We are always trying to improve our apps. Find new and fun ways for the user to interact with our apps. One way we try to make our apps fun is to make them more user interactive and immersive. In our drawing & colouring app, Cloud Doodle, we decide to make the postcard section more immersive. When we initially published the app the postcard screen was very static. Here is how it looked when the user entered the screen.
The user was shown two big buttons, front of the postcard and the back of the postcard. By pressing either of the buttons, the user would be taken to the edit screen where they would create the postcard.
We have provide a lot of backgrounds and borders for the user to start the creative process. This a picture of postcard after the user has selected a background for the front of the postcard and a border for the back of the postcard.
As you can see the screen is functional and easy to use. Since this is a fun drawing app, we wanted to make the postcard screen fun and immersive. After few meetings with the team we decide to create a 3d postcard that the user could interact with. Here is how the final implementation looks like.
With the use of OpenGL we were able to create an interactive and immersive postcard that is fun to play with. Since this a blog for the dev’s let’s get into the code.
CodeThe OpenGL code for the postcard activity can broken down into three different section, GL surface view, the renderer, and the postcard 3d object. To display OpenGL you need a container and GLSurfaceView is what you need to use to display OpenGL rendering. You can create GLSurfaceView with in your activity or in your layout xml. Since , we have a toolbar at the top of the screen it was easier just to add the GLSurfaceView to the layout xml and find it by id in the activity. The GLSurfaceView is also responsible for processing user input. In this case the user can rotate the the postcard around the y-axis (landsacpe). Hence, we attached on touch listener to the GLSurfaceView to capture the user sliding a finger across the screen. The drawing in the GLSurfaceView is delegate to the renderer, you can to create a renderer and assigne it to the GLSurfaceView. That is all the configuration that we did for the GLSurfaceView. Here is the code. Since there is a lot of stuff going on in the activity I’ll will be just showing only the relevant code.
mGLSurfaceView = (GLSurfaceView) findViewById(R.id.cardHolder);
mRenderer = new PostCardRenderer();
mGLSurfaceView.setRenderer(mRenderer);
mGLSurfaceView.setRenderMode(GLSurfaceView.RENDERMODE_WHEN_DIRTY);
mGLSurfaceView.setOnTouchListener(new View.OnTouchListener() {
@Override
public boolean onTouch(View view, MotionEvent e) {
float x = e.getX();
switch (e.getAction()) {
case MotionEvent.ACTION_MOVE:
float dx = x - mPreviousX;
mRenderer.mAngleX += dx * TOUCH_SCALE_FACTOR;
mGLSurfaceView.requestRender();
}
mPreviousX = x;
return true;
}
});
The on touch listener captures the change in the finger movement and sets the angle of the postcard that will be rendered. We’ve set the render mode to RENDERMODE_WHEN_DIRTY and because of that we need to call the requestRender() method after every angle change to render the frame. The rendering is done by the renderer assigned to the GLSurfaceView. The Renderer interface has three basic method that you have implement. One that is called when the view is created, this a good place to initialize your OpenGL properties that will not change throughout the life cycle of the view. Also, it’s a good place to load your textures.
@Override
public void onSurfaceCreated(GL10 gl, EGLConfig config) {
gl.glDisable(GL10.GL_DITHER); //Reduces quality but improves performance
gl.glHint(GL10.GL_PERSPECTIVE_CORRECTION_HINT, GL10.GL_FASTEST);
gl.glClearColor(0.05f, 0.5f, 0.98f, 0); //Background color
gl.glEnable(GL10.GL_TEXTURE_2D); //We will be using textures
gl.glEnable(GL10.GL_CULL_FACE); //Only polygons that face the screen will be drawn
gl.glShadeModel(GL10.GL_SMOOTH);
gl.glEnable(GL10.GL_DEPTH_TEST);
// Load the texture for the postcard
postCard.loadGLTexture(gl);
}
The other method that you need to implement is onSurfacechange, which is called when the size of the view has change. Here is standard code that creates the viewport and set the projection mode and restes the matrix.
@Override
public void onSurfaceChanged(GL10 gl, int width, int height) {
gl.glViewport(0, 0, width, height);
// make adjustments for screen ratio
float ratio = (float) width / height;
gl.glMatrixMode(GL10.GL_PROJECTION); // set matrix to projection mode
gl.glLoadIdentity(); // reset the matrix to its default state
gl.glFrustumf(-ratio, ratio, -1, 1, 1, 10); // apply the projection matrix
}
The third method is the onDrawFrame method that is called to draw the current frame.
@Override
public void onDrawFrame(GL10 gl) {
gl.glClear(GL10.GL_COLOR_BUFFER_BIT | GL10.GL_DEPTH_BUFFER_BIT); //clear the screen
gl.glMatrixMode(GL10.GL_MODELVIEW);
gl.glLoadIdentity();
gl.glTranslatef(0, 0, -3.0f);
gl.glRotatef(mAngleX, 0, 1, 0); //Here is where the angle is changed
postCard.draw(gl); //Here we draw the postcard
}
The importan calls before the postcard is drawn in the onDrawFrame method are the glTranslatef and glRotatef. The glTranslatef method moves the object in-closer or further on the screen, z-axis. If the object is too close you will not see the whole object. If the object is too far it might be appear too small. The glRotatef method uses the angle from the on touch listener and rotates the postcard around the y-axis. The last method in the onDrawFrame draws the postcard object.
Before you can draw the object you need to load some textures or all you will see is just some primitive shapes. The textures are load in the onSurfaceCreated method and here is the code.
public void loadGLTexture(GL10 gl) {
Bitmap bitmap1;
Bitmap bitmap2;
//The size are very important and not having correct
//sizes will prevent your texture from loading correctly
int width = 1024;
int height = 512;
//We down size for smaller screen
if (getWindow().getWindowManager().getDefaultDisplay().getWidth() < 1000) {
width = 512;
height = 256;
}
//Load your bitmaps here with the width and height defined above (create scaled bitmap)
//Here we flip the back postcard bitmap
//This can be done in the draw method by flipping the vertices
Matrix matrix = new Matrix();
matrix.setScale(-1.0f, 1.0f);
bitmap2 = Bitmap.createBitmap(temp, 0, 0, temp.getWidth(), temp.getHeight(), matrix, false);
temp.recycle();
gl.glGenTextures(2, textures, 0);
gl.glBindTexture(GL10.GL_TEXTURE_2D, textures[0]);
gl.glTexParameterf(GL10.GL_TEXTURE_2D, GL10.GL_TEXTURE_MIN_FILTER, GL10.GL_NEAREST);
gl.glTexParameterf(GL10.GL_TEXTURE_2D, GL10.GL_TEXTURE_MAG_FILTER, GL10.GL_LINEAR);
gl.glTexParameterf(GL10.GL_TEXTURE_2D, GL10.GL_TEXTURE_WRAP_S, GL10.GL_CLAMP_TO_EDGE);
gl.glTexParameterf(GL10.GL_TEXTURE_2D, GL10.GL_TEXTURE_WRAP_T, GL10.GL_CLAMP_TO_EDGE);
gl.glTexEnvf(GL10.GL_TEXTURE_ENV, GL10.GL_TEXTURE_ENV_MODE, GL10.GL_REPLACE);
GLUtils.texImage2D(GL10.GL_TEXTURE_2D, 0, bitmap1, 0);
bitmap1.recycle();
gl.glBindTexture(GL10.GL_TEXTURE_2D, textures[1]);
gl.glTexParameterf(GL10.GL_TEXTURE_2D, GL10.GL_TEXTURE_MIN_FILTER, GL10.GL_NEAREST);
gl.glTexParameterf(GL10.GL_TEXTURE_2D, GL10.GL_TEXTURE_MAG_FILTER, GL10.GL_LINEAR);
gl.glTexParameterf(GL10.GL_TEXTURE_2D, GL10.GL_TEXTURE_WRAP_S, GL10.GL_CLAMP_TO_EDGE);
gl.glTexParameterf(GL10.GL_TEXTURE_2D, GL10.GL_TEXTURE_WRAP_T, GL10.GL_CLAMP_TO_EDGE);
gl.glTexEnvf(GL10.GL_TEXTURE_ENV, GL10.GL_TEXTURE_ENV_MODE, GL10.GL_REPLACE);
GLUtils.texImage2D(GL10.GL_TEXTURE_2D, 0, bitmap2, 0);
bitmap2.recycle();
}
The most importnat part of loading the textures on Android OS is to have the correct width and height. The width and the height should be, and if you want the code to work on most device it must be, a power of 2. A lot of the Android device have issues displaying non power of 2 bitmaps as textures. Some of the clues that will tell you that the width and height are not correct are blank textures and incorrectly drawn textures. Here is an example from the Nexus S phone and the Galaxy Tablet.
As you can see the texture is blank on the Nexus S and drawn incorrectly on the Galaxy tablet.
After both of the bitmaps are created we create the textures using the bitmaps.
Before we can draw the postcard we need to define the vertices for the 3d object. The postcard is a simple rectangle so all we need to do is define four vertices. Also, we need to map the texture to the vertices. The object vertices are define in 3d space using x,y,z coordinates. The texture is define using UV mapping, which gives us a way to represent 2d image in 3d space. Here are the two arrays defining the object and the texture.
private float vertices[] = {
-3.6f, -2.0f, 0.0f,
-3.6f, 2.0f, 0.0f,
3.6f, -2.0f, 0.0f,
3.6f, 2.0f, 0.0f
};
private float texture[] = {
0.0f, 1.0f,
0.0f, 0.0f,
1.0f, 1.0f,
1.0f, 0.0f
};
Before you can use the coordinates in OpenGL you have to convert the arrays to bytes with native byte order.
// a float has 4 bytes so we allocate for each coordinate 4 bytes
ByteBuffer byteBuffer = ByteBuffer.allocateDirect(vertices.length * 4);
byteBuffer.order(ByteOrder.nativeOrder());
FloatBuffer vertexBuffer = byteBuffer.asFloatBuffer();
vertexBuffer.put(vertices);
vertexBuffer.position(0);
byteBuffer = ByteBuffer.allocateDirect(texture.length * 4);
byteBuffer.order(ByteOrder.nativeOrder());
FloatBuffer textureBuffer = byteBuffer.asFloatBuffer();
textureBuffer.put(texture);
textureBuffer.position(0);
The last piece of code that is left is the postcard draw method. Here is the code.
public void draw(GL10 gl) {
gl.glCullFace(GL_BACK);
gl.glBindTexture(GL10.GL_TEXTURE_2D, textures[0]);
// Point to our buffers
gl.glEnableClientState(GL10.GL_VERTEX_ARRAY);
gl.glEnableClientState(GL10.GL_TEXTURE_COORD_ARRAY);
// Set the face rotation
gl.glFrontFace(GL10.GL_CW);
// Point to our vertex buffer
gl.glVertexPointer(3, GL10.GL_FLOAT, 0, vertexBuffer);
gl.glTexCoordPointer(2, GL10.GL_FLOAT, 0, textureBuffer);
// Draw the vertices as triangle strip
gl.glDrawArrays(GL10.GL_TRIANGLE_STRIP, 0, vertices.length / 3);
gl.glCullFace(GL_FRONT);
gl.glBindTexture(GL10.GL_TEXTURE_2D, textures[1]);
// Point to our buffers
gl.glEnableClientState(GL10.GL_VERTEX_ARRAY);
gl.glEnableClientState(GL10.GL_TEXTURE_COORD_ARRAY);
// Set the face rotation
gl.glFrontFace(GL10.GL_CW);
// Point to our vertex buffer
gl.glVertexPointer(3, GL10.GL_FLOAT, 0, vertexBuffer);
gl.glTexCoordPointer(2, GL10.GL_FLOAT, 0, textureBuffer);
// Draw the vertices as triangle strip
gl.glDrawArrays(GL10.GL_TRIANGLE_STRIP, 0, vertices.length / 3);
//Disable the client state before leaving
gl.glDisableClientState(GL10.GL_VERTEX_ARRAY);
gl.glDisableClientState(GL10.GL_TEXTURE_COORD_ARRAY);
}
The draw method draws the front of the postcard followed by the back of the postcard. Depending how you use the glCullFace and glFronFace setting the textures will either be drawn correctly or flipped horizontally. Here we cull the front and the back of the images and set the front and the back face to clock wise. If you don’t want to cull the polygons you can remove the glCullFace method and change the second glFrontFace to GL_CCW, which is counter clock wise. This will flip the texture.
This the basic code that you need to create a 3d object and apply texture to it and make it move. As you can see OpenGL is not only for gaming, but it can be used to enhance the user experience by making your UI more interactive and immersive.
January 22nd, 2013
Subscribe
0 Comments
See more:
December 10th, 2021
Cloud Composer – Terraform Deployment
December 2nd, 2021
Provision Kubernetes: Securing Virtual Machines
by August 6th, 2023
The Critical Need for Application Modernization in SMEs