teaching machines

CS 491 Lecture 20 – OpenGL ES

November 29, 2011 by . Filed under cs491 mobile, fall 2011, lectures.

Agenda

OpenGL ES

It comes time in everyone’s life for a little 3-D. The process of taking 3-D objects and showing them on a 2-D screen is purely a mathematical one that you could solve yourself with a little time and hard thinking, but there are a couple of APIs out there to help you. Microsoft offers Direct3D and the Khronos Group offers OpenGL and company. Only the latter is cross-platform, and that is why we will ignore the former. Learning OpenGL gives you access to desktop 3-D, mobile 3-D, Apple 3-D, Android 3-D, browser 3-D, Linux 3-D, Windows 3-D, and the list goes on. Everything we discuss today will apply equally to desktop OpenGL and the embedded systems version of OpenGL called OpenGL ES.

At its heart, modern core profile OpenGL is a geometry manager. It offers nothing in the way of loading geometry from files, playing sounds, or transforming your scenes. It’s purpose is to manage the geometry you give it quickly. Following are the basic pieces we need to get our geometry on our screens.

Drawing surface

The drawing surface or canvas is platform-dependent, so most OSes ship a glue library that links OpenGL to the underlying window manager. Linux has GLX, Windows has WGL, Apple has AGL, and mobile devices have EGL. If you use a cross-platform windowing toolkit like Qt, wxWidgets, or SDL, the library will hide away these different OS-specific library calls for you.

Geometry

We don’t draw pixel-by-pixel. That’d be tedious. Instead, we coarsely describe our models using nothing but triangles. It turns out any shape or surface can be decomposed into a set of triangles. For instance:

Subway's cheese placement strategy

And here’s a little WebGL page that shows how we can represent a sphere using just triangles.

Triangles are specified by their corners, or vertices. Attached to each vertex are various attributes, the most obvious one being a vertex’s spatial position. Vertices typically also have color, an association with a texture image, and a vector called a normal that indicates which direction the surface is facing. The last of these is important for shading the geometry according to its relation to the light source.

Today, we’ll upload each set of attributes as a separate buffer. These will be stored as vertex buffer objects (VBOs) directly on the GPU (at least, that’s what we hope). There’s also a special vertex buffer object that defines the connectivity between vertices.

Let’s work through an example using a simple square.

…

Shaders

Old OpenGL did everything for you. If you took a class on computer graphics while it was running rampant, you talked about how shading was done and how transformations worked, but then you just called routines that hid away all these details. Modern core OpenGL has stripped away many of these abstractions. Theory and practice are now much closer.

You ask the OpenGL library to draw your geometry for you. If you ask it to draw triangles, it will march through your index VBO, three indices at a time, extracting the corresponding vertex attributes from your other VBOs. On each vertex, a special piece of code called a vertex shader is run. It’s the job of the vertex shader to position the vertex in a special coordinate space called clip space, which we will not discuss further. Here’s a simple shader that assumes the incoming spatial position is already in clip space:

attribute vec3 position;

void main() {
  gl_Position = vec4(position, 1.0);
}

We’ve defined all our data at the vertices and described how the vertices will plop down on the drawing surface. But somebody’s got to fill in the pixels between vertices. This process is called rasterization, the “gridification” of our geometry. OpenGL will march through all the contained pixels and execute a special piece of code called a fragment shader. It’s the job of the fragment shader to assign a pixel color. Here’s a simple example assigning each pixel the color red:

void main() {
  gl_FragColor = vec4(1.0, 0.0, 0.0, 1.0);
}

Vertex Array Object

A vertex array object binds the VBOs you load up to the named attributes of the vertex shader. It’s basically a map.

After working with OpenGL for a while, I’ve come up a small library that abstracts away the inner workings of the API and that makes me reasonably productive. Let’s use my code so that we can still call this a mobile development class.

A Rectangle

Let’s draw a rectangle.

A Sphere

Let’s draw a sphere. A lit one. We can walk through the parametric equations for a sphere and sample every so often to pick out vertices. The finer our sampling, the smoother our sphere.

const int NSLICES = 10;
const int NSTACKS = 5;

float lat_delta = td::PI / NSTACKS;
float lon_delta = 2.0f * td::PI / NSLICES;

float *positions = new float[(NSTACKS + 1) * (NSLICES + 1) * 3];
float *position = positions;
float lat = 0.0f;
for (int r = 0; r <= NSTACKS; ++r, lat += lat_delta) {
  float lon = 0.0f;
  for (int c = 0; c <= NSLICES; ++c, lon += lon_delta) {
    position[0] = sin(lat) * sin(lon);
    position[1] = cos(lat);
    position[2] = sin(lat) * cos(lon);
    position += 3;
  }
}

We can then establish the connectivity, which is best explained on a markerboard:

int *faces = new int[NSTACKS * NSLICES * 2 * 3];
int *face = faces;
for (int r = 0; r < NSTACKS; ++r) {
  for (int c = 0; c < NSLICES; ++c) {
    int next_stack = (r + 1) % (NSTACKS + 1);
    int next_slice = (c + 1) % (NSLICES + 1); 

    face[0] = r * (NSLICES + 1) + c;
    face[1] = r * (NSLICES + 1) + next_slice;
    face[2] = next_stack * (NSLICES + 1) + c;
    face[3] = face[2];
    face[4] = face[1];
    face[5] = next_stack * (NSLICES + 1) + next_slice;
    face += 6;
  }
}

Let’s think about lighting for a moment. Surfaces facing a light source should be brightly lit. Surfaces facing away should be dim. How can we measure “facing?” We need to know where the light source is, or in which direction it is, and how the surface is oriented at the vertex. This orientation is defined by the normal, which is just a vector perpendicular to the surface. On a unit sphere, what is the normal?

Our measure of facing can be expressed in terms of the angle between the normal vector and a vector to the light source. If that angle is 0, what can we say? If that angle is x, what can we say? Is there a function that nicely expresses this relationship?

Now let’s augment our shaders to light the sphere. With that done, let’s get our sphere on a mobile device.

TODO