teaching machines

CS 488: Lecture 1 – Points

January 20, 2021 by . Filed under graphics-3d, lectures, spring-2021.

Dear students:

Welcome to CS 488 – Computer Graphics Applications. This is a class where we learn how to make output that non-computer scientists can enjoy. In particular, the output is 3D and interactive. You can show the 3D worlds that you make in this class to others, and you will be a hero.

Graphics Biography

As your professor, I think it’s important for you to know that I am a human and not a robot. Therefore, I will offer up a little biographical information that I think is relevant. I’m not usually the professor who teaches this course, but I petitioned to teach it because graphics has between intertwined with my computer science journey from the very start. In the mid-1990s, my family got our first computer, a Macintosh Performa, and we got the internet a short time later. I discovered POV-Ray, a raytracing programming that let me program geometric scenes. I used it to design art for websites and to help me with me calculus homework. My interest in 3D was fed by the wireframe renderings I saw in Nintendo Power of the forthcoming Donkey Kong Country.

Then college happened, and I got a degree in computer science. As graduation neared, I wasn’t sure what to do. I wasn’t excited by the prospect of working for a corporation. A professor convinced me to attend graduate school. When I showed up to pick classes for my first semester, I saw computer graphics. A wave of nostalgia for my early days of computing rushed over me and I signed up. As soon as the class started, I regretted my decision. The math was too much for this poor farm kid. I dropped the class and grieved over my failure.

The textbook sat on my desk and haunted me. I decided to keep working on the programming assignments even though I wasn’t registered anymore. And it was such fun. Eventually I met with the instructor and explained that I was interested in the subject but ill-prepared. He suggested that I take the class with him as an independent study the next semester. I did, and he eventually became my advisor. I worked on papers with him about finding interesting features in large datasets. Around that time, NVIDIA and ATI (now AMD) started selling graphics cards that you could write programs for. I spent most of my dissertation time writing a renderer that would display volumetric data sets like CT and MRI scans.

Since graduate school, I’ve been making games, making spotty attempts to learn 3D modeling with Blender, and designing a programming language for algorithmic constructing 3D models. My programming assignments in all classes are often centered around producing graphical output. That brings me here to this course.

What brings you?

Modern Graphics

Entering the graphics scene today is awkward. 30 years ago, there was no scene. You would write a renderer from scratch, which was a lot of work. Or you could license John Carmack’s engine from id Software. 20 years ago, the industry was converging on two graphics libraries: OpenGL and Direct3D. 10 years ago the market started to fracture. Apple stopped updating OpenGL. They wanted everyone to use their propietary low-level graphics toolkit named Metal. Vulkan appeared as an open competitor to Metal. Game engines like Unity and Unreal Engine became free or at least inexpensive. For someone interested in graphics, there is no clear starting point.

In this class, I am choosing to use WebGL, an open standard for writing graphical applications that run in a web browser. Unlike Direct3D, which runs on Windows, and unlike OpenGL, which has been crippled on macOS, WebGL runs on all major operating systems, including mobile.

Graphics software developers are often very concerned about performance and therefore write in C++, a language that has only a little runtime overhead and no non-determinism like garbage collection. Some of you may have signed up for this course hoping to learn more about C++. Bad news. We’re using JavaScript, the language of the web browser. Ultimately this class is about computer graphics, and we will have more time to focus on this subject if we target WebGL and JavaScript. I have taught this course before. We used C++ and wasted considerable time in platform-specific issues. It’s just not worth it.

You are not expected to know JavaScript to succeed in this course. You are expected to learn what you need to know, which is not a big ask for someone studying computer science.

The same core ideas lie at the heart of all the graphics libraries. Here are the primitives we have to work with in all of these libraries:

We have direct control over the final three items of this list. The driver of our graphics card will update the framebuffer according to the WebGL commands we issue.

Hello, Framebuffer

Let’s see these four primitives in action by writing our first WebGL program. We’re going to write some code. I encourage you to listen and ask questions. Writing the code along with me is not sustainable. Consult the class notes later on to get the code.

We start with this HTML file:

<!DOCTYPE html>
<html>
<head>
  <title>Points</title>
  <link rel="icon" href="data:,">
  <link rel="stylesheet" href="style.css">
</head>
<body>
  <canvas id="canvas"></canvas>
  <script src="dist/bundle.js"></script>
</body>
</html>

The canvas element will be the entity that gets its pixels from the framebuffer. The script element will run our JavaScript code that creates the array of data and the vertex and fragment programs.

The HTML references this stylesheet, which makes the canvas fill the browser window:

body {
  margin: 0;
}

#canvas {
  position: fixed;
  left: 0;
  right: 0;
  width: 100vw;
  height: 100vh;
}

In our JavaScript file, we have some setup to do. First we get a handle on our canvas element and request a WebGL context:

const canvas = document.getElementById('canvas');
window.gl = canvas.getContext('webgl2');

We say window.gl here to make the context truly global. Without it, code inside other files may not be able to see it.

The first thing to do when starting a new graphics application is clear the framebuffer to a background color. We’ll put this code in a render function that, ultimately, we’ll call whenever we want to redraw.

function render() {
  gl.clearColor(1, 0.5, 0, 1);
  gl.clear(gl.COLOR_BUFFER_BIT);
}

render();

What do you suppose those four numbers in the clearColor function represent? They are the red, green, and blue intensities and the opacity that will get written to each pixel in the framebuffer. They are numbers in [0, 1]. Probably they are stored in the framebuffer as unsigned bytes in [0, 255], but that’s a decision we leave up to the graphics driver.

You should notice something funny. The framebuffer doesn’t fill the screen. That’s because the browser doesn’t automatically resize the canvas. This is an unfortunate quirk of HTML and CSS that we can fix by adding a handler for resize events:

function onSizeChanged() {
  canvas.width = canvas.clientWidth;
  canvas.height = canvas.clientHeight;
  render();
}

async function initialize() {
  window.addEventListener('resize', onSizeChanged);
  onSizeChanged();
}

initialize();

Note that clientWidth and clientHeight are the dimensions of the canvas element within the page, whereas width and height are the dimensions of the content. We force these to be the same.

I put async in front of initialize because we’ll need it later when we read in files.

Points

With a framebuffer in place, we can start creating vertices. At the very least, we need to know about each vertex’s position. There will be other data later on. Let’s start with a single vertex at the origin:

const positions = [
  0, 0, 0,
];

const attributes = new VertexAttributes();
attributes.addAttribute('position', 1, 3, positions);

This code uploads the array of vertex data to the graphics card and stores it as a vertex buffer object. Most of the details of how this is done are hidden away in the VertexAttributes class that I provide you. Generally, I don’t like scaffolding code, but I don’t want to talk concretely before abstractly.

The next step is to write a vertex program that will be run on each vertex. This program will be run directly on the graphics card and be fed in the attribute data from the vertex buffer object. The program is written in a separate language called the OpenGL Shading Language (GLSL). These programs are thus sometimes called shaders.

Our first vertex shader mostly passes the vertex’s position straight through:

const vertexSource = `
in vec3 position;

void main() {
  gl_Position = vec4(position, 1.0);
}
`;

Every vertex shader must assign to a builtin variable with the grotesque name gl_Position. It has four coordinates, not three, which should seem odd to you right now. We’re going to let that stay odd for the time being. For the time being, just tack on a 1 for the fourth coordinate.

Usually the vertices are grouped in a triplet, forming a triangle. The graphics card proceeds to color all the pixels in this triangle by running a fragment shader on each one. The job of the fragment shader is to write a color. Here we make every affected pixel black:

const fragmentSource = `
out vec4 fragmentColor;

void main() {
  fragmentColor = vec4(0.0, 0.0, 0.0, 1.0);
}
`;

Next we pair the two shaders. We also pair the shaders and vertex data that we made earlier:

shaderProgram = new ShaderProgram(vertexSource, fragmentSource);
vertexArray = new VertexArray(shaderProgram, attributes);

The marriage of a shader program and vertex attributes is called a vertex array object.

In render, we draw the vertex array object with this little dance:

shaderProgram.bind();
vertexArray.bind();
vertexArray.drawSequence(gl.POINTS);
vertexArray.unbind();
shaderProgram.unbind();

Eventually we will have many shader programs and many vertex arrays. We must make them active in order to draw them.

We should see a very small speck in the middle of the screen. On some browsers, we can bump up the size of a POINT with this line in the vertex shader:

gl_PointSize = 10.0;

Exercises

For the remainder of our class time, let’s work through the following challenges.

  1. Place a second point on the right side of the canvas.
  2. Plot points along a line.
  3. Plot a circle.

TODO

Here’s your very first TODO list:

See you next time.

Sincerely,

P.S. It’s time for a haiku!

Those blasted pirates
I’ll never find their treasure
X marks a whole plane