What Even Is An OpenGL?

Friday. August 04, 2017

Whenever I talk to a non-developer or non-game-developer about what I do for a living, there’s a good chance I’m going to need to explain what OpenGL is. It’s tricky… because to be honest, I don’t fully understand what OpenGL is myself. That must sound ludicrous given that I interact with it every single day… and most days most of what I do relates to it in some way.

But you can say about OpenGL the same thing that you can say about computers as a whole: it’s a tremendously expansive field with so many subtopics that you could never explore them all. Just as there is no single human who understands how to build an entire modern computer from scratch, there is no single human who understands wholly what takes place when a modern graphics card makes an image appear on a monitor. Still, there are humans who understand parts of this process… and I certainly am one of those humans. So I’ll tell you what OpenGL is to me, a 2D mobile game developer.

But First, Let Me Google That For You

Just to demonstrate what an enormous idea OpenGL is, let’s look it up on the internet. Wikipedia seems to think this:

Open Graphics Library (OpenGL) is a cross-language, cross-platform application programming interface (API) for rendering 2D and 3D vector graphics. The API is typically used to interact with a graphics processing unit (GPU), to achieve hardware-accelerated rendering.

Thanks, Wikipedia! I don’t know about you, but I didn’t get much out of that. I mean there were lots of words, and it definitely said something, but when you put it all together it mostly evaporates. I think the problem is that we’re trying to ingest the entire burrito in one bite. OpenGL means too many things to too many different people to summarize all of those things effectively in two sentences.

Basically, we can gather that OpenGL is an industry-standard system that lets computer hardware draw pictures on screens. Duh. But what does that really mean?

Start Over

Okay, so let’s try a different approach. Like good little software engineers, we’ll start by limiting the scope. Let’s look at a specific example of OpenGL in action and try to understand where, exactly, it fits in. Think of an iPhone game.

That’s a screenshot from Stir Fry, an iOS game I wrote long, long ago when the App Store was young. Sorry–you can’t download it anymore. But look at those happy vegetables!

As iPhone game architectures go, Stir Fry’s is pretty standard. I wrote the game using an open-source game engine called Cocos2D. But hold on–what’s a game engine? It’s an important component of a typical game’s structure. Like so many other concepts in Computerworld, an iOS game’s graphical architecture looks a lot like a layer cake.

We’ll break down these layers one-at-a-time.

• Game Logic - This is the part of the game that you can actually play. It’s how all of the elements (characters, powerups, scenery, et cetera) are modeled, and how those models interact according to the game’s rules. Usually when a developer says she’s writing a game, this is the part that she’s actually working on. It’s the only code that is unique to a particular game. You won’t find the code to slide around dancing vegetables in any game other than Stir Fry, since it’s part of Stir Fry’s game logic.

• Game Engine - The game engine, however, is a reusable framework that is often shared by many games. It’s not tailored to a particular game, but rather contains open-ended code to do lots of general things that any game might need to do. When we’re talking about 2D graphics, these general things might include loading an image from disk and positioning it on a screen. It might also include a standard way to decide which image gets drawn in front when two images overlap. And these days it almost always includes standard ways of animating these images–changing their positions, rotations, sizes, transparencies, and colors over time. I used the game engine Cocos2D to write Stir Fry, but these days there are many other popular options (Unity, SpriteKit, Corona SDK, to name just a few). For our latest game, Isaac and I went completely insane and decided to write the entire engine all by ourselves. This gives us a lot more control over how our graphics will end up looking and how the animations will work and feel… but it’s also a mountain of extra work that has literally added years to our development time. The main advantage of building on top of an existing game engine is time savings–instead of telling the computer how to do very basic tasks, the building blocks are already provided for you. So you can jump right in and start focusing on your game logic.

• OpenGL - If the game engine is in charge of loading images from disk and displaying them on the screen, then what’s left for OpenGL to do? Well–I kind of glossed over this earlier, but when the game engine is telling the computer how to draw things, it’s actually doing this entirely through interactions with OpenGL. For example, a game engine might have a method called addCircle, which adds a circle to be shown in the game scene. But when you call this method, what is the game engine really doing under the hood? It turns out that computers can’t actually draw circles–but they can draw polygons with so many sides that they look like circles when you try to display them across the screen’s pixels. So the game engine would try to draw a regular polygon with, say, 80 sides. The game engine contains the code to figure out exactly where each of the corners on the polygon should be. It calculates all of their x and y coordinates and keeps them in a list of points. And that’s where OpenGL comes in. OpenGL takes that list of points from the game engine along with instructions on what to do with those points. But after that it’s out of the game engine’s hands. OpenGL works its magic and, through some of the most advanced math and algorithms ever devised by humanity, it decides which pixels in memory it should paint over and which pixels it should leave alone. So you could say that OpenGL is the interface between the game engine and the hardware itself. It contains all the atomic-level methods (glLineWidth, glDrawArrays, glFlush, and so on) that the game engine choreographs in order to form the building blocks that the game logic in turn arranges to make even bigger things happen (like moving a character across the screen).

• Graphics Hardware/Memory - Those atomic-level methods I just brought up are only an interface. They’re a list of extremely rudimentary tasks that a game engine might need to perform in order to draw graphics (like filling a triangle or connecting two points with a line). But as I mentioned, these seemingly-simple tasks rely on mind-blowingly complex algorithms. Those algorithms are implemented in a combination of hardware (the chips in your phone’s graphics unit) and the software that supports that hardware (the graphics drivers that the chip manufacturer provides). I won’t even try to go into how these components work… but suffice it to say, they are impressive. Honestly, I don’t understand very much about their inner-workings myself… but as a software developer–even one who writes his own game engine–I mostly don’t have to. It’s kind of like piloting an airplane. I’m sure there are many things about an aircraft’s hardware that every pilot needs to know, but it’s not necessary to be able to assemble a jet engine in order to fly around with one. The point is that the graphics hardware (and its drivers) perform a staggeringly large number of calculations stupidly fast, and they use those calculations to decide how to manipulate the representation of your screen’s pixels in memory.

• Screen - Finally, we’ve reached the screen itself. Compared to what we’ve just witnessed, this part is pretty simple to wrap your head around. Each of the several hundred-thousand pixels on your phone’s display will pull three values out of its associated chunk of memory. Those values tell it how intensely to light up in three different colors: red, green, and blue. When you blend those colors together and put all the pixels next to each other, they form a mosaic that human brains can interpret as a smiling vegetable.

And there you have it! I’m sure I oversimplified and conflated more than a few concepts in there, but I hope you were able to get some sense of what OpenGL is and where it fits in a game’s graphical architecture. The borders tend to blur together, but you get the idea. In the real world, there’s a lot more to consider whenever you’re working with OpenGL. I didn’t even touch on performance or engine design or shaders or anything. But you have to start somewhere! Hopefully I’ll get a chance to tackle some of those more advanced topics soon. Until then!