The Inner Workings of GPU Rendering for 2D and 3D Graphics

The canvas is no longer just a piece of fabric stretched over a wooden frame. It has transformed into the glowing screens of our computers, where artists and engineers alike create visual masterpieces. But how do these electronic canvases come to life with vibrant graphics? The answer lies in the intricate dance between software and hardware, orchestrated by the graphics engine.

GPU is powering our graphics experience already from the decades so it should not be a mystery how it works. It doesn’t matter if we are using 2D or 3D graphics, our GPU is a performance monster. Let’s dive deep first into what the GPU is to get rid of the buzzwords and mysteries.

What is the GPU why is it important?

A GPU stands for graphics processing unit. It’s a specialized piece of hardware responsible for rendering images, videos, and 2D or 3D animations on your computer screen.

Unlike the CPU (central processing unit), which focuses on sequential serial processing, a GPU excels at parallel processing.

Here are the key points:

1. Rendering Images and Videos: Every PC uses a GPU to display visual content, whether you’re browsing the web, watching videos, or playing games.
2. Math Calculations: GPUs perform quick mathematical calculations related to graphics, freeing up the CPU to handle other tasks.
3. Thousands of Cores: A GPU has thousands of smaller cores designed for multitasking. These cores work together to process graphics data efficiently.
4. Graphics Cards: GPUs are commonly found in graphics cards (also known as video cards). These cards contain the actual graphics processor chip and RAM to handle demanding graphics loads, such as gaming.
5. Integrated vs. Discrete GPUs:

• Integrated Graphics: Some processors (like Intel’s) have integrated graphics built right into the chip. These are cost-effective but less powerful.
• Discrete Graphics: Desktop PCs can use separate graphics cards (discrete GPUs) for better performance. Laptops may also have discrete GPUs, although they’re not as easy to upgrade.

In summary, the GPU is supplemental hardware to the brain of our devices that helps its creative end to please our eyes.

They are designed to perform complicated mathematical calculations and algorithms that are backing moving bits into pixels on our screens so we can enjoy all of beauty of the art on our desktop or even better - in our pockets.

Types of computer graphics

Let’s start easy. At the heart of computer graphics are two fundamental types: raster and vector.

Raster graphics

Also known as bitmap images, they are composed of tiny dots called pixels. Each pixel is a small square that can be assigned a color. When combined, these pixels form the images we see on our screens. This method is akin to pointillism, where numerous small dots of color create a detailed painting.

Vector graphics

Those graphics use mathematical formulas to represent images. These formulas define shapes, lines, and curves that are filled with color. Unlike raster graphics, vector images can be scaled to any size without losing quality, making them ideal for logos, fonts, and illustrations that require precision and scalability.

Our monitor is 2D canvas and 2D graphics is 2D so there should be no revelations that it’s pretty straightforward to paint it on the screen. But it’s not when it comes to rendering graphics; current system designs and graphics cards prefer using 3D space even for rendering 2D graphics which we will cover later.

Designing 3D images

The 3D graphics is a bit more complex piece of cake, basically, our display is still a 2D canvas and we can’t directly draw on this surface 3 dimension objects because that’s how physics works. So we are taking the chunk of reality from 3D space; basically, it’s called viewport and camera.

The primary compound of the virtual 3D space is a vertex - a point in 3D space having 3 coordinates, X, Y, Z. When we combine 3 vertices we have face. When we combine multiple faces we are getting the mesh - for example a game character or a cube.

When we apply the 2D image which is called texture or, more professionally, material we are getting the final 3D object.

After that, we can apply shading and other filters to make the user delighted by the visual experience. To tackle the issue that our screen is 2D and graphics is 3D, we need some tools and some tricks to convert the 3D space into flat 2D picture.

And then update it, many times, usually 60 per second so the user is tricked into believing that they are looking at something with depth and they have control over it. But basically, the user is looking at very rapidly changing 2D pictures moving the viewport inside a real 3D space.

The camera is looking at a specific viewport and, just like with human eyes, we are casting rays of light that are touching the 3D virtual space. We convert these reflections into a 2D picture.

The graphics engine: A symphony of code

The graphics engine is the maestro conducting the creation of these images. It’s a complex piece of software that takes digital information from the computer’s central processing unit (CPU) and translates it into visual content that can be displayed on the screen.

This process involves several steps, each meticulously carried out to ensure the final image is rendered correctly.

Step 1: The vertex processor

The journey begins with the vertex processor, a stage where the basic building blocks of 3D graphics, known as vertices, are processed. Vertices are points in 3D space that define the corners of shapes.

The vertex processor takes these points and performs transformations on them, changing their positions based on various factors like the viewer’s perspective and the object’s location in the virtual world.

This is a really important stage because it is moving local coordinates into world coordinates so we can displace 3D elements to their proper place. For example, a tree in a game needs to stay at a correct place, not in the center, when it’s representing its default state.

Step 2: The tessellation and geometry shaders

Next, the tessellation and geometry shaders add complexity to the shapes. Tessellation breaks down the shapes into smaller parts, allowing for more detailed and smoother surfaces. The geometry shader can then manipulate these parts, adding effects like shadows and reflections.

Step 3: The rasterizer

The rasterizer converts the 3D shapes into 2D images by determining which pixels on the screen correspond to the vertices. This step is crucial as it bridges the gap between the 3D models and the 2D screen.

Step 4: The pixel processor

Once the shapes are rasterized, the pixel processor comes into play. It calculates the color of each pixel based on lighting, texture, and other visual effects. This stage is where the image truly comes to life with rich colors and realistic details.

Step 5: The output merger

Finally, the output merger combines all the processed pixels to form the complete image.It also handles depth, ensuring that objects closer to the viewer appear in front of those further away.

Step 6: Frame buffer

After our graphics is processed, rasterized (converted to pixels) we are ready to display it on the screen. The frame buffer is an object responsible for holding pixels that we are seeing on our display.

Currently, the graphics is double buffered which means our graphics card is holding two frames. One is displayed to us, the other one is being painted. Once painting is done, we are swapping them and begin the process from the scratch again, and again, until we turn off our device.

These frequent updates are tricking our brains that we see smooth animations even if we are only looking at rapidly changing static 2D images.

Conclusion: The masterpiece revealed

As the graphics engine and GPU work together, they bring the digital world to our screens. From the simplest shapes to the most complex 3D models, the process remains the same: a meticulous method of transforming data into visual art.

It’s a process that continues to evolve, pushing the boundaries of what’s possible in the realm of computer graphics.