This section is a bit unusual because not only does this console have multiple screens to draw, but also bundles traditional tile engines working alongside a modern renderer.
Comparison of screen resolution and aspect ratio between the Game Boy, Game Boy Advance, and Nintendo DS series.
Let's begin with the physical attributes: the Nintendo DS contains two LCD screens, each 256 x 192 pixels in resolution, which is roughly 20% more pixels than the GBA. They can display 262,144 colours (18-bit) and refresh at ~60 Hz.
Architecture
The graphics subsystem can render both 2D and 3D objects. The former is composed of two-dimensional geometry - in other words, 8 x 8-pixel bitmaps called 'tiles'. The latter draws three-dimensional objects (polygons) using vertices.
Diving into the internal chip that operates those screens, we can observe this console has distinctive hardware for 2D and 3D geometry. The 2D data is operated by a familiar engine, the Picture Processing Unit or 'PPU' (now just called 2D engine), while 3D data is handled by a completely new subsystem. That said, the console houses two 2D engines and one 3D engine. It's worth mentioning that, while this is not the first Nintendo console to debut 3D graphics, it's the first one to feature an in-house design to render 3D shapes.
Layout of the graphics subsystem.
Now, each engine can only be linked to one of the screens. This is not an issue for 2D-only games, as each screen has a 2D engine available. However, for those games that want to show off cutting-edge features, there's only one 3D engine available. As a consequence, 3D capabilities are only available on one screen at a time.
... And what about mixing 2D and 3D objects? Absolutely, let me explain each engine separately so we can discuss this afterwards.
Constructing a frame with 2D graphics
Before we review each stage, I recommend reading about the GBA's PPU since here I will only highlight the changes that led to the 'next-generation' of 2D games.
To set the stage, as there are two 2D engines, the first one is named Main while the second one is called Sub. This doesn't necessarily imply which screen each one is connected to. On the other hand, Main provides a bit more functionality than Sub.
To support the explanations, this time I will be using New Super Mario Bros as an example.
Tiles
Some tiles found in VRAM. For demonstration purposes, a default palette is used.
By now we all know how a basic tile engine works, but how are tiles particularly managed on this console? Well, there's a total of 656 KB of Video RAM (VRAM) available , and this chunk is split into different banks: four of 128 KB, one of 64 KB, one of 32 KB and three of 16 KB. Programmers are free to fill these banks with drawings and then point the engine to the required data. Furthermore, both engines can read from any of the banks, but they cannot access the same one concurrently.
Nonetheless, there are some limitations on how data can be distributed. For instance, the ARM7 can only access two 128 KB banks. At the same time, these two banks can't store sprites, and Sub is the only engine capable of accessing the last 16 KB bank. The list goes on, but you get the idea.
One last thing: the 3D engine, which we will dive into later on, can access some of these banks to fetch textures.
Background types
Ever since the Super Nintendo made its debut, subsequent designs of the PPU have been leaning towards providing more flexibility for background layers. Fourteen years later, this culminated in a chip capable of fetching tiles, applying many affine transformations, and, if that were not enough, assembling a layer directly from a frame buffer.
Before we discuss the different modes that the 2D engine uses to generate backgrounds, let me show you this list. It specifies the types of backgrounds the engine can generate:
- Tile type group: These background types follow the traditional tile system. In other words, the frame is rendered by filling it with tiles.
- Static background: An ordinary background, up to 512 x 512 pixels wide, using 256 colours and 16 palettes. It includes all the typical effects (H/V flipping and scrolling, mosaic, and alpha blending) plus an extra fading effect. Up to 1,024 tiles can be used.
- Affine background: A background with affine transformations. However, it doesn't allow H/V flipping and can only fetch 256 tiles (one quarter of the maximum). The size of this layer is 1024 x 1024 pixels.
- Affine Extended: Same as affine but restores the full amount of tiles and supports H/V flipping.
- Bitmap type group: Instead of processing tiles, the engine treats VRAM as a frame buffer.
- Affine Extended (256 colours): Inherit all the effects available in the tiled 'Affine Extended'. The difference is that they are now applied to a single 512 x 512-pixel bitmap.
- Affine Extended (Direct colour): Similar to the previous type, but the frame buffer now supports up to 32,768 colours (15-bit).
- Large screen: Eats up a whole 128 KB chunk of VRAM to render a large 1024 x 512-pixel frame buffer.
- 3D background: Displays the output of the 3D engine as a background layer, which is essential for showing whatever the 3D engine has processed. Although it doesn't provide a lot of 2D effects, there are interesting features like horizontal scrolling and alpha blending (with other background layers). Also, it's the only type that supports up to 262,144 colours (18-bit).
These modes can't be chosen arbitrarily; instead, the console provides a set of background modes with predefined combinations. Be that as it may, whenever a background mode states to support 'Affine Extended', developers are free to choose the flavour ('Tiled', '256 colours', and 'Direct colour').
Background modes
Background Layer 0 (BG0). This particular layer is shifted horizontally at certain scan lines to simulate the clouds moving.
Background Layer 2 (BG2).
Background Layer 3 (BG3).
This is where the background types are put into action. Main and Sub offer multiple modes of operation. All of them generate four background layers, however, each layer has different capabilities depending on the mode activated :
- Mode 0: 4 Static layers.
- Mode 1: 3 Static layers + 1 Affine layer.
- Mode 2: 2 Static layers + 2 Affine layers.
- Mode 3: 3 Static layers + 1 Affine Extended layer.
- Mode 4: 2 Static layers + 1 Affine layer + 1 Affine Extended layer.
- Mode 5: 2 Static layers + 2 Affine Extended layers.
- This is a popular mode, as it's highly flexible.
- Mode 6: 1 3D background layer + 1 Large screen.
- Because there is only enough space for a single frame buffer, this mode is available only on 'Main'.
Additionally, in Modes 0 to 5, the Main engine can set the first static layer as a 3D background instead. The 3D capabilities will be covered in a second.
Sprites
Rendered Sprite layer.
Sprites or 'Objects' inherit the same functionality as the GBA's PPU, but with two considerable additions.
Firstly, Object Attribute Memory or 'OAM' (the region where sprites entries are stored) is now 2 KB wide , enabling to display up to 128 sprites per frame per screen. Thus, 1 KB is allocated per engine.
Secondly, OAM can now reference bitmaps from VRAM as opposed to only using tiles and palettes . This is another departure from the tiling system. In fact, both sprite 'modes' can coexist in the same frame, since this option is set for each individual sprite.
Result
All layers merged... is there something missing?
As each layer renders on-the-fly, the final stage is tasked with merging everything and sending it to the selected screen. This is pretty much what happens with previous PPU-based consoles, does that mean we are done here?
Not yet! Main still has to fetch a layer from another engine, the most powerful one.
The 3D accelerator
If you have played with a Nintendo DS before, you know by now that this console can display a particular amount of 3D graphics. Unlike some GBA games, however, these are not rendered by the CPU. Instead, CPU NTR includes two components that together constitute the 3D engine, a design that reminds me of SGI's Reality Coprocessor.
Cycling back to the 'Background modes' section, you'll notice that every mode has at least one static background. This is because you can fill that layer with graphics produced by the 3D engine. The only caveat is that only Main can do this. This also explains why Mode 6 is only available on Main.
Geometry Engine
Architecture of the Geometry Engine.
If you read any of the articles from the 5th or 6th generation, you may be wondering... Where's the SIMD processor? That's a good question because the ARM9 is not particularly good at vector operations and I don't think the dedicated divider is enough. That's why Nintendo embedded a component called Geometry Engine that takes care of vertex transformations, projection, lighting, clipping, culling and polygon sorting, the latter is essential to properly use the transparency features .
This engine has some strict limitations. Specifically, the count of polygons it can process: there's an extra 248 KB of RAM used to store processed geometry. This amount accounts for up to 2048 triangles or 1706 quadrilaterals , although this limit can be stretched by using polygon strips (as opposed to individual polygons). Another point to bear in mind, is that the screen resolution of this console is relatively small, so that also compensates.
Anyway, this engine is commanded using a Command FIFO which is filled by the CPU or DMA . The FIFO stores 256 entries, yet it's complemented by another buffer called PIPE that stores four more commands (allowing a total of 260).
Rendering Engine
The architecture of the Rendering Engine.
The Rendering Engine is in charge of converting vectors to pixels (i.e. rasterising), colouring them (i.e. texture mapping), and applying lighting and other effects . It relies on perspective correction and Gouraud shading for interpolating textures and light, respectively. Moreover, the unit provides modern features like fog, alpha blending, depth buffering (either Z-buffering or a variant called W-buffering), stencil tests and anti-aliasing. However, the latter is very primitive (it just sets the outer edges of polygons as transparent) and it only works with opaque pixels.
The rendering system is a mix of old and new: instead of outputting a frame buffer, it employs line buffer rendering, where it fills scan lines (similarly to the 2D engine) and stores the results in a smaller buffer. This is because the 3D engine must work at the same pace as the 2D drawer.
Without the traditional frame buffer, the rasteriser employs scan line rendering, traversing each scan line to process any polygon edges found within. Arisotura (the developer of the MelonDS emulator) reported that, for each quadrangle, the renderer can only fill one span per scan line . This can be a bit troubling, since the result gets messy if the quad is concave or contains crossed edges, for instance.
Regarding effects, the unit also provides shadowing and a distinct feature called Toon Shading (another name for Cel Shading): Even though this unit is not programmable, the lighting parameters can be altered to achieve a cartoony effect .
Result
Ah, that's more like it.
Instead of writing the results back to a frame buffer for display, the rendering engine writes to a block called Colour Buffer which stores up to 48 scan lines . Each scan line is fetched by the 2D engine to fill the BG0 layer in a 'First In, First Out' (FIFO) manner.
3D rendering starts before the 2D one, enabling the latter to apply transformations on the new layer if required. Main also allows to capture the 2D, 3D or combined frame generated, blend it with another frame in VRAM, and write the result back to VRAM - which can be displayed afterwards.
In terms of control, the rendering engine also enables its parameters to be altered mid-frame, thanks to double-buffering mechanism that preserves a copy of the previous state until the current frame has finished drawing. Thus, no tearing appears.
Famous comparisons
Some of the early games released for this console attempt to resemble Nintendo 64 counterparts. So I found this a good opportunity to summarise why players may notice certain patterns between the two versions.
Super Mario 64 (1996).
Rendered at 320×240 pixels.
Super Mario 64 DS (2004).
Rendered at 256x192 pixels.
Mario Kart 64 (1996).
Rendered at 320×240 pixels.
Mario Kart DS (2005).
Rendered at 256x192 pixels.
To explain what's happening here, I've organised the different explanations based on what some people said on forums:
- NDS' textures look more blocky: The rendering engine does not employ any filter. So, textures are interpolated using the 'nearest neighbour' approach.
- NDS' textures look richer: The rendering engine is not limited by a 4 KB Texture Memory block. Instead, there are up to 512 KB of VRAM available and plenty compression mechanisms provided. Thus, naturally more data can be loaded.
- NDS' models contain pixelated edges: NDS models render at a lower resolution compared to the N64 counterparts.
- NDS' textures appear distorted when seen from a distance: The rasteriser operates fixed-point coordinates. Low resolution and lack of mip-mapping also contribute to aliasing.
That's pretty much in a nutshell. For more specialised cases, you'll have to dive deeper into both engines and possibly disassemble both games to investigate which functions are being used and how.
Model designs
The following are 3D models visualisations from games designed for this console. Notice the use of detailed texture maps, albeit limited to nearest-neighbour filtering. This results in a 'modernish' pixelated look, an icon of Nintendo DS graphics.
Interactive model available in the modern edition
Nintendogs (2005).
750 triangles.
Interactive model available in the modern edition
New Super Mario Bros (2006).
636 triangles.
Despite having talked about many limitations of the graphics subsystem, I have to say, lots of games knew how to leverage it.