GameCube Architecture

After the loss of SGI's dominance in the graphics market, Nintendo needed new players to partner up with.

A promising candidate seems to be IBM: Apart from their famous work on mainframes, they recently joined forces with Motorola and Apple to create a CPU powerful enough to compete with Intel's ruling in the PC market. The resulting product is a series of processors carrying the name PowerPC, which were selected to power 99% of Apple's Macintoshes, IBM's workstations and some embedded systems.

The PowerPC Gekko chip. This is what the GameCube houses.

To understand what this means for the GameCube, let's first take a look at the chain of innovation that culminated in the PowerPC CPU.

The origins of PowerPC

IBM was one of the three early forces pushing for the emerging RISC CPU design into the mainstream market. During the 80s, while Berkeley was busy developing the 'RISC CPU' and Stanford academics just founded MIPS, IBM had already produced the 801 and ROMP CPUs. These were ground-breaking yet commercially-unsuccessful silicon implementing a set of guidelines later known as the 'RISC model' .

The commercial milestone

Entering the 90s, IBM tried again with a new series of UNIX workstations called 'IBM RS/6000', and within it, a new in-house RISC CPU: POWER1. With a focus on instruction-level parallelism, the latter featured attractive advancements such as :

• A complete 32-bit instruction set called POWER.
• A 64-bit floating-point unit.
• A Harvard cache architecture, this segregates data and instruction space to increase bandwidth.
• The ability to execute two instructions at the same time by distributing them across its three separate units (branch, fixed-point and floating-point). Hence, POWER1 is recognised as being two-way superscalar.
• Static branch prediction, which turns control hazards into opportunities for accelerating execution.
• Out-of-order execution of floating-point operations, through register renaming . All in all, this greatly increments the amount of instructions the CPU executes per time (and takes care of data hazards).
  • The core design originates from IBM's mainframe era, when it was published as Tomasulo's algorithm and subsequently implemented on the IBM System/360 (1966). With POWER1, IBM managed to bring part of it to workstation equipment.

Nevertheless, the POWER1 CPU was a large and expensive package comprised of multiple chips. So, for their follow-up venture, IBM shrank down its design to fit in a single chip. While still POWER-compliant, this came at the cost of housing a 32-bit FPU, a von Neumann cache architecture and reverting to in-order execution. The new chip was called RISC Single Chip (or 'RSC') and shipped with the low-end line of their RS/6000 workstations.

Reaching the average user

IBM's POWER development board (1992), used by Apple.

In the midsts of these developments, IBM also agreed to join forces with Apple and Motorola to tackle the Intel-Microsoft monopoly in the desktop market, forming the AIM alliance. Thus, a new project was conceived for a competitive CPU in the low-end arena. This would be made of intellectual property from the three companies, including:

• IBM's RSC processor design.
• Motorola's bus architecture (found in their in-house RISC processor, the Motorola 88110).
• Apple and Motorola's knowledge of the end-user's needs.

Keith Diefendorff, a member of the Motorola 88110 team, was recruited as the lead architect and, in 1993, the project culminated in the following:

• The PowerPC instruction set: A subset of POWER, with additional instructions for multiplication, support for a symmetric multiprocessor setup and an optional 64-bit mode.
  • From then on, IBM's POWER CPU line would implement the PowerPC ISA instead. This was first apparent with the release of POWER3 in 1998, a high-performance CPU implementing the 64-bit specification of PowerPC.
  • We won't see a symmetric multiprocessor setup in this article series until the Xbox 360 and Wii U arrive.
• The PowerPC CPU, a new line of desktop CPUs implementing the PowerPC ISA, starting with the PowerPC 601. This was a cost-effective version of the RSC microarchitecture.
  • Motorola also dropped the development of the 88000 CPU altogether to focus on this new series.

Popularising the PowerPC

To make sure the new line would be both technically competitive and commercially viable, the PowerPC 601 attempted to bring certain advancements of instruction parallelism to the masses, to name a few :

• The implementation of both POWER and PowerPC ISAs, to assist POWER developers in transitioning to PowerPC.
• Three-way superscalar execution using three separate execution units (FPU, branch and ALU) . An improvement from previous designs.
• Based on Motorola's model, a new bus design called Bus Interface Unit, providing:
  • A 64-bit data bus and a 32-bit address bus, the former is critical for taking advantage of the superscalar capabilities.
  • Burst transactions, enabling to transfer 32 Bytes of memory (the size of L1 cache) with a single instruction .
• Memory Management Unit (MMU), offering virtual memory in the same package.

For the end user, this new CPU would now be found in IBM's low-end RS/6000 series and in Apple's new line of Macintosh computers called 'Power Macintosh'.

Contemporary work

The PowerPC 601 was meant to kickstart momentum for the PowerPC line, but the following years saw turbulent changes in the microarchitecture.

Individual developments

Once the 601 shipped, Motorola and IBM decided to separately work on the follow-up generation. This would be a pure PowerPC implementation (removing traces of the POWER ISA) materialising as two separate product lines :

• The low-end PowerPC 603, headed by Motorola, was designed for the portable market. In doing so, it delivered a smaller L1 cache (now based on the Harvard architecture), two additional execution units and no multiprocessor support. The design decisions resulted in an overall consumption rate of 1.8-2.0 Watts .
• The high-end PowerPC 604 headed by IBM. In exchange for a premium price and high power consumption (14.5-18.5 Watts), it provided advanced parallelism capabilities (à la MIPS), such as 4-issue execution, the return of out-of-order execution, dynamic branch prediction and multiprocessor support .

Even though the 603 should've shined for its power efficiently, existing business applications ended up eclipsing its novelties. For instance, Apple's software architecture was still relying on emulating 68000 instructions, which bottlenecked the 603's small cache size.

As a side note, IBM also developed the PowerPC 4xx series, conceived as a microcontroller solution instead of a desktop CPU . They bundle small cache and skip sophisticated modules such as the MMU and FPU. However, their variants were offered as a customisable package to tailor it to the manufacturer's needs. That being said, while the 4xx is not of particular interest for this article, a famous competitor did adopt it for secondary tasks.

Joining forces again

Ultimately, the second generation of desktop PowerPC chips was deemed either too expensive or downright uncompetitive against Intel. Thus, Apple got IBM and Motorola to collaborate again in a new unified generation that brought out the best of both worlds. The energy-efficient 603 was chosen as the foundation for the new design. To improve upon this basis, certain decisions were taken:

• Addressing its limitations by providing larger cache and higher memory bandwidth.
• Incorporating design ideas of the 604, such as dynamic branch prediction and out-of-order execution (limited to memory operations, for now).
• Implementing new enhancements, such as an extra ALU for greater parallelism.

This became the 750 line, popularised by Apple as the PowerPC G3. From then on, IBM and Motorola continued working on variants and enhancements of the 750. These focused on greater clock speeds, larger cache and smaller fabrication process.

Interestingly, it's always the most energy-efficient CPUs that manage to weather development turmoil, as others (MIPS, Intel and ARM) will later corroborate.

The band splits for good

As time passed, the three companies became increasingly distant. Two years after the PowerPC G3 was unveiled, Motorola delivered by itself a new line called the 7400 (which Apple named the G4). This featured a 64-bit FPU, a faster bus architecture called 'MPX' and a set of SIMD instructions called 'Altivec'. Despite its popularity in the desktop market (thanks to Apple), IBM only devoted attention to its exclusive POWER CPU line.

Years later, in 2003, Motorola finally gave up on the CPU business and divested its semiconductor division, leading to the incorporation of 'Freescale'. The latter wasn't interested in working on PowerPC chips either, and so the AIM alliance came to an end. Be as it may, Apple still needed new CPUs, so IBM continued the line of succession by grabbing its POWER4 design and scaling it down, leading to the PowerPC 970 CPU (also called the 'G5').

And this is where the history section ends. Thus, it's time to take a look at the GameCube's unique CPU (which sits between the 750/G3 and the 7400/G4). Now, I'll be focusing on the GameCube hardware from now on, but if this detour got your attention, you may want to read the PlayStation 3, Xbox 360 and Wii U studies next.

The PowerPC Gekko

Back to the year 2001, Nintendo required something powerful but cheap. So, to comply with those red lines, IBM grabbed one of its past designs, an enhanced G3 called the PowerPC 750CXe (found on the late iMac G3, known as the Early-Summer 2001), and beefed it up with capabilities that would please game developers. The result was the PowerPC Gekko and runs at 486 MHz.

Construction of Gekko.

Let's find out what makes Gekko so special, and to do that we need to first look at the offerings of the 750CXe. Now, having gone through the history of PowerPC, you may find that a lot this information overlaps with previous designs (that's the point of these studies!). That being said, the 750CXe offers :

• The PowerPC ISA: This comes as no surprise. The only additional information is that the 750CXe implements the v1.10 specification.
• External 64-bit data bus: As you've seen before, while the ISA can fit in a 32-bit bus, we still need to move wider chunks of data without hitting performance penalties.
• Triple-issue superscalar: If the required units are available, the CPU may process up to two instructions at the same stage of the pipeline. In the case the queue includes a branching instruction, the number of possible concurrent instructions is raised to three.
• Out-of-order execution: The CPU can re-order the sequence of instructions to keep all of its units working, thus increasing efficiency and performance.
• Two Integer Units: Combined with the superscalar and out-of-order model, it increments the number of integer operations done per unit of time.
• Integrated FPU with 32-bit and 64-bit registers: Accelerates operations with floats and doubles.
• Four-stage pipeline (with bonus): Here is a previous introduction to instruction pipelining. In the 750CXe, FPU operations are divided into three more stages (7 stages in total) while load-store operations are divided into two (5 stages in total).
  • All in all, this increments the instruction throughput without getting out of hand.

Additionally, this CPU also includes dedicated units to speed up specific computations :

• Branch Prediction Unit: Whenever there is a condition that needs to be evaluated (which would decide if the CPU should follow path 'A' or path 'B'), the CPU will instead follow one of the paths based on previous executions and then evaluate the condition. If the prediction happens to be right then the CPU will have saved some time, if not, it will reverse and follow the correct path.
• Dedicated load-store unit: Separates the units that manipulate registers and the ones handling main memory.
• Integrated Memory Management Unit: Interfaces all memory access from the CPU.
  • Previous consoles included this component as another co-processor, the MMU is now found inside the same chip, reducing manufacturing costs.

And, of course, some cache is also included to speed up memory bandwidth:

• 64 KB L1 cache: Divided into 32 KB for instructions and 32 KB for data.
• 256 KB L2 cache: It can be filled with instructions and data, which greatly improves bandwidth.

IBM's enhancements

While the previous lists of features are very appreciated (compared to previous generations), this CPU still falls behind others on gaming performance (let's not forget that this is still a general-purpose CPU, good at spreadsheets but average at physics). To compensate, IBM added the following tweaks that will constitute Gekko :

• Enhanced instruction set with 50 new SIMD instructions: These operate two 32-bit floating-point numbers or one 64-bit floating-point number using only one cycle. Consequently, the new SIMD instructions will speed up vector calculations, particularly useful during geometry transformations.
  • Not to be confused with Motorola's SIMD extension (AltiVec) which shipped on the PowerPC 7400/G4. They are not compatible with each other. Curiously enough, IBM did eventually implement Altivec in its POWER4 CPU and, by extension, into the 970/G5 series, Cell and Xenon.
• 32 floating-point registers: These come with the new SIMD instructions.
• Write Gather pipe: A special memory writing mechanism available to use. If enabled, instead of performing single-beat transfers, it holds all memory write requests in a 128-byte buffer until it's 25% full, then performs the requested writes using a technique called burst transaction which can move blocks of 32 bytes of data at once.
  • As you can imagine, this saves a lot of bandwidth by making full utilisation of the available buses.
• Locked L1 cache: Programs can take away 16 KB of L1 data cache to use as 'scratchpad' (incredibly fast memory).

Apart from handling the game logic (physics, collisions, etc), these enhancements will allow the CPU to implement parts of the graphics pipeline (geometry transformations, lighting, etc) with acceptable performance. This is very important since the GPU can only accelerate a limited set of operations, so the end result is not conditioned by the GPU's limitations.

A step forward or a step backwards?

On your Nintendo 64 article, you explained that the system has a 64-bit CPU, but the GameCube one is 32-bit. Did Nintendo downgrade their console?

Indeed Gekko implements a 32-bit PowerPC specification, while the MIPS R4300i can switch between 32-bit and 64-bit modes. To answer whether this is an improvement or not, you have to ask yourself: Why would you need '64-bitness'?

• To address more than 4 GB of memory -> The GameCube doesn't have near that amount of memory locations, so this is not a requirement.
• To operate larger chunks of data using fewer cycles and bandwidth -> That's covered by Gekko's new SIMD instructions, the 64-bit FPU and data bus; and the write-gather pipe.
• To come up with more advertising terms -> Yeah... I don't think that persuades people anymore.

As you can see, the GameCube already enjoys the advantages of 64-bit systems without being called a '64-bit console'. This is why you and I can't summarise two complex machines by their 'number of bits'.

Clever memory system

During the design of the next-gen architecture, Nintendo's architects performed a post-mortem analysis of their previous design and discovered that using a Unified Memory architecture together with some high-latency components (RDRAM) resulted in one of the biggest causes of bottlenecks (almost 50% of CPU cycles were wasted while idling) . Moreover, the inclusion of multiple independent units contributed to a concerning competition for the memory bus.

For that reason the GameCube's architects came up with a new memory system strictly based on providing dedicated memory space and using low-latency chips. With the new design, GPU and CPU will no longer compete for the same RAM (causing fill-rate issues) since the GPU will now have its own internal and amazingly fast memory. On the other side, the GPU will still be in charge of arbitrating access to I/O too.

Memory layout of this system.

The result was a system organised with two main buses:

• The Northbridge: It's 64-bit wide and connects the CPU with the GPU. It runs 3 times slower than the CPU clock, so tasks will have to be optimised to not rely so much on the GPU. Other components like the DMA and cache may come in handy.
• The Southbridge: It's also 64-bit wide and connects the GPU with 24 MB of 1T-SRAM called 'Splash'. This type of RAM is made of DRAM (which is the most popular type but also cheaper and slower) however it's enhanced with extra circuitry to behave like SRAM (faster and expensive, mostly used for cache). The bus is twice as fast as the GPU, possibly to enable the GPU to provide a steady bandwidth between the CPU and main memory.

Additionally, this design contains an additional (yet unusual) bus where more memory can be found:

• The Eastbridge: It connects the GPU with another memory chip called Audio RAM or 'ARAM' . It provides 16 MB of SRAM available for general purposes, which is pretty big for spare memory. However, the bus is not accessible through normal means (memory addresses). Instead, it's connected to an 8-bit serial endpoint (clocked two times slower than the GPU and four times slower than the CPU), which is only accessible through DMA.

Overall, this means that while ARAM provides a considerable amount of RAM, it will be limited to less critical tasks, like acting as an audio buffer or being used by certain accessories (explained in the I/O section).

Making the most of ARAM

So far, we've seen that, on paper, the memory capabilities are undoubtedly superior to its predecessor, but there's still room for improvement. For instance, Nintendo could've fitted more hardware to incorporate ARAM into the CPU's memory map.

On a related note, let's revisit the MMU used in Gekko. The CPU, with its 32-bit address bus, can access up to 4 GB of memory, but the system houses nowhere near that quantity. So, to prevent exposing unpopulated (and unpredictable) memory addresses, 'virtual memory' addressing is activated by default to mask physical addresses with a safer, easily-cacheable and continuous 'virtual' address map .

To make this work, Gekko (and other PowerPC architectures) translate virtual addresses to physical ones with the following process:

1. Perform Block Address Translation (BAT): There are eight pairs of programmable registers (four for data and four for instructions) where each pair map a range of virtual address to a continuous range of physical addresses. The MMU attempts to find the physical address if it's found within those ranges.
2. If BAT didn't work, read the Page Table: The MMU also stores a table that catalogues the physical location of pages (block of virtual addresses).
   • The MMU can take time to read a page table, so a Translation look-aside buffer (TLB) is included to cache recent reads.
   • Other architectures such as x86 or MIPS provide paging as well, though not all of them will offer a TLB.
3. Finally, if the requested virtual address can't still be translated, then the MMU triggers a 'page fault' exception in the CPU and lets the operating system decide what to do next.

So what use does this have for developers? Well, it turns out Nintendo released some libraries that extend main RAM using ARAM with the help of paging. To recap, ARAM is not addressable, but the CPU may call DMA to fetch and store data from there. Thus, the CPU can move pages out of main RAM to make room for other resources and temporarily store them in ARAM. Afterwards, whenever a page fault occurs, the OS contains routines to look for the missing pages in ARAM and restore them to their original location in main RAM.

In conclusion, with some clever tricks, these general-purpose capabilities enabled GameCube games to enjoy more memory than technically allowed, thereby reaching higher levels of quality. However, it's important to bear in mind that such tricks may come with some performance penalties (especially if they're taken for granted).