If you thought that Cell, with all its quirks, could take care of every task of this console, then let me tell you something hysterical: Sony fitted a separate chip for 3D graphics.
Uncharted 3: Drake's Deception (2011).
The Elder Scrolls V: Skyrim (2011).
Killzone 3 (2011).
One Piece: Pirate Warriors (2012).
It appears that even with a supercomputer chip, Sony still had to fetch a GPU to finalise the PlayStation 3. This makes you wonder if IBM/Sony/Toshiba hit a wall while trying to scale Cell further, so Sony had no option but to get help from a graphics company.
We had started this ICE [Initiative For A Common Engine] team with the intent of developing some core technology that could be shared amongst all the first parties (...) For a while, [PS3 had] no GPU, it was going to run everything with SPUs. The ICE team proved to Japan that it was just impossible. It would be ridiculous. Performance-wise, it would be a disaster. That's why they finally added the GPU, closer to the end .
- Unnamed sources from Naughty Dog
What I am sure is that the PS3 contains a GPU chip manufactured by Nvidia meant to offload part of the graphics pipeline. The chip is called Reality Synthesizer or 'RSX' and runs at 500 MHz . Its clock speed looks concerning when compared to Cell's (3.2 GHz), though you'll soon see that the GPU is better equipped for computing huge amounts of operations in parallel. So it's a matter of finding a balance between Cell and RSX when it comes to building the graphics pipeline (though I must confess this sounds simpler on paper than it is in practice).
I will now perform the same level of analysis previously done with Cell, this time focusing on RSX and its graphics capabilities.
Overview
It's been five years since Nvidia debuted the GeForce3/NV30 lineup in 2001, and by then the arena was battled by strong players like 3dfx, S3 and ArtX/ATI. Though in subsequent years, the number of companies slowly reduced to the point that by 2006, only ATI and Nvidia remained as the flagship video card suppliers in the PC market.
RSX chip next to Cell.
The RSX inherits existing Nvidia technology, it's reported to be based on the 7800 GTX model sold for PCs, which implements the GeForce7 (or NV47) architecture , also named 'Curie'.
In my previous Xbox analysis, I talked about the GeForce3 and their debuting pixel shaders, so what has changed since then? There have been some ups and downs, but mostly incremental changes, so nothing too groundbreaking compared to GeForce3's pixel shaders.
On the other side, while the 7800 GTX relies on the PCI Express protocol to communicate with the CPU, the RSX has been remodelled to work with a proprietary protocol called Flex I/O , a distinct interface within Cell designed to connect to neighbouring chips. Flex I/O operates in two modes:
- BIC mode for connecting other Cell processors (to be used in multi-processor environments).
- The slower IOIF mode for connecting up to two peripherals, a 'fast' one and a 'slow' one.
Alas, the RSX is not Cell, so it goes through the IOIF protocol, using the fastest slot.
For comparison purposes, IOIF behaves as a 32-bit parallel bus with a theoretical bandwidth of up to 20 GB/s, while the PCI-express used in the 7800 GTX (x16 1.0) is a 16-bit serial bus with a theoretical bandwidth of up to 4 GB/s.
Organising the content
RSX has 256 MB of dedicated GDDR3 SDRAM at its disposal. Surprisingly, it's the same memory type found in the Wii. The memory bus runs at 650 MHz with a theoretical bandwidth of up to 20.8 GB/s.
Example of how data is organised across the memory available. Notice how RSX can access its content from different memory chips.
Inside those 256 MB, Cell can place everything that RSX will need to render a frame. That includes vertex data, shaders, textures and commands. Now, thanks to Cell's Flex I/O bus, RSX can also utilise the aforementioned 256 MB of XDR memory (CPU's main RAM) as working space, though this will come with some performance penalties. This comes in handy if the rendered frame will be post-processed by an SPU, for instance.
As you can see, while this console didn't implement a UMA architecture, it can still distribute graphics data across different memory chips if programmers decide to do so. I mention this because I wish many 'technical explainers' would read more about this feature before shouting over-summarising statements like "The PS3 was limited because it didn't have UMA". That may be true in certain cases, but unless they mention these, that generic claim is, in my opinion, misguiding.
Finally, RSX supports many forms of data optimisation to save bandwidth, examples include 4:1 colour compression, z-compression and 'tiled' mode (I'll explain more about it later on).
Constructing the frame
Let's now take a look at how RSX processes and renders 3D scenes.
Pipeline overview of the RSX.
Its pipeline model is very similar to the GeForce3, but super-charged with five years of technological progress. So I suggest checking out that article beforehand since this one will focus on the new features, I also recommend reading about the PlayStation Portable's GPU because a lot of new developments and needs overlap with that chip. That being said, let's see what we've got here...
Commands
Diagram of the command stage.
As with any other GPU, there must be a block of circuitry in charge of receiving the orders from outside. With RSX, this is handled by two blocks, Host and Graphics Front End.
The Host is responsible for reading commands from memory (either at local or main) and translating them into internal signals that other components in RSX understand, this is done with the use of four sub-blocks:
- Pusher: fetches graphics commands from memory and interprets branch instructions. It also contains 1 KB of prefetch buffer. The processed commands are sent to the First In, First Out (FIFO) Cache.
- FIFO Cache: stores up to 512 commands decoded by the Pusher in a FIFO manner to provide quick access.
- Puller: as the name indicates, it pulls commands from the FIFO cache whenever RSX is ready to render and sends them to the next unit.
- Graphics FIFO: stores up to eight commands that will be read by the Graphics Front End.
The Graphics Front end then reads from the Graphics FIFO and signals the required units inside RSX to compute the operations. If you remember, this is equivalent to the 'pfifo' in the GeForce3.
As you can see, commands and data pass through many buffers and caches before reaching the final destination. This is intentional, as it prevents stalling the pipeline due to different units and buses operating at different speeds. So, cached memory takes advantage of fast bandwidth whenever it's possible.
Vertex Shader
Diagram of the vertex stage process, notice that the Vertex Processing Engines (VPE) are bypassed if the vertices don't need further processing by the vertex shader.
The next unit is the Geometry Processing block, an evolution of the 'Vertex Block' in the GeForce3 that performs vertex transformation. It's still programmable with the use of vertex shaders, which is now a widely adopted feature in the graphics industry. Furthermore, the instruction limit has been increased to 512 instructions minimum (originally, 136 was the limit!).
The block that executes shaders is called Vertex Processing Engine (VPE) and it can process one vertex per clock cycle. As if that wasn't enough, there are eight VPEs working in parallel. Since the GeForce6 series, Nvidia has aligned its shader programming interface to a model called 'Vertex Shader Model 3' or 'vs_3_0_', a standard developed by Microsoft for use with their DirectX 9.0c libraries . The VPEs also support the non-proprietary OpenGL 2.1 model and Nvidia's own variant (Cg) .
Compared to the GeForce3, we've got new instructions available for branching and subroutine calls. Also, the VPE contains four texture samplers that pull texture colours during this stage, in case programmers want to use this unit to perform some operations on them.
The Geometry Processing block works like this:
- The Index Vertex Processor (IDX) fetches and caches vertex data and textures from Video RAM (VRAM). Afterwards, it sends the data to the VAB.
- The Vertex Attribute Buffer (VAB) pulls data from the IDX cache and redirects it to each VPE.
- Each VPE processes the data based on the shader loaded. It computes one shader instruction per clock.
- The result of each VPE is sent to the Post Transform Cache, which caches results to skip identical computations over the same vertex. This only applies if vertex indices are used instead of vertex data.
- The final result is stored in the Viewport Cull Unit (VPC), which applies scissoring to discard vertices found outside the viewport; and the Attribute RAM (ATR), which caches vertex attributes (texture, colour, fog, etc) to be read at the next stages.
Rasterization
Simplified diagram of the rasterization stage. RSX embeds different units to calculate values used for interpolation of pixels and colours.
Moving on, it's time to convert (rasterize) vertices into pixels. The RSX's rasterizer is quite fast, it can rasterize up to 8x8 pixels (64) per cycle and works with frame-buffers of up to 4096x4096 pixels (though developers may need less than that).
The rasterizer accepts points, lines (including strips and closed types), triangles (including strips and fans), quadrilaterals and regular polygons. Naturally, as with this console generation, the rasterizer works with sub-pixel coordinates, where the sampling points are half-coordinates (0.5) of pixels. This allows the unit to apply anti-aliasing methods such as Multisampling afterwards. Multisampling consists in rasterizing the same geometry multiple times but shifting a few sub-pixels apart at every batch (the RSX supports four different shifting modes) and then computes an average. This results in a smoothed image.
Furthermore, this unit also performs z-culling using dedicated RAM within the RSX (it has a capacity of around three million pixels). This saves processing pixels and stencils that have already been rendered and allows to perform early z-test on incoming geometry.
A separate unit is used for rasterizing 2D objects (sprites), although this one is isolated from the 3D pipeline. Consequently, RSX works in two modes, 2D and 3D, but intermittently switching between them is expensive in terms of performance.
Pixel Shader
Pixel/Fragment stage diagram.
Next in line we've got the Fragment Shader & Texture block, this is a programmable unit (through the use of 'fragment programs' or 'shader') that applies texture mapping and other effects.
Being an advanced successor of GeForce3's texture units, the new block contains six fragment units (also called 'pipes'), each one processes 2x2 texels (named 'quads'). To organise multiple units working concurrently, another sub-block called Shader Quad Distributor (SQD) is placed to dispatch quads to each fragment unit. Afterwards, each fragment unit loads the fragment program.
To compute operations, each pipe contains the enormous amount of 1536 128-bit registers. On top of all that, each pipe can process multiple quads in parallel (multi-threading), though the number of quads processed in parallel depends on the number of registers allocated to the fragment program (no. threads = 1536 ÷ no. registers reserved by shader). In global terms, up to 460 quads can be processed in parallel. Furthermore, up to three fragment pipes can process two instructions at the same time (dual-issuing, like the PPU) as long as the instructions don't depend on each other.
The fragment units provide similar arithmetic-type instructions to the vertex unit, with the addition of texture-related opcodes, such as multiple types of texture fetching (since textures can be encoded using many structures and then compressed) and unpacking. Similarly to the vertex block, the fragment shader abides by DirectX's Pixel Shader 3.0 model , OpenGL's NV_fragment_program2 profile and Cg's 'fp40' profile . All of this to ease programming and avoid learning low-level APIs from the ground up.
Finally, since the units will be constantly fetching texture pieces from video RAM or main RAM, this block includes three texture caches: 4 KB of L1 cache for each pipe, 48 KB of L2 cache for video RAM fetches; and 96 KB of L2 cache for main RAM. Notice that main RAM cache is significantly larger, this was a conscious decision made to compensate for higher latency.
Pixel operations
Post-processing stage.
Before writing the results into the frame-buffer (stored either in VRAM or Main RAM), a final block called Raster Operation Block (ROP) performs final tests on the resulting pixels.
There are two sets of ROPs made of four blocks each (eight in total). Each group performs z-testing, alpha blending and the final write to memory. All in all, this circuitry can process up to 16 z-values and 8 pixel colours per clock. Curiously enough, the PC-variant Nvidia 7800 GTX features 16 ROPs instead of 8, maybe this cut was done to prioritise memory bandwidth consumed by the SPUs?
To save further bandwidth, ROPs also provide colour compression and z-compression. Additionally, there's a Tiling mode available for optimising memory access by the video encoder. In tiling mode, the frame buffer is stored into continuous 128 B blocks ordered in the same way they would be broadcasted/scanned. Thus, saving the GPU from doing page swaps (used for memory addressing) while transmitting the frame buffer for display, consequently improving bandwidth. These 'tiles' are stored in marked locations in memory exclusive for this type of addressing.
A unified Video Output
Gone are the days of console-proprietary video sockets and dozens of analogue signals squashed together in a single socket to accommodate every region on the earth. The PlayStation 3 finally incorporated a unified video signal soon to be adopted worldwide: the High Definition Media Interface (HDMI), used for transferring both audio and video at the same time.
Back of the PS3, HDMI output on the left side and at the other extreme there's the old Multi A/V for analogue video out.
The HDMI connector is made of 19 pins , all in a single socket. It transfers a digital signal, meaning that the picture and audio are broadcast using discrete zeroes and ones (and not a range of continuous values found in analogue signals). Consequently, it doesn't suffer from the interference or image degradation that previous equipment did, such as screen artefacts produced by cheap SCART cables.
To this day, the HDMI protocol is continuously being revised , with new versions of the specification offering more features (i.e. larger image resolution, refresh rate, alternative colour spaces, etc) while retaining the same physical medium for backwards compatibility.
Throughout the PS3 lifecycle, Sony added certain HDMI features of new revisions to the PS3 through software updates . The last protocol compatible with the PS3 is version 1.4, which most notably brought support for '3D television'. Although, other capabilities, such as higher video resolutions, remained capped at 1920x1080 pixels, with most games rendering their frame buffers at just 1280x720 pixels.
'Real' 3D vision/projection
So what was that '3D television' I mentioned before? Well, it so happens that the lifetime of this console overlapped with a short-lived fever for 3D tellies (the so-called 3DTV) . To support these, Sony updated their Software Development Kit (SDK) to assist the rendering of stereoscopic frames in RSX and implemented the '3D specification' in their HDMI encoder. What's happening behind the scenes is that the encoder broadcasts two frames at a time, and the television alternates them similarly to what the Master System's 3D glasses used to do 30 years before.