Everything you've just read has to be protected somehow against 'unauthorised' access. If you want to know an overview of how Sony carried out that, you are in for a treat.
Security foundation overview
Many parts of the console already provide security features that don't require any manual implementation in software:
- SysCon, the obscure proprietary chip (briefly mentioned in the boot process), controls the power lines of Cell, RSX and Southbridge. Its EEPROM contains records read by the operating system's modules to determine which functions are enabled and which are not .
- Though I use the word 'obscure', SysCon is just a microcontroller, either an off-the-shelf ARM7TDMI-S (that's right, the PS3 shares some of its DNA with the Game Boy Advance and even late PS2 revisions) enhanced with MagicGate support, or a custom NEC 78K0R variant . SysCon's internal firmware is what intrigues the most.
- SysCon and Cell communicate to each other using a serial interface (SPI) which plugs to Cell's TEST component . TEST provides many debugging functions on Cell, although SysCon only connects to the 'Pervasive logic' port, enabling SysCon to manage areas like power or thermal .
- Cell houses a hidden ROM that stores unencrypted boot routines without fear of prying eyes.
- Cell's privilege modes and SPE's isolated mode prevents programs from accessing unauthorised resources.
- The Southbridge seamlessly encrypts the hard drive's content using AES.
- The Blu-ray subsystem is another walled fortress, and its disc content is encrypted using a key found in the 'ROM mark' area of the disc (inaccessible by conventional readers) .
On top of this, Sony implemented the following protections in software:
- A complex Chain of trust that starts with Cell's unencrypted boot ROM and ends with a graphical user interface (XMB) that only loads encrypted binaries (by Sony) under a kernel and hypervisor.
- The chain of trust implements multiple encryption algorithms, including asymmetric ones like RSA and ECDSA and symmetric systems like AES; combined with HMAC and SHA-1 (to confirm the integrity of data).
- Some encryption keys are produced during manufacturing, meaning if hackers find and leak these keys, they will not work on other consoles. Though this comes at a cost of Sony not being able to patch software encrypted with those keys once the console leaves the factory.
- These special keys are used for
bootldrandmetldr(the early boot stages).
- These special keys are used for
- Games must call the kernel to access the hardware, which in turn asks the Hypervisor. This 'abstraction onion' prevents game exploits from escalating privileges, in theory.
Defeat
You've seen how much this console is capable of, did you expect hackers to settle with the limited features of OtherOS? I guess Sony didn't either, the company tried hard to protect some areas, while leaving others half-closed, as hackers will later demonstrate.
Let's take a look at how some of its strongholds were cracked by independent hackers across the world, bear in mind the PS3 hacking community was very active, with many tools and documentation produced every year. So, I'm going to focus on a few milestones that pave the way to an influx of content and homebrew development, but you can find more info at PS3History .
Hypervisor bypass
In 2010, after three quiet years in the hacking scene, the community took a turn for the best. George Hotz, a hacker known for previously unlocking the first iPhone model (a.k.a the '2G') so it can work with any network (originally only on Cingular/AT&T), managed to read and write protected areas in memory without being stopped by the Hypervisor. He then published his exploit along with a short summary in his blog .
The exploit requires two materials: A Linux installation running under OtherOS (for arbitrary, yet limited, code execution); and an external glitcher connected to the XDR bus (interfacing main RAM). To make a long story short, the hypervisor uses a hash table stored in main RAM to catalogue memory addresses along with their privilege levels, so user programs can't access protected memory spaces. The attack works by breaking the integrity of such table to be able to write over it, and then use that privilege to modify the entries to grant the current program access to every corner in memory.
In summary, Hotz discovered that under Linux/OtherOS, programs can request the Hypervisor many blocks of memory pointing to the same physical address, but if the program deallocates them while there's an external interference in the XDR bus (due to a glitcher sending electrical pulses), the deallocation process ends half-done . As a consequence, the hypervisor's hash table (residing in RAM) still contains an entry of the allocated addresses, but at the same time, it thinks that that space has been freed. Hotz' exploit then proceeds to request more blocks so the Hypervisor extends its table with more entries, and the process continues until an entry of the hash table overlaps the memory location of the block that was supposed to deallocate. Since the hash table kept the old entry granting the user access to that address, the hypervisor ends up giving the user access to modify a hash table entry! Thus, the exploit amends the entry to extend access to all memory space.
While this exploit required a Linux-running-under-OtherOS environment, it was a huge step towards further reverse-engineering and research projects, since hackers were now able to investigate critical areas of the system that were originally inaccessible. It's worth mentioning that, during the same time, Sony released software update 3.21 removing OtherOS. You would think that this would deter hackers from continuing their work, but it just gave them more reasons to speed it up.
PSJailbreak
Later in 2010, a group identified as 'PS Jailbreak' announced (and later released) a unique solution to run Homebrew directly from the console's native shell (XMB, under GameOS) without requiring to tamper with the console's hardware. All of this at the dismay of Sony, who will soon take the legal route to block sales of that product.
'PS Jailbreak' consisted of a USB dongle that was plugged into the front USB port before the console was turned on. The user would then have to press the power button and shortly after, the eject button. If the instructions were carried out successfully, users would then see the normal XMB interface but with the addition of an 'Install PKG' option and a couple of homebrew apps made to dump Blu-ray games into the hard disk, and then load them.
Behind the scenes, this dongle carries out a huge amount of work, which can be divided into two groups :
- The USB exploit: Once the console is turned on, the dongle tricks the system into thinking it's connected to a six-port USB hub, and then performs a complex sequence of USB commands until it reaches a heap overflow and escalates access to the PS3's Kernel (level 2), it then proceeds to execute a payload.
- The Payload: this is another complex package that patches the original shell to enable hidden functions only available on debug units (i.e. the 'Install PKG' entry), disable signature verification (to load any arbitrary module/package); and redirect Blu-ray commands to the hard drive instead (for loading games from the hard drive). The fact this program can alter so much from the kernel level makes you wonder what's the hypervisor good at?
- To complement this, I was later told by M4j0r: "Interestingly it doesn't even exploit Sony's code, this part of lv2 was written by Logitech and the developers of that exploit might had access to the source code (due to the 2008 hack)." .
This product was subsequently reversed-engineered by other communities and shortly after, open-source clones appeared (i.e. PS Groove) which removed many restrictions (for instance, users could now unplug the device after the exploit finishes). Some forks were even deployed on a Texas Instruments calculator . In any case, Sony acted quickly with software update 3.42 to remove this gold mine , although the door for Homebrew was already opened.
Honorable mentions
Before I finally talk about the grand prize of the PS3 homebrew scene, let me describe to you a couple of methods that were developed around the same time:
- The USB Jig: another USB stick, this time programmed to trick the console into entering Factory Service Mode, which is only intended for servicing the console by authorised personnel. The program embedded in the Jig replicates what Sony provides to their engineers. The main advantage of service mode is to enable to downgrade the console to a PSJailbreak-compatible version. The payload was also available in the form of a Homebrew app for the PSP . Sony responded by patching service mode to make it harder to restore it to 'normal' mode or alter the firmware from it, discouraging users from resorting to Service mode.
- The Optical Drive Emulator (ODE): a series of hardware products that different companies (Cobra, E3, etc) shipped. Instead of tampering with the console's firmware, these tampered with the Blu-ray's SATA/PATA interface. ODEs are boards that sit between the motherboard and the Blu-ray drive, thereby acting as a middleman that tricks the console into thinking it contains a valid disc game, but it's instead loading a disc image from an external USB drive. The hacking history of the PS3 contains long gaps of 'unhackable periods' where there was no software exploit available for new consoles. So, at a hefty price tag, ODEs came to fill that gap.
- The Downgrader: as Sony kept mitigating exploits with more software updates, users were left with no viable options but to downgrade to an exploitable firmware. Thus, there were companies like E3 that shipped specialised equipment that could overwrite the console's system the 'hard way'. That is, by directly flashing the NAND or NOR chips. For obvious reasons, this method required more skill and patience compared to the USB-based ones.
- Isolated leaks: this one is for research purposes, as opposed to a 'feature' the user will see (but imperative for further developments, nonetheless). Anyway, the revocation data (used to blacklist compromised certificates) is parsed by
lv2ldr, so far so good? Well, it was discovered that this process contained many vulnerabilities. Firstly and for some unexplained reason, revocation data is writable in userland. Secondly, the parser doesn't perform bounds checking on the data fetched (here we go again). Therefore, hackers managed to craft custom revocation data that could produce a buffer overflow and ultimately enable them to run arbitrary code inside the SPU's isolated mode. This allowed them to get access to confidential data (i.e. keys) that was presumably protected from the rest of the system .
The fall of encryption
Just like the PSP saga, the initial exploits required a lot of effort and could be easily patched by Sony, thereby resulting in a disadvantageous cat-and-mouse game. However, as it happened with the PSP, it was a matter of time before a discovery broke the fundamental security of this system: its chain of trust.
In 2011, George Hotz (along with the fail0verflow team) published another breakthrough, the private encryption keys that Sony uses to sign binaries executed by metldr. Binaries loaded during that boot stage are signed with an ECDSA key. Being an asymmetric encryption system, this means that anyone that possesses the private key (Sony, and now others) can encrypt and sign binaries, consequently, they will look 'authentic' in metldr's eyes. Since metldr is the third boot stage before loading lv1 (the hypervisor), this means hackers will be able to customise or develop their own hypervisor, kernel and anything underneath it. On top of that, every PlayStation 3 in the market will also think the custom binaries are authentic. All in all, a Pandora-style exploit completely done with software.
The discovery of this key, which should have been computationally unfeasible, is possible thanks to what is considered a 'blunder' in Sony's implementation of the ECDSA algorithm. To make a long story short, the mathematical formula used for ECDSA uses a random value which Sony never changed in all the update files they distributed , turning that number into a constant, thereby making it easier to solve the other variables, which is what happened eventually.
The effects of this discovery are described in the next paragraphs.
The Custom Firmware (CFW) era
Cracking metldr meant everyone was now able to create 'official' systems for the PS3, this resulted in an influx of GameOS 'flavours' which different communities produced with various customisations. These systems were modifications of Sony's official firmware files (that Sony distributed as updates) and re-packaged using Sony's leaked keys, so they could be installed anywhere. The result was called Custom Firmware (CFW) and became the de-facto method for hacking this console, that is until Sony responded with tough measures.

My CFW installation with 'VSH menu' opened. This variant (called 'Rebug') also enabled me to turn my console into a debugging station (notice the IP address at the bottom right corner, you need to enter it on the debugger to attach to a running process) and fiddle with my own homebrew.
In the meantime, many CFW appeared on the net with many names (i.e. 'Rebug', 'Ferrox', etc) and they contained customisations such as :
- Disabling signature verifications on any module installed or to be installed.
- Enabling to read and write (the classic peek and poke) over any memory address, either using the Hypervisor (level 1) or the Kernel (level 2).
- Activating hidden debug functions to install modules packaged as 'pkg' files. These didn't need to be signed with Sony's keys to work inside a CFW environment.
- Enabling to mount disc images as a virtual Blu-ray disc.
- Restoring OtherOS and even enhance it by removing the restrictions imposed by the Hypervisor. The result was called OtherOS++.
- Writing over Syscon EEPROM's database to enable to install any system version of choice. This is also known as QA Toggling.
- Altering the style of XMB (i.e. removing the epilepsy warning, allow to take screenshots in-game, etc).
There's also my favourite one: bring the debugging functions of a testkit, allowing any retail console to become a debugging station. This could be done either by installing a CFW with debugging capabilities, or a CFW that could convert the retail console (called 'CEX') into a debugging model (called 'DEX') by altering console-specific data in Flash memory.
Sony's strong response
Similarly to the events that happened after CFWs were invented for the PSP, Sony retaliated with two security updates:
From the software side, Sony shipped two system updates that enhanced the security system:
- With
3.56, binaries are signed with new encryption keys resilient to the previous ECDSA discovery , thus, CFW creators can't customise the new binaries (since they don't have the private keys to re-encrypt them). Furthermore, a new revision of the 'system updater' application is also shipped, this enforces the new certificates in system update files (PS3UPDAT.PUP), meaning that even if hackers manage to package a new CFW, only consoles with system version3.55or lower will be able to install it . - Later on, system update
3.60revamped the boot process, it nullifiedmetldrand promotedlv0to take over in bootstraping the loaders (lv1ldr,lv2ldr,appldrandisoldr). All in all, this meant hackers could not modify the new system files without first crackinglv0(finding its private key).- This eventually happened in late 2012, when a team called "The Three Musketeers" published the lv0 keys , which paved the way to new CFWs made from system versions newer than
3.55. Although, due to the aforementioned changes in the updater, only users on system version3.55or lower (including any CFW with signature checks disabled) can install it.
- This eventually happened in late 2012, when a team called "The Three Musketeers" published the lv0 keys , which paved the way to new CFWs made from system versions newer than
From the hardware side, not only subsequent PS3 models (late CECH-25xxx, CECH-3xxx and CECH-4xxx) came pre-installed with a system version higher than 3.55, but they also contain a different variant of bootldr/lv0ldr (called lv0ldr.1) that not only decrypts and loads lv0 but it also fetches a new system file called lv0.2. The latter contains metadata about lv0 to ensure that lv0 hasn't been tampered with. lv0.2 is signed with a new key (also invulnerable to the previous ECDSA discovery), thus, preventing hackers from taking control of the boot chain.
To this day, these models are not able to run a CFW, therefore nicknamed unhackables. Though they can run a 'Hybrid Firmware' (HFW) and we'll discuss more about it later on.
As time went by, the number of CFW-compatible consoles only decreased. Hence, PS3s that weren't updated past 3.55 became some sort of relics. In the meantime, there was a surge in demand for alternatives, like downgraders (to revert to system version 3.55 on old models) and ODEs (to play pirated games on new models).
Homebrew revival
After a long waiting period for users that missed the window to install a CFW, in late 2017, a team of hackers released PS3Xploit, a collection of exploits and utilities that brought back the ability to install a CFW on old models without needing an expensive downgrader (and skills to operate it).
PS3Xploit's main payload replicates the job of a hardware downgrader (patching CoreOS files) entirely by software, it works as follows:
- The starting point is the XMB's internet browser, built on top of WebKit. PS3Xploit uses JavaScript to gain arbitrary code execution within the system's userspace (and outside JavaScript's environment). To kickstart this, users only have to open XMB's native web browser, enter an URL pointing to the PS3Xploit's host an let it do its job.
- It so happens the kernel provides system calls that can be used to overwrite the operating system's files in Flash memory. On top of this, the Visual Shell (XMB) and its plugins store routines in memory that make use of those calls.
- PS3Xploit can't trigger those system calls directly due to the Hypervisor's 'no-execute' protection, preventing the exploit from loading new code in userland. However, it can find a way to overwrite Flash memory by 'borrowing' Visual Shell's routines.
- Consequently, PS3Xploit proceeds to modify Webkit's execution stack to redirect execution to Visual Shell's routines. This type of technique (corrupting the stack to deviate execution to other code residing in memory) is called Return Oriented Programming (ROP) and it's very popular in the InfoSec genre. One way of mitigating this is by implementing Address space layout randomisation (ASLR), which makes it difficult to guess the location of the routines (called gadgets) but, as you can guess, Sony's hypervisor lacks ASLR.
- Finally, those system calls are triggered with PS3Xploit's parameters and so they replace CoreOS files (the first part of the operating system, stored in Flash memory) with patched ones .
- The console is now able to install unofficial software updates, an opportunity the user can now exploit to install a custom firmware. However, it can't downgrade the system version, yet, but once an up-to-date CFW is installed, the user can install further utilities to downgrade the system and install a better-equipped CFW, if he/she so wishes.
As you can see, this gift from the sky brought custom firmwares back into the spotlight and rendered hardware downgraders and ODEs obsolete. On the other side, for those units which couldn't install a CFW either way (the unhackables), the team later offered PS3Hen, a different exploit package that focused on enabling a subset of CFW functions (including the ability to execute homebrew). This one installs itself as an entry in XMB and the user must run it every time they power on their console to re-enable the execution of homebrew apps.
Sony's partial response
As luck would have it, Sony only took small steps to block PS3Xploit (maybe because this turn of events happened years after the PS3's successor, the PlayStation 4, hit the stores). They released a few system updates that didn't fix this chain of exploits but removed the routine used in Webkit for bootstrapping the chain. In response, hackers published slightly modified software updates that restored such entry (and somehow, they didn't need to be re-signed) . These custom updates were called Hybrid firmware (HFW) and at the time of this writing, they are the de-facto option used to enable homebrew on unhackable systems.
And here concludes the anti-piracy/homebrew saga. In my humble opinion, I don't think Sony is interested in putting more effort into this console. So I wouldn't expect any more cat-and-mouse games in this field.