At first, I had assumed the Wii U implemented its audio functions through software (like its competitors started doing). But then, other questions arose: How is the complex DSP of the Wii emulated? Is Espresso powerful enough?
Well, that's not all. Unlike the Wii, the audio pipeline of the Wii U must service three endpoints:
- The Television, expecting up to 6 channels (typically only 2/stereo).
- The GamePad with up to 4 channels (a.k.a surround audio).
- Up to four Wii Remotes (with one low-quality channel each).
Lo and behold, Latte also houses a custom DSP that performs audio synthesis. It's operated through Nintendo's libraries, although developers can extend the audio pipeline through software (at the cost of CPU cycles) if it's not enough for them.
Overview of the audio pipeline. In this example, the game is loaded from the disc drive.
The DSP helps with mixing and sequencing tasks, the rest (i.e. filtering, effects, etc) is done with software (part of Espresso's workload). Moreover, Nintendo's libraries take care of workload management, where audio tasks are first assigned to the DSP and, in the event of full capacity, these are sent to Espresso.
The amount of workload and the resulting performance will depend on many factors, including the type of encoding streamed (i.e. ADPCM takes up more resources than PCM). Programmers must first test their code using the audio profiling tools provided to come up with an efficient implementation.