Scanlime's debugging setup with new FPGA

Sorting through the data we get from this setup is still a considerable challenge. Here’s a trace taken while the video camera is actually capturing video:

http://dl.getdropbox.com/u/1926728/dsi/camera-trace-20090914.raw.bz2

There’s some code for decoding this trace format in scanlime’s svn repo: http://svn.navi.cx/misc/trunk/nds/dsi/ram-tracer/decoder/

If you’d like to play along, see if you can distinguish between:

• Instruction fetches from RAM
• Reads/writes to RAM buffers (statically or dynamically allocated) by code running on either processor
• Reads/writes to control flags, used for e.g. synchronization between the ARM7 and ARM9
• DMA writes from the camera hardware to RAM of the video data

The video data makes up the vast majority of the data in this dump; if you’re working on homebrew code to talk to the camera, this might be helpful. For the rest of you — can you make a tool to visualize the data flows in these traces, or a tool to decode the video frames in scanlime’s dump?

There’s also a hidden message in the video =)

I personally haven’t had much luck with my DSi.  I tried to dump the flash on it, and managed to blow a fuse in the process (it’s hard to keep that battery aligned with the case removed…).  I can’t run any of the savegame hacks, because there are no DSi-mode cartridge-based games for the Japanese DSi yet.  I decided to get a bit more aggressive and see if we could sniff the RAM:

bushing's DSi with RAM breakout

This didn’t actually end up working out so well.   The thing stopped powering on, and well, yeah.

I did have a little more luck dumping the SPI flash onboard the WiFi dongle:

dumping SPI flash with GoodFET

I used Travis Goodspeed‘s awesome GoodFET device (thanks Travis!).  Some of you may remember FlashMe on the older Nintendo DS — this was the program that let you reflash the “firmware” that was stored on the SPI flash on the WiFi daughtercard to e.g. bypass RSA verification of Wireless MultiBoot games.  The DSi has the same chip in the same place, but it only holds a small amount of configuration data, with the “firmware” being stored elsewhere.  Bummer.

Meanwhile, at least two people have successfully dumped the 256MB MMC NAND flash; in at least one case, an old SD card reader was just soldered to 4 vias on the PCB! We’ve learned that there is approximately 1MB of that flash chip dedicated to the “firmware” used to boot the DSi — more on that below — and that second-stage loader is encrypted with a key that is shared among all DSis and stored in ROM (much like boot1 on the Wii).  Of that 1MB, about 300K is actually used.  The remainder of the NAND flash is encrypted uniquely per console, so as with the Wii, you can’t take the contents of one flash chip and boot it on another console.  (However, like with the Wii, it IS possible to back up your encrypted flash, upgrade or modify it, and then reflash back your old image to restore the old state.  The nice part is that it only requires 4 wires to do so on the DSi, as compared to 16 on the Wii.)

Once it became clear that something was seriously wrong with my DSi and it was never going to be useful in its current form, it gave its life to science.  From that, we were able to find points to solder to all of the address, data and control lines on the DSi’s Pseudo-SRAM chip without removing any chips, and a promising new member of our close-knit community was able to successfully wire up a working DSi to an FPGA:

scanlime's RAM tracing setup

scanlime really does some excellent work; through a bit of EE cleverness, he was able to slow down the clock of the DSi enough such that he could sniff all of the RAM traffic and dump it over USB to his computer for analysis.

We have no grand master exploit, but have learned the following things:

• There is a considerable amount of ROM (128K+?) and RAM (1MB+?) inside the CPU
• The internal ROM is quite sophisticated, compared to that of the Starlet (boot0) — it is able to initialize both LCD panels, read from the SPI flash and the MMC NAND flash, and decrypt the contents of the 2nd-stage bootloader from NAND into internal RAM.  If there is an error, it can display an error code on the top LCD.
• The second stage bootloader is analogous to boot2 on the Wii — it can read the TMD for the System Menu from the NAND filesystem and load the contents into memory.   Like the Wii, the code seems to be stored in NAND unencrypted (inside an encrypted filesystem).  Unlike the Wii, it seems to actually verify the contents and signature of the TMD before executing it.
• Most of the interesting keys seem to be stored inside internal RAM, safely out of reach from us.  They are cleared when a cartridge is loaded, and probably even when a DSiWare app is loaded.

While this is a tremendous boon, there are still many challenges ahead.  We have not yet found keys to decrypt most of the system software, and even finding those will not be enough to actually let us execute code; that will require a further exploit.  Furthermore,  we are currently drowning in data — a full RAM trace from scanlime’s setup from startup to installation of a channel is over 10 gigabytes.  We have some software that can convert this into a RAM dump — please don’t even ask me for it — but most of the interesting bits are only stored in memory temporarily. Work is still ongoing to let us analyze the flow of memory accesses and pick out those interesting, ephemeral bits.

In some ways, this is similar to what happened with the Wii — fiddling with the RAM gave us much greater insight into how the system worked, but it still took several months of dedicated reverse-engineering before we had a usable exploit. Maybe this one will be faster, or maybe Nintendo will have learned a thing or two from their previous mistakes.

• Wii Hardware: a history
• IOS: history, build process

A few months back, we started getting reports of “unsoftmoddable Wiis”, aka “LU64+” (among other things). Normally, I wouldn’t care, but we discovered that our HackMii Installer would not work on any of those Wiis. I started making the claim that this was due to an innocuous hardware change, coinciding with the release of boot2v4, but I never really explained why. Here’s my explanation.

“LU64+” is a really silly way to describe these Wiis — it refers to the first four characters in the serial number of the first Wiis seen to be like this — but I can’t really blame people, because the serial number is the only real visible marking as to when the console was manufactured.

Starting around early 2009 (?), the following statements were true about all new purchased Wiis:

1. All versions of IOS have the strncmp() bug fixed, access to /dev/flash blocked, and “ES_Identify” was broken.
2. boot1 had strncmp() bug fixed
3. crap installed into the slots for IOS3, IOS4 and IOS254

Beginning in March, a fairly well-defined set of symptoms of running some kinds of homebrew software emerged on “newish” Wiis:

1. Old (but valid) versions of IOS would not run at all; they would hang if you tried, which prevented downgrading IOS. This did not apply to the leaked IOS16.
2. Attempts at reloading IOS by libogc would sometimes fail; depending on the program, this would look like a hang or would just cause all IOS calls to fail

There seemed to be no difference in the installed software on one of these “newish” Wiis, compared to what you would get by doing an update through the System Menu.
People proposed theories such as “Nintendo added special hack-detection code into boot1 / boot2, but they put a ‘back-door’ in for IOS16.” As I’ve explained in the past, boot1 and boot2 are finished executing long before you see anything on your screen, and certainly don’t affect anything past that. Rather, boot2 is responsible for loading IOS, and IOS is responsible for loading the PowerPC code for the system menu or game. If you insert a game disc that uses a different version of IOS, the currently running IOS is responsible for loading and running the new IOS, which then loads the PPC code, and maybe eventually another IOS to get back to the system menu, etc. (This exact order of steps becomes important later.)

This was a curiosity I mostly ignored, because it didn’t affect any of our software … until we released the HackMii Installer.

The reason we created one common installer for BootMii, the Homebrew Channel and DVDX was that it became increasingly complicated to install software on updated Wiis. Different approaches were required, depending on the configuration of the Wii, and it became apparent that duplicating our decision-making and exploit code across multiple installers would be silly. We ended up coming up with some code that would examine the state of the Wii it was running on, pick an appropriate strategy, and then automatically install the desired software.

There are a couple of different possible paths for the Installer to use — if it can, it will just use an old IOS that still has the unpatched hash comparison function and unfettered access to /dev/flash (reloading to that IOS as necessary). If not, it would choose one of a list of IOS exploits we had found, depending on the versions of IOS installed (each exploit needs to be customized to a specific major/minor version of IOS), and use it to load MINI into memory and execute it. From there, we could directly access the NAND flash and detect the installed version of boot1 and boot2. Finally, having made the decision of whether or not we could allow BootMii to be installed as boot2, we would reload back into a normal IOS so that we could access the WiiMote. Of course, reloading back into IOS isn’t exactly trivial; MINI has a function that launches an arbitrary title from NAND by reading boot2 from the beginning of NAND, patching the call to ES_LaunchTitle(1-2) to the desired title ID, and then executing the patched boot2.

All of this happened automatically and hopefully seamlessly, but it could take a few seconds. Having been burned by the pay-for-homebrew wankers, we have an “anti-scam warning screen” that we make the user sit through before installing our software. The delay there made for a great opportunity for us to do our leet haxIOS magic, so we ran that “in the background” while the user sat through our nag screen.

This didn’t work as well as we had hoped on some new Wiis — suspiciously, it would reboot in the middle of the nag screen on “unsoftmoddable Wiis”. Suddenly this became an interesting problem to me!

Unfortunately, we’re all old farts with old Wiis. None of us could reproduce the problem ourselves – we tried to get people with troubled Wiis to run test code for us, but you really need a USBGecko to be able to get debug output from low-level code. People who were new to the Wii were the least likely to have a USBGecko for testing, so … the only way we could proceed was to track down one of these “unsoftmoddable” Wiis.

I finally did so, and ran a debugging version of the installer on it. I was surprised to see that it was actually getting all the way through our set of exploits, and was actually dying when it tried to reload back to IOS by way of boot2. It turned out that these new Wiis had a new version of boot2 — boot2v4 — and our “patch boot2 to reload IOS instead of the system menu” routine was failing to find the code to patch. Instead of giving an error message, it was just executing the unmodified new boot2 — which then re-ran the system menu instead of proceeding with our installer. Oops.

That was certainly easy enough to fix — I added a patch for boot2v4 — and the installer seemed to successfully reload IOS, only to hang inside __IOS_InitializeSubsystems() in libogc (specifically, ios_open(“/dev/es”)). This started to match the reported failures in libogc programs during IOS_Reload().

svpe said he had run into some timing problems when working on booting back and forth between IOS and MINI, and suggested I try adding a delay. When I added a delay of 5 seconds into our code between when we asked MINI to reload IOS, and when we actually reinitialized libogc, then everything started work. I found this rather odd, since the code had worked just fine with boot2v3, and it also worked on boot2v4 if we reloaded to the System Menu (since that’s what had been happening to people when our boot2 patch failed).

After another sleepless night or two, it became clear that boot2v4 actually took somewhat longer to load than boot2v2/3. If I checked the IOS version number right before calling __IOS_InitializeSubsystem(), it would report back the correct IOS version on boot2v3, but 0 on the boot2v4 system. This indicates that boot2 was still running — boot2 sets that version number to 0 when it starts.

The answer to this problem seemed to be simple enough — we just needed to add some code that would work like the following:

1. Send IPC to MINI asking for it to boot into IOS
2. delay until IOS version number is set to 0 (meaning, boot2 has fully loaded and is now executing)
3. delay until IOS version is the expected IOS version
4. proceed to initialize IOS

Again, doing this made a little difference, but not a good one. Now we no longer froze inside of __IOS_InitializeSubsystems(), but all of our IOS called failed once we booted into the installer — and they failed with strange error codes like “-900074497″. svpe then pointed out that the IOS load process looks something like this:

1. boot2 starts
2. boot2 sets IOS version number to 0
3. boot2 executes es_main()
4. es_main calls ES_LaunchTitle(1-2) (or in our case, IOSxx since we patched this parameter)
5. ES_LaunchTitle recognizes that we are trying to load an IOS, and then sets up the IOS version number in memory so that the newly-loaded IOS will know what version it is
6. ES_LaunchTitle transfers control to the new IOS
7. IOS starts and sets up the IPC registers to talk to the PPC

We were hitting a timing race between when ES_LaunchTitle changed the IOS version number, and when IPC actually started. If we tried to make IPC calls to the Starlet before the version number changed, then we would actually be trying to make IPC calls to boot2 while it was in the middle of rebooting — so we’d never get a response and libogc would hang. If we waited until after the version number was set, libogc would try to talk to the new IOS before IPC was initialized; as part of that initialization, IOS would send back a bogus “ack” message that made libogc think that the ios_open(“/dev/es”) call failed, and then it would never properly initialize ES and all later calls would fail. If we waited in our own code for this “ack” message (as suggested by isobel), then everything else worked.

Armed with this knowledge, I was able to add some timing code to our installer, and get the following numbers:

On old hardware:
Time to load boot2v3: 958 ms
Time to execute through boot2v3 and launch IOS: <1 ms
Time to initialize IPC: <1ms

On new hardware:
Time to load boot2v4: 134 ms
Time to execute through boot2v4 and launch IOS: 178 ms
Time to initialize IPC: 499 ms

Thus, we had our solution … more or less … to the problem of boot2v4 and the HackMii Installer — we just needed to add some delays. So why is the timing so much different on the new Wii, and why doesn’t this affect normal operations of the Wii?

Let’s summarize the facts:

• Wiis recently began shipping with a new boot2v4, which is based on the same code present in the set of IOSes released last October
• Reloading IOS is often “broken” on new Wiis — especially when trying to reload to an “older” version of IOS (pre-October, taken from an older Wii)
• Launching PPC titles works just fine on new Wiis, even if that requires reloading to a different version of IOS (as happens when launching the Homebrew Channel
• The same versions of IOS work differently on older Wiis, and have none of the problems noted above. The only difference in the software loaded on these Wiis is in boot1 and boot2.
• The “leaked IOS16″ seems to be exempt from these problems

I think I can explain all but the last point.

From previous research, we can add the following tidbits of knowledge:

• boot1 and bc have each gone through 4 revisions; of those, 1 was to fix the strncmp bug and 3 were to change low-level hardware initialization routines
• The strncmp bug fix first appeared in IOS37 in March, 2008; IOS, bc, and boot1 all apparently were fixed internally at Nintendo / BroadOn in Feb-Mar. 2008, but not released until a few months later
• bc (and presumably boot1) was rebuilt with more minor hardware init changes in June 2008
• new versions of boot2 and all of the IOSes were built on July 11th, 2008
• The new versions of IOS were put up on the Nintendo Update Servers in October
• Wiis began shipping with the new IOSes (but with boot2v4) in late 2008
• Two PCB revs were made at the end of 2008/beginning of 2009 — all of these Wiis shipped with the “new” IOSes and boot2v4, and do not seem to work well with older versions of IOS
• The only substantial changes made in the PCB were to power-supply circuitry; Hollywood (where IOS runs) was unchanged (still at “Hollywood AA”), and there may have been some inconsequential changes made to the Broadway

A scenario begins to emerge:

1. Nintendo develops a new PCB design to reduce cost by simplifying circuitry in mid-2008; the resulting design has fewer parts but takes a few milliseconds longer for the power supply voltages to stabilize
2. The new hardware requires a more-tolerant IOS kernel, so BroadOn rebuilds boot2 and IOS for Nintendo do use in testing the new PCB design. These IOS versions are still backwards-compatible with C/RVL-CPU-01.
3. Nintendo releases the new IOSes as an update in late 2008, to quash the strncmp() bug (as well as /dev/flash access and ES_Identify()
4. Nintendo starts producing C/RVL-CPU-01 boards with the new IOSes on them. (boot2 is not updated, since there’s no benefit to doing so)
5. Nintendo eventually starts mass-producing C/RVL-CPU-20, which requires an updated boot2(v4) in order to work, along with the new IOS versions which are already well-tested by this point.
6. Consumers receive these new C/RVL-CPU-20 Wiis. Some of them try installing old versions of IOS on them and then running libogc code that uses them
7. libogc doesn’t know that it needs to wait longer for IOS to reload; depending on the timing, this can either cause a hang or it can cause IOS initialization to fail.

I imagine that Nintendo never even realized this would be a problem. Here’s how the system normally boots:

1. System turns on
2. boot0 / boot1 / boot2 run on Starlet
3. boot2 loads the TMD for the System Menu, reads the required IOS version, then chainloads to that version of IOS
4. Newly loaded version of IOS loads System Menu from NAND into memory, then turns on PPC and starts it executing
5. User selects channel or disc; IOS loads TMD, reloads into new IOS
6. IOS loads game from NAND or disc into RAM, then bootstraps PPC

Here’s what happens when someone runs some USB loader crap from the Homebrew Channel:

1. User selects HBC from system menu; menu calls ES_LaunchTitle(00010000-HAXX)
2. IOS reads HBC’s TMD from NAND, sees that it needs IOS61 (or whatever), and reloads to it
3. IOS61 initializes hardware, then loads the HBC content into RAM and bootstraps PPC
4. User selects warezloader from SD card
5. HBC loads ELF from SD into PPC, and then jumps to it directly
6. ELF calls (e.g.) IOS_Reload(249) to load patched version of IOS
7. libogc makes IPC calls to IOS to reload IOS, but does not wait long enough for the process to finish, and then becomes confused

In the normal Nintendo scenario, IOS is always the one reloading itself, and then the new version of IOS loads the PPC and starts the code. There is never any PPC code executing while that process happens, and the PPC code can’t possibly start until the ARM is ready. In contrast, the process used by libogc is “backwards”, with the PPC reloading the ARM code; insuffcient synchronization code is used to prevent this race condition because the old Wiis reloaded so quickly that it was never an issue.

Here’s what happens when someone runs E.G. a “forwarder channel” from the System Menu — a banner with a dummy TMD that loads a (probably patched) older version of IOS, and then launches something else:

1. User selects “forwarder channel” from system menu; menu calls ES_LaunchTitle() on title
2. IOS reads channel’s TMD from NAND, sees that it needs (old) IOS36 (or whatever), and reloads to it
3. (old) IOS36 fails to properly initialize the hardware, and the system hangs

This is not a bulletproof explanation. I can’t explain:

• Why the leaked IOS16 still worked
• What specific functions are needed in an IOS for it to work properly on a new Wii — the bootup process of IOS is multithreaded and difficult to follow, and I no longer have a Wii to test this on, and honestly I don’t care THAT much
• Why, specifically, the new code is necessary
• Why the latest version of the HackMii Installer still doesn’t work on all boot2v4 Wiis, even with the added synchronization code in place

Even still, I think this is much better explanation than “Nintendo did something magic to make unsoftmoddable Wiis”. Don’t get me wrong — I’m sure there were some smiles at Nintendo when they saw that hacks stopped working — but Nintendo’s anti-hacker efforts are generally far less subtle — for example, blocking /dev/flash and fixing the IOS exploit we used before in the HBC installer.

If you, the reader, would like to play around with this, I wrote a little test program, which you can download source and binary for here: reloadtest.zip. It’s mostly a version of the libogc IOS_Reload code, but with some debugging output (to the TV) and some timing code. It’s not exactly the same as the code I talked about above — boot2 is not involved with the process — but those of you with access to Wiis with boot2v4 (and for whom the HackMii installer doesn’t work) might be able to gain some insight as to what’s going wrong. Feel free to change the two versions of IOS (“from” and “to”).

When I run it on my Korean Wii, I get numbers like:

stats: id=0x06bcc32d boot2v3 IOS46->IOS22 IOSticks=32 IPCticks=475
stats: id=0x06bcc32d boot2v3 IOS46->IOS20 IOSticks=32 IPCticks=1041

On my old NTSC Wii (which originally came with boot2v2), I get:

stats: id=0x0408cafa boot2v3 IOS35->IOS22 IOSticks=32 IPCticks=497
stats: id=0x0408cafa boot2v3 IOS35->IOS20 IOSticks=32 IPCticks=1052

I’d be interested to hear if people see substantially different results on different hardware.

1. Yield improvements — fix flaws that caused lots of warranty returns
2. Cost reduction — use cheaper or fewer parts, so that profit margin increases or the retail cost can be lowered (or both)
3. Part refresh — sometimes manufacturers stop making parts (for example, 64MB memory chips), so they’ll start using “better” chips because they are the only ones now available
4. Security — anti-modchip modifications

Most people only think about the last one, but the first three are far more common. I think Sony has most actively revised their designs — aren’t there something like 30+ revs of the PS1, 15+ revs of the PS2, and several of the PSP and PS3? Microsoft had 5 or 6 revs of the Xbox1, and 3 of the Xbox360. Nintendo has had relatively few — perhaps this is because they have never sold their consoles at a loss, so they have less incentive to retool their factories to bring their costs down. (It’s rather expensive to redesign a PCB and then retool your factories to make new ones, and you add some risk of creating new hardware problems and increasing your failure rate.)

Focussing back on our Wii, we can consider the following things as being more or less independent:

• Updates to the disc drive
• Updates to the “big chips” (Hollywood, Broadway)
• Updates to the main PCB

We’ve seen several updates to the disc drive; these are so well-known that people even put up entire websites that track them! Off the top of my head, there was DMS/D2A, D2B (like D2A, but changed mask ROM on one of the chips — probably for reason #1), D2B with cut legs (reason #4), D2C (reason #4), D2C2 (reason #4), D2E (unknown — reason #1?), D2E + epoxy (reason #4), “D2nothing” (perhaps reason #2, but most likely reason #4). Clearly, these were mostly motivated by anti-piracy concerns.

We have also seen a few revisions to the Broadway and Hollywood chips. The oldest photo I can find online of these chips is this one, dated November 17, 2006:

Note that this is the first revision of each chip — called simply “Broadway” and “Hollywood”. Protip: We can tell that the Hollywood chip was made in the 32nd week of 2006 (“0632″) and the Broadway chip was made in the 31st week (“0631″) — so, August or so.

Unfortunately, not many people share my same sick fascination with taking their expensive consoles apart and photographing them, so it’s hard to find photos of a wide range of chips online. On the Korean Wii I bought, it had a “Hollywood AA” (date code 0812) and a “Broadway B” (0744). I assume that there was a “Hollywood A”; if anyone has one of these, please send me a photo or at least give me the date code from the chip so I can build a proper timeline. (I also have a “Hollywood AA” (0801) with a “Broadway A” (0747)), and a “Hollywood” (0636) with a “Broadway A” (0641).

The chip differences aren’t really very meaningful to us. All of the chips have to run the same code. It’s likely that the revisions were done to fix minor glitches; for example, there are a few places in IOS where it checks to see if the Hollywood chip is older than some version, and if so, executes some additional code (redundant memory writes, presumably to work around a bug in the silicon).

Much more interesting to me are the PCB changes. The first couple of years of the Wii saw only one PCB, “C/RVL-CPU-01″; within the past year, there have apparently been two more (-20 and -30, as shown below):

Wii PCB comparison - C/RVL-CPU-01 vs -20 vs -30

Note that the major difference between -01 and -20 seems to be that there are a few missing parts. Segher says that this is probably simplified power-supply circuitry; this will be important later. Leaving these parts off may have allowed them to save an extra dollar or so; I can’t think of much other reason for the change. It’s also important to note that neither the Hollywood or Broadway seem to have changed — they are still at Hollywood AA and Broadway B.

The -30 PCB is what has me stumped. This photo came from someone on IRC who bought a new (“unsoftmoddable”) Wii in Australia; he sent it to me as part of our efforts to debug the HackMii Installer. The only difference between those two is … the Broadway chip package is much, much smaller. I’m told that it’s not likely that they actually shrunk the die; rather, they probably just shrunk the metal package, and placed the balls of the BGA package closer together. Unfortunately, I can’t read the writing on the Broadway chip in this photo; if anyone has a Wii with a small chip like that, please take me some pictures of the top and bottom of the PCB!

My theory is that the switch from -01 to -20 coincided with the switch from boot2v3 to boot2v4, and that is when we started seeing reports of “unsoftmoddable Wiis”. I’ll cover the software aspects of this in a separate post.

1. We could position it as a piracy-related concern, and Nintendo has some channels for reporting piracy
2. We didn’t really care if they fixed the bug, since it wasn’t really that useful for legitimate homebrew
3. There was probably not much they could do to fix it, anyway, since it was more of a design flaw

The bug (as we intended to report it) wasn’t so much that you could poke a register to enable DVD video mode:

 #define HW_DIFLAGS 0x0d800180

 set32(HW_DIFLAGS, 0x200000);

… it was that you could just set a bit in the TMD (in the “access rights” field) and it would let you send DVD video commands.  You didn’t even need to patch IOS!  If you set that bit in your TMD, when your title gets launched, ES reads the “access rights” field (offset 0x1d8) and checks a couple of bits.   If bit 1 is set, it opens /dev/di and calls ioctl 0x8E, which in turns calls syscall 0×50, which does the above register poke.  This seems to set some state in the DI controller chunk of the Starlet that allows DVD video commands to go through.  This is how DVDX works — that bit is set in the TMD for DVDX and that makes the magic happen.

All of this is more or less academic, because if you can forge a signature to modify the TMD, then you can just patch the content of IOS — and that’s what most people (everyone else) did.  We think our approach is cleaner, but oh well.

Looking through the IOS code, ES checks the TMD for two bits, not one — but we couldn’t really figure out what the other one did, so we ignored it and moved on to more pressing things.  I only discovered its purpose a year later when trying to work on a BootMii bug which prevented the front-panel buttons from working when we were booted from IOS instead of as boot2.

If you refer back to my Wii Hardware Architecture diagram, you’ll see that the PPC (where most code gets executed) has to communicate to all the peripherals through an I/O interface that’s built into the Starlet; this interface selectively bridges some address ranges to different peripherals on the AHB / APB busses.   (AHB and APB are two standard ways of connecting other devices to an ARM core — the first one is higher-performance and used for things like USB and NAND, and the latter is slower and used for things like buttons and joypads.)

Normally, you have to access peripherals on those busses by making calls to IOS, which always has access to everything.  I thought this acted as a “de facto” firewall — that there simply was no silicon pathway from the PPC to e.g. the NAND, but there was one for e.g. the EXI bus and gamepads.  I should have been suspicious, however, at the fact that shape of this firewall can change somewhat when MIOS runs.  In normal IOS, the PPC cannot talk to the optical drive interface (DI), but in GameCube mode it can — and it has to work this way in order for GameCube games to be run without patching the games on the fly.   (I wonder why Nintendo didn’t make MIOS trap those DI commands and then translate them into IOS ioctls — this may always be a mystery.)

Another reason I should have been suspicious was the wording of some of the tests we saw in the factory logs I scraped:

sdiIMemA "SDI IOP API MemRW (sort)" "use IOP API in this test. Need SDCd" 0x0B09 sdio_api_memrw.dol "-auto"
sdi0RegM "SDI Register Check for WP (manual)" 0x0B01 sdio_hcreg.dol "-sdiboth" sdi0MemS

NandChk "NandFlashTest" "NAND FLASH Phsycal layer test from BW access" 0xA088 NandManu.dol
NandIOS "NAND Flash Auto by IOS-API" "WAIKIKI output." 0x0CA2 NandIOS.dol

Note that all of this code is running on the PPC. Some of it specifically refers to talking to hardware via IOS (IOP API or IOS-API), and some of it specifically talks about accessing hardware directly from the PPC (“test from [Broadway] access”).

Well, we can now answer one of the mysteries from my factory 2 post. Towards the end of the manufacturing process, they insert a disk — 122E — into the Wii, which installs a “DataChk.wad” — title ID of 00010000-30303032 (“0002“), and then they run a bunch of DOLs from an SD card. This seems unnecessary — why can’t the disk just run stuff?

As it turns out, the TMD for “0002” has a different bit set in its TMD “access rights” field — bit 0. The code for this looks something like:

#define HW_MEMMIRR 0x0d800060
#define HW_AHBPROT 0x0d800064 // defaults to 0xFFFFFFFF on boot

void check_tmd(struct TMD *tmd) {
    // stuff ...
    enable_dvd_video_mode(tmd.access & 2);
    syscall_54(tmd.access & 1);
    // more stuff ...
}

void syscall_54(int factory_mode) {
    if (factory_mode) {
        set32(HW_MEMMIRR, 8); // this probably enables access to Hollywood regs from PPC
        set32(HW_AHBPROT, 0x80000DFE); // re-enable PPC access to disabled hardware devices
        write16(0x0d8b4202, 0); // dunno what this does
     } else {
        clear32(HW_MEMMIRR, 8); // this probably disables to Hollywood regs from PPC
        clear32(HW_AHBPROT, 0x80000DFE); // disable access to some hardware devices from PPC
        write16(0x0d8b4200, 0x18); // dunno what this does
    }
}

At normal boot, boot2 will read the System Menu (1-2)’s TMD, find a zero, and call syscall_54(0). Once it reloads into this “0002” title, IOS will call syscall_54(1), which — get this — enables access to every single fucking piece of hardware, directly from the PPC. This lets their factory tests access all of the hardware in the simplest way possible — you can access NAND, the SD card, WiFi, everything from a DOL running on the PPC. This makes the tests much easier to write — many of them would have required IOS modifications, otherwise — and probably saved them a lot of time — also, it suggests that originally there was no restriction and that they went in and placed that artificial restriction towards the end of the Wii’s development cycle.

When I wrote the code to read the front panel buttons for BootMii, I did all of my testing using BootMii/boot2. This meant that — although I didn’t realize it — I had full access to the normally-guarded hardware, including the GPIO pins that are connected to the front panel buttons. If I had tested more with BootMii/IOS, I would have realized that the real boot2 and IOS both lock down those registers, which is why buttons didn’t work in BootMii/IOS. That’s how I discovered the HW_AHBPROT register, and it reminded me of the above syscall_54(), and here we are today.

So you’ve read this far — what does this all mean? Sadly, not too much. We wrote MINI to act as a proxy so we could talk to hardware from the PPC which was normally locked out. We still need it to boot up the PPC and start code executing there, but in theory the rest is redundant. The simplest fix for the button problem was to just make MINI set that HW_AHBPROT register to 0xFFFFFFFF — it’s probably one bit per device — but there’s no real point in rewriting the rest of BootMii.

(Before anyone asks — no, this doesn’t really make it easier to access USB / Bluetooth from BootMii, because it just means that you could write your USB stack for PPC instead of for the ARM chip.)

The major winner here is gc-linux, which can now reuse most of the normal MMIO Linux drivers to talk to hardware instead of proxying through MINI. It’s not a functionality improvement, but more of a performance boost. Nevertheless, I hope you found this interesting.

I’ll give more info about all of this in my next post, but in brief — I have only ever seen three PCB revisions on the Wii:

C/RVL-CPU-01 vs C/RVL-CPU-30:

C/RVL-CPU-20:

From C/RVL-CPU-20

The PCB revision ID is found in the lower-right of the PCB, below the “Nintendo” logo. I would be interested in any photos people can give me of the PCBs of their Wii, particularly if

1. The heatsink is removed, and all of the writing on the two large chips (Hollywood and Broadway) is readable
2. The Wii was purchased in 2009 and/or is “unsoftmoddable” / “LU64+” / has boot2v4

Any more details you can provide along with the picture would be appreciated — specifically, console ID, boot1 and boot2 versions, approximate date of purchase, serial number printed on outside of Wii.  I won’t publish this info individually, but I’m trying to show how the progression over time matches up with the software changes in my previous post.

You can either post the info and photo links here as comments, or email them to me at bushing at gmail.

Thanks!

Here’s the device:

The “Labtool-48UXP” is an “Intelligent Universal Programmer”, with an impressive list of features. Some of the ones that caught my eye were “Four DACs for Vcc, Vpp1, Vpp2 and Vpp3 with 8-bit resolution. Software controllable rises time and current limit protection”, “Logic driver supports pull-up/ pull-down or high impedance on all 48 pins with 2.0V-5V level,” and “The LabTool-48UXP performs device insertion and contact checks before it programs each device. It can detect poor pin contact and devices inserted upside down or in the wrong position.”

I had high hopes for this device — maybe I could write a Mac client for it, or add support for the Wii’s NAND flash chips to it. In any case, I could find virtually no info about this thing online; surely someone had hacked on one of these things before? In any case, before any more fun could be had, I had to get it working, and that required understanding what’s inside. Shall we take a look?

When you open the thing up, you’re faced with 2 stacked boards and a dual-voltage power supply (12v/5v).

The lower board is marked “LabTool-48UXP Control Unit”.

In some order, here are the major parts on the board:

1. DB25 connector for parallel port
2. USB connector
3. Cypress “EZ-USB” FX2 microcontroller, a common USB2.0 interface that is an 8051-derivative
4. Serial EEPROM to store USB VID/PID
5. Main processor – an Intel 80C32, another ROMless 8051 derivative. This chip is as old as I am!
6. Firmware for the 80C32 — 64KB, only the first half is used
7. 32KB SRAM for the 80C32 — overlays the top 32KB of the address space. Also, I think some of the address space is used for memory-mapped IO with the XC3030 (see below) and the parallel port / FX2
8. Lattice CPLD — possibly used for the MMIO mentioned above
9. TLC7226C — 4-output 8-bit DAC. Presumably used to drive the 4 power supplies (Vcc, Vpp1, Vpp2, Vpp3) mentioned in the marketing copy
10. Power supply section — what we have here is an LM317T (used as a general voltage regulator?), LT1170CT (used to boost the +12v to +30v), and then 4 power op-amps which are configured as non-inverting amplifiers with a gain of about 10. Meaning, the DAC feeds each a voltage from 0-3.0v, and the opamp outputs 0-30v.
11. A6833 (x2) — these are 32-bit, high drive-capacity shift registers.
12. NEC B772 PNP power transistors. There are 48 of them, but I don’t think that they necessarily map 1:1 with the output pins. These seem to be driven by the output of the shift registers?
13. Reed relays — 10 of them. Why 10? Dunno. They seem to connect to the voltage sources, although 3 of them seem to pull stuff down to ground.

The chips marked with ? have the tops sanded off them. The chips marked with * are standard 74HCT logic-series chips — from top to bottom, left to right (like reading a book) we have ’373, ’245, ’373, ’14A, ’245, ’244, ’373, ’374, ’244, ’244.

The upper board is marked “LabTool-48 Driver Unit”.

1. TPIC6B259 “Power Logic 8-bit Addressable Latch”. This thing has 8 150ma open-drain outputs (wewt!) and only needs 4 control lines — 3 to specify 1 of 8 bits, 1 to actually set the bit. 6 of these, total, so one bit per pin.
2. UDN2987 “8-Channel Source Driver with Overcurrent Protections”. This part has 8 buffers which are presumably used in combination with the open-drain latches (above) to form a push-pull circuit for each pin. Each driver can source 100mA, and it has an overcurrent detection output, which is fed back into the controller circuitry and causes the device to give you an error message when you put your chips in backwards. Again, 6 of these.
3. 74HC259 — 8-bit addressable latch. There’s one of these for every UDN2987, so you can latch the state of the pull-up driver and latch the state of the pull-down driver for each pin.
4. 74HC138 and 74HC273, presumably acting as glue logic for the …
5. XC3030A — Xilinx 3000-gate FPGA. Digikey lists this as a $17 part, but that must just be because they are old and hard to find! This was state-of-the-art, 10 years ago. If nothing else, it provides the “software controllable rise time”. I think it’s safe to say this drives the push-pull latches that cover the rest of this board.

Still unclear is exactly how the transistors and relays on the first board interact with the latches on the second board. My next post will talk about the software provided by the manufacturer to drive this thing.

The goal: mount two full NAND Flash chips at the same time inside a Wii, so that you can switch back and forth between them.

This isn’t exactly self-explanatory, so let me explain what’s going on here. He’s found some extra chips (see below) that are the same as the ones inside a Wii — he then desoldered his System-Menu 3.1U Wii’s flash chip from the Wii, and cloned it onto the two extra chips using an Infectus chip and amoxiflash. This way, if anything goes wrong, he can always desolder this hack from his Wii and go back to the normal one. (All of this soldering and desoldering is difficult and risky, but hey — you do what you gotta do. Carefully.)

NAND flash uses an 8-bit data bus and 7 control lines. One of those is Chip Enable (CE) — if Chip Enable is deselected, then the chip almost acts as if it’s not connected it all (all input and outputs go to tristate). Therefore, if we can make sure that only one of those two chips will have its CE pin active at any given time, we can just solder the rest of everything together. Then, to switch between the two chips, you just write up a 2-way switch. In one position, CE of one chip is connected back to CE from the Wii board. In the other position, the other CE is connected.

A schematic may make this clearer:

Now, let me be perfectly clear here — this is a neat hardware hack, but this is not something most people will be able to pull off, nor is it something most people will find useful. It will not help you fix a broken Wii, or downgrade a Wii, or anything of the sort.

In order to make use of this, you will need:

• 1 or 2 extra NAND flash chips.  You can take these from a dead Wii, or certain (very specific) flash-based devices.   (Think USB flash sticks, CompactFlash cards, shitty MP3 players.)

I did some hacking around, and was able to get amoxiflash to build under XP.  I’ve posted a binary for Windows here: amoxiflash-win32-02.   In order to run it, you will also need to install the libusb-win32 filter from here: libusb-win32 filter driver

Of course, the source is included there (for what I’m calling “Version v0.2″), and you can compile it under Mac OS X and Linux, too!  I’ve tested it a bit under Windows, but I make no promises.  On my MacBook Pro using VMWare and XP, it seems to run at about half speed under Windows as compared to natively under OS X.  Ideas for improving the speed would be greatly appreciated.

Here’s the current usage info:

amoxiflash version 0.2, (c) 2008 bushing
Usage: ./amoxiflash command -[tvwdf] [-b blocksize] filename
          -t            test mode -- do not erase or write
          -v            verify every byte of written data
          -w            wait for status after programming
          -f            force: ignore safety checks. Dangerous!
          -d            debug (enable debugging output)
          -b blocksize  set blocksize; see docs for more info.  Default: 0x2c0
          -s blockno    start block -- skip this number of blocks
                        before proceeding

Valid commands are:
         check        check ECC data in file
         strip        strip ECC data from file
         dump         read from flash chip and dump to file
         program      compare file to flash contents, reprogram flash
                        to match file

It seems to be a bit fragile under Windows — if you see something like this, then unplug the Infectus from your computer, power the Wii off, turn the Wii back on (shorting D0 while doing so), plug the USB cable back in, and try again:

Here’s what you want to see:

As always, the source is available in Subversion.