Notes from inside your Wii

HackMii header image 2

Timing is everything (the case of the “unsoftmoddable Wii”)

August 29th, 2009 by bushing · 24 Comments

note: If you haven’t already, you should probably go back and read these posts, because they were written in preparation for this one:

A few months back, we started getting reports of “unsoftmoddable Wiis”, aka “LU64+” (among other things). Normally, I wouldn’t care, but we discovered that our HackMii Installer would not work on any of those Wiis. I started making the claim that this was due to an innocuous hardware change, coinciding with the release of boot2v4, but I never really explained why. Here’s my explanation.

“LU64+” is a really silly way to describe these Wiis — it refers to the first four characters in the serial number of the first Wiis seen to be like this — but I can’t really blame people, because the serial number is the only real visible marking as to when the console was manufactured.

Starting around early 2009 (?), the following statements were true about all new purchased Wiis:

  1. All versions of IOS have the strncmp() bug fixed, access to /dev/flash blocked, and “ES_Identify” was broken.
  2. boot1 had strncmp() bug fixed
  3. crap installed into the slots for IOS3, IOS4 and IOS254

Beginning in March, a fairly well-defined set of symptoms of running some kinds of homebrew software emerged on “newish” Wiis:

  1. Old (but valid) versions of IOS would not run at all; they would hang if you tried, which prevented downgrading IOS. This did not apply to the leaked IOS16.
  2. Attempts at reloading IOS by libogc would sometimes fail; depending on the program, this would look like a hang or would just cause all IOS calls to fail

There seemed to be no difference in the installed software on one of these “newish” Wiis, compared to what you would get by doing an update through the System Menu.
People proposed theories such as “Nintendo added special hack-detection code into boot1 / boot2, but they put a ‘back-door’ in for IOS16.” As I’ve explained in the past, boot1 and boot2 are finished executing long before you see anything on your screen, and certainly don’t affect anything past that. Rather, boot2 is responsible for loading IOS, and IOS is responsible for loading the PowerPC code for the system menu or game. If you insert a game disc that uses a different version of IOS, the currently running IOS is responsible for loading and running the new IOS, which then loads the PPC code, and maybe eventually another IOS to get back to the system menu, etc. (This exact order of steps becomes important later.)

This was a curiosity I mostly ignored, because it didn’t affect any of our software … until we released the HackMii Installer.

The reason we created one common installer for BootMii, the Homebrew Channel and DVDX was that it became increasingly complicated to install software on updated Wiis. Different approaches were required, depending on the configuration of the Wii, and it became apparent that duplicating our decision-making and exploit code across multiple installers would be silly. We ended up coming up with some code that would examine the state of the Wii it was running on, pick an appropriate strategy, and then automatically install the desired software.

There are a couple of different possible paths for the Installer to use — if it can, it will just use an old IOS that still has the unpatched hash comparison function and unfettered access to /dev/flash (reloading to that IOS as necessary). If not, it would choose one of a list of IOS exploits we had found, depending on the versions of IOS installed (each exploit needs to be customized to a specific major/minor version of IOS), and use it to load MINI into memory and execute it. From there, we could directly access the NAND flash and detect the installed version of boot1 and boot2. Finally, having made the decision of whether or not we could allow BootMii to be installed as boot2, we would reload back into a normal IOS so that we could access the WiiMote. Of course, reloading back into IOS isn’t exactly trivial; MINI has a function that launches an arbitrary title from NAND by reading boot2 from the beginning of NAND, patching the call to ES_LaunchTitle(1-2) to the desired title ID, and then executing the patched boot2.

All of this happened automatically and hopefully seamlessly, but it could take a few seconds. Having been burned by the pay-for-homebrew wankers, we have an “anti-scam warning screen” that we make the user sit through before installing our software. The delay there made for a great opportunity for us to do our leet haxIOS magic, so we ran that “in the background” while the user sat through our nag screen.

This didn’t work as well as we had hoped on some new Wiis — suspiciously, it would reboot in the middle of the nag screen on “unsoftmoddable Wiis”. Suddenly this became an interesting problem to me!

Unfortunately, we’re all old farts with old Wiis. None of us could reproduce the problem ourselves – we tried to get people with troubled Wiis to run test code for us, but you really need a USBGecko to be able to get debug output from low-level code. People who were new to the Wii were the least likely to have a USBGecko for testing, so … the only way we could proceed was to track down one of these “unsoftmoddable” Wiis.

I finally did so, and ran a debugging version of the installer on it. I was surprised to see that it was actually getting all the way through our set of exploits, and was actually dying when it tried to reload back to IOS by way of boot2. It turned out that these new Wiis had a new version of boot2 — boot2v4 — and our “patch boot2 to reload IOS instead of the system menu” routine was failing to find the code to patch. Instead of giving an error message, it was just executing the unmodified new boot2 — which then re-ran the system menu instead of proceeding with our installer. Oops.

That was certainly easy enough to fix — I added a patch for boot2v4 — and the installer seemed to successfully reload IOS, only to hang inside __IOS_InitializeSubsystems() in libogc (specifically, ios_open(“/dev/es”)). This started to match the reported failures in libogc programs during IOS_Reload().

svpe said he had run into some timing problems when working on booting back and forth between IOS and MINI, and suggested I try adding a delay. When I added a delay of 5 seconds into our code between when we asked MINI to reload IOS, and when we actually reinitialized libogc, then everything started work. I found this rather odd, since the code had worked just fine with boot2v3, and it also worked on boot2v4 if we reloaded to the System Menu (since that’s what had been happening to people when our boot2 patch failed).

After another sleepless night or two, it became clear that boot2v4 actually took somewhat longer to load than boot2v2/3. If I checked the IOS version number right before calling __IOS_InitializeSubsystem(), it would report back the correct IOS version on boot2v3, but 0 on the boot2v4 system. This indicates that boot2 was still running — boot2 sets that version number to 0 when it starts.

The answer to this problem seemed to be simple enough — we just needed to add some code that would work like the following:

  1. Send IPC to MINI asking for it to boot into IOS
  2. delay until IOS version number is set to 0 (meaning, boot2 has fully loaded and is now executing)
  3. delay until IOS version is the expected IOS version
  4. proceed to initialize IOS

Again, doing this made a little difference, but not a good one. Now we no longer froze inside of __IOS_InitializeSubsystems(), but all of our IOS called failed once we booted into the installer — and they failed with strange error codes like “-900074497”. svpe then pointed out that the IOS load process looks something like this:

  1. boot2 starts
  2. boot2 sets IOS version number to 0
  3. boot2 executes es_main()
  4. es_main calls ES_LaunchTitle(1-2) (or in our case, IOSxx since we patched this parameter)
  5. ES_LaunchTitle recognizes that we are trying to load an IOS, and then sets up the IOS version number in memory so that the newly-loaded IOS will know what version it is
  6. ES_LaunchTitle transfers control to the new IOS
  7. IOS starts and sets up the IPC registers to talk to the PPC

We were hitting a timing race between when ES_LaunchTitle changed the IOS version number, and when IPC actually started. If we tried to make IPC calls to the Starlet before the version number changed, then we would actually be trying to make IPC calls to boot2 while it was in the middle of rebooting — so we’d never get a response and libogc would hang. If we waited until after the version number was set, libogc would try to talk to the new IOS before IPC was initialized; as part of that initialization, IOS would send back a bogus “ack” message that made libogc think that the ios_open(“/dev/es”) call failed, and then it would never properly initialize ES and all later calls would fail. If we waited in our own code for this “ack” message (as suggested by isobel), then everything else worked.

Armed with this knowledge, I was able to add some timing code to our installer, and get the following numbers:

On old hardware:
Time to load boot2v3: 958 ms
Time to execute through boot2v3 and launch IOS: <1 ms
Time to initialize IPC: <1ms
On new hardware:
Time to load boot2v4: 134 ms
Time to execute through boot2v4 and launch IOS: 178 ms
Time to initialize IPC: 499 ms

Thus, we had our solution … more or less … to the problem of boot2v4 and the HackMii Installer — we just needed to add some delays. So why is the timing so much different on the new Wii, and why doesn’t this affect normal operations of the Wii?

Let’s summarize the facts:

  • Wiis recently began shipping with a new boot2v4, which is based on the same code present in the set of IOSes released last October
  • Reloading IOS is often “broken” on new Wiis — especially when trying to reload to an “older” version of IOS (pre-October, taken from an older Wii)
  • Launching PPC titles works just fine on new Wiis, even if that requires reloading to a different version of IOS (as happens when launching the Homebrew Channel
  • The same versions of IOS work differently on older Wiis, and have none of the problems noted above. The only difference in the software loaded on these Wiis is in boot1 and boot2.
  • The “leaked IOS16” seems to be exempt from these problems

I think I can explain all but the last point.

From previous research, we can add the following tidbits of knowledge:

  • boot1 and bc have each gone through 4 revisions; of those, 1 was to fix the strncmp bug and 3 were to change low-level hardware initialization routines
  • The strncmp bug fix first appeared in IOS37 in March, 2008; IOS, bc, and boot1 all apparently were fixed internally at Nintendo / BroadOn in Feb-Mar. 2008, but not released until a few months later
  • bc (and presumably boot1) was rebuilt with more minor hardware init changes in June 2008
  • new versions of boot2 and all of the IOSes were built on July 11th, 2008
  • The new versions of IOS were put up on the Nintendo Update Servers in October
  • Wiis began shipping with the new IOSes (but with boot2v4) in late 2008
  • Two PCB revs were made at the end of 2008/beginning of 2009 — all of these Wiis shipped with the “new” IOSes and boot2v4, and do not seem to work well with older versions of IOS
  • The only substantial changes made in the PCB were to power-supply circuitry; Hollywood (where IOS runs) was unchanged (still at “Hollywood AA”), and there may have been some inconsequential changes made to the Broadway

A scenario begins to emerge:

  1. Nintendo develops a new PCB design to reduce cost by simplifying circuitry in mid-2008; the resulting design has fewer parts but takes a few milliseconds longer for the power supply voltages to stabilize
  2. The new hardware requires a more-tolerant IOS kernel, so BroadOn rebuilds boot2 and IOS for Nintendo do use in testing the new PCB design. These IOS versions are still backwards-compatible with C/RVL-CPU-01.
  3. Nintendo releases the new IOSes as an update in late 2008, to quash the strncmp() bug (as well as /dev/flash access and ES_Identify()
  4. Nintendo starts producing C/RVL-CPU-01 boards with the new IOSes on them. (boot2 is not updated, since there’s no benefit to doing so)
  5. Nintendo eventually starts mass-producing C/RVL-CPU-20, which requires an updated boot2(v4) in order to work, along with the new IOS versions which are already well-tested by this point.
  6. Consumers receive these new C/RVL-CPU-20 Wiis. Some of them try installing old versions of IOS on them and then running libogc code that uses them
  7. libogc doesn’t know that it needs to wait longer for IOS to reload; depending on the timing, this can either cause a hang or it can cause IOS initialization to fail.

I imagine that Nintendo never even realized this would be a problem. Here’s how the system normally boots:

  1. System turns on
  2. boot0 / boot1 / boot2 run on Starlet
  3. boot2 loads the TMD for the System Menu, reads the required IOS version, then chainloads to that version of IOS
  4. Newly loaded version of IOS loads System Menu from NAND into memory, then turns on PPC and starts it executing
  5. User selects channel or disc; IOS loads TMD, reloads into new IOS
  6. IOS loads game from NAND or disc into RAM, then bootstraps PPC

Here’s what happens when someone runs some USB loader crap from the Homebrew Channel:

  1. User selects HBC from system menu; menu calls ES_LaunchTitle(00010000-HAXX)
  2. IOS reads HBC’s TMD from NAND, sees that it needs IOS61 (or whatever), and reloads to it
  3. IOS61 initializes hardware, then loads the HBC content into RAM and bootstraps PPC
  4. User selects warezloader from SD card
  5. HBC loads ELF from SD into PPC, and then jumps to it directly
  6. ELF calls (e.g.) IOS_Reload(249) to load patched version of IOS
  7. libogc makes IPC calls to IOS to reload IOS, but does not wait long enough for the process to finish, and then becomes confused

In the normal Nintendo scenario, IOS is always the one reloading itself, and then the new version of IOS loads the PPC and starts the code. There is never any PPC code executing while that process happens, and the PPC code can’t possibly start until the ARM is ready. In contrast, the process used by libogc is “backwards”, with the PPC reloading the ARM code; insuffcient synchronization code is used to prevent this race condition because the old Wiis reloaded so quickly that it was never an issue.

Here’s what happens when someone runs E.G. a “forwarder channel” from the System Menu — a banner with a dummy TMD that loads a (probably patched) older version of IOS, and then launches something else:

  1. User selects “forwarder channel” from system menu; menu calls ES_LaunchTitle() on title
  2. IOS reads channel’s TMD from NAND, sees that it needs (old) IOS36 (or whatever), and reloads to it
  3. (old) IOS36 fails to properly initialize the hardware, and the system hangs

This is not a bulletproof explanation. I can’t explain:

  • Why the leaked IOS16 still worked
  • What specific functions are needed in an IOS for it to work properly on a new Wii — the bootup process of IOS is multithreaded and difficult to follow, and I no longer have a Wii to test this on, and honestly I don’t care THAT much
  • Why, specifically, the new code is necessary
  • Why the latest version of the HackMii Installer still doesn’t work on all boot2v4 Wiis, even with the added synchronization code in place

Even still, I think this is much better explanation than “Nintendo did something magic to make unsoftmoddable Wiis”. Don’t get me wrong — I’m sure there were some smiles at Nintendo when they saw that hacks stopped working — but Nintendo’s anti-hacker efforts are generally far less subtle — for example, blocking /dev/flash and fixing the IOS exploit we used before in the HBC installer.

If you, the reader, would like to play around with this, I wrote a little test program, which you can download source and binary for here: reloadtest.zip. It’s mostly a version of the libogc IOS_Reload code, but with some debugging output (to the TV) and some timing code. It’s not exactly the same as the code I talked about above — boot2 is not involved with the process — but those of you with access to Wiis with boot2v4 (and for whom the HackMii installer doesn’t work) might be able to gain some insight as to what’s going wrong. Feel free to change the two versions of IOS (“from” and “to”).

When I run it on my Korean Wii, I get numbers like:

stats: id=0x06bcc32d boot2v3 IOS46->IOS22 IOSticks=32 IPCticks=475
stats: id=0x06bcc32d boot2v3 IOS46->IOS20 IOSticks=32 IPCticks=1041

On my old NTSC Wii (which originally came with boot2v2), I get:

stats: id=0x0408cafa boot2v3 IOS35->IOS22 IOSticks=32 IPCticks=497
stats: id=0x0408cafa boot2v3 IOS35->IOS20 IOSticks=32 IPCticks=1052

I’d be interested to hear if people see substantially different results on different hardware.

Tags: Wii

24 responses so far ↓

  • 1 Dood77 // Aug 29, 2009 at 10:44 pm

    “Suddenly this because an interesting problem to me!” Eh? Is this some new meme I don’t know about yet?

    Great read, as always.

  • 2 alphas // Aug 29, 2009 at 10:47 pm

    I got on my old NTSC/J Wii :
    stats: id=0x040cf4d4 boot2v2 IOSticks=32 IPCticks=434

  • 3 ChuckBartowski // Aug 29, 2009 at 10:48 pm

    Keep making articles like this and youll have yourself the best wii blog in the world*

    *in my opinion.

    Oh and i hope i beat squidman to first….

  • 4 warll // Aug 29, 2009 at 10:55 pm

    Awesome, a double feature!

    I really love your posts, well not all of them, really just the detailed ones like this one.

  • 5 SquidMan // Aug 29, 2009 at 11:30 pm

    Wow, nice detailed post. Also, 2 posts in one day, you’re awesome πŸ˜€
    I don’t have a newer Wii, so I can’t help much, but it’s definitely neat to see how this all works. When was IOS16 built? I mean, judging by the number, I’d guess it was before they made the new PCB revision and needed a test IOS, but as we saw with IOS57, IOS number is not always the order in which it was built.

  • 6 Mike // Aug 29, 2009 at 11:48 pm

    yeah two posts! this is great. thanks, i hope everything goes well, my cousins have a wii lu64 i believe. so all this may one day help

  • 7 djdynamite123 // Aug 30, 2009 at 1:40 am

    Great read, this is obviously the reason certain apps which reload another IOS or whatever fail causing a hang, and blimey SquidMan weren’t first!

  • 8 SpyroDragon // Aug 30, 2009 at 3:49 am

    Very interesting and detailed post.
    Even made me register now, to thank you. πŸ™‚

    I like how you write detailed, but still understandable.

    I wonder if BootMii will ever made compatible with Boot2v4, or if I’ll have to bargain-hunt for a Boot2v3 Wii, once my current one breaks.

  • 9 tech3475 // Aug 30, 2009 at 7:08 am

    Another interesting article.

    BTW in regards to IOS16, could there be code in it or an exception somewhere in the system which chainboots into IOS16 which allows for it to work with the newer wii’s?

  • 10 bushing // Aug 30, 2009 at 11:14 am

    @tech3475: No. The differences between IOS15 and IOS16 are pretty small, and there is no code in all of the other versions of IOS that treats IOS16 any differently than anything else. I never tried loading it, myself, so I don’t know if there was something special people were doing when they loaded it that made it work.

  • 11 adr990 // Aug 30, 2009 at 11:18 am

    Awesome post!

    Thanks for taking your time again dude.
    Always a pleasure to read your posts.

    I have a old Wii and a “LU64+ Wii” at home, to eventually test stuff. πŸ™‚ (Both PAL.)


  • 12 Luke Usher // Aug 30, 2009 at 1:24 pm

    So the whole situation with boot2v4 Wii’s is just that the hardware takes longer to boot up, and the older IOS versions do not take this into account. I wasn’t expecting it to be something that simple.

    Also, these technical posts are always an interesting insight into the workings of the Wii. These are the kind of posts I hope to see whenever I visit this blog.

  • 13 senorclean // Aug 30, 2009 at 4:13 pm

    I just ran it on the 2 wiis I had handy… (I have another Launch wii, but I guess you guys are more interested in the new ones).

    Older March ’08 wii (4.1E) – stats: id=0x03af3133 boot2v3 IOSticks=32 IPCticks=422

    Brand new August ’09 wii (4.0E) – stats: id=0x057d6fbf boot2v4 IOSticks=30 IPCticks=421

  • 14 someone // Aug 31, 2009 at 8:35 am

    If IOS15 and IOS16 are very similar and IOS16 (which is very old and has never been updated) works on newer hardware, presumably early versions of IOS15 also work on newer hardware and later versions do not work on it because they were updated with new hardware initialisation code.

    I suppose once you reach a certain version of IOS15, it will suddenly stop working on newer hardware because it’s trying and failing to initialise hardware that IOS16 is (assumedly) not even bothering to initialise?

    After all, the hardware should already be initialised anyway, it’s just that later versions of IOS15 are trying to re-initialise it and IOS16 isn’t (if my theory is right).

    By the way, here’s my summer ’07 Wii currently on 4.1E moving from a DOP’d IOS36 12.22 to IOS34 12.19…

    stats: id=0x252138f boot2v2 IOSticks=33 IPCticks=447.

  • 15 SuperAxel // Aug 31, 2009 at 8:38 am

    Korean Wii (Hollywood – May, 08):

    stats: id = 0x060d101d boot2v3 IOSticks= 31 IPCticks 430

  • 16 TwoBladedKnight // Sep 1, 2009 at 4:03 am

    God I love reading your posts on how the wii works

  • 17 pembo // Sep 1, 2009 at 11:18 am

    That’s an excellent, well thought out, and well written post.

    Thanks for giving me a good insight into the wii boot process πŸ™‚

  • 18 takaneko // Sep 1, 2009 at 3:46 pm

    NTSC-U Wii purchased mid-late July ’09.

    stats: id=0x03d65556 IOS36->IOS34 boot2v4 IOSticks=31 IPCticks=430

    How is it that I have boot2v4 and yet I have successfully softmodded my Wii? Did I just get lucky? Is there any way I can contribute to your research?

    BTW you rock.

  • 19 feraligatr // Sep 2, 2009 at 12:25 am

    Very great article!

    At the beginning, it says, on new Wiis

    “1.Old (but valid) versions of IOS would not run at all; they would hang if you tried, which prevented downgrading IOS. This did not apply to the leaked IOS16.
    2.Attempts at reloading IOS by libogc would sometimes fail; depending on the program, this would look like a hang or would just cause all IOS calls to fail”

    This article mainly focuses on the second issue. But I cannot get clue about the first one. Why doesn’t older IOSes work at all? If the system follows the IOS->IOS->ppc sequence strictly, that timing problem shouldn’t affect official softwares including older IOSes.

  • 20 bushing // Sep 4, 2009 at 12:28 pm

    @feraligatr: I don’t really know. My best guess is that the the new hardware takes longer to start up, and the old IOSes don’t delay long enough when they try to load. The system then locks up.

    (Guessing is all I can really do; we don’t have many good ways to debug this sort of thing.)

  • 21 patterp // Sep 6, 2009 at 10:35 am

    On my Wii 3.2E/PAL purchases end 08
    I get:
    IOS Version: IOS36 12.22 / boot2v3

    IOS Version: IOS34 4.15
    stats: id=0x06a08e16 boot2v3 IOSticks=32 IPCticks=441

    … good work

  • 22 RogueFive // Sep 17, 2009 at 10:39 pm

    Fascinating read. I have been following your blog for quite some time now and this has been the most intriguing post by far.
    The dedication and detective work reminds of my early assembler days writing demos. (This is going to be really boring for younger people and non-geeks, but I thought you would appreciate it).
    We faced a similar situation when (and I am really going to show my age here) someone found by accident that you can mess with the screen display on the Atari ST by switching the screen refresh frequency in the middle of screen redraws.
    The first exploit allowed the removal of the lower screen border … you had to switch from 50 to 60hz when the raster electron ray was in the last part of the last regular line, just before the VBL. The top border was next to go (by “simply” writing your own VBL interrupt and switching frequencies before the ray would get to “normal”, then the right border (doing your own HBL interrupt) and finally the left border when someone switched to 71hz for a few cycles.
    Talk about timing. We also faced problems with hardware revisions, but in the end we did fullscreen demos on the Atari that nobody thought were possible. It took the coder community 4 years to get this done … but we didn’t have the internet back then.
    I applaud your efforts … it is very refreshing that in this day & age when high processing power tends to create lazy programmers someone still has the dedication to think in milliseconds. Keep up the good work!

  • 23 KingLewy // Oct 1, 2009 at 1:11 pm

    Wow. You know a year ago I first heard of Homebrew and I didn’t want anything to do with it. 2 months ago I found out I could get ScummVM on my Wii through HBC and practically jumped on it. My entire opinion of homebrewing has completely changed and after reading these blogs (in the past two days, since I stupidly upgraded to 4.2) a USBGecko is on my shopping list. Where can I learn how to do all this stuff, as I’m new to programming?

    Many thanks bushing for some great incites into the workings of my Wii.

  • 24 techneek54115 // Mar 5, 2010 at 1:21 am

    i have a originally 3.4u normal to 4.0 then bannerbombed installed bootmii as boot2 then uninstalled it(retard) then installed bad wad , bannerbricked , then recovered after getting drivekey with sddm and indiana pwns combo. i did also put in a patched nsmb before using indiana pwns which updated me to 4.1usa and now my boot2 is v24 wth i thought boot1 couldn’t be patched outside the factory. i don’t know where to post this since it contradicts everything i’ve read

You must log in to post a comment.