Rumor has it that sometime next week some good folks from NOA are going to be reading this blog in order to give a final yay / nay on whether or not I’ll be able to submit all of my improvements to MESS.
With that in mind, I looked over my blog myself, and I realized that at no point have I really laid out any kind of mission statement as to just what I’m doing and what I plan to do. With that in mind:
The Story
In the mid to late 90’s, emulator authors in pursuit of kudos from people looking for wholesale piracy while the N64 was still alive and kicking were attempting to write N64 emulators using the traditional method at the time: Have a C or C++ implementation of each opcode of each CPU in the relevant system, and simulate the display processor through software rasterization. By late 1998, the best emulators of their ilk were able to display the opening 2D legal screens of a handful of games, including Waverace 64 and Mortal Kombat Trilogy.
Depending on one’s perspective, January 28th, 1999 marked the point at which everything came up roses (if you’re in it for free games, you pirate) or went straight to hell (if you’re a company that cares about your IP), which was when a brand new emulator hit the scene: UltraHLE.
UltraHLE introduced two concepts of emulation that were relatively unknown at the time: Dynamic Recompilation (DRC) and library black-boxing, dubbed “High-Level Emulation” or “HLE”.
The former is a straightforward method of speeding up the emulation of high-speed processors in order to achieve more efficiency on commodity hardware. Unlike a standard “interpreter” core that requires the overhead with a function call for every emulated opcode, a DRC core will convert blocks of opcodes into sequences of assembly code that can be executed in one go, often caching off heavily-used translated blocks in order to mitigate the overhead of block conversion.
The latter is also straightforward in principle, but is a bit of a square peg trying to be forced into a round hole when applied to the N64. When applied to systems that have a coprocessor with a fixed program, the principle involves figuring out what the program is doing, and then simply simulating the effects of the commands that are sent to it rather than attempting to simulate the coprocessor itself. This is applicable to the N64 as well, with one rather humongous caveat: There are a number (around 10) of different microcode programs that games uploaded to the RSP (the N64’s vector coprocessor), and not all of them are fully understood. The end result is emulators with many per-game hacks in order to make them work properly.
HLE, in fact, is most likely what Nintendo is using for Virtual Console - this is mostly obvious by the fact that all games appear to be running at a full 640×480 resolution, there is no visible dithering in alpha blending, the Invisible Cap Mario in Super Mario 64 is drawn as completely translucent rather than using a noise dither pattern, and the same issue occurs when Mario’s model fades in and out during teleportation. The latter two issues are kind of sad, actually, as I can think of at least one method that could accomplish the effect on the GX.
Unfortunately, the nature of people wanting free games without supporting their publisher being what it is, the end result of HLE was that popular games were made to run fine, whereas lesser-known games or games that push the RSP in unusual ways run in either a glitchy manner or not at all. I actually suspect that this is the primary reason why certain popular games may be delayed or be completely absent from Virtual Console entirely - Blast Corps relies on the processing speed of the RSP and the drawing speed of the RDP to slow things down rather than run out of control, Goldeneye (licensing notwithstanding) uses a bizarre method for drawing the skybox, and Animal Crossing has many graphical issues on most all HLE-based emulators.
The Plan
I’ve always been an accuracy-minded fellow, and it always pained me that low-level emulation of the N64 went by the wayside in favor of haphazard grabs at popularity and kudos from people interested in little other than piracy. Redemption came in the form of MAME, around two and a half years ago. MAME emulates arcade machines, but there were a number of arcade machines made for the Seta Aleck64 hardware, which is little other than a Nintendo 64 on a custom arcade board and custom per-game inputs. Emulator author Ville Linde wrote a first-pass implementation of the Nintendo 64’s hardware, which ran well enough that Magical Tetris Challenge ran, as it used almost entirely 2D commands.
Over the coming six months during the first half of 2006, Ville put together a Nintendo 64 driver for MESS, the sister project to MAME; whereas MAME concentrates on arcade machines, MESS concentrates on computers and game consoles. A number of games booted, but the majority of them had issues ranging from crashing MESS to major graphical glitches.
In early 2007 I came to the decision that I would start contributing to MESS’s implementation of the Nintendo 64, as Ville went Missing In Action to the siren song of World of Warcraft. Someone was kind enough to donate a Nintendo 64 cartridge copier, and another friend of mine helped me get a cross-compiling version of GCC running that could compile code for the Nintendo 64. The project is located here, uses no official Nintendo development tools or libraries, and as such is completely legal.
Once the groundwork was put down I began to write a test suite for the Nintendo 64 that could run RSP code in parallel with MESS’s implementation, verifying register contents after each opcode was executed. For good measure I threw in a memory viewer, an input tester, and a couple other bits and bobs. Thus was born NUTS: Nintendo Ultra64 Test Suite.
Until late 2007, all of my work was done to produce an initial version of NUTS. However, once NUTS was mostly done, I became interested in tackling the RDP as well, since Ville was well and truly engrossed in World of Warcraft and nobody else was stepping up to the plate. In the past six months I’ve begun working on a new tool, PIN64: Polygon Inspector for Nintendo 64. The end goal is to be able to capture a frame’s worth of RDP commands from MESS, then play back that frame in a separate program, being able to view the individual commands and the RDP’s state while the frame is being built. In addition, a user should be able to then play back that frame on a real Nintendo 64 as well, in order to compare how the RDP commands would behave on real hardware versus the current emulation state. You can see the current state of PIN64 in previous blog posts.
The Rules
The rules are simple: Do everything legally, accurately, and non-competetively.
From a legal standpoint, my intention is for everything to be on the up-and-up. This means that I will not use any confidential Nintendo documents, tools or hardware.
From an accuracy standpoint, I want to make MESS good enough that anything that will run on real hardware will run on MESS with no modifications and no degradation in graphical or aural quality. This means fully emulating every aspect of the RDP including coverage.
From a competition standpoint, unlike the folks who wrote HLE-based emulators, I am not in this for kudos, and I am not in it for free games. My goal is expressly for MESS not to draw attention away from Nintendo’s Virtual Console service. My accuracy goal will probably do this for me, as accuracy will in turn affect the overall emulation speed; the current inaccurate RDP implementation and interpreter RSP implementation causes games to run at an average of 10% of full speed on my average-spec laptop. Things will likely only grow slower as I make the RDP implementation more accurate.
This is not to say that I will reject any patches from other developers that increase the speed of MESS’s Nintendo 64 implementation without affecting accuracy, but I myself will not go out of my way to seek out kludgey optimizations.
In short, my entire goal can be summarized as follows: Do not upset and do no harm to Nintendo.
So far everything is on the up-and-up; now we can only hope that NoA agrees.
