Wednesday, November 4, 2009

NES Emulators and Applescript

One of the things I planned to do after leaving Google was port an NES emulator written in Java to JavaScript. With a JavaScript emulator, it would be possible to play Nintendo games on an iPhone via a web page. (The alternative would be to write an emulator in Objective-C and package it as an iPhone app, but that would be unlikely to make it through Apple's approval process.)

Fortunately, before I sat down to start coding, I did a quick Google search to see whether someone had already done this. In turns out that a few weeks earlier, a student named Ben Firshman released JSNES, a JavaScript NES emulator!

One of the most striking things about JSNES is that it runs at full speed in Google Chrome, but barely runs on Firefox 3.5 or Safari 4. This makes me think we've been going about browser benchmarks all wrong -- I don't care which one can calculate the nth Fibonacci number the fastest, I just care about which one lets me play Contra! (Though to be fair, this probably has more to do with performance differences with a browser's <canvas> implementation rather than its JavaScript engine.)

So there I was, all ready to write all this JavaScript code to find out that it had already been done. Duplicating it seemed like a bad idea, but I still had the NES on my mind, so I started looking around to see what other advances in emulation had come along since I last played around with it a couple of years ago.

I knew Craig had bought a kit some years back which, after some soldering, made it possible for him to plug a real NES controller into a USB port. Fortunately, technology has improved and now you can buy
an adapter with an NES port on one end and a USB port on the other. No soldering required! Since the "USB NES RetroPort" costs $19 and a used NES controller only costs $3, it seemed like a good idea to buy a couple of RetroPorts and a stack of controllers so I can just swap in new controllers when the buttons wear out.

I bought some hardware (the guy who runs retrousb.com was very helpful) and started playing with different emulators to see which ones would support my new controllers. I learned that many emulators do not let you configure your joystick directly; instead, you are expected to install JoyToKey to convert joystick input to keyboard input and then map your joystick to the key commands required for your emulator. Honestly, it worked fine, but I wanted an all-in-one solution.

On Windows, VirtuaNES worked quite well and had support for configuring controllers, but the web site was in Japanese, so it took me awhile to figure out how to do that. Once I confirmed that my controllers were working, I started looking at emulators for Mac because I wanted to rekindle my project from over two years ago of using a PowerPC Mac Mini as my NES emulation hub.

Emulator options are much more limited on Mac. I started out by looking for emulators written in Java, since those should be cross-platform. I took another look at NESCafe (which I reported did not support sound in 2007, but seems to now), but as far as I could tell, it only supported one controller, so that was a deal-breaker. Then I took a look at vNES, which seemed much more promising, except for this FAQ that claimed for reasons unknown, it would not work on a PowerPC Mac.

However, the code for vNES is open sourced under the GPL 3, so I thought I would take a stab at it. The bug turned out to be very simple (it seems like the type of thing that FindBugs should be able to pick out easily):

if(mixerInfo==null || mixerInfo.length==0){
//System.out.println("No audio mixer available, sound disabled.");
Globals.enableSound = false;
return;
}

mixer = AudioSystem.getMixer(mixerInfo[1]);

If you look carefully, you'll see that although there is a check to determine whether mixerInfo is empty, it uses mixerInfo[1] for no documented reason. When I looked at the mixerInfo array, I discovered it only had one value on my PowerPC Mac Mini, two values on my Intel Mac Mini, and nine values on my Vista Thinkpad! I changed the code to use mixerInfo[0] and all was well.

(Aside: Why is the code for every emulator I look at so messy? There are never any comments -- it makes me think one guy figured out how the NES worked and everyone else has just cloned it, so there aren't any comments because no one really knows what is going on. Also, all the classes are in the default package, there are println statements commented out all over the place, etc. In debugging vNES, I tried to clean things up a bit by putting the code in a com.virtualnes package, adding a build.xml file, and refactoring things so it could be run as either an application or an applet. I am making the zip with my changes to vNES 2.11 available on bolinfest.com. I would have tried to contribute a patch to vNES, but the Google Code project appears to be empty.)

Although I got vNES working on my PPC Mini, it was prohibitively slow. Since I had already been playing around with the source code, I considered trying to optimize it, but because of the "no comments in the source code" thing, I realized that could take days. Instead, I went back to Nestopia.

Richard Bannister's Nestopia is a solid emulator for the Mac. It runs at full speed on the PPC and looks great when output to my flatscreen TV. The sound works, both of my controllers hooked up via my RetroPorts work -- this is the real deal.

The only thing it doesn't do is take the path to the ROM as a command-line option, and this is what kept me up past 4am last night. You see, I want to build a Cover Flow UI on top of the emulator for selecting the game to play. To do that, I need to be able to programmatically open Nestopia with a particular ROM file.

If Nestopia were open source, I could have tried to fix it myself to support this feature. In the release notes bundled with Nestopia 1.4.1, the author notes
Martin Freij has generously agreed to license Nestopia to me under a closed-source license for the present. As soon as I have my API kit ready, a buildable version of Nestopia will be released with my shell library. The license for this has yet to be decided but most likely will be normal GPL with my shell excluded under section three of the license. This has been postponed repeatedly due to lack of time but will be released one day - honest!
That dates back to September 27, 2008, so I wasn't going to hold my breath waiting for the source to be released. Besides, from his list of projects, Richard seems to have a lot going on, so I can imagine that he doesn't have the time for this sort of thing.

Regardless, I want my Cover Flow! Because I couldn't change the code for Nestopia, I tried to automate it with AppleScript instead. This is when I should have put the coffee on. According to Google Web History, I did over 100 searches last night while developing my script.

The first thing I that I got to work (after much experimentation) was the following:

on run argv
tell application "Nestopia"
quit
end tell

tell application "Nestopia"
activate
delay 1
end tell

tell application "System Events"
tell process "Nestopia"
-- Click the "Maybe later" button when asked to register.
click at {700, 350}

delay 1
-- Cancel the "Quick Start" mode using the Escape key.
-- It's possible to disable Quick Start in Preferences,
-- but it's useful to have when not using this script.
key code 53

-- Hit command+shift+G to get the "Open Folder" dialog.
delay 1
keystroke "g" using {command down, shift down}

-- because the "Open Folder" dialog only deals with
-- folders and not files, we put each .nes file in
-- its own folder so we can open the folder and
-- then reliably select the only item that comes up
delay 1
keystroke (item 1 of argv)
key code 36

-- use the down arrow to select the file and hit enter
delay 1
key code 125
key code 36

-- go into fullscreen mode using the keyboard shortcut
keystroke "`" using {command down}
end tell
end tell
end run

The part of this script that is particularly gross is the logic with the "Open Folder" dialog. Nestopia displays what appears to be a standard "File Open" dialog, but I could not, for the life of me, figure out how to script it. As you can see, I resorted to using key and mouse events to type in the value I wanted, and had I relied on this, I would have had to have an individual folder for each ROM because Finder (at least on 10.4.11, which is what my PPC Mini runs) lets you type in folder names, but not path names. If there are any AppleScript masters out there, I'd be very curious to see how else you would do this.

Unfortunately, it was not until hours after I started this project that I rediscovered the code I wrote in 2007. Back then, I wrote a CGI script in Perl which would build up some AppleScript and run it from the command line. At the time, this was the easiest way to send a command via HTTP to my Mini to kick off Nestopia:

#!/usr/bin/perl
use CGI qw(param);

# let's get this out of the way before we forget!
print "Content-type: text/html\n\n";

my $rom = param("rom");

# "/Library/WebServer/Documents/nintendo/roms/"
my $folder = 'of folder "roms" ' .
'of folder "nintendo" ' .
'of folder "Documents" ' .
'of folder "WebServer" ' .
'of folder "Library" ' .
'of startup disk';

my $cmd = "osascript -e 'tell application \"Finder\"' " .
" -e 'open file \"$rom\" $folder' " .
" -e 'end tell' ";

system($cmd);
print $cmd;

One thing that you'll notice is that file paths in Finder are gross. I ended up doing the of folder thing because that was the code Script Editor produced when I used Record to help figure out the AppleScript I needed to write. The one good thing about this script, however, was that it reminded me that simply opening the file would trigger Nestopia because it is the application associated with ROM files on my Mac. This helped me clean up my current script considerably:

on run argv
tell application "Finder" to open file ((POSIX file (item 1 of argv)) as string)

delay 1

tell application "Nestopia" to activate

-- Make sure "Enable access for assistive devices" is enabled in
-- Universal Access under System Preferences or else
-- System Events won't work:
-- http://dougscripts.com/itunes/itinfo/keycodes.php
tell application "System Events"
tell process "Nestopia"
-- click the "Maybe later" button when asked to register
click at {700, 350}

-- go into fullscreen mode using the keyboard shortcut
delay 1
keystroke "`" using {command down}
end tell
end tell
end run

It took me at least half an hour of Googling until I came across a solution for passing in the file path as an argument. Apparently AppleScript only deals with HFS paths instead of POSIX paths like everyone else. It is particularly frustrating that Script Editor allows you to write POSIX file "/Users/bolinfest/drmario.nes", but as soon as you compile the code, it becomes file "Macintosh HD:Users:bolinfest:drmario.nes". What kind of editor rewrites your code into some kind of unmaintainable equivalent when you compile it?

I'm exhausted, so I haven't even started working on the Cover Flow part of the project yet, but at least I've resolved one of the big issues. It looks like there are working examples of Cover Flow UIs in JavaScript, so I will likely set up a web server on my Mac Mini with a similar CGI script that will shell out to my compiled AppleScript to launch the ROM. That way, I'll be able to browse my NES catalog from Safari on my iPhone and kick things off from there!

6 comments:

  1. I have an emulator, a NES rom, and even a USB gamepad... why doesn't my gamepad just work? Maybe you know? Do I need to do something to make it work?
    alanearing@yahoo.com

    ReplyDelete
  2. It depends on the emulator. Unfortunately, most joysticks do not "just work" with an emulator. Emulators with joystick support generally have a preferences pane where you select the button on the NES controller you are trying to emulate and then press the button on your joystick that you would like to map to that button. It is basically the same as configuring the keyboard mapping if you don't have a joystick. Unfortunately, most of the emulators that I tried did not make this obvious from their configuration UI.

    ReplyDelete
  3. This all worked fine for me, except that whenever I play a game, Nestopia sets itself as the default application for the rom file. Have you run into this problem?

    ReplyDelete
  4. I believe that's how my setup works, though I don't really consider it a problem -- are you bouncing back and forth between emulators?

    ReplyDelete
  5. Ah, it makes sense that this wouldn't be a problem for you - I got stuck in my own world and forgot about the differences between our setups. I used Platypus (http://www.sveinbjorn.org/platypus) to create a proper app from the applescript and set the rom files to open with this new app by default. Therefore, in this setup, you could have a list of games (e.g., in a stack) that would open properly (full screen, etc) with just one click; I wasn't planning on going quite as far as you with the interface. I did the same setup successfully with Kega Fusion (http://www.eidolons-inn.net/tiki-index.php?page=kega), an excellent Genesis emulator, since Kega Fusion doesn't try to set itself as the default application for a file after opening it. Nestopia, on the other hand, does set itself as the default application for a rom file after opening it, so the second time I try to run the rom, my custom script is out of the loop. I'm fairly positive that the creator type can be set via applescript (e.g., after Nestopia opens the file), but I wasn't able to successfully do it the other night.

    ReplyDelete
  6. I actually was looking for the same effect you were and was rather successful with my efforts. I though did purchase Enhance Emulator so getting rid of the dialogue box was not an issue for me. I have setup a Remote Buddy custom menu with thumbnails and screenshots to browse both Nestopia, Genesis Plus, BSNES and TGEmu games which launch fullscreen with the selected game.

    I simply used the following and saved it as an app.


    tell application "genesisplus" (or Nestopia, BSNES, TGEmu)

    open "/mypathtomyroms/romname.smd"
    activate
    end tell

    ReplyDelete