Saturday, March 31, 2007

Request for Help: XMPP library for Firefox



For quite some time, I have wanted to have a Jabber library in Firefox that I could build extensions around. For my experiments with Google Talk on the desktop, I have been using the Smack API. Smack has served me well, so I decided to try to use it as a model for an equivalent library written in JavaScript for Firefox, leveraging XPCOM. It turns out that Smack has a kickass debug mode that shows all of the XML messages that are sent back and forth while communicating with the server. Below, you can see the messages that Smack and Google Talk exchange when initiating a connection (client messages are in red, server messages are in blue):

<stream:stream to="gmail.com" xmlns="jabber:client" xmlns:stream="http://etherx.jabber.org/streams" version="1.0">


<stream:stream from="gmail.com" id="ABAC7E59A46C522E" version="1.0" xmlns:stream="http://etherx.jabber.org/streams" xmlns="jabber:client">
<stream:features>
<starttls xmlns="urn:ietf:params:xml:ns:xmpp-tls">
<required/>
</starttls>
<mechanisms xmlns="urn:ietf:params:xml:ns:xmpp-sasl">
<mechanism>X-GOOGLE-TOKEN</mechanism>
</mechanisms>
</stream:features>


<starttls xmlns="urn:ietf:params:xml:ns:xmpp-tls"/>


<proceed xmlns="urn:ietf:params:xml:ns:xmpp-tls"/>


By following a XULPlanet tutorial on sockets, I successfully duplicated the above exchange in Firefox using trusted JavaScript. Unfortunately, then I hit a snag when I tried to replicate the following Java code in JavaScript:


/**
* proceedTLSReceived() in org.jivesoftware.smack.XMPPConnection
*/
void proceedTLSReceived() throws Exception {
SSLContext context = SSLContext.getInstance("TLS");
// Verify certificate presented by the server
context.init(null, // KeyManager not required
new javax.net.ssl.TrustManager[]{new ServerTrustManager(serviceName, configuration)},
new java.security.SecureRandom());
Socket plain = socket;
// Secure the plain connection
socket = context.getSocketFactory().createSocket(plain,
plain.getInetAddress().getHostName(), plain.getPort(), true);
socket.setSoTimeout(0);
socket.setKeepAlive(true);
// Initialize the reader and writer with the new secured version
initReaderAndWriter();
// Proceed to do the handshake
((SSLSocket) socket).startHandshake();

// Set that TLS was successful
usingTLS = true;

// Set the new writer to use
packetWriter.setWriter(writer);
// Send a new opening stream to the server
packetWriter.openStream();
}


As you can see, this requires some logic to deal with security. Although it appears that Firefox has code to handle SSL and TLS, I haven't figured out a way to access it from JavaScript. One option would be to write my own XPCOM object in C++, which is what I suspect Process-one did for their XUL-based XMPP client. Unfortunately, I am not that comfortable with C++, nor am I interested in the overhead involved in compiling for multiple platforms, which is really why I was looking for an existing, scriptable XPCOM object to do the heavy-lifting for me in the first place.

Another possibility is writing an XPCOM object in a language other than C++. I started to explore PyXPCOM for writing modules in Python, but got discouraged when I discovered that Brendan Eich himself commented that PyXPCOM does not make it practical to write Firefox extensions in Python because it requires bundling Python with the extension. I'm guessing that the XPCOM language bindings for Perl, Ruby, and Java have similar limitations.

Then again, what about Java? Unlike Perl and Ruby, there is a standard Java plugin for Firefox, so maybe using JavaXPCOM is a possibility. And even if not, LiveConnect might do the trick, since we have successfully used it for Java-JavaScript communication in Chickenfoot. Being able to leverage Smack as-is would certainly be a windfall if it worked!

Though I am likely going to pursue the Java path to make this happen, I think it would be great if someone with the right know-how created a C++ XPCOM object that exposed a scriptable interface that would let you connect to a Jabber server (using TLS) and provide access to input and output streams for that socket. If anyone knows his way around the GAIM code, he or she could likely leverage what is already there to make this happen. I expect the "plumbing" involved in bringing in the existing C++ code, creating the IDL, and compiling for the major platforms, is likely where more than 80% of the work is – the new code that needs to be written should be fairly minimal.

For the end-user, this would provide the best experience since he would be able to install a Firefox extension without having to install the Java plugin on top of it. Also, the C++ XPCOM object would likely be faster, because in developing Chickenfoot, I learned that crossing the Java/C++ process boundary can be expensive. (This is described in section 8.1.1 of my thesis.) So if anyone is interested in taking on this project, or if you know of an existing solution, then please let me know!

Thursday, March 29, 2007

MLB 2007 Schedules for Google Calendar

I wrote a Chickenfoot script to generate the 2007 Major League Baseball schedules from mlb.com. Instead of putting the schedules directly into Google Calendar, I posted them on wikicalendars.com. Just visit the site, click on your favorite team (let's go Mets!), and click on the "Google Calendar" button above the calendar preview to add the schedule to your Calendar account (or click the iCal icon to add it to Apple iCal).

Subscribe to New York Mets 2007 Schedule from wikicalendars.com



But wait, "What is wikicalendars.com?" you wonder. I realized that I forgot to post about it before, so I should explain. Last fall, I wrote about how I created mitcalendars.com with Mike Lambert to make it easy to add MIT course schedules to Google Calendar. After getting feedback from students, we realized that there were two major issues with our site:


  • Students could not update or correct the calendars we had auto-generated.
  • We had to guess which permutations of calendars for a course we should generate: if there are 2 lecture choices , 8 recitation choices, and 3 lab choices, should we generate 48 calendars for the course? How do we present all 48 options a to user?


To address the first issue, we decided to store the calendar data in a wiki so that anyone on the web could update it. This way, if a class were rescheduled, or if a final exam were added, anyone from the class could adjust the course's calendar accordingly. Obviously spam would be a concern, so we decided to go with the ever-popular MediaWiki, since we assumed that Wikipedia has more experience with spam than anyone, so they likely have the best tools to combat it. Also, because it was open-source, it was easy for Mike and I to hack it up as we needed.

To address the second issue, we added a feature called "calendar splice" that enables you to combine multiple calendars from wikicalendars.com to dynamically create a new calendar. This way, we could create a calendar for each lecture and recitation offering for a course, but leave it up to the students to combine a lecture and recitation into one calendar. Unfortunately, this is not obvious from the wikicalendars.com user interface yet, but the feature is there. (Incidentally, we offer a number of output formats for calendar data, facilitating the use of wikicalendar data in mashups, etc.)

Because we wanted to make it as easy as possible to edit calendar data, we decided that we needed to create our own wiki-language for the site. Here's a snippet from the NY Mets 2007 calendar:

__CALENDAR__
CalendarName: New York Mets 2007 MLB Schedule
CalendarTimezone: America/New_York
CalendarURL: http://mlb.mlb.com/schedule/sortable.jsp?c_id=nym

DefaultEventDuration: 3h

EventWhen: 4/1/2007 8:05pm
EventWhat: Mets vs. Cardinals (Away)
EventWhere: Away

Unfortunately, unlike WikiMedia's wiki-language, our format strongly favors English speakers -- especially those who use American date and time formats. Though we were not thrilled with this decision, we felt it was acceptable for our first version of the site. We tried to focus on simplicity, and I think many will agree that 4/1/2007 8:05pm is much simpler for the average person to work with than 200704012005/200704012305. The calendar syntax is documented in detail on the web site.

So given all of this, what makes a good wikicalendar? Arguably, any calendar whose events come from a community of people rather than from one well-defined leader. For example, I started a calendar for release dates of Wii games which would ideally take contributions from those who troll gaming rumor boards and web sites. (I would love to see people start contributing!) I also created a calendar to track when NYC Marathon qualifier races are, as there are many of us who aren't interested in the non-qualifier races. I try to keep the calendar fairly up-to-date myself, but if I fall behind, hopefully another calendar subscriber will notice and pick up the slack!

Which brings me to why I decided to store the MLB calendars on wikicalendars.com: though the schedule data is correct now, the baseball season is subject to rain outs and reschedulings, and I can't stay on top of all of those changes, but Time's Person of the Year can! This is, in many ways, an experiment. Will anyone keep these calendars up to date? Will they lie dormant and become inaccurate, or even worse, become littered with spam? I guess we'll see!

Sunday, March 25, 2007

I found a use for my Mac Mini -- running 1980s technology!

I have a PowerPC Mac Mini that I bought in the summer of 2005. The best way to describe it is sloooooooowwwwwww. It lags far behind the T42 that I bought the year before, so when Apple made the switch to Intel and all the Macheads were rejoicing about what a brilliant move it was, I was trying to figure out why no one else was angry that Apple had been selling unnecessarily slow computers to its users for years. Oh, right.

So now that I've moved on to faster machines, what am I supposed to do with my Mini? One thought was to use it as a DVD player, but we already have an Xbox 360 that does that, so that's not very exciting. (Also, my Mac Mini predates Apple Front Row, so I built my own makeshift version which required running Apache as root so it could execute CGI scripts to run AppleScript to control the Mac's DVD player so I could have a webapp that I could access from my laptop to use as a remote, but that's a whole other story...)

As I mentioned previously, I wished it were possible to use one of the web-based NES emulators on Wii Opera so it would be possible to play old games using a Wiimote. And then it dawned on me – what if I could just get a Wiimote working with my computer and use the emulator there instead? On sites like Engadget, I saw that some people had already built libraries for handling Wiimote input, so I did some searching and found the leading Wiimote library for Mac: DarwiinRemote.

Though there were loads of blogs pointing to DarwiinRemote, none of them actually explained how to use it (neither does the DawiinRemote SourceForge project page), but eventually I found a nice writeup on using DarwiinRemote that got me started. Though Darwiin's "Mouse Mode" was a little wonky, it was clear that my Mac was receiving Wiimote button presses perfectly.

The next step was to find a good NES emulator. I started out with NESCafe, which I had previously enjoyed using on Windows, but on the Mac it (1) ran too slow, (2) couldn't fill the screen, and (3) could not support sound emulation. I decided that a Mac-specific emulator was probably the way to go, so I did a brief survey of emulators available for Mac, settling on Nestopia.

Then it was time to put the two pieces of software together. I was reading the release notes for DarwiinRemote 0.5 which said that support had been added for the Classic Controller. Excellent -- that would be even more fun than using a Wiimote! Unfortunately, from my testing, it appeared that only partial support was available: for example, the "up" and "left" buttons on the D-pad registered, but not "right" and "down" (if you know any game where you only need to move northwest, pleast let me know!) Though the release notes for DarwiinRemote 0.5 list claim that PowerPC bugs were fixed, the release notes for WiiRemoteFramework 0.5 admit "#1623337 PPC bugs might be fixed." I have checked out the code to see if I can fix this myself, though I imagine if I am patient, the developers will have this fixed in the next release.

In the interim, I decided to try using the emulator by turning my Wiimote sideways. Here is a screenshot of how I configured both DarwiinRemote and Nestopia:

Configuring DarwiinRemote and Nestopia

Like I said, I hold the Wiimote horizontally when I use it to play with the emulator, so that's why the D-pad mapping in DarwiinRemote may appear odd. Also, DarwiinRemote currently only has a fixed set of keys that you can map Wiimote buttons to, which is why I'm using F1 and F2 for B and A, respectively.

Now that I have everything working, I must say that the experience is fantastic! The emulator and the Wiimote have been working beautifully together: everything seems to be responding as it should. There are only a few open issues:


  • As mentioned above, I would prefer to use the Classic Controller over the Wiimote. It looks like a lot of great progress has been made on DarwiinRemote in recent history, so I'm optimistic about that.
  • I would like to make a more Wiimote-friendly File->Open dialog for Nestopia. Ideally, one that could be navigated quickly with the D-pad, shows the covers of the games, etc. Ultimately, I want to connect the Mac Mini directly into my TV such that everything can be done easily with the Wiimote so I don't have to leave a keyboard or mouse hooked up to it.
  • I need to get support for two-player mode. Honeslty, I tried playing Contra by myself yesterday and it wasn't that much fun (the 30 life code worked just fine, though!). It looks like Nestopia supports input from two players, but DarwiinRemote only seems to take input from one Wiimote at a time right now. I fear that adding that feature to DarwiinRemote may take awhile, though I suppose if the Classic Controller were working, you could have one player use the Wiimote horizontally and the other use the Classic while tethered to it since together they compose one bluetooth device to Darwiin.


Nevertheless, I hope these issues don't deter you from trying out this setup because it's really a lot of fun! Seriously, if you have a bluetooth-enabled Mac, for the cost of a Wiimote ($40), you can have your own complete Virtual Console!

PS If you're looking for ROMs, I recommend checking out http://everyvideogame.com/. They use NESCafe as a Java applet to play the games on the site and it looks like each ROM lies in a zip file identified in the <PARAM> tag for the applet, so they are pretty easy to find. Though I obviously often promote Chickenfoot for this sort of task, I admit that Perl and wget will likely do the job just fine in this case.

Sunday, March 11, 2007

JavaScript library for Wii Opera

When I heard that Opera was going to be bundled with the Wii, my first thought was whether they would make it be possible to catch Wii button events in JavaScript, and fortunately they did! Though a number of web sites have documented the button keycodes, I couldn't find a simple drop-in library for handling the button events in JavaScript, so I created my own. It is pretty simple, but hopefully it will help some eager Wii hackers get started. (Though it allows you to control a web site with your Wiimote, it also has a mode that supports control from the keyboard, making it easier to develop your Wii-friendly web site from your laptop.)

PS It is my dream that someone combine (or "mashup" in the parlance of our modern Web 2.0 times) NESCafe and the Google Web Toolkit to build the ultimate Wii-friendly web site. What can I say? They just can't release old NES titles on the Virtual Console fast enough for me!