Chromebook Pixel gives me an excuse to fork JSNES

For a long time, I have been intrigued by NES emulators. I was extremely excited when Ben Firshman released an NES emulator in JavaScript (JSNES) over three years ago. At the time, I noted that JSNES ran at almost 60fps in Chrome, but barely trickled along in Firefox. It's pretty shocking to see that that is still the case today: in Firefox 21.0 on Ubuntu, I am seeing at most 2fps while sitting idle on the title screen for Dr. Mario. That's pretty sad. (It turns out I was getting 1fps because I had Firebug enabled. Maybe that's why my perception of Firefox has diminished over time. Someone at Mozilla should look into that...) This is the only browser benchmark I care about these days.

When the Gamepad API was first announced for Chrome, I tried to get my USB NES RetroPort controllers to work, but Chrome did not seem to recognize them. I made a mental node to check back later, assuming the API would eventually be more polished. Fast-forward to this week where I was fortunate enough to attend Google I/O and score a Chromebook Pixel. It seemed like it was time to give my controllers another try.

Last night, I plugged a RetroPort into the Pixel and visited the Gamepad API test page, and it worked! Obviously the next thing I had to do was wire this up to JSNES, so that was the first thing I did when I woke up this morning. I now have my own fork of the JSNES project where I added support for the RetroPort controllers as well as loading local ROMs from disk. As I admit in my README.md, there are already outstanding pull requests for these types of things, but I wanted to have the fun of doing it myself (and an excuse to poke around the JSNES code).

Finally, the one outstanding feature I hoped to add was loading ROMs from Dropbox or GDrive using pure JavaScript. Neither product appears to have a simple JavaScript API that will give you access to file data like the W3C File API does. Perhaps I'll host my fork of JSNES if I can ever add such a feature...

P.S. I should admit that one does not need a Pixel to do these types of things. However, having a new piece of hardware and APIs that have been around long enough that you expect them to be stable is certainly a motivating factor. It's nice to have a project that doesn't involve any yak-shaving, such as figuring out how to install a version of Chrome from the Beta channel!

Generating Google Closure JavaScript from TypeScript

Over a year ago, I created a prototype for generating Google Closure JavaScript from CoffeeScript. Today, I am releasing a new, equivalent prototype that uses TypeScript as the source language.

<shamelessplug>I will be speaking at mloc.js next month in Budapest, Hungary, which is a conference on scaling JavaScript development. There, I will discuss various tools to help keep large codebases in check, so if you are interested in hearing more about this work, come join me in eastern Europe in February!</shamelessplug>

I have been interested in the challenge of Web programming in the large for quite some time. I believe that Google Closure is currently still the best option for large-scale JavaScript/Web development, but that it will ultimately be replaced by something that is less verbose. Although Dart shows considerable promise, I am still dismayed by the size of the JavaScript that it generates. By comparision, if TypeScript can be directly translated to JavaScript that can be compiled using the advanced mode of the Closure Compiler, then we can have all the benefits of optimized JavaScript from Closure without the verbosity. What's more, because TypeScript is a superset of JavaScript, I believe that its syntax extensions have a chance of making it into the ECMAScript standard at some point, whereas the chances of Dart being supported natively in all major browsers is pretty low.

In terms of hacking on TypeScript itself, getting started with the codebase was a pain because I develop on Linux, and presumably TypeScript's developers do not. Apparently Microsoft does not care that cloning a CodePlex repository using Git on Linux does not work. Therefore, in order to get the TypeScript source code, I had to switch to a Mac, clone the repo there, and then import it into GitHub so I could clone it from my Linux box.

Once I got the code on my machine, the next step was figuring out how to build it. TypeScript includes a Makefile, but it contains batch commands rather than Bash commands, so of course those did not work on Linux, either. Modifying the Makefile was a bit of work, and unfortunately will likely be a pain to reconcile with future upstream changes. It would be extremely helpful if Microsoft switched to a cross-platform build tool (or maintained two versions of the Makefile themselves).

Once I was able to build, I wanted to start hacking and exploring the TypeScript codebase. I suspected that reading the .ts files as plaintext without any syntax highlighting would be painful, so I Googled to see if there were any good alternatives to Visual Studio with TypeScript support that would run on Linux. Fortunately, JetBrains recently released a beta of PhpStorm and WebStorm 6.0 that includes support for TypeScript, so I decided to give it a try. Given that both TypeScript and support for it in WebStorm are very new, I am quite impressed with how well it works today. Syntax highlighting and ctrl+clicking to definitions work as expected, though PhpStorm reports many false positives in terms of TypeScript errors. I am optimistic that this will get better over time.

Now that I have all of the setup out of the way, the TypeScript codebase is pretty nice to work with. The source code does not contain much in the way of comments, but the code is well-structured and the class and variable names are well-chosen, so it is not too hard to find your way. I would certainly like to upstream some of my changes and contribute other fixes to TypeScript, though if that Git bug on CodePlex doesn't get fixed, it is unlikely that I will become an active contributor.

Want to learn more about Closure? Pick up a copy of my book, Closure: The Definitive Guide (O'Reilly), and learn how to build sophisticated web applications like Gmail and Google Maps!

New Essay: Caret Navigation in Web Applications

For those of you who don't follow me on Twitter, yesterday I announced a new essay: Caret Navigation in Web Applications. There I talk about some of the things that I worked hard to get right when implementing the UI for Google Tasks.

This essay may not be for JavaScript lightweights, though it didn't seem to gain much traction on Hacker News or on Reddit (which is particularly sad because Reddit readers were much of the reason that I took the time to write it in the first place). I suppose it would have been more popular if I had made it a lot shorter, but I believe it would have been much less useful if I did. That's just my style, I guess: Closure: The Definitive Guide was originally projected to be 250-300 pages and it ended up at 550.

I also wanted to take this opportunity to make some more comparisons about Google Tasks vs. Asana that weren't really appropriate for the essay:

• It's annoying that Asana does not support real hierarchy. Asana has responded to this on Quora, but their response is really about making Asana's life easier, not yours. What I think is really interesting is that if they ever decide to reverse their decision, it will really screw with their keyboard shortcuts since they currently heavily leverage Tab as a modifier, but that may not work so well when users expect Tab to indent (and un-indent).
• It's pretty subtle, but when you check something off of your list in Google Tasks, there is an animated strikethrough. I did this by creating a timing function that redraws the task with the </s> in an increasing position to achieve the effect. This turned out to be tricky to get right because you want the user to be able to see it (independent of task text length) without it feeling slow for long tasks. Since so many people seem to delight in crossing things off their list, this always seemed like a nice touch in Google Tasks. I don't find crossing things off my list in Asana as satisfying.
• Maybe I haven't given it enough time yet, but I have a lot of trouble with Asana's equivalent of the details pane. The thing seems to slide over when I don't want it to, or I just want to navigate through my tasks and have the details pane update and it starts bouncing on me...I don't know. Am I alone? In Tasks, it's shift-enter to get in and shift-enter to get out. It's not as sexy since there's no animation, but it's fast, it works, and it doesn't get in my way.

Finally, some friends thought I should include a section in the essay about "what browser vendors can do to help" since clearly the browser does not have the right APIs to make things like Google Tasks easy to implement. As you may imagine, I was getting pretty tired of writing the essay, so I didn't include it, but this is still an important topic. At a minimum, an API to return the current (x, y) position of the cursor as well as one to place it at an (x, y) position would help, though even that is ambiguous since a cursor is generally a line, not a point. It would be interesting to see how other UI toolkits handle this.

Oh, and I should mention that if I were to reimplement Google Tasks again today, I would not implement it the way that I did, if I had enough resources. If you look at Google Docs, you will see that your cursor is not a real cursor, but a blinking <div>. That means that Docs is taking responsibility for turning all of your mouse and keyboard input into formatted text. It eliminates all of the funny stuff with contentEditable by basically rebuilding...well, everything related to text editing in the browser. If I had libraries like that to work with (that already have support for pushing changes down to multiple editors in real time), then I would leverage those instead. Of course, if you don't have code like that available, then that is a pretty serious investment to make. Engineering is all about picking the right tradeoffs!

Want to learn more the technologies used to build Google Tasks? Pick up a copy of my new book, Closure: The Definitive Guide (O'Reilly).

I want a magical operator to assuage my async woes (and a pony)

Lately, I have spent a lot of time thinking about how I could reduce the tedium of async programming in JavaScript. For example, consider a typical implementation of using an XMLHttpRequest to do a GET that returns a Deferred (this example uses jQuery's implementation of Deferred, but there are many other reasonable implementations, and there is a great need to settle on a standard API, but that is a subject for another post):

/** @return {Deferred} */
var simpleGet = function(url) {
  var deferred = new $.Deferred();

  var xhr = new XMLHttpRequest();
  xhr.onreadystatechange = function() {
    if (xhr.readyState == 4) {
      if (xhr.status == 200) {
        deferred.resolve(xhr.responseText);
      } else {
        deferred.reject(xhr.status);
      }
    }
  };
  xhr.open('GET', url, true /* async */);
  xhr.send(null);  

  return deferred;
};

What I want is a magical ~ operator that requires (and understands) an object that implements a well-defined Deferred contract so I can write my code in a linear fashion:

/** @return {Deferred} */
var getTitle = function(url) {
  if (url.substring(0, 7) != 'http://') url = 'http://' + url;
  var html = ~simpleGet(url);
  var title = html.match(/<title>(.*)<\/title>/)[1];
  return title;
};

/** Completes asynchronously, but does not return a value. */
var logTitle = function(url) {
  try {
    var title = ~getTitle(url);
    console.log(title);
  } catch (e) {
    console.log('Could not extract title from ' + url);
  }
};

Unfortunately, to get this type of behavior today, I have to write something like the following (and even then, I am not sure whether the error handling is quite right):

/** @return {Deferred} */
var getTitle = function(url) {
  if (url.substring(0, 7) != 'http://') url = 'http://' + url;

  var deferred = new $.Deferred();
  
  simpleGet(url).then(function(html) {
    var title = html.match(/<title>(.*)<\/title>/)[1];
    deferred.resolve(title);
  }, function(error) {
    deferred.reject(error);
  });

  return deferred;
};

/** Completes asynchronously, but does not return a value. */
var logTitle = function(url) {
  var deferred = getTitle(url);
  deferred.then(function(title) {
    console.log(title);
  }, function(error) {
    console.log('Could not extract title from ' + url);
  })
};

I am curious how difficult it would be to programmatically translate the first into the second. I spent some time playing with generators in Firefox, but I could not seem to figure out how to emulate my desired behavior. I also spent some time looking at the ECMAScript wiki, but it is unclear whether they are talking about exactly the same thing.

In terms of modern alternatives, it appears that C#'s await and async keywords are the closest thing to what I want right now. Unfortunately, I want to end up with succinct JavaScript that runs in the browser, so I'm hoping that either CoffeeScript or Dart will solve this problem, unless the ECMAScript committee gets to it first!

Please feel free to add pointers to related resources in the comments. There is a lot out there to read these days (the Dart mailing list alone is fairly overwhelming), so there's a good chance that there is something important that I have missed.

Update (Fri Oct 28, 6:15pm): I might be able to achieve what I want using deferred functions in Traceur. Apparently I should have been looking at the deferred functions strawman proposal more closely: I skimmed it and assumed it was only about defining a Deferred API.

Want to learn about a suite of tools to help manage a large JavaScript codebase? Pick up a copy of my new book, Closure: The Definitive Guide (O'Reilly), and learn how to build sophisticated web applications like Gmail and Google Maps!

An Examination of goog.base()

A few weeks ago, I started working on adding an option to CoffeeScript to spit out Closure-Compiler-friendly JavaScript. In the process, I discovered that calls to a superclass constructor in CoffeeScript look slightly different than they do in the Closure Library. For example, if you have a class Foo and a subclass Bar, then in CoffeeScript, the call [in the generated JavaScript] to invoke Foo's constructor from Bar's looks like:

Bar.__super__.constructor.call(this, a, b);

whereas in the Closure Library, the canonical thing is to do the following, identifying the superclass function directly:

Foo.call(this, a, b);

The two are functionally equivalent, though CoffeeScript's turns out to be slightly simpler to use as a developer because it does not require the author to know the name of the superclass when writing the line of code. In the case of CoffeeScript, where JavaScript code generation is being done, this localization of information makes the translation of CoffeeScript to JavaScript easier to implement.

The only minor drawback to using the CoffeeScript form when using Closure (though note that you would have to use superClass_ instead of __super__) is that the CoffeeScript call is more bytes of code. Unfortunately, the Closure Compiler does not know that Bar.superClass_.constructor is equivalent to Foo, so it does not rewrite it as such, though such logic could be added to the Compiler.

This piqued my curiosity about how goog.base() is handled by the Compiler, so I ended up taking a much deeper look at goog.base() than I ever had before. I got so caught up in it that I ended up composing a new essay on what I learned: "An Examination of goog.base()."

The upshot of all this is that in my CoffeeScript-to-Closure translation code, I am not going to translate any of CoffeeScript's super() calls into goog.base() calls because avoiding goog.base() eliminates a couple of issues. I will still use goog.base() when writing Closure code by hand, but if Closure code is being autogenerated anyway, then using goog.base() is not as compelling.

Finally, if you're wondering why I started this project a few weeks ago and have not made any progress on the code since then, it is because I got married and went on a honeymoon, so at least my wife and I would consider that a pretty good excuse!

Want to learn more about Closure? Pick up a copy of my new book, Closure: The Definitive Guide (O'Reilly), and learn how to build sophisticated web applications like Gmail and Google Maps!

Writing useful JavaScript applications in less than half the size of jQuery

Not too long ago, I tried to bring attention to how little of the jQuery library many developers actually use and argue that frontend developers should consider what sort of improvements their users would see if they could compile their code with the Advanced mode of the Closure Compiler. Today I would like to further that argument by taking a look at TargetAlert, my browser extension that I re-released this week for Chrome.

TargetAlert is built using Closure and plovr using the template I created for developing a Chrome extension. The packaged version of the extension includes three JavaScript files:

Name	Size (bytes)	Description
targetalert.js	19475	content script that runs on every page
options.js	19569	logic for the TargetAlert options page
targetalert.js	3590	background page that channels information from the options to the content script
Total	42634

By comparison, the minified version of jQuery 1.6 is 91342 bytes, which is more than twice the size of the code for TargetAlert. (The gzipped sizes are 14488 vs. 31953, so the relative sizes are the same, even when gzipped.)

And to put things in perspective, here is the set of goog.require() statements that appear in TargetAlert code, which reflects the extent of its dependencies:

goog.require('goog.array');
goog.require('goog.dispose');
goog.require('goog.dom');
goog.require('goog.dom.NodeIterator');
goog.require('goog.dom.NodeType');
goog.require('goog.dom.TagName');
goog.require('goog.events');
goog.require('goog.events.EventType');
goog.require('goog.string');
goog.require('goog.ui.Component');
goog.require('goog.Uri');
goog.require('soy');

I include this to demonstrate that there was no effort to re-implement parts of the Closure Library in order to save bytes. On the contrary, one of the primary reasons to use Closure is that you can write code in a natural, more readable way (which may be slightly more verbose), and make the Compiler responsible for minification. Although competitions like JS1K are fun and I'm amazed to see how small JS can get when it is hand-optimized, even the first winner of JS1K, Marijn Haverbeke, admits, "In terms of productivity, this is an awful way of coding."

When deploying a packaged browser extension, there are no caching benefits to consider: if your extension includes a copy of jQuery, then it adds an extra 30K to the user's download, even when gzipped. To avoid this, your extension could reference https://ajax.googleapis.com/ajax/libs/jquery/1.6.1/jquery.min.js (or equivalent) from its code, but then it may not work when offline. Bear in mind that in some parts of the world (including the US! think about data plans for tablets), users have quotas for how much data they download, so you're helping them save a little money if you can package your resources more efficiently.

Further, if your browser extension has a content script that runs on every page, keeping your JS small reduces the amount of code that will be executed on every page load, minimizing the impact of your extension on the user's browsing experience. As users may have many extensions installed, if everyone starts including an extra 30K of JS, then this additional tax can really start to add up! Maybe it's time you gave Closure a good look, if you haven't already.

Want to learn more about Closure? Pick up a copy of my new book, Closure: The Definitive Guide (O'Reilly), and learn how to build sophisticated web applications like Gmail and Google Maps!

The Triumphant Return of TargetAlert!

About seven years ago, my adviser and I were sitting in his office Googling things as part of research for my thesis. I can't remember what we were looking for, but just after we clicked on a promising search result, the Adobe splash screen popped up. As if on cue, we both let out a groan in unison as we waited for the PDF plugin to load. In that instant, it struck me that I could build a small Firefox extension to make browsing the Web just a little bit better.

Shortly thereafter, I created TargetAlert: a browser extension that would warn you when you were about to click on a PDF. It used the simple heuristic of checking whether the link ended in pdf, and if so, it inserted a PDF icon at the end of the link as shown on the original TargetAlert home page.

And that was it! My problem was solved. Now I was able to avoid inadvertently starting up Adobe Reader as I browsed the Web.

But then I realized that there were other things on the Web that were irritating, too! Specifically, links that opened in new tabs without warning or those that started up Microsoft Office. Within a week, I added alerts for those types of links, as well.

After adding those features, I should have been content with TargetAlert as it was and put it aside to focus on my thesis, but then something incredible happened: I was Slashdotted! Suddenly, I had a lot more traffic to my site and many more users of TargetAlert, and I did not want to disappoint them, so I added a few more features and updated the web site. Bug reports came in (which I recorded), but it was my last year at MIT, and I was busy interviewing and TAing on top of my coursework and research, so updates to TargetAlert were sporadic after that. It wasn't until the summer between graduation and starting at Google that I had time to dig into TargetAlert again.

Though the primary reason that TargetAlert development slowed is that Firefox extension development should have been fun, but it wasn't. At the time, every time you made a change to your extension, you had to restart Firefox to pick up the change. As you can imagine, that made for a slow edit-reload-test cycle, inhibiting progress. Also, instead of using simple web technologies like HTML and JSON, Firefox encouraged the use of more obscure things, such as XUL and RDF. The bulk of my energy was spent on getting information into and out of TargetAlert's preferences dialog (because I actually tried to use XUL and RDF, as recommended by Mozilla), whereas the fun part of the extension was taking the user's preferences and applying them to the page.

The #1 requested feature for TargetAlert was for users to be able to define their own alerts (as it were, users could only enable or disable the alerts that were built into TargetAlert). Conceptually, this was not a difficult problem, but realizing the solution in XUL and RDF was an incredible pain. As TargetAlert didn't generate any revenue and I had other personal projects (and work projects!) that were more interesting to me, I never got around to satisfying this feature request.

Fast-forward to 2011 when I finally decommissioned a VPS that I had been paying for since 2003. Even though I had rerouted all of its traffic to a new machine years ago and it was costing me money to keep it around, I put off taking it down because I knew that I needed to block out some time to get all of the important data off of it first, which included the original CVS repository for TargetAlert.

As part of the data migration, I converted all of my CVS repositories to SVN and then to Hg, preserving all of the version history (it should have been possible to convert from CVS to Hg directly, but I couldn't get hg convert to work with CVS). Once I had all of my code from MIT in a modern version control system, I started poking around to see which projects would still build and run. It turns out that I have been a stickler for creating build.xml files for personal projects for quite some time, so I was able to compile more code than I would have expected!

But then I took a look at TargetAlert. The JavaScript that I wrote in 2004 and 2005 looks gross compared to the way I write JavaScript now. It's not even that it was totally disorganized -- it's just that I had been trying to figure out what the best practices were for Firefox/JavaScript development at the time, and they just didn't exist yet.

Further, TargetAlert worked on pre-Firefox 1.0 releases through Firefox 2.0, so the code is full of hacks to make it work on those old versions of the browser that are now irrelevant. Oh, and what about XUL? Well, my go-to resource for XUL back in the day was xulplanet.com, though the site owners have decided to shut it down, which made making sense of that old code even more discouraging. Once again, digging into Firefox extension development to get TargetAlert to work on Firefox 4.0 did not appear to be much fun.

Recently, I have been much more interested in building Chrome apps and extensions (Chrome is my primary browser, and unlike most people, I sincerely enjoy using a Cr-48), so I decided to port TargetAlert to Chrome. This turned out to be a fun project, especially because it forced me to touch a number of features of the Chrome API, so I ended up reading almost all of the documentation to get a complete view of what the API has to offer (hooray learning!).

Compared to Firefox, the API for Chrome extension development seems much better designed and documented. Though to be fair, I don't believe that Chrome's API would be this good if it weren't able to leverage so many of the lessons learned from years of Firefox extension development. For example, Greasemonkey saw considerable success as a Firefox extension, which made it obvious that Chrome should make content scripts an explicit part of its API. (It doesn't hurt that the creator of Greasemonkey, Aaron Boodman, works on Chrome.) Also, where Firefox uses a wacky, custom manifest file format for one metadata file and an ugly ass RDF format for another metadata file, Chrome uses a single JSON file, which is a format that all web developers understand. (Though admittedly, having recently spent a bit of time with manifest.json files for Chrome, I feel that the need for my suggested improvements to JSON is even more compelling.)

As TargetAlert was not the first Chrome extension I had developed, I already had some idea of how I would structure my new extension. I knew that I wanted to use both Closure and plovr for development, which meant that there would be a quick build step so that I could benefit from the static checking of the Closure Compiler. Although changes to Chrome extensions do not require a restart to pick up any changes, they do often require navigating to chrome://extensions and clicking the Reload button for your extension. I decided that I wanted to eliminate that step, so I created a template for a Chrome extension that uses plovr in order to reduce the length of the edit-reload-test cycle. This enabled me to make fast progress and finally made extension development fun again! (The README file for the template project has the details on how to use it to get up and running quickly.)

I used the original code for TargetAlert as a guide (it had some workarounds for web page quirks that I wanted to make sure made it to the new version), and within a day, I had a new version of TargetAlert for Chrome! It had the majority of the features of the original TargetAlert (as well as some bug fixes), and I felt like I could finally check "resurrect TargetAlert" off of my list.

Except I couldn't.

A week after the release of my Chrome extension, I only had eight users according to my Chrome developer dashboard. Back in the day, TargetAlert had tens of thousands of users! This made me sad, so I decided that it was finally time to make the Chrome version of TargetAlert better than the original Firefox verison: I was finally going to support user-defined alerts! Once I actually sat myself down to do the work, it was not very difficult at all. Because Chrome extensions have explicit support for an options page in HTML/JS/CSS that has access to localStorage, building a UI that could read and write preferences was a problem that I have solved many times before. Further, being able to inspect and edit the localStorage object from the JavaScript console in Chrome was much more pleasant than mucking with user preferences in about:config in Firefox ever was.

So after years of feature requests, Target Alert 0.6 for Chrome is my gift to you. Please install it and try it out! With the exception of the lack of translations in the Chrome version of TargetAlert (the Firefox version had a dozen), the Chrome version is a significant improvement over the Firefox one: it's faster, supports user-defined alerts, and with the seven years of Web development experience that I've gained since the original, I fixed a number of bugs, too.

Want to learn more about web development and Closure? Pick up a copy of my new book, Closure: The Definitive Guide (O'Reilly), and learn how to build sophisticated web applications like Gmail and Google Maps!