Bolinfest Changeblog

Saturday, January 2, 2010

google.com uses AJAX to update search results in place?

My Gmail account appears to be in an experiment where searches from google.com update in place. That is, a search for bolinfest takes me to:

http://www.google.com/#hl=en&source=hp&q=bolinfest&aq=f&aqi=g10&oq=&fp=cbc2f75bf9d43a8f

Rather than the classic:

http://www.google.com/search?hl=en&source=hp&q=bolinfest&aq=f&oq=&aqi=g-p1g2

Apparently this was first reported in January 2009. There was immediately an outcry because the fragment (with includes the keywords used in the Google search) was not included in the referrer, so sites who analyze their traffic using that information were at a loss.

In my version of the experiment, this is no longer an issue because the search results use a URL redirector (which I do not believe is a new practice). Although the first search result is for www.bolinfest.com/, and when I hover over the link in Firefox, the status bar says http://www.bolinfest.com/, the actual destination of the link is:

http://www.google.com/url?sa=t&source=web&ct=res&cd=1&ved=0CAoQFjAA&url=http%3A%2F%2Fwww.bolinfest.com%2F&rct=j&q=bolinfest&ei=kYU_S8nZAdC0lAeJ0fmVBw&usg=AFQjCNEgHH2q8kppSggtQt45aSSazDTynA&sig2=G54nS0q_UtfEChH7icC_NA

When I follow the link and run javascript:alert(document.referrer) in the address bar, I see the same thing:

http://www.google.com/url?sa=t&source=web&ct=res&cd=1&ved=0CAoQFjAA&url=http%3A%2F%2Fwww.bolinfest.com%2F&rct=j&q=bolinfest&ei=kYU_S8nZAdC0lAeJ0fmVBw&usg=AFQjCNEgHH2q8kppSggtQt45aSSazDTynA&sig2=G54nS0q_UtfEChH7icC_NA

Note that this URL contains the query in the URL parameter q=bolinfest, which should appease those who are concerned with keyword tracking.

What is interesting is that the AJAX request to load the search results appears to be an ordinary GET request to:

http://www.google.com/search?hl=en&q=bolinfest&aq=&aqi=g2&oq=bolinfest&fp=cbc2f75bf9d43a8f

The content of that URL is an HTML page with 21 <script> tags in the <head> and an empty <body> tag.

Also, copying and pasting http://www.google.com/#hl=en&source=hp&q=bolinfest&aq=f&aqi=g10&oq=&fp=cbc2f75bf9d43a8f into the address bar of a new tab works as expected (though I do get a JavaScript error when I paste the URL into a new tab that I do not get when doing the initial search from google.com).

It's hard to say what this really means, if anything. Presumably, Google's objective is to make search results load faster without pissing off advertisers. As I'm not in the SEO racket, I'm not terribly interested in the latter, but I am curious whether Google is on to a new technique for loading web content faster!

Monday, December 21, 2009

Closure Lite - Just in time for Christmas!

When I was an undergrad at MIT, the end of the fall semester meant I had about 5 weeks of freedom to hack on stuff before the start of the spring semester. At MIT, instead of starting classes back up in January, they have what is called the Independent Activities Period (IAP). Although IAP is optional, many students return for it (except the Hawaiians I knew, who generally decided to stay home in their tropical paradise during one of the coldest months in Boston) because there are so many great opportunities: you can get scuba certified, do research, or my personal favorite -- engage in programming competitions!

For you holiday hackers who haven't found the time to play around with Closure yet, I've tried to make it a little easier to get started by creating Closure Lite. Closure Lite is a single JavaScript file that you can include on a web page to start using a subset of the Closure Library. This is similar to the approach used by other popular JavaScript libraries such as jQuery. But as the Closure Lite documentation explains, although Closure Lite is a good way to start learning the Library, it is recommended to learn and use the Closure Compiler on your production JavaScript.

I hope that both Closure and Closure Lite are useful to MIT students who are competing in 6.470 this IAP!

Wednesday, December 9, 2009

Check out Speed Tracer (it's not just for GWT applications!)

Last night, Google hosted a Campfire One event introducing Google Web Toolkit (GWT) 2.0. The Campfire event is divided into six segments that are all available on YouTube.

If you do not feel like watching the entire presentation (it's a little over an hour), I recommend watching Parts 3 and 5. In Part 3, Bruce Johnson discusses GWT's JavaScript compiler (which is different from the Closure Compiler) and GWT 2.0's new code splitting feature. From the presentation, it sounds like the GWT compiler has added some of the optimizations that the Closure Compiler has had for some time. (I've heard of instances of the Closure Compiler reducing compiled code from GWT by 20%, so clearly there was room for improvement.) Historically, the GWT and Closure compilers were separate codebases because one was open-sourced and the other was not, but since that is no longer the case, perhaps we will see some convergence in the future. I wouldn't hold my breath, though.

But what was more impressive was the ease with which GWT was able to introduce code-splitting. That is, dividing up a large JavaScript file into smaller files, the majority of which get loaded asynchronously by the application as the features that depend on them are accessed. The Closure Compiler has support for such a feature, but it is undocumented and requires a bit of work by the developer, even if he knows what he is doing. The code-splitting feature in GWT 2.0 (introduced about 10 minutes into Part 3) is much more elegant and straightforward. I hope that the Closure Tools suite evolves to make this just as simple.

Then in Part 5, Kelly Norton introduces Speed Tracer, which is a Chrome extension that gives an unprecedented amount of insight into what Chrome is doing when it runs a web application. It is more similar to dynaTrace than it is to Firebug. Page Tracer is informative and so snappy that you might not believe the UI is written in HTML5 -- try it out!

However, my one gripe with the Campfire presentation is that you might come away from it believing that Speed Tracer works only with GWT applications, but that is not the case at all! Although its documentation lives under GWT on code.google.com and Speed Tracer was written using GWT, it can be downloaded and used completely independently from GWT. This morning, I installed it to explore the performance of some webapps I used to work on (which were written using Closure), such as Google Tasks. (I found some areas for improvement which I forwarded to the team.) I strongly recommend evaluating your own web applications using Speed Tracer as you may be surprised at what you discover.

Also, if you're like me, you may not notice the links to additional Speed Tracer documentation because they appear below the fold on the landing page. Under the Tools heading in the left-hand-nav, there are links to Hints, the Data Dump Format, and the Logging API.

If you stop and think about it, this level of tool support is essential for the Chrome OS initiative to succeed. If the browser is going to substitute for the desktop as a platform, then it must be fast (which is where Chrome comes in), it needs to have a kickass API (which is where HTML5 comes in), and it needs to have best-of-breed developer tools (which is where GWT, Closure, and Speed Tracer come in) so it is possible to build web applications that can compete with (and ideally exceed) desktop applications. When Chrome OS was originally announced, I was a naysayer, but now that more of the pieces are starting to come together, I'm getting a bit more optimistic.

Monday, November 23, 2009

Practical Chickenfoot Presentation

Today I gave a guest lecture on Chickenfoot to Monzy's course on mashups at NYU's Tisch School of the Arts. The presentation is available online as a Google Doc. Unlike previous talks on Chickenfoot, this one focused on practical examples of how to use it rather than the research behind it. This was a nice change because I actually got to sit down with a classroom full of first-time users of Chickenfoot as they went through the exercises. Thanks to Monzy and his students at ITP for having me!

Wednesday, November 18, 2009

Example of using Closure Compiler to attain an 85% improvement over YUI Compressor when minifying Closure Library code

For those of you who do not care about caveats or grains of salt: here is my test showing Closure Compiler beating YUI Compressor by 85% when compiling a simple JavaScript file that uses a widget from the Closure Library.

I chose the title of this blog post very carefully: I do not claim that the Closure Compiler will always minify 85% better than YUI Compressor, and I wanted to make it clear that the dramatic results are specific to Closure Library code which is written in a style that was designed to be minified by the Closure Compiler. Although both tools can be used independently, they were meant to be used together.

That is not to say that other libraries cannot get dramatic minification benefits from the Compiler -- they can! But doing so requires following the style guidelines explained in the Advanced Compilation and Externs article on the Google Code web site.

And even if you think my test is unfair, or that I rigged it to butcher the YUI Compressor, I hope you can at least benefit from these two code samples that came out of creating these tests:

Monday, November 16, 2009

Updates to Externs File for Maps V2 API

Kai found some missing items in my Maps V2 externs file. I was missing methods from the following classes: GControl, GMapType, GMarker, GProjection, and GTileLayer. I also apparently missed GPolyline.fromEncoded() and GPolygon.fromEncoded() because they were listed under Factory Methods, which my script did not know to look for.

Apparently Kai has also found an inconsistency between the online documentation for GPolyline.fromEncoded() and its implementation, but I want to get someone from the Maps team to confirm before updating that.

Wednesday, November 11, 2009

Externs file for the Google Maps API V2

If you're using the Closure Compiler with ADVANCED_OPTIMIZATIONS, then you need to make sure that you are declaring your externs appropriately. The Compiler includes many common externs by default, such as document and window, but externs files for Google JavaScript libraries do not appear to be available yet.

Using Chickenfoot, I generated an externs file for the Google Maps API V2. I strongly encourage you to use it when compiling JavaScript that uses Google Maps! If you find that the externs file has some mistakes, please fix the Chickenfoot script rather than the externs file itself. If you look carefully at the script, I had to add a number of hacks to work around bugs in Google's online HTML documentation, so there could be others that I have missed!