Tuesday, January 18, 2011

What are the most important classes for high school students to succeed in software engineering?

What are the most important classes for high school students to succeed in software engineering? That is the question that I try to answer in an essay of same name.

Also, this is the first essay I have written using NJEdit, which is the editing software that I built (and have since open sourced) in order to write Closure: The Definitive Guide. It helps me focus more on content while worrying less about formatting, though it still has a ways to go before becoming my "one click" publishing solution.

A unique feature of NJEdit is that when I produce the HTML to publish my essay, I also produce the DocBook XML version as a by-product! It's not a big selling point today, but if I ever want to publish anything to print again, I'll be ready! For open-source projects that are slowly creating HTML documentation that they hope to publish as a print book one day, NJEdit might be the solution.

And if it is, maybe someone will help me fix its bugs...

Want to learn more about Closure? Pick up a copy of my new book, Closure: The Definitive Guide (O'Reilly), and learn how to build sophisticated web applications like Gmail and Google Maps!

Wednesday, January 12, 2011

My latest Closure talks

Last Thursday, January 6, I had a doubleheader, as I gave one talk in the afternoon at an undergraduate MIT Web programming competition (6.470), and another talk in the evening at the Boston JavaScript Meetup. It was a long day (I took the train up from NYC that morning), but I got to talk to so many interesting people that I was far more energized at the end of the day than exhausted!

I cleaned up the slide decks from both talks so they are suitable for sharing. Because I was informed that the MIT programming competition had a lot of freshmen in it, I didn't go into as much detail about Closure as I normally would have because I didn't want to overwhelm them. (I ended up taking so much out of my "standard" talk that I ended up finishing ~30 minutes early, though that was likely a welcome relief to the students as I was the last speaker in a full day of lecture.) As you can see, the MIT talk is only 17 slides:



The talk I prepared for the Boston JS Meetup was the most technical one I have given to date. It was my first time presenting for a group that actually has "JavaScript" in the name, so it was refreshing not to have to spend the first 15 minutes explaining to the audience that JavaScript is a real language and that you can do serious things with it. By comparison, my second talk went much longer (about an hour and a half?), as there was a lot more material to cover as well as tons of great questions from the (very astute!) audience during my presentation:



The one thing that these presentations do not capture is the plovr demo that I have been giving during my talks. (This was also the first time that I demoed using plovr with jQuery, as I had just attempted using jQuery with plovr for the first time myself the night before. I have an open topic on the plovr discussion group about how to make plovr easier to use for jQuery developers, so please contribute if you have thoughts!) At some point, I'm planning to do a webcast with O'Reilly on Closure, so that might be a good opportunity to record a plovr demo that can be shared with everyone.

Want to learn more about Closure? Pick up a copy of my new book, Closure: The Definitive Guide (O'Reilly), and learn how to build sophisticated web applications like Gmail and Google Maps!

Tuesday, January 11, 2011

Web Content Wizard is Fixed AGAIN!

Back in 2006, I released this little tool for Google Calendar called the Web Content Wizard. With it, you can add your own "web content events" in Google Calendar. Unfortunately, there is no way to add such events so via the Google Calendar UI, so normally your only alternative is to do some gnarly web programming to talk to GData.

Things were going well until one day, the Web Content wizard just broke. The GData team changed something on their end which caused my poor little app to stop working. If I remember correctly, it was because they created this new thing about registering your web application. I learned of the problem because some users wrote in, and eventually sometime I finally found the time to debug the problem and fix it.

Then things were going well again for awhile, but then in late 2009 (when I had some newfound free time), I decided that it was finally time to migrate bolinfest.com from the Red Hat 9 server it had been running on since 2003! I had a VPS from RimuHosting (who rocks btw!) that I acquired in 2006 for my mitcalendars.com project with Mike Lambert, but I had never had the time to move my content from bolinfest.com over there. (Getting PHP 5 with its native JSON parsing methods was what finally put my over the edge, as I repeatedly failed at compiling them myself for the version of PHP 4 that was running on Shrike.)

As you can imagine, not everything worked when I did the migration in 2009. Honestly, I probably should have just gotten an even newer VPS then and migrated both servers, as now bolinfest.com is running on a dusty version of Ubuntu 8.04! (I'm running version 0.9.5 of Mercurial, which means that I don't even have the rebase extension!) Anyway, when I did the migration, I was far more preoccupied with migrating my conf files for Apache from 1.3 to 2.0 than I was with getting the Web Content Wizard to work again. Every endeavor of mine involving GData always involves a minimum of 3 hours work and a lot of cursing, and I just wasn't up for it.

Then today I got another polite email asking me why the Web Content Wizard wasn't working. I have amassed quite a few of these over the years, and I generally star them in my inbox, yet never make the time to investigate what was wrong. But for some reason, tonight I felt motivated, so I dug in and decided to debug it. I [wrongly] assumed that GData had changed on me again, so I focused my efforts there first. Surprisingly, the bulk of my time was spent getting the right error message out of PHP, as all it told me was that my request to the GData server was returning a 400. Now, I know GData is pretty lousy at a lot of things, but when it gives you a 400, in my experience, at least it gives you a reason!

I use cURL under the hood to communicate with GData from PHP, and that has never been pretty. I thought I remembered editing something in /etc/php.ini (which was now at /etc/php5/apache2/php.ini) to get cURL working correctly on the old bolinfest.com, but I could not remember what it was, nor did I appear to have documented it.

The key insight occurred when I changed this call to curl_setopt() to set the CURLOPT_FAILONERROR option to false:
curl_setopt($ch, CURLOPT_FAILONERROR, false);
Suddenly, my requests to GData contained a real error message:
The value following "version" in the XML declaration must be a quoted string
This did not make any sense because I was sure that I was sending well-formed XML to GData and it contained the <?xml version="1.0"?> preamble as it should. I decided to print it back out to make sure that things were being sent as I expected, but when I did so, I noticed that all of my double-quotes were inexplicably escaped with backslashes. That explained why the error message was complaining about well-formed XML, but not where the backslashes came from.

Then it dawned on me: php.ini has a bunch of stupid settings with the word magic in them. Sure enough, I did a search for [php HTTP_POST_VARS escape quotes], which eventually led me to the documentation for the PHP function stripslashes(), which finally led me to the documentation for magic_quotes_gpc, which basically says: "Because the average PHP developer is so likely to Little Bobby Tables himself, we put the coding behaviors for adults on a high shelf where little PHP developers can't reach them and automatically backslash escape quotes because said developers are so likely to cram that string directly into a SQL statement without escaping it."

I immediately changed the line in php.ini to magic_quotes_gpc = Off, restarted Apache, and all of a sudden, the Web Content Wizard was fixed AGAIN!

At least until my next server upgrade...

Want to learn more about Closure? Pick up a copy of my new book, Closure: The Definitive Guide (O'Reilly), and learn how to build sophisticated web applications like Gmail and Google Maps!

Wednesday, January 5, 2011

Released a new version of plovr so I can show it off at my two Closure lectures this Thursday

I just released a new version of plovr in time for the two talks on Closure I am giving this Thursday. The first is a guest lecture for an MIT web programming competition: 6.470. My understanding is that the course attracts a lot of freshmen, so my hope is that plovr can help those who are new to JavaScript leverage the Closure Compiler to help them identify their JavaScript errors (of which there are likely to be many!). This motivated me to fix some annoying issues in plovr that have been lingering for awhile (which the early adopters have been nice enough to put up with for far too long :).

The second talk I am going to give is part of the Boston JS Meetup, which I am very excited to attend for the first time. I'm looking forward to talking to the group and showing off what both Closure and plovr can offer to those in industry.

Also, up until now, I have been making decisions autonomously about new plovr features and config options because I did not have a good way to gather feedback from users other than the bug reports I received. To that end, I finally decided to create a Google Group for plovr where users can help influence plovr development and ask questions about how to use plovr. As I noted in the Closure Library group, I would like to extend plovr to help with CSS management and developing against jQuery (among other things), and I believe these features will only be useful to others if prospective users participate in the discussion. We'll see how all of that works out.

Finally, one interesting engineering challenge that has plagued plovr since its inception (but has finally been slayed!) is the question of how to manage its dependency on the Closure Library, Closure Compiler, and Closure Templates. With a considerable amount of help from Ilia Mirkin (and a lot of SVN and Hg gymnastics), I finally have a system that I am happy with. Basically, it requires creating a branch for each dependent project to which updates are periodically added and then merged into the main repository. As this is a general problem for open-source projects, I documented it thoroughly on the plovr wiki.

Although it was tricky to set up, it is now possible to build plovr without checking out the other Closure projects. (Previously, a new version of plovr would include my own local checkouts of the closure-library, closure-compiler, and closure-templates repositories, which were at somewhat arbitrary revisions with possibly even more arbitrary/undocumented patches of my own in them.) Further, because plovr has its own clones of the Closure projects in its repository, it is far easier to modify Closure code if it needs to be edited for plovr. For example, it was now trivial to make the constructor of com.google.javascript.jscomp.Result public, whereas previously I would have had to create a completely separate binary for the Closure Compiler with this one modification and then replace it with the one in plovr's lib directory. Further, I would forever have to keep my checkout of the Closure Compiler with the modification around so that I could make further modifications in the future while no outside developer would have any insight into that process -- no more!

This also allowed me to clean up the build process for plovr and remove some unnecessary and/or duplicate dependencies. The release of plovr I did back in December was 11.7 MB, but the new release is down to 8.4 MB, which is the smallest plovr jar I have ever published! Things are definitely moving in the right direction for the project.

Want to learn more about Closure? Pick up a copy of my new book, Closure: The Definitive Guide (O'Reilly), and learn how to build sophisticated web applications like Gmail and Google Maps!

Monday, December 27, 2010

I got a new computer for Christmas! Sorta...

About a year ago, I bought myself a Lenovo ThinkPad (T500). Because I had been working at Google from 2005-2009 and my work laptop was sufficient for surfing the Web while at home, my last personal laptop purchase was in 2004 when I bought an IBM ThinkPad (T42). I loved my ThinkPad, as I spent many hours of quality time with it implementing my Master's thesis at MIT.

Therefore, when it came time to buy a new machine in 2009, I decided to look at ThinkPads. Yes, I had heard that the quality of the ThinkPad had dropped considerably ever since Lenovo had taken things over from IBM. Yes, all my friends were developing on Macs, but I found them to be irritating to type on and my primary development applications (Google Chrome and Eclipse) felt slow on OS X. And yes, Windows has more than its share of annoyances. At the time, ThinkPads were shipping with Vista (arguably one of the worst Windows releases ever), but they came with a voucher for a free copy of Windows 7, which was reported to be a substantial improvement.

After weighing all of my options, I decided to save some cash and buck the trend of my contemporaries: I chose to be a PC instead of a Mac.

From the abysmal experience of making my online purchase of a Lenovo ThinkPad, I should have known that my experience with the machine itself would also be a disaster. The Lenovo web site was slow and failed to communicate what I was buying. I inadvertently bought a machine with a 32-bit processor (I still don't know why any new machine in the fall of 2009 would not be 64-bit), which could not take advantage of the 4GB of RAM that I paid for. (Searching online forums, I do not appear to be the only one who has made this mistake.) Further, it took two to three weeks to get my laptop. If I had decided to buy a Mac, I could have just walked to the Apple store and had one in my hot little hands that day.

Then there was Vista. Vista makes me angry. Sometimes when my laptop goes to sleep, it also turns off the wireless radio and fails to turn it back on when the machine wakes up. Sometimes the physical buttons to control the volume of the laptop fail to communicate with Vista. (Being unable to control the volume of your computer can be an embarrassing experience.) Sometimes the built-in video camera works, sometimes it doesn't, but either way, rebooting (the go-to solution for any Windows user) is an unreliable fix for the problem. The DVD drive is also finicky, as it inexplicably pops open from time to time. Overall, this is a horrible experience.

But hey, what about that Windows 7 voucher? Apparently you had to redeem it by calling Lenovo customer support after Windows 7 was officially released, which was several weeks after I got my ThinkPad. "Call Lenovo" sat near the bottom of my to-do list for months (who is eager to call customer support?), so by the time I made time to call Lenovo, they told me that my voucher had expired. I complained and they said they would speak to someone and contact me with a response, but they never did.

After all of this, I was incredibly unhappy with my ThinkPad. I wanted to try to put Linux on it, but I had a number of speaking engagements coming up where I was going to present from my laptop, so it did not seem prudent to wipe my machine before embarking on a series of talks. Though even after my speaking series was over, it was hard to find the time to sit down and install Linux. I had never done it by myself before, and I wanted to take the time to read up on it so I could understand what I was doing and have a backup plan in case I was unsuccessful and needed to revert back.

This year, on Christmas Day, I decided to just go for it and install Ubuntu on my ThinkPad. I copied the .iso onto a USB stick, wedged it in the side of my ThinkPad, and rebooted. I hit F1 to change the bootloader to run off the USB stick and made it to the Ubuntu installation screen. I spent a bit of time playing with options to repartition my hard disk so that I could preserve Vista and my files in case I needed to get them back (for me, this has always been the most confusing part when trying to step through a Linux installation), but I got fed up and decided that all of my important data was in the cloud (which includes Mercurial and SVN repos on other machines) and that I had no desire to go back to Vista anyway, so I decided to dedicate the entire hard disk to Ubuntu and abandon Vista forever.

When I decided to blow away my operating system, I knew that it would likely be a rash decision that I would pay for later, but so far, it has been FREAKING AWESOME! The physical volume buttons on the ThinkPad just work. The Google Talk plugin for Linux also just works. And instead of the dual, competing gaudy wireless managers in Vista (one provided by Windows and the other provided by Lenovo), there is now a single, simple icon/dropdown interface in the toolbar to select a wireless network, just like on OS X.

Using my laptop is now enjoyable again. It feels like I got a brand new machine, so even though I didn't manage to get the new computer that I really wanted for Christmas, this certainly seems like the next best thing.

Want to learn more about Closure? Pick up a copy of my new book, Closure: The Definitive Guide (O'Reilly), and learn how to build sophisticated web applications like Gmail and Google Maps!

Thursday, October 21, 2010

Slides from MIT Talk

Last week, I gave two talks about Closure in Cambridge: one at Google and the other at MIT CSAIL. I had a great time and fielded lots of good questions from sharp individuals, so I'm looking forward to my next two speaking events in New York City.

As someone requested that I upload my slides, I am more than happy to share them:



I would argue that one of the best parts of the talk is the plovr demo, which sadly is not communicated through the slides. Though if you are in the NYC area, you can come and see such a demo live if you come to one of my upcoming talks! Also, O'Reilly provides me with three copies of my book to give away at each event, so even if you aren't interested in plovr, you are probably interested in the chance to win a free book! (Though I am embarrassed to admit that I don't have the hang of how to run a "giveaway" yet, so at the MIT talk, I ended up giving books to those who hung out the longest to ask questions.)

Though after spending quite a bit of time preparing my talks last week, I am pretty disappointed with my experience creating a presentation with Google Docs. Given that the UI is displayed using HTML and CSS, it should be dead simple to change the color palette of a presentation. For example, I am generally interested in three styles: slide title, plain text, and sample code. Every time I want to set text as sample code, I have to select the text, choose "Courier New" from the font dropdown and then select my code color from the font color picker. Under the hood, there are tons of <div>s with the same inline CSS styles whereas if I created my slides by hand in HTML, I would have many <code> tags that could easily be restyled with a single CSS rule.

Trying to use different themes for Google and MIT for the same presentation would have taken hours because of all the time mousing around to select text and change all the styles, so I didn't even try. I know that Google Docs is nowhere near Keynote or Powerpoint in terms of its feature set, but I didn't realize how far behind it is. It only comes with 15 themes, and I'd argue that none of them are particularly good for a developer who wants to display code samples. I would switch to a Microsoft or an Apple solution, but I am often switching between computers, and the thought of emailing a slide deck around and clogging my inbox with twenty versions of the same presentation is depressing.

There's a good chance that I'll build my own lightweight presentation editor in HTML that is tailored for creating talks by software engineers. I basically had to do something similar in order to write my book, but that's an exciting topic for a future blog post!

Want to learn more about Closure? Pick up a copy of my new book, Closure: The Definitive Guide (O'Reilly), and learn how to build sophisticated web applications like Gmail and Google Maps!

Monday, October 11, 2010

Upcoming Talks

Over the next couple of months, I will be giving several talks about Closure:

October 13: Getting the most out of Closure
Google: Cambridge, MA

October 15: Building a 100K LOC JavaScript application with Google Closure
MIT CSAIL: Cambridge, MA

October 27: Introduction to Closure Tools and plovr
Google: New York, NY (as part of the NYC-GTUG Meetup)

November 10: Compiling your web site: Closure compiler
Location TBA: New York, NY (as part of the Web Performance NY Meetup)

If you are in the area for any of these talks, please come and say hello! I'm also more than happy to sign books (and may even be giving some away!) at each of these talks.

Hopefully, I will also get a chance to speak at both the Boston and NYC JavaScript meetups. I also hope to make it out to the Bay Area, Austin, Chicago, and Sydney (where I seem to have an open invitation!). As I have no official "book tour," my personal travel basically determines when and where I set up speaking events.

Want to learn more about Closure? Pick up a copy of my new book, Closure: The Definitive Guide (O'Reilly), and learn how to build sophisticated web applications like Gmail and Google Maps!