Web Content Wizard is Fixed AGAIN!

Back in 2006, I released this little tool for Google Calendar called the Web Content Wizard. With it, you can add your own "web content events" in Google Calendar. Unfortunately, there is no way to add such events so via the Google Calendar UI, so normally your only alternative is to do some gnarly web programming to talk to GData.

Things were going well until one day, the Web Content wizard just broke. The GData team changed something on their end which caused my poor little app to stop working. If I remember correctly, it was because they created this new thing about registering your web application. I learned of the problem because some users wrote in, and eventually sometime I finally found the time to debug the problem and fix it.

Then things were going well again for awhile, but then in late 2009 (when I had some newfound free time), I decided that it was finally time to migrate bolinfest.com from the Red Hat 9 server it had been running on since 2003! I had a VPS from RimuHosting (who rocks btw!) that I acquired in 2006 for my mitcalendars.com project with Mike Lambert, but I had never had the time to move my content from bolinfest.com over there. (Getting PHP 5 with its native JSON parsing methods was what finally put my over the edge, as I repeatedly failed at compiling them myself for the version of PHP 4 that was running on Shrike.)

As you can imagine, not everything worked when I did the migration in 2009. Honestly, I probably should have just gotten an even newer VPS then and migrated both servers, as now bolinfest.com is running on a dusty version of Ubuntu 8.04! (I'm running version 0.9.5 of Mercurial, which means that I don't even have the rebase extension!) Anyway, when I did the migration, I was far more preoccupied with migrating my conf files for Apache from 1.3 to 2.0 than I was with getting the Web Content Wizard to work again. Every endeavor of mine involving GData always involves a minimum of 3 hours work and a lot of cursing, and I just wasn't up for it.

Then today I got another polite email asking me why the Web Content Wizard wasn't working. I have amassed quite a few of these over the years, and I generally star them in my inbox, yet never make the time to investigate what was wrong. But for some reason, tonight I felt motivated, so I dug in and decided to debug it. I [wrongly] assumed that GData had changed on me again, so I focused my efforts there first. Surprisingly, the bulk of my time was spent getting the right error message out of PHP, as all it told me was that my request to the GData server was returning a 400. Now, I know GData is pretty lousy at a lot of things, but when it gives you a 400, in my experience, at least it gives you a reason!

I use cURL under the hood to communicate with GData from PHP, and that has never been pretty. I thought I remembered editing something in /etc/php.ini (which was now at /etc/php5/apache2/php.ini) to get cURL working correctly on the old bolinfest.com, but I could not remember what it was, nor did I appear to have documented it.

The key insight occurred when I changed this call to curl_setopt() to set the CURLOPT_FAILONERROR option to false:

curl_setopt($ch, CURLOPT_FAILONERROR, false);

Suddenly, my requests to GData contained a real error message:

The value following "version" in the XML declaration must be a quoted string

This did not make any sense because I was sure that I was sending well-formed XML to GData and it contained the <?xml version="1.0"?> preamble as it should. I decided to print it back out to make sure that things were being sent as I expected, but when I did so, I noticed that all of my double-quotes were inexplicably escaped with backslashes. That explained why the error message was complaining about well-formed XML, but not where the backslashes came from.

Then it dawned on me: php.ini has a bunch of stupid settings with the word magic in them. Sure enough, I did a search for [php HTTP_POST_VARS escape quotes], which eventually led me to the documentation for the PHP function stripslashes(), which finally led me to the documentation for magic_quotes_gpc, which basically says: "Because the average PHP developer is so likely to Little Bobby Tables himself, we put the coding behaviors for adults on a high shelf where little PHP developers can't reach them and automatically backslash escape quotes because said developers are so likely to cram that string directly into a SQL statement without escaping it."

I immediately changed the line in php.ini to magic_quotes_gpc = Off, restarted Apache, and all of a sudden, the Web Content Wizard was fixed AGAIN!

At least until my next server upgrade...

Want to learn more about Closure? Pick up a copy of my new book, Closure: The Definitive Guide (O'Reilly), and learn how to build sophisticated web applications like Gmail and Google Maps!