Some Screen-Scraped Calendars for Google Calendar

I noticed that there are some concert calendars for concerts in the Bay Area that do not have calendar feeds that you can overlay in Google Calendar. So sad. I decided to do some screen-scraping to create calendars for them.

I wanted to write something that would automatically screen-scrape the concert web sites once an hour. Unfortunately, since I don't know how to run Firefox headless (i.e., a Chickenfoot with its head cut off), I had to use something other than Chickenscratch, which is unfortunate because it turned out that doing it in Python was more tedious (JavaScript has better utilities for working with the DOM, and it's nice to be able to use literal regexps instead of having to "compile" them.)

I struggled for awhile to get mod_python working so I could serve a PSP or something that would generate the .ics on the fly, but that failed, so I decided to schedule a Python script with cron that would generate the .ics once an hour. Since I was no longer in Chickenfoot, I needed something besides Firefox to be a fault-tolerant HTML parser, and fortunately I found xml.dom.ext.reader which worked just fine.

I hit a pretty serious snag, though, when I discovered that my server has Python 2.2 instead of 2.4. I didn't think that it would be a big deal that I didn't have the latest version of Python; however, it turns out that date handling in 2.2 is abysmal. Instead of trying to figure out how to upgrade Python (since trying to install mod_python was so much fun), I found an open-source project that implemented the Python datetime library, so I had to download that and extract the appropriate file. Despite what I read about the backport, the implementation of pythonweb's datetime.py was far from complete (it implements the complete API, but its implementation raises many "not supported" exceptions). Nevertheless, I got things working (there are still some TODOs in the code where things could be improved), so I hope you check out the code and overlay my calendars on your Google Calendar.