Election 2008 and the internet: lessons learned

Two days after the election, and enough of the aftermath and peak traffic of the day has settled so that we can learn a few lessons from what happened.

0. Party in the streets!

Shawn Smith captured this moment in Ann Arbor.

1.  Design for infinite traffic.

Election reporting is perhaps the best reason to look at cloud computing as a system for dealing with peak load needs.  Demand for election information is very intense for a few hours, and then goes away.  Any system that you build that involves a stack of your own equipment sitting in some machine room is going to be too big for most of the time, and then too small when you need it most.

As an example, the fancy maps from Twitter Vote Report were down mid-day, but came back up when they reconfigured the site. 

RT networkredux "@votereport increase to 8 3.0Ghz cores on the primary web VM seemed to steady things a bit."

As another example, Nate Silver's FiveThirtyEight.com is hosted on Google's Blogger platform.  How many cores did that site run on?  As many as Google had available for the task.  And when push came to shove, Blogger wasn't up to the task at the heat of the moment either.

We're here and trying to publish, just can't.  They can't handle the traffic.  Sorry everybody.

Election night is an enormously busy night; if there was ever a time to design for one day of traffic, this is it.  I'm not sure how you can budget for infinite demand.

2.  In 2008, every page is about the elections.

Whether it's internet advertising, campaign coverage, partisan bickering, technology development, or just reporting on what's going on around town, it seems like every page on the whole Internet was about the elections.  It reminded me eerily of 1999, when every page was about the Year 2000, and a legion of programmers was employed fixing things; will the end of the 2008 campaign ad spend (projected at $17 million online) lead to the end of the web 2.0 era?

3.  Sending "I voted" to your network gets your network to vote.

I don't know this for sure; it's hard to tell from anecdotal reports, but my sense is that people who are using systems like Facebook and Twitter which make it easy to report that they voted are more likely to get their friends to vote. 

Facebook in particular made this easy with the Causes application that let you "donate your status message" in advance to pre-scheduled reminders to your friends to vote.  This get out the vote effort was given front page status with a rolling ticker on election day.  Causes recorded 1.7 million voters, who voted 70% to 21% for Obama, and now they know at least as much as any exit poll ever did.

Seth Godin writes more on this topic.

4.  There are problems with voting systems everywhere.

The sad conclusion of watching Twitter and #votereport for a few hours makes it clear that as voting turnout increases, the chances for things to go wrong go up.  Whether it's inadequate support at polling places leading to long lines, bad line control putting people in the wrong line to vote, or problems with ballots or machines, it's easy to find people who ran into snags and snafus.

Andrew Turner posts a recap of the amazing vote reports that the #votereport project gathered – both good stories and bad about how things worked.  There's a lot more to learn from that project and they have an awesome dataset to look at for review of where things went wrong (and where they went right).

5.  Practice makes perfect.

I voted using the Accumark system.  It was mostly painless, helped in large part by having done it enough times that I knew what was likely to fail.  The poll worker there knew that I needed to tear off the strip on top of the ballot before feeding it into the marking machine – the simple solution to most of the reported problems with the device – in part because this was my third time through this same precinct with the same voting machine and we figured it out before.

6.  (something about polls)

Nate Silver at FiveThirtyEight.com is still going over the differences between poll data and actual vote counts, but it's clear: there's a widespread, systematic, and unpredictable difference between what people say they are going to do to pollsters and what they actually do in the voting booth.   FiveThirtyEight did an amazing job at correcting for systematic poll bias, but even with all that there were still a few surprises (Indiana?!?) where you just couldn't know.

A paper by Andrew Gelman and Nate Silver circulated in the afternoon of election day, on statistics and conditional probabilities, predicting Obama 99% if he wins Virginia based on scenario analysis.

7.  Prepare ahead of time

My Michigan Election Results page this year was easy to write because I had written the same page in previous elections, knew where the results would be, and knew crucially which search terms people would be using.  I made it relevant before results came in by posting where the election parties would be.  You know these things are going to happen because they happened before; there's no reason to need to scramble at the last minute to put something together.


Lessons learned from previous events, for better or worse:

Hurricane season – Gustav lessons learned
Disaster relief and reporting – the process of the "after action review" as derived from US Army practice
Aviation safety compared to computer security

1 thought on “Election 2008 and the internet: lessons learned

Leave a comment