The Flickr engineering team is obsessed with making pages load as quickly as possible. To that end, we're refactoring large amounts of our code to do only the essential work up front, and rely on our queueing system to do the rest.
Questions Answered:
What kinds of tasks does our system do?
What happens when tasks encounter errors?
How do we structure our tasks?
What kinds of priorities do we deal with?
How fast should offline tasks run?
What's a message queueing system?
Why should my website use a queueing system, even though it doesn't get as much traffic as Flickr?
Why did we write our own queue servers?
When I upload a photo to Flickr, what, exactly, happens?
How have we been so stupid, and what have we done to be less stupid over time?