It Takes a Village to Archive the Federal Web

This talk will describe ongoing collaborative efforts to create and sustain an End of Term Web Archive of born digital, federal government resources. The Archive was identified and captured during recent Presidential Elections, including the months that preceded and followed each election in 2008 and 2012. Included in the talk will be detailed insights into the current size and character of the US government’s web presence from Twitter, YouTube, Flickr and Facebook to the many web sites found on .gov, .mil. & .org, across all branches of government - Executive, Legislative & Judicial. Additionally, you’ll hear from project participants about the tools, techniques, and resources required to curate, populate, mine, analyze, replay, and search the archive using available open source software packages, about weaknesses and gaps in the data, and the challenges associated with and limitations of voluntary archival collaborations.

Additional Supporting Materials

http://eotarchive.cdlib.org/

Questions Answered

Why was the End of Term Web Archive created? How was it created? Which organizations were involved in its creation? What is in it? How does the archive compare to what’s on the live web?
What does it take to collect and preserve the web presence of the US Federal Government? How do you ensure all the online resources of interest are identified and captured? How do you confirm that you archived and can replay social media, videos, blogs, and tweets as well as official web sites?
What can you learn as you mine and analyze government resources published exclusively online? What does it take to assemble and prepare the raw web captures for analyses? What open source tools are being used to facilitate data mining, and access?
What EOT services are publicly available? How can I get access to those services and/or data sets?
What makes voluntary collaborations across institutional domains work? When do they fall apart? What can be gained from collaborative data curation, technology and service development, and web data mining of the US national government presence on the web over time?