Documenting the Now

About

DocNow responds to the public's use of social media for chronicling historically significant events as well as demand from scholars, students, and archivists, among others, seeking a user-friendly means of collecting and preserving this type of digital content. DocNow has a strong commitment to prioritizing ethical practices when working with social media content, especially in terms of collection and long-term preservation. This commitment extends to Twitter's notion of honoring user intent and the rights of content creators. The project is a collaborative effort among the University of Maryland, University of California at Riverside, and Washington University in St. Louis. We are extremely grateful for funding from the Andrew W. Mellon Foundation.

Tools

We're building a variety of tools to help researchers work with Twitter data. Some of these require tech know-how to get running, reach out to us if you are interested in using them but not sure where to start!

DocNow

Appraise Twitter for building archival and research collections.

Tweet Catalog

A catalog of publicly shared Tweet ID sets. Add yours here!

Hydrator

"Rehydrate" your tweet ID sets into full tweets with metadata.

Twarc

Archive Twitter JSON using this command line tool.

Diff Engine

Track changes in news articles through their RSS feeds.

These tools are built by our friends in the field, check 'em out!

Webrecorder

Social Feed Manager

Open Archive

Mukurtu

Get Involved

Read and respond to articles on Medium.

Join the social media archiving conversation on Slack.

Participate in our opensource project on GitHub.

The Team

Bergis Jules
Community Lead

Ed Summers
Technical Lead

Alexandra Dolan-Mescal
UX Designer

Francis Kayiwa
DevOps Engineer

Former Team Members

Vernon Mitchell, WUSTL - Administrative Lead 2017

Chris Freeland, WUSTL - Administrative Lead 2016

Timothy Cole, WUSTL - Project Manager 2017

Desiree Smith-Jones - Project Manager 2016

Rafaele Viglianti, MITH - Developer Summer 2017

Dan Chudnov - Data Engineer 2016-2017

Meetings

Building a community of practice around social media and Web archiving is one of the most significant aspects to the work we are doing in the Documenting the Now project. Mellon has generously funded us to hold two face-to-face meetings, one in 2016 and another in 2017. These meetings allow us to bring our advisory board together and incorporate live streaming to engage with a larger audience.

March 22-24, 2018 Ethics in Archiving the Web Symposium (New York, NY)
December 11-12, 2017 Digital Blackness in the Archive: A Documenting the Now Symposium (Ferguson/St. Louis, MO)
February 15, 2017 #DocNowCommunity and #SNAPRT Twitter Chat: Ethics and Archiving Social Media: Perspectives from Students and New Archives Professionals
August 21-23, 2016 Advisory Board Meeting at Washington University in St. Louis, St. Louis, Missouri
August 11, 2016 #DocNowCommunity Twitter Chat 2: Social Media Research
July 14, 2016 #DocNowCommunity Twitter Chat 1: Project Introduction

Advisory Board

Advisory board members are active participants in the development of DocNow. Board members were carefully chosen because of their expertise as archivists, librarians, computer scientists, historians, software developers, librarians, sociologists, and journalists, and in many cases their work has directly influenced how the DocNow project was designed. We’re extremely thankful to this group and their continued support and active involvement in the project.

Natalie Baur
Preservation Librarian, El Colegio de México
Meredith Clark
Assistant Professor of Journalism, University of North Texas
Tressie McMillan Cottom
Assistant Professor of Sociology, Virginia Commonwealth University
Brian Dietz
Digital Program Librarian, North Carolina Statue University
Jarrett Drake
Digital Archivist, Princeton University
Meredith Evans
Director, Jimmy Carter Presidential Library and Museum
Jonathan Fenderson
Assistant Professor of African & American Studies, Washington University St. Louis
Deen Freelon
Assistant Professor of Communication, American University
Jessica Johnson
Assistant Professor of History, Michigan State University
Robin Katz
Public Services Librarian, University of California at Riverside

David Kim
Project Manager, Colored Conventions, University of Delaware
Mark Anthony Neal
Professor of African American Studies, Duke University
Michael Nelson
Professor of Computer Science, Old Dominion University
Yvonne Ng
Senior Archivist, WITNESS
Matt Phillips
Lead Developer, Library Innovation Lab at Harvard University
Rashawn Ray
Assistant Professor of Sociology, University of Maryland
Nicholas Taylor
Web Archiving Service Manager, Stanford University
Dexter Thomas
Writer, Los Angeles Times
Stacie Williams
Learning Lab Manager, University of Kentucky
Micah Zeller
Copyright and Digital Access Librarian, Washington University St Louis

News

Below is a curated list of news about the Documenting the Now project. If you notice any that are missing please get in touch. Also be sure to check out the stories on our Medium blog and our latest status updates on Twitter.

April, 2018 Ethical Considerations for Archiving Social Media Content Generated by Contemporary Social Movements white paper published.

Aug 29, 2017 Meet the digital librarians saving social media posts to protect human rights. Antony Funnell. Australian Broadcasting Corporation.

Aug 27, 2017 Human rights and the archiving opportunities and challenges of the digital world. Future Tense.

Aug 2, 2017 In 100 years, will today's digital files be accessible? Planning for digital obsolescence. St. Louis Public Radio.

Jan 5, 2017 Digital Spotlight: DocNow, Chronicling Historically Significant Events, American Theological Library Association.
Nov 2016 Doing Right Online: Archivists Shape an Ethics for the Digital Age, American Historical Association.
July 25, 2016 Universities finding new ways to preserve history in the making – like Ferguson, St. Louis Dispatch.
Jun 21, 2016 How an Archive of the Internet Could Change History, The New York Times.
May 02, 2016 Documenting the Now Builds Social Media Archive, Library Journal.
Mar 18, 2016 Archive the Future part II: Documenting the Now, Lost in the Stacks Podcast.
Feb 16, 2016 Introducing Documenting the Now, Maryland Institute for Technology in the Humanities.
Jan 26, 2016 UCR receives $517,000 to collaborate on social media archiving project, The Highlander.
Jan 20, 2016 Documenting the Now: Archiving Social Media for Generations to Come, WUSTL University Libraries.
May 15, 2015 The Politics Of Archiving #BlackLivesMatter, Baltimore Public Radio.