Here is a thorough description of the Distant Reader, the Distant Reader Toolbox, and Distant Reader study carrels, drawing from both the provided documentation and live data from the system.
The Distant Reader is a system created by Eric Lease Morgan at the Navari Family Center for Digital Scholarship, Hesburgh Libraries, University of Notre Dame. Its purpose is twofold: first, to address the problem of information overload, and second, to facilitate reading at scale—that is, making it possible to analyze and understand large corpora of narrative text, from a dozen journal articles up to hundreds of books. The Distant Reader takes collections of files as input and outputs structured data sets called study carrels, which are designed to be read by computers as well as people. The project's home page is at https://distantreader.org.
In practice, the Distant Reader enables researchers, students, and scholars to address a wide range of questions, from the mundane to the sublime:
As an illustration of the scale involved, the "government" carrel currently available in the local library contains 484 items totaling approximately 13,982,108 words, with an overall Flesch Readability score of 53—placing it in a moderately difficult reading range. This is just one of 53 carrels currently accessible from the local library, spanning subjects from Homer and Shakespeare to CRISPR and Ebola.
The Distant Reader Toolbox is the companion software to the Distant Reader. It is implemented as a set of Python modules and can be installed from the command line:
pip install reader-toolboxor from GitHub source:
git clone https://github.com/ericleasemorgan/reader-toolbox.git
cd reader-toolbox
pip install -e .Once installed, the rdr command provides a wide array of sub-commands for building, analyzing, and managing study carrels. Running rdr without arguments reveals a menu of commands including:
The Toolbox can also be imported into Python scripts, allowing users to combine its functionality with other Python modules for custom analysis. Full documentation is available at https://reader-toolbox.readthedocs.io/.
Study carrels are data sets—structured directories containing original documents, plain-text versions, and extracted features from a corpus. They are designed to be platform-independent and network-independent: no special software is required to read them, and once saved locally, no Internet connection is needed. This means they are designed to stand the test of time.
Each study carrel is a directory containing several subdirectories:
Additional subdirectories contain tab-delimited files of extracted features:
| Directory | Contents |
|---|---|
| adr | Email addresses |
| bib | Bibliographics (authors, titles, dates, etc.) |
| ent | Named entities (people, organizations, places, etc.) |
| pos | Parts-of-speech (nouns, verbs, adjectives, etc.) |
| urls | URLs and their domains |
| wrd | Statistically significant keywords |
| figures | Images |
| etc | Everything else, including stop word lists, bag-of-word representations, and a SQLite database |
Although the files in these subdirectories lack .tsv or .tab extensions, they are all tab-delimited and can be imported into any spreadsheet, database, or programming language.
At the root of each carrel, there may also be files readable by a wide variety of desktop applications:
Study carrels support analysis at multiple levels:
The documentation recommends several desktop applications for working with study carrels:
The "government" carrel serves as a useful illustration. As noted above, it contains 484 items totaling roughly 14 million words with a Flesch score of 53. The bibliography includes works by Emma Goldman (Anarchism and Other Essays), Petr Kropotkin (The Conquest of Bread), Voltairine de Cleyre (Selected Works), Alexander Berkman (Prison Memoirs of an Anarchist), and many others, along with PDF articles from the American Journal of Islamic Social Sciences and other scholarly publications. Each item has associated keywords, a summary, a Flesch score, and a word count. The carrel's 53 Flesch score suggests the corpus is written at a moderately advanced reading level, consistent with its mix of political philosophy, historical documents, and academic articles.
The local library currently contains 53 study carrels, including:
In essence, the Distant Reader and its Toolbox transform large collections of text into structured, analyzable data sets called study carrels. These carrels are platform-independent, network-independent, and designed for both human and computational reading. Through extracted features—bibliographics, parts-of-speech, named entities, keywords, and more—users can address research questions ranging from the simple ("How big is this corpus?") to the complex ("What themes have ebbed and flowed over time, and how do they relate across authors?"). Whether using desktop applications like OpenRefine and Gephi, the command-line rdr tool, or custom Python scripts, the system makes it possible to supplement traditional reading with computational analysis, making research both more thorough and more comprehensive.