Repository logo

Concept Lab: Precomputed Associations for Shared Lexis Tool and Associated Files (Public)

Change log


Recchia, Gabriel 
Jones, Ewan 
Nulty, Paul 
de Bolla, Peter 
Regan, John 


This dataset consists of:

I. Source code and documentation for the "Shared Lexis Tool", a Windows desktop application that provides a means of exploring all of the words that are statistically associated with a word provided by the user, in a given corpus of text (for certain predefined corpora), over a given date range.

II. Source code and documentation for the "Coassociation Grapher", a Windows desktop application. Given a particular word of interest (a “focal token”) in a particular corpus of text, the Coassociation Grapher allows you to view the relative probability of observing other terms (“bound tokens”) before or after the focal token.

III. Numerous precomputed files that need to be hosted on a webserver in order for the Shared Lexis Tool to function properly;

IV. Files that were created in the course of conducting the research described in "Tracing shifting conceptual vocabularies through time" and "The idea of liberty" (full citations in above section 'SHARING/ACCESS INFORMATION'), including "cliques" ( of words that frequently appear together;

V. Source code of text-processing scripts developed by the Concept Lab, primarily for the purpose of generating precomputed files described in section III, and associated data.

The Shared Lexis Tool and Coassociation Grapher (and the required precomputed files) are also being hosted at from 2018 to 2023, and therefore those who are merely interested in using the tools within this time frame will have no use for the present dataset. However, these files may be useful for individuals who wish to host the files on their own webserver, for example, in order to use the Shared Lexis tool past 2023. See README.txt for more information.


Software / Usage instructions

The Shared Lexis Tool and Coassociation Grapher (and the required precomputed files) are being hosted at from 2018 to 2023. Currently, when either of these Windows applications require precomputed files, they retrieve them from If these files are no longer hosted at, they will need to be hosted elsewhere in order to make use of either of these tools. The majority of the precomputed files described in Section III above need to be hosted in a directory called "counts" in order for the Shared Lexis Tool to be able to find them. This directory should be an immediate subfolder of the primary domain - e.g. . Each file with a filename of the form 5-counts-[CORPUSNAME]..clean-dist[DISTANCE].tar contains many smaller files with the extension .gz. In order to allow the Shared Lexis Tool to find these files, they must each be untarred and placed at the following directory within the primary domain: counts/[CORPUSNAME]..clean/dist_[DISTANCE]/FILENAME For example, 5-counts-eebo..clean-dist10.tar includes the file 000000.the.csv.gz, among others. It needs to be placed at counts/eebo..clean/dist_10/000000.the.csv.gz. To accomplish this, the contents of 5-counts-eebo..clean-dist10.tar would need to be untarred and the contents placed in the folder counts/eebo..clean/dist_10 on your online server. It is only necessary to do this with the corpora you plan to use. Furthermore, if you plan to access metadata for ECCO using the Shared Lexis Tool, the accompanying files, ecco-donut..gini.tar.gz, ecco-donut..spec.tar.gz, and ecco-donut..stdev.tar.gz need to be untarred and placed in directories named, ecco-donut..gini, ecco-donut..spec, and ecco-donut..stdev, respectively. These directories should be immediate subfolders of the primary domain - e.g. See README.txt for more information.


text and data mining, digital humanities



CC BY (except for, which is made available under Public Domain Dedication and License v1.0)
Foundation for the Future
Foundation for the Future, DIGITAL KNOWLEDGE, RG74515