A Review of Biographical Databases
Last week (27-28 May), we organized a two-day Expert meeting on biographical databases. Instead of a linear account of this meeting, what we propose here is a critical review of existing biographical databases (DB). This review is not meant to be exhaustive but rather to identify and reflect upon the recent trends in historical research based on such databases (for detailed information on individual DBs, see the resources page).
Research– vs. public– driven databases
We can identify two main generations of DB projects. Pioneering projects launched in the late 1990s-early 2000s (Harvard CDBD, Bases des élites suisses) incorporate multiple yet elementary information (person, institution, date, location) and simple relational models (person to person, person to institution). They usually rely on predefined categories that allow for little flexibility and overlap (spheres of social life for Swiss elites, Harvard dynastic chronology). They provide basic search engines (allowing simple and multifaceted searches) and modes of display (mostly list views) to navigate through data. CDBD offers more sophisticated features, such as the use of Geographical Information Systems for mapping locations and other spatial data, and Social Network Analysis (SNA) to visualize relationships between people. Most of them, however, are still in progress or remain at a very basic level due to technological constraints. Marylin Levine‘s pioneering project on “Chinese Biographical Database and Cohort of Chinese Political Development” (CBD) (1998-2006) went much further by including temporal analysis (date, duration, activity-based, event-based). The CBD has been transformed to an actual dataset capable of multivariate and network analysis.
At a more substantial level, it is useful to note that nearly all biographical projects are nation-based, even though they increasingly integrate the transnational trajectories of individuals or institutions (Swiss elites, Rockefeller fellows). This is probably not so much an issue of research focus or competencies, but rather of financial and institutional constraints (sources of funding, institutional base).
In terms of approach, the first generation favored prosopographical studies (based on predefined groups of elites and limited samples), while the second one deliberately starts from a looser definition of elites and attempts to take the massive and heterogeneous digital corpora now available with the ambition to enlarge the scale of data extraction and analysis (ENP-China’s “big historical data”).
As for primary sources, current projects generally rely on well-structured documents (most often biographical dictionaries, lists of persons, directories or recorder cards) from which data can be relatively easily extracted (though many problems subsist in the post-processing phase, such as fuzzy datation, missing or conflicting information). None of them, however, has ever attempted to handle such complex and messy sources as newspapers for the contemporary period. If we are to go beyond raw lists of individuals or the linear retrospect to be found in dictionaries and “Who’s Who”, such complex sources became nonetheless inevitable. By providing unique insights into the daily life of individuals, they promise to enrich our knowledge of the process of elites-making and our understanding of their contingent trajectories. But they will also complicate our choices regarding the granularity of analysis and the technical solutions for handling this increased complexity.
Database infrastructures come under two main categories. Relational or table-driven database are based on data atomization and a predefined – more or less flexible – structure: Fichoz (relying on FileMaker), SymoGIH (concerned with ontology and interoperability) and Heurist (which emphasizes collaboration, users’ autonomy and public interface). This category more generally includes MySQL databases and such softwares as “Microsoft Access”. The second group includes the emerging generation of “No SQL” databases (MongoDB being one of the most popular examples). In their ambition to overcome the rigidity of relational DB, the most innovative tools today can be document-driven (Freizo) or graph-based (Padagraph). Both are more intuitive and in both cases, the structure of the DB “naturally” derives from the historians’ direct interaction with primary sources. Freizo enables users to isolate and annotate fragments of documents, offering customized granularity, automatic extraction and direct link to original documents. Padagraph offers alternate lenses for exploring massive corpora that resist mainstream tables-driven infrastructures, based on iterative queries and the combination of multiple ways of querying data.
From the infrastructural standpoint, first-generation projects rely almost exclusively on relational (table-based) softwares (FileMaker, Access). Most recent projects still hesitate or favor a combination of infrastructure models. We are still in an experimental phase and we will see in the near future whether a new dominating model emerges or whether multiple models can coexist and allow for more flexibility — while also making choice more difficult for researchers and engineers involved in DB building…