A Review of Biographical Databases

Last week (27-28 May), we organized a two-day Expert meeting on biographical databases. Instead of a linear account of this meeting, what we propose here is a critical review of existing biographical databases (DB). This review is not meant to be exhaustive but rather to identify and reflect upon the recent trends in historical research based on such databases (for detailed information on individual DBs, see the resources page).

Research– vs. public– driven databases

From the late 1980s, databases have become ubiquitous. This ubiquity has created much ambiguity and confusion. The very term database in now imbued with multiple meanings and comes under a variety of forms, definitions, uses and users. We first need to clarify the notion if we are to see clearer through the proliferating supply of DBs over the last decade. Let’s start with a series of elementary distinctions: databases for research – or research-driven DB – and databases for genealogists or the general public. Among the latter, some are more akin to online biographical dictionaries, such as Richard R. Wertz’s Biographical Database of the Republic of China (Taiwan) (1949- Present). More sophisticated platforms like the Biographical Database of Australia (BDA Online) allow users to browse lists of persons or index of records, but as a volunteer (declared “not-for-profit”) enterprise, users who want to access actual information need to pay a subscription fee. Although they could be valuable resources for professional historians, such DBs are not driven by research questions and show no concern for developing appropriate research tools.
As historians, we are more interested in research-driven databases (RDB). They will be our main focus in this post. RDBs fall into two main groups: infrastructures and applied projects.

Database projects

We can identify two main generations of DB projects. Pioneering projects launched in the late 1990s-early 2000s (Harvard CDBD, Bases des élites suisses) incorporate multiple yet elementary information (person, institution, date, location) and simple relational models (person to person, person to institution). They usually rely on predefined categories that allow for little flexibility and overlap (spheres of social life for Swiss elites, Harvard dynastic chronology). They provide basic search engines (allowing simple and multifaceted searches) and modes of display (mostly list views) to navigate through data. CDBD offers more sophisticated features, such as the use of Geographical Information Systems for mapping locations and other spatial data, and Social Network Analysis (SNA) to visualize relationships between people. Most of them, however, are still in progress or remain at a very basic level due to technological constraints. Marylin Levine‘s pioneering project on “Chinese Biographical Database and Cohort of Chinese Political Development” (CBD) (1998-2006) went much further by including temporal analysis (date, duration, activity-based, event-based). The CBD has been transformed to an actual dataset capable of multivariate and network analysis.

During the last decade, a new generation of emerging projects (still in infancy) aim to integrate more complex data (not only persons and institutions but also events and multiple relations) and to allow researchers to modulate the granularity of their enquiries. Due to this increased complexity, they are also concerned with developing more efficient tools for browsing, mining and extracting information and for analyzing results. These new tools go far beyond previous techniques of manual input and basic search engines, using a combination of innovative methods such as natural language processing, machine learning, sequence analysis (Biography.net, Lee-Campbell Research Group). They are also concerned  with providing a more interactive interface that allows researchers to perform complex queries, to navigate through social networks (Singapore Biographical DB, Lee-Campbell) or to enrich original documents (APIS‘ annotating tools).
In this transitional phase, however, any classifying attempt is doomed to fail and the most sustainable projects stand somewhere in between. Among the pioneers, Harvard CDBD or the Swiss Elites database are currently developing new features in response to researchers’ evolving needs and the changing digital environment. In this adaptive process, however, they are often constrained by their previous technical choices.

At a more substantial level, it is useful to note that nearly all biographical projects are nation-based, even though they increasingly integrate the transnational trajectories of individuals or institutions (Swiss elites, Rockefeller fellows). This is probably not so much an issue of research  focus or competencies, but rather of financial and institutional constraints (sources of funding, institutional base).

In terms of approach, the first generation favored prosopographical studies (based on predefined groups of elites and limited samples), while the second one deliberately starts from a looser definition of elites and attempts to take the massive and heterogeneous digital corpora now available with the ambition to enlarge the scale of data extraction and analysis (ENP-China’s “big historical data”).

As for primary sources, current projects generally rely on well-structured documents (most often biographical dictionaries, lists of persons, directories or recorder cards) from which data can be relatively easily extracted (though many problems subsist in the post-processing phase, such as fuzzy datation, missing or conflicting information). None of them, however, has ever attempted to handle such complex and messy sources as newspapers for the contemporary period. If we are to go beyond raw lists of individuals or the linear retrospect to be found in dictionaries and “Who’s Who”, such complex sources became nonetheless inevitable. By providing unique insights into the daily life of individuals, they promise to enrich our knowledge of the process of elites-making and our understanding of their contingent trajectories. But they will also complicate our choices regarding the granularity of analysis and the technical solutions for handling this increased complexity.

Database infrastructures

Database infrastructures come under two main categories. Relational or table-driven database are based on data atomization and a predefined – more or less flexible – structure: Fichoz (relying on FileMaker), SymoGIH (concerned with ontology and interoperability) and Heurist (which emphasizes collaboration, users’ autonomy and public interface). This category more generally includes MySQL databases and such softwares as “Microsoft Access”. The second group includes the emerging generation of “No SQL” databases (MongoDB being one of the most popular examples). In their ambition to overcome the rigidity of relational  DB, the most innovative tools today can be document-driven (Freizo) or graph-based (Padagraph). Both are more intuitive and in both cases, the structure of the DB “naturally” derives from the historians’ direct interaction with primary sources. Freizo enables users to isolate and annotate fragments of documents, offering customized granularity, automatic extraction and direct link to original documents. Padagraph offers alternate lenses for exploring massive corpora that resist mainstream tables-driven infrastructures, based on iterative queries and the combination of  multiple ways of querying data.

From the infrastructural standpoint, first-generation projects rely almost exclusively on relational (table-based) softwares (FileMaker, Access). Most recent projects still hesitate or favor a combination of infrastructure models. We are still in an experimental phase and we will see in the near future whether a new dominating model emerges or whether multiple models can coexist and allow for more flexibility — while also making choice more difficult for researchers and engineers involved in DB building…


Cécile Armand

Ancienne élève de l'Ecole Normale Supérieure (ENS) de Lyon (2006), agrégée d'histoire (2009), docteure en histoire (2017), postdoctorante à Stanford University (2017-8) puis Aix-Marseille Université dans le cadre du projet ERC “Elites, Networks and Power in modern China” (2018-). Ma thèse soutenue à l’ENS Lyon en juin 2017 proposait une histoire spatiale de la publicité à Shanghai (1905-1949). Mon nouveau projet porte sur l’invention du consommateur et de l’expertise sur le marché en Chine républicaine. Au-delà, mes centres d'intérêt couvrent l'histoire urbaine, spatiale et sociale, les outils et méthodes numériques en sciences sociales, l'historiographie et l'épistémologie en général.

You may also like...

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.