Update on the Wikipedia Biographies

In the previous blog post, we described how we retrieved Chinese biographies by the use of a machine learning-classifier. Since then, we updated the collection with English biographies that are linked to the Chinese pages, resulting in a total number of biographies is 338,857 (228,144 in Chinese and 110,713 in English), and derived two forms of metadata from the texts: named entities and inter-language links.

Named entities are text elements that constitute everything that can referred to with a name or a noun (‘Amandine’, ‘Belgium’, etc.). There are different types of named entities, but in case of our project, we focus on extracting four types of entities: persons, organizations, locations and geopolitical entities. The purpose of such information is to explore the content of an individual biography as well as link a group of biographies that shared the same persons, organizations, locations and/or geopolitical entities.

As we have a bilingual corpus, we attempt to create a bridge between the Chinese and English corpus by linking the named entities in different languages. For that, we rely on the links that a wikipedia provides with the names. These links help us distinguish different entities sharing the same name (“Flanders”, “John Johnson”, etc.) and, via them, we can automatically access the article page of a named entity where we retrieved the inter-language link associated with that name. This kind of mapping will not always be complete as it depends on the availability of the links in each text.

If one wishes to learn more about the dataset as well as the creation process, we wrote an article, titled “Creating Biographical Networks from Chinese and English Wikipedia” that will appear in Journal of Historical Network Research (link will be provided). One can also find the dataset on Zenodo. 

For those who are interested in the topic of Named entity Recognition, there is a good blog post written by Baptiste Blouin about that.