On Chinese Wikipedia biographies

In the last few months we tried to build a corpus based on the biographies of the Chinese Wikipedia. The reason for having such a corpus is that we can use it for future NLP experiments such as information extraction and we believe that Wikipedia is an ideal ’training ground’ for that due to its vast variety of documents. In this blog post we want to share some details about what we have achieved so far.

SOURCE AND PRE-PROCESSING

An offline copy of the Chinese Wikipedia can be obtained on dumps.wikimedia.org, either in xml or html.¹ The Chinese wikidump as it is also called contains around two million pages (by comparison the English Wikipedia has around six million).  These pages are not only articles but also disambiguation pages (i.e. pages that list different topics that can be found with the same search term), redirects, lists, category, project descriptions etc. In order to reduce the number of non-articles in the dump, we used a python tool called WikiExtractor to do some filtering, which trimmed the corpus to only 1.046.744 pages.²

RULE-BASED CLASSIFIER VERSUS MACHINE LEARNING-BASED CLASSIFIER

In the beginning, we did some experiments on the method for extracting biographies from the offline copy. We wanted to figure out whether we should use a script that kept the documents based on a matching with predetermined language patterns such as (出)生在+ (‘Born in’), 出身 (‘family background’), etc. (rule-based classification) or whether we should rely on an algorithm trained to recognize biographies (Machine learning-based classification). After a lot of back and forth, we concluded that the latter was the most effective approach.

The reason for this is that the wikidump itself does not categorise pages in any meaningful way for us to detect biographies. Although a lot of metadata (page Id, redirect status, title, etc.) can be found in the offline copy, the data does not indicate anything about the type of topic we are dealing with. One could argue that the text of an article itself provides us with some clues about its content. However, due to the size of the dump (not to mention the vast variety of topics) it is impossible to produce a complete list of words and sentence patterns that captures all the nuances of a biography while minimizing the number of unwanted articles in the corpus.

MODEL TRAINING 

Before implementing our chosen method on the whole Wikipedia corpus, two steps had to be taken for preparing such a model. First, we trained the model by providing it with examples and counterexamples. The algorithm was fed with the first three sentences of each example page and was set to detect a difference in language of the biographies and non-biographies. Then we used a randomized 10% of the training sample to assess its accuracy. This process had to be iterated multiple times before the accuracy of the algorithm reached a satisfactory level. In the second step, we did a final test of its performance with a set of 414 manualy labeled unknown articles.³ As outcome of the last assesment, ten articles of the test sample were wrongly classified as biographies and only one biography is skipped.

The challenging part of the preparation was the composition of the examples. For the biographies, we generated a list of person names (in Chinese) via Wikidata SPARQL endpoint and used this list for the Special:export wikipage.⁴ There we were able to download the pages in the form of one big xml-file, which underwent some extra filtering to expunge potential non-articles. As for collecting bad examples, it was not easy as it seemed first. Although every article that was not about a person could be counted as a bad example to the machine, it did not guarantee that the algorithm could naturally recognize biographies. We conducted an error analysis and found out that the model mistook pages of fictional characters (superman, Lu Zhishen, etc.) for biographies, as well as pages of films, manga, books, etc. As a consequence, we collected a more balanced sample of ‘bad’ pages via the Wikidata:

• 2,860 fake persons (categorized in Wikidata as ‘fictional characters’, ‘fictional humans’, ‘literary characters’, ’comics characters’, ‘Videogame characters’, etc.)
• 3,040 media examples (categorized as ‘films’, ’television series’, ‘literary works’, etc.)

We also added another 2,984 random examples which were generated by the Wikipedia API. We removed the persons in this sample by use of web-scraping techniques. The XML-pages of non-biographies were obtained in the same manner as the person pages.

RESULTS

1. The version of Chinese wikipedia we used is the zhwiki-2020-02-01
2.  https://github.com/attardi/wikiextractor
3. Originally there were 415 articles. One Chinese wikipage had to be discarded as it turned out to be written in English.
4. Wikidata SPARQL: https://query.wikidata.org/; Special:export wikipage: https://zh.wikipedia.org/wiki/Special:%E5%AF%BC%E5%87%BA%E9%A1%B5%E9%9D%A2
5. https://github.com/DavidGrangier/wikipedia-biography-dataset; For more information on the dataset: R. Lebret, D. Grangier and M. Auli. 2016. Neural Text Generation from Structured Data with Application to the Biography Domain. arXiv preprint arXiv:1603.07771. (http://arxiv.org/abs/1603.07771)