The making of the MCGD Gazetteer

The ENP-China project (ERC AdG ) is pleased to announce the release of three geospatial datasets of its Modern China Geospatial Database. The main dataset — MCGD_Data_V2 —is the historical gazetteer that we have been developing to identify place names in China under their various denominations (in Chinese) and spellings (in different historical transliteration systems and in English) and their geocoordinates (Latitude/Longitude).

This is meant to serve as a reference tool for place names in modern China and facilitate the identification of places in their varying designations in historical sources. To this end, the ENP-China project has developed a visual interface to be presented in the next post on this blog. All through the preparation of the gazetteer, we have been keeping a record of how we developed it, which I will present in the next sections.

Note: As a rule, all the names in Chinese are provided in traditional script, except for MCGD_PRC_V1 for which we added a column with the names in simplified Chinese.

We built the initial place name data set based on data from the Geonames database and from the Harvard Historical GIS database. We did not work directly from these two databases, but from the Filemaker database (GeoGeneral) produced by Jean-Pierre Dedieu in which he had compiled the entire Geonames database (except Taiwan) and the Harvard Historical GIS database.

From Geo_General to Geo-General (China)
We first extracted from the Geo_General filemaker file all the place names listed under “China” as a country and created a new file (China_ALL3), which served as the basis for the processing and refining of the data. We deleted all the duplicates. Altogether we dropped 461,615 entries. Then we removed all the classes of place names that were not human settlements:          

•    Hydraulics: basically all waterways     

•    Relief: basically mountains and peaks     

•    Buildings: built structures like temple, etc.     

•    Geoarea: it should designate geographical areas (like North China, Gobi Desert), but in fact it also contains built-up structures (e.g. gates of  Beijing)

At the end of this process, there remained 526,550 entries (place names) that included three categories:    

 •    Populated    

 •    Administrative districts     

•    Unknown    In the absence of information about  the nature of the « Unknown » tag, we kept them pending further identification.

All the records in Geo-General (China) were now unique. Yet each place could appear under different names, but only once as a location with its Latitude/Longitude coordinates. At this stage we could estimate that the actual number of places would be cut by one half since they appeared at least in Chinese and pinyin. Yet many places also appeared with a single name (either in Chinese, or pinyin or Wade-Giles). Finally, we sorted by Latitude and Longitude, and deleted all the duplicates to assess the real number of unique locations. This left 251,763 unique sets of coordinates. This figure indicate exactly how many unique places the database contains for China proper. Yet it should be noted that this file does not include Taiwan.

Integrating the Harvard China Historical GIS data
We merged the data from Geo_General (China) and the data from China Historical GIS. This created a number of duplicates with varying geospatial coordinates. We addressed the issue of duplicates in the following way:     

* When the same location appeared with diverging  coordinates, we pasted the best coordinates (Google Maps verification) in the Google Coordinates field.     

* We selected all the lines and columns and searched for duplicates that met all criteria. Around 8,000 records were deleted. 

Yet there remained an issue with place names that represented the same locations, but with only a small difference in the Latitude/Longitude. We developed a small script to identify all such cases and homogenize the coordinates. The process was a mix of automatic processing and manual processing.

Refining the data: removal of built-up structures
In examining the data, we realized that it contained elements that were human settlements, but at a sub-settlement level, such as a post office, a farm, a factory, a production brigade, etc. We weeded out all the named locations that were not actual populated places (City, town, village). We split the data set into two files: Locations and Names. The Location file contained only the Location ID, the Name ID and the set of geospatial coordinates (Latitude/Longitude). The Names file contained the place name (in any language and transliteration), the name of the province in Chinese and pinyin, and the Name ID. The two files shared the Name ID that served to attach the geospatial coordinates to each place name.

We filtered out all the place names that were built-up features in both Chinese and pinyin: e.g., gongshe, nongchang, gongchang, jiedao, linchang, dian, gongdui, dadui, qingniandian, fenchang, dui (with numbers), gongmu, nonmuchang, muchang, famuchang, niuchang, 公社, 農場, 工廠, 街道, 牛場， 木廠, etc. The list is actually much longer and can be found in the R scripts that we used to clean the data. We also identified all the place names that included an administrative level in their name such as fu, xian, ting, zhou, zizhixian, zizhizhou, 府, 縣, 廳, 州, 自治縣, 自治州.  We created separate columns for each category and list them as an “Administrative_Level” category. This also served to extract the name of the locations without their administrative suffix and to list them as names of cities/towns. This created duplicates when a city/town was already listed under the same name. We removed the duplicates (same name, same LocID).

Adding and processing of the Taiwan data
We obtained the Taiwan data from the Genonames database. We downloaded the data here. 

The file contained 45,540 entries. There were two duplicated columns — Name — one of which was deleted right away. One column contained all the alternative names for each Name entry. This information had to be split up to have all the possible alternative names with their respective IDs and coordinates. This was performed in R-studio and produced a new file with all the individualized entries and related data. The file (TWR1) contained 275,704 entries.

The data for place names in the TWR1 file was coded by type of place. We decided to separate the place names and distribute them — as for Geo_General (China) — under different tabs. Our purpose was to retain only the entities marked as administrative units and populated places (city, town, village, etc.). We  attributed unique identifiers (ID_NameLoc) to each place name (Format: T+7 digits starting from T0000001). This process produced a file (TWR1_Tabs.xlsx) with the full data tab and six extra tabs: Populated, Parks, Hydraulics, Buildings, Relief, Rail & Roads. The data in the Populated tab in the TWR1_Tabs.xlsx was saved as  **Taiwan_GIS.csv**. The file contained 135,202 entries.

To sum up, we have three reference files for Taiwan:

1. TWRLL.csv: original Genonames file (45,540 entries)

2. TWSPLIT.csv: all individualized place name entries (275,704 entries)

3. TWGIS.csv: all the Populated places (135,202 entries): The data in the file was added to the main Geo_General (China) file

Integration of additional data
We integrated three external data sets that intersected with the main data set:

• Crow Directories: this is a list of 123 cities cited in the Crow Newspapers Directories (see Cécile Armand’s project on her blog and her data sets on Zenodo) with verified coordinates. Our purpose was to homogenize the data and make sure we had all the 123 cities in the main data set.

• Chinese engineers: CERD is a database developed at Leipzig University by Elizabeth Kaske and Thorben Pelzer. Our purpose was to make sure we had all the place names listed in their database and homogenize the geocoordinates whenever necessary for the same cities.

• 1931 xian: this is a separate project conducted by the ENP-China project to recreate the administrative geography of China in 1931 at the level of xian (county). This project produced a dataset with 1,956 locations that we integrated in the main dataset. The dataset —Republican China Dataset — is available on Zenodo (see below).

The integration of the 1931 xian data produced duplicates, especially in cases when the place names were attached to former provinces (Chahar, Suiyuan, etc.) or when the place name was present only as a full name (with the xian suffix). After processing and removing all the duplicates, we saved the resulting file as Geogen13 with 464,342 entries (including non-China place names in Europe, United States, etc.). Geogen13 was renamed MCGD_Data_V1. This is the final reference file for our main gazetteer.

• MCGD_PRC: this is a data set that lists all the 2,461 locations ranked as cities in the PRC. For details, see the presentation below. 

The integration of the MCGD_PRC data produced duplicates, although all these locations should have been included already in the MCGD_Data_V1 file. Yet the join by city name and province also listed cities that were not present in the MCGD_Data_V1 file. After removing the duplicates manually, we merged the 916 additional locations (but 1,832 names in Chinese/pinyin) in the MCGD_Data_V1 file. We saved the compilation as MCGD_Data_V2.

Republican China dataset

Because MCGD_Data contains a very high number of entries (>800,000), matching files extracted from sources for locations usually produces too many matches due to the existence of duplicated names within the same province (especially at village level) or between provinces. The full file is also cumbersome for processing most matching needs when we extract place names from historical sources. The decision was made to produce a dataset that included the major cities not just in Republication China (despite the name of the file), but in China in the first half of the 20th century.

MCGD_Rep is not a reduction of MCGD_Data. We proceeded in two ways to produce a new file:

• we compiled existing files that had a time stamp (1904, 1911), but we limited the content to the seat of counties (xian), prefectures, and provinces. We also added a group of cities extracted from the Crow directories for newspapers (1931-1937).
• we filtered MCGD_Data for all the names ending with 縣， 府， 聽， 州， 市 to produce a similar file, although this one did not include a time stamp on the names (except for the Harvard China Historical GIS data).

We included the following data from Harvard China Historical GIS data in MCGD_Rep for the year 1911:

• Locations_Towns_All: this is the reference file for the name of cities (or towns). It contains 2,213 names. It is based on the compilation of the set of files below. This file has two tabs: the “Data” tab contains the name of locations stripped down to their simple name, although the other columns provide the full name with the administrative suffix, the administrative level, in Chinese and in pinyin, as well as other types of information. 
• cnty_pts_1911_v6 = 1911_cnty_pts_v6: this file contains the list of the 1,864 county seats in the Harvard China Historical GIS data, with the original coordinates, except for the cities that we extracted from the Crow directories for newspapers (1931-1937). We used the coordinates established with this set of cities.
• pref_1911_v6 = 1911_pref_v6: this file contains the list of the prefecture seats in the Harvard China Historical GIS data, with the original coordinates, except for the cities that we extracted from the Crow directories for newspapers (1931-1937). We used the coordinates established with this set of cities.
• prov _1911_v6: list of 1911 provincial capitals
• cnty-pref_pts_1911_v6 = 1911_cnty-pref_ pts_v6: this is the combined file of 1911_cnty_pts_v6 and 1911_pref_v6.

• China_Xian_1931: we processed the Xingzhengqu (行政區) “Administrative areas” section in the original 1931 Diming dacidian (地名大辭典, see full source below) manually by checking all entries in the Diming dictionary with the entries in the file for our 1934 xian dataset based on data created manually by extracting them from the 1934 atlas published by the Shenbao and made available online (TWS tile service) by Academia Sinica. The 1934 data, therefore, is not an exhaustive and most accurate list. It reflects what the coder and GIS specialist were able to identify on the map. All the 1,720 entities in the 1934 list correspond to 1720 polygons. We used the 1934 list to match it to the 1931 list. A total of 141 xian in the 1934 list were not found in the 1931 list. This is probably due to a change of names. On the opposite, we found 487 xian names in the 1931 list that did not appear in the 1934 list. At the end of the iterative process between the source and the 1934 map (as well as other cartographic sources), we obtained a file with 1,956 xian names.

Apart from the xian, the 1931 Diming dictionary also listed other administrative units in the provinces with non-han populations. We found eight types of other different administrative units: 便委 (1), 土司 (6), 把總 (4), 旗 (17), 汛長 (10), 督辦 (2), 行政委員 (13), 設治局 (12) that we chose not to process at this time due to the lack of sufficiently detailed historical administrative maps to draw the boundaries of these entities. 

PRC Dataset
Although the ENP-China project does not usually deal with place names in the People’s Republic of China, we found useful to establish a resource that can help to locate places in China with a focus on the cities, not all the possible locations in China. We used two different files to establish a base for our dataset: 

• China_3607_Full: this file provided a list of 3,607 locations in Chinese, including cities (368), city districts (2853), xian, and a few jiedao (街道) with their coordinates, the name of the province and municipality. We checked that the coordinates were accurate.
• China_2014Cities: this file provided a list of 2,014 cities in pinyin, with coordinates, the name of the province in pinyin, and partial population data (2010) for 449 cities only. We added the Chinese name to the original data.
• Cities_821: this file provided a list of 821 cities as listed on Wikipedia for 2020.

The combined data produced a file with  with 2,461 names including Hong Kong and Macao, and their geospatial coordinates. 

Brief description of the files:

File	Content
MCGD_Data_V2	MCGD_Data_V2 contains all the data that we have collected on locations in modern China. Altogether there are 464,887 entries. The data include the name of locations and their variants in Chinese, pinyin, and any recorded transliteration; the name of the province in Chinese and in pinyin; Province ID; the latitude and longitude; the Name ID and Location ID, and NameID_Legacy. The Name IDs all start with H followed by seven digits. This is the internal ID system of MCGD (the NameID_Legacy column records the Name IDs in their original format depending on the source). Locations IDs that start with “DH” are data points extracted from China Historical GIS (Harvard University); those that start with “D” are locations extracted from the data points in Geonames; those that have only digits (8 digits) are data points we have added from various map sources.ts (8 digits) are data points we have added from various map sources.
MCGD_Rep_V2	MCGD_Rep_V2 is a sample of data extracted from  MCGD_Data_V1 for the Republican period. It is based mostly on the list of xian (county) seats in 1931 [Source: Zang, Lihe  臧励龢, ed. Zhongguo gujin diming da cidian 中国古今地名大辞典. Shanghai 上海: Commercial Press, 1931], with the addition of some external sources. By and large, it presents a list of the major locations in China between 1900 and 1949. It contains 1,977 entries with the name in Chinese, in pinyin; the name of the province in Chinese and in pinyin; the latitude and longitude, and the Name ID and Location ID.
MCGD_PRC_V1	MCGD_PRC_V1 is a list of cities in today’s PRC. It includes 2,461 locations, with their name in Chinese (both traditional and simplified Chinese) and in pinyin, the name of the province in Chinese and in pinyin; the latitude and longitude.

References:

Ding, Wenjiang 丁文江. 1939. Zhongguo fensheng xin tu Shenbao liushi zhounian jinian 中國分省新圖 申報六十週年紀念. Shanghai 上海: Shenbaoguan.

Ding, Wenjiang 丁文江, Weng Wenhao 翁文灝, and Zeng Shiying 曾世英. 1934. Zhonghua min guo xin ditu: Shenbao liu shi zhounian jinian 中華民國新地圖 : 申報六十週年紀念. Shanghai: Shenbao guan.

Zang, Lihe 臧励龢, ed. Zhongguo gujin diming da cidian 中国古今地名大辞典. 1931. Shanghai 上海: Commercial Press.

Elfie Swerts, « A data base on Chinese urbanization: ChinaCities », Cybergeo: European Journal of Geography [Online], Data Papers, document 830, Online since 21 September 2017, connection on 07 January 2022. URL : http://journals.openedition.org/cybergeo/28554 ; DOI : https://doi.org/10.4000/cybergeo.28554

---

---