Philippa Brant is a Lowy Institute Research Associate.

Earlier this week a massive database of Chinese development finance activities in Africa (warning: big PDF) was launched by AidData and the Center for Global Development.

This endeavour involved a team of researchers and has taken more than 18 months, and sought to find and classify all instances of Chinese development finance to Africa from 2001 to 2011. Anyone who has attempted to quantify Chinese aid would know this is no mean feat. Despite recent efforts on the part of the Chinese Government to release more information about its foreign aid program, details at a project, sector or indeed country level remain difficult to determine. This database thus potentially offers a valuable resource.

According to the accompanying report, China's involvement in Africa amounts to some 1673 'officially financed' projects totaling US$75 billion over the past decade. This figure includes foreign aid (ODA), 'other official flows' (OOF), and financing that probably fits in one of these categories. It also includes 'committed' projects that have not yet (and may never) be started. It should therefore not be compared directly to the foreign aid data of DAC donors.

Questions have been raised about methodology. It relies primarily on media reports of Chinese development activities. China Africa expert Deborah Brautigam summed up her concerns on her blog:

The main problem is that the teams that have been collecting the data and their supervisors simply don't know enough about China in Africa, or how to check media reports, track down the realities of a project, and dig into the story to find out what really happened. You can start with media reports, but it is highly problematic to stop there.

To be fair, the authors have been at pains to outline the limitations of their data collection methods. They've published a detailed document outlining their process step-by-step (link is down at time of writing). Indeed, they should be lauded for developing an open-access database and encouraging improvements through crowdsourcing. The ability to disaggregate by sector, country, financing instrument, or project status is of immense help to other researchers.

The authors admit that this methodology is most effective as a research tool when (a) the data are made widely accessible to users with the knowledge to identify errors and (b) it leverages complementary qualitative data-collection methods (such as in-country fieldwork and outreach with personnel involved with specific projects) to prevent over-reliance on media reports.

However, as Brautigam warns, the problem with publishing unclean data is that it will be reported as gospel. This is already proving true. The findings have been reported widely. The nuance and caveats are lost. My taxi driver even quoted the figure to me. Their crucial caveat – 'we issue a firm warning to all data users: the available reporting on Chinese finance in Africa is far too nuanced and imperfect for sweeping generalizations' – has not been heeded.

Understanding and calculating Chinese foreign aid is a challenging task. It requires a deep understanding of how the Chinese aid system works, how to classify different forms of financing, as well as the politics surrounding the reporting of data. In my own research into Chinese aid in the Pacific and Southeast Asia over the past five years, I have confronted many issues with quantifying data. I'm still learning. It is crucial to triangulate data. Media reports are often a great place to start, but you cannot stop there. An announcement of a Chinese aid loan for US$200 million in 2006 can turn out to be a commercial loan of Fiji $200 million that wasn't disbursed until 2009, for example.

You have to be willing to be a detective and 'follow the money', something the authors of this database acknowledge but haven’t yet done to the full extent necessary (47% of their project records rely on a single source). What do local budgets report? What about Chinese sources? It may be a matter of translating a report in Portuguese from a development partner meeting in Timor-Leste and comparing it with announcements on the Chinese Embassy website and Timor-Leste Ministry of Finance budget documents. It may be necessary to go to Papua New Guinea and ask their officials and the Chinese Embassy to confirm details. Or even better, engage a local partner to assist. It isn't always easy. It may not always work. But it can be done.

Despite my reservations, I think this database can be a good resource if used wisely. Now that it is published, I hope other researchers will take up the authors' call to arms to improve and clean the data. I only wish they had waited for more accurate data before publishing. Like the ill-fated Congressional Research Service data before it, the CGD/AidData US$75 billion Chinese 'aid' figure will unfortunately be circulated for years to come.

Readers can follow the ongoing discussion in the comments thread of Brautigam's blog and on the AidData website.

Photo by Flickr user GovernmentZA.