Charles Martin-Shields is Director of Conflict Management and Peacebuilding at TechChange, Washington DC. Gerard McCarthy is TechChange’s Project Manager (Asia-Pacific), based in Sydney.

Since 2006 the private sector has been using social media and SMS text messaging to crowdsource consumer behaviour and trends. Using data mining and mass communication tools, it is now possible to aggregate social media and text message data from thousands of people in real time, analyse market dynamics and instantaneously communicate a micro-targeted response using SMS, Twitter, Facebook or other online platforms.

Senior diplomats, foreign services and peacekeeping operations are beginning to integrate these tools into their ediplomacy and population engagement strategies, recognising the vital foreign policy insights that crowdsourced data can provide on rapidly changing dynamics at the local level.

The UK and the US have highly developed policies on ediplomacy and technology for development. As Australia reflects on how to catch up with them, there are unique technical and institutional challenges to consider in the design and implementation of crowdsourcing operations in the Asia Pacific.

The first challenge is technical and includes access to hardware and managing data security. The second is how to make crowdsourced data flows manageable for responding agencies such as the Department of Foreign Affairs and Trade, AusAID and the Department of Defence.

Access to technology in the Asia Pacific is becoming less of a hurdle as mobile phone penetration increases and security risks are better understood. According to the Mobile and Development Intelligence program, GSM mobile phone penetration in the Oceania region is beyond saturated in many Pacific Island countries. These communication tools have significant implications for diplomatic and political engagement strategies. The rapid dissemination of false rumours about Australia's High Commissioner to PNG via Facebook and Twitter highlights the increasing potential of mobile technologies to significantly impact diplomatic relations within the region almost instantaneously.

However, while mobile phones are ubiquitous within the Asia Pacific, internet access is not. Though the expansion of Digicel and other telcos throughout the Pacific is rapidly increasing 3G coverage in countries such as PNG, internet penetration rates stand at only 2% in PNG, 4.5% in the Solomon Islands and 18% in Fiji. Digital mapping platforms like Ushahidi require reliable internet connectivity, which creates a challenge for diplomatic agencies depending on stable information flows.

The nature of state surveillance also poses data security problems. As Syrian government hacking of online content has demonstrated, open data is often insecure, particularly SMS text messages. Any collection process supported by Australian government agencies or the private sector must account for the potential for screening by local intelligence organs.

This brings us to the institutional challenge. Should the Department of Defence run a crowdsourcing program in operations such as the Regional Assistance Mission to the Solomon Islands? Not necessarily. Even though they do not provide the type of granular details offered by crowdsourced data, the Department of Defence, DFAT and Australian intelligence agencies already have their own high-volume data streams. Asking an analyst to parse thousands of disparate text messages and social media posts, along with handling a large volume of cable traffic, could possibly lead to internal paralysis at the peak of a regional diplomatic crisis.

A way around this is looking to trusted local NGOs running crowdsourcing initiatives. Regional examples include the mapping platforms deployed by the AusAID-funded Australia Indonesia Facility for Disaster Reduction, particularly the recently launched Indonesia Scenario Assessment for Emergencies (InaSAFE), which uses mapping tools to better prepare for tsunami relief and reconstruction activities.

Using cleaned and verified data from local partners such as these can make the mass of digital information more manageable for large agencies and can improve the technical capacity on the ground vital to correcting false rumours, coordinating tangible responses to diplomatic and humanitarian crises and collecting accurate data on aid and development projects.

Crowdsourcing technology is exciting, particularly since it can provide granular localised data to peacekeepers, foreign services and development agencies. But without a solid understanding of the operating environment and well developed local partnerships, we end up with a pile of data that can slow the analytic process right when aid workers, diplomats and peacekeepers need information as quickly as possible.

Photo by Flickr user kahunapulej.