Associate Professor Marta Poblet, Dr Mari Fitzpatrick and Professor Prem Chhetri
RMIT University, Melbourne, Victoria.
Submitted: 16 December 2016. Accepted: 22 February 2017.
The ubiquity of Internet technologies enables the creation of new forms of digital labour that leverage the data-processing skills of the crowd. Prior barriers to human mobility, participation and engagement such as geographical remoteness and incumbent organisations and practices are rapidly dissolving in the digital age (Manyika et al. 2014). The transition from hierarchal to distributed network structures, from proprietary ownership to open-source standards and models of exchange that include contributory as well as market transactions (Rejeski 2012, Benkler et al. 2013) is underwritten by a multiplicity of established and emergent rules and practices.
The role and contribution of distributed human-computation practices to the development of virtual information and communication platforms for crisis and emergency management is opening new areas for research. Human-computation practices were initially created by ‘spontaneous’ groups of locally affected citizens using mobile devices and social media networks. These organically self-organised groups shared information and insights as emergencies and natural disasters unfolded. Subsequent to these early instances of information crowdsourcing, innovative and self-organising work units have developed that use the ‘cognitive surplus’ of the crowd and ‘aggregated intellectual skills’ to gather and process critical emergency information.
To date, much of the research emphasises the role of technology over the social implications of digital labour. This paper examines the structures and practices of these virtual processes through a focus on digital labour and microtasking for emergency management as an evolving socio-technical adaptation. Microtasking merges the capabilities of distributed human cognition with communication technology to address a range of information management challenges during emergency situations that could improve logistics response.
Microtasking is sometimes conflated with terms such as ‘crowdsourcing’, ‘microwork’, ‘crowdwork’ and in some cases ‘human computing’. Likewise, the term crowdsourcing has been used alongside human computation, collective intelligence, or social computing (e.g. Quinn & Bederson 2011, Michelucci 2013). The intersections between these domains are noted as they coincide in their focus on horizontal processes that engage large numbers of individuals working towards clearly defined goals. Research on crowdsourcing has already provided comprehensive reviews of the many definitions of the term (e.g. Estellés-Arolas & González-Ladrón-de-Guevara 2012, Hossain & Kauranen 2015). Yet, microtasking as a specific modality of crowdsourcing procedures, has received little attention until recently.
The methodology for this inquiry began with a literature review conducted through online searches of various databases. An emergency case-study approach was used to examine the microtasking approaches of three distinct entities to consider the implications and logistic potential of these practices.
Microtasking, as shown in Figure 1, is a special sub-type of human computation where tasks involving different degrees of complexity are divided into smaller and independent microtasks (Luz et al. 2014). The literature has identifies some defining elements when it comes to:
Microtasking entails the modularisation of problems into microtasks of varying granularity that are processed by a distributed digital labour force. These microtasks are published on computational platforms (e.g. Mechanical Turk, CrowdFlower or ShortTask) that distribute tasks to crowds of workers. The most recent microtasking platforms include the use of blockchain technologies to support large-scale, decentralised collaboration based on distributed governance models (e.g. Backfeed). Through the use of blockchain technology, an open distributed database can be established to record inputs from volunteers, which once entered cannot be altered retroactively.
The key distinction between microtasking initiatives such as crowd science projects, commercial platforms and virtual information management for disasters and emergencies is the open availability of the information products that these systems produce. Crowdscience initiatives are premised on the ‘open sharing of intermediate inputs’. Commercial platforms have exclusive property rights, while the crowd-sourced emergency information is subject to increasing demands for privacy and confidentiality (due to the vulnerability of disaster-affected populations, particularly unaccompanied minors).
There are two basic models of microtasking practice that are differentiated on the basis of task definition, process management, participant incentives and the nature and purpose of the final product (Novak 2013: pp. 422–425). The first model, Amazon’s Mechanical Turk, invites participants to conduct ‘small-scale, granular tasks for a few cents apiece’ (Bollier 2014: p. 33). This model is structured as a linear workflow system whereby distributed individuals execute basic tasks or ‘atomic units’ requiring minimal skills or ‘little cognitive effort’ for financial reward (Novak 2013, p. 422, 431–33). The tasks are predetermined and conducted independently as ‘parallel work’ and in some cases are aggregated later towards a larger task (Novak 2013, p. 423). ‘Atomic’ tasks occupy a problem area that is ‘well-structured’ with modes of execution that are ‘well mapped out’ and require little interactivity between individual workers (Franzoni & Sauermann 2013, p. 10). The purpose of this form of microtask is to minimise costs but obtain ‘high-quality results’ (Saito et al. 2014, p. 401). However, the emphasis on labour flexibility as a cost-saving strategy has drawn criticism that this type of crowdwork is ‘exploitative labour’ (Kittur et al. 2013) and may be regarded as the reinvention of digital and virtual ‘sweatshops’ (Blumberg 2013a, p. 3, Bollier 2014, p. 34); a new form of Tayloristic assembly line production (Novak 2013, p. 422) or unsatisfying ‘assembly-line piecework’ (Kittur et al. 2013, p. 1).
Platforms such as UpDesk allow skilled individuals to access fee-for-service projects, and InnoCentive, invites participants to select research and technical tasks for payment as a form of ‘enterprise crowdsourcing’ (Bollier 2014, p. 6, 34). The tasks offered on these platforms conform to the definition of ‘macro’ tasking as specified by Saito and colleagues (2014, p. 400). The atomic or primitive microtask requires individuals with basic skills to perform simple tasks that are centrally managed as commercial projects (Novak 2013, p. 422). These projects solicit open mass participation but both their processes and products are closed and subject to intellectual property agreements (Franzoni & Sauermann 2013, p. 9).
Blumberg (2013b, pp. 6–7) identifies a set of common characteristics for atomic microtasks:
He contrasts these features with an evolved form of crowdsourcing that entails recruiting ‘many minds’ for sophisticated problem-solving projects (2013b, pp. 5–7). The literature suggests that microtasking has evolved from the atomic prototype to also include new forms of knowledge production that requires workers to interact in order to address complex problems.
The common features of microwork that span these initiatives in emergency management include:
Differences emerge in how each initiative is structured and managed, how volunteers are recruited and the extent to which processes are formalised or remain flexible. The following case studies review three different examples of operational practices that use microtasking as a workflow methodology. The focus is on task definition, process management and the nature and purpose of their information management goals.
Mission 4636 is an example of an emerging, organic, volunteering initiative that arose during the 2010 Haiti earthquake crisis. A detailed empirical analysis of Mission 4636 can be found in Munro (2013). Mission 4636 was established in partnership with the local telecommunications provider Digicel, to allow the local population to send information through SMS about their situation and needs. At the time, 75 per cent of the population owned a cellphone and around 70-80 per cent of cell phone towers were still operational after the earthquake). The number 4636 was advertised as the medical emergency number through local and diaspora radio stations and through word-of-mouth.
The purpose of Mission 4636 was to gather the SMS information and process it into structured reports. The messages received on the 4636 site were mostly in Kreyòl or French and did not encode the sender’s location. Within weeks 2000 Kreyòl and French speaking volunteers from 49 countries were recruited to translate the messages. They were mainly recruited through Facebook and personal networks as there were concerns about reliability, privacy and security issues with an open-call process.
Using a basic microtasking platform, volunteers read, translated and structured messages according to different categories and geolocated callers onto a map and documented missing person information. These formed the four classes of sub-tasks: translation, categorisation, mapping and documenting.
Tasks were undertaken by volunteers on computers using a split screen format with the unstructured report on the left and the relevant plug-in on the right. After these microtasks were completed, the restructured and classified data was sent in English to relevant international response agencies. These agencies defined the types of reports that could be ‘actionable’ and later in the process specified the categories of data that would be most useful (Munro 2013, p. 216). In this case the leading agency was the U.S. military under the supervision of the U.S. State Department.
The respective labour allocation ratios for all tasks (translation, categorisation, mapping and documenting) were respectively 25:5:20:10. The largest task was the translation component, which underwrote and enabled the following tasks and, as Munro (2013) states, the work undertaken by other agencies such as Ushahidi-Haiti (2013, p. 229). Task execution was also supported by a collaborative facility of a basic chat room application. Approximately 1000 workers used this facility to discuss issues such as correct translation for vernacular idioms and acronyms as well as the correct location of areas with non-official place names. This system of peer review improved the quality and accuracy of information (Munro 2013, pp. 230–36).
The main purpose of Mission 4636 was not search and rescue or targeted medical response but to establish situational awareness, monitor changing conditions, track needs and vulnerabilities and direct aid to large populations (Munro 2013, pp. 218–19). These populations included at-risk and vulnerable groups, as well as hospitals and clinics outside the national capital that required supplies. The U.S. military received these structured reports as ‘the main responders to messages sent to Mission 4636’ (Munro 2013, p. 216, 218, 255).
Mission 4636 established an operational model that has become the template for subsequent virtual initiatives, with the ‘development of workflows and protocols to...inform response, recovery, and rebuilding efforts’ (Liu 2014, p. 403). At that time the mission was an emergent and improvised response but has become an established or extending model and virtual crisis information is now an established ‘feature of crisis events’ (Cobb et al. 2014, p. 3).
The second operational practice is illustrated by Humanity Road, an initiative that transitioned from beginnings in 2010 as an ‘emergent’ digital volunteer group to an incorporated not-for-profit organisation. Starbird and Palen (2013) demonstrate how a virtual organisation provides a technology-supported ‘civic response’ to emergencies by monitoring ‘social media’ posts (mainly Twitter) and processing received data to ‘create information resources for victims and responders’ (Starbird & Palen 2013, p. 1). The organisation structure comprises a core group of volunteers who act as a leadership group and recruit from a global spread of ‘episodic volunteers’ during a crisis so that volunteers can formally register or activate when needed. The work of the organisation spans the emergency cycle and operates between declared emergencies (Starbird & Palen 2013, p. 3).
The workflow during an emergency is structured by ‘pre-articulated tasks’ that have been tested in prior events and are also flexible and adaptable to accommodate necessary changes and improvise for unanticipated contingencies. These adaptations may subsequently be formally incorporated as routine work practices (Starbird & Palen 2013, p. 5). The main objective is to collect, verify, filter and synthesise relevant information from social media sources and restructure data into standardised reports as resources for the disaster-affected populations and response agencies (Starbird & Palen 2013). After a disaster strikes the Humanity Road management group decide how they can contribute and what resources are required.
The main objective is to ascertain situational awareness by:
An Event Diary document is established and is coordinated by an editor. Once an event is posted into the Urgent Events window, the leadership group activate a Disaster Desk on a Skype chat platform. This desk has designated areas as virtual workrooms where volunteers can access and process relevant disaster-related information.
Different categories of information are posted into segregated windows and include an account of the disaster event; websites and Twitter accounts to consult, official hashtags to follow, official warnings and hospital and shelter details. Useful information and updates from official sources are also ‘amplified’ and re-routed to target groups and to the public. The activity responds directly with updates to people within disaster zones (2013, p. 4).
As Cobb and co-authors (2014, p. 3) suggest ‘...emergent organisations of remote actors connected through social media are now a feature of the disaster response milieu’. Humanity Road spans both ‘emergent’ and ‘established’ dynamics. It has established stable routines that have been tested in prior events and incorporates ‘episodic’ volunteers that converge for single events and are open to and actively incorporate spontaneous ‘emergent’ volunteer practices.
The Virtual Operations Support Team (VOST) model was established in 2011 during a series of emergencies in the U.S. and has been subsequently replicated on a global scale. VOSTs process crowd-sourced information on behalf of emergency agencies that lack relavant capacity or resources. They also mediate between agencies and the engaged community (Reuter 2014). VOST organisations comprise both volunteer and emergency personnel as known ‘trusted agents’ who are pre-accredited to perform tasks when emergencies unfold.
The role of a VOST is to process crowd-sourced information through a distributed task-assignment structure using cloud-based tools in alignment with formal response agencies (Cloutier 2014). During an emergency, VOSTs operate as a virtual organisation but are distinguished from other volunteer groups as they have ‘a formal connection with an emergency response team during an event’ (Cobb et al. 2014, p. 6). They respond to requests from official agencies that also determine reporting parameters and have structural interoperability and procedural standardisation with these organisations (Cobb et al. 2014, p. 6).
The VOST workflow begins with the establishment of an event-specific incident workbook segmented into different work pages. Remote volunteers sign in through Skype to the team leader account and log their details on the General Availability Table work page. The team leader sets out the workflow tasks and tools for volunteers. The main tasks for designated workers include:
There is also a collaborative facility (chat room) that hosts a ‘backchannel conversation’ process whereby workers discuss the value and accuracy of incoming information (Cobb et al. 2014, p. 5). The team leader will aggregate the data to a predesigned template and send reports to agencies at times predetermined by an agency manager. As Cobb and co-authors (2014) suggest, although the workflow practices of a VOST is ‘collaborative, the interoperability between VOSTs and official agencies accrues to an alignment in organisational structures’ (2014, p. 5). The microtasking function to process information sits within a hierarchical structure.
The VOST acts as an information management bridge between official agencies and the crowdsourcing public during emergencies. The VOST model may have originated as an ‘extending’ formation (non-routine tasks with existing structures) but has quickly become an ‘established’ formation (formal tasks with existing structures) with an authorised organisational structure and a formal reporting relationship with official agencies on whose behalf it manages information in the digital sphere. In this respect they are also an ‘extending’ structure as they conduct what are currently non-routine tasks for an existing emergency management structure. VOSTs use volunteers with professional emergency experience in the digital sphere although there will be a point when digital operations will become an ‘established’ practice. Although the VOST model has been replicated across many countries it is an operational format that is focused on domestic emergencies.
This paper examined the role, types and forms of virtual microtasking for emergency information management to enhance collective intelligence processes to improve emergency response. The three examples presented demonstrate the functions of microtasking and the role in emergency management. The continuum that was drawn from microtasking for emergency management shows clear differences in terms of task structure and complexity as well as task management. While some tasks are simple or atomic, for example, checking and listing relevant websites, other tasks are more complex, such as translation, interpretation and classifying visual data.
Case studies demonstrate that tasks are generally structured and not ill-defined although the eventual aggregation of processed data contributes to a defined understanding or situational analysis. The modularisation and structure of the workflow is such that experienced volunteers can self-select tasks that are more complex or require technical knowledge with new entrants assigned to simple tasks. In this respect the skills and experience of digital labour contributes to a structuring effect and supports self-organising. Task management along the continuum also requires mixed approaches, that is, combinations of orchestrating, monitoring, guiding, trouble-shooting and directing. The information management continuum requires informed decision-making whereby task modules are structured to facilitate a workflow and processed information is aggregated into reports. Thus, microtasking contributes to different stages of the emergency cycle.
The account of Mission 4636 and HR indicate the role of a ‘management team’ with oversight over the workflow, and decision-making responsibilities. However, these roles are tactical and not directive. The management team appear to make decisions collectively or decisions are brought about by the logic of the process. The collaborative function evident in all three studies also addresses the significant ‘cognitive load’ and the stress that volunteers experience when processing a large volume of incoming data within a restricted time window and provides a virtual timeout with mutual support.
The case-study approach adopted in this paper has limitations. All the case studies represent post-event occurrences, which didn’t consider real-time social media uptake of volunteered information. Future research should evaluate the value of microtasking for emergency management agencies over the emergency lifecycle which includes pre, during and post disaster phase. Furthermore, the relative merits and costs of the three approaches should be examined to assess their appropriateness to different types of disasters. Time ambiguity and resource scarcity often impede the effective and efficient response to an emergency call. Emergency organisations should update the processes, tools, training and organisational culture to enhance organisational resilience to enable rapid response to emergency needs and changes in situated environments (Mees et al. 2016) while still controlling costs and quality.
Benkler Y, Shaw A & Mako Hill B 2013, Peer production: A Form of collective Intelligence, Unpublished draft. At: https://mako.cc/academic/benkler_shaw_hill-peer_production_ci.pdf.
Blumberg M 2013a, Foundations in Human Computation, in P. Michelucci (Ed), Handbook of Human Computation, Springer, New York, pp. 3–4.
Blumberg M 2013b, Patterns of Connection, in P. Michelucci (Ed), Handbook of Human Computation, Springer, New York, pp. 5–12.
Bollier D 2014, The Weightless Marketplace: Coming to Terms with Innovative, Payment Systems, Digital Currencies and Online Labor Markets, The Aspen Institute. At: http://csreports.aspeninstitute.org/documents/The_Weightless_Marketplace_PDF_for_Printer.pdf.
Cloutier P 2014, Digital Volunteers and Emergency Management, International Safety Research, pp.1–13. At: www.slideshare.net/patricecloutier/digital-volunteers-and-emergency-management.
Cobb C, McCarthy T, Perkins A, Bharadwaj A, Comis J, Do B & Starbird K 2014, Designing for the Deluge: Understanding and Supporting the Distributed, Collaborative Work of Crisis Volunteers, Computer Supported Cooperative Work, vol. 14, pp. 1–12. At: http://dx.doi.org/10.1145/2531602.2531712
Estellés-Arolas E & González-Ladrón-de-Guevara F 2012, Towards an integrated crowdsourcing definition, Journal of Information Science, vol. 38, pp. 189–200. doi 10.1177/0165551512437638
Franzoni C & Sauermann H 2013, Crowd Science: The organization of scientific research in open collaborative projects, Research Policy, vol. 43, pp. 1–20. At: http://dx.doi.org/10.1016/j.respol.2013.07.005
Hossain M & Kauranen I 2015, Crowdsourcing: a comprehensive literature review, Strategic Outsourcing: An International Journal, vol. 8, no. 1, pp. 2–22.
Kittur A, Nickerson JV, Bernstein MS, Gerber EM, Shaw A, Zimmerman J, Lease M & Horton JJ 2013, The Future of Crowd Work, Proceedings of the 2013 conference on Computer Supported Cooperative Work, pp. 1301–1318. At: http://hci.stanford.edu/publications/2013/CrowdWork/futureofcrowdwork-cscw2013.pdf.
Liu S 2014, Crisis Crowdsourcing Framework: Designing Strategic Configurations of Crowdsourcing for the Emergency Management Domain, Computer Supported Cooperative Work vol. 23, pp. 389–443.
Luz N, Silva N & Novais P 2014, A survey of task-oriented crowdsourcing, Artificial Intelligence Review, pp. 1–27.
Manyika J, Bughin J, Lund S, Nottebohm O, Poulter D, Jauch S & Ramaswamy S 2014, Global Flows in a digital age: how trade, finance, people, and data connect the world economy. McKinsey Global Institute, pp. 1–180.
Mees B, McMurray A & Chhetri P 2016, Organisational resilience and emergency management, Australian Journal of Emergency Management, vol. 31, no. 2, pp. 38–43. At: https://ajem.infoservices.com.au/items/AJEM-31-02-08.
Michelucci P (Ed) 2013, Handbook of Human Computation, Springer, New York.
Munro R 2013, Crowdsourcing and the crisis-affected community: Lessons learned and looking forward from Mission 4636, Journal of Information Retrieval, vol. 16, no. 2, pp. 210–266. At: http://link.springer.com/article/10.1007%2Fs10791-012-9203-2.
Novak J 2013, Collective Action and Human Computation: From Crowd-Workers to Social Collectives, in Michelucci P. (Ed.), Handbook of Human Computation. Springer, pp. 421–446.
Quinn JA & Bederson BB 2011, Human Computation: A survey and Taxonomy of a Growing Field, CHI Conference, 7–12 May 2011, Vancouver, Canada. At: http://alexquinn.org/papers/Human%20Computation,%20A%20Survey%20and%20Taxonomy%20of%20a%20Growing%20Field%20(CHI%202011).pdf.
Rejeski D 2012, Governing on the Edge of Change: A Report from the Next Policy Frontier, Policy Brief, Wilson Center, pp. 1–6. At: www.americanbar.org/content/dam/aba/events/environment_energy_resources/2014/10/22nd-fall-conference/course_materials/1b-cruden-paper.authcheckdam.pdf.
Reuter S 2014, Virtual VOAD: Creating and Operating Virtual Social Media Teams for VOADs, Presentation, National VOAD Conference 2014. At: www.slideshare.net/ScottReuter1/virtual-voad-creating-and-operating-virtual-social-media-teams.
Saito S, Watanabe T, Kobayashi M & Takagi H 2014, Skill Development Framework for Micro-Tasking, in Stephanidis C & Antona M (Eds). Universal Access in Human-Computer Interaction. doi 10.1007/978-3-319-07440-5_37
Starbird K & Palen L 2013, Working and Sustaining the Virtual Disaster Desk. Proceedings of the Computer-Supported Cooperative Work Conference, 2013, pp. 491–502. At: http://epic.cs.colorado.edu/wp-content/uploads/cscw2013_final.pdf.
Dr Marta Poblet is an Associate Professor at RMIT University. Her research interests cover areas in the intersection of law and technology, including mobile technologies, open data, crowdsourcing, conflict and disaster management and crisis mapping.
Dr Mari Fitzpatrick works as a researcher at RMIT University, Graduate School of Business and Law. She has a PhD in Social Anthropology.
Dr Prem Chhetri is Professor of geo-logistics at RMIT University. His research interests include urban fire and emergency planning, logistics clusters, innovation and growth and port logistics.