Microtask programming is a form of crowdsourcing for programming in which implementation work is decomposed into short, self-contained microtasks. Each microtask offers a specific goal (e.g., write a unit test) as well as all of the required context and environment support necessary to accomplish this goal. Key to microtasking is the choice of workflow, which delineates the microtasks developers may complete and how contributions from each are aggregated to generate the final software product. Existing approaches either rely on a single developer to manually generate all microtasks, limiting their potential scalability, or impose coordination requirements which limit their effectiveness. Inspired by behavior-driven development, we describe a novel workflow for decomposing programming into microtasks in which each microtask involves identifying, testing, implementing, and debugging an individual behavior within a single function. We apply this approach to the implementation of microservices, demonstrating the first approach for implementing a microservice through microtasks. To evaluate our approach, we conducted a user study in which a small crowd worked to implement a simple microservice and test suite. We found that the crowd was able to use a behavior-driven microtask workflow to successfully complete 350 microtasks and implement 13 functions, quickly onboard and submit their first microtask in less than 24 minutes, contribute new behaviors in less than 5 minutes, and together implement a functioning microservice with only four defects. We discuss these findings and their implications for incorporating microtask work into open source projects.
 Abstract The term ‘crowdsourcing’ was initially introduced in 2006 to describe an emerging distributed problem-solving model by online workers. Since then it has been widely studied and practiced to support software engineering. In this paper we provide a comprehensive survey of the use of crowdsourcing in software engineering, seeking to cover all literature on this topic. We first review the definitions of crowdsourcing and derive our definition of Crowdsourcing Software Engineering together with its taxonomy. Then we summarise industrial crowdsourcing practice in software engineering and corresponding case studies. We further analyse the software engineering domains, tasks and applications for crowdsourcing and the platforms and stakeholders involved in realising Crowdsourced Software Engineering solutions. We conclude by exposing trends, open issues and opportunities for future research on Crowdsourced Software Engineering. Paid crowd work offers remarkable opportunities for improving productivity, social mobility, and the global economy by engaging a geographically distributed workforce to complete complex tasks on demand and at scale. But it is also possible that crowd work will fail to achieve its potential, focusing on assembly-line piecework. Can we foresee a future crowd workplace in which we would want our children to participate? This paper frames the major challenges that stand in the way of this goal. Drawing on theory from organizational behavior and distributed computing, as well as direct feedback from workers, we outline a framework that will enable crowd work that is complex, collaborative, and sustainable. The framework lays out research challenges in twelve major areas: workflow, task assignment, hierarchy, real-time response, synchronous collaboration, quality control, crowds guiding AIs, AIs guiding crowds, platforms, job design, reputation, and motivation.