How to take multiple factors into account when evaluating a Game with a Purpose? How is player behaviour or participation influenced by different incentives? How does player engagement impact their accuracy in solving tasks? In this paper, we present a detailed investigation of multiple factors affecting the evaluation of a GWAP and we show how they impact on the achieved results. We inform our study with the experimental assessment of a GWAP designed to solve a multinomial classification task.
Participation incentives are usually classified as intrinsic or extrinsic motivation @cite_4 . Some comparative analysis of incentives exists for GWAPs @cite_5 , especially in contrast to different methods like micro-working @cite_8 @cite_26 @cite_14 or machine learning @cite_18 . The effect of competition and tangible rewards on participation and quality of results has also been explored, both in the context of GWAPs @cite_21 and online citizen science campaigns @cite_22 , revealing the pros and cons of designing different motivation mechanisms.
@cite_18 @cite_14 @cite_26 @cite_4 @cite_22 @cite_8 @cite_21 @cite_5
2058911993 2039922069 2088940490 2124576904
User participation is one of the most important elements in participatory sensing application for providing adequate level of service quality. However, incentive mechanism and its economic model for user participation have been less addressed so far in this research domain. This paper studies the economic model of user participation incentive in participatory sensing applications. To stimulate user participation, we design and evaluate a novel reverse auction based dynamic pricing incentive mechanism where users can sell their sensing data to a service provider with users' claimed bid prices. The proposed incentive mechanism focuses on minimizing and stabilizing the incentive cost while maintaining adequate level of participants by preventing users from dropping out of participatory sensing applications. Compared with random selection based fixed pricing incentive mechanism, the proposed mechanism not only reduces the incentive cost for retaining the same number of participants but also improves the fairness of incentive distribution and social welfare. It also helps us to achieve the geographically balanced sensing measurements and, more importantly, can remove the burden of accurate price decision for user data that is the most difficult step in designing incentive mechanism. In crowdsensing, appropriate rewards are always expected to compensate the participants for their consumptions of physical resources and involvements of manual efforts. While continuous low quality sensing data could do harm to the availability and preciseness of crowdsensing based services, few existing incentive mechanisms have ever addressed the issue of sensing data's quality. The design of quality based incentive mechanism is motivated by its potential to avoid inefficient sensing and unnecessary rewards. In this paper, we incorporate the consideration of data quality into the design of incentive mechanism for crowdsensing, and propose to pay the participants as how well they do, to motivate the rational participants to perform data sensing efficiently. This mechanism estimates the quality of sensing data, and offers each participant a reward based on her effective contribution. We also implement the mechanism and evaluate the improvements in terms of quality of service and profit of service provider. The evaluation results show that our mechanism achieves superior performance when compared to the uniform pricing scheme. This paper studies economic models of user participation incentive in participatory sensing applications. User participation is the most important element in participatory sensing applications for providing adequate level of service quality. However, incentive mechanism and its economic model for user participation have never been addressed so far in this research domain. In order to stimulate user participation, we design and evaluate a novel Reverse Auction based Dynamic Price (RADP) incentive mechanism, where users can sell their sensing data to a service provider with users' claimed bid prices. The proposed incentive mechanism focuses on minimizing and stabilizing incentive cost while maintaining adequate number of participants by preventing users from dropping out of participatory sensing applications. Compared with a Random Selection with Fixed Price (RSFP) incentive mechanism, the proposed mechanism not only reduces the incentive cost for retaining same number of participants by more than 60 but also improves the fairness of incentive distribution and social welfare. More importantly, RADP can remove burden of accurate pricing for user sensing data, the most difficult step in RSFP. In this paper we present and analyze a model in which users select among, and subsequently compete in, a collection of contests offering various rewards. The objective is to capture the essential features of a crowdsourcing system, an environment in which diverse tasks are presented to a large community. We aim to demonstrate the precise relationship between incentives and participation in such systems. We model contests as all-pay auctions with incomplete information; as a consequence of revenue equivalence, our model may also be interpreted more broadly as one in which users select among auctions of heterogeneous goods. We present two regimes in which we find an explicit correspondence in equilibrium between the offered rewards and the users' participation levels. The regimes respectively model situations in which different contests require similar or unrelated skills. Principally, we find that rewards yield logarithmically diminishing returns with respect to participation levels. We compare these results to empirical data from the crowdsourcing site Taskcn.com; we find that as we condition the data on more experienced users, the model more closely conforms to the empirical data.