Editor’s Choice: This paper underscores a very important point to keep in mind when designing a citizen science project: make sure you understand what motivates your target participants and how your project design feeds into this. This is particularly true when considering whether to “gameify” your project – some people are actually disincentivized by overt competition such as that displayed through a leaderboard. –LFF–
Abstract: Quantum Moves is a citizen science game that investigates the ability of humans to solve complex physics challenges that are intractable for computers. During the launch of Quantum Moves in April 2016 the game’s leaderboard function broke down resulting in a “no leaderboard” game experience for some players for a couple of days (though their scores were still displayed). The subsequent quick fix of an all-time Top 5 leaderboard, and the following long-term implementation of a personalized relative-position (infinite) leaderboard provided us with a unique opportunity to compare and investigate the effect of different leaderboard implementations on player performance in a points-driven citizen science game. All three conditions were live sequentially during the game’s initial influx of more than 150.000 players that stemmed from global press attention on Quantum Moves due the publication of a Nature paper about the use of Quantum Moves in solving a specific quantum physics problem. Thus, it has been possible to compare the three conditions and their influence on the performance (defined as a player’s quality of game play related to a high-score) of over 4500 new players. These 4500 odd players in our three leaderboard-conditions have a similar demographic background based upon the time-window over which the implementations occurred and controlled against Player ID tags. Our results placed Condition 1 experience over condition 3 and in some cases even over condition 2 which goes against the general assumption that leaderboards enhance gameplay and its subsequent overuse as a an oft-relied upon element that designers slap onto a game to enhance said appeal. Our study thus questions the use of leaderboards as general performance enhancers in gamification contexts and brings some empirical rigor to an often under-reported but overused phenomenon.
Source: Kock Pedersen, M., Ravn Rasmussen, N., Sherson, J.F., Vaid Basaiawmoit, R., 2017. Leaderboard Effects on Player Performance in a Citizen Science Game. arXiv:1707.03704