Browse wiki

Jump to: navigation, search
Crowdsourcing user studies with Mechanical Turk
Abstract User studies are important for many aspectUser studies are important for many aspects of the design process and involve techniques ranging from informal surveys to rigorous laboratory studies. However, the costs involved in engaging users often requires practitioners to trade off between sample size, time requirements, and monetary costs. Micro-task markets, such as Amazon's Mechanical Turk, offer a potential paradigm for engaging a large number of users for low time and monetary costs. Here we investigate the utility of a micro-task market for collecting user measurements, and discuss design considerations for developing remote micro user evaluation tasks. Although micro-task markets have great potential for rapidly collecting user measurements at low costs, we found that special care is needed in formulating tasks in order to harness the capabilities of the approach. Copyright 2008 ACM.ities of the approach. Copyright 2008 ACM.
Abstractsub User studies are important for many aspectUser studies are important for many aspects of the design process and involve techniques ranging from informal surveys to rigorous laboratory studies. However, the costs involved in engaging users often requires practitioners to trade off between sample size, time requirements, and monetary costs. Micro-task markets, such as Amazon's Mechanical Turk, offer a potential paradigm for engaging a large number of users for low time and monetary costs. Here we investigate the utility of a micro-task market for collecting user measurements, and discuss design considerations for developing remote micro user evaluation tasks. Although micro-task markets have great potential for rapidly collecting user measurements at low costs, we found that special care is needed in formulating tasks in order to harness the capabilities of the approach. Copyright 2008 ACM.ities of the approach. Copyright 2008 ACM.
Bibtextype inproceedings  +
Doi 10.1145/1357054.1357127  +
Has author Aniket Kittur + , Chi E.H. + , Bongwon Suh +
Has extra keyword Human engineering + , Design considerations + , Design process + , Laboratory studies + , Low costs + , Mechanical Turk + , Micro task + , Monetary costs + , Remote user study + , Sample sizes + , Time requirements + , Trade offs + , User evaluations + , User studies + , Wikipedia + , Costs +
Has keyword Mechanical Turk + , Micro task + , Remote user study + , Wikipedia +
Isbn 9781605580111  +
Language English +
Number of citations by publication 0  +
Number of references by publication 0  +
Pages 453–456  +
Published in Conference on Human Factors in Computing Systems - Proceedings +
Title Crowdsourcing user studies with Mechanical Turk +
Type conference paper  +
Year 2008 +
Creation dateThis property is a special property in this wiki. 7 November 2014 15:13:19  +
Categories Publications without license parameter  + , Publications without remote mirror parameter  + , Publications without archive mirror parameter  + , Publications without paywall mirror parameter  + , Conference papers  + , Publications without references parameter  + , Publications  +
Modification dateThis property is a special property in this wiki. 7 November 2014 15:13:19  +
DateThis property is a special property in this wiki. 2008  +
hide properties that link here 
Crowdsourcing user studies with Mechanical Turk + Title
 

 

Enter the name of the page to start browsing from.