Abstract

User trials provide valuable information on how users respond to interfaces in practice. However, it can be hard to ensure a representative sample. We propose a methodology to improve the understanding of the sample’s skew and to identify the characteristics of those who are missing. This can improve the interpretation of results and inform further recruitment to improve the sample. The methodology involves comparing samples with survey data from the UK population on technology experience, competence and attitudes. We provide a case study of this methodology in practice. 30 participants were recruited using quota sampling with significant effort to obtain people with low technology experience. Nevertheless, comparison with the survey data identified four key groups of people not included in the sample, covering 29% of the population. We discuss how these missing people would likely respond on the tasks, based on the characteristics of similar people in the survey.

Keywords

user trials, survey data, sampling, inclusive design

Creative Commons License

Creative Commons Attribution-NonCommercial 4.0 International License
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License

Conference Track

Research Paper

Share

COinS
 
Jun 25th, 9:00 AM

Improving our understanding of user trial samples using survey data

User trials provide valuable information on how users respond to interfaces in practice. However, it can be hard to ensure a representative sample. We propose a methodology to improve the understanding of the sample’s skew and to identify the characteristics of those who are missing. This can improve the interpretation of results and inform further recruitment to improve the sample. The methodology involves comparing samples with survey data from the UK population on technology experience, competence and attitudes. We provide a case study of this methodology in practice. 30 participants were recruited using quota sampling with significant effort to obtain people with low technology experience. Nevertheless, comparison with the survey data identified four key groups of people not included in the sample, covering 29% of the population. We discuss how these missing people would likely respond on the tasks, based on the characteristics of similar people in the survey.

 

To view the content in your browser, please download Adobe Reader or, alternately,
you may Download the file to your hard drive.

NOTE: The latest versions of Adobe Reader do not support viewing PDF files within Firefox on Mac OS and if you are using a modern (Intel) Mac, there is no official plugin for viewing PDF files within the browser window.