Sample size and the stability of usability estimates obtained through subjective assessment
Are you already subscribed?
Login to check
whether this content is already included on your personal or institutional subscription.
Abstract
The use of small samples for assessing interface usability is one of the most critical aspects in ergonomics research and practice. Small samples are usually employed to cope with the needs of companies that often impose tight schedule for the development of the study, and allow only a limited budget. In the present study this issue has been addressed in relation to the subjective assessments conducted by using the Usability Evaluation questionnaire (Us.E.; Di Nocera, Ferlazzo e Renzi, 2003). Using this instrument it is possible obtaining web site usability estimates along three dimensions: (Mental) Handling, Satisfaction, and Attractiveness. In general, Us.E. allows rapid userbased assessments to be used for orienting successive in-depth analyses and re-design choices. Given this "filter" nature, it is crucial to identify what is the minimum sample size needed for granting stable usability estimates. Results showed that, in order to obtain unbiased estimates in all Us.E. subscales, it is necessary to use a sample of at least 30 users. Smaller sample size would lead to instability in two of the three subscales.
Keywords
- Usability
- internet
- sample size
- resampling
- bootstrap