Details
| Original language | English |
|---|---|
| Pages (from-to) | 369-378 |
| Number of pages | 10 |
| Journal | Social science computer review |
| Volume | 36 |
| Issue number | 3 |
| Publication status | Published - 1 Jun 2018 |
| Externally published | Yes |
Abstract
Web surveys are commonly used in social research because they are usually cheaper, faster, and simpler to conduct than other modes. They also enable researchers to capture paradata such as response times. Particularly, the determination of proper values to define outliers in response time analyses has proven to be an intricate challenge. In fact, to a certain degree, researchers determine them arbitrarily. In this study, we use “SurveyFocus (SF)”—a paradata tool that records the activity of the web-survey pages—to assess outlier definitions based on response time distributions. Our analyses reveal that these common procedures provide relatively sufficient results. However, they are unable to detect all respondents who temporarily leave the survey, causing bias in the response times. Therefore, we recommend a two-step procedure consisting of the utilization of SF and a common outlier definition to attain a more appropriate analysis and interpretation of response times.
Keywords
- JavaScript, response behavior, response quality, survey participation, usability study
ASJC Scopus subject areas
- Social Sciences(all)
- General Social Sciences
- Computer Science(all)
- Computer Science Applications
- Social Sciences(all)
- Library and Information Sciences
- Social Sciences(all)
- Law
Cite this
- Standard
- Harvard
- Apa
- Vancouver
- BibTeX
- RIS
In: Social science computer review, Vol. 36, No. 3, 01.06.2018, p. 369-378.
Research output: Contribution to journal › Article › Research › peer review
}
TY - JOUR
T1 - Investigating the Adequacy of Response Time Outlier Definitions in Computer-Based Web Surveys Using Paradata SurveyFocus
AU - Höhne, Jan Karem
AU - Schlosser, Stephan
N1 - Publisher Copyright: © 2017, © The Author(s) 2017.
PY - 2018/6/1
Y1 - 2018/6/1
N2 - Web surveys are commonly used in social research because they are usually cheaper, faster, and simpler to conduct than other modes. They also enable researchers to capture paradata such as response times. Particularly, the determination of proper values to define outliers in response time analyses has proven to be an intricate challenge. In fact, to a certain degree, researchers determine them arbitrarily. In this study, we use “SurveyFocus (SF)”—a paradata tool that records the activity of the web-survey pages—to assess outlier definitions based on response time distributions. Our analyses reveal that these common procedures provide relatively sufficient results. However, they are unable to detect all respondents who temporarily leave the survey, causing bias in the response times. Therefore, we recommend a two-step procedure consisting of the utilization of SF and a common outlier definition to attain a more appropriate analysis and interpretation of response times.
AB - Web surveys are commonly used in social research because they are usually cheaper, faster, and simpler to conduct than other modes. They also enable researchers to capture paradata such as response times. Particularly, the determination of proper values to define outliers in response time analyses has proven to be an intricate challenge. In fact, to a certain degree, researchers determine them arbitrarily. In this study, we use “SurveyFocus (SF)”—a paradata tool that records the activity of the web-survey pages—to assess outlier definitions based on response time distributions. Our analyses reveal that these common procedures provide relatively sufficient results. However, they are unable to detect all respondents who temporarily leave the survey, causing bias in the response times. Therefore, we recommend a two-step procedure consisting of the utilization of SF and a common outlier definition to attain a more appropriate analysis and interpretation of response times.
KW - JavaScript
KW - response behavior
KW - response quality
KW - survey participation
KW - usability study
UR - http://www.scopus.com/inward/record.url?scp=85041555758&partnerID=8YFLogxK
U2 - 10.1177/0894439317710450
DO - 10.1177/0894439317710450
M3 - Article
AN - SCOPUS:85041555758
VL - 36
SP - 369
EP - 378
JO - Social science computer review
JF - Social science computer review
SN - 0894-4393
IS - 3
ER -