TY - JOUR

T1 - Question order effects

T2 - how robust are survey measures on political solidarities with reference to Germany and Europe?

AU - Höhne, Jan Karem

AU - Goerres, Achim

N1 - Funding Information: The inspection of the response distributions shows that they are skewed. However, the order of the questions did not substantially impact people’s response behavior. There is no higher willingness (in the form of agreeing answers) to endorse governmental support depending on the question order. This impression was supported by the results of two chi-square tests: Germany [χ = 3.80, df = 4, p-value = 0.43] and Europe [χ = 3.36, df = 4, p-value = 0.50]. Both tests indicated no statically significant differences.

PY - 2024/3

Y1 - 2024/3

N2 - The measurement of political solidarities and related concepts is an important endeavor in numerous scientific disciplines, such as political and social science research. European surveys, such as the Eurobarometer, frequently measure these concepts for people’s home country and Europe raising questions with respect to the order of precedence. Research has shown that the order of asking questions can have a profound impact on answer behavior compromising data quality. In this study, we therefore investigated the occurrence of question order effects in a German-European context using two questions on political solidarities. For this purpose, we conducted an experiment in a German online panel (N = 874) and analyzed response behavior and effort in terms of response times. In contrast to previous research, we found no empirical evidence for question order effects impacting people’s responses. Even though there were no response time differences between the question order conditions, the first question always took longer to respond to than the remaining one. Overall, our findings indicate the robustness of questions on political solidarities against question order effects. One potential explanation is that people have comparatively strong (or crystallized) attitudes when it comes to political solidarities.

AB - The measurement of political solidarities and related concepts is an important endeavor in numerous scientific disciplines, such as political and social science research. European surveys, such as the Eurobarometer, frequently measure these concepts for people’s home country and Europe raising questions with respect to the order of precedence. Research has shown that the order of asking questions can have a profound impact on answer behavior compromising data quality. In this study, we therefore investigated the occurrence of question order effects in a German-European context using two questions on political solidarities. For this purpose, we conducted an experiment in a German online panel (N = 874) and analyzed response behavior and effort in terms of response times. In contrast to previous research, we found no empirical evidence for question order effects impacting people’s responses. Even though there were no response time differences between the question order conditions, the first question always took longer to respond to than the remaining one. Overall, our findings indicate the robustness of questions on political solidarities against question order effects. One potential explanation is that people have comparatively strong (or crystallized) attitudes when it comes to political solidarities.

KW - data quality

KW - non-probability panel

KW - Response behavior

KW - survey question order

KW - web survey

UR - http://www.scopus.com/inward/record.url?scp=85163157320&partnerID=8YFLogxK

U2 - 10.1080/13645579.2023.2227011

DO - 10.1080/13645579.2023.2227011

M3 - Article

AN - SCOPUS:85163157320

VL - 27

SP - 249

EP - 254

JO - International Journal of Social Research Methodology

JF - International Journal of Social Research Methodology

SN - 1364-5579

IS - 2

ER -