Does a Questionnaire’s Content Matter? Experimental Evidence on the Use of Different Treatments to Improve Retention Rates of Politically Disengaged Persons in Longitudinal Political Science Surveys

Political science surveys are prone to overestimate turnout due to measurement and non-response bias. While several studies attempted to address the former problem, researchers have only recently begun investigating strategies to increase politically disengaged persons’ response rates. Even common approaches, such as monetary incentives or shorter surveys, do not seem to resolve the non-response bias in political science surveys. This challenge becomes even more urgent in longitudinal studies as the response burden increases with repetition, presumably leading to a higher retention rate of politically engaged participants. Recent research provides evidence that politically disengaged persons are more likely to be interested in non-political survey content rather than political content. Consequently, adding additional selected non-political question modules could positively impact their survey participation decisions. Although this approach seems promising, fundamental research on the survey contents’ impact on participation decisions is required. I aim to resolve this research gap by examining (1) whether politically engaged participants are more likely to continue participating in longitudinal political surveys than politically disengaged participants and (2) whether participation decisions of politically disengaged persons depend on the survey’s content. To answer these questions, I will use a survey experiment in the probability-based GESIS Panel that varies (i) the content, (ii) the length, and (iii) the monetary incentive of a questionnaire. After completing different core modules, respondents were randomly assigned to either a module with questions on political attitudes and participation (which could also vary in length and the amount of incentive given) or a module with questions on nature and environment that panelists rated as more interesting in a previous wave. This strategy allows an additional comparison of the effectiveness of different approaches aimed at motivating politically disengaged persons to participate in longitudinal surveys. These results could not only be useful to increase response and retention rates among politically disengaged persons but also to evaluate the utility of adaptive designs for political science surveys.