Assessing Response Behaviour in Voting Advice Applications

Assessing Response Behaviour in Voting Advice Applications The questionnaire forms the heart of all VAAs. Without their ability to correctly assess the political attitudes of its users and transform these into meaningful matches with political parties, VAAs would be of little use. It is therefore important for the designers of VAAs to assess if there VAA does just that. This means ensuring that the users correctly understood the questions the VAA asked them, that they correctly understood how to respond to them using the response scales, and that they gave each question the time and attention required. Doing so is often complicated, however, as the preferred experimental set-ups can be expensive and time-consuming. Here, I propose a different way which VAA designers can use to study the way their users responded to their questions. Using a combination of Multiple Correspondence Analysis and categorical Principal Component Analysis, I show that designers and scholars can easily visualize the response structure of their users. This allows them to assess whether responses for the “Neutral” category were indeed neutral, or were used to hide an “I do not know” response; whether users really saw the 5- or 7-point scale as such or simplified it; and whether a clusters of questions assumed to belong to the same dimension, indeed do so. As an example, I will use the 2014 EUVox data to assess the general structure of response quality, whether there are any differences between the countries, and how these differences can be explained and, if possible, addressed. In addition, the paper will describe various ways in which designers can implement this approach, not only after the VAA has been completed, but also during its run to improve the response quality of the VAA while it is still operating.