Improving the Measurement of Political Behavior by Integrating Survey Data and Digital Trace Data

Computer-mediated communication has become deeply ingrained in political life. People use digital technologies to get political information and news, directly follow political actors, discuss politics with friends, advocate for political causes or mobilize offline protests. The measurement and analysis of these activities pose considerable challenges for researchers since they are distributed across multiple channels and platforms, intertwined, and ephemeral. Political science has traditionally studied these phenomena using survey methods. These, however, suffer from the unreliability of self-reported media use (Prior, 2009). Studies from the emerging field of computational social science, on the other hand, collect digital traces of human behavior in a non-intrusive way. At the same time, these approaches oftentimes do not collect the necessary attributes of research subjects (e.g., sociodemographic or personality characteristics) and/or outcome variables (e.g., voting) which are necessary for answering (causal) questions about the relationship between information exposure and political behavior. Our project synthesizes these two paradigms as they have the potential to compensate for their respective weaknesses when combined in a systematic way. We use a dataset that links web browsing histories from 2,000 German online users to their responses in a survey. That way, we can objectively measure people’s online behavior while at the same time surveying them for sociodemographic variables and political attitudes that are difficult to measure or can – at best – be approximately inferred from digital traces. Our sample is recruited from a large German online access panel that provides web browsing histories for a subset of their panelists. The respondents were incentivized and consented to be tracked online. In April, our project will start collecting data for 12 months. We have several research goals that aim at contributing to ongoing debates on item measurement and political behavior. The few related studies that exist (e.g., Guess, 2015; Scharkow, 2016) have predominantly used such an integrated research design to investigate the validity of self-reported media use, but have not focused specifically on questions about political behavior. For our analytical purposes, we will assign the visited websites to politically relevant categories. We will primarily construct this categorization inductively based on the data, but definitely differentiate legacy media (only their /politics subdomains), online-only media and partisan/other political actors. This allows us to investigate at a more fine-grained level than previous studies: (1) How much time do people devote to political purposes online? (2) How can standard survey items such as political interest or party identification be operationalized through web browsing data? (3) How strong are the correlations between measures derived from web browsing and self-reports? (4) Which variables (e.g., sociodemographics, political knowledge) predict the measured deviations between self-reported and tracked behavior?