Enhancing Realism in Survey Experiments: Comparing Role-Playing Video Games and Text-Vignettes to Deliver Narrative Treatments

Despite their popularity, video games remain an underutilized tool in social science research. This applies particularly to role-playing video games (RPGs). RPGs are interactive simulations of real-world phenomena in a virtual environment. They could enable researchers to create strong experimental manipulations, sustain participants’ emotional engagement, and mitigate social desirability bias. Yet, we know little about how participants’ answering behavior in these virtual world simulations compare to more established data collection approaches. This case study fills that gap by evaluating the use of custom-designed, small-scale RPGs against text-based vignette designs. I use a 2x2 factorial design with a two-step random assignment of participants first to the treatment or control group and second to the interactive or text-based narrative version. Treated participants make a series of socio-economic deprivation experiences, simulating individual effects of long-term joblessness. Participants in the control group experience similar situations without deprivation. By integrating a browser-based RPG directly into an online survey, I compare participants’ answering and survey-taking behavior between the interactive simulation and a descriptive text-vignette setup. I use a diverse online sample (N = 400) to evaluate differences in treatment effects, cognitive load, emotional engagement, immersion, and accessibility. By demonstrating how researchers can design and deploy these virtual world simulations with little effort and by contrasting the results with those from text-based vignettes, this study validates a novel tool for increasing ecological validity in experimental surveys. The findings have broad implications for the use of custom small-scale RPGs to understand behavioral responses to complex socio-economic stimuli.