AI Governance and (Dis)Trust

Trust has become a pervasive desideratum in AI debates Trustworthy AI debates and studies on citizen attitudes to AI are awash with recipes about what “levers” to pull to enhance user / citizen trust in the technology, often detached from considerations as to the actual trustworthiness of underlying systems. This direction seems to stem from a specific perspective on the topic that is, by and large, concerned with “user acceptability” and “usage” of the technology – and how to optimise or even, maximise on this. More critical normative considerations and implications, especially relevant in a context where the technology is making significant inroads into public governance, seem to have fallen by the wayside. This is not without serious downsides. Attempting to “maximize” citizen trust and perceived legitimacy in AI decision-making, irrespective of actual system performance, raises the prospect of “manufactured trust”. In a context where AI is increasingly adopted across public sectors and consequential public sector domains, and where failures pertaining to the reliance on suboptimal AI technologies in this context abound, this is unsatisfactory and normatively disconcerting. On the other hand, distrust is almost always decried. However, while unconditional distrust towards AI applications in the public sector may indeed undermine the legitimacy and effectiveness of public governance, a certain measure of vigilance or “healthy skepticism” in AI systems is crucial to ensuring fairness, transparency, accountability in their development, adoption, and use. This paper reviews and synthesize recent research about these topics, discusses the main issues at stake, and proposes a new research agenda.