What is Really New About Algorithmic Governance? Reflections on Bureaucratic Decision-Making and Computer Science History

"Algorithmic ethics" and "algorithmic governance" have become common expressions, and thriving areas of research. In this presentation, I will focus on the relevance of those notions for algorithmic aids to bureaucratic decision-making. Public debates on this particular topic seem to suffer from two structural biases. The first is a technology-centered bias: it seems to be enough to see that an algorithm is used in a decision to infer that we are governed by an algorithm. This inference is naturally faulty: governments have been using algorithms to compute taxes for millenia without turning algorithms into the subjects of governance. The second is a presumption of novelty: whenever an algorithm is used, it is presupposed that this use raises new ethical and/or political issues. However, when attending conferences and debates on those topics, one may worry that those domains suffer from a serious case of the "old wine in new bottles" syndrom. Many discussions on algorithms for bureaucratic decision-making actually reactivate classic themes of critical examination of bureaucracy (deshumanisation, rigidity of categorization and reasoning), use of statistics (faulty statistical inferences, institutional obsession over reductive indicators and metrics, normative use of the statistical mean) and the philosophy of law (necessity of vagueness in legal definitions and discretionary power to face unpredictable social evolutions, tension between legal formalization and the singularity of the case). It is time to pause and explain, to the general public and to ourselves, what is really new about algorithmic governance. I with use both short and long term perspectives to begin tackling with this huge issue. Bureaucratic decision-making mixes both some of the oldest uses of algorithms in governance, e.g. tax computation, and some of the most recent technological evolutions, such as ML for predictive policies. From the long term perspective, I will give a quick reminder of the very old and deep relations between (proto)-computer science and rationalization of decision-making through proceduralization. From the short-term perspective, we will present some field work made at Etalab, the task force of the French Prime Minister in charge of Open Data & Open Algorithms Policies, in order to see how the recent evolutions of computer technology affect bureaucratic practices, with a particular eye for the introduction of Machine Learning in the French administration. We will endeavor to show the novelty of Machine Learning through the idea of non-procedural programming: ML models are programs but they are not procedures, and they enable generalization of practices closer to scientific modelization than to procedural thinking. Because of this non-procedural turn, ML models reverse the classic relation between algorithms and laws and regulations: ML models generate rules and criteria from data, they do not implement pre-existing laws and regulations, and as such they represent a new problem for the relation between code and law. I will conclude with some suggestions on how those findings should illuminate our understanding of ''algorithmic governance''