Evaluating the effectiveness of open-source LLMs for automated analysis of multilingual consultation feedback: a Swiss case study

Consultation procedures are a common practice in public policy-making, bringing many benefits to democratic systems. However, they generate a large quantity of opinions, more or less structured, whose processing and synthesis require a significant investment in time and resources on the part of administrations. To meet this challenge, this study proposes the use of open source Large Language Model (LLM) to analyze and synthesize stakeholder opinions in a multilingual context, using the case of Switzerland. The architecture presented was designed with respect for the fundamental principles underlying citizens' trust in democratic institutions. The results show that the proposed approach achieves a satisfactory level of performance in most of the cases studied, with only one obvious error identified. However, the model regularly lacked nuance or omitted certain elements during analysis, which limits its ability to fully replace human expertise. Thus, far from aiming to replace the work of civil servants, this architecture should be seen as a complementary tool, designed to support human analysis and reinforce the effectiveness of consultation procedures, with a view to the continuous improvement of democratic processes.