Abstract

This study aims to assess the political orientation of ChatGPT 3.5. It focuses on its performance in Italian compared to English, its primary training language. Employing a systematic approach, we used the Political Compass test to evaluate political stances along social and economic dimensions and determine the language models’ inherent biases. The methodology involved consistent prompts to ensure reliability, including iterations biased towards a left or right-wing position. Results show that the Italian model tends to lean more towards the left-libertarian quadrant than the English model, which also exhibited a left-libertarian orientation but with less extremity. Furthermore, the Italian model showed less susceptibility to extreme shifts when influenced by biased prompts, suggesting a more stable political orientation than the English version. These findings highlight the necessity for ongoing monitoring and development of protocols to evaluate the political biases of AI language models, particularly in less-represented languages. This research underscores the potential impact of AI-generated content on political communication, calling for increased control and regulation to mitigate bias and ensure fair AI applications in societal contexts. Further studies are recommended to validate these results and explore the dynamics of AI biases across diverse linguistic and temporal contexts.