FAIRGAME: A Framework for AI Agents Bias Recognition Using Game Theory (Extended abstract)

摘要

Letting AI agents interact in multi-agent settings in- troduces significant complexity in predicting and interpreting their collective behavior, with pro- found implications for trustworthy AI adoption in research and society. We present FAIRGAME (Framework for AI Agents Bias Recognition us- ing Game Theory), an open-source framework that simulates game-theoretic scenarios with LLM- based agents to systematically uncover biases aris- ing from model choice, language, agent personal- ity, and more. Applied to the Prisoner’s Dilemma and Battle of the Sexes across four LLMs and five human languages, FAIRGAME reveals inconsis- tencies across LLM models and consistent devia- tions from game-theoretic predictions; it also quan- tifies LLM-specific behavioral tendencies through a novel scoring system. Our results show that LLMs draw on prior world knowledge beyond payoff ma- trices, and that language and personality signifi- cantly shape strategic outcomes, supporting the use of reproducible and controlled simulation pipelines to predict the interacting behavior of LLM agents.