Self-Refine Learning in LLM Multi-Agent Systems for Legal Norm Cognition and Compliance

摘要

As large language models (LLMs) increasingly serve as autonomous agents in social simulations, ensuring their ability to understand and comply with legal norms is essential. Yet, current LLM agents frequently exhibit reward hacking (RH) behaviors by optimizing metrics at the expense of norm adherence, undermining simulation fidelity and limiting deployment. We introduce a TBC-TBA self-refine learning multi-agent framework that enables dynamic normative adaptation through iterative multi-agent feedback. This framework integrates Think-Before-Chat (social feedback processing) and Think-Before-Act (norm-guided decision making) phases, allowing agents to progressively refine their normative understanding via structured interaction cycles. Across five mainstream LLMs and 100 legal scenarios, we found that while LLMs partially recognize legal norms, they systematically exhibit RH behaviors with illegal action rates (IAR) of 14.29–37.11%. Comparison with human cognition reveals alignment in moral reasoning but sharp divergence in risk perception and probability distortion. To address these deficits, we introduce four methods to improve LLM's normative compliance. Dynamic Norm Learning Mechanism (DNLM) serves as the core, using a psychologically grounded identify–infer–implement process that reduces IAR by 15.78% on average and delivers the most significant improvement. We also introduce Deep MaxPain (DMP) for consequence based deterrence, Norm Analysis Chain-of-Thought (NA-CoT) for structured reasoning, and Few-shot Norm Learning (FNL) for case based acquisition, all of them enhance compliance. Our findings show that LLM agents can better follow legal norms when equipped with structured self-refine learning and psychologically informed mechanisms. This work improves social alignment in multi-agent systems and opens avenues for future research on scalable, norm-compliant autonomous agents.The code and data are publicly available on GitHub.