TreasureX Researchcs.AIWhy LLM Judges Are Biased — and How to Fix Them from the Inside
Zixiang Xu, Sixian Li, Huaxing Liu, Xiang Wang, Shuai Li, Zirui Song, Xiuying Chen · July 13, 2026
Large language models (LLMs) are increasingly used as automated judges to score text quality, but they suffer from systematic biases. A new paper from researchers at several Chinese universities goes beyond the usual input-output analysis and looks inside the model's hidden states. They find that biased inputs (e.g., favoring longer or more confident answers) push the model's internal activations along specific, low-dimensional directions. By steering these activations in the opposite direction, they can cancel the bias and restore fair scoring. Even more practically, a simple linear projection onto these bias directions can predict when the judge will fail on entirely new benchmarks, outperforming text-based methods. This work suggests that bias is not just noise in the output but a geometric property of the model's internal representations. For investors and tech enthusiasts, this opens the door to more reliable LLM evaluation tools, which could reduce costs and improve trust in AI-driven assessments. The method is lightweight and could be integrated into existing systems without retraining.
Read more: https://arxiv.org/abs/2607.11871
Educational content only, generated with AI assistance from public research abstracts. Not investment advice. Thank you to arXiv for use of its open access interoperability.