Blog
AI technology blog, API tutorials and industry updates
Tag: LLM EvaluationClear
The First Token Fallacy: Why Logit-Based LLM Judges Distort Position Bias
A new study reveals that reading LLM judge verdicts from the first token's logits can overstate position bias by 42 points, leading to misleading audits of AI model quality.
The Fragility of AI: How Prompts, Formats, and Elicitation Instruments Dictate LLM Performance
Recent AI research reveals a striking truth: LLM performance, memory retrieval, and stated preferences depend heavily on prompt formatting and evaluation instruments rather than stable internal capabilities.