Topic
Benchmarks
-
News AI
Audit of 114 self-improving AI studies finds most claims untested across independent reruns
Researchers evaluated 114 papers on recursive self-improvement, finding that 78% never test whether improvements hold across multiple reruns of the full loop.
-
News AI
LLM weight pruning degrades fine-grained skills that survive quantization in 35-model audit
Researchers evaluated 35 compressed checkpoints across six open-weight language models, finding that pruning causes capability collapse on complex tasks.
-
News AI
Intermediate execution errors decouple from final task completion in deep research agents
A study introduced the first systematic error taxonomy for autonomous deep search and research agents, revealing that intermediate execution failures decouple from final task completion metrics.
-
News AI
Formatting shortcuts distort AI agent safety benchmarks by up to 11 points
A study auditing AI agent safety evaluations showed that superficial cues like tool names and prompt markers distort guardrail false-positive rates by up to 11.46 percentage points without altering underlying safety.