Research

Working papers on agent tool-use evaluation, written alongside AgentGauge, the harness behind them.

Tool-Description Quality Is Not One Axis: A Regime Analysis of Where It Helps and Where It Backfires

Working paper (draft, not yet submitted)

Repo ↗Read the benchmark's case study →

Tool-description quality is widely treated as a broadly-applicable lever for agent tool-use, but it is not a single better/worse axis: the precision that helps an agent disambiguate within a family of confusable tools is orthogonal to, or actively harmful for, context-rich selection and for tool retrieval.

Powering Agent Evaluations: Variance Structure, Measurement Artifacts, and Minimum Detectable Effects in Tool-Use Benchmarks

Working paper (draft, not yet submitted)

Repo ↗Read the benchmark's case study →

Agent evaluations are routinely reported without a power analysis, without a stated detection floor, and without screening for measurement artifacts.

← Back to home