← All projects

Evals & Research

10 projects

shipped today

Reads a new GitHub issue and tells you which component it belongs to, which already-solved issues look like it, and roughly how long it will take to close.

  • TF-IDF
  • BGE + FAISS
  • LightGBM
  • Groq
k8s87.1%vscode89.8%

Warmer

Live
puzzle #68 live today

A word game you play once a day, where your only clue is how close each guess comes in meaning to the secret word.

  • Flutter
  • sentence-transformers
  • Firebase
-0.0030.813
shipped 1mo ago

An AI shopping assistant for Indian weddings that searches eight store catalogues at once and will not invent a price or a size to make the sale.

  • LangGraph
  • FAISS + BM25
  • CLIP
  • Groq
93.8% (n=211)

ShelfSense

Research
shipped 2mo ago

Forecasts retail demand a week out across all 30,490 Walmart product and store lines, so a buyer knows what to order.

  • LightGBM
  • Dagster
  • M5 dataset
0.89560.5693
shipped 2mo ago

Turns a prompt or a photo into a Japanese woodblock print, on a LoRA I fine-tuned to hold that style.

  • SDXL LoRA
  • ControlNet
  • Cloud Run

6.2GB peak VRAMFull SDXL + LoRA pipeline, 8GB consumer-GPU budget

shipped 5d ago

Answers whether rewriting an AI agent's tool descriptions actually made it better at its job, with an A/B harness for MCP servers. The research behind my tool-description paper.

pip install agentgauge-harnessv0.5.3 · 5 releases≈49 downloads/week (pypistats, excl. mirrors) Sum of the 7 most recently completed calendar days pypistats.org reports with known PyPI mirrors excluded, UTC, today's partial day not counted.
  • MCP
  • Causal measurement
  • Python

-13.3ppBLOCKING description defect, task-success effect (qwen2.5:7b)

shipped today

A PWA that tracks India's 22K gold rate and ships an honest naive flat-hold range, not a forecast, because the ML model I trained never beat that baseline.

  • Chronos-Bolt
  • GitHub Actions
  • PWA

Naive wins: ships the honest baseline, not the modelNaive baseline vs. ML model (209-fold backtest)

shipped today

Scores a Claude Code session on how efficient it was, and shows you where the tokens went to waste.

pip install tracegaugev0.16.0 · 22 releases≈193 downloads/week (pypistats, excl. mirrors) Sum of the 7 most recently completed calendar days pypistats.org reports with known PyPI mirrors excluded, UTC, today's partial day not counted.
  • Python
  • PyPI
shipped 5d ago

Fails your build when an agent starts costing more per run, and tells you, every run, the smallest rise it could actually have caught.

pip install adk-tracegaugev0.10.1 · 21 releases≈148 downloads/week (pypistats, excl. mirrors) Sum of the 7 most recently completed calendar days pypistats.org reports with known PyPI mirrors excluded, UTC, today's partial day not counted.
  • Google ADK
  • Bootstrap CI
  • PyPI
shipped 27d ago

A held-out benchmark for a class of silent-verdict-degradation bugs, plus 3 detection baselines (AST, local LLM consensus, frontier judge) that all failed a pre-registered viability gate.

  • AST
  • Ollama
  • Frontier judge

← Back to all projects