DealHunter

Flight search sites usually dump a wall of prices sorted one way, and leave you to manually weigh cost against convenience, is the cheaper flight worth a 6am departure and a layover? DealHunter answers that question directly: you type a request like "Delhi to Dubai in June," and instead of forty rows to compare, it hands back a small, explained set of trade-offs.

It's also a cost case study in its own right: the whole agentic search pipeline, language understanding, search, and ranking, runs on free-tier LLM infrastructure, which forces every design decision to account for what happens when a provider's free quota runs out mid-request.

A search starts as a POST request carrying a free-text trip description. A PlannerAgent, an LLM call, reads it and extracts a structured "travel intent" (origin, destination, date range, traveler count, preferences) from what was plain prose.

From there, a deterministic (non-LLM, rule-based) WindowSearcher explores candidate date windows within a hard budget on how many provider calls it's allowed to make, querying flight and hotel adapters, either the live Aviasales API or a synthetic provider used for testing and cost control.

An OptimizerAgent, another LLM call, takes the resulting candidates and computes a 2D Pareto frontier: a small set of options where none dominates the others (one is cheaper, one is more comfortable, and neither is simply worse than the other on both axes). It returns exactly two "archetypes" with a plain-language explanation of the trade-off, streamed to the browser as they're ready. If you ask a follow-up question, a ConversationManagerAgent re-scores the already-cached candidates instead of re-running the whole search.

A planning agent turns language into structure, a deterministic budgeted searcher does the expensive exploring, and a second agent compresses the results into two honest choices.

Cross-cutting: multi-provider LLM routing (Anthropic Haiku, Groq, OpenRouter, NVIDIA NIM, Ollama), an Upstash Redis cache with a 30-minute TTL, and Langfuse/structlog/Prometheus/Sentry observability.

1. A deterministic searcher, not another LLM agent, does the exploring

   The WindowSearcher that walks candidate date windows is plain rule-based code, not an LLM call. That makes it fully testable and lets it enforce a hard budget on provider API calls, a guarantee an LLM agent, which can't be relied on to count precisely, can't give.

2. Exactly two Pareto archetypes, not a ranked list

   Rather than ranking every candidate, the OptimizerAgent computes a Pareto frontier (the set of options where none is strictly better than another on every axis) and reports exactly two: one weighted toward value, one toward experience. That's a mathematical guarantee the two options are genuinely different trade-offs, not near-duplicates, and it's small enough to honestly explain in plain language.

3. LLM-judge evals: one judge, cross-family, median of three

   To score whether the OptimizerAgent's explanations are actually coherent, the eval harness uses a single LLM judge from a different model family than the agent being judged, and takes the median of three judge calls per case, reducing both self-preference bias and single-call noise.

4. A two-hop production fallback chain (Groq → OpenRouter), built after outages, not in anticipation of them

   The live-traffic fallback chain is two hops: Groq's llama-3.3-70b-versatile, falling back to OpenRouter's free Gemma tier if Groq's quota is exhausted. Five provider adapters exist in the codebase in total, Anthropic and NVIDIA NIM for eval and opt-in demo use, Ollama for local dev, but only the Groq→OpenRouter pair carries production requests; NIM was tried as a third production profile and abandoned. The fallback chain itself was added after Groq's free-tier daily quota was repeatedly exhausted in production, not as a theoretical resilience exercise, the more honest order.

Deterministic replay suite, per-run cost accounting, show source for Test suite

Cited in provenance.md

`agentic-travel-booking-system/README.md:115-120` (lint/typecheck/test commands + "Eval harness (planner golden set, VCR replay)")

measured 2026-08-08

Test suite · no committed, current whole-repo test count exists to cite, the live count has moved with no matching committed artifact at the new figure, so this page states what's actually verifiable instead of a number

31/31 (100%), show source for Planner baseline, correct archetype selection

Cited in provenance.md

`CURRENT_STATE.md:377-385`

verified against source 2026-10-10

Planner baseline, correct archetype selection

demo-haiku: 24/24, coherence 5.0/5 · demo-llama: 21/24, coherence 4.881, show source for Optimizer baseline, completion and judged coherence

Cited in provenance.md

`apps/api/docs/evals/baselines/README.md:19-27`

verified against source 2026-10-10

Optimizer baseline, completion and judged coherence · the llama run's lower completion count is free-tier quota exhaustion, not a quality failure

6/10, show source for Early self-audit score

Cited in provenance.md

`AUDIT_REPORT.md:9-15`

verified against source 2026-10-10

Early self-audit score · prototype-to-production readiness, the honest starting point the project then spent several phases closing

The Cloud Run backend was frozen at a stale image tag, missing three phases of work that had merged since. That alone would have been visible eventually, but it was compounded by a second, independent bug on the Vercel frontend: some required environment variables were set to an empty string rather than left unset.

The frontend's fallback logic used the nullish-coalescing operator, written as two question marks, which only kicks in for null or undefined, not for an empty string, which JavaScript treats as a perfectly valid, present value. So the app quietly sent requests with blank fields instead of falling back to a default or failing loudly, and every real search since launch had failed silently before it ever reached the backend.

The fix was process, not just code: a two-gate canary/soak deploy sequence (a new deploy first serves a small slice of traffic and has to stay healthy before it gets the rest) plus a staleness cron job that alerts if the running image tag falls behind main. The lesson generalizes past this one bug, a fallback operator is not a substitute for validating that a value is actually meaningful, not just technically present.

If you're running an agentic pipeline on free-tier or multi-provider LLM infrastructure, this is the failure mode worth guarding against: two independent bugs can silently compound, and a fallback operator is not the same as validating that a value is actually meaningful, a canary/soak deploy and a staleness check catch what a single build-passing check won't.