Gold Rate Tracker

Indian retail gold buyers want to know today's Tanishq 22K jewelry price and whether now is a good time to buy, without paying for a subscription service, and without trusting a forecast that's really just marketing dressed up as intelligence.

The project's real differentiator isn't a clever model. It's a willingness to refuse shipping a price-prediction feature that doesn't actually beat a trivially simple baseline, even though "AI predicts gold prices" is a far more exciting headline than "tomorrow will probably look like today."

A GitHub Actions cron job fetches the IBJA (India Bullion and Jewellers Association) benchmark rate every three hours, the national reference price, and the primary source since Tanishq's retail site started blocking automated access behind Cloudflare's bot protection. A plain-HTTP Tanishq scrape still runs opportunistically alongside it, falling back to a full browser (Playwright), and confirms the IBJA-calibrated estimate against real retail pricing whenever it succeeds; when it doesn't, the IBJA estimate ships on its own. A HuberRegressor (a regression method that's resistant to occasional bad data points) converts the IBJA benchmark into a retail-price estimate, calibrated against historical Tanishq readings.

The headline forecast shown to users is deliberately a naive flat-hold: tomorrow's price is predicted to be whatever today's price is. That's the model that ships, because measured honestly, nothing more sophisticated has beaten it yet.

A small time-series model, Chronos-Bolt-Tiny, still runs every cycle as a "directional companion", but its output is suppressed from the user interface (internally labeled "DARK") because it fails a pre-registered statistical bar for being trustworthy enough to show.

Everything renders as an installable static Progressive Web App on GitHub Pages, reading the committed JSON files directly with no backend server, and price-drop alerts go out over ntfy.sh. The entire stack runs for ₹0 a month.

A scraper feeds a naive baseline that ships, and a small ML model that runs in parallel but stays dark unless it earns its place.

Total infrastructure cost: ₹0/month, running entirely on GitHub Actions, GitHub Pages, and ntfy.sh free tiers.

1. Ship the naive flat-hold as the headline, not the ML model

   The original 165-fold walk-forward backtest (2026-05-19, a test that repeatedly trains on the past and checks the very next prediction, sliding forward through time) showed Chronos-Bolt-Tiny performing 10.4% worse than the naive flat-hold on mean absolute error, with the difference statistically significant (p=0.0089), the decision that shipped naive as the headline. Every larger backtest run since (see Results below) has confirmed the same conclusion, not reversed it.

2. Corrected the direction-signal baseline from a coin flip to the true base rate

   The project had been comparing its "predict tomorrow's direction" accuracy against 50%, the textbook baseline for a binary yes/no signal. But gold's price rises roughly 70% of the time in this data, so "always predict up" alone beats 50% for free, the real baseline is 69.7–75.5%, not 50%, and the model has to clear that higher bar to mean anything.

3. Model promotion is pre-registered and falsifiable

   Chronos only graduates from "directional companion" to a shown forecast if a backtest of at least 250 folds beats the naive baseline on mean absolute error with a Wilcoxon signed-rank test p-value under 0.05, a bar written down in advance, not adjusted after seeing the result.

4. Stop iterating and wait, because the sample size can't currently detect a real edge

   A Monte Carlo power analysis found that at the current sample size (93 folds), only an implausibly large ~21 percentage-point accuracy edge would even be detectable at standard statistical power. Rather than keep tuning a model that can't be evaluated yet, the project computed the sample size needed and set a revisit date instead of guessing.

249.24 vs. 292.14, show source for Naive flat-hold vs. Chronos-Bolt-Tiny (209-fold backtest, horizon 5 days)

Verified source

`gold-rate-tracker/data/backtest.json@ad42160` — pinned per this file's "Pinned refs" note above: `data/backtest.json` is bot-refreshed continuously (`weekly-backtest.yml`, 243 folds as of 2026-09-23) so an unpinned number goes stale within days. Repinned wave refresh-2026-09 (F3/F4, 2026-09-23) to `ad4216086d10a63bb93ed3107c9ccee429cb5fa0` — the exact commit the product repo's own README now cites as its frozen headline citation, so every surface (this row, `content/metrics.json`, the case study) agrees with the same artifact instead of three different snapshots. Commit `ad42160` (`backtest_run_at` 2026-08-02T05:18:28Z): `mae_5d_avg_naive: 249.24` vs `mae_5d_avg_chronos: 292.14`, `wilcoxon_signed_rank_p: 0.0`, `dir_acc_5d_chronos: 0.512`

• data/backtest.json:3 ↗

commit ad42160 · measured 2026-08-02

Naive flat-hold vs. Chronos-Bolt-Tiny (209-fold backtest, horizon 5 days) · MAE (₹/g), naive wins by ~17% (Wilcoxon signed-rank p < 0.001, the naive baseline's advantage is real, not noise)

51.2%, show source for Chronos direction accuracy

Verified source

`gold-rate-tracker/data/backtest.json@ad42160` — pinned per this file's "Pinned refs" note above: `data/backtest.json` is bot-refreshed continuously (`weekly-backtest.yml`, 243 folds as of 2026-09-23) so an unpinned number goes stale within days. Repinned wave refresh-2026-09 (F3/F4, 2026-09-23) to `ad4216086d10a63bb93ed3107c9ccee429cb5fa0` — the exact commit the product repo's own README now cites as its frozen headline citation, so every surface (this row, `content/metrics.json`, the case study) agrees with the same artifact instead of three different snapshots. Commit `ad42160` (`backtest_run_at` 2026-08-02T05:18:28Z): `mae_5d_avg_naive: 249.24` vs `mae_5d_avg_chronos: 292.14`, `wilcoxon_signed_rank_p: 0.0`, `dir_acc_5d_chronos: 0.512`

• data/backtest.json:3 ↗

commit ad42160 · measured 2026-08-02

Chronos direction accuracy · barely above a coin flip, and below the true ~70% regime base rate

Held DARK at both horizons tested, show source for Direction-signal promotion gate

Cited in provenance.md

`docs/adr/019-direction-signal-below-base-rate.md:16-35,62-72`, `docs/DIRECTION_SIGNAL_STATUS.md:15-20@fa9dfb6` — pinned per this file's "Pinned refs" note above: `DIRECTION_SIGNAL_STATUS.md` is bot-refreshed weekly (`eval-direction.yml`) and had already moved past the results-row's previously-cited n=93/92 figures (a stale 8-week-old snapshot, per a data-accumulation bug that repair commit `fa9dfb6` fixed) by the time this was checked. Commit `fa9dfb6` (2026-08-05, "docs: correct DIRECTION_SIGNAL_STATUS.md revisit dates, record the capture-timing bug (#623)"): h=1 N=130, base rate 50.8%, logistic OOS accuracy 48.5% (p=0.63); h=2 N=128, base rate 57.8%, logistic OOS accuracy 60.9% (p=0.45) — both still DARK, no gate change

measured 2026-08-05

Direction-signal promotion gate · h=1: logistic model 48.5% vs. 50.8% base rate (p=0.63, n=130); h=2: 60.9% vs. 57.8% base rate (p=0.45, n=128)

₹0/month, show source for Infrastructure cost

Cited in provenance.md

`README.md:11`

verified against source 2026-09-23

Infrastructure cost · GitHub Actions + GitHub Pages + ntfy.sh, all free tiers

Early on, the project reported directional-accuracy figures of 55.8% and 63.3% as beating "the 50% naive floor", the standard textbook baseline for a binary up/down signal. That comparison looked reasonable, and it made the model look genuinely useful.

A review caught the flaw: 50% is only the right baseline when the series is roughly balanced between up and down moves. Gold's price in this data rose roughly 70% of the time, which means a trivial rule, "always predict up," no model required, already beats 50% by a wide margin. Once that true base rate was used as the comparison instead, the model's advantage didn't just shrink. It disappeared: on every window tested, the always-predict-up rule matched or beat the trained model.

The project wrote this up as an architecture decision record rather than quietly keeping the more flattering 50%-baseline comparison, and turned the directional signal off in the live product. The honest version of the finding is less exciting than "the model beats a coin flip," but it's the one that's actually true, and it's the one that shipped.

If you're evaluating whether an ML feature is actually earning its complexity, this is the bar worth holding yourself to: define the promotion gate and the real baseline before you see the result, and if the naive baseline wins, that's the finding to ship, not the one to quietly bury.