[jehrr]

Problem

The scheduled canary has been red since 2026-10-09 on one assertion: no event carries an estimate. Every --mode earnings row on gl=US came back with revenue_estimate, eps_estimate, currency and period_end null, while the run reported status: complete and exit 0.

Cause: Google now sends each event's figure record in its sparse array encoding (protobuf-as-JSON: position i is field i+1, and far-apart high fields go into one trailing object keyed by field number):

form	example (real)	where
dense	[null ×8, 10094598350, null, 1.8334, …, "EUR", [2026,9,30]] (18)	gl=DE, 2026-10-15
mixed	[…18 slots…, {"108": 12.93, "109": 10.78}] (19)	gl=US, event already reported
all-sparse	[{"9": 1460392943930, "11": 4.52, "17": "TWD", "18": [2026,9,30], "109": 4.51}]	gl=US, event not yet reported

The parser only accepted the dense 18/106 widths. Checking further found a second silent loss with the same root: BAC:NYSE publishes its detailed quarters 105 slots wide, so --mode financials dropped all eight and its newest row was 2024 Q2.

A canary dispatch on this branch at 12:51 UTC on 2026-10-15 then found a third: --mode movers returned 0 rows (exit 4) because Google sends a list it has no data for yet as null — [null, null, [4 most-active]] before the US open, [null, [4 losers], [4 most-active]] on gl=DE early in its session — and the parser required three populated lists (the most-active rows were also leaking into --mode markets).

Fix

• _jspb_dense expands the sparse form ("9" → slot 8 = revenue estimate, "17"/"18" → slots 16/17 = currency/period end), so the existing slot map applies unchanged.
• _mover_sublists accepts a null list; the gainers/losers label check runs on whichever is present, and falls back to movers_N when neither is. Fixtures movers_us_premarket (the canary's own dump) and movers_de_partial (live gl=DE).
• _fin_figures recognises a figure record by structure (currency string at slot 16, date at slot 17) as well as by the two verified widths; detailed now means "carries every mapped slot" (≥ 39) instead of exactly 106.

Evidence

• 2026-10-14 canary artefact (canary_earnings.html): 5/5 events now have both estimates, currency and period end (ASML EUR 11,716,143,870 / 10.76; TSM TWD 1,460,392,943,930 / 4.52).
• Free live fetches 2026-10-15 with a Chrome User-Agent: gl=US (all-sparse, 5/5 read), gl=DE (dense, 5/5 read, unchanged), BAC/TSM/ASML quote pages. BAC 2026 Q2 and Q1 revenue (30.19B, 28.94B) and operating expense (18.63B, 18.53B) match the page's own rendered financials table.
• New fixtures earnings_us_sparse, earnings_de_dense, financials_bank, cut by make_fixtures.py (trimmed fixture verified to parse identically to the capture; no secret-shaped strings). Existing 22 fixtures byte-identical.
• With main's product_parser.py the new checks fail (11 value checks red, then the decoder check errors); with this branch the suite is green.
• Inference, stated as such: the field-number mapping for the all-sparse object is read from the encoding and agrees with the dense form served on gl=DE the same day; the estimates for unreported events are not in the curl-rendered HTML (the page fills them client-side), so they were not compared against rendered text.

Not changed

• The canary's workflow: it is fixed by this parser change; the canary-policy changes are in Atomic writes, CSV formula guard, one captcha allowance per page, Google Finance's own help text #5.
• make_fixtures.py now keeps already-generated fixtures whose capture is not on disk (it used to drop them all), so adding a fixture no longer needs every original capture.

🤖 Generated with Claude Code