Fix IDA hearing notice defects: participation, category modules, subsidy - #200
Merged
Merged
Conversation
User feedback on notice 20250227021 (Past IDA meetings demo) reported four related problems on public-hearing notices. Participation: strip trailing punctuation before URL dedupe so the same EDC board link is not rendered twice; label the generic NYCIDA board page honestly; share one affordance on meetings list and notice detail. Contract modules: gate Checkbook/OCP/PIN lifecycle to procurement notices only so hearings no longer stack wrong-universe gap cards. Subsidy: root-cause the Build NYC feed as Cloudflare-blocked at the edge; stop permanently caching unavailable; derive the hearing stage from the City Record IDA notice when the feed is down. Characterization: test/ida_notice_defects.test.mjs and updated coherence tests. Screenshots under docs/screenshots/ida-notice-defects/.
Resolve AGENTS.md and lifecycle coherence test conflicts with the contract-identity / paid-to-date coherence work so the PR can re-run CI against current main.
jimdc
enabled auto-merge
July 30, 2026 20:41
Keeps i18n key parity after introducing the honest IDA meetings page label for hearing participation links.
Distinguish too-soon vs not-published vs unavailable on subsidy stages (temporal sibling of paid/verified-zero/unavailable). Young hearings use check-back copy; aged 2022–2024 hearings get City Record hearing joins with later stages as not-published. Demo ids: 20220525018, 20231004016, 20240617012. Full Build NYC document join remains blocked by Cloudflare on the EDC feed — report that honestly rather than claiming full project depth without a machine-readable source.
The notice-detail select string grew to include description and printout columns for participation extraction; update the SODA field-list allowlist entry so the gate tracks the new literal without stale leftovers.
NYC Web Content Style Guide gate rejects semicolons in user-facing English strings. Split the too-soon check-back lines into short sentences instead.
Include the new subsidy gap helpers in both sandboxes and retarget the unmatched fixture to a non-IDA notice so City Record hearing derivation does not auto-match. Add a young IDA hearing case that joins without unavailable copy.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
User feedback on an IDA public-hearing notice (
#notice/20250227021, from the homepage Past IDA meetings demo) reported four related defects, plus an age-aware gap principle: demo examples must backtest on old notices, and gap copy must distinguish too soon vs not published vs unavailable.1. Duplicate participation buttons → one specific link
The notice body publishes the same EDC board URL twice, once with a trailing comma. Participation extraction now strips trailing punctuation before dedupe, keeps one outbound affordance, and labels the generic NYCIDA board page as IDA meetings page (the deepest public target this source publishes for that hearing).
2. List vs detail coherence
Meetings list cards and the notice permalink share one derivation:
normalizeHearingRow→participationLinksHTML. The permalink now shows the same IDA meetings page action people need when sharing “how do I attend.”3. Wrong-universe contract modules on hearings
Contract lifecycle, OCP award gaps, and PIN gap copy only render for procurement notices (
isContractLifecycleEligible). Hearings, rules, property disposition, and staffing no longer stack Checkbook-style “not found” cards.4. Subsidy “Could not reach” root cause
Root cause (evidence):
GET https://edc.nyc/about-nycedc/financial-public-documents-recordingsreturns HTTP 403 Cloudflare challenge HTML to edge fetch, sofetchSubsidyProjectsfailed and the worker paintedsource_status: unavailable. Unavailable rows were also permanently written to D1, so recovery never happened.Fix: treat challenge/empty feed as feed failure without permanent cache; when the feed is down, derive the hearing stage from the City Record IDA notice itself (
projectFromIdaNotice). Keep the operational “Could not reach” copy only when the feed is down and no notice-derived hearing applies.5. Age-aware gap copy + aged demos (backtest)
Three honest states for subsidy gaps: too_soon (typical lag not elapsed — check back), not_published (lag elapsed — class-b), unavailable (fetch failure only). Lag table: board ~60d, project_record ~90d, closing ~180d after hearing.
Aged demos that join the City Record hearing stage (backtest of the construct):
#notice/20220525018#notice/20231004016#notice/20240617012Honest limit: a full Build NYC document join (board packages, closing PDFs) is not available while the EDC documents page is Cloudflare-blocked to edge fetch. Aged notices prove the hearing-stage construct and age-aware later-stage copy; they do not invent board/closing rows without a public machine feed.
Young field case
#notice/20260617040(2026-07-16) correctly uses too_soon for later stages rather than “could not reach.”Before / after
Screenshots under
docs/screenshots/ida-notice-defects/(production before / local after).Test plan
node --test test/ida_notice_defects.test.mjs(participation, category gate, age-aware, aged backtest)node --test test/lifecycle_coherence_field_cases.test.mjs test/gap_taxonomy.test.mjs test/subsidy_lifecycle.test.mjspython3 test/standards/i18n_keys.pyNotes for review