Skip to content

Fix IDA hearing notice defects: participation, category modules, subsidy - #200

Merged
jimdc merged 7 commits into
mainfrom
owner-reported-defect-cluster-on-an-ida-public-h
Jul 30, 2026
Merged

Fix IDA hearing notice defects: participation, category modules, subsidy#200
jimdc merged 7 commits into
mainfrom
owner-reported-defect-cluster-on-an-ida-public-h

Conversation

@jimdc

@jimdc jimdc commented Jul 30, 2026

Copy link
Copy Markdown
Collaborator

Summary

User feedback on an IDA public-hearing notice (#notice/20250227021, from the homepage Past IDA meetings demo) reported four related defects, plus an age-aware gap principle: demo examples must backtest on old notices, and gap copy must distinguish too soon vs not published vs unavailable.

1. Duplicate participation buttons → one specific link

The notice body publishes the same EDC board URL twice, once with a trailing comma. Participation extraction now strips trailing punctuation before dedupe, keeps one outbound affordance, and labels the generic NYCIDA board page as IDA meetings page (the deepest public target this source publishes for that hearing).

2. List vs detail coherence

Meetings list cards and the notice permalink share one derivation: normalizeHearingRowparticipationLinksHTML. The permalink now shows the same IDA meetings page action people need when sharing “how do I attend.”

3. Wrong-universe contract modules on hearings

Contract lifecycle, OCP award gaps, and PIN gap copy only render for procurement notices (isContractLifecycleEligible). Hearings, rules, property disposition, and staffing no longer stack Checkbook-style “not found” cards.

4. Subsidy “Could not reach” root cause

Root cause (evidence): GET https://edc.nyc/about-nycedc/financial-public-documents-recordings returns HTTP 403 Cloudflare challenge HTML to edge fetch, so fetchSubsidyProjects failed and the worker painted source_status: unavailable. Unavailable rows were also permanently written to D1, so recovery never happened.

Fix: treat challenge/empty feed as feed failure without permanent cache; when the feed is down, derive the hearing stage from the City Record IDA notice itself (projectFromIdaNotice). Keep the operational “Could not reach” copy only when the feed is down and no notice-derived hearing applies.

5. Age-aware gap copy + aged demos (backtest)

Three honest states for subsidy gaps: too_soon (typical lag not elapsed — check back), not_published (lag elapsed — class-b), unavailable (fetch failure only). Lag table: board ~60d, project_record ~90d, closing ~180d after hearing.

Aged demos that join the City Record hearing stage (backtest of the construct):

Notice Hearing Why it demos well
#notice/20220525018 2022-06-09 Multi-year aged; hearing join + later stages not_published
#notice/20231004016 2023-11-02 Aged 2023 IDA hearing
#notice/20240617012 2024-07-18 Aged 2024 IDA hearing

Honest limit: a full Build NYC document join (board packages, closing PDFs) is not available while the EDC documents page is Cloudflare-blocked to edge fetch. Aged notices prove the hearing-stage construct and age-aware later-stage copy; they do not invent board/closing rows without a public machine feed.

Young field case #notice/20260617040 (2026-07-16) correctly uses too_soon for later stages rather than “could not reach.”

Before / after

Screenshots under docs/screenshots/ida-notice-defects/ (production before / local after).

Surface Before After
Meetings list duplicate Participation link one IDA meetings page
Notice modules Contract lifecycle + OCP + PIN + could-not-reach no contract modules; age-aware subsidy
Notice actions no participation IDA meetings page

Test plan

  • node --test test/ida_notice_defects.test.mjs (participation, category gate, age-aware, aged backtest)
  • node --test test/lifecycle_coherence_field_cases.test.mjs test/gap_taxonomy.test.mjs test/subsidy_lifecycle.test.mjs
  • python3 test/standards/i18n_keys.py
  • Post-deploy: aged demos above; young hearings show check-back not permanent gap; no contract lifecycle on hearings

Notes for review

  • Generic EDC board URL is intentional when that is the only published participation target; labeled IDA meetings page.
  • Worker + static site both need deploy for end-to-end subsidy recovery on production.

jimdc added 2 commits July 30, 2026 16:37
User feedback on notice 20250227021 (Past IDA meetings demo) reported four
related problems on public-hearing notices.

Participation: strip trailing punctuation before URL dedupe so the same
EDC board link is not rendered twice; label the generic NYCIDA board page
honestly; share one affordance on meetings list and notice detail.

Contract modules: gate Checkbook/OCP/PIN lifecycle to procurement notices
only so hearings no longer stack wrong-universe gap cards.

Subsidy: root-cause the Build NYC feed as Cloudflare-blocked at the edge;
stop permanently caching unavailable; derive the hearing stage from the
City Record IDA notice when the feed is down.

Characterization: test/ida_notice_defects.test.mjs and updated coherence
tests. Screenshots under docs/screenshots/ida-notice-defects/.
Resolve AGENTS.md and lifecycle coherence test conflicts with the
contract-identity / paid-to-date coherence work so the PR can re-run CI
against current main.
@jimdc
jimdc enabled auto-merge July 30, 2026 20:41
jimdc added 5 commits July 30, 2026 16:49
Keeps i18n key parity after introducing the honest IDA meetings page
label for hearing participation links.
Distinguish too-soon vs not-published vs unavailable on subsidy stages
(temporal sibling of paid/verified-zero/unavailable). Young hearings use
check-back copy; aged 2022–2024 hearings get City Record hearing joins
with later stages as not-published. Demo ids: 20220525018, 20231004016,
20240617012. Full Build NYC document join remains blocked by Cloudflare
on the EDC feed — report that honestly rather than claiming full project
depth without a machine-readable source.
The notice-detail select string grew to include description and printout
columns for participation extraction; update the SODA field-list allowlist
entry so the gate tracks the new literal without stale leftovers.
NYC Web Content Style Guide gate rejects semicolons in user-facing
English strings. Split the too-soon check-back lines into short
sentences instead.
Include the new subsidy gap helpers in both sandboxes and retarget the
unmatched fixture to a non-IDA notice so City Record hearing derivation
does not auto-match. Add a young IDA hearing case that joins without
unavailable copy.
@jimdc
jimdc added this pull request to the merge queue Jul 30, 2026
Merged via the queue into main with commit 8b42e9a Jul 30, 2026
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant