diff --git a/writing/PROGRESS.md b/writing/PROGRESS.md new file mode 100644 index 0000000..a81e30e --- /dev/null +++ b/writing/PROGRESS.md @@ -0,0 +1,151 @@ +# Progress ledger + +Read this first each session. Statuses: `todo` -> `drafted` -> `reviewed` (continuity pass by a +later session) -> `final`. + +- **Session 0 (31 Aug 2026)** drafted chapters 00 to 09 and the closing, the three diagrams and + `figures.md`. +- **Session 1 (4 Sep 2026)** merged `main` and extended over PRs #37 to #45. All four issues + chapter 09 called open had closed within four days of it being written; 09 was kept as the + argument at the time, three chapters added, the closing renumbered 10 to 13. +- **Session 2 (12 Sep 2026)** merged `main` and extended over PRs #46 to #54. **Two of the + series' own settled decisions were reversed that week**, and both are narrated rather than + edited away: chapter 12's account of the golden file is corrected by chapter 14, and chapter + 06's "hand-curation is deliberately out" by chapter 15. Four chapters added; the closing, + which had passed the series' own 3,000-word ceiling, split into 17a and 17b; the new chapters + numbered 13 to 16 so no number is skipped. +- **Session 3 (24 Sep 2026)** merged `main` and extended over PRs #55 to #58. Two chapters added + (17, 18) and the closing renumbered 19a and 19b. Chapter 17 closes chapter 16's two bugs and + corrects its Kishoge diagnosis; chapter 18 corrects two sentences of chapter 01. Forward + pointers added to 01, 10 and 16. **Re-measuring found August's figure had moved after August + ended** (Tullamore, A to D on 19 Sep, from pooled planned-works grace), which no PR mentions; + it is narrated in 17 as a finding, not decided. + +A later session should do the continuity and review pass, and re-check the "quoted at the date +they were measured" rows in `figures.md` against their stated sources. + +| Ch | Title | PRs / issues | Status | Words | +|---|---|---|---|---| +| 00 | The easiest of the three (intro) | - | drafted | 1,736 | +| 01 | A feed that is not about lifts | #1 | drafted | 1,722 | +| 02 | The start date that is 451 days old | #2 | drafted | 2,016 | +| 03 | Three sites, one design layer | #3 to #17 | drafted | 1,232 | +| 04 | A grade with nothing to borrow | #18 | drafted | 2,174 | +| 05 | The grade argued with the bar underneath it | #25, #27 | drafted | 2,210 | +| 06 | The data Ireland does not have | issue #24, #30 | drafted | 2,189 | +| 07 | "and" is a sequence, not a choice | #30 | drafted | 2,327 | +| 08 | The same bug, three times | #30 reviews, #34 | drafted | 1,996 | +| 09 | What one letter cannot say | issues #28, #31, #32, #33 | drafted | 2,448 | +| 10 | Two ways the page lied about time | #39, #42, #44 | drafted | 2,192 | +| 11 | The grade narrows to lifts | #38, #43 | drafted | 1,812 | +| 12 | Both legs, and who was on the stairs | #37, #45 | drafted | 2,741 | +| 13 | What a reader can take away | #49, #50 | drafted | 1,364 | +| 14 | A guard that was guarding the corpus | #51 | drafted | 1,421 | +| 15 | Building the thing that does not exist | #46 | drafted | 2,403 | +| 16 | A fourth site, and two bugs found sideways | #54, issues #52, #53 | drafted | 1,655 | +| 17 | The same thing, said differently | #55, #56 | drafted | 2,085 | +| 18 | The one code a rebuild cannot undo | #57, #58 | drafted | 2,082 | +| 19a | Closing: what the site can and cannot say | - | drafted | 2,355 | +| 19b | Closing: what I would tell someone starting the fourth | - | drafted | 1,791 | + +Total ~42,200 words, 31 concept boxes, three hand-written SVGs and one mermaid flow (ch 01). +Word counts for 00, 01, 10 and 16 include this session's refreshed figures and forward pointers. + +Now longer than both siblings (esb ~24,500 over 12, uisce ~32,600 over 18), which is a fact +about the repository rather than about the writing: it has merged 20 pull requests in the 24 +days since the first draft. The shape still holds. Chapter 03 is still the compressed one, and +**chapters 06 to 17 are 24,700 words, 59% of the series**, all of them on the access problem and +what followed from it. Chapter 18 is the first late chapter that goes back to the collector. + +## Chapter summaries (3 lines each) + +- **00** The question, the family, and the turn. Today's figures with today's date, including + the unknown share going the wrong way. AI process named once (204 commits, 135 co-authored). +- **01** Verbatim before parse, database disposable, `rebuild` replays the live path. No id, no + completion signal, and a failed run structurally unable to close anything. Boxes: source of + truth against derived index; a failed run is not an empty one. Mermaid pipeline. +- **02** The listing is the measure. Rush and Lusk's 451.6 days precede all collection; + Docklands and Hazelhatch are the watched-and-absent cases. The UTC/Dublin bucket bug. Boxes: + measure the window you watched; two clocks for one date. SVG. +- **03** The short one, on purpose. Vendored then pinned, `dependencies` empty for the Pi, the + alignment pass, the 16-hour threshold (later 10, ch 10), the 3.11 floor checked in CI. Box: + an empty dependency list as a deployment contract. +- **04** No Irish or EU target, so the bands are the site's own, counted in days. The grace rule + in its three versions. The clock-skew crash. Boxes: a scale with no anchor; a band calibrated + in the bar's unit. +- **05** Connolly A/100% over two red cells, so escalators count. Blue meant two opposite things. + E at 50% lands in a real gap. Boxes: one colour two meanings; a cut in a real gap. Reversed in + ch 11. +- **06** Every source empty; NeTEx and SIRI-FM unpublished; EU 2017/1926's "provided they exist" + clause makes the absence lawful; the snapshots turn out to be the only versioned record that + exists. Boxes: a National Access Point; a lawful absence. Ch 15 stops working around it. +- **07** The reading. Boilerplate stripped first; Hazelhatch; 29 sequences, 11 "or stairs", 2 + real alternatives, so no connective parser at all. Boxes: the safe direction of an error; an + inference that expires with its source. SVG. +- **08** Three of the second review's findings were the first review's. One shape: a predicate + over the wrong quantity. Found twice more in the collector. OSM carried, measured, removed. + Box: a guard that passes because what it checks is absent. +- **09** The four open questions as reasoning. Box: one number, two populations. Kept as the + argument stood on 31 August; all four closed by 3 September. +- **10** The build stalled, not the collector; and a notice that came back published as never + having left. Boxes: the age on the page is the age of the data; a test that exercises the easy + half. +- **11** #28 turned out to be the blocker for #32: a 15px gutter is what let escalators leave + the grade. Boxes: a conditional column is a misalignment; a rule with no instance, written + down and guarded. +- **12** The kept-platform note, leg detection, the escalator sentence, the overlap guard, the + golden file, and the reliability section. Boxes: reading a claim against the right leg; what + the code's own history says about the code. Its golden-file paragraph is corrected by ch 14. +- **13** What a visitor can take away: the "Lift out" tag, Atom feeds, a CSV, a link to the + source page, plus two tripwires for a silent drop. Boxes: a feed entry for something still + happening; a tripwire for a silent drop. +- **14** A red build from a merge that could not have caused it. The notes said the logs are + append-only and therefore a dropped notice means a bad checkout; `messages` is not append-only, + and a reword overwrites the body in place. Inputs pinned. Box: a guard over an input you do + not control. +- **15** The reversal. Provenance, refresh and audit were three named defects, so they became + three requirements: an append-only observation log, replayed like the collector's, with + page-sourced facts expiring when their quote leaves the page. Boxes: the objection is a + specification; an edge that records ignorance. Ends on a GTFS export of the file ch 06 found + missing. +- **16** A fourth site reads the same logs. The boundary written down, the location-codes trap + (288 of 497, zero lift notices), and two bugs found by pointing a different tool at the same + data. Box: a second reader of the same data is a test you did not write. +- **17** #53 answered as vocabulary, which by ch 04's account of the grace is the purpose answer + too; Kishoge's page puts the name in the code field; Tullamore's August moved after it ended. + Boxes: a literal string is a vocabulary of one; a property of the notice, published as a + property of the month. +- **18** The collector's one whole-file review: the raw line waited on the database, a probe that + passes on a full card, `sort -u` reorders so replay sorts by time. Boxes: the invariant has an + upstream edge; a merge that deduplicates also reorders. +- **19a** The figures, the two lists, the three-way table, the settled decisions. +- **19b** The moral, its three codas, and a 31-entry glossary. + +## Open threads + +- Review pass not yet done: every chapter is `drafted`. +- The three SVGs are functional and unpolished. Neither Session 1 nor Session 2 needed to change + them. Chapter 15 is the first chapter that would clearly benefit from one (the station graph); + it was left out rather than drawn badly against a five-station pilot. +- Cross-references to the sibling series are by chapter number, not URL, so they survive + uisce #43 and esb #30 merging or renumbering. Check them if either lands. +- **The pattern across three sessions is that this series' own "settled" sections keep being + reversed**, which is the repository working as intended and is now the series' most + distinctive feature. The habit to keep: date the chapter, leave it standing, and write the + correction as a later chapter. Do not edit a chapter to match today. +- **The most perishable things now.** Chapter 15's graph is a schema, a derivation and a + five-station pilot that the site does not read; the day it does, that chapter needs a + successor rather than an edit. Chapter 16's two issues are open, and #53 in particular turns + on what chapter 04's grace is *for*, so whatever is decided belongs beside that argument. + Chapter 12's entrance leg is still machinery with no live case. +- **The unknown verdict share is the number to watch**: 6 of 24, 7 of 30, 14 of 45, now 23 of 58. If + it keeps climbing, chapters 07 and 12's account of the prose derivation needs revisiting, and + it is the strongest argument in the series for chapter 15's survey. +- A root `README.md` pointer to `writing/` is deliberately left for the publish decision, as + both sibling series did. +- **Tullamore's August is a live question the series raised rather than reported.** If an issue + or a note decides it (provisional months, frozen months, or grace spent in time order), chapter + 17 needs a successor, and 19a's "cannot" list changes. As of 24 Sep 2026 no issue exists. +- The Kishoge fixup has no guard; chapter 17 says so. If a tripwire lands, 17 needs a pointer. +- The repository has merged 20 pull requests in 24 days. Check `git log origin/main` before + assuming this account is current; anything after #58 needs a new chapter or an extension. diff --git a/writing/README.md b/writing/README.md new file mode 100644 index 0000000..c2cb548 --- /dev/null +++ b/writing/README.md @@ -0,0 +1,185 @@ +# The lifts series - brief and style guide + +A chapter-by-chapter account of how this repository went from a script polling Irish Rail's +service-message feed on a Raspberry Pi to a status site that says what a lift outage did to +step-free access at each station. It is the third of three: the water site's series +([uisce PR #43](https://github.com/baz8080/uisce/pull/43)) and the power site's +([esb PR #30](https://github.com/baz8080/esb/pull/30)) came first, and this one is written +against both. + +Nothing in `writing/` is imported by the package. It is prose and diagrams only. + +## What this series is about + +The other two are stories about measurement: how to count people affected, how to grade a +utility against a promise. This one starts as the easiest of the three and ends somewhere +else entirely. + +One endpoint, one flat list of messages, no geography, no Census, no regulator. And then the +question the site exists to answer turns out to need a fact nobody in Ireland publishes: +**what does this station have?** A lift out at a station with a ramp to the other platform is +not the same event as a lift out at a station where the lift is the only way up, and there is +no machine-readable source in the country that can tell the two apart. So the back half of +this series is about the shape of an absence: what was looked for, why it is lawfully missing, +what was used instead, and how reading one hand-typed sentence the wrong way published the +opposite of the truth. + +The chapters are deliberately back-loaded. The collector and the site get one each, the shared +design layer gets one short one, four carry the problem that arrived at the end, three cover the +four days in early September when every question the fourth of those left open was answered, and +four more cover the week after that, in which the project stopped working around the missing +data and started recording it. + +## Who it is for + +The same reader as the other two series: an intelligent professional who is not a programmer. +They can follow arithmetic when it is shown and a table when it has real station names in it. +They will not tolerate a term used before it is explained, and they will notice a number that +appears without a sentence saying what it means. The chapters assume the reader *may* have +read the other two but must not require it: every comparison states the other site's approach +in a sentence before contrasting it. + +## Voice + +- First person, "I". The AI-assisted process is named once, in the intro, and not + re-litigated chapter by chapter. +- Candid. The wrong turns are the story: the reading of "and" that would have told a + wheelchair user access was fine at a station where it was gone; the grade that gave a + station an A over two red cells; the rebuild that emptied the database and reported + success. Tell what was believed, what was measured, what changed. +- Chronological within a chapter. +- Plain. Prefer "the log" to "append-only JSONL", "a notice" to "a message record". Introduce + a technical term once, in a concept box, then use it freely. +- **Careful about access.** This site makes claims about whether a disabled passenger could + get to a platform. The prose never says "the disabled" and never uses a wheelchair as a + synonym for disability: an escalator outage matters to somebody with a heart condition, a + pram or a suitcase, and the series says so where the site does not yet. + +## Rules + +The other two series' rules, restated so this file stands alone. + +1. **Every number carries a source and a date.** In text: "(PR #18, 28 Aug 2026)" or + "(measured 31 Aug 2026)". Every number quoted also gets a row in `figures.md`. +2. **No figure without a sentence saying what it means.** +3. **One concept box per hard idea**, at the point the idea first matters, <= 200 words, in a + blockquote starting `> **Concept: **`. Where a sibling series already boxed the idea, + restate it in a line and point at theirs rather than re-explaining. +4. **At least one worked example per hard concept**, using a real station and real numbers, + with the arithmetic shown. The running examples are **Hazelhatch and Celbridge** (the + misreading), **Dublin Pearse** (the F driven by an escalator alone) and **Rush and Lusk** + (the notice dated 451 days before anyone saw it). +5. **Diagrams earn their place.** A mermaid fence for a flow; small hand-written SVG in + `diagrams/` for anything spatial or temporal. Under 40 lines, no polish. +6. **Length: target ~1,500 to 2,000 words, hard ceiling 3,000.** Each post carries a + "~N min read" line (about 230 words a minute). +7. **Standalone.** Each chapter opens with a two-line *Where we are* so it works as a single + blog post. +8. **Vocabulary is fixed** (below); do not drift between synonyms. +9. **Missing number becomes `[verify: what]`** and is collected in the final pass. +10. **No em dashes, and no en dashes either.** This repo's own rule (`CLAUDE.md` § + Punctuation, 29 Aug 2026): the house dash is a spaced hyphen. Unlike the sibling series, + this one is checked rather than trusted, and it is stricter than esb's: `scripts/no-em-dash.sh` + greps tracked files for both characters, so `writing/` is covered the moment it is + committed and a numeric range has to be written "8 to 15". The uisce series is written the + other way, which is a deliberate difference of its own and is noted in the esb series. + +## The series' own mandate + +The esb series carries a rule that every fork from uisce is stated as *(their approach, ours, +and the property of the data that forced it)*. This one is three-way, because on the questions +that matter all three sites landed in different places, and none of the splits is taste. + +Where the three diverge, say it as **(what uisce does, what esb does, what this site does, and +the fact about this feed that forced it)**. The four that anchor chapters: + +| | uisce | esb | lifts | forced by | +|---|---|---|---|---| +| The operator's start time | publication time, re-stamped, so every duration is a floor | back-dated by hours, immutable, and measured from | back-dated by **months**, shown as their claim, colours nothing | Rush and Lusk is dated 451 days before its first sighting, over days the feed was polled every 30 minutes and the notice was absent | +| How big an event is | people inside a 500 m circle | ESB's own count of customers off | there is no size: a notice is listed or it is not | the feed carries no count of anything | +| What anchors the grade | its own thresholds on person-hours | ESB's published 4-hour / 95% charter aim | its own bands, counted in days | the PRM TSI sets a duty to hold a written policy, not a percentage, and Irish Rail publishes no availability figure | +| What is allowed to knock the grade | `KNOCK_CATS`, binary: health notices knock, discolouration shows and does not | planned works excluded, because the regulator excludes them; storm days kept, and said out loud | planned works excused for one week then counted in full; escalators counted for five days, then stopped | nobody excluded anything on our behalf, so every exclusion had to be argued from the data, twice | +| The second source | Census Small Areas, official and versioned | the same, borrowed from the water site | a hand-typed CMS field, and from 8 September a hand-recorded observation log beside it | no structured source exists at all, so the only way to raise the ceiling was to record facts and carry their provenance | + +That last row is the spine of the back half of the series. + +## Fixed vocabulary + +| Use | Not | Meaning | +|---|---|---| +| **the feed** | the API (except in code contexts) | Irish Rail's realtime service-message endpoint | +| **a notice** | a message, a banner, an alert | one service message as the feed publishes it | +| **an outage** | an incident, an event | one notice's listing, with same-poll reissues folded in | +| **listed** | active, open, live | present in the feed at a given poll | +| **no longer listed** | fixed, resolved, repaired | absent from a later successful poll. The site never says "fixed" | +| **a run** | a poll (as a noun), a pass | one scheduled collection attempt, every 30 minutes | +| **the log** | the archive, the JSONL | the raw append-only files; the source of truth | +| **the horizon** | last update, cutoff | the last moment a run actually reached the feed | +| **planned works** | maintenance, scheduled | a notice whose text says "planned works", "planned maintenance" or "engineering works" (the last two since 18 Sep 2026). Quote the notice's own word where it matters | +| **availability** | uptime, score | the share of days watched with nothing reported out at that station | +| **grade** | rating, mark | the A to F letter, station-month only | +| **step-free** | wheelchair-accessible, accessible | a route with no steps on it. The narrower, checkable claim | +| **a way up** | vertical access, circulation | what an escalator provides and a lift also provides. Losing one is not losing step-free access | +| **a leg** | a segment, a stage | street to concourse, or concourse to platform. Irish Rail keeps them in separate fields | +| **a stretch** | a span, a run | one continuous period a notice was on the feed. A notice can have several | +| **the survey** | the curated data, the hand file | the append-only observation log in `lifts-data/survey/`, one file per station | +| **an observation** | an entry, a record | one line of the survey: one fact, with who recorded it, when, from what, and how sure | +| **the graph** | the model, the map | the nodes and edges the survey replays into, and the reachability read off them | +| **the prose** | the description, the blurb | Irish Rail's hand-written `platformAccess` and `ticketOfficeAccess` fields | +| **the water site / the power site** | uisce / esb (except as repo names) | the two siblings | + +## Chapter template + +```markdown +# NN. Title +*~N min read · PRs #a to #b · dates* + +*Where we are:* two lines placing this chapter in the series. + +## The question that opened this stretch + +## What changed +(narrative, chronological within the chapter) + +> **Concept: ** - plain-English box, <= 200 words. + +### Worked example: +(real numbers, arithmetic shown, source and date) + +## What went wrong <- when applicable + +## Where it left the site +(the numbers as of the chapter's last PR) + +## Notes +PRs, commit subjects, `notes/` sections and code functions used; each figure's source. +``` + +## Working method + +Session 0 (31 August 2026) drafted chapters 00 to 09 and the closing in one pass from the +repository's own history: the commit messages, the pull request bodies, the three files in +`notes/`, the README and the open issues. Unlike the esb series it had the corpus to hand, so +the figures were re-measured rather than lifted. + +Session 1 (4 September 2026) merged `main` and extended the series over pull requests #37 to +#45. All four issues chapter 09 described as open had closed within four days of it being +written, so that chapter was reframed as the argument at the time with forward pointers, three +chapters were added, and the closing was renumbered 10 to 13. + +Session 2 (12 September 2026) merged `main` and extended the series over pull requests #46 to +#54. Two of the series' own settled decisions were reversed that week and both are narrated +rather than edited away: chapter 12's account of the golden file is corrected by chapter 14, and +chapter 06's "hand-curation is deliberately out" by chapter 15. Four chapters were added and the +closing, which had outgrown the series' own 3,000-word ceiling, was split into 17a and 17b. +Every current figure was re-measured against `../lifts-data` at its 12 September state. + +Session 3 (24 September 2026) merged `main` and extended the series over pull requests #55 to +#58. Two chapters were added and the closing renumbered 19a and 19b. Chapter 17 closes chapter +16's two bugs and corrects its diagnosis of one of them; chapter 18 corrects two sentences of +chapter 01. Re-measuring found something no pull request mentions: August's national figure moved +from 76% to 75% after August ended, because a planned-works notice at Tullamore came back. It is +narrated in chapter 17 as a finding, not a decision. + +`figures.md` marks which rows come from a measurement and which are quoted at the date they +were first measured. `PROGRESS.md` is the ledger for any later session. diff --git a/writing/chapters/00-the-easiest-of-the-three.md b/writing/chapters/00-the-easiest-of-the-three.md new file mode 100644 index 0000000..d4339ca --- /dev/null +++ b/writing/chapters/00-the-easiest-of-the-three.md @@ -0,0 +1,151 @@ +# 00. The easiest of the three +*~8 min read · the whole series · 8 August to 24 September 2026* + +*Where we are:* the beginning. This post says what the site answers, what it turned out to +cost, and how the twenty-one posts are arranged. + +## The question + +Which Irish Rail stations have lifts out of service, and for how long? + +It is a question you cannot answer today. Irish Rail publishes a live feed of service messages, +and a lift outage appears in it as a banner of English prose: *"The lift at platform 2 is +currently out of service. Iarnród Éireann Irish Rail apologise for the inconvenience caused."* +When the lift is working again the banner is gone. Nothing accumulates. There is no page +listing which stations broke most this year, or how long an outage typically runs, or whether +the same lift keeps failing. + +So this repository writes it down. A Raspberry Pi in a hallway asks the feed what is listed, +every 30 minutes, and appends the answer to a file. As of 24 September 2026 that file holds +2,224 runs over 46 days, from which 69 lift and escalator outages across 44 stations have been +reconstructed, and the site built from it is at +[baz8080.github.io/lifts](https://baz8080.github.io/lifts). It is the third site of a family: +[uisce](https://github.com/baz8080/uisce) does the same for Uisce Éireann's water notices, and +[esb](https://github.com/baz8080/esb) for ESB Networks' power outages. Each has a series like +this one. + +## Why this one is written separately + +I expected this to be the easy one, and for a fortnight it was. + +The water site has to work out how many people a boil-water notice touches, which means Census +Small Areas, a radius, and a long argument about what a pin on a map even means. The power site +has to merge five records into one fault, argue with a regulator's published indices, and +decide what to do about storm days. This site has one endpoint returning one flat list. No +geography. No population. No merging worth the name. The collector was written in a day and +the site in another, and the architecture was lifted almost unchanged from the power site, +which had already lifted it from the water one. + +Then the site tried to say what any of it *meant*, and hit something neither sibling has. + +A lift out at Athy is not the same event as a lift out at Dublin Connolly. At Athy the page +says "Level to platform 1, Lift to platform 2", so when the lift goes, platform 2 stops being +reachable without stairs and platform 1 is fine. At Connolly four platforms are level from the +ticket office, one has a ramp, and two are behind a lift. The outage that matters is not the +same outage, and no number on a status page is honest until it knows the difference. + +To know the difference you need an inventory: what does this station have, and which platform +does each machine serve. Ireland does not publish one. Not in GTFS, which is the format transit +apps read. Not in NaPTAN. Not in the NTA's developer API. Not in NeTEx, the European standard +written for exactly this, which Ireland does not publish at all. The regulation that was +supposed to force it obliges a member state to publish the listed data types *"provided they +exist in digital machine-readable format"*, and that clause is the whole story: the duty is to +publish what you hold, not to create it. + +What does exist is a free-text field on irishrail.ie, typed by hand, with no schema and no +obligation to be correct. It is the only machine-readable statement of what an Irish rail +station has that this project could find, and reading one sentence in it the wrong way produced +a page that told a wheelchair user access was fine at a station where it was gone. + +That is the story this series is arranged around. + +## How the posts are arranged + +Twenty-one, deliberately back-loaded. The first five are the site anyone would expect. Chapters +06 to 09 are what happened when it tried to mean something, 10 to 12 are the four days in early +September when everything 09 left open was closed, 13 to 16 are the week after that, in which +the project stopped working around the missing data and started recording it, and 17 and 18 go +back over things the series had called finished and find that two of them were not. + +| # | Title | What it covers | +|---|---|---| +| 01 | A feed that is not about lifts | The collector. Verbatim before parse, and why a failed run must never look like an empty one | +| 02 | The start date that is 451 days old | Measuring the listing rather than Irish Rail's own start date | +| 03 | Three sites, one design layer | The shared front end. The short chapter, on purpose | +| 04 | A grade with nothing to borrow | Inventing an availability scale when no regulator publishes one | +| 05 | The grade argued with the bar underneath it | Escalators, a colour that meant two things, and a sixth letter | +| 06 | The data Ireland does not have | Every source checked, why the absence is lawful, and what it costs | +| 07 | "and" is a sequence, not a choice | Reading the prose, and the misreading that nearly shipped | +| 08 | The same bug, three times | A review pass, and one bug shape found in three places | +| 09 | What one letter cannot say | Four open questions, and why they are hard | +| 10 | Two ways the page lied about time | A build that stalled, and a gap in a listing that vanished | +| 11 | The grade narrows to lifts | The 15 pixels that let escalators leave the letter | +| 12 | Both legs, and who was on the stairs | Which platform kept access, who lost a way up, and how far to trust any of it | +| 13 | What a reader can take away | Feeds, a CSV, and two tripwires for a silent drop | +| 14 | A guard that was guarding the corpus | A test that failed on Irish Rail editing a sentence | +| 15 | Building the thing that does not exist | Recording station access by hand, with provenance | +| 16 | A fourth site, and two bugs found sideways | A second reader of the same feed, and what it found | +| 17 | The same thing, said differently | Three words for planned works, a code that was a name, and a month that moved after it ended | +| 18 | The one code a rebuild cannot undo | The collector's first review, and what the log's own rule does not protect | +| 19a | Closing: what the site can and cannot say | The two lists, and the three-way table | +| 19b | Closing: what I would tell someone starting the fourth | The moral, and the glossary | + +Each post stands alone. Every number in them carries a source and a date, and every figure has +a row in `figures.md` saying where it came from. Where the three sites did the same job +differently, the chapter says what each one does and what fact about its data forced the split, +because none of those splits is taste. + +## What the site says today + +As of 24 September 2026, over 46 days of collection: + +- **69 outages across 44 stations**, of which 5 are escalators. +- **75% availability** across the 21 stations named in August, and 85% across the 30 named in + September so far. That is the share of watched days on which no lift was reported out at those + stations, and the denominator is stated on the page, because the feed names a station only + when something is wrong with it. +- The August grade mix across 21 station-months: **A 2, B 1, C 4, D 7, E 5, F 2**. +- Of the 58 notices on record, **31** are worked out to have removed step-free access to at + least one platform, **4** were escalators, and **23** come back `unknown` because Irish Rail's + own two sources disagree with each other. + +That last row is the one I would point at, and it is getting worse rather than better: it was 6 +of 24 on 31 August and it is 23 of 58 now. Every one of the twenty-three is a disagreement +between a notice and a station page. Thirteen are notices at stations whose page does not +mention a lift at all, and ten name a platform the page puts no lift at. Among them: a page +whose access description is the single word "Level" at a station whose lifts keep breaking, a +page that lists platform 1 twice and never mentions platform 2, stations where the notice and +the page put the lift on opposite platforms. The site prints "unknown" for all of them rather +than guessing, and chapter 07 is about why that is the only defensible thing to do. + +The reason it is getting worse is the interesting part: the corpus keeps reaching stations whose +pages are thinner than the ones it started with, and no amount of care with the parsing improves +a page that does not say anything. That is what chapter 15 is a response to. + +Both of the grade figures above moved twice in the first week of September, once because a bug +was making several stations look far worse than they were and once because escalators stopped +counting towards the letter. Chapters 10 and 11. The August figure has moved since, three weeks +after August ended, which is chapter 17. + +## One note on how it was built + +This repository was written with AI assistance, mostly Claude Code, working against +instructions and review rather than unattended. Of 215 commits on `main` as of 24 September +2026, 142 carry a `Co-Authored-By` trailer, across seven Claude model identifiers. The design +decisions, the corrections and the arguments in `notes/` are the interesting part and are +mine; several of the wrong turns in this series were caught by a human reading the output and +saying "no, that station does not work like that". Chapter 07 is one of those, and it is the +most important correction in the project. + +That is the last time the process is mentioned. The rest is about the data. + +## Notes + +- Figures measured 24 September 2026 by rebuilding `../lifts-data` and running the site build + and `python -m lift_access report`. Registered in `figures.md`. +- Commit and trailer counts: `git log --oneline | wc -l` and a grep for `Co-Authored-By`, + 24 September 2026. +- The regulation quoted is Commission Delegated Regulation (EU) 2017/1926, Annex; the clause is + read in full in chapter 06. +- Sibling series: [uisce #43](https://github.com/baz8080/uisce/pull/43), + [esb #30](https://github.com/baz8080/esb/pull/30). diff --git a/writing/chapters/01-a-feed-that-is-not-about-lifts.md b/writing/chapters/01-a-feed-that-is-not-about-lifts.md new file mode 100644 index 0000000..96c136e --- /dev/null +++ b/writing/chapters/01-a-feed-that-is-not-about-lifts.md @@ -0,0 +1,174 @@ +# 01. A feed that is not about lifts +*~8 min read · PR #1 · 8 to 18 August 2026* + +*Where we are:* nothing exists yet. This chapter is the collector: what it writes down, in what +order, and the one property everything else depends on. + +## The question that opened this stretch + +Irish Rail's realtime endpoint is one URL: + +``` +GET https://connect.irishrail.ie/realtime/messages?lang=en +``` + +It returns a flat list of every service message currently on display. Ask it now and you get +whatever is up now. Ask it in an hour and you get whatever is up then. There is no history, no +archive, no "show me last week". The only way to know how long a lift was out is to have been +watching. + +So the first question is not "how do I model a lift outage". It is "what exactly do I write +down, so that when I get the model wrong I can fix it without losing anything". + +## What changed + +### Write it down before you read it + +Every run appends one line to `raw/messages-YYYYMMDD.jsonl`. The line is written **before any +parsing happens at all**, and it is written whether the run succeeded or failed. A run that got +a 403 writes a line saying it got a 403. A run whose response was not valid JSON writes the +bytes it got. + +Everything else - the SQLite database, the site, the day bars, the grade - is derived from +those files and can be deleted at any time. `rebuild` replays the log from the beginning +through the same code path a live run uses, and reproduces the database exactly. The test that +matters most in the repository does precisely that: it drives a realistic history through the +live path (opens, updates, a close, a failed run that must change nothing, a reopen, a +duplicate, a schema drift), wipes the database, rebuilds from the log alone, and asserts the +two are identical. + +> **Concept: source of truth against derived index.** Two kinds of file live in this project. +> The **log** is what the feed actually said, byte for byte, with a timestamp. It is appended +> to and never edited. The **database** is a convenience: a fast, queryable summary of what the +> log means under today's understanding of it. The distinction earns its keep the first time +> the understanding is wrong. If the code that decides "these two notices are the same outage" +> has a bug, fixing it does not mean going back and correcting records: it means fixing the +> code and replaying the log. The interpretation is disposable and the observation is not. The +> water site made the opposite call, for a reason chapter 03 touches on, and pays for it with a +> rewrite whenever an interpretation changes. + +One line of that machinery is load-bearing in a way that is easy to miss. Every raw line is +written with `json.dumps(..., sort_keys=True)`. Because the keys are always in the same order, +the same observation written by two different machines produces byte-identical text, so two +collectors' logs can be merged with `sort -u` and the duplicates simply vanish. That is the +whole of the multi-machine story, and it is one keyword argument. + +That paragraph is left as it was written, and it was not the whole story. The merge removes +duplicates by sorting, and sorting reorders, so replay had to learn to sort by fetch time. And +the line written "before any parsing" was, until 24 September, written after the database had +been opened. Chapter 18. + +### The feed is not a lift feed + +This is the first data-shape trap, and it shapes everything after it. The endpoint is not +"lift outages". It is every service banner Irish Rail is currently showing: delays, +cancellations, "Station currently closed", engineering works, and lifts. As of 31 August 2026 +the database holds 234 distinct messages, of which **24 are about a lift or an escalator** +(22 lifts, 2 escalators). The other 210 are noise for this project's purposes. + +The collector does not care. It records all 234, because deciding what counts as a lift notice +is an interpretation, and interpretations belong downstream where they can be changed. The +site's `classify` function picks out heads matching `Lift(s) out of order|service` and +`Escalator(s) out of order|service`, and if that pattern turns out to miss something, the +missed notices are already on disk. + +There is a second category, and it is the useful kind of mess. 264 items in the log are +**unidentifiable**: they have an empty `locationCodes`, so there is nothing to attach them to. +Every one is a delay notice about a service rather than a place. They are stored in their own +table rather than discarded or forced into the main one, so a future question about delays has +data to work with, and a present question about lifts is not polluted by them. + +### There is no id, so identity has to be derived + +The feed gives each message no identifier. Poll twice and you get two lists of prose, with no +way to know which entry in the second is "the same" as an entry in the first. + +Identity is therefore constructed: `head` plus the sorted `locationCodes` plus `start`. Three +fields that between them are stable for as long as Irish Rail does not touch the notice. + +The failure mode is exactly what you would expect. If somebody edits the wording, or corrects +the start time by five minutes, the derived key changes, and to the collector that looks like +one message closing and an unrelated new one opening in the same poll. That limitation was +known and accepted on day one rather than engineered around, on the grounds that the raw +responses are kept forever: if it ever matters, the log can be reprocessed with smarter +matching. Chapter 02 is where it starts to matter, and where the site folds those pairs back +together. + +### There is no completion signal either + +Each message carries an `end` field, and in almost every message observed it is a placeholder +near the end of the current calendar year. It is not a repair estimate. It is not a promise. It +is a value somebody has to put in a form. + +So there is exactly one signal for "this is over": **the notice was in one successful run and +is absent from the next**. That is a weaker statement than "the lift is fixed", and the site +never upgrades it. The word used on every page is "no longer listed". + +### The invariant everything rests on + +Here is the failure that would quietly ruin the entire dataset. A run fetches the feed. The +network is down, or the API key has been rotated, or the response is an HTML error page. The +code parses it as best it can, gets an empty list, and concludes that nothing is listed, so +every currently-open notice must have been fixed at 03:30 on a Tuesday. + +That would not crash. It would not log an error. It would produce a database that looks +plausible and is wrong, and the corruption would be invisible until somebody noticed a page +claiming eleven lifts were repaired simultaneously. + +> **Concept: a run that failed is not a run that saw nothing.** These are two completely +> different observations and a status site has to keep them apart. "I asked and there were no +> lift notices" is evidence. "I asked and could not get an answer" is the absence of evidence, +> and treating it as the first one closes every open outage at once. The distinction has to +> survive every future edit by somebody who has forgotten it, which means it cannot be a +> runtime check that a later refactor could route around. In `poll.py` it is structural: every +> failure path returns before reaching `diff_and_update_messages`, the single function that can +> change a message's open or closed status. A failed run cannot reach the code that closes +> things, so it cannot close things. The module docstring says so at the top, in those words. + +The same function classifies a live response and a replayed one, so a live run and a `rebuild` +can never disagree about what a given response meant. Runs are recorded with an outcome, and +everything the site measures is bounded by the last run whose outcome was `ok`. As of 31 August +2026 that is 1,081 `ok` runs out of 1,084, with 3 `unreachable`. + +### Worked example: what one run actually does + +```mermaid +flowchart TD + A[fetch the feed] --> B[append the response to raw/messages-YYYYMMDD.jsonl] + B --> C{usable?} + C -- "network error, 4xx/5xx, not JSON, not a list" --> D[record the run as failed
exit non-zero, alert] + C -- "a list of items" --> E[diff against what was open] + E --> F[open new · touch still-listed · close the absent] + F --> G[record the run as ok] + D -.->|never reaches| E +``` + +The dotted line is the chapter. Everything else is bookkeeping. + +## Where it left the site + +A Raspberry Pi running one command every 30 minutes, a log that grows by about 2.9 MiB a +month, and a database that can be thrown away. Coverage as of 31 August 2026 runs from +2026-08-08T21:30Z, unbroken except for three unreachable runs. The alerting is an ntfy topic +on a phone, because the API key is Irish Rail's and can be rotated without notice, and a +collector that fails silently is worse than no collector. + +What none of this does yet is say anything. Twenty-four outages sit in a table with start +times that turn out to be nearly useless, and that is chapter 02. + +## Notes + +- PR #1, "Add Irish Rail lift-status collector" (18 Aug 2026): the module layout, the raw-first + ordering, `tests/test_rebuild.py`, the systemd units and the ntfy alerting. +- `lift_status/poll.py` module docstring: the structural failure-path property, quoted above. +- `lift_status/store.py:write_raw`: `json.dumps(..., sort_keys=True)`, and `CLAUDE.md` § The + invariant on why it is load-bearing. +- `README.md` §§ How it works, Storage, Known limitations: derived identity, the `end` + placeholder, the accepted identity-drift limitation. +- Measured 31 Aug 2026 against `../lifts-data`: 234 messages, 24 classifying as lift or + escalator (22 lifts, 2 escalators), 264 unidentifiable items, 1,084 runs of which 1,081 ok + and 3 unreachable, coverage from 2026-08-08T21:30:55Z, raw log 2.9 MiB. +- Sibling contrast: the water site's archive is its database (uisce series ch 1); the power + site's feed purges an outage within hours of restoration, which is what forced a Pi there + (esb series ch 1 and 2). This feed is patient, so the Pi here is inherited rather than + derived. diff --git a/writing/chapters/02-the-start-date-that-is-451-days-old.md b/writing/chapters/02-the-start-date-that-is-451-days-old.md new file mode 100644 index 0000000..6788d1a --- /dev/null +++ b/writing/chapters/02-the-start-date-that-is-451-days-old.md @@ -0,0 +1,185 @@ +# 02. The start date that is 451 days old +*~9 min read · PR #2 · 18 August 2026* + +*Where we are:* the collector from chapter 01 has been running since 8 August and the log holds +every notice it saw. This chapter is the first site, and the decision that shapes every number +on it: what interval are we actually measuring? + +## The question that opened this stretch + +Each notice in the feed carries a `start` field. It looks like the answer to "when did this +outage begin", and using it would make the site trivial: colour the days from `start` to `end`, +count them, print a duration. + +The first thing PR #2 did was check whether that field describes anything the collector saw. + +It does not. Measured against all 24 outages on record as of 31 August 2026, **23 carry a start +that predates the poll they were first seen at, and 12 of those by a week or more**: + +| station | Irish Rail's start predates the first sighting by | +|---|---| +| Rush and Lusk | 451.6 days | +| Docklands | 253.4 days | +| Dublin Pearse (lift) | 242.9 days | +| Hazelhatch and Celbridge | 237.6 days | +| Thurles | 197.4 days | +| Dublin Pearse (escalator) | 146.1 days | +| Ballinasloe | 123.9 days | +| Skerries | 118.5 days | +| Ballybrophy | 100.5 days | + +Rush and Lusk is the one to sit with. Irish Rail's notice dates the outage from 14 May 2025. +The notice was present at the very first poll this project ever made, at 21:30 on 8 August +2026, which means all 451 of those days precede collection entirely. Nobody observed anything +at Rush and Lusk in May 2025, because nothing was watching. + +Docklands and Hazelhatch make the same point from the other side, and more sharply, because +there the absence was observed. Docklands' notice is dated 3 December 2025 and first appeared +at 10:30 on 13 August 2026, five days into collection. Hazelhatch's is dated 18 December 2025 +and first appeared on 12 August, four days in. For those five and four days the feed was +queried 48 times a day and those notices were **not in it**. + +There are two readings of any of them. Either the lift really has been out since the date on +the notice and Irish Rail only got round to publishing it, or somebody typed a date. Both are +plausible and the feed gives no way to choose. What is certain is which days were watched. + +> **Concept: measure the window you actually watched.** A status site's honesty rests on the +> difference between "this was true" and "I saw this". Colouring 451 days of a bar because a +> text field claims them would publish an observation nobody made, and it would look exactly +> like an observation somebody did make: same colour, same bar, same page. So this site +> measures the **listing**: from the poll a notice was first seen at, to the poll it was first +> absent from. Irish Rail's start is still printed, as their claim, in their words ("Irish +> Rail's notice dates it from 14 May 2025, 451 days before it was listed"), and it colours +> nothing and enters no total. The claim is information. It is just not evidence about days +> nobody was looking. + +One outage runs the other way, and it is worth naming because it is the exception that shows +the field is not simply broken. Tullamore's planned-works notice carries a start **2.3 days +after** it was first listed: works announced in advance, which is exactly what the field is +for. The field is not nonsense. It is a claim about the world, published at an unrelated time, +and the site treats it as one. + +### The three-way split + +This is the sharpest fork in the whole family of sites, and all three landed differently on +the same field for reasons in their own data. + +| | what it does | why | +|---|---|---| +| **uisce** (water) | takes the notice's publication time and re-stamps it, so every duration is explicitly a **floor** | its feed gives no start at all; the earliest defensible instant is when the notice appeared | +| **esb** (power) | uses ESB's own `startTime` and **measures from it** | it was validated as immutable and back-dated by *hours* to the actual fault: 8 revisions in 1,460 records, and the median lag is about one poll | +| **lifts** | shows Irish Rail's start, **measures from the listing** | back-dated by *months*, over days that were watched with the notice absent | + +Same-shaped field, three different treatments, and none of them is a preference. The power +site's start survives because it was checked and passed. This one's is shown because it was +checked and failed. + +## What else the first site had to decide + +### "No longer listed", never "fixed" + +The word matters more than it looks. There is no completion signal in the feed (chapter 01), so +the only thing the site knows is that a notice stopped being published. + +The pattern of publication makes it worse. Notices do not trickle in one at a time as lifts +break: they arrive and vanish in **batches**. Six were present at the very first poll. Three +more appeared together at 14:30 on 10 August, four at 10:30 on 13 August, two at 14:02 on +17 August. Three were removed in the same single poll on 14 August. Lifts do not break in +threes at 14:01 and they do not get repaired in threes either. That is somebody working through +a publishing queue. + +So the site says "no longer listed" everywhere it would be tempting to say "fixed", and the +rule is written into the fixed vocabulary of this series for the same reason. + +### A reissued notice is one outage; a gap is two + +Chapter 01's derived identity comes back to bite here. Because identity is `head` plus codes +plus `start`, an edited head or a corrected start looks like one notice closing and another +opening **in the same poll**. + +The site folds those: `merge_edits` joins a same-station, same-kind successor whose first +sighting is exactly the predecessor's closing instant. Exactly, on the run timestamp, not +within a tolerance. If a notice comes back a poll or more later it stays a separate outage, +because that gap is information: Docklands closed at 14:01 on 14 August and a new notice +appeared at 14:02 on 17 August, three days apart, and those three days are days the lift was +not reported broken. Two notices open at once at one station never merge either, since the +older is still listed when the newer arrives. + +No lift notice has needed the merge yet. Non-lift ones have: `Station currently closed` became +`Station currently CLOSED` became `Station is OPEN`, three ids in a row. + +That rule was right and the machinery under it was not. A notice that vanished and came back +**unchanged** could not become a second row, because the derived identity key is unique, so the +collector revived the old row and the gap disappeared. Portlaoise was published as sixteen +continuous days when it was two short outages a fortnight apart. Chapter 10 is that bug and the +listings table that fixed it. + +### `end` is shown, not used + +For most notices `end` is a placeholder near the end of the calendar year. A handful look real. +Too few to trust, so it is printed as "listed end 30 Dec 2026" and plays no part in anything +measured. + +It later grew one rule, in PR #18: the listed end **disappears when the notice does**. On a +notice that has come down, "listed end 30 Dec 2026" reads as though the works were still +running, which is exactly how Dublin Pearse's closed lift notice read. The notice coming down +is the completion signal; a placeholder that outlived its notice is noise. + +### Two clocks for one date + +The last decision of this chapter is the one that came from a bug rather than a measurement. + +Irish Rail's start is a Dublin wall-clock time with no offset. "Since 5 May, 00:00" rendered in +UTC becomes "4 May, 23:00", which misquotes them by an hour for half the year. So every instant +a reader sees is rendered in Europe/Dublin. + +That was done first for the printed timestamps and not for the day buckets, and the two +conventions promptly disagreed. Bucketing by UTC date while printing Dublin wall-clock splits +them for four hours a day in summer: a notice first seen at the 23:15 UTC poll on 31 August lit +the 31 August cell while its own summary line read "first listed 1 Sep 2026, 00:15", and at a +month boundary it was filed under August and missing from September entirely. + +> **Concept: two clocks for one date is a bug in either direction.** It is tempting to think of +> this as a rendering detail, but a date is a bucket as well as a label. If the bucket boundary +> and the printed label come from different time zones, then for a few hours a day the site +> shows a cell in one month and a sentence about another, and no reader can tell which one the +> total believes. There is no version of this that is only cosmetic. The fix is to pick one +> convention for everything a reader can see, which here is Dublin, and to say so where the +> remaining UTC stamps are (the build time and the collection horizon, both machine facts). +> It has a real cost: a month is no longer a whole number of days, since March is 23 hours +> short and October 25 long, so the cell count comes from the calendar and never from +> subtracting two instants. Durations are computed once from offset-aware instants and shipped +> with the record, rather than recomputed from rendered strings that carry no offset, because +> subtracting those loses the hour at the October change, and did. + +### The window stops at the horizon, not at the build clock + +Taken straight from the power site. The site's window ends at the last run whose outcome was +`ok`, not at the moment the page was built. Days between the two are drawn as "no data", never +as "nothing listed", because the difference between "we looked and saw nothing" and "we did not +look" is the whole of chapter 01. + +## Where it left the site + +A page of stations with day bars, an initial payload of 30 KB against a self-imposed 500 KB +budget, and every measured interval anchored to something somebody actually observed. The +overview listed the 15 stations with a notice in August; a station's own page carried every +month since collection began. + +What it did not have was any way to say whether five days listed was bad. That is chapter 04. + +## Notes + +- PR #2, "Add a static status site for the collected lift data" (18 Aug 2026): the measured + interval, the batch-arrival evidence, the reissue rule, `end` as a placeholder, the Dublin + wall-clock decision, the 30 KB payload. +- `notes/site.md` §§ The measured interval is the listing, "No longer listed" is the word, + Notices reissued in the same poll are one outage, `end` is shown not used, Displayed instants + are Dublin wall-clock, Windows end at the collection horizon (all 18 Aug 2026). +- `notes/site.md` § Irish Rail's end date goes when the notice does (28 Aug 2026). +- Lead times re-measured 31 Aug 2026 against `../lifts-data`: 23 of 24 outages have a start + predating their first sighting, 12 by seven days or more, maximum 451.6 days (Rush and Lusk), + minimum minus 2.3 days (Tullamore). At PR #2 the same measure read 14 of 17 and 12 of 17. +- The power site's `startTime` validation: esb series ch 3 (`notes/grading.md` "Does startTime + drift?", 18 Aug 2026). The water site's floors: uisce series ch 3. +- Diagram: `diagrams/listed-not-started.svg`. diff --git a/writing/chapters/03-three-sites-one-design-layer.md b/writing/chapters/03-three-sites-one-design-layer.md new file mode 100644 index 0000000..edd8c2d --- /dev/null +++ b/writing/chapters/03-three-sites-one-design-layer.md @@ -0,0 +1,118 @@ +# 03. Three sites, one design layer +*~5 min read · PRs #3 to #17 · 19 to 26 August 2026* + +*Where we are:* the site from chapter 02 exists and is ugly in the specific way a site is when +its CSS was pasted from a sibling. This is the short chapter, on purpose: the other two series +tell this story at length from their side, and repeating it here would be the third telling of +the same week. + +## The question that opened this stretch + +Three status sites, built by one person, deliberately look-alike. Every UI fix had been made +three times by hand. The question is the boring one every shared-code decision starts from: is +this worth extracting, and if so, how, given one hard constraint that neither sibling has. + +## What changed + +### The constraint comes first + +The collector on this project runs on a Raspberry Pi, installed by copying files. Not `pip +install`, not a wheel, not a virtualenv: `scripts/install-native.sh` copies the `lift_status` +package and the `scripts` directory to `/opt/lift-status` and installs two systemd units. That +works because `pyproject.toml` declares **no runtime dependencies at all**, and a clone of the +repository runs on the standard library alone. + +> **Concept: an empty dependency list as a deployment contract.** Most projects treat +> "dependencies" as a list of things to install. Here it is a promise about how the thing is +> deployed. As long as the list is empty, the collector installs on a Pi by copying a +> directory, upgrades by pulling and re-running one script, and cannot be broken by a package +> that stops building on ARM or on the older Python that Raspberry Pi OS ships. The moment one +> entry appears the whole install story changes. So the shared design layer had to arrive +> somewhere the Pi never looks: `statusui` is declared in an optional `site` dependency group, +> `dependencies` stays literally empty, and the file-copy install is untouched. + +### Vendored, then pinned, one day apart + +The first move (PR #4, 19 August) was to vendor: copy statusui's tokens, base CSS, components +and browser helpers into the repository, inline them at build, and guard against drift with a +test that byte-compares the copy to a local checkout of the upstream. + +One day was enough to show what that costs. A shared fix meant a sync, a test run, a commit and +a pull request in each of three repositories, and the sites drifted anyway: this site and the +power site were on one statusui commit while the water site sat five UI commits behind, with +nothing failing to say so, because a byte-compare only fires against the checkout you happen to +have. + +So on the 20th (PR #9) statusui became a real package: a git dependency with a +`[tool.uv.sources]` entry, pinned to a commit in `uv.lock`, in the `site` group. The vendored +tree and the sync script went. One guard stayed, rewritten to read the installed package rather +than a vendored copy: a test that the shared JavaScript does not redeclare a global the site +also defines. + +Changing shared UI now means editing upstream, pushing, and running `rollout.sh`, which bumps +the pin in all three repositories, runs each site's tests and opens three pull requests. The +first rollout deleted a status dot. + +### The alignment pass + +On 26 August (PR #14) the three sites were reviewed side by side, a winner picked per element, +and the same language applied here. The banner takes the shared shape. The heading becomes "The +national picture in August 2026". The legend moves above the list, and its swatches stop being +inline styles and start using the same CSS rules that colour the bars, so the key cannot drift +from what it keys. Search becomes the shared component. The freshness chip, which shows an age +rather than a timestamp, replaces two separate "as of" lines. + +Two things were kept rather than aligned, and both for the same reason: station names and the +words "out of service" are longer than a county name, so this site keeps its wider name and +stats columns. The knobs exist for exactly that. + +One element went the other way and is the more interesting half. Every drill-down on all three +sites offers a link to the static page it has a permanent address at, and the *wording* is +deliberately per site. On the water and power sites the view shows one month and the page shows +every month, so the label promises what is on the other side: "Every month for County X on one +page". Here the station view already shows every month, so naming the content would promise +something the reader is already looking at. The label names the address instead: "Permanent +link to Athy station". A link's label makes a promise, and the promise has to match the +relationship, not the house style. + +### The cadence, and a threshold sized to it + +Two smaller things from the same week that matter to the numbers rather than the look. + +The Pi pushes its logs twice daily and the site rebuilds after each push, so the stale banner +had to be resized (PR #12). It trips at **16 hours**: above the widest legitimate gap between +pushes, which is about 14 hours, and below a missed midnight push, which would show as 17 or +more. A threshold has to be sized to the cadence it is watching, or it either cries wolf or +never fires. + +The cadence changed under it a week later, when GitHub's scheduled builds turned out to be +running four to ten hours late every day. Pushes went six-hourly, the build moved to firing on +the data landing rather than on a clock, and the threshold followed to 10 hours. Chapter 10. + +And the Python floor was written down and then checked (PR #13). `requires-python` says 3.11, +because that is what Raspberry Pi OS bookworm ships and the collector has to run there. The +development interpreter is 3.14. A floor that is only declared is a floor that drifts, so the +install script gates on the same number, ruff takes its target from it, and CI runs the suite +on 3.11 as well as 3.14. + +## Where it left the site + +Three sites that read as one product, a design layer with one home, and a collector whose +install story is exactly as simple as it was on day one. Nothing in this chapter changed a +number. + +The next chapter changes every number on the page. + +## Notes + +- PRs #3 to #17. The load-bearing ones: #4 (take the design layer from statusui), #9 (install + it as a pinned git dependency instead of vendoring), #12 (twice-daily pushes and the 16-hour + threshold), #13 (require 3.11 and check the floor in CI), #14 (the shared design language), + #15 and #16 (the permalink line and the test that the shared rule applies). +- `notes/site.md` §§ The design layer is shared (19 Aug 2026), The vendored copy became a + pinned dependency (20 Aug 2026), The design alignment pass (26 Aug 2026), The permalink + affordance moved out of the footer (26 Aug 2026). +- `CLAUDE.md` §§ Working in this repository, The UI is shared: the stdlib-only rule and the + `rollout.sh` workflow. +- `STALE_AFTER` in `lift_site/render.py`: 16 hours. +- The same week from the other two sides: uisce series ch 14, esb series ch 6a. diff --git a/writing/chapters/04-a-grade-with-nothing-to-borrow.md b/writing/chapters/04-a-grade-with-nothing-to-borrow.md new file mode 100644 index 0000000..6c2c5ef --- /dev/null +++ b/writing/chapters/04-a-grade-with-nothing-to-borrow.md @@ -0,0 +1,203 @@ +# 04. A grade with nothing to borrow +*~9 min read · PR #18 · 28 August 2026* + +*Where we are:* the site shows day bars anchored to observed listings (chapter 02) in a shared +design (chapter 03). Every row ends in a raw count, and this chapter is about why that count +says nothing. + +## The question that opened this stretch + +Each overview row ended with "**5** days listed". + +Five is not an answer. Is five bad? Five days out of a 31-day August is one thing; five out of +the 20 days collected so far is another; and the two cannot be compared by the number five. A +reader who wants to know whether a station is doing badly has to do arithmetic the site already +has the inputs for. + +The obvious fix is a grade. The sibling sites both have one, and the obvious way to build one +is to grade against the standard the operator publishes. That is exactly what the power site +does: ESB Networks' Customer Charter states an aim anyone can read, "restore supply within less +than 4 hours in 95% of cases", so an A on that site means the operator met its own promise and +the anchor is not the author's. + +So: what is the equivalent for lifts? + +## What changed + +### There is no target, and looking for one is the work + +This was checked properly, because a decision recorded ten days earlier had said *no grade*, +partly on the grounds that no operator standard exists. Reversing a settled decision means +re-testing the reason it was settled. + +- **The PRM TSI**, Regulation (EU) 1300/2014, is the European rulebook for accessibility of + rail for persons with reduced mobility. It sets design rules for lifts and escalators, and it + places an operational duty on the station manager to hold a written policy ensuring access + "at all operational times". A duty to have a policy. No percentage, no reporting obligation, + nothing a passenger can check. +- **Irish Rail's Passengers' Charter** promises "every effort ... available as advertised". +- **The Big Lift**, an NTA-funded programme that put lifts into 52 stations between 2020 and + 2024, publishes no availability figure. + +The regulators who do publish numbers are in other countries. Britain's ORR and Network Rail +report lift faults in the tens of thousands (8,696 in a year, 6.6 per lift, over 20 hours +average repair). Transport for London has historically reported 93.7% lift availability, or +98.8% excluding planned works. + +So there is nothing to borrow. The scale has to be this site's own, and the page has to say so +rather than dressing it up as a standard. + +> **Concept: a scale with no anchor.** A grade is a compression: it turns a measurement into a +> letter so a reader can compare things at a glance. What makes a letter meaningful is what it +> is anchored to. The power site's A is anchored to a promise ESB published, so an A is +> checkable against a document. This site has no such document, which leaves two bad options +> and one acceptable one. A **relative** scale, where the worst stations get the F, always +> fails somebody by construction and moves under the reader as other stations change. A scale +> borrowed from **another country's** operator sounds rigorous and is not, for reasons the next +> section measures. What is left is an absolute scale of the author's own devising, stated +> plainly as such, with cuts chosen so a reader can reconstruct them by counting. That is +> weaker than an anchor and it is honest about being weaker, which is the trade. + +### The bands are counted in days, because that is what the bar is drawn in + +The first attempt borrowed TfL's numbers. It was tried on paper and thrown away in one step, +because of an arithmetic property of day-granularity data that is easy to miss. + +This site colours **days**. It does not know the hour a lift came back, only which polls a +notice was present at, and the bar shows one cell per day. Over a 31-day month, a station +listed for exactly one day is 30/31 available, which is **96.8%**. A scale where 98% is the +target puts a station with a single bad day below the line, and every real station in the +bottom band. + +So the bands are calibrated in the unit the reader can see: + +| grade | availability | over a 31-day month | +|---|---|---| +| A | 100% | nothing counted | +| B | 95%+ | one day listed | +| C | 90%+ | two or three days | +| D | 75%+ | up to a week | +| F | below 75% | more than a week | + +(An E arrives in chapter 05, splitting F. Nothing above it moves.) + +> **Concept: a band calibrated in the unit the bar is drawn in.** If the bar shows days and the +> grade shows a percentage, the two are the same fact in different clothes, and a reader should +> be able to get from one to the other by counting cells. That is only true if the cuts land on +> whole numbers of days at a plausible month length. Bands imported from an operator measuring +> hourly availability do not: they encode a precision this data does not have, and a percentage +> printed to one decimal beside a bar with one red cell in it is a claim about hours that +> nothing here can support. Availability is therefore floored rather than rounded, so 100% +> cannot round up from a day that counted, and hours-based availability was rejected outright +> even though it is more precise. + +**Availability** is derived from the day bar itself: days watched with nothing counted against +them, over days watched. Deriving it from the bar rather than computing it separately means the +chip and the bar cannot disagree, which turns out to matter in chapter 05. + +### Planned works get a week, and then they count + +Planned works were masking the thing the site measures. They sit for months: of the 24 outages +on record as of 31 August 2026, 6 were planned works, and Midleton's had been listed +continuously since 12 August. + +The rule that landed: **works listed seven days or less in total cost nothing; past that, every +listed day counts, including the first week.** A week is a plausible maintenance window, and +because Irish Rail's own end dates are placeholders (chapter 02), the listing is the only +measure of how long works actually ran. + +The phrase "in total" is doing a great deal of work in that sentence, and review moved it +twice. + +### Worked example: six days of works, then four of fault + +Take one station with a ten-day story: six days of planned works, then the works notice comes +down and a fault notice goes up for four days. + +- **Version 1, per segment.** The grace applies to each folded segment separately. A notice + reissued every few days never exceeds seven days in any one segment, so a station could carry + a month of works and grade A. This is exactly the reissue the merge from chapter 02 exists to + fold, so it was a hole opened by the interaction of two rules that each looked fine alone. +- **Version 2, over the whole listing.** The grace is measured over the outage's entire + listing, works and fault together. Ten days exceeds seven, so nothing is forgiven: all ten + days count, availability is 0%, and the fault has reached back and charged the station for + the maintenance week it had already earned. +- **Version 3, the planned segments summed.** The works ran six days, which is inside the + grace, so they cost nothing. The fault's four days count. Six of the ten days are available, + which is 60%. + +Versions 2 and 3 both produce an F on the five-band scale, which is why this needed a +constructed example rather than a corpus case to see: the letter is the same and the number is +not, and the gap widens the longer the works run. Version 3 is what shipped, and the rule is +one sentence: what the works cost is measured on the works. + +On today's corpus the grace forgives Dublin Pearse's five-day lift notice and Greystones' two, +and does not forgive Limerick Junction's ten days or Midleton's nineteen. + +### The build-killer, and where it came from + +Three code-review rounds ran over this branch. One of them found a bug that would have taken +the site down, and it is a good example of a class of bug this project keeps producing. + +The day bar stops at the build clock. The availability window stops at the **collection +horizon**, the last successful run. Those are two different machines' clocks: the Pi collects, +GitHub Actions builds. When the two diverge, the bar can show a day that the window has already +excluded, so the count of days-with-something-listed exceeds the count of days-watched. +Availability goes **negative**, and the band lookup, which walks the table from A downwards +looking for the first cut the value clears, finds nothing and raises `StopIteration`. + +Reproduced at 13 hours of skew on Midleton: 20 days observed against a window of 21, and an +availability of minus 5. + +Two other review findings from the same branch are worth recording because they have the same +shape as the grace rule: a quantity computed over the wrong extent. The month list was built +from the build clock alone, so a collection horizon in a month the build clock had left dropped +that month's outages from the shards, the statistics and the headline. And the grade key +claimed "A - no days listed", when what A actually means is nothing *counted*, which is not the +same thing once works inside their grace are drawn on the bar and left out of the total. + +That last one is a wording bug with real consequences: Tullamore ships at **A / 100% +available** with planned-works cells visible on its bar - two when the review found it, four as +measured on 31 August 2026. The key has to say "100% available" and not "nothing listed", or +the page contradicts itself in the reader's first glance. + +### The three-way split + +| | what anchors the grade | what it is measured on | +|---|---|---| +| **uisce** (water) | its own thresholds | person-hours: population inside a radius, multiplied by the hours a notice ran | +| **esb** (power) | ESB's published 4-hour / 95% charter aim | the share of fault-interrupted customers restored inside four hours | +| **lifts** | its own bands, and the page says so | the share of watched days with nothing reported out | + +The water site had to invent a standard because Irish water is measured in public by nobody. +The power site did not have to invent one. This site had to invent one **and** had nothing to +build it out of: the feed carries no magnitude at all. A notice is listed or it is not. There +is no count of people affected, no count of lifts at a station, no severity. Days are the only +unit available, which is why the grade is counted in days and why the bands had to be +calibrated to them rather than to somebody else's percentage. + +## Where it left the site + +Every row carries a letter and a percentage, derived from the bar beneath it so the two cannot +disagree. Planned works are forgiven for a week and counted after it. The window ends at the +horizon and the letter is a claim about days that were watched. + +And within 24 hours, a reader could look at Dublin Connolly's row and see a green A sitting +directly above two red cells. That is chapter 05. + +## Notes + +- PR #18, "Grade stations on lift availability, one bar per kind" (28 Aug 2026): the reversal, + the standards search, the day-calibrated bands, the planned-works grace and its two wrong + versions, the three review rounds including the skew crash, the payload change. +- `notes/site.md` §§ No grade (18 Aug 2026, marked reversed), The grade is availability + (28 Aug 2026), Planned works are excused for a week (28 Aug 2026). +- Standards, all as cited in PR #18: PRM TSI (Regulation (EU) 1300/2014); Irish Rail + Passengers' Charter; NTA Big Lift, 52 stations 2020 to 2024; ORR/Network Rail 8,696 faults, + 6.6 per lift, over 20 hours average repair; TfL 93.7% and 98.8% excluding planned works. +- The 96.8% arithmetic: 30/31, floored, over a 31-day month. +- Corpus figures measured 31 Aug 2026: 24 outages, 6 planned. Grace outcomes (Pearse 5 days, + Greystones 2, Limerick Junction 10, Midleton 19) from `notes/site.md`, 28 Aug 2026. +- The skew reproduction (Midleton, 13 h, observed 20 against 21, availability minus 5) is from + PR #18's review notes. +- Sibling grades: uisce series ch 8a and 8b; esb series ch 4b. diff --git a/writing/chapters/05-the-grade-argued-with-the-bar.md b/writing/chapters/05-the-grade-argued-with-the-bar.md new file mode 100644 index 0000000..a0c0961 --- /dev/null +++ b/writing/chapters/05-the-grade-argued-with-the-bar.md @@ -0,0 +1,203 @@ +# 05. The grade argued with the bar underneath it +*~10 min read · PRs #25 and #27 · 29 August 2026* + +*Where we are:* every station row now carries a letter grade derived from its day bar +(chapter 04). This chapter is the day after, when a reader could see the letter and the bar +disagree on the same line. + +## The question that opened this stretch + +Dublin Connolly, on 29 August, read: + +> **A · 100% available** + +with two **red** cells sitting directly beneath it on the escalator strip. + +The grade was lifts only, on what looked like sound reasoning: the lift is the step-free route, +so lift availability is the thing that matters. Escalators had their own bar, added the day +before precisely so a working lift would never be painted by a broken escalator. + +Both halves were defensible and together they published a contradiction. No reader was going to +resolve "100% available" against two red cells in the site's favour. + +## What changed + +### Escalators count, and the grade becomes a smaller claim + +The fix was to put escalator days into the pool. That is one line of arithmetic and a +paragraph of consequences, and the consequences are the interesting part, because counting +escalators means the grade can no longer be called what it was being called. + +An escalator is moving stairs. A wheelchair user cannot use one: every operator prohibits it, +Irish Rail included, and it is a matter of the step geometry rather than a formality. So an +escalator going out of service **never removes step-free access**. It removes the easier route +for somebody who can manage stairs with difficulty, or who has a buggy, a stick or a suitcase. + +If escalator days count, then the grade is not step-free availability. It is something weaker +and plainer: + +> **Irish Rail reported something out at this station on this day.** + +Vertical circulation degraded, not access lost. The legend says "Lift and escalator +availability", which is the honest label for that. + +And the site could not honestly grade step-free access anyway, even if escalators were kept +out. A notice names one machine in prose ("the lift at platform 2") and there is no roll of how +many lifts a station has, so a lift notice coming down does not mean every lift at that station +works. Reserving the grade for lifts would have given it a name it could not live up to. + +Two alternatives were weighed and both lost: + +- **Lifts only, escalators visible but out of the letter.** More precise, and it was the + position until that day. What killed it is Connolly: the contradiction above is worse than + the imprecision below. +- **Two grades, a step-free one and a softer one.** The most honest of the three. It costs a + second chip on every row and a decision about which one sorts the overview, which is out of + proportion to a distinction this feed cannot support cleanly in the first place. + +The bars still split by kind, for the reason they always did. + +On the corpus that day the change moved aggregate availability from 70% to 66%: Connolly from +A to C, and Dublin Pearse from A to **F**, at 22%. Chapter 09 is about why that F is a problem +even though every step to it was correct, and chapter 11 is where it was undone: on 3 September +escalators came back out of the grade, and both stations returned to A. What changed in between +was not the argument but the row, which is the interesting part. + +### One colour said two opposite things + +The same pull request fixed a second self-contradiction, in the bar rather than beside it. + +Planned works were drawn in blue. All of them, forgiven or not. So Midleton, listed 19 days at +that point and grading 0% available, drew exactly the same blue as Dublin Pearse's six-day lift +notice, which cost nothing at all. One colour carried both "this is forgiven" and "this is the entire reason +this station is an F", and a reader comparing two blue bars had no way to tell which was which. + +> **Concept: one colour, two meanings.** A legend is a promise that a mark means one thing. +> When the same mark covers two cases that a reader would want to distinguish, the legend is +> not wrong so much as useless: it is true of both, and true of both is the same as silent. +> The failure is easy to introduce because it happens between two changes, neither of which +> is wrong on its own: a colour is chosen for "planned works", and later a rule is added that +> forgives *some* planned works. Nobody revisits the colour. The check is to ask, for every +> mark on the page, whether two cells drawn the same way could lead a reader to two different +> conclusions about what happened. + +Works past their grace are now **amber**, with their own key entry. Amber rather than a second +red, for two reasons: works that overran are not a fault and the notice text under the bar says +"planned works" either way, so recolouring them red would put two words and one colour in +disagreement, which is how this started. And at cell width a person with deuteranopia cannot +separate orange from the critical red. + +Three shades can now share a day, so the rule for which one wins had to become explicit rather +than a comparison: a day cell takes the worst of what was listed on it, ranked fault, then +overrun works, then works inside their grace. + +### The legend that keyed nothing + +A smaller thing that is a good illustration of the same discipline. + +Under the day key sat a second row: five colour swatches drawn from the grade chip fills. A +reader asking what they referred to was right to ask. A grade is read as a **letter**. The +colour behind the letter is reinforcement, and nothing else on the page is painted in it. So +the row was a key to a code the page does not use, sitting directly beneath a key where every +swatch does map to something in a bar. + +It was rebuilt to carry the chips themselves, letter and all, which is the object a reader has +been looking at on every row. That fixed *what* it keyed without fixing *where* it sat: two +legend rows stacked above the list still read as one key with two halves, only one of which +maps to anything visible. So it moved into the footer, inside a "How the grade works" +disclosure, directly under the sentences that define availability and the works grace. The key +and its explanation are one thing. The static station pages carry their own copy rather than a +link, because a station page is where a search result lands and its chip has to be explicable +without a second page load. + +### The scale grew an E + +The band table ran A, B, C, D, F. Skipping E is an American convention and Irish Rail is not +American, so the letter was added (PR #27). + +The interesting part is where to cut it. E splits the old F band and moves nothing else: every +cut from 100 down to 75 stays where it was, so no station-month graded A to D changes letter. + +**50%**, because it is the same kind of number the other cuts are: a count of days a reader can +hold in their head. Availability is floor-divided, so over a 31-day month a floor of 50 makes E +8 to 15 days listed and F 16 or more. Up to half the month, against more than half. + +### Worked example: the cut lands in a real gap + +The arithmetic above justifies 50 as a *legible* number. It does not justify it as the right +place to put a boundary, and those are different claims. So the distribution was checked. + +Over the 21 graded station-months in a rebuild of the corpus, the old F band held nine values: + +``` +0, 0, 18, 22, 22, 50, 68, 68, 72 (per cent available) +``` + +There is a real gap between 22 and 50, and the cut lands in it. E takes the four stations +listed for part of the month; F keeps the five listed for most or all of it, including two at +nothing available at all. Cuts at 60 and 40 both fall inside the same gap and split the nine +values identically, so 50 was chosen for saying something a reader can repeat, not because the +data preferred it. + +> **Concept: a cut that lands in a real gap.** A threshold is arbitrary until you check what it +> separates. Two things can be true at once and both are worth stating: the number was chosen +> for a reason that has nothing to do with the data (it is memorable, and half a month is a +> phrase), and the data happens to agree, because there is empty space on either side of it. +> If the values had been evenly spread, moving the cut by five points would have reclassified +> stations, and the honest thing would have been to say the boundary is a convention. Here it +> is not doing that work, and that is checkable rather than assertable: 60 and 40 produce the +> same split, which is what "lands in a gap" means operationally. + +The grade mix moved from A 1, B 1, C 5, D 5, F 9 to **A 1, B 1, C 5, D 5, E 4, F 5**. It still +read that way on 31 August; chapters 10 and 11 move it twice more. + +The pin bump for the sixth chip shipped in the same pull request rather than a follow-up, +because the band table and the chip that renders it are two halves of one change. It brought +some accessibility work with it: two of the chips moved to colours that take white lettering, +because dark ink on them had been chosen on one contrast standard and a newer one rated it far +worse, and the "no grade" dash moved off a token that was failing contrast in both light and +dark. This site renders that dash more than its siblings do, for a station-month with nothing +to grade. + +### Plain words on the tiles + +One last change from the same day, small and worth copying. The summary tiles read "4 lifts +with a notice up at the last poll" and "70% of days available". The first asks a reader to know +what a poll is. The second asks them to work out what "available" was measured over. + +They now read "lifts reported out when we last checked" and "of days with lifts and escalators +available, across the stations named this month". Longer, and the denominator stays, because +that is the part that could not be dropped: the feed names a station only when something is +wrong with it, so the site has no roll of the stations that have a lift, and any wider +denominator would be invented. + +"Poll" left the visitor-facing text entirely. "Listed" stayed, because a notice being listed is +exactly what the site measures, and "fixed" is the word it must not use. + +## Where it left the site + +A letter that means what its legend says, a bar where two cells of the same colour mean the +same thing, a six-band scale whose cuts a reader can reconstruct by counting days, and tiles +written for somebody who has not read the source. + +And one station, Dublin Pearse, sitting at the bottom of the scale on the strength of an +escalator alone. Every step to that F was correct. Chapter 09 is about why it is still wrong. + +## Notes + +- PR #25, "Count escalators in the grade, colour works that overran, and say plainly what the + page means" (29 Aug 2026): the Connolly contradiction, the escalator reasoning, the amber + code and the explicit day-severity ranking, the grade key, the plain-words pass, and the five + statements the review of that branch found untrue. +- PR #27, "Grade stations A to F inclusive, with E at 50%" (29 Aug 2026): the cut, the + nine-value distribution, the grade-mix move, and the statusui pin bump with its contrast + work. +- `notes/site.md` §§ An escalator out is a day the station was short of a way up, Blue said two + opposite things, The grade key keys the letter not a colour, ... and then left the top of the + page entirely, The scale grew an E, Plain words on the summary tiles (all 28 to 29 Aug 2026). +- Availability 70% to 66%, Connolly A to C, Pearse A to F at 22%: `notes/site.md`, 29 Aug 2026. + Measured again 31 Aug 2026 the same figures read 67% aggregate, Connolly C at 91%, Pearse F + at 20%; the corpus grew two days in between. +- Grade mix A 1, B 1, C 5, D 5, E 4, F 5: PR #27, and re-measured 31 Aug 2026 across 21 graded + station-months. +- The water site's binary knock rule, which chapter 09 returns to: uisce series ch 15. diff --git a/writing/chapters/06-the-data-ireland-does-not-have.md b/writing/chapters/06-the-data-ireland-does-not-have.md new file mode 100644 index 0000000..6b14b86 --- /dev/null +++ b/writing/chapters/06-the-data-ireland-does-not-have.md @@ -0,0 +1,220 @@ +# 06. The data Ireland does not have +*~10 min read · issue #24 and PR #30 · 29 to 30 August 2026* + +*Where we are:* the site counts lift outages and grades stations on them (chapters 04 and 05). +This chapter is about the question it could not answer, which is what any of that *means*, and +about the two weeks spent finding out that Ireland does not publish the answer. + +## The question that opened this stretch + +The site had a hole in it that a reader could not see, and it went like this. + +A station with no lift, and therefore no lift notices, renders exactly like a station whose +lift is working perfectly. Both are a row of green cells. One of those stations is worse for a +disabled passenger than the other, and the site was calling it better. + +And the national figure had no denominator. "67% available across the stations listed this +month" is a number divided by a set the feed chose, and the feed names a station only when +something is wrong with it. There is no roll of the stations that have a lift at all, so the +percentage floats: it cannot be a network figure, and the page has to say so. + +Underneath both is the real question, the one that would let the site say something worth +saying: + +> When a lift is out, is there another accessible way in? + +Filed as issue #24, and blocking treating the published site as usable. Three questions that +kept getting conflated were separated first, because they have different answers, different +sources and different value: how many stations have a lift at all (the denominator), does this +station have step-free access when nothing is listed, and is there another route when +something is. + +The third is the interesting one, and answering it needs a fact this project does not have: +**what does this station have, and which platform does each machine serve?** + +## What changed + +### Every source, checked + +The checks were run from a machine with real network egress on 30 August 2026, and written down +so nobody runs them again. + +**GTFS.** The General Transit Feed Specification is what transit apps read. It has an extension +built for exactly this question: `pathways.txt` models a station's interior as a graph, with a +`pathway_mode` column that distinguishes a walkway from stairs from a travelator from an +escalator from a lift. If a station's graph carries a lift edge and an escalator edge between +the same two nodes, the escalator has a lift beside it. If the only other edge is stairs, it +does not. That is the answer in machine-readable form, and it even settles the "flat escalator" +question in data rather than prose, since a travelator is a different mode from an escalator. + +There is no `pathways.txt`. Not in `GTFS_Irish_Rail.zip`, not in `GTFS_All.zip`, not in +`GTFS_Realtime.zip`. Each archive holds exactly ten files and none of them is it. There is no +`levels.txt` either. + +**GTFS, the simpler field.** Standard `stops.txt` carries `wheelchair_boarding`, a three-valued +column: no information, some vehicles or paths accessible, not accessible. It is not merely +unpopulated here. **The column does not exist.** The header is identical in all three archives +and ends at `parent_station`. `location_type` is empty for all 152 rail stops as well, so there +is not even a station-to-platform hierarchy. GTFS gives an inventory of stops and nothing else. + +**NaPTAN.** 152 rail stops keyed by station code, with `AccessArea` null on every one of them. +The string "accessib" does not appear anywhere in the 22 MB file. + +**PTIMS.** Bus street furniture. + +**The NTA developer API.** Bus only. Its own description names Dublin Bus, Bus Éireann and +Go-Ahead, and the GTFS archive it links to was already ruled out above. + +**Irish Rail's own `getAllStationsXML`.** Still up, still unkeyed, 171 stations with a +`StationCode`. It was right about the code space and carries no accessibility data at all. +Inventory only. + +**Irish Rail's lifts-and-escalators alerts page.** Found while scoping and it looked promising: +a page titled "Alerts for Lifts and Escalators" would, if it were a per-machine inventory with +status, have been a better primary source than the message feed and would have answered "how +many lifts does this station have". Checked on 29 August. It is not that. + +### The formats that were written for this, and are not published here + +There are two European standards that carry exactly the data this chapter is looking for. + +**NeTEx** is the European standard for a station's static equipment and accessibility: lifts, +escalators, ramps, entrances, and the paths between them. **SIRI-FM**, Facility Monitoring, is +its realtime companion, which is precisely a live "is this lift working" feed. If Ireland +published SIRI-FM, this repository would not need to exist. + +Neither is published for Ireland. data.gov.ie and the NTA's public transport data catalogue +were searched on 30 August 2026: 24 GTFS archives, NaPTAN, PTIMS, and nothing else. The only +occurrences of the string "accessibility" on the NTA's data page are navigation menu links. + +### The regulation, and the hole in it + +This is the part I did not expect, and it is why the absence looks permanent rather than like +an oversight somebody will fix. + +Commission Delegated Regulation (EU) 2017/1926 requires each member state to run a **National +Access Point** publishing a listed set of travel data types, with NeTEx as the required +representation. Read the Annex and you find the data types listed. Read the qualifying clause +and you find this: + +> provided they exist in digital machine-readable format + +> **Concept: a National Access Point, and a lawful absence.** A National Access Point is a +> single official place where a country publishes its transport data so that anyone, including +> Google and Apple, can build on it. Ireland has one and it works. The clause above is what +> decides what goes into it: the duty is to **publish what you hold**, not to create what you +> do not. So if Irish Rail never captured a lift and escalator inventory in machine-readable +> form, nothing in the regulation compels them to start, and the obligation is satisfied by +> publishing timetables. The absence is lawful. That is a more uncomfortable finding than a +> gap somebody forgot to fill, because there is no process that closes it: no deadline, no +> non-compliance, nobody to write to. It closes when an operator decides to build an inventory, +> or when a future revision drops the qualifying clause. + +### What the mapping apps have to work with, which is nothing + +The strongest evidence that this data was never created, rather than that I searched badly, is +that four independent consumers of Irish public transport data hit the same wall. + +Google Maps, Apple Maps, Transit, Citymapper and Moovit all consume GTFS for transit +directions, and accessible routing in all of them rests on three fields. Checked against the +live Irish Rail feed: + +| field | what it answers | present? | +|---|---|---| +| `stops.txt` `wheelchair_boarding` | can you board here | **no, the column is absent** | +| `trips.txt` `wheelchair_accessible` | does this vehicle take a wheelchair | **no, the column is absent** | +| `pathways.txt` | a step-free route from entrance to platform | **no, the file is absent** | + +So none of them can offer wheelchair routing on Irish Rail. That is a gap in the input, not a +failure of their products. Their place-level accessibility pins come from their own pipelines +instead: Google from Places and Local Guides, Apple from its own surveys. Crowd-sourced, +patchy, no platform detail, and no idea whether a lift is out today. + +> **Worked example: the four-consumer argument.** When a search comes back empty there are +> always two explanations, and the weak one is that you searched badly. The way to tell them +> apart is to look at who else would need the same data and check whether *they* have it. Five +> billion-dollar mapping products, each with commercial reasons to offer accessible routing in +> Ireland and each with a team who would have found the file if it existed, all fall back to +> their own crowd-sourced pins for Irish stations. Two European standards exist for precisely +> this and neither is published. A regulation that would have compelled it carries a clause +> that exempts it. Four independent lines of evidence pointing the same way is much stronger +> than any one search, and it is the reason `notes/station-access.md` ends with an instruction +> not to re-run these checks: the one thing worth watching for is Ireland starting to publish +> NeTEx. + +### What is left, and it is a CMS field + +The only machine-readable statement of what an Irish rail station has is a free-text field on +irishrail.ie, written by hand, with no schema, no versioning and no obligation to be accurate. + +It is at least not a scrape in the ugly sense. Each station page is server-rendered Nuxt and +serves its own data as JSON at `/_payload.json`: named fields, no HTML parsing. +`robots.txt` disallows only `/stations.csv`. The find-a-station payload carries the full list of +152 station slugs, so nothing is crawled. + +Two facts made it usable rather than merely available: + +- **The join is free.** Every payload carries a `stationCode`, and it is exactly the same code + space the message feed uses in `locationCodes`. All 15 codes that had lift notices at the + time matched, 15 out of 15, with no fuzzy matching and no mapping file to maintain. The + name-to-code join the scoping note had worried about does not exist as a problem. +- **The site already held half of it.** The notice text has always said which platform: "The + lift at platform 2", "The lifts at platform 1 and 4". Nothing was parsing it, because the + `head` says only "Dublin Pearse - Lift out of order" and that is what everything read. + +And two fields that look useful and are not, both worth naming because both would have shipped +a wrong claim: + +- **`wheelchairAvailability`** does not mean "this station is accessible". It means a + wheelchair can be **borrowed** there. Dublin Pearse says Yes, Docklands says No. Surfacing it + as accessibility would be a serious misrepresentation and it is one keyword away from + happening. +- **`alert`, `alertStart`, `alertEnd`.** 131 stations carry one and they are never cleared. + `alertEnd` values run back to 2014, 2015 and 2021. It is the last alert ever posted, not a + live one. + +### The unintended consequence + +The snapshot is stored the way everything else in this project is stored: every payload +verbatim, one per line, sorted keys, in `lifts-data/stations/`. 7.8 MB plain, about 2 MB in +git, and it greps and diffs. Never edited; the derivation is always recomputed from it. +Refreshed monthly by a workflow that opens a pull request rather than pushing, because a +reworded station page can move a verdict from "no step-free access" to "unknown" and back, and +that is not a change to land unread. + +Which produces something nobody set out to build. **The dated snapshots in +`lifts-data/stations/` appear to be the only versioned machine-readable record of Irish rail +station access that exists.** That was not the intent, it is a poor substitute for the operator +holding one, and it is a reason to keep the monthly refresh running well beyond keeping this +site's derivation fresh. + +Ten days later the project stopped working around the absence and started filling it. Chapter 15 +is a hand-recorded observation log, a reachability graph replayed from it, and an export in the +GTFS format this chapter found missing. Everything below still holds: that is what made the +absence worth documenting first. + +## Where it left the site + +152 stations with codes, of which **57 claim a lift** and 95 do not. A denominator. A per-station +statement of what the operator says the station has, versioned and diffable. + +And a new problem, which is that the source is prose, and prose has to be read. The next chapter +is about getting that wrong. + +## Notes + +- Issue #24, "Verify the accessibility data sources before the site is shared as usable" + (closed by PR #30); PR #30, "Say what a lift outage did to step-free access" (30 Aug 2026). +- `notes/accessible-routes.md` (29 Aug 2026, answered 30 Aug): the three questions separated, + the source-by-source scope, the struck sources, and the instruction not to scope them again. +- `notes/station-access.md` §§ Why scraping prose is the only option not the lazy one, The + regulation and the hole in it, What the mapping apps have to work with, The sources and the + ones that are closed, The snapshot (all 30 Aug 2026). +- GTFS checks: three NTA archives, ten files each, no `pathways.txt` and no `levels.txt`; + `stops.txt` header ending at `parent_station`; `location_type` empty on 152 rail stops; + NaPTAN 152 rail stops with `AccessArea` null and no "accessib" substring in 22 MB. +- Regulation: Commission Delegated Regulation (EU) 2017/1926, Annex, and its "provided they + exist in digital machine-readable format" qualifier. +- Station counts measured 31 Aug 2026 from `stations/irishrail-20260830.jsonl`: 152 stations, + 57 with a lift, 95 without. +- Diagram: `diagrams/what-would-carry-it.svg`. diff --git a/writing/chapters/07-and-is-a-sequence-not-a-choice.md b/writing/chapters/07-and-is-a-sequence-not-a-choice.md new file mode 100644 index 0000000..5b5766b --- /dev/null +++ b/writing/chapters/07-and-is-a-sequence-not-a-choice.md @@ -0,0 +1,232 @@ +# 07. "and" is a sequence, not a choice +*~10 min read · PR #30 · 30 August 2026* + +*Where we are:* chapter 06 established that the only source for what an Irish rail station has +is a hand-typed field on irishrail.ie. This chapter is about reading it, and about the reading +that would have told a wheelchair user access was fine at a station where it was gone. + +## The question that opened this stretch + +Here is the field, at one station: + +> **Hazelhatch and Celbridge:** "All platforms can be accessed via lifts and ramps" + +The site now knows there is a lift notice at Hazelhatch. It has that sentence. What does it +publish? + +The first version of this branch read the sentence as a list of options. Lifts **or** ramps: +two ways to reach the platforms, so if the lift is out, the ramps remain, and step-free access +survives the outage. + +That is wrong, and it is wrong in the worst available direction. + +## What changed + +### The sentence describes a route, not a menu + +Read it again as somebody who has been to the station. The ramp gets you along. The lift does +the level change. You need **both**, in sequence, and there is no way to most platforms at +Hazelhatch without the lift. + +The site would have published "access remains" at a station where access was gone. Barry, who +knows the station, caught it. + +> **Concept: the safe direction of an error.** Any derived claim will sometimes be wrong, so +> the question is not whether to be wrong but which way. A reader told access is gone when it +> was not has made one wasted check: annoying, recoverable, a phone call. A reader told access +> remains when it is gone travels to the station and is **stranded on a platform**, possibly +> after a journey they cannot easily reverse. The two errors are not symmetric and they must +> not be weighted as if they were. So every rule in this module leans the same way: default to +> "gone", say "unknown" freely, and never infer that access remains. That single principle +> decides most of the design decisions in the rest of this chapter, including several that look +> like they are about parsing. + +### Then it was checked, everywhere + +One caught misreading is an anecdote. The useful move was to check whether the conjunctive +reading holds across the whole network, and it does. Across all 61 stations whose prose +mentions a lift: + +- **"and" as a sequence: 29 stations.** "Lift and footbridge to platform 2" (Malahide, + Skerries, Portlaoise, Maynooth, Monasterevin, Portarlington, Templemore, Tullamore, + Balbriggan, Laytown) is lift up, cross, lift down. Same for "Lifts and footbridge to all + platforms" and "Platforms accessible via stairs and lifts". +- **"or" is a genuine disjunction, but it is nearly always "or stairs": 11 stations.** + Adamstown, Bayside, Clonsilla, Glenageary, Howth Junction, Shankill, Blackrock, Booterstown, + Bray, Tara Street. Stairs are not a step-free alternative, so the lift is still the only way. +- **Two stations, out of 61, name a step-free way round a lift for the same platform.** + Raheny, "Lift or ramp to platform 1", and Cork, "Ramp or lift to platform 5A, 5B and 6". + +So the question issue #24 set out to answer, *when a lift is out, is there another accessible +way in?*, has a near-constant answer on this network: **no**. + +That is worth publishing on its own. It also means no route engine is needed, and no graph, and +no pathfinding. The model works out which platforms a lift serves, assumes an outage removes +step-free access to them, and carves out the two exceptions by hand. + +### The model deliberately does not parse connectives + +This is the design decision the chapter exists to explain, and it is the opposite of what a +programmer's instinct suggests. The obvious move, having discovered that "and" means sequence +and "or" sometimes means alternative, is to write a parser that tells them apart. + +`lift_access/model.py` does not parse connectives at all. + +The two exceptions live in a constant, `STEP_FREE_ALTERNATIVES`, which is a hand-reviewed list +of two entries. Adding one is a human decision visible in a diff, never a parser output, and a +test fails if any published verdict claims an alternative that is not in the list. + +The reason is the safe direction again. A connective parser that is 95% right is a machine that +will, five times in a hundred, publish "access remains" without anybody having read the +sentence. Two entries reviewed by a person, with a test preventing a third from appearing +without review, is a smaller claim that is actually true. + +### Three things a review caught, all of them about reading + +**A summary sentence is not a per-platform claim.** Dublin Pearse's field opens "Via ramps, +stairs, escalators, and lifts." A lift, no platform number. Counted as covering the station, +that made the page's own "Ramp to platform 1" into a lift platform, so a notice about platform +1 would have published "Platform 1 is reached by lift". The rule is now that **specific beats +general**: a segment naming a lift with no platform covers every platform only when no other +segment names one. + +**Template text invents lifts.** This one is a trap with teeth: + +> "To access the lift, you must call via the help point at each landing of the lift shaft. +> Please see lift call operation page for steps to call the lift." + +Pasted verbatim at dozens of stations. At **three** of them (Greystones, Killiney, Donabate) it +is the *only* mention of a lift, so matching on the word "lift" invents lifts nobody claimed. It +is stripped before anything else runs. The arithmetic is visible in the numbers: 61 stations +mention "lift" in the raw prose, 58 still do after the boilerplate is stripped, and 57 are +recorded as having one once Dromod's explicit "(no lift at this station)" is honoured. + +Stripping it also dissolves a contradiction. Greystones' page says "Footbridge **only** to +platform 2" and elsewhere carries the lift-call boilerplate. Those cannot both be true. Once +the template is gone the real claim stands alone, and Greystones is recorded as not mentioning +a lift, which is why its two notices come back `unknown` rather than resolved. + +**A reviewed entry expires with the page it quotes.** `STEP_FREE_ALTERNATIVES` cites a +sentence, and these pages are refetched monthly because Irish Rail rewords them. If the +sentence is gone, the entry stops applying, and the verdict becomes **`unknown`, not `lost`**. + +> **Concept: an inference that expires with its source.** A hand-reviewed exception is a +> statement about a document at a moment: *on 30 August 2026, Irish Rail's page for Raheny said +> there is a ramp to platform 1*. When the document changes, that statement is no longer +> supported, and the honest move is to stop making it. The subtle part is what to fall back to. +> Falling back to "access was lost" would be treating a reworded page as evidence there is no +> ramp, which it is not: the review found a way round, and a rewrite is silence, not +> contradiction. So it falls back to "unknown" and a test fails loudly so somebody reads the +> page again. The general rule: when a claim's evidence disappears, retract the claim, do not +> invert it. + +### Where the two sources disagree, the site says so + +The most useful thing the derivation does is refuse to answer. Measured on the corpus as of +31 August 2026, of 24 notices: **16 resolve to "step-free access was lost"**, 2 are escalators, +and **6 come back `unknown`**. A quarter of everything on the site. + +Every one of the six is a real discrepancy between two hand-written sources, and none of them +is a parsing failure: + +| station | why | +|---|---| +| Limerick Junction | the access field is the single word "Level", yet the station has lift notices, and OpenStreetMap maps two lifts | +| Greystones (twice) | the prose names no lift outside the stripped boilerplate | +| Rush and Lusk | the prose reads "Level access to platform 1 / Lift and footbridge to platform 1": platform 1 twice, plainly a typo, and the notice names platform 2 | +| Portlaoise | the prose puts the lift at platform 1; the notice says platform 2 | +| Carlow | the prose puts the lift at platform 2; the notice says platform 1 | + +Papering over any of these means inventing a fact. Printing "unknown" costs the site a row of +confident prose and keeps it truthful. + +One refinement matters here, because the first version of it was too blunt. A notice naming +**more** platforms than the page accounts for used to forfeit everything it knew. Athy's notice +names platforms 1 and 2; the page has a lift at 2 and calls 1 level. Platform 2 is still +knowable, so the verdict keeps it and says the notice also named a platform the page does not +list a lift at. Partition what you know from what you do not, rather than discarding both. + +### The pill, and what it deliberately does not say + +The two exception stations get a small green pill on their row and their page: **"Step-free +route"**, with the card underneath quoting the line that earned it. + +It deliberately does not say "accessible station" and deliberately does not use the +international access symbol. Both would read as a far bigger claim than the reviewed list +makes. What the list actually says is only this: *Irish Rail's page names a step-free way to a +platform here that does not use the lift.* That is a narrow, checkable statement and the label +has to stay inside it. + +Neither Raheny nor Cork has ever had a notice in the corpus, so the pill has never rendered on +the live site. It is covered by tests rather than left to be discovered the first time it +matters. + +### The site says it is an inference, and asks to be corrected + +Every access line on this site is worked out from a page somebody typed. This project has +already found, in that source: a typo (Rush and Lusk, platform 1 twice), a self-contradiction +(Greystones), a station whose page says "Level" while its lifts break (Limerick Junction), and +an escalator omitted from the field that should carry it (Dublin Connolly). Presenting derived +sentences in a confident voice on top of that would be claiming more than is known. + +So the card carries a caveat in the site's own words: worked out from Irish Rail's page, +written by hand, wrong before, a careful reading rather than a survey, and blind to whatever +the page leaves out. + +And then it asks. A static site has no feedback channel, so the caveat ends in a prefilled +GitHub issue link: *"Know this station? Tell us what this gets wrong."* + +That is not a politeness. People who use these stations know things no source in chapter 06 +records, and a filed issue is auditable in the way this project asks every other claim to be. +It is also the only route by which a fact that exists nowhere machine-readable can ever reach +the site. + +It has never fired. Passive crowdsourcing yields nothing, which is the finding that eventually +produced chapter 15's generated questionnaire: the channel was right and leaving it open for +somebody to notice was not. + +## Worked example: what Hazelhatch actually publishes + +The station that started the chapter, as the site renders it today: + +- **The prose:** "All platforms can be accessed via lifts and ramps." +- **The notice:** a lift out of service, listed 48.5 hours in August. +- **The verdict:** step-free access to the platforms was gone while the notice was listed. +- **What it does not say:** how many lifts Hazelhatch has, whether the one named was the only + one, or whether anybody was actually stranded. + +The first version of this branch would have published the opposite of the third line, from the +same input, by treating one word as a disjunction. The distance between the two readings is one +conjunction and a station somebody has been to. + +## Where it left the site + +18 of 24 notices carry a worked-out consequence, 6 say "unknown" and say why, two stations +carry a narrow green pill, and every derived line on the site is labelled as an inference with +a link to correct it. + +Two things were left open on purpose, and both are chapter 09: the grade still counts escalator +days at full weight, on the same page where an escalator notice is told it removed nothing, and +the derivation reasons about only one leg of the journey. Both were closed on 3 September, in +chapters 11 and 12, and chapter 12 also carries the thing this chapter's caveat gestures at +without measuring: a written account of how far the derivation can be trusted. + +Before that, chapter 08 is about what the review of this branch found, which was the same bug +three times. + +## Notes + +- PR #30, "Say what a lift outage did to step-free access" (30 Aug 2026). +- `notes/station-access.md` §§ "and" is a sequence not a choice, The chip, Three things a review + caught, It is an inference and the page says so, The lift-call sentence is boilerplate, When + it says "unknown" (all 30 Aug 2026). +- The 61-station breakdown (29 sequences, 11 "or stairs", 2 alternatives) is from + `notes/station-access.md`, 30 Aug 2026. +- Re-measured 31 Aug 2026 against `../lifts-data`: 61 stations mention "lift" in the raw prose, + 58 after boilerplate stripping, 57 recorded as having one; verdicts across 24 notices are 16 + lost, 6 unknown, 2 escalator; the six unknown rows are as tabulated, from + `python -m lift_access report`. +- `lift_access/model.py`: `STEP_FREE_ALTERNATIVES`, `BOILERPLATE`, `read_platform_access`, + `verdict`. `tests/test_site_real.py` holds the exception-list guard. +- Hazelhatch's listing duration (48.5 hours) measured 31 Aug 2026. +- Diagram: `diagrams/and-is-a-sequence.svg`. diff --git a/writing/chapters/08-the-same-bug-three-times.md b/writing/chapters/08-the-same-bug-three-times.md new file mode 100644 index 0000000..1343d2d --- /dev/null +++ b/writing/chapters/08-the-same-bug-three-times.md @@ -0,0 +1,204 @@ +# 08. The same bug, three times +*~9 min read · PR #30's reviews and PR #34 · 30 August 2026* + +*Where we are:* the access derivation from chapter 07 is written and reviewed. This chapter is +about what the reviews found, which turned out to be one bug wearing several coats. + +## The question that opened this stretch + +Two code reviews ran over the access branch. The second one found three defects, and reading +them together produced an uncomfortable observation: + +**All three were the first review's findings, reappearing in the code written to fix them.** + +That is worth recording as a habit rather than as three bugs, because a habit can be looked for +and three bugs cannot. + +## What changed + +### The three repeats + +- **A stale reviewed entry forfeited every other platform.** The first review found that a + notice naming platforms the page does not list a lift at was discarding the platforms it + *did* know, and it was fixed by partitioning the known from the unknown (chapter 07, Athy). + The staleness check added afterwards, which handles a reviewed exception whose sentence has + been reworded, repeated the same mistake: a reworded Cork page would have taken platform 7 + down along with 5A. Same partition, applied again. +- **A caveat was shown where no derivation ran.** The first review found the step-free pill + rendering for a station absent from the snapshot. The "this is an inference" caveat added + afterwards was a single global flag and did exactly the same thing, so "worked out from Irish + Rail's page" could sit directly above "this station is not in the station snapshot". It is + gated per station now. +- **The correction link pointed at pages that do not exist.** The prefilled issue link was + slugged from the snapshot's station name, while the site's pages are named from the newest + notice's name. Those differ at Clondalkin and Hazelhatch. Both the slug and the issue title + now come from what the page is actually called. + +And one that was nobody's fault twice over: **a notice naming both machines read as unknown.** +`classify` puts lift first, so "Lifts and escalators out of order" is a lift notice whose text +names an escalator, and a guard designed to catch a mis-headed notice fired on it. The worst +case for a reader became the least informative verdict. The guard now distinguishes a text +naming only the *other* machine, which means the head is probably wrong, from one naming both, +which is a combined outage: whatever else broke, the lift is out, so the platforms are lost. + +### The shape underneath them + +Strip the specifics and the same defect is in all of them, and in two more found later. Call it +what it is: + +> **Concept: a guard that passes because what it checks is absent.** A guard is a predicate over +> some quantity: *every station in the snapshot has a verdict*, *the replay recovered rows*, +> *this alert reached the phone*. The failure mode is not the predicate being false. It is the +> predicate being computed over a set that is **empty or wrong**, so it comes out true +> vacuously, at precisely the moment the thing it was written to protect has gone missing. +> "All 38 stations I fetched are present" is true when only 38 of 152 were fetched. "Nothing +> was left to replay" is true when the directory is wrong. Every one of these passes cleanly, +> logs nothing alarming, and exits zero. The check for it is a question, asked of every guard: +> *what does this assert when the input is missing entirely?* If the answer is "success", the +> predicate is over the wrong quantity. + +Once that shape had a name, the rest of the codebase was audited for it, and it turned up twice +more, both in the collector and neither with anything to do with the site. + +### A partial fetch shadowed the last good snapshot + +`latest_snapshot` reads the newest file in the station directory. So a snapshot written during +an irishrail.ie wobble, holding 38 empty bodies out of 152, permanently shadows the last good +one, and the damage is invisible: those stations lose their verdicts to "not in the station +snapshot" and the denominator quietly shrinks. Nobody spots 38 empty bodies in an 8 MB diff. + +Transient failures are retried, and if any station is still missing, **nothing is written at +all**. A run whose entire job is to report which stations it could reach must not half-succeed +into the same filename slot as a full success. + +### `rebuild` emptied the database and reported success + +This one is the cleanest instance and the most alarming. + +`reset_derived_tables()` runs before the first line of the log is read, so the wipe is +unconditional. If there is nothing to replay, because `--data-dir` points at the wrong place, +or a drive is not mounted, or `raw/` has been renamed, then the guard on "recovered zero rows" +was a printed note rather than a protection, and the exit code said success. + +Measured against a copy of the real corpus: + +``` +before: messages: 228 runs: 1012 +rebuilt from 0 recorded run(s) +nothing to replay: no raw logs found +rebuild exit code: 0 +after: messages: 0 runs: 0 +``` + +It knew the logs were missing. It said so. It had already destroyed the tables, and it exited +zero. + +The invariant from chapter 01 makes this recoverable rather than fatal: the raw log is the +source of truth and the database is disposable, so re-running `rebuild` against the right +directory restores everything. What is **not** recoverable is an exit code that says the +rebuild worked. In CI the site build happens to catch it downstream, because a build with no +outages returns a failure, but nothing else does. + +The fix used something already there. `reset_derived_tables` deliberately runs inside the +caller's transaction, with a comment saying it does so a failed rebuild takes the wipe back. So +the fix is to notice that and use it: **a replay that recovered nothing, from a database that +had history in it, is refused and rolled back.** Returning 1 unconditionally would have been +wrong, because a fresh install has nothing to replay and nothing to lose, and still exits zero. +A log of nothing but truncated lines is refused the same way. + +### The alert window opened on the attempt, not the delivery + +The collector alerts to a phone when a run fails, and suppresses a repeat of an identical alert +for 24 hours so a stuck fault does not push every 30 minutes. + +The suppression marker was written **before** the webhook was tried: + +``` +delivered: False +marker on disk: {"digest": "32465c90...", "sent_at": 1788105365.6} +second attempt suppressed: True +``` + +So one transient failure at the moment the collector first breaks buys a full day of silence, +and that first attempt is exactly where silence costs most: the API key rotates without notice, +and a silent collector loses data that cannot be recovered later. + +The module is otherwise carefully fail-open, with every read error meaning "send anyway". This +was the one path that failed closed. The value needed was already in hand, since `notify` +returns whether it was delivered and the caller already consumes that, so the marker is now +written only once the webhook has taken it. + +### One that was left, on purpose + +`has_lift` tests the explicit denial before the claim, so a hypothetical per-platform "no lift +on this side" would silence a lift the same page claims elsewhere. Fixing it properly means +making the denial per-platform, which is a modelling decision with no data to design against: +Dromod is the only station using the phrase and it genuinely has no lift. It fails to +`unknown` rather than to a false claim, which is the safe direction from chapter 07. Left until +something real turns up, and written down so the next person does not rediscover it. + +Two others were latent, with no instance in the data, and were taken anyway because the fix was +a token each and the failure would have been silent: a block tag carrying an attribute, the +`
` a CMS paste produces, missed the separator pattern and joined two access lines +into one, which manufactures a false *specific* and defeats specific-beats-general from the +other side; and a response body that is not UTF-8, or an index that is an HTML error page, +raises an exception that neither of the two handlers catches, aborting a run whose entire job +is to report which stations it could not get. + +### The one that was carried, measured, and removed + +The same discipline killed a feature I liked. + +OpenStreetMap was carried as a second opinion on Irish Rail's prose. It is the only +machine-readable station graph that exists for this network: around Dublin Pearse there are +named platform ways, four lift nodes carrying floor-level tags, escalators tagged as conveying +steps, corridors, wheelchair tags. A routable topology, which is the thing `pathways.txt` would +have been. It also spots 13 stations where the prose mentions no lift and OSM maps one, +Limerick Junction among them. + +Three measurements, all against the real data, ended it: + +1. **It changed no verdict.** The lift check was consulted in exactly one place, as a test for + "not yes", and OSM could only move a station from "no" to "unknown". Both fail that test. + Checked with a synthetic digest mapping a lift at all 152 stations: 24 outages, **0 verdicts + changed**. +2. **Its one signal was redundant.** A station in those 13 that has a notice already returns + `unknown` without it. Limerick Junction: same verdict, same wording, with or without. +3. **It could not answer the street-side question**, which is the only thing that would have + earned its keep. "Which platform is reachable without a lift" needs floor-level tags on + platforms. Sampled over the 12 stations that have had notices: 12 of 12 had platforms + mapped, and **2 of 12** carried a level tag, both of them Dublin termini. + +Irish Rail's prose answers that same question at 32 of 57 stations, in words. + +So it went: about 60 lines, a monthly HTTP budget against a service that rate-limits, and the +one place the raw-artefact invariant had to be bent, since the map extracts run to roughly +450 MB and the digest had to be derived rather than stored verbatim. For nothing that reached a +reader. The note records all of it at length so nobody adds it back on the same hunch, because +the hunch is a good one. + +## Where it left the site + +No behaviour change a reader can see, which is the point. A collector that refuses to report +success after destroying its own tables, an alerting path that cannot be silenced by one failed +delivery, a fetch that is all-or-nothing, and a feature removed on measurements rather than +taste. + +This chapter has no sibling contrast. The other two series have their own review stories and +their own bugs, and none of them is this one. + +## Notes + +- PR #30's second review, recorded in `notes/station-access.md` § What the second review caught + (30 Aug 2026): the three repeats, the both-machines notice, the two latent fixes and the one + deliberately left. +- `notes/station-access.md` § Three things a review caught (30 Aug 2026): the summary-sentence + rule, the expiring reviewed entry, and the partial-fetch refusal. +- PR #34, "Refuse a rebuild that recovered nothing, and time the alert window from delivery" + (30 Aug 2026): both transcripts quoted above, `tests/test_alert.py` (new; the repeat window + had no coverage at all), two new cases in `tests/test_rebuild.py`, all four failing against + the unfixed source. +- `notes/station-access.md` § OpenStreetMap: carried, measured, removed (30 Aug 2026), and + issue #29, closed. The 32-of-57 figure it is compared against is from the same note. +- The audit that produced PR #34 was prompted by the shape turning up three times in one review + round, which is stated in PR #34's own body. diff --git a/writing/chapters/09-what-one-letter-cannot-say.md b/writing/chapters/09-what-one-letter-cannot-say.md new file mode 100644 index 0000000..b587efe --- /dev/null +++ b/writing/chapters/09-what-one-letter-cannot-say.md @@ -0,0 +1,231 @@ +# 09. What one letter cannot say +*~11 min read · issues #28, #31, #32, #33 · open as of 31 August 2026, all closed by 3 September* + +*Where we are:* the site grades stations (chapters 04 and 05) and says what each outage did to +step-free access (chapter 07). This chapter is about the four things still open, and it is +written as reasoning rather than as a backlog, because the reasoning is the part worth reading. + +> **All four closed within four days of this being written**, between 1 and 3 September 2026. +> The chapter is kept as the argument stood on 31 August, because the arguments are what the +> issues were for and each one shaped what got built: chapter 11 answers #28 and #32, and +> chapter 12 answers #31 and #33. Where a decision has since gone a particular way, a line says +> so and points forward. The figures here are as measured on 31 August, before the listing +> split of chapter 10 moved several of them. + +## The sharpest one: an F beside a sentence saying access was fine + +Open Dublin Pearse's page today and you can read both of these: + +> **F · 20% available** + +> *An escalator is moving stairs, so it was not a step-free route to begin with and its being +> out did not remove one.* + +Pearse's August notices are a lift at platform 2 for five days, which is inside the +planned-works grace and costs nothing, and an escalator at platform 2 for sixteen days when the +issue was filed and nineteen as measured on 31 August, which overran it. **The F is entirely escalator-driven.** As far as this feed shows, the step-free +route at Pearse has been fine since 13 August. + +I first wrote this issue claiming the two statements contradict each other in public. They do +not, and the correction is worth stating because it changes what the fix has to be. The site's +own explainer is accurate: *"Availability is the share of the days watched on which nothing was +reported out at this station, no lift and no escalator"*, under a heading that reads "Lift and +escalator availability". Every word of that is true and the grade measures exactly what it says +it measures. + +The problem is narrower and harder. + +> **Concept: one number, two populations.** A grade compresses a measurement into a letter so it +> can be compared at a glance, and the compression is only honest if everyone reading it wants +> the same question answered. Here two readers want different questions answered by the same +> chip. A wheelchair user wants to know whether they could get to a platform, and for them +> Pearse's August was fine. Somebody with a heart condition, a stick, a pram or a suitcase +> wants to know whether they faced a flight of stairs, and for them Pearse's August was bad. +> One letter cannot answer both, and the fine print that reconciles them is not what people +> read. The chip is what people read. So the failure is not inaccuracy, it is that a single +> compressed number is being asked to serve two audiences whose answers genuinely differ. + +Measured on the corpus in the issue (30 Aug 2026): + +| | grade as shipped | lift notices only | +|---|---|---| +| Dublin Connolly | **C** (91%) | A (100%) | +| Dublin Pearse | **F** (21%) | A (100%) | +| national figure | 67% | 70% | + +### The options, and why the obvious one is refused + +1. **Escalators show but stop knocking.** They already have their own bar. The grade becomes + step-free availability and the explainer says so. Pearse goes F to A, Connolly C to A, the + national figure 67% to 70%. This makes the grade **narrower and more honest** rather than + more forgiving: it stops claiming to measure something it measures badly. +2. **Weight escalators at some fraction.** Refused. There is nothing to calibrate a coefficient + against, and it would turn the grade into a number nobody can reconstruct by counting days, + which is the one property chapter 04 built the bands to have. +3. **Keep the grade, change the wording.** Cheapest, and the wording is already accurate, so it + fixes almost nothing. +4. **Two grades.** The most accurate and the most complexity, and the site has been deliberate + about carrying one number. + +Option 1 is the recommendation, and it is what shipped on 3 September (chapter 11). The +precedent is the water site. Its `KNOCK_CATS` is +**binary**: health-relevant quality notices knock the grade, discolouration shows on the bar and +does not. No coefficient. That is the honest way to say "this matters less" without inventing a +number calibrated against nothing. + +Which completes the three-way picture on the question the whole back half of this series turns +on: + +| | what is allowed to knock the grade | who decided | +|---|---|---| +| **uisce** (water) | health-relevant notices knock; discolouration shows and does not | the author, on a binary distinction the data supports | +| **esb** (power) | planned works excluded; storm days kept, and said out loud | the regulator excludes planned works, so the site follows; nothing identifies a storm day in the feed, so the site keeps them and states the difference | +| **lifts** | planned works excused a week then counted; escalators knock, and whether they should is **open** | nobody decided anything on our behalf, so every exclusion has to be argued from scratch | + +Nobody excluded anything for this site. That is the whole difficulty: the power site could +inherit a regulator's exclusions, and the water site had a clean binary in its own data. Here +there is neither, and the argument has to be made in public. + +### What makes option 1 honest rather than a dodge + +Dropping escalators from the number and saying nothing else would be a real loss of +information. An escalator outage stops somebody. It is not only about wheelchair access: +elderly passengers, people with a heart or lung condition, luggage, a pram or a stick can be +genuinely stopped by a flight of stairs, and the site currently has no vocabulary for that +group at all. + +So the fix is paired with **saying who an escalator outage did affect**, which is issue #33, +and the two should land together. They landed ten hours apart, which is close enough: chapter +11 is #32 and chapter 12 is #33. + +That is derivable rather than guessable: whether the platforms the escalator served still had a +lift, a ramp or a level route. Run against both cases on record, nobody was stranded. But +establishing that takes a field the derivation does not currently read, which is the other half +of #33. + +## The journey has two legs, and the model sees one + +`platformAccess` starts at the ticket office. How you get from the street to the concourse is a +**separate field**, `ticketOfficeAccess`, and the derivation does not reason about it. + +Connolly makes the gap concrete. Its notice is "The Escalator at **the main concourse**", which +is the entrance leg, and Connolly's escalator is named in `ticketOfficeAccess` and nowhere else: + +> "Escalator, lift or stairs from Amiens Street and from LUAS stop. Level access from car park." + +Checking only the platform field published "Irish Rail's page for Dublin Connolly does not +mention an escalator" at the one station where that line rendered, and it was false. Chapter +07's branch fixed the *mention check* by reading both fields, and the station page now quotes +both, labelled, because a lift or an escalator can be on either leg. + +But the **derivation still models only the platform leg**, and that is a real limit rather than +a tidy one: a lift outage at a station entrance would be reasoned about against prose describing +a different part of the building. No notice on record is of that shape, which is why it is an +issue rather than a bug, and `ticketOfficeAccess` is present at 143 of 152 stations, so there is +something to work with. + +A rule worth writing down before it is needed, from the same issue: if an escalator is ever the +**only powered way up** at a station, its outage leaves stairs only, and that is a genuine loss +for exactly the group above, so it should knock whatever else is decided. There are zero such +stations today. Only Dublin Pearse and Tara Street name an escalator in `platformAccess`, +Connolly names one in `ticketOfficeAccess`, and all three also have lifts. + +## The largest unclaimed win + +A lift out does not strand a station. It strands a **platform**. And the other platform is +frequently at street or car park level and needs no lift at all, and Irish Rail's prose says +which one in plain words: + +| station | lift serves | still step-free | +|---|---|---| +| Athy | platform 2 | "Level to platform 1" | +| Malahide | platform 2 | "Level to platform 1 (City Centre)" | +| Portlaoise | platform 1 | "Level to platform 2" | +| Skerries | platform 2 | "Level to platform 1" | +| Dublin Pearse | platform 2 | "Ramp to platform 1 (City Centre and northbound)" | +| Dublin Connolly | platforms 6, 7 | "Level access to platforms 1, 2, 3 and 4 from ticket office" | + +**32 of the 57 stations that claim a lift name at least one platform reached without one, and +12 of the 21 that have had a notice do.** That is an order of magnitude more than +`STEP_FREE_ALTERNATIVES`, which has two entries. + +It belongs in an archive because it bounds how bad a recorded outage was. "The lift to platform +2 was out for twelve days" and "the lift to platform 2 was out for twelve days, and platform 1 +was level throughout" describe different events, and the site cannot currently tell them apart. + +It is also safe to derive, unlike the connectives that caused the Hazelhatch misreading. +"Level to platform 1" is a direct statement, not an inference, and the model already works out +which platforms the lift serves, so the complement falls out of what is there. No new source, +no labelling, no hand-maintained file. + +The wording has to keep two claims apart, and this is the part that would be easy to get wrong: +`STEP_FREE_ALTERNATIVES` says *"you can still reach this platform another way"*, same platform, +alternative route. This says *"that platform was unreachable, this one was not"*, a different +platform and therefore a different train. The second is weaker and must not be dressed as the +first. + +### And what is refused + +The obvious next step is to label which direction each platform faces, so the site could say +"you could still travel towards the city". It should not be taken, and the reason is a +statement about what this site is. + +**This is an outage archive, not a travel planner.** A reader here is looking at what happened, +not deciding which train to catch, and a direction label only pays off for somebody planning a +journey. + +It is also the expensive kind of fact. Only 10 of 57 stations name a direction in their prose, +and no source checked has the rest: chapter 06's GTFS carries no platform data for Irish Rail +at all, OpenStreetMap has no floor-level tags outside the Dublin termini, NaPTAN's `AccessArea` +is null on all 152 rail stops. So it would be roughly 120 platforms labelled **by hand**, which +is precisely the unprovenanced second source the scoping note warns against, bought for a +question this site does not answer. + +## The small one + +Issue #28: the lift and escalator strips on a bar are not labelled on the overview row, so a +reader cannot tell which is which without opening the station. Labelling was tried and rejected +once, because the label column shortened that one station's bar and knocked its day cells out +of line with every other row's. The labels stay on the drill-down, where the bars are tall. A +better answer has not been found yet, and with escalators possibly leaving the grade it may +change shape entirely. + +## Where that leaves the site + +Four open questions, three of which are about the same thing from different angles: the site +knows more about what a lift outage did than it is currently saying, and the one number on the +front is carrying more weight than one number can. + +None of them is a bug. All of them are decisions that were made correctly against the question +being asked at the time, and that a later question has made uncomfortable. That is what the +`notes/` directory is for, and it is why these are issues with the numbers in them rather than +todos. + +### What actually happened next + +All four closed between 1 and 3 September, and the order surprised me. **#28, the one dismissed +above as "the small one", turned out to be the blocker.** The argument that had kept escalators +in the grade was never really about what a grade should measure; it was that Connolly's row gave +a reader no way to tell whose red cells those were. Fixing the row removed the argument. That is +chapter 11. + +#31 and #33 landed the same day and are chapter 12, along with something neither issue asked +for: a written account of how far any of this derivation can be trusted. + +## Notes + +- Issue #32, "The grade and the station page disagree about escalators" (30 Aug 2026): + the correction to its own framing, the measured table, the four options, the `KNOCK_CATS` + precedent, and the only-powered-way-up rule. +- Issue #33, "Model the entrance leg, and say who an escalator outage affected" (30 Aug 2026): + the two fields, the Connolly quotation, `ticketOfficeAccess` present at 143 of 152. +- Issue #31, "Say which platform was still step-free when a lift was out" (30 Aug 2026): the + six-station table, 32 of 57 and 12 of 21, the wording distinction, and the struck direction + labelling. +- Issue #28, "Lift and escalator bars are not disambiguated in the bar views" (29 Aug 2026); + the rejected label column is in `notes/site.md` § One bar per kind. +- `notes/station-access.md` §§ Escalators are not step-free, The other platform is often still + step-free (30 Aug 2026). +- Pearse at F / 21% and the national 67% against 70% are as measured in issue #32 on + 30 Aug 2026. Re-measured 31 Aug 2026 the same figures read Pearse F / 20% and 67% aggregate; + the corpus grew a day in between and the comparison is unchanged. diff --git a/writing/chapters/10-two-ways-the-page-lied-about-time.md b/writing/chapters/10-two-ways-the-page-lied-about-time.md new file mode 100644 index 0000000..544a1fc --- /dev/null +++ b/writing/chapters/10-two-ways-the-page-lied-about-time.md @@ -0,0 +1,213 @@ +# 10. Two ways the page lied about time +*~10 min read · PRs #39, #42 and #44 · 2 to 3 September 2026* + +*Where we are:* chapter 09 left four open questions about what the site *says*. This chapter is +about two things it was saying wrongly, both about time, and neither of them the collector's +fault. + +## The first: "collection has stopped", when it had not + +The page read: + +> Updated 22 hours ago - collection has stopped + +while `lifts-data` held data from nine hours earlier. Collection had not stopped. **The site +build had.** + +The stale banner's whole design (chapter 03) is that it states the age of the data and names no +cause, because from a browser a stalled build and a stalled collector look identical. That was +the right call and it was still not enough, because 22 hours of age is a genuine problem +whoever caused it. + +### GitHub's scheduled runs were four to ten hours late, every day + +The build asked for two crons, at 05:40 and 12:40 UTC. What actually ran, over the week to +1 September: + +| Cron | Actual run start, UTC | +|---|---| +| `40 5` | 10:24, 10:39, 11:46, 11:48, 13:38, 16:46, 17:41 | +| `40 12` | 16:51, 16:53, 16:56, 19:13, 22:33, 22:36 | + +The morning slot never once landed in the morning. This is not jitter: before 26 August, with a +single cron, runs started between 05:58 and 06:06Z, which is the 18 to 26 minutes GitHub +documents. Push-triggered and dispatch-triggered runs are unaffected and land within seconds. +Only `schedule` events are throttled this way. + +That produces exactly the string on the page. The 10:24Z build saw data to 23:22Z the previous +night. The 16:53Z build saw data to 11:19Z. Nothing built after it, so by 09:47 the next +morning the horizon was 21.5 hours old. + +**Retiming the cron was rejected**, and the reason is worth stating plainly: a delay of four to +ten hours cannot be aimed at a one-hour window. A better cron time only moves where the miss +lands. + +> **Concept: the age on the page is the age of the data, not of the build.** This is what makes +> the obvious fix wrong. The freshness chip measures from the collection horizon, the last +> moment a run actually reached the feed, against the reader's clock. Rebuilding the site later +> cannot make that number smaller: the data is as old as it is. So only two things move it, +> pushing the data more often, and building promptly after a push the site has not yet seen. +> A schedule does neither. It is a fallback for a trigger that never fired, and treating it as +> the primary mechanism means the page's freshness is bounded by the least reliable part of the +> chain. + +So the data repository now dispatches the site build on every push, and the site rebuilds +within a minute of the data landing. The crons stay on as the fallback, moved to 07:00 and +14:00 UTC. That choice has a small detail worth keeping: GitHub Actions cron has no timezone +and Dublin shifts by an hour with daylight saving, so for a one-hour local window only the hour +boundaries land inside it in both seasons. + +The Pi's push cadence went from twice daily to every six hours, which caps how old the page can +look: about 7 hours at worst (a six-hour slot, half an hour of randomised delay, one 30-minute +poll interval) against about 13 before. And the stale threshold followed it down from 16 hours +to **10**. Sized above the widest legitimate age and below a missed push, exactly as before, +just against a different cadence. + +One deployment note from that pull request is a good example of a change that is safe in the +repository and unsafe in the world: the new threshold had to reach the Pi *before* the merge, +because while the Pi was still pushing twice daily a 10-hour threshold would have shown the red +banner for the last three hours of every twelve-hour window. The fix would have caused the +symptom it was fixing. + +## The second: a notice that came back was published as never having left + +This one is worse, because nothing on the page looked wrong. + +**Portlaoise was published as 16 days listed, F, 29% available.** What actually happened: the +lift was listed for 20 hours from 10 August, then absent from **672 consecutive successful +polls** over the next fortnight, then listed again for 22 hours from 25 August. + +Two short outages a fortnight apart, published as one continuous sixteen-day one. Every one of +those 672 polls is in the raw log, exactly as collected. The site simply never looked. + +### Where the gap fell out + +Chapter 01's derived identity comes back one more time, and this is the sharpest edge on it. + +`identity_key` is `UNIQUE` on the messages table, which is what makes a notice the same notice +across polls. So when a notice came back, the collector had nowhere to put the second +appearance except the row already there: it cleared the closing timestamp, incremented a +reopen counter, and left the first-seen timestamp alone. The site then read one row as one +listing, first seen to closed, and **the absence in the middle vanished**. + +The reopen counter had recorded this happening six times in the first month, five of them lift +or escalator notices. Nothing downstream read it. No test covered it. + +That last part is the interesting bit, because there *was* a test that looked like it did: +`test_a_notice_that_comes_back_a_poll_later_is_a_separate_outage`. It gives the returning +notice a corrected start time, which changes the identity key, which makes a genuinely new row. +So it exercised the case where the collector's own mechanism produces the right answer for +free, and never the common case, where it does not. + +> **Concept: a test that exercises the easy half.** A test named for a behaviour is not evidence +> the behaviour holds. This one passed for years of commits while the thing it was named after +> was broken, because its fixture happened to take a path where the bug cannot occur. That is +> not a badly written test, it is a badly chosen input: the returning notice was given a +> *changed* start, which is the rare case, and the unchanged return, which is what actually +> happens, was never run. The general check is to ask what the fixture makes true incidentally, +> and whether the code under test would still be exercised if that incidental thing were +> removed. The related habit from chapter 08 is the same family: a predicate is only as good as +> the quantity it is computed over, and a test is only as good as the input it is computed on. + +This was never a raw-log problem, which is the whole point of chapter 01's invariant. Every gap +is in the JSONL exactly as collected. The fix is a `rebuild`, not a correction anybody has to +write down and remember. + +### One row per stretch + +A **listings** table now holds one row per stretch a notice was continuously on the feed. A +reopen opens a new row rather than reviving the old one, and the site builds one outage per +row. The messages table keeps its own first-seen and closed timestamps, which still answer +"when did we first ever see this notice", a different question and not the one the bars ask. + +Two alternatives lost. **Deriving the gaps in the site**, by walking runs against each notice's +last-seen timestamp, would have left the schema alone but puts the collector's knowledge in the +renderer and needs a scan of every run per notice. The collector is the thing that watches the +feed; when a notice stopped being on it is the collector's fact to record. And **dropping the +uniqueness constraint** to insert a fresh row per appearance would have duplicated every +notice's text and start across every stretch, for no gain, and the identity key is what makes +reissue detection work at all. + +### Splitting the listing exposed a second flaw + +Athy's lift has been listed continuously since collection began. It was absent from **exactly +one poll** on 21 August and back 29 minutes later. + +With the grace at one miss, that closed and reopened the notice, and once listings were split, +the site published two outages: one "no longer listed 21 Aug", another starting half an hour +later. That reads as fixed, then broken again, which is the one claim this site must never +make. + +The default grace is **2** now. The measurement that justifies it is the shape of the gaps in +the corpus, which sort into 1, 9, 79, 388 and 672 polls. There is nothing near the cut on +either side. The second miss only confirms the close, since the closing timestamp is still the +first poll the notice was absent from, so nothing measured moved. It is set in code rather than +in the environment file, because `rebuild` replays under whatever value is set when it runs and +a replay must not depend on a machine's local configuration. + +### Worked example: the grace week that a four-hour gap would have refreshed + +Splitting one notice into several stretches interacts with the planned-works grace from +chapter 04, and the interaction is not obvious. + +The grace forgives works listed seven days or less **in total**. Split the listing and each +stretch is now its own outage, so a notice that ran six days, blinked out for four hours, and +ran another six would present as two six-day works, each inside the grace, each forgiven. A gap +would have laundered the works. + +So the total is pooled across all of a notice's stretches before they separate. Without it, the +Dublin Pearse escalator's four-hour blip would have handed it a fresh grace week and taken it +from 20% to 41%, with nothing at all changed about the notice. + +The review of that branch found the same idea applied twice by accident: the fold that merges +reissued notices was summing the pooled total per chain member, and a chain can hold two +stretches of one notice (A reissued as B, then reverted to A), so A's total was counted twice. +Five and a half days of works reported as nine, which crosses the grace and drops the grade. + +Pooling has one consequence nobody wrote down, because it needs a notice to come back weeks +later. The grace is recomputed every time the notice reappears, so a month that has already +ended can change its letter. Tullamore's August did, on 19 September. Chapter 17. + +## What the split moved + +| station, August 2026 | before | after | +|---|---|---| +| Portlaoise | F, 29% | **D, 83%** | +| Thurles | F, 25% | **E, 54%** | +| Clondalkin Fonthill | F | **E, 70%** | +| Dublin Pearse | F, 20% | F, 20% | +| Midleton | F, 0% | F, 0% | + +The last two rows matter as much as the first three. Midleton really was listed from the day +collection began to the end of August, in 1,087 consecutive successful polls with no gaps, and +its 8 August edge is the collection horizon rather than the outage's start. The Pearse escalator +really did come down at exactly the end date Irish Rail wrote on it, which is the one place in +this corpus where that placeholder field turned out to predict something. Chasing those two, +because they *looked* wrong, is what found the notice that was actually broken. + +One smaller fix landed in the same window. The overview's "still out when the month ended" tile +read 0 for every past month, because its test could only be true in the horizon's own month. It +counts an outage that ran past the boundary, or one still open exactly at it, and excludes a +notice that came down at the boundary poll. + +## Where it left the site + +A page that is at most a few hours behind the feed instead of up to a day, a threshold sized to +the new cadence, and a bar that shows the days a notice was actually on the feed rather than +the envelope of its first and last appearance. As of 4 September 2026 the database holds 285 +listings across 281 messages, four of which have more than one stretch. + +## Notes + +- PR #39, "Publish on the data landing, not on a cron that runs hours late" (2 Sep 2026): the + cron timing table, the dispatch-on-push change, six-hourly pushes, `STALE_AFTER` 16 h to 10 h, + and the deployment ordering. +- `notes/publish-cadence.md`, and its § The banner blamed the wrong half. +- PR #42, "Record one listing row per stretch a notice was on the feed" (3 Sep 2026): Portlaoise, + the listings table, the grace default, the pooled planned total, the back-dated span for + notices open before the table existed, and the three review findings. +- `notes/site.md` § A notice that came back was published as never having left (2 Sep 2026), + which carries the rejected alternatives. +- PR #44 and issue #36: the past-month ongoing tile. +- Measured 4 Sep 2026: 285 listings across 281 messages, 4 messages with more than one stretch; + 1,264 runs, 1,261 ok. Grade moves as tabulated are from PR #42, measured 3 Sep 2026. diff --git a/writing/chapters/11-the-grade-narrows-to-lifts.md b/writing/chapters/11-the-grade-narrows-to-lifts.md new file mode 100644 index 0000000..73dd2d9 --- /dev/null +++ b/writing/chapters/11-the-grade-narrows-to-lifts.md @@ -0,0 +1,172 @@ +# 11. The grade narrows to lifts +*~8 min read · PRs #38 and #43 · 1 to 3 September 2026* + +*Where we are:* chapter 09 set out four open questions. This chapter closes two of them, and +the order they closed in is the point: the answer to the small one is what made the big one +possible. + +## The small one first, because it unlocked the other + +Issue #28 was the least interesting thing on the list. Two bars on a row, and nothing visible +saying which was the lift and which the escalator. The accessible label said it and the day-cell +caption said it on hover, neither of which a phone shows at rest. + +It had been tried once, on 28 August, and reverted. A 64px text label appeared on the one +station that had two bars, which shortened that station's bar and put its day 14 over every +other row's day 15. The whole overview stopped lining up because of one label on one row. + +The fix is one sentence: **reserve the column on every row, whether or not there is anything to +put in it.** Every bar now sits in a wrapper whose first column is a fixed 15px carrying a +glyph, a lift or an escalator icon. On a station page, where the bars are tall, the glyph keeps +its word beside it in an 84px column. Measured across the August overview, all 21 rows now +start their day cells at the same horizontal position at both 980px and 500px, paired and +unpaired alike. + +> **Concept: a conditional column is a misalignment.** A layout element that appears only where +> it has content is invisible on the rows that lack it and disruptive on the row that has it, +> and the disruption is comparative: it does not make that row wrong, it makes every *other* +> row wrong relative to it. Reserving the space unconditionally costs 15 pixels on every row +> and buys back the property the grid exists for, which is that the same day is at the same +> place on every line. Nothing about the first attempt was wrong except that it was +> conditional. + +The two other rejected shapes are worth a line each, because both would have reintroduced +older bugs. **One merged bar on the overview, split only on the drill-down** is exactly the bug +the split exists to fix: at Pearse on 13 August the lift came back and the escalator did not, +and a merged cell paints a working lift as broken. **An automatically sized label column on +station pages** would size to "Lifts" on one month card and "Escalators" on the next, and the +bars would step sideways down the page. + +The kinds became their own legend key beside the day key, which is what closes the loop: the +day key still names no kind, so the two cannot drift. + +## The big one: Pearse's F + +Chapter 09's sharpest open question was this pair of statements, both true, both on the same +page: + +> **F · 20% available** + +> *An escalator is moving stairs, so it was not a step-free route to begin with and its being +> out did not remove one.* + +Pearse's F was entirely escalator-driven. One letter was answering two questions for two +populations, and the chip is what people read. + +### Why lifts-only was possible now and had not been on 29 August + +This is the part I like. Chapter 05 records the decision to count escalators in the grade, and +what killed the alternative was a specific reader experience: Dublin Connolly reading **A / +100%** directly above two red cells, with nothing on the row saying whose those cells were. +That was not an argument about what a grade should measure. It was an argument about what a +reader could resolve. + +Issue #28 removed it. Since every bar carries its kind glyph in a fixed gutter, and the kinds +are their own legend key, red escalator cells under a green lift chip read as two facts about +two machines rather than as a contradiction. **The objection was about the row, and the row +changed.** + +So the grade is the lift bar's alone. The escalator keeps its own bar, its colours and its +count on the summary tiles, and paints nothing on the letter. The overview sort is unchanged: +any notice up leads, so a station whose lift is fine and whose escalator is out sits at the top +of the page with an A beside a red strip, which is the correct shape for exactly that +situation. Tara Street is doing it right now. + +### The key says "Lift availability", not "step-free availability" + +Issue #32 proposed the latter and the pull request refused it, and the refusal is the most +careful thing in this chapter. + +The grade counts **notices**, not access. A lift out at Raheny or Cork still knocks it, though +Irish Rail's page names a ramp round that lift and chapter 07's reviewed exception list says so. +The seven of 30 verdicts that come back `unknown` knock it too, because the safe direction is +to count them. A name that promised step-free would claim precisely what `lift_access` spends +600 lines being careful not to claim. + +A grade driven by the access verdict was considered and rejected on three grounds, all +practical: the site builds with no station snapshot at all, a quarter of the verdicts are +unknown, and the number would then move on a monthly scrape of somebody's prose rather than on +the feed. + +### The numbers + +Measured on the corpus to 3 September, after the listings split of chapter 10, which is why +they differ from the figures in issue #32: + +| month | station | before | after | +|---|---|---|---| +| August 2026 | Dublin Connolly | C, 91% | **A, 100%** | +| August 2026 | Dublin Pearse | F, 20% | **A, 100%** | +| August 2026 | national, 21 stations | 72% | **76%** | +| September so far | Tara Street | F, 33% | **A, 100%** | +| September so far | national, 6 stations | 50% | **61%** | + +Pearse and Connolly both come out at A over visible red escalator strips, which is the shape +chapter 05 called a contradiction and chapter 11 calls two facts. The difference between those +two readings is a 15px gutter. + +### The case that should knock, written down and guarded + +There is one shape where an escalator outage genuinely should count: **an escalator at a +station whose page claims no lift**. Stairs only, and that is a real loss for the people an +escalator is for. + +No station is of that shape. Pearse, Connolly and Tara Street all claim a lift. So the rule is +in the note rather than in the code, and a real-corpus test fails the day it applies, with a +message telling the next person to build the rule rather than loosen the test. The review of +that branch caught the guard's first version testing for "the page says yes", which would have +misdiagnosed a station simply missing from the snapshot as the only-powered-way-up case; it +fails only on a page that positively claims no lift, and a missing station is a different test's +problem. + +> **Concept: a rule with no instance, written down and guarded.** There are three things you can +> do with a case the data does not currently contain: code it speculatively, ignore it, or state +> it and set a tripwire. The first invents behaviour against no example and is how the wrong +> abstraction gets built. The second means the day it arrives, nothing notices. The third writes +> the rule in prose where the reasoning is, and adds a test that fails when the case appears +> with an error message saying what to build. The test is not testing the rule, because there is +> no rule to test. It is testing the *premise* the absence of the rule rests on. + +### One decision a review turned up that nothing had recorded + +The headline denominator is still the stations named that month, so a station with only an +escalator notice sits in it at 100% lift-available. A reviewer asked whether that pads the +number, which was a fair question that nobody had written an answer to. + +Narrowing to stations with a lift notice gives 75% instead of 76% for August, and 53% instead +of 61% for September so far. It was kept, on the grounds that the tile says "across the stations +named this month" and the overview lists exactly those stations, so the headline stays the sum +of the rows a reader can see. The denominator refused back on 28 August was *every station on +the network*, which would have been invented; a station the feed did name is not. + +### What makes it honest rather than a dodge + +Taking escalators off the letter without saying anything else would be a real loss. The people +an escalator serves would go from being counted wrongly to not being counted at all. + +That is why issue #32's own text said the fix should land with issue #33, and why this pull +request shipped saying "#33 stays open and is what makes this honest rather than a dodge". It +landed ten hours later. That is chapter 12. + +## Where it left the site + +A letter that means one thing for one population, a bar that says which machine it is about, and +an escalator outage that is visible everywhere except in the grade. As of 4 September the August +grade mix is A 3, B 1, C 4, D 6, E 5, F 2 across 21 station-months, and September so far is +A 2, D 3, E 1, F 2 across 8. + +## Notes + +- PR #38, "Say which bar is lifts and which is escalators" (1 Sep 2026), closing issue #28: the + fixed gutter, the alignment measurement, the three rejected shapes, the six review findings + including three tests that passed with the feature removed, and the MDI glyph licensing. +- PR #43, "Grade on lift availability, and let an escalator notice paint its bar alone" + (3 Sep 2026), closing issue #32: the reversal, the "Lift availability" naming, the numbers + table, the only-powered-way-up rule and its guard, the denominator decision, and the four + rejected alternatives. +- `notes/site.md` § The grade is lift availability, and an escalator notice stops knocking + (3 Sep 2026). The 29 August section is marked reversed in place; the lifts-only rule under + One bar per kind is marked reinstated. +- Grade figures re-measured 4 Sep 2026 against `../lifts-data`. The before/after table is from + PR #43, measured 3 Sep 2026 on the corpus to 05:00Z. +- Verdict counts (7 unknown of 30) measured 4 Sep 2026 by `python -m lift_access report`. diff --git a/writing/chapters/12-both-legs-and-who-was-on-the-stairs.md b/writing/chapters/12-both-legs-and-who-was-on-the-stairs.md new file mode 100644 index 0000000..b170307 --- /dev/null +++ b/writing/chapters/12-both-legs-and-who-was-on-the-stairs.md @@ -0,0 +1,260 @@ +# 12. Both legs, and who was on the stairs +*~12 min read · PRs #37 and #45 · 3 September 2026* + +*Where we are:* chapter 11 took escalators off the grade and shipped saying that issue #33 is +what makes that honest rather than a dodge. This chapter is #33, and #31 with it, and it ends +with the most uncomfortable section in the series: an honest account of how much any of this +derivation can be trusted. + +## The question that opened this stretch + +Two questions, actually, and they turn out to be the same shape. + +**Which platform was still fine?** A lift out does not strand a station, it strands a platform, +and Irish Rail's prose usually says which platform never needed the lift. The site knew that +and said nothing. + +**Who lost something when an escalator stopped?** The verdict said a wheelchair user lost +nothing and stopped there, which reads as *nothing happened*. Somebody with a heart condition, +a stick, a pram or a suitcase reads that sentence about the day they could not get to their +train. + +Both are the site knowing more than it was saying. + +## What changed + +### "Platform 1 needed no lift" + +A lost verdict now carries a second sentence: + +> Platform 1 needed no lift, so it kept step-free access: "Level to platform 1". + +The rule is deliberately narrow, and every clause in it is a guard. A sentence from the station +page contributes its platforms if it **says level or ramp**, **names a platform number**, and +**mentions no lift, stairs, step, footbridge, subway, escalator, level crossing, or "from +platform"**. That last exclusion list is longer than it looks because each entry is a way a +sentence can describe a route with a step in it while still using the word "level". + +The note is then withheld wherever the two hand-written sources disagree: a platform the page +puts a lift at, a platform the notice itself names, or a general "lifts to all platforms" +claim. Only lost verdicts carry it, since it makes no sense beside an unknown one. And it quotes +a sentence re-read from the live prose at every build, so a reworded page withdraws it rather +than leaving a stale claim standing, which is chapter 07's expiring-inference rule applied +again. + +Five lost verdicts gained the note: Dublin Pearse, Dún Laoghaire, Malahide, Portarlington and +Tullamore. + +The wording is the part that took the most care, and it is deliberately **not** the exception +list's wording. Chapter 07's `STEP_FREE_ALTERNATIVES` says *"you can still reach this platform +another way"*, which is the same platform by a different route. This says *"that platform was +unreachable, this one was not"*, which is a different platform and therefore a different train. +The second is a weaker claim and must not be dressed as the first. + +And the direction labelling that an earlier draft of issue #31 proposed stayed struck, for the +reason chapter 09 gave: this is an outage archive, not a travel planner. + +### The journey has two legs, and now the notice says which one it is on + +Every verdict up to this point was derived from `platformAccess`, which describes getting from +the ticket office to the platforms. Connolly's escalator notice says "at the main concourse", +which is the *other* leg, and Connolly's escalator is named only in `ticketOfficeAccess`. A +lift notice of that shape would have been reasoned about against prose describing a different +part of the building. + +> **Concept: reading a claim against the right leg.** A station is not one place. Getting from +> the street to the concourse and getting from the concourse to the platform are separate +> journeys with separate equipment, and Irish Rail keeps them in separate fields. A derivation +> that reads only one field will, for any notice about the other, produce a confident sentence +> about the wrong half of the building. The fix is not to merge the fields, which would lose the +> distinction entirely, but to work out which leg the *notice* is about and read it against the +> matching prose. When the notice does not say, the honest output is the reading the site had +> before, not a guess. + +A notice's own text says which leg it is on. A platform number or the word "platform" is the +platform leg. Failing that, "concourse", "entrance", "booking hall", "ticket office", "ticket +hall", "car park" or "street level" is the entrance leg. A notice naming neither is unlocated +and keeps today's reading. + +A platform **wins** over an entrance word, and the reason is a nice piece of source-reading: +`platformAccess` starts at the ticket office, so "the lift from the concourse to platform 2" is +already that field's leg. The page itself has put that lift on the platform side. + +Over the 24 distinct notice texts on record: **19 platform, 1 entrance** (Connolly's), **4 +unlocated** (Malahide's "at Malahide Station", Docklands' "The lift is currently out of +service", Tullamore, and Clonsilla's "on P2", which nothing here reads as a platform). No false +entrance hits. The entrance vocabulary is built from one real example and from the words +`ticketOfficeAccess` itself uses; a notice saying "main hall" or "foyer" falls to unlocated, +which is the safe direction. + +### What is actually in the entrance field + +All 152 stations carry `ticketOfficeAccess`. Nine are blank, all Northern Ireland stations. +**Twenty-six say there is no ticket office**, which is read as the page naming no lift there, +because the field is literally how to reach an office that does not exist. Eighty-nine say +level, 21 say ramp. + +**Four name a lift**: Connolly, Clondalkin, Docklands and Grand Canal Dock. **One names an +escalator**: Connolly. + +So a lift notice on the entrance leg is read against `ticketOfficeAccess` and nothing else, +before the platform claim is consulted, because a page can put a lift on the way in and claim +none to the platforms. Where the field names a lift the verdict is lost, quoting the sentence, +with the page's own level way in quoted beside it where there is one. Where it names none, or is +blank, the verdict is unknown, saying which. + +No entrance-leg lift notice has been listed yet. This is machinery with unit coverage and no +live case, and the note says exactly that: treat it as untested until a notice exercises it. + +### The escalator sentence says who lost a way up + +The verdict now reads, in full: + +> An escalator is moving stairs, so it was not a step-free route to begin with and its being out +> did not remove one. **Anyone who finds a flight of stairs hard, or has a buggy, a suitcase or +> a stick, did lose a way up.** Irish Rail's page puts a lift on the way into the station as +> well: "Escalator, lift or stairs from Amiens Street and from LUAS stop". Irish Rail's page +> names a level way into the station: "Level access from car park". + +That is Connolly's. Three verdicts moved (Pearse, Connolly, Tara Street) and no lift verdict +did. + +The label above it changed too: **"A way up lost, not step-free access"**, with a muted border +rather than the green of a reviewed step-free alternative. The colour is doing real work there: +green would read as reassurance about a station where something was genuinely lost. + +Two careful refusals in that sentence. It **never says a lift was working**, because the feed +cannot back that: it says what Irish Rail's page names on the same leg, quoted, or that the page +names none. And where the notice and the page disagree about a platform being level, it says so +and quotes both rather than picking a side. + +### The overlap guard + +Quoting "the page puts a lift on the way to platform 2 as well" beside an escalator outage +invites a reader to conclude the lift was working. The site knows one thing about that, so it +says it. + +`render.shard` is the one place that holds all of a station's outages at once, so it tells the +verdict whether a lift notice at the same station overlapped the escalator's listing. If one +did, the quoted lift is withheld with a line saying why. Half-open intervals, or both still +listed. + +Zero overlaps on the corpus, and the near miss is instructive: **Dublin Pearse's lift listing +closed at the exact poll its escalator's opened**, at 10:30:46Z on 13 August. Touching, not +overlapping, and a real-corpus test asserts the flag against independently computed interval +arithmetic rather than against the code's own answer. + +### The golden file + +Four review passes ran over this branch. An extra-high review found nine findings, a high review +of that delta found six, a third found five, and a fourth on the final commit found four. + +**Two of the fixes were themselves regressions**, and neither was caught by a test. A broadened +negation guard, added so that Kilcoole's "Not level" would not be quoted as a level way in, +dropped Carrigaloe's and Dalkey's "platform No 1" lines. And the sentence splitter was breaking +at "No." as though it were a full stop, hiding Banteer's and Booterstown's level platforms. + +What caught both was comparing the level lines and the report across all 152 stations against +`main`, by hand. + +So that comparison became a file. `tests/fixtures/access-golden.json` pins every level line, +entrance sentence and verdict across the 152 stations and the notices on record, and a test +regenerates it in memory and diffs. A regex or a sentence that moves anything now appears as a +diff in the pull request that moved it. + +It lives in this repository rather than in the data repository, and that is a deliberate +trade-off with a cost. A refreshed station snapshot merged in `lifts-data` turns this +repository's CI red until somebody regenerates the file and reads the diff. That is the monthly +report of chapter 07 made **mandatory rather than advisory**. Skipping the check on a snapshot +mismatch was considered and rejected, because the guard would then be silently off from the +first refresh nobody regenerated after. + +That paragraph is left as it was written and two of its claims did not survive the week. The +cost was not only the monthly review: because the fixture pinned outputs but re-derived them +from whatever the corpus held, **any** reword by Irish Rail broke an unrelated build, three +times in five days. And the version built to avoid being silently off was silently off for +anybody without a data checkout. Chapter 14. + +One refinement: the first version failed on any notice the file had not seen, which is wrong. A +new notice on the feed is not a regression, and the corpus gained 21 distinct texts in 26 days. + +## How reliable is any of this, honestly + +The most valuable thing to come out of that day is not a feature. It is a dated section in the +notes answering the question a reader should be asking by now, which the pull requests kept +implying and never stated. + +**The short form: defensible as an annotation on an outage archive, with the safeguards below, +and not as anything a traveller should act on.** + +Why it is defensible at all: the alternative is not a better source, it is silence. Every +structured source is empty (chapter 06). A reading of somebody's prose beats silence only if the +error direction is controlled, and that is the entire design, which is worth listing in one +place because it is scattered across five chapters: default to "gone"; say "unknown" freely; +quote the sentence each claim rests on; carry the caveat and the correction link on every page +that makes a derived claim; and keep the grade independent of all of it. A reader can falsify +any verdict against the quoted words. Remove the quoting or the one-directional bias and it +stops being defensible. + +Then, by class of sentence, on the corpus to 3 September: + +- **"Step-free access was gone"**, 18 of 27 notices. The strongest part: one direct sentence and + one notice. Its failure mode is a wrong page, and the page has already produced a typo, a + self-contradiction and an omission. Six of 27 verdicts are unknown for exactly that reason. + That figure is honest and it is also **the reliability ceiling of the source**. +- **"Platform 1 needed no lift"** and **"Level access from car park"**. Direct statements, but + "level" is Irish Rail's word. Nobody has checked one of these against a station: distance, + gates, opening hours, whether the route is usable with a buggy in the rain. The correction + link is the only feedback channel and it has never fired. +- **The escalator sentence's second half**, 3 of 27. "The page puts a lift on the way to + platform 2 as well" is true *of the page*. What a reader infers is that the lift worked. The + overlap guard covers the one thing the site knows, but the feed is not complete, since notices + appear and vanish in batches, so a lift out that Irish Rail never posted is invisible. The + site controls its words, not the inference. +- **The entrance leg**, zero lift notices and one escalator notice. Built from one real example, + on a field that is literally about reaching the ticket office, and at 26 stations it says there + is no office, so a sixth of the network is unknown on that leg by construction. Untested + machinery. + +> **Concept: what the code's own history says about the code.** The usual evidence for a +> derivation being right is that its tests pass. Here that evidence is weak and the note says +> why: three review passes on the day this was built found nine, six and five findings in about +> a thousand lines of this kind of logic, one fix was itself a regression, and every rule tested +> against the corpus was later found to have a wording it had not seen ("platform No 1", "Not +> level", "No." mid-sentence). The unit tests mostly pin strings their own author wrote, so they +> confirm the author's model of the prose rather than the prose. The checks that caught the real +> regressions compared output across all 152 pages against the previous version. That finding +> rate is itself a measurement, it belongs in the note beside the feature, and the honest +> conclusion is to assume wording gaps remain. + +## Where it left the site + +Five lost verdicts that name a platform which kept step-free access. Three escalator verdicts +that say who lost a way up and quote what the page names on the same leg. A notice read against +the leg it is about. A golden file that turns any change in this derivation into a visible diff. +And a dated section saying how far any of it should be trusted, which is the thing I would most +want a reader of the site to see. + +As of 4 September, across 30 notices on record: 20 resolve to step-free access lost, 3 are +escalators, and 7 are unknown. + +## Notes + +- PR #37, "Add note when another platform kept step-free access" (3 Sep 2026), closing issue + #31: the contribution rule and its carve-outs, the five verdicts, the wording distinction, and + the struck direction labelling. +- PR #45, "Read a notice against the leg it names, and say who an escalator outage affected" + (3 Sep 2026), closing issue #33: leg detection, the entrance-leg reading, the escalator + sentence, the overlap guard, the golden file, and the four review passes. +- `notes/station-access.md` §§ The other platform is often still step-free, The entrance leg and + who an escalator served (3 Sep 2026), How reliable this is, honestly (3 Sep 2026). +- Leg detection over 24 distinct notice texts: 19 platform, 1 entrance, 4 unlocated, no false + entrance hits (PR #45, 3 Sep 2026). +- `ticketOfficeAccess` breakdown across 152 stations: 9 blank, 26 no ticket office, 89 level, + 21 ramp, 4 naming a lift, 1 naming an escalator (same). +- The Pearse touching-not-overlapping instant, 2026-08-13 10:30:46Z (same). +- Reliability figures are quoted from the note as measured on the corpus to 3 Sep 2026 (27 + notices, 18 lost, 6 unknown, 3 escalator). Re-measured 4 Sep 2026 the same counts read 30 + notices, 20 lost, 7 unknown, 3 escalator. +- Review finding rate (9, 6, 5, 4 across four passes; two fixes that were regressions) from + PR #45 and the reliability note. diff --git a/writing/chapters/13-what-a-reader-can-take-away.md b/writing/chapters/13-what-a-reader-can-take-away.md new file mode 100644 index 0000000..5aa7b65 --- /dev/null +++ b/writing/chapters/13-what-a-reader-can-take-away.md @@ -0,0 +1,126 @@ +# 13. What a reader can take away +*~6 min read · PRs #49 and #50 · 6 to 7 September 2026* + +*Where we are:* chapters 10 to 12 closed every question chapter 09 left open. The site is now +accurate about what it measures. This chapter is about a different question, which is whether +any of it can leave the page. + +## The question that opened this stretch + +A survey of the site against a corpus of 36 outages over 28 stations, asking one thing of each +screen: **what can a visitor actually do with this?** + +Five answers came back as "nothing", and two of them were not about visitors at all. + +## What changed + +### "Lift out", beside the name + +The most-asked question of a status site is the present tense, and the site answered it only +by implication: a station with a notice up sorted to the top, and you had to know that. + +There is now a tag beside the station's name while a notice is listed, on the overview row and +on both station headers: **"Lift out"**, **"Escalator out"** or **"Lift and escalator out"**. + +The mechanism is a small piece of housekeeping worth noting because it is the shape of a lot of +the work in this repository. The row's fourth statistic was a boolean that existed only to feed +the sort. It is now a mask of which kinds are listed at the horizon, the sort treats any nonzero +value exactly as it treated the boolean, and both renderers print the label from it. Nothing new +is stored: the site was already carrying the fact and throwing away the detail. + +Placing the tag took a second pull request the next day, and the reasoning is a good short +example of measuring rather than choosing. The tag first dropped under the name at an indent +that lined up with the name's first letter, which read as stray padding and made every row +carrying a notice taller than the rest. It is now flush with the right edge of the name column, +so the pills line up down the list. Above 780px the name column widens from 170px to 230px, +which fits an ordinary name and its tag on one line, and incidentally stops "Kilkenny +(MacDonagh)" truncating. + +780px is measured, not chosen for roundness: below about 740px the row's fixed columns plus a +31-day bar at its three-pixels-a-cell floor stop fitting the viewport. Four other placements +were rendered against the real CSS and compared before this one was settled on. + +### Atom feeds, and an outage that appears twice + +`feed.xml` carries the 50 most recent outages, and `s/.xml` sits beside each station page +with every outage that station has ever had. + +The interesting decision is the dating, because a feed reader's model of an entry is a thing +that happened once and this site's model of an outage is an interval with an uncertain end. + +> **Concept: a feed entry for something that is still happening.** A feed reader shows an entry +> at its date and never again. An outage has two moments a reader cares about, the one where it +> starts and the one where it is no longer listed, and the second is the whole point of this +> site. So an entry is dated to the appearance while the notice is up, and **moves** to the +> close once it is down, with "(no longer listed)" in the title. A subscriber sees the outage +> twice, and the second showing is the completion signal. That is a deliberate abuse of a +> feed's semantics in exchange for the one thing a subscriber actually wants, and it is only +> defensible because the title says which showing they are looking at. The feed's own date is +> the collection horizon, never the build clock, which is the same rule chapter 02 fixed for +> every other date on the site. + +### A CSV, and a link back to the source + +`outages.csv` is one row per outage the site shows: the listing instants in UTC, Irish Rail's +own dates as they were written, and a blank end while a notice is still listed. Linked from the +footer. + +And the access card now links the Irish Rail page its prose was quoted from. That is three +words of markup and it closes a gap chapter 07 left open: the caveat asks the reader to check +the reading, and until now it did not say where. + +### Two guards that are not for visitors at all + +These are the ones I would keep if I had to drop the rest, and both are chapter 08's family: +a thing that could go wrong silently, made loud. + +**`unclassified_mentions`** finds any message whose head or text mentions a lift, an elevator or +an escalator that `classify` rejects. Chapter 01's classifier matches heads of a particular +shape. If Irish Rail reworded a head, its notices would simply stop appearing on the site, and +nothing anywhere would fail. The build prints any it finds and the real-corpus test fails on +them. It finds nothing today, which is the point: it is a tripwire, not a feature. + +**`thin_days`** names any Dublin day the collector reached the feed fewer than 40 times, +including a day with none. A day with two polls paints exactly like a day with 48, and chapter +02's whole argument is that the site must not colour a day it did not watch. The horizon +handles the end of the window; this handles a hole in the middle of it. + +> **Concept: a tripwire for a silent drop.** Most tests assert that code does what it should on +> an input you wrote. These assert that a category is *empty*: no message mentions a lift and +> fails to classify, no watched day has too few polls. That kind of check is cheap, it passes +> forever, and it is worth having precisely because the failure it guards is invisible. A +> notice that stops matching does not error, it just is not there, and nobody counts the things +> that are not on a page. + +### The reword, discovered in passing + +Regenerating the golden file for this branch turned up something that is not a rendering +problem at all. + +The real-corpus test was red on `main` from data drift, and one of the causes was that +**Midleton's notice text had been rewritten under the same identity key**. Chapter 01's derived +identity is head plus location codes plus start, and it deliberately excludes the body. So when +Irish Rail rewords a live banner, the key survives, and the collector overwrites the stored text +in place. The earlier planned-works stretch now carries the later fault wording. + +That was noted and not touched, because it is its own problem. It is chapter 14. + +## Where it left the site + +A visitor can see what is out right now, subscribe to a station, download the lot as a +spreadsheet, and click through to the page a claim was read from. The build fails if a notice +stops classifying or a day was barely watched. The initial load went from 64.8 KB to 65.8 KB, +and the feeds and the CSV are off it entirely, fetched only if asked for. + +## Notes + +- PR #49, "Say which stations are out now, publish feeds and a CSV, and warn about what the page + cannot show" (6 Sep 2026): the survey that produced the five items, the `NOW_KIND` mask, the + feed dating rule, the CSV, the source link, `unclassified_mentions` and `thin_days`, and the + Midleton reword noted in passing. +- PR #50, "Align the 'Lift out' tag to the right of the name column" (7 Sep 2026): the four + rendered alternatives, the 780px measurement, and the overflow check at twelve widths. +- `notes/site.md` § What a reader can take away, and its 7 September correction to the + placement described on 5 September. +- Initial load 64.8 KB to 65.8 KB, measured in PR #49 on 6 Sep 2026. As of 12 September it is + 68.3 KB, with `outages.csv` at 22.4 KB and `feed.xml` at 43.3 KB, both on demand. diff --git a/writing/chapters/14-a-guard-that-was-guarding-the-corpus.md b/writing/chapters/14-a-guard-that-was-guarding-the-corpus.md new file mode 100644 index 0000000..2656d96 --- /dev/null +++ b/writing/chapters/14-a-guard-that-was-guarding-the-corpus.md @@ -0,0 +1,148 @@ +# 14. A guard that was guarding the corpus +*~6 min read · PR #51 · 8 September 2026* + +*Where we are:* chapter 12 built a golden file to pin the access derivation's output across all +152 stations, and described its cost as a deliberate trade-off. This chapter is about the cost +being something else entirely, and about a sentence in the notes that was written down +confidently and was wrong. + +## The question that opened this stretch + +A CI run failed on `main` with: + +``` +notice RLUSK lift: dropped +``` + +on a merge that could not have caused it. + +## What changed + +### The assumption, in writing + +Chapter 12 said the golden file's cost was that a refreshed station snapshot would turn CI red +until somebody read the diff, and called that the monthly report made mandatory rather than +advisory. That was the intended cost and it was real. + +What was not intended is that the file pinned the derivation's **outputs** while re-deriving +them from whatever the data repository held at the moment CI happened to run. So it could fail +on two *other* repositories' schedules rather than on any change to this one. + +The hole was an assumption, written into `notes/station-access.md` in exactly these terms: + +> A notice that vanishes from the database still fails, because the logs are append-only and +> that can only mean a bad checkout. + +The logs are append-only. `messages` is not. + +Chapter 01's derived identity is head plus location codes plus start, and it **excludes the +body**. So when Irish Rail rewords a live banner, the identity key survives and the collector +overwrites the stored text in place. A verdict is keyed on the body, so a reword drops one +pinned key and adds another. The addition was tolerated. The drop was fatal. + +Rush and Lusk flipped from "platform 2" to "platform 1" overnight on 8 September, and the guard +built to catch a code regression fired on Irish Rail editing a sentence. + +> **Concept: a guard over an input you do not control.** A regression test is a claim that *this +> code*, on *this input*, produces *this output*. If the input is not pinned, the test is +> making a claim about three things while reporting on one, and every failure has to be +> diagnosed before it means anything. Worse, the failures arrive on somebody else's schedule: a +> data repository's six-hourly push, a monthly snapshot refresh, an upstream dependency bump. +> The tell is a red build on a merge that could not have caused it. The fix is not a better +> failure message; it is to pin the inputs into the fixture so the test replays them, and to +> move the question of "has the corpus changed" to wherever that is actually the question. + +That was the third such failure in five days, after an upstream design-layer bump and a +Midleton reword (chapter 13 found that one). Each had been answered with a regeneration, which +buys a few days. + +### The fixture now carries its inputs + +The file carries the payload node each station was read from and the body of each notice, and +the test replays them through today's code. That is 23 KB of page fragments across 152 stations, +and the file grows from 53 KB to 115 KB. + +Three things follow, and the second is my favourite. + +**The test needs no data checkout at all**, so it moved out of the real-corpus test file and +runs on a bare clone. Which surfaces a quiet irony: chapter 12 recorded that skipping on a +snapshot mismatch was rejected because the guard would then be *"silently off from the first +refresh nobody regenerated after"*. The version that was built to avoid being silently off was +itself silently off for anybody without a `lifts-data` checkout, which includes a fresh clone +and any contributor who has not set it up. + +**Regeneration is additive.** The regenerator builds over the union of what is already pinned +and what the corpus currently holds, so a wording Irish Rail has since withdrawn **stays as a +test vector**. Both Rush and Lusk bodies are in the file now. The station records carry the page +fragments they were read from, for exactly the reason chapter 01 gives for keeping the raw +logs: the derived form is the disposable one. + +**The comparison reports moves only.** What one document holds and the other does not is the +size of the corpus, which no code change decides, and the snapshot filename is provenance rather +than a comparison. The monthly review of a reworded station page stays where it already was: the +refresh workflow opens a pull request against the data repository with the report attached and a +body saying to read it. That is a better home for it than a red build in a different repository. + +### The review found the fix had a hole of its own + +Narrowing the comparison to moves took the dropped-record check off the **other** golden file +too, the one chapter 15 is about, and that one does still replay live observations. There a +dropped station can be a genuine code fault rather than a corpus event. + +Nothing else covered it. The test that looks as though it does passes today only because the +surveyed set happens to be the pilot set, and that coincidence ends at the sixth station. + +The fix counts the stations and notices off the **directory listing** rather than off the loader +that feeds the build, and the reasoning is worth keeping: comparing a build with the loader that +produced it holds however badly the loader breaks. That was the first version of the test, which +is why the fix was checked rather than assumed. It is the same defect chapter 08 named, a +predicate over the wrong quantity, turning up in the fix for something else. + +### Worked example: proving the determinism claim + +The claim this pull request makes is that the test's result no longer depends on the state of +another repository. That is a claim you can check directly rather than argue for, and it was: + +``` +git -C ../lifts-data checkout HEAD~20 # predating the reword +rebuild +run the suite +``` + +Green, where the old fixture would have disagreed. The suite also runs green with the data +directory **unset**, on both the development interpreter and the 3.11 floor, and the golden +tests are confirmed by name in the verbose output to be running rather than skipping. + +And the guard was checked to still fire, by breaking the thing it exists to protect: disabling +the boilerplate stripper from chapter 07 fails on Donabate, Greystones and Killiney, the three +stations where the lift-call sentence is the only mention of a lift. + +## What this corrects in the series + +Chapter 12 is left as written, because it records what was believed at the time, but two of its +sentences are now known to be wrong and this chapter is the correction: + +- The golden file's cost was **not** only the mandatory monthly review. It was also that any + reword anywhere, by Irish Rail, at any time, broke an unrelated build. +- The note's claim that a vanished notice "can only mean a bad checkout" rested on the logs + being append-only, which is true, and on the database inheriting that, which is not. + +## Where it left the site + +A test that tests this repository's code and nothing else, on a bare clone, deterministically. +A fixture that accumulates wordings rather than replacing them. And one assumption in +`notes/station-access.md` corrected in place rather than quietly dropped, which is the habit +that makes the rest of the notes worth reading. + +## Notes + +- PR #51, "Pin the access golden file's inputs, so it guards code and not the corpus" + (8 Sep 2026): the failing run, the append-only correction, the pinned inputs, additive + regeneration, the narrowed comparison, and the review finding on the other fixture. +- `notes/station-access.md`, whose "the logs are append-only and that can only mean a bad + checkout" sentence is corrected in place, and `notes/step-free-graph.md` on the fixture that + still replays live observations. +- Rush and Lusk's body flipped from "platform 2" to "platform 1" on 8 Sep 2026. The two earlier + regenerations were PR #48 (a statusui bump) and the Midleton reword found in PR #49. +- File size 53 KB to 115 KB, of which about 23 KB is page fragments across 152 stations; the + suite runs 516 tests with the data directory unset, measured 12 Sep 2026. diff --git a/writing/chapters/15-building-the-thing-that-does-not-exist.md b/writing/chapters/15-building-the-thing-that-does-not-exist.md new file mode 100644 index 0000000..4617973 --- /dev/null +++ b/writing/chapters/15-building-the-thing-that-does-not-exist.md @@ -0,0 +1,233 @@ +# 15. Building the thing that does not exist +*~10 min read · PR #46 · 4 to 8 September 2026* + +*Where we are:* chapter 06 established that nothing machine-readable says how an Irish station's +entrances, concourse, platforms, lifts and escalators connect. Chapter 07 read somebody's prose +instead, carefully, and chapter 12 wrote down how far that can be trusted. This chapter reverses +a decision the series has been quoting as settled since chapter 06. + +## The question that opened this stretch + +Barry's question, which is the obvious one and which I had an answer to: + +> What would it take to build one by hand? + +The answer on file was *don't*. `notes/accessible-routes.md` § What is deliberately out has said +since 29 August: + +> Hand-curating station accessibility by hand from photographs, station visits or Wikipedia. It +> would be a second source with no provenance, no refresh and no way to audit, attached to a +> site whose entire discipline is that every number traces back to a recorded observation. +> Better to say nothing. + +That paragraph is right about the failure mode and wrong about the conclusion, and the +difference between them is one word. It does not say hand-curation is impossible. It says a +hand-curated **file** has no provenance, no refresh and no audit. + +Those are three properties. Properties can be built. + +> **Concept: the objection is a specification.** A rejected approach usually gets summarised as +> "we decided against that", and then nobody can tell later whether the objection was about the +> approach or about a particular implementation of it. Writing it out in full, as three named +> defects, turns it into a list of requirements: a fact must say who recorded it, when, from +> what, and how sure they were (provenance); it must expire when its source changes (refresh); +> and it must be replayable from an append-only record rather than edited in place (audit). +> Every one of those is something this repository already does for the collector's logs. The +> objection was never to hand-curation. It was to a file somebody edits. + +## What changed + +### Three documents, and what each turned out to be for + +Barry brought three sources. Sorting them was most of the design work. + +**Irish Rail's Guide for Rail Passengers with Disabilities (2026)** has no per-station data at +all. It is a who-to-ask: an access email address, a named head of customer care and +accessibility, a quarterly Disability User Group chaired and attended by eleven named member +organisations, a hub station per zone whose staff cover the zone, and the phrase "over 50 lifts" +renewed since 2020, which means **a lift register exists somewhere**. + +**Metro Nation's Dublin rail map (July 2025)** was excluded as a source, and the call was +Barry's. Its "no step-free access" glyph marks twelve stations. Checked against Irish Rail's own +page it disagrees at three of them, on no stated definition, and it was already behind the +network when it was drawn: Athy's lift was delivered afterwards. It is kept in the note as one +sentence of evidence for the rule it teaches, which is **define the terms before asking anybody +anything**. + +**The Station Accessibility Programme preliminary business case**, Iarnród Éireann to the NTA, +October 2024, 188 pages. The most useful of the three by a distance, and the thing I would not +have found by searching for data: + +- Table 6-2 lists **the 51 stations that do not yet meet the accessibility standard, in the + order the programme means to fix them**, in three packages of 15, 15 and 21. That is a + denominator the site has never had, and it is now carried in code. +- Appendix B has a dated "current context" paragraph for the first fifteen that reads like a + route graph in prose: *"passengers wishing to travel southbound must exit the station + property, pass over a small road bridge ... and enter the station via a ramp behind Platform + 1"*. Better than Irish Rail's public page for those fifteen. Each has a delivery date beside + it, which is why the 2024 text for Athy says there is no step-free access to platform 2 and + the 2026 page says "Lift to platform 2". +- **The audit exists.** A 2014 feasibility report surveyed all 54 then-outstanding stations + against the European accessibility standard, a 2019 review updated the list, and 2021 + preliminary design reports cover the first fifteen. The business case names the teams holding + them. That is what a Freedom of Information request asks for, by name, rather than asking + hopefully for "accessibility data". +- **Its definitions**, which is what the questionnaire needed. The 2014 study scoped the minimum + as a wheelchair user boarding and alighting and entering and leaving each station, + distinguished **assisted** from **un-assisted** routes, counted un-assisted access to a + *single* platform per station, and treated routes *between* platforms separately, because + crossing the track is the expensive part. + +### The observation log + +`lifts-data/survey/.jsonl`. One file per station, one JSON object per line, sorted keys, +never edited. A questionnaire maps to one file, two labellers never collide, and a diff is +readable. + +```json +{"code": "ATHY", "observed": "2026-09-12", "confidence": "high", + "source": {"kind": "survey", "by": "Barry"}, + "fact": {"type": "edge", "id": "lift-p2", "mode": "lift", "from": "footbridge", + "to": "platform-2", "equipment": "lift-p2", "hours": "06:00-23:30"}} +``` + +The replay rule mirrors `rebuild` exactly: **lines apply in file order and the last line for a +fact key wins.** A correction is a later line with the same id. There are no line ids and nothing +to cross-reference, so appending a line by hand stays cheap, which matters when the people +filling these in are not programmers. + +`confidence` is `low` (read off a page), `medium` (told, or a reviewed sentence) or `high` +(seen). The source kind decides what the validator demands: a page-sourced fact must carry the +snapshot, the field and a verbatim quote; a surveyed one must say who; a business-case one must +give a page number; imagery must give a URL; an FOI response must give its reference. + +And the refresh property falls out of chapter 07's rule, generalised: **a page-sourced fact +expires when its quote leaves the page.** The replay drops it and says so, and a test turns red +until somebody reads the diff. An empty quote records that the field said nothing usable, and +expires the day it says something. + +### The graph, and an edge that records ignorance + +Nodes are entrances, concourses, platforms, landings. Edges are walkways, ramps, stairs, +footbridge stairs, subway stairs, lifts, escalators and gates. A lift or escalator edge belongs +to a piece of equipment, which can carry aliases so a notice worded "the lift on P2" can be +joined to it by hand. + +There is a ninth edge mode and it is the most interesting thing in the schema. + +> **Concept: an edge that records ignorance.** `unsurveyed` is a way known to exist whose nature +> nobody has recorded. It sounds like a placeholder and it is load-bearing. Without it, an +> entrance the seeder cannot read has no edges at all, so every platform behind it comes out +> "never step-free", which is a confident false claim in the exact direction chapter 07 spends +> its length avoiding. With it, the graph is **incomplete**, and incomplete is the truth. A data +> model that has no way to say "there is something here and I do not know what" will always +> encode absence as impossibility, because those are the only two states it has. + +Step-free access per platform is then reachability from any entrance over walkway, ramp, lift +and gate edges, preferring routes with fewest lifts. An outage is the named equipment's edges +removed, and a platform is **lost** if no step-free route is left, **kept** if one is, **never** +step-free if it had none to begin with, and untouched if its best route never used the machine. + +### Two rules that keep it on the safe side + +Chapter 07's safe direction had to survive the change of method, and it does, through two rules +that are both about the derivation refusing to outrun its evidence. + +**The confidence gate.** "Another step-free way" is published only where every edge on the +surviving route was recorded at medium confidence or better. Otherwise the platform reads as +lost, and the detail says the survey names a route nobody has confirmed. Seeds are low +confidence, so **a graph seeded from the page alone can never say more than the prose derivation +does**, and it earns verdicts only as lines with a human source land. + +**Nothing joined is unknown.** A notice that joins no recorded equipment is unknown, never a +guess. + +### Worked example: the pilot says less than the prose, and that is the design working + +Five stations were seeded from their pages and committed: Hazelhatch, Dublin Pearse, Dublin +Connolly, Athy and Castleknock. Every line is page-sourced, so the whole pipeline runs end to +end and publishes nothing new. + +| station and notice | the graph says | the prose says | +|---|---|---| +| Athy, lift at platforms 1 and 2 | lost 2; platform 1 never needed it; platform 1 named but no lift touches it | lost 2, same note about 1 | +| Pearse, lift at platform 2 | lost 2; platform 1 never needed it | lost 2; platform 1 kept | +| Pearse, escalator at platform 2 | **unknown**: the page's only escalator is on the way in | escalator, quoting the platform 2 lift | +| Connolly, escalator at the concourse | escalator, and a lift between the entrance and the concourse | escalator, quoting both entrance sentences | +| Hazelhatch, lift to platforms 2 and 3 | **unknown**: no platform named on the page | lost 2 and 3 | + +Read the two "unknown" rows and the new method looks worse than the old one. It is not. At +Hazelhatch the page claims a lift without naming a platform, so the seed draws no lift edge and +the notice joins nothing. The graph is being honest about a vague page where the prose +derivation was making an inference. The questionnaire asks the question. + +The confidence gate bites in the same direction at Raheny, one of the two reviewed step-free +exceptions from chapter 07: the ramp is recorded at medium confidence and the way in at low, so +the graph says lost where the prose says there is an alternative, **until somebody confirms the +door.** + +A method that produces fewer confident answers than the one it replaces, on its first day, is +what it looks like when the evidence bar goes up. + +### The questionnaire is generated, never written + +One Markdown form per station. The definitions come first, in the business case's own words, +because the three sources already in hand disagree with each other for want of them. + +Then, per station: the page's two fields quoted verbatim with "is this right, what does it leave +out"; how the site currently reads them; the page-versus-feed discrepancy where chapter 07's +unknown table has one; the business case paragraph and the station's rank in the programme where +it has them, with "this was written in 2024, what has changed since"; every distinct notice the +feed has carried about the place with "which machine is this, and which two places does it +connect"; ten common questions; and the draft observations for the person to correct rather than +a blank page. + +The seeder drafts those from the page at low confidence, quoting the sentence each line came +from, and **refuses to overwrite an existing log**, because a log is append-only and a second +seed would duplicate every line. + +One line in the note explains why this exists at all: the site's prefilled correction issue from +chapter 07 **has existed since 30 August and has never fired.** Passive crowdsourcing yields +nothing. The outreach has to be structured, and a form somebody can answer is the structure. + +## The turn at the end + +Chapter 06 is about a specific absence. GTFS `pathways.txt` is the format that would say a +station's interior is a graph, and Ireland publishes none. NeTEx is the European standard the +regulation names, and Ireland publishes none. + +This branch ships a `gtfs` command that exports stops, pathways and levels. + +It is five stations, seeded from prose, at low confidence, and it is not authoritative about +anything. But the shape of the project has changed. For three weeks this repository catalogued +an absence and worked carefully around it. It is now building the missing artefact, in the +format the mapping apps already read, with a documented path to the format the regulation +names, plus a `prose` command that renders a surveyed station in a layout drafted to hand to +Irish Rail: a paragraph per platform, "yes", "yes, by lift" or "no" as the first words, and the +failure case stated in the same breath rather than left to be inferred. + +The site does not read any of this yet. When it does, the graph verdict replaces the prose one +only at a station whose log has a human source, and the prose verdict stands everywhere else. + +## Where it left the site + +Unchanged, deliberately. A schema, a validator, a replay, a reachability derivation, a +questionnaire generator, a seeder, two exports, a fixture pinning what the survey says, five +pilot stations on a branch of the data repository, and a note carrying the design, the options +assessed and what to ask for under Freedom of Information. + +## Notes + +- PR #46, "Record station access by hand, with provenance: the observation log and the graph it + replays into" (8 Sep 2026). +- `notes/step-free-graph.md` (4 Sep 2026): the three documents, the options table, the log + schema, the graph, the two safe-side rules, the questionnaire, the seed, the pilot table, the + proposed format, and the fixture. +- `notes/accessible-routes.md` § What is deliberately out (29 Aug 2026), which this reverses, + and `notes/station-access.md` § The safe direction, which it preserves. +- The business case is Iarnród Éireann to the NTA, PBC-3.5, 30 October 2024. Table 6-2 is the + 51 stations; Appendix B the current-context paragraphs; Table 12-3 names the document holders. +- The five pilot stations are seeded on the `survey-pilot` branch of `baz8080/lifts-data`; + `main` there carries no `survey/` directory yet, which is why the graph fixture's own inputs + are not pinned (chapter 14). +- 490 tests passed at that merge; 516 as of 12 September 2026. diff --git a/writing/chapters/16-a-fourth-site-and-two-bugs-found-sideways.md b/writing/chapters/16-a-fourth-site-and-two-bugs-found-sideways.md new file mode 100644 index 0000000..21eec29 --- /dev/null +++ b/writing/chapters/16-a-fourth-site-and-two-bugs-found-sideways.md @@ -0,0 +1,165 @@ +# 16. A fourth site, and two bugs found sideways +*~7 min read · PR #54 and issues #52, #53 · 11 September 2026* + +*Where we are:* the series has been about one repository reading one feed. This chapter is about +what happened when a second reader of the same feed appeared, and about the two bugs it found +here without looking for them. + +## The question that opened this stretch + +Chapter 01 recorded that this collector stores every service banner Irish Rail publishes, not +just the lift ones, and that as of 31 August 24 of 234 messages qualified as lift or escalator +notices. The other 210 sat in the log, unread. + +Somebody was eventually going to read them. A fourth site, `rail-delays`, now does: it reads +this project's data repository and publishes the delay notices. + +Almost none of that work belongs in this repository. The question is which parts do. + +## What changed + +### The collector is not duplicated, not extended, and not touched + +One endpoint returns the whole feed in one response, and the raw write already stores it +verbatim. So a second poller buys nothing and costs three things: two pollers hitting a +credentialed endpoint that Irish Rail rotates without notice, a second install on the Pi, and +two logs that disagree about the same feed. + +A delays package living *here* was rejected too, for a reason that is about repositories rather +than code: this repository's CI would become the gate on two sites, and a repository publishes +one Pages site, so the delays would have had to claim to be part of this one. + +> **Concept: the boundary is a question about the artefact, not the code.** "Should this live +> here?" usually gets argued as a code question, about coupling and shared helpers. The +> decisions above turn on things that have nothing to do with code: how many credentials hit an +> endpoint, how many machines run an install, whose CI blocks whose merge, and what a Pages site +> claims to be. The cause reader that moved across imports nothing from any package here, which +> is what made it a copy rather than a port. The collector stayed because duplicating it would +> duplicate a credential, not because it was hard to extract. + +The note now carries a short table of which repository a change belongs in, because the +question had come up twice. The row that is easy to get wrong is "a fact about the feed", and it +had been got wrong here: two observations about delay notices, that the apology sentence ends a +particular way and that the start field is a departure time, had been written into this +repository's data-shape traps. They are traps for code that reads a cause or identifies a train, +and nothing here does either. They live across the way now. + +### The trap, which is about this collector + +The useful thing the new site found is a property of the identity model that nothing here had +noticed, and could not have. + +**Irish Rail empties a notice's location codes part-way through its life.** When that happens +the collector's validity check routes it to the unidentifiable table, and its tracked listing +ends while the notice is still on the feed. + +Counted over every notice collected since 8 August: + +| | notices | never carry codes | lose them partway | +|---|---|---|---| +| lift and escalator | 55 | 0 | 0 | +| everything else | 497 | 21 | **288** | + +**No lift or escalator notice has ever done it.** That is why nothing here noticed, and it is +why the identity model is not changing on the strength of it. It is also what fills a table this +repository otherwise never reads: chapter 01 described the unidentifiable items as delay notices +with empty location codes and left it there, and the real explanation is that most of them +arrive with codes and lose them. + +The reason to write it down is the next person who touches the validity check. The current +behaviour is correct for this site and would be a bug for a site that reads delays, and nothing +in the code says so. + +## Two bugs found sideways + +Both of the issues open on this repository today were found by somebody looking at something +else, and neither has broken anything on the published site. That combination is worth a section +because it is the argument for building the tools in the first place. + +### "planned maintenance" is not "planned works" + +The planned-works marker is the literal string "planned works", tested as a substring. Irish +Rail wrote a different word on one notice: + +> The Lift on platform 1 is currently out of service due to **planned maintenance**. + +First seen at Salthill and Monkstown on 11 September and still listed. It reads as a fault, so +its day cells are drawn in the fault colour, it gets none of chapter 04's grace week, and the +station page and the overview both describe it as something it does not say it is. + +It is the first notice in the corpus to use the phrase, so nothing published before that day was +wrong. + +**It was not found by looking for it.** The delays site's cause reader parses a cause out of any +notice's own clause, and lift notices are in its corpus because they are in the feed. Run beside +this site's planned-works test over all 416 distinct notices collected since 8 August, the two +disagree exactly **once**, on this notice. The cause reader's planned category matches "planned +works" (7 notices), "engineering works" (3) and "planned maintenance" (1), all three being +wordings the corpus actually carries. + +> **Concept: a second reader of the same data is a test you did not write.** A substring test +> against one phrase is unfalsifiable from inside: there is no input that makes it fail, only +> inputs it silently declines to match, and chapter 13's tripwires catch that shape only where +> somebody thought to add one. Two independent implementations of "is this planned" over the +> same corpus produce a disagreement count, and a disagreement count is a finding. One in 416 is +> a good result and it is still one more than staring at the regular expression would ever have +> produced. The general move: when a second consumer of your data appears, run it beside yours +> and diff the answers before you do anything else with it. + +What to do about it is genuinely open, and the issue says so rather than deciding: whether +"planned maintenance" should earn the same week of grace as "planned works" is a question about +what the grace is for, which is chapter 04's argument, not a question about a regular expression. + +It closed on 18 September, and it was less open than this paragraph makes it sound: by chapter +04's own account of what the grace is for, all three wordings claim the same kind of event. +Chapter 17. + +### Kishoge is keyed by its name + +The station facts are keyed by station code, read from the page's own code field. For one +station, Kishoge, that field is missing or unparseable on irishrail.ie, so the code falls back +to the slug. The record is keyed `"Kishoge"` instead of `"KISHO"`. + +Chapter 06 celebrated that join being free: every payload carries a code in exactly the code +space the message feed uses, all 15 codes with lift notices matched, no fuzzy matching and no +mapping file. That is still true of 151 stations. + +For this one, a real lift notice would fail to join to its station facts, print as "(no station)" +in the report, and fall back to an unknown verdict, even though Kishoge does have a lift and +real access prose. No lift or escalator notice has ever been posted there, so nothing on the +live site is wrong today, and the join would fail silently the day one is. + +It was found while investigating a user report of an outage at Kishoge that turned out not to be +in the feed at all. + +The fix is not obvious and the issue does not pretend otherwise. Falling back to the name is what +produced this; validating the code against the known code space, or refusing the record outright +the way chapter 08's partial-fetch refusal does, are both defensible and both change what the +denominator means. + +The diagnosis above is the issue's, repeated here without being checked, and it was wrong. +Nothing fell back to anything: Irish Rail's page publishes the name in the code field. Neither of +the two fixes offered here shipped either. Chapter 17. + +## Where it left the site + +Nothing on the page changed. A second site reads the same logs, the boundary between them is +written down, one property of the identity model is recorded before somebody trips over it, and +two silent joins-in-waiting are filed with their measurements. + +Chapter 08 was about the same bug turning up three times in one review. This one is about two +bugs turning up because somebody built a different tool and pointed it at the same data, which +is the cheaper way to find them. + +## Notes + +- PR #54, "Record what the delays site cost this repository, and the one trap it found" + (11 Sep 2026), documentation only: the three reasons the collector is not duplicated, the + boundary table, the location-codes measurement, and the two traps moved to the other + repository. +- `notes/delays-site.md`. +- Issue #53, "'planned maintenance' is not 'planned works'" (11 Sep 2026): the notice, its three + effects on the page, and the one disagreement in 416 notices. +- Issue #52, "Kishoge station-facts record is keyed by name, not location code" (11 Sep 2026). +- The location-codes table counts notices by head plus text plus start, over the corpus from + 8 August to 11 September 2026. diff --git a/writing/chapters/17-the-same-thing-said-differently.md b/writing/chapters/17-the-same-thing-said-differently.md new file mode 100644 index 0000000..035d027 --- /dev/null +++ b/writing/chapters/17-the-same-thing-said-differently.md @@ -0,0 +1,176 @@ +# 17. The same thing, said differently +*~9 min read · PRs #55, #56 · 18 to 24 September 2026* + +*Where we are:* chapter 16 filed two bugs, both found by pointing a second reader at the same +data. Both are fixed now, and neither fix is the one the issue proposed. Re-measuring for this +chapter turned up a third thing. It is not a bug, and nothing on the page mentions it. + +## The question that opened this stretch + +Chapter 16 said #53 was a question about what the planned-works grace is *for*, rather than +about a regular expression. It was answered as a question about vocabulary. Was that a dodge? + +## What changed + +### Three words for one claim + +PR #56, merged 18 September, turned `PLANNED_MARKER` into `PLANNED_MARKERS`: "planned works", +"planned maintenance" and "engineering works". The note gives the reason in one sentence. They +are the same claim in different words. The delays site's cause reader already put all three in +one category, and the corpus carries all three. + +To decide whether that dodges chapter 16's question, go back to what chapter 04 said the grace +is for. *A week is a plausible maintenance window, and because Irish Rail's own end dates are +placeholders, the listing is the only measure of how long works actually ran.* The grace is not +a reward for a phrase. It is a concession to a kind of event: the operator took the lift out on +purpose, for a job with an end. Maintenance is that kind of event, and so are engineering works. +The vocabulary answer and the purpose answer turn out to be the same answer. It would have been +a real question about purpose if the new wording had described a different event, and none of +the three does. + +So chapter 16 made #53 sound more open than it was. That is worth saying, because the +temptation after a bug found sideways is to treat it as deep. + +> **Concept: a literal string is a vocabulary of one.** A test for a phrase is really a test +> for how one person typed a claim once. It cannot fail. It can only decline to match, and it +> declines silently. The fix is not a cleverer pattern. It is to let the corpus act as the +> dictionary: list every wording the data actually carries for the claim, and widen to exactly +> those. After widening, the list is still closed, and the next synonym will be as invisible +> as the first one was. That is why a second reader over the same data matters. It was the +> only thing that noticed the first time, and it is the only thing that will notice the next. + +### Worked example: Salthill and Monkstown in September + +As of 24 September the station has two notices and one reissue on record: + +- *"The Lift on platform 1 is currently out of service due to planned maintenance."* Listed + 11 September 13:32Z to 15 September 14:32Z, which is four days and an hour. +- *"The Lifts on platforms 1 and 2 are currently out of service due to planned maintenance."* + Listed from 16 September 14:01Z. At 09:01Z on the 17th, in the same poll that it vanished, a + fault notice for platform 1 appeared with no cause given. Chapter 02's rule folds a reissue + in the same poll into one outage, and that outage came down at 23:02Z, which in Dublin is two + minutes into the 18th. + +September had 24 days watched by the horizon. Under the old marker, every listed day counted: +the 11th to the 18th, eight days, so 16 of 24 were available. That is **66%, an E**. Under the +new one, each maintenance notice ran well inside its week and costs nothing, and only the +fault's two Dublin days count, which leaves 22 of 24. That is **91%, a C**. + +That is two bands for one word. It moved nothing else: across September's 30 station-months the +national figure is 85% either way, planned notices go from 3 to 4, and one C replaces one E. + +One more detail from that reissue. Its head, the line the overview's rows are built from, read +*"Station - Lift out of order"*, with the word "Station" where the station's name belongs. The +site still printed "Salthill and Monkstown", because a row takes its name from the notice's list +of stops and uses the head only when that list is missing. A row is named from its newest +notice. If the name had come from the head, the overview would have renamed Salthill and +Monkstown "Station" at 09:01 that morning. + +### A code that was a name + +PR #55, also 18 September. Chapter 16 followed issue #52 in saying that Kishoge's code field +was "missing or unparseable", so the loader had fallen back to the station's name. The fix found +that this was not what happened. Irish Rail's page for Kishoge publishes the name, `"Kishoge"`, +in the field where every other page publishes a five-letter code. Nothing fell back to anything. +The loader read exactly what the page said. Chapter 16 repeated the issue's diagnosis without +checking it, and the correction lives here rather than being edited into that chapter. + +The issue offered two fixes: validate the code against the known code space, or refuse the +record. Neither shipped. What shipped is a table, `STATION_CODE_FIXUPS`, with one entry keyed by +the page's slug. It was checked by hand against a real notice in the corpus, whose location code +is `KISHO` and whose stop name is "Kishoge". + +Measured on 24 September, the feed has named 76 distinct location codes since 8 August. With the +table, all 76 join to a station record. Without it, 75 do. The one that misses is Kishoge's, and +it has appeared exactly once, on a delay notice (*"Delays of up to +15mins"*). There is still no +lift notice there, so the fix changes nothing a reader can see today. What it prevents is a +silent miss on the day there is one. + +This is the second hand-maintained constant in the access code, after chapter 07's list of two +reviewed step-free alternatives, and it has the same shape: a human decision, made in a diff, +with the evidence in the pull request. What it lacks is a guard. If another page puts a name in +the code field, that station will miss in the same silent way, and the comment the fix left in +`survey.py` says as much: *"a fixed page bug is not a promise every code stays clean."* Chapter +13's tripwire is the shape that would catch it: assert that every code in the snapshot looks like +a code, or that every code the feed names joins. Today neither assertion exists. + +## Found while measuring: August moved + +Re-measuring for this chapter, August's national figure came back as **75%**. Chapter 16 and the +closing had quoted **76%**, and today's grade mix has one more D and one fewer A. August ended on +the 31st, and nothing about August's lifts can have changed since. + +To tell a code change from a data change, the corpus was cut at the morning of 12 September and +rebuilt with today's code. The result was 76% and three A grades, exactly as quoted then. Today's +code on today's corpus gives 75% and two A grades. So the data moved it, and exactly one +station-month changed: **Tullamore, August, from A at 100% to D at 83%.** + +Tullamore's notice, *"Lifts are temporarily unavailable due to planned works"*, was first seen on +28 August at 15:01Z and came down on 1 September at 09:02Z. That is three days and eighteen +hours, inside the week, so its four August days cost nothing and the month graded A. On 16 +September at 14:01Z **the same notice came back**, with the same head, the same station and the +same start date Irish Rail claims. By chapter 01's identity it is one notice. It is still listed +at the horizon. + +Chapter 10's pooling adds a notice's planned stretches before anything is spent. By the +arithmetic, the total crossed seven days at about 20:00Z on 19 September. From the next poll, +every listed day of the notice counted, **including the four in August**, and 20 of 24 watched +days is 83%. August's letter changed nineteen days after August ended. No page, feed or file +says it changed. + +It is not a bug. The rule is doing exactly what chapter 10 built it to do, and the code's own +docstring anticipates a milder version of this: *"a fortnight of works spanning a month end +counts in both halves"*. The case that motivated pooling was Pearse's four-hour blink. Tullamore's +gap was fifteen days, and the rule does not look at the gap. Whether works fifteen days apart are +"the same works" is a question the notice itself answers. Irish Rail reissued it with the same +claimed start, 31 August, so by its own words it is one job. + +What is new is what this means for the page. A month that has ended is presented as history, +and a reader who noted Tullamore's A for August on the 19th would find a D on the 20th, with no +mark saying the month was revised. + +> **Concept: a property of the notice, published as a property of the month.** A month on a +> status page reads as settled. If any number in its grade is computed over something that +> outlives the month, such as a notice's running total, a correction or an average, then the +> month's figure depends on the future too. That is not wrong in itself. It is wrong when the +> page presents the figure as final. The check is to ask, of every number on a past month's +> card, what could still change it. If the answer is not "nothing", then either the number is +> provisional and should say so, or the computation should stop at the month's edge. Which of +> those is right depends on what the number is for, and that is a decision, not a fix. + +The shapes are easy to list and hard to choose between. One is to say on the page that a month +stays provisional while any notice in it can come back, and with no completion signal (chapter +01) that means forever. Another is to freeze a month's letter when the month closes, which lets +the August card disagree with the rule applied to the same notice today. A third is to spend the +grace in time order, so the first week is always forgiven, which reverses chapter 04's +"including the first week" and moves other grades. All three belong beside chapter 04's argument +in `notes/site.md`, with these measurements. As of 24 September none is an issue yet. + +## Where it left the site + +Salthill and Monkstown's September is a C where it was an E, because a claim of planned work is +now recognised in the three wordings the corpus uses. Kishoge will join the day a lift notice +names it. And there is one measurement the page does not show: a past month's letter can move, +and it did. + +The thread running through all three is sameness. Three phrases turned out to be one claim, a +name stood in for a code, and a notice that came back fifteen days later turned out, by its own +account, to be the same notice, which rewrote a month. + +## Notes + +- PR #56, "Treat 'planned maintenance' and 'engineering works' as planned works" (18 Sep 2026), + closing issue #53: `lift_site/model.py` `PLANNED_MARKERS`, `notes/site.md` § Planned works + (amended 2026-09-18), and the resolved paragraph removed from `notes/delays-site.md`. +- PR #55, "Fix Kishoge's station code join (issue #52)" (18 Sep 2026): `STATION_CODE_FIXUPS` + in `lift_access/model.py`, the golden fixture regenerated, and the `survey.py` comment quoted + above. +- Salthill and Monkstown's September grade under both markers, the national September figures, + the 76-code join and Tullamore's two stretches were all measured 24 Sep 2026 against + `../lifts-data` (horizon 2026-09-24T05:02:51Z), running `lift_site.model` with the marker + tuple swapped. Registered in `figures.md`. +- The August comparison: raw logs to 2026-09-12 morning, rebuilt into a scratch directory with + today's code, against the full corpus. Only Tullamore's August differs. +- The Tullamore crossing time is arithmetic, not an observed poll: 3 d 18 h 01 m before, plus + the ongoing stretch from 2026-09-16T14:01:41Z, reaches seven days at about 2026-09-19T20:00Z. +- `lift_site/model.py` `day_marks` docstring, quoted for the month-end case. diff --git a/writing/chapters/18-the-one-code-a-rebuild-cannot-undo.md b/writing/chapters/18-the-one-code-a-rebuild-cannot-undo.md new file mode 100644 index 0000000..a89b69f --- /dev/null +++ b/writing/chapters/18-the-one-code-a-rebuild-cannot-undo.md @@ -0,0 +1,184 @@ +# 18. The one code a rebuild cannot undo +*~9 min read · PRs #57, #58 · 20 to 24 September 2026* + +*Where we are:* chapter 01 built everything on one rule: the raw log is the truth, and anything +derived from it can be thrown away and rebuilt. This chapter is about the code *upstream* of the +log, where that rule gives no protection at all, and about one sentence in chapter 01 that did +not hold. + +## The question that opened this stretch + +Every part of this repository had been reviewed, line by line, as it changed, except the part +that changed least. Was that the part that needed it least? + +## What changed + +### Why the collector, and only the collector + +The collector was written on 18 August, before pull requests here went through a review agent. +It then barely changed. Three of its files were never touched again, and the poll loop changed +by sixteen lines. Everything built after it, the access derivation and most of the site, was +reviewed diff by diff from 26 August, and the real-corpus and golden tests pin what those parts +publish. + +The argument for reviewing the collector, and nothing else, as whole files on 24 September is +one sentence in the note: it is **the only code whose mistakes a rebuild cannot undo**. A parse +bug is fixed by fixing the parser and replaying the log. A response that never reached the log +is gone. + +> **Concept: the invariant has an upstream edge.** "The log is the truth and everything else is +> disposable" protects everything that runs *after* the log is written, and it makes review of +> that code cheaper, because any mistake can be replayed away. It protects nothing that runs +> *before* the write: fetching, retrying, deciding whether to write, and the order of the write +> relative to everything else. Those mistakes are permanent, and their cost is data you never +> find out you lost. So review effort should follow how reversible a mistake is, not how +> recently the code changed. By that measure the oldest and quietest code in the repository was +> the most urgent to review. + +The review made ten findings. Eight were fixed, one turned out not to be a real path, and one is +left as it is by the owner's decision. Ten is close to the nine that the first review of the +access branch found (chapter 12), so the collector was not unusually careless. It had just never +been looked at. + +### The log waited on the database + +Chapter 01 said the raw line is written **before any parsing happens at all**. That was true. +What it did not say, because nobody had looked, is that the line was written *inside* the block +that opened the SQLite database and ran its schema. So if a power cut left the database corrupt, +or something held it locked past five seconds, every poll failed before reaching the log. The +failure came with a traceback and no alert. + +The derived index could cost the source of truth, which is the dependency chapter 01 exists to +rule out, running the wrong way. Now the append happens before the database is opened, and it is +the only way a line gets written. A database failure has its own exit code, 7, and its own alert, +which says the response was kept and what to do about a lock, a full card or corruption. + +### Three more ways a line could be lost + +- **A full SD card was a traceback, not the storage alert.** The storage check touched an empty + file. An empty file needs no data block, so it can be created on a full disk. This is chapter + 08's shape again, a guard that passes because what it checks is absent. Chapter 08 audited the + collector for exactly that shape and found two, and this one was missed. An error from the + append itself now goes to the storage alert. +- **A cut-off last line took the next good one with it.** A power cut in the middle of an append + leaves a line with no newline, and the next run's line was appended onto the fragment, so + replay dropped both. The append now finishes the broken line first, and only the fragment is + lost. +- **A truncated gzip body escaped the retry.** Python reports a cut-off compressed body as an + `EOFError` or a `zlib.error`, and neither is the network error the client was catching. So the + attempt skipped the retry, the raw line and the alert. Now they count as transient. + +Two more were about the collector being stopped or silenced: + +- **systemd could kill a poll before it logged anything.** The unit allowed 60 seconds. The + client's own worst case is three attempts, each with a 15-second connect and a 15-second read, + plus backoff and DNS: well over 90 seconds. It is 300 now. +- **A second outage within a day of the first was silent.** This is the same 24-hour + repeat-suppression window chapter 08 fixed once already, when it opened on the attempt rather + than the delivery. The marker also outlived a recovery, so if the same fault came back after + a clean stretch, it was suppressed. The marker now clears after four clean runs in a row, which + is two hours. A suppressed failure resets that count, so an API that keeps flapping still sends + one alert a day. Clearing the marker on the first clean run was tried and rejected in review, + because a flapping API would then alert on every failure. + +The backup could also hang forever on a half-open ssh connection, and every later firing did +nothing while it hung. It now has timeouts on the connection, a 15-minute cap on the unit, and a +trap that makes sure hitting the cap still sends an alert. + +### The one keyword argument was not the whole story + +Chapter 01, on the line `json.dumps(..., sort_keys=True)`: + +> Because the keys are always in the same order, the same observation written by two different +> machines produces byte-identical text, so two collectors' logs can be merged with `sort -u` +> and the duplicates simply vanish. That is the whole of the multi-machine story, and it is one +> keyword argument. + +The duplicates do vanish. But `sort -u` removes duplicates *by sorting*, and sorting puts lines +in an order of its own. With sorted keys, the first key on every line is `body`, the whole feed +response. So a merged file comes out grouped by what the feed said, not by when it said it. And +replay read the file in line order, so the afternoon's polls, where a notice had come down, could +be replayed before the morning's, where it was still up. That is a different history, with +outages opened and closed at the wrong times. + +`sort -u` is also how a git conflict on a shared day file gets resolved, and the backup merges +from the remote, so this was not just a thought experiment. Replay now sorts each file by its +fetch timestamp, stably, and the order lives in the data rather than in line positions. + +> **Concept: a merge that deduplicates also reorders.** Any merge that removes duplicates by +> sorting puts its output in an order of its own choosing. If anything downstream treats line +> position as meaning, that merge has quietly rewritten the history. The fix is to make the +> order explicit: sort on a field that carries it, at the point where order matters. Then say +> what the explicit order costs, because a timestamp is only as good as the clock that wrote it. + +The cost is written into the note. After a power cut, the Pi has no hardware clock, so it +restores the last hourly save. A catch-up poll can then be stamped up to an hour *before* runs +already in the same file. A rebuild now applies it before them, where the live run applied it +after. Line order only ever covered part of that case, since a stamp that crosses midnight +already lands in the wrong day's file, and line order cannot survive a merge at all. It was the +owner's call: merging logs has to work. + +### Worked example: the claim that nothing moved + +The claim is that on the real data this changes nothing. That can be checked rather than +argued. On 24 September, all 2,224 raw lines were already in time order within their files, and +`rebuild` followed by `stats` gives identical output before and after every commit on the +branch. So the reordering hole had never fired. It was waiting for the first real merge. + +### What was not a path, and what was left + +The review also flagged that the backup merges from the remote without taking the poll lock. +That turned out not to be a real path. A merge only rewrites files that changed upstream, and +nothing but the Pi writes the raw logs. Taking the lock for a whole fetch and push would make a +poll skip, which is worse. + +One finding was left alone. After a reboot, systemd's time-sync target is reached as soon as the +time service starts, not when the clock is actually right, unless a wait service is enabled. +Enabling it risks a poll that never runs if NTP is unreachable, and a silent poll is exactly +what this collector exists to prevent. The note records what the fix would look like if it is +ever taken up: a poll that checks whether the clock is synced and still writes the line, flagged, +rather than one that waits or skips. + +### What a comment is for + +Four days earlier, PR #57 rewrote this repository's rule on code comments, because comments kept +arriving in volume. A comment now earns its place only when it records something the reader +cannot see: an external system's behaviour, a measurement, a dependency nothing else records, a +reason the obvious approach was rejected, or a trap that would otherwise be refactored away. + +The collector review is a demonstration of that list. What it added next to the code is a power +cut leaving a line with no newline, a probe file that passes on a full card, and a shell that +skips its exit trap on a signal it does not trap. None of those can be seen by reading the line +below, and each would be the first thing a tidy-minded refactor removed. + +## What this corrects in the series + +Chapter 01 is left as written. Two of its sentences are corrected here: + +- "Written before any parsing" was true, and the line was still written after the database was + opened. A broken derived index could cost a raw line until 24 September. +- `sort_keys=True` made merged logs deduplicate. It did not make them replay correctly, because + the merge reorders them. That took a second change, in replay, and it carries a stated cost. + +## Where it left the site + +Nothing on the page changed, and on the real corpus nothing in the database changed either, +which is the point. The collector now puts the raw line first unconditionally, reports each way +of failing under its own name, and can have its logs merged without rewriting their history. +The unit files and the backup script only take effect on the Pi after a reinstall. + +## Notes + +- PR #58, "Review the collector: the raw line no longer waits on the database" (24 Sep 2026): + the ten findings, the second review of the PR itself, and the verification list. +- `notes/collector-review.md`, dated 2026-09-24, which carries each finding, the rejected + first-run clearing of the alert marker, the `fake-hwclock` cost and the NTP decision. +- PR #57, "Say what a comment has to record to earn its place" (20 Sep 2026): `CLAUDE.md` + § Comments. Documentation only. +- Chapter 08's guard shape: the review findings there, the two found by auditing the collector + for it on 30 Aug 2026, the dropped-record check in chapter 14, and the storage probe here. +- 2,224 raw lines in time order within their files, measured 24 Sep 2026 and stated in the note; + `rebuild` then `stats` gives 2,224 runs (2,217 ok, 7 unreachable), coverage to + 2026-09-24T05:02:51Z. +- The client's worst case is from the note: three attempts of a 15 s connect and a 15 s read, + plus backoff and DNS, against `TimeoutStartSec=60`, now 300. diff --git a/writing/chapters/19a-closing-what-the-site-can-say.md b/writing/chapters/19a-closing-what-the-site-can-say.md new file mode 100644 index 0000000..21fc030 --- /dev/null +++ b/writing/chapters/19a-closing-what-the-site-can-say.md @@ -0,0 +1,177 @@ +# 19a. Closing: what the site can and cannot say +*~10 min read · the whole series · 24 September 2026* + +*Where we are:* the end, in two halves. This one is the account: the figures, the two lists, and +the table of where the three sites diverged. The next is what I would tell somebody starting a +fourth. + +## The question, answered + +**Which Irish Rail stations have lifts out of service, and for how long?** + +As of 24 September 2026, over 46 days of collection, 2,224 runs and 737 recorded notices: + +- **69 outages across 44 stations.** 6 planned works in August and 4 in September, 5 + escalators. +- Lift availability across the 21 stations named in August: **75%**, and **85%** across the 30 + named in September so far. That is the share of watched days on which no lift was reported + out at those stations. +- August grades across 21 station-months: **A 2, B 1, C 4, D 7, E 5, F 2.** September so far, + across 30: **A 2, B 7, C 10, D 6, E 5.** +- At the last poll: four lift notices and one escalator, across five stations. +- **31 of the 58 notices removed step-free access** to at least one platform, as worked out + from Irish Rail's own station pages. 4 were escalators. **23 are unknown**, because the two + hand-written sources disagree. + +Forty-six days is not a season and none of these numbers should be quoted as a fact about Irish +Rail. They are a fact about forty-six days, which is the honest scope, and the site says the +collection start date on every page. Several of them have moved for reasons that had nothing to +do with lifts breaking: chapters 10 and 11. And one of them moved **after its month had ended**. +August was 76% when chapter 16 was written and is 75% now, because a planned-works notice at +Tullamore came back in September and took its August grace with it (chapter 17). + +The unknown share is the row to watch, and it is still going the wrong way: 6 of 24 on 31 +August, 7 of 30 on 4 September, 14 of 45 on 12 September, **23 of 58** now, which is two in +five. Thirteen of the twenty-three are notices at stations whose page does not mention a lift. The +corpus keeps reaching stations whose pages are thinner than the ones it started with, which is +what chapter 12's reliability section meant by the page's own error rate being the ceiling, and +chapter 15 is the attempt to raise it. + +## What the site can say + +- **How long a notice was listed**, to the resolution of a 30-minute poll, over a window whose + boundaries are recorded runs rather than a clock. +- **Which stations were named**, keyed by location code, named from the newest notice. +- **How much of a month a station spent with a lift reported out**, as a share of days watched, + and a letter for that share on a scale it declares as its own. +- **How long each stretch a notice was on the feed ran**, rather than the envelope of its first + and last appearance, and **which stations are out right now**. +- **Whether a notice was a fault or planned works**, from the notice's own words in any of the + three wordings the corpus uses for it, and how long works ran past a week of grace. +- **What Irish Rail claims the start date was**, printed as their claim and used for nothing. +- **What an outage did to step-free access**, worked out from Irish Rail's own station page, + labelled as an inference, with a link inviting correction, and **which platform kept it** + where the page says so plainly and the two sources agree. +- **Which leg of the journey a notice is about**, from its own words, and therefore which of + Irish Rail's two access fields to read it against. +- **Who lost a way up** when an escalator stopped, and what the page names on the same leg. +- **How many stations have a lift**: 57 of 152, from a versioned snapshot. +- **How far to trust all of that**, in a dated section separating the strong claims from Irish + Rail's word taken on trust and from untested machinery. +- **All of it, off the page**: an Atom feed for the network and one per station, and a CSV of + every outage the site shows. + +## What it cannot + +- **Say anything is "fixed".** There is no completion signal. A notice stops being listed, and + that is all that is known. +- **Say how many lifts a station has**, or whether the one named was the only one. A notice + names one machine in prose. +- **Say a station is accessible.** It can say Irish Rail's page names a step-free way to a + platform that does not use the lift, at two stations in the country. That is a much smaller + claim and the site's wording stays inside it. +- **Say a lift was working** while an escalator was out. It can say the page names one on the + same leg and that no lift notice overlapped, and no more: the feed is not complete. +- **Give a network availability figure.** The denominator is the stations named that month, + because the feed names a station only when something is wrong with it, and the page says so. +- **Judge an entrance-leg lift outage against experience.** The machinery exists, no notice has + exercised it, and a sixth of the network says it has no ticket office. +- **Promise that a past month's letter is final.** A planned-works notice that comes back can + move a month that has already ended, and Tullamore's August did, on 19 September. Nothing on + the page says so yet (chapter 17). +- **Recognise a fourth word for planned works**, or a fifth page that puts a name in the code + field, until a second reader or a tripwire notices. Both fixes in chapter 17 are closed lists. +- **Colour a day before 8 August 2026.** Nothing was watching. + +## The three-way table + +The deliverable of this series. Every row is a place where the three sites do the same job +differently, and every one traces to a property of the data rather than a preference. + +| | uisce (water) | esb (power) | lifts | forced by | +|---|---|---|---|---| +| **The archive** | the database, built by upsert | verbatim append-only logs, database disposable | verbatim append-only logs, database disposable | that feed purges an outage hours after restoration; this one is patient, so the same design is inherited rather than derived | +| **The collector** | a cloud scheduler, twice a day | a Pi in the hall, every 30 minutes | a Pi in the hall, every 30 minutes | same | +| **An outage's start** | publication time, re-stamped; every duration a floor | the operator's own, back-dated by hours, and measured from | the operator's own, back-dated by **months**, shown and never measured | Rush and Lusk is dated 451.6 days before its first sighting, over polled days it was absent | +| **One event** | pins sharing a reference number | records merged on identical location and start time | one notice's listing, with same-poll reissues folded | no id in this feed either, but only one notice per station per machine, so merging is nearly free | +| **How big it is** | Census population in a 500 m circle | the operator's own customer count | there is no size | the feed carries no count of anything | +| **The grade** | person-hours availability, own thresholds | share restored inside 4 hours, the operator's published aim | share of watched days with no **lift** listed, own bands | electricity is regulated in public; water and lift availability are not | +| **Band calibration** | fitted against its distribution | set by arithmetic from a published target | calibrated to whole days, because the bar is days | at day granularity one bad day is already 96.8% | +| **What knocks the grade** | binary: health notices knock, discolouration does not | planned works excluded, storm days kept and stated | planned works excused a week then counted; escalators show on their own bar and do not knock | nobody excluded anything on our behalf, so every exclusion had to be argued twice: in, then back out | +| **What an outage means** | a boil notice is a boil notice | supply off is supply off | needs a station inventory that does not exist | no NeTEx, no SIRI-FM, no `pathways.txt`, no `wheelchair_boarding` | +| **The second source** | Census Small Areas, official and versioned | Census Small Areas, borrowed from the water site | a hand-typed CMS field, snapshotted monthly, and since 8 September a hand-recorded observation log beside it | the CMS field is the only machine-readable statement that exists, and its error rate is the ceiling on everything derived from it | + +### The identical column + +Some things all three do the same way, and they are the conventions worth carrying to a fourth +site: + +- Raw responses written verbatim before any parsing, and before anything else can fail, with + sorted keys, so two machines' logs merge with `sort -u`. Here that needed a second half: the + merge reorders lines, so replay sorts each file by fetch time (chapter 18). Whether the + siblings need the same is worth checking. +- A window that ends at the last **successful** collection, never at the build clock, with the + gap drawn as "no data". +- A failed run structurally unable to reach the code that closes records. +- Instants in the reader's wall-clock; machine stamps in UTC and labelled. +- A payload budget printed by every build and asserted by a test. +- Decisions written to `notes/` with the date, the numbers, and the alternatives that lost. +- One shared design layer, edited upstream and rolled out to all three. + +## The settled decisions, in plain language + +The repository keeps a table of things not to re-litigate. Translated out of its own shorthand: + +- **The site measures how long a notice was listed**, not what Irish Rail says the outage + began. Their date is printed as their claim. +- **A row is a station**, identified by its location code and named from the most recent notice + about it. +- **Escalators are included and tagged**, never excluded, and since 28 August they get their own + strip so a working lift is never painted by a broken escalator. Since 3 September that strip + carries no weight on the letter: an escalator outage is visible everywhere except the grade. +- **A bar shows the stretches a notice was actually on the feed**, and a single missed poll is + the feed blinking rather than an outage ending. +- **A notice reissued at the very poll the old one vanished is the same outage.** A gap of a + poll or more is two. +- **Planned works are whatever the notice text calls planned works, planned maintenance or + engineering works**, and they are excused for their first week and counted in full after it, + in their own colour once they are. The week is measured over every stretch of the notice, so + a notice that comes back can take back a grace it had already been given. +- **A station's grade is lift availability**: days watched with no lift reported out, on this + site's own A to F scale, because no Irish or EU target exists. It is not step-free + availability either, because it counts notices rather than access, and two in five of the + access verdicts are unknown. +- **The scale runs A to F inclusive**, with E splitting the old F band at 50%, which lands in a + real gap in the data. +- **Irish Rail's end date is printed while the notice is up and dropped once it comes down**, + and plays no part in any measure. +- **Windows end at the collection horizon**, and a notice listed for zero minutes still counts + in its month. +- **"and" in the station prose is a sequence, not a choice**, so a lift out removes step-free + access unless one of two reviewed exceptions applies. +- **An escalator is not step-free**, so its outage removes no step-free access. It does remove a + way up for anyone who finds stairs hard, and the site says so in those words. The one shape + that should still knock the grade, an escalator at a station whose page claims no lift, has no + instance and is written down with a test that fails the day it appears. +- **Which leg a notice is about is read from the notice's own text**, and each leg is read + against its own field. +- **OpenStreetMap was carried and removed** on measurements: it changed no verdict. +- **NeTEx is the one thing worth watching for.** Every other source is checked and closed. +- **A hand-recorded fact is admissible if it carries provenance, expiry and an audit trail.** + This reverses the 29 August decision to curate nothing by hand, which objected to a *file* + rather than to the method. +- **How reliable the derivation is has its own dated section**, by class of claim, including + the classes that are untested. +- **The collector writes the raw line before it opens the database**, and a database failure is + its own alert. The one piece of the collector's review left open, waiting for NTP after a + reboot, is left by decision, with the shape of the fix written down. + +## Notes + +- Corpus figures measured 24 September 2026 by rebuilding `../lifts-data` and running the site + build and `python -m lift_access report`. All registered in `figures.md`. +- The settled-decisions list is a plain-language rendering of the table in `CLAUDE.md` § + Settled - don't re-litigate without reading the note, whose rows point at `notes/site.md`, + `notes/station-access.md`, `notes/publish-cadence.md`, `notes/step-free-graph.md` and + `notes/collector-review.md`. +- Continued in **19b**, which carries the moral and the glossary. diff --git a/writing/chapters/19b-closing-what-i-would-tell-someone.md b/writing/chapters/19b-closing-what-i-would-tell-someone.md new file mode 100644 index 0000000..da864c6 --- /dev/null +++ b/writing/chapters/19b-closing-what-i-would-tell-someone.md @@ -0,0 +1,104 @@ +# 19b. Closing: what I would tell someone starting the fourth one +*~8 min read · the whole series · 24 September 2026* + +*Where we are:* the second half of the closing. 19a is the account of what the site can and +cannot say; this is what seven weeks of it taught, and a glossary of every idea the series +boxed. + +## What I would tell someone starting the fourth one + +The other two series each end with a sentence: the water site's about approximating carefully, +the power site's *collect first, interpret later, keep the bytes*. This one is different, and it +is the thing I did not know seven weeks ago: + +> **Collect first, and publish no meaning you cannot source.** + +Collecting is the easy half and it is where all the discipline usually goes: write the bytes +down, never edit them, make the interpretation disposable. That machinery was inherited from a +sibling, worked on day one, and never once let me down. + +What it does not do is tell you what any of it means. A perfectly recorded observation that +"the lift at platform 2 is out of service" is worth very little until you know whether platform +2 has another way up, and that is a fact about the world rather than about your pipeline. No +amount of care with the bytes creates it. + +And in Ireland, for rail station accessibility, nobody has created it. Not through +carelessness: the European standards exist, and the regulation that would compel it exempts +data you do not already hold, so the obligation is satisfied. The gap is lawful. Five major +mapping products have hit the same wall and fall back to crowd-sourced pins. The only +machine-readable statement of what an Irish rail station has is a free-text field somebody +types into a CMS, and reading one conjunction in it the wrong way would have told a wheelchair +user that access was fine at a station where it was gone. + +Which leaves one more thing to say, and it is the newest. For three weeks the answer to a +missing source was to work carefully around it and publish the limits. On 8 September it became +something else: if the fact does not exist in machine-readable form, record it, with who saw it +and when and how sure they were, and replay it the way the collector's logs are replayed. The +objection to doing that had been written down in full, which is the only reason it could be +read later as a specification rather than a verdict. **Write your rejections out properly. One +of them is a design document you have not recognised yet.** + +So the second half of that sentence is where the work went. Say "unknown" as often as it is +true, which here is two times in five. Publish the derivation as an inference and link a way to +correct it. Default every error to the direction that wastes a journey rather than strands one. Refuse to write a parser where +a reviewed list of two entries is the true claim. And when the number on the front of the page +starts answering a question you did not ask it, write the issue with the measurements in it +rather than adjusting the number quietly. + +There is a coda from the four days in September that closed every question chapter 09 left +open. The one that unblocked the rest was a fifteen-pixel layout fix filed as the least +interesting item on the list, and it was only visible as the blocker because the argument +against the alternative had been written out in full rather than summarised as "decided +against". A note that records why something was rejected also tells you, later, exactly what +would have to change for it to be right. + +And a coda from the last week, which is about where not to look. The code that had never been +reviewed was the collector, because it was the oldest and the quietest, and it was also the only +code whose mistakes a rebuild cannot undo. The rule that made everything else safe to change +made the thing upstream of it look safe too. Review by how permanent a mistake would be, not by +how recently the code moved. And re-measure the months you think are finished: one of them was +not. + +## Glossary + +Every concept boxed in the series, in order of appearance. Thirty-one of them. + +| Concept | Chapter | In one line | +|---|---|---| +| Source of truth against derived index | 01 | The log is what was observed; the database is what it currently means, and only one of them is disposable | +| A run that failed is not a run that saw nothing | 01 | "I could not ask" recorded as "nothing was there" closes every open outage at once | +| Measure the window you actually watched | 02 | Colouring days nobody observed publishes an observation nobody made, and it looks like a real one | +| Two clocks for one date is a bug in either direction | 02 | If the bucket and the label come from different time zones, no reader can tell which the total believes | +| An empty dependency list as a deployment contract | 03 | Keeping `dependencies` empty is what lets the collector install on a Pi by copying a directory | +| A scale with no anchor | 04 | With no published target, an absolute scale of your own, stated as such, beats a relative or borrowed one | +| A band calibrated in the unit the bar is drawn in | 04 | If the bar shows days, the cuts must land on whole days, or the grade claims a precision the data lacks | +| One colour, two meanings | 05 | A mark that covers two cases a reader would distinguish is not wrong, it is silent | +| A cut that lands in a real gap | 05 | A memorable threshold is fine if the data has empty space on both sides of it, which is checkable | +| A National Access Point, and a lawful absence | 06 | The duty is to publish what you hold, not to create it, so the missing data has no process that fills it | +| The safe direction of an error | 07 | Telling somebody access is gone costs a wasted check; telling them it remains strands them | +| An inference that expires with its source | 07 | When a claim's evidence is reworded away, retract the claim rather than inverting it | +| A guard that passes because what it checks is absent | 08 | Ask what a guard asserts when its input is missing; if the answer is "success", it is over the wrong quantity | +| One number, two populations | 09 | One letter cannot answer two audiences whose honest answers differ, and the fine print is not what people read | +| The age on the page is the age of the data | 10 | Rebuilding later cannot make the data younger, so only pushing more often and building on the push move the number | +| A test that exercises the easy half | 10 | If the fixture takes a path where the bug cannot occur, the test's name is the only evidence the behaviour holds | +| A conditional column is a misalignment | 11 | An element that appears only where it has content makes every other row wrong relative to the one that has it | +| A rule with no instance, written down and guarded | 11 | State it in prose and add a test that fails when the case first appears, rather than coding against no example | +| Reading a claim against the right leg | 12 | A station is two journeys with separate equipment; work out which the notice means before reading prose against it | +| What the code's own history says about the code | 12 | When four review passes find nine, six, five and four things, that rate is itself a measurement | +| A feed entry for something still happening | 13 | Date it to the appearance, then move it to the close, so a subscriber sees the outage twice and the second showing is the signal | +| A tripwire for a silent drop | 13 | Assert that a category is empty, because a notice that stops matching does not error, it just is not there | +| A guard over an input you do not control | 14 | The tell is a red build on a merge that could not have caused it; pin the inputs rather than improve the message | +| The objection is a specification | 15 | "No provenance, no refresh, no audit" is not a verdict on an approach, it is a list of three things to build | +| An edge that records ignorance | 15 | A model with no way to say "something is here and I do not know what" encodes absence as impossibility | +| The boundary is a question about the artefact, not the code | 16 | Whose CI blocks whose merge, and how many credentials hit an endpoint, decide where code lives more than coupling does | +| A second reader of the same data is a test you did not write | 16 | Run it beside yours and diff the answers: a disagreement count is a finding a regular expression cannot give you | +| A literal string is a vocabulary of one | 17 | A phrase test cannot fail, only decline; let the corpus list the wordings, and know the list is still closed | +| A property of the notice, published as a property of the month | 17 | If a past month's number depends on something that outlives the month, it is provisional, and should say so or stop at the edge | +| The invariant has an upstream edge | 18 | "Everything derived is disposable" protects what runs after the log is written, and nothing that runs before it | +| A merge that deduplicates also reorders | 18 | Removing duplicates by sorting imposes an order; keep order in a field, not in line position, and state what the field costs | + +## Notes + +- Corpus figures measured 24 September 2026 by rebuilding `../lifts-data` and running the site + build and `python -m lift_access report`. All registered in `figures.md`. +- The sibling closings: uisce series ch 17, esb series ch 8. diff --git a/writing/diagrams/and-is-a-sequence.svg b/writing/diagrams/and-is-a-sequence.svg new file mode 100644 index 0000000..6a0bc19 --- /dev/null +++ b/writing/diagrams/and-is-a-sequence.svg @@ -0,0 +1,43 @@ + + "All platforms can be accessed via lifts and ramps" + + + Read as a choice (wrong) + + + + + + entrance + lift + ramp + + + + + lift out, ramp remains: + "access remains" - the opposite of the truth + + + Read as a sequence (right) + + + + + + + entrance + ramp + lift + platform + + + + + The ramp gets you along. The lift does the level change. Break one link and the chain is broken: + step-free access to those platforms is gone. + diff --git a/writing/diagrams/listed-not-started.svg b/writing/diagrams/listed-not-started.svg new file mode 100644 index 0000000..0cfa3b2 --- /dev/null +++ b/writing/diagrams/listed-not-started.svg @@ -0,0 +1,35 @@ + + Rush and Lusk: the claim, and the part anyone saw + + + + Irish Rail's start, 451.6 days before the first poll + + + + polled every 30 minutes from here + + + + + + + 14 May 2025 + 8 Aug 2026 + 14 Aug + + + listed: 136.5 hours + + the only interval the site measures + + + + nobody was looking + + Not to scale: fifteen months on the left of the break, nine days on the right. + diff --git a/writing/diagrams/what-would-carry-it.svg b/writing/diagrams/what-would-carry-it.svg new file mode 100644 index 0000000..c6e3d1d --- /dev/null +++ b/writing/diagrams/what-would-carry-it.svg @@ -0,0 +1,36 @@ + + What would carry it, and what Ireland publishes + + + Written for this question + + + + + + NeTExlifts, ramps, entrances, paths between them + SIRI-FMlive "is this lift working" + GTFS pathways.txtstep-free route, entrance to platform + none published for Ireland + + + Actually published + + + + + + GTFS, 10 filesno wheelchair_boarding column, no hierarchy + NaPTAN, PTIMSAccessArea null on all 152 rail stops + irishrail.ie platformAccessfree text, typed by hand, no schema + the last one is the source this site uses + + + EU 2017/1926 obliges publishing the listed data types "provided they exist in digital + machine-readable format". The duty is to publish what you hold, not to create it, so the + absence on the left is lawful and has no process that closes it. + diff --git a/writing/figures.md b/writing/figures.md new file mode 100644 index 0000000..4fd47d0 --- /dev/null +++ b/writing/figures.md @@ -0,0 +1,449 @@ +# Figures registry + +Every number quoted in a chapter gets a row here. *Source* is a pull request number, a commit +subject, a `notes/` section heading, an issue number, a README section, or **measured**, which +means a read-only check run by the writing session against the real corpus. + +Unlike the two sibling series, this one had the data to hand, so the current figures are +measured rather than lifted. Historical figures are quoted as measured on their stated date and +say so where the number has since moved. + +## Measured 24 September 2026 (Session 3) + +Everything in this section was re-run on 24 September after merging `main` (through PR #58), +with `../lifts-data` pulled to commit `cd7ea6a`. Same commands as the 12 September block below. +Chapters 00, 17, 18, 19a and 19b quote this block; chapters 13 to 16 quote the 12 September one. + +### The corpus + +| Figure | Value | How | +|---|---|---| +| Runs recorded | 2,224 | `stats` | +| Run outcomes | 2,217 ok, 7 unreachable | `stats` | +| Coverage | 2026-08-08T21:30:55Z to 2026-09-24T05:02:51Z, 46 days | `stats` | +| Messages tracked | 737 (13 open, 724 closed, 10 reopened at least once) | `stats` | +| Unidentifiable items | 952 | `stats` | +| Raw log size | 10.5 MiB; database 1.4 MiB | `stats` | +| Outages after merging | 69, across 44 stations; 5 escalator, 9 planned | `lift_site.model.load_outages` | +| Listed at the horizon | 4 lift and 1 escalator notice across 5 stations (HWTHJ, BROCK, BTSTN, TMORE lift; PERSE escalator) | site build | +| Notices on record for the access report | 58 | `report` | +| Verdicts | 31 lost, 23 unknown, 4 escalator | `report`, counting `->` lines | +| Unknown by reason | 13 page does not mention a lift; 10 notice names a platform the page puts no lift at | same | +| Unknown share over time | 6 of 24 (31 Aug), 7 of 30 (4 Sep), 14 of 45 (12 Sep), 23 of 58 (24 Sep) | the four measured blocks | +| Tests | 526, OK, 11 skipped with `LIFT_STATUS_DATA_DIR` set; 34 skipped without | `unittest discover` | +| Commits on `main` | 215, of which 142 carry `Co-Authored-By` | `git log origin/main` | +| Trailer identifiers | seven: Opus 5 (78), Fable 5 (27), Fable 5.1 (18), Opus 5 (1M context) (9), Sonnet 5 (4), Opus 5.5 (3), bare "Claude" (3) | same | +| Merged pull requests | 48 | `gh pr list --state merged` | +| Open issues | none | `gh issue list` | + +### The site + +| Figure | Value | How | +|---|---|---| +| `index.html` | 61.5 KB | site build | +| `data.js` | 7.9 KB | site build | +| Initial load | 69.4 KB against a 500 KB budget | site build | +| `outages.csv` | 28.7 KB, on demand | site build | +| `feed.xml` | 43.0 KB, on demand | site build | +| Station pages | 1,243.7 KB over 44 files | site build | +| Shards | 32.3 KB over 44 files, largest `PERSE.js` at 2.5 KB | site build | +| August national row | 21 stations, 28 outages, 22 faults, 6 planned, **75%**, 2 still out at month end | `national_month` | +| August grade mix | A 2, B 1, C 4, D 7, E 5, F 2 | `station_month` per station | +| September national row, so far | 30 stations, 43 outages, 39 faults, 4 planned, 85%, 5 ongoing | `national_month` | +| September grade mix, so far | A 2, B 7, C 10, D 6, E 5 | `station_month` per station | + +### Chapter 17 + +| Figure | Value | How | +|---|---|---| +| `PLANNED_MARKERS` | "planned works", "planned maintenance", "engineering works" | PR #56, `lift_site/model.py` | +| Salthill and Monkstown, notice 1 | platform 1, planned maintenance, 2026-09-11T13:32:31Z to 2026-09-15T14:32:55Z; Irish Rail's start 2025-11-03 | database, messages 2049 | +| Salthill and Monkstown, notice 2 | platforms 1 and 2, planned maintenance, from 2026-09-16T14:01:41Z; reissued 2026-09-17T09:01:12Z as a fault with head "Station - Lift out of order", closed 2026-09-17T23:02:21Z | database, messages 2184 and 2189 | +| Salthill and Monkstown, September | old marker E 66% (8 of 24 days against); new marker C 91% (2 of 24) | `station_month` with the tuple swapped | +| National September, old marker vs new | 85% both; planned 3 vs 4; C 9 vs 10, E 6 vs 5 | same | +| Kishoge's `stationCode` | the literal name "Kishoge" | PR #55; `snapshot.load` with the fixup table emptied | +| Feed location codes joining a station record | 76 of 76 with the table; 75 without, the miss being KISHO | `messages.location_codes` against `snapshot.load` | +| Notices at KISHO | 1, "Delays of up to +15mins" | database | +| Snapshot codes not matching `[A-Z0-9]+` | none with the table; "Kishoge" without | same | +| August, 12 Sep corpus vs today, same code | 76% and A 3, then 75% and A 2; only Tullamore differs | raw logs cut at 2026-09-12 morning, rebuilt in a scratch dir | +| Tullamore, August | A 100% then D 83% (4 of 24 watched days against) | `station_month` | +| Tullamore's notice, stretch 1 | 2026-08-28T15:01:32Z to 2026-09-01T09:02:41Z, 3 d 18 h 01 m, planned | `load_outages` | +| Tullamore's notice, stretch 2 | from 2026-09-16T14:01:41Z, still listed; pooled planned total 11 d 09 h at the horizon | same | +| Irish Rail's start on Tullamore's notice | 31 Aug 2026 00:00 Dublin | station page | +| When the pooled total crossed a week | about 2026-09-19T20:00Z, by arithmetic | 7 d minus 3 d 18 h 01 m 09 s after stretch 2 opened | +| Tullamore, September so far | E 58% | station page | + +### Chapter 18 + +| Figure | Value | How | +|---|---|---| +| Findings in the collector review | 10: 8 fixed, 1 not a real path, 1 left | `notes/collector-review.md` | +| Collector files unchanged since 18 Aug | `parse.py`, `client.py`, `__main__.py`; `poll.py` by 16 lines | same | +| Diff-by-diff review of the rest | since 2026-08-26 | same | +| Database failure exit code | 7 | same; `lift_status/alert.py` | +| Alert marker clears after | 4 consecutive clean runs, two hours | same | +| `TimeoutStartSec` | 60, now 300; client worst case 3 attempts of 15 s connect and 15 s read, plus backoff and DNS | same | +| Backup unit cap | 15 minutes | PR #58 | +| Raw lines already in time order | all 2,224, so no rebuild moved | `notes/collector-review.md`; PR #58 verification | +| `fake-hwclock` skew after a power cut | up to an hour | `notes/collector-review.md` | +| First review of the access branch | 9 findings | ch 12 § The golden file | + +## Measured 12 September 2026 (Session 2), quoted by chapters 13 to 16 + +Session 0 measured on 31 August and Session 1 on 4 September. Everything in this section was +re-run on 12 September after merging `main`. Where a chapter quotes an earlier figure it says so, +and the row is in one of the dated blocks below. + +Run from `/Users/barry/Code/lifts` with `../lifts-data` pulled to its 12 September state, then: + +```bash +python -m lift_status --data-dir ../lifts-data rebuild +python -m lift_status --data-dir ../lifts-data stats +python -m lift_site --data-dir ../lifts-data +python -m lift_access --data-dir ../lifts-data report +``` + +### The corpus + +| Figure | Value | How | +|---|---|---| +| Runs recorded | 1,648 | `stats` | +| Run outcomes | 1,645 ok, 3 unreachable | `stats` | +| Coverage | 2026-08-08T21:30:55Z to 2026-09-12T05:01:41Z | `stats` | +| Collection horizon at build | 2026-09-12 05:01Z, 2.8 h behind the build | site build | +| Messages tracked | 497 (29 open, 468 closed, 7 reopened at least once) | `stats` | +| Unidentifiable items | 587 | `stats` | +| Raw log size | 6.2 MiB | `stats` | +| Outages after merging | 53, across 37 stations | site build | +| Notices on record for the access report | 45 | `report` | +| Listed at the horizon | 3 lift notices across 3 stations, 0 escalator | site build | +| `LIFT_STATUS_GRACE_MISSES` default | 2 | `lift_status/store.py:24` | + +### The site + +| Figure | Value | How | +|---|---|---| +| `index.html` | 61.2 KB | site build | +| `data.js` | 7.1 KB | site build | +| Initial load | 68.3 KB against a 500 KB budget | site build | +| `outages.csv` | 22.4 KB, on demand | site build | +| `feed.xml` | 43.3 KB, on demand | site build | +| Station pages | 1,038.4 KB over 37 files | site build | +| Shards | 25.4 KB over 37 files, largest `PERSE.js` at 1.8 KB | site build | +| `STALE_AFTER` | 10 hours | `lift_site/render.py` | +| Lift availability, August 2026 | 76% | `data.js` `national["2026-08"]` | +| August national row | 21 stations, 28 outages, 22 faults, 6 planned, 76%, 2 still out at month end | same | +| Lift availability, September so far | 79% | `data.js` `national["2026-09"]` | +| September national row | 22 stations, 27 outages, 26 faults, 1 planned, 79%, 3 ongoing | same | +| Grade mix, August, 21 station-months | A 3, B 1, C 4, D 6, E 5, F 2 | `data.js` `stats` against `bands` | +| Grade mix, September so far, 22 station-months | A 2, C 7, D 10, E 1, F 2 | same | +| Band table | A 100, B 95, C 90, D 75, E 50, F 0 | `data.js` `bands` | + +### Listings and start dates + +| Figure | Value | How | +|---|---|---| +| Longest lead: Rush and Lusk | 451.6 days, from a start of 2025-05-14 08:00Z against a first sighting at the very first poll, 2026-08-08T21:30:55Z | `lift_site.model.load_outages` | +| Outages whose start predates their first sighting, 31 Aug | 23 of 24, 12 by seven days or more | same, measured 31 Aug 2026 | +| Next four leads | Docklands 253.4, Dublin Pearse lift 242.9, Hazelhatch 237.6, Thurles 197.4 | same | +| Further leads quoted | Pearse escalator 146.1, Ballinasloe 123.9, Skerries 118.5, Ballybrophy 100.5 | same | +| The one negative lead | Tullamore, minus 2.3 days (works announced in advance) | same | +| Listing durations, hours | 6.5 to 541.5; median 62.25 | same | +| Shortest listing | Portarlington, 6.5 h | same | +| Longest listings | Athy and Midleton, 541.5 h and still listed | same | +| Hazelhatch listing | 48.5 h | same | +| Outages carrying a folded reissue | 0 | same | + +### Station facts + +| Figure | Value | How | +|---|---|---| +| Stations in the snapshot | 152 | `stations/irishrail-20260830.jsonl` | +| Stations recorded as having a lift | 57 | `report`, `model.has_lift` | +| Stations recorded as having none | 95 | same | +| Station snapshots on record | `irishrail-20260830.jsonl`, `irishrail-20260901.jsonl` | `lifts-data/stations/` | +| Prose mentioning "lift" before boilerplate stripping | 61 | `model.LIFT` over `platform_access` | +| ... after stripping | 58 | `model.strip_boilerplate` then `model.LIFT` | +| Difference explained | 3 boilerplate-only (Greystones, Killiney, Donabate), then Dromod's explicit denial | `notes/station-access.md` | +| `platformAccess` naming an escalator | 2 of 152: Tara Street, Dublin Pearse | `model.ESCALATOR` | +| `ticketOfficeAccess` naming an escalator | 1: Dublin Connolly | same | +| Stations with any `ticketOfficeAccess` text | 143 of 152 | snapshot | +| Verdicts across the 45 notices | 28 lost, 14 unknown, 3 escalator | `report` | +| Unknown share over time | 6 of 24 (31 Aug), 7 of 30 (4 Sep), 14 of 45 (12 Sep) | `report`, each date | +| Step-free pill rendered on the live site | never; `stepfree` is empty | `data.js` | +| Surveyed stations (`survey-pilot` branch of `lifts-data`) | 5: ATHY, CNLLY, CNOCK, HZLCH, PERSE | `git ls-tree origin/survey-pilot survey/` | +| `survey/` on `lifts-data` `main` | absent | same | + +### The repository + +| Figure | Value | How | +|---|---|---| +| Commits on `main` | 204 | `git log --oneline \| wc -l` | +| Commits with a `Co-Authored-By` trailer | 135, across six Claude model identifiers (77 Opus 5, 27 Fable 5, 18 Fable 5.1, 9 Opus 5 1M, 1 Sonnet 5, 3 unversioned) | `git log --format='%b' \| grep -o 'Co-Authored-By: [^<]*' \| sort \| uniq -c` | +| Merged pull requests | 42, numbered to #54 | GitHub, `baz8080/lifts` | +| Open issues | #52, #53 | GitHub | +| Test count | 516, passing with `LIFT_STATUS_DATA_DIR` set (11 skipped) and with it unset (34 skipped) | `python -m unittest discover -s tests -t .` | +| `notes/` files | site · station-access · accessible-routes · publish-cadence · step-free-graph · delays-site | `ls notes/` | +| First commit | 2026-08-08 | `git log --reverse` | +| Em dashes in `writing/` | 0 | `scripts/no-em-dash.sh` | + +## Measured 4 September 2026 (Session 1), quoted by chapters 10 to 12 + +| Figure | Value | +|---|---| +| Runs / outcomes | 1,264; 1,261 ok, 3 unreachable | +| Coverage | to 2026-09-04T05:01:41Z | +| Messages tracked | 281 (4 open, 277 closed, 4 reopened at least once) | +| Listings | 285 across 281 messages; 4 messages with more than one stretch | +| Unidentifiable items | 323 | +| Outages after merging | 34 across 27 stations; 8 planned, 3 escalator | +| Lift availability | August 76%, September so far 62% | +| Grade mix, August | A 3, B 1, C 4, D 6, E 5, F 2 | +| Grade mix, September so far, 8 station-months | A 2, D 3, E 1, F 2 | +| August availabilities, sorted | 0, 0, 54, 54, 70, 70, 70, 83, 83, 87, 87, 87, 87, 91, 91, 91, 91, 95, 100, 100, 100 | +| Verdicts across 30 notices | 20 lost, 7 unknown, 3 escalator | +| Lost verdicts carrying a kept-platform note | 5: Pearse, Dún Laoghaire, Malahide, Portarlington, Tullamore | +| Initial load | 64.6 KB | +| Commits / trailers / tests | 182 / 121 / 390 | + +## Measured 31 August 2026 (Session 0), quoted by chapters 00 to 09 + +Kept because chapters 02, 05, 07 and 09 quote the corpus as it stood before the September +changes, and say so where they do. + +| Figure | Value | +|---|---| +| Runs / outcomes | 1,084; 1,081 ok, 3 unreachable | +| Coverage | 2026-08-08T21:30:55Z to 2026-08-31T11:01:41Z | +| Messages tracked | 234, of which 24 classify as lift or escalator (22 lift, 2 escalator) | +| Unidentifiable items | 264 | +| Outages after merging | 24 across 21 stations; 6 planned, 2 escalator | +| Aggregate availability, August | 67%, with escalators counting | +| Grade mix, 21 station-months | A 1, B 1, C 5, D 5, E 4, F 5 | +| Availabilities, sorted | 0, 0, 20, 25, 29, 54, 66, 70, 70, 83, 87, 87, 87, 87, 91, 91, 91, 91, 91, 95, 100 | +| Dublin Pearse | F, 20%: 6 lift cells inside grace, 19 escalator cells overrun, 24 days watched | +| Dublin Connolly | C, 91%: 0 lift cells, 2 red escalator cells | +| Tullamore | A, 100%, over four planned-works cells | +| Verdicts across 24 notices | 16 lost, 6 unknown, 2 escalator | +| Listing durations | 6.5 h (Portarlington) to 541.5 h (Athy, Midleton); median 62.25 | +| Initial load | 64.3 KB | +| Commits / trailers / tests | 139 / 88 / 287 | + +Several of these are not merely stale, they were **wrong**, and chapter 10 is why: Portlaoise's +29%, Thurles' 25% and Clondalkin's grade all included a fortnight the notice was not on the +feed. The August aggregate and the grade mix moved again in chapter 11 when escalators stopped +counting. + +## Quoted at the date they were measured (not re-run) + +### Ch 01 + +| Figure | Value | Source | +|---|---|---| +| Lift/escalator notices in the first corpus | 17 of 113 | `notes/site.md` preamble, 18 Aug 2026 | +| `sort_keys=True` is load-bearing | present in `store.write_raw` | `CLAUDE.md` § The invariant | + +### Ch 02 + +| Figure | Value | Source | +|---|---|---| +| Starts predating first sighting, at PR #2 | 14 of 17, and 12 by a week or more | `notes/site.md` § The measured interval, 18 Aug 2026 | +| Batch arrivals | 6 at the first poll; 3 new at 10 Aug 14:30; 4 at 13 Aug 10:30; 2 at 17 Aug 14:02; 3 removed together 14 Aug 14:01 | same | +| `end` as a placeholder | 13 of 17 near the year end | `notes/site.md` § `end` is shown, 18 Aug 2026 | +| Docklands gap | closed 14 Aug 14:01, new notice 17 Aug 14:02 | `notes/site.md` § Notices reissued | +| The non-lift reissue chain | `Station currently closed` to `CLOSED` to `Station is OPEN`, ids 42, 45, 46 | same | +| Initial payload at PR #2 | 30 KB against 500 KB | PR #2 | +| Stations listed in August at PR #2 | 15 | PR #2 | +| The power site's `startTime` | 8 revisions in 1,460 records; median lag about one poll | esb `notes/grading.md`, 18 Aug 2026 | + +### Ch 03 + +| Figure | Value | Source | +|---|---|---| +| Drift while vendored | this site and esb on one statusui commit, uisce five UI commits behind | `notes/site.md` § The vendored copy became a pinned dependency, 20 Aug 2026 | +| `STALE_AFTER` | 16 hours; widest legitimate gap about 14 h, missed push 17 h+ | `lift_site/render.py`, PR #12 | +| Python floor | `requires-python` 3.11, development interpreter 3.14, both run in CI | PR #13, `CLAUDE.md` | + +### Ch 04 + +| Figure | Value | Source | +|---|---|---| +| PRM TSI | Regulation (EU) 1300/2014: design rules and a written-policy duty, no percentage | PR #18, 28 Aug 2026 | +| Irish Rail Passengers' Charter | "every effort ... available as advertised" | same | +| Big Lift | 52 stations, 2020 to 2024, no availability figure published | same | +| ORR / Network Rail | 8,696 lift faults in a year, 6.6 per lift, over 20 hours average repair | same | +| TfL | 93.7% lift availability, 98.8% excluding planned works | same | +| One listed day over 31 | 96.8% available | arithmetic, 30/31 floored | +| Bands at PR #18 | A 100, B 95, C 90, D 75, F below | PR #18 | +| Grace outcomes | Pearse 5 days and Greystones 2 forgiven; Limerick Junction 10 and Midleton 19 not | `notes/site.md` § Planned works are excused for a week, 28 Aug 2026 | +| The grace worked example | 6 days works then 4 of fault: version 3 gives 60% | `notes/site.md`, same section | +| The skew crash | Midleton at 13 h skew: observed 20 against 21, availability minus 5, `StopIteration` | PR #18 review notes | + +### Ch 05 + +| Figure | Value | Source | +|---|---|---| +| Availability before and after counting escalators | 70% to 66% | `notes/site.md` § An escalator out is a day the station was short of a way up, 29 Aug 2026 | +| Connolly and Pearse, same change | A to C, and A to F at 22% | same, and PR #25 | +| The old F band's nine values | 0, 0, 18, 22, 22, 50, 68, 68, 72 | PR #27, 29 Aug 2026 | +| Cuts at 60 and 40 | split the same nine values identically | same | +| Grade mix before and after E | A 1 B 1 C 5 D 5 F 9, becoming A 1 B 1 C 5 D 5 E 4 F 5 | same | +| E over a 31-day month | 8 to 15 days listed; F 16 or more | arithmetic, floor division | +| Chip contrast work | B measured Lc 38.6 on dark ink against 69.2 on white; the no-grade dash failed at 4.24:1 light and 3.90:1 dark | PR #27, via statusui#11 | + +### Ch 06 + +| Figure | Value | Source | +|---|---|---| +| GTFS archives | `GTFS_Irish_Rail.zip`, `GTFS_All.zip`, `GTFS_Realtime.zip`, ten files each | `notes/accessible-routes.md`, checked 30 Aug 2026 | +| `pathways.txt`, `levels.txt` | absent from all three | same | +| `wheelchair_boarding` | column absent; header ends at `parent_station` | same | +| `location_type` | empty for all 152 rail stops | same | +| NaPTAN | 152 rail stops, `AccessArea` null on every one, "accessib" absent from 22 MB | same | +| `getAllStationsXML` | up, unkeyed, 171 stations, no accessibility data | same | +| NTA catalogue searched | 24 GTFS archives, NaPTAN, PTIMS, nothing else | `notes/station-access.md`, 30 Aug 2026 | +| The regulation | Commission Delegated Regulation (EU) 2017/1926, Annex, "provided they exist in digital machine-readable format" | same | +| The three GTFS fields mapping apps read | `wheelchair_boarding`, `wheelchair_accessible`, `pathways.txt`, all absent | same | +| Code-space join | all 15 codes with lift notices matched, 15/15 | PR #30 | +| `alert` staleness | 131 stations carry one, never cleared, `alertEnd` back to 2014 | `notes/station-access.md` | +| Snapshot size | 7.8 MB plain, about 2 MB in git | same | + +### Ch 07 + +| Figure | Value | Source | +|---|---|---| +| The 61-station reading | 29 "and" as a sequence, 11 "or stairs", 2 real alternatives | `notes/station-access.md` § "and" is a sequence, 30 Aug 2026 | +| The two exceptions | Raheny "Lift or ramp to platform 1"; Cork "Ramp or lift to platform 5A, 5B and 6" | same | +| Boilerplate-only lift mentions | Greystones, Killiney, Donabate | same | +| Dromod | the one explicit "(no lift at this station)" | same | +| Verdicts at PR #30 | 18 of 24 resolve, 2 escalator, 6 unknown | PR #30, 30 Aug 2026 | + +### Ch 08 + +| Figure | Value | Source | +|---|---|---| +| The rebuild transcript | 228 messages and 1,012 runs before; 0 and 0 after; exit code 0 | PR #34, 30 Aug 2026 | +| The alert marker transcript | `delivered: False`, marker written, second attempt suppressed | same | +| `ALERT_REPEAT_SECONDS` | 24 hours | same | +| Partial fetch | 38 empty bodies in an 8 MB diff would go unnoticed | `notes/station-access.md` § Three things a review caught | +| OSM, verdicts changed | 0 of 24, with a synthetic digest mapping a lift at all 152 stations | `notes/station-access.md` § OpenStreetMap, 30 Aug 2026 | +| OSM, level tags | 2 of 12 sampled stations, both Dublin termini | same | +| OSM, stations it spots | 13 where the prose mentions no lift and OSM maps one | same | +| OSM, what it cost | about 60 lines, a monthly rate-limited HTTP budget, and a derived rather than verbatim artefact from roughly 450 MB of extracts | same | + +### Ch 09 + +| Figure | Value | Source | +|---|---|---| +| Pearse and Connolly, graded against lifts only | Pearse F 21% against A 100%; Connolly C 91% against A 100%; national 67% against 70% | issue #32, 30 Aug 2026 | +| Pearse's August notices | lift at platform 2 for 5 days (inside grace), escalator at platform 2 for 16 days (overran) | same | +| Platforms reached without a lift | 32 of 57 stations that claim a lift; 12 of the 21 that have had a notice | issue #31 and `notes/station-access.md`, 30 Aug 2026. Recorded, not re-derived: a quick re-derivation with a narrower rule gives 27 and 10, so the figure is sensitive to how "named without a lift" is defined and the recorded derivation is the one to trust | +| Direction named in the prose | 10 of 57 stations | issue #31 | +| Platforms that would need hand labelling | roughly 120 | same | +| `ticketOfficeAccess` present | 143 of 152 stations | issue #33 | +| Stations naming an escalator | Pearse and Tara Street in `platformAccess`, Connolly in `ticketOfficeAccess`; all three also have lifts | same | + +### Ch 10 + +| Figure | Value | Source | +|---|---|---| +| Cron actual start times, week to 1 Sep | `40 5` at 10:24, 10:39, 11:46, 11:48, 13:38, 16:46, 17:41; `40 12` at 16:51, 16:53, 16:56, 19:13, 22:33, 22:36 | PR #39, `notes/publish-cadence.md`, 2 Sep 2026 | +| Normal jitter before 26 Aug | runs started 05:58 to 06:06Z against a `40 5` cron | same | +| The reported string | a 10:24Z build saw data to 23:22Z; a 16:53Z build saw data to 11:19Z; 21.5 h old by 09:47 | same | +| Push cadence and worst-case age | six-hourly, capping age at about 7 h against about 13 h before | same | +| `STALE_AFTER` | 16 h to 10 h | same | +| Live lag measured while implementing | 10.2 h at 09:12Z | same | +| Portlaoise as published | 16 days listed, F, 29% available | PR #42, 3 Sep 2026 | +| What actually happened at Portlaoise | 20 h from 10 Aug, absent from 672 consecutive successful polls, 22 h from 25 Aug | same | +| Grade moves from the split | Portlaoise F 29% to D 83%; Thurles F 25% to E 54%; Clondalkin F to E 70%; Pearse and Midleton unmoved | same | +| Reopens the counter had recorded | 6 in the first month, 5 of them lift or escalator notices | `notes/site.md`, 2 Sep 2026 | +| Athy's blink | absent from exactly one poll on 21 Aug, back 29 minutes later | same | +| Gap sizes in the corpus | 1, 9, 79, 388 and 672 polls | same | +| Midleton's real block | 1,087 consecutive successful polls, no gaps | PR #42 | +| The pooled planned total | without it, a 4-hour blip took the Pearse escalator from 20% to 41% | same | +| The double-counted chain total | 5.5 days of works reported as 9 | same | + +### Ch 11 + +| Figure | Value | Source | +|---|---|---| +| The reverted label column | 64px, appeared on the one station with two bars, put its day 14 over every other row's day 15 | PR #38, 1 Sep 2026 | +| The gutter | 15px on every row, 84px with the word on station pages | same | +| Alignment measured | all 21 August rows start their days at the same x at 980px and 500px | same | +| Glyph cost | about 730 bytes on a 68 KB initial load | same | +| Review findings on that branch | 6, including three tests that passed with the feature removed | same | +| Grade moves from taking escalators off | Connolly C 91% to A 100%; Pearse F 20% to A 100%; August national 72% to 76%; Tara Street F 33% to A 100%; September national 50% to 61% | PR #43, 3 Sep 2026, corpus to 05:00Z | +| The narrower denominator that was rejected | 75% instead of 76% for August, 53% instead of 61% for September | same | +| Stations claiming a lift among the escalator three | Pearse, Connolly and Tara Street all claim one, so the only-powered-way-up rule has no instance | same | + +### Ch 12 + +| Figure | Value | Source | +|---|---|---| +| Lost verdicts gaining a kept-platform note | 5: Pearse, Dún Laoghaire, Malahide, Portarlington, Tullamore | PR #37, 3 Sep 2026 | +| Stations naming a platform reached without a lift | 32 of the 57 that claim a lift | same, and issue #31 | +| Leg detection over 24 distinct notice texts | 19 platform, 1 entrance (Connolly), 4 unlocated, no false entrance hits | PR #45, 3 Sep 2026 | +| `ticketOfficeAccess` across 152 stations | 9 blank, 26 no ticket office, 89 level, 21 ramp; 4 name a lift (Connolly, Clondalkin, Docklands, Grand Canal Dock), 1 names an escalator (Connolly) | same | +| Escalator verdicts that moved | 3: Pearse, Connolly, Tara Street; no lift verdict moved | same | +| Overlaps found by the guard | 0. Pearse's lift listing closed at the exact poll its escalator's opened, 2026-08-13T10:30:46Z | same | +| Review passes and findings | 9, then 6, then 5, then 4; two fixes were themselves regressions | same | +| The regressions caught by hand | Carrigaloe's and Dalkey's "platform No 1" lines dropped; Banteer's and Booterstown's hidden by a split at "No." | same | +| Corpus growth in distinct notice texts | 21 in 26 days | same | +| Reliability by class, corpus to 3 Sep (27 notices) | 18 lost, 6 unknown, 3 escalator; entrance leg has 0 lift notices and 1 escalator notice | `notes/station-access.md` § How reliable this is, honestly, 3 Sep 2026 | +| Stations with no ticket office | 26 of 152, so a sixth of the network is unknown on the entrance leg by construction | same | + +### Ch 13 + +| Figure | Value | Source | +|---|---|---| +| The survey that produced the work | the site read against a corpus of 36 outages over 28 stations | PR #49, 6 Sep 2026 | +| Initial load change | 64.8 KB to 65.8 KB; feeds and CSV off it entirely | same | +| Feed sizes | `feed.xml` carries the 50 most recent outages; `s/.xml` carries a station's every outage | same | +| `thin_days` threshold | fewer than 40 polls in a Dublin day, a day with none included | same | +| `unclassified_mentions` findings today | none | same | +| Name column widening | 170px to 230px above 780px | PR #50, 7 Sep 2026 | +| Why 780px | below about 740px the fixed columns plus a 31-day bar at its 3px-a-cell floor stop fitting | same | +| Placements compared | four, rendered against the real CSS | same | +| Overflow checked at | 500, 560, 641, 660, 700, 740, 779, 781, 800, 900, 1200, 1600px | same | + +### Ch 14 + +| Figure | Value | Source | +|---|---|---| +| The failing message | `notice RLUSK lift: dropped`, on `main` | PR #51, 8 Sep 2026 | +| The corrected assumption | "the logs are append-only and that can only mean a bad checkout" | `notes/station-access.md`, corrected in place 8 Sep 2026 | +| The reword | Rush and Lusk's body flipped from "platform 2" to "platform 1" overnight, 8 Sep 2026 | PR #51 | +| Failures in five days | 3: a statusui bump (#48), a Midleton reword (#49), this one | same | +| Fixture size | 53 KB to 115 KB, of which about 23 KB is page fragments across 152 stations | same | +| Determinism check | `lifts-data` checked out 20 commits back, predating the reword; suite green | same | +| Guard still fires | breaking `strip_boilerplate` fails on Donabate, Greystones and Killiney | same | + +### Ch 15 + +| Figure | Value | Source | +|---|---|---| +| The reversed decision | `notes/accessible-routes.md` § What is deliberately out, 29 Aug 2026 | quoted in full in the chapter | +| Business case | Iarnród Éireann to the NTA, PBC-3.5, 30 October 2024, 188 pages | `notes/step-free-graph.md`, 4 Sep 2026 | +| Table 6-2 | 51 stations not yet meeting the standard, in packages of 15, 15 and 21 | same | +| Prior audits named | 2014 feasibility report over 54 stations, 2019 review, 2021 preliminary design reports for the first fifteen | same | +| Metro Nation map | 12 stations glyphed "no step-free access"; disagrees with Irish Rail's page at Athy, Carlow and Ashtown | same | +| Confidence levels | `low` read off a page, `medium` told or a reviewed sentence, `high` seen | same | +| Pilot stations | Hazelhatch, Pearse, Connolly, Athy, Castleknock, all seeded page-sourced at low confidence | same | +| Where the graph says less than the prose | Hazelhatch (no platform named) and the Pearse escalator (page's only escalator is on the way in) come back unknown | same | +| Raheny under the confidence gate | ramp medium, way in low, so the graph says lost where the prose says alternative | same | +| The correction issue | live since 30 Aug 2026, has never fired | same | +| Tests at that merge | 490 | PR #46 | + +### Ch 16 + +| Figure | Value | Source | +|---|---|---| +| Location codes emptied part-way | lift and escalator 55 notices, 0 lose codes; everything else 497 notices, 21 never carry them, 288 lose them partway | PR #54, 11 Sep 2026 | +| The planned-maintenance notice | Salthill and Monkstown, first seen 2026-09-11T13:32:31Z, still listed | issue #53 | +| Disagreement count | 1 in 416 distinct notices, between the delays site's cause reader and `is_planned` | same | +| The cause reader's planned wordings | "planned works" (7 notices), "engineering works" (3), "planned maintenance" (1) | same | +| Kishoge | station facts keyed `"Kishoge"` rather than `"KISHO"`; no lift notice there yet | issue #52 | + +## Open `[verify:]` items + +None. Every number quoted in the chapters resolves to a row above. diff --git a/writing/outline.md b/writing/outline.md new file mode 100644 index 0000000..05be155 --- /dev/null +++ b/writing/outline.md @@ -0,0 +1,309 @@ +# Outline - 18 posts plus intro and a two-part closing, chronological + +Each entry: PRs and dates, thesis, concepts boxed, worked example, and the three-way contrast +the chapter must state. The repo's history is small enough to read directly (215 commits, 48 +merged pull requests, seven `notes/` files, no open issues), so there is no `sources/` extraction +as the uisce series needed; `figures.md` is the registry. + +The series' standing mandate, on top of the shared rules: **every fork from the sibling sites +is stated as (uisce's approach, esb's approach, ours, and the fact about this feed that forced +it).** The four that anchor chapters are tabulated in `README.md`. + +The shape of the series is the argument. Chapters 01 to 05 are the site anyone would expect: +collect, measure, publish, grade. Chapters 06 to 09 are what happened when the site tried to +say what any of it *meant*, which is the part that was not foreseen and is the reason this +series exists separately from the other two. Chapters 10 to 12 are the four days in early +September when everything chapter 09 left open closed, in an order nobody predicted. Chapters 13 +to 16 are the week after, in which the project stopped working around the missing data and +started recording it, and two of the series' own settled decisions were reversed. + +--- + +## Ch 00 - The easiest of the three · intro + +The question: which Irish Rail stations have lifts out, and for how long. The third site of a +family, built to a pattern that already worked twice. Then the turn, stated up front so the +back-loading reads as design: what a lift outage *means* needs a station inventory, Ireland +publishes none, and the only machine-readable statement of what a station has is a hand-typed +CMS field. Today's answer with today's date. AI process named once (204 commits, 135 with a +`Co-Authored-By` trailer, across six Claude model identifiers). + +## Ch 01 - A feed that is not about lifts · PR #1 · 8 to 18 Aug + +**Thesis.** Write it down before you read it: raw JSONL verbatim before any parse, the database +disposable, `rebuild` replaying the live code path, `sort_keys=True` load-bearing so two +machines' logs merge with `sort -u`. The feed is every service banner, not a lift feed: 24 of +234 messages qualify. There is no id, so identity is `head` + sorted `locationCodes` + `start`. +There is no completion signal, so an outage ends when its notice is first absent. And a failed +run must never read as an empty list, which `poll.py` enforces structurally rather than +checking. **Concepts.** Source of truth against derived index; a run that failed is not a run +that saw nothing. **Example.** The 264 unidentifiable items, and why they are the right kind of +mess to keep. **Contrast.** uisce's archive *is* its database; esb's feed purges within hours +and that is what forced a Pi. Here the feed is patient, so the Pi is inherited rather than +derived, and the cost of that inheritance is nearly zero. Mermaid pipeline diagram. + +## Ch 02 - The start date that is 451 days old · PR #2 · 18 Aug + +**Thesis.** The site measures the listing, not Irish Rail's start. 23 of 24 notices carry a +start that predates the poll they were first seen at, 12 of them by a week or more, over days +the feed was polled every 30 minutes and the notice was not there. So the bars, the month +filing and every duration run `first_seen` to `end`; the start is printed as Irish Rail's claim +and colours nothing. `end` is a year-end placeholder. "No longer listed", never "fixed", +because notices arrive and vanish in batches. Reissues fold on an exact poll match and a gap of +one poll is two outages. Everything a reader sees is Dublin wall-clock, after a bug that filed +a 31 August notice under August while its own summary read 1 September. **Concepts.** Measure +the window you actually watched; two clocks for one date is a bug in either direction. +**Example.** Rush and Lusk at 451.6 days, and Tullamore at minus 2.3. **Contrast.** The +operator-start row of the table in full: this is the sharpest three-way split in the series. +SVG `listed-not-started.svg`. + +## Ch 03 - Three sites, one design layer · PRs #3 to #17 · 19 to 26 Aug + +**Thesis.** The short chapter, and it says so in its first line, because the other two series +already tell this story from their side. Vendored on the 19th, drifted within a day, a pinned +git dependency by the 20th, with `dependencies` staying literally empty because the Pi install +is a file copy and nothing else. The design alignment pass. Twice-daily pushes and a 16-hour +stale threshold, sized above the widest legitimate gap and below a missed slot. A 3.11 floor +checked in CI rather than declared. **Concept.** An empty dependency list as a deployment +contract. **Example.** The staleness arithmetic. **Contrast.** One paragraph, pointing at +uisce ch 14 and esb ch 6a. + +## Ch 04 - A grade with nothing to borrow · PR #18 · 28 Aug + +**Thesis.** This reverses a settled decision, and the note records the reversal dated. There is +no Irish or EU availability target: the PRM TSI sets design rules and a written-policy duty, +the Passengers' Charter promises "every effort", the Big Lift programme publishes no figure. +The regulators who publish numbers are elsewhere (ORR/Network Rail, TfL). So the bands are this +site's own, calibrated in days a reader can count, because bands tuned for TfL's 98% put every +station in the bottom band at day granularity. Planned works are excused for their first week, +measured over the planned segments summed, after review killed two wrong versions of that rule. +And the build-killer: the bar stops at the build clock, the window at the collection horizon, +and those come from different machines, so availability went negative. **Concepts.** A scale +with no anchor; a band calibrated in the unit the bar is drawn in. **Example.** Six days of +works then four of fault, under all three versions of the grace rule. **Contrast.** The +grade-anchor row: esb could grade against a published promise, uisce had to invent one from +population, and this site had to invent one *and* had no magnitude to invent it out of. + +## Ch 05 - The grade argued with the bar underneath it · PRs #25, #27 · 29 Aug + +**Thesis.** Dublin Connolly graded A / 100% available directly above two red escalator cells. +Fixing that meant counting escalators, and the honest consequence is that the grade is a +weaker claim than it looked: *Irish Rail reported something out here*, not step-free +availability, because a wheelchair user cannot use an escalator. Then a colour that said two +opposite things (Midleton's 19 days and Pearse's forgiven 6 drew the same blue), amber for +works past their grace rather than a second red because a deuteranope cannot separate orange +from red at cell width, and a legend that keyed a code the page does not use and then left the +top of the page. The scale grows an E at 50%, and the measurement agrees with the arithmetic +when it did not have to. **Concepts.** One colour, two meanings; a cut that lands in a real +gap. **Example.** The E arithmetic over a 31-day month, against the nine availabilities the old +F band held. **Contrast.** The knock-the-grade row, with uisce's binary `KNOCK_CATS` named as +the precedent chapter 09 comes back to. + +## Ch 06 - The data Ireland does not have · issue #24, PR #30 · 30 Aug + +**Thesis.** The heart of the series, and the chapter the whole shape was built for. Every +source checked came back empty: GTFS `pathways.txt` absent from all three NTA archives, +`wheelchair_boarding` not merely unpopulated but column-absent, `location_type` empty on all +152 rail stops, NaPTAN's `AccessArea` null on every one, PTIMS bus street furniture, the NTA +developer API bus-only, `getAllStationsXML` an inventory with no accessibility at all, the +lifts-and-escalators alerts page checked and struck. NeTEx and SIRI-FM are the European formats +that would carry exactly this and Ireland publishes neither. Then the reason, which is the +point of the chapter: Delegated Regulation (EU) 2017/1926 obliges a National Access Point to +publish the listed data types *"provided they exist in digital machine-readable format"*, so +the duty is to publish what you hold and not to create it, and the absence is lawful rather +than an oversight somebody will fix. Then the consequence a reader can feel: Google, Apple, +Transit, Citymapper and Moovit all read the three GTFS fields that are missing, so none of them +can offer wheelchair routing on Irish Rail. Closing turn: the dated snapshots in +`lifts-data/stations/` appear to be the only versioned machine-readable record of Irish rail +station access that exists, which was never the intent. **Concepts.** A National Access Point; +a lawful absence. **Example.** The four-consumer argument. SVG `what-would-carry-it.svg`. + +## Ch 07 - "and" is a sequence, not a choice · PR #30 · 30 Aug + +**Thesis.** What was used instead, and how it was nearly read backwards. The lift-call sentence +is pasted template text and is the only mention of a lift at three stations, so it is stripped +before anything is matched, and stripping it dissolves the Greystones contradiction. Then +Hazelhatch: "All platforms can be accessed via lifts and ramps" read as a choice concludes a +lift outage left access intact, the exact opposite of the truth and in the one direction that +strands a reader. Barry caught it. The 61-station check that followed found 29 sequences, 11 +"or stairs", and two stations in the country naming a real step-free way round a lift. So the +model parses no connectives at all. Specific beats general. A reviewed entry expires with the +sentence it quotes, and expiry means `unknown`, not `lost`. Six of 24 notices come back unknown +and every one is a real discrepancy. Two fields that look useful and are not. A pill that is +deliberately not the access symbol. And a caveat that ends in a prefilled issue link, because +it is the only route by which a fact recorded nowhere can reach the site. **Concepts.** The +safe direction of an error; an inference that expires with its source. **Example.** Hazelhatch, +then the six-row unknown table. SVG `and-is-a-sequence.svg`. + +## Ch 08 - The same bug, three times · PR #30's second review, PR #34 · 30 Aug + +**Thesis.** Three of the second review's findings were the *first* review's findings +reappearing in the code written to fix them, which makes them a habit rather than three bugs. +Underneath them one shape: a predicate over the wrong quantity, which passes vacuously exactly +when the thing it should assert is missing. Audited for across the rest of the codebase, it +turned up twice more, both in the collector and neither with anything to do with the site: a +`rebuild` that wiped 228 messages and 1,012 runs and exited 0, and an alert repeat window that +opened on the attempt rather than the delivery, so one blip at the moment the collector first +breaks buys 24 hours of silence. Plus OpenStreetMap, carried as a second opinion and removed +after being measured: zero verdicts changed, its one signal redundant, and 2 of 12 sampled +stations carrying a `level` tag. **Concept.** A guard that passes because what it checks is +absent. **Example.** The rebuild transcript, before and after. **Contrast.** The one chapter +with no sibling contrast, and it says so. + +## Ch 09 - What one letter cannot say · issues #28, #31, #32, #33 · open 31 Aug, all closed by 3 Sep + +**Thesis.** The open work, written as reasoning rather than a backlog, because the reasoning is +the interesting part. #32 is the sharp one: Dublin Pearse is graded F, the worst band on the +scale, on the strength of an escalator alone, on a page that also tells the reader that outage +did not remove step-free access. Both statements are true and the fine print is honest. The +problem is that one letter is answering two different questions for two different populations, +and the chip is what people read. Weighting is refused because there is nothing to calibrate it +against and it would make a number nobody can reconstruct by counting days; uisce's binary +`KNOCK_CATS` is the precedent. What makes the fix honest rather than a dodge is #33: say who an +escalator outage *did* affect instead of dropping them from the number and saying nothing. #33 +also carries the entrance leg, which is a real limit: the derivation reasons about the platform +leg only, and Connolly's escalator is named in a field it never reads. #31 is the largest +unclaimed win, an order of magnitude bigger than the exception list. And direction labelling is +refused on principle: this is an archive, not a travel planner. **Concept.** One number, two +populations. Kept as the argument stood on 31 August, with forward pointers, because every issue +closed within four days and the arguments are what shaped what got built. + +## Ch 10 - Two ways the page lied about time · PRs #39, #42, #44 · 2 to 3 Sep + +**Thesis.** Two false statements about time, neither the collector's fault. The page read +"collection has stopped" when the *build* had: GitHub's scheduled runs were landing four to ten +hours late every day, so the site now builds on the data landing and the crons are a fallback, +pushes went six-hourly and the threshold followed 16 h to 10 h. And a notice that vanished and +came back unchanged was published as never having left, because the unique identity key left +nowhere to put the second appearance: Portlaoise as one sixteen-day outage rather than two short +ones a fortnight apart. A `listings` table holds one row per stretch; grace goes to 2 because +Athy blinked for a single poll; the planned total is pooled across stretches so a gap cannot +launder a grace week. **Concepts.** The age on the page is the age of the data, not of the +build; a test that exercises the easy half. **Example.** The four-hour blip that would have +taken Pearse from 20% to 41%. **Contrast.** The build-clock and horizon split is shared with +both siblings; this is the first time the *publishing* half of it broke here. + +## Ch 11 - The grade narrows to lifts · PRs #38, #43 · 1 to 3 Sep + +**Thesis.** The order is the point: #28, filed as the least interesting item on the list, was +the blocker. Reserving a 15px kind gutter on *every* row (the 28 August attempt was right except +that it was conditional) means red escalator cells under a green lift chip read as two facts +rather than a contradiction, which was the entire argument that had kept escalators in the +grade. So escalators come off the letter, and the key says "Lift availability" and not the +issue's "step-free availability", because the grade counts notices and a quarter of the access +verdicts are unknown. The only-powered-way-up case is written down with a test that fails the +day it applies. The denominator question a review raised, and why it was kept. **Concepts.** A +conditional column is a misalignment; a rule with no instance, written down and guarded. +**Example.** Pearse and Connolly at A over red escalator strips. **Contrast.** uisce's binary +`KNOCK_CATS`, now matched rather than cited. + +## Ch 12 - Both legs, and who was on the stairs · PRs #37, #45 · 3 Sep + +**Thesis.** #31 and #33, and what came out of them. A lost verdict names the platform that +needed no lift, quoting the sentence, withheld wherever the two sources disagree. A notice's own +text says which leg it is about (19 platform, 1 entrance, 4 unlocated over 24 texts), and the +entrance leg is read against `ticketOfficeAccess`, where four of 152 stations name a lift. The +escalator verdict says who lost a way up in those words, quotes what the page names on the same +leg, and never says a lift was working; the overlap guard covers the one thing the site knows. +The golden file, born of two fixes that were themselves regressions, and its deliberate cost. +Then the section that matters most: a dated, honest account of reliability by class of claim. +**Concepts.** Reading a claim against the right leg; what the code's own history says about the +code. **Example.** Connolly's full verdict sentence. **Contrast.** None, and it says so. + +## Ch 13 - What a reader can take away · PRs #49, #50 · 6 to 7 Sep + +**Thesis.** A survey of the site asking one thing of each screen: what can a visitor do with +this? Five answers were "nothing". A "Lift out" tag beside the name, built from a boolean the +row was already carrying and throwing away; Atom feeds per network and per station; a CSV; a +link back to the page a claim was quoted from. And two guards that are not for visitors at all: +`unclassified_mentions` (a reworded head would drop its notices with nothing failing) and +`thin_days` (a day with two polls paints like a day with 48). The tag's placement took a second +PR and four rendered alternatives; 780px is measured, not chosen. **Concepts.** A feed entry for +something still happening; a tripwire for a silent drop. **Example.** The Midleton reword, found +in passing, which is ch 14. **Contrast.** Neither sibling publishes a feed. + +## Ch 14 - A guard that was guarding the corpus · PR #51 · 8 Sep + +**Thesis.** A red build on `main` from a merge that could not have caused it. The access golden +file pinned outputs but re-derived them from whatever the data repo held when CI ran, so it +failed on two other repositories' schedules. The hole was a sentence written confidently into +the notes: "the logs are append-only and that can only mean a bad checkout". The logs are; the +`messages` table is not, because identity excludes the body, so a reword overwrites `text_raw` +in place. Rush and Lusk flipped overnight; third such failure in five days. Inputs pinned (53 KB +to 115 KB), the test moves to a bare clone, regeneration is additive, and the comparison reports +moves only. The review found the narrowing took the drop check off the other fixture. +**Concept.** A guard over an input you do not control. **Example.** The determinism claim tested +directly by checking the data repo out 20 commits back. **Contrast.** Corrects ch 12 in the +series' own record rather than by editing it. + +## Ch 15 - Building the thing that does not exist · PR #46 · 4 to 8 Sep + +**Thesis.** The big reversal. `accessible-routes.md` had ruled out hand-curation since 29 August +because a hand-curated file has no provenance, no refresh and no audit. Those are three +properties, and properties can be built: an append-only observation log per station, one fact a +line with who, when, from what and how sure; replayed the way `rebuild` replays the collector's +logs; a page-sourced fact expiring when its quote leaves the page. Then a graph, reachability +per platform, and an outage as the named equipment's edges removed. Two safe-side rules keep it +from outrunning its evidence, and the pilot demonstrably says *less* than the prose derivation, +which is the design working. The three documents Barry brought, and what the business case gave +that no search would have: 51 stations in priority order, route-like paragraphs, a named audit +to request under FOI, and the definitions. Ends on the turn: a `gtfs` export of the very file +ch 06 found absent, and a format drafted to hand to Irish Rail. **Concepts.** The objection is a +specification; an edge that records ignorance. **Example.** The five-station pilot table. +**Contrast.** Neither sibling ever had to manufacture its second source. + +## Ch 16 - A fourth site, and two bugs found sideways · PR #54, issues #52, #53 · 11 Sep + +**Thesis.** A fourth site reads the same logs and publishes the delay notices. What belongs +here: not the collector (two pollers on a rotated credential, a second Pi install, two logs that +disagree), not a package (this repo's CI would gate two sites, and a repo publishes one Pages +site). The trap that is about this collector: Irish Rail empties `locationCodes` part-way through +a notice's life, 288 of 497 non-lift notices and **zero** lift ones, which is why nothing here +noticed and why the identity model is not changing. Then two bugs found by looking at something +else: "planned maintenance" is not "planned works" (one disagreement in 416 notices, found by +running the delays site's cause reader beside `is_planned`), and Kishoge keyed by name because +its page's code field is unparseable. **Concept.** A second reader of the same data is a test +you did not write. **Contrast.** The boundary question is new; the siblings have no fourth +reader. + +## Ch 17 - The same thing, said differently · PRs #55, #56 · 18 to 24 Sep + +**Thesis.** Both of ch 16's bugs close, and neither the way its issue proposed. "planned +maintenance" and "engineering works" join "planned works" because, by ch 04's own account of +what the grace is for, all three claim the same kind of event, so ch 16 made #53 sound more open +than it was. Kishoge's page publishes the name *in* the code field (the issue's "missing or +unparseable, falls back" was wrong, and ch 16 repeated it); the fix is a one-entry hand table, +not validation or refusal, and nothing guards the next one. Then a finding from re-measuring: +Tullamore's planned works came back on 16 Sep, the pooled total crossed a week on the 19th, and +August's A became a D nineteen days after August ended, with nothing on the page saying so. +**Concepts.** A literal string is a vocabulary of one; a property of the notice, published as a +property of the month. **Example.** Salthill and Monkstown's September, E 66% to C 91%, and the +reissue whose head said "Station". **Contrast.** None needed; the month question is this site's +own. + +## Ch 18 - The one code a rebuild cannot undo · PRs #57, #58 · 20 to 24 Sep + +**Thesis.** The collector was the oldest, least-changed code and the only code whose mistakes +a rebuild cannot undo, so it got the repository's one whole-file review. Ten findings: the raw +line was written inside the block that opened the database; an empty probe file passes on a full +card (ch 08's shape, missed by ch 08's own audit); a power-cut fragment swallowed the next line; +gzip errors escaped the retry; the alert marker outlived a recovery; systemd's 60 s cap was +below the client's worst case; the backup could hang. And ch 01's "one keyword argument": +`sort -u` deduplicates by sorting, which reorders by `body`, so replay now sorts by fetch time, +at the stated cost of `fake-hwclock`'s hour. #57's comment rule as a coda. **Concepts.** The +invariant has an upstream edge; a merge that deduplicates also reorders. **Example.** 2,224 lines +already in order, so nothing moved. **Contrast.** Stated only as a question in 19a: whether the +siblings' merges need the same. + +## Ch 19a - Closing: what the site can and cannot say + +The figures with their date, the two lists, the ten-row three-way table and the identical +column, the settled decisions in plain language. Split from 19b because the closing outgrew the +series' own 3,000-word ceiling. + +## Ch 19b - Closing: what I would tell someone starting the fourth one + +The moral, which is not either sibling's: *collect first, and publish no meaning you cannot +source*, with the September coda on rejected alternatives and the newer one from ch 15: write +your rejections out properly, because one of them is a design document you have not recognised +yet. A coda from ch 17 and 18 on where not to look. Glossary of all 31 concept boxes.