Skip to content

75 citations were being counted as zero, and all 15 affected studies are South Asia - #13

Merged
Varnasr merged 1 commit into
mainfrom
claude/outdated-repos-maintenance-nbxnae
Sep 22, 2026
Merged

Varnasr merged 1 commit into
mainfrom
claude/outdated-repos-maintenance-nbxnae

Conversation

@Varnasr

@Varnasr Varnasr commented Sep 22, 2026

Copy link
Copy Markdown
Owner

The bug

countReferences() in scripts/build.mjs matched:

/## (?:References|Sources|Bibliography)\s*\n.../i

189 of the 204 studies head their citations ## References. The 15 written to the later, longer template head them ## Further Reading. The function returned 0 for all fifteen.

They are not missing citations. Each carries five, properly formatted:

  1. Imbert, Clement, and John Papp. "Labor Market Effects of Social Programs: Evidence from India's Employment Guarantee." American Economic Journal: Applied Economics 7, no. 2 (2015): 233-263.
  2. Dreze, Jean, and Reetika Khera. "Recent Social Security Initiatives in India." World Development 98 (2017): 555-572.
  3. Sukhtankar, Sandip. "India's National Rural Employment Guarantee Scheme: What Do We Really Know?" India Policy Forum 2016-17.
  4. Narayanan, Sudha, and Upasak Das. "Women Participation and Rationing in the Employment Guarantee Scheme." Economic & Political Weekly 49, no. 46 (2014).

The library reported 867 references where it holds 942.

The affected set is not random

Six India studies (MGNREGA, UPI, DBT/JAM, Kerala model, PLI, Swachh Bharat), two Bangladesh (BRAC, garments), two Nepal (community forestry, remittances), Pakistan (BISP), Sri Lanka (2022 crisis), Bhutan (GNH), the Maldives (climate), and a regional stunting study.

That is essentially the whole South Asia set, reading as uncited on a site whose README calls it an evidence library.

The warning pointed the wrong way

npm run validate printed, for each of them:

⚠ india-mgnrega.json: no parseable References section

That reads as a content gap. It was a heading the regex did not know. Acting on the warning as written would have meant adding citations to studies that already have them. This is why I checked one of the files before touching the parser.

Verified across all 204: no study carries both headings, so widening the regex counts nothing twice.

Result

before after
total references 867 942
studies with 0 refs 15 0
⚠ warnings on --check 15 0

All 15 move from refs: 0 to refs: 5. master-list.json, stats.json and sitemap.xml regenerated.

One finding recorded, not patched

Fixing the count exposes something about the evidence field that is worth stating plainly.

evidenceTier() returns strong when refs >= 4. The reference distribution across all 204 studies is:

  4 refs: 78 studies
  5 refs: 126 studies

Nothing has fewer than four. So after this fix the function returns strong for all 204, and the moderate and emerging branches are unreachable. The hasRCT and hasMeta tests are short-circuited by the refs >= 4 branch before they matter.

Before the fix, 8 studies read emerging. All 8 were among the 15 whose references were not being counted. The tier was reporting the parser bug, not the evidence.

I have not changed it. What counts as strong evidence is an editorial judgement about this library, not a bug to patch on someone's behalf. The honest options:

  1. Raise the threshold above the range the corpus actually occupies.
  2. Weight hasRCT / hasMeta rather than letting refs >= 4 short-circuit them.
  3. Drop the field, and stop implying a distinction the data does not carry.

CLAUDE.md states the position and the options so it is not rediscovered as a surprise.

Also in this PR

The link check moves out of ci.yml into its own daily link-check.yml. Two reasons: it carried fail: false on pull requests, so a contributor got a result they could neither act on nor be blocked by; and it shared a workflow run with the data validation, where a job cancelled by its own timeout-minutes cancels the whole run — a slow third-party host could have taken the validation down with it. On the schedule it fails, because external link rot needs no commit to happen.

Adds CLAUDE.md, which this repository did not have. It records the two-template split, the sitemap's query-string URLs (a path-based comparison reports all 204 as missing pages), and that the absent lockfile is correct: package.json declares no dependencies at all.

Checks

$ npm run validate
Validating case-study data…

✓ 204 studies processed cleanly (check mode).

No warnings, where there were 15.

🤖 Generated with Claude Code

https://claude.ai/code/session_01MrvR2NXsJFVRCeJZFCPuNL


Generated by Claude Code

…ck out

countReferences() matched "References", "Sources" and "Bibliography". 189 of
the 204 studies use the first of those. The 15 written to the later, longer
template head their citations "Further Reading", and that set is almost exactly
South Asia: six India studies, two Bangladesh, two Nepal, Pakistan, Sri Lanka,
Bhutan, the Maldives and a regional stunting study.

The function returned 0 for all fifteen. They carry five full academic
citations each: Imbert and Papp in AEJ: Applied 2015, Dreze and Khera in World
Development 2017, Sukhtankar in the India Policy Forum, Narayanan and Das in
EPW. So the library reported 867 references where it holds 942.

The --check run printed "no parseable References section" for each of them,
which reads as a content gap and was in fact a heading the regex did not know.
Acting on the warning as written would have meant adding citations to studies
that already have them. No study carries both headings, verified across all
204, so widening the regex counts nothing twice.

One consequence worth recording rather than patching. evidenceTier() returns
"strong" when refs >= 4, and every one of the 204 studies has four or five
references, so after this fix the function returns "strong" for all of them and
the moderate and emerging branches are unreachable. Before the fix eight
studies read "emerging", and all eight were among the fifteen whose references
were not being counted: the tier was reporting the parser bug rather than the
evidence. What counts as strong evidence is an editorial judgement about the
library, not a bug to fix here, so CLAUDE.md states the position and the
options and leaves the threshold alone.

Separately, the lychee link check moves out of ci.yml into its own daily
workflow. It carried fail: false on pull requests, so a contributor got a
result they could neither act on nor be blocked by, and it shared a workflow
run with the data validation, where a job cancelled by its own timeout cancels
the whole run and a slow third-party host could have taken the validation with
it. On the schedule it fails, because external link rot needs no commit.

Adds CLAUDE.md, which this repository did not have.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MrvR2NXsJFVRCeJZFCPuNL
@Varnasr
Varnasr merged commit 1c29381 into main Sep 22, 2026
1 check passed
@Varnasr
Varnasr deleted the claude/outdated-repos-maintenance-nbxnae branch September 22, 2026 23:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants