survey(source): trustworthy sources not yet covered — preprint servers, physics indexes, regional platforms

[sotashimozono]

A stocktake of what the 16-source matrix does not reach, recorded so the candidates are in one place. Not a work item yet — see Sequencing at the bottom.

None of the API details below are verified. They are from general knowledge, not from a primary source. #460 is what happens when an inferred contract ships; every candidate needs its endpoint, auth shape, response shape, rate limits and terms checked against the provider's own documentation before any code is written.

Gap 1 — preprint servers outside physics/maths/CS

arXiv covers physics, maths and CS. Nothing covers biology or medicine: Europe PMC indexes published literature and is not a preprint server.

candidate	why	notes
bioRxiv / medRxiv	the single largest gap by volume	believed to have a public DOI-lookup API, no key
OSF Preprints	best coverage-per-integration in the list — one API fronting PsyArXiv, SocArXiv, EngrXiv and a dozen more	DataCite resolves OSF DOIs today; OSF's own content API is a separate thing
ChemRxiv	chemistry	Cambridge Open Engage
Zenodo	author self-deposit, CERN-run, DOIs are first class	already a candidate in #461; DataCite resolves the DOI but Zenodo's API carries the file URL

Gap 2 — physics/astronomy indexes

Directly relevant to this repo's own corpus (the #407 PEVD papers, and the maintainer's condensed-matter work).

candidate	why
INSPIRE-HEP	believed free and key-less; bibliographic quality above arXiv's, and it links fulltext
NASA ADS	free token; the canonical bibliographic index for astronomy and much of physics

Gap 3 — regional platforms nobody else will cover

candidate	why
J-STAGE	the platform of record for Japanese domestic journals. No other source in the matrix reaches it, and no upstream aggregator covers it well
IRDB (ex-JAIRO)	Japanese institutional repositories
SciELO	Latin America / Iberia, fully OA, large. Already reachable on the content leg via trust_oa_registries, but there is no SciELO source, so it contributes nothing to discovery
BASE (Bielefeld)	largest repository index; named in #461. May require registration / IP allowlisting, which would put it outside the Tier-2 "no key" shape

Gap 4 — subject bibliographies (metadata only, no fulltext)

DBLP (CS, authoritative), RePEc (economics), PhilPapers (philosophy), Internet Archive Scholar / fatcat (long-tail OA at scale).

Sequencing — why this is filed rather than started

Three things make adding sources actively risky right now:

• bug(tdm): the Tier-3 chain is skipped whenever Crossref answers, so a TDM source can never close the fetch gap it was added for #458 — a Tier-3 source is not consulted for any DOI Crossref resolves, which is most of them.
• Tier 3 cannot fetch a PDF at all. All four TDM sources return pdf_bytes: None; the ADR-0019 safeguards were never wired through the orchestrator.
• test: an end-to-end use-case suite driven through the real entry points, because every "unreachable source" bug passed its unit tests #462 — nothing drives the real entry points on a schedule, which is how Source expansion: opt-in OA sources beyond Tier 1 — scope, exclusions, and the tiering decision #413, bug(orchestrator): the three Tier-3 TDM sources are never consulted by the production fetch path #442, bug(transport): the Tier-3 allowlists are never registered in the production HTTP client, so every TDM fetch fails UnknownSource #454 and bug(tdm): the Tier-3 chain is skipped whenever Crossref answers, so a TDM source can never close the fetch gap it was added for #458 each shipped implemented-and-unreachable.

Adding six sources to a chain with a known skip bug and no end-to-end verification multiplies the #442 defect class rather than improving coverage. #463 makes exactly this argument for ACM/SIAM/AMS and it holds here too.

Suggested order: land #462 first, so each new source arrives with a one-line proof that the production path reaches it. Then source additions get safer as they accumulate, instead of widening the unverified surface.

Refs #461, #462, #463, #458.

[sotashimozono]

Competitive context that changes this survey's priority, not its content or its sequencing argument.

Compared against openags/paper-search-mcp (2.5k stars, 249 forks, Python/MIT, actively maintained) on 2026-08-25: it reaches 25+ platforms, doiget reaches ~13. The overlap with this table is almost total — bioRxiv, medRxiv, chemRxiv, Zenodo, dblp, BASE, CiteSeerX, HAL, DOAJ, OpenAIRE, Europe PMC, CORE are all either already integrated here or already enumerated above. So this survey did the work; almost nothing needed adding to it.

Three things it does not cover, recorded here rather than opened as separate issues:

• PMID as an identifier — now feat(ref): PMID is a parked variant, so PubMed-identified entries are silently skipped — and life sciences is the market where that matters #500. It is an identifier gap, not a source gap, which is why it fell outside this table.
• SSRN, IACR ePrint, CiteSeerX — present in the comparison, absent here. Low value per integration; if anyone wants them they belong as rows in the Gap 4 table above, not as their own issues.
• PubMed proper — see feat(ref): PMID is a parked variant, so PubMed-identified entries are silently skipped — and life sciences is the market where that matters #500; Europe PMC already covers the corpus, what is missing is the identifier route.

Why the priority moved

The Sequencing argument above is still right: land #462 first, so each new source arrives with a proof the production path reaches it. Nothing below contradicts that.

What changed is the value of the far side. The differentiator against that competitor is not coverage and never will be — it is that it ships a functional Sci-Hub downloader (academic_platforms/sci_hub.py, 177 lines) wired into download_with_fallback with use_scihub: bool = True on by default, with a spoofed browser User-Agent and verify=False. That makes it uninstallable inside any organisation with a legal or security review, and it cannot be removed without costing them the feature that drives their adoption.

So doiget's position is "the one an institution can actually approve". The catch is that the institutions where that argument lands hardest are in the life sciences, and that is precisely where the coverage above is thinnest — which is what promotes Gap 1 (bioRxiv/medRxiv/OSF/ChemRxiv) and #500 from "nice to have eventually" to "the thing standing between a real advantage and the market that would pay for it".

Gap 2 (INSPIRE-HEP, NASA ADS) and Gap 3 (J-STAGE, IRDB, SciELO) keep their independent justification and are unaffected by this.

[sotashimozono]

Status check while working through the open issues.

The sequencing argument still holds, with one correction

This issue asks for #462 to land first, on the grounds that adding sources to a chain with a known skip bug (#458) and no end-to-end harness multiplies the #442 defect class.

#458 is fixed (closed 2026-08-26, ADR-0044), and "Tier 3 cannot fetch a PDF at all" is also out of date — ADR-0044 wired the Tier-3 content leg and tdm-aps returns bytes. So one and a half of the three reasons are gone.

The third stands, and it is the load-bearing one. Nothing drives the real entry points on a schedule. #413, #442, #454 and #458 each shipped implemented-and-unreachable, and each was found by a human driving the real thing. That argument does not weaken as sources accumulate; it strengthens.

So: #462 first is still the right order, and this survey stays parked behind it.

One item was promoted out

#500 (PMID) argues correctly that it is an identifier gap rather than a source gap: Ref::Pmid plus a resolver, routing to Europe PMC which is already integrated. It does not widen the Tier-2/Tier-3 chain, so the sequencing argument here does not cover it. It is tracked separately.

Nothing in the table is verified

Restating the issue's own opening rule, because it is the thing most likely to be forgotten by whoever picks this up: none of the API details are from a primary source. Every candidate needs endpoint, auth shape, response shape, rate limits and terms checked against the provider's own documentation before any code is written. ADR-0046 makes that a repository rule now — a claim about a vendor is normative, and the URL where the vendor says it is a pointer.