Skip to content

feat(source): the first entitled-network publisher attempt — Cambridge Core book chapters, measured end to end on a subscribing network #593

Description

@sotashimozono

doiget 0.8.9 (MCP binary) and 0.8.12 (CLI), measured 2026-09-02 from a subscribing university network.

#517 decided "change it" — doiget retrieves through routes the user is lawfully entitled to use, including the institutional entitlement of the network it is sitting on — and recorded a three-step order: (1) give the closed-DOI path a trace, (2) record the network in provenance, (3) then the publisher attempt. (1) is #505. (2) and (3) have no issue. This is the issue for (3), with a first target that was measured rather than guessed.

The measurement

Target: Chaikin & Lubensky, Principles of Condensed Matter Physics (CUP, 1995). Book DOI 10.1017/CBO9780511813467, chapter DOIs 10.1017/CBO9780511813467.00N.

step result
fetch 10.1017/CBO9780511813467 oa_status: closed, pdf.status: no_oa_url, type: monograph
fetch 10.1017/CBO9780511813467.006 oa_status: closed, pdf.status: no_oa_url, type: book-chapter, page: 213-287
attempts[] on both five not_consulted_not_needed rows, no publisher row — the #505 silence
dry_run pdf_sources one entry, key oa-publisher, 24 hosts, no cambridge.org
landing page, plain curl + UA, university IP HTTP 200; title="You have access to this product" appears 30 times
chapter PDF from the landing-page href HTTP 200, application/pdf, 4,102,500 bytes

The href pattern on the landing page is "/core/services/aop-cambridge-core/content/view/<HASH>/<isbn><chap>_p<first>-<last>_CBO.pdf/<slug>.pdf", one per chapter plus frontmatter / contents / preface / glossary / index. Fifteen of them cover pages i-699 contiguously. Text layer present (pdftotext extracts body prose).

Why the Crossref link[] URL is not the route

Crossref carries exactly one link for the chapter:

"link": [{"URL": ".../content/view/7FCF854BE4B7BD006C95BB0E2BBAA209",
          "content-type": "unspecified",
          "intended-application": "similarity-checking"}]

That URL returns the same 4,102,500 bytes. It must still not be used: #517 recorded that the intended-application filter stays, and orchestrator.rs::extract_crossref_publisher_url implements it (ADR-0052). The landing-page href is the same bytes reached the way an entitled reader reaches them, and it is the route with a defensible provenance story. Worth stating in the eventual ADR, because the two URLs being byte-identical is exactly the coincidence that invites the shortcut.

Note also that 0.8.9 still surfaces that Similarity Check URL as oa_url from doiget_resolve_paper; 0.8.12 returns null. Separate issue.

Why CUP book chapters are the right first target

Sketch

  • Capability gate parallel to Tier 3, but with no API key, because the credential is the network: an explicit opt-in plus a recorded note naming the entitlement. CapabilityProfile (lib.rs:1362) grows one field; the AgreedButNoKey / KeyButNotAgreed pair from CapabilityError does not transfer unchanged and needs its own shape.
  • can_serve: prefix 10.1017, type == "book-chapter", grant present.
  • A separate allowlist key. Not a widening of oa-publisher — a closed host in the OA list is how a subscription PDF ends up stored as if it were free.
  • The store currently means "PDF present = free to use". A subscription PDF stored with license = "unknown" is indistinguishable from an OA one to bib / csl / text. It needs access = "subscription" and the terms URL (https://www.cambridge.org/core/terms) on the entry, and the entitlement recorded per step (2).
  • Entitlement check must be positive, not by absence of error: require Content-Type: application/pdf, since an unentitled IP gets 200 + HTML.

The terms guard belongs in the code

CUP's terms permit reading under the subscription and prohibit systematic download of an entire book. That distinction is currently the user's to remember; it should be the source's to enforce:

  • refuse book-level DOIs (type: monograph) — chapters only;
  • cap per container-title within a session, and say why on refusal.

Filing this as part of the design rather than as a later hardening, because a source that will happily walk fifteen chapter DOIs is the one that makes the tool a liability.

Order

Depends on #505 for (1) and on the provenance schema migration for (2) — per #517, so that the before/after is diffable in the log rather than asserted.

Refs #517, #505, #497, #407, ADR-0019, ADR-0039, ADR-0052.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions