Extraction is single-page today (documented honestly in the FAQ). Many catalogs paginate.
Sketch: an optional pagination block on ExtractionSpec (next-link selector or URL template + max_pages), fetched through the same provenance-tagged fetch layer, rows concatenated before Sentinel assessment. Golden records then need per-page or set-membership semantics — assess against the union.
Open question: should the Surgeon heal the next-link selector too? (It's just another field with kind url.)
Extraction is single-page today (documented honestly in the FAQ). Many catalogs paginate.
Sketch: an optional
paginationblock onExtractionSpec(next-link selector or URL template + max_pages), fetched through the same provenance-tagged fetch layer, rows concatenated before Sentinel assessment. Golden records then need per-page or set-membership semantics — assess against the union.Open question: should the Surgeon heal the next-link selector too? (It's just another field with kind
url.)