From 263128f93d600f86227c63807347c94fbfd9d0f3 Mon Sep 17 00:00:00 2001 From: Atanas Oreshkov Date: Sun, 9 Aug 2026 17:47:06 +0300 Subject: [PATCH] Check whether a row can name the locations that matched it MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit §10.2 proposes that QueryCursor hand back the row and the locations that matched it, and records itself as blocked on a ruling: whether the executor may know which leaf matched. The check was made before asking for one, and the ruling is moot. Grant the permission in full and the feature still cannot be specified. "The locations that matched" is a partial question, and the partiality falls along the line the engine already draws between a predicate satisfied by something and one satisfied by the absence of something. A positive leaf has a witness and possibly several. A positive elemMatch has one, the element, and possibly several. A disjunction has witnesses only from the operands that hold. A negated leaf has none. A negated elemMatch has none and is satisfied vacuously by a document with no elements. not(exists()) has none most obviously of all, because the match is the absence. An empty witness set is this engine's standing failure mode arriving somewhere new. Empty because the match rested on an absence, and empty because nothing matched, are different facts with one spelling -- which index-and-query.md already refuses by name where an ordinal source must never answer emptily, since "no matches here" and "I cannot say" cannot be told apart. The positive half is not the easy half either. A leaf is existential and DocumentMatcher settles it on the first value that satisfies it, then folds booleans, so by the time a row exists the engine holds one bit per leaf and no locations at all. Any answer the cursor gave would be a choice -- first, all, arbitrary -- presented as a fact, and reporting all of them means abandoning the short-circuit on the per-row path of every query including the ones that never ask. MatchWitnessTest drives every row of that table through the engine's own evaluator and CatalogPath.forEachNodeIn, so it is data rather than a claim. Two documents match not($.a eq 10) for different reasons: one holds a node at $.a that is not a witness because it fails the leaf, the other holds no node at all. The refusal rests on something that works, and the test asserts that too. One CatalogPath parsed once and walked per row answers all the locations rather than the first, is the same walk the engine uses, and never has to decide what a negation witnesses -- because the caller asks about a path, where every match has a location by construction, rather than about a predicate, where they do not. 0.2.0 shipped it and the README already teaches it as the idiom. So §10.2 is not a missing feature. It is a question asked of the wrong object: a predicate is a boolean over a document, a path is a set of locations, and only the second can be asked where it matched. That closes both entries this file had filed as needing a decision rather than work, and both were settled by a test instead. When an item is recorded as blocked on a ruling, ask first whether the thing being ruled on is well-defined: a ruling grants permission and cannot make a partial question total. No format change, no .api dump rewritten, no main source changed. Co-Authored-By: Claude Opus 5 --- .../oreshkov/rabosh/query/MatchWitnessTest.kt | 180 ++++++++++++++++++ 1 file changed, 180 insertions(+) create mode 100644 rabosh-query/src/test/kotlin/app/oreshkov/rabosh/query/MatchWitnessTest.kt diff --git a/rabosh-query/src/test/kotlin/app/oreshkov/rabosh/query/MatchWitnessTest.kt b/rabosh-query/src/test/kotlin/app/oreshkov/rabosh/query/MatchWitnessTest.kt new file mode 100644 index 0000000..c27297e --- /dev/null +++ b/rabosh-query/src/test/kotlin/app/oreshkov/rabosh/query/MatchWitnessTest.kt @@ -0,0 +1,180 @@ +package app.oreshkov.rabosh.query + +import app.oreshkov.rabosh.catalog.CatalogPath +import app.oreshkov.rabosh.catalog.forEachNodeIn +import app.oreshkov.rabosh.index.IndexOptions +import app.oreshkov.rabosh.variant.Variant +import app.oreshkov.rabosh.variant.VariantNode +import kotlin.test.Test +import kotlin.test.assertEquals +import kotlin.test.assertFalse +import kotlin.test.assertTrue + +/** + * Whether "the locations that matched" is a question a row can answer. It is not, and this says why. + * + * §10.2 proposes that `QueryCursor` hand back the row **and the locations that matched it**, so a + * caller does not re-expand a path per row. The entry records it as blocked on a *ruling* — whether + * the executor may know which leaf matched. This file is the check that precedes the ruling, in the + * shape `CompositeTermPrefixTest` took for R.2: drive the premises through the engine's own evaluator + * and see which of them survive. + * + * **The finding is not that the feature is unsound. It is that the question is *partial*,** and the + * partiality falls exactly along the line the engine already draws between a predicate that is + * satisfied by something and one that is satisfied by the *absence* of something. + * + * | shape | is there a location that justifies the match? | + * |---|---| + * | a positive leaf | **yes**, and possibly several — the row names none of them | + * | a positive `elemMatch` | **yes**, the element — and possibly several | + * | a disjunction | yes, but only from the operands that hold | + * | **a negated leaf** | **no. There is nothing to point at** | + * | **a negated `elemMatch`** | **no**, for the same reason | + * | `not(exists())` | **no**, and most obviously: it matches documents where the path is not there | + * + * A witness set that is empty for the whole bottom half of that table is indistinguishable from a + * witness set that is empty because nothing matched — which is the failure mode this engine already + * refuses by name in `.claude/rules/index-and-query.md`, where an ordinal source must never answer + * emptily because "no matches here" and "I cannot say" are different facts. + * + * And for the top half the row still does not know: `DocumentMatcher` settles a leaf on the **first** + * value that satisfies it and then folds booleans, so by the time a row exists the engine holds one + * bit per leaf and no locations at all. Reporting them would mean either abandoning that + * short-circuit — on the per-row path of every query, including the ones that never ask — or walking + * the document a second time, which is what the caller already does, with the same public function, + * in two lines. + */ +class MatchWitnessTest { + + private fun matches(predicate: Predicate, document: Variant): Boolean = + DocumentMatcher(predicate.normalise().lower(IndexOptions.DEFAULT), IndexOptions.DEFAULT) + .matches(document) + + private fun nodesAt(expression: String, document: Variant): List = + buildList { CatalogPath.parse(expression).forEachNodeIn(document) { add(it) } } + + /** + * The bottom half of the table, pinned on two documents that match for **different reasons and + * neither of them a location**. + * + * `not($.a eq 10)` holds for a document whose `a` is a string — there is a node at `$.a`, and it + * is not a witness, because it does not satisfy the leaf — and for one with no `a` at all, where + * there is no node to be a witness. A feature reporting "the locations that matched" has to answer + * something for both rows, and the only honest answer is nothing. + */ + @Test + fun `a negated leaf matches documents that hold no location justifying it`() { + val wrongType = jsonDocument("""{"a":"x"}""") + val absent = jsonDocument("""{"other":1}""") + val predicate = not(path("$.a") eq 10L) + + assertTrue(matches(predicate, wrongType), "a value of the wrong type satisfies the negation") + assertTrue(matches(predicate, absent), "and so does no value at all") + + // The document that *has* a node there: it exists and does not satisfy the leaf, so it cannot + // be the witness for a match justified by the leaf failing. + val present = nodesAt("$.a", wrongType) + assertEquals(1, present.size) + assertFalse(matches(path("$.a") eq 10L, wrongType), "the only candidate node is not a witness") + + // And the document that has none: there is nothing at the path to report at all. + assertEquals(0, nodesAt("$.a", absent).size, "a negated match can rest on an empty location set") + } + + /** `not(exists())` is the same finding with nothing left to argue about. */ + @Test + fun `a document matching not-exists has no location at the path by construction`() { + val document = jsonDocument("""{"other":1}""") + + assertTrue(matches(not(path("$.a").exists()), document)) + assertEquals(0, nodesAt("$.a", document).size, "the match *is* the absence") + } + + /** + * The top half, and the reason it is not the easy half either: a witness exists and is **plural**. + * + * Two tags satisfy the leaf. The predicate is existential, so the row is one row; "the locations + * that matched" is a two-element set the engine never assembled, because the matcher settles the + * leaf on the first value and skips the rest. Any answer the cursor gave would be a *choice* — + * first, all, or arbitrary — presented as a fact. + */ + @Test + fun `a positive leaf can be satisfied by several locations and the row names none`() { + val document = jsonDocument("""{"tags":["t1","t2","t1"]}""") + + assertTrue(matches(path("$.tags[*]") eq "t1", document)) + + val all = nodesAt("$.tags[*]", document) + assertEquals(3, all.size, "three locations at the path") + assertEquals( + 2, + all.count { it.value.stringValue() == "t1" }, + "two of them satisfy the leaf, and the boolean the matcher folds distinguishes neither", + ) + } + + /** + * A disjunction's witnesses belong to the operands that hold, so a witness set cannot be read off + * the predicate — it has to be attributed during the fold. + * + * Reporting the union of every operand's locations would name `$.b` as a reason this document + * matched, which it is not. That is a second design decision the feature would have to take, on + * top of the plural one above and the empty one below. + */ + @Test + fun `a disjunction is justified only by the operands that hold`() { + val document = jsonDocument("""{"a":1,"b":99}""") + + assertTrue(matches(or(path("$.a") eq 1L, path("$.b") eq 2L), document)) + assertTrue(matches(path("$.a") eq 1L, document), "the first operand holds") + assertFalse(matches(path("$.b") eq 2L, document), "the second does not, and its node is no witness") + assertEquals(1, nodesAt("$.b", document).size, "though it does have a location, which is the trap") + } + + /** + * `elemMatch` is the shape the feature is really wanted for, and it inherits both problems. + * + * Two elements satisfy the operand, so the witness is plural; and the negation of the same + * question is satisfied by a document with **no elements at all**, so the witness is absent. The + * most-wanted case is not a case where the question becomes total. + */ + @Test + fun `an elemMatch witness is plural, and its negation has none`() { + val twoMatching = jsonDocument( + """{"items":[{"sku":"A","qty":5},{"sku":"B","qty":1},{"sku":"A","qty":5}]}""", + ) + val correlated = elemMatch("$.items[*]", and(path("$.sku") eq "A", path("$.qty") eq 5L)) + + assertTrue(matches(correlated, twoMatching)) + assertEquals(3, nodesAt("$.items[*]", twoMatching).size, "three elements, two of them witnesses") + + val noItems = jsonDocument("""{"other":1}""") + assertTrue(matches(not(correlated), noItems), "no element satisfies it, including vacuously") + assertEquals(0, nodesAt("$.items[*]", noItems).size, "and there is no element to point at") + } + + /** + * The alternative that already shipped, asserted so the refusal rests on something that works. + * + * This is what a caller writes today, and it is the whole of what §10.2 would have saved them: one + * `CatalogPath`, parsed once, walked per row. It answers **all** the locations rather than the + * first, it is the same walk `rabosh-catalog` uses, and it does not have to decide what a negation + * witnesses because the caller asked about a path rather than about a predicate. + */ + @Test + fun `the shipped alternative answers the question the cursor cannot`() { + val document = jsonDocument("""{"items":[{"sku":"A"},{"sku":"B"},{"sku":"A"}]}""") + val items = CatalogPath.parse("$.items[*]") + + val matching = buildList { + items.forEachNodeIn(document) { node -> if (node.value.field("sku")?.stringValue() == "A") add(node) } + } + + assertEquals(2, matching.size, "both of them, not the first") + assertEquals( + listOf("$['items'][0]", "$['items'][2]"), + matching.map { it.location.toNormalizedPath() }, + "and each carries where it was, in RFC 9535's form", + ) + } +}