diff --git a/changes/n698-list-ranking.md b/changes/n698-list-ranking.md new file mode 100644 index 0000000..8cd7022 --- /dev/null +++ b/changes/n698-list-ranking.md @@ -0,0 +1,31 @@ +--- +type: added +facade: changed +--- +[en] +`list` can now be ranked per type, narrowed like `next`, and paged. A plugin may declare +`[ranking.list.]` tables (and `types = [...]` for the order of the types, plus an optional +default `order`) under `[ranking.list]` — the same keys, the same load-time checks and the same +comparator as `[ranking.next]`. `nxf list --sort rank` ranks by them; a type without a table, and +every plugin without `[ranking.list]`, ranks as before by the `next` default order. The default +`list` order stays id. `nxf list` also takes `--type` more than once (the types are alternatives), +`--in ` (the container's children and contributors), `--limit` (with a `showing of ` +notice, and the `{"items", "total"}` envelope under `--json`) and `--paginate` / `--token`, which +page through the same machine-local cache `next` uses. Without `--limit`, `list --json` is the same +plain array as before. An embedding app gets the same read from the engine: +`Engine::list_query(&ListQuery)` and `Engine::list_query_value`; `Ranking::list` carries the +resolved list policy. Everything is additive; no existing signature changes. +[de] +`list` lässt sich jetzt pro Typ ranken, wie `next` eingrenzen und blättern. Ein Plugin kann unter +`[ranking.list]` Tabellen `[ranking.list.]` deklarieren (dazu `types = [...]` für die +Reihenfolge der Typen und optional eine eigene Standard-`order`) — dieselben Schlüssel, dieselben +Prüfungen beim Laden und derselbe Vergleich wie bei `[ranking.next]`. `nxf list --sort rank` rankt +danach; ein Typ ohne Tabelle und jedes Plugin ohne `[ranking.list]` rankt wie bisher nach der +Standardreihenfolge von `next`. Die Standardreihenfolge von `list` bleibt die Id. `nxf list` nimmt +außerdem `--type` mehrfach (die Typen sind Alternativen), `--in ` (die Kinder und Beitragenden +des Containers), `--limit` (mit dem Hinweis `showing of ` und unter `--json` der Hülle +`{"items", "total"}`) und `--paginate` / `--token`, die durch denselben maschinenlokalen Cache +blättern wie `next`. Ohne `--limit` ist `list --json` dasselbe schlichte Array wie bisher. Eine +einbettende App bekommt denselben Lesezugriff von der Engine: `Engine::list_query(&ListQuery)` und +`Engine::list_query_value`; `Ranking::list` trägt die aufgelöste Listen-Policy. Alles ist additiv; +keine bestehende Signatur ändert sich. diff --git a/changes/q29p-mention-reads.md b/changes/q29p-mention-reads.md new file mode 100644 index 0000000..cbbc037 --- /dev/null +++ b/changes/q29p-mention-reads.md @@ -0,0 +1,25 @@ +--- +type: added +facade: changed +--- +[en] +Mentions can now be read in both directions. `nxf mention by ` lists the items that mention an +item, in id order, as `list` records (closed and archived mentioners included). `--type`, `--in` and +`--label` narrow the list as they narrow `next`. `--limit` cuts it and says so, and `--paginate` +pages through it with a token, exactly as `next` does. Under `--json` you get a plain array, or +with `--limit` the `{"items", "total"}` envelope, plus `next_token` and `restarted` with +`--paginate`. An embedding app gets the same reads from the engine: `Engine::mentions(id)` returns +an item's outgoing targets, and `Engine::mentioned_by(id, &MentionedByQuery)` / +`Engine::mentioned_by_value` return the items that mention it. `read::mentioners_of` is the bulk +reverse lookup behind them. Everything is additive; no existing signature changes. +[de] +Erwähnungen lassen sich jetzt in beide Richtungen lesen. `nxf mention by ` listet die Items, die +ein Item erwähnen, in Id-Reihenfolge und als `list`-Datensätze (geschlossene und archivierte +eingeschlossen). `--type`, `--in` und `--label` grenzen die Liste ein wie bei `next`. `--limit` +kürzt sie und sagt das an, und `--paginate` blättert mit einem Token durch sie, genau wie bei +`next`. Mit `--json` kommt ein schlichtes Array, mit `--limit` die Hülle `{"items", "total"}`, und +mit `--paginate` zusätzlich `next_token` und `restarted`. Eine einbettende App bekommt dieselben +Lesezugriffe von der Engine: `Engine::mentions(id)` liefert die ausgehenden Ziele eines Items, +`Engine::mentioned_by(id, &MentionedByQuery)` und `Engine::mentioned_by_value` die Items, die es +erwähnen. `read::mentioners_of` ist die Massen-Rückwärtssuche dahinter. Alles ist additiv; keine +bestehende Signatur ändert sich. diff --git a/changes/x0wf-derived-ranking-keys.md b/changes/x0wf-derived-ranking-keys.md new file mode 100644 index 0000000..aa818f1 --- /dev/null +++ b/changes/x0wf-derived-ranking-keys.md @@ -0,0 +1,66 @@ +--- +type: added +facade: breaking +--- +[en] +A plugin can now rank items by their neighbours in the graph, not only by their own fields. +Besides field keys, a per-type order (`[ranking.next.]`, `[ranking.list.]`, and the +`[ranking.list]` default order) takes three derived keys: + +- `related` reads a value on the neighbours over one outbound edge: their rank in their own type's + order (`of = "rank"`) or a field, a custom field included (`of = { field = "..." }`), picking the + smallest or largest. +- `count` counts the neighbours over one edge kind and direction (`children`, `contributors`, + `dependents`, `mentioned_by`, or an outbound kind) that pass its `type`, `status` and `label` + filters. +- `sum` adds several counts with equal weight. + +So actions can rank by their project's rank, projects by their goals' declared order and their +open children, and persons by how often projects and open waiting actions mention them. Ranks are +computed bottom-up through the chain of types, once per query, with one pass over the edges. +`count = "mentioned_by"` reads the same reverse mention lookup as `nxf mention by`. A plugin is +refused at load when a key mixes kinds, names an unknown edge, type, status or field, carries a +part its kind does not take, reads the rank of a type without an order, or when the types rank by +each other in a cycle. An untyped `related ... of = "rank"` key is refused in the `[ranking.list]` +default order as well, and a table under `[ranking]` other than `next` and `list` is refused rather +than ignored. A `parent` neighbour is the item's one current parent, the one `show` and `--in` +read. Derived keys are not allowed in the `[ranking.next]` default order, which +also ranks `blocked` and the lanes. A plugin without derived keys ranks exactly as before. For an +embedding app: `RankKey` gains the derived parts and is now `#[non_exhaustive]` and refuses +unknown parts (build one with `RankKey::field_key`); `ListQuery::now` and `read::list_at` take +the reference time a `list` order needs when it reads the rank of a type `next` ranks; and the +store-free `read::next_active` refuses a plugin whose `[ranking.next]` has derived keys until the +server path computes them. +[de] +Ein Plugin kann Items jetzt nach ihren Nachbarn im Graphen ranken, nicht nur nach ihren eigenen +Feldern. Neben Feldschlüsseln nimmt eine Reihenfolge pro Typ (`[ranking.next.]`, +`[ranking.list.]` und die Standardreihenfolge von `[ranking.list]`) drei abgeleitete +Schlüssel: + +- `related` liest einen Wert an den Nachbarn über eine ausgehende Kante: ihren Rang in der + Reihenfolge ihres eigenen Typs (`of = "rank"`) oder ein Feld, auch ein eigenes Feld + (`of = { field = "..." }`), und nimmt den kleinsten oder größten. +- `count` zählt die Nachbarn über eine Kantenart und Richtung (`children`, `contributors`, + `dependents`, `mentioned_by` oder eine ausgehende Art), die seine Filter `type`, `status` und + `label` bestehen. +- `sum` addiert mehrere Zählungen mit gleichem Gewicht. + +So können Aktionen nach dem Rang ihres Projekts ranken, Projekte nach der deklarierten +Reihenfolge ihrer Ziele und ihren offenen Kindern, und Personen danach, wie oft Projekte und offene +wartende Aktionen sie erwähnen. Ränge werden von unten durch die Kette der Typen berechnet, einmal +pro Abfrage, mit einem Durchgang über die Kanten. `count = "mentioned_by"` liest dieselbe +Rückwärtssuche der Erwähnungen wie `nxf mention by`. Ein Plugin wird beim Laden abgelehnt, wenn +ein Schlüssel Arten mischt, eine unbekannte Kante, einen unbekannten Typ, Status oder ein +unbekanntes Feld nennt, einen Teil trägt, den seine Art nicht nimmt, den Rang eines Typs ohne +Reihenfolge liest, oder wenn die Typen sich im Kreis gegenseitig ranken. Ein Schlüssel +`related ... of = "rank"` ohne `type` wird auch in der Standardreihenfolge von `[ranking.list]` +abgelehnt, und eine Tabelle unter `[ranking]` außer `next` und `list` wird abgelehnt statt +ignoriert. Ein `parent`-Nachbar ist der eine aktuelle Parent des Items, den `show` und `--in` +lesen. Abgeleitete Schlüssel sind +in der Standardreihenfolge von `[ranking.next]` nicht erlaubt, die auch `blocked` und die Lanes +rankt. Ein Plugin ohne abgeleitete Schlüssel rankt genau wie bisher. Für eine einbettende App: +`RankKey` bekommt die abgeleiteten Teile, ist jetzt `#[non_exhaustive]` und lehnt unbekannte Teile +ab (bauen mit `RankKey::field_key`); `ListQuery::now` und `read::list_at` nehmen die Bezugszeit, +die eine `list`-Reihenfolge braucht, wenn sie den Rang eines Typs liest, den `next` rankt; und das +store-freie `read::next_active` lehnt ein Plugin ab, dessen `[ranking.next]` abgeleitete Schlüssel +hat, bis der Server-Pfad sie berechnet. diff --git a/crates/cli/docs/guide/de/commands.md b/crates/cli/docs/guide/de/commands.md index 04d2340..6fc3cce 100644 --- a/crates/cli/docs/guide/de/commands.md +++ b/crates/cli/docs/guide/de/commands.md @@ -54,6 +54,22 @@ $ nxf list --json [{"archived":null,"assignee":null,"belongs_to":null,"closed_at":null,"closing_comment":null,"completion_criterion":null,"created_at":"2026-06-23T00:00:00Z","defer_until":null,"deleted":null,"description":"Cut the first release","design":null,"due":null,"id":"ab12.0001","priority":"1","priority_label":"P1","status":"in_progress","title":"Ship v1","type":"epic","type_label":"epic","updated_at":"2026-06-23T00:00:00Z"},{"archived":null,"assignee":"dev","belongs_to":"ab12.0001","closed_at":null,"closing_comment":null,"completion_criterion":null,"created_at":"2026-06-23T00:00:00Z","defer_until":null,"deleted":null,"description":"Build the command-line tool","design":"Thin CLI over the core.","due":"2026-12-31","id":"ab12.0002","priority":"1","priority_label":"P1","status":"in_progress","title":"Write the CLI","type":"feature","type_label":"feature","updated_at":"2026-06-23T00:00:00Z"},{"archived":null,"assignee":null,"belongs_to":null,"closed_at":"2026-06-23T00:00:00Z","closing_comment":"approved","completion_criterion":null,"created_at":"2026-06-23T00:00:00Z","defer_until":null,"deleted":null,"description":"Approve the final spec","design":null,"due":null,"id":"ab12.0003","priority":"2","priority_label":"P2","status":"closed","title":"Spec sign-off","type":"feature","type_label":"feature","updated_at":"2026-06-23T00:00:00Z"}] ``` +`list --sort rank` rankt stattdessen: nach der `list`-Reihenfolge des Plugins — einer Reihenfolge +pro Typ, wo das Plugin eine deklariert (`[ranking.list.]`, mit `types = [...]` dafür, wie die +Typen zueinander stehen), sonst nach seiner `next`-Reihenfolge. Eine Reihenfolge kann auch nach den +Nachbarn eines Items ranken — dem Rang seines Elternteils, der Zahl seiner offenen Kinder, wie oft +es erwähnt wird — mit den abgeleiteten Schlüsseln, die unter [Plugins](nxf-plugins) beschrieben +sind. `--type` (mehrfach), `--in ` +(die Kinder und Beitragenden des Containers), `--status` und `--label` grenzen ein. `--limit` und +`--paginate` kürzen und blättern genau wie bei `next`: mit `--limit` ist die Antwort `{"items": +[...], "total": }`, und `--paginate` ergänzt `next_token` (mit `--token` zurückgeben; null auf +der letzten Seite) und `restarted`: + +```console +$ nxf list --sort rank --limit 2 --paginate --json +{"items":[{"archived":null,"assignee":null,"belongs_to":null,"closed_at":null,"closing_comment":null,"completion_criterion":null,"created_at":"2026-06-23T00:00:00Z","defer_until":null,"deleted":null,"description":"Cut the first release","design":null,"due":null,"id":"ab12.0001","priority":"1","priority_label":"P1","status":"in_progress","title":"Ship v1","type":"epic","type_label":"epic","updated_at":"2026-06-23T00:00:00Z"},{"archived":null,"assignee":"dev","belongs_to":"ab12.0001","closed_at":null,"closing_comment":null,"completion_criterion":null,"created_at":"2026-06-23T00:00:00Z","defer_until":null,"deleted":null,"description":"Build the command-line tool","design":"Thin CLI over the core.","due":"2026-12-31","id":"ab12.0002","priority":"1","priority_label":"P1","status":"in_progress","title":"Write the CLI","type":"feature","type_label":"feature","updated_at":"2026-06-23T00:00:00Z"}],"next_token":"[..].2","restarted":false,"total":3} +``` + `search` matcht über Titel, Beschreibung und Notizen: ```console @@ -118,7 +134,7 @@ $ nxf dep remove ab12.0002 ab12.0003 --json ``` `mention add` / `mention list` / `mention remove` halten Freitext-Referenzen per Short-id fest — -ein Verweis, der nie blockiert: +ein Verweis, der nie blockiert. `mention by` liest sie in der Gegenrichtung: ```console $ nxf mention add ab12.0002 ab12.0001 --json @@ -132,6 +148,24 @@ $ nxf mention list ab12.0002 --json ``` +`mention by ` listet die Items, die `` erwähnen, in Id-Reihenfolge als `list --json`-Datensätze +(geschlossene und archivierte eingeschlossen). `--type`, `--in` und `--label` grenzen ein wie bei +`next`. Ohne `--limit` ist die Antwort ein schlichtes Array; mit `--limit` ist sie `{"items": [...], +"total": }`, und `--paginate` ergänzt `next_token` (null auf der letzten Seite) und `restarted` +(true, wenn ein veraltetes Token wieder die erste Seite lieferte) — genau wie `next --limit --json`: + +```console +$ nxf mention by ab12.0001 --json +[{"archived":null,"assignee":"dev","belongs_to":"ab12.0001","closed_at":null,"closing_comment":null,"completion_criterion":null,"created_at":"2026-06-23T00:00:00Z","defer_until":null,"deleted":null,"description":"Build the command-line tool","design":"Thin CLI over the core.","due":"2026-12-31","id":"ab12.0002","priority":"1","priority_label":"P1","status":"in_progress","title":"Write the CLI","type":"feature","type_label":"feature","updated_at":"2026-06-23T00:00:00Z"}] + +``` + +```console +$ nxf mention by ab12.0001 --type feature --limit 1 --paginate --json +{"items":[{"archived":null,"assignee":"dev","belongs_to":"ab12.0001","closed_at":null,"closing_comment":null,"completion_criterion":null,"created_at":"2026-06-23T00:00:00Z","defer_until":null,"deleted":null,"description":"Build the command-line tool","design":"Thin CLI over the core.","due":"2026-12-31","id":"ab12.0002","priority":"1","priority_label":"P1","status":"in_progress","title":"Write the CLI","type":"feature","type_label":"feature","updated_at":"2026-06-23T00:00:00Z"}],"next_token":null,"restarted":false,"total":1} + +``` + ```console $ nxf mention remove ab12.0002 ab12.0001 --json {"msg":"removed mention ab12.0002 -> ab12.0001","ok":true} diff --git a/crates/cli/docs/guide/de/plugins.md b/crates/cli/docs/guide/de/plugins.md index 8d49d9f..82e3c17 100644 --- a/crates/cli/docs/guide/de/plugins.md +++ b/crates/cli/docs/guide/de/plugins.md @@ -150,3 +150,78 @@ Diese Verdopplung ist keine Redundanz — sie *ist* die Erklärung. Das `--json` das Plugin ist, wie ein bestimmtes Publikum sie liest und rankt. Ein Plugin bei `init` zu wählen ist daher eine echte Entscheidung; führen Sie `nxf init --help` aus, um die Optionen zu sehen, bevor Sie sich festlegen. + +## Ein Ranking deklarieren + +Ein Plugin deklariert unter `[ranking.next]`, wie `next` rankt, und unter `[ranking.list]`, wie +`list --sort rank` rankt. Beide nehmen dieselben Schlüssel und dieselben Regeln: + +- `order = [...]` ist die Standardreihenfolge: eine Liste von Schlüsseln, der Reihe nach + verglichen, mit der Id als letztem Entscheider. +- `[ranking.next.]` (oder `[ranking.list.]`) gibt einem Typ eine eigene `order`. +- `types = [...]` sagt, wie die Typen zueinander stehen, sobald einige eine eigene Reihenfolge + haben. + +Ein Schlüssel ist ein **Feldschlüssel**, etwa `{ field = "priority", dir = "asc" }`. Mit einer +`precedence`-Liste rankt ein Feldschlüssel stattdessen nach der Position des Werts in dieser +Liste. In einer Reihenfolge pro Typ, oder in der Standard-`order` von `[ranking.list]`, kann ein +Schlüssel auch ein **abgeleiteter Schlüssel** sein, der ein Item nach seinen Nachbarn im Graphen +rankt: + +```toml +[ranking.next] +types = ["project", "action"] +order = [ { field = "priority" } ] + +# related: ein Wert, gelesen an den Nachbarn über eine ausgehende Kante (parent, contributes_to, +# dep, mentions). `of = "rank"` liest die Position des Nachbarn in der Reihenfolge seines eigenen +# Typs; `of = { field = "..." }` liest ein Feld, auch ein eigenes Feld. `pick` ist min (Standard) +# oder max. +[ranking.next.action] +order = [ + { related = "parent", type = "project", of = "rank", nulls = "last" }, + { field = "priority" }, +] + +# count: wie viele Nachbarn passen. children, contributors, dependents und mentioned_by zählen die +# eingehenden Kanten parent, contributes_to, dep und mentions; eine ausgehende Art zählt in die +# andere Richtung. `type`, `status` (eine Liste) und `label` filtern die gezählten Nachbarn. +[ranking.next.project] +order = [ + { related = "contributes_to", type = "goal", of = { field = "order" }, nulls = "last" }, + { count = "children", type = "action", status = ["open", "in_progress"], dir = "desc" }, +] + +# sum: mehrere Zählungen, mit gleichem Gewicht addiert. `dir` gehört zur Summe. +[ranking.list] +types = ["person"] +[ranking.list.person] +order = [ + { sum = [ { count = "mentioned_by", type = "project" }, + { count = "mentioned_by", type = "action", label = "waiting" } ], dir = "desc" }, +] +``` + +Abgeleitete Schlüssel zählen nur lebende Nachbarn: ein gelöschter fällt weg, ein geschlossener +oder archivierter zählt. Ein Rang wird über die Kandidaten des Typs unter dem Verb berechnet, das +ihn rankt, und Items, die in jedem Schlüssel gleich sind, teilen sich einen Rang. Für `next` sind +diese Kandidaten seine umsetzbaren Items, für `list` die lebenden, nicht archivierten Items. Ein +Nachbar außerhalb dieser Menge, etwa ein geschlossenes Projekt unter `next`, hat keinen Rang und +liest sich als null. Ein `list`-Schlüssel darf den Rang eines Typs lesen, den nur `next` rankt; +die Kandidaten von `next` werden dann zur aktuellen Zeit genommen (`NXF_NOW` legt sie fest). Nur +`list --sort rank` liest diese Zeit. + +Das Plugin wird beim Laden abgelehnt, mit dem Schlüssel benannt, wenn: + +- ein Schlüssel Arten mischt oder keine nennt; +- ein Schlüssel eine unbekannte Kantenart, einen unbekannten Typ, Status oder ein unbekanntes Feld + nennt; +- ein Schlüssel einen Teil trägt, den seine Art nicht nimmt, etwa `pick` an einer Zählung oder + `nulls` an einer Zählung oder Summe (eine Zählung ist nie null); +- `of = "rank"` einen Typ ohne eigene Reihenfolge nennt; +- die Typen sich im Kreis gegenseitig ranken. + +Ein Schlüssel `related ... of = "rank"` ohne `type` liest den Rang jedes Typs mit einer +Reihenfolge, auch des eigenen. Das ist ein Kreis, also nennen Sie den `type` des Nachbarn; die +Standard-`order` von `[ranking.list]` lehnt einen solchen Schlüssel ebenfalls ab. Abgeleitete Schlüssel sind in der Standard-`order` von `[ranking.next]` nicht erlaubt, weil diese +Reihenfolge auch `blocked` und `--sort rank` der Lanes rankt. diff --git a/crates/cli/docs/guide/en/commands.md b/crates/cli/docs/guide/en/commands.md index 5cce6db..78806f0 100644 --- a/crates/cli/docs/guide/en/commands.md +++ b/crates/cli/docs/guide/en/commands.md @@ -52,6 +52,21 @@ $ nxf list --json [{"archived":null,"assignee":null,"belongs_to":null,"closed_at":null,"closing_comment":null,"completion_criterion":null,"created_at":"2026-06-23T00:00:00Z","defer_until":null,"deleted":null,"description":"Cut the first release","design":null,"due":null,"id":"ab12.0001","priority":"1","priority_label":"P1","status":"in_progress","title":"Ship v1","type":"epic","type_label":"epic","updated_at":"2026-06-23T00:00:00Z"},{"archived":null,"assignee":"dev","belongs_to":"ab12.0001","closed_at":null,"closing_comment":null,"completion_criterion":null,"created_at":"2026-06-23T00:00:00Z","defer_until":null,"deleted":null,"description":"Build the command-line tool","design":"Thin CLI over the core.","due":"2026-12-31","id":"ab12.0002","priority":"1","priority_label":"P1","status":"in_progress","title":"Write the CLI","type":"feature","type_label":"feature","updated_at":"2026-06-23T00:00:00Z"},{"archived":null,"assignee":null,"belongs_to":null,"closed_at":"2026-06-23T00:00:00Z","closing_comment":"approved","completion_criterion":null,"created_at":"2026-06-23T00:00:00Z","defer_until":null,"deleted":null,"description":"Approve the final spec","design":null,"due":null,"id":"ab12.0003","priority":"2","priority_label":"P2","status":"closed","title":"Spec sign-off","type":"feature","type_label":"feature","updated_at":"2026-06-23T00:00:00Z"}] ``` +`list --sort rank` ranks instead: by the plugin's `list` order — an order per type where the plugin +declares one (`[ranking.list.]`, with `types = [...]` for how the types stand against each +other), else its `next` order. An order may also rank by an item's neighbours — its parent's +rank, how many open children it has, how often it is mentioned — with the derived keys described +under [Plugins](nxf-plugins). `--type` (repeatable), `--in ` (the container's children and +contributors), `--status` and `--label` narrow it. `--limit` and `--paginate` cut and page it +exactly as they do `next`: with `--limit` the answer is `{"items": [...], "total": }`, and +`--paginate` adds `next_token` (pass it back with `--token`; null on the last page) and +`restarted`: + +```console +$ nxf list --sort rank --limit 2 --paginate --json +{"items":[{"archived":null,"assignee":null,"belongs_to":null,"closed_at":null,"closing_comment":null,"completion_criterion":null,"created_at":"2026-06-23T00:00:00Z","defer_until":null,"deleted":null,"description":"Cut the first release","design":null,"due":null,"id":"ab12.0001","priority":"1","priority_label":"P1","status":"in_progress","title":"Ship v1","type":"epic","type_label":"epic","updated_at":"2026-06-23T00:00:00Z"},{"archived":null,"assignee":"dev","belongs_to":"ab12.0001","closed_at":null,"closing_comment":null,"completion_criterion":null,"created_at":"2026-06-23T00:00:00Z","defer_until":null,"deleted":null,"description":"Build the command-line tool","design":"Thin CLI over the core.","due":"2026-12-31","id":"ab12.0002","priority":"1","priority_label":"P1","status":"in_progress","title":"Write the CLI","type":"feature","type_label":"feature","updated_at":"2026-06-23T00:00:00Z"}],"next_token":"[..].2","restarted":false,"total":3} +``` + `search` matches across title, description, and notes: ```console @@ -155,7 +170,7 @@ $ nxf dep remove ab12.0002 ab12.0003 --json ``` `mention add` / `mention list` / `mention remove` record free-text short-id references — a -citation that never blocks: +citation that never blocks. `mention by` reads them the other way round: ```console $ nxf mention add ab12.0002 ab12.0001 --json @@ -169,6 +184,24 @@ $ nxf mention list ab12.0002 --json ``` +`mention by ` lists the items that mention ``, in id order, as `list --json` records +(closed and archived mentioners included). `--type`, `--in` and `--label` narrow it as they narrow +`next`. Without `--limit` the answer is a plain array; with `--limit` it is `{"items": [...], +"total": }`, and `--paginate` adds `next_token` (null on the last page) and `restarted` (true +when a stale token answered with the first page again), exactly as `next --limit --json`: + +```console +$ nxf mention by ab12.0001 --json +[{"archived":null,"assignee":"dev","belongs_to":"ab12.0001","closed_at":null,"closing_comment":null,"completion_criterion":null,"created_at":"2026-06-23T00:00:00Z","defer_until":null,"deleted":null,"description":"Build the command-line tool","design":"Thin CLI over the core.","due":"2026-12-31","id":"ab12.0002","priority":"1","priority_label":"P1","status":"in_progress","title":"Write the CLI","type":"feature","type_label":"feature","updated_at":"2026-06-23T00:00:00Z"}] + +``` + +```console +$ nxf mention by ab12.0001 --type feature --limit 1 --paginate --json +{"items":[{"archived":null,"assignee":"dev","belongs_to":"ab12.0001","closed_at":null,"closing_comment":null,"completion_criterion":null,"created_at":"2026-06-23T00:00:00Z","defer_until":null,"deleted":null,"description":"Build the command-line tool","design":"Thin CLI over the core.","due":"2026-12-31","id":"ab12.0002","priority":"1","priority_label":"P1","status":"in_progress","title":"Write the CLI","type":"feature","type_label":"feature","updated_at":"2026-06-23T00:00:00Z"}],"next_token":null,"restarted":false,"total":1} + +``` + ```console $ nxf mention remove ab12.0002 ab12.0001 --json {"msg":"removed mention ab12.0002 -> ab12.0001","ok":true} diff --git a/crates/cli/docs/guide/en/plugins.md b/crates/cli/docs/guide/en/plugins.md index 817cd4d..c9609b4 100644 --- a/crates/cli/docs/guide/en/plugins.md +++ b/crates/cli/docs/guide/en/plugins.md @@ -146,3 +146,74 @@ LOG That doubling is not redundancy — it *is* the explanation. The `--json` is the meaning; the plugin is how a particular audience reads and ranks it. Picking a plugin at `init` is therefore a real choice; run `nxf init --help` to see the options before you commit. + +## Declaring a ranking + +A plugin declares how `next` ranks under `[ranking.next]` and how `list --sort rank` ranks under +`[ranking.list]`. Both take the same keys and the same rules: + +- `order = [...]` is the default order: a list of keys, compared in turn, with the id as the final + tiebreak. +- `[ranking.next.]` (or `[ranking.list.]`) gives one type its own `order`. +- `types = [...]` says how the types stand against each other once some have their own order. + +A key is a **field key**, such as `{ field = "priority", dir = "asc" }`. With a `precedence` list, +a field key ranks by the value's position in that list instead. In a per-type order, or in the +`[ranking.list]` default `order`, a key can also be a **derived key**, which ranks an item by its +neighbours in the graph: + +```toml +[ranking.next] +types = ["project", "action"] +order = [ { field = "priority" } ] + +# related: a value read on the neighbours over one outbound edge (parent, contributes_to, dep, +# mentions). `of = "rank"` reads the neighbour's position in its own type's order; +# `of = { field = "..." }` reads a field, a custom field included. `pick` is min (default) or max. +[ranking.next.action] +order = [ + { related = "parent", type = "project", of = "rank", nulls = "last" }, + { field = "priority" }, +] + +# count: how many neighbours match. children, contributors, dependents and mentioned_by count the +# inbound parent, contributes_to, dep and mentions edges; an outbound kind counts the other way. +# `type`, `status` (a list) and `label` filter the counted neighbours. +[ranking.next.project] +order = [ + { related = "contributes_to", type = "goal", of = { field = "order" }, nulls = "last" }, + { count = "children", type = "action", status = ["open", "in_progress"], dir = "desc" }, +] + +# sum: several counts, added with equal weight. `dir` belongs to the sum. +[ranking.list] +types = ["person"] +[ranking.list.person] +order = [ + { sum = [ { count = "mentioned_by", type = "project" }, + { count = "mentioned_by", type = "action", label = "waiting" } ], dir = "desc" }, +] +``` + +Derived keys count only live neighbours: a deleted one is gone, a closed or archived one counts. +A rank is computed over the type's candidates under the verb that ranks it, and items that tie on +every key share a rank. For `next` these candidates are its actionable items; for `list` they are +the live, unarchived items. A neighbour outside that set, such as a closed project under `next`, +has no rank and reads as null. A `list` key may read the rank of a type that only `next` ranks; +`next`'s candidates are then taken at the current time (`NXF_NOW` pins it). Only `list --sort rank` +reads that time. + +The plugin is refused at load, with the key named, when: + +- a key mixes kinds or names none; +- a key names an unknown edge kind, type, status or field; +- a key carries a part its kind does not take, such as `pick` on a count or `nulls` on a count or + sum (a count is never null); +- `of = "rank"` names a type without an order of its own; +- the types rank by each other in a cycle. + +A `related ... of = "rank"` key without `type` reads the rank of every type with an order, its own +included. That is a cycle, so name the neighbour's `type`; the `[ranking.list]` default `order` +refuses such a key too. Derived keys are not allowed in the +`[ranking.next]` default `order`, because that order also ranks `blocked` and the lanes' +`--sort rank`. diff --git a/crates/cli/src/commands/mod.rs b/crates/cli/src/commands/mod.rs index 4fbfae2..08f2b3d 100644 --- a/crates/cli/src/commands/mod.rs +++ b/crates/cli/src/commands/mod.rs @@ -1731,32 +1731,84 @@ pub fn next(json: bool, db: Option<&str>, args: NextArgs<'_>) -> Result<()> { // ---- list ------------------------------------------------------------------ -pub fn list( - json: bool, - db: Option<&str>, - status: Option<&str>, - item_type: Option<&str>, - label: Option<&str>, - sort: Option<&str>, -) -> Result<()> { +/// `nxf list`'s arguments, as clap parsed them. +pub struct ListArgs<'a> { + pub status: Option<&'a str>, + pub types: &'a [String], + pub within: Option<&'a str>, + pub label: Option<&'a str>, + pub sort: Option<&'a str>, + pub limit: Option, + pub paginate: bool, + pub token: Option<&'a str>, +} + +/// `nxf list` (6j6v.n698: filters and paging) — ONE query through the function +/// `Engine::list_query` answers from (`read::list_page`), so the filters, the order, the cut and +/// the paging cannot drift apart between the two, nor from `next`'s. +pub fn list(json: bool, db: Option<&str>, args: ListArgs<'_>) -> Result<()> { let (ws, cfg) = open(db)?; let store = ws.open_store()?; - let sort = sort.map(read::parse_sort).transpose()?; - let mut items = read::list(&cfg, &store, status, item_type, sort)?; - if let Some(label) = label { - items = read::with_label(&store, items, label)?; + let mut filter = read::NextFilter::new(); + for ty in args.types { + filter = filter.with_type(ty.as_str()); + } + if let Some(c) = args.within { + filter = filter.with_container(ws.replica.resolve_id(c)); + } + if let Some(label) = args.label { + filter = filter.with_label(label); } + let mut query = read::ListQuery::new().filter(filter); + let sort = args.sort.map(read::parse_sort).transpose()?; + // The reference time a derived ranking key of `[ranking.list]` takes `next`'s candidates at + // when it reads the rank of a type `next` ranks (6j6v.x0wf); `NXF_NOW` pins it, as for `next`. + // Read only for a RANKED list under such a plugin, so any other `list` never trips on the + // clock — a malformed `NXF_NOW` included (review of PR #48, Code Quality #6). + if sort == Some(read::SortKey::Rank) && cfg.ranking.list.has_derived() { + query = query.now(resolve_now(None)?); + } + if let Some(status) = args.status { + query = query.status(status); + } + if let Some(sort) = sort { + query = query.sort(sort); + } + if let Some(n) = args.limit { + query = query.limit(n); + } + if args.paginate { + query = query.paginate(); + } + if let Some(token) = args.token { + query = query.token(token); + } + let page = read::list_page(&cfg, &store, &query)?; - // Emit in the chosen order (default rank) — preserve it into JSON, do NOT re-sort by id. Each + // Emit in the chosen order (default id) — preserve it into JSON, do NOT re-sort by id. Each // record carries a sparse `parent_closed_reason` join for a closed-masked open child (07a.3 §5). if json { // ee2h: additive priority_label/type_label alongside the canonical record (CLI-only). - println!( - "{}", - with_presentation_labels(&cfg, read::list_to_value_with_custom(&cfg, &store, &items)?) + let value = with_presentation_labels( + &cfg, + read::list_to_value_with_custom(&cfg, &store, &page.items)?, ); + // The shape follows the FLAG, as for `next`: the bare array without `--limit` (byte-stable + // as before), the envelope with it. + if args.limit.is_some() { + println!("{}", read::next_page_envelope(value, &page, args.paginate)); + } else { + println!("{value}"); + } } else { - print!("{}", render_list(&cfg, &ws.replica.prefix, &items)); + let theme = nxs_ui::Theme::plain(); + if let Some(notice) = next_truncation_notice(&page, &theme) { + print!("{notice}"); + } + print!("{}", render_list(&cfg, &ws.replica.prefix, &page.items)); + if let Some(line) = next_token_line(&page, &theme) { + print!("{line}"); + } } Ok(()) } @@ -1999,6 +2051,69 @@ pub fn mention_list(json: bool, db: Option<&str>, id: &str) -> Result<()> { Ok(()) } +/// `nxf mention by`'s arguments, as clap parsed them. +pub struct MentionByArgs<'a> { + pub types: &'a [String], + pub within: Option<&'a str>, + pub label: Option<&'a str>, + pub limit: Option, + pub paginate: bool, + pub token: Option<&'a str>, +} + +/// List the items that mention `id` (6j6v.q29p) — through the function `Engine::mentioned_by` +/// answers from (`read::mentioned_by_page`), so the filters and the paging cannot drift apart +/// between the two, nor from `next`'s. +pub fn mention_by(json: bool, db: Option<&str>, id: &str, args: MentionByArgs<'_>) -> Result<()> { + let (ws, cfg) = open(db)?; + let store = ws.open_store()?; + let id = ws.replica.resolve_id(id); + let mut filter = read::NextFilter::new(); + for ty in args.types { + filter = filter.with_type(ty.as_str()); + } + if let Some(c) = args.within { + filter = filter.with_container(ws.replica.resolve_id(c)); + } + if let Some(label) = args.label { + filter = filter.with_label(label); + } + let mut query = read::MentionedByQuery::new().filter(filter); + if let Some(n) = args.limit { + query = query.limit(n); + } + if args.paginate { + query = query.paginate(); + } + if let Some(token) = args.token { + query = query.token(token); + } + let page = read::mentioned_by_page(&store, &id, &query)?; + if json { + let value = with_presentation_labels( + &cfg, + read::list_to_value_with_custom(&cfg, &store, &page.items)?, + ); + // The shape follows the FLAG, as for `next`: a plain array without `--limit`, the envelope + // with it. + if args.limit.is_some() { + println!("{}", read::next_page_envelope(value, &page, args.paginate)); + } else { + println!("{value}"); + } + } else { + let theme = nxs_ui::Theme::plain(); + if let Some(notice) = next_truncation_notice(&page, &theme) { + print!("{notice}"); + } + print!("{}", render_list(&cfg, &ws.replica.prefix, &page.items)); + if let Some(line) = next_token_line(&page, &theme) { + print!("{line}"); + } + } + Ok(()) +} + // ---- thread links (nxf 6j6v.8dbe) ------------------------------------------ /// Parse a `--relation` value, or a `validation` error listing the accepted tokens — the same loud diff --git a/crates/cli/src/lib.rs b/crates/cli/src/lib.rs index c5c0f5e..8bf2029 100644 --- a/crates/cli/src/lib.rs +++ b/crates/cli/src/lib.rs @@ -227,21 +227,44 @@ enum Command { #[arg(long = "reason-file")] reason_file: Option, }, - /// List items, optionally filtered by status and/or type. Id-ordered by default (so the - /// `--json` record stays plugin-independent); override with `--sort rank`. + /// List items, optionally filtered by status, type, container and/or label. Id-ordered by + /// default (so the `--json` record stays plugin-independent); `--sort rank` ranks by the active + /// plugin's `list` order. `--limit` shows only the head of it, and says so; `--paginate` adds a + /// token for the next page. List { /// Filter by status. #[arg(long)] status: Option, - /// Filter by type, using the active plugin's vocabulary (see `nxf schema`). - #[arg(long = "type")] - ty: Option, + /// Filter by type, using the active plugin's vocabulary (see `nxf schema`). Repeatable: the + /// types are alternatives. + #[arg(long = "type", value_name = "TYPE")] + ty: Vec, + /// Only items in this container: its children (whose parent it is) and the items that + /// contribute to it. + #[arg(long = "in", value_name = "ID")] + within: Option, /// Filter to items carrying this label (user vocabulary, not the display type). #[arg(long)] label: Option, - /// Sort order: `rank` | `id` (default `id`). An unknown key is a loud error. + /// Sort order: `rank` | `id` (default `id`). `rank` is the plugin's `list` ranking: an + /// order per type where it declares one (`[ranking.list.]`), else its `next` order. + /// An unknown key is a loud error. #[arg(long)] sort: Option, + /// Max items to show (default: no limit). Applied last — after `--sort` and the filters — + /// and never silently: a truncated list is headed `showing of `, and under + /// `--json` the flag wraps the records as `{"items": [...], "total": }`. + #[arg(long)] + limit: Option, + /// Page through the list `--limit` rows at a time: the answer ends with a token for the + /// next page (`next_token` and `restarted` under `--json`). The list is cached on this + /// machine for an hour; a change to an item in it, or a stale or invalid token, restarts + /// at the first page. + #[arg(long, requires = "limit")] + paginate: bool, + /// The token a previous `--paginate` answer printed: show the page after that one. + #[arg(long, value_name = "TOKEN", requires = "paginate")] + token: Option, }, /// List blocked items (open, with an open blocker or in a cycle). Ranked by default; override with `--sort id`. Blocked { @@ -329,7 +352,7 @@ enum Command { #[command(subcommand)] action: DepAction, }, - /// Add, remove, or list reference edges (free-text short-id citations). + /// Add, remove, or list reference edges (free-text short-id citations), in either direction. Mention { #[command(subcommand)] action: MentionAction, @@ -462,6 +485,41 @@ enum MentionAction { Remove { from: String, to: String }, /// List the short-ids `id` mentions. List { id: String }, + /// List the items that mention `id` — the reverse of `list` — in id order, closed and archived + /// mentioners included. `--type`, `--in` and `--label` narrow it as they narrow `next`; + /// `--limit` shows only the head and says so; `--paginate` adds a token for the next page. + /// Under `--json` the records are `list --json`'s, as a plain array — or, with `--limit`, as + /// `{"items": [...], "total": }`, plus `"next_token"` (a string, or null on the last page) + /// and `"restarted"` (bool) with `--paginate`, exactly as `next --limit --json` wraps them. + By { + /// The mentioned item. + id: String, + /// Only mentioners of this type (the stored type, as `list --type` takes it). Repeatable: + /// the types are alternatives. + #[arg(long = "type", value_name = "TYPE")] + ty: Vec, + /// Only mentioners in this container: its children (whose parent it is) and the items that + /// contribute to it. + #[arg(long = "in", value_name = "ID")] + within: Option, + /// Only mentioners carrying this label. + #[arg(long)] + label: Option, + /// Max items to show (default: no limit). Applied after the filters and never silently: + /// a truncated list is headed `showing of `, and under `--json` the flag wraps + /// the records as `{"items": [...], "total": }`. + #[arg(long)] + limit: Option, + /// Page through the list `--limit` rows at a time: the answer ends with a token for the + /// next page (`next_token` and `restarted` under `--json`). The list is cached on this + /// machine for an hour; a change to an item in it, or a stale or invalid token, restarts + /// at the first page. + #[arg(long, requires = "limit")] + paginate: bool, + /// The token a previous `--paginate` answer printed: show the page after that one. + #[arg(long, value_name = "TOKEN", requires = "paginate")] + token: Option, + }, } #[derive(Subcommand, Debug)] @@ -659,15 +717,25 @@ fn dispatch(cli: &Cli) -> Result<()> { Command::List { status, ty, + within, label, sort, + limit, + paginate, + token, } => commands::list( cli.json, db, - status.as_deref(), - ty.as_deref(), - label.as_deref(), - sort.as_deref(), + commands::ListArgs { + status: status.as_deref(), + types: ty, + within: within.as_deref(), + label: label.as_deref(), + sort: sort.as_deref(), + limit: *limit, + paginate: *paginate, + token: token.as_deref(), + }, ), Command::Blocked { sort } => commands::blocked(cli.json, db, sort.as_deref()), Command::Deferred { now, sort } => { @@ -707,6 +775,27 @@ fn dispatch(cli: &Cli) -> Result<()> { MentionAction::Add { from, to } => commands::mention_add(cli.json, db, from, to), MentionAction::Remove { from, to } => commands::mention_remove(cli.json, db, from, to), MentionAction::List { id } => commands::mention_list(cli.json, db, id), + MentionAction::By { + id, + ty, + within, + label, + limit, + paginate, + token, + } => commands::mention_by( + cli.json, + db, + id, + commands::MentionByArgs { + types: ty, + within: within.as_deref(), + label: label.as_deref(), + limit: *limit, + paginate: *paginate, + token: token.as_deref(), + }, + ), }, Command::Thread { action } => match action { ThreadAction::Link { diff --git a/crates/cli/tests/golden/issue-tracker/workflow.trycmd b/crates/cli/tests/golden/issue-tracker/workflow.trycmd index 3d8a9f3..3809b81 100644 --- a/crates/cli/tests/golden/issue-tracker/workflow.trycmd +++ b/crates/cli/tests/golden/issue-tracker/workflow.trycmd @@ -125,6 +125,20 @@ $ nxf mention list ab12.0002 --json ``` +The reverse read: the items that mention the epic — `list` records, narrowed and paged like `next`: + +```console +$ nxf mention by ab12.0001 --json +[{"archived":null,"assignee":"dev","belongs_to":"ab12.0001","closed_at":null,"closing_comment":null,"completion_criterion":null,"created_at":"2026-06-23T00:00:00Z","defer_until":null,"deleted":null,"description":"Build the command-line tool","design":"Thin CLI over the core.","due":"2026-12-31","id":"ab12.0002","priority":"1","priority_label":"P1","status":"in_progress","title":"Write the CLI","type":"feature","type_label":"feature","updated_at":"2026-06-23T00:00:00Z"}] + +``` + +```console +$ nxf mention by ab12.0001 --type feature --limit 1 --paginate --json +{"items":[{"archived":null,"assignee":"dev","belongs_to":"ab12.0001","closed_at":null,"closing_comment":null,"completion_criterion":null,"created_at":"2026-06-23T00:00:00Z","defer_until":null,"deleted":null,"description":"Build the command-line tool","design":"Thin CLI over the core.","due":"2026-12-31","id":"ab12.0002","priority":"1","priority_label":"P1","status":"in_progress","title":"Write the CLI","type":"feature","type_label":"feature","updated_at":"2026-06-23T00:00:00Z"}],"next_token":null,"restarted":false,"total":1} + +``` + ```console $ nxf mention remove ab12.0002 ab12.0001 --json {"msg":"removed mention ab12.0002 -> ab12.0001","ok":true} @@ -224,6 +238,12 @@ $ nxf list --json ``` +```console +$ nxf list --sort rank --limit 2 --paginate --json +{"items":[{"archived":null,"assignee":null,"belongs_to":null,"closed_at":null,"closing_comment":null,"completion_criterion":null,"created_at":"2026-06-23T00:00:00Z","defer_until":null,"deleted":null,"description":"Cut the first release","design":null,"due":null,"id":"ab12.0001","priority":"1","priority_label":"P1","status":"in_progress","title":"Ship v1","type":"epic","type_label":"epic","updated_at":"2026-06-23T00:00:00Z"},{"archived":null,"assignee":"dev","belongs_to":"ab12.0001","closed_at":null,"closing_comment":null,"completion_criterion":null,"created_at":"2026-06-23T00:00:00Z","defer_until":null,"deleted":null,"description":"Build the command-line tool","design":"Thin CLI over the core.","due":"2026-12-31","id":"ab12.0002","priority":"1","priority_label":"P1","status":"in_progress","title":"Write the CLI","type":"feature","type_label":"feature","updated_at":"2026-06-23T00:00:00Z"}],"next_token":"[..].2","restarted":false,"total":3} + +``` + Close it out, drop the now-irrelevant dependency, and read the session snapshot: ```console diff --git a/crates/cli/tests/golden/personal-todo/workflow.trycmd b/crates/cli/tests/golden/personal-todo/workflow.trycmd index 4a08c2c..cf83464 100644 --- a/crates/cli/tests/golden/personal-todo/workflow.trycmd +++ b/crates/cli/tests/golden/personal-todo/workflow.trycmd @@ -125,6 +125,20 @@ $ nxf mention list ab12.0002 --json ``` +The reverse read: the items that mention the project — `list` records, narrowed and paged like `next`: + +```console +$ nxf mention by ab12.0001 --json +[{"archived":null,"assignee":"me","belongs_to":"ab12.0001","closed_at":null,"closing_comment":null,"completion_criterion":null,"created_at":"2026-06-23T00:00:00Z","defer_until":null,"deleted":null,"description":"Reserve outbound and return","design":"Compare fares first.","due":"2026-12-31","id":"ab12.0002","priority":"1","priority_label":"soon","status":"in_progress","title":"Book flights","type":"todo","type_label":"todo","updated_at":"2026-06-23T00:00:00Z"}] + +``` + +```console +$ nxf mention by ab12.0001 --type todo --limit 1 --paginate --json +{"items":[{"archived":null,"assignee":"me","belongs_to":"ab12.0001","closed_at":null,"closing_comment":null,"completion_criterion":null,"created_at":"2026-06-23T00:00:00Z","defer_until":null,"deleted":null,"description":"Reserve outbound and return","design":"Compare fares first.","due":"2026-12-31","id":"ab12.0002","priority":"1","priority_label":"soon","status":"in_progress","title":"Book flights","type":"todo","type_label":"todo","updated_at":"2026-06-23T00:00:00Z"}],"next_token":null,"restarted":false,"total":1} + +``` + ```console $ nxf mention remove ab12.0002 ab12.0001 --json {"msg":"removed mention ab12.0002 -> ab12.0001","ok":true} @@ -223,6 +237,12 @@ $ nxf list --json ``` +```console +$ nxf list --sort rank --limit 2 --paginate --json +{"items":[{"archived":null,"assignee":null,"belongs_to":null,"closed_at":null,"closing_comment":null,"completion_criterion":null,"created_at":"2026-06-23T00:00:00Z","defer_until":null,"deleted":null,"description":"Get ready for the trip","design":null,"due":null,"id":"ab12.0001","priority":"1","priority_label":"soon","status":"in_progress","title":"Trip prep","type":"project","type_label":"project","updated_at":"2026-06-23T00:00:00Z"},{"archived":null,"assignee":"me","belongs_to":"ab12.0001","closed_at":null,"closing_comment":null,"completion_criterion":null,"created_at":"2026-06-23T00:00:00Z","defer_until":null,"deleted":null,"description":"Reserve outbound and return","design":"Compare fares first.","due":"2026-12-31","id":"ab12.0002","priority":"1","priority_label":"soon","status":"in_progress","title":"Book flights","type":"todo","type_label":"todo","updated_at":"2026-06-23T00:00:00Z"}],"next_token":"[..].2","restarted":false,"total":3} + +``` + Close it out, drop the dependency, and read the session snapshot: ```console diff --git a/crates/cli/tests/list.rs b/crates/cli/tests/list.rs index 6699a4d..e9e881c 100644 --- a/crates/cli/tests/list.rs +++ b/crates/cli/tests/list.rs @@ -147,3 +147,220 @@ fn deleted_items_are_excluded() { create(tmp.path(), "bug", "keep"); assert_eq!(list(tmp.path(), &[]).len(), 1); } + +// ---- filters and paging (6j6v.n698) --------------------------------------------------- + +fn run_json(dir: &Path, args: &[&str]) -> serde_json::Value { + let out = nxf() + .args(args) + .current_dir(dir) + .assert() + .success() + .get_output() + .stdout + .clone(); + serde_json::from_slice(&out).expect("json output") +} + +fn ids_of(v: &serde_json::Value) -> Vec { + v.as_array() + .expect("an array") + .iter() + .map(|i| i["id"].as_str().unwrap().to_string()) + .collect() +} + +/// An epic with a child bug and a contributing feature, plus an unrelated chore. Returns +/// `(epic, bug, feature, chore)`. +fn contained_board(dir: &Path) -> (String, String, String, String) { + init(dir); + let epic = create(dir, "epic", "E"); + let bug = create(dir, "bug", "B"); + let feature = create(dir, "feature", "F"); + let chore = create_with_prio(dir, "chore", "C", "P0"); + run_json( + dir, + &["update", &bug, "--set", &format!("parent={epic}"), "--json"], + ); + run_json(dir, &["contributes", "add", &feature, &epic, "--json"]); + (epic, bug, feature, chore) +} + +#[test] +fn list_filters_by_several_types_and_by_container_like_next() { + let tmp = TempDir::new().unwrap(); + let (epic, bug, feature, chore) = contained_board(tmp.path()); + let sorted = |mut v: Vec| { + v.sort(); + v + }; + assert_eq!( + list(tmp.path(), &["--in", &epic]), + sorted(vec![bug.clone(), feature.clone()]), + "a container holds its children and its contributors" + ); + assert_eq!( + list(tmp.path(), &["--type", "bug", "--type", "chore"]), + sorted(vec![bug.clone(), chore]), + "the types are alternatives" + ); + assert_eq!(list(tmp.path(), &["--in", &epic, "--type", "bug"]), [bug]); + // A short id resolves, as for `next --in`. + let short = epic.rsplit('.').next().unwrap().to_string(); + assert_eq!(list(tmp.path(), &["--in", &short]).len(), 2); +} + +#[test] +fn list_limit_and_paginate_answer_with_the_next_envelope() { + let tmp = TempDir::new().unwrap(); + let (_, bug, _, _) = contained_board(tmp.path()); + let all = list(tmp.path(), &[]); + + let cut = run_json(tmp.path(), &["list", "--limit", "3", "--json"]); + assert_eq!(ids_of(&cut["items"]), all[0..3]); + assert_eq!(cut["total"], 4); + assert!( + cut.get("next_token").is_none(), + "no token without --paginate" + ); + + let p1 = run_json( + tmp.path(), + &["list", "--limit", "3", "--paginate", "--json"], + ); + assert_eq!(ids_of(&p1["items"]), all[0..3]); + assert_eq!(p1["restarted"], false); + let token = p1["next_token"] + .as_str() + .expect("a row remains") + .to_string(); + let page2 = |dir: &Path| { + run_json( + dir, + &[ + "list", + "--limit", + "3", + "--paginate", + "--token", + &token, + "--json", + ], + ) + }; + let p2 = page2(tmp.path()); + assert_eq!(ids_of(&p2["items"]), all[3..]); + assert_eq!(p2["next_token"], serde_json::Value::Null, "the last page"); + + // A change to a cached item makes the token stale: the answer is the first page again. + run_json(tmp.path(), &["update", &bug, "--set", "title=B2", "--json"]); + let again = page2(tmp.path()); + assert_eq!(again["restarted"], true); + assert_eq!(ids_of(&again["items"]), all[0..3]); +} + +#[test] +fn list_sort_rank_pages_through_the_ranked_order() { + let tmp = TempDir::new().unwrap(); + contained_board(tmp.path()); + let ranked = list(tmp.path(), &["--sort", "rank"]); + // The P0 chore ranks first under issue-tracker; by id it would not necessarily. + assert_eq!(ranked.len(), 4); + let p1 = run_json( + tmp.path(), + &[ + "list", + "--sort", + "rank", + "--limit", + "2", + "--paginate", + "--json", + ], + ); + assert_eq!(ids_of(&p1["items"]), ranked[0..2]); + let token = p1["next_token"].as_str().unwrap().to_string(); + let p2 = run_json( + tmp.path(), + &[ + "list", + "--sort", + "rank", + "--limit", + "2", + "--paginate", + "--token", + &token, + "--json", + ], + ); + assert_eq!(ids_of(&p2["items"]), ranked[2..]); + // The same token under the id order is another query: it restarts. + let other = run_json( + tmp.path(), + &[ + "list", + "--limit", + "2", + "--paginate", + "--token", + &token, + "--json", + ], + ); + assert_eq!(other["restarted"], true); +} + +#[test] +fn list_human_view_discloses_the_cut_and_prints_the_token() { + let tmp = TempDir::new().unwrap(); + contained_board(tmp.path()); + let out = list_human(tmp.path(), &["--limit", "2", "--paginate"]); + // Trimmed: the CLI frames a human answer with blank lines (`output_framing.rs`). + let out = out.trim().to_string(); + assert!(out.starts_with("showing 2 of 4\n"), "{out}"); + assert_eq!(out.lines().count(), 4, "notice, two rows, token: {out}"); + assert!( + out.lines() + .last() + .unwrap() + .starts_with("next page: --paginate --token "), + "{out}" + ); + // Without --limit, the view is the plain list: no notice, no token. + let plain = list_human(tmp.path(), &[]); + assert!( + !plain.contains("showing") && !plain.contains("next page"), + "{plain}" + ); +} + +#[test] +fn list_paginate_needs_a_limit_and_a_token_needs_paginate() { + let tmp = TempDir::new().unwrap(); + contained_board(tmp.path()); + // The refusal names what is missing, not just any failure (review of PR #48, Test Quality #9). + let cases: [(&[&str], &[&str]); 3] = [ + (&["list", "--paginate"], &["required", "--limit"]), + ( + &["list", "--limit", "2", "--token", "x.2"], + &["required", "--paginate"], + ), + ( + &["list", "--limit", "0", "--paginate"], + &["paginate needs a limit of at least 1"], + ), + ]; + for (args, needles) in cases { + let out = nxf() + .args(args) + .current_dir(tmp.path()) + .assert() + .failure() + .get_output() + .stderr + .clone(); + let err = String::from_utf8(out).unwrap(); + assert!(needles.iter().all(|n| err.contains(n)), "{args:?}: {err}"); + } +} diff --git a/crates/cli/tests/mention.rs b/crates/cli/tests/mention.rs index 2a7d75d..de34e8c 100644 --- a/crates/cli/tests/mention.rs +++ b/crates/cli/tests/mention.rs @@ -7,7 +7,8 @@ use tempfile::TempDir; fn nxf(dir: &Path) -> Command { let mut c = nxs_test_support::cargo_bin("nxf"); - c.current_dir(dir); + // Sequential ids, so creation order is id order (the order `mention by` answers in). + c.current_dir(dir).env("NXF_DETERMINISTIC_IDS", "1"); c } @@ -76,3 +77,197 @@ fn mention_add_records_a_reference_that_does_not_block() { "mention does not block: {ready:?}" ); } + +// ---- mention by: the reverse read (6j6v.q29p) ------------------------------- + +fn create_typed(dir: &Path, ty: &str, title: &str, extra: &[&str]) -> String { + let mut args = vec![ + "create", + "--type", + ty, + "--title", + title, + "--description", + "d", + "--priority", + "P1", + "--json", + ]; + args.extend_from_slice(extra); + run_json(dir, &args)["id"].as_str().unwrap().to_string() +} + +fn ids(v: &serde_json::Value) -> Vec { + v.as_array() + .expect("an array of records") + .iter() + .map(|r| r["id"].as_str().unwrap().to_string()) + .collect() +} + +/// A target and four mentioners: a feature in the epic, two bugs (one contributing to the epic), +/// and a feature outside it. Returns `(target, epic, [mentioners in id order])`. +fn mentioned_board(dir: &Path) -> (String, String, Vec) { + nxf(dir) + .args(["init", "--plugin", "issue-tracker"]) + .assert() + .success(); + let target = create_typed(dir, "feature", "the cited one", &[]); + let epic = create_typed(dir, "epic", "the container", &[]); + let m = vec![ + create_typed(dir, "feature", "in the epic", &["--parent", &epic]), + create_typed(dir, "bug", "contributes", &[]), + create_typed(dir, "bug", "loose bug", &[]), + create_typed(dir, "feature", "loose feature", &[]), + ]; + run_json(dir, &["contributes", "add", &m[1], &epic, "--json"]); + for from in &m { + run_json(dir, &["mention", "add", from, &target, "--json"]); + } + (target, epic, m) +} + +#[test] +fn mention_by_lists_the_mentioners_as_list_records_and_filters_like_next() { + let tmp = TempDir::new().unwrap(); + let (target, epic, m) = mentioned_board(tmp.path()); + + let all = run_json(tmp.path(), &["mention", "by", &target, "--json"]); + assert_eq!( + ids(&all), + m, + "every mentioner, in id order, as a bare array" + ); + assert_eq!(all[0]["type"], "feature", "a list record"); + assert!( + all[0]["type_label"].is_string(), + "with list's presentation labels" + ); + + let bugs = run_json( + tmp.path(), + &["mention", "by", &target, "--type", "bug", "--json"], + ); + assert_eq!(ids(&bugs), m[1..3]); + let in_epic = run_json( + tmp.path(), + &["mention", "by", &target, "--in", &epic, "--json"], + ); + assert_eq!( + ids(&in_epic), + [m[0].clone(), m[1].clone()], + "child or contributor" + ); + let nothing = run_json(tmp.path(), &["mention", "by", &epic, "--json"]); + assert_eq!(nothing, serde_json::json!([])); +} + +#[test] +fn mention_by_pages_with_the_next_envelope() { + let tmp = TempDir::new().unwrap(); + let (target, _, m) = mentioned_board(tmp.path()); + + let cut = run_json( + tmp.path(), + &["mention", "by", &target, "--limit", "3", "--json"], + ); + assert_eq!(ids(&cut["items"]), m[0..3]); + assert_eq!(cut["total"], 4); + assert!( + cut.get("next_token").is_none(), + "no token without --paginate" + ); + + let p1 = run_json( + tmp.path(), + &[ + "mention", + "by", + &target, + "--limit", + "3", + "--paginate", + "--json", + ], + ); + assert_eq!(ids(&p1["items"]), m[0..3]); + assert_eq!(p1["restarted"], false); + let token = p1["next_token"] + .as_str() + .expect("a row remains") + .to_string(); + let p2 = run_json( + tmp.path(), + &[ + "mention", + "by", + &target, + "--limit", + "3", + "--paginate", + "--token", + &token, + "--json", + ], + ); + assert_eq!(ids(&p2["items"]), m[3..]); + assert_eq!(p2["next_token"], serde_json::Value::Null, "the last page"); + + // A stale token: a removed mention names a cached item, so the next answer restarts. + run_json(tmp.path(), &["mention", "remove", &m[3], &target, "--json"]); + let again = run_json( + tmp.path(), + &[ + "mention", + "by", + &target, + "--limit", + "3", + "--paginate", + "--token", + &token, + "--json", + ], + ); + assert_eq!(again["restarted"], true); + assert_eq!(ids(&again["items"]), m[0..3]); + assert_eq!(again["total"], 3); +} + +#[test] +fn mention_by_human_view_discloses_the_cut_and_prints_the_token() { + let tmp = TempDir::new().unwrap(); + let (target, _, _) = mentioned_board(tmp.path()); + let out = nxf(tmp.path()) + .args(["mention", "by", &target, "--limit", "2", "--paginate"]) + .assert() + .success() + .get_output() + .stdout + .clone(); + // Trimmed: the CLI frames a human answer with blank lines (`output_framing.rs`). + let out = String::from_utf8(out).unwrap().trim().to_string(); + assert!(out.starts_with("showing 2 of 4\n"), "{out}"); + assert_eq!(out.lines().count(), 4, "notice, two rows, token: {out}"); + assert!( + out.lines() + .last() + .unwrap() + .starts_with("next page: --paginate --token "), + "{out}" + ); +} + +#[test] +fn mention_by_a_missing_target_fails_and_paginate_needs_a_limit() { + let tmp = TempDir::new().unwrap(); + let (target, _, _) = mentioned_board(tmp.path()); + nxf(tmp.path()) + .args(["mention", "by", "ab12.zzzz", "--json"]) + .assert() + .failure(); + nxf(tmp.path()) + .args(["mention", "by", &target, "--paginate"]) + .assert() + .failure(); +} diff --git a/crates/cli/tests/verb_seam.rs b/crates/cli/tests/verb_seam.rs index 04674d9..450d363 100644 --- a/crates/cli/tests/verb_seam.rs +++ b/crates/cli/tests/verb_seam.rs @@ -73,6 +73,13 @@ const WAIVERS: &[Waiver] = &[ seam: "mentions", reason: "`read::mentions` is what the verb body calls; the CLI only renders it.", }, + Waiver::Alias { + verb: "mention by", + seam: "mentioned_by", + reason: "`Engine::mentioned_by` (over `read::mentioned_by_page`, which the verb body \ + calls) — the seam names the relation, the CLI the direction beside `mention \ + list` (6j6v.q29p).", + }, Waiver::Alias { verb: "note list", seam: "notes", diff --git a/crates/core/src/model.rs b/crates/core/src/model.rs index f96f0f9..e00fb57 100644 --- a/crates/core/src/model.rs +++ b/crates/core/src/model.rs @@ -48,7 +48,7 @@ impl Status { /// the OR-set replacement for the former single-parent `belongs_to` LWW register. The substrate is /// n:m-capable like every other edge; single-parent is enforced at the write seam during ②a and /// relaxed in ②c. The single current parent is projected by the `present_parent` view. -#[derive(Debug, Clone, Copy, PartialEq, Eq)] +#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)] pub enum EdgeKind { Dep, ContributesTo, diff --git a/crates/core/src/next_cache.rs b/crates/core/src/next_cache.rs index 2ffd2b9..a11dfa8 100644 --- a/crates/core/src/next_cache.rs +++ b/crates/core/src/next_cache.rs @@ -1,6 +1,8 @@ -//! The snapshot cache behind `next --paginate` (6j6v.15ed): the ordered result a first page was -//! cut from, kept so the following pages come from the SAME list instead of a list that moved -//! underneath the reader. +//! The snapshot cache behind `next --paginate` (6j6v.15ed), `mention by --paginate` (6j6v.q29p) +//! and `list --paginate` (6j6v.n698): the ordered result a first page was cut from, kept so the following pages come from +//! the SAME list instead of a list that moved underneath the reader. The table keeps its first +//! reader's name; every paged read shares it, its expiry and its caps, and tells its snapshots +//! apart by the `query` key, which names the read. //! //! **Machine-local.** It lives in the `next_page_cache` table of the workspace db. The table is not //! a view and is folded from nothing: no op writes it, no snapshot image carries it, no sync moves diff --git a/crates/core/src/store.rs b/crates/core/src/store.rs index 7f408dc..cf906c8 100644 --- a/crates/core/src/store.rs +++ b/crates/core/src/store.rs @@ -69,6 +69,41 @@ impl ItemReadStats { } } +/// How many EDGE-, LABEL- and CUSTOM-FIELD-read queries a [`Store`] has run since it was opened +/// (6j6v.x0wf) — the [`ItemReadStats`] instrument for the edge pass of the derived ranking keys +/// (docs/specs/derived-ranking-keys.md §5): a test asserts the pass runs once per query, not once +/// per candidate, by counting queries instead of timing them. Every read of an item's edges, +/// labels or custom fields is counted in one of the four buckets, so a per-candidate regression +/// through ANY of them shows (review of PR #48, Test Quality #1). +#[derive(Debug, Clone, Copy, Default, PartialEq, Eq)] +pub struct EdgeReadStats { + /// Whole-kind scans, [`Store::present_edges_of_kinds`] — one query each. + pub passes: u64, + /// Reverse mention lookups, [`Store::mentioned_by_bulk`] — one query per ≤500-target chunk. + pub mention_lookups: u64, + /// One-item reads — the edge targets ([`Store::deps_of`], [`Store::mentions_of`], + /// [`Store::contributes_to_of`], [`Store::parents_of`] and their fallible twins), + /// [`Store::children_of`], [`Store::labels_of`] and [`Store::custom_fields_of`]: one query each. + pub per_item: u64, + /// Bulk reads of labels, custom fields and `contributes_to` targets + /// ([`Store::labels_of_bulk`], [`Store::custom_fields_of_bulk`], + /// [`Store::contributes_to_of_bulk`]) — one query per ≤500-id chunk. + pub bulk: u64, +} + +impl EdgeReadStats { + /// The queries run between an `earlier` snapshot and this one, saturating like + /// [`ItemReadStats::since`]. + pub fn since(self, earlier: EdgeReadStats) -> EdgeReadStats { + EdgeReadStats { + passes: self.passes.saturating_sub(earlier.passes), + mention_lookups: self.mention_lookups.saturating_sub(earlier.mention_lookups), + per_item: self.per_item.saturating_sub(earlier.per_item), + bulk: self.bulk.saturating_sub(earlier.bulk), + } + } +} + /// flow's store over the shared substrate. pub struct Store { inner: Substrate, @@ -76,6 +111,8 @@ pub struct Store { /// `&self`; `Store` is already `!Sync` (it holds a `rusqlite::Connection`), so this adds no /// constraint — and bumping a `Cell` is noise next to the SQLite query it counts. item_reads: std::cell::Cell, + /// Query-shape instrumentation for the edge reads ([`EdgeReadStats`], 6j6v.x0wf). + edge_reads: std::cell::Cell, } impl Store { @@ -92,6 +129,7 @@ impl Store { let mut store = Store { inner, item_reads: std::cell::Cell::default(), + edge_reads: std::cell::Cell::default(), }; store.migrate_belongs_to_to_parent_edges(); store @@ -108,6 +146,7 @@ impl Store { let mut store = Store { inner, item_reads: std::cell::Cell::default(), + edge_reads: std::cell::Cell::default(), }; store.migrate_belongs_to_to_parent_edges(); Ok(store) @@ -163,6 +202,18 @@ impl Store { self.item_reads.set(c); } + /// Snapshot of the edge-read queries this store has run ([`EdgeReadStats`], 6j6v.x0wf). + pub fn edge_read_stats(&self) -> EdgeReadStats { + self.edge_reads.get() + } + + /// Add one edge-read query to the running totals: `bump` raises the bucket it belongs to. + fn count_edge_read(&self, bump: impl FnOnce(&mut EdgeReadStats)) { + let mut c = self.edge_reads.get(); + bump(&mut c); + self.edge_reads.set(c); + } + /// The current Lamport clock (highest lamport emitted/observed). pub fn clock(&self) -> i64 { self.inner.clock() @@ -998,6 +1049,7 @@ impl Store { let mut stmt = self .conn() .prepare("SELECT label FROM present_labels WHERE item_id=?1 ORDER BY label")?; + self.count_edge_read(|c| c.per_item += 1); let rows = stmt.query_map([item_id], |r| r.get::<_, String>(0))?; rows.collect() } @@ -1028,6 +1080,7 @@ impl Store { ORDER BY item_id, label" ); let mut stmt = self.conn().prepare(&sql)?; + self.count_edge_read(|c| c.bulk += 1); let rows = stmt.query_map(rusqlite::params_from_iter(chunk.iter()), |r| { Ok((r.get::<_, String>(0)?, r.get::<_, String>(1)?)) })?; @@ -1057,6 +1110,7 @@ impl Store { ORDER BY from_id, to_id" ); let mut stmt = self.conn().prepare(&sql)?; + self.count_edge_read(|c| c.bulk += 1); let rows = stmt.query_map(rusqlite::params_from_iter(chunk.iter()), |r| { Ok((r.get::<_, String>(0)?, r.get::<_, String>(1)?)) })?; @@ -1068,6 +1122,101 @@ impl Store { Ok(out) } + /// The present `mentions` SOURCES of every id in `targets`: for each target, the items that + /// mention it, sorted and distinct — the reverse of [`mentions_of`](Self::mentions_of), in the + /// [`contributes_to_of_bulk`](Self::contributes_to_of_bulk) shape (one query per 500 ids, over + /// the `edge_adds_to` index). A target nothing mentions is absent from the map. The sources are + /// edge endpoints, not checked against the items: the facade's `read::mentioners_of` resolves + /// them and drops the deleted ones (6j6v.q29p). + /// + /// A target given twice is looked up once: the targets are deduplicated before they are + /// chunked, so a repeat across two chunks cannot list its sources twice. + pub fn mentioned_by_bulk( + &self, + targets: &[&str], + ) -> rusqlite::Result>> { + const CHUNK: usize = 500; + let mut targets = targets.to_vec(); + targets.sort_unstable(); + targets.dedup(); + let mut out: BTreeMap> = BTreeMap::new(); + for chunk in targets.chunks(CHUNK) { + let placeholders = vec!["?"; chunk.len()].join(","); + let sql = format!( + "SELECT to_id, from_id FROM present_edges \ + WHERE kind = 'mentions' AND to_id IN ({placeholders}) \ + ORDER BY to_id, from_id" + ); + let mut stmt = self.conn().prepare(&sql)?; + self.count_edge_read(|c| c.mention_lookups += 1); + let rows = stmt.query_map(rusqlite::params_from_iter(chunk.iter()), |r| { + Ok((r.get::<_, String>(0)?, r.get::<_, String>(1)?)) + })?; + for row in rows { + let (to, from) = row?; + out.entry(to).or_default().push(from); + } + } + Ok(out) + } + + /// Every present edge of the given `kinds`, as `(from, to, kind)`, in `(kind, from, to)` order — + /// ONE query however many kinds and edges (6j6v.x0wf). The edge pass of the derived ranking + /// keys (docs/specs/derived-ranking-keys.md §5): one scan over the edges of the kinds a + /// plugin's keys name, so a query's cost is O(E) in those edges, never one query per candidate. + /// The endpoints are not checked against the items; the caller drops the dead ones. No kinds, + /// no query. + /// + /// **`parent` is the projection, not the OR-set** (review of PR #48, Integrity #1): its rows + /// come from the `present_parent` view — each child's ONE current parent, the LWW winner that + /// `belongs_to`, `--in` and `show` read — never from the raw `parent` edges, which hold two + /// parents for a child after a concurrent re-parent. Every other kind is its present edges. + pub fn present_edges_of_kinds( + &self, + kinds: &[EdgeKind], + ) -> rusqlite::Result> { + if kinds.is_empty() { + return Ok(Vec::new()); + } + let others: Vec<&str> = kinds + .iter() + .filter(|k| **k != EdgeKind::Parent) + .map(|k| k.as_str()) + .collect(); + let mut selects: Vec = Vec::new(); + if !others.is_empty() { + let placeholders = vec!["?"; others.len()].join(","); + selects.push(format!( + "SELECT from_id, to_id, kind FROM present_edges WHERE kind IN ({placeholders})" + )); + } + if kinds.contains(&EdgeKind::Parent) { + selects.push( + "SELECT child_id, parent_id, 'parent' AS kind FROM present_parent".to_string(), + ); + } + let sql = format!("{} ORDER BY 3, 1, 2", selects.join(" UNION ALL ")); + let mut stmt = self.conn().prepare(&sql)?; + self.count_edge_read(|c| c.passes += 1); + let names = others.into_iter(); + let rows = stmt.query_map(rusqlite::params_from_iter(names), |r| { + Ok(( + r.get::<_, String>(0)?, + r.get::<_, String>(1)?, + r.get::<_, String>(2)?, + )) + })?; + let mut out = Vec::new(); + for row in rows { + let (from, to, kind) = row?; + // A kind this binary does not know cannot match the `IN` list; skip defensively. + if let Some(kind) = EdgeKind::parse(&kind) { + out.push((from, to, kind)); + } + } + Ok(out) + } + // ---- plugin custom fields (6j6v.ekf5, T4): the read side of the folded `custom_fields` view -- /// The set custom-field values of ONE item (plugin-custom-fields §2.3), as `field → value` @@ -1082,6 +1231,7 @@ impl Store { let mut stmt = self.conn().prepare( "SELECT field, value FROM custom_fields WHERE item_id=?1 AND value <> '' ORDER BY field", )?; + self.count_edge_read(|c| c.per_item += 1); let rows = stmt.query_map([item_id], |r| { Ok((r.get::<_, String>(0)?, r.get::<_, String>(1)?)) })?; @@ -1112,6 +1262,7 @@ impl Store { ORDER BY item_id, field" ); let mut stmt = self.conn().prepare(&sql)?; + self.count_edge_read(|c| c.bulk += 1); let rows = stmt.query_map(rusqlite::params_from_iter(chunk.iter()), |r| { Ok(( r.get::<_, String>(0)?, @@ -1249,6 +1400,7 @@ impl Store { let mut stmt = self.conn().prepare( "SELECT from_id FROM present_edges WHERE to_id=?1 AND kind='parent' ORDER BY from_id", )?; + self.count_edge_read(|c| c.per_item += 1); let rows = stmt.query_map([parent], |r| r.get::<_, String>(0))?; rows.collect() } @@ -1297,6 +1449,7 @@ impl Store { let mut stmt = self.conn().prepare( "SELECT to_id FROM present_edges WHERE from_id=?1 AND kind=?2 ORDER BY to_id", )?; + self.count_edge_read(|c| c.per_item += 1); let rows = stmt.query_map(params![from, kind.as_str()], |r| r.get::<_, String>(0))?; rows.collect() } @@ -1513,6 +1666,139 @@ mod tests { ); } + #[test] + fn present_edges_of_kinds_reads_every_kind_asked_in_one_counted_query() { + // 6j6v.x0wf: the edge pass of the derived ranking keys — one query for every kind asked, + // removed edges excluded, other kinds left out; no kinds, no query. + let mut s = Store::open_in_memory(1); + for id in ["c1.a", "c1.b", "c1.c"] { + s.create_item(id, "task", "t", "t"); + } + s.add_edge("c1.a", "c1.b", EdgeKind::Mentions, "t"); + s.add_edge("c1.a", "c1.c", EdgeKind::Mentions, "t"); + s.remove_edge("c1.a", "c1.c", EdgeKind::Mentions, "t"); + s.add_edge("c1.b", "c1.c", EdgeKind::ContributesTo, "t"); + s.add_edge("c1.c", "c1.a", EdgeKind::Dep, "t"); + let before = s.edge_read_stats(); + let edges = s + .present_edges_of_kinds(&[EdgeKind::Mentions, EdgeKind::ContributesTo]) + .unwrap(); + assert_eq!( + edges, + vec![ + ( + "c1.b".to_string(), + "c1.c".to_string(), + EdgeKind::ContributesTo + ), + ("c1.a".to_string(), "c1.b".to_string(), EdgeKind::Mentions), + ] + ); + assert!(s.present_edges_of_kinds(&[]).unwrap().is_empty()); + let _ = s.mentioned_by_bulk(&["c1.b"]).unwrap(); + assert_eq!( + s.edge_read_stats().since(before), + EdgeReadStats { + passes: 1, + mention_lookups: 1, + ..EdgeReadStats::default() + } + ); + } + + #[test] + fn present_edges_of_kinds_reads_parent_as_the_projected_single_parent() { + // Review of PR #48, Integrity #1: after two replicas re-point one child concurrently, the + // OR-set holds both `parent` edges, but the pass reports only the `present_parent` winner — + // the parent `belongs_to`, `--in` and `show` read — so a derived key cannot count the child + // under both. Other kinds still come from the present edges, in the same one query. + let mut a = Store::open_in_memory(1); + for id in ["c1.P", "c1.Q", "c1.T"] { + a.create_item(id, "task", id, "x"); + } + let mut b = Store::open_in_memory(2); + b.apply(&a.export()); + a.set_parent("c1.T", "c1.P", "alice").unwrap(); + b.set_parent("c1.T", "c1.Q", "bob").unwrap(); + let (ea, eb) = (a.export(), b.export()); + a.apply(&eb); + b.apply(&ea); + a.add_edge("c1.T", "c1.P", EdgeKind::Mentions, "x"); + assert_eq!( + a.parents_of("c1.T").len(), + 2, + "precondition: two parent edges" + ); + let before = a.edge_read_stats(); + let edges = a + .present_edges_of_kinds(&[EdgeKind::Parent, EdgeKind::Mentions]) + .unwrap(); + assert_eq!( + edges, + vec![ + ("c1.T".to_string(), "c1.P".to_string(), EdgeKind::Mentions), + ("c1.T".to_string(), "c1.Q".to_string(), EdgeKind::Parent), + ], + "one parent — the projection's winner (site 2 at equal lamport) — in (kind, from, to) order" + ); + assert_eq!( + a.get_item("c1.T").unwrap().unwrap().belongs_to.as_deref(), + Some("c1.Q"), + "the same parent `belongs_to` reads" + ); + assert_eq!( + a.present_edges_of_kinds(&[EdgeKind::Parent]).unwrap().len(), + 1 + ); + assert_eq!(a.edge_read_stats().since(before).passes, 2); + } + + #[test] + fn every_edge_label_and_custom_field_read_is_counted() { + // Review of PR #48, Test Quality #1: the ratio guard of the derived keys sees a + // per-candidate regression only if every read of an item's edges, labels or custom fields + // is counted somewhere. + let mut s = Store::open_in_memory(1); + s.create_item("c1.a", "task", "a", "x"); + let before = s.edge_read_stats(); + let _ = s.deps_of("c1.a").unwrap(); + let _ = s.mentions_of("c1.a"); + let _ = s.mentions_of_result("c1.a").unwrap(); + let _ = s.contributes_to_of("c1.a").unwrap(); + let _ = s.parents_of("c1.a"); + let _ = s.parents_of_result("c1.a").unwrap(); + let _ = s.children_of("c1.a"); + let _ = s.labels_of("c1.a").unwrap(); + let _ = s.custom_fields_of("c1.a").unwrap(); + let _ = s.labels_of_bulk(&["c1.a"]).unwrap(); + let _ = s.custom_fields_of_bulk(&["c1.a"]).unwrap(); + let _ = s.contributes_to_of_bulk(&["c1.a"]).unwrap(); + assert_eq!( + s.edge_read_stats().since(before), + EdgeReadStats { + per_item: 9, + bulk: 3, + ..EdgeReadStats::default() + } + ); + } + + #[test] + fn mentioned_by_bulk_lists_a_target_repeated_across_chunks_once() { + // Review of PR #48, Integrity #4: the doc promises each target's sources distinct, and a + // target given in two 500-id chunks must not collect its sources twice. + let mut s = Store::open_in_memory(1); + s.create_item("c1.m", "task", "m", "x"); + s.create_item("c1.t", "task", "t", "x"); + s.add_edge("c1.m", "c1.t", EdgeKind::Mentions, "x"); + let fillers: Vec = (0..600).map(|n| format!("c1.f{n:03}")).collect(); + let mut targets: Vec<&str> = vec!["c1.t"]; + targets.extend(fillers.iter().map(String::as_str)); + targets.push("c1.t"); + let m = s.mentioned_by_bulk(&targets).unwrap(); + assert_eq!(m["c1.t"], ["c1.m"]); + } + #[test] fn get_items_empty_input_is_an_empty_result_no_query() { // PR-review Test Quality #2: an empty id slice must short-circuit to an empty Vec — never @@ -3761,6 +4047,27 @@ mod tests { assert!(crate::derive::blocked(s.connection()).unwrap().is_empty()); } + #[test] + fn mentioned_by_bulk_inverts_the_present_mentions_edges_per_target() { + // 6j6v.q29p: the reverse of `mentions_of`, for many targets in one read. + let mut s = Store::open_in_memory(1); + for id in ["c1.A", "c1.B", "c1.C", "c1.D", "c1.T", "c1.U"] { + s.create_item(id, "task", id, "x"); + } + s.add_edge("c1.C", "c1.T", EdgeKind::Mentions, "x"); + s.add_edge("c1.A", "c1.T", EdgeKind::Mentions, "x"); + s.add_edge("c1.A", "c1.T", EdgeKind::Mentions, "x"); // a second add: still one mentioner + s.add_edge("c1.B", "c1.U", EdgeKind::Mentions, "x"); + s.add_edge("c1.D", "c1.T", EdgeKind::Mentions, "x"); + s.remove_edge("c1.D", "c1.T", EdgeKind::Mentions, "x"); // removed: no longer a mentioner + s.add_edge("c1.B", "c1.T", EdgeKind::Dep, "x"); // another kind: not a mention + let m = s.mentioned_by_bulk(&["c1.T", "c1.U", "c1.A"]).unwrap(); + assert_eq!(m["c1.T"], ["c1.A", "c1.C"], "sorted and distinct"); + assert_eq!(m["c1.U"], ["c1.B"]); + assert!(!m.contains_key("c1.A"), "nothing mentions it: absent"); + assert!(s.mentioned_by_bulk(&[]).unwrap().is_empty()); + } + #[test] fn remap_prefix_swaps_only_the_prefix_across_ops_views_and_freetext() { let mut s = Store::open_in_memory(1); diff --git a/crates/facade/src/engine.rs b/crates/facade/src/engine.rs index a4f5221..ea3dbe5 100644 --- a/crates/facade/src/engine.rs +++ b/crates/facade/src/engine.rs @@ -11,7 +11,9 @@ use crate::error::Result; use crate::plugin::{self, PluginConfig}; -use crate::read::{self, BlockedItem, NextPage, NextQuery, PrimeReport, ShowRecord}; +use crate::read::{ + self, BlockedItem, ListQuery, MentionedByQuery, NextPage, NextQuery, PrimeReport, ShowRecord, +}; use crate::validate; use crate::workspace::{Workspace, WorkspaceExt}; use crate::write::{self, NewItem}; @@ -265,6 +267,37 @@ impl Engine { .with_state_mut(|s| write::mention_remove(&mut s.store, now, actor, from, to)) } + /// The ids `id` mentions — its outgoing reference edges, sorted (6j6v.q29p): the read + /// [`mention_add`](Engine::mention_add) writes, and what the CLI's `mention list` prints. + /// `not_found` if the item is missing/deleted, `io` on a db fault. + pub fn mentions(&self, id: &str) -> Result> { + self.handle.try_with_state(|s| read::mentions(&s.store, id)) + } + + /// The items that mention `id` (6j6v.q29p): the live mentioners — closed and archived included, + /// deleted ones not — in id order, narrowed by `query`'s filter (type, container, label, as for + /// [`next_query`](Engine::next_query)) and cut or paged exactly as `next_query` pages: the + /// same `total`/`start`/`next_token`/`restarted`, the same machine-local `next_page_cache` + /// (so the same db write another handle's [`subscribe`](Engine::subscribe) sees), the same + /// expiry and invalidation. The CLI's `mention by`. `not_found` if `id` is missing/deleted; + /// `validation` for `paginate` without a non-zero `limit`, or a token without `paginate`. + pub fn mentioned_by(&self, id: &str, query: &MentionedByQuery) -> Result { + self.handle + .try_with_state(|s| read::mentioned_by_page(&s.store, id, query)) + } + + /// [`mentioned_by`](Engine::mentioned_by) as JSON: `{"items", "total"}`, plus `next_token` + /// and `restarted` when the query paginates — `items` being the records + /// [`list_value`](Engine::list_value) gives. The envelope the CLI's + /// `mention by --limit --json` prints. + pub fn mentioned_by_value(&self, id: &str, query: &MentionedByQuery) -> Result { + self.handle.try_with_state(|s| { + let page = read::mentioned_by_page(&s.store, id, query)?; + let items = read::list_to_value_with_custom(&self.cfg, &s.store, &page.items)?; + Ok(read::next_page_envelope(items, &page, query.paginate)) + }) + } + /// Record a `from -> to` contributes-to edge (n:m; never blocks). pub fn contributes_add(&self, now: &str, actor: &str, from: &str, to: &str) -> Result<()> { self.handle @@ -457,6 +490,42 @@ impl Engine { }) } + /// [`list`](Engine::list) narrowed, ordered and paged (6j6v.n698): `--status`, `--type`, `--in`, + /// `--label`, `--sort`, `--limit`, `--paginate` and the page token, as one [`ListQuery`]. + /// Additive beside [`list`](Engine::list), whose signature and order stay. The CLI's `list`. + /// + /// The order is id unless the query sorts; `SortKey::Rank` ranks by the plugin's + /// `[ranking.list]` policy (its per-type orders `[ranking.list.]` and cross-type order + /// `types`, else the `[ranking.next]` default order). The type and container filters mean what + /// they mean for [`next_query`](Engine::next_query), and the page is cut and paged exactly as + /// `next_query` pages: the same `total`/`start`/`next_token`/`restarted`, the same + /// machine-local `next_page_cache` (so the same db write another handle's + /// [`subscribe`](Engine::subscribe) sees), the same expiry and invalidation. `validation` for + /// `paginate` without a non-zero `limit`, or a token without `paginate`. + /// + /// **`now` (6j6v.x0wf).** When the rows a ranked query keeps rank by a derived key of + /// `[ranking.list]` that reads the rank of a type `next` ranks, that rank is computed over + /// `next`'s candidates at a reference time: set it with [`ListQuery::now`], or the query + /// answers `validation` ("pass `now`"). The filters narrow the rows before they are ranked, so + /// a type filter whose order reads no such rank needs no `now`; the id order never does. A + /// malformed `now` answers `validation` too. + pub fn list_query(&self, query: &ListQuery) -> Result { + self.handle + .try_with_state(|s| read::list_page(&self.cfg, &s.store, query)) + } + + /// [`list_query`](Engine::list_query) as JSON: `{"items", "total"}`, plus `next_token` and + /// `restarted` when the query paginates — `items` being the records + /// [`list_value`](Engine::list_value) gives. The envelope the CLI's `list --limit --json` + /// prints. `now` is needed exactly as for [`list_query`](Engine::list_query). + pub fn list_query_value(&self, query: &ListQuery) -> Result { + self.handle.try_with_state(|s| { + let page = read::list_page(&self.cfg, &s.store, query)?; + let items = read::list_to_value_with_custom(&self.cfg, &s.store, &page.items)?; + Ok(read::next_page_envelope(items, &page, query.paginate)) + }) + } + /// [`next`](Engine::next) as the canonical `--json` array with each record's user labels attached /// (a sparse `labels` key per record). A malformed `now` is `validation`-rejected, as for /// [`next`](Engine::next). diff --git a/crates/facade/src/plugin/derived.rs b/crates/facade/src/plugin/derived.rs new file mode 100644 index 0000000..6b909db --- /dev/null +++ b/crates/facade/src/plugin/derived.rs @@ -0,0 +1,578 @@ +//! Derived ranking keys, the plugin side (docs/specs/derived-ranking-keys.md, 6j6v.x0wf). +//! +//! A key in a ranking `order` is a FIELD key (with or without a `precedence` list) or one of three +//! DERIVED keys that read the item's neighbours in the graph: +//! +//! - `related`: a value taken from the neighbours over one outbound edge kind (spec §3.1); +//! - `count`: how many neighbours over one edge kind and direction match (spec §3.2); +//! - `sum`: several counts added with equal weight (spec §3.3). +//! +//! This module holds their parts and the load-time rules that need the whole plugin: the filter +//! values, the fields `of` reads, which order `of = "rank"` reads, and the acyclic type chain +//! (spec §4, §7). The structural rules live with the other key rules in +//! [`RankSpec::validate_in`](super::RankSpec); the computation lives in the read layer. + +use super::{Dir, FieldDecl, Nulls, RankKey, RankSpec, Ranking, LIST_SECTION, NEXT_SECTION}; +use serde::{Deserialize, Serialize}; +use std::collections::{BTreeMap, BTreeSet}; + +/// The edge kinds a `related` key follows outward from the item (spec §3.1). +pub const RELATED_EDGES: &[&str] = &["parent", "contributes_to", "dep", "mentions"]; + +/// The kinds a `count` (and a `sum` summand) counts (spec §3.2): the four inbound names, then the +/// four outbound kinds of [`RELATED_EDGES`]. +pub const COUNT_EDGES: &[&str] = &[ + "children", + "contributors", + "dependents", + "mentioned_by", + "parent", + "contributes_to", + "dep", + "mentions", +]; + +/// How several neighbours of a `related` key become one value (spec §3.1). +#[derive(Debug, Clone, Copy, PartialEq, Eq, Deserialize, Serialize, Default)] +#[serde(rename_all = "lowercase")] +pub enum Pick { + /// The smallest value — the best rank, the lowest field value. The default. + #[default] + Min, + /// The largest value. + Max, +} + +/// What a `related` key reads on each neighbour (spec §3.1): `of = "rank"`, the neighbour's +/// position in its own type's declared order (spec §4), or `of = { field = ... }`, a field of the +/// neighbour — a canonical field or a custom field the plugin declares (spec §10.2). +#[derive(Debug, Clone, PartialEq, Eq)] +#[non_exhaustive] +pub enum RankOf { + /// The neighbour's rank in its own type's declared order. + Rank, + /// A field of the neighbour. + Field(OfField), +} + +/// `of = { field = ..., precedence = [...] }`: the neighbour's field value, or — with a +/// `precedence` list — the value's position in that list (an absent or unlisted value after every +/// listed one, as for a precedence key). +/// +/// `dir` and `nulls` are accepted at their defaults only (`asc`, `last`), so the spec's own example +/// `of = { field = "order", dir = "asc" }` loads. Direction and null placement belong to the +/// `related` key that holds the `of`; a non-default one here would be a silent no-op, so it is +/// refused at load, as `dir` on a precedence key is. +#[derive(Debug, Clone, PartialEq, Eq, Deserialize, Serialize)] +#[serde(deny_unknown_fields)] +#[non_exhaustive] +pub struct OfField { + pub field: String, + #[serde(default)] + pub dir: Dir, + #[serde(default)] + pub nulls: Nulls, + #[serde(default, skip_serializing_if = "Vec::is_empty")] + pub precedence: Vec, +} + +impl<'de> Deserialize<'de> for RankOf { + fn deserialize>(d: D) -> std::result::Result { + use serde::de::Error; + match toml::Value::deserialize(d)? { + toml::Value::String(s) if s == "rank" => Ok(RankOf::Rank), + toml::Value::String(s) => Err(D::Error::custom(format!( + "of = \"{s}\": the only name `of` takes is \"rank\"; a field is \ + `of = {{ field = \"...\" }}`" + ))), + v @ toml::Value::Table(_) => OfField::deserialize(v) + .map(RankOf::Field) + .map_err(|e| D::Error::custom(format!("of: {e}"))), + other => Err(D::Error::custom(format!( + "of = {other}: expected \"rank\" or `{{ field = \"...\" }}`" + ))), + } + } +} + +impl Serialize for RankOf { + fn serialize(&self, s: S) -> std::result::Result { + match self { + RankOf::Rank => s.serialize_str("rank"), + RankOf::Field(f) => f.serialize(s), + } + } +} + +/// One summand of a `sum` key (spec §3.3): exactly the parts of a `count` — the kind and its +/// optional `type`, `status` and `label` filters. `dir` belongs to the `sum`, so a summand refuses +/// it at parse, as it refuses any other part. A per-summand `weight` is deliberately left out +/// (spec §10.3); `#[non_exhaustive]` keeps room to add it. +#[derive(Debug, Clone, PartialEq, Eq, Deserialize, Serialize)] +#[serde(deny_unknown_fields)] +#[non_exhaustive] +pub struct CountPart { + pub count: String, + #[serde(default, rename = "type", skip_serializing_if = "Option::is_none")] + pub item_type: Option, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub status: Option>, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub label: Option, +} + +/// Which ranking policy a key stands in, and so which section it is spelled under. +#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Hash)] +pub(crate) enum Verb { + /// `[ranking.next]`. + Next, + /// `[ranking.list]`. + List, +} + +impl Verb { + /// The TOML section of this verb's policy. + pub(crate) fn section(self) -> &'static str { + match self { + Verb::Next => NEXT_SECTION, + Verb::List => LIST_SECTION, + } + } + + /// The other verb. + fn other(self) -> Verb { + match self { + Verb::Next => Verb::List, + Verb::List => Verb::Next, + } + } +} + +impl Ranking { + /// The policy of `verb`. + pub(crate) fn spec(&self, verb: Verb) -> &RankSpec { + match verb { + Verb::Next => &self.next, + Verb::List => &self.list, + } + } + + /// The declared order whose rank `of = "rank"` reads for a neighbour of type `ty`, asked from a + /// key under `verb` (spec §4: the type's rank under "the verb that ranks it"): `verb`'s own + /// `[ranking..]` when it declares one, else the other verb's. A person ranked by + /// `list` can so rank by the rank of the projects `next` ranks (spec §8). `None` when neither + /// declares an order for `ty`: such a neighbour has no rank and reads as null. + pub(crate) fn rank_source(&self, verb: Verb, ty: &str) -> Option { + [verb, verb.other()] + .into_iter() + .find(|v| self.spec(*v).by_type.contains_key(ty)) + } + + /// Whether the policy of `verb` holds a derived key anywhere. + pub(crate) fn has_derived(&self, verb: Verb) -> bool { + self.spec(verb).has_derived() + } + + /// The load-time rules of the derived keys that need the whole plugin (spec §7), checked after + /// the structural rules of each section ([`RankSpec::validate_in`]/`validate_types_in`): + /// + /// - a `status` filter names only statuses of `[vocabulary.status]`; + /// - `of = { field = ... }` names a field the ranking can read + /// ([`RANKABLE_FIELDS`](crate::record::RANKABLE_FIELDS)) or a custom field of `[fields]`, + /// and a custom field scoped `on` some types applies to the key's `type`; + /// - `of = "rank"` with a `type` names a type with a declared order + /// (`[ranking.next.]` or `[ranking.list.]`), and in the `[ranking.list]` + /// default order `of = "rank"` names a `type`; + /// - the type graph is acyclic: an edge from each per-type order to every order its keys read + /// the rank of; a cycle is refused with the cycle named (spec §4). + pub(crate) fn validate_derived( + &self, + statuses: &BTreeMap, + fields: &BTreeMap, + ) -> std::result::Result<(), String> { + for verb in [Verb::Next, Verb::List] { + let section = verb.section(); + for (scope, order) in self.spec(verb).orders(section) { + for key in order.iter().filter(|k| k.is_derived()) { + self.validate_derived_key(verb, &scope, key, statuses, fields)?; + } + } + } + self.check_type_chain() + } + + fn validate_derived_key( + &self, + verb: Verb, + scope: &str, + key: &RankKey, + statuses: &BTreeMap, + fields: &BTreeMap, + ) -> std::result::Result<(), String> { + let desc = key.describe(); + let check_status = |status: &Option>| { + for s in status.iter().flatten() { + if !statuses.contains_key(s) { + let valid: Vec<&str> = statuses.keys().map(String::as_str).collect(); + return Err(format!( + "{scope}ranking key {desc} filters on status '{s}', which \ + [vocabulary.status] does not declare; valid: {}", + valid.join(", ") + )); + } + } + Ok(()) + }; + check_status(&key.status)?; + for part in key.sum.iter().flatten() { + check_status(&part.status)?; + } + match &key.of { + Some(RankOf::Field(of)) => { + let canonical = crate::record::RANKABLE_FIELDS.contains(&of.field.as_str()); + match fields.get(&of.field) { + _ if canonical => {} + None => { + return Err(format!( + "{scope}ranking key {desc} reads `of` field '{}', which is neither a \ + field the ranking can read ({}) nor a custom field of [fields]", + of.field, + crate::record::RANKABLE_FIELDS.join(", ") + )); + } + Some(decl) => { + if let Some(ty) = &key.item_type { + if !decl.on.is_empty() && !decl.on.contains(ty) { + return Err(format!( + "{scope}ranking key {desc} reads custom field '{}' on '{ty}' \ + neighbours, but the field applies only to {} — it would \ + read null on every neighbour", + of.field, + decl.on.join(", ") + )); + } + } + } + } + } + // An untyped rank key in the default order (only `[ranking.list]`'s can hold one) + // would read the rank of every type with a declared order — a silent no-op without + // per-type tables, and with them a chain over every type that may demand `now`. The + // per-type orders refuse the same key as a self-cycle; refused here alike (review of + // PR #48, Integrity #3). + Some(RankOf::Rank) if scope.is_empty() && key.item_type.is_none() => { + return Err(format!( + "[{}] default `order`: ranking key {desc} reads `of = \"rank\"` without \ + `type`, which would read the rank of every type with a declared order — \ + name the neighbour's `type`", + verb.section() + )); + } + Some(RankOf::Rank) => { + if let Some(ty) = &key.item_type { + if self.rank_source(verb, ty).is_none() { + return Err(format!( + "{scope}ranking key {desc} reads the rank of '{ty}', which has no \ + declared order — declare [{}.{ty}] or [{}.{ty}], or read a field \ + with `of = {{ field = \"...\" }}`", + verb.section(), + verb.other().section(), + )); + } + } + } + None => {} + } + Ok(()) + } + + /// The orders a per-type order `(verb, ty)` reads the rank of: for each `of = "rank"` key, the + /// [`rank_source`](Self::rank_source) of its `type`, or of EVERY type with a declared order when + /// the key names none (any neighbour may then be of any type, the order's own type included). + pub(crate) fn rank_deps(&self, keys: &[RankKey], verb: Verb) -> BTreeSet<(Verb, String)> { + let ranked: BTreeSet<&String> = [Verb::Next, Verb::List] + .into_iter() + .flat_map(|v| self.spec(v).by_type.keys()) + .collect(); + let mut deps = BTreeSet::new(); + for key in keys.iter().filter(|k| k.of == Some(RankOf::Rank)) { + let targets: Vec<&String> = match &key.item_type { + Some(ty) => vec![ty], + None => ranked.iter().copied().collect(), + }; + for ty in targets { + if let Some(src) = self.rank_source(verb, ty) { + deps.insert((src, ty.clone())); + } + } + } + deps + } + + /// Refuse a cycle in the type graph (spec §4) — depth-first over the per-type orders, naming + /// the first cycle found as its chain of sections. + fn check_type_chain(&self) -> std::result::Result<(), String> { + let nodes: Vec<(Verb, String)> = [Verb::Next, Verb::List] + .into_iter() + .flat_map(|v| self.spec(v).by_type.keys().map(move |t| (v, t.clone()))) + .collect(); + let mut done: BTreeSet<(Verb, String)> = BTreeSet::new(); + for start in &nodes { + let mut path: Vec<(Verb, String)> = Vec::new(); + self.visit(start.clone(), &mut path, &mut done)?; + } + Ok(()) + } + + fn visit( + &self, + node: (Verb, String), + path: &mut Vec<(Verb, String)>, + done: &mut BTreeSet<(Verb, String)>, + ) -> std::result::Result<(), String> { + if done.contains(&node) { + return Ok(()); + } + if let Some(at) = path.iter().position(|n| *n == node) { + let name = |(v, t): &(Verb, String)| format!("[{}.{t}]", v.section()); + let mut chain: Vec = path[at..].iter().map(name).collect(); + chain.push(name(&node)); + return Err(format!( + "the ranking types rank by each other in a cycle: {} — a type may rank by the \ + rank of another only along a chain. A `related ... of = \"rank\"` key without \ + `type` reads the rank of every type with a declared order, its own included; \ + name the neighbour's `type`", + chain.join(" → ") + )); + } + let keys = &self.spec(node.0).by_type[&node.1].order; + let deps = self.rank_deps(keys, node.0); + path.push(node.clone()); + for dep in deps { + self.visit(dep, path, done)?; + } + path.pop(); + done.insert(node); + Ok(()) + } +} + +impl RankSpec { + /// Every order of this policy with the scope its errors name: the default order (an empty + /// scope), then each per-type order (`[section.] `). + pub(crate) fn orders(&self, section: &str) -> Vec<(String, &Vec)> { + std::iter::once((String::new(), &self.order)) + .chain( + self.by_type + .iter() + .map(|(t, o)| (format!("[{section}.{t}] "), &o.order)), + ) + .collect() + } + + /// Whether any order of this policy holds a derived key. + pub fn has_derived(&self) -> bool { + self.order + .iter() + .chain(self.by_type.values().flat_map(|o| o.order.iter())) + .any(RankKey::is_derived) + } +} + +impl RankKey { + /// Whether this is a derived key (`related`, `count` or `sum`) rather than a field key. + pub fn is_derived(&self) -> bool { + self.related.is_some() || self.count.is_some() || self.sum.is_some() + } + + /// The key as its errors name it: `''` for a field key, its kind and edge for a derived + /// one (`related = "parent"`, `count = "children"`, `sum`). + pub(crate) fn describe(&self) -> String { + if let Some(r) = &self.related { + format!("`related = \"{r}\"`") + } else if let Some(c) = &self.count { + format!("`count = \"{c}\"`") + } else if self.sum.is_some() { + "`sum`".to_string() + } else { + format!("'{}'", self.field) + } + } + + /// The structural rules of one key (spec §7), checked by + /// [`RankSpec::validate_in`](super::RankSpec) for every key of every order. `scope` prefixes + /// the error (`[ranking.next.] ` or empty for the default order); `default_next` says the + /// key stands in the `[ranking.next]` default order. + pub(crate) fn validate_kind( + &self, + scope: &str, + default_next: bool, + ) -> std::result::Result<(), String> { + let kinds = [ + (!self.field.is_empty(), "field"), + (self.related.is_some(), "related"), + (self.count.is_some(), "count"), + (self.sum.is_some(), "sum"), + ]; + let set: Vec<&str> = kinds.iter().filter(|(on, _)| *on).map(|k| k.1).collect(); + let desc = self.describe(); + match set.len() { + 0 => { + return Err(format!( + "{scope}a ranking key names no kind — give it `field`, `related`, `count` or \ + `sum`" + )) + } + 1 => {} + _ => { + return Err(format!( + "{scope}ranking key {desc} mixes kinds ({}) — a key is exactly one of field, \ + related, count or sum", + set.join(", ") + )) + } + } + let parts = [ + (self.item_type.is_some(), "type"), + (self.of.is_some(), "of"), + (self.pick.is_some(), "pick"), + (self.status.is_some(), "status"), + (self.label.is_some(), "label"), + ]; + let stray = |allowed: &[&str]| { + parts + .iter() + .find(|(on, name)| *on && !allowed.contains(name)) + .map(|p| p.1) + }; + if !self.is_derived() { + if let Some(part) = stray(&[]) { + return Err(format!( + "{scope}ranking key {desc} is a field key, and `{part}` belongs to a derived \ + key (related, count, sum)" + )); + } + return Ok(()); + } + if default_next { + return Err(format!( + "{scope}ranking key {desc} is a derived key in the [ranking.next] default \ + `order`, which also orders `blocked` and the lanes' `--sort rank`; derived keys \ + rank in a per-type order [ranking.next.] (or under [ranking.list])" + )); + } + if !self.precedence.is_empty() { + return Err(format!( + "{scope}ranking key {desc} sets a precedence list; on a derived key it belongs in \ + `of = {{ field = \"...\", precedence = [...] }}`" + )); + } + let never_null = |what: &str| { + if self.nulls != Nulls::Last { + Err(format!( + "{scope}ranking key {desc} sets `nulls`, but {what} is never null — drop it" + )) + } else { + Ok(()) + } + }; + if let Some(edge) = &self.related { + if !RELATED_EDGES.contains(&edge.as_str()) { + return Err(format!( + "{scope}ranking key {desc} follows no edge kind the ranking knows; valid: {}", + RELATED_EDGES.join(", ") + )); + } + if let Some(part) = stray(&["type", "of", "pick"]) { + return Err(format!( + "{scope}ranking key {desc} sets `{part}`, which a `related` key does not \ + take (its parts: type, of, pick, dir, nulls)" + )); + } + match &self.of { + None => { + return Err(format!( + "{scope}ranking key {desc} does not say what to read on the neighbours — \ + add `of = \"rank\"` or `of = {{ field = \"...\" }}`" + )) + } + Some(RankOf::Field(of)) if of.dir != Dir::Asc || of.nulls != Nulls::Last => { + return Err(format!( + "{scope}ranking key {desc} sets dir/nulls inside `of`; they belong to the \ + `related` key itself — move them out of `of`" + )) + } + Some(_) => {} + } + } else if let Some(edge) = &self.count { + Self::check_count_edge(scope, &desc, edge)?; + if let Some(part) = stray(&["type", "status", "label"]) { + return Err(format!( + "{scope}ranking key {desc} sets `{part}`, which a `count` key does not take \ + (its parts: type, status, label, dir)" + )); + } + Self::check_status_list(scope, &desc, &self.status)?; + never_null("a count")?; + } else if let Some(parts) = &self.sum { + if parts.is_empty() { + return Err(format!( + "{scope}ranking key {desc} adds no counts — list at least one `{{ count = ... }}`" + )); + } + if let Some(part) = stray(&[]) { + return Err(format!( + "{scope}ranking key {desc} sets `{part}`; a `sum` takes only its summands and \ + `dir` — filters belong in each summand" + )); + } + for p in parts { + Self::check_count_edge(scope, &desc, &p.count)?; + Self::check_status_list(scope, &desc, &p.status)?; + } + never_null("a sum")?; + } + Ok(()) + } + + fn check_count_edge(scope: &str, desc: &str, edge: &str) -> std::result::Result<(), String> { + if COUNT_EDGES.contains(&edge) { + Ok(()) + } else { + Err(format!( + "{scope}ranking key {desc} counts '{edge}', which is no edge kind or direction \ + the ranking knows; valid: {}", + COUNT_EDGES.join(", ") + )) + } + } + + fn check_status_list( + scope: &str, + desc: &str, + status: &Option>, + ) -> std::result::Result<(), String> { + if status.as_ref().is_some_and(Vec::is_empty) { + return Err(format!( + "{scope}ranking key {desc} filters on an empty status list, which counts nothing \ + — list the statuses, or drop `status`" + )); + } + Ok(()) + } + + /// The item types this key names (its own `type` and each summand's), for the check against + /// `[types].list`. + pub(crate) fn named_types(&self) -> Vec<&str> { + self.item_type + .iter() + .map(String::as_str) + .chain( + self.sum + .iter() + .flatten() + .filter_map(|p| p.item_type.as_deref()), + ) + .collect() + } +} diff --git a/crates/facade/src/plugin/mod.rs b/crates/facade/src/plugin/mod.rs index a3d22cc..38947a9 100644 --- a/crates/facade/src/plugin/mod.rs +++ b/crates/facade/src/plugin/mod.rs @@ -9,7 +9,10 @@ use crate::matrix::RelationshipMatrix; use serde::{Deserialize, Serialize}; use std::collections::BTreeMap; +mod derived; pub mod registry; +pub(crate) use derived::Verb; +pub use derived::{CountPart, OfField, Pick, RankOf, COUNT_EDGES, RELATED_EDGES}; pub use registry::PluginRegistration; /// The registered plugin names in canonical order — the registry-driven replacement for the old @@ -164,15 +167,31 @@ fn is_valid_field_name(name: &str) -> bool { bytes.all(|b| b.is_ascii_lowercase() || b.is_ascii_digit() || b == b'_') } -/// One ordering key in the `next` ranking policy. Two modes: a FIELD key (`dir`/`nulls` over the -/// stored value — numeric-aware for the priority ordinal, lexicographic else) and, when -/// `precedence` is non-empty, an ENUM-INDEX key (sp6.5): items sort by the position of their -/// `field` value in `precedence`, e.g. type `epic` before `bug` or status `in_progress` before -/// `open`. The list order IS the sort order, so a precedence key ignores `dir`/`nulls`; a value -/// absent from the list sorts after every listed one (the deterministic `id` tiebreak then settles -/// it). This is how a plugin expresses type/status precedence without any hardcoding in the engine. +/// One ordering key in a ranking policy. Two kinds of key share the struct: +/// +/// - A FIELD key (`field` set): `dir`/`nulls` over the stored value — numeric-aware for the +/// priority ordinal, lexicographic else — and, when `precedence` is non-empty, an ENUM-INDEX key +/// (sp6.5): items sort by the position of their `field` value in `precedence`, e.g. type `epic` +/// before `bug` or status `in_progress` before `open`. The list order IS the sort order, so a +/// precedence key ignores `dir`/`nulls`; a value absent from the list sorts after every listed +/// one (the deterministic `id` tiebreak then settles it). This is how a plugin expresses +/// type/status precedence without any hardcoding in the engine. +/// - A DERIVED key (6j6v.x0wf, docs/specs/derived-ranking-keys.md): exactly one of `related`, +/// `count` or `sum`, ranking an item by facts about its neighbours in the graph. `related` +/// reads a value on the neighbours over one outbound edge kind (`of`, combined by `pick`); +/// `count` counts the neighbours over one edge kind and direction that pass its `type`/`status`/ +/// `label` filters; `sum` adds several counts with equal weight. The value compares like a +/// field key's, with `dir` and `nulls`. +/// +/// A key that is neither, or both, or that carries a part its kind does not take, is refused at +/// load ([`RankSpec::validate`]). Unknown parts are a parse error. `#[non_exhaustive]` since +/// 6j6v.x0wf, so a later part (a summand `weight`, say) is additive. #[derive(Debug, Clone, PartialEq, Eq, Deserialize, Serialize)] +#[serde(deny_unknown_fields)] +#[non_exhaustive] pub struct RankKey { + /// The field a field key sorts on; empty on a derived key. + #[serde(default, skip_serializing_if = "String::is_empty")] pub field: String, #[serde(default)] pub dir: Dir, @@ -180,9 +199,52 @@ pub struct RankKey { pub nulls: Nulls, #[serde(default)] pub precedence: Vec, + /// `related = ""`: follow this edge kind outward ([`RELATED_EDGES`]). + #[serde(default, skip_serializing_if = "Option::is_none")] + pub related: Option, + /// `count = ""`: count the neighbours over this kind and direction ([`COUNT_EDGES`]). + #[serde(default, skip_serializing_if = "Option::is_none")] + pub count: Option, + /// `sum = [ { count = ... }, ... ]`: add these counts with equal weight. + #[serde(default, skip_serializing_if = "Option::is_none")] + pub sum: Option>, + /// `type`: only neighbours of this item type (`related`, `count`). + #[serde(default, rename = "type", skip_serializing_if = "Option::is_none")] + pub item_type: Option, + /// `of`: what a `related` key reads on each neighbour. + #[serde(default, skip_serializing_if = "Option::is_none")] + pub of: Option, + /// `pick`: how a `related` key makes one value of several neighbours (default `min`). + #[serde(default, skip_serializing_if = "Option::is_none")] + pub pick: Option, + /// `status`: a `count` counts only neighbours with one of these core statuses. + #[serde(default, skip_serializing_if = "Option::is_none")] + pub status: Option>, + /// `label`: a `count` counts only neighbours carrying this label. + #[serde(default, skip_serializing_if = "Option::is_none")] + pub label: Option, } impl RankKey { + /// A field key on `field`, ascending, nulls last, no precedence — the starting point for + /// building a key in code now that the struct is `#[non_exhaustive]`. + pub fn field_key(field: impl Into) -> RankKey { + RankKey { + field: field.into(), + dir: Dir::Asc, + nulls: Nulls::Last, + precedence: Vec::new(), + related: None, + count: None, + sum: None, + item_type: None, + of: None, + pick: None, + status: None, + label: None, + } + } + /// The value's sort ordinal for an enum-index (`precedence`) key: its position in the list, or /// `precedence.len()` when absent/none (so unlisted values sort after every listed one). Only /// meaningful when [`precedence`](Self::precedence) is non-empty. @@ -204,7 +266,8 @@ pub struct TypeOrder { pub order: Vec, } -/// The `next` ranking policy, TOML `[ranking.next]`. +/// A ranking policy: TOML `[ranking.next]`, and since 6j6v.n698 also `[ranking.list]` +/// ([`Ranking::list`]), which takes the same keys, tables and rules (its errors name its section). /// /// - `order` is the default order, and the only one a plugin without per-type orders has. Such a /// plugin ranks exactly as before 6j6v.z9jk. @@ -249,6 +312,26 @@ pub struct RankSpec { /// loses it, and "invalid type: integer" alone does not say which line under `[ranking.next]` is /// wrong (review of #44, Code #5). fn per_type_orders<'de, D>(d: D) -> std::result::Result, D::Error> +where + D: serde::Deserializer<'de>, +{ + per_type_orders_in(d, NEXT_SECTION) +} + +/// [`per_type_orders`] for `[ranking.list]` (6j6v.n698). +fn per_type_list_orders<'de, D>(d: D) -> std::result::Result, D::Error> +where + D: serde::Deserializer<'de>, +{ + per_type_orders_in(d, LIST_SECTION) +} + +/// The per-type orders under the section `section` (`ranking.next` or `ranking.list`) — one +/// implementation for both, so the two sections parse alike and their errors name their own lines. +fn per_type_orders_in<'de, D>( + d: D, + section: &str, +) -> std::result::Result, D::Error> where D: serde::Deserializer<'de>, { @@ -258,18 +341,23 @@ where .map(|(key, value)| { if !value.is_table() { return Err(D::Error::custom(format!( - "[ranking.next] key '{key}' is not a per-type order table \ - `[ranking.next.]`; the only other keys allowed under [ranking.next] are \ + "[{section}] key '{key}' is not a per-type order table \ + `[{section}.]`; the only other keys allowed under [{section}] are \ `order` and `types`" ))); } TypeOrder::deserialize(value) .map(|o| (key.clone(), o)) - .map_err(|e| D::Error::custom(format!("[ranking.next.{key}]: {e}"))) + .map_err(|e| D::Error::custom(format!("[{section}.{key}]: {e}"))) }) .collect() } +/// The TOML section of the `next` ranking policy. +const NEXT_SECTION: &str = "ranking.next"; +/// The TOML section of the `list` ranking policy (6j6v.n698). +const LIST_SECTION: &str = "ranking.list"; + impl RankSpec { /// The key rules, checked in the default order and in every per-type order (6j6v.z9jk): /// @@ -282,12 +370,23 @@ impl RankSpec { /// /// [`RANKABLE_FIELDS`]: crate::record::RANKABLE_FIELDS pub fn validate(&self) -> std::result::Result<(), String> { + self.validate_in(NEXT_SECTION) + } + + /// [`validate`](Self::validate) for the policy that stands under the TOML section `section`, + /// which the errors name (6j6v.n698: `ranking.next` and `ranking.list` share the rules). + pub(crate) fn validate_in(&self, section: &str) -> std::result::Result<(), String> { let per_type = self .by_type .iter() - .map(|(t, o)| (format!("[ranking.next.{t}] "), &o.order)); + .map(|(t, o)| (format!("[{section}.{t}] "), &o.order)); for (scope, order) in std::iter::once((String::new(), &self.order)).chain(per_type) { + let default_next = section == NEXT_SECTION && scope.is_empty(); for key in order { + key.validate_kind(&scope, default_next)?; + if key.is_derived() { + continue; + } if !crate::record::RANKABLE_FIELDS.contains(&key.field.as_str()) { return Err(format!( "{scope}ranking key '{}' names no field the ranking can read; valid: {}", @@ -320,17 +419,26 @@ impl RankSpec { pub fn validate_types( &self, declared: &std::collections::BTreeSet, + ) -> std::result::Result<(), String> { + self.validate_types_in(NEXT_SECTION, declared) + } + + /// [`validate_types`](Self::validate_types) for the policy under the TOML section `section`. + pub(crate) fn validate_types_in( + &self, + section: &str, + declared: &std::collections::BTreeSet, ) -> std::result::Result<(), String> { for (ty, o) in &self.by_type { if !declared.contains(ty) { return Err(format!( - "[ranking.next.{ty}] names a type the plugin does not declare — add '{ty}' \ + "[{section}.{ty}] names a type the plugin does not declare — add '{ty}' \ to [types].list or drop the table" )); } if o.order.iter().any(|k| k.field == "type") { return Err(format!( - "[ranking.next.{ty}] has a key on 'type'; every item it ranks is a '{ty}', \ + "[{section}.{ty}] has a key on 'type'; every item it ranks is a '{ty}', \ so the key never decides — drop it (the cross-type order is `types`)" )); } @@ -349,31 +457,44 @@ impl RankSpec { } Ok(()) }; + for (scope, order) in self.orders(section) { + for key in order { + if let Some(ty) = key + .named_types() + .into_iter() + .find(|t| !declared.contains(*t)) + { + return Err(format!( + "{scope}ranking key {} filters on type '{ty}', which the plugin does not \ + declare in [types].list", + key.describe() + )); + } + } + } distinct_declared("`types`", &self.types)?; for key in self.order.iter().filter(|k| k.field == "type") { distinct_declared("the precedence key on 'type'", &key.precedence)?; } if !self.types.is_empty() && self.by_type.is_empty() { - return Err( + return Err(format!( "`types` orders items of different types only for per-type orders, and there is \ - no [ranking.next.] table — drop `types`, or put the type key in `order`" - .to_string(), - ); + no [{section}.] table — drop `types`, or put the type key in `order`" + )); } if self.by_type.is_empty() { return Ok(()); } let cross = self.cross_type_order(); if cross.is_empty() { - return Err( + return Err(format!( "per-type orders need a cross-type order: declare `types = [...]` under \ - [ranking.next], or make the first key on 'type' in its `order` a precedence key" - .to_string(), - ); + [{section}], or make the first key on 'type' in its `order` a precedence key" + )); } if let Some(ty) = self.by_type.keys().find(|t| !cross.contains(t)) { return Err(format!( - "[ranking.next.{ty}] has its own order but '{ty}' is not in the cross-type order \ + "[{section}.{ty}] has its own order but '{ty}' is not in the cross-type order \ ({}) — list it, so where its items stand is declared, not left to its name", cross.join(", ") )); @@ -418,11 +539,69 @@ impl RankSpec { } /// The plugin's ranking policies, TOML `[ranking]`. `#[non_exhaustive]` since 6j6v.z9jk, so a -/// `list` policy (n698) is an additive field. +/// further policy is an additive field. #[derive(Debug, Clone, PartialEq, Eq, Deserialize)] +#[serde(from = "RankingToml")] #[non_exhaustive] pub struct Ranking { + /// `[ranking.next]`: the order `next` ranks by (within its finish-first tiers), and `blocked` + /// and the lane verbs' `--sort rank`. pub next: RankSpec, + /// `[ranking.list]` (6j6v.n698): the order `list --sort rank` ranks by — the same key language, + /// the same per-type tables `[ranking.list.]`, the same cross-type `types = [...]` and + /// the same load-time rules as [`next`](Self::next), and the same comparator. + /// + /// Always set, resolved at load: + /// + /// - `[ranking.list]` absent: the `[ranking.next]` default `order` alone, no per-type orders — + /// exactly what `list --sort rank` ranked by before n698; + /// - `[ranking.list]` without its own `order`: the `[ranking.next]` default `order` is its + /// default, so a type without a `[ranking.list.]` table ranks as it did before. + pub list: RankSpec, +} + +/// `[ranking]` as written: the list policy's `order` is optional until [`Ranking`] resolves it. +/// Only `next` and `list` stand under it: a misspelt `[ranking.lsit]` is refused at load rather +/// than ignored in silence (review of PR #48, Integrity #2). +#[derive(Deserialize)] +#[serde(deny_unknown_fields)] +struct RankingToml { + next: RankSpec, + #[serde(default)] + list: Option, +} + +/// `[ranking.list]` as written (6j6v.n698): [`RankSpec`]'s fields with `order` optional, and its +/// per-type tables parsed by the same code under their own section name. +#[derive(Deserialize)] +struct ListRankToml { + #[serde(default)] + order: Option>, + #[serde(default)] + types: Vec, + #[serde(flatten, deserialize_with = "per_type_list_orders")] + by_type: BTreeMap, +} + +impl From for Ranking { + fn from(raw: RankingToml) -> Ranking { + let list = match raw.list { + None => RankSpec { + order: raw.next.order.clone(), + types: Vec::new(), + by_type: BTreeMap::new(), + }, + Some(l) => RankSpec { + order: l.order.unwrap_or_else(|| raw.next.order.clone()), + types: l.types, + by_type: l.by_type, + }, + }; + Ranking { + next: raw.next, + list, + } + } } /// Priority as named variants (8qv.2): `labels` is the ordered set of variant names — index 0 @@ -657,13 +836,26 @@ fn load_registration(reg: &PluginRegistration) -> Result { cfg.types.validate().map_err(|e| { NxfError::validation(format!("plugin '{}' has an invalid [types]: {e}", reg.name)) })?; + for (section, spec) in [ + (NEXT_SECTION, &cfg.ranking.next), + (LIST_SECTION, &cfg.ranking.list), + ] { + spec.validate_in(section) + .and_then(|()| spec.validate_types_in(section, &cfg.types.list)) + .map_err(|e| { + NxfError::validation(format!( + "plugin '{}' has an invalid [{section}]: {e}", + reg.name + )) + })?; + } + // 6j6v.x0wf: the derived keys' rules that need the whole plugin — statuses, the fields `of` + // reads, the orders `of = "rank"` reads, and the acyclic type chain across both sections. cfg.ranking - .next - .validate() - .and_then(|()| cfg.ranking.next.validate_types(&cfg.types.list)) + .validate_derived(&cfg.vocabulary.status, &cfg.fields) .map_err(|e| { NxfError::validation(format!( - "plugin '{}' has an invalid [ranking.next]: {e}", + "plugin '{}' has an invalid [ranking]: {e}", reg.name )) })?; @@ -913,6 +1105,7 @@ mod tests { dir, nulls, precedence: vec!["epic".into(), "bug".into()], + ..RankKey::field_key("") }], types: Vec::new(), by_type: BTreeMap::new(), @@ -934,6 +1127,7 @@ mod tests { dir: Dir::Desc, nulls: Nulls::First, precedence: Vec::new(), + ..RankKey::field_key("") }], types: Vec::new(), by_type: BTreeMap::new(), @@ -1713,4 +1907,591 @@ mod tests { assert!(!spec.splits_by_type(), "{name}"); } } + + // ---- `[ranking.list]` (6j6v.n698) ----------------------------------------------------- + + /// `[ranking.next]` with a type precedence key, the base every `[ranking.list]` test adds to. + const NEXT_BASE: &str = r#" + [ranking.next] + order = [ { field = "priority", dir = "asc" } ] + "#; + + #[test] + fn without_ranking_list_the_list_order_is_the_next_default_order() { + // `list --sort rank` ranked by the `[ranking.next]` default order before n698; a plugin + // that declares no `[ranking.list]` keeps exactly that, and no per-type split. + let cfg = load_with_ranking(PER_TYPE).unwrap(); + let list = &cfg.ranking.list; + assert_eq!(list.order, cfg.ranking.next.order); + assert!(list.types.is_empty() && list.by_type.is_empty()); + assert!( + !list.splits_by_type(), + "next's per-type orders are next's alone" + ); + for name in ["issue-tracker", "personal-todo"] { + let r = load(name).unwrap().ranking; + assert_eq!(r.list.order, r.next.order, "{name}"); + assert!(!r.list.splits_by_type(), "{name}"); + } + } + + #[test] + fn ranking_list_per_type_orders_parse_and_fall_back_to_the_next_default_order() { + let cfg = load_with_ranking(&format!( + r#"{NEXT_BASE} + [ranking.list] + types = ["note", "project"] + [ranking.list.note] + order = [ {{ field = "due", dir = "desc", nulls = "last" }} ] + "# + )) + .unwrap(); + let list = &cfg.ranking.list; + assert!(list.splits_by_type()); + assert_eq!(list.order_for(Some("note"))[0].field, "due"); + assert_eq!( + list.order_for(Some("action")), + cfg.ranking.next.order.as_slice(), + "no `order` under [ranking.list] ⇒ the [ranking.next] default order" + ); + assert_eq!(list.cross_type_order(), ["note", "project"]); + assert!(!cfg.ranking.next.splits_by_type(), "next is untouched"); + } + + #[test] + fn ranking_list_may_declare_its_own_default_order() { + let cfg = load_with_ranking(&format!( + r#"{NEXT_BASE} + [ranking.list] + order = [ {{ field = "title", dir = "asc" }} ] + "# + )) + .unwrap(); + assert_eq!(cfg.ranking.list.order[0].field, "title"); + assert_eq!(cfg.ranking.next.order[0].field, "priority"); + } + + #[test] + fn ranking_list_is_held_to_the_same_rules_and_its_errors_name_its_section() { + // An unknown field in a per-type list table. + let err = refused(&format!( + r#"{NEXT_BASE} + [ranking.list] + types = ["note"] + [ranking.list.note] + order = [ {{ field = "dua" }} ] + "# + )); + assert_eq!(err.kind, crate::error::ErrorKind::Validation); + assert!( + err.msg.contains("invalid [ranking.list]") + && err.msg.contains("[ranking.list.note]") + && err.msg.contains("'dua'"), + "{}", + err.msg + ); + // A table for an undeclared type. + let err = refused(&format!( + "{NEXT_BASE}\n[ranking.list]\ntypes = [\"note\"]\n[ranking.list.person]\norder = []" + )); + assert!(err.msg.contains("[ranking.list.person]"), "{}", err.msg); + // A per-type table without a cross-type order. + let err = refused(&format!( + "{NEXT_BASE}\n[ranking.list.note]\norder = [ {{ field = \"due\" }} ]" + )); + assert!( + err.msg.contains("cross-type order") && err.msg.contains("[ranking.list]"), + "{}", + err.msg + ); + // `types` without a per-type table. + let err = refused(&format!("{NEXT_BASE}\n[ranking.list]\ntypes = [\"note\"]")); + assert!( + err.msg.contains("no [ranking.list.] table"), + "{}", + err.msg + ); + // A precedence key with dir in the list default order. + let err = refused(&format!( + "{NEXT_BASE}\n[ranking.list]\norder = [ {{ field = \"status\", precedence = [\"open\"], dir = \"desc\" }} ]" + )); + assert!( + err.msg.contains("invalid [ranking.list]") && err.msg.contains("dir/nulls"), + "{}", + err.msg + ); + // A key on `type` inside a per-type list table. + let err = refused(&format!( + "{NEXT_BASE}\n[ranking.list]\ntypes = [\"note\"]\n[ranking.list.note]\norder = [ {{ field = \"type\" }} ]" + )); + assert!(err.msg.contains("'type'"), "{}", err.msg); + } + + #[test] + fn a_misspelt_key_under_ranking_list_is_a_parse_error_naming_its_section() { + let err = refused(&format!( + "{NEXT_BASE}\n[ranking.list]\nordr = [ {{ field = \"due\" }} ]" + )); + assert_eq!(err.kind, crate::error::ErrorKind::Io, "{}", err.msg); + assert!( + err.msg.contains("ordr") && err.msg.contains("[ranking.list] key"), + "{}", + err.msg + ); + let err = refused(&format!( + "{NEXT_BASE}\n[ranking.list]\ntypes = [\"note\"]\n[ranking.list.note]\nordr = []" + )); + assert_eq!(err.kind, crate::error::ErrorKind::Io, "{}", err.msg); + assert!( + err.msg.contains("[ranking.list.note]") && err.msg.contains("ordr"), + "{}", + err.msg + ); + } + + // ---- derived ranking keys (6j6v.x0wf, docs/specs/derived-ranking-keys.md §7) ------------- + + /// A `[ranking.next]` whose `action` order is the single key `key`, with `project` ranked by + /// priority — the fixture the derived-key load tests vary. + fn with_action_key(key: &str) -> String { + format!( + r#" + [ranking.next] + types = ["action", "project"] + order = [ {{ field = "priority" }} ] + [ranking.next.action] + order = [ {key} ] + [ranking.next.project] + order = [ {{ field = "priority" }} ] + "# + ) + } + + /// The validation message for a plugin whose `action` order is `key`. + fn refused_key(key: &str) -> String { + let err = refused(&with_action_key(key)); + assert_eq!( + err.kind, + crate::error::ErrorKind::Validation, + "a load-time refusal, not a parse error: {}", + err.msg + ); + err.msg + } + + /// The parse error for a plugin whose `action` order is `key`. + fn unparsable_key(key: &str) -> String { + let err = refused(&with_action_key(key)); + assert_eq!(err.kind, crate::error::ErrorKind::Io, "{}", err.msg); + err.msg + } + + #[test] + fn each_derived_key_kind_parses_in_a_per_type_order() { + let cfg = load_with_ranking(&with_action_key( + r#" + { related = "parent", type = "project", of = "rank", pick = "max", nulls = "first" }, + { related = "contributes_to", of = { field = "priority", precedence = ["P1"] } }, + { count = "children", type = "note", status = ["open", "in_progress"], label = "x", dir = "desc" }, + { sum = [ { count = "mentioned_by", type = "project" }, { count = "dependents" } ], dir = "desc" }, + "#, + )) + .unwrap(); + let order = cfg.ranking.next.order_for(Some("action")); + assert_eq!(order.len(), 4); + assert!(order.iter().all(RankKey::is_derived)); + assert_eq!(order[0].related.as_deref(), Some("parent")); + assert_eq!(order[0].of, Some(RankOf::Rank)); + assert_eq!(order[0].pick, Some(Pick::Max)); + assert_eq!(order[0].nulls, Nulls::First); + match &order[1].of { + Some(RankOf::Field(of)) => assert_eq!(of.precedence, ["P1"]), + other => panic!("of = {{ field }} parses as a field: {other:?}"), + } + assert_eq!(order[2].count.as_deref(), Some("children")); + assert_eq!(order[2].label.as_deref(), Some("x")); + assert_eq!(order[3].sum.as_ref().map(Vec::len), Some(2)); + assert!(cfg.ranking.next.has_derived()); + assert!( + !cfg.ranking.list.has_derived(), + "list inherits the default order only" + ); + } + + #[test] + fn the_specs_declared_orders_load() { + // docs/specs/derived-ranking-keys.md §8, with `types` under [ranking.list] (the per-type + // rule of n698) and the goal's `order` declared as a custom field (§10.2). The `sum`'s + // `dir` stands on the line its array closes on: TOML 1.0 allows no newline inside an + // inline table outside its arrays. + load_with_ranking( + r#" + [ranking.next] + types = ["project", "action"] + order = [ { field = "priority" } ] + [ranking.next.project] + order = [ + { related = "contributes_to", type = "note", of = { field = "order", dir = "asc" }, pick = "min", nulls = "last" }, + { field = "priority", dir = "asc" }, + { count = "children", type = "action", status = ["open", "in_progress"], dir = "desc" }, + ] + [ranking.next.action] + order = [ + { related = "parent", type = "project", of = "rank", nulls = "last" }, + { field = "priority", dir = "asc" }, + ] + [ranking.list] + types = ["note"] + [ranking.list.note] + order = [ + { sum = [ { count = "mentioned_by", type = "project" }, + { count = "mentioned_by", type = "action", label = "waiting", status = ["open", "in_progress"] } ], dir = "desc" }, + { related = "contributes_to", type = "project", of = "rank", pick = "min", nulls = "last" }, + ] + [fields.order] + type = "text" + on = ["note"] + "#, + ) + .expect("the spec's orders are valid"); + } + + #[test] + fn a_key_that_mixes_kinds_or_names_none_is_refused() { + let msg = refused_key(r#"{ related = "parent", count = "children", of = "rank" }"#); + assert!(msg.contains("mixes kinds (related, count)"), "{msg}"); + let msg = refused_key(r#"{ field = "due", count = "children" }"#); + assert!(msg.contains("mixes kinds (field, count)"), "{msg}"); + let msg = refused_key(r#"{ dir = "desc" }"#); + assert!(msg.contains("names no kind"), "{msg}"); + assert!( + msg.contains("[ranking.next.action]"), + "names its table: {msg}" + ); + } + + #[test] + fn an_unknown_edge_kind_or_direction_is_refused() { + let msg = refused_key(r#"{ related = "children", of = "rank" }"#); + assert!( + msg.contains("`related = \"children\"`") && msg.contains("valid: parent"), + "related follows outbound kinds only: {msg}" + ); + let msg = refused_key(r#"{ count = "siblings" }"#); + assert!( + msg.contains("'siblings'") && msg.contains("mentioned_by"), + "{msg}" + ); + let msg = refused_key(r#"{ sum = [ { count = "kids" } ] }"#); + assert!(msg.contains("'kids'"), "{msg}"); + } + + #[test] + fn a_part_the_kind_does_not_take_is_refused() { + for (key, part) in [ + (r#"{ count = "children", pick = "max" }"#, "pick"), + (r#"{ count = "children", of = "rank" }"#, "of"), + ( + r#"{ related = "parent", of = "rank", status = ["open"] }"#, + "status", + ), + ( + r#"{ related = "parent", of = "rank", label = "x" }"#, + "label", + ), + ( + r#"{ sum = [ { count = "children" } ], type = "note" }"#, + "type", + ), + (r#"{ field = "due", type = "note" }"#, "type"), + ] { + let msg = refused_key(key); + assert!(msg.contains(&format!("`{part}`")), "{key}: {msg}"); + } + let msg = refused_key(r#"{ count = "children", nulls = "first" }"#); + assert!(msg.contains("never null"), "{msg}"); + let msg = refused_key(r#"{ sum = [ { count = "children" } ], nulls = "first" }"#); + assert!(msg.contains("never null"), "{msg}"); + let msg = refused_key(r#"{ count = "children", precedence = ["a"] }"#); + assert!(msg.contains("precedence"), "{msg}"); + } + + #[test] + fn a_related_key_must_say_what_it_reads_and_keeps_dir_out_of_of() { + let msg = refused_key(r#"{ related = "parent" }"#); + assert!(msg.contains("add `of = \"rank\"`"), "{msg}"); + let msg = + refused_key(r#"{ related = "parent", of = { field = "priority", dir = "desc" } }"#); + assert!(msg.contains("inside `of`"), "{msg}"); + let msg = + refused_key(r#"{ related = "parent", of = { field = "priority", nulls = "first" } }"#); + assert!(msg.contains("inside `of`"), "{msg}"); + let msg = unparsable_key(r#"{ related = "parent", of = "position" }"#); + assert!(msg.contains("\"rank\""), "{msg}"); + // Neither a string nor a table (review of PR #48, Test Quality #9). + let msg = unparsable_key(r#"{ related = "parent", of = 3 }"#); + assert!( + msg.contains("expected \"rank\" or `{ field = \"...\" }`"), + "{msg}" + ); + // `of = { ... }` takes only its own parts. + let msg = + unparsable_key(r#"{ related = "parent", of = { field = "priority", pick = "max" } }"#); + assert!(msg.contains("of:") && msg.contains("pick"), "{msg}"); + } + + #[test] + fn unknown_parts_are_parse_errors_and_weight_is_not_taken_yet() { + let msg = unparsable_key(r#"{ count = "children", stauts = ["open"] }"#); + assert!(msg.contains("stauts"), "{msg}"); + let msg = unparsable_key(r#"{ sum = [ { count = "children", weight = 2 } ] }"#); + assert!( + msg.contains("weight"), + "a summand takes no weight (§10.3): {msg}" + ); + let msg = unparsable_key(r#"{ sum = [ { count = "children", dir = "desc" } ] }"#); + assert!(msg.contains("dir"), "dir belongs to the sum (§3.3): {msg}"); + let msg = refused_key(r#"{ sum = [] }"#); + assert!(msg.contains("adds no counts"), "{msg}"); + } + + #[test] + fn filters_must_name_declared_types_and_statuses() { + let msg = refused_key(r#"{ count = "children", type = "epic" }"#); + assert!( + msg.contains("'epic'") && msg.contains("[types].list"), + "{msg}" + ); + let msg = refused_key(r#"{ sum = [ { count = "children", type = "epic" } ] }"#); + assert!(msg.contains("'epic'"), "{msg}"); + let msg = refused_key(r#"{ count = "children", status = ["done"] }"#); + assert!( + msg.contains("status 'done'") && msg.contains("valid: closed"), + "{msg}" + ); + let msg = refused_key(r#"{ sum = [ { count = "children", status = ["done"] } ] }"#); + assert!(msg.contains("status 'done'"), "{msg}"); + let msg = refused_key(r#"{ count = "children", status = [] }"#); + assert!(msg.contains("empty status list"), "{msg}"); + } + + #[test] + fn of_reads_a_rankable_or_a_declared_custom_field_only() { + let msg = refused_key(r#"{ related = "parent", of = { field = "order" } }"#); + assert!( + msg.contains("'order'") && msg.contains("custom field"), + "{msg}" + ); + // A declared custom field reads (spec §10.2); scoped to the neighbour's type it must apply. + let ok = format!( + "{}\n[fields.order]\ntype = \"text\"\non = [\"project\"]", + with_action_key( + r#"{ related = "parent", type = "project", of = { field = "order" } }"# + ) + ); + load_with_ranking(&ok).expect("a custom field on the neighbour's type reads"); + let off_type = format!( + "{}\n[fields.order]\ntype = \"text\"\non = [\"note\"]", + with_action_key( + r#"{ related = "parent", type = "project", of = { field = "order" } }"# + ) + ); + let msg = refused(&off_type).msg; + assert!(msg.contains("applies only to note"), "{msg}"); + } + + #[test] + fn of_rank_must_name_a_type_with_a_declared_order() { + let msg = refused_key(r#"{ related = "contributes_to", type = "note", of = "rank" }"#); + assert!( + msg.contains("rank of 'note'") && msg.contains("[ranking.next.note]"), + "{msg}" + ); + // An order under the OTHER section serves (spec §8: persons by the projects `next` ranks). + let cross = r#" + [ranking.next] + order = [ { field = "priority" } ] + [ranking.list] + types = ["note", "project"] + [ranking.list.note] + order = [ { related = "contributes_to", type = "action", of = "rank" } ] + [ranking.list.project] + order = [ { field = "priority" } ] + "#; + let msg = refused(cross).msg; + assert!(msg.contains("rank of 'action'"), "{msg}"); + let cross = format!("{cross}\n[ranking.next.action]\norder = [ {{ field = \"due\" }} ]") + .replace( + "[ranking.next]\n order", + "[ranking.next]\n types = [\"action\"]\n order", + ); + let cfg = load_with_ranking(&cross).expect("[ranking.next.action] serves a list key"); + assert_eq!( + cfg.ranking.rank_source(Verb::List, "action"), + Some(Verb::Next) + ); + assert_eq!( + cfg.ranking.rank_source(Verb::List, "project"), + Some(Verb::List), + "the asking section's own order first" + ); + } + + #[test] + fn a_cycle_in_the_type_chain_is_refused_and_named() { + // spec §4: project by action, action by project. + let err = refused( + r#" + [ranking.next] + types = ["action", "project"] + order = [ { field = "priority" } ] + [ranking.next.action] + order = [ { related = "parent", type = "project", of = "rank" } ] + [ranking.next.project] + order = [ { related = "mentions", type = "action", of = "rank", pick = "min" } ] + "#, + ); + assert_eq!(err.kind, crate::error::ErrorKind::Validation); + assert!( + err.msg + .contains("[ranking.next.action] → [ranking.next.project] → [ranking.next.action]"), + "{}", + err.msg + ); + // Across the two sections. + let err = refused( + r#" + [ranking.next] + types = ["project"] + order = [ { field = "priority" } ] + [ranking.next.project] + order = [ { related = "mentions", type = "note", of = "rank" } ] + [ranking.list] + types = ["note"] + [ranking.list.note] + order = [ { related = "contributes_to", type = "project", of = "rank" } ] + "#, + ); + assert!( + err.msg + .contains("[ranking.list.note] → [ranking.next.project] → [ranking.list.note]") + || err.msg.contains( + "[ranking.next.project] → [ranking.list.note] → [ranking.next.project]" + ), + "{}", + err.msg + ); + } + + #[test] + fn an_untyped_rank_key_in_a_per_type_order_reads_its_own_rank_and_is_refused() { + // `{ related = "parent", of = "rank" }` without `type` reads the rank of every ranked type, + // its own included — a self-cycle. Naming the neighbour's type is the fix. + let msg = refused_key(r#"{ related = "parent", of = "rank" }"#); + assert!( + msg.contains("[ranking.next.action] → [ranking.next.action]") + && msg.contains("name the neighbour's `type`"), + "{msg}" + ); + load_with_ranking(&with_action_key( + r#"{ related = "parent", type = "project", of = "rank" }"#, + )) + .expect("typed, it is a chain"); + } + + #[test] + fn derived_keys_are_refused_in_the_next_default_order_but_not_the_list_one() { + let err = refused( + r#" + [ranking.next] + order = [ { count = "children", dir = "desc" } ] + "#, + ); + assert!( + err.msg.contains("[ranking.next] default `order`"), + "it also orders `blocked` and the lanes: {}", + err.msg + ); + let cfg = load_with_ranking( + r#" + [ranking.next] + order = [ { field = "priority" } ] + [ranking.list] + order = [ { count = "children", dir = "desc" } ] + "#, + ) + .expect("the list default order takes derived keys"); + assert!(cfg.ranking.list.has_derived()); + assert!(!cfg.ranking.next.has_derived()); + } + + #[test] + fn an_untyped_rank_key_in_the_list_default_order_is_refused() { + // Review of PR #48, Integrity #3: without per-type tables the key would read nothing, and + // with them it would pull every ranked type into the chain. Typed, it loads. + for tables in [ + "", + r#" + [ranking.next.project] + order = [ { field = "priority" } ] + "#, + ] { + let next_types = if tables.is_empty() { + "" + } else { + r#"types = ["project"]"# + }; + let err = refused(&format!( + r#" + [ranking.next] + {next_types} + order = [ {{ field = "priority" }} ] + {tables} + [ranking.list] + order = [ {{ related = "parent", of = "rank" }} ] + "# + )); + assert_eq!(err.kind, crate::error::ErrorKind::Validation); + assert!( + err.msg.contains("[ranking.list] default `order`") + && err.msg.contains("name the neighbour's `type`"), + "{}", + err.msg + ); + } + load_with_ranking( + r#" + [ranking.next] + types = ["project"] + order = [ { field = "priority" } ] + [ranking.next.project] + order = [ { field = "priority" } ] + [ranking.list] + order = [ { related = "parent", type = "project", of = "rank" } ] + "#, + ) + .expect("typed, it reads one order"); + } + + #[test] + fn a_misspelt_ranking_section_is_refused() { + // Review of PR #48, Integrity #2: `[ranking.lsit]` would otherwise be ignored in silence. + let err = refused( + r#" + [ranking.next] + order = [ { field = "priority" } ] + [ranking.lsit] + order = [ { field = "priority" } ] + "#, + ); + assert!(err.msg.contains("lsit"), "{}", err.msg); + } + + #[test] + fn the_bundled_plugins_declare_no_derived_keys() { + for name in ["issue-tracker", "personal-todo"] { + let cfg = load(name).unwrap(); + assert!(!cfg.ranking.next.has_derived() && !cfg.ranking.list.has_derived()); + } + } } diff --git a/crates/facade/src/read.rs b/crates/facade/src/read.rs index fab6ee3..47d775b 100644 --- a/crates/facade/src/read.rs +++ b/crates/facade/src/read.rs @@ -20,14 +20,16 @@ //! caller decides whether that is the system clock or an override (the CLI's `--now`). use crate::error::{NxfError, Result}; -use crate::plugin::{Dir, Nulls, PluginConfig, RankKey}; +use crate::plugin::{Dir, Nulls, PluginConfig, RankKey, RankSpec, Verb}; use crate::record::{item_value, items_value, raw_field}; use nexus_flow_core::derive; use nexus_flow_core::graph::Board; use nexus_flow_core::model::{ItemRow, LinkWeight, ThreadLink}; use nexus_flow_core::store::Store; use serde_json::{json, Value}; -use std::collections::{BTreeMap, HashMap, HashSet}; +use std::collections::{BTreeMap, BTreeSet, HashMap, HashSet}; + +mod derived; // ---- schema (plugin-aware field + type-system introspection, 8qv.5 / y5j8) -------------------- // @@ -536,6 +538,16 @@ fn blocked_ids(store: &Store) -> Result> { /// per-verb default ([`DEFAULT_SORT_LIST`], id, so `list --json` stays plugin-independent); the /// order is preserved into the `--json` projection ([`items_in_order_value`]), so `--sort rank` /// surfaces in the output too (C2 #916.2). +/// +/// `--sort rank` ranks by the plugin's `[ranking.list]` policy ([`Ranking::list`], 6j6v.n698): its +/// per-type orders and cross-type order where the plugin declares them, else the `[ranking.next]` +/// default order as before. +/// +/// [`Ranking::list`]: crate::plugin::Ranking::list +/// +/// A `[ranking.list]` with a derived key that reads the rank of a type only `next` ranks needs a +/// reference time for `next`'s candidates; this has none and answers `validation` there — take +/// [`list_at`]. pub fn list( cfg: &PluginConfig, store: &Store, @@ -543,18 +555,212 @@ pub fn list( item_type: Option<&str>, sort: Option, ) -> Result> { - let mut items: Vec = store - .list_items()? - .into_iter() - .filter(|i| i.deleted.as_deref() != Some("1")) - .filter(|i| i.archived.is_none()) - .filter(|i| status.is_none_or(|s| i.status.as_deref() == Some(s))) - .filter(|i| item_type.is_none_or(|t| i.item_type.as_deref() == Some(t))) - .collect(); - order_by(cfg, &mut items, sort.unwrap_or(DEFAULT_SORT_LIST)); + list_at(cfg, store, status, item_type, sort, None) +} + +/// [`list`] with the reference time `now` (6j6v.x0wf): the instant `next`'s candidates are taken +/// at when a derived key of `[ranking.list]` reads the rank of a type that `next` ranks +/// (docs/specs/derived-ranking-keys.md §4: a type's rank is computed over its candidates under the +/// verb that ranks it). Nothing else reads `now`; without such a key it changes nothing. +pub fn list_at( + cfg: &PluginConfig, + store: &Store, + status: Option<&str>, + item_type: Option<&str>, + sort: Option, + now: Option<&str>, +) -> Result> { + list_selected(cfg, store, status, sort, now, |rows| { + Ok(rows + .into_iter() + .filter(|i| item_type.is_none_or(|t| i.item_type.as_deref() == Some(t))) + .collect()) + }) +} + +/// The live items with the core status `status` that `select` keeps, ordered by `sort` — the body +/// of [`list_at`] and [`list_page`]. `select` narrows the rows BEFORE they are ranked (review of +/// PR #48, Code Quality #1), so the derived keys' type chain covers only the orders the kept rows +/// rank by: a type filter whose order reads no `next` rank needs no `now`, and a filtered query +/// does no work for the rows it drops. The rank SCOPE stays every live item of a type, whatever +/// `status` or `select` keep (spec §12, decision 2) — filters select, keys order. A row's derived +/// values depend on that scope only, never on the other ranked rows, so narrowing first orders +/// the kept rows exactly as narrowing after would. +fn list_selected( + cfg: &PluginConfig, + store: &Store, + status: Option<&str>, + sort: Option, + now: Option<&str>, + select: impl FnOnce(Vec) -> Result>, +) -> Result> { + if let Some(now) = now { + crate::validate::iso_date(now)?; + } + let live = derived::list_scope(store)?; + let key = sort.unwrap_or(DEFAULT_SORT_LIST); + let mut items: Vec = select( + live.iter() + .filter(|i| status.is_none_or(|s| i.status.as_deref() == Some(s))) + .cloned() + .collect(), + )?; + if key == SortKey::Rank { + // The derived keys of `[ranking.list]` (6j6v.x0wf), computed once for these rows; the + // unfiltered live items are `list`'s scope for "rank" — filters select, keys order. + let derived = derived::compute( + cfg, + store, + Verb::List, + &items, + derived::Scopes { + next: None, + list: Some(&live), + now, + }, + )?; + let rank = TypeRank::list(cfg).with_derived(derived.as_ref()); + items.sort_by(|a, b| rank.cmp(a, b)); + } else { + order_by(cfg, &mut items, key); + } Ok(items) } +// ---- list: filters and paging (6j6v.n698) ----------------------------------- + +/// One `list` request with its filters and paging (6j6v.n698) — the input of [`list_page`] and +/// [`Engine::list_query`]. The paging is `next`'s ([`NextQuery`]): the same three shapes, the same +/// machine-local snapshot cache, expiry, caps and invalidation, because it is the same mechanism. +/// +/// - No `limit`: the whole list. +/// - `limit` alone: the head of it, with the total disclosed and no token. +/// - `limit` + `paginate`: the head plus a `next_token` while rows remain; a stale or invalid token +/// answers the FIRST page of a fresh list with `restarted: true`. +/// +/// The filter is a [`NextFilter`] and means what it means for `next`: the types are alternatives, +/// the container keeps an item whose parent it is OR that contributes to it, the label keeps an +/// item carrying it. `status` is `list`'s own filter (the core status). +/// +/// `#[non_exhaustive]`: build it with [`ListQuery::new`] and the builder methods. +/// +/// [`Engine::list_query`]: crate::engine::Engine::list_query +#[derive(Debug, Clone, Default, PartialEq, Eq)] +#[non_exhaustive] +pub struct ListQuery { + /// Only items with this core status. `None`: any. + pub status: Option, + /// The order; `None` is the id default ([`DEFAULT_SORT_LIST`]). `rank` is the plugin's + /// `[ranking.list]` policy. + pub sort: Option, + /// The filters; empty keeps every item. + pub filter: NextFilter, + /// The page size; `None` is no cut. + pub limit: Option, + /// Hand out a next-token (requires a non-zero `limit`). + pub paginate: bool, + /// The token of a previous page (requires `paginate`). + pub token: Option, + /// The reference time (ISO-8601 / RFC3339) `next`'s candidates are taken at when a derived key + /// of `[ranking.list]` reads the rank of a type that `next` ranks (6j6v.x0wf, see + /// [`list_at`]). `None`: such a query answers `validation`; any other ignores it. + pub now: Option, +} + +impl ListQuery { + /// Every live, non-archived item, in id order, unpaged — what [`list`] returns unfiltered. + pub fn new() -> ListQuery { + ListQuery::default() + } + + /// Only items with the core status `status`. + pub fn status(mut self, status: impl Into) -> ListQuery { + self.status = Some(status.into()); + self + } + + /// Order by `sort` instead of id. + pub fn sort(mut self, sort: SortKey) -> ListQuery { + self.sort = Some(sort); + self + } + + /// Narrow by `filter`. + pub fn filter(mut self, filter: NextFilter) -> ListQuery { + self.filter = filter; + self + } + + /// Cut to `limit` rows (a page of that size under [`paginate`](Self::paginate)). + pub fn limit(mut self, limit: usize) -> ListQuery { + self.limit = Some(limit); + self + } + + /// Ask for a next-token. + pub fn paginate(mut self) -> ListQuery { + self.paginate = true; + self + } + + /// Continue from a previous page's token (implies [`paginate`](Self::paginate)). + pub fn token(mut self, token: impl Into) -> ListQuery { + self.paginate = true; + self.token = Some(token.into()); + self + } + + /// Take `next`'s candidates at `now` where a derived key of `[ranking.list]` reads the rank of + /// a type that `next` ranks (6j6v.x0wf). + pub fn now(mut self, now: impl Into) -> ListQuery { + self.now = Some(now.into()); + self + } + + /// The cache key a snapshot is bound to: the read, status, sort, filter and the plugin's + /// `[ranking.list]` policy — never `limit`. Naming the read keeps a `list` token from answering + /// a `next` or `mentioned_by` query; the ranking's fingerprint makes a token restart under a + /// changed plugin order, as `next`'s does. + fn key(&self, cfg: &PluginConfig) -> String { + json!({ + "read": "list", + "status": self.status, + "sort": self.sort.unwrap_or(DEFAULT_SORT_LIST).as_str(), + "filter": self.filter.key(), + "rank": policy_fingerprint(cfg, Verb::List), + }) + .to_string() + } +} + +/// One `list` page (6j6v.n698): [`list`] under `q.status` and `q.sort`, narrowed by `q.filter` +/// before it is ranked (the rank scope stays every live item, see the private `list_selected`), +/// cut and — under `paginate` — cached machine-locally like a `next` page, so +/// the following pages come from the same list. A token's snapshot restarts when an op after it +/// touches one of its items, or an item in it left or moved in the fresh list. `validation` for +/// `paginate` without a non-zero `limit`, or a token without `paginate`. +/// +/// A first page whose snapshot cannot be written is still answered, with `next_token: None`, as +/// for [`next_page`]. +pub fn list_page(cfg: &PluginConfig, store: &Store, q: &ListQuery) -> Result { + list_page_at(cfg, store, q, unix_now()) +} + +/// [`list_page`] on the clock `clock` (unix seconds). +fn list_page_at(cfg: &PluginConfig, store: &Store, q: &ListQuery, clock: i64) -> Result { + let paging = Paging::check(q.limit, q.paginate, q.token.as_deref())?; + let mark = Watermark::read(store)?; + let fresh = list_selected( + cfg, + store, + q.status.as_deref(), + q.sort, + q.now.as_deref(), + |rows| filter_next_store(store, rows, &q.filter), + )?; + page_through(store, fresh, || q.key(cfg), &paging, mark, clock) +} + /// The canonical `--json` array for a derived/queried item set: records ordered by id. pub fn items_to_value(items: &[ItemRow]) -> Value { items_value(items) @@ -828,22 +1034,38 @@ pub fn next( let candidates = lanes(store, now)?.candidates; let ids: Vec = candidates.iter().map(|c| c.id.clone()).collect(); let items = resolve_items(store, &ids)?; - Ok(next_lane( - cfg, - candidates, - items, - sort.unwrap_or(DEFAULT_SORT_NEXT), - )) + let sort = sort.unwrap_or(DEFAULT_SORT_NEXT); + // Derived ranking keys (6j6v.x0wf) rank within the tiers, so only the rank order computes + // them — once for the whole candidate set; the candidates ARE `next`'s scope for "rank". + let derived = match sort { + SortKey::Rank => derived::compute( + cfg, + store, + Verb::Next, + &items, + derived::Scopes { + next: Some(&items), + list: None, + now: Some(now), + }, + )?, + _ => None, + }; + Ok(next_lane(cfg, candidates, items, sort, derived.as_ref())) } /// The `next` order over the library's candidates and their rows: the finish-first tiers under /// `rank`, the flat order under any other key. THE one mechanism behind [`next`] (rows from a /// store) and [`next_active`] (rows a server read), so the two cannot rank apart (6j6v.t1ym). +/// +/// `derived` holds the values of the policy's derived keys for these rows (6j6v.x0wf), `None` +/// when it declares none. fn next_lane( cfg: &PluginConfig, candidates: Vec, mut items: Vec, sort: SortKey, + derived: Option<&derived::Derived>, ) -> Vec { match sort { // The default: finishing beats starting. @@ -857,7 +1079,7 @@ fn next_lane( promoter: c.promoter, }) .collect(); - order_next_tiered(cfg, &candidates, &mut items) + order_next_tiered(cfg, &candidates, &mut items, derived) } // An explicit `--sort` is the escape hatch — the flat, untiered order (§4.2). key => order_by(cfg, &mut items, key), @@ -865,7 +1087,8 @@ fn next_lane( items } -/// One page of a `next` result: the rows to show, plus how many there were before the cut. +/// One page of a `next` result — or of any read paged like it (`mentioned_by`, 6j6v.q29p; `list`, +/// 6j6v.n698): the rows to show, plus how many there were before the cut. /// /// The type exists so the total cannot be dropped on the way to a renderer. A limit that cuts /// silently is worse than no limit at all — a list that shows 15 of 180 and looks like 15 of 15 @@ -1158,25 +1381,42 @@ impl NextQuery { json!({ "sort": format!("{:?}", self.sort.unwrap_or(DEFAULT_SORT_NEXT)), "filter": self.filter.key(), - "rank": rank_fingerprint(cfg), + "rank": policy_fingerprint(cfg, Verb::Next), }) .to_string() } } -/// A fixed-size fingerprint of the plugin's `next` ranking: FNV-1a (64 bit) over its canonical JSON +/// A fixed-size fingerprint of a ranking policy (`[ranking.next]`, or `[ranking.list]` for a `list` +/// snapshot, 6j6v.n698): FNV-1a (64 bit) over its canonical JSON /// (`serde_json` over the `Serialize` derive; the per-type orders are a `BTreeMap`, so the bytes /// are deterministic). FNV is spelled out here because it is stable across Rust releases, which /// `DefaultHasher` does not promise. A collision would only let a token survive a ranking change /// whose new order the snapshot check in `resume` still compares against. -fn rank_fingerprint(cfg: &PluginConfig) -> String { - let canonical = serde_json::to_string(&cfg.ranking.next).unwrap_or_default(); +fn rank_fingerprint(spec: &RankSpec) -> String { + let canonical = serde_json::to_string(spec).unwrap_or_default(); let hash = canonical.bytes().fold(0xcbf2_9ce4_8422_2325_u64, |h, b| { (h ^ u64::from(b)).wrapping_mul(0x0100_0000_01b3) }); format!("{hash:016x}") } +/// The fingerprint a snapshot of `verb`'s rank is bound to: its policy's [`rank_fingerprint`], and +/// — when that policy holds a derived key, which may read the rank of a type the OTHER policy +/// orders (6j6v.x0wf) — the other policy's too, so a token restarts when either order changes. +/// A policy without derived keys keeps exactly the fingerprint it had. +fn policy_fingerprint(cfg: &PluginConfig, verb: Verb) -> String { + let own = rank_fingerprint(cfg.ranking.spec(verb)); + if !cfg.ranking.has_derived(verb) { + return own; + } + let other = match verb { + Verb::Next => &cfg.ranking.list, + Verb::List => &cfg.ranking.next, + }; + format!("{own}+{}", rank_fingerprint(other)) +} + /// Unix seconds now — the clock the snapshot expiry reads. Not `now`: that is the caller's /// reference time for the defer boundary, and a test pins it far from the wall clock. fn unix_now() -> i64 { @@ -1273,24 +1513,76 @@ pub fn next_page(cfg: &PluginConfig, store: &Store, q: &NextQuery) -> Result Result { crate::validate::iso_date(&q.now)?; - if q.token.is_some() && !q.paginate { - return Err(NxfError::validation("a page token needs paginate")); - } - let limit = match (q.paginate, q.limit) { - (false, _) => None, - (true, Some(n)) if n > 0 => Some(n), - (true, _) => return Err(NxfError::validation("paginate needs a limit of at least 1")), - }; + let paging = Paging::check(q.limit, q.paginate, q.token.as_deref())?; // The watermark BEFORE the list: an op that lands while the list is computed is then above it // and counts as "since" — a spurious restart at worst, never a missed one. - let watermark = store.ops_watermark()?; - let watermark_op = store.op_at(watermark)?; + let mark = Watermark::read(store)?; let fresh = next_filtered(cfg, store, &q.now, q.sort, &q.filter)?; - let Some(n) = limit else { - return Ok(truncate_next(fresh, q.limit)); + page_through(store, fresh, || q.key(cfg), &paging, mark, clock) +} + +/// The paging half of a request, checked: what [`page_through`] cuts and caches by. Shared by +/// every paged read (`next`, `mentioned_by`, `list`), so their rules cannot drift apart +/// (6j6v.q29p, 6j6v.n698). +struct Paging<'a> { + /// The cut: `None` is the whole list. + limit: Option, + /// `Some(n)` when the read hands out tokens, `n` its page size (never 0). + page: Option, + /// The token of a previous page. + token: Option<&'a str>, +} + +impl<'a> Paging<'a> { + /// `validation` for a `token` without `paginate`, and for `paginate` without a non-zero + /// `limit`. + fn check(limit: Option, paginate: bool, token: Option<&'a str>) -> Result> { + if token.is_some() && !paginate { + return Err(NxfError::validation("a page token needs paginate")); + } + let page = match (paginate, limit) { + (false, _) => None, + (true, Some(n)) if n > 0 => Some(n), + (true, _) => return Err(NxfError::validation("paginate needs a limit of at least 1")), + }; + Ok(Paging { limit, page, token }) + } +} + +/// The log's position and identity when a list was computed — what a snapshot is checked against. +struct Watermark { + rowid: i64, + op: Option, +} + +impl Watermark { + fn read(store: &Store) -> Result { + let rowid = store.ops_watermark()?; + let op = store.op_at(rowid)?; + Ok(Watermark { rowid, op }) + } +} + +/// Cut the freshly computed, ordered `fresh` list into one page per `paging` — and, when it pages, +/// serve a token's page from its cached snapshot (if [`resume`] says it still answers) or cache +/// `fresh` under `key` for the next one. THE paging mechanism behind [`next_page`], +/// [`mentioned_by_page`] and [`list_page`] (6j6v.15ed, generalised by 6j6v.q29p and n698): one +/// snapshot table, one expiry, +/// one pair of caps, one invalidation rule. `key` must name the read as well as its filters, so a +/// token of one read restarts under another; it is built only when the read pages. +fn page_through( + store: &Store, + fresh: Vec, + key: impl FnOnce() -> String, + paging: &Paging<'_>, + mark: Watermark, + clock: i64, +) -> Result { + let Some(n) = paging.page else { + return Ok(truncate_next(fresh, paging.limit)); }; - let key = q.key(cfg); - if let Some(token) = &q.token { + let key = key(); + if let Some(token) = paging.token { if let Some((snap, pos)) = resume(store, token, &key, &fresh, clock)? { // `pos < len` (resume checked it); a limit up to `usize::MAX` must not overflow. let end = pos.saturating_add(n).min(snap.ids.len()); @@ -1309,15 +1601,15 @@ fn next_page_at(cfg: &PluginConfig, store: &Store, q: &NextQuery, clock: i64) -> }); } } - let restarted = q.token.is_some(); + let restarted = paging.token.is_some(); let total = fresh.len(); let next_token = if total > n { let snap = nexus_flow_core::next_cache::CachedResult { id: ulid::Ulid::new().to_string(), created: clock, query: key, - watermark, - watermark_op, + watermark: mark.rowid, + watermark_op: mark.op, ids: fresh.iter().map(|i| i.id.clone()).collect(), }; store @@ -1374,17 +1666,18 @@ const CLUSTER_CHILD: u8 = 1; /// Order the `ready ∪ in_progress` candidates into the finish-first tiers (§4.3), in place. /// -/// The sort key is `(tier, cluster, header-before-child)` with the plugin's [`NextRank`] as +/// The sort key is `(tier, cluster, header-before-child)` with the plugin's [`TypeRank`] as /// the tiebreak — which is what ranks Tier 1 and Tier 3 internally, and orders a cluster's children. /// Clusters themselves are ordered by their **header's** rank, so a cluster is contiguous and a /// low-priority child never drags its epic up (nor a high-priority backlog item slice a cluster -/// apart). `NextRank` ends in an id tiebreak, so the whole composed order is total and -/// deterministic. Per-type orders (6j6v.z9jk) live inside `NextRank`: they order WITHIN a +/// apart). `TypeRank` ends in an id tiebreak, so the whole composed order is total and +/// deterministic. Per-type orders (6j6v.z9jk) live inside `TypeRank`: they order WITHIN a /// tier and never move an item between tiers. fn order_next_tiered( cfg: &PluginConfig, candidates: &[derive::NextCandidate], items: &mut [ItemRow], + derived: Option<&derived::Derived>, ) { // Split the one derivation into the three lookups the key needs. §3.3: the Tier-2 headers are // exactly `in_progress \ finishable` — the started items that still have a live non-closed @@ -1410,7 +1703,7 @@ fn order_next_tiered( "a promoted child is ready, never in_progress — the tiers would overlap" ); - let rank = NextRank::new(cfg); + let rank = TypeRank::next(cfg).with_derived(derived); let keys: HashMap = { let by_id: HashMap<&str, &ItemRow> = items.iter().map(|i| (i.id.as_str(), i)).collect(); let mut headers: Vec<&str> = started @@ -2066,41 +2359,79 @@ pub fn list_to_value_with_custom( /// (sp6.5): it is the first `ranking.next` key both bundled plugins declare, so ranking is entirely /// plugin-driven — no command-code special-case undercuts the "all policy flows from the config" seam. /// -/// This is the DEFAULT order only. Per-type orders (6j6v.z9jk) apply to `next` alone, through -/// [`NextRank`]; `list --sort rank` and `blocked` keep this one until `list` gets its own -/// per-type orders (6j6v.n698). +/// This is the `[ranking.next]` DEFAULT order only, which `blocked` and the lane verbs' `--sort +/// rank` keep. Per-type orders apply through [`TypeRank`]: `next` over `[ranking.next]` +/// (6j6v.z9jk), `list --sort rank` over `[ranking.list]` (6j6v.n698). fn rank_cmp(cfg: &PluginConfig, a: &ItemRow, b: &ItemRow) -> std::cmp::Ordering { keys_cmp(a, b, &cfg.ranking.next.order) } /// `keys` in declared order, then the id tiebreak — the total order every rank ends in. fn keys_cmp(a: &ItemRow, b: &ItemRow, keys: &[RankKey]) -> std::cmp::Ordering { - for key in keys { - let c = cmp_rank_key(a, b, key); + keys_cmp_values(a, b, keys, None, None).then_with(|| a.id.cmp(&b.id)) +} + +/// `keys` in declared order WITHOUT the id tiebreak: a field key over the rows, a derived key +/// (6j6v.x0wf) over the rows' derived values `va`/`vb`, aligned with `keys`. A row without values +/// reads null on every derived key. The dense rank of a type (spec §4) is this order's ties. +fn keys_cmp_values( + a: &ItemRow, + b: &ItemRow, + keys: &[RankKey], + va: Option<&[derived::DVal]>, + vb: Option<&[derived::DVal]>, +) -> std::cmp::Ordering { + const NULL: derived::DVal = derived::DVal::Null; + for (n, key) in keys.iter().enumerate() { + let c = if key.is_derived() { + let x = va.and_then(|v| v.get(n)).unwrap_or(&NULL); + let y = vb.and_then(|v| v.get(n)).unwrap_or(&NULL); + derived::cmp_dval(x, y, key.dir, key.nulls) + } else { + cmp_rank_key(a, b, key) + }; if c != std::cmp::Ordering::Equal { return c; } } - a.id.cmp(&b.id) + std::cmp::Ordering::Equal } -/// The rank `next` orders by within a tier (6j6v.z9jk), built once per sort. A plugin without a -/// per-type order ranks by [`rank_cmp`], exactly as before. Otherwise two items of different types -/// compare by their type's position in the cross-type order -/// ([`RankSpec::type_ordinal`](crate::plugin::RankSpec::type_ordinal)), then by type name; two items -/// of one type by that type's order ([`RankSpec::order_for`](crate::plugin::RankSpec::order_for)), -/// then id. A lexicographic key, so the order is total — the tiers and the paging snapshot rely on -/// that. Shared by the store path and the store-free path through [`order_next_tiered`]. -struct NextRank<'a> { - cfg: &'a PluginConfig, +/// The rank a ranking policy orders by, built once per sort: `next` within a tier over +/// `[ranking.next]` (6j6v.z9jk), and `list --sort rank` over `[ranking.list]` (6j6v.n698) — one +/// comparator for both, so the two verbs cannot rank alike-declared orders differently, and a +/// later key kind (the derived keys of `docs/specs/derived-ranking-keys.md`, 6j6v.x0wf) is added +/// here once for both. +/// +/// A policy without a per-type order ranks by its default order, then id — for `[ranking.next]` +/// exactly [`rank_cmp`]. Otherwise two items of different types compare by their type's position in +/// the cross-type order ([`RankSpec::type_ordinal`](crate::plugin::RankSpec::type_ordinal)), then by +/// type name; two items of one type by that type's order +/// ([`RankSpec::order_for`](crate::plugin::RankSpec::order_for)), then id. A lexicographic key, so +/// the order is total — the tiers and the paging snapshot rely on that. Shared by the store path and +/// the store-free path through [`order_next_tiered`], and by [`list`]. +struct TypeRank<'a> { + spec: &'a RankSpec, /// `type → position` in the cross-type order, looked up once per comparison instead of a scan - /// of the list (review of #44, Integrity #5). `None` when the plugin does not split by type. + /// of the list (review of #44, Integrity #5). `None` when the policy does not split by type. ordinals: Option<(HashMap<&'a str, usize>, usize)>, + /// The values of the policy's derived keys for the rows being sorted (6j6v.x0wf), computed + /// once per query by `derived::compute`. `None` when the policy declares none. + derived: Option<&'a derived::Derived>, } -impl<'a> NextRank<'a> { - fn new(cfg: &'a PluginConfig) -> NextRank<'a> { - let spec = &cfg.ranking.next; +impl<'a> TypeRank<'a> { + /// The rank `next` orders by: `[ranking.next]`. + fn next(cfg: &'a PluginConfig) -> TypeRank<'a> { + TypeRank::new(&cfg.ranking.next) + } + + /// The rank `list --sort rank` orders by: `[ranking.list]` (6j6v.n698). + fn list(cfg: &'a PluginConfig) -> TypeRank<'a> { + TypeRank::new(&cfg.ranking.list) + } + + fn new(spec: &'a RankSpec) -> TypeRank<'a> { let ordinals = spec.splits_by_type().then(|| { let cross = spec.cross_type_order(); let mut map = HashMap::new(); @@ -2110,12 +2441,28 @@ impl<'a> NextRank<'a> { } (map, cross.len()) }); - NextRank { cfg, ordinals } + TypeRank { + spec, + ordinals, + derived: None, + } + } + + /// Rank with the derived values `derived` (6j6v.x0wf). + fn with_derived(mut self, derived: Option<&'a derived::Derived>) -> TypeRank<'a> { + self.derived = derived; + self + } + + /// The policy's keys `keys` over `a` and `b`, then id — with the rows' derived values. + fn keys_cmp(&self, a: &ItemRow, b: &ItemRow, keys: &[RankKey]) -> std::cmp::Ordering { + let values = |row: &ItemRow| self.derived.and_then(|d| d.of(&row.id)); + keys_cmp_values(a, b, keys, values(a), values(b)).then_with(|| a.id.cmp(&b.id)) } fn cmp(&self, a: &ItemRow, b: &ItemRow) -> std::cmp::Ordering { let Some((ordinals, unlisted)) = &self.ordinals else { - return rank_cmp(self.cfg, a, b); + return self.keys_cmp(a, b, &self.spec.order); }; let (ta, tb) = (a.item_type.as_deref(), b.item_type.as_deref()); if ta != tb { @@ -2125,7 +2472,7 @@ impl<'a> NextRank<'a> { }; return ord(ta).cmp(&ord(tb)).then_with(|| ta.cmp(&tb)); } - keys_cmp(a, b, self.cfg.ranking.next.order_for(ta)) + self.keys_cmp(a, b, self.spec.order_for(ta)) } } @@ -2475,11 +2822,22 @@ pub fn active_board(store: &Store) -> Result { /// store holding the same board. A malformed `now` is `validation`-rejected, as there. /// /// [`Engine::next`]: crate::engine::Engine::next +/// +/// A `[ranking.next]` with derived ranking keys (6j6v.x0wf) is refused with `validation`: an +/// [`ActiveBoard`] holds neither the edges nor the scopes those keys read, and ranking without them +/// would order apart from the store path in silence. The server path is its own cut (6j6v.1c5s). pub fn next_active(cfg: &PluginConfig, board: &ActiveBoard, now: &str) -> Result> { crate::validate::iso_date(now)?; + if cfg.ranking.has_derived(Verb::Next) { + return Err(NxfError::validation(format!( + "plugin '{}' ranks `next` by derived ranking keys, which the store-free lanes do not \ + compute yet — answer `next` from a store", + cfg.name + ))); + } let candidates = board.board.lanes(now).candidates; let items = board.items(candidates.iter().map(|c| &c.id)); - Ok(next_lane(cfg, candidates, items, DEFAULT_SORT_NEXT)) + Ok(next_lane(cfg, candidates, items, DEFAULT_SORT_NEXT, None)) } /// [`next_active`], narrowed by `filter` (6j6v.15ed): the rows [`next_filtered`] gives over a store @@ -3051,6 +3409,140 @@ pub fn mentions_to_value(mentions: &[String]) -> Value { json!(mentions) } +// ---- mentioned by: the reverse of `mentions` (6j6v.q29p) -------------------- + +/// THE reverse mention lookup: for every id in `targets`, the live items with a present `mentions` +/// edge to it, id-sorted and distinct. A target nothing live mentions — or that is no item at all — +/// is absent from the map; this is a bulk read over ids the caller already holds, so a miss is not +/// `not_found`. +/// +/// "Live" is the one rule here: a deleted mentioner is dropped, every other one is kept — closed +/// and archived included, because a reference is not work. A caller narrows the rows itself: +/// [`mentioned_by_page`] by a [`NextFilter`], and the derived ranking key +/// `count = "mentioned_by"` (docs/specs/derived-ranking-keys.md §3.2, 6j6v.x0wf) by counting the +/// rows that pass its `type`/`status`/`label` parts per candidate. Spec §11 requires the two to +/// share one implementation; this is it. One edge query per 500 targets +/// ([`Store::mentioned_by_bulk`]) and one item read for the distinct mentioners, so a whole +/// candidate set costs O(E + N), as spec §5 budgets. +pub fn mentioners_of(store: &Store, targets: &[&str]) -> Result>> { + let edges = store.mentioned_by_bulk(targets)?; + let sources: BTreeSet<&str> = edges.values().flatten().map(String::as_str).collect(); + let sources: Vec<&str> = sources.into_iter().collect(); + let live: HashMap = store + .get_items(&sources)? + .into_iter() + .filter(|i| i.deleted.as_deref() != Some("1")) + .map(|i| (i.id.clone(), i)) + .collect(); + let mut out = BTreeMap::new(); + for (target, froms) in edges { + // `froms` is id-sorted and distinct (the store's ORDER BY over a DISTINCT view). + let rows: Vec = froms.iter().filter_map(|f| live.get(f).cloned()).collect(); + if !rows.is_empty() { + out.insert(target, rows); + } + } + Ok(out) +} + +/// One `mentioned_by` request (6j6v.q29p): which mentioners to keep, and how to page them — the +/// paging exactly as [`NextQuery`]'s (the same three shapes, the same snapshot cache, expiry, caps +/// and invalidation), because it is the same mechanism. +/// +/// - No `limit`: every mentioner. +/// - `limit` alone: the head, with the total disclosed and no token. +/// - `limit` + `paginate`: the head plus a `next_token` while rows remain; a stale or invalid +/// token answers the FIRST page of a fresh list with `restarted: true`. +/// +/// The filter is a [`NextFilter`] and means what it means for `next`: the types are alternatives, +/// the container keeps an item whose parent it is OR that contributes to it, the label keeps an +/// item carrying it. There is no sort: mentioners come in id order. +/// +/// `#[non_exhaustive]`: build it with [`MentionedByQuery::new`] and the builder methods. +#[derive(Debug, Clone, Default, PartialEq, Eq)] +#[non_exhaustive] +pub struct MentionedByQuery { + /// The filters; empty keeps every mentioner. + pub filter: NextFilter, + /// The page size; `None` is no cut. + pub limit: Option, + /// Hand out a next-token (requires a non-zero `limit`). + pub paginate: bool, + /// The token of a previous page (requires `paginate`). + pub token: Option, +} + +impl MentionedByQuery { + /// Every mentioner, unpaged. + pub fn new() -> MentionedByQuery { + MentionedByQuery::default() + } + + /// Narrow by `filter`. + pub fn filter(mut self, filter: NextFilter) -> MentionedByQuery { + self.filter = filter; + self + } + + /// Cut to `limit` rows (a page of that size under [`paginate`](Self::paginate)). + pub fn limit(mut self, limit: usize) -> MentionedByQuery { + self.limit = Some(limit); + self + } + + /// Ask for a next-token. + pub fn paginate(mut self) -> MentionedByQuery { + self.paginate = true; + self + } + + /// Continue from a previous page's token (implies [`paginate`](Self::paginate)). + pub fn token(mut self, token: impl Into) -> MentionedByQuery { + self.paginate = true; + self.token = Some(token.into()); + self + } + + /// The cache key a snapshot is bound to: the read, the target and the filter — never `limit`, + /// so a reader may change the page size between pages. Naming the read keeps a + /// `mentioned_by` token from answering a `next` query, and the other way round. + fn key(&self, target: &str) -> String { + json!({ + "read": "mentioned_by", + "target": target, + "filter": self.filter.key(), + }) + .to_string() + } +} + +/// One `mentioned_by` page (6j6v.q29p): the live items that mention `target` ([`mentioners_of`]), +/// narrowed by `q.filter`, in id order, cut and — under `paginate` — cached machine-locally like a +/// `next` page, so the following pages come from the same list. A token's snapshot restarts when +/// an op after it touches one of its items (a removed mention names the mentioner) or an item in it +/// left the fresh list. `not_found` if `target` is missing or deleted (as [`mentions`]); +/// `validation` for `paginate` without a non-zero `limit` or a token without `paginate`. +pub fn mentioned_by_page(store: &Store, target: &str, q: &MentionedByQuery) -> Result { + mentioned_by_page_at(store, target, q, unix_now()) +} + +/// [`mentioned_by_page`] on the clock `clock` (unix seconds). +fn mentioned_by_page_at( + store: &Store, + target: &str, + q: &MentionedByQuery, + clock: i64, +) -> Result { + let paging = Paging::check(q.limit, q.paginate, q.token.as_deref())?; + require_item(store, target)?; + let mark = Watermark::read(store)?; + let rows = mentioners_of(store, &[target])? + .remove(target) + .unwrap_or_default(); + let fresh = filter_next_store(store, rows, &q.filter)?; + page_through(store, fresh, || q.key(target), &paging, mark, clock) +} + // ---- thread links (nxf 6j6v.8dbe) ------------------------------------------ /// The chat threads presently linked to `id`, sorted by thread id. `not_found` if the item is @@ -4080,6 +4572,7 @@ mod tests { dir: Dir::Asc, nulls: Nulls::Last, precedence: Vec::new(), + ..RankKey::field_key("") }; assert_eq!( cmp_rank_key(&item("a", Some("2")), &item("b", Some("10")), &prio), @@ -4101,6 +4594,7 @@ mod tests { dir: Dir::Asc, nulls: Nulls::Last, precedence: Vec::new(), + ..RankKey::field_key("") }; assert_eq!( cmp_rank_key(&item("ab12.0002", None), &item("ab12.0010", None), &id), @@ -4126,6 +4620,7 @@ mod tests { .iter() .map(|s| s.to_string()) .collect(), + ..RankKey::field_key("") }; assert_eq!( cmp_rank_key(&typed("a", "epic"), &typed("b", "bug"), &key), @@ -4159,6 +4654,7 @@ mod tests { dir: Dir::Asc, nulls: Nulls::Last, precedence: vec!["in_progress".to_string(), "open".to_string()], + ..RankKey::field_key("") }; assert_eq!( cmp_rank_key(&status("a", "in_progress"), &status("b", "open"), &key), @@ -4181,6 +4677,7 @@ mod tests { dir: Dir::Asc, nulls: Nulls::Last, precedence: Vec::new(), + ..RankKey::field_key("") }; let (two, ten, text) = (titled("a", "2"), titled("b", "10"), titled("c", "1a")); use std::cmp::Ordering::{Greater, Less}; @@ -4242,7 +4739,7 @@ mod tests { typed("a3", Some("action"), "0", Some("2026-07-01")), typed("a1", Some("action"), "1", Some("2026-06-01")), ]; - let rank = NextRank::new(&cfg); + let rank = TypeRank::next(&cfg); items.sort_by(|a, b| rank.cmp(a, b)); let ids: Vec<&str> = items.iter().map(|i| i.id.as_str()).collect(); assert_eq!(ids, ["a1", "a2", "a3", "x1", "m1", "m2", "n1"]); @@ -4553,6 +5050,34 @@ mod tests { ); } + #[test] + fn a_mentioned_by_token_expires_on_the_same_clock() { + // 6j6v.q29p: `mentioned_by` pages through the same mechanism, expiry included. + let (mut s, _) = paged_board(); + for n in 0..4 { + s.add_edge( + &format!("c1.000{n}"), + "c1.0004", + nexus_flow_core::model::EdgeKind::Mentions, + "x", + ); + } + let q = MentionedByQuery::new().limit(2).paginate(); + let first = mentioned_by_page_at(&s, "c1.0004", &q, T0).unwrap(); + assert_eq!(first.total, 4); + let t = first.next_token.expect("rows remain"); + let at = |clock: i64| { + mentioned_by_page_at(&s, "c1.0004", &q.clone().token(&t), clock) + .unwrap() + .restarted + }; + assert!(!at(T0 + NEXT_CACHE_TTL_SECS), "at the TTL it still answers"); + assert!( + at(T0 + NEXT_CACHE_TTL_SECS + 1), + "past it the token restarts" + ); + } + #[test] fn a_new_snapshot_evicts_only_snapshots_older_than_the_ttl() { // The sign of the expiry bound: a younger snapshot survives the next insert; one older than diff --git a/crates/facade/src/read/derived.rs b/crates/facade/src/read/derived.rs new file mode 100644 index 0000000..86e3e3a --- /dev/null +++ b/crates/facade/src/read/derived.rs @@ -0,0 +1,547 @@ +//! Derived ranking keys on the store path (docs/specs/derived-ranking-keys.md, 6j6v.x0wf): the +//! values a ranking's `related` / `count` / `sum` keys compare by, computed once per query. +//! +//! **Bottom-up, once per query** (spec §4, §5). A query ranks a set of rows by the orders of their +//! types. The `of = "rank"` keys of those orders name other per-type orders (the type chain, kept +//! acyclic at load); those may name others in turn. [`compute`] walks the chain from the leaves: +//! +//! 1. one EDGE PASS ([`Store::present_edges_of_kinds`]) over every present edge of the kinds the +//! keys name, for the ranked rows and every row in the chain's scopes at once. A `parent` edge +//! is the `present_parent` projection — each child's one current parent, the one `belongs_to`, +//! `--in` and `show` read — so a child re-parented concurrently counts under one parent only. +//! The reverse mentions come from [`mentioners_of`], the one implementation `mention by` reads +//! too (spec §11); +//! 2. one bulk read of the neighbours' rows, and of their labels / custom fields when a key needs +//! them; +//! 3. per type in the chain, leaves first: its rows' values, ONE sort, and a dense rank. A +//! neighbour's rank is then a lookup, never a recursion; +//! 4. the ranked rows' values, which the comparator reads per pair. +//! +//! So a query costs O(E + N log N) in the relevant edges and the items in scope, and a plugin +//! that declares no derived key pays nothing: [`compute`] returns `None` before any read. + +use super::{lanes, mentioners_of, resolve_items}; +use crate::error::{NxfError, Result}; +use crate::plugin::{Dir, Nulls, Pick, PluginConfig, RankKey, RankOf, Verb}; +use crate::record::{raw_field, RANKABLE_FIELDS}; +use nexus_flow_core::model::{EdgeKind, ItemRow}; +use nexus_flow_core::store::Store; +use std::cmp::Ordering; +use std::collections::{BTreeMap, BTreeSet, HashMap}; + +/// One derived key's value for one item. Compares like a field value: numbers before text, +/// nulls by the key's `nulls`. +#[derive(Debug, Clone, PartialEq, Eq)] +pub(super) enum DVal { + /// No value: no neighbour (or none with a value, or none in scope). + Null, + /// A count, a rank, a precedence ordinal, or a field value that parses as an integer. + Num(i64), + /// Any other field value. + Text(String), +} + +impl DVal { + /// A raw field value as the comparator reads it — numeric when it parses as an integer, as + /// [`cmp_rank_key`](super::cmp_rank_key) does. + fn of_raw(v: Option) -> DVal { + match v { + None => DVal::Null, + Some(s) => match s.parse::() { + Ok(n) => DVal::Num(n), + Err(_) => DVal::Text(s), + }, + } + } +} + +/// The ascending order of two values, numbers before text (review of #44, Integrity #2). A null +/// sorts after both; [`cmp_dval`] places nulls itself, so that arm only keeps the order total. +fn base_cmp(a: &DVal, b: &DVal) -> Ordering { + match (a, b) { + (DVal::Num(x), DVal::Num(y)) => x.cmp(y), + (DVal::Num(_), DVal::Text(_)) => Ordering::Less, + (DVal::Text(_), DVal::Num(_)) => Ordering::Greater, + (DVal::Text(x), DVal::Text(y)) => x.cmp(y), + (DVal::Null, DVal::Null) => Ordering::Equal, + (DVal::Null, _) => Ordering::Greater, + (_, DVal::Null) => Ordering::Less, + } +} + +/// Compare two values of one derived key with its `dir` and `nulls` — the field key's rules. +pub(super) fn cmp_dval(a: &DVal, b: &DVal, dir: Dir, nulls: Nulls) -> Ordering { + match (a, b) { + (DVal::Null, DVal::Null) => Ordering::Equal, + (DVal::Null, _) => match nulls { + Nulls::First => Ordering::Less, + Nulls::Last => Ordering::Greater, + }, + (_, DVal::Null) => match nulls { + Nulls::First => Ordering::Greater, + Nulls::Last => Ordering::Less, + }, + _ => match dir { + Dir::Asc => base_cmp(a, b), + Dir::Desc => base_cmp(a, b).reverse(), + }, + } +} + +/// The derived values of one query: for each ranked row, one value per key of the order it ranks +/// by, aligned with that order (a field key's position holds [`DVal::Null`] and is never read). +#[derive(Debug, Default)] +pub(super) struct Derived { + values: HashMap>, +} + +impl Derived { + /// The values of row `id`, aligned with its order. + pub(super) fn of(&self, id: &str) -> Option<&[DVal]> { + self.values.get(id).map(Vec::as_slice) + } +} + +/// What the caller already holds of the scopes "rank" is computed over (spec §4), so the pass +/// reads them only when the caller does not have them. +#[derive(Debug, Clone, Copy, Default)] +pub(super) struct Scopes<'a> { + /// `next`'s actionable candidates over the whole board, before any filter. + pub(super) next: Option<&'a [ItemRow]>, + /// The live items `list` returns without filters or paging (not deleted, not archived). + pub(super) list: Option<&'a [ItemRow]>, + /// The reference time, to compute `next`'s candidates when a `list` key reads the rank of a + /// type `next` ranks. + pub(super) now: Option<&'a str>, +} + +/// A per-type order in the type chain: the verb whose policy declares it, and the type. +type Node = (Verb, String); + +/// `(edge kind, endpoint) → the other endpoints`, one direction of the edge pass. +type Adjacency = HashMap<(EdgeKind, String), Vec>; + +/// The live items `list` returns without filters: not deleted, not archived. +pub(super) fn list_scope(store: &Store) -> Result> { + Ok(store + .list_items()? + .into_iter() + .filter(|i| i.deleted.as_deref() != Some("1")) + .filter(|i| i.archived.is_none()) + .collect()) +} + +/// The values the derived keys of `verb`'s policy rank `ranked` by — `None`, before any read, +/// when the policy has no derived key. `validation` when a `list` key reads the rank of a type +/// `next` ranks and the caller passed neither `next`'s candidates nor a reference time. +pub(super) fn compute( + cfg: &PluginConfig, + store: &Store, + verb: Verb, + ranked: &[ItemRow], + scopes: Scopes<'_>, +) -> Result> { + let ranking = &cfg.ranking; + let spec = ranking.spec(verb); + if !spec.has_derived() { + return Ok(None); + } + + // The chain, leaves first: the orders the ranked rows' orders read the rank of, transitively. + let ranked_types: BTreeSet> = + ranked.iter().map(|i| i.item_type.as_deref()).collect(); + let mut chain: Vec = Vec::new(); + let mut seen: BTreeSet = BTreeSet::new(); + for ty in &ranked_types { + for dep in ranking.rank_deps(spec.order_for(*ty), verb) { + post_order(cfg, dep, &mut seen, &mut chain); + } + } + + // The scopes the chain ranks over, read only when the caller does not hold them. + let next_owned; + let next_rows: &[ItemRow] = match scopes.next { + _ if !chain.iter().any(|(v, _)| *v == Verb::Next) => &[], + Some(rows) => rows, + None => { + let now = scopes.now.ok_or_else(|| { + NxfError::validation( + "the plugin's [ranking.list] reads the rank of a type that `next` ranks, \ + which is computed over `next`'s candidates at a reference time — pass `now`", + ) + })?; + let candidates = lanes(store, now)?.candidates; + let ids: Vec = candidates.into_iter().map(|c| c.id).collect(); + next_owned = resolve_items(store, &ids)?; + &next_owned + } + }; + let list_owned; + let list_rows: &[ItemRow] = match scopes.list { + _ if !chain.iter().any(|(v, _)| *v == Verb::List) => &[], + Some(rows) => rows, + None => { + list_owned = list_scope(store)?; + &list_owned + } + }; + let node_rows: Vec> = chain + .iter() + .map(|(v, ty)| { + let scope = match v { + Verb::Next => next_rows, + Verb::List => list_rows, + }; + scope + .iter() + .filter(|i| i.item_type.as_deref() == Some(ty.as_str())) + .collect() + }) + .collect(); + + // Every row a key is evaluated on, and every derived key in play. + let mut subjects: HashMap<&str, &ItemRow> = HashMap::new(); + for row in ranked.iter().chain(node_rows.iter().flatten().copied()) { + subjects.insert(row.id.as_str(), row); + } + let mut keys: Vec<&RankKey> = Vec::new(); + for ty in &ranked_types { + keys.extend(spec.order_for(*ty).iter().filter(|k| k.is_derived())); + } + for (v, ty) in &chain { + keys.extend( + ranking.spec(*v).by_type[ty] + .order + .iter() + .filter(|k| k.is_derived()), + ); + } + + let needs = Needs::of(&keys); + let mut ctx = Context { + cfg, + subjects, + out: Adjacency::new(), + inb: Adjacency::new(), + mentioners: BTreeMap::new(), + neighbours: HashMap::new(), + labels: BTreeMap::new(), + custom: BTreeMap::new(), + ranks: HashMap::new(), + }; + ctx.read_edges(store, &needs)?; + + // Bottom-up: each order of the chain ranks its scope once; its dependents look the rank up. + for (node, rows) in chain.iter().zip(&node_rows) { + let keys = &ranking.spec(node.0).by_type[&node.1].order; + let values: Vec> = rows.iter().map(|r| ctx.values(keys, node.0, r)).collect(); + let mut order: Vec = (0..rows.len()).collect(); + let cmp = |a: usize, b: usize| { + super::keys_cmp_values(rows[a], rows[b], keys, Some(&values[a]), Some(&values[b])) + }; + order.sort_by(|a, b| cmp(*a, *b).then_with(|| rows[*a].id.cmp(&rows[*b].id))); + // A DENSE rank over the declared keys (spec §4): rows equal on every key share a rank, and + // the next distinct row takes the next integer. The id orders the sort, never the rank. + let mut ranks: HashMap = HashMap::with_capacity(rows.len()); + let mut rank = 0_i64; + for (n, &i) in order.iter().enumerate() { + if n > 0 && cmp(order[n - 1], i) != Ordering::Equal { + rank += 1; + } + ranks.insert(rows[i].id.clone(), rank); + } + ctx.ranks.insert(node.clone(), ranks); + } + + let values = ranked + .iter() + .map(|r| { + let keys = spec.order_for(r.item_type.as_deref()); + (r.id.clone(), ctx.values(keys, verb, r)) + }) + .collect(); + Ok(Some(Derived { values })) +} + +/// Depth-first post-order over the type chain: every order a node reads the rank of lands before +/// it. The chain is acyclic (refused at load otherwise); `seen` is marked on entry, so even a +/// cycle could not recurse forever — its back edge would simply read no rank. +fn post_order(cfg: &PluginConfig, node: Node, seen: &mut BTreeSet, out: &mut Vec) { + if !seen.insert(node.clone()) { + return; + } + let keys = &cfg.ranking.spec(node.0).by_type[&node.1].order; + for dep in cfg.ranking.rank_deps(keys, node.0) { + post_order(cfg, dep, seen, out); + } + out.push(node); +} + +/// What the keys in play need read, gathered before the reads. +#[derive(Debug, Default)] +struct Needs { + /// The edge kinds of the one pass. + kinds: Vec, + /// A `count = "mentioned_by"` is in play: the reverse mentions through [`mentioners_of`]. + mentioned_by: bool, + /// A count filters on a label. + labels: bool, + /// An `of` reads a custom field. + custom: bool, +} + +impl Needs { + fn of(keys: &[&RankKey]) -> Needs { + let mut needs = Needs::default(); + let add = |needs: &mut Needs, kind: Option| { + if let Some(k) = kind { + if !needs.kinds.contains(&k) { + needs.kinds.push(k); + } + } + }; + for key in keys { + if let Some(edge) = &key.related { + add(&mut needs, EdgeKind::parse(edge)); + if let Some(RankOf::Field(of)) = &key.of { + needs.custom |= !RANKABLE_FIELDS.contains(&of.field.as_str()); + } + } + for c in counts_of(key) { + match c.kind { + "mentioned_by" => needs.mentioned_by = true, + kind => add(&mut needs, count_edge(kind).map(|(k, _)| k)), + } + needs.labels |= c.label.is_some(); + } + } + needs + } +} + +/// One count, from a `count` key or a `sum` summand. +struct Count<'a> { + kind: &'a str, + ty: Option<&'a str>, + status: Option<&'a [String]>, + label: Option<&'a str>, +} + +/// The counts of a key: its own for a `count`, its summands for a `sum`, none else. +fn counts_of(key: &RankKey) -> Vec> { + if let Some(kind) = &key.count { + return vec![Count { + kind, + ty: key.item_type.as_deref(), + status: key.status.as_deref(), + label: key.label.as_deref(), + }]; + } + key.sum + .iter() + .flatten() + .map(|p| Count { + kind: &p.count, + ty: p.item_type.as_deref(), + status: p.status.as_deref(), + label: p.label.as_deref(), + }) + .collect() +} + +/// A count kind as an edge kind and a direction (`true`: inbound). `mentioned_by` is not here: it +/// reads [`mentioners_of`]. +fn count_edge(kind: &str) -> Option<(EdgeKind, bool)> { + match kind { + "children" => Some((EdgeKind::Parent, true)), + "contributors" => Some((EdgeKind::ContributesTo, true)), + "dependents" => Some((EdgeKind::Dep, true)), + out => EdgeKind::parse(out).map(|k| (k, false)), + } +} + +/// Everything one query's derived values are computed from. +struct Context<'a> { + cfg: &'a PluginConfig, + /// The ranked rows and every row of the chain's scopes — live by construction. + subjects: HashMap<&'a str, &'a ItemRow>, + /// Outbound edges from a subject: `(kind, from) → [to]`. + out: Adjacency, + /// Inbound edges to a subject: `(kind, to) → [from]`. + inb: Adjacency, + /// The live items mentioning each subject ([`mentioners_of`]). + mentioners: BTreeMap>, + /// The live neighbours that are not subjects. + neighbours: HashMap, + /// The labels of the neighbours, when a count filters on one. + labels: BTreeMap>, + /// The custom fields of the neighbours, when an `of` reads one. + custom: BTreeMap>, + /// The dense rank of each order of the chain computed so far: `node → id → rank`. + ranks: HashMap>, +} + +impl Context<'_> { + /// The reads: the edge pass, the reverse mentions, the neighbours' rows, labels and custom + /// fields — each ONE bulk read (per 500 ids where the store chunks), whatever the row count. + fn read_edges(&mut self, store: &Store, needs: &Needs) -> Result<()> { + for (from, to, kind) in store.present_edges_of_kinds(&needs.kinds)? { + if self.subjects.contains_key(to.as_str()) { + self.inb + .entry((kind, to.clone())) + .or_default() + .push(from.clone()); + } + if self.subjects.contains_key(from.as_str()) { + self.out.entry((kind, from)).or_default().push(to); + } + } + if needs.mentioned_by { + let mut ids: Vec<&str> = self.subjects.keys().copied().collect(); + ids.sort_unstable(); + self.mentioners = mentioners_of(store, &ids)?; + } + let missing: BTreeSet<&str> = self + .out + .values() + .chain(self.inb.values()) + .flatten() + .map(String::as_str) + .filter(|id| !self.subjects.contains_key(id)) + .collect(); + let missing: Vec<&str> = missing.into_iter().collect(); + self.neighbours = store + .get_items(&missing)? + .into_iter() + .filter(|i| i.deleted.as_deref() != Some("1")) + .map(|i| (i.id.clone(), i)) + .collect(); + if needs.labels || needs.custom { + let mut ids: BTreeSet<&str> = self.neighbours.keys().map(String::as_str).collect(); + ids.extend(self.subjects.keys().copied()); + ids.extend(self.mentioners.values().flatten().map(|i| i.id.as_str())); + let ids: Vec<&str> = ids.into_iter().collect(); + if needs.labels { + self.labels = store.labels_of_bulk(&ids)?; + } + if needs.custom { + self.custom = store.custom_fields_of_bulk(&ids)?; + } + } + Ok(()) + } + + /// The live row `id`, a subject or a neighbour. + fn row(&self, id: &str) -> Option<&ItemRow> { + self.subjects + .get(id) + .copied() + .or_else(|| self.neighbours.get(id)) + } + + /// The values of `keys` (an order under `verb`) for `item`, aligned with the keys. + fn values(&self, keys: &[RankKey], verb: Verb, item: &ItemRow) -> Vec { + keys.iter() + .map(|key| { + if key.related.is_some() { + self.related(key, verb, item) + } else if key.is_derived() { + let n: usize = counts_of(key).iter().map(|c| self.count(c, item)).sum(); + DVal::Num(i64::try_from(n).unwrap_or(i64::MAX)) + } else { + DVal::Null + } + }) + .collect() + } + + /// A `related` key's value for `item`: `of` read on each live neighbour over the edge (of the + /// key's `type`, if it names one), then the smallest or largest non-null value by `pick`. + fn related(&self, key: &RankKey, verb: Verb, item: &ItemRow) -> DVal { + let Some(kind) = key.related.as_deref().and_then(EdgeKind::parse) else { + return DVal::Null; + }; + let Some(targets) = self.out.get(&(kind, item.id.clone())) else { + return DVal::Null; + }; + let values = targets + .iter() + .filter_map(|id| self.row(id)) + .filter(|n| key.item_type.is_none() || n.item_type == key.item_type) + .map(|n| self.read_of(key, verb, n)) + .filter(|v| *v != DVal::Null); + let pick = key.pick.unwrap_or_default(); + let best = match pick { + Pick::Min => values.min_by(base_cmp), + Pick::Max => values.max_by(base_cmp), + }; + best.unwrap_or(DVal::Null) + } + + /// What `of` reads on the neighbour `n`. + fn read_of(&self, key: &RankKey, verb: Verb, n: &ItemRow) -> DVal { + match &key.of { + Some(RankOf::Rank) => n + .item_type + .as_deref() + .and_then(|ty| { + let src = self.cfg.ranking.rank_source(verb, ty)?; + self.ranks.get(&(src, ty.to_string()))?.get(&n.id).copied() + }) + .map_or(DVal::Null, DVal::Num), + Some(RankOf::Field(of)) => { + let raw = if RANKABLE_FIELDS.contains(&of.field.as_str()) { + raw_field(n, &of.field) + } else { + self.custom + .get(&n.id) + .and_then(|c| c.get(&of.field)) + .cloned() + }; + if of.precedence.is_empty() { + DVal::of_raw(raw) + } else { + let pos = raw + .and_then(|v| of.precedence.iter().position(|p| *p == v)) + .unwrap_or(of.precedence.len()); + DVal::Num(i64::try_from(pos).unwrap_or(i64::MAX)) + } + } + // `of` is required on a `related` key (checked at load). + _ => DVal::Null, + } + } + + /// How many live neighbours of `item` over the count's kind pass its filters. + fn count(&self, c: &Count<'_>, item: &ItemRow) -> usize { + let keeps = |n: &ItemRow| { + c.ty.is_none_or(|t| n.item_type.as_deref() == Some(t)) + && c.status.is_none_or(|s| { + n.status + .as_deref() + .is_some_and(|x| s.iter().any(|y| y == x)) + }) + && c.label.is_none_or(|l| { + self.labels + .get(&n.id) + .is_some_and(|ls| ls.iter().any(|x| x == l)) + }) + }; + if c.kind == "mentioned_by" { + return self + .mentioners + .get(&item.id) + .map_or(0, |rows| rows.iter().filter(|n| keeps(n)).count()); + } + let Some((kind, inbound)) = count_edge(c.kind) else { + return 0; + }; + let side = if inbound { &self.inb } else { &self.out }; + side.get(&(kind, item.id.clone())).map_or(0, |ids| { + ids.iter() + .filter_map(|id| self.row(id)) + .filter(|n| keeps(n)) + .count() + }) + } +} diff --git a/crates/facade/src/record.rs b/crates/facade/src/record.rs index 5a4379c..066821e 100644 --- a/crates/facade/src/record.rs +++ b/crates/facade/src/record.rs @@ -77,7 +77,8 @@ pub fn items_value(items: &[ItemRow]) -> serde_json::Value { /// Every field name [`raw_field`] reads — so every field a ranking key can sort on. A plugin's /// ranking key naming anything else is refused at load (6j6v.z9jk): `raw_field` would return -/// `None` for every item and the key would never decide. Custom fields are not rankable yet (x0wf). +/// `None` for every item and the key would never decide. A custom field ranks only as a +/// neighbour's value, `related ... of = { field = ... }` (6j6v.x0wf), never as a field key. pub const RANKABLE_FIELDS: &[&str] = &[ "id", "type", diff --git a/crates/facade/tests/derived_ranking.rs b/crates/facade/tests/derived_ranking.rs new file mode 100644 index 0000000..ae831c4 --- /dev/null +++ b/crates/facade/tests/derived_ranking.rs @@ -0,0 +1,998 @@ +//! Derived ranking keys on the local SQL path (6j6v.x0wf, docs/specs/derived-ranking-keys.md): +//! `related`, `count` and `sum` keys in `[ranking.next.]` and `[ranking.list.]` +//! orders, the type chain ranked bottom-up once per query, and its cost. +//! +//! - The spec's own orders (§8) at the seam the products take — `Engine::next_query` and +//! `Engine::list_query` — over a registered fixture plugin: actions by their project's rank, +//! projects by their goals' declared order (a custom field, §10.2), persons by a `sum` of +//! mentions and then by the rank of the projects they contribute to (§9 "chains"). +//! - Each kind over hand-built boards through `read::list_at`, the function `list_query` answers +//! from (§9 "per kind"). +//! - Determinism (§9): another edge insertion order ranks alike, and ties fall to the id. +//! - Bounded cost (§9, a query-shape guard in the style of xn8s): the edge pass runs once per query +//! however many candidates there are. +//! +//! Its own test binary, so the fixture plugin does not pollute the facade lib tests' "exactly two +//! OSS plugins" assertion. + +use nexus_flow_core::model::{EdgeKind, ItemRow, MergeStrategy}; +use nexus_flow_core::store::{EdgeReadStats, ItemReadStats, Store}; +use nexus_flow_facade::engine::Engine; +use nexus_flow_facade::error::ErrorKind; +use nexus_flow_facade::plugin::{self, PluginConfig, PluginRegistration}; +use nexus_flow_facade::read::{self, ListQuery, NextFilter, NextQuery, SortKey}; +use nexus_flow_facade::workspace::{self, WorkspaceExt}; +use std::path::Path; +use tempfile::TempDir; + +const NOW: &str = "2026-06-17T00:00:00Z"; + +/// The spec's §8 orders, with `types` under `[ranking.list]` and the goals' declared sequence as +/// the custom field `order`. +const SPEC_ORDERS: &str = r#" +name = "derived-rank-fixture" +[description] +en = "x" +de = "y" +[priority] +labels = ["P0", "P1", "P2", "P3"] +[types] +list = ["goal", "project", "action", "person"] +[vocabulary.status] +open = "open" +in_progress = "in progress" +closed = "closed" +[ranking.next] +types = ["project", "action"] +order = [ + { field = "status", precedence = ["in_progress", "open"] }, + { field = "priority", dir = "asc" }, +] +[ranking.next.project] +order = [ + { related = "contributes_to", type = "goal", of = { field = "order", dir = "asc" }, pick = "min", nulls = "last" }, + { field = "priority", dir = "asc" }, + { count = "children", type = "action", status = ["open", "in_progress"], dir = "desc" }, +] +[ranking.next.action] +order = [ + { related = "parent", type = "project", of = "rank", nulls = "last" }, + { field = "priority", dir = "asc" }, +] +[ranking.list] +types = ["person"] +[ranking.list.person] +order = [ + { sum = [ { count = "mentioned_by", type = "project" }, + { count = "mentioned_by", type = "action", label = "waiting", status = ["open", "in_progress"] } ], dir = "desc" }, + { related = "contributes_to", type = "project", of = "rank", pick = "min", nulls = "last" }, +] +[presentation.list] +columns = ["id"] +[fields.order] +type = "text" +on = ["goal"] +"#; + +inventory::submit!(PluginRegistration { + name: "derived-rank-fixture", + toml: SPEC_ORDERS, + order: 130, +}); + +fn workspace(plugin: &str) -> TempDir { + let tmp = TempDir::new().unwrap(); + workspace::init_flow(tmp.path(), plugin).unwrap(); + tmp +} + +fn writer(dir: &Path) -> Store { + workspace::discover(dir).unwrap().open_store().unwrap() +} + +fn engine(tmp: &TempDir) -> Engine { + Engine::open(None, tmp.path()).unwrap() +} + +fn item(s: &mut Store, id: &str, ty: &str, priority: &str) { + s.create_item(id, ty, id, "t"); + s.set_field(id, "priority", Some(priority.into()), "t"); +} + +fn custom(s: &mut Store, id: &str, field: &str, value: &str) { + s.set_custom_field_merge(id, field, Some(value.into()), "t", MergeStrategy::Lww); +} + +fn edge(s: &mut Store, from: &str, to: &str, kind: EdgeKind) { + s.add_edge(from, to, kind, "t"); +} + +fn close(s: &mut Store, id: &str) { + s.set_field(id, "status", Some("closed".into()), "t"); +} + +fn ids(items: &[ItemRow]) -> Vec { + items.iter().map(|i| i.id.clone()).collect() +} + +/// The rows of `ty`, in the order the list holds them. +fn of_type(items: &[ItemRow], ty: &str) -> Vec { + items + .iter() + .filter(|i| i.item_type.as_deref() == Some(ty)) + .map(|i| i.id.clone()) + .collect() +} + +/// The spec's three cases on one board (every ticket open, so all of `next` is one tier): +/// +/// - goals: `g1` order 2, `g2` order 1, `g3` no order; +/// - projects, by their goals' smallest order, then priority, then open child actions (desc): +/// `p3` (g1+g2 → 1, P1), `p2` (g2 → 1, P3), `p5` (g1 → 2, P0, two open children), +/// `p1` (g1 → 2, P0, one open child), `p4` (no goal → null last). Ranks 0..4 in that order; +/// `pc` is closed, so it is not a `next` candidate and has no rank; +/// - actions, by their parent project's rank, then priority: `a3` (p3, P1), `a2` (p3, P3), +/// `a5`, `a6` (p5, P2 each — the id decides), `a1` (p1), `a4` (no parent → null last); `a7` is +/// p5's CLOSED child, so it neither counts nor ranks; +/// - persons, by the mentions of projects plus those of open actions labelled `waiting` +/// (desc), then the best rank among the projects they contribute to: +/// `u2` (p3 + waiting a2 = 2, rank 0), `u1` (p1 + p2 = 2, rank 4), `u3` (0: its mentioners are +/// the closed waiting `a7` and the deleted project `px`; rank 1), `u6` (0, rank 4 — p4), +/// `u4` (0, mentioned by the unlabelled `a3`; no project → null), `u5` (0, contributes to the +/// closed `pc` → null: outside `next`'s scope). `u4` before `u5` by id. +fn spec_board(dir: &Path) { + let mut s = writer(dir); + for (g, order) in [ + ("zz12.g1", Some("2")), + ("zz12.g2", Some("1")), + ("zz12.g3", None), + ] { + item(&mut s, g, "goal", "1"); + if let Some(o) = order { + custom(&mut s, g, "order", o); + } + } + for (p, prio) in [ + ("zz12.p1", "0"), + ("zz12.p2", "3"), + ("zz12.p3", "1"), + ("zz12.p4", "0"), + ("zz12.p5", "0"), + ("zz12.pc", "0"), + ("zz12.px", "0"), + ] { + item(&mut s, p, "project", prio); + } + close(&mut s, "zz12.pc"); + edge(&mut s, "zz12.p1", "zz12.g1", EdgeKind::ContributesTo); + edge(&mut s, "zz12.p2", "zz12.g2", EdgeKind::ContributesTo); + edge(&mut s, "zz12.p3", "zz12.g1", EdgeKind::ContributesTo); + edge(&mut s, "zz12.p3", "zz12.g2", EdgeKind::ContributesTo); + edge(&mut s, "zz12.p5", "zz12.g1", EdgeKind::ContributesTo); + edge(&mut s, "zz12.p4", "zz12.g3", EdgeKind::ContributesTo); + for (a, prio, parent) in [ + ("zz12.a1", "0", Some("zz12.p1")), + ("zz12.a2", "3", Some("zz12.p3")), + ("zz12.a3", "1", Some("zz12.p3")), + ("zz12.a4", "0", None), + ("zz12.a5", "2", Some("zz12.p5")), + ("zz12.a6", "2", Some("zz12.p5")), + ("zz12.a7", "0", Some("zz12.p5")), + ] { + item(&mut s, a, "action", prio); + if let Some(p) = parent { + s.set_parent(a, p, "t").unwrap(); + } + } + close(&mut s, "zz12.a7"); + for u in ["u1", "u2", "u3", "u4", "u5", "u6"] { + item(&mut s, &format!("zz12.{u}"), "person", "1"); + } + edge(&mut s, "zz12.p1", "zz12.u1", EdgeKind::Mentions); + edge(&mut s, "zz12.p2", "zz12.u1", EdgeKind::Mentions); + edge(&mut s, "zz12.u1", "zz12.p4", EdgeKind::ContributesTo); + edge(&mut s, "zz12.p3", "zz12.u2", EdgeKind::Mentions); + edge(&mut s, "zz12.a2", "zz12.u2", EdgeKind::Mentions); + s.add_label("zz12.a2", "waiting", "t"); + edge(&mut s, "zz12.u2", "zz12.p3", EdgeKind::ContributesTo); + edge(&mut s, "zz12.a7", "zz12.u3", EdgeKind::Mentions); + s.add_label("zz12.a7", "waiting", "t"); + edge(&mut s, "zz12.px", "zz12.u3", EdgeKind::Mentions); + s.delete_item("zz12.px", "t"); + edge(&mut s, "zz12.u3", "zz12.p2", EdgeKind::ContributesTo); + edge(&mut s, "zz12.a3", "zz12.u4", EdgeKind::Mentions); + edge(&mut s, "zz12.u5", "zz12.pc", EdgeKind::ContributesTo); + edge(&mut s, "zz12.u6", "zz12.p4", EdgeKind::ContributesTo); +} + +const PROJECTS: [&str; 5] = ["zz12.p3", "zz12.p2", "zz12.p5", "zz12.p1", "zz12.p4"]; +const ACTIONS: [&str; 6] = [ + "zz12.a3", "zz12.a2", "zz12.a5", "zz12.a6", "zz12.a1", "zz12.a4", +]; +const PERSONS: [&str; 6] = [ + "zz12.u2", "zz12.u1", "zz12.u3", "zz12.u6", "zz12.u4", "zz12.u5", +]; + +// ---- the spec's orders at the Engine seam (§9 "chains") ----------------------------------- + +#[test] +fn next_query_ranks_actions_by_project_rank_and_projects_by_goal_order() { + let tmp = workspace("derived-rank-fixture"); + spec_board(tmp.path()); + let page = engine(&tmp).next_query(&NextQuery::new(NOW)).unwrap(); + assert_eq!( + of_type(&page.items, "project"), + PROJECTS, + "goal order, priority, children" + ); + assert_eq!( + of_type(&page.items, "action"), + ACTIONS, + "the parent's rank, across three types" + ); +} + +#[test] +fn next_query_filters_after_ranking_and_pages_in_the_same_order() { + // The rank is computed over all of `next`'s candidates: a type filter removes rows, it does not + // change the project ranks the actions read. + let tmp = workspace("derived-rank-fixture"); + spec_board(tmp.path()); + let engine = engine(&tmp); + let q = || NextQuery::new(NOW).filter(NextFilter::new().with_type("action")); + assert_eq!(ids(&engine.next_query(&q()).unwrap().items), ACTIONS); + let p1 = engine.next_query(&q().limit(4).paginate()).unwrap(); + let token = p1.next_token.clone().expect("rows remain"); + let p2 = engine.next_query(&q().limit(4).token(token)).unwrap(); + assert!(!p2.restarted); + let walked: Vec = ids(&p1.items).into_iter().chain(ids(&p2.items)).collect(); + assert_eq!(walked, ACTIONS); +} + +#[test] +fn list_query_ranks_persons_by_a_sum_of_mentions_then_by_project_rank() { + let tmp = workspace("derived-rank-fixture"); + spec_board(tmp.path()); + let q = ListQuery::new() + .sort(SortKey::Rank) + .filter(NextFilter::new().with_type("person")) + .now(NOW); + let page = engine(&tmp).list_query(&q).unwrap(); + assert_eq!(ids(&page.items), PERSONS); +} + +#[test] +fn a_list_order_reading_a_next_rank_needs_now() { + // A person reads the rank of projects, which `next` ranks over its candidates at an instant. + let tmp = workspace("derived-rank-fixture"); + spec_board(tmp.path()); + let err = engine(&tmp) + .list_query(&ListQuery::new().sort(SortKey::Rank)) + .unwrap_err(); + assert_eq!(err.kind, ErrorKind::Validation); + assert!(err.msg.contains("pass `now`"), "{}", err.msg); + // The id order reads no rank, and needs no `now`. + engine(&tmp).list_query(&ListQuery::new()).unwrap(); +} + +#[test] +fn a_type_filter_whose_order_reads_no_next_rank_needs_no_now() { + // Review of PR #48, Code Quality #1: the filters narrow the rows before they are ranked, so + // the chain covers only the orders the kept rows rank by. Goals and projects rank by the + // `[ranking.list]` default order (field keys only) and need no `now`; persons read the rank + // of projects, which `next` ranks, and still do. + let tmp = workspace("derived-rank-fixture"); + spec_board(tmp.path()); + let engine = engine(&tmp); + let ranked = |ty: &str| { + ListQuery::new() + .sort(SortKey::Rank) + .filter(NextFilter::new().with_type(ty)) + }; + let goals = engine.list_query(&ranked("goal")).unwrap(); + assert_eq!(ids(&goals.items), ["zz12.g1", "zz12.g2", "zz12.g3"]); + let projects = engine.list_query(&ranked("project")).unwrap(); + assert_eq!( + projects.total, 6, + "every live project: all seven but the deleted px" + ); + let err = engine.list_query(&ranked("person")).unwrap_err(); + assert_eq!(err.kind, ErrorKind::Validation); + assert!(err.msg.contains("pass `now`"), "{}", err.msg); + // The rank scope stays every live item: a filter does not change the ranks persons read. + let persons = engine.list_query(&ranked("person").now(NOW)).unwrap(); + assert_eq!(ids(&persons.items), PERSONS); +} + +#[test] +fn a_malformed_now_is_refused() { + let tmp = workspace("derived-rank-fixture"); + spec_board(tmp.path()); + for q in [ListQuery::new(), ListQuery::new().sort(SortKey::Rank)] { + let err = engine(&tmp).list_query(&q.now("yesterday")).unwrap_err(); + assert_eq!(err.kind, ErrorKind::Validation, "{}", err.msg); + } +} + +#[test] +fn a_page_restarts_when_an_op_outside_the_snapshot_reorders_it() { + // Review of PR #48, Test Quality #4 — resume check 2. The snapshot holds actions only; the op + // sets a GOAL's declared order, so it targets no item of the snapshot (check 1 passes), but it + // reorders the projects and with them the actions. g2 → 9: p5 (g1, two open children), p1, + // p3 (now min 2), p2, p4 — so a5, a6, a1, a3, a2, a4. + let tmp = workspace("derived-rank-fixture"); + spec_board(tmp.path()); + let engine = engine(&tmp); + let q = || { + NextQuery::new(NOW) + .filter(NextFilter::new().with_type("action")) + .limit(2) + }; + let p1 = engine.next_query(&q().paginate()).unwrap(); + assert_eq!(ids(&p1.items), ACTIONS[..2]); + let token = p1.next_token.expect("rows remain"); + { + let mut s = writer(tmp.path()); + custom(&mut s, "zz12.g2", "order", "9"); + } + let p2 = engine.next_query(&q().token(token)).unwrap(); + assert!(p2.restarted, "an outside op moved the snapshot's items"); + assert_eq!(ids(&p2.items), ["zz12.a5", "zz12.a6"]); + assert_eq!(p2.start, 0); +} + +#[test] +fn the_rank_follows_the_board_on_the_next_query() { + // Moving a3 to the bottom-ranked project moves it; nothing is cached across queries. + let tmp = workspace("derived-rank-fixture"); + spec_board(tmp.path()); + let engine = engine(&tmp); + { + let mut s = writer(tmp.path()); + s.set_parent("zz12.a3", "zz12.p4", "t").unwrap(); + } + let page = engine.next_query(&NextQuery::new(NOW)).unwrap(); + assert_eq!( + of_type(&page.items, "action"), + ["zz12.a2", "zz12.a5", "zz12.a6", "zz12.a1", "zz12.a3", "zz12.a4"] + ); +} + +#[test] +fn the_store_free_lanes_refuse_a_plugin_with_derived_next_keys() { + // The server path is its own cut (1c5s); ranking without the edges would differ in silence. + let tmp = workspace("derived-rank-fixture"); + spec_board(tmp.path()); + let cfg = plugin::load("derived-rank-fixture").unwrap(); + let s = writer(tmp.path()); + let board = read::active_board(&s).unwrap(); + let err = read::next_active(&cfg, &board, NOW).unwrap_err(); + assert_eq!(err.kind, ErrorKind::Validation); + assert!(err.msg.contains("derived ranking keys"), "{}", err.msg); +} + +// ---- each kind over hand-built boards (§9 "per kind") ------------------------------------- + +/// A plugin whose `[ranking.list.item]` order is `order` — three types: `item` (ranked), `box` +/// and `note` (its neighbours). `box` ranks by priority under `[ranking.list.box]`, so a key may +/// read its rank. Parsed directly: the load-time rules are the plugin tests' business. +fn kinds_cfg(order: &str) -> PluginConfig { + toml::from_str(&format!( + r#" + name = "kinds-fixture" + [description] + en = "x" + de = "y" + [priority] + labels = ["P0", "P1", "P2", "P3"] + [types] + list = ["item", "box", "note"] + [vocabulary.status] + open = "open" + in_progress = "in progress" + closed = "closed" + [ranking.next] + order = [ {{ field = "priority" }} ] + [ranking.list] + types = ["item", "box"] + [ranking.list.item] + order = {order} + [ranking.list.box] + order = [ {{ field = "priority", dir = "asc" }} ] + [presentation.list] + columns = ["id"] + [fields.stage] + type = "text" + "# + )) + .unwrap() +} + +/// The items of `s`, ranked by `cfg`'s list order. +fn ranked_items(cfg: &PluginConfig, s: &Store) -> Vec { + let rows = read::list_at(cfg, s, None, None, Some(SortKey::Rank), None).unwrap(); + of_type(&rows, "item") +} + +/// Four items `i1`..`i4` (all P0) and boxes `b1` P2, `b2` P0, `b3` P1, plus note `n1` P3. +fn kinds_board() -> Store { + let mut s = Store::open_in_memory(1); + for i in ["zz12.i1", "zz12.i2", "zz12.i3", "zz12.i4"] { + item(&mut s, i, "item", "0"); + } + item(&mut s, "zz12.b1", "box", "2"); + item(&mut s, "zz12.b2", "box", "0"); + item(&mut s, "zz12.b3", "box", "1"); + item(&mut s, "zz12.n1", "note", "3"); + s +} + +#[test] +fn related_reads_one_neighbours_field_and_places_the_items_without_one() { + let mut s = kinds_board(); + s.set_parent("zz12.i1", "zz12.b1", "t").unwrap(); // P2 + s.set_parent("zz12.i2", "zz12.b2", "t").unwrap(); // P0 + s.set_parent("zz12.i3", "zz12.b3", "t").unwrap(); // P1 + let last = kinds_cfg(r#"[ { related = "parent", of = { field = "priority" } } ]"#); + assert_eq!( + ranked_items(&last, &s), + ["zz12.i2", "zz12.i3", "zz12.i1", "zz12.i4"], + "no parent → null, last by default" + ); + let first = kinds_cfg( + r#"[ { related = "parent", of = { field = "priority" }, dir = "desc", nulls = "first" } ]"#, + ); + assert_eq!( + ranked_items(&first, &s), + ["zz12.i4", "zz12.i1", "zz12.i3", "zz12.i2"] + ); +} + +#[test] +fn related_picks_the_min_or_max_of_several_neighbours() { + let mut s = kinds_board(); + // i1 → b1 (P2) + b2 (P0); i2 → b3 (P1); i3 → b1 (P2). + edge(&mut s, "zz12.i1", "zz12.b1", EdgeKind::ContributesTo); + edge(&mut s, "zz12.i1", "zz12.b2", EdgeKind::ContributesTo); + edge(&mut s, "zz12.i2", "zz12.b3", EdgeKind::ContributesTo); + edge(&mut s, "zz12.i3", "zz12.b1", EdgeKind::ContributesTo); + let min = kinds_cfg(r#"[ { related = "contributes_to", of = { field = "priority" } } ]"#); + assert_eq!( + ranked_items(&min, &s), + ["zz12.i1", "zz12.i2", "zz12.i3", "zz12.i4"], + "min: i1 reads 0" + ); + let max = kinds_cfg( + r#"[ { related = "contributes_to", of = { field = "priority" }, pick = "max" } ]"#, + ); + assert_eq!( + ranked_items(&max, &s), + ["zz12.i2", "zz12.i1", "zz12.i3", "zz12.i4"], + "max: i1 reads 2, ties i3 and falls to the id" + ); +} + +#[test] +fn related_honours_its_type_filter_a_precedence_and_a_custom_field() { + let mut s = kinds_board(); + edge(&mut s, "zz12.i1", "zz12.n1", EdgeKind::Mentions); // a note, P3 + edge(&mut s, "zz12.i2", "zz12.b1", EdgeKind::Mentions); // a box, P2 + let any = + kinds_cfg(r#"[ { related = "mentions", of = { field = "priority" }, dir = "desc" } ]"#); + assert_eq!(ranked_items(&any, &s)[0], "zz12.i1", "the note's P3 counts"); + let boxes = kinds_cfg( + r#"[ { related = "mentions", type = "box", of = { field = "priority" }, dir = "desc" } ]"#, + ); + assert_eq!( + ranked_items(&boxes, &s), + ["zz12.i2", "zz12.i1", "zz12.i3", "zz12.i4"], + "only box neighbours: i1 reads null" + ); + custom(&mut s, "zz12.n1", "stage", "late"); + custom(&mut s, "zz12.b1", "stage", "early"); + let stage = kinds_cfg( + r#"[ { related = "mentions", of = { field = "stage", precedence = ["early", "late"] } } ]"#, + ); + assert_eq!( + ranked_items(&stage, &s), + ["zz12.i2", "zz12.i1", "zz12.i3", "zz12.i4"], + "the custom field's precedence position: early before late; no neighbour → null" + ); +} + +#[test] +fn related_reads_a_neighbours_rank_in_its_own_order() { + let mut s = kinds_board(); + // Box ranks by priority: b2 (P0) 0, b3 (P1) 1, b1 (P2) 2. + s.set_parent("zz12.i1", "zz12.b1", "t").unwrap(); + s.set_parent("zz12.i2", "zz12.b3", "t").unwrap(); + s.set_parent("zz12.i3", "zz12.b2", "t").unwrap(); + let cfg = kinds_cfg(r#"[ { related = "parent", type = "box", of = "rank" } ]"#); + assert_eq!( + ranked_items(&cfg, &s), + ["zz12.i3", "zz12.i2", "zz12.i1", "zz12.i4"] + ); + // The rank is dense over the box order's keys: b2 and b3 tie on priority, so they share a + // rank and their items fall through to the next key (here: the id). + s.set_field("zz12.b3", "priority", Some("0".into()), "t"); + assert_eq!( + ranked_items(&cfg, &s), + ["zz12.i2", "zz12.i3", "zz12.i1", "zz12.i4"] + ); + // A deleted neighbour is no neighbour. + s.delete_item("zz12.b1", "t"); + assert_eq!(ranked_items(&cfg, &s)[2..], ["zz12.i1", "zz12.i4"]); +} + +#[test] +fn count_counts_children_by_type_status_and_label() { + let mut s = kinds_board(); + // i1: three children — two notes (one closed), one box. i2: one open note labelled `x`. + for (id, ty) in [ + ("zz12.c1", "note"), + ("zz12.c2", "note"), + ("zz12.c3", "box"), + ("zz12.c4", "note"), + ] { + item(&mut s, id, ty, "0"); + } + s.set_parent("zz12.c1", "zz12.i1", "t").unwrap(); + s.set_parent("zz12.c2", "zz12.i1", "t").unwrap(); + s.set_parent("zz12.c3", "zz12.i1", "t").unwrap(); + s.set_parent("zz12.c4", "zz12.i2", "t").unwrap(); + close(&mut s, "zz12.c2"); + s.add_label("zz12.c4", "x", "t"); + let order = |s: &Store, key: &str| ranked_items(&kinds_cfg(&format!("[ {key} ]")), s); + assert_eq!( + order(&s, r#"{ count = "children", dir = "desc" }"#), + ["zz12.i1", "zz12.i2", "zz12.i3", "zz12.i4"], + "3, 1, 0, 0" + ); + assert_eq!( + order(&s, r#"{ count = "children", dir = "asc" }"#), + ["zz12.i3", "zz12.i4", "zz12.i2", "zz12.i1"], + "a count is never null: zero sorts as zero" + ); + assert_eq!( + order(&s, r#"{ count = "children", type = "box", dir = "desc" }"#)[0], + "zz12.i1" + ); + // Open notes only: i1 1, i2 1 → the id. + assert_eq!( + order( + &s, + r#"{ count = "children", type = "note", status = ["open"], dir = "desc" }"# + ), + ["zz12.i1", "zz12.i2", "zz12.i3", "zz12.i4"] + ); + assert_eq!( + order(&s, r#"{ count = "children", label = "x", dir = "desc" }"#), + ["zz12.i2", "zz12.i1", "zz12.i3", "zz12.i4"] + ); + // A deleted child is no child. + s.delete_item("zz12.c1", "t"); + s.delete_item("zz12.c3", "t"); + assert_eq!( + order( + &s, + r#"{ count = "children", status = ["open"], dir = "desc" }"# + ), + ["zz12.i2", "zz12.i1", "zz12.i3", "zz12.i4"], + "i1 has only its closed child left" + ); +} + +#[test] +fn count_reads_every_direction_and_kind() { + let mut s = kinds_board(); + // Inbound: b1 → i3 contributes; b2, b3 → i3 dep (i3 has two dependents); n1 → i4 mentions. + edge(&mut s, "zz12.b1", "zz12.i3", EdgeKind::ContributesTo); + edge(&mut s, "zz12.b2", "zz12.i3", EdgeKind::Dep); + edge(&mut s, "zz12.b3", "zz12.i3", EdgeKind::Dep); + edge(&mut s, "zz12.n1", "zz12.i4", EdgeKind::Mentions); + // Outbound: i2 → b1, b2 dep; i2 → n1 mentions; i1 → b3 contributes. + edge(&mut s, "zz12.i2", "zz12.b1", EdgeKind::Dep); + edge(&mut s, "zz12.i2", "zz12.b2", EdgeKind::Dep); + edge(&mut s, "zz12.i2", "zz12.n1", EdgeKind::Mentions); + edge(&mut s, "zz12.i1", "zz12.b3", EdgeKind::ContributesTo); + let first = |s: &Store, kind: &str| { + ranked_items( + &kinds_cfg(&format!(r#"[ {{ count = "{kind}", dir = "desc" }} ]"#)), + s, + )[0] + .clone() + }; + assert_eq!(first(&s, "contributors"), "zz12.i3"); + assert_eq!(first(&s, "dependents"), "zz12.i3"); + assert_eq!(first(&s, "mentioned_by"), "zz12.i4"); + assert_eq!(first(&s, "dep"), "zz12.i2"); + assert_eq!(first(&s, "mentions"), "zz12.i2"); + assert_eq!(first(&s, "contributes_to"), "zz12.i1"); + assert_eq!(first(&s, "parent"), "zz12.i1", "all zero: the id decides"); + // One item with a parent: an outbound `parent` counts it (review of PR #48, Test Quality #7). + s.set_parent("zz12.i4", "zz12.b1", "t").unwrap(); + assert_eq!(first(&s, "parent"), "zz12.i4"); +} + +#[test] +fn count_mentioned_by_keeps_closed_and_archived_mentioners_and_drops_deleted_ones() { + // The reverse mention lookup is q29p's `mentioners_of` (spec §11): live is its one rule. + let mut s = kinds_board(); + for n in ["zz12.m1", "zz12.m2", "zz12.m3"] { + item(&mut s, n, "note", "0"); + } + edge(&mut s, "zz12.m1", "zz12.i2", EdgeKind::Mentions); + edge(&mut s, "zz12.m2", "zz12.i2", EdgeKind::Mentions); + edge(&mut s, "zz12.m3", "zz12.i1", EdgeKind::Mentions); + edge(&mut s, "zz12.n1", "zz12.i1", EdgeKind::Mentions); + close(&mut s, "zz12.m1"); + s.set_field("zz12.m2", "archived", Some(NOW.into()), "t"); + s.delete_item("zz12.m3", "t"); + let cfg = kinds_cfg(r#"[ { count = "mentioned_by", dir = "desc" } ]"#); + assert_eq!( + ranked_items(&cfg, &s), + ["zz12.i2", "zz12.i1", "zz12.i3", "zz12.i4"], + "i2: 2 (closed + archived), i1: 1 (the deleted one is gone)" + ); + // Exactly what `mentioners_of` reads for the same targets. + let m = read::mentioners_of(&s, &["zz12.i1", "zz12.i2"]).unwrap(); + assert_eq!(m["zz12.i2"].len(), 2); + assert_eq!(m["zz12.i1"].len(), 1); +} + +#[test] +fn sum_adds_its_counts_with_equal_weight() { + let mut s = kinds_board(); + // i1: 2 box mentions + 0 children; i2: 1 box mention + 2 children; i3: 0 + 1. + edge(&mut s, "zz12.b1", "zz12.i1", EdgeKind::Mentions); + edge(&mut s, "zz12.b2", "zz12.i1", EdgeKind::Mentions); + edge(&mut s, "zz12.b3", "zz12.i2", EdgeKind::Mentions); + for c in ["zz12.c1", "zz12.c2", "zz12.c3"] { + item(&mut s, c, "note", "0"); + } + s.set_parent("zz12.c1", "zz12.i2", "t").unwrap(); + s.set_parent("zz12.c2", "zz12.i2", "t").unwrap(); + s.set_parent("zz12.c3", "zz12.i3", "t").unwrap(); + let cfg = kinds_cfg( + r#"[ { sum = [ { count = "mentioned_by", type = "box" }, { count = "children" } ], dir = "desc" } ]"#, + ); + assert_eq!( + ranked_items(&cfg, &s), + ["zz12.i2", "zz12.i1", "zz12.i3", "zz12.i4"], + "3, 2, 1, 0 — not mentions first" + ); +} + +#[test] +fn a_concurrently_reparented_child_counts_under_its_projected_parent_only() { + // Review of PR #48, Integrity #1: two replicas re-point one child at once. The OR-set then + // holds both `parent` edges, but the derived keys read the `present_parent` projection — the + // one parent `belongs_to`, `--in` and `show` read (site 2 wins at equal lamport). + let mut a = kinds_board(); + let mut b = Store::open_in_memory(2); + b.apply(&a.export()); + // The note n1 under i2 (site 1) and under i3 (site 2); the item i1 under b1 P2 (site 1) and + // under b2 P0 (site 2); i2 under b3 P1 everywhere. + a.set_parent("zz12.n1", "zz12.i2", "alice").unwrap(); + b.set_parent("zz12.n1", "zz12.i3", "bob").unwrap(); + a.set_parent("zz12.i1", "zz12.b1", "alice").unwrap(); + b.set_parent("zz12.i1", "zz12.b2", "bob").unwrap(); + let (ea, eb) = (a.export(), b.export()); + a.apply(&eb); + b.apply(&ea); + a.set_parent("zz12.i2", "zz12.b3", "t").unwrap(); + assert_eq!(a.parents_of("zz12.n1").len(), 2, "precondition: two edges"); + assert_eq!( + a.get_item("zz12.n1") + .unwrap() + .unwrap() + .belongs_to + .as_deref(), + Some("zz12.i3") + ); + assert_eq!( + a.get_item("zz12.i1") + .unwrap() + .unwrap() + .belongs_to + .as_deref(), + Some("zz12.b2") + ); + let children = kinds_cfg(r#"[ { count = "children", dir = "desc" } ]"#); + assert_eq!( + ranked_items(&children, &a), + ["zz12.i3", "zz12.i1", "zz12.i2", "zz12.i4"], + "n1 counts under i3 alone, not under i2 as well" + ); + let max = kinds_cfg(r#"[ { related = "parent", of = { field = "priority" }, pick = "max" } ]"#); + assert_eq!( + ranked_items(&max, &a), + ["zz12.i1", "zz12.i2", "zz12.i3", "zz12.i4"], + "i1 reads b2's P0 — max over the one projected parent, not b1's P2 as well" + ); +} + +#[test] +fn a_list_rank_scope_ignores_the_status_filter() { + // spec §12, decision 2, pinned (review of PR #48, Test Quality #2): under `--status open` the + // closed box b2 is filtered out of the rows, but it is still in the scope its type ranks over, + // so its rank (0, P0) is still read by the item under it. + let mut s = kinds_board(); + close(&mut s, "zz12.b2"); + s.set_parent("zz12.i3", "zz12.b2", "t").unwrap(); // rank 0, closed + s.set_parent("zz12.i1", "zz12.b1", "t").unwrap(); // rank 2 + let cfg = kinds_cfg(r#"[ { related = "parent", type = "box", of = "rank" } ]"#); + let rows = read::list_at(&cfg, &s, Some("open"), None, Some(SortKey::Rank), None).unwrap(); + assert!(!ids(&rows).contains(&"zz12.b2".to_string()), "filtered out"); + assert_eq!( + of_type(&rows, "item"), + ["zz12.i3", "zz12.i1", "zz12.i2", "zz12.i4"] + ); +} + +#[test] +fn a_derived_key_ranks_in_the_list_default_order_without_per_type_tables() { + // Review of PR #48, Test Quality #5: the `[ranking.list]` default order takes derived keys + // (spec §12) and ranks every type by them when no per-type order is declared. + let cfg: PluginConfig = toml::from_str( + r#" + name = "list-default-fixture" + [description] + en = "x" + de = "y" + [priority] + labels = ["P0", "P1", "P2", "P3"] + [types] + list = ["item", "box", "note"] + [vocabulary.status] + open = "open" + in_progress = "in progress" + closed = "closed" + [ranking.next] + order = [ { field = "priority" } ] + [ranking.list] + order = [ { count = "children", dir = "desc" }, { field = "priority" } ] + [presentation.list] + columns = ["id"] + "#, + ) + .unwrap(); + assert!(cfg.ranking.list.has_derived() && !cfg.ranking.list.splits_by_type()); + let mut s = kinds_board(); + // b3 two children, i4 one; everyone else none, then priority (P0 items, b2 P0, b1 P2, n1 P3). + s.set_parent("zz12.i1", "zz12.b3", "t").unwrap(); + s.set_parent("zz12.i2", "zz12.b3", "t").unwrap(); + s.set_parent("zz12.n1", "zz12.i4", "t").unwrap(); + let rows = read::list_at(&cfg, &s, None, None, Some(SortKey::Rank), None).unwrap(); + assert_eq!( + ids(&rows), + ["zz12.b3", "zz12.i4", "zz12.b2", "zz12.i1", "zz12.i2", "zz12.i3", "zz12.b1", "zz12.n1"] + ); +} + +/// A plugin whose list ranks items by the rank of their parent box, and whose `next` ranks boxes +/// by `box_order` — the cross-section read whose fingerprint a list token carries. +fn cross_cfg(box_order: &str) -> PluginConfig { + toml::from_str(&format!( + r#" + name = "cross-fixture" + [description] + en = "x" + de = "y" + [priority] + labels = ["P0", "P1", "P2", "P3"] + [types] + list = ["item", "box", "note"] + [vocabulary.status] + open = "open" + in_progress = "in progress" + closed = "closed" + [ranking.next] + types = ["box"] + order = [ {{ field = "priority" }} ] + [ranking.next.box] + order = {box_order} + [ranking.list] + types = ["item"] + [ranking.list.item] + order = [ {{ related = "parent", type = "box", of = "rank" }} ] + [presentation.list] + columns = ["id"] + "# + )) + .unwrap() +} + +#[test] +fn a_derived_list_token_restarts_when_only_the_next_order_changes() { + // Review of PR #48, Test Quality #3: a list order that reads a `next` rank binds its token to + // `[ranking.next]` too. The changed `next` order adds a key that decides nothing here (every + // box is open), so the list order is the same and only the fingerprint can restart the token. + let mut s = kinds_board(); + s.set_parent("zz12.i1", "zz12.b1", "t").unwrap(); + s.set_parent("zz12.i2", "zz12.b2", "t").unwrap(); + s.set_parent("zz12.i3", "zz12.b3", "t").unwrap(); + let before = cross_cfg(r#"[ { field = "priority", dir = "asc" } ]"#); + let after = + cross_cfg(r#"[ { field = "priority", dir = "asc" }, { field = "status", dir = "asc" } ]"#); + let q = || { + ListQuery::new() + .sort(SortKey::Rank) + .filter(NextFilter::new().with_type("item")) + .now(NOW) + .limit(2) + }; + let p1 = read::list_page(&before, &s, &q().paginate()).unwrap(); + assert_eq!( + ids(&p1.items), + ["zz12.i2", "zz12.i3"], + "b2 P0, b3 P1, b1 P2" + ); + let token = p1.next_token.expect("rows remain"); + let same = read::list_page(&before, &s, &q().token(token.clone())).unwrap(); + assert!(!same.restarted, "the same plugin resumes"); + assert_eq!(ids(&same.items), ["zz12.i1", "zz12.i4"]); + let fresh = read::list_page(&after, &s, &q()).unwrap(); + assert_eq!(ids(&fresh.items), ids(&p1.items), "the order did not move"); + let changed = read::list_page(&after, &s, &q().token(token)).unwrap(); + assert!( + changed.restarted, + "a changed [ranking.next.box] restarts the list token" + ); +} + +#[test] +fn a_plugin_without_derived_keys_ranks_list_and_next_as_before() { + let cfg = plugin::load("issue-tracker").unwrap(); + let mut s = Store::open_in_memory(1); + s.create_item("zz12.e1", "epic", "e", "t"); + s.create_item("zz12.t1", "task", "t", "t"); + let before = s.edge_read_stats(); + read::list_at(&cfg, &s, None, None, Some(SortKey::Rank), None).unwrap(); + read::next(&cfg, &s, NOW, None).unwrap(); + assert_eq!( + s.edge_read_stats().since(before), + EdgeReadStats::default(), + "no derived key, no edge pass" + ); +} + +// ---- determinism (§9) ----------------------------------------------------------------- + +#[test] +fn another_edge_order_ranks_alike_and_ties_fall_to_the_id() { + let cfg = kinds_cfg( + r#"[ { count = "mentioned_by", dir = "desc" }, { related = "contributes_to", type = "box", of = "rank", pick = "min" } ]"#, + ); + let edges = [ + ("zz12.b1", "zz12.i3", EdgeKind::Mentions), + ("zz12.n1", "zz12.i3", EdgeKind::Mentions), + ("zz12.b2", "zz12.i2", EdgeKind::Mentions), + ("zz12.i2", "zz12.b1", EdgeKind::ContributesTo), + ("zz12.i1", "zz12.b3", EdgeKind::ContributesTo), + ("zz12.i1", "zz12.b1", EdgeKind::ContributesTo), + ("zz12.b3", "zz12.i1", EdgeKind::Mentions), + // i4 ties i2 on both keys (one mention, best box b1): a FULL tie. + ("zz12.n1", "zz12.i4", EdgeKind::Mentions), + ("zz12.i4", "zz12.b1", EdgeKind::ContributesTo), + ]; + let build = |order: &[usize]| { + let mut s = kinds_board(); + for &n in order { + let (from, to, kind) = edges[n]; + edge(&mut s, from, to, kind); + } + ranked_items(&cfg, &s) + }; + // The edge reads are ordered, so the insertion order cannot leak into the rank today; this + // pins that against a future read that is not (a hash map, an unordered scan). + let forward = build(&[0, 1, 2, 3, 4, 5, 6, 7, 8]); + assert_eq!(build(&[8, 7, 6, 5, 4, 3, 2, 1, 0]), forward); + assert_eq!(build(&[3, 7, 0, 6, 1, 8, 5, 2, 4]), forward); + // i3: 2 mentions; i1, i2 and i4: 1 each, then the best box rank (b3 rank 1 vs b1 rank 2): i1; + // i2 and i4 tie on every key and fall to the id, as every order does. + assert_eq!(forward, ["zz12.i3", "zz12.i1", "zz12.i2", "zz12.i4"]); + // The full tie, from the other side: renamed so that the id order is reversed, the tied pair + // swaps, and nothing else moves. + let mut s = kinds_board(); + item(&mut s, "zz12.i0", "item", "0"); + for &(from, to, kind) in &edges { + let swap = |id: &'static str| if id == "zz12.i4" { "zz12.i0" } else { id }; + edge(&mut s, swap(from), swap(to), kind); + } + s.delete_item("zz12.i4", "t"); + assert_eq!( + ranked_items(&cfg, &s), + ["zz12.i3", "zz12.i1", "zz12.i0", "zz12.i2"] + ); +} + +// ---- bounded cost (§9: a ratio guard in the style of xn8s) ---------------------------- + +/// A board of `n` projects under the spec's orders, each with two open actions, a goal, and a +/// person it mentions — enough graph for every key kind and the full three-type chain. +fn cost_board(n: usize) -> Store { + let mut s = Store::open_in_memory(1); + for g in 0..4 { + let id = format!("zz12.g{g}"); + item(&mut s, &id, "goal", "1"); + custom(&mut s, &id, "order", &g.to_string()); + } + for p in 0..n { + let pid = format!("zz12.p{p:04}"); + item(&mut s, &pid, "project", &(p % 4).to_string()); + edge( + &mut s, + &pid, + &format!("zz12.g{}", p % 4), + EdgeKind::ContributesTo, + ); + let uid = format!("zz12.u{p:04}"); + item(&mut s, &uid, "person", "1"); + edge(&mut s, &pid, &uid, EdgeKind::Mentions); + edge(&mut s, &uid, &pid, EdgeKind::ContributesTo); + for a in 0..2 { + let aid = format!("zz12.a{p:04}{a}"); + item(&mut s, &aid, "action", &((p + a) % 4).to_string()); + s.set_parent(&aid, &pid, "t").unwrap(); + } + } + s +} + +/// The reads one ranked `next` and one ranked `list` run on a board of `n` projects. +fn reads(n: usize) -> (usize, EdgeReadStats, ItemReadStats) { + let cfg = plugin::load("derived-rank-fixture").unwrap(); + let s = cost_board(n); + let (edges, items) = (s.edge_read_stats(), s.item_read_stats()); + let next = read::next(&cfg, &s, NOW, None).unwrap(); + read::list_at(&cfg, &s, None, None, Some(SortKey::Rank), Some(NOW)).unwrap(); + ( + next.len(), + s.edge_read_stats().since(edges), + s.item_read_stats().since(items), + ) +} + +/// The edge pass runs ONCE per query, not once per candidate (spec §5, §9). Asserts the query +/// shape — the store's own counts of edge and item reads — not the clock, for the reason the xn8s +/// guard gives (a timing ratio reports the machine's load as readily as the code's): the counts +/// are identical on a board of 12 projects and on one five times its size, so the ratio of reads +/// to candidates falls with the board instead of staying flat. A per-candidate edge read, or a +/// per-type rank recomputed per neighbour, turns them into numbers that grow with `n`. +#[test] +fn the_edge_pass_runs_once_per_query_however_many_candidates() { + let (small_n, small_edges, small_items) = reads(12); + let (large_n, large_edges, large_items) = reads(60); + assert!( + large_n > 4 * small_n && small_n > 20, + "fixture regressed: {small_n} / {large_n} candidates" + ); + // `next`: one pass; `list`: one pass, plus `next`'s candidates for the project rank (no new + // edge read) and one reverse-mention lookup for the persons' `sum`. + assert_eq!( + small_edges, + EdgeReadStats { + passes: 2, + mention_lookups: 1, + ..small_edges + }, + "one edge pass per query, one mention lookup for the one query that counts mentions" + ); + // Every other read of edges, labels or custom fields is counted too (review of PR #48, Test + // Quality #1): none runs per item, and the bulk ones do not grow with the board (the equality + // below), so a per-candidate regression through any of them goes red here. + assert_eq!( + small_edges.per_item, 0, + "no per-item edge read: {small_edges:?}" + ); + assert_eq!( + large_edges, small_edges, + "the edge reads do not grow with the board" + ); + assert_eq!(small_items.single, 0, "no per-item read: {small_items:?}"); + assert_eq!( + large_items, small_items, + "the item reads do not grow with the board" + ); +} diff --git a/crates/facade/tests/list_query.rs b/crates/facade/tests/list_query.rs new file mode 100644 index 0000000..00e99a3 --- /dev/null +++ b/crates/facade/tests/list_query.rs @@ -0,0 +1,446 @@ +//! `list` with a plugin-declared order per type, the type and container filters, and paging +//! (6j6v.n698): `[ranking.list.]` ranks `list --sort rank`, `types = [...]` orders the types +//! against each other, a type without a table falls back to the `[ranking.next]` default order, and +//! `Engine::list_query` narrows and pages exactly as `next_query` does — through the same snapshot +//! cache. Tested at the seam the products take. Its own test binary, so the fixture plugins do not +//! pollute the facade lib tests' "exactly two OSS plugins" assertion. + +use nexus_flow_core::model::{EdgeKind, ItemRow}; +use nexus_flow_core::store::Store; +use nexus_flow_facade::engine::Engine; +use nexus_flow_facade::error::ErrorKind; +use nexus_flow_facade::plugin::{self, PluginRegistration}; +use nexus_flow_facade::read::{self, ListQuery, MentionedByQuery, NextFilter, NextQuery, SortKey}; +use nexus_flow_facade::workspace::{self, WorkspaceExt}; +use std::path::Path; +use tempfile::TempDir; + +const NOW: &str = "2026-06-17T00:00:00Z"; +const ACTOR: &str = "alice"; + +/// The fixture: four types. Under `list --sort rank`, persons rank by title, projects by priority +/// DESCENDING, and both stand before the types without a list table (`action`, `goal`), which take +/// the `[ranking.next]` default order. `next` never sees `[ranking.list]`. +const LIST_RANKED: &str = r#" +name = "list-rank-fixture" +[description] +en = "x" +de = "y" +[priority] +labels = ["P0", "P1", "P2", "P3"] +[types] +list = ["goal", "project", "person", "action"] +[vocabulary.status] +open = "open" +in_progress = "in progress" +closed = "closed" +[ranking.next] +order = [ + { field = "status", precedence = ["in_progress", "open"] }, + { field = "priority", dir = "asc" }, +] +[ranking.list] +types = ["person", "project"] +[ranking.list.person] +order = [ { field = "title", dir = "asc" } ] +[ranking.list.project] +order = [ { field = "priority", dir = "desc" } ] +[presentation.list] +columns = ["id"] +"#; + +inventory::submit!(PluginRegistration { + name: "list-rank-fixture", + toml: LIST_RANKED, + order: 120, +}); + +// The same plugin without `[ranking.list]`: `list --sort rank` ranks by the `[ranking.next]` +// default order alone, as before n698. +inventory::submit!(PluginRegistration { + name: "list-flat-rank-fixture", + toml: r#" +name = "list-flat-rank-fixture" +[description] +en = "x" +de = "y" +[priority] +labels = ["P0", "P1", "P2", "P3"] +[types] +list = ["goal", "project", "person", "action"] +[vocabulary.status] +open = "open" +in_progress = "in progress" +closed = "closed" +[ranking.next] +order = [ + { field = "status", precedence = ["in_progress", "open"] }, + { field = "priority", dir = "asc" }, +] +[presentation.list] +columns = ["id"] +"#, + order: 121, +}); + +fn workspace(plugin: &str) -> TempDir { + let tmp = TempDir::new().unwrap(); + workspace::init_flow(tmp.path(), plugin).unwrap(); + tmp +} + +/// Another writer on the workspace — what a second process, or a sync pull, is to the engine. +fn writer(dir: &Path) -> Store { + workspace::discover(dir).unwrap().open_store().unwrap() +} + +fn item(s: &mut Store, id: &str, ty: &str, title: &str, priority: &str) { + s.create_item(id, ty, title, "t"); + s.set_field(id, "priority", Some(priority.into()), "t"); +} + +/// Seven items whose per-type list orders DISAGREE with both the id order and the flat rank: +/// +/// - persons: `m001` "Zoe" P1, `m002` "Adam" P2 (label `vip`) → by title: m002, m001; +/// - projects: `p001` P0, `p002` P3 → by priority desc: p002, p001; +/// - actions (no table): `a001` P3 child of p001, `a002` P0 in progress, contributes to p001 → +/// the default order: a002, a001; +/// - goal (no table): `g001` P1. +/// +/// `m001` contributes to `p001` as well, so the container `p001` holds a001, a002 and m001. +fn board(dir: &Path) { + let mut s = writer(dir); + item(&mut s, "zz12.m001", "person", "Zoe", "1"); + item(&mut s, "zz12.m002", "person", "Adam", "2"); + item(&mut s, "zz12.p001", "project", "Build", "0"); + item(&mut s, "zz12.p002", "project", "Archive", "3"); + item(&mut s, "zz12.a001", "action", "Write", "3"); + item(&mut s, "zz12.a002", "action", "Read", "0"); + item(&mut s, "zz12.g001", "goal", "Grow", "1"); + s.set_parent("zz12.a001", "zz12.p001", "t").unwrap(); + s.add_edge("zz12.a002", "zz12.p001", EdgeKind::ContributesTo, "t"); + s.add_edge("zz12.m001", "zz12.p001", EdgeKind::ContributesTo, "t"); + s.set_field("zz12.a002", "status", Some("in_progress".into()), "t"); + s.add_label("zz12.m002", "vip", "t"); +} + +/// `list --sort rank` under the fixture: persons by title, projects by priority descending, then +/// the unlisted types by name (`action` before `goal`), each by the `[ranking.next]` default order. +const RANKED: [&str; 7] = [ + "zz12.m002", + "zz12.m001", + "zz12.p002", + "zz12.p001", + "zz12.a002", + "zz12.a001", + "zz12.g001", +]; + +const BY_ID: [&str; 7] = [ + "zz12.a001", + "zz12.a002", + "zz12.g001", + "zz12.m001", + "zz12.m002", + "zz12.p001", + "zz12.p002", +]; + +fn ids(items: &[ItemRow]) -> Vec { + items.iter().map(|i| i.id.clone()).collect() +} + +fn engine(tmp: &TempDir) -> Engine { + Engine::open(None, tmp.path()).unwrap() +} + +fn page(engine: &Engine, q: ListQuery) -> read::NextPage { + engine.list_query(&q).unwrap() +} + +fn ranked() -> ListQuery { + ListQuery::new().sort(SortKey::Rank) +} + +// ---- the order ---------------------------------------------------------------------- + +#[test] +fn the_default_order_stays_id_whatever_the_plugin_declares() { + let tmp = workspace("list-rank-fixture"); + board(tmp.path()); + let e = engine(&tmp); + let p = page(&e, ListQuery::new()); + assert_eq!(ids(&p.items), BY_ID); + assert_eq!((p.total, p.start, p.restarted), (7, 0, false)); + assert_eq!(p.next_token, None); + assert_eq!(ids(&e.list(None, None).unwrap()), BY_ID, "Engine::list too"); +} + +#[test] +fn sort_rank_takes_the_per_type_list_orders_and_the_cross_type_order() { + let tmp = workspace("list-rank-fixture"); + board(tmp.path()); + let p = page(&engine(&tmp), ranked()); + assert_eq!(ids(&p.items), RANKED); + // The free read the CLI's plain `list --sort rank` and `flow_list` take ranks alike. + let s = writer(tmp.path()); + let cfg = plugin::load("list-rank-fixture").unwrap(); + let free = read::list(&cfg, &s, None, None, Some(SortKey::Rank)).unwrap(); + assert_eq!(ids(&free), RANKED); +} + +#[test] +fn without_ranking_list_sort_rank_ranks_by_the_next_default_order_as_before() { + // Status precedence first (a002 is in progress), then priority ascending, then id — across + // all types, with no type split. + let tmp = workspace("list-flat-rank-fixture"); + board(tmp.path()); + assert_eq!( + ids(&page(&engine(&tmp), ranked()).items), + [ + "zz12.a002", + "zz12.p001", + "zz12.g001", + "zz12.m001", + "zz12.m002", + "zz12.a001", + "zz12.p002" + ] + ); +} + +#[test] +fn the_bundled_plugins_rank_list_exactly_as_before() { + // issue-tracker: status → priority → type → due → id — the `next` default order, as + // `list --sort rank` ranked before n698. + let tmp = workspace("issue-tracker"); + { + let mut s = writer(tmp.path()); + item(&mut s, "it12.0001", "epic", "e", "2"); + item(&mut s, "it12.0002", "feature", "f", "1"); + item(&mut s, "it12.0003", "bug", "b", "1"); + item(&mut s, "it12.0004", "chore", "c", "0"); + } + assert_eq!( + ids(&page(&engine(&tmp), ranked()).items), + ["it12.0004", "it12.0003", "it12.0002", "it12.0001"] + ); +} + +#[test] +fn ranking_list_never_changes_next() { + let ranked_ws = workspace("list-rank-fixture"); + board(ranked_ws.path()); + let flat_ws = workspace("list-flat-rank-fixture"); + board(flat_ws.path()); + assert_eq!( + ids(&engine(&ranked_ws).next(NOW).unwrap()), + ids(&engine(&flat_ws).next(NOW).unwrap()), + "both plugins share [ranking.next]; [ranking.list] is list's alone" + ); +} + +// ---- the filters ---------------------------------------------------------------------- + +#[test] +fn list_query_filters_by_type_container_label_and_status_keeping_the_order() { + let tmp = workspace("list-rank-fixture"); + board(tmp.path()); + let e = engine(&tmp); + let q = |q: ListQuery| ids(&page(&e, q).items); + + assert_eq!( + q(ranked().filter(NextFilter::new().with_type("person"))), + ["zz12.m002", "zz12.m001"] + ); + assert_eq!( + q(ranked().filter(NextFilter::new().with_type("project").with_type("person"))), + ["zz12.m002", "zz12.m001", "zz12.p002", "zz12.p001"], + "the types are alternatives" + ); + // In p001: its child a001, and its contributors a002 and m001. + assert_eq!( + q(ranked().filter(NextFilter::new().with_container("zz12.p001"))), + ["zz12.m001", "zz12.a002", "zz12.a001"] + ); + assert_eq!( + q(ListQuery::new().filter(NextFilter::new().with_container("zz12.p001"))), + ["zz12.a001", "zz12.a002", "zz12.m001"], + "id order without a sort" + ); + assert_eq!( + q(ListQuery::new().filter(NextFilter::new().with_label("vip"))), + ["zz12.m002"] + ); + assert_eq!(q(ListQuery::new().status("in_progress")), ["zz12.a002"]); + assert!(q(ListQuery::new().filter(NextFilter::new().with_container("zz12.g001"))).is_empty()); +} + +// ---- paging, as `next` pages ---------------------------------------------------------- + +#[test] +fn a_limit_alone_cuts_and_discloses_but_hands_out_no_token() { + let tmp = workspace("list-rank-fixture"); + board(tmp.path()); + let p = page(&engine(&tmp), ranked().limit(3)); + assert_eq!(ids(&p.items), RANKED[0..3]); + assert_eq!(p.total, 7); + assert!(p.truncated()); + assert_eq!(p.next_token, None); +} + +#[test] +fn paginate_walks_the_whole_ranked_list_in_order() { + let tmp = workspace("list-rank-fixture"); + board(tmp.path()); + let e = engine(&tmp); + let mut seen = Vec::new(); + let mut p = page(&e, ranked().limit(3).paginate()); + loop { + assert!(!p.restarted); + assert_eq!(p.start, seen.len()); + assert_eq!(p.total, 7); + seen.extend(ids(&p.items)); + match p.next_token.clone() { + Some(t) => p = page(&e, ranked().limit(3).token(t)), + None => break, + } + } + assert_eq!(seen, RANKED); +} + +#[test] +fn a_change_to_a_cached_item_restarts_at_the_first_page() { + let tmp = workspace("list-rank-fixture"); + board(tmp.path()); + let e = engine(&tmp); + let p1 = page(&e, ranked().limit(3).paginate()); + // p001 is on the page not yet served: the op bears on the loaded list. + e.update(NOW, ACTOR, "zz12.p001", &["title=Built".to_string()]) + .unwrap(); + let p = page(&e, ranked().limit(3).token(p1.next_token.unwrap())); + assert!(p.restarted, "the snapshot is stale"); + assert_eq!( + (ids(&p.items), p.start), + (RANKED[0..3].iter().map(|s| s.to_string()).collect(), 0) + ); +} + +#[test] +fn a_new_item_does_not_invalidate_and_waits_for_a_fresh_query() { + let tmp = workspace("list-rank-fixture"); + board(tmp.path()); + let e = engine(&tmp); + let p1 = page(&e, ranked().limit(4).paginate()); + // A person titled "Aaron" would rank first; it is not in the snapshot, so this pager keeps on. + item(&mut writer(tmp.path()), "zz12.m003", "person", "Aaron", "1"); + let p2 = page(&e, ranked().limit(4).token(p1.next_token.unwrap())); + assert!(!p2.restarted); + assert_eq!(ids(&p2.items), RANKED[4..]); + let fresh = page(&e, ranked()); + assert_eq!((fresh.total, fresh.items[0].id.as_str()), (8, "zz12.m003")); +} + +#[test] +fn a_token_of_another_sort_filter_status_or_read_restarts() { + let tmp = workspace("list-rank-fixture"); + board(tmp.path()); + let e = engine(&tmp); + let t = page(&e, ranked().limit(2).paginate()).next_token.unwrap(); + + let by_id = page(&e, ListQuery::new().limit(2).token(&t)); + assert!(by_id.restarted, "another sort"); + assert_eq!(ids(&by_id.items), BY_ID[0..2]); + let filtered = page( + &e, + ranked() + .filter(NextFilter::new().with_type("person")) + .limit(2) + .token(&t), + ); + assert!(filtered.restarted, "another filter"); + assert!( + page(&e, ranked().status("open").limit(2).token(&t)).restarted, + "another status" + ); + let as_next = e + .next_query(&NextQuery::new(NOW).limit(2).token(&t)) + .unwrap(); + assert!(as_next.restarted, "a list token is not a next token"); + let as_mentioned_by = e + .mentioned_by("zz12.p001", &MentionedByQuery::new().limit(2).token(&t)) + .unwrap(); + assert!(as_mentioned_by.restarted, "nor a mentioned-by token"); + + // And the other way round: a next token does not page a list. + let next_token = e + .next_query(&NextQuery::new(NOW).limit(2).paginate()) + .unwrap() + .next_token + .unwrap(); + assert!(page(&e, ranked().limit(2).token(next_token)).restarted); + + let garbage = page(&e, ranked().limit(2).token("nope")); + assert!(garbage.restarted); + assert_eq!(ids(&garbage.items), RANKED[0..2]); +} + +#[test] +fn a_token_from_another_plugin_list_order_restarts() { + let tmp = workspace("list-rank-fixture"); + board(tmp.path()); + let s = writer(tmp.path()); + let ranked_cfg = plugin::load("list-rank-fixture").unwrap(); + let flat_cfg = plugin::load("list-flat-rank-fixture").unwrap(); + let p1 = read::list_page(&ranked_cfg, &s, &ranked().limit(2).paginate()).unwrap(); + let token = p1.next_token.unwrap(); + let under_flat = read::list_page(&flat_cfg, &s, &ranked().limit(2).token(&token)).unwrap(); + assert!(under_flat.restarted, "another [ranking.list] restarts"); + let p2 = read::list_page(&ranked_cfg, &s, &ranked().limit(2).token(&token)).unwrap(); + assert!(!p2.restarted); + assert_eq!(ids(&p2.items), RANKED[2..4]); +} + +#[test] +fn paginate_needs_a_limit_and_a_token_needs_paginate() { + let tmp = workspace("list-rank-fixture"); + board(tmp.path()); + let e = engine(&tmp); + for q in [ + ListQuery::new().paginate(), + ListQuery::new().limit(0).paginate(), + ] { + assert_eq!(e.list_query(&q).unwrap_err().kind, ErrorKind::Validation); + } + let mut q = ListQuery::new().limit(2); + q.token = Some("x.2".into()); + assert_eq!(e.list_query(&q).unwrap_err().kind, ErrorKind::Validation); +} + +#[test] +fn list_query_value_is_the_paging_envelope_of_list_records() { + let tmp = workspace("list-rank-fixture"); + board(tmp.path()); + let e = engine(&tmp); + let v = e.list_query_value(&ranked().limit(2).paginate()).unwrap(); + assert_eq!(v["total"], 7); + assert_eq!(v["restarted"], false); + assert!(v["next_token"].is_string()); + let items = v["items"].as_array().unwrap(); + assert_eq!(items.len(), 2); + assert_eq!(items[0]["id"], "zz12.m002"); + assert_eq!(items[0]["type"], "person"); + // The records are `list_value`'s. + let listed = e.list_value(None, None).unwrap(); + let same = listed + .as_array() + .unwrap() + .iter() + .find(|r| r["id"] == "zz12.m002") + .unwrap(); + assert_eq!(&items[0], same); + // Without paginate: no token keys, as for `next`. + let plain = e.list_query_value(&ListQuery::new()).unwrap(); + assert_eq!(plain["total"], 7); + assert!(plain.get("next_token").is_none()); +} diff --git a/crates/facade/tests/mentioned_by.rs b/crates/facade/tests/mentioned_by.rs new file mode 100644 index 0000000..5007e90 --- /dev/null +++ b/crates/facade/tests/mentioned_by.rs @@ -0,0 +1,375 @@ +//! Mention edges read over the engine seam (6j6v.q29p): `Engine::mentions` (an item's outgoing +//! targets) and `Engine::mentioned_by` (the items that mention a target), the latter narrowed by +//! type and container and paged through the same machine-local snapshot `next` pages through +//! (6j6v.15ed). Tested at the seam the products take. + +use nexus_flow_core::model::EdgeKind; +use nexus_flow_core::store::{EdgeReadStats, Store}; +use nexus_flow_facade::engine::Engine; +use nexus_flow_facade::error::ErrorKind; +use nexus_flow_facade::read::{self, MentionedByQuery, NextFilter, NextQuery}; +use nexus_flow_facade::workspace::{self, WorkspaceExt}; +use std::path::Path; +use tempfile::TempDir; + +const NOW: &str = "2026-06-17T00:00:00Z"; +const ACTOR: &str = "alice"; +const TARGET: &str = "ab12.t001"; + +fn workspace() -> TempDir { + let tmp = TempDir::new().unwrap(); + workspace::init_flow(tmp.path(), "issue-tracker").unwrap(); + tmp +} + +/// Another writer on the workspace — what a second process, or a sync pull, is to the engine. +fn writer(dir: &Path) -> Store { + workspace::discover(dir).unwrap().open_store().unwrap() +} + +fn item(s: &mut Store, id: &str, ty: &str) { + s.create_item(id, ty, id, "t"); + s.set_field(id, "priority", Some("2".into()), "t"); +} + +fn ids(items: &[nexus_flow_core::model::ItemRow]) -> Vec { + items.iter().map(|i| i.id.clone()).collect() +} + +fn by(engine: &Engine, q: MentionedByQuery) -> read::NextPage { + engine.mentioned_by(TARGET, &q).unwrap() +} + +/// A target, two containers, and mentioners of every kind: +/// +/// - `0001` task, parent e001 — mentions the target +/// - `0002` bug, contributes to e002 — mentions the target +/// - `0003` task, parent e002 AND contributes to e001 — mentions the target +/// - `0004` bug, in nothing, closed — mentions the target +/// - `0005` task, deleted — mentions the target +/// - `0006` task — mentions only e001, not the target +/// - `0007` task — its mention of the target was removed +fn board(dir: &Path) { + let mut s = writer(dir); + item(&mut s, TARGET, "feature"); + item(&mut s, "ab12.e001", "epic"); + item(&mut s, "ab12.e002", "epic"); + for (id, ty) in [ + ("ab12.0001", "task"), + ("ab12.0002", "bug"), + ("ab12.0003", "task"), + ("ab12.0004", "bug"), + ("ab12.0005", "task"), + ("ab12.0006", "task"), + ("ab12.0007", "task"), + ] { + item(&mut s, id, ty); + } + s.set_parent("ab12.0001", "ab12.e001", "t").unwrap(); + s.set_parent("ab12.0003", "ab12.e002", "t").unwrap(); + s.add_edge("ab12.0002", "ab12.e002", EdgeKind::ContributesTo, "t"); + s.add_edge("ab12.0003", "ab12.e001", EdgeKind::ContributesTo, "t"); + for from in [ + "ab12.0003", + "ab12.0001", + "ab12.0002", + "ab12.0004", + "ab12.0005", + "ab12.0007", + ] { + s.add_edge(from, TARGET, EdgeKind::Mentions, "t"); + } + s.add_edge("ab12.0006", "ab12.e001", EdgeKind::Mentions, "t"); + s.remove_edge("ab12.0007", TARGET, EdgeKind::Mentions, "t"); + s.set_field("ab12.0004", "status", Some("closed".into()), "t"); + s.delete_item("ab12.0005", "t"); +} + +const LIVE: [&str; 4] = ["ab12.0001", "ab12.0002", "ab12.0003", "ab12.0004"]; + +// ---- Engine::mentions ---------------------------------------------------------- + +#[test] +fn mentions_reads_the_outgoing_targets_over_the_seam() { + let tmp = workspace(); + board(tmp.path()); + let engine = Engine::open(None, tmp.path()).unwrap(); + assert_eq!(engine.mentions("ab12.0006").unwrap(), ["ab12.e001"]); + assert_eq!(engine.mentions("ab12.0001").unwrap(), [TARGET]); + assert!(engine.mentions("ab12.0007").unwrap().is_empty(), "removed"); + // A write through the engine reads back through it. + engine + .mention_add(NOW, ACTOR, "ab12.0006", "ab12.e002") + .unwrap(); + assert_eq!( + engine.mentions("ab12.0006").unwrap(), + ["ab12.e001", "ab12.e002"] + ); + let err = engine.mentions("ab12.zzzz").unwrap_err(); + assert_eq!(err.kind, ErrorKind::NotFound); +} + +// ---- Engine::mentioned_by: the set and its filters ------------------------------- + +#[test] +fn mentioned_by_lists_the_live_mentioners_in_id_order() { + let tmp = workspace(); + board(tmp.path()); + let engine = Engine::open(None, tmp.path()).unwrap(); + let p = by(&engine, MentionedByQuery::new()); + // Closed counts (a reference is not work); deleted, removed and other targets' do not. + assert_eq!(ids(&p.items), LIVE); + assert_eq!((p.total, p.start, p.restarted), (4, 0, false)); + assert_eq!(p.next_token, None); + // Nothing mentions e002. + let none = engine + .mentioned_by("ab12.e002", &MentionedByQuery::new()) + .unwrap(); + assert!(none.items.is_empty()); + assert_eq!(none.total, 0); +} + +#[test] +fn an_archived_mentioner_still_counts_at_the_seam() { + // Live is the one rule (spec §11): archived is not deleted, so `Engine::mentioned_by` keeps + // it, as `mentioners_of` and `count = "mentioned_by"` do (review of PR #48, Test Quality #9). + let tmp = workspace(); + board(tmp.path()); + { + let mut s = writer(tmp.path()); + s.set_field("ab12.0002", "archived", Some(NOW.into()), "t"); + } + let engine = Engine::open(None, tmp.path()).unwrap(); + assert_eq!(ids(&by(&engine, MentionedByQuery::new()).items), LIVE); +} + +#[test] +fn mentioned_by_a_missing_or_deleted_target_is_not_found() { + let tmp = workspace(); + board(tmp.path()); + let engine = Engine::open(None, tmp.path()).unwrap(); + for id in ["ab12.zzzz", "ab12.0005"] { + let err = engine + .mentioned_by(id, &MentionedByQuery::new()) + .unwrap_err(); + assert_eq!(err.kind, ErrorKind::NotFound, "{id}"); + } +} + +#[test] +fn mentioned_by_filters_by_type_and_by_container() { + let tmp = workspace(); + board(tmp.path()); + let engine = Engine::open(None, tmp.path()).unwrap(); + let q = |f: NextFilter| ids(&by(&engine, MentionedByQuery::new().filter(f)).items); + + assert_eq!( + q(NextFilter::new().with_type("bug")), + ["ab12.0002", "ab12.0004"] + ); + assert_eq!( + q(NextFilter::new().with_type("bug").with_type("task")), + LIVE, + "the types are alternatives" + ); + // In e001: its child 0001 and its contributor 0003. In e002: contributor 0002, child 0003. + assert_eq!( + q(NextFilter::new().with_container("ab12.e001")), + ["ab12.0001", "ab12.0003"] + ); + assert_eq!( + q(NextFilter::new().with_container("ab12.e002")), + ["ab12.0002", "ab12.0003"] + ); + assert_eq!( + q(NextFilter::new() + .with_container("ab12.e002") + .with_type("task")), + ["ab12.0003"] + ); + assert!(q(NextFilter::new().with_type("epic")).is_empty()); +} + +// ---- Engine::mentioned_by: paging, as `next` pages --------------------------------- + +#[test] +fn a_limit_alone_cuts_and_discloses_but_hands_out_no_token() { + let tmp = workspace(); + board(tmp.path()); + let engine = Engine::open(None, tmp.path()).unwrap(); + let p = by(&engine, MentionedByQuery::new().limit(3)); + assert_eq!( + (ids(&p.items), p.total), + (LIVE[0..3].iter().map(|s| s.to_string()).collect(), 4) + ); + assert!(p.truncated()); + assert_eq!(p.next_token, None); +} + +#[test] +fn paginate_walks_the_whole_list_in_order() { + let tmp = workspace(); + board(tmp.path()); + let engine = Engine::open(None, tmp.path()).unwrap(); + let p1 = by(&engine, MentionedByQuery::new().limit(3).paginate()); + assert_eq!(ids(&p1.items), LIVE[0..3]); + let t = p1.next_token.expect("a row remains"); + let p2 = by(&engine, MentionedByQuery::new().limit(3).token(&t)); + assert_eq!( + (ids(&p2.items), p2.start, p2.total), + (vec![LIVE[3].to_string()], 3, 4) + ); + assert!(!p2.restarted); + assert_eq!(p2.next_token, None, "the last page"); +} + +#[test] +fn removing_a_mention_of_a_cached_item_restarts_at_the_first_page() { + let tmp = workspace(); + board(tmp.path()); + let engine = Engine::open(None, tmp.path()).unwrap(); + let p1 = by(&engine, MentionedByQuery::new().limit(2).paginate()); + // The edge op names the mentioner on the page not yet served: it bears on the loaded list. + engine + .mention_remove(NOW, ACTOR, "ab12.0003", TARGET) + .unwrap(); + let p = by( + &engine, + MentionedByQuery::new() + .limit(2) + .token(p1.next_token.unwrap()), + ); + assert!(p.restarted, "the snapshot is stale"); + assert_eq!( + (ids(&p.items), p.start, p.total), + (vec!["ab12.0001".to_string(), "ab12.0002".to_string()], 0, 3) + ); +} + +#[test] +fn an_outside_op_does_not_invalidate_and_a_new_mentioner_waits_for_a_fresh_query() { + let tmp = workspace(); + board(tmp.path()); + let engine = Engine::open(None, tmp.path()).unwrap(); + let p1 = by(&engine, MentionedByQuery::new().limit(2).paginate()); + // 0006 is not in the snapshot; its new mention of the target is not served by this pager. + engine.mention_add(NOW, ACTOR, "ab12.0006", TARGET).unwrap(); + let p2 = by( + &engine, + MentionedByQuery::new() + .limit(2) + .token(p1.next_token.unwrap()), + ); + assert!(!p2.restarted); + assert_eq!(ids(&p2.items), ["ab12.0003", "ab12.0004"]); + assert_eq!(p2.next_token, None); + assert_eq!( + by(&engine, MentionedByQuery::new()).total, + 5, + "a fresh query sees it" + ); +} + +#[test] +fn a_token_of_another_target_filter_or_read_restarts() { + let tmp = workspace(); + board(tmp.path()); + let mut s = writer(tmp.path()); + item(&mut s, "ab12.t002", "feature"); + for from in ["ab12.0001", "ab12.0002", "ab12.0003"] { + s.add_edge(from, "ab12.t002", EdgeKind::Mentions, "t"); + } + let engine = Engine::open(None, tmp.path()).unwrap(); + let t = by(&engine, MentionedByQuery::new().limit(2).paginate()) + .next_token + .unwrap(); + + let other_target = engine + .mentioned_by("ab12.t002", &MentionedByQuery::new().limit(2).token(&t)) + .unwrap(); + assert!(other_target.restarted, "another target"); + let other_filter = by( + &engine, + MentionedByQuery::new() + .filter(NextFilter::new().with_type("task")) + .limit(2) + .token(&t), + ); + assert!(other_filter.restarted, "another filter"); + let as_next = engine + .next_query(&NextQuery::new(NOW).limit(2).token(&t)) + .unwrap(); + assert!( + as_next.restarted, + "a mentioned-by token is not a next token" + ); + let garbage = by(&engine, MentionedByQuery::new().limit(2).token("nope")); + assert!(garbage.restarted); + assert_eq!(ids(&garbage.items), LIVE[0..2]); +} + +#[test] +fn paginate_needs_a_limit_and_a_token_needs_paginate() { + let tmp = workspace(); + board(tmp.path()); + let engine = Engine::open(None, tmp.path()).unwrap(); + for q in [ + MentionedByQuery::new().paginate(), + MentionedByQuery::new().limit(0).paginate(), + ] { + let err = engine.mentioned_by(TARGET, &q).unwrap_err(); + assert_eq!(err.kind, ErrorKind::Validation); + } + let mut q = MentionedByQuery::new().limit(2); + q.token = Some("x.2".into()); + let err = engine.mentioned_by(TARGET, &q).unwrap_err(); + assert_eq!(err.kind, ErrorKind::Validation); +} + +#[test] +fn mentioned_by_value_is_the_paging_envelope_of_list_records() { + let tmp = workspace(); + board(tmp.path()); + let engine = Engine::open(None, tmp.path()).unwrap(); + let v = engine + .mentioned_by_value(TARGET, &MentionedByQuery::new().limit(2).paginate()) + .unwrap(); + assert_eq!(v["total"], 4); + assert_eq!(v["restarted"], false); + assert!(v["next_token"].is_string()); + let items = v["items"].as_array().unwrap(); + assert_eq!(items.len(), 2); + assert_eq!(items[0]["id"], "ab12.0001"); + assert_eq!(items[0]["type"], "task"); + // Without paginate: no token keys, as for `next`. + let plain = engine + .mentioned_by_value(TARGET, &MentionedByQuery::new()) + .unwrap(); + assert_eq!(plain["total"], 4); + assert!(plain.get("next_token").is_none()); +} + +// ---- the shared reverse lookup ------------------------------------------------------ + +#[test] +fn mentioners_of_answers_many_targets_in_one_read() { + // THE reverse lookup `mentioned_by` and the derived ranking key `count = "mentioned_by"` + // (6j6v.e354 §11) share. + let tmp = workspace(); + board(tmp.path()); + let s = writer(tmp.path()); + let before = s.edge_read_stats(); + let m = read::mentioners_of(&s, &[TARGET, "ab12.e001", "ab12.e002", "ab12.zzzz"]).unwrap(); + assert_eq!( + s.edge_read_stats().since(before), + EdgeReadStats { + mention_lookups: 1, + ..EdgeReadStats::default() + }, + "one reverse lookup for all four targets, and no per-item edge read" + ); + assert_eq!(ids(&m[TARGET]), LIVE); + assert_eq!(ids(&m["ab12.e001"]), ["ab12.0006"]); + assert!(!m.contains_key("ab12.e002"), "nothing mentions it"); + assert!(!m.contains_key("ab12.zzzz"), "not an item, not an error"); +} diff --git a/crates/nxs/src/mcp/flow.rs b/crates/nxs/src/mcp/flow.rs index 639ddf5..0aaa4bd 100644 --- a/crates/nxs/src/mcp/flow.rs +++ b/crates/nxs/src/mcp/flow.rs @@ -56,12 +56,19 @@ pub(super) fn list(db: Option<&str>, start: &Path, p: &FlowListParams) -> Result let (ws, cfg) = open(db, start)?; let store = ws.open_store()?; let sort = p.sort.as_deref().map(read::parse_sort).transpose()?; - let items = read::list( + // `now` only matters to a derived ranking key of `[ranking.list]` that reads the rank of a + // type `next` ranks (6j6v.x0wf) — resolved as `nxf list` resolves it, and only for a ranked + // list under such a plugin, so the seams agree. + let now = (sort == Some(read::SortKey::Rank) && cfg.ranking.list.has_derived()) + .then(|| resolve_now(None)) + .transpose()?; + let items = read::list_at( &cfg, &store, p.status.as_deref(), p.item_type.as_deref(), sort, + now.as_deref(), )?; read::list_to_value_with_custom(&cfg, &store, &items) } diff --git a/docs/generated/cli.md b/docs/generated/cli.md index 93fffbf..3dc68de 100644 --- a/docs/generated/cli.md +++ b/docs/generated/cli.md @@ -27,6 +27,7 @@ This document contains the help content for the `nxf` command-line program. * [`nxf mention add`↴](#nxf-mention-add) * [`nxf mention remove`↴](#nxf-mention-remove) * [`nxf mention list`↴](#nxf-mention-list) +* [`nxf mention by`↴](#nxf-mention-by) * [`nxf contributes`↴](#nxf-contributes) * [`nxf contributes add`↴](#nxf-contributes-add) * [`nxf contributes remove`↴](#nxf-contributes-remove) @@ -65,7 +66,7 @@ nexus-flow agent CLI * `archive` — Archive one or more items — "closed and put away". Cascades DOWN: each root and its whole `belongs_to` subtree are archived, but only when the root is closed and every descendant is closed (already-archived counts as closed). Atomic per root, independent between roots; the result reports each root's outcome plus the full list of ids actually archived (incl. cascaded). Archiving is reversible — see `unarchive` * `unarchive` — Unarchive one or more items, making them visible again. Cascades UP only: each item and its ancestor chain resurface, but its children/siblings stay archived. Partial per root; the result reports each root's outcome plus the full list of ids actually unarchived * `close` — Close an item, recording why. Closing is final (no hard delete), so a reason is required -* `list` — List items, optionally filtered by status and/or type. Id-ordered by default (so the `--json` record stays plugin-independent); override with `--sort rank` +* `list` — List items, optionally filtered by status, type, container and/or label. Id-ordered by default (so the `--json` record stays plugin-independent); `--sort rank` ranks by the active plugin's `list` order. `--limit` shows only the head of it, and says so; `--paginate` adds a token for the next page * `blocked` — List blocked items (open, with an open blocker or in a cycle). Ranked by default; override with `--sort id` * `deferred` — List deferred items (open, unblocked, with a future defer date) — the lane that is neither ready nor blocked. Ordered by defer-date ascending (soonest first); override with `--sort` * `closed` — List closed items (excluding archived). Ordered by close-date descending (most recent first); override with `--sort` @@ -73,7 +74,7 @@ nexus-flow agent CLI * `recap` — Recap the most recently closed work — "what got finished lately" — newest close first, archived closes INCLUDED. A recency recall view, distinct from `closed` (the lane, which drops archived). `--limit` caps the count (default 10); `--since` keeps only later closes * `next` — List actionable work — ready plus already-claimed, unblocked, not deferred — ranked by the active plugin's `next` policy and tiered to finish before starting: work you can close now, then each started epic with its open children, then the backlog. `--sort id` gives the flat, untiered order; `--type`, `--in` and `--label` narrow it without re-ranking; `--limit` shows only the head of it, and says so; `--paginate` adds a token for the next page * `dep` — Add or remove a dependency edge. Direction: `dep add A B` makes A depend on B, so B blocks A — A cannot be ready until B closes -* `mention` — Add, remove, or list reference edges (free-text short-id citations) +* `mention` — Add, remove, or list reference edges (free-text short-id citations), in either direction * `contributes` — Add, remove, or list contributes-to edges: `from` contributes to `to` (n:m). Like a mention it never blocks — it records a structural "feeds into" relation, not a dependency * `thread` — Link a chat thread to a board item, or list what a thread is about. This is how the conversation that did the work and the ticket it was about stop being two artefacts: a later reader of the item sees that there are conversations, without having to go looking * `label` — Add, remove, or list user labels (free-text tags) on an item. A label is user vocabulary, distinct from the plugin's display type; filter `list`/`next` with `--label` @@ -221,16 +222,20 @@ Close an item, recording why. Closing is final (no hard delete), so a reason is ## `nxf list` -List items, optionally filtered by status and/or type. Id-ordered by default (so the `--json` record stays plugin-independent); override with `--sort rank` +List items, optionally filtered by status, type, container and/or label. Id-ordered by default (so the `--json` record stays plugin-independent); `--sort rank` ranks by the active plugin's `list` order. `--limit` shows only the head of it, and says so; `--paginate` adds a token for the next page **Usage:** `nxf list [OPTIONS]` ###### **Options:** * `--status ` — Filter by status -* `--type ` — Filter by type, using the active plugin's vocabulary (see `nxf schema`) +* `--type ` — Filter by type, using the active plugin's vocabulary (see `nxf schema`). Repeatable: the types are alternatives +* `--in ` — Only items in this container: its children (whose parent it is) and the items that contribute to it * `--label