From 32fd14786e736dc4658d1c59bfce660eef25db27 Mon Sep 17 00:00:00 2001 From: Stefan Ziegler Date: Fri, 6 Nov 2026 18:42:12 +0100 Subject: [PATCH 1/6] Set transfer validation resource target and check API readiness NFR-011 and STACK-013 target a 2 GiB Java heap for representative data validation. Keep host memory, temporary storage, I/O and runtime measurable independently. Jenesis 0.18.1 on Java 25.0.2 executed 42 selected compiler contracts with no failures or skips. The public schema probe covered the P4.0 originals and the explicitly labelled Reference24 derivative. Independent readiness gate PASS with OpenCode 2.2.5 and deepseek/deepseek-flash (Flash-1017). Its 2.3 and 2.4 association counter-probe verified inherited role order, target refinement and explicit NO OID. XML placement remains adapter-specific. --- docs/contracts/requirements.md | 4 ++-- docs/design/transfer-validation-database.md | 4 +++- 2 files changed, 5 insertions(+), 3 deletions(-) diff --git a/docs/contracts/requirements.md b/docs/contracts/requirements.md index f7b0fe36..341fe09d 100644 --- a/docs/contracts/requirements.md +++ b/docs/contracts/requirements.md @@ -51,7 +51,7 @@ Der [Compilerkatalog](../../testdata/verification/compiler-contracts.json) binde | `NFR-008` | Öffentliche Resultate MÜSSEN Nullfreiheit, Reihenfolge und Lebensdauer dokumentieren. | Geltend | [Vertrag](../architecture.md) | API-Vertragstests und Register. | | `NFR-009` | Der Build MUSS mit gepinnten Werkzeugen reproduzierbar ausführbar sein. Jenesis baut die JVM-Produkte. Der Gradle-Wrapper baut die beiden portablen Backends. | Geltend | [Vertrag](../development/build.md) | Frischer Checkout, gepinnte Toolchain und dokumentierte Kommandos. | | `NFR-010` | Der Kern MUSS unabhängig von einer laufenden Serverinfrastruktur verwendbar sein. | Geltend | [Vertrag](../architecture.md) | Lokale Bibliotheks-/CLI-Ausführung ohne Spring, Datenbank oder Container. | -| `NFR-011` | Produktionsgeometrievalidierung MUSS später begrenzte Caches, Streaming und explizite Ressourcenbudgets anbieten. | Geplant | [Vertrag](../design/transfer-validation-database.md) | Grössenkorpus und definierte Reaktion auf Budgeterschöpfung; 64 GB nicht überschritten im festgelegten Profil. | +| `NFR-011` | Produktionsgeometrievalidierung MUSS später begrenzte Caches, Streaming und explizite Ressourcenbudgets anbieten. Ziel für repräsentative produktive Datenprüfungen, auch grösserer Datensätze, ist das Standardprofil `-Xmx2g` (2'147'483'648 Byte Java-Heap). | Geplant | [Vertrag](../design/transfer-validation-database.md#geometrie-und-speicherbudget) | Grössenkorpus und kontrollierte Reaktion auf Budgeterschöpfung gemäss `STACK-013`. | | `NFR-012` | Abhängigkeiten und technische Neuerungen MÜSSEN einen tatsächlichen aktuellen Nutzen haben. | Geltend | [Vertrag](../architecture.md) | Kurzer Zweck je Dependency; kein Framework nur für künftige Eventualitäten. | ## ARCH @@ -321,7 +321,7 @@ Der [Compilerkatalog](../../testdata/verification/compiler-contracts.json) binde | `STACK-010` | Datenvalidierung MUSS lokale und transferweite Regeln unterscheiden; ein lokaler Modus darf keine vollständige Validität behaupten. | Geplant | [Vertrag](../design/transfer-validation-database.md) | Bericht mit geprüften und ausgelassenen Regelklassen. | | `STACK-011` | Ausdrucksauswertung MUSS die getypte Compilersemantik und exakte INTERLIS-Zahlendarstellung verwenden. | Geplant | [Vertrag](../design/transfer-validation-database.md) | Ausdrücke werden nicht erneut aus Rohtext interpretiert; exakte Grenzfalltests. | | `STACK-012` | Geometrieprüfungen MÜSSEN wiederholte Arbeit vermeiden, fachlich relevante Rundungs-/Bogenregeln aber erhalten. | Geplant | [Vertrag](../design/transfer-validation-database.md) | Geometrieadapter- und Cachetests mit Bögen, Präzision und geänderten Optionen. | -| `STACK-013` | Validierung MUSS ein explizites Ressourcenbudget unterstützen; für die vorgesehene grosse Betriebsvariante ist ein Gesamtbudget von höchstens 64 GB planbar. | Geplant | [Vertrag](../design/transfer-validation-database.md#geometrie-und-speicherbudget) | Lasttest berücksichtigt Heap, Native Memory, Caches und Puffer; Überschreitung wird kontrolliert behandelt. | +| `STACK-013` | Validierung MUSS ein explizites Ressourcenbudget unterstützen. Das Standardprofil strebt repräsentative produktive Datenprüfungen, auch grösserer Datensätze, unter `-Xmx2g` (2'147'483'648 Byte Java-Heap) an. Fachlich extrem komplexe Eingaben dürfen nach Begründung höhere Budgets benötigen. | Geplant | [Vertrag](../design/transfer-validation-database.md#geometrie-und-speicherbudget) | Lasttest erfasst Heap, Gesamtprozessspeicher (RSS), native Allokationen, Cache, temporären Spitzenplatz, I/O und Laufzeit getrennt. Budgeterschöpfung wird kontrolliert behandelt. | | `STACK-014` | Laufzeitabhängige GENERIC-/DEFERRED-Konkretisierung MUSS als gültiger Modellzustand erhalten und über passenden Kontext geprüft werden. | Geltend | [Vertrag](../design/transfer-validation-database.md#basketabhängige-domänenkonkretisierung) | Gültige abstrakte Domäne kompiliert; fehlende oder ungültige Transferkonkretisierung wird separat diagnostiziert. | | `STACK-015` | Konfigurations- und CRS-/Übersetzungs-Overlays MÜSSEN explizit, unveränderlich und Teil des Plan-Fingerprints sein. | Geltend | [Vertrag](../design/transfer-validation-database.md) | Verschiedene Overlays teilen keine inkorrekten Pläne oder Caches. | diff --git a/docs/design/transfer-validation-database.md b/docs/design/transfer-validation-database.md index bfba38c0..b810b239 100644 --- a/docs/design/transfer-validation-database.md +++ b/docs/design/transfer-validation-database.md @@ -216,7 +216,9 @@ Die Schnittstelle `GeometryOperations` kapselt die konkret gewählte Bibliothek. Teure vorbereitete Geometrien oder räumliche Indizes dürfen wiederverwendet werden, wenn die jeweilige Bibliothek dies für unveränderte Eingaben erlaubt. Konvertierung, Normalisierung und Validierung werden nicht für jede identische Teilprüfung blind wiederholt. Jede Optimierung muss Ergebnisgleichheit einschliesslich Fehlerfällen nachweisen. -`ResourceBudget` enthält mindestens Gesamtziel, maximalen Heapanteil, Cachegrenzen, temporären Speicherplatz und Grenzen einzelner Objekte beziehungsweise Geometrien. Ein Betriebsprofil mit höchstens **64 GB Gesamt-RAM** ist ein explizites Ziel für grosse Validierungen, keine bereits erreichte Leistungszusage. Die API verwendet Bytewerte; bis zu einer ausdrücklich anderen Festlegung gilt konservativ `64_000_000_000` Byte. GB und GiB dürfen nicht unbemerkt gleichgesetzt werden. Der Heap darf nicht einfach auf das gesamte Budget gesetzt werden, wenn native Speicheranteile, Threads, Direct Buffers oder weitere Prozesse ebenfalls Platz benötigen. Der konkrete Host reserviert eine gemessene Sicherheitsmarge und stoppt kontrolliert vor einer absehbaren Überschreitung; `OutOfMemoryError` ist keine reguläre Budgetstrategie. +Das Standardbetriebsprofil strebt repräsentative produktive Datenprüfungen, einschliesslich grösserer Datensätze, mit **`-Xmx2g` (2 × 2^30 = 2'147'483'648 Byte Java-Heap)** an. Das Ziel ist anhand repräsentativer Lasten zu überprüfen und gilt ohne universelle Leistungs- oder Kapazitätsgarantie. Fachlich extrem komplexe Eingaben dürfen nach Begründung höhere Budgets benötigen. + +Ein künftiges `ResourceBudget` führt den Java-Heap, den Gesamtprozessspeicher (RSS), native Allokationen, Cachegrenzen und den temporären Spitzenplatz als getrennte Grössen. Für einzelne Objekte und Geometrien gelten eigene Grenzen. I/O und Laufzeit werden separat gemessen. Der Host plant eine gemessene Sicherheitsmarge ein und stoppt kontrolliert vor einer absehbaren Überschreitung. Transferweite Indizes benötigen eine `IndexStore`-Abstraktion mit kleinen In-Memory- und später bedarfsgerecht ausgelagerten Implementierungen. Die Spezifikation schreibt keinen verteilten Cluster vor. Eine lokale, nachvollziehbare Spill-Strategie wird erst anhand realer Datenmengen gewählt und muss Datentypen, Ordnung und Identitäten verlustfrei erhalten. From 0522331ea68142d83c10a36f0fc2f5dc1527b8f3 Mon Sep 17 00:00:00 2001 From: Stefan Ziegler Date: Fri, 6 Nov 2026 18:53:30 +0100 Subject: [PATCH 2/6] Project checked schemas into neutral transfer plans Register interlis-transfer with a model-only production module path. Preserve snapshot-bound names, effective slots, object and basket rules, and bound generic alternatives. Unsupported transfer features return qualified issues. Jenesis 0.18.1 with Java 25.0.2 ran six planner contracts without failures or skips. Independent OpenCode 2.2.5 / deepseek/deepseek-flash gate PASS. Its counterprobe covered association slots, primitive BAG typing, translation identity, topic inheritance and explicit custom line-form rejection. Transient declarations remain schema metadata; wire layout is adapter scope. --- build/interlis/JvmBuildInputs.java | 3 +- build/interlis/ModulePathContract.java | 10 +- docs/architecture.md | 3 +- docs/development/build.md | 2 +- .../build.jenesis/test-inputs.properties | 4 + interlis-transfer/pom.xml | 41 +++ .../guru/interlis/transfer/AttributeSlot.java | 37 +++ .../interlis/transfer/ModelDescriptor.java | 31 ++ .../interlis/transfer/ObjectDescriptor.java | 51 +++ .../java/guru/interlis/transfer/RoleSlot.java | 25 ++ .../interlis/transfer/TopicDescriptor.java | 40 +++ .../guru/interlis/transfer/TransferName.java | 21 ++ .../guru/interlis/transfer/TransferPlan.java | 148 +++++++++ .../interlis/transfer/TransferPlanner.java | 310 ++++++++++++++++++ .../transfer/TransferPlanningResult.java | 50 +++ .../guru/interlis/transfer/TransferSlot.java | 16 + .../src/main/java/module-info.java | 5 + .../transfer/TransferPlannerContractTest.java | 255 ++++++++++++++ ...TransferPlannerUnsupportedFeatureTest.java | 35 ++ .../transfer/TransferTestSchemas.java | 85 +++++ pom.xml | 1 + 21 files changed, 1169 insertions(+), 4 deletions(-) create mode 100644 interlis-transfer/build.jenesis/test-inputs.properties create mode 100644 interlis-transfer/pom.xml create mode 100644 interlis-transfer/src/main/java/guru/interlis/transfer/AttributeSlot.java create mode 100644 interlis-transfer/src/main/java/guru/interlis/transfer/ModelDescriptor.java create mode 100644 interlis-transfer/src/main/java/guru/interlis/transfer/ObjectDescriptor.java create mode 100644 interlis-transfer/src/main/java/guru/interlis/transfer/RoleSlot.java create mode 100644 interlis-transfer/src/main/java/guru/interlis/transfer/TopicDescriptor.java create mode 100644 interlis-transfer/src/main/java/guru/interlis/transfer/TransferName.java create mode 100644 interlis-transfer/src/main/java/guru/interlis/transfer/TransferPlan.java create mode 100644 interlis-transfer/src/main/java/guru/interlis/transfer/TransferPlanner.java create mode 100644 interlis-transfer/src/main/java/guru/interlis/transfer/TransferPlanningResult.java create mode 100644 interlis-transfer/src/main/java/guru/interlis/transfer/TransferSlot.java create mode 100644 interlis-transfer/src/main/java/module-info.java create mode 100644 interlis-transfer/src/test/java/guru/interlis/transfer/TransferPlannerContractTest.java create mode 100644 interlis-transfer/src/test/java/guru/interlis/transfer/TransferPlannerUnsupportedFeatureTest.java create mode 100644 interlis-transfer/src/test/java/guru/interlis/transfer/TransferTestSchemas.java diff --git a/build/interlis/JvmBuildInputs.java b/build/interlis/JvmBuildInputs.java index ef32fb77..8ea27d08 100644 --- a/build/interlis/JvmBuildInputs.java +++ b/build/interlis/JvmBuildInputs.java @@ -12,7 +12,8 @@ public final class JvmBuildInputs { public static final List MODULES = List.of("interlis-model", "interlis-compiler", "interlis-workspace", "interlis-repository", "interlis-projection", "interlis-documentation", - "interlis-wasm-bridge", "interlis-compiler-cli", "interlis-lsp", "interlis-compiler-gui", "interlis-verification"); + "interlis-transfer", "interlis-wasm-bridge", "interlis-compiler-cli", "interlis-lsp", + "interlis-compiler-gui", "interlis-verification"); public static void requireProject(Path root) throws Exception { if (Runtime.version().feature() != 25) throw new IllegalStateException("Java25 required"); diff --git a/build/interlis/ModulePathContract.java b/build/interlis/ModulePathContract.java index cc6c8aeb..93228025 100644 --- a/build/interlis/ModulePathContract.java +++ b/build/interlis/ModulePathContract.java @@ -25,7 +25,14 @@ public static void run(Path root)throws Exception { var result=JenesisBridge.moduleResult(outcome,project); if(!result.explicitDescriptor()||!result.moduleName().orElseThrow().equals(name(project)))throw new IllegalStateException("Wrong own module: "+project); own.add(name(project)); - for(var artifact:ModuleArtifacts.runtime(root,project,outcome)) { + var runtime=ModuleArtifacts.runtime(root,project,outcome); + if(project.equals("interlis-transfer")) { + String prefix="maven/guru.interlis/", version="/"+JvmBuildInputs.projectVersion(root); + var expected=Set.of(prefix+"interlis-transfer"+version,prefix+"interlis-model"+version); + var actual=runtime.stream().map(ModuleArtifacts.Artifact::coordinates).collect(java.util.stream.Collectors.toSet()); + if(!actual.equals(expected))throw new IllegalStateException("Unexpected transfer runtime closure: "+actual); + } + for(var artifact:runtime) { Path prior=files.putIfAbsent(artifact.fileName(),artifact.file()); if(prior!=null&&!JvmBuildInputs.sha256(prior).equals(artifact.sha256()))throw new IllegalStateException("Conflicting module-path artifact"); } @@ -63,6 +70,7 @@ public static void run(Path root)throws Exception { "guru.interlis.repository","guru.interlis.repository.RepositoryClient", "guru.interlis.projection","guru.interlis.projection.DiagramProjector", "guru.interlis.documentation","guru.interlis.documentation.DocumentationProjector", + "guru.interlis.transfer","guru.interlis.transfer.TransferPlanner", "guru.interlis.wasm.bridge","guru.interlis.bridge.BridgeDispatcher"); String extra=String.join("\n",own.stream().filter(entrypoints::containsKey).map(name-> "if (named && !"+entrypoints.get(name)+".class.getModule().getName().equals(\""+name+"\")) throw new AssertionError(\"Wrong public entrypoint module\");").toList()); diff --git a/docs/architecture.md b/docs/architecture.md index 3a0b4a92..81419f46 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -20,13 +20,14 @@ Die zugehörigen POMs beschreiben die konkreten Abhängigkeiten und Scopes. | `interlis-repository` | workspace, model | Datei-/HTTP-Host, Indexe, Beschaffung, Cache und Lock | | `interlis-projection` | compiler, model | Diagramm-DTOs und reine Projektion | | `interlis-documentation` | model, projection | Lesende Dokumentationsprojektion und Rendering | +| `interlis-transfer` | model | Formatunabhängige Transferpläne aus gültigen Schemasichten | | `interlis-compiler-cli` | compiler, model, repository, workspace, projection | CLI, Prozessgrenzen, Präsentation und Ausgabe | | `interlis-compiler-gui` | model, compiler, workspace, repository | Desktop-Modellprüfung, Diagnosen, Abhängigkeiten und Repositoryexplorer | | `interlis-lsp` | workspace, compiler, projection, model, repository, documentation | LSP-/Editorhost, Scheduling und Ausgaben | | `interlis-wasm-bridge` | workspace, compiler, projection, model | Gemeinsames JSON-Protokoll und Sitzungen | | `interlis-verification` | compiler, model, workspace, repository, wasm-bridge | Prüfwerkzeuge ausserhalb der SDK-Auslieferung | -Alle elf JVM-Module werden über denselben Builder erzeugt. Die +Alle zwölf JVM-Module werden über denselben Builder erzeugt. Die jeweils ausgewählten JPMS-Closures stehen im [Buildvertrag](development/build.md). TeaVM und Web Image konsumieren denselben expliziten JVM-Snapshot. Browser und Editor teilen den neutralen diff --git a/docs/development/build.md b/docs/development/build.md index d21a4bff..0e29fd6b 100644 --- a/docs/development/build.md +++ b/docs/development/build.md @@ -1,6 +1,6 @@ # Build und Toolchains -Jenesis 0.18.1 erzeugt die elf JVM-Module mit Java 25 ohne Preview. Die POMs +Jenesis 0.18.1 erzeugt die zwölf JVM-Module mit Java 25 ohne Preview. Die POMs definieren Group, Artifact, Version, Scopes und Pins. Der Upstreampin gehört zu den [Jenesis-Quellen](../../build/JENESIS-UPSTREAM.json). Der gepinnte Gradle-Wrapper baut ausschliesslich TeaVM und GraalVM Web Image. diff --git a/interlis-transfer/build.jenesis/test-inputs.properties b/interlis-transfer/build.jenesis/test-inputs.properties new file mode 100644 index 00000000..655b369e --- /dev/null +++ b/interlis-transfer/build.jenesis/test-inputs.properties @@ -0,0 +1,4 @@ +input.0=testdata/models/base24.ili +input.1=testdata/models/survey24.ili +input.2=testdata/models/legacy23.ili +input.3=testdata/reference/iox/ili23/Reference23.ili diff --git a/interlis-transfer/pom.xml b/interlis-transfer/pom.xml new file mode 100644 index 00000000..ad469dde --- /dev/null +++ b/interlis-transfer/pom.xml @@ -0,0 +1,41 @@ + + + 4.0.0 + guru.interlisinterlis-java0.2.0-SNAPSHOT + interlis-transfer + + guru.interlisinterlis-model${project.version} + guru.interlisinterlis-compiler${project.version}test + org.junit.jupiterjunit-jupitertest + org.junit.platformjunit-platform-launchertest + + + diff --git a/interlis-transfer/src/main/java/guru/interlis/transfer/AttributeSlot.java b/interlis-transfer/src/main/java/guru/interlis/transfer/AttributeSlot.java new file mode 100644 index 00000000..b910198d --- /dev/null +++ b/interlis-transfer/src/main/java/guru/interlis/transfer/AttributeSlot.java @@ -0,0 +1,37 @@ +package guru.interlis.transfer; + +import guru.interlis.model.AttributeSymbol; +import guru.interlis.model.Cardinality; +import guru.interlis.model.EffectiveMember; +import guru.interlis.model.SymbolRef; +import guru.interlis.model.TypeRef; + +import java.util.Objects; + +/** + * An effective attribute in a plan object type. All references belong to the plan snapshot. + * + * @param symbol attribute declaration + * @param name attribute names and original identity + * @param member effective position and provenance + * @param attribute bound declaration with source flags and override identity + * @param effectiveType resolved type reference + * @param cardinality effective value multiplicity + */ +public record AttributeSlot(SymbolRef symbol, TransferName name, EffectiveMember member, + AttributeSymbol attribute, TypeRef effectiveType, Cardinality cardinality) + implements TransferSlot { + public AttributeSlot { + Objects.requireNonNull(symbol, "symbol"); + Objects.requireNonNull(name, "name"); + Objects.requireNonNull(member, "member"); + Objects.requireNonNull(attribute, "attribute"); + Objects.requireNonNull(effectiveType, "effectiveType"); + Objects.requireNonNull(cardinality, "cardinality"); + } + + /** Returns the declaration's type, including any declared domain reference. */ + public TypeRef declaredType() { + return new TypeRef(symbol.snapshot(), attribute.declaredType()); + } +} diff --git a/interlis-transfer/src/main/java/guru/interlis/transfer/ModelDescriptor.java b/interlis-transfer/src/main/java/guru/interlis/transfer/ModelDescriptor.java new file mode 100644 index 00000000..8ca74a4a --- /dev/null +++ b/interlis-transfer/src/main/java/guru/interlis/transfer/ModelDescriptor.java @@ -0,0 +1,31 @@ +package guru.interlis.transfer; + +import guru.interlis.model.LanguageVersion; +import guru.interlis.model.SymbolRef; + +import java.util.List; +import java.util.Objects; + +/** + * One model in the checked transfer closure. References belong to the plan snapshot. + * + * @param symbol model declaration + * @param name model names and original identity + * @param languageVersion INTERLIS language version of the declaration + * @param modelVersion version string declared by the model + * @param xmlNamespace effective XML namespace published by the schema + * @param incrementalTransferAllowed whether the model permits incremental transfers + * @param topics immutable list of model topics in declaration order + */ +public record ModelDescriptor(SymbolRef symbol, TransferName name, LanguageVersion languageVersion, + String modelVersion, String xmlNamespace, boolean incrementalTransferAllowed, + List topics) { + public ModelDescriptor { + Objects.requireNonNull(symbol, "symbol"); + Objects.requireNonNull(name, "name"); + Objects.requireNonNull(languageVersion, "languageVersion"); + Objects.requireNonNull(modelVersion, "modelVersion"); + Objects.requireNonNull(xmlNamespace, "xmlNamespace"); + topics = List.copyOf(topics); + } +} diff --git a/interlis-transfer/src/main/java/guru/interlis/transfer/ObjectDescriptor.java b/interlis-transfer/src/main/java/guru/interlis/transfer/ObjectDescriptor.java new file mode 100644 index 00000000..dde36880 --- /dev/null +++ b/interlis-transfer/src/main/java/guru/interlis/transfer/ObjectDescriptor.java @@ -0,0 +1,51 @@ +package guru.interlis.transfer; + +import guru.interlis.model.Cardinality; +import guru.interlis.model.Modifier; +import guru.interlis.model.OidRule; +import guru.interlis.model.SymbolKind; +import guru.interlis.model.SymbolRef; + +import java.util.List; +import java.util.Objects; +import java.util.Optional; +import java.util.Set; + +/** + * A class, structure or association in the checked closure. References belong to + * the plan snapshot. Each role belongs to its semantic association. + * + * @param symbol object type declaration + * @param name object type names and original identity + * @param model declaring model + * @param topic declaring topic, if present + * @param kind class, structure or association kind + * @param baseType immediate base declaration, if present + * @param slots immutable attributes and roles in effective member order + * @param oidRules immutable effective OID rules with their original provenance + * @param modifiers immutable declaration modifiers + * @param associationCardinality effective link multiplicity for an association, empty for other kinds + */ +public record ObjectDescriptor(SymbolRef symbol, TransferName name, SymbolRef model, + Optional topic, SymbolKind kind, Optional baseType, + List slots, List oidRules, Set modifiers, + Optional associationCardinality) { + public ObjectDescriptor { + Objects.requireNonNull(symbol, "symbol"); + Objects.requireNonNull(name, "name"); + Objects.requireNonNull(model, "model"); + Objects.requireNonNull(topic, "topic"); + Objects.requireNonNull(kind, "kind"); + Objects.requireNonNull(baseType, "baseType"); + slots = List.copyOf(slots); + oidRules = List.copyOf(oidRules); + modifiers = Set.copyOf(modifiers); + Objects.requireNonNull(associationCardinality, "associationCardinality"); + } + + /** Whether the type can occur as a concrete object in a neutral basket stream. */ + public boolean transferable() { + return topic.isPresent() && !modifiers.contains(Modifier.ABSTRACT) + && (kind == SymbolKind.CLASS || kind == SymbolKind.ASSOCIATION); + } +} diff --git a/interlis-transfer/src/main/java/guru/interlis/transfer/RoleSlot.java b/interlis-transfer/src/main/java/guru/interlis/transfer/RoleSlot.java new file mode 100644 index 00000000..9dc4b9f1 --- /dev/null +++ b/interlis-transfer/src/main/java/guru/interlis/transfer/RoleSlot.java @@ -0,0 +1,25 @@ +package guru.interlis.transfer; + +import guru.interlis.model.EffectiveMember; +import guru.interlis.model.RoleSymbol; +import guru.interlis.model.SymbolRef; + +import java.util.Objects; + +/** + * An effective association role. References in the role declaration belong to the plan snapshot. + * + * @param symbol role declaration + * @param name role names and original identity + * @param member effective position and provenance + * @param role bound association end, including targets, restrictions, cardinality and ordering + */ +public record RoleSlot(SymbolRef symbol, TransferName name, EffectiveMember member, RoleSymbol role) + implements TransferSlot { + public RoleSlot { + Objects.requireNonNull(symbol, "symbol"); + Objects.requireNonNull(name, "name"); + Objects.requireNonNull(member, "member"); + Objects.requireNonNull(role, "role"); + } +} diff --git a/interlis-transfer/src/main/java/guru/interlis/transfer/TopicDescriptor.java b/interlis-transfer/src/main/java/guru/interlis/transfer/TopicDescriptor.java new file mode 100644 index 00000000..9396192f --- /dev/null +++ b/interlis-transfer/src/main/java/guru/interlis/transfer/TopicDescriptor.java @@ -0,0 +1,40 @@ +package guru.interlis.transfer; + +import guru.interlis.model.OidRule; +import guru.interlis.model.SymbolRef; +import guru.interlis.model.TopicSymbol; + +import java.util.List; +import java.util.Objects; +import java.util.Optional; + +/** + * Topic and basket rules. References, including IDs within model descriptors, + * belong to the plan snapshot. + * + * @param symbol topic declaration + * @param name topic names and original identity + * @param model declaring model + * @param baseTopic immediate base topic, if present + * @param dependencies immutable list of directly declared topic dependencies in source order + * @param objectTypes immutable list of object type references in plan order + * @param oidRules immutable effective OID rules, with the basket rule before the object default + * @param genericBindings immutable effective generic bindings in first-use order + * @param abstractTopic whether the topic is abstract + */ +public record TopicDescriptor(SymbolRef symbol, TransferName name, SymbolRef model, + Optional baseTopic, List dependencies, + List objectTypes, + List oidRules, List genericBindings, + boolean abstractTopic) { + public TopicDescriptor { + Objects.requireNonNull(symbol, "symbol"); + Objects.requireNonNull(name, "name"); + Objects.requireNonNull(model, "model"); + Objects.requireNonNull(baseTopic, "baseTopic"); + dependencies = List.copyOf(dependencies); + objectTypes = List.copyOf(objectTypes); + oidRules = List.copyOf(oidRules); + genericBindings = List.copyOf(genericBindings); + } +} diff --git a/interlis-transfer/src/main/java/guru/interlis/transfer/TransferName.java b/interlis-transfer/src/main/java/guru/interlis/transfer/TransferName.java new file mode 100644 index 00000000..bd82a983 --- /dev/null +++ b/interlis-transfer/src/main/java/guru/interlis/transfer/TransferName.java @@ -0,0 +1,21 @@ +package guru.interlis.transfer; + +import guru.interlis.model.QualifiedName; +import guru.interlis.model.SymbolRef; + +import java.util.Objects; + +/** + * Names and canonical identity of a declaration within one schema snapshot. + * + * @param declared qualified name of the projected declaration + * @param transfer qualified model name chosen for the neutral transfer vocabulary + * @param canonical original declaration identity, following checked translations + */ +public record TransferName(QualifiedName declared, QualifiedName transfer, SymbolRef canonical) { + public TransferName { + Objects.requireNonNull(declared, "declared"); + Objects.requireNonNull(transfer, "transfer"); + Objects.requireNonNull(canonical, "canonical"); + } +} diff --git a/interlis-transfer/src/main/java/guru/interlis/transfer/TransferPlan.java b/interlis-transfer/src/main/java/guru/interlis/transfer/TransferPlan.java new file mode 100644 index 00000000..06cf6148 --- /dev/null +++ b/interlis-transfer/src/main/java/guru/interlis/transfer/TransferPlan.java @@ -0,0 +1,148 @@ +package guru.interlis.transfer; + +import guru.interlis.model.CompiledSchema; +import guru.interlis.model.SchemaFingerprint; +import guru.interlis.model.SnapshotId; +import guru.interlis.model.SnapshotMismatchException; +import guru.interlis.model.Symbol; +import guru.interlis.model.SymbolRef; +import guru.interlis.model.TypeDescriptor; +import guru.interlis.model.TypeRef; +import guru.interlis.model.query.QueryResult; +import guru.interlis.model.query.SemanticQueries; + +import java.util.List; +import java.util.Map; +import java.util.Optional; +import java.util.function.Function; +import java.util.stream.Collectors; + +/** + * Immutable neutral transfer vocabulary derived from one checked schema snapshot. + * Lists preserve plan order. Lookups reject handles from other snapshots. + */ +public final class TransferPlan { + private final SnapshotId snapshotId; + private final SchemaFingerprint schemaFingerprint; + private final SemanticQueries queries; + private final List rootModels; + private final List models; + private final List topics; + private final List objects; + private final Map modelsBySymbol; + private final Map topicsBySymbol; + private final Map objectsBySymbol; + + TransferPlan(CompiledSchema schema, List models, List topics, + List objects) { + snapshotId = schema.snapshotId(); + schemaFingerprint = schema.fingerprint(); + queries = schema.queries(); + rootModels = List.copyOf(schema.rootModels()); + this.models = List.copyOf(models); + this.topics = List.copyOf(topics); + this.objects = List.copyOf(objects); + modelsBySymbol = index(this.models, ModelDescriptor::symbol); + topicsBySymbol = index(this.topics, TopicDescriptor::symbol); + objectsBySymbol = index(this.objects, ObjectDescriptor::symbol); + } + + /** Returns the snapshot to which the plan and its handles belong. */ + public SnapshotId snapshotId() { + return snapshotId; + } + + /** Returns the fingerprint of the checked schema used for projection. */ + public SchemaFingerprint schemaFingerprint() { + return schemaFingerprint; + } + + /** Returns the immutable root model references in schema order. */ + public List rootModels() { + return rootModels; + } + + /** Returns the immutable model descriptors in plan order. */ + public List models() { + return models; + } + + /** Returns the immutable topic descriptors in plan order. */ + public List topics() { + return topics; + } + + /** Returns the immutable object type descriptors in plan order. */ + public List objects() { + return objects; + } + + /** + * Finds a model declaration in this plan. + * @param ref model reference from this snapshot + * @return descriptor, or empty for a reference absent from the model index + * @throws SnapshotMismatchException if the reference belongs to a different snapshot + */ + public Optional model(SymbolRef ref) { + checkSnapshot(ref.snapshot()); + return Optional.ofNullable(modelsBySymbol.get(ref)); + } + + /** + * Finds a topic declaration in this plan. + * @param ref topic reference from this snapshot + * @return descriptor, or empty for a reference absent from the topic index + * @throws SnapshotMismatchException if the reference belongs to a different snapshot + */ + public Optional topic(SymbolRef ref) { + checkSnapshot(ref.snapshot()); + return Optional.ofNullable(topicsBySymbol.get(ref)); + } + + /** + * Finds a class, structure or association declaration in this plan. + * @param ref object type reference from this snapshot + * @return descriptor, or empty for a reference absent from the object type index + * @throws SnapshotMismatchException if the reference belongs to a different snapshot + */ + public Optional object(SymbolRef ref) { + checkSnapshot(ref.snapshot()); + return Optional.ofNullable(objectsBySymbol.get(ref)); + } + + /** + * Resolves a type in the plan's checked schema. + * @param ref type reference from this snapshot + * @return type or the schema query's typed rejection + * @throws SnapshotMismatchException if the reference belongs to a different snapshot + */ + public QueryResult type(TypeRef ref) { + checkSnapshot(ref.snapshot()); + return queries.type(ref); + } + + /** + * Resolves a symbol in the plan's checked schema, including predefined symbols. + * @param ref symbol reference from this snapshot + * @return symbol or the schema query's typed rejection + * @throws SnapshotMismatchException if the reference belongs to a different snapshot + */ + public QueryResult symbol(SymbolRef ref) { + checkSnapshot(ref.snapshot()); + return queries.symbol(ref); + } + + private Map index(List descriptors, Function reference) { + return descriptors.stream().collect(Collectors.toUnmodifiableMap(descriptor -> { + SymbolRef ref = reference.apply(descriptor); + checkSnapshot(ref.snapshot()); + return ref; + }, Function.identity())); + } + + private void checkSnapshot(SnapshotId actual) { + if (!snapshotId.equals(actual)) { + throw new SnapshotMismatchException(snapshotId, actual); + } + } +} diff --git a/interlis-transfer/src/main/java/guru/interlis/transfer/TransferPlanner.java b/interlis-transfer/src/main/java/guru/interlis/transfer/TransferPlanner.java new file mode 100644 index 00000000..8104687b --- /dev/null +++ b/interlis-transfer/src/main/java/guru/interlis/transfer/TransferPlanner.java @@ -0,0 +1,310 @@ +package guru.interlis.transfer; + +import guru.interlis.model.AssociationSymbol; +import guru.interlis.model.AttributeSymbol; +import guru.interlis.model.Cardinality; +import guru.interlis.model.ClassSymbol; +import guru.interlis.model.CollectionType; +import guru.interlis.model.CompiledSchema; +import guru.interlis.model.CoordinateType; +import guru.interlis.model.DomainReferenceType; +import guru.interlis.model.GeometryType; +import guru.interlis.model.GraphicSymbol; +import guru.interlis.model.ModelSymbol; +import guru.interlis.model.Modifier; +import guru.interlis.model.Origin; +import guru.interlis.model.QualifiedName; +import guru.interlis.model.RoleSymbol; +import guru.interlis.model.StructureSymbol; +import guru.interlis.model.Symbol; +import guru.interlis.model.SymbolId; +import guru.interlis.model.SymbolRef; +import guru.interlis.model.TopicSymbol; +import guru.interlis.model.TypeDescriptor; +import guru.interlis.model.TypeId; +import guru.interlis.model.TypeRef; +import guru.interlis.model.UnsupportedSymbol; +import guru.interlis.model.ViewSymbol; +import guru.interlis.model.query.QueryResult; +import guru.interlis.model.query.SemanticQueries; +import guru.interlis.transfer.TransferPlanningResult.Feature; +import guru.interlis.transfer.TransferPlanningResult.Issue; + +import java.util.ArrayList; +import java.util.HashMap; +import java.util.HashSet; +import java.util.LinkedHashSet; +import java.util.List; +import java.util.Map; +import java.util.Objects; +import java.util.Optional; + +/** Projects the checked closure of one schema into a format-neutral transfer vocabulary. + * Association slots describe the relationship itself. Adapters select any format-specific + * placement of those slots and encode the selected names. */ +public final class TransferPlanner { + private TransferPlanner() { } + + /** Name selection for the neutral vocabulary, independent of language version. */ + public enum Names { + /** Use names of the canonical originals of translated declarations. */ + ORIGINAL, + /** Use names as declared in each model. */ + DECLARED + } + + /** Projects a nonnull checked schema using original names, without accessing host resources. */ + public static TransferPlanningResult plan(CompiledSchema schema) { + return plan(schema, Names.ORIGINAL); + } + + /** Projects a nonnull checked schema using the selected nonnull vocabulary. + * Issues are reported in deterministic model and member order. A result with issues + * carries no partial plan. Neither the schema nor its graph is modified. */ + public static TransferPlanningResult plan(CompiledSchema schema, Names names) { + return new Projection(schema.queries(), Objects.requireNonNull(names, "names")).project(schema); + } + + private static final class Projection { + private final SemanticQueries queries; + private final Names names; + private final Map originals = new HashMap<>(); + private final Map qualifiedNames = new HashMap<>(); + private final List models = new ArrayList<>(); + private final List topics = new ArrayList<>(); + private final List objects = new ArrayList<>(); + private final LinkedHashSet issues = new LinkedHashSet<>(); + + Projection(SemanticQueries queries, Names names) { + this.queries = queries; + this.names = names; + queries.graph().qualifiedNames().forEach((name, ids) -> + ids.forEach(id -> qualifiedNames.put(id, name))); + } + + TransferPlanningResult project(CompiledSchema schema) { + try { + var declarations = schema.models().stream().map(ref -> { + var path = QualifiedName.of(ref.symbol().value()); + if (known(queries.symbol(ref), path) instanceof ModelSymbol model) { + return model; + } + throw new MissingModelInformation(path); + }).toList(); + declarations.forEach(this::indexTranslations); + declarations.forEach(this::projectModel); + } catch (MissingModelInformation missing) { + issues.add(new Issue(missing.path, Feature.MISSING_MODEL_INFORMATION)); + } + if (!issues.isEmpty()) { + return new TransferPlanningResult.Unsupported(List.copyOf(issues)); + } + return new TransferPlanningResult.Ready(new TransferPlan(schema, models, topics, objects)); + } + + private void indexTranslations(ModelSymbol model) { + model.translation().ifPresent(translation -> { + for (var link : translation.correspondences()) { + if (!link.original().equals(link.translated())) { + var previous = originals.putIfAbsent(link.translated(), link.original()); + if (previous != null && !previous.equals(link.original())) { + throw new MissingModelInformation(qualifiedName(model)); + } + } + } + }); + } + + private void projectModel(ModelSymbol model) { + SymbolRef modelRef = ref(model.id()); + var modelTopics = new ArrayList(); + for (var member : model.members()) { + Symbol declaration = symbol(member); + if (declaration instanceof TopicSymbol topic) { + modelTopics.add(ref(topic.id())); + projectTopic(modelRef, topic); + } else { + projectObject(modelRef, Optional.empty(), declaration); + } + } + models.add(new ModelDescriptor(modelRef, name(model), model.languageVersion(), + model.modelVersion(), model.transferNamespace(), + model.options().incrementalTransferAllowed(), modelTopics)); + } + + private void projectTopic(SymbolRef model, TopicSymbol topic) { + SymbolRef topicRef = ref(topic.id()); + topic.members().forEach(id -> projectObject(model, Optional.of(topicRef), symbol(id))); + for (var binding : topic.genericBindings()) { + for (var alternative : binding.alternatives()) { + checkType(typeOf(ref(alternative), qualifiedName(topic)), qualifiedName(topic)); + } + } + var objectTypes = known(queries.membersOf(topicRef), qualifiedName(topic)).stream() + .filter(member -> isBasketObject(symbol(member.symbol()))) + .toList(); + topics.add(new TopicDescriptor(topicRef, name(topic), model, topic.baseTopic().map(this::ref), + topic.dependencies().stream().map(this::ref).toList(), objectTypes, + topic.header().oidRules(), topic.genericBindings(), topic.header().has(Modifier.ABSTRACT))); + } + + private boolean isBasketObject(Symbol symbol) { + return !symbol.header().has(Modifier.ABSTRACT) + && (symbol instanceof ClassSymbol || symbol instanceof AssociationSymbol association + && association.derivedFrom().isEmpty()); + } + + private void projectObject(SymbolRef model, Optional topic, Symbol declaration) { + Optional base; + Optional associationCardinality = Optional.empty(); + switch (declaration) { + case ClassSymbol cls -> base = cls.baseType(); + case StructureSymbol structure -> base = structure.baseType(); + case AssociationSymbol association -> { + base = association.baseAssociation(); + associationCardinality = Optional.of(association.cardinality()); + if (association.derivedFrom().isPresent()) { + issues.add(new Issue(qualifiedName(declaration), Feature.DERIVED_ASSOCIATION)); + return; + } + } + case ViewSymbol view -> { + if (!view.header().has(Modifier.TRANSIENT) + && topic.map(ref -> symbol(ref.symbol()).header().has(Modifier.VIEW)).orElse(false)) { + issues.add(new Issue(qualifiedName(view), Feature.VIEW)); + } + return; + } + case GraphicSymbol graphic -> { + if (!graphic.header().has(Modifier.ABSTRACT)) { + issues.add(new Issue(qualifiedName(graphic), Feature.GRAPHIC)); + } + return; + } + case UnsupportedSymbol unsupported -> { + issues.add(new Issue(qualifiedName(unsupported), Feature.MISSING_MODEL_INFORMATION)); + return; + } + default -> { return; } + } + var path = qualifiedName(declaration); + if (declaration.name().synthetic()) { + issues.add(new Issue(path, Feature.ANONYMOUS_OBJECT)); + return; + } + if (topic.isEmpty() && isBasketObject(declaration)) { + issues.add(new Issue(path, Feature.OBJECT_OUTSIDE_TOPIC)); + return; + } + SymbolRef owner = ref(declaration.id()); + var slots = new ArrayList(); + for (var member : known(queries.effectiveMembers(owner), path)) { + Symbol slot = symbol(member.declaredSymbol()); + var memberPath = path.child(slot.name().value()); + switch (slot) { + case AttributeSymbol attribute -> { + var attributeRef = ref(attribute.id()); + var type = typeOf(attributeRef, memberPath); + if (!attribute.transientAttribute()) { + checkType(type, memberPath); + } + var cardinality = type instanceof CollectionType collection ? collection.cardinality() + : Cardinality.of(attribute.mandatory() ? 1 : 0, 1); + slots.add(new AttributeSlot(attributeRef, name(attribute), member, attribute, + typeRef(type.id()), cardinality)); + } + case RoleSymbol role -> slots.add(new RoleSlot(ref(role.id()), name(role), member, role)); + default -> throw new MissingModelInformation(memberPath); + } + } + objects.add(new ObjectDescriptor(owner, name(declaration), model, topic, declaration.kind(), + base.map(this::ref), slots, declaration.header().oidRules(), declaration.header().modifiers(), + associationCardinality)); + } + + private void checkType(TypeDescriptor type, QualifiedName path) { + switch (type.kind()) { + case TEXT, BOOLEAN, NUMERIC, ENUMERATION, ALIGNMENT, FORMATTED, OID, + REFERENCE, STRUCTURE_VALUE, GENERIC, DEFERRED -> { } + case COLLECTION -> checkType(known(queries.type(typeRef(((CollectionType) type).elementType())), path), path); + case DOMAIN_REFERENCE -> checkType(typeOf(ref(((DomainReferenceType) type).targetDomain()), path), path); + case COORDINATE -> { + if (((CoordinateType) type).dimensions() < 2) { + issues.add(new Issue(path, Feature.COORDINATE_DIMENSION)); + } + } + case GEOMETRY -> checkGeometry((GeometryType) type, path); + case ERROR, PENDING -> issues.add(new Issue(path, Feature.MISSING_MODEL_INFORMATION)); + default -> issues.add(new Issue(path, Feature.VALUE_TYPE)); + } + } + + private void checkGeometry(GeometryType geometry, QualifiedName path) { + for (var form : geometry.lineForms()) { + var symbol = symbol(form); + if (!(symbol.origin() instanceof Origin.PredefinedOrigin predefined) + || !(predefined.key().equals("INTERLIS.STRAIGHTS") || predefined.key().equals("INTERLIS.ARCS"))) { + issues.add(new Issue(path, Feature.CUSTOM_LINE_FORM)); + } + } + if (geometry.lineAttributes().isPresent()) { + issues.add(new Issue(path, Feature.LINE_ATTRIBUTES)); + } + geometry.vertexDomain().ifPresent(vertex -> checkType(typeOf(ref(vertex), path), path)); + } + + private TypeDescriptor typeOf(SymbolRef symbol, QualifiedName path) { + var effective = known(queries.effectiveType(symbol), path); + return known(queries.type(typeRef(effective.effectiveType())), path); + } + + private TransferName name(Symbol symbol) { + SymbolId canonical = symbol.id(); + var visited = new HashSet(); + while (originals.containsKey(canonical)) { + if (!visited.add(canonical)) { + throw new MissingModelInformation(qualifiedName(symbol)); + } + canonical = originals.get(canonical); + } + QualifiedName declared = qualifiedName(symbol); + QualifiedName transfer = names == Names.ORIGINAL && !canonical.equals(symbol.id()) + ? qualifiedName(symbol(canonical)) : declared; + return new TransferName(declared, transfer, ref(canonical)); + } + + private QualifiedName qualifiedName(Symbol symbol) { + var indexed = qualifiedNames.get(symbol.id()); + if (indexed != null) { + return indexed; + } + // Synthetic declarations are absent from the qualified-name index. + return symbol.owner().map(owner -> qualifiedName(symbol(owner)).child(symbol.name().value())) + .orElseGet(() -> QualifiedName.of(symbol.name().value())); + } + + private Symbol symbol(SymbolId id) { + return known(queries.symbol(ref(id)), QualifiedName.of(id.value())); + } + + private SymbolRef ref(SymbolId id) { + return new SymbolRef(queries.snapshotId(), id); + } + + private TypeRef typeRef(TypeId id) { + return new TypeRef(queries.snapshotId(), id); + } + + private static T known(QueryResult result, QualifiedName path) { + return result.known().orElseThrow(() -> new MissingModelInformation(path)); + } + } + + private static final class MissingModelInformation extends RuntimeException { + private final QualifiedName path; + + MissingModelInformation(QualifiedName path) { + this.path = path; + } + } +} diff --git a/interlis-transfer/src/main/java/guru/interlis/transfer/TransferPlanningResult.java b/interlis-transfer/src/main/java/guru/interlis/transfer/TransferPlanningResult.java new file mode 100644 index 00000000..65ff2140 --- /dev/null +++ b/interlis-transfer/src/main/java/guru/interlis/transfer/TransferPlanningResult.java @@ -0,0 +1,50 @@ +package guru.interlis.transfer; + +import guru.interlis.model.QualifiedName; + +import java.util.List; +import java.util.Objects; + +/** Outcome of projecting a checked schema into the neutral transfer vocabulary. */ +public sealed interface TransferPlanningResult + permits TransferPlanningResult.Ready, TransferPlanningResult.Unsupported { + + /** A complete plan for the supported scope. */ + record Ready(TransferPlan plan) implements TransferPlanningResult { + public Ready { + Objects.requireNonNull(plan, "plan"); + } + } + + /** An immutable, nonempty list of issues in the order discovered by the planner. */ + record Unsupported(List issues) implements TransferPlanningResult { + public Unsupported { + issues = List.copyOf(issues); + if (issues.isEmpty()) { + throw new IllegalArgumentException("An unsupported projection must identify its issues"); + } + } + } + + /** One unsupported feature at a qualified declaration path in the checked schema. */ + record Issue(QualifiedName path, Feature feature) { + public Issue { + Objects.requireNonNull(path, "path"); + Objects.requireNonNull(feature, "feature"); + } + } + + /** Reasons a checked schema cannot be projected in full. */ + enum Feature { + MISSING_MODEL_INFORMATION, + ANONYMOUS_OBJECT, + OBJECT_OUTSIDE_TOPIC, + VIEW, + GRAPHIC, + DERIVED_ASSOCIATION, + VALUE_TYPE, + COORDINATE_DIMENSION, + CUSTOM_LINE_FORM, + LINE_ATTRIBUTES + } +} diff --git a/interlis-transfer/src/main/java/guru/interlis/transfer/TransferSlot.java b/interlis-transfer/src/main/java/guru/interlis/transfer/TransferSlot.java new file mode 100644 index 00000000..e10ba81f --- /dev/null +++ b/interlis-transfer/src/main/java/guru/interlis/transfer/TransferSlot.java @@ -0,0 +1,16 @@ +package guru.interlis.transfer; + +import guru.interlis.model.EffectiveMember; +import guru.interlis.model.SymbolRef; + +/** One immutable member of an object type, bound to the same snapshot as its owner. */ +public sealed interface TransferSlot permits AttributeSlot, RoleSlot { + /** Returns the effective member's declaration. */ + SymbolRef symbol(); + + /** Returns the declaration's names and original identity. */ + TransferName name(); + + /** Returns the effective position, inheritance path and override provenance. */ + EffectiveMember member(); +} diff --git a/interlis-transfer/src/main/java/module-info.java b/interlis-transfer/src/main/java/module-info.java new file mode 100644 index 00000000..3966843a --- /dev/null +++ b/interlis-transfer/src/main/java/module-info.java @@ -0,0 +1,5 @@ +/** Format-independent transfer contracts over immutable compiled schemas. */ +module guru.interlis.transfer { + requires transitive guru.interlis.model; + exports guru.interlis.transfer; +} diff --git a/interlis-transfer/src/test/java/guru/interlis/transfer/TransferPlannerContractTest.java b/interlis-transfer/src/test/java/guru/interlis/transfer/TransferPlannerContractTest.java new file mode 100644 index 00000000..614c079c --- /dev/null +++ b/interlis-transfer/src/test/java/guru/interlis/transfer/TransferPlannerContractTest.java @@ -0,0 +1,255 @@ +package guru.interlis.transfer; + +import guru.interlis.model.Cardinality; +import guru.interlis.model.CollectionKind; +import guru.interlis.model.CollectionType; +import guru.interlis.model.CompiledSchema; +import guru.interlis.model.EffectiveMember; +import guru.interlis.model.EffectiveMemberOrigin; +import guru.interlis.model.GeometryKind; +import guru.interlis.model.GeometryType; +import guru.interlis.model.LanguageVersion; +import guru.interlis.model.NumericType; +import guru.interlis.model.OidRule; +import guru.interlis.model.QualifiedName; +import guru.interlis.model.SnapshotMismatchException; +import guru.interlis.model.StructureValueType; +import guru.interlis.model.SymbolKind; +import guru.interlis.model.SymbolRef; +import guru.interlis.model.TopicSymbol; +import guru.interlis.model.TypeRef; +import guru.interlis.source.SourceRevision; +import guru.interlis.source.SourceText; +import org.junit.jupiter.api.Test; + +import java.io.IOException; +import java.math.BigDecimal; +import java.util.List; +import java.util.Optional; + +import static guru.interlis.transfer.TransferTestSchemas.compile; +import static guru.interlis.transfer.TransferTestSchemas.compileInline; +import static guru.interlis.transfer.TransferTestSchemas.fixture; +import static guru.interlis.transfer.TransferTestSchemas.ref; +import static guru.interlis.transfer.TransferTestSchemas.survey; +import static org.junit.jupiter.api.Assertions.assertEquals; +import static org.junit.jupiter.api.Assertions.assertFalse; +import static org.junit.jupiter.api.Assertions.assertInstanceOf; +import static org.junit.jupiter.api.Assertions.assertNotEquals; +import static org.junit.jupiter.api.Assertions.assertSame; +import static org.junit.jupiter.api.Assertions.assertThrows; +import static org.junit.jupiter.api.Assertions.assertTrue; + +class TransferPlannerContractTest { + @Test + void projectsEffectiveMemberOrderAndOriginAcrossTheImportClosure() throws IOException { + var schema = survey(); + var snapshot = schema.snapshotId(); + var fingerprint = schema.fingerprint(); + var graph = schema.semanticGraph(); + var plan = ready(schema); + + var survey = ref(schema, "Survey"); + var base = ref(schema, "Base"); + var data = ref(schema, "Survey", "Data"); + var common = ref(schema, "Base", "Common"); + var building = ref(schema, "Survey", "Data", "Building"); + var baseBuilding = ref(schema, "Base", "Common", "BaseBuilding"); + var ownerLink = ref(schema, "Survey", "Data", "OwnerLink"); + + assertEquals(List.of(survey), plan.rootModels()); + assertEquals(List.of(data), plan.model(survey).orElseThrow().topics()); + assertEquals(List.of(common), plan.model(base).orElseThrow().topics()); + var model = plan.model(survey).orElseThrow(); + assertEquals(LanguageVersion.ILI24, model.languageVersion()); + assertEquals("1.0", model.modelVersion()); + assertEquals("http://www.interlis.ch/xtf/2.4/Survey", model.xmlNamespace()); + var topic = plan.topic(data).orElseThrow(); + assertEquals(survey, topic.model()); + assertEquals(List.of(common), topic.dependencies()); + assertEquals(List.of(building, ownerLink), topic.objectTypes()); + + var object = plan.object(building).orElseThrow(); + assertEquals(SymbolKind.CLASS, object.kind()); + assertEquals(Optional.of(baseBuilding), object.baseType()); + assertEquals(Optional.of(data), object.topic()); + assertEquals(List.of("height", "label", "ownerName"), slotNames(object)); + var members = object.slots().stream().map(TransferSlot::member).toList(); + assertEquals(List.of(0, 1, 2), members.stream().map(EffectiveMember::position).toList()); + assertEquals(List.of(EffectiveMemberOrigin.OVERRIDDEN, EffectiveMemberOrigin.INHERITED, + EffectiveMemberOrigin.LOCAL), members.stream().map(EffectiveMember::origin).toList()); + assertEquals(List.of(true, false, false), members.stream().map(EffectiveMember::overridden).toList()); + assertEquals(List.of(building.symbol()), members.getFirst().inheritancePath()); + assertEquals(List.of(baseBuilding.symbol(), building.symbol()), members.get(1).inheritancePath()); + var height = assertInstanceOf(AttributeSlot.class, object.slots().getFirst()); + assertEquals(Optional.of(ref(schema, "Base", "Common", "BaseBuilding", "height").symbol()), + height.attribute().overrides()); + assertEquals(Cardinality.of(0, 1), height.cardinality()); + assertInstanceOf(NumericType.class, plan.type(height.effectiveType()).known().orElseThrow()); + + var uuid = ref(schema, "INTERLIS", "UUIDOID"); + assertEquals(plan.topic(common).orElseThrow().oidRules(), object.oidRules()); + assertEquals(Optional.of(uuid.symbol()), object.oidRules().getFirst().domain()); + assertTrue(plan.symbol(uuid).known().isPresent()); + assertTrue(plan.model(ref(schema, "INTERLIS")).isEmpty()); + var association = plan.object(ownerLink).orElseThrow(); + assertEquals(SymbolKind.ASSOCIATION, association.kind()); + assertEquals(List.of("building", "ownerRole"), slotNames(association)); + var ownerRole = assertInstanceOf(RoleSlot.class, association.slots().get(1)).role(); + assertEquals(List.of(baseBuilding.symbol()), ownerRole.targetRoots()); + assertEquals(Cardinality.exactly(1), ownerRole.cardinality()); + assertTrue(ownerRole.external()); + + assertThrows(UnsupportedOperationException.class, () -> plan.rootModels().clear()); + assertThrows(UnsupportedOperationException.class, () -> topic.objectTypes().clear()); + assertThrows(UnsupportedOperationException.class, () -> object.slots().clear()); + assertSame(graph, schema.semanticGraph()); + assertEquals(snapshot, schema.snapshotId()); + assertEquals(snapshot, plan.snapshotId()); + assertEquals(fingerprint, schema.fingerprint()); + assertEquals(fingerprint, plan.schemaFingerprint()); + } + + @Test + void retainsStructureListsAssociationRolesAndExactGeometryFromTheReferenceModel() throws IOException { + var schema = compile("Reference23", fixture("reference/iox/ili23/Reference23.ili")); + var plan = ready(schema); + var records = ref(schema, "Reference23", "Records"); + var detail = ref(schema, "Reference23", "Records", "Detail"); + var parent = ref(schema, "Reference23", "Records", "Parent"); + var item = ref(schema, "Reference23", "Records", "Item"); + var link = ref(schema, "Reference23", "Records", "ParentLink"); + var parcel = ref(schema, "Reference23", "Records", "Parcel"); + var model = plan.model(ref(schema, "Reference23")).orElseThrow(); + assertEquals(LanguageVersion.ILI23, model.languageVersion()); + assertEquals("2026-11-06", model.modelVersion()); + assertEquals("http://www.interlis.ch/INTERLIS2.3", model.xmlNamespace()); + assertEquals(List.of(parent, item, link, parcel), plan.topic(records).orElseThrow().objectTypes()); + + var structure = plan.object(detail).orElseThrow(); + assertEquals(SymbolKind.STRUCTURE, structure.kind()); + assertFalse(structure.transferable()); + assertEquals(List.of("description"), slotNames(structure)); + var itemObject = plan.object(item).orElseThrow(); + assertEquals(List.of("code", "details"), slotNames(itemObject)); + var details = assertInstanceOf(AttributeSlot.class, itemObject.slots().get(1)); + var list = assertInstanceOf(CollectionType.class, plan.type(details.effectiveType()).known().orElseThrow()); + assertEquals(CollectionKind.LIST, list.collectionKind()); + assertEquals(Cardinality.unboundedFrom(0), details.cardinality()); + var element = assertInstanceOf(StructureValueType.class, + plan.type(new TypeRef(plan.snapshotId(), list.elementType())).known().orElseThrow()); + assertEquals(List.of(detail.symbol()), element.targetRoots()); + + var association = plan.object(link).orElseThrow(); + assertEquals(List.of("parent", "children"), slotNames(association)); + var children = assertInstanceOf(RoleSlot.class, association.slots().get(1)).role(); + assertEquals(List.of(item.symbol()), children.targetRoots()); + assertEquals(Cardinality.unboundedFrom(0), children.cardinality()); + + var shape = assertInstanceOf(AttributeSlot.class, plan.object(parcel).orElseThrow().slots().getFirst()); + var area = assertInstanceOf(GeometryType.class, plan.type(shape.effectiveType()).known().orElseThrow()); + assertEquals(GeometryKind.AREA, area.geometryKind()); + assertEquals(List.of(ref(schema, "INTERLIS", "STRAIGHTS").symbol(), ref(schema, "INTERLIS", "ARCS").symbol()), + area.lineForms()); + assertTrue(area.withoutOverlaps()); + var tolerance = assertInstanceOf(GeometryType.Explicit.class, area.overlap().orElseThrow()).value(); + assertEquals("0.01", tolerance.lexeme()); + assertEquals(new BigDecimal("0.01"), tolerance.exactValue()); + } + + @Test + void keepsDeclaredTransferAndCanonicalNamesSeparateForTranslations() { + var schema = compileInline("Target", """ + INTERLIS 2.3; + MODEL Source AT "https://example.interlis.ch" VERSION "1.0" = + TOPIC T = CLASS A = a : TEXT*10; END A; END T; + END Source. + MODEL Target (de) AT "https://example.interlis.ch" VERSION "1.0" TRANSLATION OF Source ["1.0"] = + TOPIC U = CLASS B = b : TEXT*10; END B; END U; + END Target. + """); + var originalPlan = ready(schema); + var declaredPlan = assertInstanceOf(TransferPlanningResult.Ready.class, + TransferPlanner.plan(schema, TransferPlanner.Names.DECLARED)).plan(); + var translatedRef = ref(schema, "Target", "U", "B"); + var originalRef = ref(schema, "Source", "T", "A"); + var original = originalPlan.object(translatedRef).orElseThrow(); + var declared = declaredPlan.object(translatedRef).orElseThrow(); + assertNotEquals(originalRef, translatedRef); + assertEquals(translatedRef, original.symbol()); + assertEquals(new TransferName(QualifiedName.of("Target", "U", "B"), QualifiedName.of("Source", "T", "A"), + originalRef), original.name()); + assertEquals(new TransferName(QualifiedName.of("Target", "U", "B"), QualifiedName.of("Target", "U", "B"), + originalRef), declared.name()); + assertEquals(new TransferName(QualifiedName.of("Target", "U", "B", "b"), + QualifiedName.of("Source", "T", "A", "a"), ref(schema, "Source", "T", "A", "a")), + original.slots().getFirst().name()); + assertEquals(QualifiedName.of("Target", "U", "B", "b"), declared.slots().getFirst().name().transfer()); + assertEquals(originalPlan.schemaFingerprint(), declaredPlan.schemaFingerprint()); + } + + @Test + void retainsSeparateBasketOidRulesAndDeferredDomainAlternatives() { + var schema = compileInline("GenericExample", """ + INTERLIS 2.4; + MODEL GenericExample AT "https://example.org" VERSION "1" = + DOMAIN BasketId = OID TEXT*20; ObjectId = OID TEXT*20; + G (GENERIC) = COORD NUMERIC, NUMERIC; + A = COORD 0..100,0..100; B = COORD 200..300,200..300; + CONTEXT Default = G = A OR B; + TOPIC T = + BASKET OID AS BasketId; OID AS ObjectId; + DEFERRED GENERICS G; + CLASS C = NO OID; p: G; END C; + END T; + TOPIC U = CLASS D = END D; END U; + END GenericExample. + """); + var plan = ready(schema); + var topic = plan.topic(ref(schema, "GenericExample", "T")).orElseThrow(); + assertEquals(List.of(true, false), topic.oidRules().stream().map(OidRule::basket).toList()); + assertEquals(List.of(ref(schema, "GenericExample", "BasketId").symbol(), + ref(schema, "GenericExample", "ObjectId").symbol()), topic.oidRules().stream() + .map(rule -> rule.domain().orElseThrow()).toList()); + var rules = plan.object(ref(schema, "GenericExample", "T", "C")).orElseThrow().oidRules(); + assertEquals(1, rules.size()); + assertFalse(rules.getFirst().basket()); + assertEquals(Optional.empty(), rules.getFirst().domain()); + assertEquals(List.of(), plan.object(ref(schema, "GenericExample", "U", "D")).orElseThrow().oidRules()); + + assertEquals(List.of(new TopicSymbol.GenericBinding(ref(schema, "GenericExample", "G").symbol(), + List.of(ref(schema, "GenericExample", "A").symbol(), ref(schema, "GenericExample", "B").symbol()), + List.of(ref(schema, "GenericExample", "Default").symbol()), TopicSymbol.BindingState.DEFERRED)), + topic.genericBindings()); + } + + @Test + void rejectsReferencesFromAnotherRevisionBeforeLookingUpTheId() throws IOException { + var source = fixture("models/legacy23.ili"); + var schema = compile("Legacy", source); + var foreignSchema = compile("Legacy", new SourceText(source.id(), new SourceRevision(2), source.text())); + var plan = ready(schema); + var foreignPlan = ready(foreignSchema); + var item = ref(schema, "Legacy", "Data", "Item"); + var foreignItem = ref(foreignSchema, "Legacy", "Data", "Item"); + assertNotEquals(plan.snapshotId(), foreignPlan.snapshotId()); + assertEquals(List.of("label", "count"), slotNames(plan.object(item).orElseThrow())); + assertThrows(SnapshotMismatchException.class, () -> plan.object(foreignItem)); + assertThrows(SnapshotMismatchException.class, + () -> plan.object(new SymbolRef(foreignPlan.snapshotId(), item.symbol()))); + assertThrows(SnapshotMismatchException.class, () -> plan.symbol(foreignItem)); + assertThrows(SnapshotMismatchException.class, () -> plan.topic(ref(foreignSchema, "Legacy", "Data"))); + assertThrows(SnapshotMismatchException.class, () -> plan.model(ref(foreignSchema, "Legacy"))); + var foreignCount = assertInstanceOf(AttributeSlot.class, foreignPlan.object(foreignItem).orElseThrow() + .slots().get(1)); + assertThrows(SnapshotMismatchException.class, () -> plan.type(foreignCount.effectiveType())); + } + + private static TransferPlan ready(CompiledSchema schema) { + return assertInstanceOf(TransferPlanningResult.Ready.class, TransferPlanner.plan(schema)).plan(); + } + + private static List slotNames(ObjectDescriptor object) { + return object.slots().stream().map(slot -> slot.name().declared().last()).toList(); + } +} diff --git a/interlis-transfer/src/test/java/guru/interlis/transfer/TransferPlannerUnsupportedFeatureTest.java b/interlis-transfer/src/test/java/guru/interlis/transfer/TransferPlannerUnsupportedFeatureTest.java new file mode 100644 index 00000000..f6772e03 --- /dev/null +++ b/interlis-transfer/src/test/java/guru/interlis/transfer/TransferPlannerUnsupportedFeatureTest.java @@ -0,0 +1,35 @@ +package guru.interlis.transfer; + +import guru.interlis.model.QualifiedName; +import org.junit.jupiter.api.Test; + +import java.util.List; + +import static guru.interlis.transfer.TransferTestSchemas.compileInline; +import static org.junit.jupiter.api.Assertions.assertEquals; +import static org.junit.jupiter.api.Assertions.assertInstanceOf; +import static org.junit.jupiter.api.Assertions.assertThrows; + +class TransferPlannerUnsupportedFeatureTest { + @Test + void rejectsAValidSchemaWithABlackboxSlotAtItsQualifiedPath() { + var schema = compileInline("Payload", """ + INTERLIS 2.4; + MODEL Payload AT "https://example.org" VERSION "1" = + DOMAIN Raw = BLACKBOX XML; + TOPIC Data = + CLASS Entry = + label : TEXT*20; + body : Raw; + END Entry; + END Data; + END Payload. + """); + var fingerprint = schema.fingerprint(); + var result = assertInstanceOf(TransferPlanningResult.Unsupported.class, TransferPlanner.plan(schema)); + assertEquals(List.of(new TransferPlanningResult.Issue(QualifiedName.of("Payload", "Data", "Entry", "body"), + TransferPlanningResult.Feature.VALUE_TYPE)), result.issues()); + assertThrows(UnsupportedOperationException.class, () -> result.issues().clear()); + assertEquals(fingerprint, schema.fingerprint()); + } +} diff --git a/interlis-transfer/src/test/java/guru/interlis/transfer/TransferTestSchemas.java b/interlis-transfer/src/test/java/guru/interlis/transfer/TransferTestSchemas.java new file mode 100644 index 00000000..98535b21 --- /dev/null +++ b/interlis-transfer/src/test/java/guru/interlis/transfer/TransferTestSchemas.java @@ -0,0 +1,85 @@ +package guru.interlis.transfer; + +import guru.interlis.compiler.CompilationOutcome; +import guru.interlis.compiler.CompilationRequest; +import guru.interlis.compiler.CompilerEngines; +import guru.interlis.compiler.CompilerOptions; +import guru.interlis.compiler.ImportBinding; +import guru.interlis.compiler.ImportResolution; +import guru.interlis.compiler.ImportSite; +import guru.interlis.compiler.ModelLocation; +import guru.interlis.compiler.ModelSelector; +import guru.interlis.compiler.ResolutionFingerprint; +import guru.interlis.compiler.SourceSet; +import guru.interlis.execution.CancellationToken; +import guru.interlis.model.CompiledSchema; +import guru.interlis.model.QualifiedName; +import guru.interlis.model.SymbolRef; +import guru.interlis.source.SourceId; +import guru.interlis.source.SourceRevision; +import guru.interlis.source.SourceText; +import guru.interlis.syntax.ParseMode; +import guru.interlis.syntax.ParseOptions; +import guru.interlis.syntax.SyntaxServices; + +import java.io.IOException; +import java.nio.file.Files; +import java.nio.file.Path; +import java.util.List; +import java.util.Map; +import java.util.Optional; + +import static org.junit.jupiter.api.Assertions.assertEquals; +import static org.junit.jupiter.api.Assertions.assertInstanceOf; + +final class TransferTestSchemas { + private static final CancellationToken TOKEN = CancellationToken.none(); + + private TransferTestSchemas() {} + + static CompiledSchema survey() throws IOException { + SourceText base = fixture("models/base24.ili"); + SourceText survey = fixture("models/survey24.ili"); + var parser = SyntaxServices.create(); + var options = ParseOptions.defaults(ParseMode.EDITOR); + var importer = parser.parse(survey, options, TOKEN).root().models().getFirst(); + var target = parser.parse(base, options, TOKEN).root().models().getFirst(); + var site = new ImportSite(importer.imports().getFirst().range(), "Survey", "Base"); + var location = new ModelLocation(base.id(), base.revision(), "Base", + Optional.of(target.modelVersion()), target.range().startOffset()); + var binding = new ImportBinding(site, new ImportResolution.Resolved(location)); + var sources = SourceSet.of(Map.of(base.id(), base, survey.id(), survey), + List.of(binding), List.of(), ResolutionFingerprint.none()); + return compile("Survey", sources); + } + + static SourceText fixture(String relativePath) throws IOException { + var path = Path.of(System.getProperty("testdata.dir")).resolve(relativePath); + return new SourceText(new SourceId(path.toUri().toString()), new SourceRevision(1), Files.readString(path)); + } + + static CompiledSchema compileInline(String model, String text) { + return compile(model, new SourceText(new SourceId("memory:/" + model + ".ili"), + new SourceRevision(1), text)); + } + + static CompiledSchema compile(String model, SourceText source) { + return compile(model, SourceSet.of(Map.of(source.id(), source))); + } + + static SymbolRef ref(CompiledSchema schema, String... path) { + var refs = schema.queries().findQualifiedName(QualifiedName.of(path)); + assertEquals(1, refs.size(), () -> "Expected one symbol for " + QualifiedName.of(path)); + return refs.getFirst(); + } + + private static CompiledSchema compile(String model, SourceSet sources) { + var request = new CompilationRequest( + List.of(new ModelSelector.ByName(model, Optional.empty(), Optional.empty())), + sources, CompilerOptions.defaults()); + var outcome = CompilerEngines.create().compile(request, TOKEN); + return assertInstanceOf(CompilationOutcome.Succeeded.class, outcome, + () -> outcome instanceof CompilationOutcome.Rejected rejected + ? rejected.analysis().requestDiagnostics().toString() : outcome.toString()).schema(); + } +} diff --git a/pom.xml b/pom.xml index 55d74ddb..a3118c0f 100644 --- a/pom.xml +++ b/pom.xml @@ -11,6 +11,7 @@ interlis-repository interlis-projection interlis-documentation + interlis-transfer interlis-wasm-bridge interlis-compiler-cli interlis-lsp From ba739aab1180fcc293540701622b9b22a0cef7b8 Mon Sep 17 00:00:00 2001 From: Stefan Ziegler Date: Fri, 6 Nov 2026 19:03:06 +0100 Subject: [PATCH 3/6] Represent transfer values, identities, and exact geometry Keep absence, undefined and empty values distinct. Preserve collection order, exact number lexemes, reference metadata, and independently supplied TID, OID and basket identity. Retain curve partitions, signed radii and surface boundaries without topology operations. Jenesis 0.18.1 on Java 25.0.2 passed 15 targeted value tests with no skips. OpenCode 2.2.5 and deepseek/deepseek-flash returned independent PASS with 31 counterprobe checks. Format-level clipping markers, XTF link placement and version-specific radius checks remain adapter scope. --- .../interlis/transfer/TransferGeometry.java | 182 ++++++++++++++++++ .../interlis/transfer/TransferIdentity.java | 69 +++++++ .../interlis/transfer/TransferObject.java | 68 +++++++ .../guru/interlis/transfer/TransferValue.java | 110 +++++++++++ .../transfer/TransferGeometryValueTest.java | 173 +++++++++++++++++ .../transfer/TransferIdentityTest.java | 106 ++++++++++ .../interlis/transfer/TransferValueTest.java | 143 ++++++++++++++ 7 files changed, 851 insertions(+) create mode 100644 interlis-transfer/src/main/java/guru/interlis/transfer/TransferGeometry.java create mode 100644 interlis-transfer/src/main/java/guru/interlis/transfer/TransferIdentity.java create mode 100644 interlis-transfer/src/main/java/guru/interlis/transfer/TransferObject.java create mode 100644 interlis-transfer/src/main/java/guru/interlis/transfer/TransferValue.java create mode 100644 interlis-transfer/src/test/java/guru/interlis/transfer/TransferGeometryValueTest.java create mode 100644 interlis-transfer/src/test/java/guru/interlis/transfer/TransferIdentityTest.java create mode 100644 interlis-transfer/src/test/java/guru/interlis/transfer/TransferValueTest.java diff --git a/interlis-transfer/src/main/java/guru/interlis/transfer/TransferGeometry.java b/interlis-transfer/src/main/java/guru/interlis/transfer/TransferGeometry.java new file mode 100644 index 00000000..217e22a8 --- /dev/null +++ b/interlis-transfer/src/main/java/guru/interlis/transfer/TransferGeometry.java @@ -0,0 +1,182 @@ +package guru.interlis.transfer; + +import guru.interlis.transfer.TransferValue.NumberValue; + +import java.util.List; +import java.util.Objects; +import java.util.Optional; +import java.util.function.ToIntFunction; + +/** + * Exact geometry values with immutable component lists in source order. Ordinates + * and radii retain their original numeric lexemes. An enclosing transfer object + * can supply the source identity, and the component lists retain each part's + * position in that object. Constructors check shape and dimensional consistency. + */ +public sealed interface TransferGeometry extends TransferValue + permits TransferGeometry.Coordinate, TransferGeometry.MultiCoordinate, + TransferGeometry.Polyline, TransferGeometry.MultiPolyline, + TransferGeometry.Surface, TransferGeometry.MultiSurface { + + /** A position with exactly two or three ordinates in model axis order. */ + record Coordinate(List ordinates) implements TransferGeometry { + public Coordinate { + ordinates = List.copyOf(ordinates); + if (ordinates.size() != 2 && ordinates.size() != 3) { + throw new IllegalArgumentException("A coordinate requires two or three ordinates"); + } + } + + private int dimensions() { + return ordinates.size(); + } + } + + /** A possibly empty sequence of positions of one dimension. */ + record MultiCoordinate(List coordinates) implements TransferGeometry { + public MultiCoordinate { + coordinates = List.copyOf(coordinates); + requireHomogeneousDimensions(coordinates, Coordinate::dimensions); + } + } + + /** + * A polyline with its parts in source order. All parts have the same dimension. + * An empty list represents a supplied line with no parts, as in an empty extract. + */ + record Polyline(List parts) implements TransferGeometry { + public Polyline { + parts = List.copyOf(parts); + requireHomogeneousDimensions(parts, CurvePart::dimensions); + } + + private int dimensions() { + return parts.isEmpty() ? 0 : parts.getFirst().dimensions(); + } + } + + /** A possibly empty sequence of polylines of one dimension. */ + record MultiPolyline(List lines) implements TransferGeometry { + public MultiPolyline { + lines = List.copyOf(lines); + requireHomogeneousDimensions(lines, Polyline::dimensions); + } + } + + /** + * One outer boundary and its inner boundaries, of the same dimension. + * The order of holes is retained. + */ + record Surface(Ring shell, List holes) implements TransferGeometry { + public Surface { + Objects.requireNonNull(shell, "shell"); + holes = List.copyOf(holes); + requireDimensions(holes, shell.dimensions(), Ring::dimensions); + } + + private int dimensions() { + return shell.dimensions(); + } + } + + /** A possibly empty sequence of surfaces of one dimension. */ + record MultiSurface(List surfaces) implements TransferGeometry { + public MultiSurface { + surfaces = List.copyOf(surfaces); + requireHomogeneousDimensions(surfaces, Surface::dimensions); + } + } + + /** + * One boundary with its original partition into one or more nonempty polylines + * of the same dimension. Closure and continuity are geometric properties + * of the supplied boundary. + */ + record Ring(List lines) { + public Ring { + lines = List.copyOf(lines); + if (lines.isEmpty() || lines.stream().anyMatch(line -> line.parts().isEmpty())) { + throw new IllegalArgumentException("A ring requires one or more nonempty polylines"); + } + requireHomogeneousDimensions(lines, Polyline::dimensions); + } + + private int dimensions() { + return lines.getFirst().dimensions(); + } + } + + /** + * A start position followed by zero or more segments in traversal order. + * Every segment starts at the preceding end position, or at {@code start} + * for the first segment. All end positions have the start's dimension. + */ + record CurvePart(Coordinate start, List segments) { + public CurvePart { + Objects.requireNonNull(start, "start"); + segments = List.copyOf(segments); + requireDimensions(segments, start.dimensions(), segment -> segment.end().dimensions()); + } + + private int dimensions() { + return start.dimensions(); + } + } + + /** A straight or circular segment within a curve part. */ + sealed interface Segment permits Straight, Arc { + /** Returns the exact end position, of the same dimension as the part. */ + Coordinate end(); + } + + /** A straight segment ending at the supplied position. */ + record Straight(Coordinate end) implements Segment { + public Straight { + Objects.requireNonNull(end, "end"); + } + } + + /** + * A circular segment with a two-dimensional intermediate position and a + * two- or three-dimensional end. An optional nonzero radius retains its + * supplied sign, including the direction encoded by an INTERLIS 2.3 radius. + */ + record Arc(Coordinate support, Coordinate end, Optional radius) implements Segment { + public Arc { + Objects.requireNonNull(support, "support"); + Objects.requireNonNull(end, "end"); + Objects.requireNonNull(radius, "radius"); + if (support.dimensions() != 2) { + throw new IllegalArgumentException("An arc support position requires two ordinates"); + } + if (radius.isPresent() && radius.orElseThrow().exactValue().signum() == 0) { + throw new IllegalArgumentException("An arc radius must be nonzero"); + } + } + } + + private static void requireHomogeneousDimensions(List values, ToIntFunction dimensions) { + // Zero means the component has no positions from which to derive a dimension. + int expected = 0; + for (T value : values) { + int actual = dimensions.applyAsInt(value); + if (expected == 0) { + expected = actual; + } else if (actual != 0) { + requireSameDimension(expected, actual); + } + } + } + + private static void requireDimensions(List values, int expected, ToIntFunction dimensions) { + for (T value : values) { + requireSameDimension(expected, dimensions.applyAsInt(value)); + } + } + + private static void requireSameDimension(int expected, int actual) { + if (expected != actual) { + throw new IllegalArgumentException("Geometry component dimensions must agree"); + } + } +} diff --git a/interlis-transfer/src/main/java/guru/interlis/transfer/TransferIdentity.java b/interlis-transfer/src/main/java/guru/interlis/transfer/TransferIdentity.java new file mode 100644 index 00000000..de8a6a23 --- /dev/null +++ b/interlis-transfer/src/main/java/guru/interlis/transfer/TransferIdentity.java @@ -0,0 +1,69 @@ +package guru.interlis.transfer; + +import guru.interlis.source.SourceId; + +import java.util.Objects; +import java.util.Optional; + +/** Identity and provenance of one object occurrence. + * The object's descriptor carries the applicable OID domain and inheritance rules. + * + * @param currentBasket basket containing this occurrence + * @param originalBasket original basket when supplied independently by the source + * @param tid object identifier scoped to the transfer, if present + * @param oid independently supplied stable object identifier, if present + * @param source opaque data source identifier, if known + */ +public record TransferIdentity(BasketId currentBasket, Optional originalBasket, + Optional tid, Optional oid, Optional source) { + public TransferIdentity { + Objects.requireNonNull(currentBasket, "currentBasket"); + Objects.requireNonNull(originalBasket, "originalBasket"); + Objects.requireNonNull(tid, "tid"); + Objects.requireNonNull(oid, "oid"); + Objects.requireNonNull(source, "source"); + } + + /** Provenance of a transfer-scoped identifier. */ + public enum IdOrigin { + /** Supplied by the transfer source. */ + SOURCE, + /** Assigned for this transfer occurrence. */ + SYNTHETIC + } + + /** Identifier of a basket in a transfer. */ + public record BasketId(String value, IdOrigin origin) { + public BasketId { + requireIdentifier(value); + Objects.requireNonNull(origin, "origin"); + } + } + + /** Identity carried by a reference. Its variant determines the scope. */ + public sealed interface ObjectKey permits TransferId, ObjectOid { + /** Returns the exact opaque identifier as supplied. */ + String value(); + } + + /** Transfer-scoped object identity with explicit source or synthetic provenance. */ + public record TransferId(String value, IdOrigin origin) implements ObjectKey { + public TransferId { + requireIdentifier(value); + Objects.requireNonNull(origin, "origin"); + } + } + + /** Stable object identity supplied separately from any transfer-scoped identifier. */ + public record ObjectOid(String value) implements ObjectKey { + public ObjectOid { + requireIdentifier(value); + } + } + + private static void requireIdentifier(String value) { + if (Objects.requireNonNull(value, "value").isEmpty()) { + throw new IllegalArgumentException("An identifier must have a value"); + } + } +} diff --git a/interlis-transfer/src/main/java/guru/interlis/transfer/TransferObject.java b/interlis-transfer/src/main/java/guru/interlis/transfer/TransferObject.java new file mode 100644 index 00000000..7761fdb0 --- /dev/null +++ b/interlis-transfer/src/main/java/guru/interlis/transfer/TransferObject.java @@ -0,0 +1,68 @@ +package guru.interlis.transfer; + +import guru.interlis.model.SnapshotMismatchException; +import guru.interlis.model.SymbolRef; + +import java.util.ArrayDeque; +import java.util.Collections; +import java.util.IdentityHashMap; +import java.util.Iterator; +import java.util.Objects; +import java.util.Optional; + +/** One immutable, owned object and its already constructed fields. Values can be retained + * after advancing a reader. The object holds only its own data and any immutable shared + * subtrees. Adapters choose input depth and size limits before constructing an object. + * Snapshot checks walk nested containers iteratively. Constructors take defensive copies + * of their inputs, so the value graph can be assembled from existing values without cycles. + * + * @param identity the identity and provenance of this occurrence + * @param data object type and supplied values from one snapshot + */ +public record TransferObject(TransferIdentity identity, TransferValue.Structured data) { + public TransferObject { + Objects.requireNonNull(identity, "identity"); + Objects.requireNonNull(data, "data"); + requireSnapshot(data); + } + + /** Returns the snapshot-bound object type. */ + public SymbolRef type() { + return data.type(); + } + + /** Returns a supplied value or empty for an absent slot. */ + public Optional value(SymbolRef slot) { + return data.value(slot); + } + + private static void requireSnapshot(TransferValue.Structured data) { + var snapshot = data.type().snapshot(); + var seen = Collections.newSetFromMap(new IdentityHashMap()); + var pending = new ArrayDeque>(); + pending.push(data.values().values().iterator()); + while (!pending.isEmpty()) { + var fields = pending.peek(); + if (!fields.hasNext()) { + pending.pop(); + continue; + } + switch (fields.next()) { + case TransferValue.Structured structure -> { + if (!snapshot.equals(structure.type().snapshot())) { + throw new SnapshotMismatchException(snapshot, structure.type().snapshot()); + } + if (seen.add(structure)) { + pending.push(structure.values().values().iterator()); + } + } + case TransferValue.Repeated repeated -> { + if (seen.add(repeated)) { + pending.push(repeated.values().iterator()); + } + } + default -> { } + } + } + } +} diff --git a/interlis-transfer/src/main/java/guru/interlis/transfer/TransferValue.java b/interlis-transfer/src/main/java/guru/interlis/transfer/TransferValue.java new file mode 100644 index 00000000..2f7aa6b6 --- /dev/null +++ b/interlis-transfer/src/main/java/guru/interlis/transfer/TransferValue.java @@ -0,0 +1,110 @@ +package guru.interlis.transfer; + +import guru.interlis.model.CollectionKind; +import guru.interlis.model.SnapshotMismatchException; +import guru.interlis.model.SymbolRef; + +import java.math.BigDecimal; +import java.math.BigInteger; +import java.util.Collections; +import java.util.LinkedHashMap; +import java.util.List; +import java.util.Map; +import java.util.Objects; +import java.util.Optional; + +/** Immutable, owned transfer value. A containing object represents an absent value by + * omitting its slot. Values and collection entries are nonnull. */ +public sealed interface TransferValue permits TransferValue.Text, TransferValue.BooleanValue, + TransferValue.Enumeration, TransferValue.NumberValue, TransferValue.Structured, + TransferValue.Repeated, TransferValue.Reference, TransferValue.Undefined, TransferGeometry { + + /** Exact text, including the empty string. */ + record Text(String value) implements TransferValue { + public Text { + Objects.requireNonNull(value, "value"); + } + } + + /** Boolean value. */ + record BooleanValue(boolean value) implements TransferValue { } + + /** An enumeration value in root-to-leaf component order. */ + record Enumeration(List path) implements TransferValue { + public Enumeration { + path = List.copyOf(path); + if (path.isEmpty() || path.stream().anyMatch(String::isBlank)) { + throw new IllegalArgumentException("An enumeration path needs nonblank components"); + } + } + } + + /** Exact decimal value and source spelling. The adapter supplies the interpretation + * of its own numeric notation. Both components participate in value equality. */ + record NumberValue(String lexeme, BigDecimal exactValue) implements TransferValue { + public NumberValue { + if (Objects.requireNonNull(lexeme, "lexeme").isBlank()) { + throw new IllegalArgumentException("A number needs its lexical spelling"); + } + Objects.requireNonNull(exactValue, "exactValue"); + } + } + + /** Fields of an object or a nested structure. The immutable index preserves the supplied + * iteration order. The plan's slots define model order. The type and every slot key belong + * to the same snapshot. Values are + * already constructed and defensively captured, so nested values can share immutable + * subtrees. TransferObject checks snapshot ownership across the complete object. */ + record Structured(SymbolRef type, Map values) implements TransferValue { + public Structured { + Objects.requireNonNull(type, "type"); + var copy = new LinkedHashMap(); + for (var field : values.entrySet()) { + checkSlot(type, field.getKey()); + copy.put(field.getKey(), Objects.requireNonNull(field.getValue(), "value")); + } + values = Collections.unmodifiableMap(copy); + } + + /** Returns a supplied value or empty for an absent slot. + * @throws SnapshotMismatchException if the key belongs to a different snapshot + */ + public Optional value(SymbolRef slot) { + checkSlot(type, slot); + return Optional.ofNullable(values.get(slot)); + } + + private static void checkSlot(SymbolRef type, SymbolRef slot) { + if (!type.snapshot().equals(slot.snapshot())) { + throw new SnapshotMismatchException(type.snapshot(), slot.snapshot()); + } + } + } + + /** BAG or LIST of primitive or structured values. The supplied order and duplicates + * are retained for both kinds. Only LIST gives that order model semantics. */ + record Repeated(CollectionKind kind, List values) implements TransferValue { + public Repeated { + Objects.requireNonNull(kind, "kind"); + values = List.copyOf(values); + } + } + + /** Reference to an object identity. Basket and position are retained when supplied. + * A position is absolute in the ordered relation within the transfer basket and is one-based. */ + record Reference(TransferIdentity.ObjectKey target, Optional targetBasket, + boolean external, Optional orderPosition) implements TransferValue { + public Reference { + Objects.requireNonNull(target, "target"); + Objects.requireNonNull(targetBasket, "targetBasket"); + Objects.requireNonNull(orderPosition, "orderPosition").ifPresent(position -> { + if (position.signum() <= 0) { + throw new IllegalArgumentException("A reference position must be positive"); + } + }); + } + } + + /** A supplied value explicitly marked undefined by the input. */ + enum Undefined implements TransferValue { INSTANCE } +} diff --git a/interlis-transfer/src/test/java/guru/interlis/transfer/TransferGeometryValueTest.java b/interlis-transfer/src/test/java/guru/interlis/transfer/TransferGeometryValueTest.java new file mode 100644 index 00000000..874b0453 --- /dev/null +++ b/interlis-transfer/src/test/java/guru/interlis/transfer/TransferGeometryValueTest.java @@ -0,0 +1,173 @@ +package guru.interlis.transfer; + +import guru.interlis.transfer.TransferGeometry.Arc; +import guru.interlis.transfer.TransferGeometry.Coordinate; +import guru.interlis.transfer.TransferGeometry.CurvePart; +import guru.interlis.transfer.TransferGeometry.MultiCoordinate; +import guru.interlis.transfer.TransferGeometry.MultiPolyline; +import guru.interlis.transfer.TransferGeometry.MultiSurface; +import guru.interlis.transfer.TransferGeometry.Polyline; +import guru.interlis.transfer.TransferGeometry.Ring; +import guru.interlis.transfer.TransferGeometry.Segment; +import guru.interlis.transfer.TransferGeometry.Straight; +import guru.interlis.transfer.TransferGeometry.Surface; +import guru.interlis.transfer.TransferValue.NumberValue; +import org.junit.jupiter.api.Test; + +import java.math.BigDecimal; +import java.util.ArrayList; +import java.util.Collections; +import java.util.List; +import java.util.Optional; + +import static org.junit.jupiter.api.Assertions.assertEquals; +import static org.junit.jupiter.api.Assertions.assertThrows; + +class TransferGeometryValueTest { + @Test + void retainsExactOrdinatesLexemesAndAxisOrder() { + var ordinates = new ArrayList<>(List.of(number("100000000000000000.001"), number("0.000000000000001"))); + var coordinate = new Coordinate(ordinates); + ordinates.clear(); + assertEquals(List.of("100000000000000000.001", "0.000000000000001"), + coordinate.ordinates().stream().map(NumberValue::lexeme).toList()); + assertEquals(List.of(new BigDecimal("100000000000000000.001"), new BigDecimal("0.000000000000001")), + coordinate.ordinates().stream().map(NumberValue::exactValue).toList()); + assertThrows(UnsupportedOperationException.class, () -> coordinate.ordinates().clear()); + assertEquals(List.of(number("1.00"), number("2.00"), number("3.00")), + coordinate("1.00", "2.00", "3.00").ordinates()); + } + + @Test + void retainsCurvePartsStraightSegmentsAndArcPointsWithSignedRadius() { + var start = coordinate("0", "0", "1.00"); + var straightEnd = coordinate("2", "0", "2.00"); + var support = coordinate("3.00000000001", "1.00000000001"); + var arcEnd = coordinate("2", "2", "3.00"); + var segments = new ArrayList(List.of(new Straight(straightEnd), + new Arc(support, arcEnd, Optional.of(number("-1.000"))))); + var part = new CurvePart(start, segments); + segments.clear(); + var next = new CurvePart(coordinate("5", "5", "6.00"), List.of()); + var parts = new ArrayList<>(List.of(part, next)); + var line = new Polyline(parts); + parts.clear(); + + assertEquals(List.of(part, next), line.parts()); + assertEquals(start, part.start()); + assertEquals(List.of(new Straight(straightEnd), new Arc(support, arcEnd, Optional.of(number("-1.000")))), + part.segments()); + var arc = (Arc) part.segments().get(1); + assertEquals(support, arc.support()); + assertEquals(arcEnd, arc.end()); + assertEquals("-1.000", arc.radius().orElseThrow().lexeme()); + assertEquals(new BigDecimal("-1.000"), arc.radius().orElseThrow().exactValue()); + assertEquals(Optional.of(number("1.000")), new Arc(support, arcEnd, Optional.of(number("1.000"))).radius()); + assertEquals(Optional.empty(), new Arc(support, arcEnd, Optional.empty()).radius()); + assertEquals(List.of(), next.segments()); + assertThrows(UnsupportedOperationException.class, () -> part.segments().clear()); + assertThrows(UnsupportedOperationException.class, () -> line.parts().clear()); + assertEquals(line, new Polyline(List.of(new CurvePart(start, List.copyOf(part.segments())), next))); + } + + @Test + void retainsRingPartitionsHoleOrderAndMultiGeometryComponents() { + var origin = coordinate("0", "0"); + var corner = coordinate("4", "4"); + var lower = line(origin, coordinate("4", "0"), corner); + var upper = line(corner, coordinate("0", "4"), origin); + var parts = new ArrayList<>(List.of(lower, upper)); + var shell = new Ring(parts); + parts.clear(); + var hole = new Ring(List.of(line(coordinate("1", "1"), coordinate("1", "2"), + coordinate("2", "2"), coordinate("1", "1")))); + var secondHole = new Ring(List.of(line(coordinate("3", "3"), coordinate("3", "3.5"), + coordinate("3.5", "3.5"), coordinate("3", "3")))); + var holes = new ArrayList<>(List.of(hole, secondHole)); + var surface = new Surface(shell, holes); + holes.clear(); + var secondSurface = new Surface(hole, List.of()); + var surfaces = new ArrayList<>(List.of(surface, secondSurface)); + var multi = new MultiSurface(surfaces); + surfaces.clear(); + + assertEquals(List.of(lower, upper), multi.surfaces().getFirst().shell().lines()); + assertEquals(List.of(hole, secondHole), surface.holes()); + assertEquals(new MultiSurface(List.of(new Surface(new Ring(List.of(lower, upper)), + List.of(hole, secondHole)), secondSurface)), multi); + assertEquals(List.of(origin, corner, origin), new MultiCoordinate(List.of(origin, corner, origin)).coordinates()); + assertEquals(List.of(upper, lower), new MultiPolyline(List.of(upper, lower)).lines()); + assertEquals(List.of(), new MultiCoordinate(List.of()).coordinates()); + assertEquals(List.of(), new MultiPolyline(List.of()).lines()); + assertEquals(List.of(), new MultiSurface(List.of()).surfaces()); + assertThrows(UnsupportedOperationException.class, () -> shell.lines().clear()); + assertThrows(UnsupportedOperationException.class, () -> surface.holes().clear()); + assertThrows(UnsupportedOperationException.class, () -> multi.surfaces().clear()); + } + + @Test + void keepsAnEmptyExtractLineWithoutAssigningADimension() { + var empty = new Polyline(List.of()); + var line2 = line(coordinate("0", "0")); + var line3 = line(coordinate("0", "0", "1")); + + assertEquals(List.of(), empty.parts()); + assertEquals(List.of(empty, line2, empty), new MultiPolyline(List.of(empty, line2, empty)).lines()); + assertEquals(List.of(line3, empty), new MultiPolyline(List.of(line3, empty)).lines()); + assertThrows(IllegalArgumentException.class, () -> new MultiPolyline(List.of(line2, empty, line3))); + assertThrows(IllegalArgumentException.class, () -> new Ring(List.of(empty))); + } + + @Test + void rejectsMissingGeometryComponents() { + var point = coordinate("0", "0"); + assertThrows(NullPointerException.class, () -> new Coordinate(null)); + assertThrows(NullPointerException.class, () -> new MultiCoordinate(Collections.singletonList(null))); + assertThrows(NullPointerException.class, () -> new Straight(null)); + assertThrows(NullPointerException.class, () -> new Arc(null, point, Optional.empty())); + assertThrows(NullPointerException.class, () -> new Arc(point, point, null)); + assertThrows(NullPointerException.class, () -> new CurvePart(point, Collections.singletonList(null))); + assertThrows(IllegalArgumentException.class, () -> new Ring(List.of())); + assertThrows(NullPointerException.class, () -> new Surface(null, List.of())); + } + + @Test + void rejectsInconsistentDimensionsAndZeroRadius() { + var xy = coordinate("0", "0"); + var xyz = coordinate("0", "0", "1"); + var line2 = line(xy); + var line3 = line(xyz); + var ring2 = new Ring(List.of(line2)); + var ring3 = new Ring(List.of(line3)); + + assertThrows(IllegalArgumentException.class, () -> coordinate("0")); + assertThrows(IllegalArgumentException.class, () -> coordinate("0", "0", "1", "1")); + assertThrows(IllegalArgumentException.class, () -> new Arc(xyz, xy, Optional.empty())); + assertThrows(IllegalArgumentException.class, () -> new Arc(xy, xyz, Optional.of(number("0.00")))); + assertThrows(IllegalArgumentException.class, () -> new CurvePart(xy, List.of(new Straight(xyz)))); + assertThrows(IllegalArgumentException.class, () -> new CurvePart(xy, List.of(new Arc(xy, xyz, Optional.empty())))); + assertThrows(IllegalArgumentException.class, () -> new Polyline(List.of(line2.parts().getFirst(), line3.parts().getFirst()))); + assertThrows(IllegalArgumentException.class, () -> new MultiCoordinate(List.of(xy, xyz))); + assertThrows(IllegalArgumentException.class, () -> new MultiPolyline(List.of(line2, line3))); + assertThrows(IllegalArgumentException.class, () -> new Ring(List.of(line2, line3))); + assertThrows(IllegalArgumentException.class, () -> new Surface(ring2, List.of(ring3))); + assertThrows(IllegalArgumentException.class, + () -> new MultiSurface(List.of(new Surface(ring2, List.of()), new Surface(ring3, List.of())))); + } + + private static Polyline line(Coordinate start, Coordinate... ends) { + List segments = new ArrayList<>(); + for (Coordinate end : ends) { + segments.add(new Straight(end)); + } + return new Polyline(List.of(new CurvePart(start, segments))); + } + + private static Coordinate coordinate(String... ordinates) { + return new Coordinate(List.of(ordinates).stream().map(TransferGeometryValueTest::number).toList()); + } + + private static NumberValue number(String lexeme) { + return new NumberValue(lexeme, new BigDecimal(lexeme)); + } +} diff --git a/interlis-transfer/src/test/java/guru/interlis/transfer/TransferIdentityTest.java b/interlis-transfer/src/test/java/guru/interlis/transfer/TransferIdentityTest.java new file mode 100644 index 00000000..3eaa8b7e --- /dev/null +++ b/interlis-transfer/src/test/java/guru/interlis/transfer/TransferIdentityTest.java @@ -0,0 +1,106 @@ +package guru.interlis.transfer; + +import guru.interlis.model.CollectionKind; +import guru.interlis.model.SnapshotId; +import guru.interlis.model.SnapshotMismatchException; +import guru.interlis.model.SymbolId; +import guru.interlis.model.SymbolRef; +import guru.interlis.source.SourceId; +import org.junit.jupiter.api.Test; + +import java.util.List; +import java.util.Map; +import java.util.Optional; + +import static guru.interlis.transfer.TransferIdentity.IdOrigin.SOURCE; +import static guru.interlis.transfer.TransferIdentity.IdOrigin.SYNTHETIC; +import static org.junit.jupiter.api.Assertions.assertEquals; +import static org.junit.jupiter.api.Assertions.assertNotEquals; +import static org.junit.jupiter.api.Assertions.assertThrows; + +class TransferIdentityTest { + @Test + void keepsSourceAndSyntheticTransferIdsSeparateFromTheStableObjectOid() { + var current = new TransferIdentity.BasketId("basket", SYNTHETIC); + var original = new TransferIdentity.BasketId("original", SOURCE); + var source = new SourceId("memory:/transfer"); + var synthetic = new TransferIdentity.TransferId("42", SYNTHETIC); + var explicit = new TransferIdentity.TransferId("42", SOURCE); + var stable = new TransferIdentity.ObjectOid("42"); + var identity = new TransferIdentity(current, Optional.of(original), Optional.of(synthetic), + Optional.empty(), Optional.of(source)); + var identified = new TransferIdentity(current, Optional.of(original), Optional.of(explicit), + Optional.of(stable), Optional.of(source)); + + assertEquals(current, identity.currentBasket()); + assertEquals(SYNTHETIC, identity.currentBasket().origin()); + assertEquals(Optional.of(original), identity.originalBasket()); + assertEquals(Optional.of(source), identity.source()); + assertEquals(Optional.of(synthetic), identity.tid()); + assertEquals(Optional.empty(), identity.oid()); + assertEquals(Optional.of(explicit), identified.tid()); + assertEquals(Optional.of(stable), identified.oid()); + assertEquals(synthetic.value(), stable.value()); + assertNotEquals(synthetic, explicit); + assertNotEquals(synthetic, stable); + assertNotEquals(explicit, stable); + } + + @Test + void keepsAnUnidentifiedObjectAndItsDataWithoutInventingAKey() { + var basket = new TransferIdentity.BasketId("basket", SOURCE); + var identity = new TransferIdentity(basket, Optional.empty(), Optional.empty(), Optional.empty(), + Optional.empty()); + var snapshot = new SnapshotId("transfer-identity-fixture"); + var type = new SymbolRef(snapshot, new SymbolId("type")); + var slot = new SymbolRef(snapshot, new SymbolId("name")); + var value = new TransferValue.Text("same object data"); + var data = new TransferValue.Structured(type, Map.of(slot, value)); + var object = new TransferObject(identity, data); + + assertEquals(object, new TransferObject(identity, new TransferValue.Structured(type, data.values()))); + assertEquals(identity, object.identity()); + assertEquals(Optional.empty(), object.identity().tid()); + assertEquals(Optional.empty(), object.identity().oid()); + assertEquals(type, object.type()); + assertEquals(Optional.of(value), object.value(slot)); + assertEquals(Optional.empty(), object.value(new SymbolRef(snapshot, new SymbolId("missing")))); + } + + @Test + void rejectsAForeignNestedStructureInsideARepeatedValue() { + var basket = new TransferIdentity.BasketId("basket", SOURCE); + var identity = new TransferIdentity(basket, Optional.empty(), Optional.empty(), Optional.empty(), + Optional.empty()); + var snapshot = new SnapshotId("transfer-identity-fixture"); + var type = new SymbolRef(snapshot, new SymbolId("type")); + var slot = new SymbolRef(snapshot, new SymbolId("children")); + var foreignType = new SymbolRef(new SnapshotId("other-identity-fixture"), new SymbolId("structure")); + var nested = new TransferValue.Structured(foreignType, Map.of()); + var list = new TransferValue.Repeated(CollectionKind.LIST, List.of(nested)); + + assertThrows(SnapshotMismatchException.class, + () -> new TransferObject(identity, new TransferValue.Structured(type, Map.of(slot, list)))); + } + + @Test + void checksDeepValuesWithoutRecursingThroughTheJavaStack() { + var basket = new TransferIdentity.BasketId("basket", SOURCE); + var identity = new TransferIdentity(basket, Optional.empty(), Optional.empty(), Optional.empty(), + Optional.empty()); + var snapshot = new SnapshotId("deep-transfer"); + var type = new SymbolRef(snapshot, new SymbolId("type")); + var slot = new SymbolRef(snapshot, new SymbolId("child")); + TransferValue value = new TransferValue.Text("leaf"); + for (int depth = 0; depth < 8_192; depth++) { + value = new TransferValue.Structured(type, Map.of(slot, value)); + } + + var object = new TransferObject(identity, (TransferValue.Structured) value); + TransferValue actual = object.data(); + for (int depth = 0; depth < 8_192; depth++) { + actual = ((TransferValue.Structured) actual).value(slot).orElseThrow(); + } + assertEquals(new TransferValue.Text("leaf"), actual); + } +} diff --git a/interlis-transfer/src/test/java/guru/interlis/transfer/TransferValueTest.java b/interlis-transfer/src/test/java/guru/interlis/transfer/TransferValueTest.java new file mode 100644 index 00000000..57ca13ca --- /dev/null +++ b/interlis-transfer/src/test/java/guru/interlis/transfer/TransferValueTest.java @@ -0,0 +1,143 @@ +package guru.interlis.transfer; + +import guru.interlis.model.CollectionKind; +import guru.interlis.model.SnapshotId; +import guru.interlis.model.SnapshotMismatchException; +import guru.interlis.model.SymbolId; +import guru.interlis.model.SymbolRef; +import org.junit.jupiter.api.Test; + +import java.math.BigDecimal; +import java.math.BigInteger; +import java.util.ArrayList; +import java.util.LinkedHashMap; +import java.util.List; +import java.util.Map; +import java.util.Optional; + +import static org.junit.jupiter.api.Assertions.assertEquals; +import static org.junit.jupiter.api.Assertions.assertFalse; +import static org.junit.jupiter.api.Assertions.assertInstanceOf; +import static org.junit.jupiter.api.Assertions.assertNotEquals; +import static org.junit.jupiter.api.Assertions.assertThrows; +import static org.junit.jupiter.api.Assertions.assertTrue; + +class TransferValueTest { + private static final SnapshotId SNAPSHOT = new SnapshotId("transfer-value-fixture"); + + @Test + void separatesMissingUndefinedAndPresentEmptyValues() { + var type = ref("type"); + var missing = ref("missing"); + var undefined = ref("undefined"); + var emptyText = ref("empty-text"); + var emptyBag = ref("empty-bag"); + var emptyList = ref("empty-list"); + var values = new LinkedHashMap(); + values.put(undefined, TransferValue.Undefined.INSTANCE); + values.put(emptyText, new TransferValue.Text("")); + values.put(emptyBag, new TransferValue.Repeated(CollectionKind.BAG, List.of())); + values.put(emptyList, new TransferValue.Repeated(CollectionKind.LIST, List.of())); + var structure = new TransferValue.Structured(type, values); + values.put(missing, new TransferValue.BooleanValue(true)); + + assertEquals(Optional.empty(), structure.value(missing)); + assertEquals(Optional.of(TransferValue.Undefined.INSTANCE), structure.value(undefined)); + assertEquals(Optional.of(new TransferValue.Text("")), structure.value(emptyText)); + assertEquals(Optional.of(new TransferValue.Repeated(CollectionKind.BAG, List.of())), structure.value(emptyBag)); + assertEquals(Optional.of(new TransferValue.Repeated(CollectionKind.LIST, List.of())), structure.value(emptyList)); + assertEquals(List.of(undefined, emptyText, emptyBag, emptyList), List.copyOf(structure.values().keySet())); + assertThrows(UnsupportedOperationException.class, () -> structure.values().clear()); + + values.put(missing, null); + assertThrows(NullPointerException.class, () -> new TransferValue.Structured(type, values)); + assertThrows(NullPointerException.class, () -> new TransferValue.Text(null)); + } + + @Test + void preservesPrecisionAndOriginalLexemeSeparately() { + var value = new TransferValue.NumberValue("000123456789012345678901234567890.0012300", + new BigDecimal("123456789012345678901234567890.0012300")); + + assertEquals("000123456789012345678901234567890.0012300", value.lexeme()); + assertEquals(new BigDecimal("123456789012345678901234567890.0012300"), value.exactValue()); + assertEquals(7, value.exactValue().scale()); + assertNotEquals(value, new TransferValue.NumberValue("123456789012345678901234567890.0012300", + value.exactValue())); + assertThrows(IllegalArgumentException.class, () -> new TransferValue.NumberValue(" ", BigDecimal.ZERO)); + } + + @Test + void roundTripsNestedStructuresAndPrimitiveRepetitionsWithTheirOrderAndKind() { + var type = ref("object-type"); + var innerType = ref("structure-type"); + var bagSlot = ref("bag-slot"); + var listSlot = ref("list-slot"); + var structureSlot = ref("structure-slot"); + var enumSlot = ref("enum-slot"); + var falseValue = new TransferValue.BooleanValue(false); + var trueValue = new TransferValue.BooleanValue(true); + var primitives = new ArrayList(List.of(falseValue, trueValue, falseValue)); + var bag = new TransferValue.Repeated(CollectionKind.BAG, primitives); + var list = new TransferValue.Repeated(CollectionKind.LIST, primitives); + primitives.clear(); + + var enumPath = new ArrayList<>(List.of("surface", "roof")); + var enumeration = new TransferValue.Enumeration(enumPath); + enumPath.clear(); + var inner = new TransferValue.Structured(innerType, Map.of(enumSlot, enumeration)); + var original = new TransferValue.Structured(type, Map.of(bagSlot, bag, listSlot, list, structureSlot, inner)); + var replay = new TransferValue.Structured(original.type(), original.values()); + + assertEquals(original, replay); + assertEquals(CollectionKind.BAG, bag.kind()); + assertEquals(CollectionKind.LIST, list.kind()); + assertEquals(List.of(falseValue, trueValue, falseValue), bag.values()); + assertEquals(List.of(falseValue, trueValue, falseValue), list.values()); + assertNotEquals(bag, list); + var nested = assertInstanceOf(TransferValue.Structured.class, replay.value(structureSlot).orElseThrow()); + assertEquals(Optional.of(new TransferValue.Enumeration(List.of("surface", "roof"))), nested.value(enumSlot)); + assertThrows(UnsupportedOperationException.class, () -> list.values().clear()); + assertThrows(UnsupportedOperationException.class, () -> enumeration.path().clear()); + } + + @Test + void retainsReferenceTargetBasketAndOrderIndependentlyOfTheTargetKey() { + var tid = new TransferIdentity.TransferId("target", TransferIdentity.IdOrigin.SOURCE); + var oid = new TransferIdentity.ObjectOid("target"); + var targetBasket = new TransferIdentity.BasketId("destination", TransferIdentity.IdOrigin.SOURCE); + var position = new BigInteger("18446744073709551616"); + var local = new TransferValue.Reference(tid, Optional.empty(), false, Optional.empty()); + var external = new TransferValue.Reference(oid, Optional.of(targetBasket), true, Optional.of(position)); + + assertEquals(tid, local.target()); + assertEquals(Optional.empty(), local.targetBasket()); + assertFalse(local.external()); + assertEquals(Optional.empty(), local.orderPosition()); + assertEquals(oid, external.target()); + assertEquals(Optional.of(targetBasket), external.targetBasket()); + assertTrue(external.external()); + assertEquals(Optional.of(position), external.orderPosition()); + assertNotEquals(local, external); + assertThrows(IllegalArgumentException.class, + () -> new TransferValue.Reference(tid, Optional.empty(), false, Optional.of(BigInteger.ZERO))); + } + + @Test + void rejectsForeignSlotsEvenWhenTheSymbolIdMatches() { + var type = ref("type"); + var slot = ref("attribute"); + var foreignSlot = new SymbolRef(new SnapshotId("foreign-value-fixture"), slot.symbol()); + var text = new TransferValue.Text("value"); + var structure = new TransferValue.Structured(type, Map.of(slot, text)); + + assertEquals(Optional.of(text), structure.value(slot)); + assertThrows(SnapshotMismatchException.class, () -> structure.value(foreignSlot)); + assertThrows(SnapshotMismatchException.class, + () -> new TransferValue.Structured(type, Map.of(foreignSlot, text))); + } + + private static SymbolRef ref(String id) { + return new SymbolRef(SNAPSHOT, new SymbolId(id)); + } +} From a2d52ff41953df81bb7e20c50d8746cffc5ce12a Mon Sep 17 00:00:00 2001 From: Stefan Ziegler Date: Fri, 6 Nov 2026 19:22:08 +0100 Subject: [PATCH 4/6] Stream checked transfer events through reader and writer boundaries Enforce order, completeness, snapshot identity, declared models and effective basket bindings. Retain only scoped IDs and active context for protocol checks. Release adapter resources once and distinguish cancellation, incomplete EOF, invalid input and failed I/O. Jenesis 0.18.1 on Java 25.0.2 passed 11 targeted protocol and streaming tests. OpenCode 2.2.5 / deepseek/deepseek-flash initially found five invalid acceptances. The corrected candidate passed re-review, the original reproduction and a new 36-check counterprobe. The 2.3 XTF cross-namespace rule and complete UPDATE handling remain adapter scope. --- .../guru/interlis/transfer/TransferEvent.java | 134 +++++++ .../transfer/TransferProtocolException.java | 38 ++ .../interlis/transfer/TransferReader.java | 71 ++++ .../interlis/transfer/TransferSequence.java | 269 +++++++++++++ .../guru/interlis/transfer/TransferValue.java | 4 +- .../interlis/transfer/TransferWriter.java | 64 +++ .../transfer/TransferEventProtocolTest.java | 375 ++++++++++++++++++ .../TransferStreamingContractTest.java | 196 +++++++++ 8 files changed, 1150 insertions(+), 1 deletion(-) create mode 100644 interlis-transfer/src/main/java/guru/interlis/transfer/TransferEvent.java create mode 100644 interlis-transfer/src/main/java/guru/interlis/transfer/TransferProtocolException.java create mode 100644 interlis-transfer/src/main/java/guru/interlis/transfer/TransferReader.java create mode 100644 interlis-transfer/src/main/java/guru/interlis/transfer/TransferSequence.java create mode 100644 interlis-transfer/src/main/java/guru/interlis/transfer/TransferWriter.java create mode 100644 interlis-transfer/src/test/java/guru/interlis/transfer/TransferEventProtocolTest.java create mode 100644 interlis-transfer/src/test/java/guru/interlis/transfer/TransferStreamingContractTest.java diff --git a/interlis-transfer/src/main/java/guru/interlis/transfer/TransferEvent.java b/interlis-transfer/src/main/java/guru/interlis/transfer/TransferEvent.java new file mode 100644 index 00000000..305cc9ce --- /dev/null +++ b/interlis-transfer/src/main/java/guru/interlis/transfer/TransferEvent.java @@ -0,0 +1,134 @@ +package guru.interlis.transfer; + +import guru.interlis.model.LanguageVersion; +import guru.interlis.model.SymbolRef; +import guru.interlis.source.SourceId; +import guru.interlis.transfer.TransferIdentity.BasketId; + +import java.util.List; +import java.util.Objects; +import java.util.Optional; + +/** + * Owned, immutable event in one neutral transfer stream. Symbol references + * are bound to the stream's plan snapshot. Event metadata is carried in source order. + */ +public sealed interface TransferEvent permits TransferEvent.StartTransfer, TransferEvent.StartBasket, + TransferEvent.ObjectEvent, TransferEvent.EndBasket, TransferEvent.EndTransfer { + + /** + * Transfer header. + * + * @param source opaque source identifier, if known + * @param sender supplied sender, if present + * @param comment supplied comment, if present + * @param models immutable list of model declarations in source order, covering every + * basket topic and object model used in the stream + */ + record StartTransfer(Optional source, Optional sender, Optional comment, + List models) implements TransferEvent { + public StartTransfer { + Objects.requireNonNull(source, "source"); + Objects.requireNonNull(sender, "sender"); + Objects.requireNonNull(comment, "comment"); + models = List.copyOf(models); + } + } + + /** + * One model declared by the source. + * + * @param model model declaration in the plan snapshot + * @param languageVersion INTERLIS language version of the model + * @param modelVersion supplied version string, if present, matching the plan's model version + * @param issuer supplied model issuer, if present + */ + record ModelDeclaration(SymbolRef model, LanguageVersion languageVersion, Optional modelVersion, + Optional issuer) { + public ModelDeclaration { + Objects.requireNonNull(model, "model"); + Objects.requireNonNull(languageVersion, "languageVersion"); + Objects.requireNonNull(modelVersion, "modelVersion"); + Objects.requireNonNull(issuer, "issuer"); + } + } + + /** + * Header for one basket. Incremental metadata is carried for later interpretation. + * + * @param id identity of this basket occurrence within the transfer + * @param topic primary topic in the plan snapshot + * @param additionalTopics effective extension topics in the supplied order + * @param kind transfer kind + * @param consistency supplied or defaulted consistency state + * @param startState opaque initial state of an incremental basket, if supplied + * @param endState opaque final state of an incremental basket, if supplied + * @param stableOid independently supplied stable basket identity, if present + * @param domainBindings immutable list of permitted runtime bindings for the active topics, + * covering every generic domain used by those topics, in input order + */ + record StartBasket(BasketId id, SymbolRef topic, List additionalTopics, BasketKind kind, + Consistency consistency, Optional startState, Optional endState, + Optional stableOid, List domainBindings) implements TransferEvent { + public StartBasket { + Objects.requireNonNull(id, "id"); + Objects.requireNonNull(topic, "topic"); + additionalTopics = List.copyOf(additionalTopics); + Objects.requireNonNull(kind, "kind"); + Objects.requireNonNull(consistency, "consistency"); + Objects.requireNonNull(startState, "startState"); + Objects.requireNonNull(endState, "endState"); + Objects.requireNonNull(stableOid, "stableOid"); + domainBindings = List.copyOf(domainBindings); + } + } + + /** One supplied binding from a generic domain to a concrete domain in the plan snapshot. */ + record DomainBinding(SymbolRef generic, SymbolRef concrete) { + public DomainBinding { + Objects.requireNonNull(generic, "generic"); + Objects.requireNonNull(concrete, "concrete"); + } + } + + /** + * One object occurrence and its metadata. Values can be retained after advancing a reader. + * + * @param object immutable data and identity of the occurrence + * @param operation supplied incremental operation, if present + * @param consistency supplied or defaulted consistency state + */ + record ObjectEvent(TransferObject object, Optional operation, Consistency consistency) + implements TransferEvent { + public ObjectEvent { + Objects.requireNonNull(object, "object"); + Objects.requireNonNull(operation, "operation"); + Objects.requireNonNull(consistency, "consistency"); + } + } + + /** Ends the active basket, identified by the same transfer-scoped ID as its start. */ + record EndBasket(BasketId id) implements TransferEvent { + public EndBasket { + Objects.requireNonNull(id, "id"); + } + } + + /** Ends a complete transfer after its last basket. */ + record EndTransfer() implements TransferEvent { } + + /** Full dataset, initial incremental state, or update of an incremental state. */ + enum BasketKind { + FULL, INITIAL, UPDATE + } + + /** Transfer consistency metadata shared by baskets and objects. */ + enum Consistency { + COMPLETE, INCOMPLETE, INCONSISTENT, ADAPTED + } + + /** Incremental operation supplied for an object occurrence. */ + enum Operation { + INSERT, UPDATE, DELETE + } +} diff --git a/interlis-transfer/src/main/java/guru/interlis/transfer/TransferProtocolException.java b/interlis-transfer/src/main/java/guru/interlis/transfer/TransferProtocolException.java new file mode 100644 index 00000000..d99bc3ba --- /dev/null +++ b/interlis-transfer/src/main/java/guru/interlis/transfer/TransferProtocolException.java @@ -0,0 +1,38 @@ +package guru.interlis.transfer; + +import java.util.Objects; + +/** Failure at the neutral transfer stream boundary. */ +public final class TransferProtocolException extends IllegalStateException { + private final Reason reason; + + TransferProtocolException(Reason reason, String message) { + super(message); + this.reason = Objects.requireNonNull(reason, "reason"); + } + + /** Returns the stable category of the failure. */ + public Reason reason() { + return reason; + } + + /** Stream, lifecycle and context failure categories. */ + public enum Reason { + /** The event cannot occur at the current point in the stream. */ + UNEXPECTED_EVENT, + /** The stream stopped before a valid end event. */ + INCOMPLETE_TRANSFER, + /** The reader or writer is closed. */ + CLOSED, + /** An earlier stream failure prevents further use. */ + FAILED, + /** A model handle belongs to a different schema snapshot. */ + FOREIGN_SNAPSHOT, + /** The referenced model element is absent from the plan or has the wrong kind. */ + UNKNOWN_MODEL_ELEMENT, + /** A basket ID or full-transfer object ID has already occurred in its scope. */ + DUPLICATE_ID, + /** The event conflicts with its declared model or active basket. */ + CONTEXT_MISMATCH + } +} diff --git a/interlis-transfer/src/main/java/guru/interlis/transfer/TransferReader.java b/interlis-transfer/src/main/java/guru/interlis/transfer/TransferReader.java new file mode 100644 index 00000000..acf7eb01 --- /dev/null +++ b/interlis-transfer/src/main/java/guru/interlis/transfer/TransferReader.java @@ -0,0 +1,71 @@ +package guru.interlis.transfer; + +import guru.interlis.execution.CancellationToken; + +import java.util.Objects; +import java.util.Optional; + +/** Pull reader over one checked transfer sequence. Use from a single caller and close it + * after reading or abandoning the transfer. Each delivered event is immutable and owned by + * the caller. The reader keeps model declarations and the active basket context. For + * uniqueness it keeps only distinct basket ID and FULL object TID strings until completion, + * failure or close. Memory for this check grows with the number of distinct IDs. These two + * identity scopes are separate. */ +public abstract class TransferReader implements AutoCloseable { + private final TransferSequence sequence; + private boolean exhausted; + + /** Binds this reader to a nonnull plan. */ + protected TransferReader(TransferPlan plan) { + sequence = new TransferSequence(plan); + } + + /** Reads one event. Empty means the source ended after a valid EndTransfer. + * Read through the empty result to confirm that the source has no trailing events. + * A cancellation already requested on entry leaves the reader at its current position. + * If an exception occurs while reading or checking an event, only close remains available. + * Adapter errors are propagated as supplied by the adapter. + * @throws TransferProtocolException if the stream is incomplete, invalid, failed or closed + * @throws guru.interlis.execution.OperationCancelledException on cancellation + */ + public final Optional readNext(CancellationToken cancellation) { + sequence.requireOpen(); + Objects.requireNonNull(cancellation, "cancellation").throwIfCancellationRequested(); + if (exhausted) { + return Optional.empty(); + } + try { + var next = Objects.requireNonNull(readEvent(cancellation), "source result"); + cancellation.throwIfCancellationRequested(); + if (next.isPresent()) { + sequence.accept(next.orElseThrow()); + } else { + if (!sequence.ended()) { + throw new TransferProtocolException(TransferProtocolException.Reason.INCOMPLETE_TRANSFER, + "Source ended before EndTransfer"); + } + exhausted = true; + } + return next; + } catch (RuntimeException failure) { + sequence.fail(); + throw failure; + } + } + + /** Pulls one immutable event or reports the underlying source's end. Long reads + * should observe the cancellation token. */ + protected abstract Optional readEvent(CancellationToken cancellation); + + /** Releases the underlying source once, including after partial or failed reads. */ + protected abstract void release(); + + /** Closes this reader and releases its source. Closing early abandons the transfer. + * Subsequent closes are no-ops. */ + @Override + public final void close() { + if (sequence.beginClose()) { + release(); + } + } +} diff --git a/interlis-transfer/src/main/java/guru/interlis/transfer/TransferSequence.java b/interlis-transfer/src/main/java/guru/interlis/transfer/TransferSequence.java new file mode 100644 index 00000000..ed66db81 --- /dev/null +++ b/interlis-transfer/src/main/java/guru/interlis/transfer/TransferSequence.java @@ -0,0 +1,269 @@ +package guru.interlis.transfer; + +import guru.interlis.model.DomainSymbol; +import guru.interlis.model.SymbolId; +import guru.interlis.model.SymbolRef; +import guru.interlis.transfer.TransferEvent.BasketKind; +import guru.interlis.transfer.TransferEvent.EndBasket; +import guru.interlis.transfer.TransferEvent.EndTransfer; +import guru.interlis.transfer.TransferEvent.ObjectEvent; +import guru.interlis.transfer.TransferEvent.StartBasket; +import guru.interlis.transfer.TransferEvent.StartTransfer; +import guru.interlis.transfer.TransferIdentity.BasketId; + +import java.util.ArrayList; +import java.util.HashMap; +import java.util.HashSet; +import java.util.List; +import java.util.Objects; +import java.util.Set; + +import static guru.interlis.transfer.TransferProtocolException.Reason.CLOSED; +import static guru.interlis.transfer.TransferProtocolException.Reason.CONTEXT_MISMATCH; +import static guru.interlis.transfer.TransferProtocolException.Reason.DUPLICATE_ID; +import static guru.interlis.transfer.TransferProtocolException.Reason.FAILED; +import static guru.interlis.transfer.TransferProtocolException.Reason.FOREIGN_SNAPSHOT; +import static guru.interlis.transfer.TransferProtocolException.Reason.UNEXPECTED_EVENT; +import static guru.interlis.transfer.TransferProtocolException.Reason.UNKNOWN_MODEL_ELEMENT; + +/** Shared sequence checks with the active basket context and transfer-scoped identity sets. */ +final class TransferSequence { + private final TransferPlan plan; + private State state = State.BEFORE_TRANSFER; + private Set declaredModels = Set.of(); + private Set basketIds = new HashSet<>(); + private Set fullObjectIds = new HashSet<>(); + private ActiveBasket basket; + private boolean closed; + private boolean failed; + + TransferSequence(TransferPlan plan) { + this.plan = Objects.requireNonNull(plan, "plan"); + } + + void requireOpen() { + if (closed) { + throw new TransferProtocolException(CLOSED, "Transfer stream is closed"); + } + if (failed) { + throw new TransferProtocolException(FAILED, "Transfer stream has failed"); + } + } + + boolean beginClose() { + if (closed) { + return false; + } + closed = true; + releaseState(); + return true; + } + + boolean failed() { + return failed; + } + + void fail() { + failed = true; + releaseState(); + } + + boolean ended() { + return state == State.ENDED; + } + + void accept(TransferEvent event) { + requireOpen(); + switch (event) { + case StartTransfer start -> { + requireState(State.BEFORE_TRANSFER, event); + declaredModels = validateModels(start); + state = State.BETWEEN_BASKETS; + } + case StartBasket start -> { + requireState(State.BETWEEN_BASKETS, event); + var next = validateBasket(start); + requireNewId(basketIds, start.id().value(), "basket"); + basket = next; + state = State.IN_BASKET; + } + case ObjectEvent occurrence -> { + requireState(State.IN_BASKET, event); + validateObject(occurrence.object()); + } + case EndBasket end -> { + requireState(State.IN_BASKET, event); + requireBasketId(end.id()); + basket = null; + state = State.BETWEEN_BASKETS; + } + case EndTransfer ignored -> { + requireState(State.BETWEEN_BASKETS, event); + state = State.ENDED; + releaseState(); + } + } + } + + private Set validateModels(StartTransfer start) { + var declared = new HashSet(); + for (var declaration : start.models()) { + SymbolRef ref = declaration.model(); + requireSnapshot(ref); + var model = plan.model(ref).orElseThrow(() -> unknown("model", ref)); + if (model.languageVersion() != declaration.languageVersion() + || declaration.modelVersion().filter(version -> !version.equals(model.modelVersion())).isPresent()) { + throw new TransferProtocolException(CONTEXT_MISMATCH, + "Declared version differs from the transfer model " + model.name().declared()); + } + declared.add(ref); + } + return declared; + } + + private ActiveBasket validateBasket(StartBasket start) { + TopicDescriptor primary = concreteTopic(start.topic()); + requireDeclaredModel(primary.model()); + Set types = new HashSet<>(primary.objectTypes()); + var topics = new ArrayList(); + topics.add(primary); + for (SymbolRef ref : start.additionalTopics()) { + TopicDescriptor extension = concreteTopic(ref); + requireDeclaredModel(extension.model()); + if (!extendsTopic(extension, primary.symbol())) { + throw new TransferProtocolException(CONTEXT_MISMATCH, + extension.name().declared() + " does not extend " + primary.name().declared()); + } + types.addAll(extension.objectTypes()); + topics.add(extension); + } + validateDomains(start.domainBindings(), topics); + return new ActiveBasket(start.id(), start.kind(), types); + } + + private void validateObject(TransferObject object) { + requireBasketId(object.identity().currentBasket()); + SymbolRef ref = object.type(); + requireSnapshot(ref); + var type = plan.object(ref).orElseThrow(() -> unknown("object type", ref)); + if (!type.transferable() || !basket.objectTypes().contains(ref)) { + throw new TransferProtocolException(CONTEXT_MISMATCH, + type.name().declared() + " cannot occur in basket " + basket.id().value()); + } + requireDeclaredModel(type.model()); + if (basket.kind() == BasketKind.FULL) { + object.identity().tid().ifPresent(tid -> requireNewId(fullObjectIds, tid.value(), "object")); + } + } + + private void requireBasketId(BasketId id) { + if (!basket.id().equals(id)) { + throw new TransferProtocolException(CONTEXT_MISMATCH, "Basket ID differs from the active basket"); + } + } + + private TopicDescriptor topic(SymbolRef ref) { + requireSnapshot(ref); + return plan.topic(ref).orElseThrow(() -> unknown("topic", ref)); + } + + private TopicDescriptor concreteTopic(SymbolRef ref) { + TopicDescriptor topic = topic(ref); + if (topic.abstractTopic()) { + throw new TransferProtocolException(CONTEXT_MISMATCH, + "Basket topic " + topic.name().declared() + " is abstract"); + } + return topic; + } + + private boolean extendsTopic(TopicDescriptor extension, SymbolRef ancestor) { + while (extension.baseTopic().isPresent()) { + SymbolRef base = extension.baseTopic().orElseThrow(); + if (base.equals(ancestor)) { + return true; + } + extension = topic(base); + } + return false; + } + + private void validateDomains(List bindings, List topics) { + var selected = new HashMap(); + for (var binding : bindings) { + var generic = requireDomain(binding.generic()); + var concrete = requireDomain(binding.concrete()); + if (!generic.generic() || concrete.generic() + || selected.putIfAbsent(generic.id(), concrete.id()) != null) { + throw new TransferProtocolException(CONTEXT_MISMATCH, + "Expected one generic-to-concrete assignment for " + generic.name().value()); + } + } + var required = new HashSet(); + for (var topic : topics) { + for (var binding : topic.genericBindings()) { + required.add(binding.genericDomain()); + SymbolId concrete = selected.get(binding.genericDomain()); + if (concrete == null || !binding.alternatives().contains(concrete)) { + var generic = requireDomain(new SymbolRef(plan.snapshotId(), binding.genericDomain())); + throw new TransferProtocolException(CONTEXT_MISMATCH, + "Basket for " + topic.name().declared() + " needs a permitted assignment for " + + generic.name().value()); + } + } + } + if (!selected.keySet().equals(required)) { + throw new TransferProtocolException(CONTEXT_MISMATCH, "Basket assigns an unused generic domain"); + } + } + + private DomainSymbol requireDomain(SymbolRef ref) { + requireSnapshot(ref); + if (plan.symbol(ref).known().orElse(null) instanceof DomainSymbol domain) { + return domain; + } + throw unknown("domain", ref); + } + + private void requireDeclaredModel(SymbolRef model) { + if (!declaredModels.contains(model)) { + throw new TransferProtocolException(CONTEXT_MISMATCH, + "Transfer header must declare " + plan.model(model).orElseThrow().name().declared()); + } + } + + private static void requireNewId(Set ids, String id, String kind) { + if (!ids.add(id)) { + throw new TransferProtocolException(DUPLICATE_ID, "Duplicate " + kind + " ID " + id); + } + } + + private void releaseState() { + basket = null; + declaredModels = Set.of(); + basketIds = Set.of(); + fullObjectIds = Set.of(); + } + + private void requireSnapshot(SymbolRef ref) { + if (!plan.snapshotId().equals(ref.snapshot())) { + throw new TransferProtocolException(FOREIGN_SNAPSHOT, "Model reference belongs to another snapshot"); + } + } + + private void requireState(State expected, TransferEvent event) { + if (state != expected) { + throw new TransferProtocolException(UNEXPECTED_EVENT, + event.getClass().getSimpleName() + " cannot occur in state " + state); + } + } + + private static TransferProtocolException unknown(String kind, SymbolRef ref) { + return new TransferProtocolException(UNKNOWN_MODEL_ELEMENT, "Unknown " + kind + " " + ref.symbol().value()); + } + + private enum State { + BEFORE_TRANSFER, BETWEEN_BASKETS, IN_BASKET, ENDED + } + + private record ActiveBasket(BasketId id, BasketKind kind, Set objectTypes) { } +} diff --git a/interlis-transfer/src/main/java/guru/interlis/transfer/TransferValue.java b/interlis-transfer/src/main/java/guru/interlis/transfer/TransferValue.java index 2f7aa6b6..38b09351 100644 --- a/interlis-transfer/src/main/java/guru/interlis/transfer/TransferValue.java +++ b/interlis-transfer/src/main/java/guru/interlis/transfer/TransferValue.java @@ -91,7 +91,9 @@ record Repeated(CollectionKind kind, List values) implements Tran } /** Reference to an object identity. Basket and position are retained when supplied. - * A position is absolute in the ordered relation within the transfer basket and is one-based. */ + * The external flag records the EXTERNAL reference semantics that permit a target in + * another basket. The supplied target basket is independent of that flag. A position + * is absolute in the ordered relation within the transfer basket and is one-based. */ record Reference(TransferIdentity.ObjectKey target, Optional targetBasket, boolean external, Optional orderPosition) implements TransferValue { public Reference { diff --git a/interlis-transfer/src/main/java/guru/interlis/transfer/TransferWriter.java b/interlis-transfer/src/main/java/guru/interlis/transfer/TransferWriter.java new file mode 100644 index 00000000..e1f5d65a --- /dev/null +++ b/interlis-transfer/src/main/java/guru/interlis/transfer/TransferWriter.java @@ -0,0 +1,64 @@ +package guru.interlis.transfer; + +import guru.interlis.execution.CancellationToken; + +import java.util.Objects; + +/** Push writer over one checked transfer sequence. Use from a single caller and close it + * to release the destination. Each accepted event is delivered synchronously. The writer + * keeps model declarations and the active basket context. For uniqueness it keeps only + * distinct basket ID and FULL object TID strings until completion, failure or close. Memory + * for this check grows with the number of distinct IDs. These two identity scopes are separate. */ +public abstract class TransferWriter implements AutoCloseable { + private final TransferSequence sequence; + + /** Binds this writer to a nonnull plan. */ + protected TransferWriter(TransferPlan plan) { + sequence = new TransferSequence(plan); + } + + /** Writes an event without a cancellation request. */ + public final void write(TransferEvent event) { + write(event, CancellationToken.none()); + } + + /** Checks and writes one nonnull event. Pre-cancellation and protocol rejection leave the + * writer at its current position. If writing to the destination throws, only close remains + * available, since the adapter may have written part of the event. The failure is propagated. + * @throws TransferProtocolException on invalid event order or context, or a failed or closed writer + * @throws guru.interlis.execution.OperationCancelledException on cancellation + */ + public final void write(TransferEvent event, CancellationToken cancellation) { + sequence.requireOpen(); + Objects.requireNonNull(cancellation, "cancellation").throwIfCancellationRequested(); + sequence.accept(Objects.requireNonNull(event, "event")); + try { + writeEvent(event, cancellation); + } catch (RuntimeException failure) { + sequence.fail(); + throw failure; + } + } + + /** Writes one checked event to the destination. Long writes should observe cancellation. */ + protected abstract void writeEvent(TransferEvent event, CancellationToken cancellation); + + /** Releases the destination once, including after partial or failed writes. */ + protected abstract void release(); + + /** Closes the writer and releases its destination. Subsequent closes are no-ops. + * @throws TransferProtocolException if the destination was released successfully + * and the event stream had neither ended nor previously failed + */ + @Override + public final void close() { + boolean incomplete = !sequence.ended() && !sequence.failed(); + if (sequence.beginClose()) { + release(); + if (incomplete) { + throw new TransferProtocolException(TransferProtocolException.Reason.INCOMPLETE_TRANSFER, + "Writer closed before EndTransfer"); + } + } + } +} diff --git a/interlis-transfer/src/test/java/guru/interlis/transfer/TransferEventProtocolTest.java b/interlis-transfer/src/test/java/guru/interlis/transfer/TransferEventProtocolTest.java new file mode 100644 index 00000000..39d2127e --- /dev/null +++ b/interlis-transfer/src/test/java/guru/interlis/transfer/TransferEventProtocolTest.java @@ -0,0 +1,375 @@ +package guru.interlis.transfer; + +import guru.interlis.execution.CancellationReason; +import guru.interlis.execution.CancellationToken; +import guru.interlis.execution.OperationCancelledException; +import guru.interlis.model.CompiledSchema; +import guru.interlis.model.LanguageVersion; +import guru.interlis.model.SymbolRef; +import guru.interlis.source.SourceId; +import guru.interlis.source.SourceRevision; +import guru.interlis.source.SourceText; +import guru.interlis.transfer.TransferEvent.DomainBinding; +import guru.interlis.transfer.TransferEvent.EndBasket; +import guru.interlis.transfer.TransferEvent.EndTransfer; +import guru.interlis.transfer.TransferEvent.ModelDeclaration; +import guru.interlis.transfer.TransferEvent.ObjectEvent; +import guru.interlis.transfer.TransferEvent.StartBasket; +import guru.interlis.transfer.TransferEvent.StartTransfer; +import guru.interlis.transfer.TransferIdentity.BasketId; +import guru.interlis.transfer.TransferIdentity.TransferId; +import guru.interlis.transfer.TransferStreamingContractTest.MemoryReader; +import guru.interlis.transfer.TransferStreamingContractTest.MemoryWriter; +import org.junit.jupiter.api.BeforeAll; +import org.junit.jupiter.api.Test; +import org.junit.jupiter.api.function.Executable; + +import java.util.ArrayList; +import java.util.List; +import java.util.Map; +import java.util.Optional; +import java.util.concurrent.atomic.AtomicBoolean; +import java.util.concurrent.atomic.AtomicInteger; + +import static guru.interlis.transfer.TransferEvent.BasketKind.FULL; +import static guru.interlis.transfer.TransferEvent.BasketKind.UPDATE; +import static guru.interlis.transfer.TransferEvent.Consistency.ADAPTED; +import static guru.interlis.transfer.TransferEvent.Consistency.COMPLETE; +import static guru.interlis.transfer.TransferIdentity.IdOrigin.SOURCE; +import static guru.interlis.transfer.TransferIdentity.IdOrigin.SYNTHETIC; +import static guru.interlis.transfer.TransferProtocolException.Reason.CLOSED; +import static guru.interlis.transfer.TransferProtocolException.Reason.CONTEXT_MISMATCH; +import static guru.interlis.transfer.TransferProtocolException.Reason.DUPLICATE_ID; +import static guru.interlis.transfer.TransferProtocolException.Reason.FAILED; +import static guru.interlis.transfer.TransferProtocolException.Reason.FOREIGN_SNAPSHOT; +import static guru.interlis.transfer.TransferProtocolException.Reason.INCOMPLETE_TRANSFER; +import static guru.interlis.transfer.TransferProtocolException.Reason.UNEXPECTED_EVENT; +import static guru.interlis.transfer.TransferStreamingContractTest.ready; +import static guru.interlis.transfer.TransferTestSchemas.compile; +import static guru.interlis.transfer.TransferTestSchemas.compileInline; +import static guru.interlis.transfer.TransferTestSchemas.ref; +import static org.junit.jupiter.api.Assertions.assertEquals; +import static org.junit.jupiter.api.Assertions.assertSame; +import static org.junit.jupiter.api.Assertions.assertThrows; + +class TransferEventProtocolTest { + private static final CancellationToken TOKEN = CancellationToken.none(); + private static final String MODEL = "ProtocolFixture"; + private static final String MODEL_TEXT = """ + INTERLIS 2.4; + MODEL ProtocolFixture AT "https://example.org" VERSION "1" = + TOPIC Data = CLASS Entry = END Entry; END Data; + END ProtocolFixture. + """; + private static final BasketId BASKET = new BasketId("basket", SOURCE); + private static CompiledSchema schema; + private static TransferPlan plan; + + @BeforeAll + static void preparePlan() { + schema = compileInline(MODEL, MODEL_TEXT); + plan = ready(schema); + } + + @Test + void rejectsObjectsBeforeABasketAndLetsAWriterRetryRejectedEvents() { + var start = startTransfer(); + var basket = startBasket(); + var wrongBasket = new BasketId("wrong-basket", SOURCE); + var object = object(BASKET); + var writer = new MemoryWriter(plan); + try (writer) { + expect(UNEXPECTED_EVENT, () -> writer.write(object)); + writer.write(start); + expect(UNEXPECTED_EVENT, () -> writer.write(object)); + writer.write(basket); + expect(CONTEXT_MISMATCH, () -> writer.write(object(wrongBasket))); + expect(CONTEXT_MISMATCH, () -> writer.write(new EndBasket(wrongBasket))); + writer.write(object); + writer.write(new EndBasket(BASKET)); + writer.write(new EndTransfer()); + expect(UNEXPECTED_EVENT, () -> writer.write(new EndTransfer())); + } + assertEquals(List.of(start, basket, object, new EndBasket(BASKET), new EndTransfer()), writer.events); + + try (var reader = new MemoryReader(plan, List.of(object).iterator())) { + expect(UNEXPECTED_EVENT, () -> reader.readNext(TOKEN)); + } + } + + @Test + void failsAnIncompleteReadAndFreesAnIncompleteWriterExactlyOnce() { + var start = startTransfer(); + var basket = startBasket(); + try (var reader = new MemoryReader(plan, List.of(start, basket).iterator())) { + assertEquals(Optional.of(start), reader.readNext(TOKEN)); + assertEquals(Optional.of(basket), reader.readNext(TOKEN)); + expect(INCOMPLETE_TRANSFER, () -> reader.readNext(TOKEN)); + } + var writer = new MemoryWriter(plan); + writer.write(start); + writer.write(basket); + expect(INCOMPLETE_TRANSFER, writer::close); + writer.close(); + assertEquals(1, writer.releases); + expect(CLOSED, () -> writer.write(new EndBasket(BASKET))); + } + + @Test + void canAbandonAnActiveReaderWithoutReadingThroughTheEnd() { + var start = startTransfer(); + var basket = startBasket(); + var reader = new MemoryReader(plan, + List.of(start, basket, new EndBasket(BASKET), new EndTransfer()).iterator()); + assertEquals(Optional.of(start), reader.readNext(TOKEN)); + assertEquals(Optional.of(basket), reader.readNext(TOKEN)); + + reader.close(); + reader.close(); + + assertEquals(2, reader.pulls); + assertEquals(1, reader.releases); + expect(CLOSED, () -> reader.readNext(TOKEN)); + } + + @Test + void detectsAnExtraEndAfterEmittingEndTransfer() { + var start = startTransfer(); + var end = new EndTransfer(); + try (var reader = new MemoryReader(plan, List.of(start, end, end).iterator())) { + assertEquals(Optional.of(start), reader.readNext(TOKEN)); + assertEquals(Optional.of(end), reader.readNext(TOKEN)); + expect(UNEXPECTED_EVENT, () -> reader.readNext(TOKEN)); + } + } + + @Test + void preCancellationDoesNotConsumeAndRepeatedCloseReleasesOnlyOnce() { + var start = startTransfer(); + var end = new EndTransfer(); + var cancelled = CancellationToken.requested(CancellationReason.SUPERSEDED); + var reader = new MemoryReader(plan, List.of(start, end).iterator()); + try (reader) { + var cancellation = assertThrows(OperationCancelledException.class, () -> reader.readNext(cancelled)); + assertEquals(CancellationReason.SUPERSEDED, cancellation.reason()); + assertEquals(0, reader.pulls); + assertEquals(Optional.of(start), reader.readNext(TOKEN)); + assertEquals(Optional.of(end), reader.readNext(TOKEN)); + assertEquals(Optional.empty(), reader.readNext(TOKEN)); + } + reader.close(); + assertEquals(1, reader.releases); + expect(CLOSED, () -> reader.readNext(TOKEN)); + + var writer = new MemoryWriter(plan); + try (writer) { + var cancellation = assertThrows(OperationCancelledException.class, () -> writer.write(start, cancelled)); + assertEquals(CancellationReason.SUPERSEDED, cancellation.reason()); + assertEquals(List.of(), writer.events); + writer.write(start); + writer.write(end); + } + writer.close(); + assertEquals(List.of(start, end), writer.events); + assertEquals(1, writer.releases); + expect(CLOSED, () -> writer.write(end)); + } + + @Test + void cancellationDuringAReadAndAdapterWriteFailureLeaveTheStreamFailed() { + var cancelled = new AtomicBoolean(); + var readReleases = new AtomicInteger(); + var start = startTransfer(); + var reader = new TransferReader(plan) { + @Override + protected Optional readEvent(CancellationToken cancellation) { + cancelled.set(true); + return Optional.of(start); + } + + @Override + protected void release() { + readReleases.incrementAndGet(); + } + }; + try (reader) { + assertThrows(OperationCancelledException.class, () -> reader.readNext(cancelled::get)); + expect(FAILED, () -> reader.readNext(TOKEN)); + } + assertEquals(1, readReleases.get()); + + var failure = new IllegalStateException("destination failed after a partial write"); + var writeReleases = new AtomicInteger(); + var writer = new TransferWriter(plan) { + @Override + protected void writeEvent(TransferEvent event, CancellationToken cancellation) { + throw failure; + } + + @Override + protected void release() { + writeReleases.incrementAndGet(); + } + }; + try (writer) { + assertSame(failure, assertThrows(IllegalStateException.class, () -> writer.write(start))); + expect(FAILED, () -> writer.write(start)); + } + assertEquals(1, writeReleases.get()); + } + + @Test + void rejectsForeignModelAndTopicReferencesDespiteEqualSymbolIds() { + var foreign = compile(MODEL, new SourceText(new SourceId("memory:/" + MODEL + ".ili"), + new SourceRevision(2), MODEL_TEXT)); + var model = new SymbolRef(foreign.snapshotId(), ref(schema, MODEL).symbol()); + var topic = new SymbolRef(foreign.snapshotId(), startBasket().topic().symbol()); + var start = startTransfer(); + var wrongModel = new StartTransfer(Optional.empty(), Optional.empty(), Optional.empty(), + List.of(new ModelDeclaration(model, LanguageVersion.ILI24, Optional.empty(), Optional.empty()))); + var wrongBasket = new StartBasket(BASKET, topic, List.of(), FULL, COMPLETE, + Optional.empty(), Optional.empty(), Optional.empty(), List.of()); + + var writer = new MemoryWriter(plan); + try (writer) { + expect(FOREIGN_SNAPSHOT, () -> writer.write(wrongModel)); + writer.write(start); + expect(FOREIGN_SNAPSHOT, () -> writer.write(wrongBasket)); + writer.write(new EndTransfer()); + } + assertEquals(List.of(start, new EndTransfer()), writer.events); + try (var reader = new MemoryReader(plan, List.of(start, wrongBasket).iterator())) { + assertEquals(Optional.of(start), reader.readNext(TOKEN)); + expect(FOREIGN_SNAPSHOT, () -> reader.readNext(TOKEN)); + } + } + + @Test + void requiresTheBasketModelInTheHeaderAndChecksTheSuppliedVersion() { + var emptyHeader = new StartTransfer(Optional.empty(), Optional.empty(), Optional.empty(), List.of()); + var incorrectVersion = new StartTransfer(Optional.empty(), Optional.empty(), Optional.empty(), List.of( + new ModelDeclaration(ref(schema, MODEL), LanguageVersion.ILI24, Optional.of("wrong"), + Optional.empty()))); + var writer = new MemoryWriter(plan); + try (writer) { + expect(CONTEXT_MISMATCH, () -> writer.write(incorrectVersion)); + writer.write(emptyHeader); + expect(CONTEXT_MISMATCH, () -> writer.write(startBasket())); + writer.write(new EndTransfer()); + } + assertEquals(List.of(emptyHeader, new EndTransfer()), writer.events); + try (var reader = new MemoryReader(plan, + List.of(emptyHeader, startBasket()).iterator())) { + assertEquals(Optional.of(emptyHeader), reader.readNext(TOKEN)); + expect(CONTEXT_MISMATCH, () -> reader.readNext(TOKEN)); + } + } + + @Test + void checksBasketAndFullObjectIdsAcrossBasketsInSeparateNamespaces() { + var second = new BasketId("second", SOURCE); + var duplicateBid = new BasketId(BASKET.value(), SYNTHETIC); + var tid = new TransferId(BASKET.value(), SOURCE); + var duplicateTid = new TransferId(tid.value(), SYNTHETIC); + var firstObject = object(BASKET, Optional.of(tid)); + var secondObject = object(second, Optional.of(new TransferId("other", SOURCE))); + var writer = new MemoryWriter(plan); + try (writer) { + writer.write(startTransfer()); + writer.write(startBasket()); + writer.write(firstObject); + writer.write(new EndBasket(BASKET)); + expect(DUPLICATE_ID, () -> writer.write(startBasket(duplicateBid))); + writer.write(startBasket(second)); + expect(DUPLICATE_ID, () -> writer.write(object(second, Optional.of(duplicateTid)))); + writer.write(secondObject); + writer.write(new EndBasket(second)); + writer.write(new EndTransfer()); + } + assertEquals(List.of(firstObject, secondObject), writer.events.stream() + .filter(ObjectEvent.class::isInstance).toList()); + } + + @Test + void preservesIncrementalStatesConsistencyAndRuntimeDomainAssignments() { + var generic = compileInline("GenericExample", """ + INTERLIS 2.4; + MODEL GenericExample AT "https://example.org" VERSION "1" = + DOMAIN G (GENERIC) = COORD NUMERIC, NUMERIC; + A = COORD 0..100,0..100; B = COORD 200..300,200..300; + Other = COORD 400..500,400..500; + CONTEXT Default = G = A OR B; + TOPIC Data = DEFERRED GENERICS G; CLASS Entry = point: G; END Entry; END Data; + END GenericExample. + """); + var genericPlan = ready(generic); + var start = new StartTransfer(Optional.empty(), Optional.empty(), Optional.empty(), List.of( + new ModelDeclaration(ref(generic, "GenericExample"), LanguageVersion.ILI24, + Optional.of("1"), Optional.empty()))); + var genericDomain = ref(generic, "GenericExample", "G"); + var concreteDomain = ref(generic, "GenericExample", "B"); + var assignment = new DomainBinding(genericDomain, concreteDomain); + var assignments = new ArrayList<>(List.of(assignment)); + var basket = updatedBasket(generic, assignments); + assignments.clear(); + assertEquals(List.of(assignment), basket.domainBindings()); + var identity = new TransferIdentity(BASKET, Optional.empty(), Optional.empty(), + Optional.of(new TransferIdentity.ObjectOid("entry-oid")), Optional.empty()); + var entry = new TransferObject(identity, new TransferValue.Structured( + ref(generic, "GenericExample", "Data", "Entry"), Map.of())); + var update = new ObjectEvent(entry, Optional.of(TransferEvent.Operation.UPDATE), ADAPTED); + List expected = List.of(start, basket, update, new EndBasket(BASKET), new EndTransfer()); + var writer = new MemoryWriter(genericPlan); + try (writer) { + writer.write(start); + var reversed = new DomainBinding(concreteDomain, genericDomain); + var invalidAlternative = new DomainBinding(genericDomain, ref(generic, "GenericExample", "Other")); + expect(CONTEXT_MISMATCH, () -> writer.write(updatedBasket(generic, List.of()))); + expect(CONTEXT_MISMATCH, () -> writer.write(updatedBasket(generic, List.of(reversed)))); + expect(CONTEXT_MISMATCH, () -> writer.write(updatedBasket(generic, List.of(invalidAlternative)))); + expect(CONTEXT_MISMATCH, () -> writer.write(updatedBasket(generic, List.of(assignment, assignment)))); + expected.subList(1, expected.size()).forEach(writer::write); + } + try (var reader = new MemoryReader(genericPlan, writer.events.iterator())) { + for (TransferEvent event : expected) { + assertEquals(Optional.of(event), reader.readNext(TOKEN)); + } + assertEquals(Optional.empty(), reader.readNext(TOKEN)); + } + } + + private static StartBasket updatedBasket(CompiledSchema schema, List assignments) { + return new StartBasket(BASKET, ref(schema, "GenericExample", "Data"), List.of(), UPDATE, ADAPTED, + Optional.of("state-9"), Optional.of("state-10"), Optional.of("basket-oid"), assignments); + } + + private static void expect(TransferProtocolException.Reason reason, Executable operation) { + assertEquals(reason, assertThrows(TransferProtocolException.class, operation).reason()); + } + + private static StartTransfer startTransfer() { + return new StartTransfer(Optional.empty(), Optional.empty(), Optional.empty(), List.of( + new ModelDeclaration(ref(schema, MODEL), LanguageVersion.ILI24, Optional.of("1"), Optional.empty()))); + } + + private static StartBasket startBasket() { + return startBasket(BASKET); + } + + private static StartBasket startBasket(BasketId id) { + return new StartBasket(id, ref(schema, MODEL, "Data"), List.of(), FULL, COMPLETE, + Optional.empty(), Optional.empty(), Optional.empty(), List.of()); + } + + private static ObjectEvent object(BasketId basket) { + return object(basket, Optional.empty()); + } + + private static ObjectEvent object(BasketId basket, Optional tid) { + var identity = new TransferIdentity(basket, Optional.empty(), tid, Optional.empty(), + Optional.empty()); + return new ObjectEvent(new TransferObject(identity, + new TransferValue.Structured(ref(schema, MODEL, "Data", "Entry"), Map.of())), + Optional.empty(), COMPLETE); + } +} diff --git a/interlis-transfer/src/test/java/guru/interlis/transfer/TransferStreamingContractTest.java b/interlis-transfer/src/test/java/guru/interlis/transfer/TransferStreamingContractTest.java new file mode 100644 index 00000000..c3cb700a --- /dev/null +++ b/interlis-transfer/src/test/java/guru/interlis/transfer/TransferStreamingContractTest.java @@ -0,0 +1,196 @@ +package guru.interlis.transfer; + +import guru.interlis.execution.CancellationToken; +import guru.interlis.model.CollectionKind; +import guru.interlis.model.CompiledSchema; +import guru.interlis.model.LanguageVersion; +import guru.interlis.model.SymbolRef; +import guru.interlis.source.SourceId; +import guru.interlis.transfer.TransferEvent.EndBasket; +import guru.interlis.transfer.TransferEvent.EndTransfer; +import guru.interlis.transfer.TransferEvent.ModelDeclaration; +import guru.interlis.transfer.TransferEvent.ObjectEvent; +import guru.interlis.transfer.TransferEvent.StartBasket; +import guru.interlis.transfer.TransferEvent.StartTransfer; +import guru.interlis.transfer.TransferGeometry.Arc; +import guru.interlis.transfer.TransferGeometry.Coordinate; +import guru.interlis.transfer.TransferGeometry.CurvePart; +import guru.interlis.transfer.TransferGeometry.Polyline; +import guru.interlis.transfer.TransferGeometry.Ring; +import guru.interlis.transfer.TransferGeometry.Straight; +import guru.interlis.transfer.TransferGeometry.Surface; +import guru.interlis.transfer.TransferIdentity.BasketId; +import guru.interlis.transfer.TransferIdentity.TransferId; +import guru.interlis.transfer.TransferValue.NumberValue; +import guru.interlis.transfer.TransferValue.Reference; +import guru.interlis.transfer.TransferValue.Structured; +import guru.interlis.transfer.TransferValue.Text; +import org.junit.jupiter.api.Test; + +import java.io.IOException; +import java.math.BigDecimal; +import java.util.ArrayList; +import java.util.Iterator; +import java.util.LinkedHashMap; +import java.util.List; +import java.util.Map; +import java.util.Optional; + +import static guru.interlis.transfer.TransferEvent.BasketKind.FULL; +import static guru.interlis.transfer.TransferEvent.Consistency.COMPLETE; +import static guru.interlis.transfer.TransferEvent.Consistency.INCOMPLETE; +import static guru.interlis.transfer.TransferIdentity.IdOrigin.SOURCE; +import static guru.interlis.transfer.TransferTestSchemas.compileInline; +import static guru.interlis.transfer.TransferTestSchemas.fixture; +import static guru.interlis.transfer.TransferTestSchemas.ref; +import static org.junit.jupiter.api.Assertions.assertEquals; +import static org.junit.jupiter.api.Assertions.assertInstanceOf; +import static org.junit.jupiter.api.Assertions.assertSame; + +class TransferStreamingContractTest { + @Test + void streamsTwoBasketsWithOwnedNestedValuesRolesAndGeometry() throws IOException { + var original = fixture("reference/iox/ili23/Reference23.ili").text(); + var withLinkPayload = original.replace("children -- {0..*} Item;", + "children -- {0..*} Item;\n label : TEXT*20;"); + var schema = compileInline("Reference23", withLinkPayload); + var fingerprint = schema.fingerprint(); + var graph = schema.semanticGraph(); + var plan = ready(schema); + var source = new SourceId("memory:/reference-transfer"); + var model = ref(schema, "Reference23"); + var topic = ref(schema, "Reference23", "Records"); + var item = ref(schema, "Reference23", "Records", "Item"); + var detail = ref(schema, "Reference23", "Records", "Detail"); + var description = ref(schema, "Reference23", "Records", "Detail", "description"); + var code = ref(schema, "Reference23", "Records", "Item", "code"); + var details = ref(schema, "Reference23", "Records", "Item", "details"); + var parcel = ref(schema, "Reference23", "Records", "Parcel"); + var shape = ref(schema, "Reference23", "Records", "Parcel", "shape"); + var parent = ref(schema, "Reference23", "Records", "Parent"); + var link = ref(schema, "Reference23", "Records", "ParentLink"); + var parentRole = ref(schema, "Reference23", "Records", "ParentLink", "parent"); + var childrenRole = ref(schema, "Reference23", "Records", "ParentLink", "children"); + var label = ref(schema, "Reference23", "Records", "ParentLink", "label"); + var first = new BasketId("basket-1", SOURCE); + var second = new BasketId("basket-2", SOURCE); + var itemId = new TransferId("item-1", SOURCE); + var parentId = new TransferId("parent-1", SOURCE); + var ownedDetails = new TransferValue.Repeated(CollectionKind.LIST, List.of( + new Structured(detail, Map.of(description, new Text("first"))), + new Structured(detail, Map.of(description, new Text("second"))))); + var fields = new LinkedHashMap(); + fields.put(code, new Text("ABC")); + fields.put(details, ownedDetails); + var itemEvent = object(source, first, Optional.of(itemId), item, fields); + fields.clear(); + + var start = new StartTransfer(Optional.of(source), Optional.of("reference sender"), Optional.of("two baskets"), + List.of(new ModelDeclaration(model, LanguageVersion.ILI23, Optional.of("2026-11-06"), + Optional.of("https://example.invalid/iox-reference")))); + var surface = surface(); + var linkEvent = object(source, second, Optional.empty(), link, Map.of( + parentRole, new Reference(parentId, Optional.empty(), false, Optional.empty()), + childrenRole, new Reference(itemId, Optional.of(first), false, Optional.empty()), + label, new Text("relationship payload"))); + List expected = List.of(start, + new StartBasket(first, topic, List.of(), FULL, COMPLETE, + Optional.empty(), Optional.empty(), Optional.empty(), List.of()), + itemEvent, object(source, first, Optional.of(new TransferId("parcel-1", SOURCE)), + parcel, Map.of(shape, surface)), + new EndBasket(first), + new StartBasket(second, topic, List.of(), FULL, INCOMPLETE, + Optional.empty(), Optional.empty(), Optional.empty(), List.of()), + object(source, second, Optional.of(parentId), parent, Map.of()), + linkEvent, new EndBasket(second), new EndTransfer()); + + var writer = new MemoryWriter(plan); + try (writer) { + for (TransferEvent event : expected) { + writer.write(event); + } + } + assertEquals(expected, writer.events); + assertEquals(1, writer.releases); + var reader = new MemoryReader(plan, writer.events.iterator()); + try (reader) { + assertEquals(0, reader.pulls); + for (int index = 0; index < expected.size(); index++) { + assertEquals(Optional.of(expected.get(index)), reader.readNext(CancellationToken.none())); + assertEquals(index + 1, reader.pulls); + } + assertEquals(Optional.empty(), reader.readNext(CancellationToken.none())); + } + assertEquals(1, reader.releases); + assertEquals(Optional.of(new Text("ABC")), itemEvent.object().value(code)); + assertEquals(Optional.of(ownedDetails), itemEvent.object().value(details)); + assertEquals(Optional.of(new Reference(itemId, Optional.of(first), false, Optional.empty())), + linkEvent.object().value(childrenRole)); + assertSame(graph, schema.semanticGraph()); + assertEquals(fingerprint, schema.fingerprint()); + assertEquals(fingerprint, plan.schemaFingerprint()); + } + + private static ObjectEvent object(SourceId source, BasketId basket, Optional id, + SymbolRef type, Map fields) { + var identity = new TransferIdentity(basket, Optional.empty(), id, Optional.empty(), Optional.of(source)); + return new ObjectEvent(new TransferObject(identity, new Structured(type, fields)), Optional.empty(), COMPLETE); + } + + private static Surface surface() { + var start = point("0.00", "0.00"); + var path = new CurvePart(start, List.of(new Straight(point("10.00", "0.00")), + new Arc(point("15.0000001", "5.0000001"), point("10.00", "10.00"), Optional.empty()), + new Straight(point("0.00", "10.00")), new Straight(start))); + return new Surface(new Ring(List.of(new Polyline(List.of(path)))), List.of()); + } + + private static Coordinate point(String x, String y) { + return new Coordinate(List.of(new NumberValue(x, new BigDecimal(x)), new NumberValue(y, new BigDecimal(y)))); + } + + static TransferPlan ready(CompiledSchema schema) { + return assertInstanceOf(TransferPlanningResult.Ready.class, TransferPlanner.plan(schema)).plan(); + } + + static final class MemoryReader extends TransferReader { + private final Iterator events; + int pulls; + int releases; + + MemoryReader(TransferPlan plan, Iterator events) { + super(plan); + this.events = events; + } + + @Override + protected Optional readEvent(CancellationToken cancellation) { + pulls++; + return events.hasNext() ? Optional.of(events.next()) : Optional.empty(); + } + + @Override + protected void release() { + releases++; + } + } + + static final class MemoryWriter extends TransferWriter { + final List events = new ArrayList<>(); + int releases; + + MemoryWriter(TransferPlan plan) { + super(plan); + } + + @Override + protected void writeEvent(TransferEvent event, CancellationToken cancellation) { + events.add(event); + } + + @Override + protected void release() { + releases++; + } + } +} From 475968df99e64420f7026a8d7bd9b7fab644db57 Mon Sep 17 00:00:00 2001 From: Stefan Ziegler Date: Fri, 6 Nov 2026 19:27:57 +0100 Subject: [PATCH 5/6] Document the integrated format-neutral transfer core Describe the delivered plan and stream contracts and the lifetime of the transfer identity checks. Record transient attributes as schema metadata and leave wire encoding at adapter boundaries. Jenesis 0.18.1 with Temurin 25.0.2 passed 32 transfer tests with no skips. The single module-check passed for all 12 JVM products, including named and classpath consumers and the expected forbidden-access rejection. The transfer runtime closure contains the transfer and model artifacts. Final independent OpenCode 2.2.5 / deepseek/deepseek-flash gate PASS on the entire branch diff. A separate 40-check probe compiled the real Base/Survey 2.4 closure using the module-check artifacts and exercised projection, streaming, wrong-basket rejection and retry, and schema immutability. --- docs/architecture.md | 2 +- docs/design/transfer-validation-database.md | 41 +++++-------------- .../guru/interlis/transfer/AttributeSlot.java | 1 + 3 files changed, 12 insertions(+), 32 deletions(-) diff --git a/docs/architecture.md b/docs/architecture.md index 81419f46..7eef1cfd 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -20,7 +20,7 @@ Die zugehörigen POMs beschreiben die konkreten Abhängigkeiten und Scopes. | `interlis-repository` | workspace, model | Datei-/HTTP-Host, Indexe, Beschaffung, Cache und Lock | | `interlis-projection` | compiler, model | Diagramm-DTOs und reine Projektion | | `interlis-documentation` | model, projection | Lesende Dokumentationsprojektion und Rendering | -| `interlis-transfer` | model | Formatunabhängige Transferpläne aus gültigen Schemasichten | +| `interlis-transfer` | model | Formatunabhängige Transferpläne, Objektwerte und Streaming-Protokoll | | `interlis-compiler-cli` | compiler, model, repository, workspace, projection | CLI, Prozessgrenzen, Präsentation und Ausgabe | | `interlis-compiler-gui` | model, compiler, workspace, repository | Desktop-Modellprüfung, Diagnosen, Abhängigkeiten und Repositoryexplorer | | `interlis-lsp` | workspace, compiler, projection, model, repository, documentation | LSP-/Editorhost, Scheduling und Ausgaben | diff --git a/docs/design/transfer-validation-database.md b/docs/design/transfer-validation-database.md index b810b239..cb1cc1f2 100644 --- a/docs/design/transfer-validation-database.md +++ b/docs/design/transfer-validation-database.md @@ -1,6 +1,6 @@ # Entwurf für Transfer, Validierung und Datenbanken -Status **geplant**. Der Compiler stellt die semantische Grundlage bereit. Transfer, Datenvalidierung und Datenbankabbildung sind künftige Verbraucher mit eigenen Plänen, Fähigkeiten und Abnahmen. Die API-Skizzen beschreiben Entwurfsrollen. +Der formatunabhängige Transferkern steht in `interlis-transfer` für INTERLIS 2.3 und 2.4 zur Verfügung. Formatadapter, Datenvalidierung und Datenbankabbildung sind geplant. Die API-Skizzen beschreiben den vorgesehenen Ausbau. Die Verantwortung wird entlang fachlicher Produkte geschnitten: @@ -13,7 +13,7 @@ CompiledSchema + explizite Verbraucheroptionen Ein `CompiledSchema` bleibt ein unveränderliches, fachlich geprüftes Modell. Projektionen erzeugen weitere unveränderliche Pläne. Laufzeitdaten, Caches, Objektidentitäten, Datenbankverbindungen und Zwischenergebnisse einer Datenvalidierung gehören in laufgebundene Kontexte. -Die späteren Module heissen vorläufig `interlis-transfer`, `interlis-validation` und `interlis-db`. Sie werden erst angelegt, wenn reale implementierte Funktionalität vorliegt. Der Compiler hängt niemals von diesen Modulen ab. Die öffentlichen Plan- und Laufzeitverträge sind geplant. Der heutige Compiler erhält die nötigen Modellinformationen und belegt sie mit Metamodell- und Projektionstests. +`interlis-transfer` verwendet die öffentliche, snapshotgebundene Modellsicht. Als weitere Verbraucher sind `interlis-validation` und `interlis-db` vorgesehen. Der Compiler stellt die Modellinformationen bereit und belegt sie mit Metamodell- und Projektionstests. ## Gemeinsame Planverträge @@ -71,29 +71,15 @@ Pläne werden vollständig geprüft veröffentlicht. Ein abgebrochener Aufbau ve ### Eingaben und Planner -`TransferOptions` enthält das explizite `TransferProfileId`, Namens-/Übersetzungswahl, relevante Formatoptionen und Ressourcenlimits. Die konkrete Liste kompatibler Optionen wird gegen die bisherigen Werkzeuge aufgenommen. Eine unbestimmte Option `legacy=true` genügt nicht. +`TransferPlanner.plan(CompiledSchema)` verwendet die Namen der kanonischen Originaldeklarationen. Die Überladung mit `TransferPlanner.Names.DECLARED` wählt die deklarierten Namen und behält die kanonische Identität. Der Planner übernimmt Typen, Reihenfolge und Herkunft aus dem geprüften Schema. `TransferPlanningResult.Ready` enthält den vollständigen Plan im unterstützten Umfang. `Unsupported` nennt betroffene Pfade und Merkmale. Legale generische Bindungen werden für die spätere Basketkonkretisierung erhalten. -`TransferMetadata` enthält benötigte Originalmodellinformationen und provenance-bezogene Daten, die nicht zwingend dauerhaft in jedem semantischen Objekt gespeichert werden müssen. Die Quellenhaltung wird mit [Repository-Auflösung](../contracts/repositories.md) abgestimmt. Ein späterer Writer darf kein Modell aus dem Netz nachladen, nur um fehlende ursprüngliche Modelltexte zu rekonstruieren. - -`TransferPlanner` besitzt: - -```java -PlanningOutcome plan( - CompiledSchema schema, - TransferOptions options, - TransferMetadata metadata, - CancellationToken cancellation); -``` - -**Vorbedingungen:** Das Schema wurde durch das Compilation-Gate erzeugt; Optionen und Metadaten entsprechen ihren Schemaversionen; Quell- und Symbolbezüge passen zur Schemaherkunft. - -**Nachbedingungen:** `Ready` enthält eine kompakte, selbstkonsistente Beschreibung aller für den gewählten Transfer nötigen Topics, Klassen, Strukturen, Attribute, Rollen und Namen. Die Reihenfolge ist explizit. Symbolbindung und Typinformationen werden aus dem geprüften Schema übernommen und nicht aus Text erneut geraten. - -**Fehler:** Fehlende Verbraucherfähigkeit ergibt `Unsupported`. Unzulässige Kombinationen von Transferoptionen ergeben `Rejected`. Unbekannte interne Graphreferenzen ergeben `Failed`. Eine rechtmässig zur Laufzeit zu konkretisierende Domäne ist für sich kein Planungsfehler. +Explizite Transferprofile und zusätzliche Quellmetadaten gehören zum späteren Ausbau an den Adaptergrenzen. Die Quellenhaltung wird mit der [Repository-Auflösung](../contracts/repositories.md) abgestimmt. ### Planstrukturen +Die folgende Tabelle beschreibt den vorgesehenen Ausbau der Deskriptoren. + | Typ | Wesentliche Informationen | |---|---| | `TransferPlan` | Provenienz, Profil, Modelle/Topics, Tagindex, Objektdeskriptoren, benötigte Laufzeitbindungen | @@ -104,23 +90,16 @@ PlanningOutcome plan( | `RuntimeTypeDescriptor` | Aufgelöste statische Typinformation plus gegebenenfalls rechtmässige Laufzeitbindung | | `BasketBindingRequirements` | Erwartete generische beziehungsweise aufgeschobene Konkretisierungen | -`TransferPlan.findObjectByTag(String tag)` liefert ein explizites Treffer-/Nichttrefferergebnis. `TransferPlan.object(SymbolRef ref)` prüft sowohl Snapshotzugehörigkeit als auch die enthaltene Symbol-ID. Eine nackte `SymbolId` genügt an dieser öffentlichen Grenze nicht. Ein Übersetzungsname darf nicht als kanonischer Schlüssel intern die ursprüngliche Identität ersetzen. +Die Zuordnung formatspezifischer Tags gehört zum späteren Adapter. `TransferPlan.object(SymbolRef ref)` prüft die Snapshotzugehörigkeit und liefert den Deskriptor für die enthaltene Symbol-ID, falls er im Plan liegt. Der Plan hält kanonische Identität und gewählte Transferbezeichnung getrennt. `ObjectDescriptor.effectiveSlots()` liefert die fachlich richtige, unveränderliche Reihenfolge. Die Position eines überschriebenen Attributes darf nicht allein durch das Auftauchen in einer neuen lokalen Liste verändert werden. Eine Assoziation mit eigenen Attributen oder höherer Stelligkeit muss als eigenständiger semantischer Sachverhalt erhalten bleiben. ### Transferereignisse und Objektmodell -Die initiale Streaming-API orientiert sich an einem Ereignisstrom mit `StartTransfer`, `StartBasket`, `ObjectEvent`, `EndBasket` und `EndTransfer`. Die genaue historische Ereignisreihenfolge und Zusatzinformationen werden als iox-Kompatibilitätsvertrag aufgenommen. Ein `TransferReader` besitzt konzeptionell: - -```java -Optional readNext(CancellationToken cancellation); -void close(); -``` - -I/O-Ausnahmen werden im konkreten I/O-Modul definiert und in ein strukturiertes Laufzeitergebnis überführt. Der neutrale Compilerkern ist davon unabhängig. Der Reader darf das nach aussen gegebene Ereignis nicht beim nächsten `readNext` heimlich wiederverwenden und verändern. Für hohe Leistung kann später eine ausdrücklich ausgeliehene Cursor-API hinzukommen; sie muss eine andere, dokumentierte Besitzsemantik besitzen. +`TransferReader.readNext(CancellationToken)` und `TransferWriter.write(...)` prüfen die Folge `StartTransfer`, Baskets mit `StartBasket`, `ObjectEvent` und `EndBasket`, sowie `EndTransfer`. `Optional.empty()` bestätigt das Quellenende nach `EndTransfer`. Ereignisse sind unveränderlich und bleiben nach dem nächsten Aufruf lesbar. Assoziationsereignisse erhalten die Rollen und die eigenen Attribute der Beziehung. Die Einbettung im konkreten Format bestimmt der Adapter. -Ein `TransferObject` hält Identität, Typdeskriptorbezug und Attribut-/Rollenwerte eines Objekts. Ein einzelnes Objekt darf verschachtelte Strukturen besitzen. Das ist keine Aufforderung, einen ganzen Basket im Speicher zu halten. Ein übergrosses Einzelobjekt unterliegt einem eigenen Budget und einer eindeutigen Fehlerbehandlung. +Ein `TransferObject` hält Identität, Typbezug und Attribut-/Rollenwerte eines einzelnen Objekts. Adapter legen Grössen- und Tiefengrenzen für einzelne Objekte fest. Der Kern hält die Modelldeklarationen, den aktiven Basketkontext und getrennte Mengen für BID und FULL-TID. Der Speicher für diese Kennungen wächst mit ihrer Anzahl und wird bei Transferende, Fehler oder Close freigegeben. Das Datenmodell unterscheidet mindestens: @@ -134,7 +113,7 @@ Das Datenmodell unterscheidet mindestens: Eine Java-`null`-Sammlung ist kein Ersatz für diese Unterscheidungen. Ein allgemeines `Map` darf an einer Adaptergrenze hilfreich sein, aber nicht der einzige typisierte Vertrag des neuen Stacks werden. -`BasketRuntimeContext` verbindet Basketidentität, Topic, Laufzeitdomänen und Transferprofil. Die Prüfung von GENERIC-/DEFERRED-Bindungen erzeugt einen unveränderlichen `ResolvedBasketContext`; sie verändert das globale Schema nicht. Unterschiedliche Baskets dürfen unterschiedliche rechtmässige Konkretisierungen besitzen. +`StartBasket` erhält Basketidentität, Topics, Zustandsmetadaten und Laufzeitdomänen. Das Protokoll prüft die Zuordnungen gegen die vom Modell veröffentlichten Alternativen. Unterschiedliche Baskets dürfen unterschiedliche rechtmässige Konkretisierungen besitzen. Die Auswertung inkrementeller Änderungen gehört zum späteren Verbraucher. ### IOM-/iox-Adapter diff --git a/interlis-transfer/src/main/java/guru/interlis/transfer/AttributeSlot.java b/interlis-transfer/src/main/java/guru/interlis/transfer/AttributeSlot.java index b910198d..e04318fb 100644 --- a/interlis-transfer/src/main/java/guru/interlis/transfer/AttributeSlot.java +++ b/interlis-transfer/src/main/java/guru/interlis/transfer/AttributeSlot.java @@ -10,6 +10,7 @@ /** * An effective attribute in a plan object type. All references belong to the plan snapshot. + * Transient attributes describe schema metadata. Adapters omit them from transferred payloads. * * @param symbol attribute declaration * @param name attribute names and original identity From 00bd3d24fd55f1bff80ed7f05bc2c5193ce3c671 Mon Sep 17 00:00:00 2001 From: Stefan Ziegler Date: Fri, 6 Nov 2026 20:19:00 +0100 Subject: [PATCH 6/6] Honor model-fixed generic domains in transfer baskets Accept omitted CONCRETE assignments and require any supplied assignment to agree with the effective model. Preserve explicit DEFERRED selection, reject conflicting active topic choices, and keep abstract topics out of baskets. Verify neutral BID and FULL-TID scopes for 2.3 and 2.4, including both collision orders and provenance. Document the separate XTF adapter obligations. Jenesis 0.18.1 on Java 25.0.2 passed 12 baseline protocol tests and 36 final transfer tests, with no skips. The independent OpenCode 2.2.5 review used the provider alias for DeepSeek V4.1 Flash 1017 and passed a new 33-check counterprobe. --- docs/design/transfer-validation-database.md | 6 +- .../guru/interlis/transfer/TransferEvent.java | 5 +- .../interlis/transfer/TransferSequence.java | 21 +- .../transfer/TransferEventProtocolTest.java | 216 ++++++++++++++++-- 4 files changed, 215 insertions(+), 33 deletions(-) diff --git a/docs/design/transfer-validation-database.md b/docs/design/transfer-validation-database.md index cb1cc1f2..18bcf54d 100644 --- a/docs/design/transfer-validation-database.md +++ b/docs/design/transfer-validation-database.md @@ -101,6 +101,8 @@ Die Zuordnung formatspezifischer Tags gehört zum späteren Adapter. `TransferPl Ein `TransferObject` hält Identität, Typbezug und Attribut-/Rollenwerte eines einzelnen Objekts. Adapter legen Grössen- und Tiefengrenzen für einzelne Objekte fest. Der Kern hält die Modelldeklarationen, den aktiven Basketkontext und getrennte Mengen für BID und FULL-TID. Der Speicher für diese Kennungen wächst mit ihrer Anzahl und wird bei Transferende, Fehler oder Close freigegeben. +Der Kern vergleicht die Zeichenwerte innerhalb dieser Namensräume unabhängig von der Herkunft `SOURCE` oder `SYNTHETIC`. Die OID-Regeln für INITIAL und UPDATE gehören zum jeweiligen Transferprofil. Der XTF-2.3-Adapter muss für FULL zusätzlich BID und TID gemeinsam prüfen, auch wenn die gleichlautende Kennung zuerst als TID erscheint. In XTF 2.4 gilt die transferweite Eindeutigkeit der FULL-TIDs und die allgemeine Objektstromregel für Behälter- und Objektidentifikationen. Die Akzeptanz gleicher Zeichenwerte in den getrennten Namensräumen des Kerns trifft keine Aussage über die Gültigkeit einer konkreten XTF-Codierung. [Gepinnte Normbelege und P4.0-Befunde](../reference/iox-legacy.md#normbelege) + Das Datenmodell unterscheidet mindestens: - nicht vorhandenen Slot beziehungsweise Wert; @@ -113,7 +115,9 @@ Das Datenmodell unterscheidet mindestens: Eine Java-`null`-Sammlung ist kein Ersatz für diese Unterscheidungen. Ein allgemeines `Map` darf an einer Adaptergrenze hilfreich sein, aber nicht der einzige typisierte Vertrag des neuen Stacks werden. -`StartBasket` erhält Basketidentität, Topics, Zustandsmetadaten und Laufzeitdomänen. Das Protokoll prüft die Zuordnungen gegen die vom Modell veröffentlichten Alternativen. Unterschiedliche Baskets dürfen unterschiedliche rechtmässige Konkretisierungen besitzen. Die Auswertung inkrementeller Änderungen gehört zum späteren Verbraucher. +`StartBasket` erhält Basketidentität, Topics, Zustandsmetadaten und Laufzeitdomänen. Im neutralen Kern gilt bei `CONCRETE` die einzige vom Modell festgelegte Alternative. Eine mitgelieferte Zuordnung muss damit übereinstimmen. `DEFERRED` erfordert eine explizite Auswahl aus den veröffentlichten Alternativen. `ABSTRACT` beschreibt eine unaufgelöste Bindung. Ein Basket braucht ein konkretes Topic. Für alle wirksamen Verwendungen derselben generischen Domäne im Basket gilt dieselbe Konkretisierung, einschliesslich geerbter Verwendungen. + +Der XTF-2.4-Adapter muss nach §4.3.6 auch schemafeste generische Verwendungen in `domains` explizit zuordnen. Die Auswertung inkrementeller Änderungen gehört zum späteren Verbraucher. ### IOM-/iox-Adapter diff --git a/interlis-transfer/src/main/java/guru/interlis/transfer/TransferEvent.java b/interlis-transfer/src/main/java/guru/interlis/transfer/TransferEvent.java index 305cc9ce..3cee194f 100644 --- a/interlis-transfer/src/main/java/guru/interlis/transfer/TransferEvent.java +++ b/interlis-transfer/src/main/java/guru/interlis/transfer/TransferEvent.java @@ -64,8 +64,9 @@ record ModelDeclaration(SymbolRef model, LanguageVersion languageVersion, Option * @param startState opaque initial state of an incremental basket, if supplied * @param endState opaque final state of an incremental basket, if supplied * @param stableOid independently supplied stable basket identity, if present - * @param domainBindings immutable list of permitted runtime bindings for the active topics, - * covering every generic domain used by those topics, in input order + * @param domainBindings immutable list of supplied assignments for the active topics in input order. + * Deferred domains require an assignment. A supplied assignment for a fixed + * domain must agree with its model binding. */ record StartBasket(BasketId id, SymbolRef topic, List additionalTopics, BasketKind kind, Consistency consistency, Optional startState, Optional endState, diff --git a/interlis-transfer/src/main/java/guru/interlis/transfer/TransferSequence.java b/interlis-transfer/src/main/java/guru/interlis/transfer/TransferSequence.java index ed66db81..e3423033 100644 --- a/interlis-transfer/src/main/java/guru/interlis/transfer/TransferSequence.java +++ b/interlis-transfer/src/main/java/guru/interlis/transfer/TransferSequence.java @@ -198,20 +198,31 @@ private void validateDomains(List bindings, List(); + var effective = new HashMap(); for (var topic : topics) { for (var binding : topic.genericBindings()) { - required.add(binding.genericDomain()); - SymbolId concrete = selected.get(binding.genericDomain()); + SymbolId genericId = binding.genericDomain(); + SymbolId supplied = selected.get(genericId); + SymbolId concrete = switch (binding.state()) { + case CONCRETE -> supplied == null ? binding.alternatives().getFirst() : supplied; + case DEFERRED -> supplied; + case ABSTRACT -> null; + }; if (concrete == null || !binding.alternatives().contains(concrete)) { - var generic = requireDomain(new SymbolRef(plan.snapshotId(), binding.genericDomain())); + var generic = requireDomain(new SymbolRef(plan.snapshotId(), genericId)); throw new TransferProtocolException(CONTEXT_MISMATCH, "Basket for " + topic.name().declared() + " needs a permitted assignment for " + generic.name().value()); } + SymbolId previous = effective.putIfAbsent(genericId, concrete); + if (previous != null && !previous.equals(concrete)) { + var generic = requireDomain(new SymbolRef(plan.snapshotId(), genericId)); + throw new TransferProtocolException(CONTEXT_MISMATCH, + "Basket topics resolve " + generic.name().value() + " to different domains"); + } } } - if (!selected.keySet().equals(required)) { + if (!effective.keySet().containsAll(selected.keySet())) { throw new TransferProtocolException(CONTEXT_MISMATCH, "Basket assigns an unused generic domain"); } } diff --git a/interlis-transfer/src/test/java/guru/interlis/transfer/TransferEventProtocolTest.java b/interlis-transfer/src/test/java/guru/interlis/transfer/TransferEventProtocolTest.java index 39d2127e..1fe7c442 100644 --- a/interlis-transfer/src/test/java/guru/interlis/transfer/TransferEventProtocolTest.java +++ b/interlis-transfer/src/test/java/guru/interlis/transfer/TransferEventProtocolTest.java @@ -6,9 +6,12 @@ import guru.interlis.model.CompiledSchema; import guru.interlis.model.LanguageVersion; import guru.interlis.model.SymbolRef; +import guru.interlis.model.TopicSymbol.BindingState; +import guru.interlis.model.TopicSymbol.GenericBinding; import guru.interlis.source.SourceId; import guru.interlis.source.SourceRevision; import guru.interlis.source.SourceText; +import guru.interlis.transfer.TransferEvent.BasketKind; import guru.interlis.transfer.TransferEvent.DomainBinding; import guru.interlis.transfer.TransferEvent.EndBasket; import guru.interlis.transfer.TransferEvent.EndTransfer; @@ -17,12 +20,15 @@ import guru.interlis.transfer.TransferEvent.StartBasket; import guru.interlis.transfer.TransferEvent.StartTransfer; import guru.interlis.transfer.TransferIdentity.BasketId; +import guru.interlis.transfer.TransferIdentity.ObjectOid; import guru.interlis.transfer.TransferIdentity.TransferId; import guru.interlis.transfer.TransferStreamingContractTest.MemoryReader; import guru.interlis.transfer.TransferStreamingContractTest.MemoryWriter; import org.junit.jupiter.api.BeforeAll; import org.junit.jupiter.api.Test; import org.junit.jupiter.api.function.Executable; +import org.junit.jupiter.params.ParameterizedTest; +import org.junit.jupiter.params.provider.EnumSource; import java.util.ArrayList; import java.util.List; @@ -32,6 +38,7 @@ import java.util.concurrent.atomic.AtomicInteger; import static guru.interlis.transfer.TransferEvent.BasketKind.FULL; +import static guru.interlis.transfer.TransferEvent.BasketKind.INITIAL; import static guru.interlis.transfer.TransferEvent.BasketKind.UPDATE; import static guru.interlis.transfer.TransferEvent.Consistency.ADAPTED; import static guru.interlis.transfer.TransferEvent.Consistency.COMPLETE; @@ -265,29 +272,93 @@ void requiresTheBasketModelInTheHeaderAndChecksTheSuppliedVersion() { } } - @Test - void checksBasketAndFullObjectIdsAcrossBasketsInSeparateNamespaces() { + @ParameterizedTest + @EnumSource(LanguageVersion.class) + void checksBasketAndFullObjectIdsAcrossBasketsInSeparateNamespaces(LanguageVersion language) { + var fixture = compileInline(MODEL, MODEL_TEXT.replace("INTERLIS 2.4;", "INTERLIS " + language.literal() + ";")); + var fixturePlan = ready(fixture); var second = new BasketId("second", SOURCE); var duplicateBid = new BasketId(BASKET.value(), SYNTHETIC); - var tid = new TransferId(BASKET.value(), SOURCE); - var duplicateTid = new TransferId(tid.value(), SYNTHETIC); - var firstObject = object(BASKET, Optional.of(tid)); - var secondObject = object(second, Optional.of(new TransferId("other", SOURCE))); - var writer = new MemoryWriter(plan); + var firstTid = new TransferId(second.value(), SYNTHETIC); + var duplicateTid = new TransferId(firstTid.value(), SOURCE); + var secondTid = new TransferId(BASKET.value(), SOURCE); + var firstObject = object(fixture, BASKET, Optional.of(firstTid), Optional.empty()); + var secondObject = object(fixture, second, Optional.of(secondTid), Optional.empty()); + var start = startTransfer(fixture, language); + var firstBasket = startBasket(fixture, BASKET, FULL); + var secondBasket = startBasket(fixture, second, FULL); + List expected = List.of(start, firstBasket, firstObject, new EndBasket(BASKET), + secondBasket, secondObject, new EndBasket(second), new EndTransfer()); + var writer = new MemoryWriter(fixturePlan); try (writer) { - writer.write(startTransfer()); - writer.write(startBasket()); + writer.write(start); + writer.write(firstBasket); writer.write(firstObject); writer.write(new EndBasket(BASKET)); - expect(DUPLICATE_ID, () -> writer.write(startBasket(duplicateBid))); - writer.write(startBasket(second)); - expect(DUPLICATE_ID, () -> writer.write(object(second, Optional.of(duplicateTid)))); + expect(DUPLICATE_ID, () -> writer.write(startBasket(fixture, duplicateBid, FULL))); + writer.write(secondBasket); + expect(DUPLICATE_ID, () -> writer.write(object(fixture, second, Optional.of(duplicateTid), Optional.empty()))); writer.write(secondObject); writer.write(new EndBasket(second)); writer.write(new EndTransfer()); } - assertEquals(List.of(firstObject, secondObject), writer.events.stream() - .filter(ObjectEvent.class::isInstance).toList()); + assertEquals(expected, writer.events); + assertReads(fixturePlan, writer.events); + + var independentWriter = new MemoryWriter(fixturePlan); + try (independentWriter) { + expected.forEach(independentWriter::write); + } + assertEquals(expected, independentWriter.events); + assertReads(fixturePlan, independentWriter.events); + } + + @Test + void tracksBasketIdsAcrossAllKindsAndKeepsIncrementalKeysOutsideTheFullTidScope() { + var initial = new BasketId("initial", SOURCE); + var update = new BasketId("update", SOURCE); + var last = new BasketId("last", SOURCE); + var fullTid = new TransferId("full-id", SOURCE); + var incrementalTid = new TransferId("incremental-id", SOURCE); + var stableOid = new ObjectOid(fullTid.value()); + var firstObject = object(schema, BASKET, Optional.of(fullTid), Optional.empty()); + var initialObject = new ObjectEvent(objectData(schema, initial, Optional.of(incrementalTid), + Optional.of(stableOid)), Optional.of(TransferEvent.Operation.INSERT), COMPLETE); + var updatedObject = new ObjectEvent(objectData(schema, update, Optional.of(incrementalTid), + Optional.of(stableOid)), Optional.of(TransferEvent.Operation.UPDATE), COMPLETE); + var lastObject = object(schema, last, Optional.of(incrementalTid), Optional.empty()); + var start = startTransfer(); + var fullBasket = startBasket(); + var initialBasket = startBasket(schema, initial, INITIAL); + var updatedBasket = startBasket(schema, update, UPDATE); + var lastBasket = startBasket(schema, last, FULL); + var writer = new MemoryWriter(plan); + try (writer) { + writer.write(start); + writer.write(fullBasket); + writer.write(firstObject); + writer.write(new EndBasket(BASKET)); + expect(DUPLICATE_ID, () -> writer.write(startBasket(schema, BASKET, INITIAL))); + writer.write(initialBasket); + writer.write(initialObject); + writer.write(new EndBasket(initial)); + expect(DUPLICATE_ID, () -> writer.write(startBasket(schema, initial, UPDATE))); + writer.write(updatedBasket); + writer.write(updatedObject); + writer.write(new EndBasket(update)); + expect(DUPLICATE_ID, () -> writer.write(startBasket(schema, update, FULL))); + writer.write(lastBasket); + expect(DUPLICATE_ID, () -> writer.write(object(schema, last, Optional.of(fullTid), Optional.empty()))); + writer.write(lastObject); + writer.write(new EndBasket(last)); + writer.write(new EndTransfer()); + } + List expected = List.of(start, fullBasket, firstObject, new EndBasket(BASKET), + initialBasket, initialObject, + new EndBasket(initial), updatedBasket, updatedObject, new EndBasket(update), lastBasket, lastObject, + new EndBasket(last), new EndTransfer()); + assertEquals(expected, writer.events); + assertReads(plan, writer.events); } @Test @@ -338,6 +409,84 @@ void preservesIncrementalStatesConsistencyAndRuntimeDomainAssignments() { } } + @Test + void acceptsAnImplicitOrMatchingConcreteAssignmentForInheritedUses() { + var fixture = compileInline(MODEL, """ + INTERLIS 2.4; + MODEL ProtocolFixture AT "https://example.org" VERSION "1" = + DOMAIN G (GENERIC) = COORD NUMERIC, NUMERIC; + A = COORD 0..100,0..100; + B = COORD 200..300,200..300; + CONTEXT Default = G = A; + TOPIC Base = + CLASS Entry = first: G; second: G; END Entry; + END Base; + TOPIC Data EXTENDS Base = END Data; + END ProtocolFixture. + """); + var fixturePlan = ready(fixture); + var topic = ref(fixture, MODEL, "Data"); + var generic = ref(fixture, MODEL, "G"); + var fixed = ref(fixture, MODEL, "A"); + var descriptor = fixturePlan.topic(topic).orElseThrow(); + assertEquals(List.of(ref(fixture, MODEL, "Base", "Entry")), descriptor.objectTypes()); + assertEquals(List.of(new GenericBinding(generic.symbol(), List.of(fixed.symbol()), + List.of(ref(fixture, MODEL, "Default").symbol()), BindingState.CONCRETE)), + descriptor.genericBindings()); + + var start = startTransfer(fixture, LanguageVersion.ILI24); + var implicit = domainBasket(BASKET, topic, List.of()); + var other = new BasketId("explicit", SOURCE); + var matching = domainBasket(other, topic, List.of(new DomainBinding(generic, fixed))); + var mismatched = domainBasket(BASKET, topic, List.of(new DomainBinding(generic, ref(fixture, MODEL, "B")))); + List expected = List.of(start, implicit, new EndBasket(BASKET), + matching, new EndBasket(other), new EndTransfer()); + var writer = new MemoryWriter(fixturePlan); + try (writer) { + writer.write(start); + expect(CONTEXT_MISMATCH, () -> writer.write(mismatched)); + expected.subList(1, expected.size()).forEach(writer::write); + } + assertEquals(expected, writer.events); + assertReads(fixturePlan, expected); + } + + @Test + void keepsAnAbstractGenericInThePlanAndRejectsItsBasket() { + var fixture = compileInline(MODEL, """ + INTERLIS 2.4; + MODEL ProtocolFixture AT "https://example.org" VERSION "1" = + DOMAIN G (GENERIC) = COORD NUMERIC, NUMERIC; + A = COORD 0..100,0..100; + TOPIC Data (ABSTRACT) = + CLASS Entry = point: G; END Entry; + END Data; + END ProtocolFixture. + """); + var fixturePlan = ready(fixture); + var topic = ref(fixture, MODEL, "Data"); + var generic = ref(fixture, MODEL, "G"); + assertEquals(List.of(new GenericBinding(generic.symbol(), List.of(), List.of(), BindingState.ABSTRACT)), + fixturePlan.topic(topic).orElseThrow().genericBindings()); + + var start = startTransfer(fixture, LanguageVersion.ILI24); + var supplied = new DomainBinding(generic, ref(fixture, MODEL, "A")); + var writer = new MemoryWriter(fixturePlan); + try (writer) { + writer.write(start); + expect(CONTEXT_MISMATCH, () -> writer.write(domainBasket(BASKET, topic, List.of()))); + expect(CONTEXT_MISMATCH, () -> writer.write(domainBasket(BASKET, topic, List.of(supplied)))); + writer.write(new EndTransfer()); + } + assertEquals(List.of(start, new EndTransfer()), writer.events); + assertReads(fixturePlan, writer.events); + } + + private static StartBasket domainBasket(BasketId id, SymbolRef topic, List bindings) { + return new StartBasket(id, topic, List.of(), FULL, COMPLETE, + Optional.empty(), Optional.empty(), Optional.empty(), bindings); + } + private static StartBasket updatedBasket(CompiledSchema schema, List assignments) { return new StartBasket(BASKET, ref(schema, "GenericExample", "Data"), List.of(), UPDATE, ADAPTED, Optional.of("state-9"), Optional.of("state-10"), Optional.of("basket-oid"), assignments); @@ -347,29 +496,46 @@ private static void expect(TransferProtocolException.Reason reason, Executable o assertEquals(reason, assertThrows(TransferProtocolException.class, operation).reason()); } + private static void assertReads(TransferPlan plan, List events) { + try (var reader = new MemoryReader(plan, events.iterator())) { + for (TransferEvent event : events) { + assertEquals(Optional.of(event), reader.readNext(TOKEN)); + } + assertEquals(Optional.empty(), reader.readNext(TOKEN)); + } + } + private static StartTransfer startTransfer() { + return startTransfer(schema, LanguageVersion.ILI24); + } + + private static StartTransfer startTransfer(CompiledSchema fixture, LanguageVersion language) { return new StartTransfer(Optional.empty(), Optional.empty(), Optional.empty(), List.of( - new ModelDeclaration(ref(schema, MODEL), LanguageVersion.ILI24, Optional.of("1"), Optional.empty()))); + new ModelDeclaration(ref(fixture, MODEL), language, Optional.of("1"), Optional.empty()))); } private static StartBasket startBasket() { - return startBasket(BASKET); + return startBasket(schema, BASKET, FULL); } - private static StartBasket startBasket(BasketId id) { - return new StartBasket(id, ref(schema, MODEL, "Data"), List.of(), FULL, COMPLETE, + private static StartBasket startBasket(CompiledSchema fixture, BasketId id, BasketKind kind) { + return new StartBasket(id, ref(fixture, MODEL, "Data"), List.of(), kind, COMPLETE, Optional.empty(), Optional.empty(), Optional.empty(), List.of()); } private static ObjectEvent object(BasketId basket) { - return object(basket, Optional.empty()); + return object(schema, basket, Optional.empty(), Optional.empty()); + } + + private static ObjectEvent object(CompiledSchema fixture, BasketId basket, Optional tid, + Optional oid) { + return new ObjectEvent(objectData(fixture, basket, tid, oid), Optional.empty(), COMPLETE); } - private static ObjectEvent object(BasketId basket, Optional tid) { - var identity = new TransferIdentity(basket, Optional.empty(), tid, Optional.empty(), - Optional.empty()); - return new ObjectEvent(new TransferObject(identity, - new TransferValue.Structured(ref(schema, MODEL, "Data", "Entry"), Map.of())), - Optional.empty(), COMPLETE); + private static TransferObject objectData(CompiledSchema fixture, BasketId basket, Optional tid, + Optional oid) { + var identity = new TransferIdentity(basket, Optional.empty(), tid, oid, Optional.empty()); + return new TransferObject(identity, new TransferValue.Structured(ref(fixture, MODEL, "Data", "Entry"), + Map.of())); } }