feat(karta): consumer Workload interface with shape-blind pod mutation - #345
Draft
AviadHayumi wants to merge 10 commits into
Draft
feat(karta): consumer Workload interface with shape-blind pod mutation#345AviadHayumi wants to merge 10 commits into
AviadHayumi wants to merge 10 commits into
Conversation
pkg/karta is the single front door for consumers: New pairs a workload object with its Karta definition (validated eagerly, deep-copied, GVK checked, suspend actions verified), Tree reads, UpdatePods mutates, Suspend/Resume act on every suspendable component, Components lists each component's statically writable pod fields, and Object returns a deep copy ready to apply. mutation is an intent patch (all-pointer PodPatch) routed as individual raw leaf assignments under the definition's pod path - no typed round-trips and no whole-subtree write-back, so unknown sibling fields survive and definitions where podSpecPath and metadataPath point at the same object stay intact. a field is writable only when its jq path is a statically assignable pure path (fields, literal-string keys, []); the shipped catalog's computed projections and formulas classify as typed unsupported errors before any jq runs. containerPath is read-only. capability failures list every offending field behind one ErrNotSupported sentinel, mutating calls are atomic via scratch/adopt, and explicit empty scalar values are rejected uniformly. a catalog conformance gate exercises every definition in docs/catalog: each writable field must apply cleanly to a synthetic object built from the definition's own paths, with canaries proving only the named leaves changed. pkg/karta/kartatest ships a recording fake whose capability behavior is seeded from the same WritablePodFields production uses. the tree gains a depth-first Instances iterator, and the quickstart consumes the front door end to end on JobSet and LeaderWorkerSet. Signed-off-by: aviadh <aviad.hayumi@gmail.com>
Signed-off-by: aviadh <aviad.hayumi@gmail.com>
pkg/jq becomes internal/jq: the evaluator and its validation are plumbing behind the consumer surfaces (pkg/karta, pkg/resource, pkg/tree, the v1alpha1 validator) and could previously be imported directly, bypassing every capability check the front door enforces. all in-repo importers move with it; nested modules under the github.com/run-ai/karta path prefix can still reach it, external importers cannot. BREAKING CHANGE: github.com/run-ai/karta/pkg/jq and pkg/jq/execution are no longer importable outside the repository. consumers use pkg/karta (or pkg/resource) instead. Signed-off-by: aviadh <aviad.hayumi@gmail.com>
every recorded real-cluster state replays through both eras: extraction (tree.Build then vs Workload.Tree now) must be identical, and the same scheduler+label intent is applied through the old typed component path and the new leaf-write front door. the new way must change only the intended leaves; the typed path's damage is recorded as evidence - on the core Pod recordings its whole-template write-back at path "." replaces the object and loses apiVersion, kind and status entirely. Signed-off-by: aviadh <aviad.hayumi@gmail.com>
… builder Signed-off-by: aviadh <aviad.hayumi@gmail.com>
the verb updates the component's pod template inside the workload object - it never touches Pod resources. the old name read like a pod operation. Signed-off-by: aviadh <aviad.hayumi@gmail.com>
…engine
UpdatePodTemplate now takes a PodTemplateUpdate: the typed PodPatch or a
raw Patch (the {metadata, spec} pod template view). Both compile to the
same merge patch and go through one router, so the typed vocabulary can
never drift from what the raw door routes.
The raw patch is strict-decoded against a partial corev1.PodTemplateSpec,
so a typo like spec.labels fails validation instead of silently aliasing
a route. metadata is restricted to labels and annotations. Nulls are
rejected with every offending path listed; scalars replace, including
zero values. spec.containers merges by name, and a single unnamed entry
targets the sole container - on fragmented shapes it bridges image and
resources to their own paths.
WithInstances now validates each physical route against instanceIdPath
instead of one shape-level base, which makes per-instance targeting work
on fragmented shapes whose paths iterate with the ids.
PodField constants become logical view paths, so capability errors from
both doors carry one vocabulary.
Signed-off-by: aviadh <aviad.hayumi@gmail.com>
Performance: NewPrimitiveRunner and NewComponentFactoryFromPrimitiveObject skip the defensive JSON round-trip when the data is already JSON-primitive (a runner's own output). A mutation on a 1MB object drops from ~16ms to ~8.6ms; ten raw scalar leaves drop from ~9ms to ~1.6ms. Number canonicalization to float64 stays as is and is now documented on Object() and the runner. The kartatest fake no longer drifts from production: NewFromKarta fakes validate every update through the exported ValidatePodTemplateUpdate - the exact compile path production runs - so raw patches, empty patches and unknown components fail identically. ComponentInfos and AllPodFields are exported and shared, removing the copy-pasted component describer and field universe. Also: single karta: prefix in printed errors (the sentinel text is no longer embedded twice), mergeRawMap takes the canonical map directly (three duplicated stringify builders removed), the pureBase closure threading replaced with a package function, slices/maps stdlib over hand-rolled loops, syntheticObject moved next to its only caller. Signed-off-by: aviadh <aviad.hayumi@gmail.com>
Three review agents (a zero-context reader, a stdlib-precedent reviewer and a dead-code audit) drove this pass. Renames - every invented word replaced with the plain or established one: withScratch is now mutateAtomically (run on a copy, keep only on success), the leaf/raw vocabulary becomes patchField/pathWrite with a fieldPath (the downward API's own word), flattenPatch flattens, routeFields/routeFragmentedField/routeUnderFragment route, toUnstructured matches the k8s converter it mirrors, mergeMap matches apimachinery's own patch engine, joinPath joins, and the targets puzzle (targeted.set[targeted.order[i]]) reads as instances.selected[instances.all[i]]. The pure-path jargon is gone: parseWritablePath and writableBase align with isWritablePath and WritablePodFields, and pathpure.go is writable_path.go. Deletions - about 150 lines with no behavior any test asserts: the fake's never-executed static capability path and its speculative knobs (interceptors, canned tree/object results), the orphaned SetFields, AllPodFields and stringMapToAny, both unused error predicates (errors.As covers them), the merge-map rule inlined at its only use, and dead error plumbing in the patch walker. Signed-off-by: aviadh <aviad.hayumi@gmail.com>
…nguage A definition language whose fields are not jq paths (#330) plugs into the router at one point: writableBase, the single constructor of write routes. Document that seam and the shape-classification contract on specShape, say 'no write route' instead of naming paths in the route errors, and add the handshake fixture - a definition whose spec group exists with no writable paths: reads work, every write fails with one typed ErrNotSupported, and the capability list is empty. A future language flips exactly that fixture from rejected to routed. Signed-off-by: aviadh <aviad.hayumi@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
implements #341 - the consumer front door, designed so it cannot corrupt any definition in the shipped catalog.
what this adds
pkg/karta- one small interface:the mutation design
UpdatePodTemplatetakes aPodTemplateUpdate- the typedPodPatchor a rawPatchmap. the typed patch compiles into the raw form, so ONE router serves both and the typed fields cannot drift from what the engine routes.corev1.PodTemplateSpec- k8s itself is the schema, a typo likespec.labelsfails before anything runs.metadatais restricted to labels and annotations in v1.spec.containers, which merges by container name. one unnamed containers entry means the sole container, and on fragmented shapes it routes image and resources to their own definition paths. null never deletes in v1, every offending path listed at once.{requests: ...}, KServe's select-pipecontainerPath, every// 1). those return a typed unsupported error before any jq runs.podSpecPathandmetadataPathpoint at the same object (KServe's transformer) are safe by construction.WithInstancesvalidates every physical route againstinstanceIdPath, so per-instance targeting works on any shape whose paths iterate with the ids - fragmented included.spec.tolerations,metadata.labels) in one*UnsupportedFieldsError, classifiable witherrors.Is(err, karta.ErrNotSupported).replicasPathin the catalog carries formulas and literals. extension path documented in add a small karta.Workload interface - one clear front door for consumers #341.how it is verified
docs/catalog: every writable field applied to a synthetic object built from the definition's own paths, with canaries proving only the named leaves changed.test/e2e/recorded_data) through both eras: extraction byte-identical everywhere; mutation changed zero unintended leaves. the old typed path destroyed core Pod recordings outright (whole-object replacement, status lost) - recorded in the spec as evidence.make checkgreen, 32 specs, lint 0 issues.performance
measured on a 1MB workload: one typed field 8.5ms, ten raw scalar fields 1.4ms. the engine skips the defensive json round-trip when data is already json-primitive (
NewPrimitiveRunner). numbers canonicalize to float64 (documented onObject()); integers above 2^53 are not supported - pre-existing engine behavior, now stated.the jq engine moves under internal/ (breaking, deliberate)
pkg/jq->internal/jq. leaving the evaluator importable lets a consumer bypass every capability check the front door enforces. withinternal/the compiler owns the boundary: consumers getpkg/karta(andpkg/resourceas the power api), nothing else compiles. all in-repo importers moved; nested modules undergithub.com/run-ai/karta/still build (go's internal rule is prefix based). externalpkg/jqimporters break at compile time - that is the point, shout if anyone depends on it today.also
pkg/karta/kartatest: a recording fake that validates every update throughkarta.ValidatePodTemplateUpdate- the exact code path production runs, so the fake cannot drift from it.tree.Instances(): depth-first (component, instance) iterator.PodView) is deferred: normalized PodView: shape-blind pod reads to match the shape-blind mutation api #344.size: +1598 logic, +1141 tests, +16 docs (the
internal/move is rename-detected, ~0).