feat(nfs): real modal.NetworkFileSystem -> EFS mount support (calque#91 Workstream B) - #186
Merged
Merged
Conversation
…91 Workstream B) A real NetworkFileSystem.from_name(name) used as a network_file_systems= value (a separate decorator kwarg from volumes=, never nested inside it) now resolves to a real EFS-over-NFS mount, bring-your-own only — calque never auto-creates an EFS filesystem (create_if_missing=True is a distinct leak, not a blocker), discovering the pre-provisioned filesystem via a calque:nfs-name=<name> tag convention. pyast.py structurally tracks the from_name(...) binding (mirroring Volume's own zero-leak-on-binding posture) and the network_file_systems= decorator kwarg; parse.go threads it through ir.Function/Class.NetworkFileSystems; internal/plan/efs.go resolves the real EFS filesystem/mount-target coverage/security group; internal/exec/bootstrap.go splices the mount lines; cmd/calque/realrun.go and fleetrun.go narrow the AZ sweep to AZs with live mount-target coverage (a hard error, not a leak, if that narrows to zero) and attach a self-referential NFS/2049-ingress security group via spawnaws.LaunchConfig.SecurityGroupIDs. IAM (ClientMount/ClientWrite) is explicitly out of scope for this pass. This closes out both real-mapping workstreams calque#91 was tracking (Workstream A, CloudBucketMount -> S3, shipped in #185/v0.5.1). modal.Dict/Queue/App.include remain deliberately leak-only, out of scope.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
modal.NetworkFileSystem→ EFS support, the second real-mapping workstream tracked by calque#91 (Workstream A,modal.CloudBucketMount→ S3, shipped in feat(volume): real modal.CloudBucketMount -> S3 mount support (calque#91 Workstream A) #185 / v0.5.1).create_if_missing=Falsedefault already describes "the filesystem must exist" — calque never creates or mounts-target-provisions an EFS filesystem itself.create_if_missing=Truein the source script is a distinct leak ("pre-provision an EFS filesystem taggedcalque:nfs-name=<name>"), never a hard failure.network_file_systems=is a separate decorator kwarg fromvolumes=(never nested inside it, unlikeCloudBucketMount). Confirmed real-world usage exists (testdata/real-world/README.md's prior "unmodeled-arg leak" note).What's new
tools/pyast/pyast.py:NetworkFileSystem.from_name(...)is now structurally tracked (self.network_file_systems, mirroringVolume.from_name's own zero-leak-on-binding posture — no more generic "recognized but not modeled" leak just for the binding). New_network_file_systems_mapextractsnetwork_file_systems={mount: nfs}into{mount_path: var_name}.create_if_missing=Truegets its own distinct leak.internal/ir/ir.go:NetworkFileSystemMounttype;App.NetworkFileSystems(var→from_name indirection, mirrorsApp.Volumes);Function/Class.NetworkFileSystems(mount path → resolved mount).internal/parse/parse.go: widenedreadConfigKwargsto also returnnfs map[string]string; newdecodeNetworkFileSystems; class→method inheritance mirrors CloudBucketMounts/Volumes.internal/plan/efs.go(new):ResolveNetworkFileSystems(pure Go, dedupe/conflict, offline-tested);DiscoverEFSFilesystem(matches acalque:nfs-name=<name>tag viaDescribeFileSystems— EFS returns tags inline, no separateListTagsForResourcecall needed);ResolveMountTargetsForAZs(AZ coverage viaDescribeMountTargets);EnsureNFSSecurityGroup(self-referential TCP/2049 ingress, idempotent);DefaultVPCID;NFSMountCommands(rendersmount -t nfs4lines using spawn'spkg/storage.GetEFSProfilemount-options string).internal/exec/bootstrap.go: newBootstrapConfig.NFSMountLines, spliced at the same point asCloudBucketMountLines.cmd/calque/realrun.go/fleetrun.go: newnetworkFileSystemSpecsForApp— resolves the real EFS filesystem, narrows the AZ sweep to AZs with live mount-target coverage for every required mount (a hard error, not a leak, if that narrows to zero — no placement would make the mount work), and attaches the NFS security group viaspawnaws.LaunchConfig.SecurityGroupIDs(confirmed this field exists and is honored — no leak-and-move-on needed here, unlike the original plan's contingency).internal/plan/spawn.go:SpawnLauncher.SecurityGroupIDsthreads through tospawnaws.LaunchConfig.elasticfilesystem:ClientMount/ClientWrite) is explicitly out of scope for this pass — not touched.Architectural note (deviation from the original plan)
The plan anticipated
spawnaws.LaunchConfigmight have no security-group field, requiring a "leak and move on" fallback. It actually does (LaunchConfig.SecurityGroupIDs, applied to the instance's network interface) — so the NFS security group is genuinely attached, not just leaked about.Test plan
go build ./...go vet ./...gofmt -l .(clean)go test -count=1 ./...(all packages pass)golangci-lint run ./...(0 issues)ruff check .(all checks passed)internal/parse/parse_test.go(positive resolve, class/method inheritance,create_if_missing=Trueleak),internal/plan/efs_test.go(ResolveNetworkFileSystemsdedupe/conflict,NFSMountCommandsshape/order/options — offline only;DiscoverEFSFilesystem/ResolveMountTargetsForAZs/EnsureNFSSecurityGroupare real AWS round-trips with no offline tier, matchingacquire.go's own documented precedent),internal/exec/bootstrap_demo_test.go(NFSMountLinessplice, coexistence withCloudBucketMountLines),internal/plan/spawn_test.go(SecurityGroupIDsthreading).testdata/scripts/network_file_system.py,testdata/scripts/network_file_system_create_if_missing.py.