Skip to content

feat(nfs): real modal.NetworkFileSystem -> EFS mount support (calque#91 Workstream B) - #186

Merged
scttfrdmn merged 1 commit into
mainfrom
feat/calque-91b-networkfilesystem
Aug 15, 2026
Merged

feat(nfs): real modal.NetworkFileSystem -> EFS mount support (calque#91 Workstream B)#186
scttfrdmn merged 1 commit into
mainfrom
feat/calque-91b-networkfilesystem

Conversation

@scttfrdmn

Copy link
Copy Markdown
Contributor

Summary

  • Adds real modal.NetworkFileSystem → EFS support, the second real-mapping workstream tracked by calque#91 (Workstream A, modal.CloudBucketMount → S3, shipped in feat(volume): real modal.CloudBucketMount -> S3 mount support (calque#91 Workstream A) #185 / v0.5.1).
  • Design: bring-your-own EFS, never auto-created. Modal's own create_if_missing=False default already describes "the filesystem must exist" — calque never creates or mounts-target-provisions an EFS filesystem itself. create_if_missing=True in the source script is a distinct leak ("pre-provision an EFS filesystem tagged calque:nfs-name=<name>"), never a hard failure.
  • network_file_systems= is a separate decorator kwarg from volumes= (never nested inside it, unlike CloudBucketMount). Confirmed real-world usage exists (testdata/real-world/README.md's prior "unmodeled-arg leak" note).

What's new

  • tools/pyast/pyast.py: NetworkFileSystem.from_name(...) is now structurally tracked (self.network_file_systems, mirroring Volume.from_name's own zero-leak-on-binding posture — no more generic "recognized but not modeled" leak just for the binding). New _network_file_systems_map extracts network_file_systems={mount: nfs} into {mount_path: var_name}. create_if_missing=True gets its own distinct leak.
  • internal/ir/ir.go: NetworkFileSystemMount type; App.NetworkFileSystems (var→from_name indirection, mirrors App.Volumes); Function/Class.NetworkFileSystems (mount path → resolved mount).
  • internal/parse/parse.go: widened readConfigKwargs to also return nfs map[string]string; new decodeNetworkFileSystems; class→method inheritance mirrors CloudBucketMounts/Volumes.
  • internal/plan/efs.go (new): ResolveNetworkFileSystems (pure Go, dedupe/conflict, offline-tested); DiscoverEFSFilesystem (matches a calque:nfs-name=<name> tag via DescribeFileSystems — EFS returns tags inline, no separate ListTagsForResource call needed); ResolveMountTargetsForAZs (AZ coverage via DescribeMountTargets); EnsureNFSSecurityGroup (self-referential TCP/2049 ingress, idempotent); DefaultVPCID; NFSMountCommands (renders mount -t nfs4 lines using spawn's pkg/storage.GetEFSProfile mount-options string).
  • internal/exec/bootstrap.go: new BootstrapConfig.NFSMountLines, spliced at the same point as CloudBucketMountLines.
  • cmd/calque/realrun.go / fleetrun.go: new networkFileSystemSpecsForApp — resolves the real EFS filesystem, narrows the AZ sweep to AZs with live mount-target coverage for every required mount (a hard error, not a leak, if that narrows to zero — no placement would make the mount work), and attaches the NFS security group via spawnaws.LaunchConfig.SecurityGroupIDs (confirmed this field exists and is honored — no leak-and-move-on needed here, unlike the original plan's contingency).
  • internal/plan/spawn.go: SpawnLauncher.SecurityGroupIDs threads through to spawnaws.LaunchConfig.
  • IAM (elasticfilesystem:ClientMount/ClientWrite) is explicitly out of scope for this pass — not touched.

Architectural note (deviation from the original plan)

The plan anticipated spawnaws.LaunchConfig might have no security-group field, requiring a "leak and move on" fallback. It actually does (LaunchConfig.SecurityGroupIDs, applied to the instance's network interface) — so the NFS security group is genuinely attached, not just leaked about.

Test plan

  • go build ./...
  • go vet ./...
  • gofmt -l . (clean)
  • go test -count=1 ./... (all packages pass)
  • golangci-lint run ./... (0 issues)
  • ruff check . (all checks passed)
  • New tests: internal/parse/parse_test.go (positive resolve, class/method inheritance, create_if_missing=True leak), internal/plan/efs_test.go (ResolveNetworkFileSystems dedupe/conflict, NFSMountCommands shape/order/options — offline only; DiscoverEFSFilesystem/ResolveMountTargetsForAZs/EnsureNFSSecurityGroup are real AWS round-trips with no offline tier, matching acquire.go's own documented precedent), internal/exec/bootstrap_demo_test.go (NFSMountLines splice, coexistence with CloudBucketMountLines), internal/plan/spawn_test.go (SecurityGroupIDs threading).
  • New fixtures: testdata/scripts/network_file_system.py, testdata/scripts/network_file_system_create_if_missing.py.

…91 Workstream B)

A real NetworkFileSystem.from_name(name) used as a network_file_systems=
value (a separate decorator kwarg from volumes=, never nested inside it)
now resolves to a real EFS-over-NFS mount, bring-your-own only — calque
never auto-creates an EFS filesystem (create_if_missing=True is a distinct
leak, not a blocker), discovering the pre-provisioned filesystem via a
calque:nfs-name=<name> tag convention.

pyast.py structurally tracks the from_name(...) binding (mirroring Volume's
own zero-leak-on-binding posture) and the network_file_systems= decorator
kwarg; parse.go threads it through ir.Function/Class.NetworkFileSystems;
internal/plan/efs.go resolves the real EFS filesystem/mount-target
coverage/security group; internal/exec/bootstrap.go splices the mount
lines; cmd/calque/realrun.go and fleetrun.go narrow the AZ sweep to AZs
with live mount-target coverage (a hard error, not a leak, if that narrows
to zero) and attach a self-referential NFS/2049-ingress security group via
spawnaws.LaunchConfig.SecurityGroupIDs. IAM (ClientMount/ClientWrite) is
explicitly out of scope for this pass.

This closes out both real-mapping workstreams calque#91 was tracking
(Workstream A, CloudBucketMount -> S3, shipped in #185/v0.5.1).
modal.Dict/Queue/App.include remain deliberately leak-only, out of scope.
@scttfrdmn
scttfrdmn merged commit c6991ce into main Aug 15, 2026
7 checks passed
@scttfrdmn
scttfrdmn deleted the feat/calque-91b-networkfilesystem branch August 15, 2026 07:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant