diff --git a/docs/prd/plugins/extensions/component-engines.md b/docs/prd/plugins/extensions/component-engines.md index 7ec9726dbfa..fbe913989e8 100644 --- a/docs/prd/plugins/extensions/component-engines.md +++ b/docs/prd/plugins/extensions/component-engines.md @@ -54,7 +54,7 @@ Generated content is evaluated bytes, not a second template. Validate destinatio Installation must not enable discovery. Disabled providers must cause no startup, downloads, or directory scans during ordinary commands. -Add discovered definitions before commands reject projects without YAML stacks. Feed imports and discovery into the same resolved stack map. Reuse naming precedence and stable logical coordinates. Cache paths must not become instance names. +Add discovered definitions before commands reject projects without YAML stacks. Discovery contributes provider-derived stacks. An explicit import contributes components to the importing stack and never a stack of its own: the host merges the returned components into the manifest that declares the import, so mixins and catalogs inherit them, and it removes imported units from automatic discovery. Reuse naming precedence and stable logical coordinates. Cache paths must not become instance names. A provider reports non-fatal findings, such as an unmatched pattern or a skipped symlink, as warning-severity diagnostics. Discovery defaults to no commands, network, secret decryption, writes, or state reads. Host API grants must enforce the policy. These restrictions do not sandbox trusted native code at the operating-system level. Report denied effects; never omit instances or substitute empty values. @@ -101,7 +101,7 @@ Cancellation must stop invocations, processes, callbacks, and descendants. Test |---|---|---| | Protocol and SDK | Independent services, lossless values, optional registration, native transport tests. | Implemented in this branch; not activated by installation. | | Host activation | Static descriptors, provider settings schemas, private registry, package selection, and service adapters. | Implemented; opt-in CLI discovery is wired. | -| Host composition | Flags, scoped services, discovery/import and engine adapters. | Scoped discovery and native preparation are wired. Imports and typed plugin flags remain. | +| Host composition | Flags, scoped services, discovery/import and engine adapters. | Scoped discovery, imports into the importing stack, and native preparation are wired. Typed plugin flags remain. | | Terragrunt plugin | Separately built SDK package runs the advanced native fixture on stock Atmos. | Local package runs advanced and nested-stack fixtures through the CLI. Published installation remains. | | Integration | Lists, state functions, affected, dependencies, UI, saved plans, and Pro share instances. | Lists and dependency scheduling are wired. Remaining integrations require validation. | | Compatibility | Differential tests for supported constructs and explicit failures for unsupported behavior. | Required. | diff --git a/docs/prd/terragrunt-support.md b/docs/prd/terragrunt-support.md index 2be1fbad38e..1d62bf8cc7c 100644 --- a/docs/prd/terragrunt-support.md +++ b/docs/prd/terragrunt-support.md @@ -66,8 +66,14 @@ Provide settings for the executable/version in executable mode, project/discover - Expand stacks without treating generated cache directories as separate projects. - Dispatch `terragrunt://` through a generic import interface. Share the evaluator with discovery. - Add definitions before commands require YAML stacks. Reuse naming precedence and path fallback. -- Keep logical coordinates stable across imports, discovery, generated directories, and machines. -- Deduplicate imports/discovery; fail on conflicting definitions, including YAML collisions. +- Name automatically discovered units from paths: the component is the unit directory name; the stack is the directory above it (`default` and `root` at the base path); an explicit stack file names the stack after its own directory and the component after the unit's path inside the stack. Name templates and patterns do not apply to discovered stacks. +- An import merges its units into the importing stack as components. Imports never create a stack. A mixin's import applies to every stack that imports the mixin; the same unit may be imported by several stacks. Imports ignore `included_paths` and `excluded_paths`, work without automatic discovery, and fail when they select no unit. An imported unit is removed from automatic discovery. +- A YAML component with an imported unit's name is an override, never a conflict: the unit is the lowest layer of the importing manifest and all stack configuration deep-merges over it. Precedence for `vars` is the unit's evaluated inputs, then the stack's merged `vars`; stack `env` wins over the unit's `extra_arguments` environment. Only two different units with one name in a stack conflict. The unit owns its state backend, module (`metadata.component`, `source.uri`), and engine; overriding those is an error, never a silent ignore. +- Keep logical coordinates stable across discovery, generated directories, and machines. An imported unit's coordinates are the importing stack and the unit's name. +- Use one pattern language for `included_paths` and `excluded_paths`: doublestar globs relative to the base path that select the matched path and everything below it. Report an include pattern that selects nothing, a `terragrunt.hcl` without `terraform.source`, and skipped symlinks as warnings. +- Follow Terragrunt v1.2.1 for symlinks: do not follow directory links, treat a symlinked `terragrunt.hcl` or `terragrunt.stack.hcl` as a unit at the link's location, and report what is skipped. +- Fail on two different units with one component name in one stack, naming both definitions. A discovered stack that has the name of a stack manifest fails too. This rule does not apply to imported units: an imported unit with a YAML component of the same name merges instead (see above). +- Do not attribute a plugin-provided component to an unrelated YAML manifest in provenance output. - Leave source configuration and state unchanged during listing. Report denied effects. Imported units have type `terraform`. Native mode reports engine `terraform` or `opentofu`; executable mode reports `terragrunt`. Do not infer the engine from HCL. diff --git a/errors/errors.go b/errors/errors.go index a50acdeeff5..4d788acaea2 100644 --- a/errors/errors.go +++ b/errors/errors.go @@ -243,6 +243,9 @@ var ( // ErrPlanVerificationFailed is returned when a stored planfile differs from the current state during --verify-plan. ErrPlanVerificationFailed = errors.New("plan verification failed: stored plan differs from current state") + // ErrSavedPlanNotFound is returned when applying a plugin-managed component's saved plan and none exists. + ErrSavedPlanNotFound = errors.New("no saved plan found") + // ErrStoredPlanfileMissing is returned when a deploy expected a stored planfile to verify against but none was found. ErrStoredPlanfileMissing = errors.New("plan verification failed: no stored planfile was found to verify against") diff --git a/errors/plugin.go b/errors/plugin.go index 425a8af2220..abd53c586de 100644 --- a/errors/plugin.go +++ b/errors/plugin.go @@ -42,6 +42,9 @@ var ( ErrPluginUnauthorized = errors.New("plugin API invocation is not authorized") ErrPluginPayloadTooLarge = errors.New("plugin payload exceeds size limit") + // ErrPluginComponentsFailed identifies selected components whose configuration provider reported a failure. + ErrPluginComponentsFailed = errors.New("selected components failed configuration discovery; no infrastructure command was run") + // ErrPluginBundlePathForbidden rejects hidden, credential-like, or special files in a plugin bundle. ErrPluginBundlePathForbidden = errors.New("plugin bundle contains a forbidden file") // ErrPluginBundleExecutableInvalid rejects an executable that is not a native binary for its declared platform. @@ -72,10 +75,20 @@ var ( ErrPluginUnlockedInCI = errors.New("plugin is declared but not locked; automatic installation is disabled in CI") // ErrPluginFlagConflict identifies flags that cannot be combined. ErrPluginFlagConflict = errors.New("plugin flags cannot be combined") + // ErrConfigurationImportConflict identifies two different imported provider units that would become + // one component of a stack. A YAML component of the same name is an override, not a conflict. + ErrConfigurationImportConflict = errors.New("provider import conflicts with another component definition") + // ErrConfigurationOverrideUnsupported identifies stack configuration that tries to change a setting + // a configuration provider owns, such as the state backend or the module of a unit. + ErrConfigurationOverrideUnsupported = errors.New("stack configuration cannot override a setting owned by the configuration provider") + // ErrConfigurationImportEmpty identifies a provider import that resolved to no components. + ErrConfigurationImportEmpty = errors.New("provider import matched no components") // ErrComponentHookFailed identifies a provider-supplied hook that exited unsuccessfully or could not run. ErrComponentHookFailed = errors.New("component hook failed") // ErrComponentVarFileMissing identifies a required variable file that does not exist. ErrComponentVarFileMissing = errors.New("required variable file does not exist") + // ErrComponentHookWorkingDirectory identifies a hook whose working directory does not exist. + ErrComponentHookWorkingDirectory = errors.New("hook working directory does not exist") // ErrComponentOutputCapture identifies command output that could not be recorded for error hook matching. ErrComponentOutputCapture = errors.New("failed to capture command output for error hooks") diff --git a/examples/terragrunt-native/README.md b/examples/terragrunt-native/README.md index 9c91f90a3c3..754f4d92e62 100644 --- a/examples/terragrunt-native/README.md +++ b/examples/terragrunt-native/README.md @@ -31,29 +31,40 @@ atmos plugin build plugins/terragrunt export ATMOS_TERRAGRUNT_PLUGIN_PATH="$PWD/plugins/terragrunt" ``` -Copy the example into a new directory, then configure the package path: +Copy the example into a new directory and select its `dev` profile, which points the +plugin package at your build: ```shell -atmos init ./examples/terragrunt-native ./terragrunt-native --copy --no-git +atmos init ./examples/terragrunt-native ./terragrunt-native \ + --copy --no-git cd terragrunt-native -atmos config set plugins.packages.terragrunt.path "$ATMOS_TERRAGRUNT_PLUGIN_PATH" +export ATMOS_PROFILE=dev ``` -The example already enables native discovery. To enable the same behavior in an +The example already enables native discovery, and `profiles/dev/atmos.yaml` reads the +plugin path from `ATMOS_TERRAGRUNT_PLUGIN_PATH`. To enable the same behavior in an existing project, run these commands from that project's root: ```shell -atmos config set plugins.packages.terragrunt.path "$ATMOS_TERRAGRUNT_PLUGIN_PATH" +atmos config set plugins.packages.terragrunt.path \ + "$ATMOS_TERRAGRUNT_PLUGIN_PATH" +atmos config set components.terraform.base_path \ + components/terraform atmos config set components.terraform.command tofu atmos config set components.terraform.terragrunt.enabled true atmos config set components.terraform.terragrunt.mode native -atmos config set --type=yaml components.terraform.terragrunt.discovery.included_paths '["live/**"]' -atmos config set components.terraform.terragrunt.discovery.enabled true +atmos config set --type=yaml \ + components.terraform.terragrunt.discovery.included_paths \ + '["live/**"]' +atmos config set \ + components.terraform.terragrunt.discovery.enabled true ``` -The environment variable above is only a shell convenience for these commands. -The plugin path is stored in `atmos.yaml`; Atmos does not require that variable on -subsequent runs. Adjust `included_paths` to match your Terragrunt directory layout. +In an existing project the plugin path is stored in `atmos.yaml`, so Atmos does not +need the environment variable on subsequent runs. Adjust `included_paths` to match your Terragrunt directory layout; +see [discovery paths](https://atmos.tools/migration/terragrunt#discovery-paths) for the +pattern rules. `components.terraform.base_path` is required by Atmos even though +Terragrunt units do not live there; the directory does not need to exist. If you use Terraform instead, set `components.terraform.command` to `terraform` and install Terraform 1.4 or newer. diff --git a/examples/terragrunt-native/atmos.yaml b/examples/terragrunt-native/atmos.yaml index 796c7971848..cc43645367c 100644 --- a/examples/terragrunt-native/atmos.yaml +++ b/examples/terragrunt-native/atmos.yaml @@ -5,11 +5,11 @@ components: terraform: base_path: components/terraform command: tofu + # Settings for the Terragrunt configuration provider. The provider itself comes from + # plugins.packages.terragrunt; the `dev` profile supplies a local build (see profiles/dev). terragrunt: enabled: true mode: native discovery: enabled: true included_paths: [live/**] - -# Set plugins.packages.terragrunt.path to your built local package before running. diff --git a/examples/terragrunt-native/profiles/dev/atmos.yaml b/examples/terragrunt-native/profiles/dev/atmos.yaml new file mode 100644 index 00000000000..cb44173e002 --- /dev/null +++ b/examples/terragrunt-native/profiles/dev/atmos.yaml @@ -0,0 +1,12 @@ +# Development profile: runs the Terragrunt plugin from a local build instead of a published release. +# +# export ATMOS_TERRAGRUNT_PLUGIN_PATH=/path/to/atmos/plugins/terragrunt +# export ATMOS_PROFILE=dev +# +# A package with `path` is not fetched, locked, or signature-checked, and Atmos warns once per run. +# Once the plugin is published, declare `source` and `version` under plugins.packages.terragrunt in +# atmos.yaml and drop this profile. +plugins: + packages: + terragrunt: + path: !env ATMOS_TERRAGRUNT_PLUGIN_PATH diff --git a/internal/exec/configuration_imports.go b/internal/exec/configuration_imports.go new file mode 100644 index 00000000000..2c52f9a1f36 --- /dev/null +++ b/internal/exec/configuration_imports.go @@ -0,0 +1,13 @@ +package exec + +import ( + "github.com/cloudposse/atmos/pkg/component/configuration" + "github.com/cloudposse/atmos/pkg/schema" +) + +// importConfigurationComponents routes configured provider schemes before YAML or +// remote-source resolution. The provider's units come back as a manifest fragment that +// joins the importing manifest, so they are components of the importing stack. +func importConfigurationComponents(config *schema.AtmosConfiguration, uri string, declaration *schema.StackImport, loadOptions *StackLoadOptions) (map[string]any, bool, error) { + return configuration.ImportComponents(config, uri, declaration, loadOptions != nil && loadOptions.StaticOnly) +} diff --git a/internal/exec/configuration_imports_test.go b/internal/exec/configuration_imports_test.go index 66314c9d5ed..9885fdb22e4 100644 --- a/internal/exec/configuration_imports_test.go +++ b/internal/exec/configuration_imports_test.go @@ -10,33 +10,180 @@ import ( "go.uber.org/mock/gomock" errUtils "github.com/cloudposse/atmos/errors" + componentadapter "github.com/cloudposse/atmos/pkg/plugin/adapter/component" + configurationadapter "github.com/cloudposse/atmos/pkg/plugin/adapter/configuration" + pluginv1 "github.com/cloudposse/atmos/pkg/plugin/proto/atmos/plugin/v1" "github.com/cloudposse/atmos/pkg/schema" ) +//go:generate go run go.uber.org/mock/mockgen@v0.6.0 -source=$GOFILE -destination=mock_configuration_imports_test.go -package=$GOPACKAGE -mock_names=importingRuntime=MockImportingRuntime,configurationProvider=MockConfigurationProvider + +// importingRuntime is a configuration runtime that also resolves provider imports. +type importingRuntime interface { + schema.ComponentConfigurationRuntime + schema.ComponentConfigurationImportRuntime +} + +// configurationProvider is the plugin boundary a configuration session calls. +type configurationProvider interface { + configurationadapter.Provider +} + +// The generated mocks implement the contracts that the code under test asserts. +var ( + _ importingRuntime = (*MockImportingRuntime)(nil) + _ configurationProvider = (*MockConfigurationProvider)(nil) +) + +// importRuntime holds what the generated import runtime returns and records the URIs it is asked +// for. Tests change its fields to return a specific unit, a unit per import, or an error. type importRuntime struct { - *MockComponentConfigurationRuntime imports []string err error + // unit replaces the default component returned for "app". + unit map[string]any + // units, when set, are returned one per import, in order. + units []map[string]any + // bySource returns a unit whose configuration source is the import URI. + bySource bool } -func (r *importRuntime) RegisterImport(_ context.Context, uri string) (bool, error) { - r.imports = append(r.imports, uri) - return true, r.err +// mock returns a generated runtime that answers ImportComponents from the fixture at call time. +// Discover and Prepare have no expectations, so gomock rejects them. +func (r *importRuntime) mock(t *testing.T) *MockImportingRuntime { + t.Helper() + runtime := NewMockImportingRuntime(gomock.NewController(t)) + runtime.EXPECT().ImportComponents(gomock.Any(), gomock.Any()).DoAndReturn(func(_ context.Context, uri string) (map[string]any, bool, error) { + r.imports = append(r.imports, uri) + return map[string]any{"app": r.unitFor(uri)}, true, r.err + }).AnyTimes() + return runtime } -func TestYAMLImportsRegisterBeforeDiscovery(t *testing.T) { +func (r *importRuntime) unitFor(uri string) map[string]any { + switch { + case r.bySource: + return map[string]any{"settings": map[string]any{"configuration": map[string]any{"source": uri}}} + case len(r.units) > 0: + return r.units[(len(r.imports)-1)%len(r.units)] + case r.unit != nil: + return r.unit + } + return map[string]any{"settings": map[string]any{"configuration": map[string]any{"source": "live/app/terragrunt.hcl"}}} +} + +// Imported units are merged into the manifest that imports them (and so into every stack that +// imports that manifest), exactly like components of an imported YAML mixin. +func TestYAMLImportsAddUnitsToTheImportingManifest(t *testing.T) { root := t.TempDir() + mixin := filepath.Join(root, "mixin.yaml") + require.NoError(t, os.WriteFile(mixin, []byte("import:\n - terragrunt://live\n"), 0o600)) path := filepath.Join(root, "adopt.yaml") - require.NoError(t, os.WriteFile(path, []byte("import:\n - terragrunt://live\nvars:\n stage: dev\n"), 0o600)) - runtime := &importRuntime{MockComponentConfigurationRuntime: NewMockComponentConfigurationRuntime(gomock.NewController(t))} - config := &schema.AtmosConfiguration{BasePathAbsolute: root, ComponentConfigurations: runtime} + require.NoError(t, os.WriteFile(path, []byte("import:\n - mixin\nvars:\n stage: dev\n"), 0o600)) + runtime := &importRuntime{} + config := &schema.AtmosConfiguration{BasePathAbsolute: root, ComponentConfigurations: runtime.mock(t)} config.Components.Terraform.ConfigurationProviders = map[string]schema.ComponentConfiguration{"terragrunt": {Enabled: true}} importsConfig := map[string]map[string]any{} - _, err := ProcessYAMLConfigFile(config, root, path, importsConfig, nil, false, true, false, false, nil, nil, nil, nil, "") + result, err := ProcessYAMLConfigFile(config, root, path, importsConfig, nil, false, true, false, false, nil, nil, nil, nil, "") require.NoError(t, err) require.Equal(t, []string{"terragrunt://live"}, runtime.imports) - require.Equal(t, map[string]map[string]any{"terragrunt://live": {}}, importsConfig, - "a provider import must appear in the stack import report without contributing YAML") + require.Contains(t, importsConfig, "mixin", "the YAML mixin is reported as usual") + require.Contains(t, importsConfig, "terragrunt://live", "a provider import must appear in the stack import report") + require.Empty(t, importsConfig["terragrunt://live"], "a provider import contributes no YAML") + require.Len(t, importsConfig, 2) + components := result.DeepMergedConfig["components"].(map[string]any)["terraform"].(map[string]any) + require.Contains(t, components, "app") +} + +// placeholderUnit mirrors what a provider discovers: placeholder sections that the stack may override. +func placeholderUnit() map[string]any { + return map[string]any{ + "vars": map[string]any{}, + "metadata": map[string]any{"terraform_workspace": "default"}, + "settings": map[string]any{"configuration": map[string]any{"source": "live/app/terragrunt.hcl", "provider": "terragrunt"}}, + } +} + +func processUnitManifest(t *testing.T, root, name, content string, runtime *importRuntime) map[string]any { + t.Helper() + path := filepath.Join(root, name) + require.NoError(t, os.WriteFile(path, []byte(content), 0o600)) + config := &schema.AtmosConfiguration{BasePathAbsolute: root, ComponentConfigurations: runtime.mock(t)} + config.Components.Terraform.ConfigurationProviders = map[string]schema.ComponentConfiguration{"terragrunt": {Enabled: true}} + result, err := ProcessYAMLConfigFile(config, root, path, map[string]map[string]any{}, nil, false, true, false, false, nil, nil, nil, nil, "") + require.NoError(t, err) + components := result.DeepMergedConfig["components"].(map[string]any)["terraform"].(map[string]any) + app, ok := components["app"].(map[string]any) + require.True(t, ok) + return app +} + +// A YAML component named like an imported unit is an override that deep-merges over the unit, +// whether it is written in the importing manifest or in an import that comes before the unit. +func TestYAMLComponentOverridesAnImportedUnit(t *testing.T) { + root := t.TempDir() + runtime := &importRuntime{unit: placeholderUnit()} + + app := processUnitManifest(t, root, "adopt.yaml", "import:\n - terragrunt://live\ncomponents:\n terraform:\n app:\n vars:\n a: 1\n nested:\n key: value\n settings:\n team: platform\n metadata:\n terraform_workspace: blue\n", runtime) + require.Equal(t, map[string]any{"a": 1, "nested": map[string]any{"key": "value"}}, app["vars"]) + settings := app["settings"].(map[string]any) + require.Equal(t, "platform", settings["team"]) + require.Equal(t, "live/app/terragrunt.hcl", settings["configuration"].(map[string]any)["source"], "the unit marker survives a YAML settings block") + require.Equal(t, "blue", app["metadata"].(map[string]any)["terraform_workspace"]) + + // The override sits in an import that precedes the unit's import: the unit is still the base layer. + require.NoError(t, os.WriteFile(filepath.Join(root, "catalog.yaml"), []byte("components:\n terraform:\n app:\n vars:\n from_catalog: true\n metadata:\n terraform_workspace: green\n"), 0o600)) + app = processUnitManifest(t, root, "early.yaml", "import:\n - catalog\n - terragrunt://live\n", runtime) + require.Equal(t, map[string]any{"from_catalog": true}, app["vars"]) + require.Equal(t, "green", app["metadata"].(map[string]any)["terraform_workspace"], "an earlier YAML override must not lose to the unit's placeholder") + require.Equal(t, "live/app/terragrunt.hcl", app["settings"].(map[string]any)["configuration"].(map[string]any)["source"]) +} + +func TestNestedUnitImportsPreserveYAMLPrecedence(t *testing.T) { + for _, tc := range []struct { + name string + imports string + mixin string + want string + }{ + {"catalog before nested unit", "catalog, outer", "import: [terragrunt://live]\n", "green"}, + {"catalog after nested unit", "outer, catalog", "import: [terragrunt://live]\n", "green"}, + {"explicit mixin workspace wins", "catalog, outer", "import: [terragrunt://live]\ncomponents:\n terraform:\n app:\n metadata:\n terraform_workspace: blue\n", "blue"}, + {"explicit default remains YAML", "catalog, outer", "import: [terragrunt://live]\ncomponents:\n terraform:\n app:\n metadata:\n terraform_workspace: default\n", "default"}, + } { + t.Run(tc.name, func(t *testing.T) { + root := t.TempDir() + for name, body := range map[string]string{ + "catalog": "components:\n terraform:\n app:\n metadata:\n terraform_workspace: green\n vars:\n from_catalog: true\n", + "mixin": tc.mixin, + "outer": "import: [mixin]\n", + } { + require.NoError(t, os.WriteFile(filepath.Join(root, name+".yaml"), []byte(body), 0o600)) + } + unit := placeholderUnit() + runtime := &importRuntime{unit: unit} + app := processUnitManifest(t, root, "stack.yaml", "import: ["+tc.imports+"]\n", runtime) + require.Equal(t, tc.want, app["metadata"].(map[string]any)["terraform_workspace"]) + require.Equal(t, true, app["vars"].(map[string]any)["from_catalog"]) + require.Equal(t, "live/app/terragrunt.hcl", app["settings"].(map[string]any)["configuration"].(map[string]any)["source"]) + require.Equal(t, placeholderUnit(), unit, "loading must not mutate provider data") + }) + } +} + +func TestTwoUnitsWithOneComponentNameStillConflict(t *testing.T) { + root := t.TempDir() + first := &importRuntime{unit: placeholderUnit()} + second := placeholderUnit() + second["settings"].(map[string]any)["configuration"].(map[string]any)["source"] = "live/other/terragrunt.hcl" + path := filepath.Join(root, "adopt.yaml") + require.NoError(t, os.WriteFile(path, []byte("import:\n - terragrunt://live\n - terragrunt://other\n"), 0o600)) + config := &schema.AtmosConfiguration{BasePathAbsolute: root, ComponentConfigurations: (&importRuntime{units: []map[string]any{first.unit, second}}).mock(t)} + config.Components.Terraform.ConfigurationProviders = map[string]schema.ComponentConfiguration{"terragrunt": {Enabled: true}} + _, err := ProcessYAMLConfigFile(config, root, path, map[string]map[string]any{}, nil, false, true, false, false, nil, nil, nil, nil, "") + require.ErrorIs(t, err, errUtils.ErrConfigurationImportConflict) + require.ErrorContains(t, err, "live/app/terragrunt.hcl") + require.ErrorContains(t, err, "live/other/terragrunt.hcl") } func TestStackImportReportListsProviderImports(t *testing.T) { @@ -46,8 +193,8 @@ func TestStackImportReportListsProviderImports(t *testing.T) { require.NoError(t, os.WriteFile(filepath.Join(stacks, "catalog", "base.yaml"), []byte("import:\n - terragrunt://live/demo\nvars:\n team: core\n"), 0o600)) path := filepath.Join(stacks, "dev.yaml") require.NoError(t, os.WriteFile(path, []byte("import:\n - terragrunt://live/demo\n - catalog/base\nvars:\n stage: dev\n"), 0o600)) - runtime := &importRuntime{MockComponentConfigurationRuntime: NewMockComponentConfigurationRuntime(gomock.NewController(t))} - config := &schema.AtmosConfiguration{BasePathAbsolute: root, ComponentConfigurations: runtime} + runtime := &importRuntime{} + config := &schema.AtmosConfiguration{BasePathAbsolute: root, ComponentConfigurations: runtime.mock(t)} config.Components.Terraform.ConfigurationProviders = map[string]schema.ComponentConfiguration{"terragrunt": {Enabled: true}} importsConfig := map[string]map[string]any{} @@ -124,8 +271,8 @@ func TestYAMLImportsRejectProviderImportsThatCannotBeRegistered(t *testing.T) { root := t.TempDir() path := filepath.Join(root, "adopt.yaml") require.NoError(t, os.WriteFile(path, []byte("import:\n - terragrunt://live\nvars:\n stage: dev\n"), 0o600)) - runtime := &importRuntime{MockComponentConfigurationRuntime: NewMockComponentConfigurationRuntime(gomock.NewController(t)), err: errUtils.ErrPluginInvalidConfig} - config := &schema.AtmosConfiguration{BasePathAbsolute: root, ComponentConfigurations: runtime} + runtime := &importRuntime{err: errUtils.ErrPluginInvalidConfig} + config := &schema.AtmosConfiguration{BasePathAbsolute: root, ComponentConfigurations: runtime.mock(t)} config.Components.Terraform.ConfigurationProviders = map[string]schema.ComponentConfiguration{"terragrunt": {Enabled: true}} _, err := ProcessYAMLConfigFile(config, root, path, map[string]map[string]any{}, nil, false, true, false, false, nil, nil, nil, nil, "") require.ErrorIs(t, err, errUtils.ErrPluginInvalidConfig, "a rejected provider import must fail the stack load instead of being skipped") @@ -137,3 +284,266 @@ func TestYAMLImportsRejectProviderImportsThatCannotBeRegistered(t *testing.T) { require.ErrorIs(t, err, errUtils.ErrPluginInvalidConfig) require.Len(t, runtime.imports, 1, "the disabled provider is never asked to register the import") } + +// Two mixins that import different units with one component name conflict in the manifest that +// imports both, even though that manifest has no provider import of its own. +func TestUnitsImportedThroughDifferentMixinsConflict(t *testing.T) { + root := t.TempDir() + first := &importRuntime{unit: placeholderUnit()} + second := placeholderUnit() + second["settings"].(map[string]any)["configuration"].(map[string]any)["source"] = "live/other/terragrunt.hcl" + require.NoError(t, os.WriteFile(filepath.Join(root, "first.yaml"), []byte("import:\n - terragrunt://live\n"), 0o600)) + require.NoError(t, os.WriteFile(filepath.Join(root, "second.yaml"), []byte("import:\n - terragrunt://other\n"), 0o600)) + path := filepath.Join(root, "adopt.yaml") + require.NoError(t, os.WriteFile(path, []byte("import:\n - first\n - second\n"), 0o600)) + config := &schema.AtmosConfiguration{BasePathAbsolute: root, ComponentConfigurations: (&importRuntime{units: []map[string]any{first.unit, second}}).mock(t)} + config.Components.Terraform.ConfigurationProviders = map[string]schema.ComponentConfiguration{"terragrunt": {Enabled: true}} + _, err := ProcessYAMLConfigFile(config, root, path, map[string]map[string]any{}, nil, false, true, false, false, nil, nil, nil, nil, "") + require.ErrorIs(t, err, errUtils.ErrConfigurationImportConflict) + require.ErrorContains(t, err, "live/app/terragrunt.hcl") + require.ErrorContains(t, err, "live/other/terragrunt.hcl") +} + +// A manifest's Terraform overrides reach the units it imports, directly or through a mixin, without +// changing the provider's import result. +func TestStackOverridesApplyToImportedUnits(t *testing.T) { + root := t.TempDir() + unit := placeholderUnit() + runtime := &importRuntime{unit: unit} + overrides := "terraform:\n overrides:\n vars:\n region: us-west-2\n" + + app := processUnitManifest(t, root, "direct.yaml", "import:\n - terragrunt://live\n"+overrides, runtime) + require.Equal(t, map[string]any{"vars": map[string]any{"region": "us-west-2"}}, app["overrides"]) + require.NotContains(t, unit, "overrides", "the provider's unit must not be changed") + + require.NoError(t, os.WriteFile(filepath.Join(root, "mixin.yaml"), []byte("import:\n - terragrunt://live\n"), 0o600)) + app = processUnitManifest(t, root, "nested.yaml", "import:\n - mixin\n"+overrides, runtime) + require.Equal(t, map[string]any{"vars": map[string]any{"region": "us-west-2"}}, app["overrides"]) +} + +func TestStaticLoadsRejectProviderImportsBeforeRuntime(t *testing.T) { + for _, nested := range []bool{false, true} { + name := "direct" + if nested { + name = "nested" + } + t.Run(name, func(t *testing.T) { + root := t.TempDir() + file := filepath.Join(root, "dev.yaml") + imports := "terragrunt://live" + if nested { + require.NoError(t, os.WriteFile(filepath.Join(root, "mixin.yaml"), []byte("import: [terragrunt://live]\n"), 0o600)) + imports = "mixin" + } + require.NoError(t, os.WriteFile(file, []byte("name: dev\nimport: ["+imports+"]\n"), 0o600)) + runtime := &importRuntime{} + config := &schema.AtmosConfiguration{BasePathAbsolute: root, StacksBaseAbsolutePath: root, StackConfigFilesAbsolutePaths: []string{file}, ComponentConfigurations: runtime.mock(t)} + config.Components.Terraform.ConfigurationProviders = map[string]schema.ComponentConfiguration{"terragrunt": {Enabled: true}} + _, err := ExecuteDescribeStacksWithOptions(config, "dev", nil, nil, nil, + false, false, false, true, nil, nil, false, DescribeStacksErrorOptions{StaticOnly: true}) + require.ErrorIs(t, err, errUtils.ErrInvalidImport) + require.ErrorContains(t, err, "cannot inspect provider import") + require.Empty(t, runtime.imports, "static inspection must never invoke the provider runtime") + + // The same manifest still imports units during an ordinary load. + result, err := ProcessYAMLConfigFile(config, root, file, map[string]map[string]any{}, nil, false, true, false, false, nil, nil, nil, nil, "") + require.NoError(t, err) + require.Equal(t, []string{"terragrunt://live"}, runtime.imports) + require.Contains(t, result.DeepMergedConfig["components"].(map[string]any)["terraform"], "app") + + // Even a previously used runtime must not serve a later static load. + _, handled, err := importConfigurationComponents(config, "terragrunt://live", &schema.StackImport{}, &StackLoadOptions{StaticOnly: true}) + require.True(t, handled) + require.ErrorIs(t, err, errUtils.ErrInvalidImport) + require.Len(t, runtime.imports, 1) + }) + } +} + +// validationImportProvider returns a unit both through discovery and through an unreferenced +// mixin, and counts the import requests the generated provider receives. +type validationImportProvider struct { + imports int +} + +func (p *validationImportProvider) mock(t *testing.T) *MockConfigurationProvider { + t.Helper() + provider := NewMockConfigurationProvider(gomock.NewController(t)) + provider.EXPECT().ComponentType().Return("terraform").AnyTimes() + provider.EXPECT().Discover(gomock.Any(), gomock.Any()).DoAndReturn(func(_ context.Context, request *pluginv1.DiscoverComponentsRequest) (*pluginv1.DiscoverComponentsResponse, error) { + if len(request.Imports) > 0 { + p.imports++ + } + return &pluginv1.DiscoverComponentsResponse{Components: []*pluginv1.DiscoveredComponent{{ + Address: &pluginv1.ComponentAddress{Stack: "live", Component: "app", ComponentType: "terraform"}, + Provider: "terragrunt", Engine: "opentofu", Source: "live/app/terragrunt.hcl", + }}}, nil + }).AnyTimes() + return provider +} + +func TestValidateStacksResolvesUnreferencedProviderImports(t *testing.T) { + for _, path := range []string{"live/app", "missing"} { + t.Run(path, func(t *testing.T) { + root := t.TempDir() + stacks := filepath.Join(root, "stacks") + require.NoError(t, os.MkdirAll(stacks, 0o755)) + require.NoError(t, os.MkdirAll(filepath.Join(root, "live", "app"), 0o755)) + mixin := filepath.Join(stacks, "mixin.yaml") + require.NoError(t, os.WriteFile(mixin, []byte("import:\n - terragrunt://"+path+"\n"), 0o600)) + config := &schema.AtmosConfiguration{BasePath: root, BasePathAbsolute: root, StacksBaseAbsolutePath: stacks, TerraformDirAbsolutePath: filepath.Join(root, "components")} + config.Stacks.BasePath = "stacks" + config.Components.Terraform.ConfigurationProviders = map[string]schema.ComponentConfiguration{"terragrunt": {Enabled: true, Discovery: schema.ComponentDiscoveryConfig{Enabled: true, IncludedPaths: []string{"live/**"}}}} + provider := &validationImportProvider{} + mock := provider.mock(t) + session, err := configurationadapter.New(t.Context(), config, "describe affected", func(context.Context, []string) (map[string]configurationadapter.Provider, error) { + return map[string]configurationadapter.Provider{"terragrunt": mock}, nil + }) + require.NoError(t, err) + config.ComponentConfigurations = session + // No stack manifest includes the mixin. Validation must still read it after discovery, + // as it does before describe affected compares the HEAD and target inventories. + before, err := session.Discover(t.Context()) + require.NoError(t, err) + if path == "missing" { + require.ErrorContains(t, ValidateStacks(config), "does not exist in the project") + return + } + require.NoError(t, ValidateStacks(config)) + require.NoError(t, ValidateStacks(config)) + require.Equal(t, 1, provider.imports, "both validation passes and repeat validation share the import cache") + require.Same(t, session, config.ComponentConfigurations) + after, err := session.Discover(t.Context()) + require.NoError(t, err) + require.Equal(t, before, after, "validation must not remove the discovered unit") + }) + } +} + +// Exercise real provider import translation, YAML merging, stack compilation and preparation. +func TestImportedUnitBackendOptionsBeforeNormalization(t *testing.T) { + for _, tc := range []struct { + name, yaml string + rejected bool + }{ + {"component local", "components:\n terraform:\n app:\n backend:\n local:\n path: other.tfstate\n", true}, + {"component backend type", "components:\n terraform:\n app:\n backend_type: s3\n backend:\n s3:\n bucket: other\n", true}, + {"inherited local", "components:\n terraform:\n base:\n metadata:\n type: abstract\n backend:\n local:\n path: inherited.tfstate\n app:\n metadata:\n inherits: [base]\n", true}, + {"global local", "terraform:\n backend_type: local\n backend:\n local:\n path: global.tfstate\n", false}, + {"global s3", "terraform:\n backend_type: s3\n backend:\n s3:\n bucket: global\n", false}, + } { + t.Run(tc.name, func(t *testing.T) { + root := t.TempDir() + require.NoError(t, os.MkdirAll(filepath.Join(root, "live", "app"), 0o755)) + file := filepath.Join(root, "dev.yaml") + require.NoError(t, os.WriteFile(file, []byte("import: [terragrunt://live/app]\n"+tc.yaml), 0o600)) + config := &schema.AtmosConfiguration{BasePath: root, BasePathAbsolute: root, StacksBaseAbsolutePath: root} + config.Components.Terraform.ConfigurationProviders = map[string]schema.ComponentConfiguration{"terragrunt": {Enabled: true}} + provider := &backendOptionsProvider{} + placeholder, err := componentadapter.EncodeObject(map[string]any{"backend_type": "local", "backend": map[string]any{"local": map[string]any{}}}) + require.NoError(t, err) + provider.configuration = placeholder + t.Cleanup(func() { + want := 1 + if tc.rejected { + want = 0 + } + require.Equal(t, want, provider.prepares) + }) + session, err := configurationadapter.New(t.Context(), config, "plan", func(context.Context, []string) (map[string]configurationadapter.Provider, error) { + return map[string]configurationadapter.Provider{"terragrunt": provider.mock(t)}, nil + }) + require.NoError(t, err) + config.ComponentConfigurations = session + loaded, err := ProcessYAMLConfigFile(config, root, file, map[string]map[string]any{}, nil, false, true, false, false, nil, nil, nil, nil, "") + require.NoError(t, err) + compiled, _, err := ProcessStackConfig(config, root, "", "", "", "", file, loaded.DeepMergedConfig, false, false, "", nil, loaded.ImportsConfig, false) + if tc.rejected { + require.ErrorIs(t, err, errUtils.ErrConfigurationOverrideUnsupported) + require.ErrorContains(t, err, "app") + require.ErrorContains(t, err, "dev") + return // The cleanup assertion verifies rejection before provider effects. + } + require.NoError(t, err) + app := compiled["components"].(map[string]any)["terraform"].(map[string]any)["app"].(map[string]any) + backend, err := componentadapter.EncodeObject(map[string]any{"path": "unit.tfstate"}) + require.NoError(t, err) + provider.response = &pluginv1.PrepareComponentResponse{Engine: "terraform", State: &pluginv1.ComponentStateIdentity{BackendType: "local", Backend: backend, WorkingDirectory: root}} + prepared, err := session.Prepare(t.Context(), &schema.ConfigAndStacksInfo{Stack: "dev", ComponentFromArg: "app", SubCommand: "plan", ComponentSection: app, ComponentSettingsSection: app["settings"].(map[string]any)}) + require.NoError(t, err) + require.Equal(t, filepath.Join(root, "unit.tfstate"), prepared.Backend["path"]) + }) + } +} + +// backendOptionsProvider holds what the generated provider returns and counts preparations. +type backendOptionsProvider struct { + configuration *pluginv1.ComponentValue + response *pluginv1.PrepareComponentResponse + prepares int +} + +func (p *backendOptionsProvider) mock(t *testing.T) *MockConfigurationProvider { + t.Helper() + provider := NewMockConfigurationProvider(gomock.NewController(t)) + provider.EXPECT().ComponentType().Return("terraform").AnyTimes() + provider.EXPECT().Discover(gomock.Any(), gomock.Any()).DoAndReturn(func(context.Context, *pluginv1.DiscoverComponentsRequest) (*pluginv1.DiscoverComponentsResponse, error) { + return &pluginv1.DiscoverComponentsResponse{Components: []*pluginv1.DiscoveredComponent{{Address: &pluginv1.ComponentAddress{Stack: "live", Component: "app", ComponentType: "terraform"}, Provider: "terragrunt", Engine: "terraform", Source: "live/app/terragrunt.hcl", Configuration: p.configuration}}}, nil + }).AnyTimes() + provider.EXPECT().Prepare(gomock.Any(), gomock.Any()).DoAndReturn(func(context.Context, *pluginv1.ComponentRequest) (*pluginv1.PrepareComponentResponse, error) { + p.prepares++ + return p.response, nil + }).AnyTimes() + return provider +} + +func TestConfigurationImportProvenance(t *testing.T) { + for _, tc := range []struct { + name, first, second string + conflict bool + source string + }{ + {name: "ordinary YAML source settings", first: "", second: "", source: ""}, + {name: "same unit with different YAML overrides", first: "terragrunt://same", second: "terragrunt://same", source: "terragrunt://same"}, + {name: "distinct units hidden by YAML overrides", first: "terragrunt://first", second: "terragrunt://second", conflict: true}, + {name: "unit and YAML source setting", first: "terragrunt://first", second: "", source: "terragrunt://first"}, + } { + t.Run(tc.name, func(t *testing.T) { + root := t.TempDir() + write := func(name, content string) { + require.NoError(t, os.WriteFile(filepath.Join(root, name+".yaml"), []byte(content), 0o600)) + } + for i, unit := range []string{tc.first, tc.second} { + name := []string{"first", "second"}[i] + content := "" + if unit != "" { + content = "import:\n - " + unit + "\n" + } + // Distinct units get the same YAML source to ensure merging cannot hide the conflict. + yamlSource := name + if tc.conflict { + yamlSource = "same-override" + } + write(name, content+"components:\n terraform:\n app:\n settings:\n configuration:\n source: "+yamlSource+"\n") + write(name+"-nested", "import:\n - "+name+"\n") + } + write("stack", "import:\n - first-nested\n - second-nested\n") + runtime := &importRuntime{bySource: true} + config := &schema.AtmosConfiguration{BasePathAbsolute: root, ComponentConfigurations: runtime.mock(t)} + config.Components.Terraform.ConfigurationProviders = map[string]schema.ComponentConfiguration{"terragrunt": {Enabled: true}} + result, err := ProcessYAMLConfigFile(config, root, filepath.Join(root, "stack.yaml"), map[string]map[string]any{}, nil, false, true, false, false, nil, nil, nil, nil, "") + if tc.conflict { + require.ErrorIs(t, err, errUtils.ErrConfigurationImportConflict) + return + } + require.NoError(t, err) + if tc.source == "" { + require.Empty(t, result.ImportedUnitSources) + } else { + require.Equal(t, map[string]string{"app": tc.source}, result.ImportedUnitSources) + } + app := result.DeepMergedConfig["components"].(map[string]any)["terraform"].(map[string]any)["app"].(map[string]any) + require.Equal(t, "second", app["settings"].(map[string]any)["configuration"].(map[string]any)["source"]) + }) + } +} diff --git a/internal/exec/configuration_stacks.go b/internal/exec/configuration_stacks.go index 1dec831dd72..644d8dc2085 100644 --- a/internal/exec/configuration_stacks.go +++ b/internal/exec/configuration_stacks.go @@ -2,13 +2,13 @@ package exec import ( "context" - "fmt" "path/filepath" "github.com/santhosh-tekuri/jsonschema/v5" errUtils "github.com/cloudposse/atmos/errors" "github.com/cloudposse/atmos/pkg/component/configuration" + "github.com/cloudposse/atmos/pkg/perf" "github.com/cloudposse/atmos/pkg/schema" ) @@ -32,7 +32,11 @@ func mergeConfigurationStacks(config *schema.AtmosConfiguration, stacks map[stri for file, value := range stacks { existing, _ := value.(map[string]any) if file == name || existing["name"] == name { - return fmt.Errorf("%w: stack %s", errUtils.ErrDuplicatePluginExtension, name) + return errUtils.Build(errUtils.ErrDuplicatePluginExtension). + WithExplanationf("A discovered stack is named `%s`, which is already a stack defined in the stack manifests.", name). + WithContext("stack", name). + WithHintf("Rename the stack manifest `%s`, or move the discovered units to a different directory so their derived stack name differs.", name). + Err() } } processed, deferred, err := ProcessStackConfig(config, config.StacksBaseAbsolutePath, config.TerraformDirAbsolutePath, config.HelmfileDirAbsolutePath, config.PackerDirAbsolutePath, config.AnsibleDirAbsolutePath, filepath.Join(config.StacksBaseAbsolutePath, name+".yaml"), definition, false, true, "", map[string]map[string][]string{}, map[string]map[string]any{}, false) @@ -44,6 +48,19 @@ func mergeConfigurationStacks(config *schema.AtmosConfiguration, stacks map[stri return nil } +// checkConfigurationBackendOverrides passes the component's own backend layers to the +// configuration package before normalization discards their origin. Global backends remain +// allowed; the provider replaces them during preparation. +func checkConfigurationBackendOverrides(opts *ComponentProcessorOptions, result *ComponentProcessorResult, settings map[string]any) error { + defer perf.Track(nil, "exec.checkConfigurationBackendOverrides")() + + return configuration.CheckBackendOverrides( + opts.Component, opts.StackName, settings, + configuration.BackendLayer{Type: result.BaseComponentBackendType, Settings: result.BaseComponentBackendSection}, + configuration.BackendLayer{Type: result.ComponentBackendType, Settings: result.ComponentBackendSection}, + ) +} + // validateDiscoveredManifests adapts the configuration package's manifest validation // to the stack processor's compiled-schema cache. func validateDiscoveredManifests(config *schema.AtmosConfiguration, schemaPath string) ([]string, error) { diff --git a/internal/exec/describe_affected_changed_files_index.go b/internal/exec/describe_affected_changed_files_index.go index 640b797346e..9669dd13472 100644 --- a/internal/exec/describe_affected_changed_files_index.go +++ b/internal/exec/describe_affected_changed_files_index.go @@ -2,9 +2,12 @@ package exec import ( "path/filepath" + "slices" "strings" "sync" + "golang.org/x/sync/singleflight" + cfg "github.com/cloudposse/atmos/pkg/config" "github.com/cloudposse/atmos/pkg/perf" "github.com/cloudposse/atmos/pkg/schema" @@ -24,13 +27,20 @@ type changedFilesIndex struct { // outside a component base path, such as native Helm values_files. allFilesSet map[string]struct{} - resolvedOnce sync.Once - resolved *changedFilesIndex + // directories holds every changed file and each of its ancestor directories, so a + // literal folder dependency is answered by one lookup instead of a scan of all files. + directories map[string]struct{} + directoriesOnce sync.Once + resolvedOnce sync.Once + resolved *changedFilesIndex // watchMu protects filtered views for this change set. watchMu sync.Mutex watchIndexes map[string]*changedFilesIndex + // watchBuilds coalesces concurrent builds of the same exclusion set. + watchBuilds singleflight.Group + mu sync.RWMutex } @@ -252,6 +262,40 @@ func (idx *changedFilesIndex) getAllFiles() []string { func (idx *changedFilesIndex) isChangedFile(path string) bool { idx.mu.RLock() defer idx.mu.RUnlock() + if idx.allFilesSet == nil { + // An index assembled without newChangedFilesIndex has no lookup set. + return slices.Contains(idx.allFiles, filepath.Clean(path)) + } _, ok := idx.allFilesSet[filepath.Clean(path)] return ok } + +// hasChangedUnder reports whether a changed file is the path itself or lives below it. +// It answers literal folder dependencies in constant time. +func (idx *changedFilesIndex) hasChangedUnder(path string) bool { + idx.directoriesOnce.Do(func() { + idx.mu.RLock() + defer idx.mu.RUnlock() + idx.directories = make(map[string]struct{}, len(idx.allFiles)) + for _, file := range idx.allFiles { + // An ancestor already present means all of its own ancestors are present too. + for dir := file; ; dir = filepath.Dir(dir) { + if _, seen := idx.directories[dir]; seen { + break + } + idx.directories[dir] = struct{}{} + if filepath.Dir(dir) == dir { + break + } + } + } + }) + _, ok := idx.directories[filepath.Clean(path)] + return ok +} + +// hasGlobMeta reports whether a dependency path may contain pattern syntax. Literal paths +// take the indexed fast path; anything else keeps glob matching. +func hasGlobMeta(path string) bool { + return strings.ContainsAny(filepath.ToSlash(path), "*?[]{}\\") +} diff --git a/internal/exec/describe_affected_configuration.go b/internal/exec/describe_affected_configuration.go index 6740e2d80a4..13ff72f385a 100644 --- a/internal/exec/describe_affected_configuration.go +++ b/internal/exec/describe_affected_configuration.go @@ -42,21 +42,38 @@ func configurationWatchIndex(config *schema.AtmosConfiguration, settings map[str return nil, err } key := string(keyBytes) - original.watchMu.Lock() - defer original.watchMu.Unlock() - if filtered := original.watchIndexes[key]; filtered != nil { + if filtered := original.cachedWatchIndex(key); filtered != nil { return filtered, nil } - files, err := configuration.UnexcludedWatchFiles(config.BasePathAbsolute, original.getAllFiles(), patterns) + result, err, _ := original.watchBuilds.Do(key, func() (any, error) { + // Another caller may have stored the view after our first lookup. + if filtered := original.cachedWatchIndex(key); filtered != nil { + return filtered, nil + } + files, err := configuration.UnexcludedWatchFiles(config.BasePathAbsolute, original.getAllFiles(), patterns) + if err != nil { + return nil, err + } + filtered := newChangedFilesIndex(config, files, config.BasePathAbsolute) + original.watchMu.Lock() + defer original.watchMu.Unlock() + if original.watchIndexes == nil { + original.watchIndexes = make(map[string]*changedFilesIndex) + } + original.watchIndexes[key] = filtered + return filtered, nil + }) if err != nil { return nil, err } - filtered := newChangedFilesIndex(config, files, config.BasePathAbsolute) - if original.watchIndexes == nil { - original.watchIndexes = make(map[string]*changedFilesIndex) - } - original.watchIndexes[key] = filtered - return filtered, nil + return result.(*changedFilesIndex), nil +} + +// cachedWatchIndex reads a completed view without waiting for other views to build. +func (index *changedFilesIndex) cachedWatchIndex(key string) *changedFilesIndex { + index.watchMu.Lock() + defer index.watchMu.Unlock() + return index.watchIndexes[key] } // configurationMatchInputs returns the change set and dependencies to match. diff --git a/internal/exec/describe_affected_configuration_test.go b/internal/exec/describe_affected_configuration_test.go index db6af48b53a..9db5c545f79 100644 --- a/internal/exec/describe_affected_configuration_test.go +++ b/internal/exec/describe_affected_configuration_test.go @@ -4,6 +4,7 @@ import ( "context" "os" "path/filepath" + "strconv" "testing" "github.com/cloudposse/atmos/pkg/component/configuration" @@ -203,3 +204,185 @@ func TestResolvedChangedFilesPerIndex(t *testing.T) { require.NotSame(t, initial, resolvedChangedFiles(second)) require.Same(t, initial, resolvedChangedFiles(first)) } + +// watchFixture builds a project with many units that each watch their own files and folder, +// plus a changed-file set that touches only some of them. +func watchFixture(t testing.TB, units, changed int) (*changedFilesIndex, [][]schema.ComponentDependency) { + t.Helper() + root := t.TempDir() + config := &schema.AtmosConfiguration{BasePathAbsolute: root, BasePath: root} + files := make([]string, 0, changed) + for i := 0; i < changed; i++ { + files = append(files, filepath.Join("docs", "generated", "file-"+itoa(i)+".md")) + } + files = append(files, filepath.Join("live", "unit-7", "extra.tfvars"), filepath.Join("modules", "shared", "deep", "main.tf")) + deps := make([][]schema.ComponentDependency, units) + for i := range deps { + unit := filepath.Join(root, "live", "unit-"+itoa(i)) + deps[i] = []schema.ComponentDependency{ + {Kind: "file", Path: filepath.Join(unit, "terragrunt.hcl")}, + {Kind: "file", Path: filepath.Join(unit, "extra.tfvars")}, + {Kind: "folder", Path: filepath.Join(root, "modules", "mod-"+itoa(i%10))}, + } + } + // Unit 3 consumes the shared module folder, unit 4 watches a glob. + deps[3] = append(deps[3], schema.ComponentDependency{Kind: "folder", Path: filepath.Join(root, "modules", "shared")}) + deps[4] = append(deps[4], schema.ComponentDependency{Kind: "folder", Path: filepath.Join(root, "docs", "generated", "file-1[0-9].md")}) + return newChangedFilesIndex(config, files, root), deps +} + +func itoa(i int) string { return strconv.Itoa(i) } + +func TestFileWatchSelectionUsesIndex(t *testing.T) { + index, deps := watchFixture(t, 300, 9000) + var selected []int + for unit := range deps { + changed, _, _, err := isComponentDependentFolderOrFileChangedIndexed(index, deps[unit]) + require.NoError(t, err) + if changed { + selected = append(selected, unit) + } + } + // Unit 3 watches the changed shared module, unit 7 has a changed var file, and unit 4 watches + // a glob that matches generated documents: literal and glob watches agree on the result. + require.Equal(t, []int{3, 4, 7}, selected) + + // Literal watches are answered from the index: only the one glob watch scans the change set. + // Matching every watch against every changed file made selection grow with units times files. + scans := watchPatternScans.Load() + for unit := range deps { + _, _, _, err := isComponentDependentFolderOrFileChangedIndexed(index, deps[unit]) + require.NoError(t, err) + } + require.LessOrEqual(t, watchPatternScans.Load()-scans, int64(1), "literal watches must not scan every changed file") +} + +func TestFileWatchFolderSemantics(t *testing.T) { + root := t.TempDir() + config := &schema.AtmosConfiguration{BasePathAbsolute: root, BasePath: root} + index := newChangedFilesIndex(config, []string{filepath.Join("modules", "a", "b", "main.tf"), "top.md"}, root) + for _, tc := range []struct { + name string + dep schema.ComponentDependency + want bool + }{ + {"folder contains change", schema.ComponentDependency{Kind: "folder", Path: filepath.Join(root, "modules", "a")}, true}, + {"nested folder", schema.ComponentDependency{Kind: "folder", Path: filepath.Join(root, "modules", "a", "b")}, true}, + {"project root", schema.ComponentDependency{Kind: "folder", Path: root}, true}, + {"sibling prefix is not contained", schema.ComponentDependency{Kind: "folder", Path: filepath.Join(root, "modules", "a", "b", "mai")}, false}, + {"other folder", schema.ComponentDependency{Kind: "folder", Path: filepath.Join(root, "modules", "c")}, false}, + {"exact file", schema.ComponentDependency{Kind: "file", Path: filepath.Join(root, "top.md")}, true}, + {"file is not a folder prefix", schema.ComponentDependency{Kind: "file", Path: filepath.Join(root, "modules")}, false}, + {"glob folder", schema.ComponentDependency{Kind: "folder", Path: filepath.Join(root, "modules", "*")}, true}, + } { + t.Run(tc.name, func(t *testing.T) { + changed, _, _, err := isComponentDependentFolderOrFileChangedIndexed(index, []schema.ComponentDependency{tc.dep}) + require.NoError(t, err) + require.Equal(t, tc.want, changed) + }) + } +} + +func TestConfigurationWatchIndexIsBuiltOncePerExclusionSet(t *testing.T) { + root := t.TempDir() + config := &schema.AtmosConfiguration{BasePathAbsolute: root, BasePath: root} + index := newChangedFilesIndex(config, []string{"live/a/.terragrunt-cache/x.tf", "live/a/main.hcl"}, root) + settings := map[string]any{"configuration": map[string]any{"watch_excludes": []any{"**/.terragrunt-cache/**"}}} + first, err := configurationWatchIndex(config, settings, index) + require.NoError(t, err) + second, err := configurationWatchIndex(config, settings, index) + require.NoError(t, err) + require.Same(t, first, second) + require.Equal(t, []string{filepath.Join(root, "live", "a", "main.hcl")}, first.getAllFiles()) + require.Len(t, index.getAllFiles(), 2, "the shared change set is not modified") + + other := map[string]any{"configuration": map[string]any{"watch_excludes": []any{"**/*.hcl"}}} + different, err := configurationWatchIndex(config, other, index) + require.NoError(t, err) + require.NotSame(t, first, different) + require.Equal(t, []string{filepath.Join(root, "live", "a", ".terragrunt-cache", "x.tf")}, different.getAllFiles()) +} + +func BenchmarkFileWatchSelection300Units9000Files(b *testing.B) { + index, deps := watchFixture(b, 300, 9000) + b.ResetTimer() + for i := 0; i < b.N; i++ { + for unit := range deps { + if _, _, _, err := isComponentDependentFolderOrFileChangedIndexed(index, deps[unit]); err != nil { + b.Fatal(err) + } + } + } +} + +func TestSortAffectedIsDeterministic(t *testing.T) { + // Terraform precedes helmfile inside a stack because that is the processing order. + want := []schema.Affected{ + {Stack: "a", ComponentType: "terraform", Component: "app", Affected: "component"}, + {Stack: "a", ComponentType: "terraform", Component: "app", Affected: "stack.vars"}, + {Stack: "a", ComponentType: "terraform", Component: "vpc", Affected: "component"}, + {Stack: "a", ComponentType: "helmfile", Component: "z", Affected: "stack.vars"}, + {Stack: "b", ComponentType: "terraform", Component: "app", Affected: "component"}, + {Stack: "b", ComponentType: "terraform", Component: "zeta", Affected: "component"}, + } + // Stack results arrive in any order and components within a type in map order. + arrivals := [][]int{{0, 1, 2, 3, 4, 5}, {4, 5, 3, 2, 1, 0}, {5, 4, 2, 0, 3, 1}, {3, 2, 1, 0, 5, 4}} + for _, arrival := range arrivals { + input := make([]schema.Affected, 0, len(want)) + for _, index := range arrival { + input = append(input, want[index]) + } + // Every arrival order produces the same full order, types included. + sortAffected(input) + require.Equal(t, want, input, "arrival %v", arrival) + } + input := []schema.Affected{want[5], want[4], want[2], want[0], want[1], want[3]} + sortAffected(input) + require.Equal(t, want, input) + + // Deleted components arrive after every other type, but still sort by name within their type. + deleted := []schema.Affected{ + {Stack: "a", ComponentType: "terraform", Component: "zeta", Affected: "component"}, + {Stack: "a", ComponentType: "helmfile", Component: "chart", Affected: "component"}, + {Stack: "a", ComponentType: "terraform", Component: "alpha", Affected: "deleted"}, + } + sortAffected(deleted) + require.Equal(t, []string{"alpha", "zeta", "chart"}, []string{deleted[0].Component, deleted[1].Component, deleted[2].Component}) +} + +// The BASE policy inventory resolves provider imports with a session of its own checkout; the +// HEAD inventory keeps the invocation's session, which is never replaced. +func TestAffectedPolicyInventoryUsesARevisionSession(t *testing.T) { + headRoot, _ := affectedPolicyRepo(t, map[string]string{"README": "unchanged"}) + baseRoot, _ := affectedPolicyRepo(t, map[string]string{"README": "unchanged"}) + config := affectedPolicyTestConfig(headRoot) + ctrl := gomock.NewController(t) + current := NewMockComponentConfigurationRuntime(ctrl) + previous := NewMockComponentConfigurationRuntime(ctrl) + forkedAt := "" + revisions := NewMockComponentConfigurationRevisionRuntime(ctrl) + gomock.InOrder( + revisions.EXPECT().ForRevision(gomock.Any()).DoAndReturn(func(root string) (schema.ComponentConfigurationRuntime, error) { + forkedAt = root + return previous, nil + }), + revisions.EXPECT().ForRevision(gomock.Any()).Return(nil, context.Canceled), + ) + runtime := &revisionConfigurationRuntime{MockComponentConfigurationRuntime: current, MockComponentConfigurationRevisionRuntime: revisions} + config.ComponentConfigurations = runtime + + base, err := affectedPolicyConfig(config, headRoot, baseRoot) + require.NoError(t, err) + require.Same(t, previous, base.ComponentConfigurations) + require.Equal(t, base.BasePathAbsolute, forkedAt) + require.Same(t, runtime, config.ComponentConfigurations, "the caller's session is not replaced") + + forkedAt = "" + head, err := affectedPolicyConfig(config, headRoot, headRoot) + require.NoError(t, err) + require.Same(t, runtime, head.ComponentConfigurations) + require.Empty(t, forkedAt) + + _, err = affectedPolicyConfig(config, headRoot, baseRoot) + require.ErrorIs(t, err, context.Canceled) +} diff --git a/internal/exec/describe_affected_helpers.go b/internal/exec/describe_affected_helpers.go index 860a2584ca2..c3e0d502db2 100644 --- a/internal/exec/describe_affected_helpers.go +++ b/internal/exec/describe_affected_helpers.go @@ -4,6 +4,7 @@ import ( "errors" "os" "strings" + "sync" "github.com/go-git/go-git/v5" "github.com/go-git/go-git/v5/plumbing" @@ -301,8 +302,12 @@ func ExecuteDescribeAffectedWithTargetRefCheckoutWithOptions( return nil, nil, nil, "", err } - // Deferred cleanup ensures worktree is always removed. - defer cleanupWorktree(localRepoInfo.LocalWorktreePath, worktreePath) + // Deferred cleanup ensures worktree is always removed, including when the process is + // interrupted: SIGINT and SIGTERM skip deferred functions, so the worktree and its + // `.git/worktrees` registration would otherwise leak. + cleanup := sync.OnceFunc(func() { cleanupWorktree(localRepoInfo.LocalWorktreePath, worktreePath) }) + defer removeOnSignal(cleanup)() + defer cleanup() // Open the worktree as a git repository. remoteRepo, err := g.OpenWorktreeAwareRepo(worktreePath) @@ -340,6 +345,15 @@ func ExecuteDescribeAffectedWithTargetRefCheckoutWithOptions( return affected, localRepoHead, remoteRepoHead, localRepoInfo.RepoUrl, nil } +// affectedSignalCleanups removes describe-affected worktrees when the process is interrupted. +var affectedSignalCleanups = g.NewWorktreeSignalCleanups(os.Exit) + +// removeOnSignal registers a worktree cleanup for SIGINT and SIGTERM. Call the returned function +// after normal cleanup. +func removeOnSignal(cleanup func()) func() { + return affectedSignalCleanups.Register(cleanup) +} + // cleanupWorktree removes a git worktree and its parent temp directory. func cleanupWorktree(repoPath, worktreePath string) { g.RemoveWorktree(repoPath, worktreePath) diff --git a/internal/exec/describe_affected_policy_inventory.go b/internal/exec/describe_affected_policy_inventory.go index 697e9f39bb2..10ada624760 100644 --- a/internal/exec/describe_affected_policy_inventory.go +++ b/internal/exec/describe_affected_policy_inventory.go @@ -70,6 +70,13 @@ func affectedPolicyConfig(config *schema.AtmosConfiguration, root, checkout stri *path = rebased } copyConfig.BasePath = copyConfig.BasePathAbsolute + // BASE resolves provider imports in its own checkout with its own session; the HEAD session + // caches imports against HEAD's root and would hide changed units from input.changes. + if filepath.Clean(checkout) != filepath.Clean(root) { + if err := rebaseConfigurationDiscovery(©Config); err != nil { + return nil, err + } + } includeEmpty := true copyConfig.Describe.Settings.IncludeEmpty = &includeEmpty include, err := u.JoinPaths(copyConfig.StacksBaseAbsolutePath, copyConfig.Stacks.IncludedPaths) diff --git a/internal/exec/describe_affected_utils_optimized.go b/internal/exec/describe_affected_utils_optimized.go index aa5302f54da..59ac788ead8 100644 --- a/internal/exec/describe_affected_utils_optimized.go +++ b/internal/exec/describe_affected_utils_optimized.go @@ -2,6 +2,7 @@ package exec import ( "path/filepath" + "sync/atomic" cfg "github.com/cloudposse/atmos/pkg/config" "github.com/cloudposse/atmos/pkg/schema" @@ -87,6 +88,10 @@ func areTerraformComponentModulesChangedIndexed( return false, nil } +// watchPatternScans counts dependencies that had to be matched against the whole change set +// because they contain glob syntax. Literal dependencies never increment it. +var watchPatternScans atomic.Int64 + // isComponentDependentFolderOrFileChangedIndexed checks dependencies using indexed files. // This is an optimized version that reduces file iterations. // @@ -130,8 +135,19 @@ func isComponentDependentFolderOrFileChangedIndexed( return false, "", "", err } + // Literal paths are answered from the index. Matching every dependency against every + // changed file made selection grow with units times changed files. + if !hasGlobMeta(changedFileOrFolderAbs) { + if (pathPatternSuffix == "" && filesIndex.isChangedFile(changedFileOrFolderAbs)) || + (pathPatternSuffix != "" && filesIndex.hasChangedUnder(changedFileOrFolderAbs)) { + isChanged = true + } + continue + } + pathPattern := changedFileOrFolderAbs + pathPatternSuffix + watchPatternScans.Add(1) for _, changedFile := range allChangedFiles { // Files are already absolute paths (normalized by changedFilesIndex during creation). match, err := u.PathMatch(pathPattern, changedFile) diff --git a/internal/exec/describe_affected_utils_parallel.go b/internal/exec/describe_affected_utils_parallel.go index 7475c8d3cd1..53101b4fb49 100644 --- a/internal/exec/describe_affected_utils_parallel.go +++ b/internal/exec/describe_affected_utils_parallel.go @@ -1,6 +1,8 @@ package exec import ( + "cmp" + "slices" "sync" cfg "github.com/cloudposse/atmos/pkg/config" @@ -107,9 +109,42 @@ func findAffectedParallel( } allAffected = append(allAffected, deletedAffected...) + // Stacks finish in any order and component maps iterate in any order, so the same + // comparison would otherwise print its results in a different order on every run. + sortAffected(allAffected) + return allAffected, nil } +// sortAffected makes the order of results independent of goroutine scheduling and map +// iteration: results are grouped by stack, component types within a stack follow the +// processing order (deleted components included), and components of one type are ordered +// by name. +func sortAffected(affected []schema.Affected) { + slices.SortStableFunc(affected, func(a, b schema.Affected) int { + return cmp.Or( + cmp.Compare(a.Stack, b.Stack), + cmp.Compare(affectedTypeRank(a.ComponentType), affectedTypeRank(b.ComponentType)), + cmp.Compare(a.ComponentType, b.ComponentType), + cmp.Compare(a.Component, b.Component), + cmp.Compare(a.Affected, b.Affected), + ) + }) +} + +// affectedTypeOrder lists component types in the order processStackAffected processes them. +var affectedTypeOrder = slices.Concat([]string{cfg.TerraformComponentType}, simpleAffectedComponentTypes, + []string{cfg.KubernetesComponentType, cfg.HelmComponentType, cfg.CloudFormationComponentType}) + +// affectedTypeRank orders component types by affectedTypeOrder. An unknown type sorts after the +// known ones. +func affectedTypeRank(componentType string) int { + if index := slices.Index(affectedTypeOrder, componentType); index >= 0 { + return index + } + return len(affectedTypeOrder) +} + // processStackAffected processes a single stack to find affected components. // This function is called in parallel for each stack. // diff --git a/internal/exec/describe_component.go b/internal/exec/describe_component.go index 8ebdb35cd1c..80677672331 100644 --- a/internal/exec/describe_component.go +++ b/internal/exec/describe_component.go @@ -13,6 +13,7 @@ import ( tuiTerm "github.com/cloudposse/atmos/internal/tui/templates/term" "github.com/cloudposse/atmos/pkg/auth" "github.com/cloudposse/atmos/pkg/component" + configurationadapter "github.com/cloudposse/atmos/pkg/component/configuration" "github.com/cloudposse/atmos/pkg/component/typedetect" cfg "github.com/cloudposse/atmos/pkg/config" "github.com/cloudposse/atmos/pkg/data" @@ -654,8 +655,10 @@ func ExecuteDescribeComponentWithContext(params DescribeComponentContextParams) if configAndStacksInfo.StackFile != "" { mergeContext = GetMergeContextForStack(configAndStacksInfo.StackFile) } - // Fall back to the old method if the stack file is not set - if mergeContext == nil { + // Fall back to the old method if the stack file is not set. A component supplied by a + // configuration provider has no manifest of its own: the most recently processed manifest + // is unrelated to it (and varies between runs), so it must not be reported as its origin. + if mergeContext == nil && !configurationadapter.OwnsComponent(configAndStacksInfo.ComponentSection) { mergeContext = GetLastMergeContext() } diff --git a/internal/exec/mock_configuration_imports_test.go b/internal/exec/mock_configuration_imports_test.go new file mode 100644 index 00000000000..95325fac1ff --- /dev/null +++ b/internal/exec/mock_configuration_imports_test.go @@ -0,0 +1,172 @@ +// Code generated by MockGen. DO NOT EDIT. +// Source: configuration_imports_test.go +// +// Generated by this command: +// +// mockgen -source=configuration_imports_test.go -destination=mock_configuration_imports_test.go -package=exec -mock_names=importingRuntime=MockImportingRuntime,configurationProvider=MockConfigurationProvider +// + +// Package exec is a generated GoMock package. +package exec + +import ( + context "context" + reflect "reflect" + + pluginv1 "github.com/cloudposse/atmos/pkg/plugin/proto/atmos/plugin/v1" + schema "github.com/cloudposse/atmos/pkg/schema" + gomock "go.uber.org/mock/gomock" +) + +// MockImportingRuntime is a mock of importingRuntime interface. +type MockImportingRuntime struct { + ctrl *gomock.Controller + recorder *MockImportingRuntimeMockRecorder + isgomock struct{} +} + +// MockImportingRuntimeMockRecorder is the mock recorder for MockImportingRuntime. +type MockImportingRuntimeMockRecorder struct { + mock *MockImportingRuntime +} + +// NewMockImportingRuntime creates a new mock instance. +func NewMockImportingRuntime(ctrl *gomock.Controller) *MockImportingRuntime { + mock := &MockImportingRuntime{ctrl: ctrl} + mock.recorder = &MockImportingRuntimeMockRecorder{mock} + return mock +} + +// EXPECT returns an object that allows the caller to indicate expected use. +func (m *MockImportingRuntime) EXPECT() *MockImportingRuntimeMockRecorder { + return m.recorder +} + +// Discover mocks base method. +func (m *MockImportingRuntime) Discover(arg0 context.Context) (map[string]map[string]any, error) { + m.ctrl.T.Helper() + ret := m.ctrl.Call(m, "Discover", arg0) + ret0, _ := ret[0].(map[string]map[string]any) + ret1, _ := ret[1].(error) + return ret0, ret1 +} + +// Discover indicates an expected call of Discover. +func (mr *MockImportingRuntimeMockRecorder) Discover(arg0 any) *gomock.Call { + mr.mock.ctrl.T.Helper() + return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "Discover", reflect.TypeOf((*MockImportingRuntime)(nil).Discover), arg0) +} + +// ImportComponents mocks base method. +func (m *MockImportingRuntime) ImportComponents(arg0 context.Context, arg1 string) (map[string]any, bool, error) { + m.ctrl.T.Helper() + ret := m.ctrl.Call(m, "ImportComponents", arg0, arg1) + ret0, _ := ret[0].(map[string]any) + ret1, _ := ret[1].(bool) + ret2, _ := ret[2].(error) + return ret0, ret1, ret2 +} + +// ImportComponents indicates an expected call of ImportComponents. +func (mr *MockImportingRuntimeMockRecorder) ImportComponents(arg0, arg1 any) *gomock.Call { + mr.mock.ctrl.T.Helper() + return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "ImportComponents", reflect.TypeOf((*MockImportingRuntime)(nil).ImportComponents), arg0, arg1) +} + +// Prepare mocks base method. +func (m *MockImportingRuntime) Prepare(arg0 context.Context, arg1 *schema.ConfigAndStacksInfo) (*schema.PreparedComponentConfiguration, error) { + m.ctrl.T.Helper() + ret := m.ctrl.Call(m, "Prepare", arg0, arg1) + ret0, _ := ret[0].(*schema.PreparedComponentConfiguration) + ret1, _ := ret[1].(error) + return ret0, ret1 +} + +// Prepare indicates an expected call of Prepare. +func (mr *MockImportingRuntimeMockRecorder) Prepare(arg0, arg1 any) *gomock.Call { + mr.mock.ctrl.T.Helper() + return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "Prepare", reflect.TypeOf((*MockImportingRuntime)(nil).Prepare), arg0, arg1) +} + +// MockConfigurationProvider is a mock of configurationProvider interface. +type MockConfigurationProvider struct { + ctrl *gomock.Controller + recorder *MockConfigurationProviderMockRecorder + isgomock struct{} +} + +// MockConfigurationProviderMockRecorder is the mock recorder for MockConfigurationProvider. +type MockConfigurationProviderMockRecorder struct { + mock *MockConfigurationProvider +} + +// NewMockConfigurationProvider creates a new mock instance. +func NewMockConfigurationProvider(ctrl *gomock.Controller) *MockConfigurationProvider { + mock := &MockConfigurationProvider{ctrl: ctrl} + mock.recorder = &MockConfigurationProviderMockRecorder{mock} + return mock +} + +// EXPECT returns an object that allows the caller to indicate expected use. +func (m *MockConfigurationProvider) EXPECT() *MockConfigurationProviderMockRecorder { + return m.recorder +} + +// ComponentType mocks base method. +func (m *MockConfigurationProvider) ComponentType() string { + m.ctrl.T.Helper() + ret := m.ctrl.Call(m, "ComponentType") + ret0, _ := ret[0].(string) + return ret0 +} + +// ComponentType indicates an expected call of ComponentType. +func (mr *MockConfigurationProviderMockRecorder) ComponentType() *gomock.Call { + mr.mock.ctrl.T.Helper() + return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "ComponentType", reflect.TypeOf((*MockConfigurationProvider)(nil).ComponentType)) +} + +// Discover mocks base method. +func (m *MockConfigurationProvider) Discover(arg0 context.Context, arg1 *pluginv1.DiscoverComponentsRequest) (*pluginv1.DiscoverComponentsResponse, error) { + m.ctrl.T.Helper() + ret := m.ctrl.Call(m, "Discover", arg0, arg1) + ret0, _ := ret[0].(*pluginv1.DiscoverComponentsResponse) + ret1, _ := ret[1].(error) + return ret0, ret1 +} + +// Discover indicates an expected call of Discover. +func (mr *MockConfigurationProviderMockRecorder) Discover(arg0, arg1 any) *gomock.Call { + mr.mock.ctrl.T.Helper() + return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "Discover", reflect.TypeOf((*MockConfigurationProvider)(nil).Discover), arg0, arg1) +} + +// Prepare mocks base method. +func (m *MockConfigurationProvider) Prepare(arg0 context.Context, arg1 *pluginv1.ComponentRequest) (*pluginv1.PrepareComponentResponse, error) { + m.ctrl.T.Helper() + ret := m.ctrl.Call(m, "Prepare", arg0, arg1) + ret0, _ := ret[0].(*pluginv1.PrepareComponentResponse) + ret1, _ := ret[1].(error) + return ret0, ret1 +} + +// Prepare indicates an expected call of Prepare. +func (mr *MockConfigurationProviderMockRecorder) Prepare(arg0, arg1 any) *gomock.Call { + mr.mock.ctrl.T.Helper() + return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "Prepare", reflect.TypeOf((*MockConfigurationProvider)(nil).Prepare), arg0, arg1) +} + +// ResolveState mocks base method. +func (m *MockConfigurationProvider) ResolveState(arg0 context.Context, arg1 *pluginv1.ComponentRequest) (*pluginv1.ResolveComponentStateResponse, error) { + m.ctrl.T.Helper() + ret := m.ctrl.Call(m, "ResolveState", arg0, arg1) + ret0, _ := ret[0].(*pluginv1.ResolveComponentStateResponse) + ret1, _ := ret[1].(error) + return ret0, ret1 +} + +// ResolveState indicates an expected call of ResolveState. +func (mr *MockConfigurationProviderMockRecorder) ResolveState(arg0, arg1 any) *gomock.Call { + mr.mock.ctrl.T.Helper() + return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "ResolveState", reflect.TypeOf((*MockConfigurationProvider)(nil).ResolveState), arg0, arg1) +} diff --git a/internal/exec/stack_processor_merge.go b/internal/exec/stack_processor_merge.go index bcec7d3df7a..eba850b7cb3 100644 --- a/internal/exec/stack_processor_merge.go +++ b/internal/exec/stack_processor_merge.go @@ -598,6 +598,11 @@ func mergeComponentConfigurations(atmosConfig *schema.AtmosConfiguration, opts * // Terraform-specific: process backends and add Terraform-specific fields. if opts.ComponentType == cfg.TerraformComponentType { + // Validate component-owned options while backend types are still nested and + // component inheritance is distinguishable from stack-global defaults. + if err := checkConfigurationBackendOverrides(opts, result, finalSettings); err != nil { + return nil, nil, err + } // Process backend configuration. finalComponentBackendType, finalComponentBackend, err := processTerraformBackend( &terraformBackendConfig{ diff --git a/internal/exec/stack_processor_utils.go b/internal/exec/stack_processor_utils.go index b44c44bd4a1..6ff8de8175b 100644 --- a/internal/exec/stack_processor_utils.go +++ b/internal/exec/stack_processor_utils.go @@ -19,7 +19,7 @@ import ( "gopkg.in/yaml.v3" errUtils "github.com/cloudposse/atmos/errors" - "github.com/cloudposse/atmos/pkg/component/configuration" + configurationadapter "github.com/cloudposse/atmos/pkg/component/configuration" "github.com/cloudposse/atmos/pkg/condition" cfg "github.com/cloudposse/atmos/pkg/config" "github.com/cloudposse/atmos/pkg/function/yamlsource" @@ -962,6 +962,9 @@ func ProcessYAMLConfigFileWithContext( // importFileResult holds the result of processing a single import file in parallel. type importFileResult struct { + unitFragments []map[string]any + yamlOnlyConfig map[string]any + importedUnitSources map[string]string importMetadata schema.StackImportMetadata index int importFile string @@ -1231,6 +1234,10 @@ func processYAMLConfigFileWithContextInternal( stackConfig.PreserveStackLiterals = true atmosConfig = &stackConfig var stackConfigs []map[string]any + // Units imported from a configuration provider are the base layer of the manifest: any YAML + // component of the same name, wherever it is imported, merges over the unit's own data. + var unitFragments []map[string]any + importedUnitSources := map[string]string{} inheritedNestedImports = normalizeNestedImports(inheritedNestedImports) if localBasePath == "" { localBasePath = basePath @@ -1635,10 +1642,14 @@ func processYAMLConfigFileWithContextInternal( return nil, nil, impErr } imp = renderedImp - if handled, importErr := configuration.RegisterImport(atmosConfig, imp, &importStruct); handled || importErr != nil { + if fragment, handled, importErr := importConfigurationComponents(atmosConfig, imp, &importStruct, loadOptions); handled || importErr != nil { if importErr != nil { return nil, nil, importErr } + if err := configurationadapter.MergeImportUnitSources(relativeFilePath, importedUnitSources, configurationadapter.ImportUnitSources(fragment)); err != nil { + return nil, nil, err + } + unitFragments = append(unitFragments, fragment) // Report the provider import with the stack's other imports, keyed by its URI as // remote imports are. It contributes no YAML, so its recorded configuration is empty. importsConfigLock.Lock() @@ -1857,7 +1868,10 @@ func processYAMLConfigFileWithContextInternal( index: index, importFile: file, yamlConfig: importResult.DeepMergedConfig, + yamlOnlyConfig: importResult.YAMLConfig, + unitFragments: importResult.UnitFragments, importMetadata: importResult.ImportMetadata, + importedUnitSources: importResult.ImportedUnitSources, yamlConfigRaw: importResult.StackConfig, terraformOverridesInline: importResult.TerraformOverridesInline, terraformOverridesImports: importResult.TerraformOverridesImports, @@ -1880,6 +1894,10 @@ func processYAMLConfigFileWithContextInternal( return nil, nil, result.err } + if err := configurationadapter.MergeImportUnitSources(relativeFilePath, importedUnitSources, result.importedUnitSources); err != nil { + return nil, nil, err + } + importedMetadata, err = metadataScope.Merge(importedMetadata, result.importMetadata) if err != nil { return nil, nil, err @@ -1910,8 +1928,10 @@ func processYAMLConfigFileWithContextInternal( return nil, nil, err } - // Append to stackConfigs in order. - stackConfigs = append(stackConfigs, result.yamlConfig) + // Keep provider defaults below all YAML layers, even when a unit arrives + // through a nested mixin after an earlier catalog override. + unitFragments = append(unitFragments, result.unitFragments...) + stackConfigs = append(stackConfigs, result.yamlOnlyConfig) // Record metadata for this import. // We record every time we encounter an import to track all files that import it, @@ -2012,6 +2032,23 @@ func processYAMLConfigFileWithContextInternal( stackConfigs = append(stackConfigs, stackConfigMap) } + var yamlConfig map[string]any + // Imported units are the lowest layer, so the imports and the manifest itself deep-merge over + // them with the usual rules. Only two different units that share a component name conflict, + // including units that arrive through different YAML imports, so every manifest checks. + if len(unitFragments) > 0 { + // Preserve YAML separately before adding defaults, without recording provenance twice. + yamlConfig, err = m.Merge(atmosConfig, stackConfigs) + if err != nil { + return nil, nil, err + } + if err := applyHelmTypeOverrides(atmosConfig, yamlConfig, relativeFilePath); err != nil { + return nil, nil, err + } + configurationadapter.ApplyUnitOverrides(unitFragments, finalTerraformOverrides) + stackConfigs = append(slices.Clone(unitFragments), stackConfigs...) + } + // Deep-merge the stack manifest and all the imports stackConfigsDeepMerged, err := m.MergeWithContext(atmosConfig, stackConfigs, mergeContext) if err != nil { @@ -2021,6 +2058,9 @@ func processYAMLConfigFileWithContextInternal( if err := applyHelmTypeOverrides(atmosConfig, stackConfigsDeepMerged, relativeFilePath); err != nil { return nil, nil, err } + if len(unitFragments) == 0 { + yamlConfig = stackConfigsDeepMerged + } // NOTE: We don't store merge context here because ProcessYAMLConfigFileWithContext // can be called from parallel goroutines in ProcessYAMLConfigFiles, which would create @@ -2032,6 +2072,9 @@ func processYAMLConfigFileWithContextInternal( } return &schema.StackManifestProcessingResult{ DeepMergedConfig: stackConfigsDeepMerged, + YAMLConfig: yamlConfig, + UnitFragments: unitFragments, + ImportedUnitSources: importedUnitSources, ImportMetadata: finalMetadata, ImportsConfig: importsConfig, StackConfig: stackConfigMap, diff --git a/internal/exec/validate_stacks.go b/internal/exec/validate_stacks.go index a2700ec206a..e19685d0bdf 100644 --- a/internal/exec/validate_stacks.go +++ b/internal/exec/validate_stacks.go @@ -121,6 +121,14 @@ func ValidateStacks(atmosConfig *schema.AtmosConfiguration) error { } validationErrorMessages = append(validationErrorMessages, errorList...) + // Additional manifests may contain imports absent from the selected stacks. Give + // those reads a validation view so they cannot alter the discovered inventory. + if runtime, ok := atmosConfig.ComponentConfigurations.(schema.ComponentConfigurationValidationRuntime); ok { + validationConfig := *atmosConfig + validationConfig.ComponentConfigurations = runtime.ForValidation() + atmosConfig = &validationConfig + } + // 2. Check all YAML stack manifests defined in the infrastructure // It will check YAML syntax and all the Atmos sections defined in the manifests diff --git a/pkg/component/configuration/conflicts.go b/pkg/component/configuration/conflicts.go new file mode 100644 index 00000000000..830fbcad718 --- /dev/null +++ b/pkg/component/configuration/conflicts.go @@ -0,0 +1,147 @@ +package configuration + +import ( + "fmt" + "maps" + + errUtils "github.com/cloudposse/atmos/errors" + "github.com/cloudposse/atmos/pkg/perf" +) + +// ImportFragment wraps imported units as a stack manifest fragment, so the ordinary +// import merge adds them to the importing manifest as Terraform components. +func ImportFragment(components map[string]any) map[string]any { + defer perf.Track(nil, "configuration.ImportFragment")() + + return map[string]any{"components": map[string]any{"terraform": components}} +} + +// ApplyUnitOverrides attaches a manifest's Terraform `overrides` to the units it imports, like the +// manifest's own components. Each unit section is copied first, so a provider's cached import result +// is never changed. +func ApplyUnitOverrides(fragments []map[string]any, overrides map[string]any) { + defer perf.Track(nil, "configuration.ApplyUnitOverrides")() + + if len(overrides) == 0 { + return + } + for _, fragment := range fragments { + section, _ := fragment["components"].(map[string]any) + terraform, _ := section["terraform"].(map[string]any) + for name, component := range terraform { + unit, ok := component.(map[string]any) + if !ok { + continue + } + unit = maps.Clone(unit) + unit["overrides"] = overrides + terraform[name] = unit + } + } +} + +// unitSource reads the source identity of an unmerged provider-owned component. +func unitSource(component any) string { + section, _ := component.(map[string]any) + settings, _ := section["settings"].(map[string]any) + configuration, _ := settings["configuration"].(map[string]any) + source, _ := configuration["source"].(string) + return source +} + +// ImportUnitSources extracts identity from a provider import fragment before YAML merging. +// Callers must not pass ordinary YAML or merged stack configuration. +func ImportUnitSources(fragment map[string]any) map[string]string { + defer perf.Track(nil, "configuration.ImportUnitSources")() + + units := map[string]string{} + section, _ := fragment["components"].(map[string]any) + terraform, _ := section["terraform"].(map[string]any) + for name, component := range terraform { + if source := unitSource(component); source != "" { + units[name] = source + } + } + return units +} + +// MergeImportUnitSources accumulates provider provenance across recursive imports. +func MergeImportUnitSources(manifest string, units, incoming map[string]string) error { + defer perf.Track(nil, "configuration.MergeImportUnitSources")() + + for name, source := range incoming { + if previous, seen := units[name]; seen && previous != source { + return ImportConflict(name, previous, source, manifest) + } + units[name] = source + } + return nil +} + +// CheckImportConflicts checks unmerged provider fragments for duplicate unit names. +// Ordinary YAML and merged stack configurations must not be passed here. +func CheckImportConflicts(manifest string, configs []map[string]any) error { + defer perf.Track(nil, "configuration.CheckImportConflicts")() + units := map[string]string{} + for _, config := range configs { + if err := MergeImportUnitSources(manifest, units, ImportUnitSources(config)); err != nil { + return err + } + } + return nil +} + +// OwnsComponent reports whether a configuration provider supplies a component section, so +// callers can avoid attributing it to unrelated YAML files. +func OwnsComponent(component map[string]any) bool { + defer perf.Track(nil, "configuration.OwnsComponent")() + + settings, _ := component["settings"].(map[string]any) + configuration, _ := settings["configuration"].(map[string]any) + return configuration["provider"] != nil +} + +// ImportConflict reports two units that would become one component of the importing stack. +func ImportConflict(component, first, second, where string) error { + defer perf.Track(nil, "configuration.ImportConflict")() + + return errUtils.Build(fmt.Errorf("%w: units %s and %s both become component %q (%s)", errUtils.ErrConfigurationImportConflict, first, second, component, where)). + WithContext("component", component). + WithContext("unit", first). + WithContext("other_unit", second). + WithHint("Import the units from different stack files, or import a narrower directory or a single `terragrunt.hcl`"). + Err() +} + +// BackendLayer is one component-level backend: its type and its settings. +type BackendLayer struct { + Type string + Settings map[string]any +} + +// CheckBackendOverrides rejects a component-level backend for a component a configuration provider +// owns: the unit's own backend is its state identity. A component without a provider is unaffected. +func CheckBackendOverrides(component, stack string, settings map[string]any, layers ...BackendLayer) error { + defer perf.Track(nil, "configuration.CheckBackendOverrides")() + + // Only a provider owns the backend; a bare `configuration` key, such as an empty map, does not. + if !OwnsComponent(map[string]any{"settings": settings}) { + return nil + } + for _, layer := range layers { + setting := "" + if layer.Type != "" && layer.Type != "local" { + setting = "backend_type" + } + if local, ok := layer.Settings["local"].(map[string]any); ok && len(local) > 0 { + setting = "backend.local" + } + if setting != "" { + return errUtils.Build(fmt.Errorf("%w: the state backend of component %q in stack %q is owned by the configuration provider, so `%s` cannot be set", errUtils.ErrConfigurationOverrideUnsupported, component, stack, setting)). + WithContext("component", component). + WithContext("stack", stack). + WithHint("Define the backend in the unit's remote_state block; component backend overrides cannot change the unit's state identity.").Err() + } + } + return nil +} diff --git a/pkg/component/configuration/files.go b/pkg/component/configuration/files.go index 03bcc2a27ce..4d16ef31aa3 100644 --- a/pkg/component/configuration/files.go +++ b/pkg/component/configuration/files.go @@ -1,6 +1,7 @@ package configuration import ( + "crypto/rand" "errors" "fmt" "os" @@ -25,10 +26,15 @@ const ( ) // WriteFiles materializes evaluated bytes under the provisioned module, without templating. +// +// It also keeps a manifest of what it generated. A file generated by an earlier run that the +// current run no longer names is removed, so deleting a generate block stops applying it; a +// file under if_exists = "error" is accepted again when it is the one this host wrote; and the +// adjacent Terraform lock-file copy is rewritten only when the copy beside the unit changed. func WriteFiles(directory string, files []schema.ComponentConfigurationFile) error { defer perf.Track(nil, "configuration.WriteFiles")() - if len(files) == 0 { + if len(files) == 0 && !manifestExists(directory) { return nil } root, err := os.OpenRoot(directory) @@ -41,12 +47,49 @@ func WriteFiles(directory string, files []schema.ComponentConfigurationFile) err return err } } + previous := readManifest(root) + undo := make(restorePoint) + current := make(map[string]string, len(files)) + names := make([]string, 0, len(files)) for _, file := range files { - if err := writeFile(root, file); err != nil { - return err + // Clean the key, so ./provider.tf and provider.tf name the same recorded file. + key := filepath.ToSlash(filepath.Clean(file.Path)) + names = append(names, key) + undo.capture(root, key) + digest, err := writeFile(root, file, previous[key]) + if err != nil { + return errors.Join(err, keepRecordsAfterFailure(root, previous, current, undo)) + } + if digest != "" { + current[key] = digest } } - return nil + // Record the manifest even when a stale file cannot be removed, so the files written above stay + // owned and the stale file is retried; then report the removal failure. + staleErr := removeStaleFiles(root, previous, names, current, undo) + return errors.Join(recordManifest(root, current, undo), staleErr) +} + +// recordManifest writes the manifest. When it cannot be written, the run's file changes would have +// no recorded owner, so they are rolled back and the failure is reported with any rollback problem. +func recordManifest(root *os.Root, current map[string]string, undo restorePoint) error { + err := writeManifest(root, current) + if err == nil { + return nil + } + return errors.Join(err, undo.restore(root)) +} + +// keepRecordsAfterFailure writes the manifest when a batch stops early: the files this run already +// wrote keep their new digests, and every other file keeps its earlier record, so a retry after the +// conflict is fixed still recognizes them as this host's. +func keepRecordsAfterFailure(root *os.Root, previous, current map[string]string, undo restorePoint) error { + for key, digest := range previous { + if _, written := current[key]; !written { + current[key] = digest + } + } + return recordManifest(root, current, undo) } func validateFile(file schema.ComponentConfigurationFile) error { @@ -59,43 +102,133 @@ func validateFile(file schema.ComponentConfigurationFile) error { return nil } -func writeFile(root *os.Root, file schema.ComponentConfigurationFile) error { - remove, err := checkFileOwnership(root, file) +// writeFile applies one file's policy and returns the digest to record, or empty when the host +// does not own the file afterwards (removed, or an existing file that "skip" left alone). +func writeFile(root *os.Root, file schema.ComponentConfigurationFile, recorded string) (string, error) { + remove, err := checkFileOwnership(root, file, recorded) if err != nil { - return err + return "", err } if remove { - return removeFile(root, file.Path) + return "", removeFile(root, file.Path) } - mode := writtenMode(file) + // Register sensitive bytes even when the file is kept, so a hook that prints it is masked. if file.Sensitive { atmosio.RegisterSecretValue(string(file.Content)) } + digest := contentDigest(file.Content) + existing, exists := fileDigest(root, file.Path) + if exists && existing == digest { + return unchangedFileOutcome(root, file, recorded, digest) + } + if exists { + if keep, owned, err := existingFileOutcome(file, recorded, existing, digest); keep || err != nil { + return owned, err + } + } + return digest, replaceFile(root, file, exists) +} + +// unchangedFileOutcome retains ownership only when an identical file belongs to this host. +func unchangedFileOutcome(root *os.Root, file schema.ComponentConfigurationFile, recorded, digest string) (string, error) { + if (file.IfExists == "skip" || file.IfExists == "error") && recorded != digest { + // A file the module already ships, or any file this host did not write, stays unowned, so + // removing the block keeps it. Its bytes already match, so if_exists = "error" has nothing to reject. + return "", nil + } + return digest, ensureMode(root, file) +} + +// existingFileOutcome applies the policy to a file that exists with different bytes. It reports +// whether the file is left alone, and the digest to keep recording for it. +func existingFileOutcome(file schema.ComponentConfigurationFile, recorded, existing, digest string) (bool, string, error) { + switch { + case keepsUnitLockFile(file, recorded, digest): + return true, recorded, nil + case file.IfExists == "skip": + return true, recordedIfOwned(recorded, existing), nil + case file.IfExists == "error" && (recorded == "" || recorded != existing): + return false, "", fmt.Errorf("%w: generated file %s already exists and was not generated by this unit (if_exists = \"error\"); remove it from the working directory or use overwrite_terragrunt", errUtils.ErrPluginInvalidConfig, file.Path) + } + return false, "", nil +} + +// keepsUnitLockFile leaves a lock file Terraform has updated since the unit's copy was last +// copied. The unit's copy takes over again once it changes. +func keepsUnitLockFile(file schema.ComponentConfigurationFile, recorded, digest string) bool { + return file.UnitFileCopy && file.Path == lockFileName && file.IfExists == "overwrite" && recorded == digest +} + +// recordedIfOwned keeps owning a skipped file only while it is still the bytes this host wrote. +func recordedIfOwned(recorded, existing string) string { + if recorded == existing { + return recorded + } + return "" +} + +// ensureMode applies the file's permission bits to a file written by an earlier run, which may +// predate them. Like a fresh write, it leaves the mode alone unless the file is sensitive or +// requests one. Windows keeps its ACLs. +func ensureMode(root *os.Root, file schema.ComponentConfigurationFile) error { + if (!file.Sensitive && file.Mode == 0) || runtime.GOOS == "windows" { + return nil + } + if err := root.Chmod(file.Path, writtenMode(file)); err != nil { + return generatedFileError(file.Path, err) + } + return nil +} + +// replaceFile writes the bytes with the file's clamped permission bits; sensitive files, which can +// embed backend credentials, stay private to the owner. +// +// An existing file is replaced through a temporary file renamed over it, so a failed write leaves +// the previous bytes, and the digest recorded for them, valid for a retry. +func replaceFile(root *os.Root, file schema.ComponentConfigurationFile, exists bool) error { if err := root.MkdirAll(filepath.Dir(file.Path), directoryMode); err != nil { + return generatedFileError(file.Path, err) + } + // Policy has already approved replacing an existing file; creating one must not race another writer. + if !exists { + return createFile(root, file.Path, file) + } + temporary := filepath.Join(filepath.Dir(file.Path), "."+filepath.Base(file.Path)+"."+rand.Text()+".tmp") + if err := createFile(root, temporary, file); err != nil { return err } - output, err := root.OpenFile(file.Path, openFlags(file.IfExists), mode) - if errors.Is(err, os.ErrExist) && file.IfExists == "skip" { - return nil + if err := root.Rename(temporary, file.Path); err != nil { + _ = root.Remove(temporary) + return generatedFileError(file.Path, err) } + return nil +} + +// createFile writes a new file with the permission bits the file carries and removes it again if +// the write does not complete, so a partial file is never left behind. +func createFile(root *os.Root, path string, file schema.ComponentConfigurationFile) error { + mode := writtenMode(file) + output, err := root.OpenFile(path, os.O_WRONLY|os.O_CREATE|os.O_EXCL, mode) if err != nil { - return fmt.Errorf("%w: generated file %s: %w", errUtils.ErrPluginInvalidConfig, file.Path, err) + return generatedFileError(path, err) } - // Open applies the process umask and ignores the mode of an existing file, so set it explicitly. - chmodErr := applyMode(output, file, mode) - if chmodErr != nil { - return errors.Join(chmodErr, output.Close()) + // Open applies the process umask, so set the mode explicitly. + if err := applyMode(output, file, mode); err != nil { + _ = output.Close() + _ = root.Remove(path) + return generatedFileError(path, err) } _, writeErr := output.Write(file.Content) - return errors.Join(writeErr, output.Close()) + if err := errors.Join(writeErr, output.Close()); err != nil { + _ = root.Remove(path) + return generatedFileError(path, err) + } + return nil } -// openFlags selects exclusive creation unless the policy replaces existing content. -func openFlags(policy string) int { - if policy == "overwrite" || policy == "overwrite_terragrunt" { - return os.O_WRONLY | os.O_CREATE | os.O_TRUNC - } - return os.O_WRONLY | os.O_CREATE | os.O_EXCL +// generatedFileError classifies a filesystem failure on a generated file as invalid configuration. +func generatedFileError(path string, err error) error { + return fmt.Errorf("%w: generated file %s: %w", errUtils.ErrPluginInvalidConfig, path, err) } // removeFile deletes a provider-owned file, treating an already absent file as removed. diff --git a/pkg/component/configuration/files_test.go b/pkg/component/configuration/files_test.go index 53cd7729291..0b355167e7d 100644 --- a/pkg/component/configuration/files_test.go +++ b/pkg/component/configuration/files_test.go @@ -9,6 +9,7 @@ import ( "github.com/stretchr/testify/require" errUtils "github.com/cloudposse/atmos/errors" + atmosio "github.com/cloudposse/atmos/pkg/io" "github.com/cloudposse/atmos/pkg/schema" ) @@ -20,6 +21,10 @@ func TestWriteFilesOwnershipAndLiteralContent(t *testing.T) { content, err := os.ReadFile(path) require.NoError(t, err) require.Equal(t, file.Content, content) + // Writing the same bytes again is not a conflict: the file is the one this host generated. + require.NoError(t, WriteFiles(root, []schema.ComponentConfigurationFile{file})) + // A different file of the same name that this host did not generate still is. + require.NoError(t, os.WriteFile(path, []byte("authored by someone else"), 0o600)) require.ErrorIs(t, WriteFiles(root, []schema.ComponentConfigurationFile{file}), errUtils.ErrPluginInvalidConfig) file.Content = []byte("replacement") file.IfExists = "skip" @@ -59,6 +64,60 @@ func TestWriteFilesPreflightAndContainment(t *testing.T) { require.True(t, os.IsNotExist(err)) } +// A sensitive file that already holds the generated bytes is kept as is, and its content is still +// registered for masking, so a hook that prints it in a later invocation is masked. +func TestWriteFilesRegistersSensitiveContentItKeeps(t *testing.T) { + t.Cleanup(atmosio.Reset) + atmosio.Reset() + require.NoError(t, atmosio.Initialize()) + root := t.TempDir() + secret := "kept-backend-credential-4b1d" + file := schema.ComponentConfigurationFile{Path: "backend.tf", Content: []byte(secret), IfExists: "overwrite", Sensitive: true} + require.NoError(t, os.WriteFile(filepath.Join(root, file.Path), file.Content, 0o600)) + require.NoError(t, WriteFiles(root, []schema.ComponentConfigurationFile{file})) + require.Equal(t, "key=", atmosio.MaskString("key="+secret)) +} + +// A module file that already matches a skip block stays the module's: removing the block later +// must not delete it as a stale generated file. +func TestSkipKeepsOwnershipOfAnIdenticalModuleFile(t *testing.T) { + root := t.TempDir() + path := filepath.Join(root, "versions.tf") + content := []byte("terraform {}\n") + require.NoError(t, os.WriteFile(path, content, 0o644)) + file := schema.ComponentConfigurationFile{Path: "versions.tf", Content: content, IfExists: "skip"} + require.NoError(t, WriteFiles(root, []schema.ComponentConfigurationFile{file})) + require.NoError(t, WriteFiles(root, nil)) + got, err := os.ReadFile(path) + require.NoError(t, err) + require.Equal(t, content, got) + // A skip file this host created is still its own and is removed with its block. + generated := schema.ComponentConfigurationFile{Path: "generated.tf", Content: content, IfExists: "skip"} + require.NoError(t, WriteFiles(root, []schema.ComponentConfigurationFile{generated})) + require.NoError(t, WriteFiles(root, []schema.ComponentConfigurationFile{generated})) + require.NoError(t, WriteFiles(root, nil)) + require.NoFileExists(t, filepath.Join(root, generated.Path)) +} + +func TestEnsureModeWrapsPermissionFailures(t *testing.T) { + if runtime.GOOS == "windows" { + t.Skip("ensureMode leaves permissions alone on Windows, which keeps its ACLs") + } + root, err := os.OpenRoot(t.TempDir()) + require.NoError(t, err) + t.Cleanup(func() { _ = root.Close() }) + err = ensureMode(root, schema.ComponentConfigurationFile{Path: "missing.tf", Sensitive: true}) + require.ErrorIs(t, err, errUtils.ErrPluginInvalidConfig) + require.ErrorIs(t, err, os.ErrNotExist, "the filesystem cause is preserved") + require.ErrorContains(t, err, "missing.tf") + // A file that exists is tightened without error. + require.NoError(t, root.WriteFile("present.tf", nil, 0o644)) + require.NoError(t, ensureMode(root, schema.ComponentConfigurationFile{Path: "present.tf", Sensitive: true})) + info, err := root.Stat("present.tf") + require.NoError(t, err) + require.Equal(t, os.FileMode(0o600), info.Mode().Perm()) +} + func TestWatchPaths(t *testing.T) { root := t.TempDir() for _, provider := range []bool{true, false} { @@ -78,6 +137,25 @@ func TestWatchPaths(t *testing.T) { } } +// Under if_exists = "error", a module file that already holds the generated bytes is accepted but +// stays the module's: it is not recorded, and removing the generate block keeps it. +func TestErrorPolicyDoesNotClaimAnIdenticalModuleFile(t *testing.T) { + root := t.TempDir() + path := filepath.Join(root, "versions.tf") + content := []byte("terraform {}\n") + require.NoError(t, os.WriteFile(path, content, 0o644)) + file := schema.ComponentConfigurationFile{Path: "versions.tf", Content: content, IfExists: "error"} + require.NoError(t, WriteFiles(root, []schema.ComponentConfigurationFile{file})) + written, err := os.OpenRoot(root) + require.NoError(t, err) + t.Cleanup(func() { _ = written.Close() }) + require.NotContains(t, readManifest(written), "versions.tf", "an unowned file is not recorded") + require.NoError(t, WriteFiles(root, nil)) + got, err := os.ReadFile(path) + require.NoError(t, err) + require.Equal(t, content, got, "removing the block keeps the module's file") +} + func TestWriteFilesAppliesClampedMode(t *testing.T) { if runtime.GOOS == "windows" { t.Skip("Windows ignores POSIX permission bits.") @@ -147,3 +225,39 @@ func TestWriteFilesRejectsModesBeyondPermissionBits(t *testing.T) { _, statErr := os.Stat(filepath.Join(root, "a.sh")) require.True(t, os.IsNotExist(statErr)) } + +// A manifest that cannot be written must not leave file changes without a recorded owner. The +// update is rolled back, so retrying it and later removing the generate block still removes the file. +func TestWriteFilesRollsBackWhenTheManifestCannotBeWritten(t *testing.T) { + for _, policy := range []string{"overwrite", "error"} { + t.Run(policy, func(t *testing.T) { + root := t.TempDir() + path := filepath.Join(root, "generated.tf") + manifestPath := filepath.Join(root, filepath.FromSlash(manifestName)) + generate := func(content string) []schema.ComponentConfigurationFile { + return []schema.ComponentConfigurationFile{{Path: "generated.tf", Content: []byte(content), IfExists: policy}} + } + require.NoError(t, WriteFiles(root, generate("A"))) + + require.NoError(t, os.Chmod(manifestPath, 0o400)) + t.Cleanup(func() { _ = os.Chmod(manifestPath, 0o600) }) + if probe, err := os.OpenFile(manifestPath, os.O_WRONLY, 0); err == nil { + _ = probe.Close() + t.Skip("the process can write a read-only manifest") + } + err := WriteFiles(root, generate("B")) + require.Error(t, err) + content, readErr := os.ReadFile(path) + require.NoError(t, readErr) + require.Equal(t, "A", string(content), "the update is rolled back with the manifest still recording A") + + require.NoError(t, os.Chmod(manifestPath, 0o600)) + require.NoError(t, WriteFiles(root, generate("B")), "the retry recognizes the file as this host's") + content, readErr = os.ReadFile(path) + require.NoError(t, readErr) + require.Equal(t, "B", string(content)) + require.NoError(t, WriteFiles(root, nil)) + require.NoFileExists(t, path, "removing the generate block removes the file the host owns") + }) + } +} diff --git a/pkg/component/configuration/imports.go b/pkg/component/configuration/imports.go index 51c5327dea9..90b8a60b67b 100644 --- a/pkg/component/configuration/imports.go +++ b/pkg/component/configuration/imports.go @@ -13,31 +13,45 @@ import ( "github.com/cloudposse/atmos/pkg/schema" ) -// RegisterImport routes configured provider schemes before YAML or -// remote-source resolution. Providers add instances to the shared inventory; -// importing a root does not rename its stacks or merge HCL into YAML variables. -func RegisterImport(config *schema.AtmosConfiguration, uri string, declaration *schema.StackImport) (bool, error) { - defer perf.Track(nil, "configuration.RegisterImport")() - +// ImportComponents resolves a provider import into a manifest fragment for the importing stack. +// Static inspection rejects provider imports before entering a potentially effectful runtime. +func ImportComponents(config *schema.AtmosConfiguration, uri string, declaration *schema.StackImport, staticOnly bool) (map[string]any, bool, error) { + defer perf.Track(nil, "configuration.ImportComponents")() scheme, _, found := strings.Cut(uri, "://") if !found || config == nil { - return false, nil + return nil, false, nil } settings, configured := config.Components.Terraform.ConfigurationProviders[scheme] if !configured { - return false, nil + return nil, false, nil + } + // Provider discovery can evaluate HCL; static loads must not enter the runtime. + if staticOnly { + return nil, true, fmt.Errorf("%w: cannot inspect provider import %q without evaluating provider configuration", errUtils.ErrInvalidImport, uri) + } + if err := checkProviderImport(config, scheme, &settings, declaration); err != nil { + return nil, true, err + } + imports := config.ComponentConfigurations.(schema.ComponentConfigurationImportRuntime) + components, handled, err := imports.ImportComponents(context.Background(), uri) + if err != nil || !handled { + return nil, handled, err } + return ImportFragment(components), true, nil +} + +// checkProviderImport rejects an import the configured provider cannot serve. +func checkProviderImport(config *schema.AtmosConfiguration, scheme string, settings *schema.ComponentConfiguration, declaration *schema.StackImport) error { if !settings.Enabled { - return true, fmt.Errorf("%w: configuration provider %s is disabled", errUtils.ErrPluginInvalidConfig, scheme) + return fmt.Errorf("%w: configuration provider %s is disabled", errUtils.ErrPluginInvalidConfig, scheme) } if len(declaration.Context) != 0 || declaration.SkipIfMissing || declaration.TTL != "" || declaration.NestedImports != "" { - return true, fmt.Errorf("%w: provider imports do not support context, skip_if_missing, TTL, or nested_imports", errUtils.ErrPluginUnsupportedExtension) + return fmt.Errorf("%w: provider imports do not support context, skip_if_missing, TTL, or nested_imports", errUtils.ErrPluginUnsupportedExtension) } - imports, ok := config.ComponentConfigurations.(schema.ComponentConfigurationImportRuntime) - if !ok { - return true, fmt.Errorf("%w: provider %s has no import runtime", errUtils.ErrPluginUnsupportedExtension, scheme) + if _, ok := config.ComponentConfigurations.(schema.ComponentConfigurationImportRuntime); !ok { + return fmt.Errorf("%w: provider %s has no import runtime", errUtils.ErrPluginUnsupportedExtension, scheme) } - return imports.RegisterImport(context.Background(), uri) + return nil } // WatchExcludes reads and validates the provider's watch_excludes. diff --git a/pkg/component/configuration/manifest.go b/pkg/component/configuration/manifest.go new file mode 100644 index 00000000000..b111231b26e --- /dev/null +++ b/pkg/component/configuration/manifest.go @@ -0,0 +1,205 @@ +package configuration + +import ( + "crypto/sha256" + "encoding/hex" + "encoding/json" + "errors" + "fmt" + "io" + "maps" + "os" + "path/filepath" + "runtime" + "slices" + + errUtils "github.com/cloudposse/atmos/errors" +) + +// manifestName records, per working directory, the files this host generated and the digest of +// the bytes it wrote. It lets a later run remove a file that is no longer generated and tell a +// file it wrote earlier from one that merely has the same name. It lives in the .atmos directory, +// which saved-plan digests and source refreshes already treat as bookkeeping. +const manifestName = ".atmos/generated-files.json" + +// lockFileName is Terraform's dependency lock file. It is copied from beside the unit and then +// belongs to Terraform, which rewrites it during init. +const lockFileName = ".terraform.lock.hcl" + +// manifest maps a generated path (slash separated) to the SHA-256 of the bytes written. +type manifest struct { + Files map[string]string `json:"files"` +} + +// readManifest returns the recorded digests, or none when the manifest is absent or unreadable. +// An unreadable manifest only costs stale-file cleanup for one run, so it is never an error. +func readManifest(root *os.Root) map[string]string { + file, err := root.Open(manifestName) + if err != nil { + return map[string]string{} + } + defer file.Close() + data, err := io.ReadAll(file) + if err != nil { + return map[string]string{} + } + var stored manifest + if json.Unmarshal(data, &stored) != nil || stored.Files == nil { + return map[string]string{} + } + // Clean recorded paths, so a manifest written before keys were cleaned still matches. + files := make(map[string]string, len(stored.Files)) + for path, digest := range stored.Files { + files[filepath.ToSlash(filepath.Clean(filepath.FromSlash(path)))] = digest + } + return files +} + +func writeManifest(root *os.Root, files map[string]string) error { + if len(files) == 0 { + err := root.Remove(manifestName) + if errors.Is(err, os.ErrNotExist) { + return nil + } + return err + } + data, err := json.MarshalIndent(manifest{Files: files}, "", " ") + if err != nil { + return err + } + if err := root.MkdirAll(filepath.Dir(manifestName), directoryMode); err != nil { + return err + } + return root.WriteFile(manifestName, data, fileMode) +} + +// manifestExists reports whether the working directory has a manifest, without opening a root. +func manifestExists(directory string) bool { + _, err := os.Stat(filepath.Join(directory, filepath.FromSlash(manifestName))) + return err == nil +} + +func contentDigest(content []byte) string { + sum := sha256.Sum256(content) + return hex.EncodeToString(sum[:]) +} + +// fileDigest returns the digest of an existing regular file, or false when there is none. +func fileDigest(root *os.Root, path string) (string, bool) { + info, err := root.Lstat(path) + if err != nil || !info.Mode().IsRegular() { + return "", false + } + file, err := root.Open(path) + if err != nil { + return "", false + } + defer file.Close() + data, err := io.ReadAll(file) + if err != nil { + return "", false + } + return contentDigest(data), true +} + +// removeStaleFiles deletes files an earlier run generated that the current run does not name. +// A file is removed only while it is still what was written (matching digest) or still carries +// the Terragrunt signature; anything a user or module changed since stays untouched. A file that +// cannot be removed stays recorded in kept, so a later run retries it, and the error is returned. +func removeStaleFiles(root *os.Root, previous map[string]string, current []string, kept map[string]string, undo restorePoint) error { + var failures error + for _, path := range slices.Sorted(mapKeys(previous)) { + if slices.Contains(current, path) || !filepath.IsLocal(filepath.FromSlash(path)) { + continue + } + local := filepath.FromSlash(path) + digest, exists := fileDigest(root, local) + if !exists || (digest != previous[path] && verifyTerragruntSignature(root, local) != nil) { + continue + } + undo.capture(root, path) + if err := root.Remove(local); err != nil && !errors.Is(err, os.ErrNotExist) { + kept[path] = previous[path] + failures = errors.Join(failures, fmt.Errorf("%w: remove stale generated file %s: %w", errUtils.ErrPluginInvalidConfig, path, err)) + continue + } + removeEmptyParents(root, filepath.Dir(local)) + } + return failures +} + +// restorePoint remembers the files a run changes, so a run whose manifest cannot be written leaves +// the working directory as it found it instead of with changed files that nobody owns. +type restorePoint map[string]*fileState + +// fileState is a file as it was before the run touched it. +type fileState struct { + exists bool + content []byte + mode os.FileMode +} + +// capture remembers a path's state the first time the run is about to change it. Only regular +// files and absent paths are recorded; anything else is left alone by the run's policies. +func (r restorePoint) capture(root *os.Root, path string) { + if _, seen := r[path]; seen { + return + } + local := filepath.FromSlash(path) + info, err := root.Lstat(local) + switch { + case errors.Is(err, os.ErrNotExist): + r[path] = &fileState{} + case err != nil || !info.Mode().IsRegular(): + return + default: + if content, readErr := root.ReadFile(local); readErr == nil { + r[path] = &fileState{exists: true, content: content, mode: info.Mode().Perm()} + } + } +} + +// restore puts every captured file back, removing the ones the run created. +func (r restorePoint) restore(root *os.Root) error { + var failures error + for _, path := range slices.Sorted(maps.Keys(r)) { + if err := r.restoreFile(root, path); err != nil { + failures = errors.Join(failures, fmt.Errorf("%w: restore generated file %s: %w", errUtils.ErrPluginInvalidConfig, path, err)) + } + } + return failures +} + +func (r restorePoint) restoreFile(root *os.Root, path string) error { + state, local := r[path], filepath.FromSlash(path) + if !state.exists { + return removeFile(root, local) + } + if err := root.MkdirAll(filepath.Dir(local), directoryMode); err != nil { + return err + } + if err := root.WriteFile(local, state.content, state.mode); err != nil { + return err + } + if runtime.GOOS == "windows" { + return nil + } + return root.Chmod(local, state.mode) +} + +// removeEmptyParents tidies directories that only existed for a removed generated file. +func removeEmptyParents(root *os.Root, directory string) { + for directory != "." && directory != string(filepath.Separator) && root.Remove(directory) == nil { + directory = filepath.Dir(directory) + } +} + +func mapKeys(values map[string]string) func(yield func(string) bool) { + return func(yield func(string) bool) { + for key := range values { + if !yield(key) { + return + } + } + } +} diff --git a/pkg/component/configuration/manifest_test.go b/pkg/component/configuration/manifest_test.go new file mode 100644 index 00000000000..042450e45cf --- /dev/null +++ b/pkg/component/configuration/manifest_test.go @@ -0,0 +1,315 @@ +package configuration + +import ( + "os" + "path/filepath" + "runtime" + "testing" + + "github.com/stretchr/testify/require" + + errUtils "github.com/cloudposse/atmos/errors" + "github.com/cloudposse/atmos/pkg/schema" +) + +func generated(path, content, policy string) schema.ComponentConfigurationFile { + return schema.ComponentConfigurationFile{Path: path, Content: []byte(content), IfExists: policy} +} + +func read(t *testing.T, root, path string) string { + t.Helper() + data, err := os.ReadFile(filepath.Join(root, path)) + require.NoError(t, err) + return string(data) +} + +func TestRemovedGenerateBlockStopsBeingApplied(t *testing.T) { + root := t.TempDir() + signed := "# Generated by Terragrunt. Sig: nIlQXj57tbuaRZEa\nvariable \"stale_marker\" {}\n" + unsigned := "variable \"unsigned\" {}\n" + require.NoError(t, WriteFiles(root, []schema.ComponentConfigurationFile{ + generated("provider.tf", signed, "overwrite_terragrunt"), + generated("nested/dir/extra.tf", unsigned, "overwrite"), + generated("keep.tf", unsigned, "overwrite"), + })) + require.FileExists(t, filepath.Join(root, "provider.tf")) + + // The next run generates only keep.tf: both other files were this host's, and both go. + require.NoError(t, WriteFiles(root, []schema.ComponentConfigurationFile{generated("keep.tf", unsigned, "overwrite")})) + require.NoFileExists(t, filepath.Join(root, "provider.tf")) + require.NoFileExists(t, filepath.Join(root, "nested", "dir", "extra.tf")) + require.NoDirExists(t, filepath.Join(root, "nested"), "directories left empty are removed with their file") + require.FileExists(t, filepath.Join(root, "keep.tf")) + + // Generating nothing at all removes the rest and the manifest with it. + require.NoError(t, WriteFiles(root, nil)) + require.NoFileExists(t, filepath.Join(root, "keep.tf")) + require.NoFileExists(t, filepath.Join(root, ".atmos", "generated-files.json")) +} + +func TestStaleCleanupNeverRemovesFilesTheHostDoesNotOwn(t *testing.T) { + root := t.TempDir() + require.NoError(t, WriteFiles(root, []schema.ComponentConfigurationFile{ + generated("edited.tf", "generated\n", "overwrite"), + generated("replaced.tf", "generated\n", "overwrite"), + generated("gone.tf", "generated\n", "overwrite"), + })) + require.NoError(t, os.WriteFile(filepath.Join(root, "edited.tf"), []byte("edited by hand\n"), 0o600)) + require.NoError(t, os.Remove(filepath.Join(root, "replaced.tf"))) + require.NoError(t, os.Mkdir(filepath.Join(root, "replaced.tf"), 0o755), "a directory now has the generated name") + require.NoError(t, os.WriteFile(filepath.Join(root, "module.tf"), []byte("from the module\n"), 0o600)) + require.NoError(t, os.Remove(filepath.Join(root, "gone.tf"))) + + require.NoError(t, WriteFiles(root, nil)) + + require.Equal(t, "edited by hand\n", read(t, root, "edited.tf"), "a changed file is no longer the generated one") + require.DirExists(t, filepath.Join(root, "replaced.tf")) + require.Equal(t, "from the module\n", read(t, root, "module.tf")) +} + +func TestStaleCleanupIgnoresUnsafeManifestPaths(t *testing.T) { + root := t.TempDir() + outside := filepath.Join(t.TempDir(), "victim.tf") + require.NoError(t, os.WriteFile(outside, []byte("x"), 0o600)) + require.NoError(t, os.MkdirAll(filepath.Join(root, ".atmos"), 0o755)) + relative, err := filepath.Rel(root, outside) + require.NoError(t, err) + manifest := `{"files":{"` + filepath.ToSlash(relative) + `":"` + contentDigest([]byte("x")) + `"}}` + require.NoError(t, os.WriteFile(filepath.Join(root, ".atmos", "generated-files.json"), []byte(manifest), 0o600)) + + require.NoError(t, WriteFiles(root, nil)) + require.FileExists(t, outside, "a manifest cannot direct removal outside the working directory") +} + +func TestErrorPolicyAcceptsOnlyTheFileThisHostGenerated(t *testing.T) { + root := t.TempDir() + file := generated("provider.tf", "provider \"x\" {}\n", "error") + require.NoError(t, WriteFiles(root, []schema.ComponentConfigurationFile{file})) + for range 2 { + require.NoError(t, WriteFiles(root, []schema.ComponentConfigurationFile{file}), "a rerun must not fail on its own file") + } + + // New content for the same path replaces the file this host wrote earlier. + file.Content = []byte("provider \"y\" {}\n") + require.NoError(t, WriteFiles(root, []schema.ComponentConfigurationFile{file})) + require.Equal(t, "provider \"y\" {}\n", read(t, root, "provider.tf")) + + // A file the host did not write is a conflict, and the message names the file and the way out. + other := t.TempDir() + require.NoError(t, os.WriteFile(filepath.Join(other, "provider.tf"), []byte("authored\n"), 0o600)) + err := WriteFiles(other, []schema.ComponentConfigurationFile{file}) + require.ErrorIs(t, err, errUtils.ErrPluginInvalidConfig) + require.Contains(t, err.Error(), "provider.tf") + require.Contains(t, err.Error(), "overwrite_terragrunt") + require.Equal(t, "authored\n", read(t, other, "provider.tf")) +} + +func TestOverwriteTerragruntOwnsUnsignedFilesItWrote(t *testing.T) { + root := t.TempDir() + file := generated("secret.auto.tfvars", "token = \"a\"\n", "overwrite_terragrunt") + require.NoError(t, os.WriteFile(filepath.Join(root, "secret.auto.tfvars"), []byte("authored\n"), 0o600)) + require.ErrorIs(t, WriteFiles(root, []schema.ComponentConfigurationFile{file}), errUtils.ErrPluginInvalidConfig) + + fresh := t.TempDir() + require.NoError(t, WriteFiles(fresh, []schema.ComponentConfigurationFile{file})) + file.Content = []byte("token = \"b\"\n") + require.NoError(t, WriteFiles(fresh, []schema.ComponentConfigurationFile{file}), "an unsigned file the host wrote stays owned") + require.Equal(t, "token = \"b\"\n", read(t, fresh, "secret.auto.tfvars")) +} + +// sensitiveFiles are generated files the provider marks sensitive, which the host keeps private. +func sensitiveFiles() []schema.ComponentConfigurationFile { + files := []schema.ComponentConfigurationFile{ + generated("backend.tf", "terraform { backend \"s3\" { secret_key = \"k\" } }\n", "overwrite"), + generated("provider.tf", "provider {}\n", "overwrite"), + } + for i := range files { + files[i].Sensitive = true + } + return files +} + +func TestGeneratedFilesArePrivateToTheOwner(t *testing.T) { + if runtime.GOOS == "windows" { + t.Skip("POSIX permissions") + } + root := t.TempDir() + require.NoError(t, os.WriteFile(filepath.Join(root, "backend.tf"), []byte("old"), 0o644)) + require.NoError(t, WriteFiles(root, sensitiveFiles())) + for _, name := range []string{"backend.tf", "provider.tf"} { + info, err := os.Stat(filepath.Join(root, name)) + require.NoError(t, err) + require.Equal(t, os.FileMode(0o600), info.Mode().Perm(), name) + } + // An unchanged file written by an older run with a wider mode is tightened too. + require.NoError(t, os.Chmod(filepath.Join(root, "provider.tf"), 0o644)) + require.NoError(t, WriteFiles(root, sensitiveFiles())) + info, err := os.Stat(filepath.Join(root, "provider.tf")) + require.NoError(t, err) + require.Equal(t, os.FileMode(0o600), info.Mode().Perm()) +} + +// Terraform rewrites the lock file during init. The unit's copy must not undo that on every run, +// but must take over again when the copy beside the unit changes. +func TestUnitLockFileIsCopiedOnlyWhenItChanges(t *testing.T) { + root := t.TempDir() + lock := func(content string) []schema.ComponentConfigurationFile { + file := generated(lockFileName, content, "overwrite") + file.UnitFileCopy = true + return []schema.ComponentConfigurationFile{file} + } + require.NoError(t, WriteFiles(root, lock("# pinned v1\n"))) + require.Equal(t, "# pinned v1\n", read(t, root, lockFileName)) + + require.NoError(t, os.WriteFile(filepath.Join(root, lockFileName), []byte("# upgraded by init\n"), 0o600)) + require.NoError(t, WriteFiles(root, lock("# pinned v1\n"))) + require.Equal(t, "# upgraded by init\n", read(t, root, lockFileName), "an unchanged unit copy must not overwrite what init wrote") + + require.NoError(t, WriteFiles(root, lock("# pinned v2\n"))) + require.Equal(t, "# pinned v2\n", read(t, root, lockFileName), "an edited unit copy takes over") + + require.NoError(t, os.Remove(filepath.Join(root, lockFileName))) + require.NoError(t, WriteFiles(root, lock("# pinned v2\n"))) + require.Equal(t, "# pinned v2\n", read(t, root, lockFileName), "a missing lock file is restored") +} + +func TestCorruptManifestOnlyCostsCleanup(t *testing.T) { + root := t.TempDir() + require.NoError(t, os.MkdirAll(filepath.Join(root, ".atmos"), 0o755)) + require.NoError(t, os.WriteFile(filepath.Join(root, ".atmos", "generated-files.json"), []byte("{not json"), 0o600)) + require.NoError(t, WriteFiles(root, []schema.ComponentConfigurationFile{generated("a.tf", "a\n", "overwrite")})) + require.Equal(t, "a\n", read(t, root, "a.tf")) + require.Contains(t, read(t, root, ".atmos/generated-files.json"), "a.tf") +} + +// Equivalent spellings of one path name the same generated file, so switching from ./provider.tf +// to provider.tf keeps the file instead of removing it as stale. +func TestEquivalentPathsKeepTheGeneratedFile(t *testing.T) { + root := t.TempDir() + content := "variable \"provider\" {}\n" + require.NoError(t, WriteFiles(root, []schema.ComponentConfigurationFile{generated("./provider.tf", content, "overwrite")})) + require.NoError(t, WriteFiles(root, []schema.ComponentConfigurationFile{generated("provider.tf", content, "overwrite")})) + require.FileExists(t, filepath.Join(root, "provider.tf")) + require.NoError(t, WriteFiles(root, []schema.ComponentConfigurationFile{generated("./provider.tf", content, "overwrite")})) + require.FileExists(t, filepath.Join(root, "provider.tf")) +} + +// A stale generated file that cannot be removed is reported and stays recorded, so a later run +// retries it once permissions are repaired, while the files written in the same run stay owned. +func TestStaleFileRemovalFailureIsReportedAndRetried(t *testing.T) { + root := t.TempDir() + content := "variable \"stale\" {}\n" + require.NoError(t, WriteFiles(root, []schema.ComponentConfigurationFile{ + generated("child/stale.tf", content, "overwrite"), + generated("keep.tf", content, "overwrite"), + })) + child := filepath.Join(root, "child") + require.NoError(t, os.Chmod(child, 0o500)) + t.Cleanup(func() { _ = os.Chmod(child, 0o755) }) + // A process that can bypass the restriction, such as root with CAP_DAC_OVERRIDE, would remove + // the stale file, so the failure under test cannot occur. + requireEffectiveDirectoryPermissions(t, child) + + err := WriteFiles(root, []schema.ComponentConfigurationFile{generated("keep.tf", content, "overwrite")}) + require.ErrorIs(t, err, errUtils.ErrPluginInvalidConfig) + require.ErrorContains(t, err, "child/stale.tf") + require.FileExists(t, filepath.Join(child, "stale.tf")) + + require.NoError(t, os.Chmod(child, 0o755)) + require.NoError(t, WriteFiles(root, []schema.ComponentConfigurationFile{generated("keep.tf", content, "overwrite")})) + require.NoFileExists(t, filepath.Join(child, "stale.tf")) + require.FileExists(t, filepath.Join(root, "keep.tf")) +} + +// When a later file in a batch conflicts, the files already updated keep their new digests, so a +// retry after the conflict is fixed still accepts them as this host's. +func TestBatchFailureKeepsTheRecordsOfFilesItAlreadyWrote(t *testing.T) { + root := t.TempDir() + first := generated("secret.auto.tfvars", "token = \"a\"\n", "overwrite_terragrunt") + second := generated("guarded.tf", "variable \"guarded\" {}\n", "error") + require.NoError(t, WriteFiles(root, []schema.ComponentConfigurationFile{first, second})) + + // Someone else replaces the guarded file, and the next run updates the unsigned file first. + require.NoError(t, os.WriteFile(filepath.Join(root, "guarded.tf"), []byte("authored\n"), 0o600)) + first.Content = []byte("token = \"b\"\n") + require.ErrorIs(t, WriteFiles(root, []schema.ComponentConfigurationFile{first, second}), errUtils.ErrPluginInvalidConfig) + require.Equal(t, "token = \"b\"\n", read(t, root, "secret.auto.tfvars")) + + // After the conflict is fixed, the updated file is still this host's. + require.NoError(t, os.Remove(filepath.Join(root, "guarded.tf"))) + require.NoError(t, WriteFiles(root, []schema.ComponentConfigurationFile{first, second})) + require.Equal(t, "token = \"b\"\n", read(t, root, "secret.auto.tfvars")) +} + +// An explicit generate block must keep enforcing its policy after init changes the lock. +func TestExplicitLockFilePoliciesAfterTerraformUpdates(t *testing.T) { + for _, policy := range []string{"overwrite", "skip", "error"} { + t.Run(policy, func(t *testing.T) { + root := t.TempDir() + files := []schema.ComponentConfigurationFile{generated(lockFileName, "# generated pins\n", policy)} + require.NoError(t, WriteFiles(root, files)) + require.NoError(t, os.WriteFile(filepath.Join(root, lockFileName), []byte("# upgraded by init\n"), 0o600)) + err := WriteFiles(root, files) + if policy == "error" { + require.ErrorIs(t, err, errUtils.ErrPluginInvalidConfig) + } else { + require.NoError(t, err) + } + want := "# upgraded by init\n" + if policy == "overwrite" { + want = "# generated pins\n" + } + require.Equal(t, want, read(t, root, lockFileName)) + }) + } +} + +func TestExplicitLockGenerationReplacesAdjacentCopy(t *testing.T) { + root := t.TempDir() + file := generated(lockFileName, "# pinned\n", "overwrite") + file.UnitFileCopy = true + require.NoError(t, WriteFiles(root, []schema.ComponentConfigurationFile{file})) + require.NoError(t, os.WriteFile(filepath.Join(root, lockFileName), []byte("# upgraded by init\n"), 0o600)) + // The explicit block has identical bytes to the old copy; provenance alone changes. + file.UnitFileCopy = false + require.NoError(t, WriteFiles(root, []schema.ComponentConfigurationFile{file})) + require.Equal(t, "# pinned\n", read(t, root, lockFileName)) +} + +// requireEffectiveDirectoryPermissions skips a test when the process can still create files in a +// read-only directory, as root with CAP_DAC_OVERRIDE can, or on Windows. +func requireEffectiveDirectoryPermissions(t *testing.T, directory string) { + t.Helper() + if runtime.GOOS == "windows" { + t.Skip("directory permissions do not prevent changes on Windows") + } + probe := filepath.Join(directory, ".permission-probe") + if err := os.WriteFile(probe, nil, 0o600); err == nil { + _ = os.Remove(probe) + t.Skip("the process bypasses directory permissions") + } +} + +// A replacement that cannot be written leaves the previous file and its recorded digest in place, +// so the retry still recognizes the file as this host's under an ownership-checking policy. +func TestFailedReplacementKeepsTheRecordedFile(t *testing.T) { + root := t.TempDir() + original := generated("child/backend.tf", "terraform {}\n", "overwrite_terragrunt") + require.NoError(t, WriteFiles(root, []schema.ComponentConfigurationFile{original})) + child := filepath.Join(root, "child") + require.NoError(t, os.Chmod(child, 0o500)) + t.Cleanup(func() { _ = os.Chmod(child, 0o755) }) + requireEffectiveDirectoryPermissions(t, child) + + replacement := generated("child/backend.tf", "terraform { backend \"s3\" {} }\n", "overwrite_terragrunt") + require.ErrorIs(t, WriteFiles(root, []schema.ComponentConfigurationFile{replacement}), errUtils.ErrPluginInvalidConfig) + require.Equal(t, "terraform {}\n", read(t, root, "child/backend.tf"), "the previous bytes stay intact") + entries, err := os.ReadDir(child) + require.NoError(t, err) + require.Len(t, entries, 1, "no temporary file is left behind") + require.Equal(t, "backend.tf", entries[0].Name()) + + require.NoError(t, os.Chmod(child, 0o755)) + require.NoError(t, WriteFiles(root, []schema.ComponentConfigurationFile{replacement})) + require.Equal(t, "terraform { backend \"s3\" {} }\n", read(t, root, "child/backend.tf")) +} diff --git a/pkg/component/configuration/mock_import_runtime_test.go b/pkg/component/configuration/mock_import_runtime_test.go new file mode 100644 index 00000000000..45beaba44de --- /dev/null +++ b/pkg/component/configuration/mock_import_runtime_test.go @@ -0,0 +1,88 @@ +// Code generated by MockGen. DO NOT EDIT. +// Source: registry_test.go +// +// Generated by this command: +// +// mockgen -source=registry_test.go -destination=mock_import_runtime_test.go -package=configuration -mock_names=importingRuntime=MockImportingRuntime +// + +// Package configuration is a generated GoMock package. +package configuration + +import ( + context "context" + reflect "reflect" + + schema "github.com/cloudposse/atmos/pkg/schema" + gomock "go.uber.org/mock/gomock" +) + +// MockImportingRuntime is a mock of importingRuntime interface. +type MockImportingRuntime struct { + ctrl *gomock.Controller + recorder *MockImportingRuntimeMockRecorder + isgomock struct{} +} + +// MockImportingRuntimeMockRecorder is the mock recorder for MockImportingRuntime. +type MockImportingRuntimeMockRecorder struct { + mock *MockImportingRuntime +} + +// NewMockImportingRuntime creates a new mock instance. +func NewMockImportingRuntime(ctrl *gomock.Controller) *MockImportingRuntime { + mock := &MockImportingRuntime{ctrl: ctrl} + mock.recorder = &MockImportingRuntimeMockRecorder{mock} + return mock +} + +// EXPECT returns an object that allows the caller to indicate expected use. +func (m *MockImportingRuntime) EXPECT() *MockImportingRuntimeMockRecorder { + return m.recorder +} + +// Discover mocks base method. +func (m *MockImportingRuntime) Discover(arg0 context.Context) (map[string]map[string]any, error) { + m.ctrl.T.Helper() + ret := m.ctrl.Call(m, "Discover", arg0) + ret0, _ := ret[0].(map[string]map[string]any) + ret1, _ := ret[1].(error) + return ret0, ret1 +} + +// Discover indicates an expected call of Discover. +func (mr *MockImportingRuntimeMockRecorder) Discover(arg0 any) *gomock.Call { + mr.mock.ctrl.T.Helper() + return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "Discover", reflect.TypeOf((*MockImportingRuntime)(nil).Discover), arg0) +} + +// ImportComponents mocks base method. +func (m *MockImportingRuntime) ImportComponents(arg0 context.Context, arg1 string) (map[string]any, bool, error) { + m.ctrl.T.Helper() + ret := m.ctrl.Call(m, "ImportComponents", arg0, arg1) + ret0, _ := ret[0].(map[string]any) + ret1, _ := ret[1].(bool) + ret2, _ := ret[2].(error) + return ret0, ret1, ret2 +} + +// ImportComponents indicates an expected call of ImportComponents. +func (mr *MockImportingRuntimeMockRecorder) ImportComponents(arg0, arg1 any) *gomock.Call { + mr.mock.ctrl.T.Helper() + return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "ImportComponents", reflect.TypeOf((*MockImportingRuntime)(nil).ImportComponents), arg0, arg1) +} + +// Prepare mocks base method. +func (m *MockImportingRuntime) Prepare(arg0 context.Context, arg1 *schema.ConfigAndStacksInfo) (*schema.PreparedComponentConfiguration, error) { + m.ctrl.T.Helper() + ret := m.ctrl.Call(m, "Prepare", arg0, arg1) + ret0, _ := ret[0].(*schema.PreparedComponentConfiguration) + ret1, _ := ret[1].(error) + return ret0, ret1 +} + +// Prepare indicates an expected call of Prepare. +func (mr *MockImportingRuntimeMockRecorder) Prepare(arg0, arg1 any) *gomock.Call { + mr.mock.ctrl.T.Helper() + return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "Prepare", reflect.TypeOf((*MockImportingRuntime)(nil).Prepare), arg0, arg1) +} diff --git a/pkg/component/configuration/ownership.go b/pkg/component/configuration/ownership.go index 90336cf0d12..028424da171 100644 --- a/pkg/component/configuration/ownership.go +++ b/pkg/component/configuration/ownership.go @@ -26,7 +26,10 @@ func validFilePolicy(policy string) bool { // checkFileOwnership never follows a final symlink when deciding ownership. // The os.Root handle additionally confines all ancestor traversal to the provisioned directory. -func checkFileOwnership(root *os.Root, file schema.ComponentConfigurationFile) (bool, error) { +// +// The recorded digest is what this host wrote for the path in an earlier run. A file that still +// has those bytes is owned even without a signature, which covers generate blocks that disable it. +func checkFileOwnership(root *os.Root, file schema.ComponentConfigurationFile, recorded string) (bool, error) { remove := file.IfExists == "remove" || file.IfExists == "remove_terragrunt" stat, err := root.Lstat(file.Path) if errors.Is(err, os.ErrNotExist) { @@ -41,12 +44,20 @@ func checkFileOwnership(root *os.Root, file schema.ComponentConfigurationFile) ( if file.IfExists != "overwrite_terragrunt" && file.IfExists != "remove_terragrunt" { return remove, nil } - if err := verifyTerragruntSignature(root, file.Path); err != nil { + if err := verifyOwnership(root, file.Path, recorded); err != nil { return false, err } return remove, nil } +// verifyOwnership accepts the bytes this host recorded, or else the Terragrunt signature. +func verifyOwnership(root *os.Root, path, recorded string) error { + if digest, ok := fileDigest(root, path); ok && recorded != "" && digest == recorded { + return nil + } + return verifyTerragruntSignature(root, path) +} + // verifyTerragruntSignature requires the first line of the generated file to carry the Terragrunt signature. func verifyTerragruntSignature(root *os.Root, path string) error { input, err := root.Open(path) diff --git a/pkg/component/configuration/registry_test.go b/pkg/component/configuration/registry_test.go index 56b029085ba..4061f1f8000 100644 --- a/pkg/component/configuration/registry_test.go +++ b/pkg/component/configuration/registry_test.go @@ -9,6 +9,7 @@ import ( "github.com/santhosh-tekuri/jsonschema/v5" "github.com/stretchr/testify/require" + "go.uber.org/mock/gomock" errUtils "github.com/cloudposse/atmos/errors" "github.com/cloudposse/atmos/pkg/schema" @@ -53,46 +54,55 @@ func TestFactoryRegistrationAndOptIn(t *testing.T) { require.NoError(t, Attach(t.Context(), config, "list")) } -type importRuntime struct { +//go:generate go run go.uber.org/mock/mockgen@v0.6.0 -source=$GOFILE -destination=mock_import_runtime_test.go -package=$GOPACKAGE -mock_names=importingRuntime=MockImportingRuntime + +// importingRuntime is a configuration runtime that also resolves provider imports. +type importingRuntime interface { schema.ComponentConfigurationRuntime - imports []string - err error + schema.ComponentConfigurationImportRuntime } -func (r *importRuntime) RegisterImport(_ context.Context, uri string) (bool, error) { - r.imports = append(r.imports, uri) - return true, r.err -} +// The generated mock implements the contract that ImportComponents asserts. +var _ importingRuntime = (*MockImportingRuntime)(nil) func TestConfigurationImportRouting(t *testing.T) { config := &schema.AtmosConfiguration{} for _, uri := range []string{"ordinary", "https://example.com/config.yaml"} { - handled, err := RegisterImport(config, uri, &schema.StackImport{}) + _, handled, err := ImportComponents(config, uri, &schema.StackImport{}, false) require.NoError(t, err) require.False(t, handled) } - handled, err := RegisterImport(nil, "terragrunt://live", &schema.StackImport{}) + _, handled, err := ImportComponents(nil, "terragrunt://live", &schema.StackImport{}, false) require.NoError(t, err) require.False(t, handled) config.Components.Terraform.ConfigurationProviders = map[string]schema.ComponentConfiguration{"terragrunt": {}} - _, err = RegisterImport(config, "terragrunt://live", &schema.StackImport{}) + _, _, err = ImportComponents(config, "terragrunt://live", &schema.StackImport{}, false) require.ErrorContains(t, err, "disabled") config.Components.Terraform.ConfigurationProviders["terragrunt"] = schema.ComponentConfiguration{Enabled: true} - _, err = RegisterImport(config, "terragrunt://live", &schema.StackImport{}) + _, _, err = ImportComponents(config, "terragrunt://live", &schema.StackImport{}, false) require.ErrorIs(t, err, errUtils.ErrPluginUnsupportedExtension) - runtime := &importRuntime{} + // Every expectation is explicit: an unexpected ImportComponents call, such as one from a + // static load or a rejected import declaration, fails the test. + runtime := NewMockImportingRuntime(gomock.NewController(t)) config.ComponentConfigurations = runtime for _, declaration := range []schema.StackImport{{Context: map[string]any{"stage": "dev"}}, {SkipIfMissing: true}, {TTL: "1h"}, {NestedImports: "local"}} { - _, err := RegisterImport(config, "terragrunt://live", &declaration) + _, _, err := ImportComponents(config, "terragrunt://live", &declaration, false) require.ErrorIs(t, err, errUtils.ErrPluginUnsupportedExtension) } - handled, err = RegisterImport(config, "terragrunt://live", &schema.StackImport{}) + _, handled, err = ImportComponents(config, "terragrunt://live", &schema.StackImport{}, true) + require.True(t, handled) + require.ErrorIs(t, err, errUtils.ErrInvalidImport, "static loads must not enter the provider runtime") + registration := errors.New("registration failed") + gomock.InOrder( + runtime.EXPECT().ImportComponents(gomock.Any(), "terragrunt://live").Return(map[string]any{"app": map[string]any{}}, true, nil), + runtime.EXPECT().ImportComponents(gomock.Any(), "terragrunt://live").Return(map[string]any{"app": map[string]any{}}, true, registration), + ) + fragment, handled, err := ImportComponents(config, "terragrunt://live", &schema.StackImport{}, false) require.NoError(t, err) require.True(t, handled) - require.Equal(t, []string{"terragrunt://live"}, runtime.imports) - runtime.err = errors.New("registration failed") - _, err = RegisterImport(config, "terragrunt://live", &schema.StackImport{}) - require.ErrorIs(t, err, runtime.err) + require.Equal(t, map[string]any{"components": map[string]any{"terraform": map[string]any{"app": map[string]any{}}}}, fragment) + _, _, err = ImportComponents(config, "terragrunt://live", &schema.StackImport{}, false) + require.ErrorIs(t, err, registration) } func TestWatchExcludes(t *testing.T) { @@ -111,6 +121,26 @@ func TestWatchExcludes(t *testing.T) { require.ErrorIs(t, err, errUtils.ErrPluginInvalidConfig) } +func TestApplyUnitOverridesCopiesEachUnit(t *testing.T) { + unit := map[string]any{"vars": map[string]any{"a": 1}} + terraform := map[string]any{"app": unit, "invalid": "not a section"} + fragments := []map[string]any{{"components": map[string]any{"terraform": terraform}}, {}} + + ApplyUnitOverrides(fragments, nil) + require.Equal(t, map[string]any{"vars": map[string]any{"a": 1}}, terraform["app"], "empty overrides must leave units untouched") + + overrides := map[string]any{"vars": map[string]any{"region": "us-west-2"}} + ApplyUnitOverrides(fragments, overrides) + app := terraform["app"].(map[string]any) + require.Equal(t, overrides, app["overrides"]) + require.Equal(t, map[string]any{"a": 1}, app["vars"]) + require.NotContains(t, unit, "overrides", "the provider's unit must not be changed") + require.Equal(t, "not a section", terraform["invalid"]) + + unit["vars"] = "changed later" + require.Equal(t, map[string]any{"a": 1}, app["vars"], "replacing a source section must not reach the attached unit") +} + func TestMatchDependencies(t *testing.T) { deps := []schema.ComponentDependency{{Kind: "folder", Path: "short/unit"}, {Kind: "file", Path: "short/a.tf"}} resolve := func(path string) string { return "long/" + path } @@ -186,3 +216,21 @@ func TestCompileManifestSchemaWrapsFailures(t *testing.T) { require.NoError(t, err) require.NotNil(t, compiled) } + +// Only a provider owns the backend; an empty `settings.configuration` map leaves ordinary components +// alone. Each component-level layer, inherited or the component's own, is checked. +func TestBackendOverridesRequireProviderOwnership(t *testing.T) { + s3 := BackendLayer{Type: "s3", Settings: map[string]any{"bucket": "other"}} + owned := map[string]any{"configuration": map[string]any{"provider": "terragrunt"}} + require.NoError(t, CheckBackendOverrides("app", "dev", map[string]any{"configuration": map[string]any{}}, s3)) + require.NoError(t, CheckBackendOverrides("app", "dev", map[string]any{}, s3)) + require.NoError(t, CheckBackendOverrides("app", "dev", owned, BackendLayer{Type: "local"}, BackendLayer{})) + + err := CheckBackendOverrides("app", "dev", owned, BackendLayer{}, s3) + require.ErrorIs(t, err, errUtils.ErrConfigurationOverrideUnsupported) + require.ErrorContains(t, err, "backend_type") + require.ErrorContains(t, err, `component "app" in stack "dev"`) + + inherited := BackendLayer{Type: "local", Settings: map[string]any{"local": map[string]any{"path": "inherited.tfstate"}}} + require.ErrorContains(t, CheckBackendOverrides("app", "dev", owned, inherited), "backend.local") +} diff --git a/pkg/component/execution/arguments.go b/pkg/component/execution/arguments.go index 15825cde6f3..5c6373b1279 100644 --- a/pkg/component/execution/arguments.go +++ b/pkg/component/execution/arguments.go @@ -138,9 +138,15 @@ func variableOption(argument string) (name string, hasValue bool) { func requiredVarFiles(files []string, directory string) ([]string, error) { files = keepLast(files) for _, file := range files { - if !fileExists(resolve(directory, file)) { + resolved := resolve(directory, file) + if fileExists(resolved) { + continue + } + if filepath.IsAbs(file) { return nil, fmt.Errorf("%w: %s", errUtils.ErrComponentVarFileMissing, file) } + // A relative name resolves in the provisioned module directory, not beside terragrunt.hcl. + return nil, fmt.Errorf("%w: %s (looked in %s, the directory Terraform runs in; for a file next to terragrunt.hcl use ${get_terragrunt_dir()}/%s)", errUtils.ErrComponentVarFileMissing, file, directory, file) } return files, nil } diff --git a/pkg/component/execution/execution.go b/pkg/component/execution/execution.go index 4bbb2adf9be..9475786d4b7 100644 --- a/pkg/component/execution/execution.go +++ b/pkg/component/execution/execution.go @@ -36,11 +36,15 @@ const ( ) // CommandName returns the name that hooks and extra arguments match against. -// It is "destroy" for `destroy` and for `apply` with the destroy flag set, and the command -// itself otherwise, so `plan -destroy` still matches "plan". +// It is "destroy" for `destroy` and for `apply` with the destroy flag set, and the Terraform +// command itself otherwise, so `plan -destroy` still matches "plan". Like Terragrunt, only the +// command is matched, never its subcommand: `state list` and `state rm` both match "state". func CommandName(command string, flags []string) string { defer perf.Track(nil, "execution.CommandName")() + if fields := strings.Fields(command); len(fields) > 0 { + command = fields[0] + } if command == commandDestroy || (command == commandApply && hasDestroyFlag(flags)) { return commandDestroy } diff --git a/pkg/component/execution/execution_test.go b/pkg/component/execution/execution_test.go new file mode 100644 index 00000000000..962365e125c --- /dev/null +++ b/pkg/component/execution/execution_test.go @@ -0,0 +1,53 @@ +package execution + +import ( + "os" + "path/filepath" + "testing" + + "github.com/stretchr/testify/require" + + errUtils "github.com/cloudposse/atmos/errors" + "github.com/cloudposse/atmos/pkg/schema" +) + +func TestCommandNameMatchesTheCommandNotItsSubcommand(t *testing.T) { + for _, test := range []struct { + command string + flags []string + want string + }{ + {"plan", nil, "plan"}, + {"plan", []string{"-destroy"}, "plan"}, + {"apply", []string{"-destroy"}, "destroy"}, + {"destroy", nil, "destroy"}, + {"state list", nil, "state"}, + {"state rm", []string{"x"}, "state"}, + {"workspace", nil, "workspace"}, + {"", nil, ""}, + } { + require.Equal(t, test.want, CommandName(test.command, test.flags), test.command) + } +} + +func TestRequiredVarFileErrorsSayWhereARelativeNameWasLooked(t *testing.T) { + directory := t.TempDir() + items := []schema.ComponentExtraArguments{{Commands: []string{"plan"}, RequiredVarFiles: []string{"vars.tfvars"}}} + + _, err := ApplyExtraArguments(items, &ArgumentsRequest{Command: "plan", Args: []string{"plan"}, InsertAt: 1, WorkingDirectory: directory}) + require.ErrorIs(t, err, errUtils.ErrComponentVarFileMissing) + require.Contains(t, err.Error(), "vars.tfvars") + require.Contains(t, err.Error(), directory, "the directory the name was resolved against must be named") + require.Contains(t, err.Error(), "get_terragrunt_dir()") + + absolute := filepath.Join(directory, "absent.tfvars") + items[0].RequiredVarFiles = []string{absolute} + _, err = ApplyExtraArguments(items, &ArgumentsRequest{Command: "plan", Args: []string{"plan"}, InsertAt: 1, WorkingDirectory: directory}) + require.ErrorIs(t, err, errUtils.ErrComponentVarFileMissing) + require.Contains(t, err.Error(), absolute) + require.NotContains(t, err.Error(), "get_terragrunt_dir()") + + require.NoError(t, os.WriteFile(absolute, nil, 0o600)) + _, err = ApplyExtraArguments(items, &ArgumentsRequest{Command: "plan", Args: []string{"plan"}, InsertAt: 1, WorkingDirectory: directory}) + require.NoError(t, err) +} diff --git a/pkg/component/terraform/configuration.go b/pkg/component/terraform/configuration.go index 0174c7bc483..ca111639bf3 100644 --- a/pkg/component/terraform/configuration.go +++ b/pkg/component/terraform/configuration.go @@ -1,23 +1,36 @@ package terraform import ( + "bytes" "context" "encoding/hex" + "errors" "fmt" "hash/fnv" "maps" "os" "os/exec" "path/filepath" + "runtime" + "slices" "strings" + "github.com/google/shlex" + "github.com/hashicorp/hcl/v2" + "github.com/hashicorp/hcl/v2/hclparse" + errUtils "github.com/cloudposse/atmos/errors" "github.com/cloudposse/atmos/pkg/component" + "github.com/cloudposse/atmos/pkg/component/configuration" cfg "github.com/cloudposse/atmos/pkg/config" "github.com/cloudposse/atmos/pkg/dependencies" + envpkg "github.com/cloudposse/atmos/pkg/env" + "github.com/cloudposse/atmos/pkg/perf" "github.com/cloudposse/atmos/pkg/provisioner" "github.com/cloudposse/atmos/pkg/provisioner/workdir" "github.com/cloudposse/atmos/pkg/schema" + "github.com/cloudposse/atmos/pkg/terraform/backend" + "github.com/cloudposse/atmos/pkg/ui" u "github.com/cloudposse/atmos/pkg/utils" ) @@ -31,6 +44,9 @@ func prepareConfiguration(ctx context.Context, config *schema.AtmosConfiguration if err := checkComponentRestrictions(info); err != nil { return nil, err } + if err := checkStackOverrides(info); err != nil { + return nil, err + } if info.DryRun { return nil, nil } @@ -41,8 +57,20 @@ func prepareConfiguration(ctx context.Context, config *schema.AtmosConfiguration if err != nil || prepared == nil { return prepared, err } + if err = applyStackOverrides(config, info, prepared); err != nil { + return nil, err + } + applyPreparedConfiguration(config, info, prepared, execution) + warnOnCacheLocalState(info, prepared) + return prepared, nil +} + +// applyPreparedConfiguration projects the prepared configuration onto the invocation. +func applyPreparedConfiguration(config *schema.AtmosConfiguration, info *schema.ConfigAndStacksInfo, prepared *schema.PreparedComponentConfiguration, execution *schema.ComponentExecution) { info.ComponentVarsSection = prepared.Inputs - info.ComponentExecution = execution + // The stack's env section wins over provider environment in the metadata this invocation runs, + // including the apply-time metadata of a restored saved plan. + info.ComponentExecution = withoutStackEnvironment(execution, info.ComponentEnvSection) info.ConfigurationSensitiveInputKeys = maps.Clone(prepared.SensitiveInputKeys) info.ComponentBackendType, info.ComponentBackendSection = prepared.BackendType, prepared.Backend info.TerraformWorkspace = prepared.Workspace @@ -53,16 +81,44 @@ func prepareConfiguration(ctx context.Context, config *schema.AtmosConfiguration source = make(map[string]any) } source["uri"] = prepared.Source - if isLocalModuleSource(prepared.Source) { - // A local module changes without its location changing, so the retained workdir - // would keep the code of the first run. Refreshing is a cheap local copy. - source["ttl"] = "0s" - } + // A local module changes without its location changing. The source provisioner stamps a local + // directory and refreshes the retained workdir only when that stamp changes, so no TTL is set. info.ComponentSection["source"] = source info.ComponentSection["backend_type"] = prepared.BackendType info.ComponentSection["backend"] = prepared.Backend config.Components.Terraform.AutoGenerateBackendFile = !prepared.BackendGenerated - return prepared, nil +} + +// terragruntCacheDirectory is where Terragrunt keeps its working copies of a unit. +const terragruntCacheDirectory = ".terragrunt-cache" + +// cacheLocalStatePath returns the local state file path when it lives inside a Terragrunt cache. +func cacheLocalStatePath(prepared *schema.PreparedComponentConfiguration) (string, bool) { + if prepared.BackendType != "local" { + return "", false + } + path, _ := prepared.Backend["path"].(string) + for _, segment := range strings.Split(filepath.ToSlash(path), "/") { + if segment == terragruntCacheDirectory { + return path, true + } + } + return "", false +} + +// warnOnCacheLocalState tells the user that state kept inside .terragrunt-cache is easy to lose. +// The location is deliberate (it is Terragrunt's own backend identity, so existing state keeps +// working), but cleaning the cache, or checking out to another path, silently starts from empty state. +func warnOnCacheLocalState(info *schema.ConfigAndStacksInfo, prepared *schema.PreparedComponentConfiguration) { + path, inCache := cacheLocalStatePath(prepared) + if !inCache { + return + } + ui.Warningf("Component `%s` in stack `%s` keeps its local state at `%s`, inside `%s`. "+ + "The location depends on the checkout path, and deleting the cache (for example by cleaning untracked files) loses the state.", + info.ComponentFromArg, info.Stack, path, terragruntCacheDirectory) + ui.Hint("Configure `remote_state` with a remote backend, or a local backend with a stable path such as " + + "`path = \"${get_terragrunt_dir()}/terraform.tfstate\"`, and move the existing state file there.") } // resolveHostExecutable makes info.Command the absolute path of the selected executable. @@ -102,11 +158,6 @@ func prepareOrRestore(ctx context.Context, config *schema.AtmosConfiguration, in return prepared, prepared.Execution.ForSavedPlanApply(), nil } -// isLocalModuleSource reports whether the evaluated module source is a path on this machine. -func isLocalModuleSource(uri string) bool { - return strings.HasPrefix(uri, ".") || filepath.IsAbs(uri) -} - // useRetainedWorkdir points the invocation at the directory the reviewed plan was created in. // A new process has not provisioned anything, so the path is derived the way provisioning does. func useRetainedWorkdir(config *schema.AtmosConfiguration, info *schema.ConfigAndStacksInfo) error { @@ -175,6 +226,112 @@ func (e *Executor) prepareConfigurationEnvironment(ctx context.Context, config * return tenv, nil } +// writePreparedConfiguration verifies retained files before relinquishing the host backend +// and applying provider file policies. +func writePreparedConfiguration(directory string, prepared *schema.PreparedComponentConfiguration) error { + if err := validateRetainedConfiguration(directory, prepared.Files); err != nil { + return err + } + if prepared.BackendGenerated { + if err := backend.RemoveHostBackend(directory); err != nil { + return err + } + } + return configuration.WriteFiles(directory, prepared.Files) +} + +// validateRetainedConfiguration runs before any backend cleanup or file writes. +// A skipped Terraform file may disagree with the provider's resolved state identity. +// Accept different bytes only when both versions are provably free of backend/cloud +// declarations; otherwise fail closed without replacing the user's retained file. +func validateRetainedConfiguration(directory string, files []schema.ComponentConfigurationFile) error { + defer perf.Track(nil, "terraform.validateRetainedConfiguration")() + + for _, file := range files { + if file.IfExists != "skip" || !terraformConfigurationPath(file.Path) { + continue + } + if err := validateRetainedFile(directory, file); err != nil { + return err + } + } + return nil +} + +func validateRetainedFile(directory string, file schema.ComponentConfigurationFile) error { + if !filepath.IsLocal(file.Path) { + return errUtils.ErrPluginProtocol + } + root, err := os.OpenRoot(directory) + if err != nil { + return retainedFileError(file.Path, err) + } + defer root.Close() + stat, err := root.Lstat(file.Path) + if errors.Is(err, os.ErrNotExist) { + return nil + } + if err != nil { + return retainedFileError(file.Path, err) + } + if !stat.Mode().IsRegular() { + return retainedBackendError(file.Path) + } + retained, err := root.ReadFile(file.Path) + if err != nil { + return retainedFileError(file.Path, err) + } + if bytes.Equal(retained, file.Content) { + return nil + } + if configurationHasBackend(file.Path, retained) || configurationHasBackend(file.Path, file.Content) { + return retainedBackendError(file.Path) + } + return nil +} + +// retainedFileError classifies a failure to inspect a retained generated file as invalid +// configuration, as configuration.WriteFiles does for its working directory, and keeps the cause. +func retainedFileError(path string, err error) error { + return fmt.Errorf("%w: retained generated file %s: %w", errUtils.ErrPluginInvalidConfig, path, err) +} + +func retainedBackendError(path string) error { + return fmt.Errorf("%w: cannot verify backend identity for retained generated file %s (if_exists = \"skip\"); use matching generated content or an explicit overwrite policy", errUtils.ErrPluginInvalidConfig, path) +} + +func terraformConfigurationPath(path string) bool { + lower := strings.ToLower(path) + return strings.HasSuffix(lower, ".tf") || strings.HasSuffix(lower, ".tf.json") +} + +// configurationHasBackend also returns true for unparseable configuration: absence +// of a backend cannot then be established. Diagnostics never expose retained secrets. +func configurationHasBackend(path string, content []byte) bool { + parser := hclparse.NewParser() + var file *hcl.File + var diagnostics hcl.Diagnostics + if strings.HasSuffix(strings.ToLower(path), ".json") { + file, diagnostics = parser.ParseJSON(content, path) + } else { + file, diagnostics = parser.ParseHCL(content, path) + } + if diagnostics.HasErrors() { + return true + } + body, _, diagnostics := file.Body.PartialContent(&hcl.BodySchema{Blocks: []hcl.BlockHeaderSchema{{Type: "terraform"}}}) + if diagnostics.HasErrors() { + return true + } + for _, block := range body.Blocks { + nested, _, diags := block.Body.PartialContent(&hcl.BodySchema{Blocks: []hcl.BlockHeaderSchema{{Type: "backend", LabelNames: []string{"type"}}, {Type: "cloud"}}}) + if diags.HasErrors() || len(nested.Blocks) != 0 { + return true + } + } + return false +} + // checkConfigurationWorkspace rejects a TF_WORKSPACE that would make Terraform use a different // workspace than the one the configuration provider prepared, before any initialization runs. // The component environment overrides the base environment, as it does for the Terraform process. @@ -229,3 +386,240 @@ func isRuntimeArtifact(path string) bool { } return false } + +// validatePlanOutputFlags rejects ambiguous plan destinations before hooks or Terraform run. +func validatePlanOutputFlags(command string, args []string) error { + if command != "plan" { + return nil + } + count := 0 + for i := 0; i < len(args); i++ { + arg := args[i] + if arg == "-out" || arg == "--out" { + count++ + i++ + } else if strings.HasPrefix(arg, "-out=") || strings.HasPrefix(arg, "--out=") { + count++ + } + if count > 1 { + return fmt.Errorf("%w: multiple plan output flags", errUtils.ErrPlanVerificationFailed) + } + } + return nil +} + +// validateEnvironmentPlanOutput rejects plan output flags in TF_CLI_ARGS or +// TF_CLI_ARGS_plan for a configuration-owned plan. Terraform reads these +// variables itself, so a flag there could choose a plan file that Atmos does not +// see and therefore does not bind. The check inspects the effective value of each +// variable in the subprocess environment: the sanitized process environment (or +// the inherited one when auth did not sanitize it), the global atmos.yaml +// environment, the component environment, and provider extra-argument +// environment, with later layers overriding earlier ones. Hooks run as child +// processes and cannot change it. +func validateEnvironmentPlanOutput(atmosConfig *schema.AtmosConfiguration, info *schema.ConfigAndStacksInfo, execCtx *componentExecContext, opts ...component.ExecutionOption) error { + if execCtx.preparedConfiguration == nil || info.DryRun || info.SubCommand != subcommandPlan { + return nil + } + return rejectEnvironmentPlanOutput(planEnvironment(atmosConfig, info, opts), runtime.GOOS) +} + +// rejectEnvironmentPlanOutput rejects a plan output flag in the effective TF_CLI_ARGS or +// TF_CLI_ARGS_plan of an environment listed in precedence order. Variable names are +// case-insensitive on Windows only, where the last matching entry is the effective value. +func rejectEnvironmentPlanOutput(entries []string, goos string) error { + for _, name := range []string{tfCliArgsEnvVar, tfCliArgsEnvVar + "_" + subcommandPlan} { + value, _ := lastEnvListValue(entries, name, goos) + args, err := tokenizeEnvironmentArgs(value) + if err != nil { + return fmt.Errorf("%w: %s cannot be parsed: %w", errUtils.ErrPlanVerificationFailed, name, err) + } + if containsOutputFlag(args) { + return fmt.Errorf("%w: %s sets a plan output flag; pass -out as a command argument instead", errUtils.ErrPlanVerificationFailed, name) + } + } + return nil +} + +// tokenizeEnvironmentArgs splits a TF_CLI_ARGS value the way the engine does: +// spaces, tabs, and newlines separate tokens outside quotes, and a malformed +// value (for example an unterminated quote) is an error because the engine +// refuses to run with it. The shared parseArgs only splits on spaces, so a tab +// before -out would hide the flag from the check. +func tokenizeEnvironmentArgs(value string) ([]string, error) { + return shlex.Split(value) +} + +// planEnvironment lists the environment layers a plan subprocess receives, in precedence order. +func planEnvironment(atmosConfig *schema.AtmosConfiguration, info *schema.ConfigAndStacksInfo, opts []component.ExecutionOption) []string { + entries := slices.Clone(planBaseEnvironment(info, opts)) + if atmosConfig != nil { + entries = envpkg.MergeGlobalEnv(entries, atmosConfig.Env) + } + entries = append(entries, info.ComponentEnvList...) + if info.ComponentExecution == nil { + return entries + } + for i := range info.ComponentExecution.ExtraArguments { + item := &info.ComponentExecution.ExtraArguments[i] + if !slices.Contains(item.Commands, subcommandPlan) { + continue + } + for name, value := range item.Environment { + entries = append(entries, name+"="+value) + } + } + return entries +} + +// containsOutputFlag reports any plan output flag, with or without a value. +func containsOutputFlag(args []string) bool { + return slices.ContainsFunc(args, func(arg string) bool { + return arg == "-out" || arg == "--out" || strings.HasPrefix(arg, "-out=") || strings.HasPrefix(arg, "--out=") + }) +} + +func plannedOutputPath(directory string, args []string) string { + for i, arg := range args { + path := "" + switch { + case (arg == "-out" || arg == "--out") && i+1 < len(args): + path = args[i+1] + case strings.HasPrefix(arg, "-out="): + path = strings.TrimPrefix(arg, "-out=") + case strings.HasPrefix(arg, "--out="): + path = strings.TrimPrefix(arg, "--out=") + } + if path != "" { + if filepath.IsAbs(path) { + return path + } + return filepath.Join(directory, path) + } + } + return "" +} + +// planSnapshot is the configuration a saved plan is bound to: the provider identity and the +// digest of the retained working directory, excluding the plan and the state Terraform writes. +type planSnapshot struct { + path, identity, workingDigest, sensitiveDigest string + // publicFiles holds the observed non-sensitive generated bytes as canonical JSON. + publicFiles string + // engineDigest is the digest of the Terraform executable observed around the subprocess, so a hook + // that replaces the binary afterwards cannot move the binding onto the replacement. It is empty when + // the executable could not be read, and the binding then falls back to its own observation. + engineDigest string + // planDigest is the digest of the plan file Terraform wrote, taken before any after hook runs. It is + // empty when no digest was observed, and it is not part of the before-and-after comparison. + planDigest string +} + +// planSnapshotKey carries the plan snapshotter to the main command's subprocess. +type planSnapshotKey struct{} + +// planSnapshotter observes the configuration around the Terraform subprocess, after the before +// hooks and before the after hooks. A hook that rewrites the module after Terraform read it then +// cannot move the binding onto the rewritten bytes. +type planSnapshotter struct { + ctx context.Context + config *schema.AtmosConfiguration + info *schema.ConfigAndStacksInfo + execCtx *componentExecContext + pathFrom func(phaseArgs []string) string + result *planSnapshot + err error +} + +// takePlanSnapshot fingerprints the configuration with the inputs the binding record uses. +func takePlanSnapshot(ctx context.Context, config *schema.AtmosConfiguration, info *schema.ConfigAndStacksInfo, execCtx *componentExecContext, path string) (*planSnapshot, error) { + planProvider, err := planRuntime(config) + if err != nil { + return nil, err + } + identity, err := planProvider.PlanIdentity(ctx, info, execCtx.preparedConfiguration.Sources) + if err != nil { + return nil, err + } + digest, err := configurationWorkdirDigest(execCtx.workingDir, path) + if err != nil { + return nil, err + } + sensitive, err := sensitiveDigest(execCtx.workingDir, execCtx.preparedConfiguration.Files) + if err != nil { + return nil, err + } + publicFiles, err := snapshotPublicFiles(execCtx.workingDir, execCtx.preparedConfiguration.Files) + if err != nil { + return nil, err + } + engineDigest, _ := fileDigest(info.Command) + return &planSnapshot{path: path, identity: identity, workingDigest: digest, sensitiveDigest: sensitive, engineDigest: engineDigest, publicFiles: publicFiles}, nil +} + +// around wraps one run of the Terraform subprocess. The configuration must be the same before and +// after it; the unchanged snapshot is kept for the binding, and a changed one rejects the plan. +func (s *planSnapshotter) around(phaseArgs []string, run func() error) func() error { + return func() error { + s.result, s.err = nil, nil + path := s.pathFrom(phaseArgs) + if path == "" { + return run() + } + before, beforeErr := takePlanSnapshot(s.ctx, s.config, s.info, s.execCtx, path) + runErr := run() + if beforeErr != nil { + // The command still completes and its outcome is reported; the binding is what fails. + s.err = beforeErr + return runErr + } + after, err := takePlanSnapshot(s.ctx, s.config, s.info, s.execCtx, path) + switch { + case err != nil: + s.err = err + case *after != *before: + s.err = fmt.Errorf("%w: configuration changed while Terraform ran", errUtils.ErrPlanVerificationFailed) + default: + // Terraform writes the plan itself unless a policy gate publishes it later, so the plan is + // fingerprinted now and an after hook cannot move the binding onto rewritten bytes. + if s.execCtx.planPolicy == nil { + if digest, digestErr := fileDigest(path); digestErr == nil { + after.planDigest = digest + } + } + s.result = after + } + return runErr + } +} + +// reviewedSnapshot returns the configuration Terraform read, or takes a fresh snapshot when no +// observed run recorded one for this plan. +func (c *componentExecContext) reviewedSnapshot(ctx context.Context, config *schema.AtmosConfiguration, info *schema.ConfigAndStacksInfo, path string) (*planSnapshot, error) { + if s := c.planSnapshots; s != nil { + if s.err != nil { + return nil, s.err + } + if s.result != nil && s.result.path == path { + return s.result, nil + } + } + return takePlanSnapshot(ctx, config, info, c, path) +} + +// observePlanSnapshots returns ctx carrying a snapshotter for a plan this invocation binds. Policy +// plans publish the artifact at the original output path, so that path is the one excluded from the +// digest. The returned context is unchanged when no plan is bound. +func (c *componentExecContext) observePlanSnapshots(ctx context.Context, config *schema.AtmosConfiguration, info *schema.ConfigAndStacksInfo, policyArgs []string) context.Context { + c.planSnapshots = nil + if !bindsPlan(info, c) { + return ctx + } + c.planSnapshots = &planSnapshotter{ctx: ctx, config: config, info: info, execCtx: c, pathFrom: func(phaseArgs []string) string { + if c.planPolicy != nil { + phaseArgs = policyArgs + } + return plannedOutputPath(c.componentPath, phaseArgs) + }} + return context.WithValue(ctx, planSnapshotKey{}, c.planSnapshots) +} diff --git a/pkg/component/terraform/configuration_backend_test.go b/pkg/component/terraform/configuration_backend_test.go new file mode 100644 index 00000000000..5554c017728 --- /dev/null +++ b/pkg/component/terraform/configuration_backend_test.go @@ -0,0 +1,176 @@ +package terraform + +import ( + "os" + "path/filepath" + "testing" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + + errUtils "github.com/cloudposse/atmos/errors" + "github.com/cloudposse/atmos/pkg/schema" + "github.com/cloudposse/atmos/pkg/terraform/backend" +) + +func TestPreparedBackendTakeover(t *testing.T) { + for _, name := range []string{"remote.tf", "backend.tf.json"} { + t.Run(name, func(t *testing.T) { + directory := t.TempDir() + config := &schema.AtmosConfiguration{} + config.Components.Terraform.AutoGenerateBackendFile = true + info := &schema.ConfigAndStacksInfo{ComponentBackendType: "local", ComponentSettingsSection: map[string]any{"configuration": map[string]any{}}} + require.NoError(t, backend.GenerateFile(config, info, directory)) + userFile := filepath.Join(directory, "user.tf") + require.NoError(t, os.WriteFile(userFile, []byte("# user-owned"), 0o600)) + prepared := &schema.PreparedComponentConfiguration{BackendGenerated: true, Files: []schema.ComponentConfigurationFile{{Path: name, Content: []byte("# generated backend"), IfExists: "error"}}} + require.NoError(t, writePreparedConfiguration(directory, prepared)) + content, err := os.ReadFile(filepath.Join(directory, name)) + require.NoError(t, err) + assert.Equal(t, "# generated backend", string(content)) + if name != "backend.tf.json" { + assert.NoFileExists(t, filepath.Join(directory, "backend.tf.json")) + } + content, err = os.ReadFile(userFile) + require.NoError(t, err) + assert.Equal(t, "# user-owned", string(content)) + // Subsequent preparation uses provider ownership, with no stale host receipt. + prepared.Files[0].IfExists = "overwrite" + require.NoError(t, writePreparedConfiguration(directory, prepared)) + }) + } +} + +func TestPreparedConfigurationRetainedBackend(t *testing.T) { + const expected = `terraform { + backend "local" { + path = "expected.tfstate" + } +}` + for _, test := range []struct { + name, path, generated, retained, policy string + blocked bool + }{ + {"missing backend", "remote.tf", expected, `locals { enabled = true }`, "skip", true}, + {"different identity", "remote.tf", expected, `terraform { + backend "local" { + path = "other.tfstate" + } +}`, "skip", true}, + {"unexpected backend", "provider.tf", `locals { enabled = true }`, expected, "skip", true}, + {"unexpected cloud", "provider.tf", `locals { enabled = true }`, "terraform {\n cloud {}\n}", "skip", true}, + {"json backend", "remote.tf.json", `{"terraform":{"backend":{"local":{"path":"expected"}}}}`, `{"terraform":{"backend":{"local":{"path":"other"}}}}`, "skip", true}, + {"invalid retained", "remote.tf", expected, `terraform {`, "skip", true}, + {"identical backend", "remote.tf", expected, expected, "skip", false}, + {"non backend", "provider.tf", `locals { enabled = true }`, `locals { enabled = false }`, "skip", false}, + {"auxiliary", "data.json", `{"value":true}`, `{"value":false}`, "skip", false}, + {"overwrite", "remote.tf", expected, `locals { enabled = true }`, "overwrite", false}, + } { + t.Run(test.name, func(t *testing.T) { + directory := t.TempDir() + path := filepath.Join(directory, test.path) + require.NoError(t, os.WriteFile(path, []byte(test.retained), 0o600)) + prepared := &schema.PreparedComponentConfiguration{Files: []schema.ComponentConfigurationFile{{Path: test.path, Content: []byte(test.generated), IfExists: test.policy}}} + err := writePreparedConfiguration(directory, prepared) + if test.blocked { + require.ErrorIs(t, err, errUtils.ErrPluginInvalidConfig) + } else { + require.NoError(t, err) + } + content, readErr := os.ReadFile(path) + require.NoError(t, readErr) + want := test.retained + if test.policy == "overwrite" { + want = test.generated + } + require.Equal(t, want, string(content)) + }) + } +} + +func TestRetainedBackendFailurePreservesHostBackend(t *testing.T) { + directory := t.TempDir() + config := &schema.AtmosConfiguration{} + config.Components.Terraform.AutoGenerateBackendFile = true + info := &schema.ConfigAndStacksInfo{ComponentBackendType: "local", ComponentSettingsSection: map[string]any{"configuration": map[string]any{}}} + require.NoError(t, backend.GenerateFile(config, info, directory)) + hostPath := filepath.Join(directory, "backend.tf.json") + before, err := os.ReadFile(hostPath) + require.NoError(t, err) + require.NoError(t, os.WriteFile(filepath.Join(directory, "remote.tf"), []byte("# user file"), 0o600)) + prepared := &schema.PreparedComponentConfiguration{BackendGenerated: true, Files: []schema.ComponentConfigurationFile{{Path: "remote.tf", Content: []byte("terraform {\n backend \"local\" {}\n}"), IfExists: "skip"}}} + require.ErrorIs(t, writePreparedConfiguration(directory, prepared), errUtils.ErrPluginInvalidConfig) + after, err := os.ReadFile(hostPath) + require.NoError(t, err) + require.Equal(t, before, after) +} + +func TestPreparedConfigurationSkipNewBackend(t *testing.T) { + directory := t.TempDir() + content := []byte("terraform {\n backend \"local\" {}\n}") + prepared := &schema.PreparedComponentConfiguration{BackendGenerated: true, Files: []schema.ComponentConfigurationFile{{Path: "remote.tf", Content: content, IfExists: "skip"}}} + require.NoError(t, writePreparedConfiguration(directory, prepared)) + actual, err := os.ReadFile(filepath.Join(directory, "remote.tf")) + require.NoError(t, err) + require.Equal(t, content, actual) + require.NoError(t, writePreparedConfiguration(directory, prepared)) +} + +func TestPreparedConfigurationSkipSymlink(t *testing.T) { + directory := t.TempDir() + target := filepath.Join(t.TempDir(), "outside.tf") + content := []byte("# user-owned") + require.NoError(t, os.WriteFile(target, content, 0o600)) + if err := os.Symlink(target, filepath.Join(directory, "remote.tf")); err != nil { + t.Skipf("symlink unavailable: %v", err) + } + prepared := &schema.PreparedComponentConfiguration{Files: []schema.ComponentConfigurationFile{{Path: "remote.tf", Content: content, IfExists: "skip"}}} + require.Error(t, writePreparedConfiguration(directory, prepared)) + actual, err := os.ReadFile(target) + require.NoError(t, err) + require.Equal(t, content, actual) +} + +// Failures to inspect retained files or the reviewed working directory are classified with a +// static Atmos error and keep the filesystem cause. +func TestRetainedFileFailuresUseStaticErrors(t *testing.T) { + missing := filepath.Join(t.TempDir(), "absent") + retained := schema.ComponentConfigurationFile{Path: "backend.tf", IfExists: "skip", Content: []byte("terraform {}")} + err := validateRetainedFile(missing, retained) + require.ErrorIs(t, err, errUtils.ErrPluginInvalidConfig) + require.ErrorIs(t, err, os.ErrNotExist) + require.ErrorContains(t, err, "backend.tf") + + err = restoreSensitiveContent(missing, []schema.ComponentConfigurationFile{{Path: "secret.auto.tfvars", Sensitive: true}}, "") + require.ErrorIs(t, err, errUtils.ErrPlanVerificationFailed) + require.ErrorIs(t, err, os.ErrNotExist) +} + +// A sensitive file that its entry removes is absent at apply by design, and restoring it succeeds +// while the digest still binds the reviewed state: a different reviewed digest, a file that +// reappears, or a missing file under any other policy is rejected. +func TestRestoreSensitiveContentAcceptsRemovedFiles(t *testing.T) { + for _, policy := range []string{"remove", "remove_terragrunt"} { + t.Run(policy, func(t *testing.T) { + directory := t.TempDir() + files := []schema.ComponentConfigurationFile{{Path: "secret.auto.tfvars", Sensitive: true, IfExists: policy}} + reviewed, err := sensitiveDigest(directory, files) + require.NoError(t, err) + require.NoError(t, restoreSensitiveContent(directory, files, reviewed)) + require.Nil(t, files[0].Content) + + require.ErrorIs(t, restoreSensitiveContent(directory, files, "different"), errUtils.ErrPlanVerificationFailed) + + require.NoError(t, os.WriteFile(filepath.Join(directory, "secret.auto.tfvars"), []byte("token=1"), 0o600)) + require.ErrorIs(t, restoreSensitiveContent(directory, files, reviewed), errUtils.ErrPlanVerificationFailed, "a file that appeared after review changes the digest") + }) + } + + directory := t.TempDir() + files := []schema.ComponentConfigurationFile{{Path: "secret.auto.tfvars", Sensitive: true, IfExists: "overwrite"}} + reviewed, err := sensitiveDigest(directory, files) + require.NoError(t, err) + err = restoreSensitiveContent(directory, files, reviewed) + require.ErrorIs(t, err, errUtils.ErrPlanVerificationFailed) + require.ErrorIs(t, err, os.ErrNotExist) +} diff --git a/pkg/component/terraform/configuration_overrides.go b/pkg/component/terraform/configuration_overrides.go new file mode 100644 index 00000000000..e0e6baa8ab5 --- /dev/null +++ b/pkg/component/terraform/configuration_overrides.go @@ -0,0 +1,131 @@ +package terraform + +import ( + "fmt" + "maps" + + errUtils "github.com/cloudposse/atmos/errors" + m "github.com/cloudposse/atmos/pkg/merge" + "github.com/cloudposse/atmos/pkg/schema" +) + +// A component owned by a configuration provider (for example a Terragrunt unit imported or +// discovered by the Terragrunt plugin) is configured by two layers. The provider's own evaluation +// is the base; everything the stack manifests say about the component deep-merges over it with +// the usual rules, so any setting can be overridden from stack configuration. From lowest to +// highest precedence, per section: +// +// vars provider inputs < stack vars (imports, catalog, inherits, the component, overrides) +// env provider extra_arguments environment < stack env +// settings stack settings (the provider only marks ownership under settings.configuration) +// metadata stack metadata (terraform_workspace becomes TF_WORKSPACE for the evaluation) +// command stack command selects the binary the provider evaluates and runs +// hooks, providers, auth, dependencies, locals: stack configuration only +// +// Two settings are owned by the provider and cannot be overridden, because other readers depend +// on them being the same everywhere: the state backend (dependency outputs, !terraform.output and +// saved-plan verification all resolve it from the unit) and the module (metadata.component). +// Trying to override them is an error, never a silent ignore. + +// localBackendType is the placeholder backend type the provider gives every discovered component. +const localBackendType = "local" + +// checkStackOverrides rejects stack configuration that tries to change a provider-owned setting. +func checkStackOverrides(info *schema.ConfigAndStacksInfo) error { + if setting, overridden := backendOverride(info); overridden { + return overrideError(info, setting, "the state backend", + "Define the backend in the unit's `remote_state` block (or its `generate \"backend\"` block); the unit's state identity is shared with dependency reads and saved plans.") + } + if component, _ := info.ComponentMetadataSection["component"].(string); component != "" { + return overrideError(info, "metadata.component: "+component, "the Terraform module", + "The module comes from the unit's `terraform { source = ... }`. To use another module, change the unit, or define a regular component that sets `metadata.component`.") + } + return nil +} + +// backendOverride defensively checks unnormalized callers. Stack compilation validates +// component-level backend options before flattening them; flattened local options can +// come from allowed global defaults and must not be rejected here. +func backendOverride(info *schema.ConfigAndStacksInfo) (string, bool) { + if kind, _ := info.ComponentSection["backend_type"].(string); kind != "" && kind != localBackendType { + return "backend_type: " + kind, true + } + backend, _ := info.ComponentSection["backend"].(map[string]any) + if local, _ := backend[localBackendType].(map[string]any); len(local) > 0 { + return "backend." + localBackendType, true + } + return "", false +} + +func overrideError(info *schema.ConfigAndStacksInfo, setting, owned, hint string) error { + return errUtils.Build(fmt.Errorf("%w: %s of component %q in stack %q is owned by the configuration provider, so `%s` cannot be set", errUtils.ErrConfigurationOverrideUnsupported, owned, info.ComponentFromArg, info.Stack, setting)). + WithContext("component", info.ComponentFromArg). + WithContext("stack", info.Stack). + WithContext("setting", setting). + WithHint(hint). + Err() +} + +// overlayStackVars deep-merges the stack's vars over the provider's inputs. The provider +// reevaluates its inputs for every operation, so the result is computed here, per operation, and +// neither side is stored. +func overlayStackVars(config *schema.AtmosConfiguration, inputs, stackVars map[string]any) (map[string]any, error) { + if len(stackVars) == 0 { + return inputs, nil + } + merged, err := m.Merge(config, []map[string]any{inputs, stackVars}) + if err != nil { + return nil, fmt.Errorf("component vars cannot be merged over the provider inputs: %w", err) + } + return merged, nil +} + +// withoutStackEnvironment drops provider environment entries that the stack's env section also +// sets, so the stack wins. The provider's execution data is copied, not modified in place. +func withoutStackEnvironment(execution *schema.ComponentExecution, stackEnv map[string]any) *schema.ComponentExecution { + if execution == nil || len(stackEnv) == 0 { + return execution + } + result := *execution + result.ExtraArguments = make([]schema.ComponentExtraArguments, len(execution.ExtraArguments)) + for i, item := range execution.ExtraArguments { + result.ExtraArguments[i] = item + if len(item.Environment) == 0 { + continue + } + kept := maps.Clone(item.Environment) + for key := range stackEnv { + delete(kept, key) + } + result.ExtraArguments[i].Environment = kept + } + return &result +} + +// componentMergeConfig returns the configuration that governs merging this component's sections. +// A component's own `settings.list_merge_strategy` wins over the global one, as it does when the +// stack manifests are merged. The shared configuration is copied, never modified. +func componentMergeConfig(config *schema.AtmosConfiguration, info *schema.ConfigAndStacksInfo) *schema.AtmosConfiguration { + strategy, _ := info.ComponentSettingsSection["list_merge_strategy"].(string) + if strategy == "" || strategy == config.Settings.ListMergeStrategy { + return config + } + merged := *config + merged.Settings.ListMergeStrategy = strategy + return &merged +} + +// applyStackOverrides layers the stack's vars and env over what the provider prepared. A reviewed +// saved plan already embodies its inputs, so only the environment is layered for it. +func applyStackOverrides(config *schema.AtmosConfiguration, info *schema.ConfigAndStacksInfo, prepared *schema.PreparedComponentConfiguration) error { + prepared.Execution = withoutStackEnvironment(prepared.Execution, info.ComponentEnvSection) + if usesConfigurationPlan(info) { + return nil + } + inputs, err := overlayStackVars(componentMergeConfig(config, info), prepared.Inputs, info.ComponentVarsSection) + if err != nil { + return err + } + prepared.Inputs = inputs + return nil +} diff --git a/pkg/component/terraform/configuration_overrides_test.go b/pkg/component/terraform/configuration_overrides_test.go new file mode 100644 index 00000000000..8a7d1e963ec --- /dev/null +++ b/pkg/component/terraform/configuration_overrides_test.go @@ -0,0 +1,209 @@ +package terraform + +import ( + "os" + "testing" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + "go.uber.org/mock/gomock" + + errUtils "github.com/cloudposse/atmos/errors" + "github.com/cloudposse/atmos/pkg/schema" +) + +func TestOverlayStackVarsDeepMergesStackOverProviderInputs(t *testing.T) { + config := &schema.AtmosConfiguration{} + inputs := map[string]any{"name": "from-hcl", "tags": map[string]any{"team": "hcl", "env": "dev"}, "keep": 1, "dropped": "hcl"} + stack := map[string]any{"name": "from-stack", "tags": map[string]any{"team": "stack", "extra": "x"}, "added": true, "dropped": nil} + + merged, err := overlayStackVars(config, inputs, stack) + + require.NoError(t, err) + assert.Equal(t, map[string]any{ + "name": "from-stack", + "tags": map[string]any{"team": "stack", "env": "dev", "extra": "x"}, + "keep": 1, + "added": true, + "dropped": nil, + }, merged, "maps deep-merge, scalars are replaced, and null replaces the provider input") + + // Result to source: mutating the result leaves both layers alone. + merged["tags"].(map[string]any)["team"] = "mutated" + assert.Equal(t, "hcl", inputs["tags"].(map[string]any)["team"]) + assert.Equal(t, "stack", stack["tags"].(map[string]any)["team"]) + // Source to result: mutating a source after the merge leaves the result alone. + merged, err = overlayStackVars(config, inputs, stack) + require.NoError(t, err) + inputs["tags"].(map[string]any)["env"] = "mutated" + stack["tags"].(map[string]any)["extra"] = "mutated" + assert.Equal(t, map[string]any{"team": "stack", "env": "dev", "extra": "x"}, merged["tags"]) +} + +func TestOverlayStackVarsHonorsTheListMergeStrategy(t *testing.T) { + inputs := map[string]any{"zones": []any{"a", "b"}} + stack := map[string]any{"zones": []any{"c"}} + + replaced, err := overlayStackVars(&schema.AtmosConfiguration{}, inputs, stack) + require.NoError(t, err) + assert.Equal(t, []any{"c"}, replaced["zones"]) + + config := &schema.AtmosConfiguration{} + config.Settings.ListMergeStrategy = "append" + appended, err := overlayStackVars(config, inputs, stack) + require.NoError(t, err) + assert.Equal(t, []any{"a", "b", "c"}, appended["zones"]) + + config.Settings.ListMergeStrategy = "bogus" + _, err = overlayStackVars(config, inputs, stack) + require.Error(t, err) +} + +func TestOverlayStackVarsWithoutStackVarsKeepsProviderInputs(t *testing.T) { + inputs := map[string]any{"name": "from-hcl"} + for _, stack := range []map[string]any{nil, {}} { + merged, err := overlayStackVars(&schema.AtmosConfiguration{}, inputs, stack) + require.NoError(t, err) + assert.Equal(t, inputs, merged) + } +} + +func TestWithoutStackEnvironmentLetsTheStackWin(t *testing.T) { + execution := &schema.ComponentExecution{ExtraArguments: []schema.ComponentExtraArguments{ + {Commands: []string{"plan"}, Environment: map[string]string{"TF_LOG": "TRACE", "FROM_UNIT": "1"}}, + {Commands: []string{"apply"}}, + }} + + result := withoutStackEnvironment(execution, map[string]any{"TF_LOG": "ERROR"}) + + assert.Equal(t, map[string]string{"FROM_UNIT": "1"}, result.ExtraArguments[0].Environment) + assert.Empty(t, result.ExtraArguments[1].Environment) + assert.Equal(t, []string{"plan"}, result.ExtraArguments[0].Commands) + assert.Equal(t, map[string]string{"TF_LOG": "TRACE", "FROM_UNIT": "1"}, execution.ExtraArguments[0].Environment, "the provider's data is not modified") + + assert.Same(t, execution, withoutStackEnvironment(execution, nil), "no stack env, nothing to drop") + assert.Nil(t, withoutStackEnvironment(nil, map[string]any{"A": "b"})) +} + +func TestCheckStackOverrides(t *testing.T) { + owned := func(section map[string]any, metadata map[string]any) *schema.ConfigAndStacksInfo { + return &schema.ConfigAndStacksInfo{ComponentFromArg: "vpc", Stack: "dev", ComponentSection: section, ComponentMetadataSection: metadata} + } + for _, test := range []struct { + name string + info *schema.ConfigAndStacksInfo + wantErr string + }{ + {name: "placeholder only", info: owned(map[string]any{"backend_type": "local", "backend": map[string]any{"local": map[string]any{}}}, nil)}, + {name: "no backend at all", info: owned(map[string]any{}, nil)}, + {name: "global backend is shielded by the placeholder", info: owned(map[string]any{"backend_type": "local", "backend": map[string]any{"local": map[string]any{}, "s3": map[string]any{"bucket": "b"}}}, nil)}, + {name: "backend type", info: owned(map[string]any{"backend_type": "s3"}, nil), wantErr: "backend_type: s3"}, + {name: "backend options", info: owned(map[string]any{"backend_type": "local", "backend": map[string]any{"local": map[string]any{"path": "x"}}}, nil), wantErr: "backend.local"}, + {name: "module", info: owned(map[string]any{}, map[string]any{"component": "other"}), wantErr: "metadata.component: other"}, + {name: "metadata that is not owned", info: owned(map[string]any{}, map[string]any{"enabled": true, "terraform_workspace": "blue", "inherits": []any{"base"}})}, + } { + t.Run(test.name, func(t *testing.T) { + err := checkStackOverrides(test.info) + if test.wantErr == "" { + require.NoError(t, err) + return + } + require.ErrorIs(t, err, errUtils.ErrConfigurationOverrideUnsupported) + require.ErrorContains(t, err, test.wantErr) + require.ErrorContains(t, err, `"vpc"`) + }) + } +} + +func overrideInfo(t *testing.T) *schema.ConfigAndStacksInfo { + t.Helper() + binary, err := os.Executable() + require.NoError(t, err) + return &schema.ConfigAndStacksInfo{ + Command: binary, SubCommand: "plan", Stack: "dev", ComponentFromArg: "vpc", + ComponentSection: map[string]any{}, + ComponentSettingsSection: map[string]any{"configuration": true}, + ComponentVarsSection: map[string]any{"name": "from-stack", "tags": map[string]any{"team": "stack"}}, + ComponentEnvSection: map[string]any{"TF_LOG": "ERROR"}, + } +} + +func TestPrepareConfigurationAppliesStackOverrides(t *testing.T) { + runtime := NewMockComponentConfigurationRuntime(gomock.NewController(t)) + config := &schema.AtmosConfiguration{ComponentConfigurations: runtime} + info := overrideInfo(t) + prepared := &schema.PreparedComponentConfiguration{ + Engine: "terraform", Source: "./module", BackendType: "local", Backend: map[string]any{"path": "state"}, Workspace: "default", + Inputs: map[string]any{"name": "from-hcl", "tags": map[string]any{"team": "hcl", "env": "dev"}, "region": "us-east-1"}, + Execution: &schema.ComponentExecution{ExtraArguments: []schema.ComponentExtraArguments{ + {Commands: []string{"plan"}, Environment: map[string]string{"TF_LOG": "TRACE", "UNIT": "1"}}, + }}, + } + runtime.EXPECT().Prepare(gomock.Any(), info).Return(prepared, nil) + + _, err := prepareConfiguration(t.Context(), config, info) + + require.NoError(t, err) + want := map[string]any{"name": "from-stack", "tags": map[string]any{"team": "stack", "env": "dev"}, "region": "us-east-1"} + assert.Equal(t, want, info.ComponentVarsSection) + assert.Equal(t, want, info.ComponentSection["vars"], "the generated varfile reads the merged component section") + assert.Equal(t, map[string]string{"UNIT": "1"}, info.ComponentExecution.ExtraArguments[0].Environment) +} + +func TestPrepareConfigurationRefusesProviderOwnedOverridesBeforeEvaluating(t *testing.T) { + // The mock has no expectation: evaluating the unit would fail the test. + runtime := NewMockComponentConfigurationRuntime(gomock.NewController(t)) + config := &schema.AtmosConfiguration{ComponentConfigurations: runtime} + info := overrideInfo(t) + info.ComponentSection["backend_type"] = "s3" + + _, err := prepareConfiguration(t.Context(), config, info) + + require.ErrorIs(t, err, errUtils.ErrConfigurationOverrideUnsupported) +} + +func TestApplyStackOverridesKeepsSavedPlanInputs(t *testing.T) { + info := overrideInfo(t) + info.UseTerraformPlan = true + prepared := &schema.PreparedComponentConfiguration{Inputs: map[string]any{}} + + require.NoError(t, applyStackOverrides(&schema.AtmosConfiguration{}, info, prepared)) + + assert.Empty(t, prepared.Inputs, "a reviewed plan embodies its inputs; current stack vars must not leak into the apply") +} + +func TestApplyStackOverridesReportsMergeFailures(t *testing.T) { + config := &schema.AtmosConfiguration{} + config.Settings.ListMergeStrategy = "bogus" + err := applyStackOverrides(config, overrideInfo(t), &schema.PreparedComponentConfiguration{Inputs: map[string]any{"a": 1}}) + require.Error(t, err) +} + +// A component's own settings.list_merge_strategy governs how its stack vars combine with the +// provider's inputs, whatever the global strategy is, and the shared configuration is not modified. +func TestApplyStackOverridesHonorsTheComponentListMergeStrategy(t *testing.T) { + for _, test := range []struct { + name, global, component string + want []any + }{ + {"component append over global replace", "", "append", []any{"a", "b", "c"}}, + {"component replace over global append", "append", "replace", []any{"c"}}, + {"global strategy applies without a component setting", "append", "", []any{"a", "b", "c"}}, + } { + t.Run(test.name, func(t *testing.T) { + config := &schema.AtmosConfiguration{} + config.Settings.ListMergeStrategy = test.global + info := overrideInfo(t) + info.ComponentVarsSection = map[string]any{"zones": []any{"c"}} + if test.component != "" { + info.ComponentSettingsSection["list_merge_strategy"] = test.component + } + prepared := &schema.PreparedComponentConfiguration{Inputs: map[string]any{"zones": []any{"a", "b"}}} + + require.NoError(t, applyStackOverrides(config, info, prepared)) + + assert.Equal(t, test.want, prepared.Inputs["zones"]) + assert.Equal(t, test.global, config.Settings.ListMergeStrategy, "the shared configuration is not modified") + }) + } +} diff --git a/pkg/component/terraform/configuration_plan.go b/pkg/component/terraform/configuration_plan.go index ea9fc439bb8..792f8a8a629 100644 --- a/pkg/component/terraform/configuration_plan.go +++ b/pkg/component/terraform/configuration_plan.go @@ -6,6 +6,7 @@ import ( "crypto/sha256" "encoding/hex" "encoding/json" + "errors" "fmt" "io" "io/fs" @@ -13,29 +14,42 @@ import ( "os" "path/filepath" "reflect" - "runtime" - "slices" "strings" - "github.com/google/shlex" - errUtils "github.com/cloudposse/atmos/errors" - envpkg "github.com/cloudposse/atmos/pkg/env" "github.com/cloudposse/atmos/pkg/schema" ) const configurationPlanSuffix = ".atmos.json" +// configurationPlanVersion is bumped whenever the stored binding changes shape. +const configurationPlanVersion = 3 + // subcommandPlan names the Terraform plan command. const subcommandPlan = "plan" // planBindingSuffix names the binding of a default saved plan, which Atmos writes as "*.planfile". const planBindingSuffix = ".planfile" + configurationPlanSuffix +// configurationPlan is the private record stored beside a saved plan. It binds the plan to the +// exact preparation it was made from. Evaluated inputs and backend settings are not stored: +// the plan file already embodies the inputs, and the backend is re-resolved at apply time and +// compared by BackendDigest, so credentials in backend settings never reach this file. type configurationPlan struct { Version int Stack, Component, Identity, EngineDigest, PlanDigest, WorkingDirectory, WorkingDigest string - Prepared *schema.PreparedComponentConfiguration + BackendDigest string + // SensitiveDigest binds the bytes of every sensitive generated file, including files the working + // directory digest excludes, such as those below a Terragrunt cache. + SensitiveDigest string + Prepared *schema.PreparedComponentConfiguration +} + +// ownsConfigurationPlan reports whether a provider manages this component's saved plans. +// Their plan files are reviewed artifacts bound to a record, not ephemeral scratch files. +func ownsConfigurationPlan(info *schema.ConfigAndStacksInfo) bool { + _, owned := info.ComponentSettingsSection["configuration"] + return owned } func usesConfigurationPlan(info *schema.ConfigAndStacksInfo) bool { @@ -75,18 +89,7 @@ func restoreConfigurationPlan(ctx context.Context, config *schema.AtmosConfigura if err != nil { return nil, err } - identity, err := runtime.PlanIdentity(ctx, info, plan.Prepared.Sources) - if err != nil { - return nil, err - } - artifactMatches, err := configurationArtifactMatches(config, info, plan, path) - if err != nil { - return nil, err - } - if identity != plan.Identity || !artifactMatches { - return nil, fmt.Errorf("%w: plugin, source, engine, workdir, or reviewed artifact changed; create a new plan", errUtils.ErrPlanVerificationFailed) - } - if err := verifyReviewedBackendAndWorkdir(ctx, config, info, plan, path); err != nil { + if err := verifyReviewedPlan(ctx, config, info, runtime, plan); err != nil { return nil, err } // Terraform runs inside the workdir, so a path relative to the project would not resolve there. @@ -94,11 +97,33 @@ func restoreConfigurationPlan(ctx context.Context, config *schema.AtmosConfigura if err != nil { return nil, err } + info.ConfigurationPlanImplicit = info.ConfigurationPlanImplicit || info.PlanFile == "" info.UseTerraformPlan = true info.PlanFile = absolute return plan.Prepared, nil } +// verifyReviewedPlan rejects a plan whose identity, artifact, backend, or workdir changed since review, +// then restores the sensitive content the reviewed plan depends on. +func verifyReviewedPlan(ctx context.Context, config *schema.AtmosConfiguration, info *schema.ConfigAndStacksInfo, runtime schema.ComponentConfigurationPlanRuntime, plan *configurationPlan) error { + path := configurationPlanPath(config, info) + identity, err := runtime.PlanIdentity(ctx, info, plan.Prepared.Sources) + if err != nil { + return err + } + artifactMatches, err := configurationArtifactMatches(config, info, plan, path) + if err != nil { + return err + } + if identity != plan.Identity || !artifactMatches { + return fmt.Errorf("%w: plugin, source, engine, workdir, or reviewed artifact changed; create a new plan", errUtils.ErrPlanVerificationFailed) + } + if err := verifyReviewedBackendAndWorkdir(ctx, config, info, plan, path); err != nil { + return err + } + return restoreSensitiveContent(plan.WorkingDirectory, plan.Prepared.Files, plan.SensitiveDigest) +} + // verifyReviewedBackendAndWorkdir rejects a changed backend identity or a changed retained working directory. func verifyReviewedBackendAndWorkdir(ctx context.Context, config *schema.AtmosConfiguration, info *schema.ConfigAndStacksInfo, plan *configurationPlan, path string) error { if err := verifyConfigurationPlanState(ctx, config, info, plan); err != nil { @@ -114,9 +139,15 @@ func restoresConfigurationPlan(info *schema.ConfigAndStacksInfo) bool { // loadConfigurationPlan reads the companion binding and checks it targets this component. func loadConfigurationPlan(path string, info *schema.ConfigAndStacksInfo) (*configurationPlan, error) { + if err := requireSavedPlan(path, info); err != nil { + return nil, err + } data, err := os.ReadFile(path + configurationPlanSuffix) + if errors.Is(err, fs.ErrNotExist) { + return nil, missingPlanRecordError(path, info) + } if err != nil { - return nil, fmt.Errorf("%w: plugin plan binding is missing: %w", errUtils.ErrPlanVerificationFailed, err) + return nil, fmt.Errorf("%w: plugin plan binding cannot be read: %w", errUtils.ErrPlanVerificationFailed, err) } var plan configurationPlan decoder := json.NewDecoder(strings.NewReader(string(data))) @@ -125,12 +156,39 @@ func loadConfigurationPlan(path string, info *schema.ConfigAndStacksInfo) (*conf if err := decoder.Decode(&plan); err != nil { return nil, errUtils.ErrPlanVerificationFailed } - if plan.Version != 1 || plan.Prepared == nil || plan.Stack != info.Stack || plan.Component != info.ComponentFromArg { + if plan.Version != configurationPlanVersion || plan.Prepared == nil || plan.Stack != info.Stack || plan.Component != info.ComponentFromArg { return nil, errUtils.ErrPlanVerificationFailed } + // The stored record omits inputs; the plan file already carries every value Terraform applies. + plan.Prepared.Inputs = map[string]any{} return &plan, nil } +// requireSavedPlan explains an absent plan file in terms of what the user did, not of file access. +func requireSavedPlan(path string, info *schema.ConfigAndStacksInfo) error { + if _, err := os.Stat(path); err == nil || !errors.Is(err, fs.ErrNotExist) { + return nil + } + if info.PlanFile != "" { + return errUtils.Build(errUtils.ErrSavedPlanNotFound). + WithExplanationf("The plan file `%s` does not exist.", path). + WithHintf("Create it with `atmos terraform plan %s -s %s -- -out=%s`.", info.ComponentFromArg, info.Stack, path). + Err() + } + return errUtils.Build(errUtils.ErrSavedPlanNotFound). + WithExplanationf("Component `%s` in stack `%s` has no saved plan. It was never planned, or its plan was already applied or replaced by a later apply.", info.ComponentFromArg, info.Stack). + WithHintf("Run `atmos terraform plan %s -s %s` first, then apply with `--from-plan`.", info.ComponentFromArg, info.Stack). + Err() +} + +// missingPlanRecordError reports a plan file that Atmos did not create for this component. +func missingPlanRecordError(path string, info *schema.ConfigAndStacksInfo) error { + return errUtils.Build(errUtils.ErrSavedPlanNotFound). + WithExplanationf("The plan file `%s` has no Atmos plan record (`%s%s`). Plugin-managed components apply only plans created by `atmos terraform plan`, because the record proves what was reviewed.", path, filepath.Base(path), configurationPlanSuffix). + WithHintf("Run `atmos terraform plan %s -s %s -- -out=%s` and apply that file.", info.ComponentFromArg, info.Stack, path). + Err() +} + // configurationArtifactMatches reports whether the plan file, engine binary, and working directory path are unchanged. func configurationArtifactMatches(config *schema.AtmosConfiguration, info *schema.ConfigAndStacksInfo, plan *configurationPlan, path string) (bool, error) { digest, err := fileDigest(path) @@ -212,124 +270,44 @@ func newConfigurationPlan(info *schema.ConfigAndStacksInfo, execCtx *componentEx } engineDigest = snapshot.engineDigest } - return &configurationPlan{WorkingDigest: snapshot.workingDigest, Version: 1, Stack: info.Stack, Component: info.ComponentFromArg, Identity: snapshot.identity, EngineDigest: engineDigest, PlanDigest: planDigest, WorkingDirectory: filepath.Clean(execCtx.workingDir), Prepared: execCtx.preparedConfiguration}, nil -} - -// validatePlanOutputFlags rejects ambiguous plan destinations before hooks or Terraform run. -func validatePlanOutputFlags(command string, args []string) error { - if command != "plan" { - return nil + backendDigest, err := backendIdentityDigest(execCtx.preparedConfiguration.Backend) + if err != nil { + return nil, err } - count := 0 - for i := 0; i < len(args); i++ { - arg := args[i] - if arg == "-out" || arg == "--out" { - count++ - i++ - } else if strings.HasPrefix(arg, "-out=") || strings.HasPrefix(arg, "--out=") { - count++ - } - if count > 1 { - return fmt.Errorf("%w: multiple plan output flags", errUtils.ErrPlanVerificationFailed) + // Inputs and backend values stay out of the record; see configurationPlan. Sensitive generated + // files, such as a backend file with credentials, keep only their place: apply reads their bytes + // back from the retained workdir, whose digest proves they are the reviewed ones. + stored := *execCtx.preparedConfiguration + stored.Inputs, stored.Backend = nil, nil + stored.Files = withoutSensitiveContent(stored.Files) + if snapshot.publicFiles != "" { + var observed map[string][]byte + if err := json.Unmarshal([]byte(snapshot.publicFiles), &observed); err != nil { + return nil, err } - } - return nil -} - -// validateEnvironmentPlanOutput rejects plan output flags in TF_CLI_ARGS or -// TF_CLI_ARGS_plan for a configuration-owned plan. Terraform reads these -// variables itself, so a flag there could choose a plan file that Atmos does not -// see and therefore does not bind. The check inspects the effective value of each -// variable in the subprocess environment: the sanitized process environment (or -// the inherited one when auth did not sanitize it), the global atmos.yaml -// environment, the component environment, and provider extra-argument -// environment, with later layers overriding earlier ones. Hooks run as child -// processes and cannot change it. -func validateEnvironmentPlanOutput(atmosConfig *schema.AtmosConfiguration, info *schema.ConfigAndStacksInfo, execCtx *componentExecContext) error { - if execCtx.preparedConfiguration == nil || info.DryRun || info.SubCommand != subcommandPlan { - return nil - } - return rejectEnvironmentPlanOutput(planEnvironment(atmosConfig, info), runtime.GOOS) -} - -// rejectEnvironmentPlanOutput rejects a plan output flag in the effective TF_CLI_ARGS or -// TF_CLI_ARGS_plan of an environment listed in precedence order. Variable names are -// case-insensitive on Windows only, where the last matching entry is the effective value. -func rejectEnvironmentPlanOutput(entries []string, goos string) error { - for _, name := range []string{tfCliArgsEnvVar, tfCliArgsEnvVar + "_" + subcommandPlan} { - value, _ := lastEnvListValue(entries, name, goos) - args, err := tokenizeEnvironmentArgs(value) - if err != nil { - return fmt.Errorf("%w: %s cannot be parsed: %w", errUtils.ErrPlanVerificationFailed, name, err) - } - if containsOutputFlag(args) { - return fmt.Errorf("%w: %s sets a plan output flag; pass -out as a command argument instead", errUtils.ErrPlanVerificationFailed, name) + for i := range stored.Files { + if content, ok := observed[stored.Files[i].Path]; ok && !stored.Files[i].Sensitive { + stored.Files[i].Content = content + } } } - return nil -} - -// tokenizeEnvironmentArgs splits a TF_CLI_ARGS value the way the engine does: -// spaces, tabs, and newlines separate tokens outside quotes, and a malformed -// value (for example an unterminated quote) is an error because the engine -// refuses to run with it. The shared parseArgs only splits on spaces, so a tab -// before -out would hide the flag from the check. -func tokenizeEnvironmentArgs(value string) ([]string, error) { - return shlex.Split(value) + return &configurationPlan{WorkingDigest: snapshot.workingDigest, SensitiveDigest: snapshot.sensitiveDigest, Version: configurationPlanVersion, Stack: info.Stack, Component: info.ComponentFromArg, Identity: snapshot.identity, EngineDigest: engineDigest, PlanDigest: planDigest, WorkingDirectory: filepath.Clean(execCtx.workingDir), BackendDigest: backendDigest, Prepared: &stored}, nil } -// planEnvironment lists the environment layers a plan subprocess receives, in precedence order. -func planEnvironment(atmosConfig *schema.AtmosConfiguration, info *schema.ConfigAndStacksInfo) []string { - base := info.SanitizedEnv - if base == nil { - base = os.Environ() - } - entries := slices.Clone(base) - if atmosConfig != nil { - entries = envpkg.MergeGlobalEnv(entries, atmosConfig.Env) - } - entries = append(entries, info.ComponentEnvList...) - if info.ComponentExecution == nil { - return entries - } - for i := range info.ComponentExecution.ExtraArguments { - item := &info.ComponentExecution.ExtraArguments[i] - if !slices.Contains(item.Commands, subcommandPlan) { - continue - } - for name, value := range item.Environment { - entries = append(entries, name+"="+value) - } +// backendIdentityDigest hashes the canonical JSON form of backend settings, which only ever need equality. +func backendIdentityDigest(backend map[string]any) (string, error) { + // Backend values may cross JSON/protobuf boundaries as different Go number + // types. Hash their exact normalized values so equivalent backends match. + normalized, err := normalizedBackend(backend) + if err != nil { + return "", err } - return entries -} - -// containsOutputFlag reports any plan output flag, with or without a value. -func containsOutputFlag(args []string) bool { - return slices.ContainsFunc(args, func(arg string) bool { - return arg == "-out" || arg == "--out" || strings.HasPrefix(arg, "-out=") || strings.HasPrefix(arg, "--out=") - }) -} - -func plannedOutputPath(directory string, args []string) string { - for i, arg := range args { - path := "" - switch { - case (arg == "-out" || arg == "--out") && i+1 < len(args): - path = args[i+1] - case strings.HasPrefix(arg, "-out="): - path = strings.TrimPrefix(arg, "-out=") - case strings.HasPrefix(arg, "--out="): - path = strings.TrimPrefix(arg, "--out=") - } - if path != "" { - if filepath.IsAbs(path) { - return path - } - return filepath.Join(directory, path) - } + data, err := json.Marshal(normalized) + if err != nil { + return "", err } - return "" + digest := sha256.Sum256(data) + return hex.EncodeToString(digest[:]), nil } func fileDigest(path string) (string, error) { @@ -379,9 +357,15 @@ func verifyConfigurationPlanState(ctx context.Context, config *schema.AtmosConfi if err != nil { return err } - if state == nil || state.BackendType != plan.Prepared.BackendType || state.Workspace != plan.Prepared.Workspace || !equalBackendConfiguration(state.Backend, plan.Prepared.Backend) { + if state == nil || state.BackendType != plan.Prepared.BackendType || state.Workspace != plan.Prepared.Workspace { + return fmt.Errorf("%w: reviewed backend identity changed", errUtils.ErrPlanVerificationFailed) + } + digest, err := backendIdentityDigest(state.Backend) + if err != nil || digest != plan.BackendDigest { return fmt.Errorf("%w: reviewed backend identity changed", errUtils.ErrPlanVerificationFailed) } + // The verified current backend stands in for the one the record deliberately does not store. + plan.Prepared.Backend = state.Backend return nil } @@ -398,14 +382,11 @@ func excludedWorkdirFile(root, path, planPath, name string) bool { // Root and a non-empty planPath are resolved to absolute paths first, so a relative root never // fails to exclude the plan file; an empty planPath stays empty and excludes nothing. func configurationWorkdirDigest(root, planPath string) (string, error) { - root, err := filepath.Abs(root) + // An explicit plan path may be absolute while the directory is relative, or the reverse. + // Both must compare equal, or the plan file itself would be hashed into the directory digest. + root, planPath, err := absolutePaths(root, planPath) if err != nil { - return "", planIOError(err) - } - if planPath != "" { - if planPath, err = filepath.Abs(planPath); err != nil { - return "", planIOError(err) - } + return "", err } files := make(map[string]string) err = filepath.WalkDir(root, func(path string, entry fs.DirEntry, err error) error { @@ -434,6 +415,31 @@ func configurationWorkdirDigest(root, planPath string) (string, error) { return hex.EncodeToString(digest[:]), nil } +// absolutePaths returns both paths absolute and with symlinks resolved, so one file reached through +// two spellings (for example /var and /private/var on macOS) compares equal. +func absolutePaths(root, planPath string) (string, string, error) { + absoluteRoot, err := resolvedPath(root) + if err != nil { + return "", "", err + } + absolutePlan, err := resolvedPath(planPath) + if err != nil { + return "", "", err + } + return absoluteRoot, absolutePlan, nil +} + +func resolvedPath(path string) (string, error) { + absolute, err := filepath.Abs(path) + if err != nil { + return "", err + } + if resolved, err := filepath.EvalSymlinks(absolute); err == nil { + return resolved, nil + } + return absolute, nil +} + // skipWorkdirDirectory reports whether a nested directory holds only mutable or cached data. // The .atmos directory is provisioner bookkeeping (access timestamps and metadata lock files // that even a read-only lookup of the retained directory creates), never executed content. The @@ -505,6 +511,36 @@ func recordWorkdirFile(root, path string, entry fs.DirEntry, files map[string]st return nil } +// stateChangingSubcommands make an earlier saved plan stale once they succeed. +var stateChangingSubcommands = map[string]struct{}{"apply": {}, "deploy": {}, "destroy": {}} + +// finishConfigurationPlan removes a saved plan and its record after a successful command made it +// unusable: the plan was applied, or a direct apply changed the state it was made against. Any +// other command leaves the reviewed plan alone, so plan, output, apply --from-plan keeps working. +// An explicit plan path is the user's file, so only its Atmos record is removed. +func finishConfigurationPlan(config *schema.AtmosConfiguration, info *schema.ConfigAndStacksInfo) { + if !ownsConfigurationPlan(info) || info.DryRun { + return + } + // A CI exit-code mapping can report a failed apply as success. The subprocess's own status + // decides, so a plan whose apply failed (for example on a state lock) can be applied again. + if info.ExecMetadataRawExitCode != 0 { + return + } + if _, changes := stateChangingSubcommands[info.SubCommand]; !changes { + return + } + canonical := ConstructTerraformComponentPlanfilePath(config, info) + // A plan file the user named is theirs even when it is the canonical path, so it is kept. + if info.PlanFile == "" || info.ConfigurationPlanImplicit || filepath.Clean(configurationPlanPath(config, info)) != filepath.Clean(canonical) { + removeTerraformFile(canonical, "Failed to remove consumed plan file") + } + removeTerraformFile(canonical+configurationPlanSuffix, "Failed to remove consumed plan record") + if info.PlanFile != "" { + removeTerraformFile(configurationPlanPath(config, info)+configurationPlanSuffix, "Failed to remove consumed plan record") + } +} + // equalBackendConfiguration compares JSON backend values without losing numeric precision. func equalBackendConfiguration(left, right map[string]any) bool { a, errA := normalizedBackend(left) diff --git a/pkg/component/terraform/configuration_plan_files.go b/pkg/component/terraform/configuration_plan_files.go new file mode 100644 index 00000000000..bec146f1efb --- /dev/null +++ b/pkg/component/terraform/configuration_plan_files.go @@ -0,0 +1,35 @@ +package terraform + +import ( + "encoding/json" + "errors" + "os" + + "github.com/cloudposse/atmos/pkg/schema" +) + +// snapshotPublicFiles preserves retained generated bytes after before-hooks have run. +// Missing files retain their preparation semantics; sensitive bytes never enter this record. +func snapshotPublicFiles(workdir string, files []schema.ComponentConfigurationFile) (string, error) { + root, err := os.OpenRoot(workdir) + if err != nil { + return "", planIOError(err) + } + defer root.Close() + contents := make(map[string][]byte) + for _, file := range files { + if file.Sensitive || file.IfExists == "remove" || file.IfExists == "remove_terragrunt" { + continue + } + data, err := root.ReadFile(file.Path) + if errors.Is(err, os.ErrNotExist) { + continue + } + if err != nil { + return "", planIOError(err) + } + contents[file.Path] = data + } + data, err := json.Marshal(contents) + return string(data), err +} diff --git a/pkg/component/terraform/configuration_plan_pipeline_test.go b/pkg/component/terraform/configuration_plan_pipeline_test.go index 5fcd3deb08e..44581afe9a6 100644 --- a/pkg/component/terraform/configuration_plan_pipeline_test.go +++ b/pkg/component/terraform/configuration_plan_pipeline_test.go @@ -93,7 +93,7 @@ func TestApplyOfReviewedPlanReusesReviewedPreparationAndWorkdir(t *testing.T) { require.Equal(t, project.workdir, execution.componentPath, "a reviewed plan runs in the retained directory") require.Equal(t, execution.workingDir, execution.componentPath) - require.Equal(t, map[string]any{"value": "reviewed"}, execution.preparedConfiguration.Inputs) + require.Empty(t, execution.preparedConfiguration.Inputs, "the record stores no inputs; the plan file carries them") retained, err := os.ReadFile(filepath.Join(project.workdir, "main.tf")) require.NoError(t, err) require.Equal(t, "# reviewed module", string(retained)) @@ -112,6 +112,39 @@ func TestApplyOfReviewedPlanReusesReviewedPreparationAndWorkdir(t *testing.T) { require.Contains(t, strings.Split(string(args), "\n"), planPath) } +func TestSavedPlanApplySkipsSecondComponentValidation(t *testing.T) { + project := newReviewedPlanProject(t) + executor := testExecutor(t) + validations := 0 + executor.services.ValidateComponent = func(_ *schema.AtmosConfiguration, _ string, section map[string]any, _, _ string, _ []string, _ int) (bool, error) { + validations++ + vars, _ := section["vars"].(map[string]any) + return vars["value"] == "reviewed", nil + } + planPath := project.review(t) + require.Equal(t, 1, validations, "plan creation validates the component with its values") + + apply := project.reviewedApply(t, planPath) + _, err := executor.prepareComponentExecution(t.Context(), provisioner.OutputWriters{}, project.config, apply, false) + require.NoError(t, err, "a verified saved plan must not be rejected for its intentionally empty vars") + require.Equal(t, 1, validations, "saved-plan apply must not revalidate the redacted component") +} + +func TestDirectPluginApplyStillValidatesComponent(t *testing.T) { + project := newReviewedPlanProject(t) + apply := project.info(t, "apply") + project.provider.EXPECT().Prepare(gomock.Any(), apply).Return(project.prepared(), nil) + executor := testExecutor(t) + validations := 0 + executor.services.ValidateComponent = func(_ *schema.AtmosConfiguration, _ string, _ map[string]any, _, _ string, _ []string, _ int) (bool, error) { + validations++ + return false, nil + } + _, err := executor.prepareComponentExecution(t.Context(), provisioner.OutputWriters{}, project.config, apply, false) + require.ErrorIs(t, err, errUtils.ErrInvalidComponent) + require.Equal(t, 1, validations, "a direct apply has no reviewed plan to replace validation") +} + // Terragrunt evaluates the configuration again for apply, so a hook enabled only for apply is // absent from the plan-time preparation. Applying the reviewed plan runs the apply-time hooks // recorded with it, and not the plan-time ones. @@ -193,7 +226,9 @@ func TestPrepareConfigurationRestoresReviewedPlanWithoutPreparing(t *testing.T) prepared, err := prepareConfiguration(t.Context(), project.config, apply) require.NoError(t, err) - require.Equal(t, map[string]any{"value": "reviewed"}, apply.ComponentVarsSection) + // The record keeps no inputs: the plan file carries the values Terraform applies. + require.Empty(t, apply.ComponentVarsSection) + require.NotNil(t, apply.ComponentVarsSection) require.Equal(t, prepared.Inputs, apply.ComponentSection["vars"]) // A tampered plan is rejected before any of the reviewed state is applied to the invocation. diff --git a/pkg/component/terraform/configuration_plan_test.go b/pkg/component/terraform/configuration_plan_test.go index 3417c5cef8d..b3e8494ce31 100644 --- a/pkg/component/terraform/configuration_plan_test.go +++ b/pkg/component/terraform/configuration_plan_test.go @@ -6,12 +6,14 @@ import ( "errors" "fmt" "io" + "io/fs" "os" "path/filepath" "runtime" "testing" errUtils "github.com/cloudposse/atmos/errors" + "github.com/cloudposse/atmos/pkg/component" "github.com/cloudposse/atmos/pkg/policy" "github.com/cloudposse/atmos/pkg/policy/planfile" "github.com/cloudposse/atmos/pkg/process" @@ -93,7 +95,7 @@ func planFixture(t *testing.T) (*schema.AtmosConfiguration, *schema.ConfigAndSta return config, info, execCtx, runtime, path } -func TestConfigurationPlanRestoresReviewedInputs(t *testing.T) { +func TestConfigurationPlanRestoresReviewedPreparation(t *testing.T) { config, info, execCtx, _, path := planFixture(t) err := recordConfigurationPlan(t.Context(), config, info, execCtx, []string{"plan", "-out", filepath.Base(path)}) require.NoError(t, err) @@ -108,7 +110,8 @@ func TestConfigurationPlanRestoresReviewedInputs(t *testing.T) { info.PlanFile = path prepared, err := restoreConfigurationPlan(t.Context(), config, info) require.NoError(t, err) - require.Equal(t, execCtx.preparedConfiguration.Inputs, prepared.Inputs) + require.Empty(t, prepared.Inputs, "the record stores no inputs") + require.Equal(t, execCtx.preparedConfiguration.Backend, prepared.Backend, "the verified current backend replaces the unstored one") require.True(t, info.UseTerraformPlan) require.Equal(t, path, info.PlanFile) require.FileExists(t, path) @@ -230,6 +233,97 @@ func TestConfigurationWorkdirDigestIgnoresMutableFiles(t *testing.T) { require.True(t, errors.Is(err, errUtils.ErrPluginUnsupportedExtension)) } +// The binding sits beside the plan in a directory that CI caches and uploads, so it must not +// carry evaluated input values, hook command lines are the only execution data it keeps, and +// backend settings (which can hold credentials) are reduced to a digest. +func TestConfigurationPlanRecordHoldsNoInputsOrBackendValues(t *testing.T) { + config, info, execCtx, runtime, path := planFixture(t) + execCtx.preparedConfiguration.Inputs = map[string]any{"password": "input-secret-value"} + execCtx.preparedConfiguration.Backend = map[string]any{"access_key": "backend-secret-value"} + runtime.state.Backend = execCtx.preparedConfiguration.Backend + require.NoError(t, recordConfigurationPlan(t.Context(), config, info, execCtx, []string{"-out=" + path})) + + data, err := os.ReadFile(path + configurationPlanSuffix) + require.NoError(t, err) + require.NotContains(t, string(data), "input-secret-value") + require.NotContains(t, string(data), "backend-secret-value") + require.Contains(t, string(data), "BackendDigest") + // The caller's preparation is untouched, so the rest of the plan command still sees its inputs. + require.Equal(t, "input-secret-value", execCtx.preparedConfiguration.Inputs["password"]) + require.Equal(t, "backend-secret-value", execCtx.preparedConfiguration.Backend["access_key"]) + + info.SubCommand, info.UseTerraformPlan, info.PlanFile = "apply", true, path + prepared, err := restoreConfigurationPlan(t.Context(), config, info) + require.NoError(t, err) + require.Equal(t, "backend-secret-value", prepared.Backend["access_key"]) +} + +// Generated files can hold credentials; apply restores them from the verified workdir. +func TestConfigurationPlanRecordHoldsNoSensitiveGeneratedFiles(t *testing.T) { + for _, policy := range []string{"overwrite", "overwrite_terragrunt", "skip", "error"} { + t.Run(policy, func(t *testing.T) { + config, info, execCtx, _, path := planFixture(t) + backend := `terraform { backend "s3" { access_key = "generated-backend-secret" } }` + provider := `provider "aws" { secret_key = "generated-provider-secret" }` + execCtx.preparedConfiguration.Files = []schema.ComponentConfigurationFile{ + {Path: "backend.tf", Content: []byte(backend), IfExists: policy, Sensitive: true}, + {Path: "provider.tf", Content: []byte(provider), IfExists: policy, Sensitive: true}, + {Path: "public.txt", Content: []byte("public-content"), IfExists: "overwrite"}, + {Path: "obsolete.tf", IfExists: "remove"}, + } + for _, file := range execCtx.preparedConfiguration.Files[:2] { + require.NoError(t, os.WriteFile(filepath.Join(execCtx.workingDir, file.Path), file.Content, 0o600)) + } + require.NoError(t, recordConfigurationPlan(t.Context(), config, info, execCtx, []string{"-out=" + path})) + data, err := os.ReadFile(path + configurationPlanSuffix) + require.NoError(t, err) + var record configurationPlan + require.NoError(t, json.Unmarshal(data, &record)) + require.Len(t, record.Prepared.Files, 4) + for i, file := range execCtx.preparedConfiguration.Files[:2] { + require.Empty(t, record.Prepared.Files[i].Content, "the decoded record holds no sensitive bytes") + require.NotEmpty(t, file.Content, "the caller's preparation is untouched") + require.Equal(t, policy, record.Prepared.Files[i].IfExists) + } + require.Equal(t, []byte("public-content"), record.Prepared.Files[2].Content) + info.SubCommand, info.UseTerraformPlan, info.PlanFile = "apply", true, path + prepared, err := restoreConfigurationPlan(t.Context(), config, info) + require.NoError(t, err) + require.Equal(t, []byte(backend), prepared.Files[0].Content) + require.Equal(t, []byte(provider), prepared.Files[1].Content) + require.Equal(t, []byte("public-content"), prepared.Files[2].Content) + require.Empty(t, prepared.Files[3].Content) + }) + } +} + +// The working directory digest skips cache directories, so a sensitive generated file there is +// verified against its own recorded digest before apply restores it. +func TestConfigurationPlanRejectsChangedSensitiveFileOutsideTheWorkdirDigest(t *testing.T) { + config, info, execCtx, _, path := planFixture(t) + relative := filepath.Join(".terragrunt-cache", "unit", "backend.tf") + reviewed := []byte(`terraform { backend "s3" { access_key = "reviewed-secret" } }`) + execCtx.preparedConfiguration.Files = []schema.ComponentConfigurationFile{{Path: filepath.ToSlash(relative), Content: reviewed, IfExists: "overwrite", Sensitive: true}} + require.NoError(t, os.MkdirAll(filepath.Dir(filepath.Join(execCtx.workingDir, relative)), 0o755)) + require.NoError(t, os.WriteFile(filepath.Join(execCtx.workingDir, relative), reviewed, 0o600)) + require.NoError(t, recordConfigurationPlan(t.Context(), config, info, execCtx, []string{"-out=" + path})) + info.SubCommand, info.UseTerraformPlan, info.PlanFile = "apply", true, path + + prepared, err := restoreConfigurationPlan(t.Context(), config, info) + require.NoError(t, err, "the reviewed bytes are restored") + require.Equal(t, reviewed, prepared.Files[0].Content) + + require.NoError(t, os.WriteFile(filepath.Join(execCtx.workingDir, relative), []byte(`terraform { backend "s3" { access_key = "tampered" } }`), 0o600)) + _, err = restoreConfigurationPlan(t.Context(), config, info) + require.ErrorIs(t, err, errUtils.ErrPlanVerificationFailed) + require.ErrorContains(t, err, "reviewed generated file changed") + + require.NoError(t, os.Remove(filepath.Join(execCtx.workingDir, relative))) + _, err = restoreConfigurationPlan(t.Context(), config, info) + require.ErrorIs(t, err, errUtils.ErrPlanVerificationFailed) + require.ErrorIs(t, err, fs.ErrNotExist) +} + func TestConfigurationWorkdirDigestIncludesDownloadedModules(t *testing.T) { root := t.TempDir() module := filepath.Join(root, ".terraform", "modules", "child", "script.sh") @@ -281,6 +375,208 @@ func TestPlanOutputFlagsUseEffectiveArguments(t *testing.T) { } } +// --from-plan= names the same file as the canonical location, so it must verify the same +// way even when the working directory is spelled relative and the path absolute. +func TestConfigurationPlanExplicitAbsolutePathVerifiesLikeCanonicalPath(t *testing.T) { + config, info, execCtx, _, _ := planFixture(t) + t.Chdir(config.BasePath) + config.BasePath = "." + relativeWorkdir := ConstructTerraformComponentWorkingDir(config, info) + require.False(t, filepath.IsAbs(relativeWorkdir)) + execCtx.workingDir, execCtx.componentPath = relativeWorkdir, relativeWorkdir + canonical := filepath.Join(relativeWorkdir, ConstructTerraformComponentPlanfileName(info)) + require.NoError(t, os.WriteFile(canonical, []byte("reviewed plan"), 0o600)) + require.NoError(t, recordConfigurationPlan(t.Context(), config, info, execCtx, []string{"-out=" + filepath.Base(canonical)})) + + absolute, err := filepath.Abs(canonical) + require.NoError(t, err) + for name, planFile := range map[string]string{"canonical": "", "absolute": absolute, "relative": filepath.Base(canonical)} { + t.Run(name, func(t *testing.T) { + apply := *info + apply.SubCommand, apply.UseTerraformPlan, apply.PlanFile = "apply", true, planFile + _, err := restoreConfigurationPlan(t.Context(), config, &apply) + require.NoError(t, err) + }) + } +} + +func TestConfigurationPlanMissingPlanExplainsWhatHappened(t *testing.T) { + t.Run("never planned", func(t *testing.T) { + config, info, _, _, path := planFixture(t) + require.NoError(t, os.Remove(path)) + info.SubCommand, info.UseTerraformPlan = "apply", true + _, err := restoreConfigurationPlan(t.Context(), config, info) + require.ErrorIs(t, err, errUtils.ErrSavedPlanNotFound) + require.NotErrorIs(t, err, errUtils.ErrPlanVerificationFailed) + require.Contains(t, errUtils.Format(err, errUtils.FormatterConfig{}), "no saved plan") + }) + t.Run("explicit path that does not exist", func(t *testing.T) { + config, info, _, _, path := planFixture(t) + require.NoError(t, os.Remove(path)) + info.SubCommand, info.UseTerraformPlan, info.PlanFile = "apply", true, path + _, err := restoreConfigurationPlan(t.Context(), config, info) + require.ErrorIs(t, err, errUtils.ErrSavedPlanNotFound) + }) + t.Run("plan file without an Atmos record", func(t *testing.T) { + config, info, execCtx, _, path := planFixture(t) + require.NoError(t, recordConfigurationPlan(t.Context(), config, info, execCtx, []string{"-out=" + path})) + require.NoError(t, os.Remove(path+configurationPlanSuffix)) + info.SubCommand, info.UseTerraformPlan, info.PlanFile = "apply", true, path + _, err := restoreConfigurationPlan(t.Context(), config, info) + require.ErrorIs(t, err, errUtils.ErrSavedPlanNotFound) + }) +} + +func TestConfigurationPlanRestoreThenFinish(t *testing.T) { + for _, subcommand := range []string{"apply", "deploy"} { + for _, selection := range []string{"implicit", "explicit absolute", "explicit relative"} { + t.Run(subcommand+"/"+selection, func(t *testing.T) { + config, info, execCtx, _, _ := planFixture(t) + canonical := ConstructTerraformComponentPlanfilePath(config, info) + require.NoError(t, os.WriteFile(canonical, []byte("reviewed plan"), 0o600)) + require.NoError(t, recordConfigurationPlan(t.Context(), config, info, execCtx, []string{"-out=" + canonical})) + info.SubCommand, info.UseTerraformPlan = subcommand, true + switch selection { + case "explicit absolute": + info.PlanFile = canonical + case "explicit relative": + info.PlanFile = filepath.Base(canonical) + } + _, err := restoreConfigurationPlan(t.Context(), config, info) + require.NoError(t, err) + require.Equal(t, canonical, info.PlanFile) + // Failed applies and dry runs preserve both files for another attempt. + info.ExecMetadataRawExitCode = 1 + finishConfigurationPlan(config, info) + require.FileExists(t, canonical) + require.FileExists(t, canonical+configurationPlanSuffix) + info.ExecMetadataRawExitCode, info.DryRun = 0, true + finishConfigurationPlan(config, info) + require.FileExists(t, canonical) + require.FileExists(t, canonical+configurationPlanSuffix) + info.DryRun = false + finishConfigurationPlan(config, info) + if selection == "implicit" { + require.NoFileExists(t, canonical) + } else { + require.FileExists(t, canonical) + } + require.NoFileExists(t, canonical+configurationPlanSuffix) + }) + } + } +} + +func TestFinishConfigurationPlan(t *testing.T) { + setup := func(t *testing.T, subcommand string) (*schema.AtmosConfiguration, *schema.ConfigAndStacksInfo, string) { + config, info, execCtx, _, _ := planFixture(t) + canonical := ConstructTerraformComponentPlanfilePath(config, info) + require.NoError(t, os.WriteFile(canonical, []byte("reviewed plan"), 0o600)) + require.NoError(t, recordConfigurationPlan(t.Context(), config, info, execCtx, []string{"-out=" + canonical})) + info.SubCommand = subcommand + return config, info, canonical + } + for _, subcommand := range []string{"apply", "deploy", "destroy"} { + t.Run(subcommand+" removes the consumed plan and its record", func(t *testing.T) { + config, info, canonical := setup(t, subcommand) + finishConfigurationPlan(config, info) + require.NoFileExists(t, canonical) + require.NoFileExists(t, canonical+configurationPlanSuffix) + }) + } + for _, subcommand := range []string{"plan", "output", "validate", "init", "show", "refresh", "state list"} { + t.Run(subcommand+" keeps the reviewed plan", func(t *testing.T) { + config, info, canonical := setup(t, subcommand) + finishConfigurationPlan(config, info) + require.FileExists(t, canonical) + require.FileExists(t, canonical+configurationPlanSuffix) + }) + } + t.Run("an explicit plan path keeps the user's file and drops its record", func(t *testing.T) { + config, info, canonical := setup(t, "apply") + explicit := filepath.Join(filepath.Dir(canonical), "explicit.planfile") + require.NoError(t, os.WriteFile(explicit, []byte("explicit"), 0o600)) + require.NoError(t, os.WriteFile(explicit+configurationPlanSuffix, []byte("{}"), 0o600)) + info.PlanFile = explicit + finishConfigurationPlan(config, info) + require.FileExists(t, explicit) + require.NoFileExists(t, explicit+configurationPlanSuffix) + }) + t.Run("an explicit plan path that names the canonical file keeps it and drops its record", func(t *testing.T) { + config, info, canonical := setup(t, "apply") + info.PlanFile = canonical + finishConfigurationPlan(config, info) + require.FileExists(t, canonical) + require.NoFileExists(t, canonical+configurationPlanSuffix) + }) + t.Run("a failed apply keeps the plan and its record even when its exit code is mapped to success", func(t *testing.T) { + config, info, canonical := setup(t, "apply") + info.ExecMetadataRawExitCode = 1 + finishConfigurationPlan(config, info) + require.FileExists(t, canonical) + require.FileExists(t, canonical+configurationPlanSuffix) + }) + t.Run("a dry run and an ordinary component keep every file", func(t *testing.T) { + config, info, canonical := setup(t, "apply") + info.DryRun = true + finishConfigurationPlan(config, info) + info.DryRun = false + info.ComponentSettingsSection = map[string]any{} + finishConfigurationPlan(config, info) + require.FileExists(t, canonical) + require.FileExists(t, canonical+configurationPlanSuffix) + }) +} + +// Inspection commands keep the saved plan of a plugin-managed and an ordinary component alike, +// while a completed destroy consumes it for both. +func TestCleanupTerraformFilesKeepsReviewedPlanForOwnedComponents(t *testing.T) { + for _, tc := range []struct { + subcommand string + owned bool + kept bool + }{ + {"destroy", true, false}, + {"destroy", false, false}, + {"output", true, true}, + {"output", false, true}, + } { + config, info, _, _, _ := planFixture(t) + if !tc.owned { + info.ComponentSettingsSection = map[string]any{} + } + canonical := ConstructTerraformComponentPlanfilePath(config, info) + require.NoError(t, os.WriteFile(canonical, []byte("plan"), 0o600)) + info.SubCommand = tc.subcommand + cleanupTerraformFiles(config, info) + if tc.kept { + require.FileExists(t, canonical, "%s owned=%t", tc.subcommand, tc.owned) + } else { + require.NoFileExists(t, canonical, "%s owned=%t", tc.subcommand, tc.owned) + } + } +} + +// The same directory reached through a symlink (/var and /private/var on macOS) must not make the +// plan file look like part of the reviewed working directory. +func TestConfigurationWorkdirDigestIgnoresPlanReachedThroughSymlink(t *testing.T) { + root := t.TempDir() + workdir := filepath.Join(root, "real") + require.NoError(t, os.Mkdir(workdir, 0o755)) + require.NoError(t, os.WriteFile(filepath.Join(workdir, "main.tf"), []byte("module"), 0o600)) + plan := filepath.Join(workdir, "reviewed.planfile") + require.NoError(t, os.WriteFile(plan, []byte("plan"), 0o600)) + link := filepath.Join(root, "link") + if err := os.Symlink(workdir, link); err != nil { + t.Skipf("symlink unavailable: %v", err) + } + direct, err := configurationWorkdirDigest(workdir, plan) + require.NoError(t, err) + viaLink, err := configurationWorkdirDigest(workdir, filepath.Join(link, "reviewed.planfile")) + require.NoError(t, err) + require.Equal(t, direct, viaLink) +} + func TestRecoveredPlanBindsEffectiveOutput(t *testing.T) { config, info, execCtx, _, _ := planFixture(t) config.Components.Terraform.Init.Reconfigure = schema.TerraformInitReconfigureAuto @@ -450,6 +746,86 @@ func TestPolicyPlanBindsPublishedOutput(t *testing.T) { require.FileExists(t, path+configurationPlanSuffix) } +func TestDetailedExitCodePlanBinding(t *testing.T) { + for _, test := range []struct { + name string + hooks, recoverInit, hookFailure, bindingFailure, noFlag bool + code int + }{ + {name: "plain plan", code: 2}, + {name: "hooks complete", hooks: true, code: 2}, + {name: "init recovery", hooks: true, recoverInit: true, code: 2}, + {name: "after hook fails with status two", hooks: true, hookFailure: true, code: 2}, + {name: "binding fails", hooks: true, bindingFailure: true, code: 2}, + {name: "plan fails", hooks: true, code: 1}, + {name: "status two without flag", hooks: true, noFlag: true, code: 2}, + } { + t.Run(test.name, func(t *testing.T) { + config, info, execCtx, runtime, path := planFixture(t) + config.Components.Terraform.Init.Reconfigure = schema.TerraformInitReconfigureAuto + args := []string{"plan", "-out=" + path} + if !test.noFlag { + args = append(args, "-detailed-exitcode") + } + if test.hooks { + info.ComponentExecution = &schema.ComponentExecution{AfterHooks: []schema.ComponentHook{execHook("after", "plan")}} + } + bindingErr := errors.New("binding identity unavailable") + if test.bindingFailure { + runtime.err = bindingErr + } + attempts, initCalls, hookCalls := 0, 0, 0 + shell := newShellFixture(t, func(call *shellCall) error { + if call.command == "hook-after" { + hookCalls++ + if test.hookFailure { + return exitCodeError(2) + } + return nil + } + attempts++ + if test.recoverInit && attempts == 1 { + return errors.New("Backend configuration changed") + } + return exitCodeError(test.code) + }) + executor := testExecutor(t) + executor.services.ExecuteShellCommand = shell.mock.Execute + executor.executeTerraformInitForcedFn = func(*schema.AtmosConfiguration, *schema.ConfigAndStacksInfo, string, string, bool, bool, ...ShellCommandOption) error { + initCalls++ + return nil + } + err := executor.runMainCommandPhase(config, info, execCtx, args, execCtx.componentPath, false) + require.Error(t, err) + if test.bindingFailure { + require.ErrorIs(t, err, bindingErr) + } else { + require.Equal(t, test.code, errUtils.GetExitCode(err)) + } + if test.recoverInit { + require.Equal(t, 2, attempts) + require.Equal(t, 1, initCalls) + } else { + require.Equal(t, 1, attempts) + require.Zero(t, initCalls) + } + if test.hooks && test.code == 2 && !test.noFlag { + require.Equal(t, 1, hookCalls) + } else { + require.Zero(t, hookCalls) + } + if test.hookFailure || test.bindingFailure || test.noFlag || test.code != 2 { + require.NoFileExists(t, path+configurationPlanSuffix) + return + } + require.FileExists(t, path+configurationPlanSuffix) + info.SubCommand, info.UseTerraformPlan, info.PlanFile = "apply", true, path + _, err = restoreConfigurationPlan(t.Context(), config, info) + require.NoError(t, err, "the status-two plan must be usable by --from-plan") + }) + } +} + // TestPlanOutputFlagsInEnvironmentAreRejected covers output flags that Terraform // reads from TF_CLI_ARGS or TF_CLI_ARGS_plan, which Atmos cannot bind. func TestPlanOutputFlagsInEnvironmentAreRejected(t *testing.T) { @@ -506,6 +882,75 @@ func TestPlanOutputFlagsInEnvironmentAreRejected(t *testing.T) { } } +func TestDetailedExitCodeBindingUsesSubprocessEnvironment(t *testing.T) { + t.Setenv("TF_CLI_ARGS", "") + for _, test := range []struct { + name, ambient string + processEnv, componentEnv []string + success bool + }{ + {name: "inherit ambient", ambient: "-detailed-exitcode", success: true}, + {name: "replacement enables", processEnv: []string{"TF_CLI_ARGS_plan=-detailed-exitcode"}, success: true}, + {name: "empty replacement excludes ambient", ambient: "-detailed-exitcode", processEnv: []string{}}, + {name: "replacement excludes ambient", ambient: "-detailed-exitcode", processEnv: []string{"OTHER=value"}}, + {name: "component enables", processEnv: []string{}, componentEnv: []string{"TF_CLI_ARGS_plan=-detailed-exitcode"}, success: true}, + {name: "component disables", processEnv: []string{"TF_CLI_ARGS_plan=-detailed-exitcode"}, componentEnv: []string{"TF_CLI_ARGS_plan="}}, + } { + t.Run(test.name, func(t *testing.T) { + t.Setenv("TF_CLI_ARGS_plan", test.ambient) + config, info, execCtx, _, path := planFixture(t) + info.ComponentExecution = detailedExitCodeInfo().ComponentExecution + info.ComponentEnvList = test.componentEnv + shell := newShellFixture(t, func(call *shellCall) error { + if call.command == info.Command { + return exitCodeError(2) + } + return nil + }) + executor := testExecutor(t) + executor.services.ExecuteShellCommand = shell.mock.Execute + err := executor.runMainCommandPhase(config, info, execCtx, []string{"plan", "-out=" + path}, execCtx.componentPath, false, component.WithEnvironment(test.processEnv)) + require.Equal(t, 2, errUtils.GetExitCode(err)) + require.Len(t, shell.calls, 2) + if test.success { + require.Equal(t, "hook-after", shell.calls[1].command) + require.FileExists(t, path+configurationPlanSuffix) + } else { + require.Equal(t, "hook-alert", shell.calls[1].command) + require.NoFileExists(t, path+configurationPlanSuffix) + } + }) + } +} + +func TestDetailedExitCodePolicyPlanBinding(t *testing.T) { + for _, deny := range []bool{false, true} { + t.Run(fmt.Sprintf("deny=%t", deny), func(t *testing.T) { + config, info, execCtx, _, path := planFixture(t) + var engine policy.Engine + require.NoError(t, engine.Register(policy.CELEngine, &policy.CEL{})) + execCtx.planPolicy = &planfile.Guard{Engine: &engine, Show: newPlanShowRunner(t), Rules: []policy.Rule{{ID: "guard", On: policy.TerraformPlan, Engine: policy.CELEngine, Expression: fmt.Sprint(deny), Effects: policy.Effects{Deny: true}}}} + require.NoError(t, os.Remove(path)) + shell := newShellFixture(t, func(call *shellCall) error { + require.NoError(t, os.WriteFile(plannedOutputPath(execCtx.componentPath, call.args), []byte("reviewed"), 0o600)) + return exitCodeError(2) + }) + executor := testExecutor(t) + executor.services.ExecuteShellCommand = shell.mock.Execute + err := executor.runMainCommandPhase(config, info, execCtx, []string{"plan", "-out=" + path, "-detailed-exitcode"}, execCtx.componentPath, false) + if deny { + require.Error(t, err) + require.NotEqual(t, 2, errUtils.GetExitCode(err)) + require.NoFileExists(t, path) + require.NoFileExists(t, path+configurationPlanSuffix) + } else { + require.Equal(t, 2, errUtils.GetExitCode(err)) + require.FileExists(t, path+configurationPlanSuffix) + } + }) + } +} + // TestRejectEnvironmentPlanOutputNameCase matches the variable names case-insensitively on Windows only. func TestRejectEnvironmentPlanOutputNameCase(t *testing.T) { for _, test := range []struct { @@ -554,6 +999,29 @@ func TestEnvironmentPlanOutputIgnoresUnownedPlans(t *testing.T) { require.NoError(t, validateEnvironmentPlanOutput(config, info, execCtx)) } +// The plan output check inspects the process environment the subprocess receives: an environment +// chosen for the execution replaces the sanitized one, including when it is empty. +func TestEnvironmentPlanOutputUsesExecutionEnvironment(t *testing.T) { + t.Setenv("TF_CLI_ARGS_plan", "") + t.Setenv("TF_CLI_ARGS", "") + config, info, execCtx, _, _ := planFixture(t) + flag := "TF_CLI_ARGS_plan=-out=env.plan" + + info.SanitizedEnv = []string{"OTHER=value"} + require.NoError(t, validateEnvironmentPlanOutput(config, info, execCtx)) + require.ErrorIs(t, validateEnvironmentPlanOutput(config, info, execCtx, component.WithEnvironment([]string{flag})), errUtils.ErrPlanVerificationFailed) + + info.SanitizedEnv = []string{flag} + require.ErrorIs(t, validateEnvironmentPlanOutput(config, info, execCtx), errUtils.ErrPlanVerificationFailed) + require.NoError(t, validateEnvironmentPlanOutput(config, info, execCtx, component.WithEnvironment([]string{})), "an empty execution environment replaces the sanitized one") + require.NoError(t, validateEnvironmentPlanOutput(config, info, execCtx, component.WithEnvironment([]string{"OTHER=value"}))) + require.ErrorIs(t, validateEnvironmentPlanOutput(config, info, execCtx, component.WithEnvironment(nil)), errUtils.ErrPlanVerificationFailed, "a nil environment keeps the sanitized one") + + info.SanitizedEnv = nil + info.ComponentEnvList = []string{flag} + require.ErrorIs(t, validateEnvironmentPlanOutput(config, info, execCtx, component.WithEnvironment([]string{})), errUtils.ErrPlanVerificationFailed, "component layers still apply") +} + func TestConfigurationPlanRoundTripNumericBackend(t *testing.T) { config, info, execCtx, fixture, path := planFixture(t) execCtx.preparedConfiguration.Backend["port"] = 1234 @@ -576,7 +1044,7 @@ func TestNewConfigurationPlanRejectsAPlanRewrittenAfterTerraformWroteIt(t *testi captured, err := fileDigest(path) require.NoError(t, err) info := &schema.ConfigAndStacksInfo{Command: binary, Stack: "dev", ComponentFromArg: "app"} - execCtx := &componentExecContext{workingDir: dir} + execCtx := &componentExecContext{workingDir: dir, preparedConfiguration: &schema.PreparedComponentConfiguration{Engine: "terraform"}} snapshot := &planSnapshot{path: path, identity: "identity", workingDigest: "working", planDigest: captured} plan, err := newConfigurationPlan(info, execCtx, snapshot, path) @@ -618,3 +1086,29 @@ func TestConfigurationWorkdirDigestResolvesRelativePaths(t *testing.T) { require.NotEqual(t, absolute, digest) }) } + +func TestConfigurationPlanPreservesReviewedOverwriteBytes(t *testing.T) { + config, info, execCtx, _, path := planFixture(t) + execCtx.preparedConfiguration.Files = []schema.ComponentConfigurationFile{{Path: "generated.tf", Content: []byte("# original A"), IfExists: "overwrite"}} + generated := filepath.Join(execCtx.workingDir, "generated.tf") + reviewed := []byte("# reviewed B") + require.NoError(t, os.WriteFile(generated, reviewed, 0o600)) + snapshot, err := takePlanSnapshot(t.Context(), config, info, execCtx, path) + require.NoError(t, err) + execCtx.planSnapshots = &planSnapshotter{result: snapshot} + // An after-hook must not change the bytes captured in the record. + require.NoError(t, os.WriteFile(generated, []byte("# after C"), 0o600)) + require.NoError(t, recordConfigurationPlan(t.Context(), config, info, execCtx, []string{"-out=" + path})) + info.SubCommand, info.UseTerraformPlan, info.PlanFile = "apply", true, path + _, err = restoreConfigurationPlan(t.Context(), config, info) + require.ErrorIs(t, err, errUtils.ErrPlanVerificationFailed) + require.NoError(t, os.WriteFile(generated, reviewed, 0o600)) + prepared, err := restoreConfigurationPlan(t.Context(), config, info) + require.NoError(t, err) + require.Equal(t, reviewed, prepared.Files[0].Content) + require.NoError(t, writePreparedConfiguration(execCtx.workingDir, prepared)) + actual, err := os.ReadFile(generated) + require.NoError(t, err) + require.Equal(t, reviewed, actual) + require.Equal(t, []byte("# original A"), execCtx.preparedConfiguration.Files[0].Content) +} diff --git a/pkg/component/terraform/configuration_source_pipeline_test.go b/pkg/component/terraform/configuration_source_pipeline_test.go new file mode 100644 index 00000000000..f667df0a70d --- /dev/null +++ b/pkg/component/terraform/configuration_source_pipeline_test.go @@ -0,0 +1,114 @@ +package terraform + +import ( + "context" + "os" + "path/filepath" + "testing" + + "github.com/stretchr/testify/require" + "go.uber.org/mock/gomock" + + errUtils "github.com/cloudposse/atmos/errors" + adapter "github.com/cloudposse/atmos/pkg/plugin/adapter/component" + configurationadapter "github.com/cloudposse/atmos/pkg/plugin/adapter/configuration" + pluginv1 "github.com/cloudposse/atmos/pkg/plugin/proto/atmos/plugin/v1" + "github.com/cloudposse/atmos/pkg/provisioner" + "github.com/cloudposse/atmos/pkg/schema" +) + +//go:generate go run go.uber.org/mock/mockgen@v0.6.0 -source=$GOFILE -destination=mock_configuration_provider_test.go -package=$GOPACKAGE -mock_names=configurationProvider=MockConfigurationProvider + +// configurationProvider is the plugin boundary a configuration session calls. +type configurationProvider interface { + configurationadapter.Provider +} + +// The generated mock implements the provider contract the session asserts. +var _ configurationProvider = (*MockConfigurationProvider)(nil) + +// sourcePipelineProvider holds the responses the generated provider returns and counts preparations. +// It replaces only the RPC boundary; source validation is performed by the real Session.Prepare +// reached through component preparation. +type sourcePipelineProvider struct { + unit *pluginv1.DiscoveredComponent + result *pluginv1.PrepareComponentResponse + prepares int +} + +// mock returns a generated provider that answers from the fixture. ResolveState has no +// expectation, so gomock rejects it. +func (p *sourcePipelineProvider) mock(t *testing.T) *MockConfigurationProvider { + t.Helper() + provider := NewMockConfigurationProvider(gomock.NewController(t)) + provider.EXPECT().ComponentType().Return("terraform").AnyTimes() + provider.EXPECT().Discover(gomock.Any(), gomock.Any()).DoAndReturn(func(context.Context, *pluginv1.DiscoverComponentsRequest) (*pluginv1.DiscoverComponentsResponse, error) { + return &pluginv1.DiscoverComponentsResponse{Components: []*pluginv1.DiscoveredComponent{p.unit}}, nil + }).AnyTimes() + provider.EXPECT().Prepare(gomock.Any(), gomock.Any()).DoAndReturn(func(context.Context, *pluginv1.ComponentRequest) (*pluginv1.PrepareComponentResponse, error) { + p.prepares++ + return p.result, nil + }).AnyTimes() + return provider +} + +func TestConfigurationSourceSymlinkRejectedBeforeInitAndMain(t *testing.T) { + for _, escaping := range []bool{false, true} { + name := "in-root source reaches init and validate" + if escaping { + name = "escaping source stops before init and validate" + } + t.Run(name, func(t *testing.T) { + project := newReviewedPlanProject(t) + targetRoot := project.root + if escaping { + targetRoot = t.TempDir() + } + target := filepath.Join(targetRoot, "inputs.hcl") + require.NoError(t, os.WriteFile(target, []byte("# source"), 0o600)) + if err := os.Symlink(target, filepath.Join(project.root, "source.hcl")); err != nil { + t.Skipf("symlinks unavailable: %v", err) + } + value, err := adapter.EncodeObject(map[string]any{"source": project.module}) + require.NoError(t, err) + empty, err := adapter.EncodeObject(map[string]any{}) + require.NoError(t, err) + provider := &sourcePipelineProvider{ + unit: &pluginv1.DiscoveredComponent{Provider: "terragrunt", Engine: "terraform", Source: "terragrunt.hcl", Address: &pluginv1.ComponentAddress{Stack: "dev", Component: "app", ComponentType: "terraform"}, Configuration: value}, + result: &pluginv1.PrepareComponentResponse{Engine: "terraform", ModuleSource: project.module, Inputs: empty, State: &pluginv1.ComponentStateIdentity{BackendType: "http", Backend: empty, Workspace: "default"}, Sources: []*pluginv1.ComponentSource{{Path: "source.hcl", Kind: pluginv1.ComponentSource_FILE}}}, + } + project.config.Components.Terraform.ConfigurationProviders = map[string]schema.ComponentConfiguration{"terragrunt": {Enabled: true, Discovery: schema.ComponentDiscoveryConfig{Enabled: true}}} + session, err := configurationadapter.New(t.Context(), project.config, "validate", func(context.Context, []string) (map[string]configurationadapter.Provider, error) { + return map[string]configurationadapter.Provider{"terragrunt": provider.mock(t)}, nil + }) + require.NoError(t, err) + project.config.ComponentConfigurations = session + info := project.info(t, "validate") + shell := newShellFixture(t, nil) + executor := testExecutor(t) + executor.services.ExecuteShellCommand = shell.mock.Execute + // Use the same preparation and command phases as Executor.execute, keeping + // implicit init enabled so the passing case proves both phases are reachable. + execution, err := executor.prepareComponentExecution(t.Context(), provisioner.OutputWriters{}, project.config, info, false) + if err == nil { + err = executor.executeCommandPipeline(project.config, info, execution) + } + require.Equal(t, 1, provider.prepares, "must reach the real session's preparation boundary") + if escaping { + require.ErrorIs(t, err, errUtils.ErrPluginInvalidConfig) + require.Nil(t, execution) + require.Empty(t, shell.calls, "neither Terraform init nor the main command may run") + return + } + require.NoError(t, err) + var commands []string + for _, call := range shell.calls { + if len(call.args) > 0 { + commands = append(commands, call.args[0]) + } + } + require.Contains(t, commands, "init") + require.Contains(t, commands, "validate") + }) + } +} diff --git a/pkg/component/terraform/configuration_test.go b/pkg/component/terraform/configuration_test.go index 5791743c6c1..93727312294 100644 --- a/pkg/component/terraform/configuration_test.go +++ b/pkg/component/terraform/configuration_test.go @@ -31,21 +31,20 @@ func TestPrepareConfiguration(t *testing.T) { require.Same(t, expected, result) require.True(t, filepath.IsAbs(info.Command)) require.Equal(t, expected.Inputs, info.ComponentVarsSection) - // A local module is refreshed on every run; see TestPrepareConfigurationRefreshesOnlyLocalModules. - require.Equal(t, map[string]any{"uri": expected.Source, "ttl": "0s"}, info.ComponentSection["source"]) + // Local module edits are detected by the source provisioner, so no TTL forces a refresh. + require.Equal(t, map[string]any{"uri": expected.Source}, info.ComponentSection["source"]) require.Equal(t, expected.Backend, info.ComponentBackendSection) require.Equal(t, "default", info.TerraformWorkspace) require.True(t, config.Components.Terraform.AutoGenerateBackendFile) } -func TestPrepareConfigurationRefreshesOnlyLocalModules(t *testing.T) { +func TestPrepareConfigurationNeverForcesSourceRefresh(t *testing.T) { for _, test := range []struct { name string source string - ttl any }{ - {name: "relative path", source: "./modules/unit", ttl: "0s"}, - {name: "absolute path", source: filepath.Join(t.TempDir(), "module"), ttl: "0s"}, + {name: "relative path", source: "./modules/unit"}, + {name: "absolute path", source: filepath.Join(t.TempDir(), "module")}, {name: "git", source: "git::https://example.invalid/modules.git//unit?ref=v1"}, {name: "archive", source: "https://example.invalid/unit.zip"}, } { @@ -63,7 +62,7 @@ func TestPrepareConfigurationRefreshesOnlyLocalModules(t *testing.T) { source, ok := info.ComponentSection["source"].(map[string]any) require.True(t, ok) require.Equal(t, test.source, source["uri"]) - require.Equal(t, test.ttl, source["ttl"]) + require.NotContains(t, source, "ttl", "a TTL of zero would re-copy the module and discard saved plans on every run") }) } } @@ -132,6 +131,29 @@ func TestConfigurationIdentityIsStableAndSeparatesFields(t *testing.T) { require.NotEqual(t, configurationIdentity("ab", "c"), configurationIdentity("a", "bc")) } +func TestCacheLocalStatePath(t *testing.T) { + cache := filepath.Join("live", "dev", "unit", ".terragrunt-cache", "abc", "def", "terraform.tfstate") + for _, test := range []struct { + name string + prepared schema.PreparedComponentConfiguration + want bool + }{ + {"default local state inside the cache", schema.PreparedComponentConfiguration{BackendType: "local", Backend: map[string]any{"path": cache}}, true}, + {"stable local path", schema.PreparedComponentConfiguration{BackendType: "local", Backend: map[string]any{"path": filepath.Join("live", "dev", "unit", "state", "terraform.tfstate")}}, false}, + {"similar name is not the cache", schema.PreparedComponentConfiguration{BackendType: "local", Backend: map[string]any{"path": filepath.Join("a", "x.terragrunt-cache", "terraform.tfstate")}}, false}, + {"remote backend", schema.PreparedComponentConfiguration{BackendType: "s3", Backend: map[string]any{"path": cache}}, false}, + {"no backend settings", schema.PreparedComponentConfiguration{BackendType: "local"}, false}, + } { + t.Run(test.name, func(t *testing.T) { + path, found := cacheLocalStatePath(&test.prepared) + require.Equal(t, test.want, found) + if found { + require.Equal(t, cache, path) + } + }) + } +} + func TestCheckConfigurationWorkspace(t *testing.T) { for _, test := range []struct { name string @@ -154,6 +176,22 @@ func TestCheckConfigurationWorkspace(t *testing.T) { } } +// Warning about cache-local state must never change where the state lives or fail the command. +func TestPrepareConfigurationKeepsCacheLocalStateLocation(t *testing.T) { + runtime := NewMockComponentConfigurationRuntime(gomock.NewController(t)) + config := &schema.AtmosConfiguration{ComponentConfigurations: runtime} + binary, err := os.Executable() + require.NoError(t, err) + info := &schema.ConfigAndStacksInfo{Command: binary, SubCommand: "plan", Stack: "live/dev", ComponentFromArg: "app", ComponentSection: map[string]any{}, ComponentSettingsSection: map[string]any{"configuration": true}} + cache := filepath.Join(t.TempDir(), ".terragrunt-cache", "a", "b", "terraform.tfstate") + runtime.EXPECT().Prepare(gomock.Any(), info).Return(&schema.PreparedComponentConfiguration{Engine: "terraform", Source: "./module", BackendType: "local", Backend: map[string]any{"path": cache}}, nil) + + _, err = prepareConfiguration(t.Context(), config, info) + + require.NoError(t, err) + require.Equal(t, cache, info.ComponentBackendSection["path"]) +} + func TestRejectRuntimeArtifacts(t *testing.T) { require.NoError(t, rejectRuntimeArtifacts([]schema.ComponentConfigurationFile{{Path: "backend.tf", IfExists: "overwrite"}})) for _, path := range []string{"examples/terraform.tfstate", "nested/provider.tf"} { @@ -211,3 +249,23 @@ func TestGetTerraformStateStopsWhenProviderCannotResolveBackend(t *testing.T) { require.ErrorIs(t, err, failure, "the provider's failure is returned as is, not retried against a guessed backend") require.Nil(t, result) } + +func TestConfigurationCommandPrecedence(t *testing.T) { + for _, tc := range []struct{ name, global, stack, flag, want string }{ + {"default", "", "", "", "terraform"}, + {"global", "tofu", "", "", "tofu"}, + {"component", "tofu", "terraform", "", "terraform"}, + {"inherited command", "tofu", "/inherited/terraform", "", "/inherited/terraform"}, + {"override command", "tofu", "/overrides/terraform", "", "/overrides/terraform"}, + {"explicit flag wins over global and stack", "tofu", "/stack/tofu", "/cli/terraform", "/cli/terraform"}, + } { + t.Run(tc.name, func(t *testing.T) { + config := &schema.AtmosConfiguration{} + config.Components.Terraform.Command = tc.global + info := &schema.ConfigAndStacksInfo{Command: tc.stack, ComponentCommand: tc.flag} + resolveTerraformCommand(config, info) + require.Equal(t, tc.want, info.Command) + require.Equal(t, tc.flag, info.ComponentCommand, "explicit flag provenance survives resolution") + }) + } +} diff --git a/pkg/component/terraform/execute_command.go b/pkg/component/terraform/execute_command.go index 100dd105f11..1c8b25c1415 100644 --- a/pkg/component/terraform/execute_command.go +++ b/pkg/component/terraform/execute_command.go @@ -253,9 +253,10 @@ func cleanupTerraformFiles(atmosConfig *schema.AtmosConfiguration, info *schema. return } + finishConfigurationPlan(atmosConfig, info) // Only a completed apply/destroy consumes the default plan. Inspection commands // also pass through this pipeline and must preserve plans saved for later use. - if (info.SubCommand == subcommandApply || info.SubCommand == "destroy") && info.PlanFile == "" { + if (info.SubCommand == subcommandApply || info.SubCommand == "destroy") && info.PlanFile == "" && !ownsConfigurationPlan(info) { planFilePath := ConstructTerraformComponentPlanfilePath(atmosConfig, info) removeTerraformFile(planFilePath, "Failed to remove plan file during cleanup") } diff --git a/pkg/component/terraform/execute_hooks.go b/pkg/component/terraform/execute_hooks.go index 5e2a25c80f6..389dbb51b4b 100644 --- a/pkg/component/terraform/execute_hooks.go +++ b/pkg/component/terraform/execute_hooks.go @@ -6,21 +6,21 @@ import ( "fmt" "io" "io/fs" - "maps" "os" "path/filepath" "slices" + "strconv" "strings" "sync" errUtils "github.com/cloudposse/atmos/errors" "github.com/cloudposse/atmos/pkg/component" "github.com/cloudposse/atmos/pkg/component/execution" + envpkg "github.com/cloudposse/atmos/pkg/env" log "github.com/cloudposse/atmos/pkg/logger" "github.com/cloudposse/atmos/pkg/perf" "github.com/cloudposse/atmos/pkg/process" "github.com/cloudposse/atmos/pkg/schema" - "github.com/cloudposse/atmos/pkg/terraform/autoinit" "github.com/cloudposse/atmos/pkg/ui" ) @@ -85,16 +85,18 @@ func (e *Executor) runMainShellCommand(atmosConfig *schema.AtmosConfiguration, i } return e.runExecutionPhase(atmosConfig, info, phase, opts, func(phaseInfo *schema.ConfigAndStacksInfo, phaseArgs []string, phaseOpts []component.ExecutionOption) error { ctx := component.ContextFromOptions(phaseOpts...) + env := slices.Concat(envpkg.ConvertMapToSlice(atmosConfig.Env), phaseInfo.ComponentEnvList) if captured, ok := ctx.Value(mainCommandArgsKey{}).(*[]string); ok { *captured = slices.Clone(phaseArgs) } run := func() error { - return ExecuteShellCommandWithRetry( + var changesPresent error + err := ExecuteShellCommandWithRetry( atmosConfig, phaseInfo, info.SubCommand, func(o ...component.ExecutionOption) error { - return e.executeStreamingOrShell(atmosConfig, phaseInfo, &streamingExecRequest{ + err := e.executeStreamingOrShell(atmosConfig, phaseInfo, &streamingExecRequest{ componentPath: componentPath, args: phaseArgs, gatePhase: info.SubCommand, @@ -102,15 +104,27 @@ func (e *Executor) runMainShellCommand(atmosConfig *schema.AtmosConfiguration, i redirectStdErr: info.RedirectStdErr, shellOpts: o, }) + // Classify with the effective arguments and environment before legacy + // output matching can retry a plan that already succeeded. + if isDetailedExitCodeChanges(phase.command, phaseArgs, env, err, o...) { + changesPresent = err + return nil + } + return err }, phaseOpts..., ) + if err != nil { + return err + } + return changesPresent } if snapshots, ok := ctx.Value(planSnapshotKey{}).(*planSnapshotter); ok { run = snapshots.around(phaseArgs, run) } err := run() - if err == nil || !lifecycleOwnsRecovery(info) { + // A plan that reports changes completed; only a real failure is given to init recovery. + if err == nil || !lifecycleOwnsRecovery(info) || isDetailedExitCodeChanges(phase.command, phaseArgs, env, err, phaseOpts...) { return err } if recovery, ok := ctx.Value(mainCommandRecoveryKey{}).(*mainCommandRecovery); ok { @@ -147,7 +161,7 @@ func (e *Executor) runExecutionPhase(atmosConfig *schema.AtmosConfiguration, inf if err := validatePlanOutputFlags(phase.command, phase.args); err != nil { return err } - return action(info, phase.args, opts) + return successfulPlanResult(phase.command, phase.args, info.ComponentEnvList, action(info, phase.args, opts), opts...) } applied, err := execution.ApplyExtraArguments(config.ExtraArguments, &execution.ArgumentsRequest{ @@ -178,19 +192,36 @@ func (e *Executor) runExecutionPhase(atmosConfig *schema.AtmosConfiguration, inf defer capture.cleanup() } runner := &hookRunner{executor: e, atmosConfig: atmosConfig, info: &phaseInfo, phase: phase, opts: hookOptions(opts), capture: capture} - // A detailed plan exit code of 2 reports changes, not a failure: ordinary after hooks run and - // error hooks do not. The status itself is returned unchanged once the hooks have run. - var detailedResult error + // With -detailed-exitcode, exit status 2 reports a plan with changes. Hooks must not see that + // as a failure, but the status still has to reach the caller once the hooks are done. + var changesPresent error + env := slices.Concat(envpkg.ConvertMapToSlice(atmosConfig.Env), phaseInfo.ComponentEnvList) err = execution.Run(component.ContextFromOptions(opts...), config, phase.command, runner, func() (execution.Output, error) { - output, err := runPhaseAction(&phaseInfo, applied.Args, opts, capture, action) - if isDetailedPlanChanges(phase, applied.Args, err) { - detailedResult = err + output, actionErr := runPhaseAction(&phaseInfo, applied.Args, opts, capture, action) + if isDetailedExitCodeChanges(phase.command, applied.Args, env, actionErr, opts...) { + changesPresent = actionErr return output, nil } - return output, err + return output, actionErr }) - if err == nil { - return detailedResult + if err != nil { + return err + } + return successfulPlanResult(phase.command, applied.Args, env, changesPresent, opts...) +} + +// successfulPlanChanges distinguishes a completed plan (including its hooks) from a +// failed command or hook that happens to return the same exit status. +type successfulPlanChanges struct{ error } + +func (e *successfulPlanChanges) Unwrap() error { + defer perf.Track(nil, "terraform.successfulPlanChanges.Unwrap")() + return e.error +} + +func successfulPlanResult(command string, args, env []string, err error, opts ...component.ExecutionOption) error { + if isDetailedExitCodeChanges(command, args, env, err, opts...) { + return &successfulPlanChanges{error: err} } return err } @@ -208,9 +239,94 @@ func runPhaseAction(info *schema.ConfigAndStacksInfo, args []string, opts []comp return file.finish(), err } -// isDetailedPlanChanges reports whether err is plan's "-detailed-exitcode" status for changes present. -func isDetailedPlanChanges(phase *executionPhase, args []string, err error) bool { - return err != nil && phase.command == "plan" && detailedExitCodeEnabled(args) && resolveExitCode(err) == detailedExitCodeChangesDetected +// detailedExitCodeChanges is the status `plan -detailed-exitcode` returns when changes are present. +const detailedExitCodeChanges = 2 + +// isDetailedExitCodeChanges reports whether err is exit status 2 of a command run with +// -detailed-exitcode, which Terragrunt treats as success for hooks and the success of the unit. +// A failed hook is never that status, even when the hook itself exited with 2. +func isDetailedExitCodeChanges(command string, args, env []string, err error, opts ...component.ExecutionOption) bool { + if command != "plan" || err == nil || errors.Is(err, errUtils.ErrComponentHookFailed) || errUtils.GetExitCode(err) != detailedExitCodeChanges { + return false + } + return detailedExitCodeEnabled(command, args, env, opts...) +} + +// detailedExitCodeEnabled reports whether -detailed-exitcode is in effect for the command. +// Terraform inserts TF_CLI_ARGS_, then TF_CLI_ARGS, before the command-line arguments, +// and the last setting of a flag wins, so an explicit command-line setting takes precedence. +// Terraform reads the variables as shell words, so they are tokenized like the engine does: quoted +// values are unquoted and tabs and newlines separate flags. A value that cannot be tokenized makes +// the engine refuse to run, so it contributes no flags here. +func detailedExitCodeEnabled(command string, args, env []string, opts ...component.ExecutionOption) bool { + suffix := strings.NewReplacer("-", "_", " ", "_").Replace(command) + words := slices.Concat( + environmentWords(environmentValue(env, "TF_CLI_ARGS_"+suffix, opts...)), + environmentWords(environmentValue(env, "TF_CLI_ARGS", opts...)), + args, + ) + enabled := false + for _, word := range words { + name, value, hasValue := strings.Cut(strings.TrimLeft(word, "-"), "=") + if !strings.HasPrefix(word, "-") || name != "detailed-exitcode" { + continue + } + if !hasValue { + enabled = true + continue + } + if parsed, err := strconv.ParseBool(value); err == nil { + enabled = parsed + } + } + return enabled +} + +// environmentWords tokenizes a TF_CLI_ARGS value like the engine. A malformed value is refused by +// the engine, so it yields no words. +func environmentWords(value string) []string { + words, err := tokenizeEnvironmentArgs(value) + if err != nil { + return nil + } + return words +} + +// planBaseEnvironment is the process environment layer of a plan subprocess: the environment +// chosen for the execution when there is one, otherwise the sanitized environment, otherwise the +// inherited one. +func planBaseEnvironment(info *schema.ConfigAndStacksInfo, opts []component.ExecutionOption) []string { + var options component.ExecutionOptions + for _, opt := range opts { + opt(&options) + } + switch { + case options.ProcessEnv != nil: + return options.ProcessEnv + case info.SanitizedEnv != nil: + return info.SanitizedEnv + } + return os.Environ() +} + +// environmentValue returns the value of key the subprocess sees: the component environment is +// appended to the selected process environment, so its last entry wins. +func environmentValue(env []string, key string, opts ...component.ExecutionOption) string { + var options component.ExecutionOptions + for _, opt := range opts { + opt(&options) + } + base := options.ProcessEnv + if base == nil { + base = os.Environ() + } + merged := slices.Concat(base, env) + for i := len(merged) - 1; i >= 0; i-- { + if name, value, found := strings.Cut(merged[i], "="); found && name == key { + return value + } + } + return "" } // hookOptions carries over only what a hook shares with the command: cancellation, the process @@ -252,7 +368,7 @@ type hookRunner struct { func (r *hookRunner) RunHook(_ context.Context, kind execution.HookKind, hook *schema.ComponentHook) (execution.Output, error) { defer perf.Track(r.atmosConfig, "terraform.hookRunner.RunHook")() - ui.Info(fmt.Sprintf("Executing %s: %s", strings.ReplaceAll(string(kind), "_", " "), hook.Name)) + ui.Info(fmt.Sprintf("Executing %s: %s%s", strings.ReplaceAll(string(kind), "_", " "), hook.Name, hookUnitLabel(r.info))) var base component.ExecutionOptions for _, opt := range r.opts { @@ -284,23 +400,48 @@ func (r *hookRunner) RunHook(_ context.Context, kind execution.HookKind, hook *s hookEnvCommand+"="+r.phase.command, hookEnvHookName+"="+hook.Name, ) - err := r.executor.services.ExecuteShellCommand(*r.atmosConfig, hook.Execute[0], hook.Execute[1:], r.workingDirectory(hook), env, false, "", opts...) + directory, err := r.workingDirectory(hook) + if err != nil { + return nil, err + } + err = r.executor.services.ExecuteShellCommand(*r.atmosConfig, hook.Execute[0], hook.Execute[1:], directory, env, false, "", opts...) if file == nil { return nil, err } return file.finish(), err } +// hookUnitLabel names the component and stack a hook runs for, so output of a bulk run can be attributed. +func hookUnitLabel(info *schema.ConfigAndStacksInfo) string { + if info.ComponentFromArg == "" { + return "" + } + return fmt.Sprintf(" (%s in %s)", info.ComponentFromArg, info.Stack) +} + // workingDirectory defaults to the provisioned module directory; relative paths resolve there. -func (r *hookRunner) workingDirectory(hook *schema.ComponentHook) string { +// A directory that does not exist is reported as such, instead of as a failure to start the program. +func (r *hookRunner) workingDirectory(hook *schema.ComponentHook) (string, error) { switch { case hook.WorkingDirectory == "": - return r.phase.componentPath + return r.phase.componentPath, nil case filepath.IsAbs(hook.WorkingDirectory): - return hook.WorkingDirectory + return hook.WorkingDirectory, requireDirectory(hook.WorkingDirectory, "") default: - return filepath.Join(r.phase.componentPath, hook.WorkingDirectory) + resolved := filepath.Join(r.phase.componentPath, hook.WorkingDirectory) + return resolved, requireDirectory(resolved, hook.WorkingDirectory) + } +} + +// requireDirectory fails with the missing directory's name; relative names also say where they resolved. +func requireDirectory(directory, relative string) error { + if info, err := os.Stat(directory); err == nil && info.IsDir() { + return nil + } + if relative == "" { + return fmt.Errorf("%w: %s", errUtils.ErrComponentHookWorkingDirectory, directory) } + return fmt.Errorf("%w: %s (working_dir %q resolves against the provisioned module directory, not the directory of terragrunt.hcl; use ${get_terragrunt_dir()}/%s for a directory beside it)", errUtils.ErrComponentHookWorkingDirectory, directory, relative, relative) } // expandPolicyPlanArguments gives a plan policy the effective plan invocation. The policy @@ -462,92 +603,3 @@ func (c *captureFile) Open() (io.Reader, error) { func captureError(err error) error { return fmt.Errorf("%w: %w", errUtils.ErrComponentOutputCapture, err) } - -// initFileFlags are the init options whose value can name a file Terraform reads during init. -var initFileFlags = []string{"-backend-config", "-var-file"} - -// withProviderInitSettings adds the init settings that provider extra arguments contribute to smart -// init's inputs: the effective init arguments and environment, and the contents of files those -// arguments name. Without them, an edited init-only backend file would leave a stale initialization -// trusted. Values only contribute to the fingerprint digest; they are never recorded or logged. -// Settings that cannot be evaluated return nil inputs, so no recorded marker is trusted; the init -// phase itself then reports the problem. -func withProviderInitSettings(in *autoinit.Inputs, info *schema.ConfigAndStacksInfo, componentPath string) *autoinit.Inputs { - config := info.ComponentExecution - if in == nil || config.IsEmpty() { - return in - } - applied, err := execution.ApplyExtraArguments(config.ExtraArguments, &execution.ArgumentsRequest{ - Command: subcommandInit, Args: []string{subcommandInit}, InsertAt: 1, WorkingDirectory: componentPath, - }) - if err != nil { - return nil - } - in.EnvLookup = overlayEnvironment(applied.Environment, in.EnvLookup) - extra := maps.Clone(in.Extra) - if extra == nil { - extra = map[string]string{} - } - arguments := applied.Args[1:] - extra["provider-init-arguments"] = strings.Join(arguments, "\x00") - extra["provider-init-environment"] = strings.Join(applied.Environment, "\x00") - for _, path := range initFileArguments(arguments) { - if !filepath.IsAbs(path) { - path = filepath.Join(componentPath, path) - } - // A missing or unreadable file is recorded as such, so creating it changes the digest. - digest, err := fileDigest(path) - if err != nil { - digest = "unreadable" - } - extra["provider-init-file:"+path] = digest - } - in.Extra = extra - return in -} - -// overlayEnvironment looks up KEY=value entries first and then base. A nil base reports a key -// absent, so the caller falls back to the process environment. -func overlayEnvironment(entries []string, base func(string) (string, bool)) func(string) (string, bool) { - environment := map[string]string{} - for _, entry := range entries { - if key, value, ok := strings.Cut(entry, "="); ok { - environment[key] = value - } - } - return func(key string) (string, bool) { - if value, ok := environment[key]; ok { - return value, true - } - if base == nil { - return "", false - } - return base(key) - } -} - -// initFileArguments returns the file paths named by -backend-config and -var-file options, in -// the "-flag=value" and "-flag value" forms, each with a single or double dash. A -backend-config value holding "=" sets one -// setting instead of naming a file; it is already part of the arguments. -func initFileArguments(arguments []string) []string { - var paths []string - for i := 0; i < len(arguments); i++ { - name, value, inline := strings.Cut(strings.TrimLeft(arguments[i], "-"), "=") - name = "-" + name - if !slices.Contains(initFileFlags, name) { - continue - } - if !inline { - if i+1 >= len(arguments) { - break - } - i++ - value = arguments[i] - } - if name == "-backend-config" && strings.Contains(value, "=") { - continue - } - paths = append(paths, value) - } - return paths -} diff --git a/pkg/component/terraform/execute_hooks_test.go b/pkg/component/terraform/execute_hooks_test.go index 2c12232aadf..8496596ecc7 100644 --- a/pkg/component/terraform/execute_hooks_test.go +++ b/pkg/component/terraform/execute_hooks_test.go @@ -170,6 +170,7 @@ func TestMainCommandMissingRequiredVarFileFailsBeforeAnyProcess(t *testing.T) { func TestHooksRunWithTheInvocationEnvironmentAndDirectory(t *testing.T) { dir := t.TempDir() absolute := t.TempDir() + require.NoError(t, os.Mkdir(filepath.Join(dir, "sub"), 0o755)) shell := newShellFixture(t, nil) e := hookExecutor(shell) ctx := context.WithValue(t.Context(), hookContextKey{}, "marker") @@ -521,6 +522,201 @@ func TestPreflightRequiredVarFiles(t *testing.T) { require.NoError(t, preflightRequiredVarFiles(workspace, dir), "workspace management runs no main command") } +// exitCodeError mimics the error a Terraform subprocess returns with a given status. +func exitCodeError(code int) error { return errUtils.ExitCodeError{Code: code} } + +func detailedExitCodeInfo() *schema.ConfigAndStacksInfo { + return hookInfo("plan", &schema.ComponentExecution{ + AfterHooks: []schema.ComponentHook{execHook("after", "plan")}, + ErrorHooks: []schema.ComponentHook{{Name: "alert", Commands: []string{"plan"}, Execute: []string{"hook-alert"}, OnErrors: []string{".*"}}}, + }) +} + +// Exit status 2 of plan -detailed-exitcode means "changes present": hooks must see success, +// and the caller must still receive status 2. +func TestDetailedExitCodeChangesAreNotAHookFailure(t *testing.T) { + shell := newShellFixture(t, func(call *shellCall) error { + if call.command == "terraform" { + return exitCodeError(2) + } + return nil + }) + err := hookExecutor(shell).runMainTerraformCommand(&schema.AtmosConfiguration{}, detailedExitCodeInfo(), []string{"plan", "-detailed-exitcode"}, t.TempDir()) + + require.Error(t, err) + require.Equal(t, 2, errUtils.GetExitCode(err), "the status must still reach the caller") + require.Equal(t, []string{"terraform plan", "hook-after arg"}, shell.names(), "the after hook runs and the error hook does not") +} + +func TestDetailedExitCodeIsOnlyToleratedWhereTerraformDefinesIt(t *testing.T) { + for _, test := range []struct { + name string + args []string + code int + }{ + {"status 2 without the flag is a failure", []string{"plan"}, 2}, + {"status 1 with the flag is a failure", []string{"plan", "-detailed-exitcode"}, 1}, + {"double dash flag", []string{"plan", "--detailed-exitcode"}, 2}, + {"explicit true value", []string{"plan", "-detailed-exitcode=true"}, 2}, + } { + t.Run(test.name, func(t *testing.T) { + shell := newShellFixture(t, func(call *shellCall) error { + if call.command == "terraform" { + return exitCodeError(test.code) + } + return nil + }) + err := hookExecutor(shell).runMainTerraformCommand(&schema.AtmosConfiguration{}, detailedExitCodeInfo(), test.args, t.TempDir()) + require.Equal(t, test.code, errUtils.GetExitCode(err)) + tolerated := test.code == 2 && len(test.args) > 1 + if tolerated { + require.Equal(t, []string{"terraform plan", "hook-after arg"}, shell.names()) + return + } + require.Equal(t, []string{"terraform plan", "hook-alert"}, shell.names(), "a real failure runs the error hook and skips the after hook") + }) + } +} + +func TestHookWorkingDirectoryErrorsNameTheDirectory(t *testing.T) { + dir := t.TempDir() + missing := filepath.Join(t.TempDir(), "absent") + for name, test := range map[string]struct { + directory string + contains []string + }{ + "absolute": {missing, []string{missing}}, + "relative": {"scripts", []string{filepath.Join(dir, "scripts"), "get_terragrunt_dir()"}}, + } { + t.Run(name, func(t *testing.T) { + shell := newShellFixture(t, nil) + info := hookInfo("plan", &schema.ComponentExecution{BeforeHooks: []schema.ComponentHook{ + {Name: "bad", Commands: []string{"plan"}, Execute: []string{"hook-bad"}, WorkingDirectory: test.directory}, + }}) + err := hookExecutor(shell).runMainTerraformCommand(&schema.AtmosConfiguration{}, info, []string{"plan"}, dir) + require.ErrorIs(t, err, errUtils.ErrComponentHookWorkingDirectory) + require.ErrorIs(t, err, errUtils.ErrComponentHookFailed) + for _, want := range test.contains { + require.Contains(t, err.Error(), want) + } + require.Empty(t, shell.calls, "the program must not be started in a missing directory") + }) + } +} + +func TestHookUnitLabel(t *testing.T) { + require.Empty(t, hookUnitLabel(&schema.ConfigAndStacksInfo{})) + require.Equal(t, " (network in live/dev)", hookUnitLabel(&schema.ConfigAndStacksInfo{ComponentFromArg: "network", Stack: "live/dev"})) +} + +// A hook listing "state" matches every state subcommand, as in Terragrunt. +func TestHooksMatchTheTerraformCommandNotItsSubcommand(t *testing.T) { + shell := newShellFixture(t, nil) + info := hookInfo("state list", &schema.ComponentExecution{BeforeHooks: []schema.ComponentHook{execHook("state-hook", "state")}}) + require.NoError(t, hookExecutor(shell).runMainTerraformCommand(&schema.AtmosConfiguration{}, info, []string{"state", "list"}, t.TempDir())) + require.Equal(t, []string{"hook-state-hook arg", "terraform state"}, shell.names()) +} + +// The flag can come from TF_CLI_ARGS_plan or TF_CLI_ARGS, and a command-line setting takes precedence. +func TestDetailedExitCodeEnabledFollowsTerraformPrecedence(t *testing.T) { + t.Setenv("TF_CLI_ARGS_plan", "") + t.Setenv("TF_CLI_ARGS", "") + for _, test := range []struct { + name string + args []string + env []string + want bool + }{ + {"absent", []string{"plan"}, nil, false}, + {"bare flag", []string{"plan", "-detailed-exitcode"}, nil, true}, + {"explicit true", []string{"plan", "-detailed-exitcode=true"}, nil, true}, + {"explicit false", []string{"plan", "-detailed-exitcode=false"}, nil, false}, + {"command environment", []string{"plan"}, []string{"TF_CLI_ARGS_plan=-input=false -detailed-exitcode"}, true}, + {"global environment", []string{"plan"}, []string{"TF_CLI_ARGS=-detailed-exitcode"}, true}, + {"global wins over command environment", []string{"plan"}, []string{"TF_CLI_ARGS_plan=-detailed-exitcode", "TF_CLI_ARGS=-detailed-exitcode=false"}, false}, + {"command line disables environment", []string{"plan", "-detailed-exitcode=false"}, []string{"TF_CLI_ARGS_plan=-detailed-exitcode"}, false}, + {"command line enables over environment", []string{"plan", "-detailed-exitcode"}, []string{"TF_CLI_ARGS_plan=-detailed-exitcode=false"}, true}, + {"other command environment", []string{"plan"}, []string{"TF_CLI_ARGS_apply=-detailed-exitcode"}, false}, + {"later component value wins", []string{"plan"}, []string{"TF_CLI_ARGS_plan=-detailed-exitcode", "TF_CLI_ARGS_plan="}, false}, + {"single-quoted environment value", []string{"plan"}, []string{"TF_CLI_ARGS_plan=-input=false -detailed-exitcode='true'"}, true}, + {"double-quoted global value", []string{"plan"}, []string{`TF_CLI_ARGS=-detailed-exitcode="true"`}, true}, + {"quoted false disables", []string{"plan"}, []string{"TF_CLI_ARGS_plan=-detailed-exitcode", "TF_CLI_ARGS=-detailed-exitcode='false'"}, false}, + {"tab separates flags", []string{"plan"}, []string{"TF_CLI_ARGS_plan=-input=false\t-detailed-exitcode"}, true}, + {"newline separates global flags", []string{"plan"}, []string{"TF_CLI_ARGS=-input=false\n-detailed-exitcode"}, true}, + {"tab separated false disables", []string{"plan"}, []string{"TF_CLI_ARGS_plan=-detailed-exitcode", "TF_CLI_ARGS=-lock=false\t-detailed-exitcode=false"}, false}, + {"quoted tab stays inside one word", []string{"plan"}, []string{"TF_CLI_ARGS_plan=-var 'a\t-detailed-exitcode'"}, false}, + {"malformed value contributes no flags", []string{"plan"}, []string{`TF_CLI_ARGS_plan=-detailed-exitcode "unterminated`}, false}, + } { + t.Run(test.name, func(t *testing.T) { + require.Equal(t, test.want, detailedExitCodeEnabled("plan", test.args, test.env)) + }) + } + t.Setenv("TF_CLI_ARGS_plan", "-detailed-exitcode") + require.True(t, detailedExitCodeEnabled("plan", []string{"plan"}, nil), "the inherited process environment applies") +} + +func TestMainCommandLegacyRetryDetailedPlan(t *testing.T) { + t.Setenv("TF_CLI_ARGS", "") + t.Setenv("TF_CLI_ARGS_plan", "") + for _, test := range []struct { + name string + args []string + env []string + extra schema.ComponentExtraArguments + processEnv []string + globalEnv map[string]string + code int + success bool + }{ + {name: "global plan environment", args: []string{"plan"}, globalEnv: map[string]string{"TF_CLI_ARGS_plan": "-detailed-exitcode"}, code: 2, success: true}, + {name: "global environment overrides process", args: []string{"plan"}, processEnv: []string{"TF_CLI_ARGS=-detailed-exitcode=false"}, globalEnv: map[string]string{"TF_CLI_ARGS": "-detailed-exitcode"}, code: 2, success: true}, + {name: "component overrides global", args: []string{"plan"}, globalEnv: map[string]string{"TF_CLI_ARGS": "-detailed-exitcode"}, env: []string{"TF_CLI_ARGS=-detailed-exitcode=false"}, code: 2}, + {name: "command flag", args: []string{"plan", "-detailed-exitcode"}, code: 2, success: true}, + {name: "extra arguments", args: []string{"plan"}, extra: schema.ComponentExtraArguments{Arguments: []string{"-detailed-exitcode"}}, code: 2, success: true}, + {name: "extra environment", args: []string{"plan"}, extra: schema.ComponentExtraArguments{Environment: map[string]string{"TF_CLI_ARGS_plan": "-detailed-exitcode"}}, code: 2, success: true}, + {name: "component environment", args: []string{"plan"}, env: []string{"TF_CLI_ARGS=-detailed-exitcode"}, code: 2, success: true}, + {name: "process environment", args: []string{"plan"}, processEnv: []string{"TF_CLI_ARGS_plan=-detailed-exitcode"}, code: 2, success: true}, + {name: "flag disabled", args: []string{"plan", "-detailed-exitcode=false"}, env: []string{"TF_CLI_ARGS_plan=-detailed-exitcode"}, code: 2}, + {name: "flag absent", args: []string{"plan"}, code: 2}, + {name: "real failure", args: []string{"plan", "-detailed-exitcode"}, code: 1}, + {name: "apply failure", args: []string{"apply"}, env: []string{"TF_CLI_ARGS=-detailed-exitcode"}, code: 2}, + } { + t.Run(test.name, func(t *testing.T) { + info := detailedExitCodeInfo() + info.SubCommand = test.args[0] + info.ComponentEnvList = test.env + test.extra.Commands = []string{info.SubCommand} + info.ComponentExecution.ExtraArguments = []schema.ComponentExtraArguments{test.extra} + info.ComponentExecution.AfterHooks[0].Commands = []string{info.SubCommand} + info.ComponentExecution.ErrorHooks[0].Commands = []string{info.SubCommand} + info.ComponentRetrySection = &schema.RetryConfig{MaxAttempts: intPtr(2), Conditions: []string{"/.*/"}} + attempts := 0 + shell := newShellFixture(t, func(call *shellCall) error { + if call.command == "terraform" { + attempts++ + _, writeErr := io.WriteString(call.options.StdoutCapture, "plan output\n") + require.NoError(t, writeErr) + return exitCodeError(test.code) + } + return nil + }) + var opts []component.ExecutionOption + if test.processEnv != nil { + opts = append(opts, component.WithEnvironment(test.processEnv)) + } + err := hookExecutor(shell).runMainShellCommand(&schema.AtmosConfiguration{Env: test.globalEnv}, info, test.args, t.TempDir(), opts) + require.Equal(t, test.code, errUtils.GetExitCode(err)) + if test.success { + require.Equal(t, 1, attempts) + require.Equal(t, []string{"terraform plan", "hook-after arg"}, shell.names()) + } else { + require.Equal(t, 2, attempts) + require.Equal(t, "hook-alert", shell.names()[len(shell.names())-1]) + } + }) + } +} + // TestPolicyPlanUsesProviderOutputArguments verifies that a plan policy selects and binds // the output path Terraform receives, including one supplied only by provider extra arguments. func TestPolicyPlanUsesProviderOutputArguments(t *testing.T) { @@ -712,21 +908,13 @@ func TestDetailedExitCodeEnabled(t *testing.T) { {name: "empty value", args: []string{"-detailed-exitcode="}, want: false}, {name: "last wins disabled", args: []string{"-detailed-exitcode", "-detailed-exitcode=false"}, want: false}, {name: "last wins enabled", args: []string{"-detailed-exitcode=false", "-detailed-exitcode=true"}, want: true}, - {name: "unparsable after enabled", args: []string{"-detailed-exitcode", "-detailed-exitcode=x"}, want: false}, + // Terraform rejects an invalid boolean, so the value is ignored and the earlier setting stands. + {name: "unparsable after enabled is ignored", args: []string{"-detailed-exitcode", "-detailed-exitcode=x"}, want: true}, {name: "similar flag", args: []string{"-detailed-exitcode-extra=true"}, want: false}, } for _, tt := range tests { t.Run(tt.name, func(t *testing.T) { - require.Equal(t, tt.want, detailedExitCodeEnabled(tt.args)) + require.Equal(t, tt.want, detailedExitCodeEnabled(subcommandPlan, tt.args, nil)) }) } } - -// TestIsExplicitDetailedPlanChangesHonorsExplicitValue checks the plan predicate end to end. -func TestIsExplicitDetailedPlanChangesHonorsExplicitValue(t *testing.T) { - info := &schema.ConfigAndStacksInfo{SubCommand: subcommandPlan} - err := errUtils.ExitCodeError{Code: detailedExitCodeChangesDetected} - require.True(t, isExplicitDetailedPlanChanges(info, []string{"-detailed-exitcode=true"}, err)) - require.False(t, isExplicitDetailedPlanChanges(info, []string{"-detailed-exitcode=false"}, err)) - require.False(t, isExplicitDetailedPlanChanges(info, []string{"-detailed-exitcode", "-detailed-exitcode=false"}, err)) -} diff --git a/pkg/component/terraform/execute_init_settings.go b/pkg/component/terraform/execute_init_settings.go new file mode 100644 index 00000000000..a55a8bcd34f --- /dev/null +++ b/pkg/component/terraform/execute_init_settings.go @@ -0,0 +1,101 @@ +package terraform + +import ( + "maps" + "path/filepath" + "slices" + "strings" + + "github.com/cloudposse/atmos/pkg/component/execution" + "github.com/cloudposse/atmos/pkg/schema" + "github.com/cloudposse/atmos/pkg/terraform/autoinit" +) + +// initFileFlags are the init options whose value can name a file Terraform reads during init. +var initFileFlags = []string{"-backend-config", "-var-file"} + +// withProviderInitSettings adds the init settings that provider extra arguments contribute to smart +// init's inputs: the effective init arguments and environment, and the contents of files those +// arguments name. Without them, an edited init-only backend file would leave a stale initialization +// trusted. Values only contribute to the fingerprint digest; they are never recorded or logged. +// Settings that cannot be evaluated return nil inputs, so no recorded marker is trusted; the init +// phase itself then reports the problem. +func withProviderInitSettings(in *autoinit.Inputs, info *schema.ConfigAndStacksInfo, componentPath string) *autoinit.Inputs { + config := info.ComponentExecution + if in == nil || config.IsEmpty() { + return in + } + applied, err := execution.ApplyExtraArguments(config.ExtraArguments, &execution.ArgumentsRequest{ + Command: subcommandInit, Args: []string{subcommandInit}, InsertAt: 1, WorkingDirectory: componentPath, + }) + if err != nil { + return nil + } + in.EnvLookup = overlayEnvironment(applied.Environment, in.EnvLookup) + extra := maps.Clone(in.Extra) + if extra == nil { + extra = map[string]string{} + } + arguments := applied.Args[1:] + extra["provider-init-arguments"] = strings.Join(arguments, "\x00") + extra["provider-init-environment"] = strings.Join(applied.Environment, "\x00") + for _, path := range initFileArguments(arguments) { + if !filepath.IsAbs(path) { + path = filepath.Join(componentPath, path) + } + // A missing or unreadable file is recorded as such, so creating it changes the digest. + digest, err := fileDigest(path) + if err != nil { + digest = "unreadable" + } + extra["provider-init-file:"+path] = digest + } + in.Extra = extra + return in +} + +// overlayEnvironment looks up KEY=value entries first and then base. A nil base reports a key +// absent, so the caller falls back to the process environment. +func overlayEnvironment(entries []string, base func(string) (string, bool)) func(string) (string, bool) { + environment := map[string]string{} + for _, entry := range entries { + if key, value, ok := strings.Cut(entry, "="); ok { + environment[key] = value + } + } + return func(key string) (string, bool) { + if value, ok := environment[key]; ok { + return value, true + } + if base == nil { + return "", false + } + return base(key) + } +} + +// initFileArguments returns the file paths named by -backend-config and -var-file options, in +// the "-flag=value" and "-flag value" forms, each with a single or double dash. A -backend-config value holding "=" sets one +// setting instead of naming a file; it is already part of the arguments. +func initFileArguments(arguments []string) []string { + var paths []string + for i := 0; i < len(arguments); i++ { + name, value, inline := strings.Cut(strings.TrimLeft(arguments[i], "-"), "=") + name = "-" + name + if !slices.Contains(initFileFlags, name) { + continue + } + if !inline { + if i+1 >= len(arguments) { + break + } + i++ + value = arguments[i] + } + if name == "-backend-config" && strings.Contains(value, "=") { + continue + } + paths = append(paths, value) + } + return paths +} diff --git a/pkg/component/terraform/execute_pipeline.go b/pkg/component/terraform/execute_pipeline.go index f378756072e..dc753f760e2 100644 --- a/pkg/component/terraform/execute_pipeline.go +++ b/pkg/component/terraform/execute_pipeline.go @@ -3,18 +3,21 @@ package terraform import ( "bytes" "context" + "crypto/sha256" + "encoding/hex" "errors" "fmt" + "io/fs" + "maps" + "os" "path/filepath" "slices" - "strconv" "strings" errUtils "github.com/cloudposse/atmos/errors" auth "github.com/cloudposse/atmos/pkg/auth" "github.com/cloudposse/atmos/pkg/ci" "github.com/cloudposse/atmos/pkg/component" - "github.com/cloudposse/atmos/pkg/component/configuration" "github.com/cloudposse/atmos/pkg/dependencies" log "github.com/cloudposse/atmos/pkg/logger" "github.com/cloudposse/atmos/pkg/policy/planfile" @@ -62,48 +65,27 @@ func (e *Executor) prepareComponentExecution( if err != nil { return nil, err } + // A restored plugin plan was validated when it was created. Its binding omits + // variable values, so validating the reconstructed component would test empty vars. + restoredConfigurationPlan := prepared != nil && isRestoredConfigurationPlan(info) - var componentPath string - if prepared != nil && usesConfigurationPlan(info) { - // A reviewed plan uses the retained workdir; provisioning could replace its modules. - componentPath = ConstructTerraformComponentWorkingDir(atmosConfig, info) - } else { - componentPath, err = e.resolveAndProvisionComponentPath(ctx, writers, atmosConfig, info) - if err != nil { - return nil, err - } - } - - if prepared != nil && !info.DryRun { - if err := rejectRuntimeArtifacts(prepared.Files); err != nil { - return nil, err - } - if err := configuration.WriteFiles(componentPath, prepared.Files); err != nil { - return nil, err - } - } - - if err = checkComponentRestrictions(info); err != nil { + componentPath, err := e.resolveConfiguredComponentPath(ctx, writers, atmosConfig, info, prepared) + if err != nil { return nil, err } if shouldProcess && tenv == nil { - tenv, err = resolveAndInstallToolchainDeps(atmosConfig, info) - if err != nil { + if tenv, err = installToolchainForCommand(atmosConfig, info); err != nil { return nil, err } - info.Command = tenv.Resolve(info.Command) } varFile := ConstructTerraformComponentVarfileName(info) - testVarFile := "" - if hasTerraformTestVars(info) { - testVarFile = ConstructTerraformComponentTestVarfileName(info) - } + testVarFile := terraformTestVarfileName(info) planFile := ConstructTerraformComponentPlanfileName(info) workingDir := ConstructTerraformComponentWorkingDir(atmosConfig, info) - if err = e.runPreExecutionSteps(atmosConfig, info, workingDir, tenv); err != nil { + if err = e.runPreExecutionSteps(atmosConfig, info, workingDir, tenv, restoredConfigurationPlan); err != nil { return nil, err } @@ -118,6 +100,64 @@ func (e *Executor) prepareComponentExecution( }, nil } +// terraformTestVarfileName returns the test var file name, or "" when the component has no test vars. +func terraformTestVarfileName(info *schema.ConfigAndStacksInfo) string { + if !hasTerraformTestVars(info) { + return "" + } + return ConstructTerraformComponentTestVarfileName(info) +} + +// isRestoredConfigurationPlan reports whether the invocation applies a provider-owned saved plan. +func isRestoredConfigurationPlan(info *schema.ConfigAndStacksInfo) bool { + return ownsConfigurationPlan(info) && usesConfigurationPlan(info) && restoresConfigurationPlan(info) +} + +// installToolchainForCommand installs the toolchain dependencies and resolves the command against them. +func installToolchainForCommand(atmosConfig *schema.AtmosConfiguration, info *schema.ConfigAndStacksInfo) (*dependencies.ToolchainEnvironment, error) { + tenv, err := resolveAndInstallToolchainDeps(atmosConfig, info) + if err != nil { + return nil, err + } + info.Command = tenv.Resolve(info.Command) + return tenv, nil +} + +// resolveConfiguredComponentPath resolves the component directory and, for a provider-owned +// component, writes its prepared files there. +func (e *Executor) resolveConfiguredComponentPath( + ctx context.Context, + writers provisioner.OutputWriters, + atmosConfig *schema.AtmosConfiguration, + info *schema.ConfigAndStacksInfo, + prepared *schema.PreparedComponentConfiguration, +) (string, error) { + var componentPath string + if prepared != nil && usesConfigurationPlan(info) { + // A reviewed plan uses the retained workdir; provisioning could replace its modules. + componentPath = ConstructTerraformComponentWorkingDir(atmosConfig, info) + } else { + var err error + componentPath, err = e.resolveAndProvisionComponentPath(ctx, writers, atmosConfig, info) + if err != nil { + return "", err + } + } + + if prepared != nil && !info.DryRun { + if err := rejectRuntimeArtifacts(prepared.Files); err != nil { + return "", err + } + if err := writePreparedConfiguration(componentPath, prepared); err != nil { + return "", fmt.Errorf("component %s in stack %s, working directory %s: %w", info.ComponentFromArg, info.Stack, componentPath, err) + } + } + if err := checkComponentRestrictions(info); err != nil { + return "", err + } + return componentPath, nil +} + // runPreExecutionSteps performs validation, auth pre-hook, config file generation, // env var warnings, and env var assembly. Extracted from prepareComponentExecution // to keep cyclomatic complexity within bounds. @@ -126,6 +166,7 @@ func (e *Executor) runPreExecutionSteps( info *schema.ConfigAndStacksInfo, workingDir string, tenv *dependencies.ToolchainEnvironment, + restoredConfigurationPlan bool, ) error { // `(computed)` is an inspection-only degradation marker. Never permit it to // reach Terraform via a generated tfvars file or TF_VAR_ environment value. @@ -142,21 +183,16 @@ func (e *Executor) runPreExecutionSteps( return err } - if err := e.validateTerraformComponent(atmosConfig, info); err != nil { - return err + if !restoredConfigurationPlan { + if err := e.validateTerraformComponent(atmosConfig, info); err != nil { + return err + } } if !schema.ConfigurationOwned(info.ComponentSettingsSection) || info.DryRun { - if err := auth.TerraformPreHook(atmosConfig, info); err != nil { - // The error is returned and rendered once by the command boundary, so it is only logged - // at debug level here to avoid printing the same failure twice. - log.Debug("Error executing 'atmos auth terraform pre-hook'", - logFieldComponent, info.ComponentFromArg, "error", err) - // Pre-hook failures terminate execution — this matches the original terraform.go behavior. - // Authentication setup failures must not silently produce unauthenticated terraform commands. + if err := runAuthPreHook(atmosConfig, info); err != nil { return err } - disableTerraformPluginCacheForExecution(atmosConfig, info) } // Generate backend and provider-override files. When AutoGenerateFiles=true, @@ -169,6 +205,12 @@ func (e *Executor) runPreExecutionSteps( warnOnConflictingEnvVars() + return finishPreExecutionEnvironment(atmosConfig, info, tenv) +} + +// finishPreExecutionEnvironment assembles the environment: provider-owned components check their +// workspace and append secrets, every other component gets the full environment assembly. +func finishPreExecutionEnvironment(atmosConfig *schema.AtmosConfiguration, info *schema.ConfigAndStacksInfo, tenv *dependencies.ToolchainEnvironment) error { if schema.ConfigurationOwned(info.ComponentSettingsSection) && !info.DryRun { if err := checkConfigurationWorkspace(info); err != nil { return err @@ -179,6 +221,21 @@ func (e *Executor) runPreExecutionSteps( return assembleComponentEnvVars(atmosConfig, info, tenv) } +// runAuthPreHook runs the auth pre-hook and disables the Terraform plugin cache for this execution. +func runAuthPreHook(atmosConfig *schema.AtmosConfiguration, info *schema.ConfigAndStacksInfo) error { + if err := auth.TerraformPreHook(atmosConfig, info); err != nil { + // The error is returned and rendered once by the command boundary, so it is only logged + // at debug level here to avoid printing the same failure twice. + log.Debug("Error executing 'atmos auth terraform pre-hook'", + logFieldComponent, info.ComponentFromArg, "error", err) + // Pre-hook failures terminate execution — this matches the original terraform.go behavior. + // Authentication setup failures must not silently produce unauthenticated terraform commands. + return err + } + disableTerraformPluginCacheForExecution(atmosConfig, info) + return nil +} + // executeCommandPipeline runs the full terraform command pipeline after the component // has been prepared: optional init pre-step, argument construction, workspace setup, // TTY guard, and final command execution + cleanup. @@ -194,30 +251,45 @@ func terraformPhaseLabel(info *schema.ConfigAndStacksInfo, phase string) string return strings.TrimSpace(tool + " " + phase) } +// runGroupedInitPhase runs the init pre-step, when needed, and returns the possibly updated component path. +func (e *Executor) runGroupedInitPhase( + atmosConfig *schema.AtmosConfiguration, + info *schema.ConfigAndStacksInfo, + execCtx *componentExecContext, + componentPath string, + opts ...component.ExecutionOption, +) (string, error) { + if !shouldRunTerraformInit(atmosConfig, info) { + return componentPath, nil + } + // Phase-level CI log grouping (Dimension "phase"): fold the `init` phase + // into its own collapsible group. A no-op unless ci.groups.mode is "auto" + // and this is the outermost Atmos invocation (a terraform run nested inside + // a workflow step stays flat inside the step's group). + err := ci.Group(atmosConfig, ci.DimensionPhase, terraformPhaseLabel(info, subcommandInit), func() error { + var phaseErr error + componentPath, phaseErr = e.executeTerraformInitPhase(atmosConfig, info, componentPath, execCtx.varFile, opts...) + return phaseErr + }) + if err != nil { + return componentPath, errUtils.AsPrerequisiteError(err) + } + return componentPath, nil +} + func (e *Executor) executeCommandPipeline( atmosConfig *schema.AtmosConfiguration, info *schema.ConfigAndStacksInfo, execCtx *componentExecContext, opts ...component.ExecutionOption, ) error { - componentPath := execCtx.componentPath if info.DryRun && schema.ConfigurationOwned(info.ComponentSettingsSection) { ui.Warning("Dry-run preview may omit Terragrunt extra arguments, required var files, and environment variables because Terragrunt configuration is not evaluated") } - if shouldRunTerraformInit(atmosConfig, info) { - // Phase-level CI log grouping (Dimension "phase"): fold the `init` phase - // into its own collapsible group. A no-op unless ci.groups.mode is "auto" - // and this is the outermost Atmos invocation (a terraform run nested inside - // a workflow step stays flat inside the step's group). - err := ci.Group(atmosConfig, ci.DimensionPhase, terraformPhaseLabel(info, subcommandInit), func() error { - var phaseErr error - componentPath, phaseErr = e.executeTerraformInitPhase(atmosConfig, info, componentPath, execCtx.varFile, opts...) - return phaseErr - }) - if err != nil { - return errUtils.AsPrerequisiteError(err) - } + componentPath, err := e.runGroupedInitPhase(atmosConfig, info, execCtx, execCtx.componentPath, opts...) + if err != nil { + return err } handleDeploySubcommand(atmosConfig, info) @@ -274,7 +346,7 @@ func (e *Executor) runMainCommandPhase( //nolint:revive // argument-limit: every // this, incidental init/workspace-select output (e.g. a stray "No changes." // lookalike) can poison the exec-metadata parser's extraction even though the // real plan/apply's own output is correct (research.md Decision 32). - if err := validateEnvironmentPlanOutput(atmosConfig, info, execCtx); err != nil { + if err := validateEnvironmentPlanOutput(atmosConfig, info, execCtx, opts...); err != nil { return err } allArgsAndFlags, restore, err := expandPolicyPlanArguments(info, execCtx, allArgsAndFlags, componentPath) @@ -303,13 +375,14 @@ func (e *Executor) runMainCommandPhase( //nolint:revive // argument-limit: every return e.executeGovernedTerraformCommand(atmosConfig, info, allArgsAndFlags, componentPath, uploadStatusFlag, execCtx.planPolicy, mainOpts...) }) info.ExecMetadataRawOutput = combineExecMetadataOutput(&execMetadataStdoutBuf, &execMetadataStderrBuf) - if mainErr != nil && !lifecycleOwnsRecovery(info) { + // A completed -detailed-exitcode plan reports changes with status 2 after writing the plan, so + // it is neither recovered nor returned before the reviewed plan is recorded. A component with + // provider-supplied hooks already recovered inside its lifecycle. + if mainErr != nil && !completedPlanWithChanges(mainErr) && !lifecycleOwnsRecovery(info) { mainErr = e.recoverFromInitRequired(atmosConfig, info, execCtx, allArgsAndFlags, componentPath, uploadStatusFlag, mainErr, &execMetadataStdoutBuf, &execMetadataStderrBuf, append(slices.Clone(opts), captureArgs)...) } - // An explicit -detailed-exitcode plan reports changes with status 2 after writing the - // plan, so the reviewed plan is recorded before that status is returned. - if mainErr != nil && !isExplicitDetailedPlanChanges(info, effectiveArgs, mainErr) { + if mainErr != nil && !completedPlanWithChanges(mainErr) { return mainErr } // Policy plans run against a private artifact that is removed after the guard @@ -324,145 +397,11 @@ func (e *Executor) runMainCommandPhase( //nolint:revive // argument-limit: every return mainErr } -// planSnapshot is the configuration a saved plan is bound to: the provider identity and the -// digest of the retained working directory, excluding the plan and the state Terraform writes. -type planSnapshot struct { - path, identity, workingDigest string - // engineDigest is the digest of the Terraform executable observed around the subprocess, so a hook - // that replaces the binary afterwards cannot move the binding onto the replacement. It is empty when - // the executable could not be read, and the binding then falls back to its own observation. - engineDigest string - // planDigest is the digest of the plan file Terraform wrote, taken before any after hook runs. It is - // empty when no digest was observed, and it is not part of the before-and-after comparison. - planDigest string -} - -// planSnapshotKey carries the plan snapshotter to the main command's subprocess. -type planSnapshotKey struct{} - -// planSnapshotter observes the configuration around the Terraform subprocess, after the before -// hooks and before the after hooks. A hook that rewrites the module after Terraform read it then -// cannot move the binding onto the rewritten bytes. -type planSnapshotter struct { - ctx context.Context - config *schema.AtmosConfiguration - info *schema.ConfigAndStacksInfo - execCtx *componentExecContext - pathFrom func(phaseArgs []string) string - result *planSnapshot - err error -} - -// takePlanSnapshot fingerprints the configuration with the inputs the binding record uses. -func takePlanSnapshot(ctx context.Context, config *schema.AtmosConfiguration, info *schema.ConfigAndStacksInfo, execCtx *componentExecContext, path string) (*planSnapshot, error) { - runtime, err := planRuntime(config) - if err != nil { - return nil, err - } - identity, err := runtime.PlanIdentity(ctx, info, execCtx.preparedConfiguration.Sources) - if err != nil { - return nil, err - } - digest, err := configurationWorkdirDigest(execCtx.workingDir, path) - if err != nil { - return nil, err - } - engineDigest, _ := fileDigest(info.Command) - return &planSnapshot{path: path, identity: identity, workingDigest: digest, engineDigest: engineDigest}, nil -} - -// around wraps one run of the Terraform subprocess. The configuration must be the same before and -// after it; the unchanged snapshot is kept for the binding, and a changed one rejects the plan. -func (s *planSnapshotter) around(phaseArgs []string, run func() error) func() error { - return func() error { - s.result, s.err = nil, nil - path := s.pathFrom(phaseArgs) - if path == "" { - return run() - } - before, err := takePlanSnapshot(s.ctx, s.config, s.info, s.execCtx, path) - if err != nil { - return err - } - runErr := run() - after, err := takePlanSnapshot(s.ctx, s.config, s.info, s.execCtx, path) - switch { - case err != nil: - s.err = err - case *after != *before: - s.err = fmt.Errorf("%w: configuration changed while Terraform ran", errUtils.ErrPlanVerificationFailed) - default: - // Terraform writes the plan itself unless a policy gate publishes it later, so the plan is - // fingerprinted now and an after hook cannot move the binding onto rewritten bytes. - if s.execCtx.planPolicy == nil { - if digest, digestErr := fileDigest(path); digestErr == nil { - after.planDigest = digest - } - } - s.result = after - } - return runErr - } -} - -// reviewedSnapshot returns the configuration Terraform read, or takes a fresh snapshot when no -// observed run recorded one for this plan. -func (c *componentExecContext) reviewedSnapshot(ctx context.Context, config *schema.AtmosConfiguration, info *schema.ConfigAndStacksInfo, path string) (*planSnapshot, error) { - if s := c.planSnapshots; s != nil { - if s.err != nil { - return nil, s.err - } - if s.result != nil && s.result.path == path { - return s.result, nil - } - } - return takePlanSnapshot(ctx, config, info, c, path) -} - -// observePlanSnapshots returns ctx carrying a snapshotter for a plan this invocation binds. Policy -// plans publish the artifact at the original output path, so that path is the one excluded from the -// digest. The returned context is unchanged when no plan is bound. -func (c *componentExecContext) observePlanSnapshots(ctx context.Context, config *schema.AtmosConfiguration, info *schema.ConfigAndStacksInfo, policyArgs []string) context.Context { - c.planSnapshots = nil - if !bindsPlan(info, c) { - return ctx - } - c.planSnapshots = &planSnapshotter{ctx: ctx, config: config, info: info, execCtx: c, pathFrom: func(phaseArgs []string) string { - if c.planPolicy != nil { - phaseArgs = policyArgs - } - return plannedOutputPath(c.componentPath, phaseArgs) - }} - return context.WithValue(ctx, planSnapshotKey{}, c.planSnapshots) -} - -// isExplicitDetailedPlanChanges reports plan's "changes present" status from -detailed-exitcode. -// A failed hook is never that status, even when the hook itself exited with 2. -func isExplicitDetailedPlanChanges(info *schema.ConfigAndStacksInfo, args []string, err error) bool { - if errors.Is(err, errUtils.ErrComponentHookFailed) { - return false - } - return info.SubCommand == subcommandPlan && detailedExitCodeEnabled(args) && resolveExitCode(err) == detailedExitCodeChangesDetected -} - -// detailedExitCodeEnabled reports whether args enable -detailed-exitcode. It accepts the bare flag -// and the explicit boolean forms (-detailed-exitcode=true, --detailed-exitcode=false), and the last -// occurrence wins, matching Go flag semantics. An unparsable value counts as not enabled. -func detailedExitCodeEnabled(args []string) bool { - enabled := false - for _, arg := range args { - name, value, hasValue := strings.Cut(arg, "=") - if name != detailedExitCodeFlag && name != "-"+detailedExitCodeFlag { - continue - } - if !hasValue { - enabled = true - continue - } - parsed, err := strconv.ParseBool(value) - enabled = err == nil && parsed - } - return enabled +// completedPlanWithChanges accepts only the completed phase itself, not a joined +// or wrapped failure that also contains a status-two result. +func completedPlanWithChanges(err error) bool { + var changes *successfulPlanChanges + return errors.As(err, &changes) && err == changes //nolint:err113,errorlint // Identity is required: errors.Is would also accept joined failures. } // recoverFromInitRequired is smart init's plan/apply-time safety net: when the main command @@ -629,3 +568,97 @@ func addTerraformTestVarfileArg(info *schema.ConfigAndStacksInfo, testVarFile st } info.AdditionalArgsAndFlags = append([]string{varFileFlag, testVarFile}, info.AdditionalArgsAndFlags...) } + +// withoutSensitiveContent copies files, dropping the bytes of every sensitive one. +func withoutSensitiveContent(files []schema.ComponentConfigurationFile) []schema.ComponentConfigurationFile { + stored := slices.Clone(files) + for i := range stored { + if stored[i].Sensitive { + stored[i].Content = nil + } + } + return stored +} + +// restoreSensitiveContent reads the bytes of each sensitive file back from the working directory and +// restores them only when they are the reviewed ones. The working directory digest excludes some +// locations, so the reviewed bytes are verified here against the digest recorded with the plan. +func restoreSensitiveContent(workdir string, files []schema.ComponentConfigurationFile, reviewed string) error { + contents, err := readSensitiveFiles(workdir, files) + if err != nil { + return err + } + if contents == nil { + return nil + } + for i := range files { + // A file the entry removes is absent by design, and the digest still binds that absence. + if files[i].Sensitive && contents[files[i].Path] == nil && !slices.Contains([]string{"remove", "remove_terragrunt"}, files[i].IfExists) { + return fmt.Errorf("%w: reviewed generated file %s is missing: %w", errUtils.ErrPlanVerificationFailed, files[i].Path, fs.ErrNotExist) + } + } + if digestSensitiveContents(contents) != reviewed { + return fmt.Errorf("%w: reviewed generated file changed", errUtils.ErrPlanVerificationFailed) + } + for i := range files { + if files[i].Sensitive { + files[i].Content = contents[files[i].Path] + } + } + return nil +} + +// sensitiveDigest fingerprints the sensitive generated files as they are in the working directory. +// It is empty when no file is sensitive. +func sensitiveDigest(workdir string, files []schema.ComponentConfigurationFile) (string, error) { + contents, err := readSensitiveFiles(workdir, files) + if err != nil || contents == nil { + return "", err + } + return digestSensitiveContents(contents), nil +} + +// readSensitiveFiles reads every sensitive file below workdir, keyed by path. A missing file maps +// to nil content. It returns nil when no file is sensitive. +func readSensitiveFiles(workdir string, files []schema.ComponentConfigurationFile) (map[string][]byte, error) { + if !slices.ContainsFunc(files, func(file schema.ComponentConfigurationFile) bool { return file.Sensitive }) { + return nil, nil + } + root, err := os.OpenRoot(workdir) + if err != nil { + return nil, planIOError(err) + } + defer root.Close() + contents := make(map[string][]byte) + for i := range files { + if !files[i].Sensitive { + continue + } + content, err := root.ReadFile(files[i].Path) + if errors.Is(err, fs.ErrNotExist) { + contents[files[i].Path] = nil + continue + } + if err != nil { + return nil, fmt.Errorf("%w: reviewed generated file %s cannot be read: %w", errUtils.ErrPlanVerificationFailed, files[i].Path, err) + } + if content == nil { + content = []byte{} + } + contents[files[i].Path] = content + } + return contents, nil +} + +// digestSensitiveContents hashes the paths and bytes of the sensitive files in a stable order. +func digestSensitiveContents(contents map[string][]byte) string { + hash := sha256.New() + for _, path := range slices.Sorted(maps.Keys(contents)) { + if contents[path] == nil { + fmt.Fprintf(hash, "%s\x00absent\n", path) + continue + } + fmt.Fprintf(hash, "%s\x00%x\n", path, sha256.Sum256(contents[path])) + } + return hex.EncodeToString(hash.Sum(nil)) +} diff --git a/pkg/component/terraform/mock_configuration_provider_test.go b/pkg/component/terraform/mock_configuration_provider_test.go index d214c882de3..ea520f76b30 100644 --- a/pkg/component/terraform/mock_configuration_provider_test.go +++ b/pkg/component/terraform/mock_configuration_provider_test.go @@ -1,9 +1,9 @@ // Code generated by MockGen. DO NOT EDIT. -// Source: github.com/cloudposse/atmos/pkg/plugin/adapter/configuration (interfaces: Provider) +// Source: configuration_source_pipeline_test.go // // Generated by this command: // -// mockgen -destination=pkg/component/terraform/mock_configuration_provider_test.go -package=terraform -mock_names=Provider=MockConfigurationProvider github.com/cloudposse/atmos/pkg/plugin/adapter/configuration Provider +// mockgen -source=configuration_source_pipeline_test.go -destination=mock_configuration_provider_test.go -package=terraform -mock_names=configurationProvider=MockConfigurationProvider // // Package terraform is a generated GoMock package. @@ -17,7 +17,7 @@ import ( gomock "go.uber.org/mock/gomock" ) -// MockConfigurationProvider is a mock of Provider interface. +// MockConfigurationProvider is a mock of configurationProvider interface. type MockConfigurationProvider struct { ctrl *gomock.Controller recorder *MockConfigurationProviderMockRecorder diff --git a/pkg/git/worktree_signal.go b/pkg/git/worktree_signal.go new file mode 100644 index 00000000000..327aba8cfdb --- /dev/null +++ b/pkg/git/worktree_signal.go @@ -0,0 +1,107 @@ +package git + +import ( + "os" + "os/signal" + "sync" + "syscall" + + "github.com/cloudposse/atmos/pkg/perf" +) + +// signalExitBase is the conventional exit status offset for a process ended by a signal. +const signalExitBase = 128 + +// WorktreeSignalCleanups removes temporary worktrees when the process receives SIGINT or SIGTERM. +// SIGINT and SIGTERM skip deferred functions, so a worktree and its `.git/worktrees` registration +// would otherwise leak. One watcher owns every active cleanup, so none can exit ahead of another. +type WorktreeSignalCleanups struct { + mu sync.Mutex + watcher *worktreeSignalWatcher + // exit ends the process with a status; callers inject it so tests can observe it. + exit func(code int) +} + +type worktreeSignalWatcher struct { + signals chan os.Signal + done chan struct{} + cleanups map[*worktreeSignalCleanup]struct{} +} + +type worktreeSignalCleanup struct { + run func() +} + +// NewWorktreeSignalCleanups returns cleanups that end the process with exit after a signal. +func NewWorktreeSignalCleanups(exit func(code int)) *WorktreeSignalCleanups { + return &WorktreeSignalCleanups{exit: exit} +} + +// Register runs cleanup on SIGINT or SIGTERM, together with every other registered cleanup, before +// exiting with the conventional 128+signal status. Call the returned function after normal cleanup. +func (c *WorktreeSignalCleanups) Register(cleanup func()) func() { + defer perf.Track(nil, "git.WorktreeSignalCleanups.Register")() + + c.mu.Lock() + defer c.mu.Unlock() + watcher := c.watcher + if watcher == nil { + watcher = &worktreeSignalWatcher{ + signals: make(chan os.Signal, 1), + done: make(chan struct{}), + cleanups: make(map[*worktreeSignalCleanup]struct{}), + } + c.watcher = watcher + signal.Notify(watcher.signals, syscall.SIGINT, syscall.SIGTERM) + go c.watch(watcher) + } + key := &worktreeSignalCleanup{run: cleanup} + watcher.cleanups[key] = struct{}{} + return sync.OnceFunc(func() { + c.mu.Lock() + defer c.mu.Unlock() + delete(watcher.cleanups, key) + if len(watcher.cleanups) == 0 { + signal.Stop(watcher.signals) + close(watcher.done) + c.watcher = nil + } + }) +} + +func (c *WorktreeSignalCleanups) watch(watcher *worktreeSignalWatcher) { + select { + case received := <-watcher.signals: + c.terminate(watcher, received) + case <-watcher.done: + // A signal that arrived before the final release still terminates the process. + select { + case received := <-watcher.signals: + c.terminate(watcher, received) + default: + } + } +} + +// terminate runs the cleanups still registered and exits with the signal's status. Normal release +// may already have retired this watcher and run every cleanup; the received signal still ends the +// process with its documented status instead of letting the command report success. A signal +// queued for a retired watcher can arrive after another registration started a new watcher, so the +// cleanups that are active now run too, or their worktrees and Git registrations would leak when +// the process exits. +func (c *WorktreeSignalCleanups) terminate(watcher *worktreeSignalWatcher, received os.Signal) { + c.mu.Lock() + defer c.mu.Unlock() + active := c.watcher + if active == nil { + active = watcher + } + for cleanup := range active.cleanups { + cleanup.run() + } + code := 1 + if number, ok := received.(syscall.Signal); ok { + code = signalExitBase + int(number) + } + c.exit(code) +} diff --git a/pkg/git/worktree_signal_test.go b/pkg/git/worktree_signal_test.go new file mode 100644 index 00000000000..63d51dc54be --- /dev/null +++ b/pkg/git/worktree_signal_test.go @@ -0,0 +1,51 @@ +package git + +import ( + "syscall" + "testing" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" +) + +// A signal queued for a watcher that its final release retired still cleans up the registrations +// that another caller made afterwards, instead of exiting and leaking their worktrees. +func TestRetiredWatcherSignalCleansActiveWorktrees(t *testing.T) { + var exits []int + cleanups := NewWorktreeSignalCleanups(func(code int) { exits = append(exits, code) }) + var cleanedA, cleanedB int + + releaseA := cleanups.Register(func() { cleanedA++ }) + cleanups.mu.Lock() + retired := cleanups.watcher + cleanups.mu.Unlock() + require.NotNil(t, retired) + releaseA() + + releaseB := cleanups.Register(func() { cleanedB++ }) + defer releaseB() + + cleanups.terminate(retired, syscall.SIGTERM) + + assert.Equal(t, 1, cleanedB, "the registration made after the first watcher retired is cleaned") + assert.Zero(t, cleanedA, "a released registration is not cleaned again") + assert.Equal(t, []int{signalExitBase + int(syscall.SIGTERM)}, exits) +} + +// A retired watcher with nothing registered afterwards runs no cleanup but still ends the process. +func TestRetiredWatcherSignalWithoutActiveWorktreesStillExits(t *testing.T) { + var exits []int + cleanups := NewWorktreeSignalCleanups(func(code int) { exits = append(exits, code) }) + cleaned := 0 + + release := cleanups.Register(func() { cleaned++ }) + cleanups.mu.Lock() + retired := cleanups.watcher + cleanups.mu.Unlock() + release() + + cleanups.terminate(retired, syscall.SIGINT) + + assert.Zero(t, cleaned) + assert.Equal(t, []int{signalExitBase + int(syscall.SIGINT)}, exits) +} diff --git a/pkg/plugin/adapter/component/calls.go b/pkg/plugin/adapter/component/calls.go index caabe91b79e..6de1487c165 100644 --- a/pkg/plugin/adapter/component/calls.go +++ b/pkg/plugin/adapter/component/calls.go @@ -116,8 +116,8 @@ func (p *Provider) ResolveState(ctx context.Context, input *pluginv1.ComponentRe if result == nil { return errUtils.ErrPluginProtocol } - if len(result.Diagnostics) != 0 { - return errUtils.ErrPluginInvalidConfig + if err := firstDiagnosticError(result.Diagnostics); err != nil { + return err } if !validState(result.State) || !validSources(result.Sources) { return errUtils.ErrPluginProtocol diff --git a/pkg/plugin/adapter/component/calls_test.go b/pkg/plugin/adapter/component/calls_test.go index dc0db2f681d..f68ea0e1c61 100644 --- a/pkg/plugin/adapter/component/calls_test.go +++ b/pkg/plugin/adapter/component/calls_test.go @@ -4,9 +4,12 @@ import ( "context" "os" "path/filepath" + "strings" "testing" "time" + "github.com/santhosh-tekuri/jsonschema/v5" + "github.com/stretchr/testify/assert" "github.com/stretchr/testify/require" "github.com/zclconf/go-cty/cty" ctyjson "github.com/zclconf/go-cty/cty/json" @@ -262,7 +265,8 @@ func TestDiscoveryRejectsInvalidOwnershipIdentityAndProvenance(t *testing.T) { require.ErrorIs(t, p.checkDiscovery(nil), errUtils.ErrPluginProtocol) require.ErrorIs(t, p.checkDiscovery(&pluginv1.DiscoverComponentsResponse{Components: []*pluginv1.DiscoveredComponent{nil}}), errUtils.ErrPluginProtocol) require.ErrorIs(t, p.checkDiscovery(&pluginv1.DiscoverComponentsResponse{Components: []*pluginv1.DiscoveredComponent{unit, unit}}), errUtils.ErrDuplicatePluginExtension) - require.ErrorIs(t, p.checkDiscovery(&pluginv1.DiscoverComponentsResponse{Diagnostics: []*pluginv1.Diagnostic{{Message: "failure"}}}), errUtils.ErrPluginInvalidConfig) + // Diagnostics never invalidate the response; the discovery session interprets them per unit. + require.NoError(t, p.checkDiscovery(&pluginv1.DiscoverComponentsResponse{Diagnostics: []*pluginv1.Diagnostic{{Message: "failure"}}})) } func TestPreparationRejectsUnsafeArtifactsAndMissingStateIdentity(t *testing.T) { @@ -371,3 +375,54 @@ func TestUnsupportedAndInvalidCallsDoNotStartExecutable(t *testing.T) { _, err = p.Discover(ctx, &pluginv1.DiscoverComponentsRequest{RootDirectory: request.RootDirectory}) require.ErrorIs(t, err, context.Canceled) } + +const settingsSchema = `{ + "type": "object", + "properties": { + "enabled": {"type": "boolean"}, + "mode": {"const": "native"}, + "discovery": {"type": "object", "properties": {"enabled": {"type": "boolean"}}, "additionalProperties": false} + }, + "required": ["enabled"], + "additionalProperties": false +}` + +func providerWithSchema(t *testing.T) *Provider { + t.Helper() + compiler := jsonschema.NewCompiler() + require.NoError(t, compiler.AddResource("settings.json", strings.NewReader(settingsSchema))) + compiled, err := compiler.Compile("settings.json") + require.NoError(t, err) + return &Provider{name: "terragrunt", schema: compiled} +} + +func TestSettingsViolationsNameKeysAndHideValues(t *testing.T) { + provider := providerWithSchema(t) + require.NoError(t, provider.ValidateSettings(map[string]any{"enabled": true, "mode": "native"})) + + for name, test := range map[string]struct { + settings map[string]any + want string + }{ + "misspelled key": {map[string]any{"enabled": true, "discoverey": map[string]any{}}, "components.terraform.terragrunt.discoverey"}, + "nested unknown": {map[string]any{"enabled": true, "discovery": map[string]any{"included": "x"}}, "components.terraform.terragrunt.discovery.included"}, + "missing": {map[string]any{}, "components.terraform.terragrunt.enabled"}, + "wrong type": {map[string]any{"enabled": "hunter2"}, "`components.terraform.terragrunt.enabled` has the wrong type"}, + "bad const": {map[string]any{"enabled": true, "mode": "hunter2"}, "`components.terraform.terragrunt.mode` has an unsupported value"}, + } { + t.Run(name, func(t *testing.T) { + err := provider.ValidateSettings(test.settings) + require.ErrorIs(t, err, errUtils.ErrPluginInvalidConfig) + text := strings.Join(errUtils.AllDetails(err), "\n") + assert.Contains(t, text, test.want) + assert.NotContains(t, text, "hunter2", "schema messages may echo values and must stay out") + assert.NotEmpty(t, errUtils.AllHints(err)) + }) + } +} + +func TestNonSchemaSettingsErrorStillNamesTheSection(t *testing.T) { + err := providerWithSchema(t).settingsError(assert.AnError) + require.ErrorIs(t, err, errUtils.ErrPluginInvalidConfig) + assert.Contains(t, strings.Join(errUtils.AllDetails(err), "\n"), "components.terraform.terragrunt") +} diff --git a/pkg/plugin/adapter/component/diagnostics.go b/pkg/plugin/adapter/component/diagnostics.go new file mode 100644 index 00000000000..51b84a37730 --- /dev/null +++ b/pkg/plugin/adapter/component/diagnostics.go @@ -0,0 +1,203 @@ +package component + +import ( + "errors" + "fmt" + "strings" + + "google.golang.org/grpc/codes" + "google.golang.org/grpc/status" + + errUtils "github.com/cloudposse/atmos/errors" + atmosio "github.com/cloudposse/atmos/pkg/io" + log "github.com/cloudposse/atmos/pkg/logger" + "github.com/cloudposse/atmos/pkg/perf" + pluginv1 "github.com/cloudposse/atmos/pkg/plugin/proto/atmos/plugin/v1" + "github.com/cloudposse/atmos/pkg/ui" +) + +// debugHint tells the user how to reveal the masked diagnostic a plugin marked as not safe to print. +const debugHint = "Run with `ATMOS_LOGS_LEVEL=Debug` to log the full diagnostic from the plugin, with secrets masked." + +// Failure is a plugin failure that carries the plugin's structured description. Its message is the +// plugin's headline with secrets masked; it unwraps to the Atmos sentinel for the failure class. +type Failure struct { + Detail *pluginv1.ErrorDetail + sentinel error +} + +// Error returns the masked headline. +// +//nolint:lintroller // Trivial accessor; exempt from performance tracking. +func (f *Failure) Error() string { return atmosio.MaskString(headline(f.Detail)) } + +// Unwrap exposes the sentinel so errors.Is keeps classifying the failure. +// +//nolint:lintroller // Trivial accessor; exempt from performance tracking. +func (f *Failure) Unwrap() error { return f.sentinel } + +func headline(detail *pluginv1.ErrorDetail) string { + if detail.GetSummary() != "" { + return detail.GetSummary() + } + if detail.GetCode() != "" { + return detail.GetCode() + } + return errUtils.ErrPluginAPI.Error() +} + +func sentinelFor(code codes.Code) error { + switch code { + case codes.Unimplemented: + return errUtils.ErrPluginUnsupportedExtension + case codes.InvalidArgument, codes.FailedPrecondition, codes.NotFound: + return errUtils.ErrPluginInvalidConfig + default: + return errUtils.ErrPluginAPI + } +} + +// detailFromStatus extracts the first structured plugin failure carried by a gRPC status error. +func detailFromStatus(err error) (*pluginv1.ErrorDetail, bool) { + var carrier interface{ GRPCStatus() *status.Status } + if !errors.As(err, &carrier) { + return nil, false + } + for _, item := range carrier.GRPCStatus().Details() { + if detail, ok := item.(*pluginv1.ErrorDetail); ok { + return detail, true + } + } + return nil, false +} + +// DetailError renders a plugin's structured failure with the Atmos error builder: the headline as the +// message, an explanation naming the unit, stack, file, setting, and operation, the plugin's hints, and +// the detail text when the plugin marked it safe. Detail text that is not safe stays out of the error +// and goes to the masked debug log. Every field passes through secret masking before display. +func DetailError(detail *pluginv1.ErrorDetail, class codes.Code) error { + defer perf.Track(nil, "componentadapter.DetailError")() + + if detail == nil { + return errUtils.ErrPluginAPI + } + builder := errUtils.Build(&Failure{Detail: detail, sentinel: sentinelFor(class)}) + if where := describeWhere(detail); where != "" { + builder.WithExplanation(where) + } + if detail.UserSafe && detail.Detail != "" { + builder.WithExplanation(mask(detail.Detail)) + } + for _, hint := range detail.Hints { + builder.WithHint(mask(hint)) + } + if logUnsafeDetail(detail) { + builder.WithHint(debugHint) + } + if location := detail.GetLocation(); location.GetFile() != "" { + // Mask before attaching: annotation escaping can hide a mask literal from the final masker. + builder.WithLocation(mask(location.File), int(location.Line)) + } + return builder.WithContext("plugin_error_code", detail.Code).Err() +} + +func mask(text string) string { return atmosio.MaskString(text) } + +// logUnsafeDetail logs a detail the plugin did not mark safe at Debug level, masked, and reports +// whether it did, so the caller can point at the debug log. +func logUnsafeDetail(detail *pluginv1.ErrorDetail) bool { + if detail.UserSafe || detail.Detail == "" { + return false + } + log.Debug("Plugin diagnostic", "code", detail.Code, "unit", detail.Unit, "stack", detail.Stack, "operation", detail.Operation, "detail", mask(detail.Detail)) + return true +} + +// describeWhere states where the problem is, using only the always-safe fields. +func describeWhere(detail *pluginv1.ErrorDetail) string { + var parts []string + if detail.Unit != "" { + unit := fmt.Sprintf("Component `%s`", detail.Unit) + if detail.Stack != "" { + unit += fmt.Sprintf(" in stack `%s`", detail.Stack) + } + parts = append(parts, unit) + } + if file := detail.GetLocation().GetFile(); file != "" { + parts = append(parts, "defined in `"+formatLocation(detail.Location)+"`") + } + if detail.ConfigKey != "" { + parts = append(parts, fmt.Sprintf("setting `%s`", detail.ConfigKey)) + } + if detail.Operation != "" { + parts = append(parts, "during "+detail.Operation) + } + return mask(strings.Join(parts, ", ")) +} + +func formatLocation(location *pluginv1.SourceLocation) string { + text := location.File + if location.Line > 0 { + text += fmt.Sprintf(":%d", location.Line) + if location.Column > 0 { + text += fmt.Sprintf(":%d", location.Column) + } + } + return text +} + +// DiagnosticError converts one plugin diagnostic into an error, or nil for a warning. +func DiagnosticError(diagnostic *pluginv1.Diagnostic) error { + defer perf.Track(nil, "componentadapter.DiagnosticError")() + + detail := diagnostic.GetDetail() + if detail == nil { + // The legacy form carries a field and a message the plugin may not have vetted, so the + // message is kept as unsafe detail: logged masked at Debug level, never shown. + detail = &pluginv1.ErrorDetail{Code: "invalid_configuration", Summary: "Plugin rejected the configuration", ConfigKey: diagnostic.GetField(), Detail: diagnostic.GetMessage()} + } + if detail.Severity == pluginv1.Severity_SEVERITY_WARNING { + return nil + } + return DetailError(detail, codes.InvalidArgument) +} + +// firstDiagnosticError returns the first error-severity diagnostic as an error. Warnings are +// shown on the UI channel and never fail the call. +func firstDiagnosticError(diagnostics []*pluginv1.Diagnostic) error { + var failure error + for _, diagnostic := range diagnostics { + if err := DiagnosticError(diagnostic); err != nil { + if failure == nil { + failure = err + } + continue + } + WarnDiagnostic(diagnostic) + } + return failure +} + +// WarnDiagnostic prints a warning-severity diagnostic on the UI channel with secrets masked. +func WarnDiagnostic(diagnostic *pluginv1.Diagnostic) { + defer perf.Track(nil, "componentadapter.WarnDiagnostic")() + + detail := diagnostic.GetDetail() + if detail == nil || detail.Severity != pluginv1.Severity_SEVERITY_WARNING { + return + } + text := headline(detail) + if where := describeWhere(detail); where != "" { + text += " (" + where + ")" + } + if detail.UserSafe && detail.Detail != "" { + text += ": " + detail.Detail + } + ui.Warning(mask(text)) + for _, hint := range detail.Hints { + ui.Hint(mask(hint)) + } + if logUnsafeDetail(detail) { + ui.Hint(debugHint) + } +} diff --git a/pkg/plugin/adapter/component/diagnostics_test.go b/pkg/plugin/adapter/component/diagnostics_test.go new file mode 100644 index 00000000000..3197d2b4dda --- /dev/null +++ b/pkg/plugin/adapter/component/diagnostics_test.go @@ -0,0 +1,133 @@ +package component + +import ( + "strings" + "testing" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + "google.golang.org/grpc/codes" + "google.golang.org/grpc/status" + "google.golang.org/protobuf/proto" + + errUtils "github.com/cloudposse/atmos/errors" + atmosio "github.com/cloudposse/atmos/pkg/io" + pluginv1 "github.com/cloudposse/atmos/pkg/plugin/proto/atmos/plugin/v1" +) + +func prepareWith(t *testing.T, command string) error { + t.Helper() + request := &pluginv1.ComponentRequest{RootDirectory: t.TempDir(), Address: fixtureAddress(), Operation: &pluginv1.ComponentOperation{Command: command}} + _, err := fixtureProvider(t).Validate(t.Context(), request) + return err +} + +func TestStructuredFailureRendersSafeDetail(t *testing.T) { + err := prepareWith(t, "detail-safe") + require.ErrorIs(t, err, errUtils.ErrPluginUnsupportedExtension) + assert.Equal(t, "Unsupported Terragrunt setting", err.Error()) + + text := strings.Join(errUtils.AllDetails(err), "\n") + for _, want := range []string{"Component `vpc` in stack `live/dev`", "live/dev/vpc/terragrunt.hcl:7:3", "setting `terraform_binary`", "during prepare", "terraform_binary is not implemented."} { + assert.Contains(t, text, want) + } + assert.Equal(t, []string{"Remove terraform_binary."}, errUtils.AllHints(err)) + location, found := errUtils.GetLocation(err) + require.True(t, found) + assert.Equal(t, "live/dev/vpc/terragrunt.hcl", location.File) + assert.Equal(t, 7, location.Line) +} + +func TestStructuredFailureWithholdsUnsafeDetail(t *testing.T) { + err := prepareWith(t, "detail-unsafe") + require.ErrorIs(t, err, errUtils.ErrPluginUnsupportedExtension) + rendered := err.Error() + strings.Join(errUtils.AllDetails(err), "\n") + assert.NotContains(t, rendered, "ghp_SECRET") + hints := errUtils.AllHints(err) + require.Len(t, hints, 2) + assert.Equal(t, "Remove terraform_binary.", hints[0]) + assert.Contains(t, hints[1], "ATMOS_LOGS_LEVEL=Debug") +} + +func TestErrorSeverityDiagnosticFailsAndWarningDoesNot(t *testing.T) { + request := &pluginv1.ComponentRequest{RootDirectory: t.TempDir(), Address: fixtureAddress(), Operation: &pluginv1.ComponentOperation{Command: "diagnostic-error"}} + _, err := fixtureProvider(t).Prepare(t.Context(), request) + require.ErrorIs(t, err, errUtils.ErrPluginInvalidConfig) + assert.Equal(t, "Bad setting", err.Error()) + + warning := &pluginv1.Diagnostic{Detail: &pluginv1.ErrorDetail{Code: "skipped", Summary: "Skipped", Severity: pluginv1.Severity_SEVERITY_WARNING}} + require.NoError(t, firstDiagnosticError([]*pluginv1.Diagnostic{warning})) + require.ErrorIs(t, firstDiagnosticError([]*pluginv1.Diagnostic{warning, {Field: "x", Message: "legacy"}}), errUtils.ErrPluginInvalidConfig) +} + +func TestMissingDetailFallsBackToSentinel(t *testing.T) { + require.ErrorIs(t, DetailError(nil, 0), errUtils.ErrPluginAPI) +} + +// A legacy diagnostic keeps its message as unsafe detail: never shown, but pointed at by the debug hint. +func TestLegacyDiagnosticKeepsItsMessageForDebugLogging(t *testing.T) { + err := DiagnosticError(&pluginv1.Diagnostic{Field: "region", Message: "region ghp_LEGACY is invalid"}) + require.ErrorIs(t, err, errUtils.ErrPluginInvalidConfig) + rendered := err.Error() + strings.Join(errUtils.AllDetails(err), "\n") + assert.NotContains(t, rendered, "ghp_LEGACY") + hints := errUtils.AllHints(err) + require.NotEmpty(t, hints) + assert.Contains(t, hints[len(hints)-1], "ATMOS_LOGS_LEVEL=Debug") + var failure *Failure + require.ErrorAs(t, err, &failure) + assert.Equal(t, "region ghp_LEGACY is invalid", failure.Detail.Detail) + assert.False(t, failure.Detail.UserSafe) +} + +// The attached location is masked before annotation escaping can hide a mask literal from the +// final masker. +func TestDiagnosticLocationIsMaskedBeforeItIsAttached(t *testing.T) { + t.Cleanup(atmosio.Reset) + atmosio.Reset() + require.NoError(t, atmosio.Initialize()) + atmosio.RegisterSecret("tok,en-7d3a") + err := DetailError(&pluginv1.ErrorDetail{Code: "bad", Summary: "Bad", Location: &pluginv1.SourceLocation{File: "live/tok,en-7d3a/terragrunt.hcl", Line: 3}}, 0) + location, found := errUtils.GetLocation(err) + require.True(t, found) + assert.NotContains(t, location.File, "tok,en-7d3a") + assert.Equal(t, 3, location.Line) +} + +// An InvalidArgument failure that carries a structured detail stays on the structured path: the plugin's +// safe detail reaches the user and an unsafe detail does not. A bare InvalidArgument status still +// discloses nothing from its RPC message. +func TestInvalidArgumentStructuredDetailKeepsSafeContract(t *testing.T) { + for _, safe := range []bool{true, false} { + detail := &pluginv1.ErrorDetail{Code: "evaluation_failed", Summary: "Evaluation failed", Location: &pluginv1.SourceLocation{File: "live/app/terragrunt.hcl", Line: 2}, UserSafe: safe} + if safe { + detail.Detail = "locals.region is not defined." + } else { + detail.Detail = "token ghp_INVALIDARG leaked" + } + wire, err := status.New(codes.InvalidArgument, "secret RPC payload").WithDetails(detail) + require.NoError(t, err) + // Round-trip the status protobuf as the RPC transport does. + data, err := proto.Marshal(wire.Proto()) + require.NoError(t, err) + decoded := wire.Proto() + require.NoError(t, proto.Unmarshal(data, decoded)) + + result := invocationError(status.FromProto(decoded).Err()) + require.ErrorIs(t, result, errUtils.ErrPluginInvalidConfig) + var failure *Failure + require.ErrorAs(t, result, &failure, "structured detail must not degrade to a bare sentinel") + rendered := result.Error() + strings.Join(errUtils.AllDetails(result), "\n") + assert.Equal(t, "Evaluation failed", result.Error()) + assert.NotContains(t, rendered, "secret RPC payload") + assert.Contains(t, rendered, "live/app/terragrunt.hcl:2") + if safe { + assert.Contains(t, rendered, "locals.region is not defined.") + } else { + assert.NotContains(t, rendered, "ghp_INVALIDARG") + } + } + + bare := invocationError(status.Error(codes.InvalidArgument, "secret RPC payload")) + require.ErrorIs(t, bare, errUtils.ErrPluginAPI) + assert.NotContains(t, bare.Error(), "secret") +} diff --git a/pkg/plugin/adapter/component/invoke.go b/pkg/plugin/adapter/component/invoke.go index cba19908f9e..ccf20ce679f 100644 --- a/pkg/plugin/adapter/component/invoke.go +++ b/pkg/plugin/adapter/component/invoke.go @@ -49,6 +49,10 @@ func invocationError(err error) error { return err } } + // A structured detail is the plugin's own vetted description of the failure. + if detail, ok := detailFromStatus(err); ok { + return DetailError(detail, status.Code(err)) + } switch status.Code(err) { case codes.Canceled: return errors.Join(errUtils.ErrPluginAPI, context.Canceled) @@ -104,10 +108,7 @@ func (p *Provider) validateOperation(ctx context.Context, handle *native.Handle, if result == nil { return errUtils.ErrPluginProtocol } - if len(result.Diagnostics) != 0 { - return errUtils.ErrPluginInvalidConfig - } - return nil + return firstDiagnosticError(result.Diagnostics) } // discoveryDiagnostic accepts only the versioned discovery contract. Messages, diff --git a/pkg/plugin/adapter/component/main_test.go b/pkg/plugin/adapter/component/main_test.go index 7fa6f709d7f..b75fe05ede2 100644 --- a/pkg/plugin/adapter/component/main_test.go +++ b/pkg/plugin/adapter/component/main_test.go @@ -76,10 +76,31 @@ func (*componentFixture) Validate(ctx context.Context, request *pluginv1.Compone return nil, status.Error(codes.Internal, "must-not-disclose-secret-value") case "unsupported": return nil, status.Error(codes.Unimplemented, "unavailable") + case "detail-safe", "detail-unsafe": + return nil, fixtureFailure(request.GetOperation().GetCommand() == "detail-safe") + case "diagnostic-error": + return &pluginv1.ValidateResponse{Diagnostics: []*pluginv1.Diagnostic{{Field: "mode", Message: "Bad setting", Detail: &pluginv1.ErrorDetail{Code: "bad", Summary: "Bad setting", ConfigKey: "mode"}}}}, nil } return &pluginv1.ValidateResponse{}, nil } +func fixtureFailure(safe bool) error { + detail := &pluginv1.ErrorDetail{ + Code: "unsupported_setting", Summary: "Unsupported Terragrunt setting", Operation: "prepare", Unit: "vpc", Stack: "live/dev", + ConfigKey: "terraform_binary", Location: &pluginv1.SourceLocation{File: "live/dev/vpc/terragrunt.hcl", Line: 7, Column: 3}, Hints: []string{"Remove terraform_binary."}, UserSafe: safe, + } + if safe { + detail.Detail = "terraform_binary is not implemented." + } else { + detail.Detail = "token ghp_SECRETSECRETSECRET leaked" + } + withDetail, err := status.New(codes.Unimplemented, detail.Summary).WithDetails(detail) + if err != nil { + return err + } + return withDetail.Err() +} + func fixtureAddress() *pluginv1.ComponentAddress { return &pluginv1.ComponentAddress{Stack: "live", Component: "dev/app", ComponentType: "terraform"} } diff --git a/pkg/plugin/adapter/component/registry.go b/pkg/plugin/adapter/component/registry.go index a2a411c57c5..dd3261a7703 100644 --- a/pkg/plugin/adapter/component/registry.go +++ b/pkg/plugin/adapter/component/registry.go @@ -144,8 +144,8 @@ func (p *Provider) ComponentType() string { return p.componentType } //nolint:lintroller // Pure lookup or validation; exempt from performance tracking. func (p *Provider) ValidateSettings(settings map[string]any) error { if err := p.schema.Validate(settings); err != nil { - // Schema diagnostics can contain secret field values; do not expose them. - return fmt.Errorf("%w: settings for configuration provider %s", errUtils.ErrPluginInvalidConfig, p.name) + // Schema diagnostics can contain secret field values; report only the keys involved. + return p.settingsError(err) } return nil } diff --git a/pkg/plugin/adapter/component/responses.go b/pkg/plugin/adapter/component/responses.go index d92afc9bf04..2992e48f950 100644 --- a/pkg/plugin/adapter/component/responses.go +++ b/pkg/plugin/adapter/component/responses.go @@ -45,9 +45,8 @@ func (p *Provider) checkDiscovery(result *pluginv1.DiscoverComponentsResponse) e if result == nil { return errUtils.ErrPluginProtocol } - if len(result.Diagnostics) != 0 { - return errUtils.ErrPluginInvalidConfig - } + // Diagnostics are per-component or project-level findings that the discovery session + // interprets; they never invalidate the healthy components in the same response. seen := make(map[[3]string]bool) for _, item := range result.Components { if !p.validDiscoveredComponent(item) { @@ -73,8 +72,8 @@ func checkPreparation(result *pluginv1.PrepareComponentResponse) error { if result == nil { return errUtils.ErrPluginProtocol } - if len(result.Diagnostics) != 0 { - return errUtils.ErrPluginInvalidConfig + if err := firstDiagnosticError(result.Diagnostics); err != nil { + return err } if result.Engine == "" || result.ModuleSource == "" || !filepath.IsAbs(result.WorkingDirectory) || !validState(result.State) || !validSources(result.Sources) || !validDependencies(result.Dependencies) { return errUtils.ErrPluginProtocol diff --git a/pkg/plugin/adapter/component/settings_error.go b/pkg/plugin/adapter/component/settings_error.go new file mode 100644 index 00000000000..b5ee5e9e133 --- /dev/null +++ b/pkg/plugin/adapter/component/settings_error.go @@ -0,0 +1,75 @@ +package component + +import ( + "errors" + "regexp" + "strings" + + "github.com/santhosh-tekuri/jsonschema/v5" + + errUtils "github.com/cloudposse/atmos/errors" +) + +// keySeparator joins the parts of a configuration key. +const keySeparator = "." + +// quotedName matches a property name that a schema message quotes, as in "additionalProperties 'x' not allowed". +var quotedName = regexp.MustCompile(`'([^']+)'`) + +// settingsError describes a schema violation by the keys involved. Schema messages can echo the +// offending value, so only key names, the failing keyword, and the section are reported. +func (p *Provider) settingsError(err error) error { + section := "components.terraform." + p.name + builder := errUtils.Build(errUtils.ErrPluginInvalidConfig).WithContext("config_section", section) + var violation *jsonschema.ValidationError + if !errors.As(err, &violation) { + return builder.WithExplanationf("The settings under `%s` do not match the schema of configuration provider `%s`.", section, p.name).Err() + } + seen := map[string]bool{} + for _, leaf := range leaves(violation) { + line := describeViolation(section, leaf) + if seen[line] { + continue + } + seen[line] = true + builder.WithExplanation(line) + } + return builder.WithHintf("Fix the setting named above. `atmos config set ` edits atmos.yaml without loading plugin settings, so it works even while this error is shown.").Err() +} + +func leaves(err *jsonschema.ValidationError) []*jsonschema.ValidationError { + if len(err.Causes) == 0 { + return []*jsonschema.ValidationError{err} + } + var result []*jsonschema.ValidationError + for _, cause := range err.Causes { + result = append(result, leaves(cause)...) + } + return result +} + +func describeViolation(section string, leaf *jsonschema.ValidationError) string { + key := section + strings.ReplaceAll(leaf.InstanceLocation, "/", keySeparator) + keyword := leaf.KeywordLocation[strings.LastIndex(leaf.KeywordLocation, "/")+1:] + switch keyword { + case "additionalProperties": + if name := quotedName.FindStringSubmatch(leaf.Message); name != nil { + return "Unknown setting `" + joinKey(key, name[1]) + "`. Check the spelling." + } + return "Unknown setting under `" + key + "`. Check the spelling." + case "required": + if name := quotedName.FindStringSubmatch(leaf.Message); name != nil { + return "Missing required setting `" + joinKey(key, name[1]) + "`." + } + return "A required setting is missing under `" + key + "`." + case "type": + return "`" + key + "` has the wrong type." + default: + return "`" + key + "` has an unsupported value." + } +} + +// joinKey appends a property name to a key path, tolerating a trailing separator. +func joinKey(key, name string) string { + return strings.TrimSuffix(key, keySeparator) + keySeparator + name +} diff --git a/pkg/plugin/adapter/configuration/discovery.go b/pkg/plugin/adapter/configuration/discovery.go index 697d8684a81..de4d3c92960 100644 --- a/pkg/plugin/adapter/configuration/discovery.go +++ b/pkg/plugin/adapter/configuration/discovery.go @@ -2,7 +2,6 @@ package configuration import ( "context" - "fmt" "path/filepath" errUtils "github.com/cloudposse/atmos/errors" @@ -20,35 +19,88 @@ func addressKey(a *pluginv1.ComponentAddress) [3]string { } func (s *Session) discover(ctx context.Context) error { - names := make([]string, 0, len(s.requests)) - for _, request := range s.requests { - names = append(names, request.Provider) + if s.configErr != nil { + return s.configErr + } + definitions := make(map[string]map[string]any) + instances := make(map[[3]string]*pluginv1.DiscoveredComponent) + if err := s.checkProviderOptions(ctx); err != nil { + return err + } + if len(s.requests) == 0 { + // Only explicit imports are in play and they were resolved with the stack manifests. + s.definitions, s.instances = definitions, instances + return nil } - registry, err := s.loader(ctx, names) + registry, err := s.providers(ctx) if err != nil { return err } - definitions := make(map[string]map[string]any) - instances := make(map[[3]string]*pluginv1.DiscoveredComponent) + failures, err := s.discoverRequests(ctx, registry, definitions, instances) + if err != nil { + return err + } + // A unit with a missing or failed dependency fails alone; every healthy unit stays usable. + settleFailures(definitions, instances, failures) + + s.definitions, s.instances, s.failures = definitions, instances, failures + return nil +} + +// checkProviderOptions validates the provider options against the registered providers when any are set. +func (s *Session) checkProviderOptions(ctx context.Context) error { + if !s.hasProviderOptions() { + return nil + } + registry, err := s.providers(ctx) + if err != nil { + return err + } + return s.validateOptions(registry) +} + +// discoverRequests asks each requested provider for its units and records them, collecting per-unit failures. +func (s *Session) discoverRequests( + ctx context.Context, + registry map[string]Provider, + definitions map[string]map[string]any, + instances map[[3]string]*pluginv1.DiscoveredComponent, +) (map[failedKey]*failedUnit, error) { + failures := make(map[failedKey]*failedUnit) for _, request := range s.requests { provider, exists := registry[request.Provider] if !exists || provider.ComponentType() != terraformType { - return errUtils.ErrPluginUnsupportedExtension + return nil, errUtils.ErrPluginUnsupportedExtension } discovered, err := provider.Discover(ctx, request) if err != nil { - return err + return nil, err } - if err := addDiscovered(definitions, instances, discovered); err != nil { - return err + if err := recordDiagnostics(request.Provider, discovered.Diagnostics, failures); err != nil { + return nil, err + } + if err := addDiscovered(definitions, instances, s.withoutImported(discovered)); err != nil { + return nil, err } } - if err := checkDependencies(instances); err != nil { - return err - } + return failures, nil +} - s.registry, s.definitions, s.instances = registry, definitions, instances - return nil +// withoutImported drops units that a stack imported explicitly: an imported unit is a +// component of the importing stack, not also a component of a derived stack. +func (s *Session) withoutImported(discovered *pluginv1.DiscoverComponentsResponse) *pluginv1.DiscoverComponentsResponse { + s.importMu.lock() + defer s.importMu.unlock() + if len(s.imported) == 0 { + return discovered + } + kept := &pluginv1.DiscoverComponentsResponse{Diagnostics: discovered.Diagnostics} + for _, instance := range discovered.Components { + if s.imported[instance.Source] == nil { + kept.Components = append(kept.Components, instance) + } + } + return kept } func addInstance(definitions map[string]map[string]any, instance *pluginv1.DiscoveredComponent) error { @@ -56,12 +108,31 @@ func addInstance(definitions map[string]map[string]any, instance *pluginv1.Disco if !filepath.IsLocal(address.Stack) || !filepath.IsLocal(address.Component) { return errUtils.ErrPluginProtocol } + config, err := instanceConfig(instance, false) + if err != nil { + return err + } + definition := definitions[address.Stack] + if definition == nil { + definition = map[string]any{"name": address.Stack, componentsKey: map[string]any{terraformType: map[string]any{}}} + definitions[address.Stack] = definition + } + definition[componentsKey].(map[string]any)[terraformType].(map[string]any)[address.Component] = config + return nil +} + +// instanceConfig builds the component section for one discovered unit. A unit imported into +// a stack depends on its siblings in that stack, so its dependencies carry no stack. +func instanceConfig(instance *pluginv1.DiscoveredComponent, imported bool) (map[string]any, error) { + if !filepath.IsLocal(instance.Address.Component) { + return nil, errUtils.ErrPluginProtocol + } if instance.Engine != terraformType && instance.Engine != "opentofu" { - return errUtils.ErrPluginUnsupportedExtension + return nil, errUtils.ErrPluginUnsupportedExtension } config, err := adapter.DecodeObject(instance.Configuration) if err != nil { - return err + return nil, err } settings, _ := config["settings"].(map[string]any) if settings == nil { @@ -74,28 +145,12 @@ func addInstance(definitions map[string]map[string]any, instance *pluginv1.Disco configuration["provider"], configuration["engine"], configuration["source"] = instance.Provider, instance.Engine, instance.Source settings["configuration"] = configuration config["settings"] = settings - config["dependencies"] = mergeDependencies(config["dependencies"], instanceDependencies(instance)) - definition := definitions[address.Stack] - if definition == nil { - definition = map[string]any{"name": address.Stack, componentsKey: map[string]any{terraformType: map[string]any{}}} - definitions[address.Stack] = definition - } - definition[componentsKey].(map[string]any)[terraformType].(map[string]any)[address.Component] = config - return nil + // Dependencies the unit declares in its configuration merge with the discovered ones. + config["dependencies"] = mergeDependencies(config["dependencies"], instanceDependencies(instance, imported)) + return config, nil } -func checkDependencies(instances map[[3]string]*pluginv1.DiscoveredComponent) error { - for _, instance := range instances { - for _, dep := range instance.Dependencies { - if _, exists := instances[addressKey(dep.Target)]; !exists { - return fmt.Errorf("%w: dependency %s/%s is outside discovery scope", errUtils.ErrPluginInvalidConfig, dep.Target.Stack, dep.Target.Component) - } - } - } - return nil -} - -func instanceDependencies(instance *pluginv1.DiscoveredComponent) map[string]any { +func instanceDependencies(instance *pluginv1.DiscoveredComponent, imported bool) map[string]any { deps := map[string]any{"files": []any{}, "folders": []any{}, componentsKey: []any{}} for _, source := range instance.Sources { field := "files" @@ -105,7 +160,11 @@ func instanceDependencies(instance *pluginv1.DiscoveredComponent) map[string]any deps[field] = append(deps[field].([]any), source.Path) } for _, dependency := range instance.Dependencies { - deps[componentsKey] = append(deps[componentsKey].([]any), map[string]any{"name": dependency.Target.Component, "stack": dependency.Target.Stack, "kind": dependency.Target.ComponentType}) + target := map[string]any{"name": dependency.Target.Component, "kind": dependency.Target.ComponentType} + if !imported { + target["stack"] = dependency.Target.Stack + } + deps[componentsKey] = append(deps[componentsKey].([]any), target) } return deps } @@ -134,8 +193,13 @@ func mergeDependencies(declared any, discovered map[string]any) map[string]any { func addDiscovered(definitions map[string]map[string]any, instances map[[3]string]*pluginv1.DiscoveredComponent, discovered *pluginv1.DiscoverComponentsResponse) error { for _, instance := range discovered.Components { key := addressKey(instance.Address) - if _, exists := instances[key]; exists { - return errUtils.ErrDuplicatePluginExtension + if previous, exists := instances[key]; exists { + return errUtils.Build(errUtils.ErrDuplicatePluginExtension). + WithExplanationf("Component `%s` in stack `%s` is produced by both %s and %s.", key[2], key[0], previous.Source, instance.Source). + WithContext("component", key[2]). + WithContext("stack", key[0]). + WithHint("Remove one of the units. A symlinked `terragrunt.hcl` or overlapping `included_paths` entries can repeat a unit."). + Err() } if err := addInstance(definitions, instance); err != nil { return err diff --git a/pkg/plugin/adapter/configuration/failures.go b/pkg/plugin/adapter/configuration/failures.go new file mode 100644 index 00000000000..8559964659b --- /dev/null +++ b/pkg/plugin/adapter/configuration/failures.go @@ -0,0 +1,147 @@ +package configuration + +import ( + "context" + "maps" + "slices" + + errUtils "github.com/cloudposse/atmos/errors" + "github.com/cloudposse/atmos/pkg/perf" + adapter "github.com/cloudposse/atmos/pkg/plugin/adapter/component" + pluginv1 "github.com/cloudposse/atmos/pkg/plugin/proto/atmos/plugin/v1" +) + +// failedKey is the stack and component of a unit whose discovery failed. +type failedKey [2]string + +// failedUnit is a discovered unit that cannot run. It stays visible as a placeholder component +// so that commands naming it, and bulk runs that select it, fail with its own diagnostic while +// every healthy component keeps working. +type failedUnit struct { + provider, source string + err error +} + +// recordDiagnostics interprets a discovery response's diagnostics. Warnings print and never fail. +// An error naming a unit is held for the commands that target that unit; an error that names no +// unit is a project-level failure and is returned. +func recordDiagnostics(provider string, diagnostics []*pluginv1.Diagnostic, failures map[failedKey]*failedUnit) error { + var projectLevel error + for _, diagnostic := range diagnostics { + err := adapter.DiagnosticError(diagnostic) + if err == nil { + adapter.WarnDiagnostic(diagnostic) + continue + } + detail := diagnostic.GetDetail() + if detail.GetUnit() == "" || detail.GetStack() == "" { + if projectLevel == nil { + projectLevel = err + } + continue + } + key := failedKey{detail.Stack, detail.Unit} + if failures[key] == nil { + failures[key] = &failedUnit{provider: provider, source: detail.GetLocation().GetFile(), err: err} + } + } + return projectLevel +} + +// settleFailures removes units whose dependencies are missing or failed, then adds a placeholder +// component for every failed unit. Removal repeats until no healthy unit depends on a failure. +func settleFailures(definitions map[string]map[string]any, instances map[[3]string]*pluginv1.DiscoveredComponent, failures map[failedKey]*failedUnit) { + for { + broken := false + for _, key := range slices.SortedFunc(maps.Keys(instances), func(a, b [3]string) int { return slices.Compare(a[:], b[:]) }) { + instance := instances[key] + if err := brokenDependency(instance, instances, failures); err != nil { + failures[failedKey{instance.Address.Stack, instance.Address.Component}] = &failedUnit{provider: instance.Provider, source: instance.Source, err: err} + delete(instances, key) + removeComponent(definitions, instance.Address) + broken = true + } + } + if !broken { + break + } + } + for key, failure := range failures { + addPlaceholder(definitions, key, failure) + } +} + +func brokenDependency(instance *pluginv1.DiscoveredComponent, instances map[[3]string]*pluginv1.DiscoveredComponent, failures map[failedKey]*failedUnit) error { + for _, dep := range instance.Dependencies { + target := dep.Target.Stack + "/" + dep.Target.Component + if failure := failures[failedKey{dep.Target.Stack, dep.Target.Component}]; failure != nil { + return errUtils.Build(errUtils.ErrPluginInvalidConfig). + WithExplanationf("Component `%s` in stack `%s` depends on `%s`, which failed discovery: %s", instance.Address.Component, instance.Address.Stack, target, failure.err). + WithHintf("Fix `%s` first; run `atmos terraform plan %s -s %s` to see its error.", target, dep.Target.Component, dep.Target.Stack). + Err() + } + if _, exists := instances[addressKey(dep.Target)]; !exists { + return errUtils.Build(errUtils.ErrPluginInvalidConfig). + WithExplanationf("Component `%s` in stack `%s` (%s) depends on `%s`, which is not part of the discovered components.", instance.Address.Component, instance.Address.Stack, instance.Source, target). + WithHint("Include the dependency in `discovery.included_paths` and keep it out of `discovery.excluded_paths`."). + WithHint("A unit imported with `terragrunt://` belongs to the importing stack and is not discovered separately; import its dependents the same way."). + Err() + } + } + return nil +} + +func removeComponent(definitions map[string]map[string]any, address *pluginv1.ComponentAddress) { + if definition := definitions[address.Stack]; definition != nil { + if components, ok := definition[componentsKey].(map[string]any); ok { + if terraform, ok := components[terraformType].(map[string]any); ok { + delete(terraform, address.Component) + } + } + } +} + +// addPlaceholder declares a failed unit as a component that carries no variables and is marked +// failed, so that Prepare and the bulk pre-flight raise its diagnostic. +func addPlaceholder(definitions map[string]map[string]any, key failedKey, failure *failedUnit) { + definition := definitions[key[0]] + if definition == nil { + definition = map[string]any{"name": key[0], componentsKey: map[string]any{terraformType: map[string]any{}}} + definitions[key[0]] = definition + } + configuration := map[string]any{"provider": failure.provider, "engine": terraformType, "failed": true} + placeholder := map[string]any{ + "vars": map[string]any{}, + "settings": map[string]any{"configuration": configuration}, + } + if failure.source != "" { + configuration["source"] = failure.source + // Watch the unit's file, so an edit that leaves it broken still selects it as affected and + // a bulk run reports its diagnostic. + placeholder["dependencies"] = map[string]any{"files": []any{failure.source}} + } + definition[componentsKey].(map[string]any)[terraformType].(map[string]any)[key[1]] = placeholder +} + +// ComponentFailure returns the diagnostic of a unit that failed discovery, or nil for a healthy +// or unknown component. It waits for discovery so a bulk run can reject a broken unit before it +// runs any infrastructure command. +func (s *Session) ComponentFailure(ctx context.Context, stack, component string) error { + defer perf.Track(nil, "configurationadapter.ComponentFailure")() + + if err := s.ensure(ctx); err != nil { + return err + } + if failure := s.failures[failedKey{stack, component}]; failure != nil { + return failure.err + } + return nil +} + +// failureFor wraps a unit's discovery diagnostic for the command that targets it. +func (s *Session) failureFor(stack, component string) error { + if failure := s.failures[failedKey{stack, component}]; failure != nil { + return failure.err + } + return nil +} diff --git a/pkg/plugin/adapter/configuration/failures_test.go b/pkg/plugin/adapter/configuration/failures_test.go new file mode 100644 index 00000000000..c735683495d --- /dev/null +++ b/pkg/plugin/adapter/configuration/failures_test.go @@ -0,0 +1,99 @@ +package configuration + +import ( + "context" + "strings" + "testing" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + "go.uber.org/mock/gomock" + + errUtils "github.com/cloudposse/atmos/errors" + pluginv1 "github.com/cloudposse/atmos/pkg/plugin/proto/atmos/plugin/v1" + "github.com/cloudposse/atmos/pkg/schema" +) + +func discoverWith(t *testing.T, response *pluginv1.DiscoverComponentsResponse) (*Session, map[string]map[string]any, error) { + t.Helper() + provider := NewMockProvider(gomock.NewController(t)) + provider.EXPECT().ComponentType().Return("terraform").AnyTimes() + provider.EXPECT().Discover(gomock.Any(), gomock.Any()).Return(response, nil) + session, err := New(t.Context(), enabledConfig(t), "plan", func(context.Context, []string) (map[string]Provider, error) { + return map[string]Provider{"terragrunt": provider}, nil + }) + require.NoError(t, err) + definitions, err := session.Discover(t.Context()) + return session, definitions, err +} + +func failureDiagnostic(stack, name string) *pluginv1.Diagnostic { + return &pluginv1.Diagnostic{Detail: &pluginv1.ErrorDetail{ + Code: "unsupported_setting", Summary: "Unsupported Terragrunt setting `terraform_binary`", Stack: stack, Unit: name, ConfigKey: "terraform_binary", + Location: &pluginv1.SourceLocation{File: "live/" + name + "/terragrunt.hcl"}, Hints: []string{"Remove it."}, UserSafe: true, Operation: "discover", + }} +} + +// One failing unit leaves the healthy units usable and fails only the commands that target it. +func TestFailedUnitStaysVisibleAndFailsOnlyItsOwnCommands(t *testing.T) { + session, definitions, err := discoverWith(t, &pluginv1.DiscoverComponentsResponse{ + Components: []*pluginv1.DiscoveredComponent{unit(t, "good")}, + Diagnostics: []*pluginv1.Diagnostic{failureDiagnostic("live", "broken")}, + }) + require.NoError(t, err) + components := definitions["live"]["components"].(map[string]any)["terraform"].(map[string]any) + require.Contains(t, components, "good") + require.Contains(t, components, "broken", "a failed unit stays listed so it can be fixed") + configuration := components["broken"].(map[string]any)["settings"].(map[string]any)["configuration"].(map[string]any) + assert.Equal(t, true, configuration["failed"]) + assert.Equal(t, "live/broken/terragrunt.hcl", configuration["source"]) + // The unit's file stays watched, so an edit that leaves it broken still selects it as affected. + assert.Equal(t, map[string]any{"files": []any{"live/broken/terragrunt.hcl"}}, components["broken"].(map[string]any)["dependencies"]) + + require.NoError(t, session.ComponentFailure(t.Context(), "live", "good")) + require.NoError(t, session.ComponentFailure(t.Context(), "live", "unknown")) + failure := session.ComponentFailure(t.Context(), "live", "broken") + require.ErrorIs(t, failure, errUtils.ErrPluginInvalidConfig) + assert.Equal(t, "Unsupported Terragrunt setting `terraform_binary`", failure.Error()) + + _, err = session.Prepare(t.Context(), &schema.ConfigAndStacksInfo{Stack: "live", ComponentFromArg: "broken", Command: "tofu", SubCommand: "plan", ComponentSettingsSection: map[string]any{"configuration": true}}) + require.ErrorIs(t, err, errUtils.ErrPluginInvalidConfig) + assert.Equal(t, failure.Error(), err.Error()) +} + +func TestWarningDiagnosticsNeverFailDiscovery(t *testing.T) { + warning := &pluginv1.Diagnostic{Detail: &pluginv1.ErrorDetail{Code: "skipped", Summary: "Directory skipped", Severity: pluginv1.Severity_SEVERITY_WARNING, UserSafe: true}} + session, definitions, err := discoverWith(t, &pluginv1.DiscoverComponentsResponse{Components: []*pluginv1.DiscoveredComponent{unit(t, "good")}, Diagnostics: []*pluginv1.Diagnostic{warning}}) + require.NoError(t, err) + require.Contains(t, definitions, "live") + require.NoError(t, session.ComponentFailure(t.Context(), "live", "good")) +} + +func TestDiagnosticWithoutAUnitFailsTheProject(t *testing.T) { + _, _, err := discoverWith(t, &pluginv1.DiscoverComponentsResponse{Diagnostics: []*pluginv1.Diagnostic{{Field: "discovery", Message: "legacy"}}}) + require.ErrorIs(t, err, errUtils.ErrPluginInvalidConfig) +} + +// A unit that depends on a failed or out-of-scope unit fails alone, and the failure names the dependency. +func TestDependentsOfFailedOrMissingUnitsFailToo(t *testing.T) { + app, db, outside := unit(t, "app"), unit(t, "db"), unit(t, "lonely") + app.Dependencies = []*pluginv1.ComponentDependency{{Target: &pluginv1.ComponentAddress{Stack: "live", Component: "broken", ComponentType: "terraform"}}} + db.Dependencies = []*pluginv1.ComponentDependency{{Target: app.Address}} + outside.Dependencies = []*pluginv1.ComponentDependency{{Target: &pluginv1.ComponentAddress{Stack: "other", Component: "network", ComponentType: "terraform"}}} + session, _, err := discoverWith(t, &pluginv1.DiscoverComponentsResponse{ + Components: []*pluginv1.DiscoveredComponent{app, db, outside, unit(t, "good")}, + Diagnostics: []*pluginv1.Diagnostic{failureDiagnostic("live", "broken")}, + }) + require.NoError(t, err) + for _, name := range []string{"app", "db", "lonely", "broken"} { + require.Error(t, session.ComponentFailure(t.Context(), "live", name), name) + } + require.NoError(t, session.ComponentFailure(t.Context(), "live", "good")) + // Each dependent's diagnostic names its own dependency. + for name, dependency := range map[string]string{"app": "live/broken", "lonely": "other/network"} { + failure := session.ComponentFailure(t.Context(), "live", name) + assert.Contains(t, strings.Join(errUtils.AllDetails(failure), "\n"), dependency, name) + } + require.NotEmpty(t, errUtils.AllHints(session.ComponentFailure(t.Context(), "live", "lonely"))) + require.Contains(t, errUtils.AllDetails(session.ComponentFailure(t.Context(), "live", "db"))[0], "depends on `live/app`") +} diff --git a/pkg/plugin/adapter/configuration/imports.go b/pkg/plugin/adapter/configuration/imports.go index dbc90b018d0..c9d9c0662be 100644 --- a/pkg/plugin/adapter/configuration/imports.go +++ b/pkg/plugin/adapter/configuration/imports.go @@ -2,70 +2,233 @@ package configuration import ( "context" + "errors" "fmt" + "io/fs" + "os" "path/filepath" "strings" "google.golang.org/protobuf/proto" errUtils "github.com/cloudposse/atmos/errors" + "github.com/cloudposse/atmos/pkg/component/configuration" "github.com/cloudposse/atmos/pkg/perf" + adapter "github.com/cloudposse/atmos/pkg/plugin/adapter/component" pluginv1 "github.com/cloudposse/atmos/pkg/plugin/proto/atmos/plugin/v1" + "github.com/cloudposse/atmos/pkg/schema" ) -// RegisterImport records an explicit root without starting a plugin. The normal -// discovery pass evaluates the combined scope once after YAML imports are read. -func (s *Session) RegisterImport(ctx context.Context, uri string) (bool, error) { - defer perf.Track(nil, "configuration.Session.RegisterImport")() +// importRejectedCharacters cannot appear in a provider import path. +const importRejectedCharacters = `\?#*[{` +// ImportComponents resolves an explicit provider import into Terraform components of the +// importing stack. The provider evaluates only the imported roots; automatic discovery +// scope (included_paths and excluded_paths) never filters an explicit import. The units +// keep their provider names but drop the provider's derived stack, so the same unit can be +// imported by several stacks, or through a mixin, and each stack gets its own component. +func (s *Session) ImportComponents(ctx context.Context, uri string) (map[string]any, bool, error) { + defer perf.Track(nil, "configuration.Session.ImportComponents")() + + return s.importComponents(ctx, uri, false) +} + +func (s *Session) importComponents(ctx context.Context, uri string, validation bool) (map[string]any, bool, error) { name, path, found := strings.Cut(uri, "://") if !found { - return false, nil + return nil, false, nil + } + template := s.importProviders[name] + if template == nil { + // A configured provider with invalid settings reports them, not a remote download error. + if err := s.settingsErrs[name]; err != nil { + return nil, true, err + } + return nil, false, nil + } + key, err := s.importKey(ctx, name, template.Provider, path) + if err != nil { + return nil, true, err + } + + instances, err := s.resolveCachedImport(ctx, template, key, validation) + if err != nil { + return nil, true, err } - s.importMu.Lock() - defer s.importMu.Unlock() - request := s.importProviders[name] - if request == nil { - return false, nil + components, err := importedComponents(instances) + return components, true, err +} + +// resolveCachedImport serializes import evaluation and orders executable imports +// before discovery starts. Validation reads share results without changing inventory. +func (s *Session) resolveCachedImport(ctx context.Context, template *pluginv1.DiscoverComponentsRequest, key string, validation bool) ([]*pluginv1.DiscoveredComponent, error) { + if err := s.importMu.lockContext(ctx, s.ctx); err != nil { + return nil, err } + defer s.importMu.unlock() + // Another import may have run while this one waited; a caller that gave up meanwhile gets its + // cancellation, not that import's cached result. if err := ctx.Err(); err != nil { - return true, err + return nil, err } if err := s.ctx.Err(); err != nil { - return true, err - } - // Glob and bracket characters are rejected here, naming the import, rather than failing - // later inside provider discovery with an error that does not identify it. - if !filepath.IsLocal(path) || strings.ContainsAny(path, "\\?#*[{") { - return true, fmt.Errorf("%w: provider import %s must name a project-relative file or directory", errUtils.ErrPluginInvalidConfig, uri) - } - path = filepath.ToSlash(filepath.Clean(path)) - // The scheme may be a configuration alias; the provider only recognizes its own name. - key := request.Provider + "://" + path - if s.imports[key] { - return true, nil - } - if s.started { - return true, fmt.Errorf("%w: cannot add an import after configuration discovery", errUtils.ErrPluginInvalidConfig) - } - selected := s.discoveryRequest(request) - selected.Imports = append(selected.Imports, key) - s.imports[key] = true - return true, nil + return nil, err + } + instances, resolved := s.importResults[key] + if validation && !resolved { + instances, resolved = s.validationImports[key] + } + if resolved { + return instances, nil + } + if s.started && !validation { + return nil, fmt.Errorf("%w: cannot add import %q after configuration discovery", errUtils.ErrPluginInvalidConfig, key) + } + instances, err := s.resolveImport(ctx, template, key) + if err != nil { + return nil, err + } + s.cacheImport(key, instances, validation) + return instances, nil } -// discoveryRequest returns the pending discovery request for the template's provider, -// cloning the template into the session when none exists yet. The caller holds importMu. -func (s *Session) discoveryRequest(template *pluginv1.DiscoverComponentsRequest) *pluginv1.DiscoverComponentsRequest { - var selected *pluginv1.DiscoverComponentsRequest - for _, existing := range s.requests { - if existing.Provider == template.Provider { - selected = existing +// cacheImport stores successful results under importMu. Validation-only units must +// never become discovery exclusions or preparation sources. +func (s *Session) cacheImport(key string, instances []*pluginv1.DiscoveredComponent, validation bool) { + if validation { + if s.validationImports == nil { + s.validationImports = make(map[string][]*pluginv1.DiscoveredComponent) } + s.validationImports[key] = instances + return + } + s.importResults[key] = instances + for _, instance := range instances { + s.imported[instance.Source] = instance + } +} + +// importedComponents translates resolved units without changing the cached provider results. +func importedComponents(instances []*pluginv1.DiscoveredComponent) (map[string]any, error) { + components := make(map[string]any, len(instances)) + for _, instance := range instances { + config, err := instanceConfig(instance, true) + if err != nil { + return nil, err + } + components[instance.Address.Component] = config + } + return components, nil +} + +// importKey validates an import path and returns its canonical URI. The path must be a +// project-relative file or directory that exists. The scheme may be a configuration alias, so +// errors name the import as written while the key uses the provider's own name, the only one +// the provider recognizes. +func (s *Session) importKey(ctx context.Context, scheme, provider, path string) (string, error) { + if err := ctx.Err(); err != nil { + return "", err + } + if err := s.ctx.Err(); err != nil { + return "", err + } + // Providers read the path as a pattern root, so glob and escape characters are rejected + // here, naming the import and the reason, even inside a literal directory name. + if !filepath.IsLocal(path) || strings.ContainsAny(path, importRejectedCharacters) { + return "", fmt.Errorf("%w: provider import %s must name a project-relative file or directory without any of the characters %s", errUtils.ErrPluginInvalidConfig, scheme+"://"+path, importRejectedCharacters) } - if selected == nil { - selected = proto.Clone(template).(*pluginv1.DiscoverComponentsRequest) - s.requests = append(s.requests, selected) + clean := filepath.ToSlash(filepath.Clean(path)) + key := provider + "://" + clean + _, err := os.Stat(filepath.Join(s.root, filepath.FromSlash(clean))) + if err != nil && !errors.Is(err, fs.ErrNotExist) { + // Only a missing path is reported as missing; other failures keep their cause. + return "", fmt.Errorf("%w: import %q: cannot read %s: %w", errUtils.ErrPluginInvalidConfig, scheme+"://"+clean, clean, err) } - return selected + if err != nil { + return "", errUtils.Build(fmt.Errorf("%w: import %q: %s does not exist in the project", errUtils.ErrPluginInvalidConfig, scheme+"://"+clean, clean)). + WithHint("Provider imports name a file or directory relative to the Atmos base path (`base_path` in `atmos.yaml`), not to the importing stack file"). + Err() + } + return key, nil +} + +// resolveImport asks the provider for the units under one import. The caller holds importMu. +func (s *Session) resolveImport(ctx context.Context, template *pluginv1.DiscoverComponentsRequest, key string) ([]*pluginv1.DiscoveredComponent, error) { + // Keep caller deadlines and values while also stopping work when the session ends. + ctx, cancel := context.WithCancel(ctx) + stop := context.AfterFunc(s.ctx, cancel) + defer stop() + defer cancel() + // Cancellation may have happened while this import waited for importMu. + if err := s.ctx.Err(); err != nil { + return nil, err + } + if err := ctx.Err(); err != nil { + return nil, err + } + provider, err := s.importProvider(ctx, template.Provider) + if err != nil { + return nil, err + } + request := proto.Clone(template).(*pluginv1.DiscoverComponentsRequest) + request.IncludePaths, request.ExcludePaths, request.Imports = nil, nil, []string{key} + discovered, err := provider.Discover(ctx, request) + if err != nil { + return nil, err + } + if err := importDiagnostics(discovered.Diagnostics); err != nil { + return nil, err + } + if len(discovered.Components) == 0 { + return nil, errUtils.Build(errUtils.ErrConfigurationImportEmpty). + WithContext("import", key). + WithHint("Point the import at a directory below which a `terragrunt.hcl` with a `terraform { source = ... }` block or a `terragrunt.stack.hcl` exists, or at one of those files"). + Err() + } + names := make(map[string]*pluginv1.DiscoveredComponent, len(discovered.Components)) + for _, instance := range discovered.Components { + if previous := names[instance.Address.Component]; previous != nil { + return nil, configuration.ImportConflict(instance.Address.Component, previous.Source, instance.Source, key) + } + names[instance.Address.Component] = instance + } + return discovered.Components, nil +} + +// importProvider starts the providers and returns the Terraform provider an import names. +func (s *Session) importProvider(ctx context.Context, name string) (Provider, error) { + registry, err := s.providers(ctx) + if err != nil { + return nil, err + } + provider, exists := registry[name] + if !exists || provider.ComponentType() != terraformType { + return nil, errUtils.ErrPluginUnsupportedExtension + } + return provider, nil +} + +// importDiagnostics fails an import on a broken unit, naming the unit: the user asked for it by +// name. Other diagnostics are shown as warnings. +func importDiagnostics(diagnostics []*pluginv1.Diagnostic) error { + for _, diagnostic := range diagnostics { + if err := adapter.DiagnosticError(diagnostic); err != nil { + return err + } + adapter.WarnDiagnostic(diagnostic) + } + return nil +} + +// ForValidation shares cached imports but never registers additional executable units. +func (s *Session) ForValidation() schema.ComponentConfigurationRuntime { + defer perf.Track(nil, "configuration.Session.ForValidation")() + return &validationSession{Session: s} +} + +type validationSession struct{ *Session } + +func (s *validationSession) ImportComponents(ctx context.Context, uri string) (map[string]any, bool, error) { + defer perf.Track(nil, "configuration.validationSession.ImportComponents")() + return s.importComponents(ctx, uri, true) } diff --git a/pkg/plugin/adapter/configuration/imports_test.go b/pkg/plugin/adapter/configuration/imports_test.go index 2ae75158959..dade5cb6396 100644 --- a/pkg/plugin/adapter/configuration/imports_test.go +++ b/pkg/plugin/adapter/configuration/imports_test.go @@ -2,122 +2,699 @@ package configuration import ( "context" + "io/fs" + "os" "path/filepath" + "runtime" "sync" "testing" + "time" + cockroachErrors "github.com/cockroachdb/errors" "github.com/stretchr/testify/require" "go.uber.org/mock/gomock" + "google.golang.org/protobuf/proto" errUtils "github.com/cloudposse/atmos/errors" + "github.com/cloudposse/atmos/pkg/component/configuration" + adapter "github.com/cloudposse/atmos/pkg/plugin/adapter/component" pluginv1 "github.com/cloudposse/atmos/pkg/plugin/proto/atmos/plugin/v1" "github.com/cloudposse/atmos/pkg/schema" ) -func TestExplicitImportsWithoutAutomaticDiscovery(t *testing.T) { +// directories creates project directories so that imports naming them exist. +func directories(t *testing.T, root string, names ...string) { + t.Helper() + for _, name := range names { + require.NoError(t, os.MkdirAll(filepath.Join(root, filepath.FromSlash(name)), 0o755)) + } +} + +func importOnlyConfig(t *testing.T) *schema.AtmosConfiguration { + t.Helper() config := enabledConfig(t) + directories(t, config.BasePathAbsolute, "live/app", "empty", "late") settings := config.Components.Terraform.ConfigurationProviders["terragrunt"] settings.Discovery.Enabled = false config.Components.Terraform.ConfigurationProviders["terragrunt"] = settings + return config +} + +func TestImportedUnitsBecomeComponentsOfTheImportingStack(t *testing.T) { provider := NewMockProvider(gomock.NewController(t)) - provider.EXPECT().ComponentType().Return("terraform") + provider.EXPECT().ComponentType().Return("terraform").AnyTimes() + network, app := unit(t, "network"), unit(t, "app") + app.Dependencies = []*pluginv1.ComponentDependency{{Target: network.Address, Kind: pluginv1.ComponentDependency_OUTPUT}} provider.EXPECT().Discover(gomock.Any(), gomock.Any()).DoAndReturn(func(_ context.Context, req *pluginv1.DiscoverComponentsRequest) (*pluginv1.DiscoverComponentsResponse, error) { - require.Empty(t, req.IncludePaths) - require.Equal(t, []string{"terragrunt://live", "terragrunt://other/terragrunt.hcl"}, req.Imports) + require.Empty(t, req.IncludePaths, "imports are not limited by included_paths") + require.Empty(t, req.ExcludePaths, "imports are not filtered by excluded_paths") + require.Equal(t, []string{"terragrunt://live"}, req.Imports) require.Nil(t, req.Effects) - return &pluginv1.DiscoverComponentsResponse{Components: []*pluginv1.DiscoveredComponent{unit(t, "app")}}, nil - }) + return &pluginv1.DiscoverComponentsResponse{Components: []*pluginv1.DiscoveredComponent{network, app}}, nil + }).Times(2) // The import, then the dependency recheck that preparation runs under its own environment. + // The recheck confirms the output dependency still reads the state it was discovered with. + provider.EXPECT().ResolveState(gomock.Any(), gomock.Any()).DoAndReturn(func(_ context.Context, req *pluginv1.ComponentRequest) (*pluginv1.ResolveComponentStateResponse, error) { + require.Equal(t, network.Source, req.Source) + return &pluginv1.ResolveComponentStateResponse{State: &pluginv1.ComponentStateIdentity{BackendType: "local"}}, nil + }).Times(2) loads := 0 - session, err := New(t.Context(), config, "list", func(_ context.Context, names []string) (map[string]Provider, error) { + session, err := New(t.Context(), importOnlyConfig(t), "list", func(_ context.Context, names []string) (map[string]Provider, error) { loads++ require.Equal(t, []string{"terragrunt"}, names) return map[string]Provider{"terragrunt": provider}, nil }) require.NoError(t, err) - for _, uri := range []string{"terragrunt://live", "terragrunt://live/.", "terragrunt://other/terragrunt.hcl"} { - handled, err := session.RegisterImport(t.Context(), uri) + require.Zero(t, loads, "no plugin starts before an import or discovery needs it") + + // The same import from two stacks (or a mixin) resolves once and yields independent copies. + var first, second map[string]any + for _, target := range []*map[string]any{&first, &second} { + components, handled, err := session.ImportComponents(t.Context(), "terragrunt://live/.") require.NoError(t, err) require.True(t, handled) + *target = components } - require.Zero(t, loads) + require.Equal(t, 1, loads) + require.Len(t, first, 2) + appConfig := first["app"].(map[string]any) + require.Equal(t, []any{map[string]any{"name": "network", "kind": "terraform"}}, appConfig["dependencies"].(map[string]any)["components"], "an imported unit depends on its siblings in the importing stack") + require.Equal(t, "live/app/terragrunt.hcl", appConfig["settings"].(map[string]any)["configuration"].(map[string]any)["source"]) + appConfig["settings"].(map[string]any)["configuration"].(map[string]any)["source"] = "corrupt" + require.Equal(t, "live/app/terragrunt.hcl", second["app"].(map[string]any)["settings"].(map[string]any)["configuration"].(map[string]any)["source"]) + first["app"] = "corrupt" + require.IsType(t, map[string]any{}, second["app"]) + + // Import-only configuration has no derived stacks. definitions, err := session.Discover(t.Context()) require.NoError(t, err) - require.Contains(t, definitions, "live") - handled, err := session.RegisterImport(t.Context(), "terragrunt://live") + require.Empty(t, definitions) + + // Prepare finds the unit by source, under the stack and component the user selected. + provider.EXPECT().Prepare(gomock.Any(), gomock.Any()).DoAndReturn(func(_ context.Context, req *pluginv1.ComponentRequest) (*pluginv1.PrepareComponentResponse, error) { + require.Equal(t, "dev", req.Address.Stack) + require.Equal(t, "web", req.Address.Component) + require.Equal(t, "live/app/terragrunt.hcl", req.Source) + return &pluginv1.PrepareComponentResponse{Engine: "opentofu", ModuleSource: "/module", State: &pluginv1.ComponentStateIdentity{BackendType: "local"}}, nil + }) + _, err = session.Prepare(t.Context(), &schema.ConfigAndStacksInfo{Stack: "dev", ComponentFromArg: "web", SubCommand: "plan", ComponentSettingsSection: map[string]any{"configuration": map[string]any{"source": "live/app/terragrunt.hcl"}}}) + require.NoError(t, err) + _, err = session.Prepare(t.Context(), &schema.ConfigAndStacksInfo{Stack: "dev", ComponentFromArg: "web", SubCommand: "plan", ComponentSettingsSection: map[string]any{"configuration": map[string]any{"source": "live/other/terragrunt.hcl"}}}) + require.ErrorIs(t, err, errUtils.ErrPluginInvalidConfig) +} + +func TestImportedUnitsAreNotAlsoDiscoveredInADerivedStack(t *testing.T) { + provider := NewMockProvider(gomock.NewController(t)) + provider.EXPECT().ComponentType().Return("terraform").AnyTimes() + provider.EXPECT().Discover(gomock.Any(), gomock.Any()).DoAndReturn(func(_ context.Context, req *pluginv1.DiscoverComponentsRequest) (*pluginv1.DiscoverComponentsResponse, error) { + if len(req.Imports) > 0 { + return &pluginv1.DiscoverComponentsResponse{Components: []*pluginv1.DiscoveredComponent{unit(t, "app")}}, nil + } + return &pluginv1.DiscoverComponentsResponse{Components: []*pluginv1.DiscoveredComponent{unit(t, "app"), unit(t, "db")}}, nil + }).Times(2) + config := enabledConfig(t) + directories(t, config.BasePathAbsolute, "live/app", "late") + require.NoError(t, os.WriteFile(filepath.Join(config.BasePathAbsolute, "live", "app", "terragrunt.hcl"), nil, 0o644)) + session, err := New(t.Context(), config, "list", func(context.Context, []string) (map[string]Provider, error) { + return map[string]Provider{"terragrunt": provider}, nil + }) + require.NoError(t, err) + _, _, err = session.ImportComponents(t.Context(), "terragrunt://live/app/terragrunt.hcl") + require.NoError(t, err) + definitions, err := session.Discover(t.Context()) + require.NoError(t, err) + derived := definitions["live"]["components"].(map[string]any)["terraform"].(map[string]any) + require.NotContains(t, derived, "app", "an imported unit belongs to the importing stack only") + require.Contains(t, derived, "db") + _, _, err = session.ImportComponents(t.Context(), "terragrunt://late") + require.ErrorIs(t, err, errUtils.ErrPluginInvalidConfig, "imports after discovery would be silently ignored") +} + +// A discovered unit and an imported unit with another source that claim one stack and component +// are a conflict: preparation names both sources instead of running either unit. +func TestDiscoveredAndImportedUnitAtOneAddressConflict(t *testing.T) { + provider := NewMockProvider(gomock.NewController(t)) + provider.EXPECT().ComponentType().Return("terraform").AnyTimes() + catalog := unit(t, "db") + catalog.Source = "catalog/db/terragrunt.hcl" + provider.EXPECT().Discover(gomock.Any(), gomock.Any()).DoAndReturn(func(_ context.Context, req *pluginv1.DiscoverComponentsRequest) (*pluginv1.DiscoverComponentsResponse, error) { + if len(req.Imports) > 0 { + return &pluginv1.DiscoverComponentsResponse{Components: []*pluginv1.DiscoveredComponent{catalog}}, nil + } + return &pluginv1.DiscoverComponentsResponse{Components: []*pluginv1.DiscoveredComponent{unit(t, "app"), unit(t, "db")}}, nil + }).Times(2) + config := enabledConfig(t) + directories(t, config.BasePathAbsolute, "catalog") + session, err := New(t.Context(), config, "plan", func(context.Context, []string) (map[string]Provider, error) { + return map[string]Provider{"terragrunt": provider}, nil + }) + require.NoError(t, err) + _, _, err = session.ImportComponents(t.Context(), "terragrunt://catalog") + require.NoError(t, err) + _, err = session.Discover(t.Context()) + require.NoError(t, err) + + info := &schema.ConfigAndStacksInfo{Stack: "live", ComponentFromArg: "db", SubCommand: "plan", ComponentSettingsSection: map[string]any{"configuration": map[string]any{"source": catalog.Source}}} + _, err = session.Prepare(t.Context(), info) + require.ErrorIs(t, err, errUtils.ErrConfigurationImportConflict) + require.ErrorContains(t, err, "live/db/terragrunt.hcl") + require.ErrorContains(t, err, "catalog/db/terragrunt.hcl") + + // Without a competing import, the discovered unit is still found at its address. + info.ComponentSettingsSection = map[string]any{"configuration": map[string]any{"source": "live/db/terragrunt.hcl"}} + instance, err := session.instanceFor(info) + require.NoError(t, err) + require.Equal(t, "live/db/terragrunt.hcl", instance.Source) +} + +func TestImportOfUnitsWithTheSameNameFails(t *testing.T) { + provider := NewMockProvider(gomock.NewController(t)) + provider.EXPECT().ComponentType().Return("terraform").AnyTimes() + dev, prod := unit(t, "vpc"), unit(t, "vpc") + dev.Source, prod.Source = "live/dev/vpc/terragrunt.hcl", "live/prod/vpc/terragrunt.hcl" + provider.EXPECT().Discover(gomock.Any(), gomock.Any()).Return(&pluginv1.DiscoverComponentsResponse{Components: []*pluginv1.DiscoveredComponent{dev, prod}}, nil) + session, err := New(t.Context(), importOnlyConfig(t), "list", func(context.Context, []string) (map[string]Provider, error) { + return map[string]Provider{"terragrunt": provider}, nil + }) + require.NoError(t, err) + _, _, err = session.ImportComponents(t.Context(), "terragrunt://live") + require.ErrorIs(t, err, errUtils.ErrConfigurationImportConflict) + require.ErrorContains(t, err, "live/dev/vpc/terragrunt.hcl") + require.ErrorContains(t, err, "live/prod/vpc/terragrunt.hcl") +} + +func TestImportOfAMissingPathNamesIt(t *testing.T) { + session, err := New(t.Context(), importOnlyConfig(t), "list", func(context.Context, []string) (map[string]Provider, error) { + t.Fatal("a missing path must be rejected before the provider runs") + return nil, nil + }) require.NoError(t, err) + _, handled, err := session.ImportComponents(t.Context(), "terragrunt://live/absent") require.True(t, handled) - _, err = session.RegisterImport(t.Context(), "terragrunt://late") require.ErrorIs(t, err, errUtils.ErrPluginInvalidConfig) + require.ErrorContains(t, err, "terragrunt://live/absent") +} + +func TestImportThatMatchesNothingFails(t *testing.T) { + provider := NewMockProvider(gomock.NewController(t)) + provider.EXPECT().ComponentType().Return("terraform").AnyTimes() + provider.EXPECT().Discover(gomock.Any(), gomock.Any()).Return(&pluginv1.DiscoverComponentsResponse{}, nil) + session, err := New(t.Context(), importOnlyConfig(t), "list", func(context.Context, []string) (map[string]Provider, error) { + return map[string]Provider{"terragrunt": provider}, nil + }) + require.NoError(t, err) + _, _, err = session.ImportComponents(t.Context(), "terragrunt://empty") + require.ErrorIs(t, err, errUtils.ErrConfigurationImportEmpty) } -func TestImportValidationCancellationAndRevision(t *testing.T) { +func TestImportValidationAndCancellation(t *testing.T) { ctx, cancel := context.WithCancel(t.Context()) session, err := New(ctx, enabledConfig(t), "list", func(context.Context, []string) (map[string]Provider, error) { return nil, nil }) require.NoError(t, err) - for _, uri := range []string{"ordinary", "https://example.com/config.yaml"} { - handled, err := session.RegisterImport(t.Context(), uri) + for _, uri := range []string{"ordinary", "https://example.com/config.yaml", "other://live"} { + _, handled, err := session.ImportComponents(t.Context(), uri) require.NoError(t, err) require.False(t, handled) } - for _, path := range []string{"", "../outside", "/absolute", `live\app`, "live?ref=main", "live#fragment", "live/*", "live/[ab]", "live/{a,b}"} { - handled, err := session.RegisterImport(t.Context(), "terragrunt://"+path) + for _, path := range []string{"", "../outside", "/absolute", `live\app`, "live?ref=main", "live#fragment", "live/*", "live/[ab]", "live/[dev]", "live/{a,b}"} { + _, handled, err := session.ImportComponents(t.Context(), "terragrunt://"+path) require.True(t, handled) require.ErrorIs(t, err, errUtils.ErrPluginInvalidConfig) require.ErrorContains(t, err, "terragrunt://"+path) } - _, err = session.RegisterImport(t.Context(), "terragrunt://live") - require.NoError(t, err) - rebased, err := session.ForRevision(filepath.Join(t.TempDir(), "base")) - require.NoError(t, err) - revision := rebased.(*Session) - require.Empty(t, revision.requests[0].Imports) - require.Equal(t, []string{"live/**"}, revision.requests[0].IncludePaths) - _, err = revision.RegisterImport(t.Context(), "terragrunt://previous") - require.NoError(t, err) - require.Equal(t, []string{"terragrunt://previous"}, revision.requests[0].Imports) - require.Equal(t, []string{"terragrunt://live"}, session.requests[0].Imports) canceled, stop := context.WithCancel(t.Context()) stop() - _, err = session.RegisterImport(canceled, "terragrunt://live") + _, _, err = session.ImportComponents(canceled, "terragrunt://live") require.ErrorIs(t, err, context.Canceled) cancel() - _, err = session.RegisterImport(t.Context(), "terragrunt://live") + _, _, err = session.ImportComponents(t.Context(), "terragrunt://live") require.ErrorIs(t, err, context.Canceled) } -func TestConcurrentImportRegistrationDeduplicates(t *testing.T) { - session, err := New(t.Context(), enabledConfig(t), "list", func(context.Context, []string) (map[string]Provider, error) { return nil, nil }) +func TestConcurrentImportsResolveOnce(t *testing.T) { + provider := NewMockProvider(gomock.NewController(t)) + provider.EXPECT().ComponentType().Return("terraform").AnyTimes() + provider.EXPECT().Discover(gomock.Any(), gomock.Any()).Return(&pluginv1.DiscoverComponentsResponse{Components: []*pluginv1.DiscoveredComponent{unit(t, "app")}}, nil).Times(1) + session, err := New(t.Context(), importOnlyConfig(t), "list", func(context.Context, []string) (map[string]Provider, error) { + return map[string]Provider{"terragrunt": provider}, nil + }) require.NoError(t, err) var workers sync.WaitGroup for range 10 { workers.Add(1) go func() { defer workers.Done() - _, err := session.RegisterImport(t.Context(), "terragrunt://live") - if err != nil { + if _, _, err := session.ImportComponents(t.Context(), "terragrunt://live"); err != nil { t.Error(err) } }() } workers.Wait() - require.Len(t, session.requests, 1) - require.Equal(t, []string{"terragrunt://live"}, session.requests[0].Imports) var _ schema.ComponentConfigurationImportRuntime = session } func TestAliasedImportSchemeUsesTheProviderName(t *testing.T) { - config := enabledConfig(t) + config := importOnlyConfig(t) settings := config.Components.Terraform.ConfigurationProviders["terragrunt"] settings.Provider = "terragrunt" config.Components.Terraform.ConfigurationProviders = map[string]schema.ComponentConfiguration{"tg": settings} - session, err := New(t.Context(), config, "list", func(context.Context, []string) (map[string]Provider, error) { return nil, nil }) + provider := NewMockProvider(gomock.NewController(t)) + provider.EXPECT().ComponentType().Return("terraform").AnyTimes() + provider.EXPECT().Discover(gomock.Any(), gomock.Any()).DoAndReturn(func(_ context.Context, request *pluginv1.DiscoverComponentsRequest) (*pluginv1.DiscoverComponentsResponse, error) { + require.Equal(t, "terragrunt", request.Provider) + require.Equal(t, []string{"terragrunt://live"}, request.Imports, "the provider only recognizes its own name") + return &pluginv1.DiscoverComponentsResponse{Components: []*pluginv1.DiscoveredComponent{unit(t, "app")}}, nil + }) + session, err := New(t.Context(), config, "list", func(context.Context, []string) (map[string]Provider, error) { + return map[string]Provider{"terragrunt": provider}, nil + }) require.NoError(t, err) for _, uri := range []string{"tg://live", "tg://live/."} { - handled, err := session.RegisterImport(t.Context(), uri) + components, handled, err := session.ImportComponents(t.Context(), uri) require.NoError(t, err) require.True(t, handled) + require.Contains(t, components, "app") + } +} + +func TestImportConflictsRejectOnlyDifferentUnits(t *testing.T) { + unitComponent := func(source string) map[string]any { + return map[string]any{"settings": map[string]any{"configuration": map[string]any{"source": source}}} + } + fragment := func(name string, component map[string]any) map[string]any { + return configuration.ImportFragment(map[string]any{name: component}) + } + yaml := map[string]any{"components": map[string]any{"terraform": map[string]any{"vpc": map[string]any{"vars": map[string]any{"a": 1}}}}} + + require.NoError(t, configuration.CheckImportConflicts("stacks/dev.yaml", nil)) + require.NoError(t, configuration.CheckImportConflicts("stacks/dev.yaml", []map[string]any{fragment("vpc", unitComponent("live/vpc/terragrunt.hcl")), fragment("vpc", unitComponent("live/vpc/terragrunt.hcl"))}), "the same unit imported twice is one component") + require.NoError(t, configuration.CheckImportConflicts("stacks/dev.yaml", []map[string]any{fragment("app", unitComponent("live/app/terragrunt.hcl")), yaml})) + + err := configuration.CheckImportConflicts("stacks/dev.yaml", []map[string]any{fragment("vpc", unitComponent("live/dev/vpc/terragrunt.hcl")), fragment("vpc", unitComponent("live/prod/vpc/terragrunt.hcl"))}) + require.ErrorIs(t, err, errUtils.ErrConfigurationImportConflict) + require.ErrorContains(t, err, "live/dev/vpc/terragrunt.hcl") + require.ErrorContains(t, err, "live/prod/vpc/terragrunt.hcl") + + // A YAML component with the name of a unit is an override, wherever it comes from. + for _, configs := range [][]map[string]any{ + {fragment("vpc", unitComponent("live/vpc/terragrunt.hcl")), yaml}, + {yaml, fragment("vpc", unitComponent("live/vpc/terragrunt.hcl"))}, + {fragment("vpc", unitComponent("live/vpc/terragrunt.hcl")), yaml, fragment("vpc", unitComponent("live/vpc/terragrunt.hcl"))}, + } { + require.NoError(t, configuration.CheckImportConflicts("stacks/dev.yaml", configs)) + } +} + +func TestOwnsComponentRecognizesProviderSections(t *testing.T) { + owned := map[string]any{"settings": map[string]any{"configuration": map[string]any{"provider": "terragrunt"}}} + require.True(t, configuration.OwnsComponent(owned)) + require.False(t, configuration.OwnsComponent(map[string]any{"settings": map[string]any{"team": "platform"}})) + require.False(t, configuration.OwnsComponent(nil)) +} + +// An explicit import of a configured provider whose settings are invalid reports the setting and +// its repair hint, instead of falling through to remote import resolution. +func TestImportOfAProviderWithInvalidSettingsReportsTheSetting(t *testing.T) { + config := importOnlyConfig(t) + settings := config.Components.Terraform.ConfigurationProviders["terragrunt"] + settings.Mode = "wrapper" + config.Components.Terraform.ConfigurationProviders["terragrunt"] = settings + session, err := New(t.Context(), config, "list", func(context.Context, []string) (map[string]Provider, error) { + t.Fatal("invalid settings must be reported before the provider runs") + return nil, nil + }) + require.NoError(t, err) + _, handled, err := session.ImportComponents(t.Context(), "terragrunt://live") + require.True(t, handled) + require.ErrorIs(t, err, errUtils.ErrPluginInvalidConfig) + require.Contains(t, cockroachErrors.FlattenHints(err), "The only supported mode is `native`") + // A scheme no section configures is still not handled here. + _, handled, err = session.ImportComponents(t.Context(), "other://live") + require.NoError(t, err) + require.False(t, handled) +} + +func TestValidationImportsDoNotChangeRunningDiscovery(t *testing.T) { + config := enabledConfig(t) + directories(t, config.BasePathAbsolute, "live/app") + provider := NewMockProvider(gomock.NewController(t)) + provider.EXPECT().ComponentType().Return("terraform").AnyTimes() + started, finish := make(chan struct{}), make(chan struct{}) + app := unit(t, "app") + provider.EXPECT().Discover(gomock.Any(), gomock.Any()).DoAndReturn(func(_ context.Context, req *pluginv1.DiscoverComponentsRequest) (*pluginv1.DiscoverComponentsResponse, error) { + if len(req.Imports) == 0 { + close(started) + <-finish + } + return &pluginv1.DiscoverComponentsResponse{Components: []*pluginv1.DiscoveredComponent{app}}, nil + }).Times(2) + session, err := New(t.Context(), config, "describe affected", func(context.Context, []string) (map[string]Provider, error) { + return map[string]Provider{"terragrunt": provider}, nil + }) + require.NoError(t, err) + done := make(chan error, 1) + go func() { _, err := session.Discover(t.Context()); done <- err }() + <-started + validation := session.ForValidation().(schema.ComponentConfigurationImportRuntime) + var workers sync.WaitGroup + for range 10 { + workers.Add(1) + go func() { + defer workers.Done() + components, handled, err := validation.ImportComponents(t.Context(), "terragrunt://live/app") + if err != nil || !handled || components["app"] == nil { + t.Errorf("validation import: handled=%v, err=%v, components=%v", handled, err, components) + } + }() + } + workers.Wait() + close(finish) + require.NoError(t, <-done) + definitions, err := session.Discover(t.Context()) + require.NoError(t, err) + require.Contains(t, definitions["live"]["components"].(map[string]any)["terraform"], "app") + require.Empty(t, session.imported, "validation must not register preparation sources") + _, _, err = session.ImportComponents(t.Context(), "terragrunt://live/app") + require.ErrorIs(t, err, errUtils.ErrPluginInvalidConfig, "a validation cache entry does not authorize a late execution import") + first, _, err := validation.ImportComponents(t.Context(), "terragrunt://live/app") + require.NoError(t, err) + first["app"].(map[string]any)["settings"] = "corrupt" + again, _, err := validation.ImportComponents(t.Context(), "terragrunt://live/app") + require.NoError(t, err) + require.IsType(t, map[string]any{}, again["app"].(map[string]any)["settings"]) + revision, err := session.ForRevision(t.TempDir()) + require.NoError(t, err) + require.Empty(t, revision.(*Session).validationImports) +} + +func TestImportInProgressCancellation(t *testing.T) { + for _, phase := range []string{"loader", "discovery"} { + for _, origin := range []string{"session", "caller", "deadline"} { + t.Run(phase+"/"+origin, func(t *testing.T) { + sessionCtx, cancelSession := context.WithCancel(t.Context()) + defer cancelSession() + callerCtx, cancelCaller := context.WithCancel(t.Context()) + defer cancelCaller() + // A generous caller deadline also verifies that composition retains the deadline. + deadline := time.Now().Add(time.Minute) + if origin == "deadline" { + deadline = time.Now().Add(time.Second) + } + callerCtx, cancelDeadline := context.WithDeadline(callerCtx, deadline) + defer cancelDeadline() + started := make(chan context.Context, 1) + block := func(ctx context.Context) error { + started <- ctx + <-ctx.Done() + return ctx.Err() + } + provider := NewMockProvider(gomock.NewController(t)) + if phase == "discovery" { + provider.EXPECT().ComponentType().Return("terraform") + provider.EXPECT().Discover(gomock.Any(), gomock.Any()).DoAndReturn(func(ctx context.Context, _ *pluginv1.DiscoverComponentsRequest) (*pluginv1.DiscoverComponentsResponse, error) { + return nil, block(ctx) + }) + } + session, err := New(sessionCtx, importOnlyConfig(t), "list", func(ctx context.Context, _ []string) (map[string]Provider, error) { + if phase == "loader" { + return nil, block(ctx) + } + return map[string]Provider{"terragrunt": provider}, nil + }) + require.NoError(t, err) + done := make(chan error, 1) + go func() { + _, _, err := session.ImportComponents(callerCtx, "terragrunt://live") + done <- err + }() + select { + case ctx := <-started: + got, ok := ctx.Deadline() + require.True(t, ok) + require.Equal(t, deadline, got) + case <-time.After(5 * time.Second): + t.Fatal("import did not reach provider") + } + switch origin { + case "session": + cancelSession() + case "caller": + cancelCaller() + } + select { + case err := <-done: + want := context.Canceled + if origin == "deadline" { + want = context.DeadlineExceeded + } + require.ErrorIs(t, err, want) + case <-time.After(5 * time.Second): + t.Fatal("cancellation did not unblock import") + } + require.Empty(t, session.importResults, "failed imports must not be cached") + }) + } + } +} + +// An import waiting for another import of the same URI gives up when its context ends, and never +// returns the other import's cached units after its own cancellation. +func TestCanceledImportWaiterDoesNotReturnCachedSuccess(t *testing.T) { + provider := NewMockProvider(gomock.NewController(t)) + provider.EXPECT().ComponentType().Return("terraform").AnyTimes() + started, release := make(chan struct{}), make(chan struct{}) + provider.EXPECT().Discover(gomock.Any(), gomock.Any()).DoAndReturn(func(context.Context, *pluginv1.DiscoverComponentsRequest) (*pluginv1.DiscoverComponentsResponse, error) { + close(started) + <-release + return &pluginv1.DiscoverComponentsResponse{Components: []*pluginv1.DiscoveredComponent{unit(t, "app")}}, nil + }) + session, err := New(t.Context(), importOnlyConfig(t), "list", func(context.Context, []string) (map[string]Provider, error) { + return map[string]Provider{"terragrunt": provider}, nil + }) + require.NoError(t, err) + first := make(chan error, 1) + go func() { + _, _, err := session.ImportComponents(t.Context(), "terragrunt://live") + first <- err + }() + <-started + + waiter, cancel := context.WithCancel(t.Context()) + second := make(chan error, 1) + go func() { + _, _, err := session.ImportComponents(waiter, "terragrunt://live") + second <- err + }() + cancel() + select { + case err := <-second: + require.ErrorIs(t, err, context.Canceled) + case <-time.After(5 * time.Second): + t.Fatal("a canceled waiter must not wait for the in-flight import") + } + close(release) + require.NoError(t, <-first) + + // A later caller with a live context still shares the cached result. + components, handled, err := session.ImportComponents(t.Context(), "terragrunt://live") + require.NoError(t, err) + require.True(t, handled) + require.Contains(t, components, "app") +} + +// An import path that exists but cannot be inspected keeps its filesystem cause; only a missing +// path gets the "does not exist" diagnostic. +func TestImportPathFilesystemFailuresKeepTheirCause(t *testing.T) { + if runtime.GOOS == "windows" { + t.Skip("directory permissions do not block path inspection on Windows") + } + config := importOnlyConfig(t) + locked := filepath.Join(config.BasePathAbsolute, "locked") + require.NoError(t, os.MkdirAll(filepath.Join(locked, "unit"), 0o755)) + require.NoError(t, os.Chmod(locked, 0o000)) + t.Cleanup(func() { _ = os.Chmod(locked, 0o755) }) + if _, err := os.Stat(filepath.Join(locked, "unit")); err == nil { + t.Skip("the process bypasses directory permissions") + } + session, err := New(t.Context(), config, "list", func(context.Context, []string) (map[string]Provider, error) { + t.Fatal("an unreadable import path must be rejected before the provider runs") + return nil, nil + }) + require.NoError(t, err) + _, handled, err := session.ImportComponents(t.Context(), "terragrunt://locked/unit") + require.True(t, handled) + require.ErrorIs(t, err, errUtils.ErrPluginInvalidConfig) + require.ErrorIs(t, err, fs.ErrPermission) + require.NotContains(t, err.Error(), "does not exist") + + _, _, err = session.ImportComponents(t.Context(), "terragrunt://absent") + require.ErrorIs(t, err, errUtils.ErrPluginInvalidConfig) + require.ErrorContains(t, err, "does not exist") +} + +// A unit imported on its own may depend on a unit beyond the imported root. The target is absent from +// the baseline and from the recheck, so preparation accepts it and still rejects a changed edge. +func TestImportedUnitDependencyOutsideTheImportedRoot(t *testing.T) { + for _, test := range []struct { + name string + kind pluginv1.ComponentDependency_Kind + changed bool + want error + }{ + {name: "order dependency", kind: pluginv1.ComponentDependency_ORDER}, + {name: "output dependency", kind: pluginv1.ComponentDependency_OUTPUT}, + {name: "changed dependency target", kind: pluginv1.ComponentDependency_ORDER, changed: true, want: errUtils.ErrPluginInvalidConfig}, + } { + t.Run(test.name, func(t *testing.T) { + provider := NewMockProvider(gomock.NewController(t)) + provider.EXPECT().ComponentType().Return("terraform").AnyTimes() + app := unit(t, "app") + app.Dependencies = []*pluginv1.ComponentDependency{{Target: unit(t, "shared").Address, Kind: test.kind}} + recheck := proto.Clone(app).(*pluginv1.DiscoveredComponent) + if test.changed { + recheck.Dependencies[0].Target = unit(t, "other").Address + } + calls := 0 + provider.EXPECT().Discover(gomock.Any(), gomock.Any()).DoAndReturn(func(context.Context, *pluginv1.DiscoverComponentsRequest) (*pluginv1.DiscoverComponentsResponse, error) { + calls++ + if calls == 1 { + return &pluginv1.DiscoverComponentsResponse{Components: []*pluginv1.DiscoveredComponent{app}}, nil + } + return &pluginv1.DiscoverComponentsResponse{Components: []*pluginv1.DiscoveredComponent{recheck}}, nil + }).Times(2) + if test.want == nil { + provider.EXPECT().Prepare(gomock.Any(), gomock.Any()).Return(&pluginv1.PrepareComponentResponse{Engine: "opentofu", ModuleSource: "/module", State: &pluginv1.ComponentStateIdentity{BackendType: "local"}}, nil) + } + session, err := New(t.Context(), importOnlyConfig(t), "plan", func(context.Context, []string) (map[string]Provider, error) { + return map[string]Provider{"terragrunt": provider}, nil + }) + require.NoError(t, err) + _, _, err = session.ImportComponents(t.Context(), "terragrunt://live/app") + require.NoError(t, err) + _, err = session.Prepare(t.Context(), &schema.ConfigAndStacksInfo{Stack: "dev", ComponentFromArg: "web", SubCommand: "plan", ComponentSettingsSection: map[string]any{"configuration": map[string]any{"source": app.Source}}}) + if test.want != nil { + require.ErrorIs(t, err, test.want) + return + } + require.NoError(t, err) + }) + } +} + +// An imported dependency's prepared state is stored under the importing stack, so the dependent +// finds it there instead of re-resolving the dependency under the discovery environment. +func TestImportedDependencyUsesItsPreparedStateInTheImportingStack(t *testing.T) { + stateAt := func(path string) *pluginv1.ComponentStateIdentity { + backend, err := adapter.EncodeObject(map[string]any{"path": path}) + require.NoError(t, err) + return &pluginv1.ComponentStateIdentity{BackendType: "local", Backend: backend, Workspace: "default", WorkingDirectory: "/work"} + } + for _, test := range []struct { + name string + dependent *pluginv1.ComponentStateIdentity + want error + }{ + {name: "matches prepared dependency", dependent: stateAt("network.tfstate")}, + {name: "differs from prepared dependency", dependent: stateAt("other.tfstate"), want: errUtils.ErrPluginInvalidConfig}, + } { + t.Run(test.name, func(t *testing.T) { + provider := NewMockProvider(gomock.NewController(t)) + provider.EXPECT().ComponentType().Return("terraform").AnyTimes() + network, app := unit(t, "network"), unit(t, "app") + app.Dependencies = []*pluginv1.ComponentDependency{{Target: network.Address, Kind: pluginv1.ComponentDependency_OUTPUT}} + provider.EXPECT().Discover(gomock.Any(), gomock.Any()).Return(&pluginv1.DiscoverComponentsResponse{Components: []*pluginv1.DiscoveredComponent{network, app}}, nil).AnyTimes() + provider.EXPECT().ResolveState(gomock.Any(), gomock.Any()).DoAndReturn(func(_ context.Context, request *pluginv1.ComponentRequest) (*pluginv1.ResolveComponentStateResponse, error) { + require.Equal(t, "dependent", request.Invocation.Environment["IDENTITY"], "a prepared dependency must not be re-resolved") + return &pluginv1.ResolveComponentStateResponse{State: test.dependent}, nil + }).MinTimes(1) + provider.EXPECT().Prepare(gomock.Any(), gomock.Any()).Return(&pluginv1.PrepareComponentResponse{Engine: "opentofu", ModuleSource: "/module", State: stateAt("network.tfstate")}, nil) + if test.want == nil { + provider.EXPECT().Prepare(gomock.Any(), gomock.Any()).Return(&pluginv1.PrepareComponentResponse{Engine: "opentofu", ModuleSource: "/module", State: stateAt("app.tfstate")}, nil) + } + session, err := New(t.Context(), importOnlyConfig(t), "apply", func(context.Context, []string) (map[string]Provider, error) { + return map[string]Provider{"terragrunt": provider}, nil + }) + require.NoError(t, err) + _, _, err = session.ImportComponents(t.Context(), "terragrunt://live") + require.NoError(t, err) + info := func(component string, unit *pluginv1.DiscoveredComponent, identity string) *schema.ConfigAndStacksInfo { + return &schema.ConfigAndStacksInfo{Stack: "dev", ComponentFromArg: component, SubCommand: "apply", TerraformWorkspace: "default", ComponentSettingsSection: map[string]any{"configuration": map[string]any{"source": unit.Source}}, SanitizedEnv: []string{"IDENTITY=" + identity}} + } + _, err = session.Prepare(t.Context(), info("network", network, "network")) + require.NoError(t, err) + _, err = session.Prepare(t.Context(), info("app", app, "dependent")) + if test.want != nil { + require.ErrorIs(t, err, test.want) + return + } + require.NoError(t, err) + }) + } +} + +// An output dependency whose target was evaluated by a separate import is still checked: the +// dependent's state under its own environment must equal the target's prepared state. A target that +// no import evaluated has no state evidence and is skipped. +func TestDependencyPreparedStateIsComparedAcrossSeparateImports(t *testing.T) { + stateAt := func(path string) *pluginv1.ComponentStateIdentity { + backend, err := adapter.EncodeObject(map[string]any{"path": path}) + require.NoError(t, err) + return &pluginv1.ComponentStateIdentity{BackendType: "local", Backend: backend, Workspace: "default", WorkingDirectory: "/work"} + } + for _, test := range []struct { + name string + imports []string + dependent *pluginv1.ComponentStateIdentity + want error + }{ + {name: "matches prepared dependency", imports: []string{"network", "app"}, dependent: stateAt("network.tfstate")}, + {name: "differs from prepared dependency", imports: []string{"network", "app"}, dependent: stateAt("other.tfstate"), want: errUtils.ErrPluginInvalidConfig}, + {name: "unimported target is skipped", imports: []string{"app"}, dependent: stateAt("other.tfstate")}, + } { + t.Run(test.name, func(t *testing.T) { + provider := NewMockProvider(gomock.NewController(t)) + provider.EXPECT().ComponentType().Return("terraform").AnyTimes() + network, app := unit(t, "network"), unit(t, "app") + app.Dependencies = []*pluginv1.ComponentDependency{{Target: network.Address, Kind: pluginv1.ComponentDependency_OUTPUT}} + provider.EXPECT().Discover(gomock.Any(), gomock.Any()).DoAndReturn(func(_ context.Context, request *pluginv1.DiscoverComponentsRequest) (*pluginv1.DiscoverComponentsResponse, error) { + require.Len(t, request.Imports, 1) + if request.Imports[0] == "terragrunt://live/network" { + return &pluginv1.DiscoverComponentsResponse{Components: []*pluginv1.DiscoveredComponent{network}}, nil + } + return &pluginv1.DiscoverComponentsResponse{Components: []*pluginv1.DiscoveredComponent{app}}, nil + }).AnyTimes() + resolved := 0 + provider.EXPECT().ResolveState(gomock.Any(), gomock.Any()).DoAndReturn(func(_ context.Context, request *pluginv1.ComponentRequest) (*pluginv1.ResolveComponentStateResponse, error) { + resolved++ + require.Equal(t, "dependent", request.Invocation.Environment["IDENTITY"], "a prepared dependency must not be re-resolved") + return &pluginv1.ResolveComponentStateResponse{State: test.dependent}, nil + }).AnyTimes() + provider.EXPECT().Prepare(gomock.Any(), gomock.Any()).Return(&pluginv1.PrepareComponentResponse{Engine: "opentofu", ModuleSource: "/module", State: stateAt("network.tfstate")}, nil).AnyTimes() + config := importOnlyConfig(t) + directories(t, config.BasePathAbsolute, "live/network") + session, err := New(t.Context(), config, "apply", func(context.Context, []string) (map[string]Provider, error) { + return map[string]Provider{"terragrunt": provider}, nil + }) + require.NoError(t, err) + for _, name := range test.imports { + _, _, err = session.ImportComponents(t.Context(), "terragrunt://live/"+name) + require.NoError(t, err) + } + info := func(component string, unit *pluginv1.DiscoveredComponent, identity string) *schema.ConfigAndStacksInfo { + return &schema.ConfigAndStacksInfo{Stack: "dev", ComponentFromArg: component, SubCommand: "apply", TerraformWorkspace: "default", ComponentSettingsSection: map[string]any{"configuration": map[string]any{"source": unit.Source}}, SanitizedEnv: []string{"IDENTITY=" + identity}} + } + if len(test.imports) > 1 { + _, err = session.Prepare(t.Context(), info("network", network, "network")) + require.NoError(t, err) + } + _, err = session.Prepare(t.Context(), info("app", app, "dependent")) + if test.want != nil { + require.ErrorIs(t, err, test.want) + return + } + require.NoError(t, err) + if len(test.imports) == 1 { + require.Zero(t, resolved, "a target no import evaluated has no state to compare") + } + }) } - require.Len(t, session.requests, 1) - require.Equal(t, "terragrunt", session.requests[0].Provider) - require.Equal(t, []string{"terragrunt://live"}, session.requests[0].Imports) } diff --git a/pkg/plugin/adapter/configuration/plan.go b/pkg/plugin/adapter/configuration/plan.go index 3f2022e3832..1c8dc8d1637 100644 --- a/pkg/plugin/adapter/configuration/plan.go +++ b/pkg/plugin/adapter/configuration/plan.go @@ -169,7 +169,7 @@ func digestSource(ctx context.Context, root, path string, result map[string]stri } relative, err := filepath.Rel(root, real) if err != nil || !filepath.IsLocal(relative) { - return fmt.Errorf("%w: source leaves project root", errUtils.ErrPluginInvalidConfig) + return sourceOutsideRoot(root, path, real) } data, err := os.ReadFile(real) if err != nil { @@ -183,3 +183,100 @@ func digestSource(ctx context.Context, root, path string, result map[string]stri result[filepath.ToSlash(relative)] = hex.EncodeToString(sum[:]) return nil } + +// sourceOutsideRoot names the watched file and where its symlink leads. Atmos fingerprints a +// plan's inputs to bind the saved plan to them, and it only reads files inside the project. +func sourceOutsideRoot(root, path, real string) error { + shown, err := filepath.Rel(root, path) + if err != nil { + shown = path + } + return errUtils.Build(fmt.Errorf("%w: source %s leaves the project root (it resolves to %s)", errUtils.ErrPluginInvalidConfig, filepath.ToSlash(shown), real)). + WithContext("source", filepath.ToSlash(shown)). + WithContext("resolves_to", real). + WithHint("Replace the symlink with a copy of the file, or point it at a file inside the project"). + Err() +} + +// checkSourcesInRoot fails before any infrastructure command runs when a watched file, or a +// symlink inside a watched directory, resolves outside the project root. It only inspects +// symlinks, so it costs a directory walk and no file reads. +func checkSourcesInRoot(ctx context.Context, root string, sources []schema.ComponentConfigurationSource) error { + root, err := filepath.EvalSymlinks(root) + if err != nil { + return sourceReadError(err) + } + for _, source := range sources { + if !filepath.IsLocal(source.Path) { + return errUtils.ErrPluginInvalidConfig + } + top := filepath.Join(root, filepath.FromSlash(source.Path)) + if err := checkSourcePath(root, top); err != nil { + return err + } + if err := checkSourceTree(ctx, root, top); err != nil { + return err + } + } + return nil +} + +// checkSourcePath rejects a watched path whose own location leaves the project root through a +// symlinked parent directory, such as `shared/inputs.hcl` when `shared` links outside. A missing +// path is left for the first read to report; any other resolution failure, such as a permission +// error, is reported now, as fingerprinting would report it later. +func checkSourcePath(root, top string) error { + real, err := filepath.EvalSymlinks(top) + if errors.Is(err, fs.ErrNotExist) { + return nil + } + if err != nil { + return sourceReadError(err) + } + if relative, err := filepath.Rel(root, real); err != nil || !filepath.IsLocal(relative) { + return sourceOutsideRoot(root, top, real) + } + return nil +} + +// checkSourceTree walks one watched source and rejects links that leave the project root. +func checkSourceTree(ctx context.Context, root, top string) error { + err := filepath.WalkDir(top, func(path string, entry fs.DirEntry, walkErr error) error { + if walkErr != nil { + return sourceReadError(walkErr) + } + if err := ctx.Err(); err != nil { + return err + } + if entry.IsDir() { + if path != top && excludedPlanSource(entry.Name()) { + return filepath.SkipDir + } + return nil + } + return checkSourceLink(root, path, entry) + }) + if err != nil && !errors.Is(err, fs.ErrNotExist) { + return err + } + return nil +} + +// checkSourceLink rejects a symlink entry that resolves outside the project root. A broken +// link is left for the first read to report; other resolution failures are reported now. +func checkSourceLink(root, path string, entry fs.DirEntry) error { + if entry.Type()&fs.ModeSymlink == 0 || excludedPlanSource(entry.Name()) { + return nil + } + real, err := filepath.EvalSymlinks(path) + if errors.Is(err, fs.ErrNotExist) { + return nil + } + if err != nil { + return sourceReadError(err) + } + if relative, err := filepath.Rel(root, real); err != nil || !filepath.IsLocal(relative) { + return sourceOutsideRoot(root, path, real) + } + return nil +} diff --git a/pkg/plugin/adapter/configuration/plan_test.go b/pkg/plugin/adapter/configuration/plan_test.go index da5eaa1e89c..36142c62d75 100644 --- a/pkg/plugin/adapter/configuration/plan_test.go +++ b/pkg/plugin/adapter/configuration/plan_test.go @@ -3,9 +3,11 @@ package configuration import ( "context" "errors" + "io/fs" "math" "os" "path/filepath" + "runtime" "testing" "github.com/stretchr/testify/require" @@ -314,3 +316,111 @@ func TestExcludedPlanSource(t *testing.T) { require.Equal(t, excluded, excludedPlanSource(name), name) } } + +func TestSourceThatLeavesTheProjectIsNamedBeforeAnythingRuns(t *testing.T) { + root := t.TempDir() + outside := t.TempDir() + require.NoError(t, os.MkdirAll(filepath.Join(root, "modules", "app"), 0o755)) + require.NoError(t, os.WriteFile(filepath.Join(root, "modules", "app", "main.tf"), nil, 0o644)) + require.NoError(t, os.WriteFile(filepath.Join(outside, "victim.tf"), []byte("x"), 0o644)) + sources := []schema.ComponentConfigurationSource{{Path: "modules/app", Directory: true}, {Path: "absent", Directory: true}} + require.NoError(t, checkSourcesInRoot(t.Context(), root, sources)) + if err := os.Symlink(filepath.Join(outside, "victim.tf"), filepath.Join(root, "modules", "app", "link.tf")); err != nil { + t.Skipf("symlink unavailable: %v", err) + } + err := checkSourcesInRoot(t.Context(), root, sources) + require.ErrorIs(t, err, errUtils.ErrPluginInvalidConfig) + require.ErrorContains(t, err, "modules/app/link.tf") + require.ErrorContains(t, err, "leaves the project root") + // The plan fingerprint reports the same file instead of an anonymous failure. + _, err = sourceDigests(t.Context(), root, sources) + require.ErrorIs(t, err, errUtils.ErrPluginInvalidConfig) + require.ErrorContains(t, err, "modules/app/link.tf") + // A symlink inside the project is fine, and ignored artifacts are not inspected. + require.NoError(t, os.Remove(filepath.Join(root, "modules", "app", "link.tf"))) + require.NoError(t, os.Symlink(filepath.Join(root, "modules", "app", "main.tf"), filepath.Join(root, "modules", "app", "inside.tf"))) + require.NoError(t, os.MkdirAll(filepath.Join(root, "modules", "app", ".terraform"), 0o755)) + require.NoError(t, os.Symlink(filepath.Join(outside, "victim.tf"), filepath.Join(root, "modules", "app", ".terraform", "provider"))) + require.NoError(t, checkSourcesInRoot(t.Context(), root, sources)) +} + +// A watched file reached through a symlinked parent directory that leaves the project is rejected +// before anything runs, although the file itself is not a symlink. +func TestWatchedFileUnderASymlinkedParentThatLeavesTheProjectIsRejected(t *testing.T) { + root := t.TempDir() + outside := t.TempDir() + require.NoError(t, os.WriteFile(filepath.Join(outside, "inputs.hcl"), []byte("x"), 0o644)) + if err := os.Symlink(outside, filepath.Join(root, "shared")); err != nil { + t.Skipf("symlink unavailable: %v", err) + } + err := checkSourcesInRoot(t.Context(), root, []schema.ComponentConfigurationSource{{Path: "shared/inputs.hcl"}}) + require.ErrorIs(t, err, errUtils.ErrPluginInvalidConfig) + require.ErrorContains(t, err, "shared/inputs.hcl") + require.ErrorContains(t, err, "leaves the project root") +} + +// A watched link whose target cannot be resolved for a reason other than absence is reported +// during preparation, as fingerprinting would report it, while a dangling link is still allowed. +func TestSourceLinkResolutionFailuresAreReported(t *testing.T) { + if runtime.GOOS == "windows" { + t.Skip("directory permissions do not block path resolution on Windows") + } + root, err := filepath.EvalSymlinks(t.TempDir()) + require.NoError(t, err) + locked := filepath.Join(root, "locked") + require.NoError(t, os.MkdirAll(filepath.Join(locked, "inner"), 0o755)) + require.NoError(t, os.WriteFile(filepath.Join(locked, "inner", "inputs.hcl"), nil, 0o600)) + require.NoError(t, os.MkdirAll(filepath.Join(root, "watched"), 0o755)) + if err := os.Symlink(filepath.Join(locked, "inner", "inputs.hcl"), filepath.Join(root, "watched", "inputs.hcl")); err != nil { + t.Skipf("symlinks unavailable: %v", err) + } + require.NoError(t, os.Symlink(filepath.Join(root, "absent.hcl"), filepath.Join(root, "watched", "dangling.hcl"))) + require.NoError(t, os.Symlink(filepath.Join(locked, "inner", "inputs.hcl"), filepath.Join(root, "direct.hcl"))) + require.NoError(t, os.Chmod(locked, 0o000)) + t.Cleanup(func() { _ = os.Chmod(locked, 0o755) }) + if _, err := filepath.EvalSymlinks(filepath.Join(root, "direct.hcl")); err == nil { + t.Skip("the process bypasses directory permissions") + } + + for _, source := range []string{"watched", "direct.hcl"} { + err := checkSourcesInRoot(t.Context(), root, []schema.ComponentConfigurationSource{{Path: source}}) + require.ErrorIs(t, err, errUtils.ErrReadFile, source) + require.ErrorIs(t, err, fs.ErrPermission, source) + } + require.NoError(t, os.Chmod(locked, 0o755)) + require.NoError(t, checkSourcesInRoot(t.Context(), root, []schema.ComponentConfigurationSource{{Path: "watched"}}), "a dangling link is left for the first read") +} + +// Filesystem failures from the source preflight carry the same classification as fingerprint +// read failures, and a cancelled context is reported as itself. +func TestSourcePreflightWrapsFilesystemFailures(t *testing.T) { + missing := filepath.Join(t.TempDir(), "absent-root") + err := checkSourcesInRoot(t.Context(), missing, []schema.ComponentConfigurationSource{{Path: "watched"}}) + require.ErrorIs(t, err, errUtils.ErrReadFile) + require.ErrorIs(t, err, fs.ErrNotExist) + + if runtime.GOOS != "windows" { + root, evalErr := filepath.EvalSymlinks(t.TempDir()) + require.NoError(t, evalErr) + locked := filepath.Join(root, "watched", "locked") + require.NoError(t, os.MkdirAll(locked, 0o755)) + require.NoError(t, os.Chmod(locked, 0o000)) + t.Cleanup(func() { _ = os.Chmod(locked, 0o755) }) + if _, readErr := os.ReadDir(locked); readErr == nil { + t.Log("the process bypasses directory permissions") + } else { + err = checkSourcesInRoot(t.Context(), root, []schema.ComponentConfigurationSource{{Path: "watched", Directory: true}}) + require.ErrorIs(t, err, errUtils.ErrReadFile) + require.ErrorIs(t, err, fs.ErrPermission) + } + } + + root := t.TempDir() + require.NoError(t, os.MkdirAll(filepath.Join(root, "watched"), 0o755)) + require.NoError(t, os.WriteFile(filepath.Join(root, "watched", "a.tf"), nil, 0o644)) + ctx, cancel := context.WithCancel(t.Context()) + cancel() + err = checkSourcesInRoot(ctx, root, []schema.ComponentConfigurationSource{{Path: "watched", Directory: true}}) + require.ErrorIs(t, err, context.Canceled) + require.NotErrorIs(t, err, errUtils.ErrReadFile) +} diff --git a/pkg/plugin/adapter/configuration/preparation.go b/pkg/plugin/adapter/configuration/preparation.go index 8374419f287..d6604cd9fc7 100644 --- a/pkg/plugin/adapter/configuration/preparation.go +++ b/pkg/plugin/adapter/configuration/preparation.go @@ -3,6 +3,7 @@ package configuration import ( "context" "fmt" + "maps" "os" "path/filepath" "slices" @@ -11,6 +12,8 @@ import ( "google.golang.org/protobuf/proto" errUtils "github.com/cloudposse/atmos/errors" + componentconfiguration "github.com/cloudposse/atmos/pkg/component/configuration" + "github.com/cloudposse/atmos/pkg/downloader" "github.com/cloudposse/atmos/pkg/perf" adapter "github.com/cloudposse/atmos/pkg/plugin/adapter/component" pluginv1 "github.com/cloudposse/atmos/pkg/plugin/proto/atmos/plugin/v1" @@ -31,6 +34,31 @@ func (s *Session) Prepare(ctx context.Context, info *schema.ConfigAndStacksInfo) if err != nil { return nil, err } + result, err := s.runProviderPrepare(ctx, info, selected) + if err != nil { + return nil, err + } + prepared, err := preparedConfiguration(result) + if err != nil { + return nil, err + } + if err := checkSourcesInRoot(ctx, s.root, prepared.Sources); err != nil { + return nil, err + } + return prepared, nil +} + +// runProviderPrepare asks the owning provider to reevaluate the selected unit with every effect +// granted, after checking module ownership and engine selection. Only an explicit --command +// flag may switch the engine discovered from the global configuration. The dependencies the +// scheduler ordered are rechecked first, so no provider effect runs when they changed. +func (s *Session) runProviderPrepare(ctx context.Context, info *schema.ConfigAndStacksInfo, selected *selectedComponent) (*pluginv1.PrepareComponentResponse, error) { + if err := checkSourceOverride(info, selected.instance); err != nil { + return nil, err + } + if engine := expectedEngine(info, selected.instance); engine != selected.instance.Engine && info.ComponentCommand == "" { + return nil, engineMismatch(info, selected.instance.Engine, engine) + } if err := s.checkPreparationDependencies(ctx, selected.instance, selected.request.Invocation.Environment, info.SubCommand); err != nil { return nil, err } @@ -45,11 +73,55 @@ func (s *Session) Prepare(ctx context.Context, info *schema.ConfigAndStacksInfo) if result.State != nil { s.prepared.Store(addressKey(selected.instance.Address), proto.Clone(result.State)) } - return preparedConfiguration(result) + return result, nil +} + +// checkSourceOverride rejects a stack that points the component at another module. The module +// belongs to the unit's own `terraform { source }`; every other `source` attribute (version, ttl, +// included paths) is ordinary configuration and merges normally. +func checkSourceOverride(info *schema.ConfigAndStacksInfo, instance *pluginv1.DiscoveredComponent) error { + discovered, err := adapter.DecodeObject(instance.Configuration) + if err != nil { + return err + } + want, got := sourceURI(discovered["source"]), sourceURI(info.ComponentSection["source"]) + if want == "" || got == "" || want == got { + return nil + } + // Either URI can carry credentials, so the error shows only their redacted forms. + return errUtils.Build(fmt.Errorf("%w: the Terraform module of component %q in stack %q is owned by its unit, so `source.uri: %s` cannot replace `%s`", errUtils.ErrConfigurationOverrideUnsupported, info.ComponentFromArg, info.Stack, downloader.RedactSource(got), downloader.RedactSource(want))). + WithContext("component", info.ComponentFromArg). + WithContext("stack", info.Stack). + WithHint("Change `terraform { source = ... }` in the unit's `terragrunt.hcl`, or define a regular component with its own `source`"). + Err() +} + +// sourceURI returns the module location of a component `source`, written as a string or as a map with `uri`. +func sourceURI(source any) string { + switch value := source.(type) { + case string: + return value + case map[string]any: + uri, _ := value["uri"].(string) + return uri + } + return "" +} + +// engineMismatch explains a command that selects a different engine than the one the component was +// discovered with. Stack configuration may pick another version or path of the same engine, but +// not switch between Terraform and OpenTofu, because the unit's lock file and state belong to one. +func engineMismatch(info *schema.ConfigAndStacksInfo, discovered, selected string) error { + return errUtils.Build(fmt.Errorf("%w: command %q selects the %s engine, but component %q was discovered for %s", errUtils.ErrConfigurationOverrideUnsupported, info.Command, selected, info.ComponentFromArg, discovered)). + WithContext("component", info.ComponentFromArg). + WithContext("stack", info.Stack). + WithContext("command", info.Command). + WithHint("Set `command` to another " + discovered + " executable, or change `components.terraform.command` in `atmos.yaml` to switch every unit to " + selected). + Err() } // expectedEngine is the engine of the command this invocation runs. Discovery records the engine of -// the globally configured command, but --command or a component command may select another one, +// the globally configured command, but --command may explicitly select another one, // and the provider prepares for the binary it is given. Any other engine is a protocol violation. func expectedEngine(info *schema.ConfigAndStacksInfo, instance *pluginv1.DiscoveredComponent) string { if info.Command == "" { @@ -74,11 +146,27 @@ func (s *Session) componentRequest(ctx context.Context, info *schema.ConfigAndSt if err := s.ensure(ctx); err != nil { return nil, err } - instance := s.instances[[3]string{info.Stack, "terraform", info.ComponentFromArg}] + // A unit that failed discovery reports its own diagnostic, not a generic lookup failure. + if err := s.failureFor(info.Stack, info.ComponentFromArg); err != nil { + return nil, err + } + instance, err := s.instanceFor(info) + if err != nil { + return nil, err + } if instance == nil { - return nil, errUtils.ErrPluginInvalidConfig + return nil, errUtils.Build(errUtils.ErrPluginInvalidConfig). + WithExplanation("The component is not provided by any enabled configuration provider."). + WithContext("component", info.ComponentFromArg). + WithContext("stack", info.Stack). + WithHint("Run `atmos list instances` to see the discovered and imported components"). + Err() + } + registry, err := s.providers(ctx) + if err != nil { + return nil, err } - provider, ok := s.registry[instance.Provider] + provider, ok := registry[instance.Provider] if !ok { return nil, errUtils.ErrPluginUnsupportedExtension } @@ -94,38 +182,150 @@ func (s *Session) componentRequest(ctx context.Context, info *schema.ConfigAndSt return &selectedComponent{request: request, provider: provider, instance: instance}, nil } +// instanceFor finds the unit behind a component. A discovered unit is addressed by its derived +// stack; an imported unit lives in whichever stack imported it, so it is found by its source file +// and addressed as the stack and component the user selected. When a discovered unit and an +// imported unit claim the same stack and component, neither is chosen: the conflict is reported. +func (s *Session) instanceFor(info *schema.ConfigAndStacksInfo) (*pluginv1.DiscoveredComponent, error) { + configuration, _ := info.ComponentSettingsSection["configuration"].(map[string]any) + source, _ := configuration["source"].(string) + s.importMu.lock() + imported := s.imported[source] + s.importMu.unlock() + if discovered := s.instances[[3]string{info.Stack, "terraform", info.ComponentFromArg}]; discovered != nil { + if imported != nil && imported.Source != discovered.Source { + return nil, componentconfiguration.ImportConflict(info.ComponentFromArg, discovered.Source, imported.Source, "stack "+info.Stack+": one discovered, one imported") + } + return discovered, nil + } + if imported == nil { + return nil, nil + } + instance := proto.Clone(imported).(*pluginv1.DiscoveredComponent) + instance.Address = &pluginv1.ComponentAddress{Stack: info.Stack, Component: info.ComponentFromArg, ComponentType: terraformType} + return instance, nil +} + // Reevaluate the inventory without preparation effects before any dependency // outputs are read. Identity and component environment overrides must not change // the units that the scheduler ordered using the original discovery result. func (s *Session) checkPreparationDependencies(ctx context.Context, instance *pluginv1.DiscoveredComponent, env map[string]string, command string) error { - for _, original := range s.requests { - if original.Provider != instance.Provider { + baseline := s.preparationBaseline(instance) + if baseline == nil { + return errUtils.ErrPluginInvalidConfig + } + request := proto.Clone(baseline.request).(*pluginv1.DiscoverComponentsRequest) + request.Invocation = &pluginv1.Invocation{Environment: env} + request.Operation = &pluginv1.ComponentOperation{Command: command, Mode: "native"} + response, err := s.registry[instance.Provider].Discover(ctx, request) + if err != nil { + return err + } + current := make(map[[3]string]*pluginv1.DiscoveredComponent) + if err := addDiscovered(make(map[string]map[string]any), current, response); err != nil { + return err + } + selected := current[addressKey(baseline.self.Address)] + if !sameDependencyUnit(baseline.self, selected) || !sameDependencies(baseline.self.Dependencies, selected.Dependencies) { + return fmt.Errorf("%w: preparation changed discovered dependency paths", errUtils.ErrPluginInvalidConfig) + } + for _, dependency := range selected.Dependencies { + key := addressKey(dependency.Target) + before, after := baseline.units[key], current[key] + // A target outside the evaluated scope, such as a dependency beyond a single imported + // unit's root, is absent from both inventories. The unchanged edge is already compared above. + if before == nil && after == nil { continue } - request := proto.Clone(original).(*pluginv1.DiscoverComponentsRequest) - request.Invocation = &pluginv1.Invocation{Environment: env} - request.Operation = &pluginv1.ComponentOperation{Command: command, Mode: "native"} - response, err := s.registry[instance.Provider].Discover(ctx, request) - if err != nil { - return err + if !sameDependencyUnit(before, after) { + return fmt.Errorf("%w: preparation changed discovered dependency paths", errUtils.ErrPluginInvalidConfig) } - current := make(map[[3]string]*pluginv1.DiscoveredComponent) - if err := addDiscovered(make(map[string]map[string]any), current, response); err != nil { - return err + } + return s.checkDependencyStates(ctx, baseline, selected.Dependencies, env, command) +} + +// preparationEvidence is the discovery request that found a unit, every unit that request found, +// and the unit as that request reported it, all under the provider's own addresses. +type preparationEvidence struct { + request *pluginv1.DiscoverComponentsRequest + units map[[3]string]*pluginv1.DiscoveredComponent + self *pluginv1.DiscoveredComponent + // importStack is the stack that imported the unit; it is empty for a discovered unit. + importStack string +} + +// preparationBaseline finds the evidence for a discovered unit in its provider's discovery +// request, and for an imported unit in the import that resolved it. An imported unit is +// addressed by the importing stack, so its provider address comes from the cached import. +func (s *Session) preparationBaseline(instance *pluginv1.DiscoveredComponent) *preparationEvidence { + if discovered := s.instances[addressKey(instance.Address)]; discovered != nil && discovered.Source == instance.Source { + for _, request := range s.requests { + if request.Provider == instance.Provider { + return &preparationEvidence{request: request, units: s.instances, self: discovered} + } } - selected := current[addressKey(instance.Address)] - if !sameDependencyUnit(instance, selected) || !sameDependencies(instance.Dependencies, selected.Dependencies) { - return fmt.Errorf("%w: preparation changed discovered dependency paths", errUtils.ErrPluginInvalidConfig) + return nil + } + return s.importBaseline(instance) +} + +// importBaseline finds the cached import that resolved an imported unit. +func (s *Session) importBaseline(instance *pluginv1.DiscoveredComponent) *preparationEvidence { + s.importMu.lock() + defer s.importMu.unlock() + reported := s.imported[instance.Source] + if reported == nil { + return nil + } + for key, units := range s.importResults { + if !slices.Contains(units, reported) { + continue } - for _, dependency := range selected.Dependencies { - key := addressKey(dependency.Target) - if !sameDependencyUnit(s.instances[key], current[key]) { - return fmt.Errorf("%w: preparation changed discovered dependency paths", errUtils.ErrPluginInvalidConfig) + template := s.importTemplate(instance.Provider) + if template == nil { + return nil + } + request := proto.Clone(template).(*pluginv1.DiscoverComponentsRequest) + request.IncludePaths, request.ExcludePaths, request.Imports = nil, nil, []string{key} + byAddress := make(map[[3]string]*pluginv1.DiscoveredComponent, len(units)) + for _, unit := range units { + byAddress[addressKey(unit.Address)] = unit + } + return &preparationEvidence{request: request, units: byAddress, self: reported, importStack: instance.Address.Stack} + } + return nil +} + +// otherImportTarget finds a dependency target among the units of the cached imports, with the +// discovery request that evaluated it. It returns nil when no import evaluated the target. +func (s *Session) otherImportTarget(key [3]string) (*pluginv1.DiscoveredComponent, *pluginv1.DiscoverComponentsRequest) { + s.importMu.lock() + defer s.importMu.unlock() + for _, importKey := range slices.Sorted(maps.Keys(s.importResults)) { + for _, unit := range s.importResults[importKey] { + if addressKey(unit.Address) != key { + continue + } + template := s.importTemplate(unit.Provider) + if template == nil { + return nil, nil } + request := proto.Clone(template).(*pluginv1.DiscoverComponentsRequest) + request.IncludePaths, request.ExcludePaths, request.Imports = nil, nil, []string{importKey} + return unit, request } - return s.checkDependencyStates(ctx, original, selected.Dependencies, env, command) } - return errUtils.ErrPluginInvalidConfig + return nil, nil +} + +// importTemplate returns the import request template of a provider, under any configured alias. +func (s *Session) importTemplate(provider string) *pluginv1.DiscoverComponentsRequest { + for _, template := range s.importProviders { + if template.Provider == provider { + return template + } + } + return nil } // checkDependencyStates keeps output reads on the dependency's own state. The @@ -133,18 +333,26 @@ func (s *Session) checkPreparationDependencies(ctx context.Context, instance *pl // environment, so an identity or component override can select different state. // The reference is the identity the dependency was prepared with in this // invocation, or else its identity under the discovery environment. -func (s *Session) checkDependencyStates(ctx context.Context, discovery *pluginv1.DiscoverComponentsRequest, dependencies []*pluginv1.ComponentDependency, env map[string]string, command string) error { +func (s *Session) checkDependencyStates(ctx context.Context, baseline *preparationEvidence, dependencies []*pluginv1.ComponentDependency, env map[string]string, command string) error { for _, dependency := range dependencies { if dependency.Kind != pluginv1.ComponentDependency_OUTPUT { continue } key := addressKey(dependency.Target) - target := s.instances[key] + target, discovery := baseline.units[key], baseline.request + if target == nil { + // The dependent's own request did not evaluate the target, but a separate import may have. + target, discovery = s.otherImportTarget(key) + } + if target == nil { + // No evidence evaluated the target, so there is no state to compare. + continue + } current, err := s.dependencyState(ctx, discovery, target, env, command) if err != nil { return err } - expected, err := s.referenceState(ctx, discovery, target, key, command) + expected, err := s.referenceState(ctx, discovery, target, preparedKey(baseline, key), command) if err != nil { return err } @@ -155,6 +363,16 @@ func (s *Session) checkDependencyStates(ctx context.Context, discovery *pluginv1 return nil } +// preparedKey is the address under which a dependency's prepared state is stored: the address +// the dependency was selected with. An imported unit's dependencies are its siblings in the +// importing stack, so they take that stack rather than the provider's own. +func preparedKey(baseline *preparationEvidence, key [3]string) [3]string { + if baseline.importStack != "" { + key[0] = baseline.importStack + } + return key +} + func (s *Session) referenceState(ctx context.Context, discovery *pluginv1.DiscoverComponentsRequest, target *pluginv1.DiscoveredComponent, key [3]string, command string) (*pluginv1.ComponentStateIdentity, error) { if prepared, ok := s.prepared.Load(key); ok { return prepared.(*pluginv1.ComponentStateIdentity), nil @@ -221,7 +439,7 @@ func preparedConfiguration(result *pluginv1.PrepareComponentResponse) (*schema.P } prepared.Execution = execution for _, file := range result.Files { - prepared.Files = append(prepared.Files, schema.ComponentConfigurationFile{Path: file.Path, Content: file.Content, IfExists: file.IfExists, Sensitive: file.Sensitive, Mode: file.Mode}) + prepared.Files = append(prepared.Files, schema.ComponentConfigurationFile{Path: file.Path, Content: file.Content, IfExists: file.IfExists, Sensitive: file.Sensitive, Mode: file.Mode, UnitFileCopy: file.UnitFileCopy}) } for _, source := range result.Sources { prepared.Sources = append(prepared.Sources, schema.ComponentConfigurationSource{Path: source.Path, Directory: source.Kind == pluginv1.ComponentSource_DIRECTORY}) @@ -264,7 +482,7 @@ func preparationEnvironment(info *schema.ConfigAndStacksInfo) map[string]string func (s *Session) providerSettings(name string) *pluginv1.ComponentValue { var settings *pluginv1.ComponentValue - for _, request := range s.requests { + for _, request := range s.importProviders { if request.Provider == name { settings = request.ProviderConfiguration } diff --git a/pkg/plugin/adapter/configuration/preparation_test.go b/pkg/plugin/adapter/configuration/preparation_test.go index b7a699bfa26..c4ef54491fe 100644 --- a/pkg/plugin/adapter/configuration/preparation_test.go +++ b/pkg/plugin/adapter/configuration/preparation_test.go @@ -44,6 +44,7 @@ func TestPrepareTranslatesGeneratedArtifactsAndWatchedSources(t *testing.T) { Files: []*pluginv1.ComponentGeneratedFile{ {Path: "backend.tf.json", Content: []byte("{}"), IfExists: "overwrite", Sensitive: true}, {Path: "provider.tf", Content: []byte("p"), IfExists: "skip"}, + {Path: ".terraform.lock.hcl", Content: []byte("pins"), IfExists: "overwrite", UnitFileCopy: true}, }, Sources: []*pluginv1.ComponentSource{ {Path: "root.hcl", Kind: pluginv1.ComponentSource_FILE}, @@ -58,6 +59,7 @@ func TestPrepareTranslatesGeneratedArtifactsAndWatchedSources(t *testing.T) { require.Equal(t, []schema.ComponentConfigurationFile{ {Path: "backend.tf.json", Content: []byte("{}"), IfExists: "overwrite", Sensitive: true}, {Path: "provider.tf", Content: []byte("p"), IfExists: "skip"}, + {Path: ".terraform.lock.hcl", Content: []byte("pins"), IfExists: "overwrite", UnitFileCopy: true}, }, prepared.Files) require.Equal(t, []schema.ComponentConfigurationSource{{Path: "root.hcl"}, {Path: "modules", Directory: true}}, prepared.Sources) @@ -66,6 +68,17 @@ func TestPrepareTranslatesGeneratedArtifactsAndWatchedSources(t *testing.T) { require.Equal(t, "-var=a=1", sent.Operation.TerraformArguments[0], "the request owns a copy of the arguments") } +func TestEngineMismatchNamesTheCommandAndBothEngines(t *testing.T) { + session, _, info := preparationSession(t) + info.Command = "terraform" + + _, err := session.Prepare(t.Context(), info) + + require.ErrorIs(t, err, errUtils.ErrConfigurationOverrideUnsupported) + require.ErrorContains(t, err, `command "terraform" selects the terraform engine`) + require.ErrorContains(t, err, "discovered for opentofu") +} + func TestPrepareRejectsDependenciesChangedSinceDiscovery(t *testing.T) { provider := NewMockProvider(gomock.NewController(t)) provider.EXPECT().ComponentType().Return("terraform").AnyTimes() @@ -99,7 +112,7 @@ func TestPrepareFailsClosed(t *testing.T) { {"provider failure", func(p *MockProvider, _ *schema.ConfigAndStacksInfo) { p.EXPECT().Prepare(gomock.Any(), gomock.Any()).Return(nil, failure) }, failure}, - {"engine changed by provider", func(p *MockProvider, _ *schema.ConfigAndStacksInfo) { + {"provider returns an unexpected engine", func(p *MockProvider, _ *schema.ConfigAndStacksInfo) { p.EXPECT().Prepare(gomock.Any(), gomock.Any()).Return(&pluginv1.PrepareComponentResponse{Engine: "terraform", State: &pluginv1.ComponentStateIdentity{}}, nil) }, errUtils.ErrPluginProtocol}, {"saved plan", func(_ *MockProvider, info *schema.ConfigAndStacksInfo) { info.UseTerraformPlan = true }, errUtils.ErrPluginUnsupportedExtension}, @@ -126,8 +139,11 @@ func TestPrepareAcceptsTheEngineOfTheSelectedCommand(t *testing.T) { } { t.Run(test.name, func(t *testing.T) { session, provider, info := preparationSession(t) - info.Command = test.command - provider.EXPECT().Prepare(gomock.Any(), gomock.Any()).Return(&pluginv1.PrepareComponentResponse{Engine: test.engine, State: &pluginv1.ComponentStateIdentity{}}, nil) + info.Command, info.ComponentCommand = test.command, test.command + provider.EXPECT().Prepare(gomock.Any(), gomock.Any()).DoAndReturn(func(_ context.Context, request *pluginv1.ComponentRequest) (*pluginv1.PrepareComponentResponse, error) { + require.Equal(t, test.command, request.Invocation.Environment["ATMOS_TERRAFORM_BINARY"]) + return &pluginv1.PrepareComponentResponse{Engine: test.engine, State: &pluginv1.ComponentStateIdentity{}}, nil + }) _, err := session.Prepare(t.Context(), info) if test.want == nil { require.NoError(t, err) @@ -156,6 +172,44 @@ func TestPreparedExecutionRejectsNilEntriesInEveryList(t *testing.T) { } } +func TestPrepareRejectsAnotherModuleSourceButAllowsOtherSourceSettings(t *testing.T) { + for _, test := range []struct { + name string + source any + want bool + redacted []string + }{ + {name: "no source in the section"}, + {name: "same string", source: "./modules/unit"}, + {name: "same uri", source: map[string]any{"uri": "./modules/unit", "version": "1.2.3"}}, + {name: "settings without a uri", source: map[string]any{"version": "1.2.3"}}, + {name: "another uri", source: map[string]any{"uri": "./modules/other"}, want: true}, + {name: "another string", source: "./modules/other", want: true}, + {name: "unsupported shape", source: 42}, + {name: "credentials in another uri", source: "git::https://user:token-4f2a@example.com/org/modules.git?ref=v1&sshkey=key-9c1e", want: true, redacted: []string{"token-4f2a", "key-9c1e"}}, + } { + t.Run(test.name, func(t *testing.T) { + session, provider, info := preparationSession(t) + info.ComponentSection = map[string]any{"source": test.source} + if test.want { + _, err := session.Prepare(t.Context(), info) + require.ErrorIs(t, err, errUtils.ErrConfigurationOverrideUnsupported) + require.ErrorContains(t, err, "./modules/unit") + for _, secret := range test.redacted { + require.NotContains(t, err.Error(), secret) + } + if test.redacted == nil { + require.ErrorContains(t, err, "./modules/other") + } + return + } + provider.EXPECT().Prepare(gomock.Any(), gomock.Any()).Return(&pluginv1.PrepareComponentResponse{Engine: "opentofu", State: &pluginv1.ComponentStateIdentity{}}, nil) + _, err := session.Prepare(t.Context(), info) + require.NoError(t, err) + }) + } +} + func TestSessionResolvesStateWithoutPreparation(t *testing.T) { config := enabledConfig(t) mock := NewMockProvider(gomock.NewController(t)) diff --git a/pkg/plugin/adapter/configuration/revision.go b/pkg/plugin/adapter/configuration/revision.go index ac2a520237b..6bd923d754d 100644 --- a/pkg/plugin/adapter/configuration/revision.go +++ b/pkg/plugin/adapter/configuration/revision.go @@ -23,9 +23,12 @@ func (s *Session) ForRevision(root string) (schema.ComponentConfigurationRuntime if !filepath.IsAbs(root) { return nil, errUtils.ErrPluginInvalidConfig } - s.importMu.Lock() - defer s.importMu.Unlock() - revision := &Session{ctx: s.ctx, root: root, command: s.command, loader: s.loader, done: make(chan struct{}), imports: make(map[string]bool), importProviders: make(map[string]*pluginv1.DiscoverComponentsRequest)} + s.importMu.lock() + defer s.importMu.unlock() + revision := &Session{ctx: s.ctx, root: root, command: s.command, loader: s.loader, done: make(chan struct{}), importResults: make(map[string][]*pluginv1.DiscoveredComponent), imported: make(map[string]*pluginv1.DiscoveredComponent), importProviders: make(map[string]*pluginv1.DiscoverComponentsRequest)} + revision.settingsErrs = s.settingsErrs + // The revision rejects the same invalid configuration as HEAD instead of discovering units HEAD would refuse. + revision.configErr = s.configErr // Explicit imports come from the revision's YAML, not the HEAD inventory. for scheme, request := range s.importProviders { cloned := proto.Clone(request).(*pluginv1.DiscoverComponentsRequest) diff --git a/pkg/plugin/adapter/configuration/session.go b/pkg/plugin/adapter/configuration/session.go index 5936b08191e..1b21442a06d 100644 --- a/pkg/plugin/adapter/configuration/session.go +++ b/pkg/plugin/adapter/configuration/session.go @@ -3,6 +3,7 @@ package configuration import ( "context" + "errors" "maps" "os" "path/filepath" @@ -44,11 +45,26 @@ type Session struct { registry map[string]Provider definitions map[string]map[string]any instances map[[3]string]*pluginv1.DiscoveredComponent + failures map[failedKey]*failedUnit err error - importMu sync.Mutex + importMu importLock importProviders map[string]*pluginv1.DiscoverComponentsRequest - imports map[string]bool - started bool + // importResults caches each resolved import by URI; imported indexes the same units by + // source so Prepare can find a unit under whichever stack imported it. + importResults map[string][]*pluginv1.DiscoveredComponent + // validationImports caches extra manifest reads without registering executable units. + validationImports map[string][]*pluginv1.DiscoveredComponent + imported map[string]*pluginv1.DiscoveredComponent + started bool + loadMu sync.Mutex + // configErr holds a settings problem found while reading configuration. Reading never fails on it, + // so `atmos config set` can always repair the file; the first use of the provider reports it. + configErr error + // settingsErrs holds the settings problem of each enabled section that could not be registered, + // by section name, so an explicit import of that scheme reports it instead of falling through. + settingsErrs map[string]error + // options holds the provider-owned keys of each enabled section, by section name. + options map[string]map[string]any // prepared records the state identity each instance was prepared with, so a // dependent's output reads can be checked against it within this invocation. prepared sync.Map @@ -65,34 +81,59 @@ func New(ctx context.Context, config *schema.AtmosConfiguration, command string, if !filepath.IsAbs(root) || loader == nil { return nil, errUtils.ErrPluginInvalidConfig } - s := &Session{ctx: ctx, root: root, command: command, loader: loader, done: make(chan struct{}), importProviders: make(map[string]*pluginv1.DiscoverComponentsRequest), imports: make(map[string]bool)} + s := &Session{ctx: ctx, root: root, command: command, loader: loader, done: make(chan struct{}), importProviders: make(map[string]*pluginv1.DiscoverComponentsRequest), importResults: make(map[string][]*pluginv1.DiscoveredComponent), imported: make(map[string]*pluginv1.DiscoveredComponent), options: make(map[string]map[string]any)} seen := make(map[string]bool) for _, name := range slices.Sorted(maps.Keys(config.Components.Terraform.ConfigurationProviders)) { settings := config.Components.Terraform.ConfigurationProviders[name] if !settings.Enabled { continue } - - scheme := name - if settings.Provider != "" { - name = settings.Provider - } - if seen[name] { - return nil, errUtils.ErrDuplicatePluginExtension - } - seen[name] = true - request, err := discoveryRequest(config, &settings, name, root, command) - if err != nil { - return nil, err - } - s.importProviders[scheme] = proto.Clone(request).(*pluginv1.DiscoverComponentsRequest) - if settings.Discovery.Enabled { - s.requests = append(s.requests, request) - } + s.addProvider(config, name, &settings, seen) } return s, nil } +// addProvider records one enabled `components.terraform` section. A section that repeats a provider or +// cannot be read is remembered in configErr, so reading configuration never fails on it. +func (s *Session) addProvider(config *schema.AtmosConfiguration, scheme string, settings *schema.ComponentConfiguration, seen map[string]bool) { + name := scheme + if settings.Provider != "" { + name = settings.Provider + } + if seen[name] { + s.rejectSection(scheme, errUtils.Build(errUtils.ErrDuplicatePluginExtension). + WithExplanationf("More than one section under `components.terraform` enables the configuration provider `%s`.", name). + WithContext("config_key", settingsKey(scheme)). + WithHint("Keep one enabled section per provider, or point the other section at a different provider with `provider:`.").Err()) + return + } + seen[name] = true + if err := validateSettings(scheme, settings); err != nil { + s.rejectSection(scheme, err) + return + } + request, err := discoveryRequest(config, settings, name, s.root, s.command) + if err != nil { + s.rejectSection(scheme, invalidSetting(settingsKey(scheme), "The settings cannot be read", "Remove keys under the section that repeat a built-in setting such as `enabled`, `provider`, `mode`, or `discovery`.")) + return + } + s.options[scheme] = settings.Options + s.importProviders[scheme] = proto.Clone(request).(*pluginv1.DiscoverComponentsRequest) + if settings.Discovery.Enabled { + s.requests = append(s.requests, request) + } +} + +// rejectSection remembers a section's settings problem for the first use of the provider, and for +// an explicit import of its scheme. +func (s *Session) rejectSection(scheme string, err error) { + s.configErr = errors.Join(s.configErr, err) + if s.settingsErrs == nil { + s.settingsErrs = make(map[string]error) + } + s.settingsErrs[scheme] = err +} + // Discover returns isolated definitions and shares only immutable evaluator results within this invocation. func (s *Session) Discover(ctx context.Context) (map[string]map[string]any, error) { defer perf.Track(nil, "configurationadapter.Discover")() @@ -111,6 +152,35 @@ func (s *Session) Discover(ctx context.Context) (map[string]map[string]any, erro return result, nil } +// providers loads every enabled provider once per session, so imports resolved while the +// stack manifests are read and the later discovery share one plugin activation. +func (s *Session) providers(ctx context.Context) (map[string]Provider, error) { + s.loadMu.Lock() + defer s.loadMu.Unlock() + if s.configErr != nil { + return nil, s.configErr + } + if s.registry != nil { + return s.registry, nil + } + names := make([]string, 0, len(s.importProviders)) + for _, request := range s.importProviders { + if !slices.Contains(names, request.Provider) { + names = append(names, request.Provider) + } + } + slices.Sort(names) + registry, err := s.loader(ctx, names) + if err != nil { + return nil, err + } + if registry == nil { + registry = map[string]Provider{} + } + s.registry = registry + return registry, nil +} + func (s *Session) ensure(ctx context.Context) error { if err := ctx.Err(); err != nil { return err @@ -119,9 +189,9 @@ func (s *Session) ensure(ctx context.Context) error { return err } s.once.Do(func() { - s.importMu.Lock() + s.importMu.lock() s.started = true - s.importMu.Unlock() + s.importMu.unlock() go func() { defer close(s.done) // Shared discovery is owned by the session: one caller giving up must not @@ -151,24 +221,16 @@ func environment() map[string]string { } func discoveryRequest(config *schema.AtmosConfiguration, settings *schema.ComponentConfiguration, name, root, command string) (*pluginv1.DiscoverComponentsRequest, error) { - if settings.Mode != "" && settings.Mode != "native" { - return nil, errUtils.ErrPluginUnsupportedExtension - } - if settings.Discovery.Enabled && len(settings.Discovery.IncludedPaths) == 0 { - return nil, errUtils.ErrPluginInvalidConfig - } - for _, pattern := range append(slices.Clone(settings.Discovery.IncludedPaths), settings.Discovery.ExcludedPaths...) { - if !filepath.IsLocal(pattern) { - return nil, errUtils.ErrPluginInvalidConfig - } - } value, err := adapter.EncodeObject(settings) if err != nil { return nil, err } includes := slices.Clone(settings.Discovery.IncludedPaths) - if !settings.Discovery.Enabled { + switch { + case !settings.Discovery.Enabled: includes = nil + case len(includes) == 0: + includes = slices.Clone(defaultIncludedPaths) } env := environment() for key, value := range config.Env { @@ -177,3 +239,38 @@ func discoveryRequest(config *schema.AtmosConfiguration, settings *schema.Compon env["ATMOS_TERRAFORM_BINARY"] = config.Components.Terraform.Command return &pluginv1.DiscoverComponentsRequest{Provider: name, RootDirectory: root, IncludePaths: includes, ExcludePaths: slices.Clone(settings.Discovery.ExcludedPaths), ProviderConfiguration: value, Operation: &pluginv1.ComponentOperation{Command: command, Mode: "native"}, Invocation: &pluginv1.Invocation{Environment: env}}, nil } + +// importLock serializes import evaluation and the session's import records. A waiter for an +// import can give up when its context ends, instead of waiting for another import to finish. +// The zero value is unlocked. +type importLock struct { + once sync.Once + channel chan struct{} +} + +func (l *importLock) slot() chan struct{} { + l.once.Do(func() { l.channel = make(chan struct{}, 1) }) + return l.channel +} + +// lock waits for the lock. +func (l *importLock) lock() { + l.slot() <- struct{}{} +} + +// unlock releases the lock. +func (l *importLock) unlock() { + <-l.slot() +} + +// lockContext waits for the lock until either context ends, and reports why it gave up. +func (l *importLock) lockContext(ctx, session context.Context) error { + select { + case l.slot() <- struct{}{}: + return nil + case <-ctx.Done(): + return ctx.Err() + case <-session.Done(): + return session.Err() + } +} diff --git a/pkg/plugin/adapter/configuration/session_test.go b/pkg/plugin/adapter/configuration/session_test.go index f78510dc965..ce72461d77b 100644 --- a/pkg/plugin/adapter/configuration/session_test.go +++ b/pkg/plugin/adapter/configuration/session_test.go @@ -4,6 +4,7 @@ import ( "context" "encoding/json" "errors" + "fmt" "path/filepath" "sync" "testing" @@ -94,8 +95,7 @@ func TestSessionRejectsInvalidScope(t *testing.T) { change func(*schema.ComponentConfiguration) want error }{ - {"mode", func(c *schema.ComponentConfiguration) { c.Mode = "cli" }, errUtils.ErrPluginUnsupportedExtension}, - {"empty", func(c *schema.ComponentConfiguration) { c.Discovery.IncludedPaths = nil }, errUtils.ErrPluginInvalidConfig}, + {"mode", func(c *schema.ComponentConfiguration) { c.Mode = "cli" }, errUtils.ErrPluginInvalidConfig}, {"escape", func(c *schema.ComponentConfiguration) { c.Discovery.IncludedPaths = []string{"../**"} }, errUtils.ErrPluginInvalidConfig}, {"options", func(c *schema.ComponentConfiguration) { c.Options = map[string]any{"enabled": false} }, errUtils.ErrPluginInvalidConfig}, } { @@ -104,11 +104,15 @@ func TestSessionRejectsInvalidScope(t *testing.T) { settings := config.Components.Terraform.ConfigurationProviders["terragrunt"] test.change(&settings) config.Components.Terraform.ConfigurationProviders["terragrunt"] = settings - _, err := New(t.Context(), config, "list", func(context.Context, []string) (map[string]Provider, error) { + // Reading configuration never fails on plugin settings, so `atmos config set` can repair them. + session, err := New(t.Context(), config, "list", func(context.Context, []string) (map[string]Provider, error) { t.Fatal("must not load") return nil, nil }) + require.NoError(t, err) + _, err = session.Discover(t.Context()) require.ErrorIs(t, err, test.want) + require.NotEmpty(t, errUtils.AllHints(err), "the error must say how to fix the setting") }) } config := enabledConfig(t) @@ -117,10 +121,32 @@ func TestSessionRejectsInvalidScope(t *testing.T) { settings := config.Components.Terraform.ConfigurationProviders["terragrunt"] settings.Provider = "terragrunt" config.Components.Terraform.ConfigurationProviders["duplicate"] = settings - _, err = New(t.Context(), config, "list", func(context.Context, []string) (map[string]Provider, error) { return nil, nil }) + session, err := New(t.Context(), config, "list", func(context.Context, []string) (map[string]Provider, error) { return nil, nil }) + require.NoError(t, err) + _, err = session.Discover(t.Context()) require.ErrorIs(t, err, errUtils.ErrDuplicatePluginExtension) } +// Enabling discovery without included_paths scopes it to the whole project instead of failing. +func TestDiscoveryWithoutIncludedPathsDefaultsToTheProject(t *testing.T) { + config := enabledConfig(t) + settings := config.Components.Terraform.ConfigurationProviders["terragrunt"] + settings.Discovery.IncludedPaths = nil + config.Components.Terraform.ConfigurationProviders["terragrunt"] = settings + provider := NewMockProvider(gomock.NewController(t)) + provider.EXPECT().ComponentType().Return("terraform") + provider.EXPECT().Discover(gomock.Any(), gomock.Any()).DoAndReturn(func(_ context.Context, request *pluginv1.DiscoverComponentsRequest) (*pluginv1.DiscoverComponentsResponse, error) { + require.Equal(t, []string{"**"}, request.IncludePaths) + return &pluginv1.DiscoverComponentsResponse{}, nil + }) + session, err := New(t.Context(), config, "list", func(context.Context, []string) (map[string]Provider, error) { + return map[string]Provider{"terragrunt": provider}, nil + }) + require.NoError(t, err) + _, err = session.Discover(t.Context()) + require.NoError(t, err) +} + func TestSessionCancellationAndFailure(t *testing.T) { ctx, cancel := context.WithCancel(context.Background()) started := make(chan struct{}) @@ -197,9 +223,6 @@ func TestSessionRejectsConflictingDiscovery(t *testing.T) { {name: "unsafe stack", change: func(c *pluginv1.DiscoveredComponent) { c.Address.Stack = "../outside" }, want: errUtils.ErrPluginProtocol}, {name: "unsupported engine", change: func(c *pluginv1.DiscoveredComponent) { c.Engine = "terragrunt" }, want: errUtils.ErrPluginUnsupportedExtension}, {name: "invalid values", change: func(c *pluginv1.DiscoveredComponent) { c.Configuration = &pluginv1.ComponentValue{} }, want: errUtils.ErrPluginProtocol}, - {name: "out of scope dependency", change: func(c *pluginv1.DiscoveredComponent) { - c.Dependencies = []*pluginv1.ComponentDependency{{Target: &pluginv1.ComponentAddress{Stack: "other", Component: "network", ComponentType: "terraform"}}} - }, want: errUtils.ErrPluginInvalidConfig}, {name: "duplicate", duplicate: true, want: errUtils.ErrDuplicatePluginExtension}, } { t.Run(test.name, func(t *testing.T) { @@ -404,36 +427,65 @@ func TestRevisionDiscoveryIsolated(t *testing.T) { require.ErrorIs(t, err, context.Canceled) } -func TestRevisionDropsImportOnlyScopeButKeepsImportProviders(t *testing.T) { +// A revision rejects the same invalid configuration as the HEAD session instead of discovering +// units that HEAD would refuse. +func TestRevisionInheritsConfigurationRejection(t *testing.T) { + config := enabledConfig(t) + provider := NewMockProvider(gomock.NewController(t)) + provider.EXPECT().ComponentType().Return("terraform").AnyTimes() + head, err := New(t.Context(), config, "plan", func(context.Context, []string) (map[string]Provider, error) { + return map[string]Provider{"terragrunt": provider}, nil + }) + require.NoError(t, err) + rejection := fmt.Errorf("%w: duplicate section", errUtils.ErrDuplicatePluginExtension) + head.rejectSection("terragrunt", rejection) + _, err = head.Discover(t.Context()) + require.ErrorIs(t, err, errUtils.ErrDuplicatePluginExtension) + + revision, err := head.ForRevision(t.TempDir()) + require.NoError(t, err) + _, err = revision.Discover(t.Context()) + require.ErrorIs(t, err, errUtils.ErrDuplicatePluginExtension) +} + +func TestRevisionKeepsImportProvidersButNotImportedUnits(t *testing.T) { config := enabledConfig(t) settings := config.Components.Terraform.ConfigurationProviders["terragrunt"] settings.Discovery.Enabled = false config.Components.Terraform.ConfigurationProviders["terragrunt"] = settings - head, err := New(t.Context(), config, "list", func(context.Context, []string) (map[string]Provider, error) { return nil, nil }) + directories(t, config.BasePathAbsolute, "live") + provider := NewMockProvider(gomock.NewController(t)) + provider.EXPECT().ComponentType().Return("terraform").AnyTimes() + provider.EXPECT().Discover(gomock.Any(), gomock.Any()).DoAndReturn(func(_ context.Context, request *pluginv1.DiscoverComponentsRequest) (*pluginv1.DiscoverComponentsResponse, error) { + return &pluginv1.DiscoverComponentsResponse{Components: []*pluginv1.DiscoveredComponent{unit(t, "app")}}, nil + }).Times(2) + head, err := New(t.Context(), config, "list", func(context.Context, []string) (map[string]Provider, error) { + return map[string]Provider{"terragrunt": provider}, nil + }) require.NoError(t, err) - handled, err := head.RegisterImport(t.Context(), "terragrunt://live") + components, handled, err := head.ImportComponents(t.Context(), "terragrunt://live") require.NoError(t, err) require.True(t, handled) - require.Len(t, head.requests, 1, "HEAD imports create a pending import-only request") + require.Contains(t, components, "app") baseRoot := t.TempDir() + directories(t, baseRoot, "previous") rebased, err := head.ForRevision(baseRoot) require.NoError(t, err) revision := rebased.(*Session) - require.Empty(t, revision.requests, "the revision must not inherit HEAD's import-only scope") + require.Empty(t, revision.requests, "the revision must not inherit HEAD's discovery scope") + require.Empty(t, revision.imported, "the revision must not inherit HEAD's imported units") require.Equal(t, baseRoot, revision.importProviders["terragrunt"].RootDirectory) require.Equal(t, config.BasePathAbsolute, head.importProviders["terragrunt"].RootDirectory, "HEAD's provider template is unchanged") // The revision accepts its own YAML imports independently of HEAD's. - handled, err = revision.RegisterImport(t.Context(), "terragrunt://previous") + components, handled, err = revision.ImportComponents(t.Context(), "terragrunt://previous") require.NoError(t, err) require.True(t, handled) - require.Equal(t, []string{"terragrunt://previous"}, revision.requests[0].Imports) - require.Equal(t, []string{"terragrunt://live"}, head.requests[0].Imports) + require.Contains(t, components, "app") + require.Contains(t, revision.imported, "live/app/terragrunt.hcl") } -// Output reads must use the state the dependency was prepared with, not the -// state its remote_state selects under the dependent's environment. func TestPreparationRejectsDependencyStateDrift(t *testing.T) { stateAt := func(path string) *pluginv1.ComponentStateIdentity { backend, err := adapter.EncodeObject(map[string]any{"path": path}) diff --git a/pkg/plugin/adapter/configuration/settings.go b/pkg/plugin/adapter/configuration/settings.go new file mode 100644 index 00000000000..cda57477e41 --- /dev/null +++ b/pkg/plugin/adapter/configuration/settings.go @@ -0,0 +1,114 @@ +package configuration + +import ( + "fmt" + "path/filepath" + "slices" + "strings" + + errUtils "github.com/cloudposse/atmos/errors" + adapter "github.com/cloudposse/atmos/pkg/plugin/adapter/component" + pluginv1 "github.com/cloudposse/atmos/pkg/plugin/proto/atmos/plugin/v1" + "github.com/cloudposse/atmos/pkg/schema" +) + +// defaultIncludedPaths is the discovery scope when `discovery.enabled` is true and +// `discovery.included_paths` is not set: every unit below the project root. Terragrunt itself +// discovers recursively from the working directory, and failing here would make the natural +// first step (`atmos config set ...discovery.enabled true`) lock the project out of every command. +var defaultIncludedPaths = []string{"**"} + +// settingsValidator is implemented by providers that check their own settings schema locally. +type settingsValidator interface { + ValidateSettings(map[string]any) error +} + +// settingsKey returns the configuration key of a provider setting, for example +// `components.terraform.terragrunt.discovery.included_paths`. +func settingsKey(section string, parts ...string) string { + return strings.Join(append([]string{"components", "terraform", section}, parts...), ".") +} + +// invalidSetting names the offending key and how to repair it. The repair is always possible with +// `atmos config set`, which never loads or validates plugin settings. +func invalidSetting(key, problem, hint string) error { + return errUtils.Build(errUtils.ErrPluginInvalidConfig). + WithExplanationf("%s: `%s`.", problem, key). + WithContext("config_key", key). + WithHint(hint). + Err() +} + +// validateSettings checks one enabled provider section without starting a plugin. +func validateSettings(section string, settings *schema.ComponentConfiguration) error { + if settings.Mode != "" && settings.Mode != "native" { + return invalidSetting(settingsKey(section, "mode"), "Unsupported mode", "The only supported mode is `native`. Remove `mode`, or fix it with `atmos config set "+settingsKey(section, "mode")+" native`.") + } + for _, field := range []struct { + name string + patterns []string + }{{"included_paths", settings.Discovery.IncludedPaths}, {"excluded_paths", settings.Discovery.ExcludedPaths}} { + for _, pattern := range field.patterns { + if !filepath.IsLocal(pattern) { + key := settingsKey(section, "discovery", field.name) + return invalidSetting(key, fmt.Sprintf("The pattern %q must be relative to the project root and stay inside it", pattern), "Use a pattern such as `live/**`; absolute paths and `..` are not allowed.") + } + } + } + return nil +} + +// validateProviderOptions asks the provider to check keys the host does not own, such as a +// misspelled `discoverey`, so they fail loudly instead of silently doing nothing. +func validateProviderOptions(section string, provider Provider, request *pluginv1.DiscoverComponentsRequest) error { + validator, ok := provider.(settingsValidator) + if !ok { + return nil + } + values, err := adapter.DecodeObject(request.ProviderConfiguration) + if err != nil { + return err + } + if err := validator.ValidateSettings(values); err != nil { + return errUtils.Build(err).WithContext("config_section", settingsKey(section)).Err() + } + return nil +} + +// sortedSections lists the configured provider sections in stable order. +func (s *Session) sortedSections() []string { + sections := make([]string, 0, len(s.importProviders)) + for section := range s.importProviders { + sections = append(sections, section) + } + slices.Sort(sections) + return sections +} + +// validateOptions checks the provider-owned keys of every enabled section that sets any. +func (s *Session) validateOptions(registry map[string]Provider) error { + for _, section := range s.sortedSections() { + request := s.importProviders[section] + if len(s.options[section]) == 0 { + continue + } + provider := registry[request.Provider] + if provider == nil { + continue + } + if err := validateProviderOptions(section, provider, request); err != nil { + return err + } + } + return nil +} + +// hasProviderOptions reports whether any enabled section sets keys the host does not own. +func (s *Session) hasProviderOptions() bool { + for _, options := range s.options { + if len(options) > 0 { + return true + } + } + return false +} diff --git a/pkg/plugin/bootstrap/components.go b/pkg/plugin/bootstrap/components.go index 2ae00ca2d09..b80b7d80651 100644 --- a/pkg/plugin/bootstrap/components.go +++ b/pkg/plugin/bootstrap/components.go @@ -2,9 +2,9 @@ package bootstrap import ( "context" - "fmt" "maps" "slices" + "strings" errUtils "github.com/cloudposse/atmos/errors" "github.com/cloudposse/atmos/pkg/component/configuration" @@ -45,7 +45,7 @@ func LoadComponents(ctx context.Context, atmos *schema.AtmosConfiguration, refer return nil, err } if len(selected.Plugins.Packages) == 0 { - return nil, errUtils.ErrPluginUnsupportedExtension + return nil, missingProviderError(references) } temporary, err := PrepareAvailable(ctx, atmos, options...) if err != nil { @@ -66,7 +66,7 @@ func requireComponentReferences(registry *componentadapter.Registry, references for _, reference := range references { provider, exists := registry.Get(reference.Provider) if !exists || !provider.Supports(reference.Capability) { - return fmt.Errorf("%w: %s (%s)", errUtils.ErrPluginUnsupportedExtension, reference.Provider, reference.Capability) + return missingProviderError([]ComponentReference{reference}) } } return nil @@ -159,7 +159,11 @@ func selectComponentOwner(alias string, references []ComponentReference, exports continue } if owner := owners[reference]; owner != "" && owner != alias { - return false, errUtils.ErrDuplicatePluginExtension + return false, errUtils.Build(errUtils.ErrDuplicatePluginExtension). + WithExplanationf("Plugin packages `%s` and `%s` both provide `%s` (%s).", owner, alias, reference.Provider, reference.Capability). + WithContext("provider", reference.Provider). + WithHint("Keep one of them under `plugins.packages`, or disable the other with `enabled: false`."). + Err() } owners[reference] = alias matched = true @@ -194,6 +198,26 @@ func componentPackageExports(alias string, state *config.Selections, file *plugi return result, exists, nil } +// missingProviderError explains that an enabled configuration provider has no plugin behind it and how +// to supply one. +func missingProviderError(references []ComponentReference) error { + names := make([]string, 0, len(references)) + for _, reference := range references { + if !slices.Contains(names, reference.Provider) { + names = append(names, reference.Provider) + } + } + slices.Sort(names) + provider := strings.Join(names, ", ") + return errUtils.Build(errUtils.ErrPluginUnsupportedExtension). + WithExplanationf("An enabled section under `components.terraform` selects the `%s` configuration provider, but no installed plugin package provides it.", provider). + WithContext("provider", provider). + WithHint("Declare the package under `plugins.packages` with `source` and `version`, or `path` for a local build, then check it with `atmos plugin list`."). + WithHint("If the project ships a profile that supplies a local build, such as the Terragrunt example's `dev` profile, activate it with `--profile` or `ATMOS_PROFILE`."). + WithHint("To stop using the provider, set `enabled` to false in the section under `components.terraform` that selects it."). + Err() +} + func init() { // Register the constructor, never an invocation's provider or discovered stack map. if err := configuration.Register("plugin", newConfigurationSession); err != nil { diff --git a/pkg/plugin/bootstrap/local.go b/pkg/plugin/bootstrap/local.go index ddae47bd3b1..58c00197053 100644 --- a/pkg/plugin/bootstrap/local.go +++ b/pkg/plugin/bootstrap/local.go @@ -15,29 +15,34 @@ import ( "github.com/cloudposse/atmos/pkg/ui" ) -// announcedLocal remembers which aliases already printed the local-path warning for a configuration, -// so one invocation warns once per alias even though several steps register the same selection. +// announcedLocal remembers which path packages already printed the local-path warning. A CLI process +// is one invocation, but it copies its configuration for every component run and starts the plugin +// once per call, so the key is the alias and path, not the configuration object. Tests that model a +// second invocation call resetAnnouncedLocal. var announcedLocal = struct { sync.Mutex - seen map[*schema.AtmosConfiguration]map[string]bool -}{seen: make(map[*schema.AtmosConfiguration]map[string]bool)} + seen map[string]bool +}{seen: make(map[string]bool)} -// announceLocal warns once per alias that a package runs from a local path. A path package skips the -// lock, the cache, signature evidence and version selection, so the warning says so loudly. -func announceLocal(atmos *schema.AtmosConfiguration, state *config.Selections) { +// resetAnnouncedLocal forgets every announcement, as a new process would. +func resetAnnouncedLocal() { + announcedLocal.Lock() + defer announcedLocal.Unlock() + announcedLocal.seen = make(map[string]bool) +} + +// announceLocal warns once per alias and path that a package runs from a local path. A path package +// skips the lock, the cache, signature evidence and version selection, so the warning says so loudly. +func announceLocal(_ *schema.AtmosConfiguration, state *config.Selections) { announcedLocal.Lock() defer announcedLocal.Unlock() - seen := announcedLocal.seen[atmos] - if seen == nil { - seen = make(map[string]bool) - announcedLocal.seen[atmos] = seen - } for _, alias := range slices.Sorted(maps.Keys(state.Packages)) { entry := state.Packages[alias] - if !entry.Local || seen[alias] { + key := alias + "\x00" + entry.Path + if !entry.Local || announcedLocal.seen[key] { continue } - seen[alias] = true + announcedLocal.seen[key] = true ui.Warningf("Plugin `%s` runs from local path `%s`; lock, signature and version checks are bypassed.", alias, config.DisplayPath(entry.Path)) } } diff --git a/pkg/plugin/bootstrap/local_test.go b/pkg/plugin/bootstrap/local_test.go index df60348dab3..973a7cd1531 100644 --- a/pkg/plugin/bootstrap/local_test.go +++ b/pkg/plugin/bootstrap/local_test.go @@ -124,8 +124,14 @@ func TestPathPackageWarnsOncePerAliasPerInvocation(t *testing.T) { assert.Contains(t, plain, "lock, signature and version checks are bypassed") assert.Contains(t, plain, local) - // A later invocation (a new configuration) warns again. + // A copied configuration within the same process does not warn again. next := *atmos + selection := &config.Selections{Packages: map[string]config.Selection{"fixture": {Path: local, Local: true}}} + announceLocal(&next, selection) + assert.Equal(t, 1, strings.Count(ansi.Strip(output.String()), "runs from local path")) + + // A later invocation is a new process, which starts with nothing announced. + resetAnnouncedLocal() announceLocal(&next, &config.Selections{Packages: map[string]config.Selection{"fixture": {Path: local, Local: true}, "released": {Path: "elsewhere"}}}) plain = ansi.Strip(output.String()) assert.Equal(t, 2, strings.Count(plain, "runs from local path")) @@ -180,3 +186,18 @@ func TestSelectedPackagesLimitsLocalSelectionToTheRequestedAlias(t *testing.T) { require.Len(t, state.Packages, 1) require.True(t, state.Packages["other"].Local) } + +func TestMissingProviderNamesTheProviderAndHowToSupplyIt(t *testing.T) { + err := missingProviderError([]ComponentReference{{Provider: "terragrunt", Capability: "component.discovery"}, {Provider: "terragrunt", Capability: "component.preparation"}}) + require.ErrorIs(t, err, errUtils.ErrPluginUnsupportedExtension) + details := strings.Join(errUtils.AllDetails(err), "\n") + // A section may select a provider under another name, so the provider name is never a section name. + assert.NotContains(t, details, "components.terraform.terragrunt") + assert.Contains(t, details, "enabled section under `components.terraform`") + assert.Equal(t, 1, strings.Count(details, "terragrunt` configuration provider"), "a provider is named once however many capabilities were requested") + hints := strings.Join(errUtils.AllHints(err), "\n") + assert.Contains(t, hints, "plugins.packages") + assert.Contains(t, hints, "ATMOS_PROFILE") + assert.NotContains(t, hints, "components.terraform.terragrunt") + assert.Contains(t, hints, "section under `components.terraform`") +} diff --git a/pkg/plugin/proto/atmos/plugin/v1/plugin.pb.go b/pkg/plugin/proto/atmos/plugin/v1/plugin.pb.go index 26aa21b2b58..57f75fe9c17 100644 --- a/pkg/plugin/proto/atmos/plugin/v1/plugin.pb.go +++ b/pkg/plugin/proto/atmos/plugin/v1/plugin.pb.go @@ -25,6 +25,58 @@ const ( _ = protoimpl.EnforceVersion(protoimpl.MaxVersion - 20) ) +// Severity classifies an ErrorDetail. A WARNING never fails a command: the host +// prints it on the UI channel and continues. An ERROR fails the commands that +// target the named unit, or the whole command when no unit is named. +type Severity int32 + +const ( + Severity_SEVERITY_UNSPECIFIED Severity = 0 + Severity_SEVERITY_ERROR Severity = 1 + Severity_SEVERITY_WARNING Severity = 2 +) + +// Enum value maps for Severity. +var ( + Severity_name = map[int32]string{ + 0: "SEVERITY_UNSPECIFIED", + 1: "SEVERITY_ERROR", + 2: "SEVERITY_WARNING", + } + Severity_value = map[string]int32{ + "SEVERITY_UNSPECIFIED": 0, + "SEVERITY_ERROR": 1, + "SEVERITY_WARNING": 2, + } +) + +func (x Severity) Enum() *Severity { + p := new(Severity) + *p = x + return p +} + +func (x Severity) String() string { + return protoimpl.X.EnumStringOf(x.Descriptor(), protoreflect.EnumNumber(x)) +} + +func (Severity) Descriptor() protoreflect.EnumDescriptor { + return file_atmos_plugin_v1_plugin_proto_enumTypes[0].Descriptor() +} + +func (Severity) Type() protoreflect.EnumType { + return &file_atmos_plugin_v1_plugin_proto_enumTypes[0] +} + +func (x Severity) Number() protoreflect.EnumNumber { + return protoreflect.EnumNumber(x) +} + +// Deprecated: Use Severity.Descriptor instead. +func (Severity) EnumDescriptor() ([]byte, []int) { + return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{0} +} + type ComponentSource_Kind int32 const ( @@ -61,11 +113,11 @@ func (x ComponentSource_Kind) String() string { } func (ComponentSource_Kind) Descriptor() protoreflect.EnumDescriptor { - return file_atmos_plugin_v1_plugin_proto_enumTypes[0].Descriptor() + return file_atmos_plugin_v1_plugin_proto_enumTypes[1].Descriptor() } func (ComponentSource_Kind) Type() protoreflect.EnumType { - return &file_atmos_plugin_v1_plugin_proto_enumTypes[0] + return &file_atmos_plugin_v1_plugin_proto_enumTypes[1] } func (x ComponentSource_Kind) Number() protoreflect.EnumNumber { @@ -74,7 +126,7 @@ func (x ComponentSource_Kind) Number() protoreflect.EnumNumber { // Deprecated: Use ComponentSource_Kind.Descriptor instead. func (ComponentSource_Kind) EnumDescriptor() ([]byte, []int) { - return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{30, 0} + return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{32, 0} } type ComponentDependency_Kind int32 @@ -110,11 +162,11 @@ func (x ComponentDependency_Kind) String() string { } func (ComponentDependency_Kind) Descriptor() protoreflect.EnumDescriptor { - return file_atmos_plugin_v1_plugin_proto_enumTypes[1].Descriptor() + return file_atmos_plugin_v1_plugin_proto_enumTypes[2].Descriptor() } func (ComponentDependency_Kind) Type() protoreflect.EnumType { - return &file_atmos_plugin_v1_plugin_proto_enumTypes[1] + return &file_atmos_plugin_v1_plugin_proto_enumTypes[2] } func (x ComponentDependency_Kind) Number() protoreflect.EnumNumber { @@ -123,7 +175,7 @@ func (x ComponentDependency_Kind) Number() protoreflect.EnumNumber { // Deprecated: Use ComponentDependency_Kind.Descriptor instead. func (ComponentDependency_Kind) EnumDescriptor() ([]byte, []int) { - return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{31, 0} + return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{33, 0} } // Invocation identifies the live call authorized to use the Atmos API broker. @@ -981,17 +1033,229 @@ func (*AuthenticateEvent_Interaction) isAuthenticateEvent_Event() {} func (*AuthenticateEvent_Result) isAuthenticateEvent_Event() {} +// SourceLocation points at a position in a user-authored file. Paths are +// relative to the project root when the file lives below it; line and column +// are 1-based and zero means unknown. +type SourceLocation struct { + state protoimpl.MessageState `protogen:"open.v1"` + File string `protobuf:"bytes,1,opt,name=file,proto3" json:"file,omitempty"` + Line uint32 `protobuf:"varint,2,opt,name=line,proto3" json:"line,omitempty"` + Column uint32 `protobuf:"varint,3,opt,name=column,proto3" json:"column,omitempty"` + unknownFields protoimpl.UnknownFields + sizeCache protoimpl.SizeCache +} + +func (x *SourceLocation) Reset() { + *x = SourceLocation{} + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[12] + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + ms.StoreMessageInfo(mi) +} + +func (x *SourceLocation) String() string { + return protoimpl.X.MessageStringOf(x) +} + +func (*SourceLocation) ProtoMessage() {} + +func (x *SourceLocation) ProtoReflect() protoreflect.Message { + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[12] + if x != nil { + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + if ms.LoadMessageInfo() == nil { + ms.StoreMessageInfo(mi) + } + return ms + } + return mi.MessageOf(x) +} + +// Deprecated: Use SourceLocation.ProtoReflect.Descriptor instead. +func (*SourceLocation) Descriptor() ([]byte, []int) { + return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{12} +} + +func (x *SourceLocation) GetFile() string { + if x != nil { + return x.File + } + return "" +} + +func (x *SourceLocation) GetLine() uint32 { + if x != nil { + return x.Line + } + return 0 +} + +func (x *SourceLocation) GetColumn() uint32 { + if x != nil { + return x.Column + } + return 0 +} + +// ErrorDetail is the structured, user-facing description of a plugin failure. +// A plugin returns it as a gRPC status detail (google.rpc.Status.details) on a +// failed call, or inside Diagnostic.detail for a per-item failure that must not +// fail the whole call. The host renders it with the Atmos error builder. +// +// Secret safety contract: code, summary, hints, location, unit, stack, +// config_key, and operation are always shown and MUST NOT contain evaluated +// values. detail is shown only when user_safe is true. When user_safe is false +// the host omits detail from the error, names the unit and operation, hints at +// the debug log level, and records the masked detail in the debug log only. +// The host also applies secret masking to every field before display. +type ErrorDetail struct { + state protoimpl.MessageState `protogen:"open.v1"` + // Stable machine-readable identifier such as "unsupported_setting" or "syntax_error". + Code string `protobuf:"bytes,1,opt,name=code,proto3" json:"code,omitempty"` + // One-line headline naming what failed, for example "Unsupported Terragrunt setting". + Summary string `protobuf:"bytes,2,opt,name=summary,proto3" json:"summary,omitempty"` + // Longer explanation. Shown only when user_safe is true. + Detail string `protobuf:"bytes,3,opt,name=detail,proto3" json:"detail,omitempty"` + // Actionable remedies, shown as hints in order. + Hints []string `protobuf:"bytes,4,rep,name=hints,proto3" json:"hints,omitempty"` + // Where the problem is authored. + Location *SourceLocation `protobuf:"bytes,5,opt,name=location,proto3" json:"location,omitempty"` + // Component (unit) name, in Atmos terms. + Unit string `protobuf:"bytes,6,opt,name=unit,proto3" json:"unit,omitempty"` + // Stack the unit belongs to. + Stack string `protobuf:"bytes,7,opt,name=stack,proto3" json:"stack,omitempty"` + // Configuration key, setting, or block that caused the failure, for example "terraform_binary". + ConfigKey string `protobuf:"bytes,8,opt,name=config_key,json=configKey,proto3" json:"config_key,omitempty"` + // True when detail contains no evaluated values and is safe to print. + UserSafe bool `protobuf:"varint,9,opt,name=user_safe,json=userSafe,proto3" json:"user_safe,omitempty"` + // The operation that failed, such as "discover", "prepare", or "validate". + Operation string `protobuf:"bytes,10,opt,name=operation,proto3" json:"operation,omitempty"` + // Severity of the problem. Unset means ERROR. + Severity Severity `protobuf:"varint,11,opt,name=severity,proto3,enum=atmos.plugin.v1.Severity" json:"severity,omitempty"` + unknownFields protoimpl.UnknownFields + sizeCache protoimpl.SizeCache +} + +func (x *ErrorDetail) Reset() { + *x = ErrorDetail{} + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[13] + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + ms.StoreMessageInfo(mi) +} + +func (x *ErrorDetail) String() string { + return protoimpl.X.MessageStringOf(x) +} + +func (*ErrorDetail) ProtoMessage() {} + +func (x *ErrorDetail) ProtoReflect() protoreflect.Message { + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[13] + if x != nil { + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + if ms.LoadMessageInfo() == nil { + ms.StoreMessageInfo(mi) + } + return ms + } + return mi.MessageOf(x) +} + +// Deprecated: Use ErrorDetail.ProtoReflect.Descriptor instead. +func (*ErrorDetail) Descriptor() ([]byte, []int) { + return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{13} +} + +func (x *ErrorDetail) GetCode() string { + if x != nil { + return x.Code + } + return "" +} + +func (x *ErrorDetail) GetSummary() string { + if x != nil { + return x.Summary + } + return "" +} + +func (x *ErrorDetail) GetDetail() string { + if x != nil { + return x.Detail + } + return "" +} + +func (x *ErrorDetail) GetHints() []string { + if x != nil { + return x.Hints + } + return nil +} + +func (x *ErrorDetail) GetLocation() *SourceLocation { + if x != nil { + return x.Location + } + return nil +} + +func (x *ErrorDetail) GetUnit() string { + if x != nil { + return x.Unit + } + return "" +} + +func (x *ErrorDetail) GetStack() string { + if x != nil { + return x.Stack + } + return "" +} + +func (x *ErrorDetail) GetConfigKey() string { + if x != nil { + return x.ConfigKey + } + return "" +} + +func (x *ErrorDetail) GetUserSafe() bool { + if x != nil { + return x.UserSafe + } + return false +} + +func (x *ErrorDetail) GetOperation() string { + if x != nil { + return x.Operation + } + return "" +} + +func (x *ErrorDetail) GetSeverity() Severity { + if x != nil { + return x.Severity + } + return Severity_SEVERITY_UNSPECIFIED +} + +// Diagnostic describes one problem. field and message keep their original meaning; +// detail adds the structured form and takes precedence when present. type Diagnostic struct { state protoimpl.MessageState `protogen:"open.v1"` Field string `protobuf:"bytes,1,opt,name=field,proto3" json:"field,omitempty"` Message string `protobuf:"bytes,2,opt,name=message,proto3" json:"message,omitempty"` + Detail *ErrorDetail `protobuf:"bytes,3,opt,name=detail,proto3" json:"detail,omitempty"` unknownFields protoimpl.UnknownFields sizeCache protoimpl.SizeCache } func (x *Diagnostic) Reset() { *x = Diagnostic{} - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[12] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[14] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -1003,7 +1267,7 @@ func (x *Diagnostic) String() string { func (*Diagnostic) ProtoMessage() {} func (x *Diagnostic) ProtoReflect() protoreflect.Message { - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[12] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[14] if x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -1016,7 +1280,7 @@ func (x *Diagnostic) ProtoReflect() protoreflect.Message { // Deprecated: Use Diagnostic.ProtoReflect.Descriptor instead. func (*Diagnostic) Descriptor() ([]byte, []int) { - return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{12} + return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{14} } func (x *Diagnostic) GetField() string { @@ -1033,6 +1297,13 @@ func (x *Diagnostic) GetMessage() string { return "" } +func (x *Diagnostic) GetDetail() *ErrorDetail { + if x != nil { + return x.Detail + } + return nil +} + type ValidateResponse struct { state protoimpl.MessageState `protogen:"open.v1"` Diagnostics []*Diagnostic `protobuf:"bytes,1,rep,name=diagnostics,proto3" json:"diagnostics,omitempty"` @@ -1042,7 +1313,7 @@ type ValidateResponse struct { func (x *ValidateResponse) Reset() { *x = ValidateResponse{} - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[13] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[15] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -1054,7 +1325,7 @@ func (x *ValidateResponse) String() string { func (*ValidateResponse) ProtoMessage() {} func (x *ValidateResponse) ProtoReflect() protoreflect.Message { - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[13] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[15] if x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -1067,7 +1338,7 @@ func (x *ValidateResponse) ProtoReflect() protoreflect.Message { // Deprecated: Use ValidateResponse.ProtoReflect.Descriptor instead. func (*ValidateResponse) Descriptor() ([]byte, []int) { - return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{13} + return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{15} } func (x *ValidateResponse) GetDiagnostics() []*Diagnostic { @@ -1087,7 +1358,7 @@ type Unsupported struct { func (x *Unsupported) Reset() { *x = Unsupported{} - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[14] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[16] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -1099,7 +1370,7 @@ func (x *Unsupported) String() string { func (*Unsupported) ProtoMessage() {} func (x *Unsupported) ProtoReflect() protoreflect.Message { - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[14] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[16] if x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -1112,7 +1383,7 @@ func (x *Unsupported) ProtoReflect() protoreflect.Message { // Deprecated: Use Unsupported.ProtoReflect.Descriptor instead. func (*Unsupported) Descriptor() ([]byte, []int) { - return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{14} + return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{16} } func (x *Unsupported) GetOperation() string { @@ -1139,7 +1410,7 @@ type RefreshRequest struct { func (x *RefreshRequest) Reset() { *x = RefreshRequest{} - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[15] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[17] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -1151,7 +1422,7 @@ func (x *RefreshRequest) String() string { func (*RefreshRequest) ProtoMessage() {} func (x *RefreshRequest) ProtoReflect() protoreflect.Message { - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[15] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[17] if x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -1164,7 +1435,7 @@ func (x *RefreshRequest) ProtoReflect() protoreflect.Message { // Deprecated: Use RefreshRequest.ProtoReflect.Descriptor instead. func (*RefreshRequest) Descriptor() ([]byte, []int) { - return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{15} + return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{17} } func (x *RefreshRequest) GetProvider() *ProviderRequest { @@ -1194,7 +1465,7 @@ type RefreshResponse struct { func (x *RefreshResponse) Reset() { *x = RefreshResponse{} - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[16] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[18] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -1206,7 +1477,7 @@ func (x *RefreshResponse) String() string { func (*RefreshResponse) ProtoMessage() {} func (x *RefreshResponse) ProtoReflect() protoreflect.Message { - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[16] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[18] if x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -1219,7 +1490,7 @@ func (x *RefreshResponse) ProtoReflect() protoreflect.Message { // Deprecated: Use RefreshResponse.ProtoReflect.Descriptor instead. func (*RefreshResponse) Descriptor() ([]byte, []int) { - return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{16} + return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{18} } func (x *RefreshResponse) GetResult() isRefreshResponse_Result { @@ -1272,7 +1543,7 @@ type EnvironmentResponse struct { func (x *EnvironmentResponse) Reset() { *x = EnvironmentResponse{} - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[17] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[19] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -1284,7 +1555,7 @@ func (x *EnvironmentResponse) String() string { func (*EnvironmentResponse) ProtoMessage() {} func (x *EnvironmentResponse) ProtoReflect() protoreflect.Message { - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[17] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[19] if x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -1297,7 +1568,7 @@ func (x *EnvironmentResponse) ProtoReflect() protoreflect.Message { // Deprecated: Use EnvironmentResponse.ProtoReflect.Descriptor instead. func (*EnvironmentResponse) Descriptor() ([]byte, []int) { - return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{17} + return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{19} } func (x *EnvironmentResponse) GetEnvironment() map[string]string { @@ -1317,7 +1588,7 @@ type PrepareEnvironmentRequest struct { func (x *PrepareEnvironmentRequest) Reset() { *x = PrepareEnvironmentRequest{} - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[18] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[20] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -1329,7 +1600,7 @@ func (x *PrepareEnvironmentRequest) String() string { func (*PrepareEnvironmentRequest) ProtoMessage() {} func (x *PrepareEnvironmentRequest) ProtoReflect() protoreflect.Message { - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[18] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[20] if x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -1342,7 +1613,7 @@ func (x *PrepareEnvironmentRequest) ProtoReflect() protoreflect.Message { // Deprecated: Use PrepareEnvironmentRequest.ProtoReflect.Descriptor instead. func (*PrepareEnvironmentRequest) Descriptor() ([]byte, []int) { - return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{18} + return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{20} } func (x *PrepareEnvironmentRequest) GetProvider() *ProviderRequest { @@ -1371,7 +1642,7 @@ type CredentialPath struct { func (x *CredentialPath) Reset() { *x = CredentialPath{} - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[19] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[21] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -1383,7 +1654,7 @@ func (x *CredentialPath) String() string { func (*CredentialPath) ProtoMessage() {} func (x *CredentialPath) ProtoReflect() protoreflect.Message { - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[19] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[21] if x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -1396,7 +1667,7 @@ func (x *CredentialPath) ProtoReflect() protoreflect.Message { // Deprecated: Use CredentialPath.ProtoReflect.Descriptor instead. func (*CredentialPath) Descriptor() ([]byte, []int) { - return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{19} + return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{21} } func (x *CredentialPath) GetLocation() string { @@ -1436,7 +1707,7 @@ type PathsResponse struct { func (x *PathsResponse) Reset() { *x = PathsResponse{} - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[20] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[22] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -1448,7 +1719,7 @@ func (x *PathsResponse) String() string { func (*PathsResponse) ProtoMessage() {} func (x *PathsResponse) ProtoReflect() protoreflect.Message { - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[20] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[22] if x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -1461,7 +1732,7 @@ func (x *PathsResponse) ProtoReflect() protoreflect.Message { // Deprecated: Use PathsResponse.ProtoReflect.Descriptor instead. func (*PathsResponse) Descriptor() ([]byte, []int) { - return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{20} + return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{22} } func (x *PathsResponse) GetPaths() []*CredentialPath { @@ -1481,7 +1752,7 @@ type IdentityAuthenticateRequest struct { func (x *IdentityAuthenticateRequest) Reset() { *x = IdentityAuthenticateRequest{} - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[21] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[23] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -1493,7 +1764,7 @@ func (x *IdentityAuthenticateRequest) String() string { func (*IdentityAuthenticateRequest) ProtoMessage() {} func (x *IdentityAuthenticateRequest) ProtoReflect() protoreflect.Message { - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[21] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[23] if x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -1506,7 +1777,7 @@ func (x *IdentityAuthenticateRequest) ProtoReflect() protoreflect.Message { // Deprecated: Use IdentityAuthenticateRequest.ProtoReflect.Descriptor instead. func (*IdentityAuthenticateRequest) Descriptor() ([]byte, []int) { - return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{21} + return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{23} } func (x *IdentityAuthenticateRequest) GetIdentity() *IdentityRequest { @@ -1533,7 +1804,7 @@ type IdentityPrepareRequest struct { func (x *IdentityPrepareRequest) Reset() { *x = IdentityPrepareRequest{} - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[22] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[24] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -1545,7 +1816,7 @@ func (x *IdentityPrepareRequest) String() string { func (*IdentityPrepareRequest) ProtoMessage() {} func (x *IdentityPrepareRequest) ProtoReflect() protoreflect.Message { - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[22] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[24] if x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -1558,7 +1829,7 @@ func (x *IdentityPrepareRequest) ProtoReflect() protoreflect.Message { // Deprecated: Use IdentityPrepareRequest.ProtoReflect.Descriptor instead. func (*IdentityPrepareRequest) Descriptor() ([]byte, []int) { - return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{22} + return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{24} } func (x *IdentityPrepareRequest) GetIdentity() *IdentityRequest { @@ -1588,7 +1859,7 @@ type CommandRequest struct { func (x *CommandRequest) Reset() { *x = CommandRequest{} - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[23] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[25] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -1600,7 +1871,7 @@ func (x *CommandRequest) String() string { func (*CommandRequest) ProtoMessage() {} func (x *CommandRequest) ProtoReflect() protoreflect.Message { - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[23] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[25] if x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -1613,7 +1884,7 @@ func (x *CommandRequest) ProtoReflect() protoreflect.Message { // Deprecated: Use CommandRequest.ProtoReflect.Descriptor instead. func (*CommandRequest) Descriptor() ([]byte, []int) { - return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{23} + return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{25} } func (x *CommandRequest) GetInvocation() *Invocation { @@ -1653,7 +1924,7 @@ type CommandExit struct { func (x *CommandExit) Reset() { *x = CommandExit{} - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[24] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[26] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -1665,7 +1936,7 @@ func (x *CommandExit) String() string { func (*CommandExit) ProtoMessage() {} func (x *CommandExit) ProtoReflect() protoreflect.Message { - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[24] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[26] if x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -1678,7 +1949,7 @@ func (x *CommandExit) ProtoReflect() protoreflect.Message { // Deprecated: Use CommandExit.ProtoReflect.Descriptor instead. func (*CommandExit) Descriptor() ([]byte, []int) { - return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{24} + return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{26} } func (x *CommandExit) GetCode() uint32 { @@ -1702,7 +1973,7 @@ type CommandEvent struct { func (x *CommandEvent) Reset() { *x = CommandEvent{} - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[25] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[27] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -1714,7 +1985,7 @@ func (x *CommandEvent) String() string { func (*CommandEvent) ProtoMessage() {} func (x *CommandEvent) ProtoReflect() protoreflect.Message { - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[25] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[27] if x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -1727,7 +1998,7 @@ func (x *CommandEvent) ProtoReflect() protoreflect.Message { // Deprecated: Use CommandEvent.ProtoReflect.Descriptor instead. func (*CommandEvent) Descriptor() ([]byte, []int) { - return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{25} + return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{27} } func (x *CommandEvent) GetEvent() isCommandEvent_Event { @@ -1801,7 +2072,7 @@ type ComponentValue struct { func (x *ComponentValue) Reset() { *x = ComponentValue{} - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[26] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[28] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -1813,7 +2084,7 @@ func (x *ComponentValue) String() string { func (*ComponentValue) ProtoMessage() {} func (x *ComponentValue) ProtoReflect() protoreflect.Message { - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[26] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[28] if x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -1826,7 +2097,7 @@ func (x *ComponentValue) ProtoReflect() protoreflect.Message { // Deprecated: Use ComponentValue.ProtoReflect.Descriptor instead. func (*ComponentValue) Descriptor() ([]byte, []int) { - return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{26} + return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{28} } func (x *ComponentValue) GetTypeJson() []byte { @@ -1859,7 +2130,7 @@ type ComponentValuePath struct { func (x *ComponentValuePath) Reset() { *x = ComponentValuePath{} - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[27] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[29] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -1871,7 +2142,7 @@ func (x *ComponentValuePath) String() string { func (*ComponentValuePath) ProtoMessage() {} func (x *ComponentValuePath) ProtoReflect() protoreflect.Message { - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[27] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[29] if x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -1884,7 +2155,7 @@ func (x *ComponentValuePath) ProtoReflect() protoreflect.Message { // Deprecated: Use ComponentValuePath.ProtoReflect.Descriptor instead. func (*ComponentValuePath) Descriptor() ([]byte, []int) { - return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{27} + return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{29} } func (x *ComponentValuePath) GetSteps() []*ComponentValueStep { @@ -1907,7 +2178,7 @@ type ComponentValueStep struct { func (x *ComponentValueStep) Reset() { *x = ComponentValueStep{} - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[28] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[30] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -1919,7 +2190,7 @@ func (x *ComponentValueStep) String() string { func (*ComponentValueStep) ProtoMessage() {} func (x *ComponentValueStep) ProtoReflect() protoreflect.Message { - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[28] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[30] if x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -1932,7 +2203,7 @@ func (x *ComponentValueStep) ProtoReflect() protoreflect.Message { // Deprecated: Use ComponentValueStep.ProtoReflect.Descriptor instead. func (*ComponentValueStep) Descriptor() ([]byte, []int) { - return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{28} + return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{30} } func (x *ComponentValueStep) GetSelector() isComponentValueStep_Selector { @@ -1989,7 +2260,7 @@ type ComponentAddress struct { func (x *ComponentAddress) Reset() { *x = ComponentAddress{} - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[29] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[31] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -2001,7 +2272,7 @@ func (x *ComponentAddress) String() string { func (*ComponentAddress) ProtoMessage() {} func (x *ComponentAddress) ProtoReflect() protoreflect.Message { - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[29] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[31] if x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -2014,7 +2285,7 @@ func (x *ComponentAddress) ProtoReflect() protoreflect.Message { // Deprecated: Use ComponentAddress.ProtoReflect.Descriptor instead. func (*ComponentAddress) Descriptor() ([]byte, []int) { - return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{29} + return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{31} } func (x *ComponentAddress) GetStack() string { @@ -2051,7 +2322,7 @@ type ComponentSource struct { func (x *ComponentSource) Reset() { *x = ComponentSource{} - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[30] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[32] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -2063,7 +2334,7 @@ func (x *ComponentSource) String() string { func (*ComponentSource) ProtoMessage() {} func (x *ComponentSource) ProtoReflect() protoreflect.Message { - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[30] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[32] if x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -2076,7 +2347,7 @@ func (x *ComponentSource) ProtoReflect() protoreflect.Message { // Deprecated: Use ComponentSource.ProtoReflect.Descriptor instead. func (*ComponentSource) Descriptor() ([]byte, []int) { - return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{30} + return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{32} } func (x *ComponentSource) GetPath() string { @@ -2110,7 +2381,7 @@ type ComponentDependency struct { func (x *ComponentDependency) Reset() { *x = ComponentDependency{} - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[31] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[33] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -2122,7 +2393,7 @@ func (x *ComponentDependency) String() string { func (*ComponentDependency) ProtoMessage() {} func (x *ComponentDependency) ProtoReflect() protoreflect.Message { - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[31] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[33] if x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -2135,7 +2406,7 @@ func (x *ComponentDependency) ProtoReflect() protoreflect.Message { // Deprecated: Use ComponentDependency.ProtoReflect.Descriptor instead. func (*ComponentDependency) Descriptor() ([]byte, []int) { - return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{31} + return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{33} } func (x *ComponentDependency) GetTarget() *ComponentAddress { @@ -2167,7 +2438,7 @@ type DiscoveredComponent struct { func (x *DiscoveredComponent) Reset() { *x = DiscoveredComponent{} - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[32] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[34] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -2179,7 +2450,7 @@ func (x *DiscoveredComponent) String() string { func (*DiscoveredComponent) ProtoMessage() {} func (x *DiscoveredComponent) ProtoReflect() protoreflect.Message { - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[32] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[34] if x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -2192,7 +2463,7 @@ func (x *DiscoveredComponent) ProtoReflect() protoreflect.Message { // Deprecated: Use DiscoveredComponent.ProtoReflect.Descriptor instead. func (*DiscoveredComponent) Descriptor() ([]byte, []int) { - return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{32} + return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{34} } func (x *DiscoveredComponent) GetAddress() *ComponentAddress { @@ -2259,7 +2530,7 @@ type ComponentEffects struct { func (x *ComponentEffects) Reset() { *x = ComponentEffects{} - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[33] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[35] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -2271,7 +2542,7 @@ func (x *ComponentEffects) String() string { func (*ComponentEffects) ProtoMessage() {} func (x *ComponentEffects) ProtoReflect() protoreflect.Message { - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[33] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[35] if x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -2284,7 +2555,7 @@ func (x *ComponentEffects) ProtoReflect() protoreflect.Message { // Deprecated: Use ComponentEffects.ProtoReflect.Descriptor instead. func (*ComponentEffects) Descriptor() ([]byte, []int) { - return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{33} + return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{35} } func (x *ComponentEffects) GetCommands() bool { @@ -2339,7 +2610,7 @@ type ComponentOperation struct { func (x *ComponentOperation) Reset() { *x = ComponentOperation{} - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[34] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[36] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -2351,7 +2622,7 @@ func (x *ComponentOperation) String() string { func (*ComponentOperation) ProtoMessage() {} func (x *ComponentOperation) ProtoReflect() protoreflect.Message { - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[34] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[36] if x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -2364,7 +2635,7 @@ func (x *ComponentOperation) ProtoReflect() protoreflect.Message { // Deprecated: Use ComponentOperation.ProtoReflect.Descriptor instead. func (*ComponentOperation) Descriptor() ([]byte, []int) { - return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{34} + return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{36} } func (x *ComponentOperation) GetCommand() string { @@ -2415,7 +2686,7 @@ type DiscoverComponentsRequest struct { func (x *DiscoverComponentsRequest) Reset() { *x = DiscoverComponentsRequest{} - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[35] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[37] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -2427,7 +2698,7 @@ func (x *DiscoverComponentsRequest) String() string { func (*DiscoverComponentsRequest) ProtoMessage() {} func (x *DiscoverComponentsRequest) ProtoReflect() protoreflect.Message { - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[35] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[37] if x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -2440,7 +2711,7 @@ func (x *DiscoverComponentsRequest) ProtoReflect() protoreflect.Message { // Deprecated: Use DiscoverComponentsRequest.ProtoReflect.Descriptor instead. func (*DiscoverComponentsRequest) Descriptor() ([]byte, []int) { - return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{35} + return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{37} } func (x *DiscoverComponentsRequest) GetInvocation() *Invocation { @@ -2514,16 +2785,20 @@ func (x *DiscoverComponentsRequest) GetProviderConfiguration() *ComponentValue { } type DiscoverComponentsResponse struct { - state protoimpl.MessageState `protogen:"open.v1"` - Components []*DiscoveredComponent `protobuf:"bytes,1,rep,name=components,proto3" json:"components,omitempty"` - Diagnostics []*Diagnostic `protobuf:"bytes,2,rep,name=diagnostics,proto3" json:"diagnostics,omitempty"` + state protoimpl.MessageState `protogen:"open.v1"` + // Healthy components. One failing component must not hide the others. + Components []*DiscoveredComponent `protobuf:"bytes,1,rep,name=components,proto3" json:"components,omitempty"` + // Per-component failures. A diagnostic whose detail names a unit and stack is + // reported only to commands that target that component; others keep working. + // A diagnostic without a unit is a project-level failure. + Diagnostics []*Diagnostic `protobuf:"bytes,2,rep,name=diagnostics,proto3" json:"diagnostics,omitempty"` unknownFields protoimpl.UnknownFields sizeCache protoimpl.SizeCache } func (x *DiscoverComponentsResponse) Reset() { *x = DiscoverComponentsResponse{} - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[36] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[38] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -2535,7 +2810,7 @@ func (x *DiscoverComponentsResponse) String() string { func (*DiscoverComponentsResponse) ProtoMessage() {} func (x *DiscoverComponentsResponse) ProtoReflect() protoreflect.Message { - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[36] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[38] if x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -2548,7 +2823,7 @@ func (x *DiscoverComponentsResponse) ProtoReflect() protoreflect.Message { // Deprecated: Use DiscoverComponentsResponse.ProtoReflect.Descriptor instead. func (*DiscoverComponentsResponse) Descriptor() ([]byte, []int) { - return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{36} + return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{38} } func (x *DiscoverComponentsResponse) GetComponents() []*DiscoveredComponent { @@ -2582,7 +2857,7 @@ type ComponentRequest struct { func (x *ComponentRequest) Reset() { *x = ComponentRequest{} - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[37] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[39] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -2594,7 +2869,7 @@ func (x *ComponentRequest) String() string { func (*ComponentRequest) ProtoMessage() {} func (x *ComponentRequest) ProtoReflect() protoreflect.Message { - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[37] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[39] if x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -2607,7 +2882,7 @@ func (x *ComponentRequest) ProtoReflect() protoreflect.Message { // Deprecated: Use ComponentRequest.ProtoReflect.Descriptor instead. func (*ComponentRequest) Descriptor() ([]byte, []int) { - return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{37} + return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{39} } func (x *ComponentRequest) GetInvocation() *Invocation { @@ -2687,7 +2962,7 @@ type ComponentStateIdentity struct { func (x *ComponentStateIdentity) Reset() { *x = ComponentStateIdentity{} - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[38] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[40] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -2699,7 +2974,7 @@ func (x *ComponentStateIdentity) String() string { func (*ComponentStateIdentity) ProtoMessage() {} func (x *ComponentStateIdentity) ProtoReflect() protoreflect.Message { - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[38] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[40] if x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -2712,7 +2987,7 @@ func (x *ComponentStateIdentity) ProtoReflect() protoreflect.Message { // Deprecated: Use ComponentStateIdentity.ProtoReflect.Descriptor instead. func (*ComponentStateIdentity) Descriptor() ([]byte, []int) { - return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{38} + return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{40} } func (x *ComponentStateIdentity) GetBackendType() string { @@ -2754,14 +3029,17 @@ type ComponentGeneratedFile struct { // Optional POSIX permission bits. Zero means the host default (0644, or 0600 // for sensitive files). The host keeps only owner/group/other read and // execute bits plus owner write, and ignores the field on Windows. - Mode uint32 `protobuf:"varint,5,opt,name=mode,proto3" json:"mode,omitempty"` + Mode uint32 `protobuf:"varint,5,opt,name=mode,proto3" json:"mode,omitempty"` + // True only for a literal file copied from beside the configuration unit, + // never for an explicit generate block. Absent means normal policy handling. + UnitFileCopy bool `protobuf:"varint,6,opt,name=unit_file_copy,json=unitFileCopy,proto3" json:"unit_file_copy,omitempty"` unknownFields protoimpl.UnknownFields sizeCache protoimpl.SizeCache } func (x *ComponentGeneratedFile) Reset() { *x = ComponentGeneratedFile{} - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[39] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[41] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -2773,7 +3051,7 @@ func (x *ComponentGeneratedFile) String() string { func (*ComponentGeneratedFile) ProtoMessage() {} func (x *ComponentGeneratedFile) ProtoReflect() protoreflect.Message { - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[39] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[41] if x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -2786,7 +3064,7 @@ func (x *ComponentGeneratedFile) ProtoReflect() protoreflect.Message { // Deprecated: Use ComponentGeneratedFile.ProtoReflect.Descriptor instead. func (*ComponentGeneratedFile) Descriptor() ([]byte, []int) { - return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{39} + return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{41} } func (x *ComponentGeneratedFile) GetPath() string { @@ -2824,6 +3102,13 @@ func (x *ComponentGeneratedFile) GetMode() uint32 { return 0 } +func (x *ComponentGeneratedFile) GetUnitFileCopy() bool { + if x != nil { + return x.UnitFileCopy + } + return false +} + // Evaluated command metadata; the host executes it with the invocation's // environment, toolchain, cancellation, and I/O. Lists retain declaration order. type ComponentExecutionConfiguration struct { @@ -2844,7 +3129,7 @@ type ComponentExecutionConfiguration struct { func (x *ComponentExecutionConfiguration) Reset() { *x = ComponentExecutionConfiguration{} - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[40] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[42] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -2856,7 +3141,7 @@ func (x *ComponentExecutionConfiguration) String() string { func (*ComponentExecutionConfiguration) ProtoMessage() {} func (x *ComponentExecutionConfiguration) ProtoReflect() protoreflect.Message { - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[40] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[42] if x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -2869,7 +3154,7 @@ func (x *ComponentExecutionConfiguration) ProtoReflect() protoreflect.Message { // Deprecated: Use ComponentExecutionConfiguration.ProtoReflect.Descriptor instead. func (*ComponentExecutionConfiguration) Descriptor() ([]byte, []int) { - return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{40} + return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{42} } func (x *ComponentExecutionConfiguration) GetExtraArguments() []*ComponentExtraArguments { @@ -2920,7 +3205,7 @@ type ComponentExtraArguments struct { func (x *ComponentExtraArguments) Reset() { *x = ComponentExtraArguments{} - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[41] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[43] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -2932,7 +3217,7 @@ func (x *ComponentExtraArguments) String() string { func (*ComponentExtraArguments) ProtoMessage() {} func (x *ComponentExtraArguments) ProtoReflect() protoreflect.Message { - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[41] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[43] if x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -2945,7 +3230,7 @@ func (x *ComponentExtraArguments) ProtoReflect() protoreflect.Message { // Deprecated: Use ComponentExtraArguments.ProtoReflect.Descriptor instead. func (*ComponentExtraArguments) Descriptor() ([]byte, []int) { - return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{41} + return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{43} } func (x *ComponentExtraArguments) GetCommands() []string { @@ -3000,7 +3285,7 @@ type ComponentHook struct { func (x *ComponentHook) Reset() { *x = ComponentHook{} - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[42] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[44] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -3012,7 +3297,7 @@ func (x *ComponentHook) String() string { func (*ComponentHook) ProtoMessage() {} func (x *ComponentHook) ProtoReflect() protoreflect.Message { - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[42] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[44] if x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -3025,7 +3310,7 @@ func (x *ComponentHook) ProtoReflect() protoreflect.Message { // Deprecated: Use ComponentHook.ProtoReflect.Descriptor instead. func (*ComponentHook) Descriptor() ([]byte, []int) { - return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{42} + return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{44} } func (x *ComponentHook) GetName() string { @@ -3092,13 +3377,31 @@ type PrepareComponentResponse struct { // emit its default backend file. BackendGenerated bool `protobuf:"varint,10,opt,name=backend_generated,json=backendGenerated,proto3" json:"backend_generated,omitempty"` Execution *ComponentExecutionConfiguration `protobuf:"bytes,11,opt,name=execution,proto3" json:"execution,omitempty"` - unknownFields protoimpl.UnknownFields - sizeCache protoimpl.SizeCache + // Maps each input name to the authored file, line, and column that produced + // its value, for provenance reporting. Missing entries mean unknown origin. + InputOrigins map[string]*SourceLocation `protobuf:"bytes,12,rep,name=input_origins,json=inputOrigins,proto3" json:"input_origins,omitempty" protobuf_key:"bytes,1,opt,name=key" protobuf_val:"bytes,2,opt,name=value"` + // Fields 13-18 are reserved for Terragrunt parity. The host does not read + // them yet; a plugin that cannot honor a setting must reject it with an + // ErrorDetail instead of returning it, and must leave these fields unset. + // Engine executable selected by terraform_binary. + TerraformBinary string `protobuf:"bytes,13,opt,name=terraform_binary,json=terraformBinary,proto3" json:"terraform_binary,omitempty"` + // errors block: retry and ignore policies. + Errors *ComponentErrorsConfiguration `protobuf:"bytes,14,opt,name=errors,proto3" json:"errors,omitempty"` + // exclude block: conditional unit exclusion. + Exclude *ComponentExclude `protobuf:"bytes,15,opt,name=exclude,proto3" json:"exclude,omitempty"` + // iam_role, iam_assume_role_session_name, iam_assume_role_duration, iam_web_identity_token. + Iam *ComponentIam `protobuf:"bytes,16,opt,name=iam,proto3" json:"iam,omitempty"` + // encryption block as opaque, provider-defined data. + Encryption *ComponentValue `protobuf:"bytes,17,opt,name=encryption,proto3" json:"encryption,omitempty"` + // feature flag name to its evaluated value rendered as a string. + FeatureFlags map[string]string `protobuf:"bytes,18,rep,name=feature_flags,json=featureFlags,proto3" json:"feature_flags,omitempty" protobuf_key:"bytes,1,opt,name=key" protobuf_val:"bytes,2,opt,name=value"` + unknownFields protoimpl.UnknownFields + sizeCache protoimpl.SizeCache } func (x *PrepareComponentResponse) Reset() { *x = PrepareComponentResponse{} - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[43] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[45] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -3110,7 +3413,7 @@ func (x *PrepareComponentResponse) String() string { func (*PrepareComponentResponse) ProtoMessage() {} func (x *PrepareComponentResponse) ProtoReflect() protoreflect.Message { - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[43] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[45] if x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -3123,7 +3426,7 @@ func (x *PrepareComponentResponse) ProtoReflect() protoreflect.Message { // Deprecated: Use PrepareComponentResponse.ProtoReflect.Descriptor instead. func (*PrepareComponentResponse) Descriptor() ([]byte, []int) { - return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{43} + return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{45} } func (x *PrepareComponentResponse) GetEngine() string { @@ -3203,6 +3506,380 @@ func (x *PrepareComponentResponse) GetExecution() *ComponentExecutionConfigurati return nil } +func (x *PrepareComponentResponse) GetInputOrigins() map[string]*SourceLocation { + if x != nil { + return x.InputOrigins + } + return nil +} + +func (x *PrepareComponentResponse) GetTerraformBinary() string { + if x != nil { + return x.TerraformBinary + } + return "" +} + +func (x *PrepareComponentResponse) GetErrors() *ComponentErrorsConfiguration { + if x != nil { + return x.Errors + } + return nil +} + +func (x *PrepareComponentResponse) GetExclude() *ComponentExclude { + if x != nil { + return x.Exclude + } + return nil +} + +func (x *PrepareComponentResponse) GetIam() *ComponentIam { + if x != nil { + return x.Iam + } + return nil +} + +func (x *PrepareComponentResponse) GetEncryption() *ComponentValue { + if x != nil { + return x.Encryption + } + return nil +} + +func (x *PrepareComponentResponse) GetFeatureFlags() map[string]string { + if x != nil { + return x.FeatureFlags + } + return nil +} + +// ComponentErrorsConfiguration mirrors the Terragrunt errors block. Reserved; not implemented. +type ComponentErrorsConfiguration struct { + state protoimpl.MessageState `protogen:"open.v1"` + Retry []*ComponentRetryPolicy `protobuf:"bytes,1,rep,name=retry,proto3" json:"retry,omitempty"` + Ignore []*ComponentIgnorePolicy `protobuf:"bytes,2,rep,name=ignore,proto3" json:"ignore,omitempty"` + unknownFields protoimpl.UnknownFields + sizeCache protoimpl.SizeCache +} + +func (x *ComponentErrorsConfiguration) Reset() { + *x = ComponentErrorsConfiguration{} + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[46] + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + ms.StoreMessageInfo(mi) +} + +func (x *ComponentErrorsConfiguration) String() string { + return protoimpl.X.MessageStringOf(x) +} + +func (*ComponentErrorsConfiguration) ProtoMessage() {} + +func (x *ComponentErrorsConfiguration) ProtoReflect() protoreflect.Message { + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[46] + if x != nil { + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + if ms.LoadMessageInfo() == nil { + ms.StoreMessageInfo(mi) + } + return ms + } + return mi.MessageOf(x) +} + +// Deprecated: Use ComponentErrorsConfiguration.ProtoReflect.Descriptor instead. +func (*ComponentErrorsConfiguration) Descriptor() ([]byte, []int) { + return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{46} +} + +func (x *ComponentErrorsConfiguration) GetRetry() []*ComponentRetryPolicy { + if x != nil { + return x.Retry + } + return nil +} + +func (x *ComponentErrorsConfiguration) GetIgnore() []*ComponentIgnorePolicy { + if x != nil { + return x.Ignore + } + return nil +} + +type ComponentRetryPolicy struct { + state protoimpl.MessageState `protogen:"open.v1"` + Name string `protobuf:"bytes,1,opt,name=name,proto3" json:"name,omitempty"` + // Regular expressions matched against command errors. + RetryableErrors []string `protobuf:"bytes,2,rep,name=retryable_errors,json=retryableErrors,proto3" json:"retryable_errors,omitempty"` + MaxAttempts uint32 `protobuf:"varint,3,opt,name=max_attempts,json=maxAttempts,proto3" json:"max_attempts,omitempty"` + SleepIntervalSeconds uint32 `protobuf:"varint,4,opt,name=sleep_interval_seconds,json=sleepIntervalSeconds,proto3" json:"sleep_interval_seconds,omitempty"` + unknownFields protoimpl.UnknownFields + sizeCache protoimpl.SizeCache +} + +func (x *ComponentRetryPolicy) Reset() { + *x = ComponentRetryPolicy{} + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[47] + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + ms.StoreMessageInfo(mi) +} + +func (x *ComponentRetryPolicy) String() string { + return protoimpl.X.MessageStringOf(x) +} + +func (*ComponentRetryPolicy) ProtoMessage() {} + +func (x *ComponentRetryPolicy) ProtoReflect() protoreflect.Message { + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[47] + if x != nil { + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + if ms.LoadMessageInfo() == nil { + ms.StoreMessageInfo(mi) + } + return ms + } + return mi.MessageOf(x) +} + +// Deprecated: Use ComponentRetryPolicy.ProtoReflect.Descriptor instead. +func (*ComponentRetryPolicy) Descriptor() ([]byte, []int) { + return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{47} +} + +func (x *ComponentRetryPolicy) GetName() string { + if x != nil { + return x.Name + } + return "" +} + +func (x *ComponentRetryPolicy) GetRetryableErrors() []string { + if x != nil { + return x.RetryableErrors + } + return nil +} + +func (x *ComponentRetryPolicy) GetMaxAttempts() uint32 { + if x != nil { + return x.MaxAttempts + } + return 0 +} + +func (x *ComponentRetryPolicy) GetSleepIntervalSeconds() uint32 { + if x != nil { + return x.SleepIntervalSeconds + } + return 0 +} + +type ComponentIgnorePolicy struct { + state protoimpl.MessageState `protogen:"open.v1"` + Name string `protobuf:"bytes,1,opt,name=name,proto3" json:"name,omitempty"` + // Regular expressions matched against command errors. + IgnorableErrors []string `protobuf:"bytes,2,rep,name=ignorable_errors,json=ignorableErrors,proto3" json:"ignorable_errors,omitempty"` + Message string `protobuf:"bytes,3,opt,name=message,proto3" json:"message,omitempty"` + // Extra signals written for the caller, keyed by name. + Signals map[string]*ComponentValue `protobuf:"bytes,4,rep,name=signals,proto3" json:"signals,omitempty" protobuf_key:"bytes,1,opt,name=key" protobuf_val:"bytes,2,opt,name=value"` + unknownFields protoimpl.UnknownFields + sizeCache protoimpl.SizeCache +} + +func (x *ComponentIgnorePolicy) Reset() { + *x = ComponentIgnorePolicy{} + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[48] + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + ms.StoreMessageInfo(mi) +} + +func (x *ComponentIgnorePolicy) String() string { + return protoimpl.X.MessageStringOf(x) +} + +func (*ComponentIgnorePolicy) ProtoMessage() {} + +func (x *ComponentIgnorePolicy) ProtoReflect() protoreflect.Message { + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[48] + if x != nil { + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + if ms.LoadMessageInfo() == nil { + ms.StoreMessageInfo(mi) + } + return ms + } + return mi.MessageOf(x) +} + +// Deprecated: Use ComponentIgnorePolicy.ProtoReflect.Descriptor instead. +func (*ComponentIgnorePolicy) Descriptor() ([]byte, []int) { + return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{48} +} + +func (x *ComponentIgnorePolicy) GetName() string { + if x != nil { + return x.Name + } + return "" +} + +func (x *ComponentIgnorePolicy) GetIgnorableErrors() []string { + if x != nil { + return x.IgnorableErrors + } + return nil +} + +func (x *ComponentIgnorePolicy) GetMessage() string { + if x != nil { + return x.Message + } + return "" +} + +func (x *ComponentIgnorePolicy) GetSignals() map[string]*ComponentValue { + if x != nil { + return x.Signals + } + return nil +} + +// ComponentExclude mirrors the Terragrunt exclude block. Reserved; not implemented. +type ComponentExclude struct { + state protoimpl.MessageState `protogen:"open.v1"` + // Evaluated "if" condition. + Condition bool `protobuf:"varint,1,opt,name=condition,proto3" json:"condition,omitempty"` + // Commands the exclusion applies to; "all" and "all_except_output" are plugin vocabulary. + Actions []string `protobuf:"bytes,2,rep,name=actions,proto3" json:"actions,omitempty"` + ExcludeDependencies bool `protobuf:"varint,3,opt,name=exclude_dependencies,json=excludeDependencies,proto3" json:"exclude_dependencies,omitempty"` + unknownFields protoimpl.UnknownFields + sizeCache protoimpl.SizeCache +} + +func (x *ComponentExclude) Reset() { + *x = ComponentExclude{} + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[49] + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + ms.StoreMessageInfo(mi) +} + +func (x *ComponentExclude) String() string { + return protoimpl.X.MessageStringOf(x) +} + +func (*ComponentExclude) ProtoMessage() {} + +func (x *ComponentExclude) ProtoReflect() protoreflect.Message { + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[49] + if x != nil { + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + if ms.LoadMessageInfo() == nil { + ms.StoreMessageInfo(mi) + } + return ms + } + return mi.MessageOf(x) +} + +// Deprecated: Use ComponentExclude.ProtoReflect.Descriptor instead. +func (*ComponentExclude) Descriptor() ([]byte, []int) { + return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{49} +} + +func (x *ComponentExclude) GetCondition() bool { + if x != nil { + return x.Condition + } + return false +} + +func (x *ComponentExclude) GetActions() []string { + if x != nil { + return x.Actions + } + return nil +} + +func (x *ComponentExclude) GetExcludeDependencies() bool { + if x != nil { + return x.ExcludeDependencies + } + return false +} + +// ComponentIam mirrors the Terragrunt IAM settings. Reserved; not implemented. +// The web identity token is a secret and must be carried as a sensitive value. +type ComponentIam struct { + state protoimpl.MessageState `protogen:"open.v1"` + Role string `protobuf:"bytes,1,opt,name=role,proto3" json:"role,omitempty"` + SessionName string `protobuf:"bytes,2,opt,name=session_name,json=sessionName,proto3" json:"session_name,omitempty"` + DurationSeconds uint32 `protobuf:"varint,3,opt,name=duration_seconds,json=durationSeconds,proto3" json:"duration_seconds,omitempty"` + WebIdentityToken *ComponentValue `protobuf:"bytes,4,opt,name=web_identity_token,json=webIdentityToken,proto3" json:"web_identity_token,omitempty"` + unknownFields protoimpl.UnknownFields + sizeCache protoimpl.SizeCache +} + +func (x *ComponentIam) Reset() { + *x = ComponentIam{} + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[50] + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + ms.StoreMessageInfo(mi) +} + +func (x *ComponentIam) String() string { + return protoimpl.X.MessageStringOf(x) +} + +func (*ComponentIam) ProtoMessage() {} + +func (x *ComponentIam) ProtoReflect() protoreflect.Message { + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[50] + if x != nil { + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + if ms.LoadMessageInfo() == nil { + ms.StoreMessageInfo(mi) + } + return ms + } + return mi.MessageOf(x) +} + +// Deprecated: Use ComponentIam.ProtoReflect.Descriptor instead. +func (*ComponentIam) Descriptor() ([]byte, []int) { + return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{50} +} + +func (x *ComponentIam) GetRole() string { + if x != nil { + return x.Role + } + return "" +} + +func (x *ComponentIam) GetSessionName() string { + if x != nil { + return x.SessionName + } + return "" +} + +func (x *ComponentIam) GetDurationSeconds() uint32 { + if x != nil { + return x.DurationSeconds + } + return 0 +} + +func (x *ComponentIam) GetWebIdentityToken() *ComponentValue { + if x != nil { + return x.WebIdentityToken + } + return nil +} + type ResolveComponentStateResponse struct { state protoimpl.MessageState `protogen:"open.v1"` State *ComponentStateIdentity `protobuf:"bytes,1,opt,name=state,proto3" json:"state,omitempty"` @@ -3214,7 +3891,7 @@ type ResolveComponentStateResponse struct { func (x *ResolveComponentStateResponse) Reset() { *x = ResolveComponentStateResponse{} - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[44] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[51] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -3226,7 +3903,7 @@ func (x *ResolveComponentStateResponse) String() string { func (*ResolveComponentStateResponse) ProtoMessage() {} func (x *ResolveComponentStateResponse) ProtoReflect() protoreflect.Message { - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[44] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[51] if x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -3239,7 +3916,7 @@ func (x *ResolveComponentStateResponse) ProtoReflect() protoreflect.Message { // Deprecated: Use ResolveComponentStateResponse.ProtoReflect.Descriptor instead. func (*ResolveComponentStateResponse) Descriptor() ([]byte, []int) { - return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{44} + return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{51} } func (x *ResolveComponentStateResponse) GetState() *ComponentStateIdentity { @@ -3346,11 +4023,30 @@ const file_atmos_plugin_v1_plugin_proto_rawDesc = "" + "\bprogress\x18\x01 \x01(\v2\x19.atmos.plugin.v1.ProgressH\x00R\bprogress\x12@\n" + "\vinteraction\x18\x02 \x01(\v2\x1c.atmos.plugin.v1.InteractionH\x00R\vinteraction\x126\n" + "\x06result\x18\x03 \x01(\v2\x1c.atmos.plugin.v1.CredentialsH\x00R\x06resultB\a\n" + - "\x05event\"<\n" + + "\x05event\"P\n" + + "\x0eSourceLocation\x12\x12\n" + + "\x04file\x18\x01 \x01(\tR\x04file\x12\x12\n" + + "\x04line\x18\x02 \x01(\rR\x04line\x12\x16\n" + + "\x06column\x18\x03 \x01(\rR\x06column\"\xe1\x02\n" + + "\vErrorDetail\x12\x12\n" + + "\x04code\x18\x01 \x01(\tR\x04code\x12\x18\n" + + "\asummary\x18\x02 \x01(\tR\asummary\x12\x16\n" + + "\x06detail\x18\x03 \x01(\tR\x06detail\x12\x14\n" + + "\x05hints\x18\x04 \x03(\tR\x05hints\x12;\n" + + "\blocation\x18\x05 \x01(\v2\x1f.atmos.plugin.v1.SourceLocationR\blocation\x12\x12\n" + + "\x04unit\x18\x06 \x01(\tR\x04unit\x12\x14\n" + + "\x05stack\x18\a \x01(\tR\x05stack\x12\x1d\n" + + "\n" + + "config_key\x18\b \x01(\tR\tconfigKey\x12\x1b\n" + + "\tuser_safe\x18\t \x01(\bR\buserSafe\x12\x1c\n" + + "\toperation\x18\n" + + " \x01(\tR\toperation\x125\n" + + "\bseverity\x18\v \x01(\x0e2\x19.atmos.plugin.v1.SeverityR\bseverity\"r\n" + "\n" + "Diagnostic\x12\x14\n" + "\x05field\x18\x01 \x01(\tR\x05field\x12\x18\n" + - "\amessage\x18\x02 \x01(\tR\amessage\"Q\n" + + "\amessage\x18\x02 \x01(\tR\amessage\x124\n" + + "\x06detail\x18\x03 \x01(\v2\x1c.atmos.plugin.v1.ErrorDetailR\x06detail\"Q\n" + "\x10ValidateResponse\x12=\n" + "\vdiagnostics\x18\x01 \x03(\v2\x1b.atmos.plugin.v1.DiagnosticR\vdiagnostics\"C\n" + "\vUnsupported\x12\x1c\n" + @@ -3494,13 +4190,14 @@ const file_atmos_plugin_v1_plugin_proto_rawDesc = "" + "\fbackend_type\x18\x01 \x01(\tR\vbackendType\x129\n" + "\abackend\x18\x02 \x01(\v2\x1f.atmos.plugin.v1.ComponentValueR\abackend\x12\x1c\n" + "\tworkspace\x18\x03 \x01(\tR\tworkspace\x12+\n" + - "\x11working_directory\x18\x04 \x01(\tR\x10workingDirectory\"\x95\x01\n" + + "\x11working_directory\x18\x04 \x01(\tR\x10workingDirectory\"\xbb\x01\n" + "\x16ComponentGeneratedFile\x12\x12\n" + "\x04path\x18\x01 \x01(\tR\x04path\x12\x18\n" + "\acontent\x18\x02 \x01(\fR\acontent\x12\x1b\n" + "\tif_exists\x18\x03 \x01(\tR\bifExists\x12\x1c\n" + "\tsensitive\x18\x04 \x01(\bR\tsensitive\x12\x12\n" + - "\x04mode\x18\x05 \x01(\rR\x04mode\"\x81\x03\n" + + "\x04mode\x18\x05 \x01(\rR\x04mode\x12$\n" + + "\x0eunit_file_copy\x18\x06 \x01(\bR\funitFileCopy\"\x81\x03\n" + "\x1fComponentExecutionConfiguration\x12Q\n" + "\x0fextra_arguments\x18\x01 \x03(\v2(.atmos.plugin.v1.ComponentExtraArgumentsR\x0eextraArguments\x12A\n" + "\fbefore_hooks\x18\x02 \x03(\v2\x1e.atmos.plugin.v1.ComponentHookR\vbeforeHooks\x12?\n" + @@ -3526,7 +4223,8 @@ const file_atmos_plugin_v1_plugin_proto_rawDesc = "" + "\frun_on_error\x18\x05 \x01(\bR\n" + "runOnError\x12'\n" + "\x0fsuppress_stdout\x18\x06 \x01(\bR\x0esuppressStdout\x12\x1b\n" + - "\ton_errors\x18\a \x03(\tR\bonErrors\"\xfd\x04\n" + + "\ton_errors\x18\a \x03(\tR\bonErrors\"\x85\n" + + "\n" + "\x18PrepareComponentResponse\x12\x16\n" + "\x06engine\x18\x01 \x01(\tR\x06engine\x12+\n" + "\x11working_directory\x18\x02 \x01(\tR\x10workingDirectory\x12#\n" + @@ -3539,11 +4237,55 @@ const file_atmos_plugin_v1_plugin_proto_rawDesc = "" + "\vdiagnostics\x18\t \x03(\v2\x1b.atmos.plugin.v1.DiagnosticR\vdiagnostics\x12+\n" + "\x11backend_generated\x18\n" + " \x01(\bR\x10backendGenerated\x12N\n" + - "\texecution\x18\v \x01(\v20.atmos.plugin.v1.ComponentExecutionConfigurationR\texecution\"\xd9\x01\n" + + "\texecution\x18\v \x01(\v20.atmos.plugin.v1.ComponentExecutionConfigurationR\texecution\x12`\n" + + "\rinput_origins\x18\f \x03(\v2;.atmos.plugin.v1.PrepareComponentResponse.InputOriginsEntryR\finputOrigins\x12)\n" + + "\x10terraform_binary\x18\r \x01(\tR\x0fterraformBinary\x12E\n" + + "\x06errors\x18\x0e \x01(\v2-.atmos.plugin.v1.ComponentErrorsConfigurationR\x06errors\x12;\n" + + "\aexclude\x18\x0f \x01(\v2!.atmos.plugin.v1.ComponentExcludeR\aexclude\x12/\n" + + "\x03iam\x18\x10 \x01(\v2\x1d.atmos.plugin.v1.ComponentIamR\x03iam\x12?\n" + + "\n" + + "encryption\x18\x11 \x01(\v2\x1f.atmos.plugin.v1.ComponentValueR\n" + + "encryption\x12`\n" + + "\rfeature_flags\x18\x12 \x03(\v2;.atmos.plugin.v1.PrepareComponentResponse.FeatureFlagsEntryR\ffeatureFlags\x1a`\n" + + "\x11InputOriginsEntry\x12\x10\n" + + "\x03key\x18\x01 \x01(\tR\x03key\x125\n" + + "\x05value\x18\x02 \x01(\v2\x1f.atmos.plugin.v1.SourceLocationR\x05value:\x028\x01\x1a?\n" + + "\x11FeatureFlagsEntry\x12\x10\n" + + "\x03key\x18\x01 \x01(\tR\x03key\x12\x14\n" + + "\x05value\x18\x02 \x01(\tR\x05value:\x028\x01\"\x9b\x01\n" + + "\x1cComponentErrorsConfiguration\x12;\n" + + "\x05retry\x18\x01 \x03(\v2%.atmos.plugin.v1.ComponentRetryPolicyR\x05retry\x12>\n" + + "\x06ignore\x18\x02 \x03(\v2&.atmos.plugin.v1.ComponentIgnorePolicyR\x06ignore\"\xae\x01\n" + + "\x14ComponentRetryPolicy\x12\x12\n" + + "\x04name\x18\x01 \x01(\tR\x04name\x12)\n" + + "\x10retryable_errors\x18\x02 \x03(\tR\x0fretryableErrors\x12!\n" + + "\fmax_attempts\x18\x03 \x01(\rR\vmaxAttempts\x124\n" + + "\x16sleep_interval_seconds\x18\x04 \x01(\rR\x14sleepIntervalSeconds\"\x9c\x02\n" + + "\x15ComponentIgnorePolicy\x12\x12\n" + + "\x04name\x18\x01 \x01(\tR\x04name\x12)\n" + + "\x10ignorable_errors\x18\x02 \x03(\tR\x0fignorableErrors\x12\x18\n" + + "\amessage\x18\x03 \x01(\tR\amessage\x12M\n" + + "\asignals\x18\x04 \x03(\v23.atmos.plugin.v1.ComponentIgnorePolicy.SignalsEntryR\asignals\x1a[\n" + + "\fSignalsEntry\x12\x10\n" + + "\x03key\x18\x01 \x01(\tR\x03key\x125\n" + + "\x05value\x18\x02 \x01(\v2\x1f.atmos.plugin.v1.ComponentValueR\x05value:\x028\x01\"}\n" + + "\x10ComponentExclude\x12\x1c\n" + + "\tcondition\x18\x01 \x01(\bR\tcondition\x12\x18\n" + + "\aactions\x18\x02 \x03(\tR\aactions\x121\n" + + "\x14exclude_dependencies\x18\x03 \x01(\bR\x13excludeDependencies\"\xbf\x01\n" + + "\fComponentIam\x12\x12\n" + + "\x04role\x18\x01 \x01(\tR\x04role\x12!\n" + + "\fsession_name\x18\x02 \x01(\tR\vsessionName\x12)\n" + + "\x10duration_seconds\x18\x03 \x01(\rR\x0fdurationSeconds\x12M\n" + + "\x12web_identity_token\x18\x04 \x01(\v2\x1f.atmos.plugin.v1.ComponentValueR\x10webIdentityToken\"\xd9\x01\n" + "\x1dResolveComponentStateResponse\x12=\n" + "\x05state\x18\x01 \x01(\v2'.atmos.plugin.v1.ComponentStateIdentityR\x05state\x12:\n" + "\asources\x18\x02 \x03(\v2 .atmos.plugin.v1.ComponentSourceR\asources\x12=\n" + - "\vdiagnostics\x18\x03 \x03(\v2\x1b.atmos.plugin.v1.DiagnosticR\vdiagnostics2\x9f\x01\n" + + "\vdiagnostics\x18\x03 \x03(\v2\x1b.atmos.plugin.v1.DiagnosticR\vdiagnostics*N\n" + + "\bSeverity\x12\x18\n" + + "\x14SEVERITY_UNSPECIFIED\x10\x00\x12\x12\n" + + "\x0eSEVERITY_ERROR\x10\x01\x12\x14\n" + + "\x10SEVERITY_WARNING\x10\x022\x9f\x01\n" + "\x04Info\x12O\n" + "\bDescribe\x12 .atmos.plugin.v1.DescribeRequest\x1a!.atmos.plugin.v1.DescribeResponse\x12F\n" + "\tConfigure\x12!.atmos.plugin.v1.ConfigureRequest\x1a\x16.google.protobuf.Empty2\xd3\x04\n" + @@ -3586,191 +4328,217 @@ func file_atmos_plugin_v1_plugin_proto_rawDescGZIP() []byte { } var ( - file_atmos_plugin_v1_plugin_proto_enumTypes = make([]protoimpl.EnumInfo, 2) - file_atmos_plugin_v1_plugin_proto_msgTypes = make([]protoimpl.MessageInfo, 53) + file_atmos_plugin_v1_plugin_proto_enumTypes = make([]protoimpl.EnumInfo, 3) + file_atmos_plugin_v1_plugin_proto_msgTypes = make([]protoimpl.MessageInfo, 63) file_atmos_plugin_v1_plugin_proto_goTypes = []any{ - ComponentSource_Kind(0), // 0: atmos.plugin.v1.ComponentSource.Kind - ComponentDependency_Kind(0), // 1: atmos.plugin.v1.ComponentDependency.Kind - (*Invocation)(nil), // 2: atmos.plugin.v1.Invocation - (*Credentials)(nil), // 3: atmos.plugin.v1.Credentials - (*ServiceVersionRange)(nil), // 4: atmos.plugin.v1.ServiceVersionRange - (*Export)(nil), // 5: atmos.plugin.v1.Export - (*DescribeRequest)(nil), // 6: atmos.plugin.v1.DescribeRequest - (*DescribeResponse)(nil), // 7: atmos.plugin.v1.DescribeResponse - (*ConfigureRequest)(nil), // 8: atmos.plugin.v1.ConfigureRequest - (*ProviderRequest)(nil), // 9: atmos.plugin.v1.ProviderRequest - (*IdentityRequest)(nil), // 10: atmos.plugin.v1.IdentityRequest - (*Progress)(nil), // 11: atmos.plugin.v1.Progress - (*Interaction)(nil), // 12: atmos.plugin.v1.Interaction - (*AuthenticateEvent)(nil), // 13: atmos.plugin.v1.AuthenticateEvent - (*Diagnostic)(nil), // 14: atmos.plugin.v1.Diagnostic - (*ValidateResponse)(nil), // 15: atmos.plugin.v1.ValidateResponse - (*Unsupported)(nil), // 16: atmos.plugin.v1.Unsupported - (*RefreshRequest)(nil), // 17: atmos.plugin.v1.RefreshRequest - (*RefreshResponse)(nil), // 18: atmos.plugin.v1.RefreshResponse - (*EnvironmentResponse)(nil), // 19: atmos.plugin.v1.EnvironmentResponse - (*PrepareEnvironmentRequest)(nil), // 20: atmos.plugin.v1.PrepareEnvironmentRequest - (*CredentialPath)(nil), // 21: atmos.plugin.v1.CredentialPath - (*PathsResponse)(nil), // 22: atmos.plugin.v1.PathsResponse - (*IdentityAuthenticateRequest)(nil), // 23: atmos.plugin.v1.IdentityAuthenticateRequest - (*IdentityPrepareRequest)(nil), // 24: atmos.plugin.v1.IdentityPrepareRequest - (*CommandRequest)(nil), // 25: atmos.plugin.v1.CommandRequest - (*CommandExit)(nil), // 26: atmos.plugin.v1.CommandExit - (*CommandEvent)(nil), // 27: atmos.plugin.v1.CommandEvent - (*ComponentValue)(nil), // 28: atmos.plugin.v1.ComponentValue - (*ComponentValuePath)(nil), // 29: atmos.plugin.v1.ComponentValuePath - (*ComponentValueStep)(nil), // 30: atmos.plugin.v1.ComponentValueStep - (*ComponentAddress)(nil), // 31: atmos.plugin.v1.ComponentAddress - (*ComponentSource)(nil), // 32: atmos.plugin.v1.ComponentSource - (*ComponentDependency)(nil), // 33: atmos.plugin.v1.ComponentDependency - (*DiscoveredComponent)(nil), // 34: atmos.plugin.v1.DiscoveredComponent - (*ComponentEffects)(nil), // 35: atmos.plugin.v1.ComponentEffects - (*ComponentOperation)(nil), // 36: atmos.plugin.v1.ComponentOperation - (*DiscoverComponentsRequest)(nil), // 37: atmos.plugin.v1.DiscoverComponentsRequest - (*DiscoverComponentsResponse)(nil), // 38: atmos.plugin.v1.DiscoverComponentsResponse - (*ComponentRequest)(nil), // 39: atmos.plugin.v1.ComponentRequest - (*ComponentStateIdentity)(nil), // 40: atmos.plugin.v1.ComponentStateIdentity - (*ComponentGeneratedFile)(nil), // 41: atmos.plugin.v1.ComponentGeneratedFile - (*ComponentExecutionConfiguration)(nil), // 42: atmos.plugin.v1.ComponentExecutionConfiguration - (*ComponentExtraArguments)(nil), // 43: atmos.plugin.v1.ComponentExtraArguments - (*ComponentHook)(nil), // 44: atmos.plugin.v1.ComponentHook - (*PrepareComponentResponse)(nil), // 45: atmos.plugin.v1.PrepareComponentResponse - (*ResolveComponentStateResponse)(nil), // 46: atmos.plugin.v1.ResolveComponentStateResponse - nil, // 47: atmos.plugin.v1.Invocation.EnvironmentEntry - nil, // 48: atmos.plugin.v1.ConfigureRequest.BrokerIdsEntry - nil, // 49: atmos.plugin.v1.Interaction.FieldsEntry - nil, // 50: atmos.plugin.v1.EnvironmentResponse.EnvironmentEntry - nil, // 51: atmos.plugin.v1.PrepareEnvironmentRequest.EnvironmentEntry - nil, // 52: atmos.plugin.v1.IdentityPrepareRequest.EnvironmentEntry - nil, // 53: atmos.plugin.v1.ComponentOperation.OptionsEntry - nil, // 54: atmos.plugin.v1.ComponentExtraArguments.EnvironmentEntry - (*timestamppb.Timestamp)(nil), // 55: google.protobuf.Timestamp - (*structpb.Struct)(nil), // 56: google.protobuf.Struct - (*emptypb.Empty)(nil), // 57: google.protobuf.Empty + Severity(0), // 0: atmos.plugin.v1.Severity + ComponentSource_Kind(0), // 1: atmos.plugin.v1.ComponentSource.Kind + ComponentDependency_Kind(0), // 2: atmos.plugin.v1.ComponentDependency.Kind + (*Invocation)(nil), // 3: atmos.plugin.v1.Invocation + (*Credentials)(nil), // 4: atmos.plugin.v1.Credentials + (*ServiceVersionRange)(nil), // 5: atmos.plugin.v1.ServiceVersionRange + (*Export)(nil), // 6: atmos.plugin.v1.Export + (*DescribeRequest)(nil), // 7: atmos.plugin.v1.DescribeRequest + (*DescribeResponse)(nil), // 8: atmos.plugin.v1.DescribeResponse + (*ConfigureRequest)(nil), // 9: atmos.plugin.v1.ConfigureRequest + (*ProviderRequest)(nil), // 10: atmos.plugin.v1.ProviderRequest + (*IdentityRequest)(nil), // 11: atmos.plugin.v1.IdentityRequest + (*Progress)(nil), // 12: atmos.plugin.v1.Progress + (*Interaction)(nil), // 13: atmos.plugin.v1.Interaction + (*AuthenticateEvent)(nil), // 14: atmos.plugin.v1.AuthenticateEvent + (*SourceLocation)(nil), // 15: atmos.plugin.v1.SourceLocation + (*ErrorDetail)(nil), // 16: atmos.plugin.v1.ErrorDetail + (*Diagnostic)(nil), // 17: atmos.plugin.v1.Diagnostic + (*ValidateResponse)(nil), // 18: atmos.plugin.v1.ValidateResponse + (*Unsupported)(nil), // 19: atmos.plugin.v1.Unsupported + (*RefreshRequest)(nil), // 20: atmos.plugin.v1.RefreshRequest + (*RefreshResponse)(nil), // 21: atmos.plugin.v1.RefreshResponse + (*EnvironmentResponse)(nil), // 22: atmos.plugin.v1.EnvironmentResponse + (*PrepareEnvironmentRequest)(nil), // 23: atmos.plugin.v1.PrepareEnvironmentRequest + (*CredentialPath)(nil), // 24: atmos.plugin.v1.CredentialPath + (*PathsResponse)(nil), // 25: atmos.plugin.v1.PathsResponse + (*IdentityAuthenticateRequest)(nil), // 26: atmos.plugin.v1.IdentityAuthenticateRequest + (*IdentityPrepareRequest)(nil), // 27: atmos.plugin.v1.IdentityPrepareRequest + (*CommandRequest)(nil), // 28: atmos.plugin.v1.CommandRequest + (*CommandExit)(nil), // 29: atmos.plugin.v1.CommandExit + (*CommandEvent)(nil), // 30: atmos.plugin.v1.CommandEvent + (*ComponentValue)(nil), // 31: atmos.plugin.v1.ComponentValue + (*ComponentValuePath)(nil), // 32: atmos.plugin.v1.ComponentValuePath + (*ComponentValueStep)(nil), // 33: atmos.plugin.v1.ComponentValueStep + (*ComponentAddress)(nil), // 34: atmos.plugin.v1.ComponentAddress + (*ComponentSource)(nil), // 35: atmos.plugin.v1.ComponentSource + (*ComponentDependency)(nil), // 36: atmos.plugin.v1.ComponentDependency + (*DiscoveredComponent)(nil), // 37: atmos.plugin.v1.DiscoveredComponent + (*ComponentEffects)(nil), // 38: atmos.plugin.v1.ComponentEffects + (*ComponentOperation)(nil), // 39: atmos.plugin.v1.ComponentOperation + (*DiscoverComponentsRequest)(nil), // 40: atmos.plugin.v1.DiscoverComponentsRequest + (*DiscoverComponentsResponse)(nil), // 41: atmos.plugin.v1.DiscoverComponentsResponse + (*ComponentRequest)(nil), // 42: atmos.plugin.v1.ComponentRequest + (*ComponentStateIdentity)(nil), // 43: atmos.plugin.v1.ComponentStateIdentity + (*ComponentGeneratedFile)(nil), // 44: atmos.plugin.v1.ComponentGeneratedFile + (*ComponentExecutionConfiguration)(nil), // 45: atmos.plugin.v1.ComponentExecutionConfiguration + (*ComponentExtraArguments)(nil), // 46: atmos.plugin.v1.ComponentExtraArguments + (*ComponentHook)(nil), // 47: atmos.plugin.v1.ComponentHook + (*PrepareComponentResponse)(nil), // 48: atmos.plugin.v1.PrepareComponentResponse + (*ComponentErrorsConfiguration)(nil), // 49: atmos.plugin.v1.ComponentErrorsConfiguration + (*ComponentRetryPolicy)(nil), // 50: atmos.plugin.v1.ComponentRetryPolicy + (*ComponentIgnorePolicy)(nil), // 51: atmos.plugin.v1.ComponentIgnorePolicy + (*ComponentExclude)(nil), // 52: atmos.plugin.v1.ComponentExclude + (*ComponentIam)(nil), // 53: atmos.plugin.v1.ComponentIam + (*ResolveComponentStateResponse)(nil), // 54: atmos.plugin.v1.ResolveComponentStateResponse + nil, // 55: atmos.plugin.v1.Invocation.EnvironmentEntry + nil, // 56: atmos.plugin.v1.ConfigureRequest.BrokerIdsEntry + nil, // 57: atmos.plugin.v1.Interaction.FieldsEntry + nil, // 58: atmos.plugin.v1.EnvironmentResponse.EnvironmentEntry + nil, // 59: atmos.plugin.v1.PrepareEnvironmentRequest.EnvironmentEntry + nil, // 60: atmos.plugin.v1.IdentityPrepareRequest.EnvironmentEntry + nil, // 61: atmos.plugin.v1.ComponentOperation.OptionsEntry + nil, // 62: atmos.plugin.v1.ComponentExtraArguments.EnvironmentEntry + nil, // 63: atmos.plugin.v1.PrepareComponentResponse.InputOriginsEntry + nil, // 64: atmos.plugin.v1.PrepareComponentResponse.FeatureFlagsEntry + nil, // 65: atmos.plugin.v1.ComponentIgnorePolicy.SignalsEntry + (*timestamppb.Timestamp)(nil), // 66: google.protobuf.Timestamp + (*structpb.Struct)(nil), // 67: google.protobuf.Struct + (*emptypb.Empty)(nil), // 68: google.protobuf.Empty } ) var file_atmos_plugin_v1_plugin_proto_depIdxs = []int32{ - 55, // 0: atmos.plugin.v1.Invocation.deadline:type_name -> google.protobuf.Timestamp - 47, // 1: atmos.plugin.v1.Invocation.environment:type_name -> atmos.plugin.v1.Invocation.EnvironmentEntry - 55, // 2: atmos.plugin.v1.Credentials.expires_at:type_name -> google.protobuf.Timestamp - 56, // 3: atmos.plugin.v1.Credentials.data:type_name -> google.protobuf.Struct - 4, // 4: atmos.plugin.v1.DescribeRequest.host_supported:type_name -> atmos.plugin.v1.ServiceVersionRange - 4, // 5: atmos.plugin.v1.DescribeResponse.services:type_name -> atmos.plugin.v1.ServiceVersionRange - 5, // 6: atmos.plugin.v1.DescribeResponse.exports:type_name -> atmos.plugin.v1.Export - 48, // 7: atmos.plugin.v1.ConfigureRequest.broker_ids:type_name -> atmos.plugin.v1.ConfigureRequest.BrokerIdsEntry - 56, // 8: atmos.plugin.v1.ConfigureRequest.settings:type_name -> google.protobuf.Struct - 2, // 9: atmos.plugin.v1.ProviderRequest.invocation:type_name -> atmos.plugin.v1.Invocation - 56, // 10: atmos.plugin.v1.ProviderRequest.spec:type_name -> google.protobuf.Struct - 2, // 11: atmos.plugin.v1.IdentityRequest.invocation:type_name -> atmos.plugin.v1.Invocation - 56, // 12: atmos.plugin.v1.IdentityRequest.spec:type_name -> google.protobuf.Struct - 56, // 13: atmos.plugin.v1.IdentityRequest.principal:type_name -> google.protobuf.Struct - 49, // 14: atmos.plugin.v1.Interaction.fields:type_name -> atmos.plugin.v1.Interaction.FieldsEntry - 11, // 15: atmos.plugin.v1.AuthenticateEvent.progress:type_name -> atmos.plugin.v1.Progress - 12, // 16: atmos.plugin.v1.AuthenticateEvent.interaction:type_name -> atmos.plugin.v1.Interaction - 3, // 17: atmos.plugin.v1.AuthenticateEvent.result:type_name -> atmos.plugin.v1.Credentials - 14, // 18: atmos.plugin.v1.ValidateResponse.diagnostics:type_name -> atmos.plugin.v1.Diagnostic - 9, // 19: atmos.plugin.v1.RefreshRequest.provider:type_name -> atmos.plugin.v1.ProviderRequest - 3, // 20: atmos.plugin.v1.RefreshRequest.current:type_name -> atmos.plugin.v1.Credentials - 3, // 21: atmos.plugin.v1.RefreshResponse.credentials:type_name -> atmos.plugin.v1.Credentials - 16, // 22: atmos.plugin.v1.RefreshResponse.unsupported:type_name -> atmos.plugin.v1.Unsupported - 50, // 23: atmos.plugin.v1.EnvironmentResponse.environment:type_name -> atmos.plugin.v1.EnvironmentResponse.EnvironmentEntry - 9, // 24: atmos.plugin.v1.PrepareEnvironmentRequest.provider:type_name -> atmos.plugin.v1.ProviderRequest - 51, // 25: atmos.plugin.v1.PrepareEnvironmentRequest.environment:type_name -> atmos.plugin.v1.PrepareEnvironmentRequest.EnvironmentEntry - 21, // 26: atmos.plugin.v1.PathsResponse.paths:type_name -> atmos.plugin.v1.CredentialPath - 10, // 27: atmos.plugin.v1.IdentityAuthenticateRequest.identity:type_name -> atmos.plugin.v1.IdentityRequest - 3, // 28: atmos.plugin.v1.IdentityAuthenticateRequest.upstream:type_name -> atmos.plugin.v1.Credentials - 10, // 29: atmos.plugin.v1.IdentityPrepareRequest.identity:type_name -> atmos.plugin.v1.IdentityRequest - 52, // 30: atmos.plugin.v1.IdentityPrepareRequest.environment:type_name -> atmos.plugin.v1.IdentityPrepareRequest.EnvironmentEntry - 2, // 31: atmos.plugin.v1.CommandRequest.invocation:type_name -> atmos.plugin.v1.Invocation - 56, // 32: atmos.plugin.v1.CommandRequest.settings:type_name -> google.protobuf.Struct - 26, // 33: atmos.plugin.v1.CommandEvent.exit:type_name -> atmos.plugin.v1.CommandExit - 29, // 34: atmos.plugin.v1.ComponentValue.sensitive_paths:type_name -> atmos.plugin.v1.ComponentValuePath - 30, // 35: atmos.plugin.v1.ComponentValuePath.steps:type_name -> atmos.plugin.v1.ComponentValueStep - 28, // 36: atmos.plugin.v1.ComponentValueStep.index:type_name -> atmos.plugin.v1.ComponentValue - 0, // 37: atmos.plugin.v1.ComponentSource.kind:type_name -> atmos.plugin.v1.ComponentSource.Kind - 31, // 38: atmos.plugin.v1.ComponentDependency.target:type_name -> atmos.plugin.v1.ComponentAddress - 1, // 39: atmos.plugin.v1.ComponentDependency.kind:type_name -> atmos.plugin.v1.ComponentDependency.Kind - 31, // 40: atmos.plugin.v1.DiscoveredComponent.address:type_name -> atmos.plugin.v1.ComponentAddress - 28, // 41: atmos.plugin.v1.DiscoveredComponent.configuration:type_name -> atmos.plugin.v1.ComponentValue - 32, // 42: atmos.plugin.v1.DiscoveredComponent.sources:type_name -> atmos.plugin.v1.ComponentSource - 33, // 43: atmos.plugin.v1.DiscoveredComponent.dependencies:type_name -> atmos.plugin.v1.ComponentDependency - 53, // 44: atmos.plugin.v1.ComponentOperation.options:type_name -> atmos.plugin.v1.ComponentOperation.OptionsEntry - 2, // 45: atmos.plugin.v1.DiscoverComponentsRequest.invocation:type_name -> atmos.plugin.v1.Invocation - 36, // 46: atmos.plugin.v1.DiscoverComponentsRequest.operation:type_name -> atmos.plugin.v1.ComponentOperation - 35, // 47: atmos.plugin.v1.DiscoverComponentsRequest.effects:type_name -> atmos.plugin.v1.ComponentEffects - 28, // 48: atmos.plugin.v1.DiscoverComponentsRequest.provider_configuration:type_name -> atmos.plugin.v1.ComponentValue - 34, // 49: atmos.plugin.v1.DiscoverComponentsResponse.components:type_name -> atmos.plugin.v1.DiscoveredComponent - 14, // 50: atmos.plugin.v1.DiscoverComponentsResponse.diagnostics:type_name -> atmos.plugin.v1.Diagnostic - 2, // 51: atmos.plugin.v1.ComponentRequest.invocation:type_name -> atmos.plugin.v1.Invocation - 31, // 52: atmos.plugin.v1.ComponentRequest.address:type_name -> atmos.plugin.v1.ComponentAddress - 36, // 53: atmos.plugin.v1.ComponentRequest.operation:type_name -> atmos.plugin.v1.ComponentOperation - 35, // 54: atmos.plugin.v1.ComponentRequest.effects:type_name -> atmos.plugin.v1.ComponentEffects - 28, // 55: atmos.plugin.v1.ComponentRequest.configuration:type_name -> atmos.plugin.v1.ComponentValue - 28, // 56: atmos.plugin.v1.ComponentRequest.provider_configuration:type_name -> atmos.plugin.v1.ComponentValue - 28, // 57: atmos.plugin.v1.ComponentStateIdentity.backend:type_name -> atmos.plugin.v1.ComponentValue - 43, // 58: atmos.plugin.v1.ComponentExecutionConfiguration.extra_arguments:type_name -> atmos.plugin.v1.ComponentExtraArguments - 44, // 59: atmos.plugin.v1.ComponentExecutionConfiguration.before_hooks:type_name -> atmos.plugin.v1.ComponentHook - 44, // 60: atmos.plugin.v1.ComponentExecutionConfiguration.after_hooks:type_name -> atmos.plugin.v1.ComponentHook - 44, // 61: atmos.plugin.v1.ComponentExecutionConfiguration.error_hooks:type_name -> atmos.plugin.v1.ComponentHook - 42, // 62: atmos.plugin.v1.ComponentExecutionConfiguration.apply:type_name -> atmos.plugin.v1.ComponentExecutionConfiguration - 54, // 63: atmos.plugin.v1.ComponentExtraArguments.environment:type_name -> atmos.plugin.v1.ComponentExtraArguments.EnvironmentEntry - 28, // 64: atmos.plugin.v1.PrepareComponentResponse.inputs:type_name -> atmos.plugin.v1.ComponentValue - 40, // 65: atmos.plugin.v1.PrepareComponentResponse.state:type_name -> atmos.plugin.v1.ComponentStateIdentity - 41, // 66: atmos.plugin.v1.PrepareComponentResponse.files:type_name -> atmos.plugin.v1.ComponentGeneratedFile - 33, // 67: atmos.plugin.v1.PrepareComponentResponse.dependencies:type_name -> atmos.plugin.v1.ComponentDependency - 32, // 68: atmos.plugin.v1.PrepareComponentResponse.sources:type_name -> atmos.plugin.v1.ComponentSource - 14, // 69: atmos.plugin.v1.PrepareComponentResponse.diagnostics:type_name -> atmos.plugin.v1.Diagnostic - 42, // 70: atmos.plugin.v1.PrepareComponentResponse.execution:type_name -> atmos.plugin.v1.ComponentExecutionConfiguration - 40, // 71: atmos.plugin.v1.ResolveComponentStateResponse.state:type_name -> atmos.plugin.v1.ComponentStateIdentity - 32, // 72: atmos.plugin.v1.ResolveComponentStateResponse.sources:type_name -> atmos.plugin.v1.ComponentSource - 14, // 73: atmos.plugin.v1.ResolveComponentStateResponse.diagnostics:type_name -> atmos.plugin.v1.Diagnostic - 28, // 74: atmos.plugin.v1.ComponentOperation.OptionsEntry.value:type_name -> atmos.plugin.v1.ComponentValue - 6, // 75: atmos.plugin.v1.Info.Describe:input_type -> atmos.plugin.v1.DescribeRequest - 8, // 76: atmos.plugin.v1.Info.Configure:input_type -> atmos.plugin.v1.ConfigureRequest - 9, // 77: atmos.plugin.v1.AuthProvider.Validate:input_type -> atmos.plugin.v1.ProviderRequest - 9, // 78: atmos.plugin.v1.AuthProvider.Authenticate:input_type -> atmos.plugin.v1.ProviderRequest - 17, // 79: atmos.plugin.v1.AuthProvider.Refresh:input_type -> atmos.plugin.v1.RefreshRequest - 9, // 80: atmos.plugin.v1.AuthProvider.Logout:input_type -> atmos.plugin.v1.ProviderRequest - 9, // 81: atmos.plugin.v1.AuthProvider.Environment:input_type -> atmos.plugin.v1.ProviderRequest - 9, // 82: atmos.plugin.v1.AuthProvider.Paths:input_type -> atmos.plugin.v1.ProviderRequest - 20, // 83: atmos.plugin.v1.AuthProvider.PrepareEnvironment:input_type -> atmos.plugin.v1.PrepareEnvironmentRequest - 10, // 84: atmos.plugin.v1.AuthIdentity.Validate:input_type -> atmos.plugin.v1.IdentityRequest - 23, // 85: atmos.plugin.v1.AuthIdentity.Authenticate:input_type -> atmos.plugin.v1.IdentityAuthenticateRequest - 10, // 86: atmos.plugin.v1.AuthIdentity.Logout:input_type -> atmos.plugin.v1.IdentityRequest - 10, // 87: atmos.plugin.v1.AuthIdentity.Environment:input_type -> atmos.plugin.v1.IdentityRequest - 10, // 88: atmos.plugin.v1.AuthIdentity.Paths:input_type -> atmos.plugin.v1.IdentityRequest - 24, // 89: atmos.plugin.v1.AuthIdentity.PrepareEnvironment:input_type -> atmos.plugin.v1.IdentityPrepareRequest - 25, // 90: atmos.plugin.v1.Commands.Run:input_type -> atmos.plugin.v1.CommandRequest - 37, // 91: atmos.plugin.v1.ComponentDiscovery.Discover:input_type -> atmos.plugin.v1.DiscoverComponentsRequest - 39, // 92: atmos.plugin.v1.ComponentValidation.Validate:input_type -> atmos.plugin.v1.ComponentRequest - 39, // 93: atmos.plugin.v1.ComponentPreparation.Prepare:input_type -> atmos.plugin.v1.ComponentRequest - 39, // 94: atmos.plugin.v1.ComponentState.Resolve:input_type -> atmos.plugin.v1.ComponentRequest - 7, // 95: atmos.plugin.v1.Info.Describe:output_type -> atmos.plugin.v1.DescribeResponse - 57, // 96: atmos.plugin.v1.Info.Configure:output_type -> google.protobuf.Empty - 15, // 97: atmos.plugin.v1.AuthProvider.Validate:output_type -> atmos.plugin.v1.ValidateResponse - 13, // 98: atmos.plugin.v1.AuthProvider.Authenticate:output_type -> atmos.plugin.v1.AuthenticateEvent - 18, // 99: atmos.plugin.v1.AuthProvider.Refresh:output_type -> atmos.plugin.v1.RefreshResponse - 57, // 100: atmos.plugin.v1.AuthProvider.Logout:output_type -> google.protobuf.Empty - 19, // 101: atmos.plugin.v1.AuthProvider.Environment:output_type -> atmos.plugin.v1.EnvironmentResponse - 22, // 102: atmos.plugin.v1.AuthProvider.Paths:output_type -> atmos.plugin.v1.PathsResponse - 19, // 103: atmos.plugin.v1.AuthProvider.PrepareEnvironment:output_type -> atmos.plugin.v1.EnvironmentResponse - 15, // 104: atmos.plugin.v1.AuthIdentity.Validate:output_type -> atmos.plugin.v1.ValidateResponse - 13, // 105: atmos.plugin.v1.AuthIdentity.Authenticate:output_type -> atmos.plugin.v1.AuthenticateEvent - 57, // 106: atmos.plugin.v1.AuthIdentity.Logout:output_type -> google.protobuf.Empty - 19, // 107: atmos.plugin.v1.AuthIdentity.Environment:output_type -> atmos.plugin.v1.EnvironmentResponse - 22, // 108: atmos.plugin.v1.AuthIdentity.Paths:output_type -> atmos.plugin.v1.PathsResponse - 19, // 109: atmos.plugin.v1.AuthIdentity.PrepareEnvironment:output_type -> atmos.plugin.v1.EnvironmentResponse - 27, // 110: atmos.plugin.v1.Commands.Run:output_type -> atmos.plugin.v1.CommandEvent - 38, // 111: atmos.plugin.v1.ComponentDiscovery.Discover:output_type -> atmos.plugin.v1.DiscoverComponentsResponse - 15, // 112: atmos.plugin.v1.ComponentValidation.Validate:output_type -> atmos.plugin.v1.ValidateResponse - 45, // 113: atmos.plugin.v1.ComponentPreparation.Prepare:output_type -> atmos.plugin.v1.PrepareComponentResponse - 46, // 114: atmos.plugin.v1.ComponentState.Resolve:output_type -> atmos.plugin.v1.ResolveComponentStateResponse - 95, // [95:115] is the sub-list for method output_type - 75, // [75:95] is the sub-list for method input_type - 75, // [75:75] is the sub-list for extension type_name - 75, // [75:75] is the sub-list for extension extendee - 0, // [0:75] is the sub-list for field type_name + 66, // 0: atmos.plugin.v1.Invocation.deadline:type_name -> google.protobuf.Timestamp + 55, // 1: atmos.plugin.v1.Invocation.environment:type_name -> atmos.plugin.v1.Invocation.EnvironmentEntry + 66, // 2: atmos.plugin.v1.Credentials.expires_at:type_name -> google.protobuf.Timestamp + 67, // 3: atmos.plugin.v1.Credentials.data:type_name -> google.protobuf.Struct + 5, // 4: atmos.plugin.v1.DescribeRequest.host_supported:type_name -> atmos.plugin.v1.ServiceVersionRange + 5, // 5: atmos.plugin.v1.DescribeResponse.services:type_name -> atmos.plugin.v1.ServiceVersionRange + 6, // 6: atmos.plugin.v1.DescribeResponse.exports:type_name -> atmos.plugin.v1.Export + 56, // 7: atmos.plugin.v1.ConfigureRequest.broker_ids:type_name -> atmos.plugin.v1.ConfigureRequest.BrokerIdsEntry + 67, // 8: atmos.plugin.v1.ConfigureRequest.settings:type_name -> google.protobuf.Struct + 3, // 9: atmos.plugin.v1.ProviderRequest.invocation:type_name -> atmos.plugin.v1.Invocation + 67, // 10: atmos.plugin.v1.ProviderRequest.spec:type_name -> google.protobuf.Struct + 3, // 11: atmos.plugin.v1.IdentityRequest.invocation:type_name -> atmos.plugin.v1.Invocation + 67, // 12: atmos.plugin.v1.IdentityRequest.spec:type_name -> google.protobuf.Struct + 67, // 13: atmos.plugin.v1.IdentityRequest.principal:type_name -> google.protobuf.Struct + 57, // 14: atmos.plugin.v1.Interaction.fields:type_name -> atmos.plugin.v1.Interaction.FieldsEntry + 12, // 15: atmos.plugin.v1.AuthenticateEvent.progress:type_name -> atmos.plugin.v1.Progress + 13, // 16: atmos.plugin.v1.AuthenticateEvent.interaction:type_name -> atmos.plugin.v1.Interaction + 4, // 17: atmos.plugin.v1.AuthenticateEvent.result:type_name -> atmos.plugin.v1.Credentials + 15, // 18: atmos.plugin.v1.ErrorDetail.location:type_name -> atmos.plugin.v1.SourceLocation + 0, // 19: atmos.plugin.v1.ErrorDetail.severity:type_name -> atmos.plugin.v1.Severity + 16, // 20: atmos.plugin.v1.Diagnostic.detail:type_name -> atmos.plugin.v1.ErrorDetail + 17, // 21: atmos.plugin.v1.ValidateResponse.diagnostics:type_name -> atmos.plugin.v1.Diagnostic + 10, // 22: atmos.plugin.v1.RefreshRequest.provider:type_name -> atmos.plugin.v1.ProviderRequest + 4, // 23: atmos.plugin.v1.RefreshRequest.current:type_name -> atmos.plugin.v1.Credentials + 4, // 24: atmos.plugin.v1.RefreshResponse.credentials:type_name -> atmos.plugin.v1.Credentials + 19, // 25: atmos.plugin.v1.RefreshResponse.unsupported:type_name -> atmos.plugin.v1.Unsupported + 58, // 26: atmos.plugin.v1.EnvironmentResponse.environment:type_name -> atmos.plugin.v1.EnvironmentResponse.EnvironmentEntry + 10, // 27: atmos.plugin.v1.PrepareEnvironmentRequest.provider:type_name -> atmos.plugin.v1.ProviderRequest + 59, // 28: atmos.plugin.v1.PrepareEnvironmentRequest.environment:type_name -> atmos.plugin.v1.PrepareEnvironmentRequest.EnvironmentEntry + 24, // 29: atmos.plugin.v1.PathsResponse.paths:type_name -> atmos.plugin.v1.CredentialPath + 11, // 30: atmos.plugin.v1.IdentityAuthenticateRequest.identity:type_name -> atmos.plugin.v1.IdentityRequest + 4, // 31: atmos.plugin.v1.IdentityAuthenticateRequest.upstream:type_name -> atmos.plugin.v1.Credentials + 11, // 32: atmos.plugin.v1.IdentityPrepareRequest.identity:type_name -> atmos.plugin.v1.IdentityRequest + 60, // 33: atmos.plugin.v1.IdentityPrepareRequest.environment:type_name -> atmos.plugin.v1.IdentityPrepareRequest.EnvironmentEntry + 3, // 34: atmos.plugin.v1.CommandRequest.invocation:type_name -> atmos.plugin.v1.Invocation + 67, // 35: atmos.plugin.v1.CommandRequest.settings:type_name -> google.protobuf.Struct + 29, // 36: atmos.plugin.v1.CommandEvent.exit:type_name -> atmos.plugin.v1.CommandExit + 32, // 37: atmos.plugin.v1.ComponentValue.sensitive_paths:type_name -> atmos.plugin.v1.ComponentValuePath + 33, // 38: atmos.plugin.v1.ComponentValuePath.steps:type_name -> atmos.plugin.v1.ComponentValueStep + 31, // 39: atmos.plugin.v1.ComponentValueStep.index:type_name -> atmos.plugin.v1.ComponentValue + 1, // 40: atmos.plugin.v1.ComponentSource.kind:type_name -> atmos.plugin.v1.ComponentSource.Kind + 34, // 41: atmos.plugin.v1.ComponentDependency.target:type_name -> atmos.plugin.v1.ComponentAddress + 2, // 42: atmos.plugin.v1.ComponentDependency.kind:type_name -> atmos.plugin.v1.ComponentDependency.Kind + 34, // 43: atmos.plugin.v1.DiscoveredComponent.address:type_name -> atmos.plugin.v1.ComponentAddress + 31, // 44: atmos.plugin.v1.DiscoveredComponent.configuration:type_name -> atmos.plugin.v1.ComponentValue + 35, // 45: atmos.plugin.v1.DiscoveredComponent.sources:type_name -> atmos.plugin.v1.ComponentSource + 36, // 46: atmos.plugin.v1.DiscoveredComponent.dependencies:type_name -> atmos.plugin.v1.ComponentDependency + 61, // 47: atmos.plugin.v1.ComponentOperation.options:type_name -> atmos.plugin.v1.ComponentOperation.OptionsEntry + 3, // 48: atmos.plugin.v1.DiscoverComponentsRequest.invocation:type_name -> atmos.plugin.v1.Invocation + 39, // 49: atmos.plugin.v1.DiscoverComponentsRequest.operation:type_name -> atmos.plugin.v1.ComponentOperation + 38, // 50: atmos.plugin.v1.DiscoverComponentsRequest.effects:type_name -> atmos.plugin.v1.ComponentEffects + 31, // 51: atmos.plugin.v1.DiscoverComponentsRequest.provider_configuration:type_name -> atmos.plugin.v1.ComponentValue + 37, // 52: atmos.plugin.v1.DiscoverComponentsResponse.components:type_name -> atmos.plugin.v1.DiscoveredComponent + 17, // 53: atmos.plugin.v1.DiscoverComponentsResponse.diagnostics:type_name -> atmos.plugin.v1.Diagnostic + 3, // 54: atmos.plugin.v1.ComponentRequest.invocation:type_name -> atmos.plugin.v1.Invocation + 34, // 55: atmos.plugin.v1.ComponentRequest.address:type_name -> atmos.plugin.v1.ComponentAddress + 39, // 56: atmos.plugin.v1.ComponentRequest.operation:type_name -> atmos.plugin.v1.ComponentOperation + 38, // 57: atmos.plugin.v1.ComponentRequest.effects:type_name -> atmos.plugin.v1.ComponentEffects + 31, // 58: atmos.plugin.v1.ComponentRequest.configuration:type_name -> atmos.plugin.v1.ComponentValue + 31, // 59: atmos.plugin.v1.ComponentRequest.provider_configuration:type_name -> atmos.plugin.v1.ComponentValue + 31, // 60: atmos.plugin.v1.ComponentStateIdentity.backend:type_name -> atmos.plugin.v1.ComponentValue + 46, // 61: atmos.plugin.v1.ComponentExecutionConfiguration.extra_arguments:type_name -> atmos.plugin.v1.ComponentExtraArguments + 47, // 62: atmos.plugin.v1.ComponentExecutionConfiguration.before_hooks:type_name -> atmos.plugin.v1.ComponentHook + 47, // 63: atmos.plugin.v1.ComponentExecutionConfiguration.after_hooks:type_name -> atmos.plugin.v1.ComponentHook + 47, // 64: atmos.plugin.v1.ComponentExecutionConfiguration.error_hooks:type_name -> atmos.plugin.v1.ComponentHook + 45, // 65: atmos.plugin.v1.ComponentExecutionConfiguration.apply:type_name -> atmos.plugin.v1.ComponentExecutionConfiguration + 62, // 66: atmos.plugin.v1.ComponentExtraArguments.environment:type_name -> atmos.plugin.v1.ComponentExtraArguments.EnvironmentEntry + 31, // 67: atmos.plugin.v1.PrepareComponentResponse.inputs:type_name -> atmos.plugin.v1.ComponentValue + 43, // 68: atmos.plugin.v1.PrepareComponentResponse.state:type_name -> atmos.plugin.v1.ComponentStateIdentity + 44, // 69: atmos.plugin.v1.PrepareComponentResponse.files:type_name -> atmos.plugin.v1.ComponentGeneratedFile + 36, // 70: atmos.plugin.v1.PrepareComponentResponse.dependencies:type_name -> atmos.plugin.v1.ComponentDependency + 35, // 71: atmos.plugin.v1.PrepareComponentResponse.sources:type_name -> atmos.plugin.v1.ComponentSource + 17, // 72: atmos.plugin.v1.PrepareComponentResponse.diagnostics:type_name -> atmos.plugin.v1.Diagnostic + 45, // 73: atmos.plugin.v1.PrepareComponentResponse.execution:type_name -> atmos.plugin.v1.ComponentExecutionConfiguration + 63, // 74: atmos.plugin.v1.PrepareComponentResponse.input_origins:type_name -> atmos.plugin.v1.PrepareComponentResponse.InputOriginsEntry + 49, // 75: atmos.plugin.v1.PrepareComponentResponse.errors:type_name -> atmos.plugin.v1.ComponentErrorsConfiguration + 52, // 76: atmos.plugin.v1.PrepareComponentResponse.exclude:type_name -> atmos.plugin.v1.ComponentExclude + 53, // 77: atmos.plugin.v1.PrepareComponentResponse.iam:type_name -> atmos.plugin.v1.ComponentIam + 31, // 78: atmos.plugin.v1.PrepareComponentResponse.encryption:type_name -> atmos.plugin.v1.ComponentValue + 64, // 79: atmos.plugin.v1.PrepareComponentResponse.feature_flags:type_name -> atmos.plugin.v1.PrepareComponentResponse.FeatureFlagsEntry + 50, // 80: atmos.plugin.v1.ComponentErrorsConfiguration.retry:type_name -> atmos.plugin.v1.ComponentRetryPolicy + 51, // 81: atmos.plugin.v1.ComponentErrorsConfiguration.ignore:type_name -> atmos.plugin.v1.ComponentIgnorePolicy + 65, // 82: atmos.plugin.v1.ComponentIgnorePolicy.signals:type_name -> atmos.plugin.v1.ComponentIgnorePolicy.SignalsEntry + 31, // 83: atmos.plugin.v1.ComponentIam.web_identity_token:type_name -> atmos.plugin.v1.ComponentValue + 43, // 84: atmos.plugin.v1.ResolveComponentStateResponse.state:type_name -> atmos.plugin.v1.ComponentStateIdentity + 35, // 85: atmos.plugin.v1.ResolveComponentStateResponse.sources:type_name -> atmos.plugin.v1.ComponentSource + 17, // 86: atmos.plugin.v1.ResolveComponentStateResponse.diagnostics:type_name -> atmos.plugin.v1.Diagnostic + 31, // 87: atmos.plugin.v1.ComponentOperation.OptionsEntry.value:type_name -> atmos.plugin.v1.ComponentValue + 15, // 88: atmos.plugin.v1.PrepareComponentResponse.InputOriginsEntry.value:type_name -> atmos.plugin.v1.SourceLocation + 31, // 89: atmos.plugin.v1.ComponentIgnorePolicy.SignalsEntry.value:type_name -> atmos.plugin.v1.ComponentValue + 7, // 90: atmos.plugin.v1.Info.Describe:input_type -> atmos.plugin.v1.DescribeRequest + 9, // 91: atmos.plugin.v1.Info.Configure:input_type -> atmos.plugin.v1.ConfigureRequest + 10, // 92: atmos.plugin.v1.AuthProvider.Validate:input_type -> atmos.plugin.v1.ProviderRequest + 10, // 93: atmos.plugin.v1.AuthProvider.Authenticate:input_type -> atmos.plugin.v1.ProviderRequest + 20, // 94: atmos.plugin.v1.AuthProvider.Refresh:input_type -> atmos.plugin.v1.RefreshRequest + 10, // 95: atmos.plugin.v1.AuthProvider.Logout:input_type -> atmos.plugin.v1.ProviderRequest + 10, // 96: atmos.plugin.v1.AuthProvider.Environment:input_type -> atmos.plugin.v1.ProviderRequest + 10, // 97: atmos.plugin.v1.AuthProvider.Paths:input_type -> atmos.plugin.v1.ProviderRequest + 23, // 98: atmos.plugin.v1.AuthProvider.PrepareEnvironment:input_type -> atmos.plugin.v1.PrepareEnvironmentRequest + 11, // 99: atmos.plugin.v1.AuthIdentity.Validate:input_type -> atmos.plugin.v1.IdentityRequest + 26, // 100: atmos.plugin.v1.AuthIdentity.Authenticate:input_type -> atmos.plugin.v1.IdentityAuthenticateRequest + 11, // 101: atmos.plugin.v1.AuthIdentity.Logout:input_type -> atmos.plugin.v1.IdentityRequest + 11, // 102: atmos.plugin.v1.AuthIdentity.Environment:input_type -> atmos.plugin.v1.IdentityRequest + 11, // 103: atmos.plugin.v1.AuthIdentity.Paths:input_type -> atmos.plugin.v1.IdentityRequest + 27, // 104: atmos.plugin.v1.AuthIdentity.PrepareEnvironment:input_type -> atmos.plugin.v1.IdentityPrepareRequest + 28, // 105: atmos.plugin.v1.Commands.Run:input_type -> atmos.plugin.v1.CommandRequest + 40, // 106: atmos.plugin.v1.ComponentDiscovery.Discover:input_type -> atmos.plugin.v1.DiscoverComponentsRequest + 42, // 107: atmos.plugin.v1.ComponentValidation.Validate:input_type -> atmos.plugin.v1.ComponentRequest + 42, // 108: atmos.plugin.v1.ComponentPreparation.Prepare:input_type -> atmos.plugin.v1.ComponentRequest + 42, // 109: atmos.plugin.v1.ComponentState.Resolve:input_type -> atmos.plugin.v1.ComponentRequest + 8, // 110: atmos.plugin.v1.Info.Describe:output_type -> atmos.plugin.v1.DescribeResponse + 68, // 111: atmos.plugin.v1.Info.Configure:output_type -> google.protobuf.Empty + 18, // 112: atmos.plugin.v1.AuthProvider.Validate:output_type -> atmos.plugin.v1.ValidateResponse + 14, // 113: atmos.plugin.v1.AuthProvider.Authenticate:output_type -> atmos.plugin.v1.AuthenticateEvent + 21, // 114: atmos.plugin.v1.AuthProvider.Refresh:output_type -> atmos.plugin.v1.RefreshResponse + 68, // 115: atmos.plugin.v1.AuthProvider.Logout:output_type -> google.protobuf.Empty + 22, // 116: atmos.plugin.v1.AuthProvider.Environment:output_type -> atmos.plugin.v1.EnvironmentResponse + 25, // 117: atmos.plugin.v1.AuthProvider.Paths:output_type -> atmos.plugin.v1.PathsResponse + 22, // 118: atmos.plugin.v1.AuthProvider.PrepareEnvironment:output_type -> atmos.plugin.v1.EnvironmentResponse + 18, // 119: atmos.plugin.v1.AuthIdentity.Validate:output_type -> atmos.plugin.v1.ValidateResponse + 14, // 120: atmos.plugin.v1.AuthIdentity.Authenticate:output_type -> atmos.plugin.v1.AuthenticateEvent + 68, // 121: atmos.plugin.v1.AuthIdentity.Logout:output_type -> google.protobuf.Empty + 22, // 122: atmos.plugin.v1.AuthIdentity.Environment:output_type -> atmos.plugin.v1.EnvironmentResponse + 25, // 123: atmos.plugin.v1.AuthIdentity.Paths:output_type -> atmos.plugin.v1.PathsResponse + 22, // 124: atmos.plugin.v1.AuthIdentity.PrepareEnvironment:output_type -> atmos.plugin.v1.EnvironmentResponse + 30, // 125: atmos.plugin.v1.Commands.Run:output_type -> atmos.plugin.v1.CommandEvent + 41, // 126: atmos.plugin.v1.ComponentDiscovery.Discover:output_type -> atmos.plugin.v1.DiscoverComponentsResponse + 18, // 127: atmos.plugin.v1.ComponentValidation.Validate:output_type -> atmos.plugin.v1.ValidateResponse + 48, // 128: atmos.plugin.v1.ComponentPreparation.Prepare:output_type -> atmos.plugin.v1.PrepareComponentResponse + 54, // 129: atmos.plugin.v1.ComponentState.Resolve:output_type -> atmos.plugin.v1.ResolveComponentStateResponse + 110, // [110:130] is the sub-list for method output_type + 90, // [90:110] is the sub-list for method input_type + 90, // [90:90] is the sub-list for extension type_name + 90, // [90:90] is the sub-list for extension extendee + 0, // [0:90] is the sub-list for field type_name } func init() { file_atmos_plugin_v1_plugin_proto_init() } @@ -3783,16 +4551,16 @@ func file_atmos_plugin_v1_plugin_proto_init() { (*AuthenticateEvent_Interaction)(nil), (*AuthenticateEvent_Result)(nil), } - file_atmos_plugin_v1_plugin_proto_msgTypes[16].OneofWrappers = []any{ + file_atmos_plugin_v1_plugin_proto_msgTypes[18].OneofWrappers = []any{ (*RefreshResponse_Credentials)(nil), (*RefreshResponse_Unsupported)(nil), } - file_atmos_plugin_v1_plugin_proto_msgTypes[25].OneofWrappers = []any{ + file_atmos_plugin_v1_plugin_proto_msgTypes[27].OneofWrappers = []any{ (*CommandEvent_Stdout)(nil), (*CommandEvent_Stderr)(nil), (*CommandEvent_Exit)(nil), } - file_atmos_plugin_v1_plugin_proto_msgTypes[28].OneofWrappers = []any{ + file_atmos_plugin_v1_plugin_proto_msgTypes[30].OneofWrappers = []any{ (*ComponentValueStep_Attribute)(nil), (*ComponentValueStep_Index)(nil), } @@ -3801,8 +4569,8 @@ func file_atmos_plugin_v1_plugin_proto_init() { File: protoimpl.DescBuilder{ GoPackagePath: reflect.TypeOf(x{}).PkgPath(), RawDescriptor: unsafe.Slice(unsafe.StringData(file_atmos_plugin_v1_plugin_proto_rawDesc), len(file_atmos_plugin_v1_plugin_proto_rawDesc)), - NumEnums: 2, - NumMessages: 53, + NumEnums: 3, + NumMessages: 63, NumExtensions: 0, NumServices: 8, }, diff --git a/pkg/provisioner/source/floating_ref.go b/pkg/provisioner/source/floating_ref.go new file mode 100644 index 00000000000..bcc25d24947 --- /dev/null +++ b/pkg/provisioner/source/floating_ref.go @@ -0,0 +1,93 @@ +package source + +import ( + "fmt" + "net/url" + "regexp" + "strings" + + cfg "github.com/cloudposse/atmos/pkg/config" + "github.com/cloudposse/atmos/pkg/schema" +) + +var ( + // A Git object id, abbreviated or full. + commitRefPattern = regexp.MustCompile(`^[0-9a-f]{7,64}$`) + // A release tag such as v1, 1.2.3, v1.2.3-rc.1, or 2024.01. + releaseRefPattern = regexp.MustCompile(`^v?[0-9]+([.][0-9]+)*([-+][0-9A-Za-z.+-]+)?$`) +) + +// floatingRef returns the Git ref that the source follows when it can move: a branch +// or any name that is neither a commit id nor a release tag. A Git source without any +// ref follows its default branch, which is reported as floating with an empty ref. A +// fetched copy of such a source is never refreshed unless a TTL expires or the user +// pulls with --force. +func floatingRef(spec *schema.VendorComponentSource) (string, bool) { + if spec == nil || spec.Uri == "" || isLocalSource(spec.Uri) { + return "", false + } + ref := spec.Version + if _, query, found := strings.Cut(spec.Uri, "?"); found { + if values, err := url.ParseQuery(query); err == nil && values.Get("ref") != "" { + ref = values.Get("ref") + } + } + if ref == "" { + return "", isGitSource(spec.Uri) + } + ref = strings.TrimPrefix(ref, "refs/tags/") + if commitRefPattern.MatchString(ref) || releaseRefPattern.MatchString(ref) { + return "", false + } + return ref, true +} + +// isGitSource reports whether a remote source is fetched with Git: a forced `git::` source, +// a scheme-less GitHub, GitLab, or Bitbucket path, or a repository path ending in `.git`. +// Archives and object-store sources have no branch, so they never float. +func isGitSource(uri string) bool { + if strings.HasPrefix(uri, "git::") { + return true + } + for _, host := range []string{"github.com/", "gitlab.com/", "bitbucket.org/"} { + if strings.HasPrefix(uri, host) { + return true + } + } + repository, _, _ := strings.Cut(uri, "?") + if scheme, rest, found := strings.Cut(repository, "://"); found { + repository = scheme + "://" + strings.SplitN(rest, "//", 2)[0] + } else { + repository = strings.SplitN(repository, "//", 2)[0] + } + return strings.HasSuffix(repository, ".git") +} + +// floatingRefNotice explains why a cached copy of a branch-pinned source is reused, and +// names the pull command of the component's own type. It returns an empty string when the +// source cannot drift or a TTL already governs it. +func floatingRefNotice(spec *schema.VendorComponentSource, componentType, component, stack string) string { + ref, floating := floatingRef(spec) + if !floating || spec.TTL != "" { + return "" + } + target := component + if stack != "" { + target += " -s " + stack + } + follows := fmt.Sprintf("the floating ref %q", ref) + if ref == "" { + follows = "its default branch" + } + return fmt.Sprintf("Source %s follows %s and is not re-fetched automatically; "+ + "refresh it with `%s %s --force`, or set a source ttl", component, follows, sourcePullCommand(componentType), target) +} + +// sourcePullCommand returns the `source pull` command of a component type, such as +// `atmos helmfile source pull` or `atmos aws cloudformation source pull`. +func sourcePullCommand(componentType string) string { + if componentType == "" { + componentType = cfg.TerraformComponentType + } + return "atmos " + strings.ReplaceAll(componentType, "/", " ") + " source pull" +} diff --git a/pkg/provisioner/source/local_stamp.go b/pkg/provisioner/source/local_stamp.go new file mode 100644 index 00000000000..b36c90f1eb7 --- /dev/null +++ b/pkg/provisioner/source/local_stamp.go @@ -0,0 +1,300 @@ +package source + +import ( + "encoding/hex" + "fmt" + "hash/fnv" + "io" + "io/fs" + "os" + "path/filepath" + + "github.com/cloudposse/atmos/pkg/perf" + "github.com/cloudposse/atmos/pkg/provisioner/workdir" + "github.com/cloudposse/atmos/pkg/schema" + "github.com/cloudposse/atmos/pkg/ui" + "github.com/cloudposse/atmos/pkg/vendor" +) + +// localStampSkippedDirs are runtime directories omitted from the ordinary stamp walk. +// A copied directory link can expose their contents outside these runtime paths. +var localStampSkippedDirs = map[string]struct{}{".git": {}, ".terraform": {}, ".terragrunt-cache": {}} + +// localSourceRoot returns the directory a local directory source copies from, resolved the same +// way as vendoring resolves it. It reports false for remote sources and for missing directories. +func localSourceRoot(spec *schema.VendorComponentSource) (string, bool) { + uri := vendor.NormalizeURI(resolveSourceURI(spec)) + if vendor.IsLocalPath(uri) && !filepath.IsAbs(uri) { + absolute, err := resolveLocalSourcePath(uri, "") + if err != nil { + return "", false + } + uri = absolute + } + dir, ok, err := localDirectorySource(uri) + if err != nil || !ok { + return "", false + } + return dir, true +} + +// localSourceStamp digests file metadata and contents of the local directory source. +// Contents are streamed so equal-size edits with unchanged timestamps are detected without +// buffering entire files. It returns an empty string for other sources or unreadable files. +// +// The stamp is a change-detection token compared across processes, not a credential hash, so it uses +// a non-cryptographic 128-bit FNV-1a digest. It stays stable for an unchanged source. +func localSourceStamp(spec *schema.VendorComponentSource) string { + root, ok := localSourceRoot(spec) + if !ok { + return "" + } + // Walk the target of a symlinked source root; WalkDir never descends into a root link. + if resolved, err := filepath.EvalSymlinks(root); err == nil { + root = resolved + } + hash := fnv.New128a() + links := &stampLinks{root: root, spec: spec, seen: make(map[string]bool)} + skip := vendor.CreateSkipFunc(root, spec.IncludedPaths, spec.ExcludedPaths) + if err := filepath.WalkDir(root, links.walk(hash, skip)); err != nil { + return "" + } + return hex.EncodeToString(hash.Sum(nil)) +} + +// stampSkipFunc reports whether the source copy excludes a path. +type stampSkipFunc func(os.FileInfo, string, string) (bool, error) + +// walk digests every file the source copy keeps and the targets of the links it copies. +func (links *stampLinks) walk(hash io.Writer, skip stampSkipFunc) fs.WalkDirFunc { + return func(path string, entry fs.DirEntry, err error) error { + if err != nil { + return err + } + if done, err := links.skipEntry(path, entry, skip); done { + return err + } + if err := stampEntry(hash, links.root, path, entry); err != nil { + return err + } + return links.stampTarget(hash, path, entry) + } +} + +// skipEntry reports whether entry is not stamped itself: excluded entries and directories. Excluded +// directories and runtime directories below the root are not descended either. +func (links *stampLinks) skipEntry(path string, entry fs.DirEntry, skip stampSkipFunc) (bool, error) { + info, err := entry.Info() + if err != nil { + return true, err + } + if excluded, err := skip(info, path, ""); err != nil || excluded { + if err == nil && entry.IsDir() { + return true, filepath.SkipDir + } + return true, err + } + if !entry.IsDir() { + return false, nil + } + if _, skipped := localStampSkippedDirs[entry.Name()]; skipped && path != links.root { + return true, filepath.SkipDir + } + return true, nil +} + +// stampLinks covers directories exposed by copied links, even when their real +// paths are runtime directories skipped by the ordinary walk. Real target paths +// make the traversal finite for directory cycles and repeated aliases. +type stampLinks struct { + root string + spec *schema.VendorComponentSource + seen map[string]bool +} + +func (links *stampLinks) stampTarget(hash io.Writer, path string, entry fs.DirEntry) error { + if follow, err := links.copiesLink(path, entry); !follow || err != nil { + return err + } + target, err := filepath.EvalSymlinks(path) + if err != nil { + return err + } + targetInfo, err := os.Stat(target) + if err != nil { + return err + } + if targetInfo.Mode().IsRegular() { + return stampFileContents(hash, target) + } + if !targetInfo.IsDir() || links.seen[target] { + return nil + } + links.seen[target] = true + return links.stampDirectory(hash, target) +} + +// copiesLink reports whether entry is a link that the source copy keeps and follows. +func (links *stampLinks) copiesLink(path string, entry fs.DirEntry) (bool, error) { + if entry.Type()&os.ModeSymlink == 0 { + return false, nil + } + info, err := entry.Info() + if err != nil { + return false, err + } + skip := vendor.CreateSkipFunc(links.root, links.spec.IncludedPaths, links.spec.ExcludedPaths) + if excluded, err := skip(info, path, ""); err != nil || excluded { + return false, err + } + opts := vendor.CopyOptions{IncludedPaths: links.spec.IncludedPaths, ExcludedPaths: links.spec.ExcludedPaths} + return vendor.CanCopySymlink(links.root, path, opts), nil +} + +func (links *stampLinks) stampDirectory(hash io.Writer, target string) error { + skip := vendor.CreateSkipFunc(links.root, links.spec.IncludedPaths, links.spec.ExcludedPaths) + return filepath.WalkDir(target, func(path string, entry fs.DirEntry, err error) error { + if err != nil { + return err + } + info, err := entry.Info() + if err != nil { + return err + } + if excluded, err := skip(info, path, ""); err != nil || excluded { + if err == nil && entry.IsDir() { + return filepath.SkipDir + } + return err + } + if entry.IsDir() { + return nil + } + if err := stampEntry(hash, links.root, path, entry); err != nil { + return err + } + return links.stampTarget(hash, path, entry) + }) +} + +// stampEntry adds one non-directory entry to the manifest digest. +func stampEntry(hash io.Writer, root, path string, entry fs.DirEntry) error { + info, err := entry.Info() + if err != nil { + return err + } + relative, err := filepath.Rel(root, path) + if err != nil { + return err + } + fmt.Fprintf(hash, "%s\x00%d\x00%d\x00%d\n", filepath.ToSlash(relative), info.Size(), info.ModTime().UnixNano(), info.Mode()) + if info.Mode().IsRegular() { + return stampFileContents(hash, path) + } + if info.Mode()&os.ModeSymlink != 0 { + // A link edited through its target changes the module without touching the link. + if target, statErr := os.Stat(path); statErr == nil { + fmt.Fprintf(hash, "->%d\x00%d\n", target.Size(), target.ModTime().UnixNano()) + } + } + return nil +} + +// stampFileContents adds a fixed-size content digest, keeping entry boundaries unambiguous. +func stampFileContents(hash io.Writer, path string) error { + file, err := os.Open(path) + if err != nil { + return err + } + defer file.Close() + content := fnv.New128a() + if _, err := io.Copy(content, file); err != nil { + return err + } + _, err = fmt.Fprintf(hash, "content:%x\n", content.Sum(nil)) + return err +} + +// localSourceChanged reports whether a local directory source was edited, or removed, since the +// workdir recorded its stamp. An explicit TTL stays authoritative. A local workdir without a stamp +// (provisioned before stamps existed, or a failed stamp write) is refreshed once so it records one, and so is one whose source cannot be +// stamped; a remote source never has a stamp, and URI, version, and TTL decide for it. +func localSourceChanged(metadata *workdir.WorkdirMetadata, spec *schema.VendorComponentSource) (bool, string) { + if spec.TTL != "" { + return false, "" + } + stamp := localSourceStamp(spec) + if metadata.SourceStamp == "" { + if stamp != "" { + return true, "Local source has no recorded stamp, refreshing the working directory" + } + if localSourceMissing(spec) { + // Reusing the copy would hide that the source is gone; refreshing reports it by name. + return true, "Local source is missing or unreadable, refreshing the working directory" + } + if _, local := localSourceRoot(spec); local { + // The directory exists but a file in it could not be read, so no stamp exists to compare. + // Reusing the copy would hide that; refreshing reports the read failure by name. + return true, "Local source could not be stamped, refreshing the working directory" + } + return false, "" + } + if stamp == metadata.SourceStamp { + return false, "" + } + return true, "Local source changed, refreshing the working directory" +} + +// localSourceMissing reports whether a local source names a path that does not exist or cannot +// be read. It reports false for remote sources. +func localSourceMissing(spec *schema.VendorComponentSource) bool { + uri := vendor.NormalizeURI(resolveSourceURI(spec)) + switch { + case vendor.IsFileURI(uri): + path, err := fileURIPath(uri) + if err != nil || path == "" { + return true + } + uri = path + case !vendor.IsLocalPath(uri): + return false + } + if !filepath.IsAbs(uri) { + absolute, err := resolveLocalSourcePath(uri, "") + if err != nil { + return true + } + uri = absolute + } + _, err := os.Stat(uri) + return err != nil +} + +// recordSourceStamp stores the stamp taken before the local source was copied. A failure only +// causes one extra refresh on the next run, because the stored stamp then differs. +func recordSourceStamp(targetDir, stamp string, out *ui.Output) { + defer perf.Track(nil, "source.recordSourceStamp")() + + if stamp == "" { + return + } + metadata, err := workdir.ReadMetadata(targetDir) + if err == nil && metadata != nil { + metadata.SourceStamp = stamp + err = workdir.WriteMetadata(targetDir, metadata) + } + if err != nil { + out.Warningf("Failed to record the local source stamp: %s", err) + } +} + +// sameLocalSource reports whether two local source declarations name one directory, so a +// relative and an absolute spelling of the same path do not look like a changed source. +func sameLocalSource(left, right string) bool { + if !isLocalSource(left) || !isLocalSource(right) { + return false + } + absoluteLeft, errLeft := filepath.Abs(left) + absoluteRight, errRight := filepath.Abs(right) + return errLeft == nil && errRight == nil && absoluteLeft == absoluteRight +} diff --git a/pkg/provisioner/source/local_stamp_test.go b/pkg/provisioner/source/local_stamp_test.go new file mode 100644 index 00000000000..55dbd5d5cb8 --- /dev/null +++ b/pkg/provisioner/source/local_stamp_test.go @@ -0,0 +1,538 @@ +package source + +import ( + "crypto/sha256" + "encoding/hex" + "io" + "os" + "path/filepath" + "testing" + "time" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + + errUtils "github.com/cloudposse/atmos/errors" + "github.com/cloudposse/atmos/pkg/downloader" + "github.com/cloudposse/atmos/pkg/provisioner" + "github.com/cloudposse/atmos/pkg/provisioner/workdir" + "github.com/cloudposse/atmos/pkg/schema" + "github.com/cloudposse/atmos/pkg/ui" + "github.com/cloudposse/atmos/pkg/vendor" +) + +func writeStampFile(t *testing.T, path, content string) { + t.Helper() + require.NoError(t, os.MkdirAll(filepath.Dir(path), 0o755)) + require.NoError(t, os.WriteFile(path, []byte(content), 0o644)) +} + +func TestLocalSourceStamp(t *testing.T) { + root := t.TempDir() + spec := &schema.VendorComponentSource{Uri: root} + writeStampFile(t, filepath.Join(root, "main.tf"), "# one") + base := localSourceStamp(spec) + require.NotEmpty(t, base) + assert.Equal(t, base, localSourceStamp(spec), "an untouched source has a stable stamp") + assert.Len(t, base, 32, "the stamp is a hex-encoded 128-bit digest") + + t.Run("edit changes the stamp", func(t *testing.T) { + writeStampFile(t, filepath.Join(root, "main.tf"), "# two, longer") + edited := localSourceStamp(spec) + assert.NotEqual(t, base, edited) + base = edited + }) + t.Run("same size edit with a new modification time changes the stamp", func(t *testing.T) { + path := filepath.Join(root, "main.tf") + before, err := os.Stat(path) + require.NoError(t, err) + writeStampFile(t, path, "# two, longer") + require.NoError(t, os.Chtimes(path, time.Now(), before.ModTime().Add(time.Hour))) + assert.NotEqual(t, base, localSourceStamp(spec)) + base = localSourceStamp(spec) + }) + t.Run("added and removed files change the stamp", func(t *testing.T) { + extra := filepath.Join(root, "nested", "extra.tf") + writeStampFile(t, extra, "# extra") + added := localSourceStamp(spec) + assert.NotEqual(t, base, added) + require.NoError(t, os.Remove(extra)) + assert.NotEqual(t, added, localSourceStamp(spec)) + }) + t.Run("ignored directories do not change the stamp", func(t *testing.T) { + before := localSourceStamp(spec) + for _, dir := range []string{".git", ".terraform", ".terragrunt-cache"} { + writeStampFile(t, filepath.Join(root, dir, "state"), "x") + } + assert.Equal(t, before, localSourceStamp(spec)) + }) + t.Run("remote and missing sources have no stamp", func(t *testing.T) { + assert.Empty(t, localSourceStamp(&schema.VendorComponentSource{Uri: "git::https://example.com/org/repo.git//module?ref=v1"})) + assert.Empty(t, localSourceStamp(&schema.VendorComponentSource{Uri: filepath.Join(root, "absent")})) + }) +} + +func TestLocalSourceStampRelativeSource(t *testing.T) { + root := t.TempDir() + t.Chdir(root) + writeStampFile(t, filepath.Join(root, "modules", "unit", "main.tf"), "# one") + relative := localSourceStamp(&schema.VendorComponentSource{Uri: "./modules/unit"}) + absolute := localSourceStamp(&schema.VendorComponentSource{Uri: filepath.Join(root, "modules", "unit")}) + require.NotEmpty(t, relative) + assert.Equal(t, absolute, relative, "a relative and an absolute spelling name one directory") +} + +func TestNeedsProvisioningLocalSourceStamp(t *testing.T) { + sourceDir := t.TempDir() + writeStampFile(t, filepath.Join(sourceDir, "main.tf"), "# one") + spec := &schema.VendorComponentSource{Uri: sourceDir} + target := t.TempDir() + writeStampFile(t, filepath.Join(target, "main.tf"), "# one") + require.NoError(t, writeWorkdirMetadata(target, "app", "dev", spec)) + + t.Run("metadata written by an explicit pull records the stamp and is reused", func(t *testing.T) { + metadata, err := workdir.ReadMetadata(target) + require.NoError(t, err) + assert.Equal(t, localSourceStamp(spec), metadata.SourceStamp) + needs, reason := needsProvisioning(target, spec, true) + assert.False(t, needs, reason) + }) + t.Run("a local workdir without a stamp is refreshed once", func(t *testing.T) { + metadata, err := workdir.ReadMetadata(target) + require.NoError(t, err) + metadata.SourceStamp = "" + require.NoError(t, workdir.WriteMetadata(target, metadata)) + needs, reason := needsProvisioning(target, spec, true) + assert.True(t, needs) + assert.Contains(t, reason, "no recorded stamp") + }) + + recordSourceStamp(target, localSourceStamp(spec), ui.New(nil)) + t.Run("an unchanged source is reused", func(t *testing.T) { + needs, reason := needsProvisioning(target, spec, true) + assert.False(t, needs, reason) + }) + t.Run("an edited source is refreshed", func(t *testing.T) { + writeStampFile(t, filepath.Join(sourceDir, "main.tf"), "# two") + needs, reason := needsProvisioning(target, spec, true) + assert.True(t, needs) + assert.Contains(t, reason, "Local source changed") + }) + t.Run("an explicit TTL stays authoritative over edits", func(t *testing.T) { + withTTL := &schema.VendorComponentSource{Uri: sourceDir, TTL: "1h"} + needs, reason := needsProvisioning(target, withTTL, true) + assert.False(t, needs, reason) + }) + t.Run("a removed source is refreshed so the failure names it", func(t *testing.T) { + // Keep the same spec, so URI-change detection cannot be what reports it. + stamp := localSourceStamp(spec) + require.NotEmpty(t, stamp) + recordSourceStamp(target, stamp, ui.New(nil)) + needs, reason := needsProvisioning(target, spec, true) + require.False(t, needs, reason) + require.NoError(t, os.RemoveAll(sourceDir)) + needs, reason = needsProvisioning(target, spec, true) + assert.True(t, needs) + assert.Contains(t, reason, "Local source changed") + }) +} + +// A relative and an absolute spelling of the same local source must not look like a source change. +func TestCheckMetadataChangesTreatsSpellingsOfOnePathAsEqual(t *testing.T) { + root := t.TempDir() + t.Chdir(root) + metadata := &workdir.WorkdirMetadata{Source: filepath.Join(root, "modules", "unit")} + changed, reason := checkMetadataChanges(metadata, &schema.VendorComponentSource{Uri: "./modules/unit"}) + assert.False(t, changed, reason) + changed, _ = checkMetadataChanges(metadata, &schema.VendorComponentSource{Uri: "./modules/other"}) + assert.True(t, changed) +} + +// Provisioning a local source through the workdir twice picks up an edit, copies nothing when +// nothing changed, and leaves runtime artifacts such as a saved plan in place. +func TestAutoProvisionSourceLocalEditDetection(t *testing.T) { + sourceDir := t.TempDir() + writeStampFile(t, filepath.Join(sourceDir, "main.tf"), "# v1") + config := &schema.AtmosConfiguration{BasePath: t.TempDir()} + newComponent := func() map[string]any { + return map[string]any{"component": "example", "atmos_stack": "dev", "source": map[string]any{"uri": sourceDir}, "provision": map[string]any{"workdir": map[string]any{"enabled": true}}} + } + provision := func() string { + component := newComponent() + require.NoError(t, AutoProvisionSource(t.Context(), config, "terraform", component, nil, provisioner.OutputWriters{})) + target, err := ResolveTarget(config, "terraform", "example", component) + require.NoError(t, err) + return target.Dir + } + + dir := provision() + assertContent := func(want string) { + t.Helper() + data, err := os.ReadFile(filepath.Join(dir, "main.tf")) + require.NoError(t, err) + assert.Equal(t, want, string(data)) + } + assertContent("# v1") + metadata, err := workdir.ReadMetadata(dir) + require.NoError(t, err) + require.NotEmpty(t, metadata.SourceStamp, "provisioning a local directory records its stamp") + + // Provisioning rewrites the metadata, so an unchanged timestamp proves nothing was copied. + marker := filepath.Join(dir, "marker.txt") + require.NoError(t, os.WriteFile(marker, []byte("keep"), 0o644)) + provision() + again, err := workdir.ReadMetadata(dir) + require.NoError(t, err) + assert.True(t, metadata.UpdatedAt.Equal(again.UpdatedAt), "an unchanged source must not be re-copied") + + // Editing the source refreshes the workdir in place: the new code appears, runtime state stays. + writeStampFile(t, filepath.Join(sourceDir, "main.tf"), "# v2 edited") + provision() + assertContent("# v2 edited") + refreshed, err := workdir.ReadMetadata(dir) + require.NoError(t, err) + assert.True(t, refreshed.UpdatedAt.After(metadata.UpdatedAt)) + assert.NotEqual(t, metadata.SourceStamp, refreshed.SourceStamp) + assert.FileExists(t, marker, "a refresh must preserve files the source does not own") +} + +// A source that is a directory symlink is stamped through its target, so an edit beneath the +// target refreshes the workdir. +func TestLocalSourceStampFollowsASymlinkedRoot(t *testing.T) { + root := t.TempDir() + target := filepath.Join(root, "modules", "unit") + writeStampFile(t, filepath.Join(target, "main.tf"), "# one") + link := filepath.Join(root, "linked") + trySymlink(t, target, link) + spec := &schema.VendorComponentSource{Uri: link} + before := localSourceStamp(spec) + require.NotEmpty(t, before) + writeStampFile(t, filepath.Join(target, "nested", "extra.tf"), "# extra") + assert.NotEqual(t, before, localSourceStamp(spec)) +} + +// An unstamped workdir of a local source that is gone is refreshed, so provisioning names the +// missing source instead of reusing stale configuration. A remote source is unaffected. +func TestLocalSourceChangedReportsAMissingUnstampedSource(t *testing.T) { + missing := &schema.VendorComponentSource{Uri: filepath.Join(t.TempDir(), "absent")} + changed, reason := localSourceChanged(&workdir.WorkdirMetadata{}, missing) + assert.True(t, changed) + assert.Contains(t, reason, "missing") + + remote := &schema.VendorComponentSource{Uri: "git::https://example.com/org/repo.git//module?ref=v1"} + changed, reason = localSourceChanged(&workdir.WorkdirMetadata{}, remote) + assert.False(t, changed, reason) + + present := t.TempDir() + writeStampFile(t, filepath.Join(present, "main.tf"), "# one") + changed, _ = localSourceChanged(&workdir.WorkdirMetadata{SourceStamp: localSourceStamp(&schema.VendorComponentSource{Uri: present})}, &schema.VendorComponentSource{Uri: present}) + assert.False(t, changed) +} + +// A local directory that exists but cannot be fully read has no stamp. An unstamped workdir of it +// is refreshed rather than reused, so the read failure is reported by provisioning. +func TestLocalSourceChangedRefreshesAnUnstampableLocalDirectory(t *testing.T) { + dir := t.TempDir() + writeStampFile(t, filepath.Join(dir, "main.tf"), "# one") + unreadable := filepath.Join(dir, "nested", "secret.tf") + writeStampFile(t, unreadable, "# two") + require.NoError(t, os.Chmod(unreadable, 0o000)) + t.Cleanup(func() { _ = os.Chmod(unreadable, 0o600) }) + spec := &schema.VendorComponentSource{Uri: dir} + if localSourceStamp(spec) != "" { + t.Skip("the process can read files regardless of their permissions") + } + require.False(t, localSourceMissing(spec), "the directory exists") + + changed, reason := localSourceChanged(&workdir.WorkdirMetadata{}, spec) + assert.True(t, changed) + assert.Contains(t, reason, "could not be stamped") + + // A remote source never has a stamp and is still reused. + remote := &schema.VendorComponentSource{Uri: "git::https://example.com/org/repo.git//module?ref=v1"} + changed, reason = localSourceChanged(&workdir.WorkdirMetadata{}, remote) + assert.False(t, changed, reason) +} + +func TestLocalSourceStampCopiedDirectoryLinks(t *testing.T) { + for _, dir := range []string{".terraform", ".terragrunt-cache"} { + t.Run(dir, func(t *testing.T) { + root := t.TempDir() + file := filepath.Join(root, dir, "nested", "main.tf") + writeStampFile(t, file, "# before") + trySymlink(t, dir, filepath.Join(root, "linked")) + spec := &schema.VendorComponentSource{Uri: root} + before := localSourceStamp(spec) + require.NotEmpty(t, before) + assert.Equal(t, before, localSourceStamp(spec)) + first := t.TempDir() + require.NoError(t, vendor.CopyToTarget(root, first, vendor.CopyOptions{})) + data, err := os.ReadFile(filepath.Join(first, "linked", "nested", "main.tf")) + require.NoError(t, err) + assert.Equal(t, "# before", string(data)) + parentInfo, err := os.Stat(filepath.Dir(file)) + require.NoError(t, err) + writeStampFile(t, file, "# after, longer") + // No directory metadata change can explain the changed stamp. + require.NoError(t, os.Chtimes(filepath.Dir(file), parentInfo.ModTime(), parentInfo.ModTime())) + assert.NotEqual(t, before, localSourceStamp(spec)) + changed, _ := localSourceChanged(&workdir.WorkdirMetadata{SourceStamp: before}, spec) + assert.True(t, changed) + second := t.TempDir() + require.NoError(t, vendor.CopyToTarget(root, second, vendor.CopyOptions{})) + data, err = os.ReadFile(filepath.Join(second, "linked", "nested", "main.tf")) + require.NoError(t, err) + assert.Equal(t, "# after, longer", string(data)) + }) + } +} + +func TestLocalSourceStampDirectoryLinkExclusions(t *testing.T) { + for _, tc := range []struct { + name string + target string + excludes []string + includes []string + external bool + }{ + {name: "git target", target: ".git"}, + {name: "external target", external: true}, + {name: "excluded link", target: ".terraform", excludes: []string{"linked"}}, + {name: "excluded target", target: ".terraform", excludes: []string{".terraform"}}, + {name: "excluded descendant", target: ".terraform", excludes: []string{"**/main.tf"}}, + {name: "include filter", target: ".terraform", includes: []string{"**/*.txt"}}, + } { + t.Run(tc.name, func(t *testing.T) { + root := t.TempDir() + target := filepath.Join(root, tc.target) + if tc.external { + target = t.TempDir() + } + file := filepath.Join(target, "nested", "main.tf") + writeStampFile(t, file, "# before") + trySymlink(t, target, filepath.Join(root, "linked")) + spec := &schema.VendorComponentSource{Uri: root, ExcludedPaths: tc.excludes, IncludedPaths: tc.includes} + before := localSourceStamp(spec) + require.NotEmpty(t, before) + writeStampFile(t, file, "# after, longer") + assert.Equal(t, before, localSourceStamp(spec)) + copyDir := t.TempDir() + require.NoError(t, vendor.CopyToTarget(root, copyDir, vendor.CopyOptions{IncludedPaths: tc.includes, ExcludedPaths: tc.excludes})) + assert.NoFileExists(t, filepath.Join(copyDir, "linked", "nested", "main.tf")) + }) + } +} + +func TestLocalSourceStampDirectoryLinkCycles(t *testing.T) { + root := t.TempDir() + cache := filepath.Join(root, ".terraform") + file := filepath.Join(cache, "main.tf") + writeStampFile(t, file, "# before") + trySymlink(t, cache, filepath.Join(root, "linked")) + trySymlink(t, cache, filepath.Join(root, "alias")) + trySymlink(t, root, filepath.Join(cache, "back")) + trySymlink(t, "missing", filepath.Join(cache, "dangling")) + spec := &schema.VendorComponentSource{Uri: root} + before := localSourceStamp(spec) + require.NotEmpty(t, before) + assert.Equal(t, before, localSourceStamp(spec)) + writeStampFile(t, file, "# after, longer") + assert.NotEqual(t, before, localSourceStamp(spec)) +} + +func replaceStampContents(t *testing.T, path string) { + t.Helper() + before, err := os.Stat(path) + require.NoError(t, err) + require.NoError(t, os.WriteFile(path, []byte("# two"), before.Mode())) + require.NoError(t, os.Chtimes(path, before.ModTime(), before.ModTime())) + after, err := os.Stat(path) + require.NoError(t, err) + require.Equal(t, before.Size(), after.Size()) + require.Equal(t, before.Mode(), after.Mode()) + require.Equal(t, before.ModTime(), after.ModTime()) +} + +func TestLocalSourceStampEqualMetadataContents(t *testing.T) { + for _, kind := range []string{"regular", "file link", "directory link"} { + t.Run(kind, func(t *testing.T) { + root := t.TempDir() + path := filepath.Join(root, "main.tf") + if kind != "regular" { + path = filepath.Join(root, ".terraform", "main.tf") + } + writeStampFile(t, path, "# one") + switch kind { + case "file link": + trySymlink(t, path, filepath.Join(root, "main.tf")) + case "directory link": + trySymlink(t, filepath.Dir(path), filepath.Join(root, "module")) + } + spec := &schema.VendorComponentSource{Uri: root} + before := localSourceStamp(spec) + require.NotEmpty(t, before) + require.Equal(t, before, localSourceStamp(spec)) + replaceStampContents(t, path) + after := localSourceStamp(spec) + require.NotEmpty(t, after) + assert.NotEqual(t, before, after) + assert.Equal(t, after, localSourceStamp(spec)) + changed, reason := localSourceChanged(&workdir.WorkdirMetadata{SourceStamp: before}, spec) + assert.True(t, changed) + assert.Contains(t, reason, "Local source changed") + }) + } +} + +func TestLocalSourceStampExcludedContents(t *testing.T) { + for _, kind := range []string{"excluded file", "include filter", "runtime directory", "external link"} { + t.Run(kind, func(t *testing.T) { + root := t.TempDir() + path := filepath.Join(root, "main.tf") + spec := &schema.VendorComponentSource{Uri: root} + switch kind { + case "excluded file": + spec.ExcludedPaths = []string{"main.tf"} + case "include filter": + spec.IncludedPaths = []string{"**/*.txt"} + case "runtime directory": + path = filepath.Join(root, ".terraform", "main.tf") + case "external link": + path = filepath.Join(t.TempDir(), "main.tf") + trySymlink(t, path, filepath.Join(root, "main.tf")) + } + writeStampFile(t, path, "# one") + before := localSourceStamp(spec) + require.NotEmpty(t, before) + replaceStampContents(t, path) + assert.Equal(t, before, localSourceStamp(spec)) + }) + } +} + +func TestStampFileContentsReadErrors(t *testing.T) { + // A directory opens successfully but cannot be read as file contents. + assert.Error(t, stampFileContents(io.Discard, t.TempDir())) + assert.Error(t, stampFileContents(io.Discard, filepath.Join(t.TempDir(), "missing"))) +} + +func TestSourceRefreshLegacyRemovedResources(t *testing.T) { + for _, name := range []string{"resources.tf", "exports/resources.tf", "legacy/main.tf", "outputs.tf.json"} { + t.Run(name, func(t *testing.T) { + src, target := t.TempDir(), t.TempDir() + refreshWrite(t, src, "main.tf", "# updated") + refreshWrite(t, target, "main.tf", "# original") + refreshWrite(t, target, name, "resource \"null_resource\" \"old\" {}") + runtime := map[string]string{ + "terraform.tfstate": "state", ".terraform/modules/example/resources.tf": "cached module", + ".terraform.lock.hcl": "provider lock", "reviewed-plan": "plan", "notes.txt": "user file", + } + for path, content := range runtime { + refreshWrite(t, target, path, content) + } + spec := &schema.VendorComponentSource{Uri: src} + require.NoError(t, writeWorkdirMetadata(target, "app", "dev", spec)) + metadata, err := workdir.ReadMetadata(target) + require.NoError(t, err) + metadata.SourceStamp = "" + require.NoError(t, workdir.WriteMetadata(target, metadata)) + needs, _ := needsProvisioning(target, spec, true) + require.True(t, needs) + before, err := downloader.TreeSHA256(target) + require.NoError(t, err) + err = VendorSource(t.Context(), nil, spec, target, WithRuntimePreservation(nil)) + require.ErrorIs(t, err, errUtils.ErrSourceCopyFailed) + assert.ErrorContains(t, err, name) + assert.ErrorContains(t, err, "inspect this file") + after, err := downloader.TreeSHA256(target) + require.NoError(t, err) + assert.Equal(t, before, after, "failure must preserve all source, runtime files and metadata") + assert.NoFileExists(t, filepath.Join(target, sourceFilesManifest)) + // Once the caller confirms and removes the obsolete file, migration succeeds. + require.NoError(t, os.Remove(filepath.Join(target, name))) + require.NoError(t, VendorSource(t.Context(), nil, spec, target, WithRuntimePreservation(nil))) + assert.FileExists(t, filepath.Join(target, sourceFilesManifest)) + for path, content := range runtime { + actual, err := os.ReadFile(filepath.Join(target, path)) + require.NoError(t, err) + assert.Equal(t, content, string(actual)) + } + }) + } +} + +func TestSourceRefreshLegacyIncomingResourcesBecomeOwned(t *testing.T) { + src, target := t.TempDir(), t.TempDir() + refreshWrite(t, src, "resources.tf", "# current") + refreshWrite(t, target, "resources.tf", "# previous") + spec := &schema.VendorComponentSource{Uri: src} + require.NoError(t, VendorSource(t.Context(), nil, spec, target, WithRuntimePreservation(nil))) + actual, err := os.ReadFile(filepath.Join(target, "resources.tf")) + require.NoError(t, err) + assert.Equal(t, "# current", string(actual)) + require.NoError(t, os.Remove(filepath.Join(src, "resources.tf"))) + refreshWrite(t, src, "main.tf", "# remaining") + require.NoError(t, VendorSource(t.Context(), nil, spec, target, WithRuntimePreservation(nil))) + assert.NoFileExists(t, filepath.Join(target, "resources.tf")) +} + +// Terraform files Atmos writes into the workdir on every run are not stale source configuration, so +// they do not block migrating a legacy workdir without a source ownership receipt. +func TestSourceRefreshLegacyKeepsHostGeneratedFiles(t *testing.T) { + src, target := t.TempDir(), t.TempDir() + refreshWrite(t, src, "main.tf", "# updated") + refreshWrite(t, target, "main.tf", "# original") + refreshWrite(t, target, "backend.tf.json", `{"terraform":{"backend":{"local":{}}}}`) + generated := "# generated by a configuration provider" + refreshWrite(t, target, "provider.tf", generated) + sum := sha256.Sum256([]byte(generated)) + refreshWrite(t, target, ".atmos/generated-files.json", `{"files":{"provider.tf":"`+hex.EncodeToString(sum[:])+`"}}`) + spec := &schema.VendorComponentSource{Uri: src} + require.NoError(t, writeWorkdirMetadata(target, "app", "dev", spec)) + metadata, err := workdir.ReadMetadata(target) + require.NoError(t, err) + metadata.SourceStamp = "" + require.NoError(t, workdir.WriteMetadata(target, metadata)) + + require.NoError(t, VendorSource(t.Context(), nil, spec, target, WithRuntimePreservation(nil))) + assert.FileExists(t, filepath.Join(target, sourceFilesManifest)) + content, err := os.ReadFile(filepath.Join(target, "main.tf")) + require.NoError(t, err) + assert.Equal(t, "# updated", string(content)) +} + +// A file the generated-files manifest names is exempt from the legacy guard only while it still +// holds the bytes the host recorded. A file edited since, or replaced, is inspected like any other. +func TestSourceRefreshLegacyRejectsChangedHostGeneratedFiles(t *testing.T) { + for _, test := range []struct { + name string + contents string + want bool + }{ + {"unchanged generated file is kept", "# generated by a configuration provider", false}, + {"edited generated file is rejected", "# edited by a user", true}, + } { + t.Run(test.name, func(t *testing.T) { + src, target := t.TempDir(), t.TempDir() + refreshWrite(t, src, "main.tf", "# updated") + refreshWrite(t, target, "main.tf", "# original") + sum := sha256.Sum256([]byte("# generated by a configuration provider")) + refreshWrite(t, target, "provider.tf", test.contents) + refreshWrite(t, target, ".atmos/generated-files.json", `{"files":{"provider.tf":"`+hex.EncodeToString(sum[:])+`"}}`) + spec := &schema.VendorComponentSource{Uri: src} + require.NoError(t, writeWorkdirMetadata(target, "app", "dev", spec)) + metadata, err := workdir.ReadMetadata(target) + require.NoError(t, err) + metadata.SourceStamp = "" + require.NoError(t, workdir.WriteMetadata(target, metadata)) + + err = VendorSource(t.Context(), nil, spec, target, WithRuntimePreservation(nil)) + if !test.want { + require.NoError(t, err) + return + } + require.ErrorIs(t, err, errUtils.ErrSourceCopyFailed) + require.ErrorContains(t, err, "provider.tf") + }) + } +} diff --git a/pkg/provisioner/source/provision_hook.go b/pkg/provisioner/source/provision_hook.go index 8cc7ca04273..abeafd0443d 100644 --- a/pkg/provisioner/source/provision_hook.go +++ b/pkg/provisioner/source/provision_hook.go @@ -153,6 +153,7 @@ func AutoProvisionSource( return err } // No provisioning needed - touch LastAccessed and return. + logProvisionMessage(ctx, writers, out, floatingRefNotice(sourceSpec, componentType, component, stack)) if isWorkdir { if err := workdir.UpdateLastAccessed(targetDir); err != nil { // Non-critical error - log and continue. @@ -164,19 +165,18 @@ func AutoProvisionSource( } // Log reason for re-provisioning. - if reason != "" { - if !workdir.OutputSuppressed(ctx) { - ui.Info(reason) - } else if writers.Stderr != nil { - out.Info(reason) - } - } + logProvisionMessage(ctx, writers, out, reason) if authContext != nil { ctx = downloader.WithAWSAuthContext(ctx, authContext.AWS) } hadSource := isNonEmptyDir(targetDir) + // Take the stamp before copying so an edit made during the copy is seen on the next run. + stamp := "" + if isWorkdir { + stamp = localSourceStamp(sourceSpec) + } // Vendor the source to target directory. var receipt downloader.ResolvedArtifact options := []VendorSourceOption{WithArtifactReceipt(&receipt)} @@ -195,6 +195,7 @@ func AutoProvisionSource( } } if isWorkdir { + recordSourceStamp(targetDir, stamp, out) setProvisionedWorkdir(componentConfig, targetDir) // Only first materialization has no backend cache. Refresh preserves .terraform; // normal init fingerprinting decides whether its actual inputs changed. @@ -382,6 +383,7 @@ func determineSourceTargetDirectory( // - URI has changed from what's in metadata. // - No metadata exists (fresh workdir). // - A remote source TTL expires (workdir or a shared directory with provenance). +// - A local directory source was edited since the workdir recorded its stamp. // // Returns (false, "") if the cached source can be reused. func needsProvisioning(targetDir string, sourceSpec *schema.VendorComponentSource, isWorkdir bool) (bool, string) { @@ -407,6 +409,11 @@ func needsProvisioning(targetDir string, sourceSpec *schema.VendorComponentSourc return changed, reason } + // An edited local directory source refreshes the workdir without waiting for a TTL. + if changed, reason = localSourceChanged(metadata, sourceSpec); changed { + return changed, reason + } + // Check TTL-based cache expiration. if sourceSpec.TTL != "" { expired, ttlReason := isSourceCacheExpired(sourceSpec.TTL, metadata.UpdatedAt) @@ -418,6 +425,19 @@ func needsProvisioning(targetDir string, sourceSpec *schema.VendorComponentSourc return false, "" } +// logProvisionMessage prints a one-line provisioning status unless output is suppressed +// and no dedicated writer exists. +func logProvisionMessage(ctx context.Context, writers provisioner.OutputWriters, out *ui.Output, message string) { + if message == "" { + return + } + if !workdir.OutputSuppressed(ctx) { + ui.Info(message) + } else if writers.Stderr != nil { + out.Info(message) + } +} + // isNonEmptyDir checks if the path exists, is a directory, and contains at least one entry // (excluding the .atmos metadata directory). func isNonEmptyDir(path string) bool { @@ -462,7 +482,7 @@ func checkMetadataChanges(metadata *workdir.WorkdirMetadata, sourceSpec *schema. if previous == "" { previous = metadata.SourceURI } - if sourceSpec.Uri != previous { + if sourceSpec.Uri != previous && !sameLocalSource(sourceSpec.Uri, previous) { return true, fmt.Sprintf("Source URI changed (%s → %s)", downloader.RedactSource(previous), downloader.RedactSource(sourceSpec.Uri)) } @@ -599,6 +619,9 @@ func writeWorkdirMetadata(workdirPath, component, stack string, sourceSpec *sche UpdatedAt: now, LastAccessed: now, ContentHash: "", // Content hash is computed separately for local sources. + // An explicit pull records the stamp too, so later edits to a local source still refresh + // the workdir. Automatic provisioning replaces it with the stamp taken before copying. + SourceStamp: localSourceStamp(sourceSpec), } // Use the fetch receipt captured before Git metadata is stripped. Compatibility // callers without one get an honest content digest, never the enclosing repo HEAD. diff --git a/pkg/provisioner/source/refresh.go b/pkg/provisioner/source/refresh.go index 8f7ae458fa7..20515163de6 100644 --- a/pkg/provisioner/source/refresh.go +++ b/pkg/provisioner/source/refresh.go @@ -1,8 +1,9 @@ package source import ( - "archive/zip" "context" + "crypto/sha256" + "encoding/hex" "encoding/json" "errors" "fmt" @@ -67,7 +68,10 @@ func vendorSourcePreserving(ctx context.Context, config *schema.AtmosConfigurati // Concurrent publishers of one checkout run one after another: a waiter later finds the target // up to date or republishes the same files. The wait is bounded by the caller's context. lock := filelock.New(provisioner.LockCoordPath(canonical+".source-refresh") + ".lock") - warning := time.AfterFunc(sourceRefreshLockWarnAfter, func() { warnSourceRefreshLockWait(ctx, target) }) + // The timer callback runs on its own goroutine, so it uses the hook captured here rather than + // reading the package variable when it fires (tests replace that variable). + warn := warnSourceRefreshLockWait + warning := time.AfterFunc(sourceRefreshLockWarnAfter, func() { warn(ctx, target) }) defer warning.Stop() err = lock.WithExclusive(ctx, func() error { warning.Stop() @@ -148,7 +152,11 @@ func publishSource(ctx context.Context, stage, target string, config map[string] if err := rejectActiveStateLock(target); err != nil { return err } - changes, incoming, err := sourceChanges(stage, target, next, previous, runtimePaths(target, config)) + protected := runtimePaths(target, config) + if err := rejectLegacyResources(target, next, protected, hostGeneratedFiles(target, moduleRoots(target, config))); err != nil { + return err + } + changes, incoming, err := sourceChanges(stage, target, next, previous, protected) if err != nil { return err } @@ -167,6 +175,110 @@ func publishSource(ctx context.Context, stage, target string, config map[string] return publication.publish(ctx, changes, incoming) } +// Without a receipt, neither metadata nor a source stamp proves ownership of a +// removed file. Refuse to bless stale configuration; let the caller reconcile it. +// Every Terraform configuration file absent from the incoming source counts, except +// runtime files and the files Atmos itself writes into the workdir on every run. +func rejectLegacyResources(target string, next sourceFiles, protected, generated map[string]bool) error { + if _, err := os.Lstat(filepath.Join(target, sourceFilesManifest)); !os.IsNotExist(err) { + return err + } + incoming := make(map[string]bool, len(next.Files)) + for _, name := range next.Files { + incoming[name] = true + } + err := filepath.WalkDir(target, func(path string, entry fs.DirEntry, err error) error { + if err != nil { + return err + } + relative, err := filepath.Rel(target, path) + if err != nil { + return err + } + relative = filepath.ToSlash(relative) + if protectedRuntimePath(path, relative, protected) { + if entry.IsDir() { + return filepath.SkipDir + } + return nil + } + if !staleLegacyFile(entry, relative, incoming, generated) { + return nil + } + return fmt.Errorf("%w: legacy workdir %s has no source ownership receipt and contains %s absent from the incoming source; inspect this file and move it outside the workdir or remove it if obsolete, then retry", errUtils.ErrSourceCopyFailed, target, relative) + }) + if os.IsNotExist(err) { + return nil + } + return err +} + +// hostWorkdirFiles are Terraform files Atmos generates in a component workdir on every run. +var hostWorkdirFiles = map[string]bool{"backend.tf.json": true, "providers_override.tf.json": true} + +// generatedFilesManifest lists, per module root, the files a configuration provider generated. +const generatedFilesManifest = ".atmos/generated-files.json" + +// staleLegacyFile reports a Terraform configuration file that the incoming source no longer has +// and that Atmos did not write into the workdir itself. +func staleLegacyFile(entry fs.DirEntry, relative string, incoming, generated map[string]bool) bool { + if entry.IsDir() || !terraformConfigurationFile(entry.Name()) { + return false + } + return !incoming[relative] && !generated[relative] && !hostWorkdirFiles[entry.Name()] +} + +func terraformConfigurationFile(name string) bool { + return strings.HasSuffix(name, ".tf") || strings.HasSuffix(name, ".tf.json") +} + +// matchesRecordedDigest reports whether path is a regular file whose bytes hash to the digest the +// generated-files manifest recorded. +func matchesRecordedDigest(path, digest string) bool { + info, err := os.Lstat(path) + if err != nil || !info.Mode().IsRegular() { + return false + } + data, err := os.ReadFile(path) + if err != nil { + return false + } + sum := sha256.Sum256(data) + return hex.EncodeToString(sum[:]) == digest +} + +// hostGeneratedFiles returns the target-relative paths of files recorded as generated in each +// module root and still holding the recorded bytes. An unreadable manifest records nothing, so its +// files are inspected like any other. +func hostGeneratedFiles(target string, roots []string) map[string]bool { + generated := map[string]bool{} + for _, root := range roots { + data, err := os.ReadFile(filepath.Join(root, filepath.FromSlash(generatedFilesManifest))) + if err != nil { + continue + } + var recorded struct { + Files map[string]string `json:"files"` + } + if json.Unmarshal(data, &recorded) != nil { + continue + } + for name, digest := range recorded.Files { + path := filepath.Join(root, filepath.FromSlash(name)) + // A file that no longer holds the recorded bytes was changed after the host wrote it, so + // the exemption does not apply and the legacy guard inspects it like any other file. + if !matchesRecordedDigest(path, digest) { + continue + } + relative, err := filepath.Rel(target, path) + if err == nil && filepath.IsLocal(relative) { + generated[filepath.ToSlash(relative)] = true + } + } + } + return generated +} + func readSourceInventory(target string) (sourceFiles, error) { previous := sourceFiles{} data, err := os.ReadFile(filepath.Join(target, sourceFilesManifest)) @@ -431,95 +543,3 @@ func underProtectedRoot(path string, protected map[string]bool) bool { } return false } - -// checkSourceDestination rejects a destination that traverses a symlink and, for an incoming file, -// one whose type differs from the source path. -func checkSourceDestination(target, name string, install bool) error { - if err := rejectSourceSymlinkAncestors(target, name); err != nil { - return err - } - if !install { - return nil - } - return rejectSourceTypeChange(target, name) -} - -// rejectSourceTypeChange fails before publication when an incoming file would replace a nonempty -// directory, or would need a directory where a file exists. Refresh replaces files in place and cannot -// change a path's type transactionally, so the error names the path and how to recover. An empty -// directory is still replaced by publication, as before. -func rejectSourceTypeChange(target, name string) error { - const recovery = "; refresh cannot change a path between a file and a directory in place. Move only that path out of the component directory (keep anything it holds that you still need), then refresh again; do not delete the whole directory, which would also remove local state, `.terraform` and saved plans" - local := filepath.FromSlash(name) - if entries, err := os.ReadDir(filepath.Join(target, local)); err == nil && len(entries) > 0 { - return fmt.Errorf("%w: source file %s conflicts with a nonempty directory in the target%s", - errUtils.ErrSourceCopyFailed, name, recovery) - } - for dir := filepath.Dir(local); dir != "." && dir != string(filepath.Separator); dir = filepath.Dir(dir) { - if info, err := os.Lstat(filepath.Join(target, dir)); err == nil && info.Mode().IsRegular() { - return fmt.Errorf("%w: source path %s needs a directory where the target has the file %s%s", - errUtils.ErrSourceCopyFailed, name, filepath.ToSlash(dir), recovery) - } - } - return nil -} - -func isTerraformPlan(path string) bool { - archive, err := zip.OpenReader(path) - if err != nil { - return false - } - defer archive.Close() - for _, file := range archive.File { - if file.Name == "tfplan" { - return true - } - } - return false -} - -// sourceChange distinguishes files owned by the source from runtime artifacts. -type sourceChangeFlags struct{ install, owned bool } - -func sourceChange(stage, target, name string, flags sourceChangeFlags, protected map[string]bool) (bool, error) { - install, owned := flags.install, flags.owned - if err := checkSourceDestination(target, name, install); err != nil { - return false, err - } - path := filepath.Join(target, filepath.FromSlash(name)) - runtime, err := runtimeOwned(path, name, protected) - if err != nil { - return false, fmt.Errorf("%w: inspect runtime artifact %s: %w", errUtils.ErrSourceCopyFailed, name, err) - } - if runtime { - return false, validateRuntimeSource(stage, path, name, flags, protected) - } - if !install { - _, err := os.Lstat(path) - return !os.IsNotExist(err), nil - } - if !owned { - if err := validateUnownedIncoming(stage, target, name); err != nil { - return false, err - } - } - return true, nil -} - -// identicalSourceFile permits unchanged source fixtures without replacing runtime-modified contents. -func identicalSourceFile(stage, path, name string) bool { - old, oldErr := os.ReadFile(path) - fresh, freshErr := os.ReadFile(filepath.Join(stage, filepath.FromSlash(name))) - return oldErr == nil && freshErr == nil && string(old) == string(fresh) -} - -// validateRuntimeSource allows only preserved locks and unchanged source-owned fixtures. -func validateRuntimeSource(stage, path, name string, flags sourceChangeFlags, protected map[string]bool) error { - if !flags.install || strings.HasSuffix(strings.ToLower(filepath.Base(name)), ".terraform.lock.hcl") { - return nil - } - if flags.owned && !protectedRuntimePath(path, name, protected) && identicalSourceFile(stage, path, name) { - return nil - } - return fmt.Errorf("%w: downloaded source collides with runtime artifact %s", errUtils.ErrSourceCopyFailed, name) -} diff --git a/pkg/provisioner/source/refresh_destination.go b/pkg/provisioner/source/refresh_destination.go new file mode 100644 index 00000000000..43789261a6b --- /dev/null +++ b/pkg/provisioner/source/refresh_destination.go @@ -0,0 +1,103 @@ +package source + +import ( + "archive/zip" + "fmt" + "os" + "path/filepath" + "strings" + + errUtils "github.com/cloudposse/atmos/errors" +) + +// checkSourceDestination rejects a destination that traverses a symlink and, for an incoming file, +// one whose type differs from the source path. +func checkSourceDestination(target, name string, install bool) error { + if err := rejectSourceSymlinkAncestors(target, name); err != nil { + return err + } + if !install { + return nil + } + return rejectSourceTypeChange(target, name) +} + +// rejectSourceTypeChange fails before publication when an incoming file would replace a nonempty +// directory, or would need a directory where a file exists. Refresh replaces files in place and cannot +// change a path's type transactionally, so the error names the path and how to recover. An empty +// directory is still replaced by publication, as before. +func rejectSourceTypeChange(target, name string) error { + const recovery = "; refresh cannot change a path between a file and a directory in place. Move only that path out of the component directory (keep anything it holds that you still need), then refresh again; do not delete the whole directory, which would also remove local state, `.terraform` and saved plans" + local := filepath.FromSlash(name) + if entries, err := os.ReadDir(filepath.Join(target, local)); err == nil && len(entries) > 0 { + return fmt.Errorf("%w: source file %s conflicts with a nonempty directory in the target%s", + errUtils.ErrSourceCopyFailed, name, recovery) + } + for dir := filepath.Dir(local); dir != "." && dir != string(filepath.Separator); dir = filepath.Dir(dir) { + if info, err := os.Lstat(filepath.Join(target, dir)); err == nil && info.Mode().IsRegular() { + return fmt.Errorf("%w: source path %s needs a directory where the target has the file %s%s", + errUtils.ErrSourceCopyFailed, name, filepath.ToSlash(dir), recovery) + } + } + return nil +} + +func isTerraformPlan(path string) bool { + archive, err := zip.OpenReader(path) + if err != nil { + return false + } + defer archive.Close() + for _, file := range archive.File { + if file.Name == "tfplan" { + return true + } + } + return false +} + +// sourceChange distinguishes files owned by the source from runtime artifacts. +type sourceChangeFlags struct{ install, owned bool } + +func sourceChange(stage, target, name string, flags sourceChangeFlags, protected map[string]bool) (bool, error) { + install, owned := flags.install, flags.owned + if err := checkSourceDestination(target, name, install); err != nil { + return false, err + } + path := filepath.Join(target, filepath.FromSlash(name)) + runtime, err := runtimeOwned(path, name, protected) + if err != nil { + return false, fmt.Errorf("%w: inspect runtime artifact %s: %w", errUtils.ErrSourceCopyFailed, name, err) + } + if runtime { + return false, validateRuntimeSource(stage, path, name, flags, protected) + } + if !install { + _, err := os.Lstat(path) + return !os.IsNotExist(err), nil + } + if !owned { + if err := validateUnownedIncoming(stage, target, name); err != nil { + return false, err + } + } + return true, nil +} + +// identicalSourceFile permits unchanged source fixtures without replacing runtime-modified contents. +func identicalSourceFile(stage, path, name string) bool { + old, oldErr := os.ReadFile(path) + fresh, freshErr := os.ReadFile(filepath.Join(stage, filepath.FromSlash(name))) + return oldErr == nil && freshErr == nil && string(old) == string(fresh) +} + +// validateRuntimeSource allows only preserved locks and unchanged source-owned fixtures. +func validateRuntimeSource(stage, path, name string, flags sourceChangeFlags, protected map[string]bool) error { + if !flags.install || strings.HasSuffix(strings.ToLower(filepath.Base(name)), ".terraform.lock.hcl") { + return nil + } + if flags.owned && !protectedRuntimePath(path, name, protected) && identicalSourceFile(stage, path, name) { + return nil + } + return fmt.Errorf("%w: downloaded source collides with runtime artifact %s", errUtils.ErrSourceCopyFailed, name) +} diff --git a/pkg/provisioner/source/source.go b/pkg/provisioner/source/source.go index 32ae52175e5..ae8136424c7 100644 --- a/pkg/provisioner/source/source.go +++ b/pkg/provisioner/source/source.go @@ -102,6 +102,12 @@ func Provision(ctx context.Context, params *ProvisionParams) error { if params.ComponentType == "terraform" { options = append(options, WithRuntimePreservation(params.ComponentConfig), withProvenance(target, params.Stack, sourceSpec, &receipt)) } + // Like automatic provisioning, take the stamp before copying, so an edit made during the + // copy refreshes the workdir on the next run. + stamp := "" + if target.IsWorkdir { + stamp = localSourceStamp(sourceSpec) + } err = spinner.ExecWithSpinner(progressMsg, completedMsg, func() error { if err := VendorSource(ctx, params.AtmosConfig, sourceSpec, targetDir, options...); err != nil { return err @@ -120,6 +126,9 @@ func Provision(ctx context.Context, params *ProvisionParams) error { "stack": params.Stack, }, "Verify source URI is accessible and credentials are valid") } + if target.IsWorkdir { + recordSourceStamp(targetDir, stamp, ui.New(nil)) + } restoreInstanceLock(targetDir, params.ComponentConfig) return nil diff --git a/pkg/provisioner/workdir/types.go b/pkg/provisioner/workdir/types.go index 9cd2ce60ac2..30eb6772df3 100644 --- a/pkg/provisioner/workdir/types.go +++ b/pkg/provisioner/workdir/types.go @@ -52,6 +52,11 @@ type WorkdirMetadata struct { // ContentHash is a hash of the source content for change detection (local sources only). ContentHash string `json:"content_hash,omitempty"` + + // SourceStamp is a stat manifest digest (path, size, modification time) of a local + // directory source taken before it was copied. A differing stamp means the source was + // edited since the workdir was provisioned. It is empty for remote sources. + SourceStamp string `json:"source_stamp,omitempty"` } // SourceType indicates the type of component source. diff --git a/pkg/provisioner/workdir/workdir.go b/pkg/provisioner/workdir/workdir.go index 569ee6d7bb4..7c9331be469 100644 --- a/pkg/provisioner/workdir/workdir.go +++ b/pkg/provisioner/workdir/workdir.go @@ -636,6 +636,11 @@ func buildLocalMetadata(params *localMetadataParams) *WorkdirMetadata { if !params.changed { metadata.UpdatedAt = params.existingMetadata.UpdatedAt } + // A stamp recorded for this same local source stays valid; dropping it would force a + // needless refresh on the next run. A stamp of another source is not carried over. + if params.existingMetadata.Source == params.componentPath { + metadata.SourceStamp = params.existingMetadata.SourceStamp + } } return metadata diff --git a/pkg/provisioner/workdir/workdir_test.go b/pkg/provisioner/workdir/workdir_test.go index 3e3f42c92f8..3b22aa0ba41 100644 --- a/pkg/provisioner/workdir/workdir_test.go +++ b/pkg/provisioner/workdir/workdir_test.go @@ -2197,3 +2197,12 @@ func TestServiceProvision_FailsClosedWhenExistingWorkdirBelongsToDifferentIdenti _, ok := componentConfig[WorkdirPathKey] assert.False(t, ok, "WorkdirPathKey must not be set when the existing workdir fails identity verification") } + +// A stamp recorded for the same local source survives a sync; a stamp of another source does not. +func TestBuildLocalMetadata_PreservesStampOfTheSameSource(t *testing.T) { + existing := &WorkdirMetadata{Source: "components/terraform/vpc", SourceStamp: "stamp"} + same := buildLocalMetadata(&localMetadataParams{componentPath: "components/terraform/vpc", existingMetadata: existing}) + assert.Equal(t, "stamp", same.SourceStamp) + other := buildLocalMetadata(&localMetadataParams{componentPath: "components/terraform/other", existingMetadata: existing}) + assert.Empty(t, other.SourceStamp) +} diff --git a/pkg/scheduler/adapters/mock_failure_runtime_test.go b/pkg/scheduler/adapters/mock_failure_runtime_test.go new file mode 100644 index 00000000000..542563b5743 --- /dev/null +++ b/pkg/scheduler/adapters/mock_failure_runtime_test.go @@ -0,0 +1,86 @@ +// Code generated by MockGen. DO NOT EDIT. +// Source: terraform_preflight_test.go +// +// Generated by this command: +// +// mockgen -source=terraform_preflight_test.go -destination=mock_failure_runtime_test.go -package=adapters -mock_names=failureCheckingRuntime=MockFailureCheckingRuntime +// + +// Package adapters is a generated GoMock package. +package adapters + +import ( + context "context" + reflect "reflect" + + schema "github.com/cloudposse/atmos/pkg/schema" + gomock "go.uber.org/mock/gomock" +) + +// MockFailureCheckingRuntime is a mock of failureCheckingRuntime interface. +type MockFailureCheckingRuntime struct { + ctrl *gomock.Controller + recorder *MockFailureCheckingRuntimeMockRecorder + isgomock struct{} +} + +// MockFailureCheckingRuntimeMockRecorder is the mock recorder for MockFailureCheckingRuntime. +type MockFailureCheckingRuntimeMockRecorder struct { + mock *MockFailureCheckingRuntime +} + +// NewMockFailureCheckingRuntime creates a new mock instance. +func NewMockFailureCheckingRuntime(ctrl *gomock.Controller) *MockFailureCheckingRuntime { + mock := &MockFailureCheckingRuntime{ctrl: ctrl} + mock.recorder = &MockFailureCheckingRuntimeMockRecorder{mock} + return mock +} + +// EXPECT returns an object that allows the caller to indicate expected use. +func (m *MockFailureCheckingRuntime) EXPECT() *MockFailureCheckingRuntimeMockRecorder { + return m.recorder +} + +// ComponentFailure mocks base method. +func (m *MockFailureCheckingRuntime) ComponentFailure(ctx context.Context, stack, component string) error { + m.ctrl.T.Helper() + ret := m.ctrl.Call(m, "ComponentFailure", ctx, stack, component) + ret0, _ := ret[0].(error) + return ret0 +} + +// ComponentFailure indicates an expected call of ComponentFailure. +func (mr *MockFailureCheckingRuntimeMockRecorder) ComponentFailure(ctx, stack, component any) *gomock.Call { + mr.mock.ctrl.T.Helper() + return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "ComponentFailure", reflect.TypeOf((*MockFailureCheckingRuntime)(nil).ComponentFailure), ctx, stack, component) +} + +// Discover mocks base method. +func (m *MockFailureCheckingRuntime) Discover(arg0 context.Context) (map[string]map[string]any, error) { + m.ctrl.T.Helper() + ret := m.ctrl.Call(m, "Discover", arg0) + ret0, _ := ret[0].(map[string]map[string]any) + ret1, _ := ret[1].(error) + return ret0, ret1 +} + +// Discover indicates an expected call of Discover. +func (mr *MockFailureCheckingRuntimeMockRecorder) Discover(arg0 any) *gomock.Call { + mr.mock.ctrl.T.Helper() + return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "Discover", reflect.TypeOf((*MockFailureCheckingRuntime)(nil).Discover), arg0) +} + +// Prepare mocks base method. +func (m *MockFailureCheckingRuntime) Prepare(arg0 context.Context, arg1 *schema.ConfigAndStacksInfo) (*schema.PreparedComponentConfiguration, error) { + m.ctrl.T.Helper() + ret := m.ctrl.Call(m, "Prepare", arg0, arg1) + ret0, _ := ret[0].(*schema.PreparedComponentConfiguration) + ret1, _ := ret[1].(error) + return ret0, ret1 +} + +// Prepare indicates an expected call of Prepare. +func (mr *MockFailureCheckingRuntimeMockRecorder) Prepare(arg0, arg1 any) *gomock.Call { + mr.mock.ctrl.T.Helper() + return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "Prepare", reflect.TypeOf((*MockFailureCheckingRuntime)(nil).Prepare), arg0, arg1) +} diff --git a/pkg/scheduler/adapters/terraform.go b/pkg/scheduler/adapters/terraform.go index 0bb757ecc5a..ee049c67715 100644 --- a/pkg/scheduler/adapters/terraform.go +++ b/pkg/scheduler/adapters/terraform.go @@ -173,6 +173,9 @@ func ExecuteTerraform(ctx context.Context, opts TerraformOptions) error { ui.Success("No components matched") return nil } + if err := preflightTerraformGraph(ctx, opts.AtmosConfig, graph); err != nil { + return err + } maxConcurrency := effectiveTerraformMaxConcurrency(opts.Info) if maxConcurrency > 1 { diff --git a/pkg/scheduler/adapters/terraform_preflight.go b/pkg/scheduler/adapters/terraform_preflight.go new file mode 100644 index 00000000000..6fc6c0f18c8 --- /dev/null +++ b/pkg/scheduler/adapters/terraform_preflight.go @@ -0,0 +1,33 @@ +package adapters + +import ( + "context" + + errUtils "github.com/cloudposse/atmos/errors" + "github.com/cloudposse/atmos/pkg/dependency" + "github.com/cloudposse/atmos/pkg/schema" +) + +// preflightTerraformGraph rejects selected components that their configuration provider could not +// evaluate, before any component runs. Without it a bulk run would apply the healthy components +// that sort first and only then fail on the broken one. Every broken component is reported. +func preflightTerraformGraph(ctx context.Context, atmosConfig *schema.AtmosConfiguration, graph *dependency.Graph) error { + checker, ok := atmosConfig.ComponentConfigurations.(schema.ComponentConfigurationFailureRuntime) + if !ok { + return nil + } + var failures []error + for _, id := range sortedGraphNodeIDs(graph) { + node := graph.Nodes[id] + if node == nil { + continue + } + if err := checker.ComponentFailure(ctx, node.Stack, node.Component); err != nil { + failures = append(failures, err) + } + } + if len(failures) == 0 { + return nil + } + return errUtils.Build(errUtils.ErrPluginComponentsFailed).WithCause(errUtils.JoinPreservingHints(failures...)).Err() +} diff --git a/pkg/scheduler/adapters/terraform_preflight_test.go b/pkg/scheduler/adapters/terraform_preflight_test.go new file mode 100644 index 00000000000..462a546ad8d --- /dev/null +++ b/pkg/scheduler/adapters/terraform_preflight_test.go @@ -0,0 +1,107 @@ +package adapters + +import ( + "context" + "errors" + "sync/atomic" + "testing" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + "go.uber.org/mock/gomock" + + errUtils "github.com/cloudposse/atmos/errors" + "github.com/cloudposse/atmos/pkg/dependency" + "github.com/cloudposse/atmos/pkg/schema" +) + +//go:generate go run go.uber.org/mock/mockgen@v0.6.0 -source=$GOFILE -destination=mock_failure_runtime_test.go -package=$GOPACKAGE -mock_names=failureCheckingRuntime=MockFailureCheckingRuntime + +// failureCheckingRuntime is a configuration runtime that reports components it could not evaluate. +type failureCheckingRuntime interface { + schema.ComponentConfigurationRuntime + schema.ComponentConfigurationFailureRuntime +} + +// The generated mock implements the contract that preflight asserts. +var _ failureCheckingRuntime = (*MockFailureCheckingRuntime)(nil) + +// failureRuntime holds the failures the generated runtime reports and records every component it +// is asked about, so tests can assert that each selected component is checked before dispatch. +type failureRuntime struct { + failures map[string]error + asked []string +} + +// mock returns a generated runtime that answers ComponentFailure from the fixture. Discover and +// Prepare have no expectations, so gomock rejects them. +func (r *failureRuntime) mock(t *testing.T) *MockFailureCheckingRuntime { + t.Helper() + runtime := NewMockFailureCheckingRuntime(gomock.NewController(t)) + runtime.EXPECT().ComponentFailure(gomock.Any(), gomock.Any(), gomock.Any()).DoAndReturn(func(_ context.Context, stack, component string) error { + r.asked = append(r.asked, stack+"/"+component) + return r.failures[stack+"/"+component] + }).AnyTimes() + return runtime +} + +func graphOf(t *testing.T, ids ...[2]string) *dependency.Graph { + t.Helper() + builder := dependency.NewBuilder() + for _, id := range ids { + require.NoError(t, builder.AddNode(&dependency.Node{ID: terraformNodeID(id[1], id[0]), Stack: id[0], Component: id[1]})) + } + graph, err := builder.Build() + require.NoError(t, err) + return graph +} + +func TestPreflightRejectsEveryBrokenComponentBeforeAnyRuns(t *testing.T) { + runtime := &failureRuntime{failures: map[string]error{"live/a": errors.New("first"), "live/c": errors.New("second")}} + config := &schema.AtmosConfiguration{ComponentConfigurations: runtime.mock(t)} + graph := graphOf(t, [2]string{"live", "a"}, [2]string{"live", "b"}, [2]string{"live", "c"}) + + err := preflightTerraformGraph(t.Context(), config, graph) + require.ErrorIs(t, err, errUtils.ErrPluginComponentsFailed) + assert.ErrorContains(t, err, "first") + assert.ErrorContains(t, err, "second") + assert.Len(t, runtime.asked, 3, "every selected component is checked") +} + +func TestPreflightIgnoresHealthyGraphsAndProvidersWithoutFailures(t *testing.T) { + graph := graphOf(t, [2]string{"live", "a"}) + require.NoError(t, preflightTerraformGraph(t.Context(), &schema.AtmosConfiguration{}, graph)) + require.NoError(t, preflightTerraformGraph(t.Context(), &schema.AtmosConfiguration{ComponentConfigurations: (&failureRuntime{}).mock(t)}, graph)) +} + +func TestExecuteTerraformPreflightRejectsFailuresBeforeDispatch(t *testing.T) { + runtime := &failureRuntime{failures: map[string]error{ + "dev/b-broken": errors.New("first component failed validation"), + "dev/c-broken": errors.New("second component failed validation"), + }} + // The healthy component sorts first and has no dependencies, so it would + // be eligible for dispatch if validation happened only as each node ran. + stacks := terraformStacksFixture(map[string]any{ + "a-healthy": map[string]any{}, + "b-broken": map[string]any{}, + "c-broken": map[string]any{}, + }) + var executorCalls atomic.Int32 + + err := ExecuteTerraform(t.Context(), TerraformOptions{ + AtmosConfig: &schema.AtmosConfiguration{ComponentConfigurations: runtime.mock(t)}, + Info: &schema.ConfigAndStacksInfo{All: true, SubCommand: "plan"}, + Stacks: stacks, + Executor: func(TerraformExecution) (TerraformExecutionResult, error) { + executorCalls.Add(1) + return TerraformExecutionResult{}, nil + }, + }) + + assert.Zero(t, executorCalls.Load(), "validation must finish before any component is dispatched") + require.ErrorIs(t, err, errUtils.ErrPluginComponentsFailed) + assert.ErrorContains(t, err, "first component failed validation") + assert.ErrorContains(t, err, "second component failed validation") + assert.ElementsMatch(t, []string{"dev/a-healthy", "dev/b-broken", "dev/c-broken"}, runtime.asked, + "every selected component must be validated") +} diff --git a/pkg/schema/component_configuration.go b/pkg/schema/component_configuration.go index 512f36474d4..232751fa3d8 100644 --- a/pkg/schema/component_configuration.go +++ b/pkg/schema/component_configuration.go @@ -36,12 +36,25 @@ type ComponentConfigurationRuntime interface { Prepare(context.Context, *ConfigAndStacksInfo) (*PreparedComponentConfiguration, error) } +// ComponentConfigurationFailureRuntime reports components that a provider discovered but could not +// evaluate. Bulk commands use it to reject every selected broken component before they run any +// infrastructure command. A nil error means the component is healthy or unknown to the provider. +type ComponentConfigurationFailureRuntime interface { + ComponentFailure(ctx context.Context, stack, component string) error +} + // ComponentConfigurationRevisionRuntime supplies isolated discovery for Git comparisons. // Providers without this capability must fail affected selection rather than reuse HEAD. type ComponentConfigurationRevisionRuntime interface { ForRevision(root string) (ComponentConfigurationRuntime, error) } +// ComponentConfigurationValidationRuntime supplies a manifest-reading view that can +// resolve additional imports without changing discovery or preparation inventory. +type ComponentConfigurationValidationRuntime interface { + ForValidation() ComponentConfigurationRuntime +} + // PreparedComponentConfiguration contains evaluated inputs and literal artifacts. type PreparedComponentConfiguration struct { Sources []ComponentConfigurationSource @@ -68,6 +81,8 @@ type ComponentConfigurationFile struct { // Mode carries optional POSIX permission bits. Zero selects the host default. // The writer clamps it, so a provider cannot request setuid or writable-by-others files. Mode uint32 + // UnitFileCopy identifies a literal adjacent unit-file copy, not explicit generation. + UnitFileCopy bool } // ConfigurationOwned reports whether a configuration provider owns a component's settings. diff --git a/pkg/schema/component_configuration_import.go b/pkg/schema/component_configuration_import.go index dc9a6c3e3e2..63e612e136c 100644 --- a/pkg/schema/component_configuration_import.go +++ b/pkg/schema/component_configuration_import.go @@ -2,10 +2,11 @@ package schema import "context" -// ComponentConfigurationImportRuntime accepts explicit provider imports before -// discovery. Imported definitions retain the provider's logical coordinates. +// ComponentConfigurationImportRuntime resolves explicit provider imports. The returned map +// holds the imported units as Terraform components, keyed by component name, so they +// become components of the importing stack instead of creating a stack of their own. type ComponentConfigurationImportRuntime interface { - RegisterImport(context.Context, string) (bool, error) + ImportComponents(context.Context, string) (components map[string]any, handled bool, err error) } // HasComponentConfiguration reports provider enablement without starting plugins. diff --git a/pkg/schema/schema.go b/pkg/schema/schema.go index 780f77ac633..58dfb9166ee 100644 --- a/pkg/schema/schema.go +++ b/pkg/schema/schema.go @@ -2245,6 +2245,8 @@ type ConfigAndStacksInfo struct { UseTerraformPlan bool PlanFile string StoredPlanFile string + // ConfigurationPlanImplicit retains implicit selection after restoration fills PlanFile. + ConfigurationPlanImplicit bool `yaml:"-" json:"-" mapstructure:"-"` // VerifyPlanMode carries the explicit CLI planfile-verify override for `deploy` // (--verify-plan => fail, --verify-plan=false => off; empty when the flag is unset). // It is resolved against config + CI mode via planfile.ResolveVerifyMode at the diff --git a/pkg/schema/stack_manifest_processing_result.go b/pkg/schema/stack_manifest_processing_result.go index 232d3a36674..131e0296208 100644 --- a/pkg/schema/stack_manifest_processing_result.go +++ b/pkg/schema/stack_manifest_processing_result.go @@ -3,6 +3,13 @@ package schema // StackManifestProcessingResult is the result of recursively processing a stack // manifest file and deep-merging all its imports. type StackManifestProcessingResult struct { + // UnitFragments retain provider defaults separately through recursive imports. + UnitFragments []map[string]any + // YAMLConfig excludes provider defaults, so a parent can place all units below + // every YAML layer without discarding explicit YAML values equal to defaults. + YAMLConfig map[string]any + // ImportedUnitSources records provider unit identity independently of merged YAML settings. + ImportedUnitSources map[string]string // ImportMetadata preserves literal metadata separately from rendered configuration. ImportMetadata StackImportMetadata // DeepMergedConfig is the final stack configuration after the manifest and diff --git a/pkg/terraform/backend/file.go b/pkg/terraform/backend/file.go index cade8b6c70d..ae58396ed79 100644 --- a/pkg/terraform/backend/file.go +++ b/pkg/terraform/backend/file.go @@ -1,8 +1,10 @@ package backend import ( + "fmt" "path/filepath" + errUtils "github.com/cloudposse/atmos/errors" log "github.com/cloudposse/atmos/pkg/logger" "github.com/cloudposse/atmos/pkg/perf" "github.com/cloudposse/atmos/pkg/schema" @@ -28,7 +30,15 @@ func GenerateFile(atmosConfig *schema.AtmosConfiguration, info *schema.ConfigAnd if err != nil { return err } - return u.WriteToFileAsJSON(backendFileName, componentBackendConfig, backendFileMode) + if err := u.WriteToFileAsJSON(backendFileName, componentBackendConfig, backendFileMode); err != nil { + return err + } + if _, owned := info.ComponentSettingsSection["configuration"]; owned { + if err := recordHostBackend(workingDir); err != nil { + return fmt.Errorf("%w: record ownership of %s: %w", errUtils.ErrBackendFileGeneration, backendFileName, err) + } + } + return nil } const backendFileMode = 0o600 diff --git a/pkg/terraform/backend/file_test.go b/pkg/terraform/backend/file_test.go index 78c5991055b..4fc1e7ff9a0 100644 --- a/pkg/terraform/backend/file_test.go +++ b/pkg/terraform/backend/file_test.go @@ -70,3 +70,77 @@ func TestGenerateFileConfiguration(t *testing.T) { }) } } + +func TestHostBackendOwnership(t *testing.T) { + for _, mode := range []string{"unchanged", "modified", "unrecorded", "symlink", "missing", "corrupt receipt"} { + t.Run(mode, func(t *testing.T) { + dir := t.TempDir() + config := &schema.AtmosConfiguration{} + config.Components.Terraform.AutoGenerateBackendFile = true + info := &schema.ConfigAndStacksInfo{ComponentBackendType: "local", ComponentSettingsSection: map[string]any{"configuration": map[string]any{}}} + require.NoError(t, GenerateFile(config, info, dir)) + path := filepath.Join(dir, hostBackendName) + switch mode { + case "modified": + require.NoError(t, os.WriteFile(path, []byte("user changes"), 0o600)) + case "unrecorded": + require.NoError(t, os.Remove(filepath.Join(dir, hostBackendReceipt))) + case "symlink": + require.NoError(t, os.Rename(path, filepath.Join(dir, "user.tf.json"))) + require.NoError(t, os.Symlink("user.tf.json", path)) + case "missing": + require.NoError(t, os.Remove(path)) + case "corrupt receipt": + require.NoError(t, os.WriteFile(filepath.Join(dir, hostBackendReceipt), []byte("invalid"), 0o600)) + } + err := RemoveHostBackend(dir) + if mode == "modified" || mode == "symlink" || mode == "corrupt receipt" { + require.ErrorIs(t, err, errUtils.ErrPluginInvalidConfig) + assert.FileExists(t, path) + } else { + require.NoError(t, err) + if mode == "unrecorded" { + assert.FileExists(t, path) + } else { + assert.NoFileExists(t, path) + } + } + }) + } +} + +// A provider-owned backend whose ownership receipt cannot be written fails with the backend +// generation error and keeps the filesystem cause. +func TestGenerateFileWrapsReceiptFailures(t *testing.T) { + dir := t.TempDir() + // A file where the receipt directory belongs makes the receipt impossible to create. + require.NoError(t, os.WriteFile(filepath.Join(dir, ".atmos"), nil, 0o600)) + config := &schema.AtmosConfiguration{} + config.Components.Terraform.AutoGenerateBackendFile = true + info := &schema.ConfigAndStacksInfo{ + SubCommand: "plan", ComponentBackendType: "local", ComponentBackendSection: map[string]any{"path": "state.tfstate"}, + ComponentSettingsSection: map[string]any{"configuration": map[string]any{"provider": "terragrunt"}}, + } + err := GenerateFile(config, info, dir) + require.ErrorIs(t, err, errUtils.ErrBackendFileGeneration) + require.ErrorContains(t, err, "backend.tf.json") + var pathErr *os.PathError + require.ErrorAs(t, err, &pathErr, "the filesystem cause is preserved") +} + +// Filesystem failures while relinquishing the host backend are classified with repository +// sentinels and keep their causes; a missing receipt is still not an error. +func TestRemoveHostBackendWrapsFilesystemFailures(t *testing.T) { + err := RemoveHostBackend(filepath.Join(t.TempDir(), "absent")) + require.ErrorIs(t, err, errUtils.ErrOpenFile) + require.ErrorIs(t, err, os.ErrNotExist) + + require.NoError(t, RemoveHostBackend(t.TempDir()), "no receipt means nothing to remove") + + dir := t.TempDir() + // A directory where the receipt file belongs cannot be read as a receipt. + require.NoError(t, os.MkdirAll(filepath.Join(dir, ".atmos", "host-backend.sha256"), 0o700)) + err = RemoveHostBackend(dir) + require.ErrorIs(t, err, errUtils.ErrReadFile) + require.ErrorContains(t, err, "host-backend.sha256") +} diff --git a/pkg/terraform/backend/ownership.go b/pkg/terraform/backend/ownership.go new file mode 100644 index 00000000000..58dd177b42c --- /dev/null +++ b/pkg/terraform/backend/ownership.go @@ -0,0 +1,101 @@ +package backend + +import ( + "crypto/sha256" + "encoding/hex" + "errors" + "fmt" + "os" + + errUtils "github.com/cloudposse/atmos/errors" + "github.com/cloudposse/atmos/pkg/perf" +) + +// The receipt records only the fixed host backend, never a caller-supplied path. +// Keep it separate from provider-generated files and out of module content digests. +const ( + hostBackendReceipt = ".atmos/host-backend.sha256" + hostBackendName = "backend.tf.json" + hostBackendDirectoryMode = 0o700 +) + +func backendDigest(root *os.Root) (string, error) { + info, err := root.Lstat(hostBackendName) + if err != nil { + return "", err + } + if !info.Mode().IsRegular() { + return "", fmt.Errorf("%w: host backend is not a regular file", errUtils.ErrPluginInvalidConfig) + } + data, err := root.ReadFile(hostBackendName) + if err != nil { + return "", err + } + sum := sha256.Sum256(data) + return hex.EncodeToString(sum[:]), nil +} + +func recordHostBackend(directory string) error { + root, err := os.OpenRoot(directory) + if err != nil { + return err + } + defer root.Close() + digest, err := backendDigest(root) + if err != nil { + return err + } + if err := root.MkdirAll(".atmos", hostBackendDirectoryMode); err != nil { + return err + } + return root.WriteFile(hostBackendReceipt, []byte(digest), backendFileMode) +} + +// RemoveHostBackend removes only the unchanged file recorded by the host writer. +// Older, unrecorded backends remain user-owned: never infer ownership from a name +// or matching backend settings. Terraform will reject any remaining duplicate. +func RemoveHostBackend(directory string) error { + defer perf.Track(nil, "backend.RemoveHostBackend")() + root, err := os.OpenRoot(directory) + if err != nil { + return hostBackendError(errUtils.ErrOpenFile, "open working directory", err) + } + defer root.Close() + receipt, err := root.ReadFile(hostBackendReceipt) + if errors.Is(err, os.ErrNotExist) { + return nil + } + if err != nil { + return hostBackendError(errUtils.ErrReadFile, "read "+hostBackendReceipt, err) + } + digest, err := backendDigest(root) + if errors.Is(err, os.ErrNotExist) { + return removeHostBackendFile(root, hostBackendReceipt) + } + if err != nil { + if errors.Is(err, errUtils.ErrPluginInvalidConfig) { + return err + } + return hostBackendError(errUtils.ErrReadFile, "read "+hostBackendName, err) + } + if digest != string(receipt) { + return fmt.Errorf("%w: refusing to remove modified host backend %s", errUtils.ErrPluginInvalidConfig, hostBackendName) + } + if err := removeHostBackendFile(root, hostBackendName); err != nil { + return err + } + return removeHostBackendFile(root, hostBackendReceipt) +} + +// removeHostBackendFile removes one host-owned file and classifies a failure. +func removeHostBackendFile(root *os.Root, name string) error { + if err := root.Remove(name); err != nil { + return hostBackendError(errUtils.ErrFileOperation, "remove "+name, err) + } + return nil +} + +// hostBackendError wraps a filesystem failure with a repository sentinel and keeps its cause. +func hostBackendError(sentinel error, action string, err error) error { + return fmt.Errorf("%w: host backend: %s: %w", sentinel, action, err) +} diff --git a/pkg/vendor/copy.go b/pkg/vendor/copy.go index 50f97e638b3..d0afeadef11 100644 --- a/pkg/vendor/copy.go +++ b/pkg/vendor/copy.go @@ -56,6 +56,15 @@ func CopyToTarget(srcDir, dstDir string, opts CopyOptions) error { return cp.Copy(srcDir, dstDir, copyOptions) } +// CanCopySymlink reports whether the copier follows a link, including its source +// containment and resolved-target exclusion checks. Callers must also apply +// CreateSkipFunc to the link itself. +func CanCopySymlink(srcDir, src string, opts CopyOptions) bool { + defer perf.Track(nil, "vendor.CanCopySymlink")() + + return createSymlinkHandler(srcDir, opts)(src) == cp.Deep +} + // createSymlinkHandler returns an OnSymlink handler that validates symlink targets // resolve within srcDir before following them, and applies the same skip policy // (e.g., .git exclusion, exclude patterns) to the resolved target path. diff --git a/plugins/terragrunt/README.md b/plugins/terragrunt/README.md index cd1e9860bcf..a33b12e805c 100644 --- a/plugins/terragrunt/README.md +++ b/plugins/terragrunt/README.md @@ -50,25 +50,45 @@ Run `atmos list instances`, `atmos describe stacks`, and `atmos terraform plan - Preparation receives Atmos's sanitized environment after authentication, the selected executable, toolchain PATH, and managed Terraform RC configuration. Child commands use the invocation PATH. Neither identity changes nor HCL evaluation replace the plugin process environment. +## Names, imports, patterns, and symlinks + +User-facing rules are in [the migration guide](https://atmos.tools/migration/terragrunt#component-and-stack-names). The implementation points to know: + +- Discovery derives the component from the unit directory name and the stack from the directory above it (`default` and `root` at the base path; the stack file's directory and the in-stack unit path for `terragrunt.stack.hcl`). `engine/discovery.go` owns these names. +- A `terragrunt://` import is resolved by the host while stack manifests are read (`pkg/plugin/adapter/configuration/imports.go`). The plugin returns the units; the host drops the derived stack and merges them as components of the importing manifest. Imports ignore `included_paths` and `excluded_paths`, and a unit that is imported is removed from automatic discovery. +- A YAML component with the name of an imported unit is an override, not a conflict. The host merges every imported unit as the lowest layer of the manifest (`internal/exec/stack_processor_utils.go`), so YAML anywhere in the import chain deep-merges over it; only two different units with one name conflict (`CheckImportConflicts`). At run time the host merges the stack's `vars` over the unit's evaluated inputs and lets the stack `env` win over the unit's `extra_arguments` environment (`pkg/component/terraform/configuration_overrides.go`). The backend, `metadata.component`, `source.uri`, and a switch of engine through `command` are owned by the unit and are rejected with `ErrConfigurationOverrideUnsupported`. The migration guide lists the precedence per section. +- `included_paths` and `excluded_paths` share one matcher (`engine/discovery_paths.go`): a pattern selects what it matches and everything below it. Unmatched include patterns, shared `terragrunt.hcl` files without `terraform.source`, and skipped directory symlinks are returned as warning diagnostics, not failures. +- Symlinks follow Terragrunt v1.2.1: directory links are not followed (a warning names them) and a symlinked `terragrunt.hcl` is a unit at the link's location (`engine.logicalPath`). + ## Affected selection The `describe affected --base ` and `terraform plan --affected --ref ` paths discover HEAD and BASE independently. Both checkouts use the invocation's trusted plugin and discovery scope. Historical HCL never reuses the current inventory or reads dependency state. The affected acceptance runner uses temporary Git repositories and dry runs. It checks additions, deletions, renames, changed dependency targets, shared includes, local module files, nested explicit stacks, mixed YAML/HCL graphs, stack filters, and dependent depth. Both output and ordering edges participate in `--include-dependents`. The runner requires Git but no Terraform or Terragrunt executable. -Source tracking covers files read during partial discovery, declared HCL, local module directories, stack template provenance, and files and globs named by input expressions. A dynamic path that cannot be resolved without running the expression falls back to treating the unit as affected by any change in its scope. Effect-dependent discovery still fails. This is not a claim that every Terragrunt expression has complete affected provenance. +A unit is selected when any file it reads changes: `terragrunt.hcl` and its includes; the module directory (for a `dir//subdir` source the whole directory before `//`, and the target of a symlinked module); `extra_arguments` var files; files read by `generate` and `remote_state` blocks; hook scripts and working directories inside the project; `.terraform.lock.hcl` beside the unit; and files read by `file()`, `templatefile()`, `read_terragrunt_config()` and similar functions in `inputs`. Paths built from `get_terragrunt_dir()`, `get_parent_terragrunt_dir()`, `get_repo_root()`, `path_relative_to_include()`, `find_in_parent_folders()`, `abspath()`, `format()`, `join()` and literal locals resolve to exact files. A path that depends on the environment (`get_env`), dependency outputs, or another unevaluated value makes the unit depend on everything in its scope. Effect-dependent discovery still fails. This is not a claim that every Terragrunt expression has complete affected provenance. Results are sorted by stack, then by name within each component type. ## Scope The fixtures cover local sources, local backend paths, includes, locals, nested stack values, literal generation, order and output dependencies, command-scoped mocks, fresh outputs, and native bulk execution. Unit tests cover absolute and relative workdir overrides, ownership conflicts, repeated provisioning without state loss, concurrent environment isolation, unit-relative child commands, cancellation, and rejected effects. The plugin workflow runs its unit tests on Linux, macOS, and Windows. These checks do not establish a compatibility percentage. -The host executes `extra_arguments` and `before_hook`, `after_hook`, and `error_hook` blocks with the invocation's environment, identity, toolchain PATH, and cancellation. The plugin only evaluates them. Hooks run once per Atmos phase, and an implicit `init` matches hooks for `init`, not for the requested command. Hook output goes to stderr. Workspace selection, `providers lock`, dependency-output reads, and `atmos terraform shell` do not run hooks. A missing required var file fails before any process starts. Saved-plan apply reuses the execution data recorded at plan time and requires the original working directory. +The host executes `extra_arguments` and `before_hook`, `after_hook`, and `error_hook` blocks with the invocation's environment, identity, toolchain PATH, and cancellation. The plugin only evaluates them. Hooks run once per Atmos phase, and an implicit `init` matches hooks for `init`, not for the requested command. Hook output goes to stderr. Workspace selection, `providers lock`, dependency-output reads, and `atmos terraform shell` do not run hooks. Hooks match the first word of the Terraform command, so `commands = ["state"]` runs for every `state` subcommand. A `plan -detailed-exitcode` status of 2 (changes present) is not a hook failure: after hooks run, error hooks do not, and Atmos still exits with 2. A relative hook `working_dir` or `required_var_files` entry resolves against the provisioned working directory; use `${get_terragrunt_dir()}` for paths beside `terragrunt.hcl`. A missing required var file fails before any process starts. + +A saved plan is kept until it is applied. `output`, `validate`, `init` and `state list` leave it in place. A successful `apply`, `deploy` or `destroy` removes the plan and its Atmos record, so applying the same plan twice reports that no saved plan exists. Apply reuses the execution data recorded at plan time and requires the original working directory. The record holds no input values and no backend settings; Atmos compares the backend by digest at apply time. -Remote module sources (`git::https`, `git::http`, `git::ssh`, `git::file`, `https`, `http`) keep their getter URI for the Atmos source provisioner. Local module sources are re-copied on every run (`source.ttl: 0s`) so edits take effect. Registry sources and remote stack catalogs are not supported. Backend configuration is adopted, including cloud backends, but Atmos does not provision the backend resources. +Remote module sources (`git::https`, `git::http`, `git::ssh`, `git::file`, `https`, `http`) keep their getter URI for the Atmos source provisioner. A local module is copied into the working directory again only when its files change (path, size, or modification time); an explicit `source.ttl` takes precedence. A source that follows a branch (`?ref=main`) is fetched once and reused, and Atmos prints a notice when it reuses such a copy: run `atmos terraform source pull -s --force` to refresh it, or set `source.ttl`. Release tags and commit ids are never reported. Registry sources and remote stack catalogs are not supported. Backend configuration is adopted, including cloud backends, but Atmos does not provision the backend resources. + +Atmos removes a generated file when its `generate` block is removed, but only while the file is unchanged or still carries the Terragrunt signature. `if_exists = "error"` accepts the file Atmos generated on an earlier run, and all generated files are mode 0600. A `generate` block whose contents declare a `backend` or `cloud` block is rejected because Atmos cannot verify its state-reader identity. Declare the backend with `remote_state` and use `remote_state.generate` instead. Backend-only state reads inspect generation without evaluating inputs or commands: Terraform generate blocks must have literal paths and contents that can be proven not to declare a backend; dynamic or attribute-form generation is rejected in this phase, including in parent configurations. Automatic includes are also rejected for state reads because the upstream partial parser does not merge them; use explicit includes for backend configuration. The `.terraform.lock.hcl` beside `terragrunt.hcl` is copied into the working directory when it changes; Atmos does not copy changes back, so copy `.workdir/.../.terraform.lock.hcl` yourself after `init -upgrade`. + +A unit without a stable `remote_state` path keeps local state under `.terragrunt-cache`, which moves with the checkout path and disappears when the cache is deleted. Atmos keeps that location, to preserve existing state identity, and warns when it applies. + +`dependencies.tools` is read from stack manifests and component sections, not from `atmos.yaml`. An imported unit takes `dependencies.tools` from the importing stack's configuration, and those pins take precedence over `.tool-versions`. A discovered unit has no stack manifest, so it takes its tool pins from `.tool-versions`. Unsupported semantics fail before any infrastructure command and never switch executors. Currently rejected: `terraform_binary`; `errors`, `exclude`, feature flags, and IAM role or web-identity settings; `remote_state` with `disable_init` or encryption; inline modules without `terraform.source`; output dependencies with a non-default workspace during discovery; the `read-config`, `terragrunt-read-config`, and `init-from-module` hook lifecycles; hooks that run a built-in integration such as `tflint`; and an invalid `on_errors` expression. Executable mode, typed provider flags and help, exact cty types and sensitivity, scoped Terragrunt logging, migration and setup generation, and Atmos Pro integration remain in the PRDs. -Only enabled/mode/provider/discovery settings are accepted by this package. Additional plugin settings and typed CLI option handling remain planned. Discovery that needs command execution, decryption, network access, or identity-dependent effects fails. The native plugin is trusted code, not an OS sandbox. +Unsupported settings fail with an error that names the unit, the file, and the setting, plus the line and column when the evaluator reports a position (settings rejected after decoding, such as `terraform_binary`, name only the file). A broken unit does not affect other units or YAML components: it stays listed, and only commands that target it fail. `apply`, `plan`, and `destroy` with `--all` validate every selected unit first and fail with "no infrastructure command was run". Evaluation errors can contain secrets, so Atmos shows the file, line, unit, and a hint but not the evaluator's text; run with `ATMOS_LOGS_LEVEL=Debug` to log the full message with secrets masked. + +Plugin settings are validated when first used, not when `atmos.yaml` loads, so `atmos config set` can always repair them. Unknown keys under `components.terraform.terragrunt` and invalid `mode` values fail with the key name. With `discovery.enabled: true` and no `included_paths`, discovery covers every unit under the project root. Only enabled/mode/provider/discovery settings are accepted by this package. Additional plugin settings and typed CLI option handling remain planned. Discovery that needs command execution, decryption, network access, or identity-dependent effects fails. The native plugin is trusted code, not an OS sandbox. ## Maintained upstream facade diff --git a/plugins/terragrunt/bridge/execution_test.go b/plugins/terragrunt/bridge/execution_test.go index a1866bc0998..5fe4bb59ac2 100644 --- a/plugins/terragrunt/bridge/execution_test.go +++ b/plugins/terragrunt/bridge/execution_test.go @@ -6,8 +6,13 @@ import ( "os" "os/exec" "path/filepath" + "runtime" "sync" "testing" + + "github.com/gruntwork-io/terragrunt/pkg/config" + "github.com/gruntwork-io/terragrunt/pkg/log" + "github.com/spf13/afero" ) func TestEnvironmentLookupUsesInvocationPath(t *testing.T) { @@ -49,3 +54,99 @@ func TestEnvironmentLookupUsesInvocationPath(t *testing.T) { t.Fatal("environment mutated") } } + +// accessLog records every path a filesystem is asked to open or stat. +type accessLog struct { + afero.Fs + mu sync.Mutex + paths []string +} + +func (a *accessLog) record(name string) { + a.mu.Lock() + defer a.mu.Unlock() + a.paths = append(a.paths, name) +} + +func (a *accessLog) Open(name string) (afero.File, error) { + a.record(name) + return a.Fs.Open(name) +} + +func (a *accessLog) Stat(name string) (os.FileInfo, error) { + a.record(name) + return a.Fs.Stat(name) +} + +func (a *accessLog) LstatIfPossible(name string) (os.FileInfo, bool, error) { + a.record(name) + return a.Fs.(afero.Lstater).LstatIfPossible(name) +} + +// The unit directory is used as the root of copy-rule globs. A glob metacharacter in its path +// must neither widen the scan to the unit's parent nor change which files the overlay carries. +func TestUnitOverlayStaysInsideUnitWithGlobMetacharacters(t *testing.T) { + base := t.TempDir() + unit := filepath.Join(base, "unit{with-brace") + sibling := filepath.Join(base, "sibling") + for _, dir := range []string{unit, sibling} { + if err := os.MkdirAll(dir, 0o755); err != nil { + t.Fatal(err) + } + } + for path, content := range map[string]string{ + filepath.Join(unit, "main.tf"): "# unit", + filepath.Join(unit, ".tflint.hcl"): "# lint", + filepath.Join(sibling, "other.tf"): "# sibling", + filepath.Join(sibling, "nested.tfx"): "# sibling", + } { + if err := os.WriteFile(path, []byte(content), 0o644); err != nil { + t.Fatal(err) + } + } + recorder := &accessLog{Fs: afero.NewOsFs()} + env := Environment(map[string]string{}, nil, nil).WithFS(recorder) + files, _, err := UnitOverlay(log.New(), env, unit, &config.TerraformConfig{}) + if err != nil { + t.Fatal(err) + } + if string(files["main.tf"]) != "# unit" || string(files[".tflint.hcl"]) != "# lint" || len(files) != 2 { + t.Fatalf("overlay files = %v", files) + } + recorder.mu.Lock() + defer recorder.mu.Unlock() + for _, path := range recorder.paths { + if rel, err := filepath.Rel(unit, path); err != nil || !filepath.IsLocal(rel) { + t.Errorf("overlay read %q outside the unit directory", path) + } + } +} + +// An unreadable sibling of a unit whose path holds a glob metacharacter must not fail the overlay. +func TestUnitOverlayIgnoresUnreadableSiblingDirectories(t *testing.T) { + if runtime.GOOS == "windows" || os.Geteuid() == 0 { + t.Skip("directory permissions are not enforced here") + } + base := t.TempDir() + unit := filepath.Join(base, "unit{with-brace") + locked := filepath.Join(base, "locked") + for _, dir := range []string{unit, locked} { + if err := os.MkdirAll(dir, 0o755); err != nil { + t.Fatal(err) + } + } + if err := os.WriteFile(filepath.Join(unit, "main.tf"), []byte("# unit"), 0o644); err != nil { + t.Fatal(err) + } + if err := os.Chmod(locked, 0); err != nil { + t.Fatal(err) + } + t.Cleanup(func() { _ = os.Chmod(locked, 0o755) }) + files, _, err := UnitOverlay(log.New(), Environment(map[string]string{}, nil, nil), unit, &config.TerraformConfig{}) + if err != nil { + t.Fatal(err) + } + if string(files["main.tf"]) != "# unit" { + t.Fatalf("overlay files = %v", files) + } +} diff --git a/plugins/terragrunt/bridge/overlay.go b/plugins/terragrunt/bridge/overlay.go index aef719d6418..574f9904ff6 100644 --- a/plugins/terragrunt/bridge/overlay.go +++ b/plugins/terragrunt/bridge/overlay.go @@ -24,8 +24,13 @@ var ErrUnsupportedOverlay = errors.New("unit file cannot be exported through the // effect on platforms with POSIX permission bits. func UnitOverlay(logger log.Logger, env *Venv, unit string, terraform *config.TerraformConfig) (map[string][]byte, map[string]uint32, error) { memory := afero.NewMemMapFs() - fsys := afero.NewCopyOnWriteFs(afero.NewReadOnlyFs(env.FS), memory) - destination := filepath.Join(filepath.Dir(unit), ".atmos-unit-overlay") + // Upstream joins the copy source with each include and exclude pattern and expands the result as + // a glob, so a `*` or `{` anywhere in the unit's real path would turn part of the path into a + // pattern and widen the walk to its parent. The copy runs on a view of the unit rooted at a + // fixed path that holds no glob characters, which bounds every read to the unit's own tree. + view := newUnitView(unit) + fsys := afero.NewCopyOnWriteFs(afero.NewReadOnlyFs(view.mount(env.FS)), memory) + unit, destination := view.source, view.destination include := []string{".tflint.hcl"} var exclude []string if terraform.IncludeInCopy != nil { @@ -54,6 +59,11 @@ func UnitOverlay(logger log.Logger, env *Venv, unit string, terraform *config.Te if name == ".terragrunt-module-manifest" { return nil } + // The lock file beside the unit is handed to the host as a unit file copy with its own + // ownership rules, so the overlay leaves it out rather than rejecting it. + if filepath.ToSlash(name) == util.TerraformLockFile { + return nil + } if err := checkOverlayFile(name); err != nil { return err } @@ -62,7 +72,9 @@ func UnitOverlay(logger log.Logger, env *Venv, unit string, terraform *config.Te return err } files[filepath.ToSlash(name)] = content - perm := restrictByDirectories(env.FS, unit, filepath.Dir(name), info.Mode().Perm()) + // The directories are read through the unit view, because unit is the view's fixed source path + // and does not exist in the host filesystem. + perm := restrictByDirectories(fsys, unit, filepath.Dir(name), info.Mode().Perm()) if exportsMode(perm) { modes[filepath.ToSlash(name)] = uint32(perm) } @@ -71,6 +83,102 @@ func UnitOverlay(logger log.Logger, env *Venv, unit string, terraform *config.Te return files, modes, err } +// unitView maps a fixed, glob-safe source path onto the unit's real directory. +type unitView struct { + unit, source, destination string +} + +// newUnitView places the view's source and destination side by side at the root of the unit's volume. +func newUnitView(unit string) unitView { + root := filepath.VolumeName(unit) + string(filepath.Separator) + return unitView{ + unit: filepath.Clean(unit), + source: filepath.Join(root, "atmos-unit-source"), + destination: filepath.Join(root, "atmos-unit-overlay"), + } +} + +// mount returns base with the view's source path served from the unit directory. +func (v unitView) mount(base afero.Fs) afero.Fs { + return rebaseFs{Fs: base, view: v} +} + +// rebaseFs serves paths under the view's source from the unit directory and reports the view's +// destination as absent, because that tree exists only in the in-memory layer above it. Every other +// path passes through, so a symlink target outside the unit still reads normally. The overlay copy +// only reads through it, behind a read-only layer. +type rebaseFs struct { + afero.Fs + view unitView +} + +// resolve returns the real path to read for name. +func (r rebaseFs) resolve(op, name string) (string, error) { + clean := filepath.Clean(filepath.FromSlash(name)) + if under(r.view.destination, clean) { + return "", &os.PathError{Op: op, Path: name, Err: os.ErrNotExist} + } + if rel, err := filepath.Rel(r.view.source, clean); err == nil && filepath.IsLocal(rel) { + return filepath.Join(r.view.unit, rel), nil + } + return name, nil +} + +// under reports whether path is root or lies below it. +func under(root, path string) bool { + rel, err := filepath.Rel(root, path) + return err == nil && filepath.IsLocal(rel) +} + +func (r rebaseFs) Open(name string) (afero.File, error) { + target, err := r.resolve("open", name) + if err != nil { + return nil, err + } + return r.Fs.Open(target) +} + +func (r rebaseFs) OpenFile(name string, flag int, perm os.FileMode) (afero.File, error) { + target, err := r.resolve("open", name) + if err != nil { + return nil, err + } + return r.Fs.OpenFile(target, flag, perm) +} + +func (r rebaseFs) Stat(name string) (os.FileInfo, error) { + target, err := r.resolve("stat", name) + if err != nil { + return nil, err + } + return r.Fs.Stat(target) +} + +// LstatIfPossible lets the copy tell a symlink from the file it points to. +func (r rebaseFs) LstatIfPossible(name string) (os.FileInfo, bool, error) { + target, err := r.resolve("lstat", name) + if err != nil { + return nil, false, err + } + if lstater, ok := r.Fs.(afero.Lstater); ok { + return lstater.LstatIfPossible(target) + } + info, err := r.Fs.Stat(target) + return info, false, err +} + +// ReadlinkIfPossible reports a link's target, which stays a real path. +func (r rebaseFs) ReadlinkIfPossible(name string) (string, error) { + target, err := r.resolve("readlink", name) + if err != nil { + return "", err + } + if reader, ok := r.Fs.(afero.LinkReader); ok { + return reader.ReadlinkIfPossible(target) + } + return "", &os.PathError{Op: "readlink", Path: target, Err: afero.ErrNoReadlink} +} + // restrictByDirectories conservatively narrows a file's permission bits by the // subdirectories that contain it. The overlay carries file modes only, so a // readable file inside a private directory (for example 0644 under 0700) would diff --git a/plugins/terragrunt/fixtures/toolprobe/main.go b/plugins/terragrunt/fixtures/toolprobe/main.go new file mode 100644 index 00000000000..feb5b5756df --- /dev/null +++ b/plugins/terragrunt/fixtures/toolprobe/main.go @@ -0,0 +1,63 @@ +// Command toolprobe stands in for a version-pinned OpenTofu in acceptance tests. It records +// its arguments in invocations.log beside itself and then runs the real executable named in +// real-tool.txt, so a test can prove which commands used the pinned copy. +package main + +import ( + "errors" + "fmt" + "os" + "os/exec" + "path/filepath" + "strings" +) + +const ( + logMode = 0o600 + failureCode = 127 +) + +func main() { + os.Exit(run()) +} + +func run() int { + self, err := os.Executable() + if err != nil { + return fail(err) + } + directory := filepath.Dir(self) + real, err := os.ReadFile(filepath.Join(directory, "real-tool.txt")) + if err != nil { + return fail(err) + } + if err := record(filepath.Join(directory, "invocations.log"), os.Args[1:]); err != nil { + return fail(err) + } + // The target is the path a test wrote beside this fixture, never external input. + command := exec.Command(strings.TrimSpace(string(real)), os.Args[1:]...) //nolint:gosec // Test fixture runs the tool it was told to wrap. + command.Stdin, command.Stdout, command.Stderr = os.Stdin, os.Stdout, os.Stderr + if err := command.Run(); err != nil { + var exit *exec.ExitError + if errors.As(err, &exit) { + return exit.ExitCode() + } + return fail(err) + } + return 0 +} + +func record(path string, arguments []string) error { + file, err := os.OpenFile(path, os.O_APPEND|os.O_CREATE|os.O_WRONLY, logMode) + if err != nil { + return err + } + defer file.Close() + _, err = fmt.Fprintln(file, strings.Join(arguments, " ")) + return err +} + +func fail(err error) int { + fmt.Fprintln(os.Stderr, "toolprobe:", err) + return failureCode +} diff --git a/plugins/terragrunt/go.mod b/plugins/terragrunt/go.mod index e6c2d8092fb..838b739976c 100644 --- a/plugins/terragrunt/go.mod +++ b/plugins/terragrunt/go.mod @@ -14,8 +14,8 @@ require ( github.com/spf13/afero v1.15.0 github.com/stretchr/testify v1.12.1 github.com/zclconf/go-cty v1.19.0 - google.golang.org/genproto/googleapis/rpc v0.0.0-20261019180522-ce908d29299b google.golang.org/grpc v1.84.0 + google.golang.org/protobuf v1.36.12 gopkg.in/yaml.v3 v3.0.1 ) @@ -231,7 +231,7 @@ require ( google.golang.org/api v0.308.0 // indirect google.golang.org/genproto v0.0.0-20260720171339-e059f2f05d78 // indirect google.golang.org/genproto/googleapis/api v0.0.0-20260928230214-8a89bd6388cc // indirect - google.golang.org/protobuf v1.36.12 // indirect + google.golang.org/genproto/googleapis/rpc v0.0.0-20261019180522-ce908d29299b // indirect gopkg.in/ini.v1 v1.67.3 // indirect oras.land/oras-go/v2 v2.7.0 // indirect ) diff --git a/plugins/terragrunt/internal/engine/backend.go b/plugins/terragrunt/internal/engine/backend.go index 52640b50a33..b7954a50fd9 100644 --- a/plugins/terragrunt/internal/engine/backend.go +++ b/plugins/terragrunt/internal/engine/backend.go @@ -2,7 +2,6 @@ package engine import ( "errors" - "fmt" "github.com/gruntwork-io/terragrunt/pkg/atmosbridge" "github.com/gruntwork-io/terragrunt/pkg/config" @@ -17,11 +16,11 @@ func resolveBackend(cfg *config.TerragruntConfig, result *model.Resolution) erro if cfg.RemoteState == nil { return nil } - if cfg.RemoteState.DisableInit { - return ErrUnsupportedRemoteState - } - if len(cfg.RemoteState.Encryption) > 0 { - return fmt.Errorf("%w: remote_state.encryption", ErrUnsupportedRemoteState) + if cfg.RemoteState.DisableInit || len(cfg.RemoteState.Encryption) > 0 { + return &model.Problem{ + Code: model.CodeUnsupportedSetting, Summary: "Unsupported Terragrunt setting in `remote_state`", Detail: "The native adaptation does not implement `disable_init` or `remote_state.encryption`.", + Hints: []string{"Remove `disable_init` and `encryption` from `remote_state`. To manage the backend with Atmos instead, migrate the unit to a regular Atmos component with its own `backend` settings."}, File: result.Path, Key: "remote_state", Safe: true, Err: errors.Join(ErrUnsupportedExecution, ErrUnsupportedRemoteState), + } } result.BackendType = cfg.RemoteState.BackendName result.Backend = atmosbridge.BackendConfig(cfg.RemoteState.Config) diff --git a/plugins/terragrunt/internal/engine/backend_generation.go b/plugins/terragrunt/internal/engine/backend_generation.go new file mode 100644 index 00000000000..5a86f3b953a --- /dev/null +++ b/plugins/terragrunt/internal/engine/backend_generation.go @@ -0,0 +1,133 @@ +package engine + +import ( + "errors" + "io/fs" + "path/filepath" + "strings" + + "github.com/hashicorp/hcl/v2" + "github.com/hashicorp/hcl/v2/hclparse" + "github.com/spf13/afero" + "github.com/zclconf/go-cty/cty" + + "github.com/cloudposse/atmos/plugins/terragrunt/internal/model" +) + +func unresolvedGeneratedBackend(path string) error { + return &model.Problem{ + Code: model.CodeUnsupportedSetting, Summary: "Cannot verify the backend identity of a `generate` block", + Detail: "State readers cannot use an unverified generated backend or cloud configuration.", + Hints: []string{"Declare the backend in `remote_state` and use `remote_state.generate` to generate it. For state reads, other generate blocks must have literal paths and contents that can be verified without evaluating inputs or commands."}, + File: path, Key: "generate", Safe: true, Err: ErrUnsupportedExecution, + } +} + +// checkBackendGeneration inspects files read by the partial backend parse, including +// parents. Unknown generation expressions fail closed without evaluating inputs, +// hooks or commands. Non-HCL auxiliary files are not configuration declarations. +func checkBackendGeneration(filesystem afero.Fs, paths []string) error { + for _, path := range paths { + // Upstream partial parsing does not merge automatic includes. They can add + // either generation or a different remote_state identity, so do not guess. + auto := filepath.Join(filepath.Dir(path), "terragrunt.autoinclude.hcl") + if _, err := filesystem.Stat(auto); err == nil { + return &model.Problem{ + Code: model.CodeUnsupportedSetting, Summary: "Cannot verify backend identity from an automatic include", + Detail: "Backend-only parsing does not merge terragrunt.autoinclude.hcl.", + Hints: []string{"Use an explicit include for backend configuration so state reads and execution resolve the same identity."}, + File: auto, Key: "include", Safe: true, Err: ErrUnsupportedExecution, + } + } else if !errors.Is(err, fs.ErrNotExist) { + return err + } + content, err := afero.ReadFile(filesystem, path) + if err != nil { + return err + } + parser := hclparse.NewParser() + var file *hcl.File + var diagnostics hcl.Diagnostics + if strings.HasSuffix(path, ".json") { + file, diagnostics = parser.ParseJSON(content, path) + } else { + file, diagnostics = parser.ParseHCL(content, path) + } + // The upstream parser already validates configuration files. + if diagnostics.HasErrors() { + continue + } + if err := checkBackendGenerationFile(file); err != nil { + return err + } + } + return nil +} + +func checkBackendGenerationFile(file *hcl.File) error { + content, _, diagnostics := file.Body.PartialContent(&hcl.BodySchema{ + Attributes: []hcl.AttributeSchema{{Name: "generate"}}, + Blocks: []hcl.BlockHeaderSchema{{Type: "generate", LabelNames: []string{"name"}}}, + }) + if diagnostics.HasErrors() { + return diagnostics + } + if attr, ok := content.Attributes["generate"]; ok { + return unresolvedGeneratedBackend(attr.Range.Filename) + } + for _, block := range content.Blocks { + if !staticallyNonBackendGeneration(block.Body) { + return unresolvedGeneratedBackend(block.DefRange.Filename) + } + } + return nil +} + +func staticallyNonBackendGeneration(body hcl.Body) bool { + content, _, diagnostics := body.PartialContent(&hcl.BodySchema{Attributes: []hcl.AttributeSchema{ + {Name: "disable"}, {Name: "path"}, {Name: "contents"}, + }}) + if diagnostics.HasErrors() { + return false + } + if disabled, known := literalGenerateDisabled(content.Attributes["disable"]); !known || disabled { + return known && disabled + } + path, ok := literalGenerateString(content.Attributes["path"]) + if !ok || !filepath.IsLocal(path) || path == "." { + return false + } + if !terraformConfigurationFile(path) { + return true + } + text, ok := literalGenerateString(content.Attributes["contents"]) + return ok && !declaresBackend(path, text) +} + +func literalGenerateString(attr *hcl.Attribute) (string, bool) { + if attr == nil { + return "", false + } + value, diagnostics := attr.Expr.Value(nil) + if diagnostics.HasErrors() || !value.IsKnown() || value.IsNull() || value.Type() != cty.String { + return "", false + } + return value.AsString(), true +} + +func literalGenerateDisabled(attr *hcl.Attribute) (bool, bool) { + if attr == nil { + return false, true + } + value, diags := attr.Expr.Value(nil) + if diags.HasErrors() || !value.IsKnown() || value.IsNull() || value.Type() != cty.Bool { + return false, false + } + return value.True(), true +} + +// terraformConfigurationFile identifies generated files Terraform can load. +func terraformConfigurationFile(path string) bool { + lower := strings.ToLower(path) + return strings.HasSuffix(lower, ".tf") || strings.HasSuffix(lower, ".tf.json") +} diff --git a/plugins/terragrunt/internal/engine/backend_test.go b/plugins/terragrunt/internal/engine/backend_test.go index e4fab0e4708..05189ce1e38 100644 --- a/plugins/terragrunt/internal/engine/backend_test.go +++ b/plugins/terragrunt/internal/engine/backend_test.go @@ -1,7 +1,10 @@ package engine import ( + "cmp" "errors" + "fmt" + "os" "path/filepath" "testing" @@ -62,6 +65,7 @@ func TestRemoteBackendGenerationRetainsPathAndPolicy(t *testing.T) { require.True(t, result.BackendGenerated) path := filepath.Join("custom", "backend.tf") require.Equal(t, "overwrite_terragrunt", result.GeneratedPolicies[path]) + require.True(t, result.SensitiveGenerated[path], "a generated backend can hold credentials") require.Contains(t, result.Generated[path], `backend "s3"`) require.Contains(t, result.Generated[path], `assume_role = {`) require.NotContains(t, result.Generated[path], "skip_bucket_versioning") @@ -75,22 +79,22 @@ func TestUnsupportedBackendSemanticsAndPaths(t *testing.T) { backend = "s3" disable_init = true config = { bucket = "existing" } -}`, "disable_init or encryption is not implemented"}, +}`, "disable_init"}, {"encryption", `remote_state { backend = "s3" encryption = { key_provider = "pbkdf2", passphrase = "0123456789abcdef" } config = { bucket = "existing" } -}`, "disable_init or encryption is not implemented"}, +}`, "disable_init"}, {"backend path escapes", `remote_state { backend = "local" generate = { path = "../escape.tf", if_exists = "overwrite" } config = { path = "state" } -}`, "unsupported backend generated path: ../escape.tf"}, +}`, "escapes the working directory"}, {"backend path is the directory", `remote_state { backend = "local" generate = { path = ".", if_exists = "overwrite" } config = { path = "state" } -}`, "unsupported backend generated path: ."}, +}`, "escapes the working directory"}, {"backend path duplicates generate block", `remote_state { backend = "local" generate = { path = "same.tf", if_exists = "overwrite" } @@ -100,7 +104,7 @@ generate "duplicate" { path = "same.tf" if_exists = "overwrite" contents = "" -}`, "duplicate backend generated path: same.tf"}, +}`, "write the same file"}, {"backend path duplicates a normalized generate path", `remote_state { backend = "local" generate = { path = "./same.tf", if_exists = "overwrite" } @@ -110,7 +114,7 @@ generate "duplicate" { path = "same.tf" if_exists = "overwrite" contents = "" -}`, "duplicate backend generated path: ./same.tf"}, +}`, "write the same file"}, } { t.Run(test.name, func(t *testing.T) { result, err := Resolve(t.Context(), nativeUnit(t, test.hcl)) @@ -120,3 +124,115 @@ generate "duplicate" { }) } } + +func TestGeneratedBackendIdentityRejectedAcrossPhases(t *testing.T) { + for _, phase := range []string{"", "backend"} { + for backendIndex, backend := range []string{ + "terraform {\n backend \"local\" { path = \"other.tfstate\" }\n}", + "terraform {\n backend \"s3\" {\n bucket = \"other-bucket\"\n key = \"state\"\n region = \"us-east-1\"\n }\n}", + "terraform {\n cloud { organization = \"example\" }\n}", + } { + for remoteIndex, remote := range []string{"", "remote_state {\n backend = \"local\"\n config = { path = \"known.tfstate\" }\n}\n"} { + // The subtest name becomes part of its temp directory, so it must stay free of the + // HCL text and its glob metacharacters. + t.Run(fmt.Sprintf("phase-%s-backend-%d-remote-%d", cmp.Or(phase, "default"), backendIndex, remoteIndex), func(t *testing.T) { + request := nativeUnit(t, remote+fmt.Sprintf("generate \"backend\" {\n path = \"custom.tf\"\n if_exists = \"skip\"\n contents = %q\n}", backend)) + request.Phase = phase + owned := filepath.Join(filepath.Dir(request.Path), "custom.tf") + require.NoError(t, os.WriteFile(owned, []byte("user-owned"), 0o600)) + result, err := Resolve(t.Context(), request) + require.ErrorIs(t, err, ErrUnsupportedExecution) + require.ErrorContains(t, err, "Cannot verify the backend identity") + require.Nil(t, result) + bytes, err := os.ReadFile(owned) + require.NoError(t, err) + require.Equal(t, "user-owned", string(bytes)) + require.NoDirExists(t, filepath.Join(filepath.Dir(request.Path), ".terragrunt-cache")) + }) + } + } + } +} + +func TestBackendGenerationPartialParseFailsClosed(t *testing.T) { + for name, extra := range map[string]string{ + "json": "path = \"backend.tf.json\"\n contents = \"{\\\"terraform\\\":{\\\"backend\\\":{\\\"s3\\\":{}}}}\"", + "dynamic": "path = \"backend.tf\"\n contents = run_cmd(\"must-not-execute\")", + "path": "path = run_cmd(\"must-not-execute\")\n contents = \"# harmless\"", + "disable": "path = \"backend.tf\"\n disable = run_cmd(\"must-not-execute\")\n contents = \"# harmless\"", + } { + t.Run(name, func(t *testing.T) { + request := nativeUnit(t, "generate \"x\" {\n if_exists = \"overwrite\"\n"+extra+"\n}\ninputs = { value = run_cmd(\"must-not-evaluate-inputs\") }") + request.Phase = "backend" + result, err := Resolve(t.Context(), request) + require.ErrorContains(t, err, "Cannot verify the backend identity") + require.NotContains(t, err.Error(), "must-not") + require.Nil(t, result) + }) + } +} + +func TestBackendGenerationPartialParseInspectsIncludes(t *testing.T) { + request := nativeUnit(t, "include \"parent\" { path = \"parent.hcl\" }") + request.Phase = "backend" + require.NoError(t, os.WriteFile(filepath.Join(filepath.Dir(request.Path), "parent.hcl"), []byte("generate \"backend\" {\n path = \"backend.tf\"\n if_exists = \"overwrite\"\n contents = \"terraform {\\n backend \\\"local\\\" {}\\n}\"\n}"), 0o600)) + result, err := Resolve(t.Context(), request) + require.ErrorContains(t, err, "Cannot verify the backend identity") + require.Nil(t, result) +} + +func TestBackendGenerationPartialParsePreservesResolvedIdentity(t *testing.T) { + for _, backend := range []string{"local", "s3"} { + request := nativeUnit(t, fmt.Sprintf(`remote_state { + backend = %q + generate = { path = "backend.tf", if_exists = "overwrite_terragrunt" } + config = { bucket = "known", key = "state", region = "us-east-1" } +} +generate "provider" { + path = "provider.tf" + if_exists = "overwrite" + contents = "provider \"aws\" {}" +} +generate "disabled" { + path = "old.tf" + if_exists = "overwrite" + disable = true + contents = run_cmd("must-not-evaluate-disabled") +} +inputs = { value = run_cmd("must-not-evaluate-inputs") } +`, backend)) + request.Phase = "backend" + result, err := Resolve(t.Context(), request) + require.NoError(t, err) + require.Equal(t, backend, result.BackendType) + require.True(t, result.BackendGenerated) + require.Contains(t, result.Generated["backend.tf"], "backend \""+backend+"\"") + } +} + +func TestBackendGenerationPartialParseRejectsAttributeForm(t *testing.T) { + request := nativeUnit(t, `generate = { + backend = { + path = "backend.tf" + if_exists = "overwrite" + contents = "terraform {\n backend \"local\" {}\n}" + } +}`) + request.Phase = "backend" + result, err := Resolve(t.Context(), request) + require.ErrorContains(t, err, "Cannot verify the backend identity") + require.Nil(t, result) +} + +func TestBackendGenerationPartialParseInspectsAutoIncludes(t *testing.T) { + request := nativeUnit(t, "") + request.Phase = "backend" + require.NoError(t, os.WriteFile(filepath.Join(filepath.Dir(request.Path), "terragrunt.autoinclude.hcl"), []byte(`generate "backend" { + path = "backend.tf" + if_exists = "overwrite" + contents = "terraform {\n backend \"local\" {}\n}" +}`), 0o600)) + result, err := Resolve(t.Context(), request) + require.ErrorContains(t, err, "Cannot verify backend identity from an automatic include") + require.Nil(t, result) +} diff --git a/plugins/terragrunt/internal/engine/discovery.go b/plugins/terragrunt/internal/engine/discovery.go index d28fe770e71..9fe16311fdd 100644 --- a/plugins/terragrunt/internal/engine/discovery.go +++ b/plugins/terragrunt/internal/engine/discovery.go @@ -18,43 +18,6 @@ import ( "github.com/cloudposse/atmos/plugins/terragrunt/internal/model" ) -var ( - errDiscoveryDepth = errors.New("nested-stack discovery depth exceeded at") - errInlineModuleSource = errors.New("inline Terraform modules require terraform.source") - errIdentityCollision = errors.New("discovered instance collision") - errGenerationDepth = errors.New("nested-stack generation depth exceeded at") - errExpansionKey = errors.New("unsupported") - errDuplicateUnit = errors.New("duplicate unit instance") - errDuplicateStack = errors.New("duplicate stack instance") - errDependencyWorkspace = errors.New("native output dependencies with non-default workspaces require a dependency backend adapter") -) - -// DiscoveryError carries only a filesystem identity and a fixed category across -// the RPC boundary. Cause remains local for debugging and cancellation checks. -type DiscoveryError struct { - File string - Category string - Cause error -} - -//nolint:lintroller // Pure error delegation in a standalone plugin; no host tracker dependency. -func (e *DiscoveryError) Error() string { return e.Cause.Error() } - -//nolint:lintroller // Preserve the original cause without introducing the host tracker dependency. -func (e *DiscoveryError) Unwrap() error { return e.Cause } - -func discoveryError(root, path, category string, cause error) error { - var existing *DiscoveryError - if errors.As(cause, &existing) { - return cause - } - relative, err := filepath.Rel(root, path) - if err != nil || !filepath.IsLocal(relative) { - return cause - } - return &DiscoveryError{File: filepath.ToSlash(relative), Category: category, Cause: cause} -} - // canonicalPath preserves physical ancestor identity for files generated only in // the discovery overlay. Actual existence is checked through the active FS. func canonicalPath(path string) (string, error) { @@ -76,11 +39,25 @@ func canonicalPath(path string) (string, error) { return filepath.Join(resolved, filepath.Base(path)), nil } +// logicalPath resolves symlinks in a configuration file's directory but not in the file +// itself. Like upstream Terragrunt, a symlinked terragrunt.hcl is a unit at the place the +// link lives, and the configuration is read through the link. +func logicalPath(path string) (string, error) { + dir, err := canonicalPath(filepath.Dir(path)) + if err != nil { + return "", err + } + return filepath.Join(dir, filepath.Base(path)), nil +} + func within(path, directory string) bool { rel, err := filepath.Rel(directory, path) return err == nil && rel != ".." && !strings.HasPrefix(rel, ".."+string(filepath.Separator)) } +// discoveryResolver evaluates one configuration file in the discovery overlay. +type discoveryResolver func(context.Context, *model.Request, afero.Fs) (*model.Resolution, error) + // projectDiscovery holds one inventory pass. Its overlay is invocation-local: // generated HCL retains its original virtual path and no source files or state // are moved or written. @@ -88,45 +65,66 @@ type projectDiscovery struct { ctx context.Context request *model.Request root string + evaluate discoveryResolver overlay afero.Fs result *model.Resolution templates map[string]bool // generated records units reached through stack traversal so the standalone // pass does not discover the same generated unit again. generated map[string]bool + // childFailures holds generated children that failed, with their generation root, so the + // failures of stacks that turn out to be templates can be dropped. + childFailures []childFailure +} + +// childFailure is a generated child recorded at its own address so its siblings stay usable. +type childFailure struct { + generationRoot string + failure model.UnitFailure } // discoverProject discovers live stacks first, then standalone units that no // stack owns, and rejects colliding Atmos identities. func discoverProject(ctx context.Context, request *model.Request) (*model.Resolution, error) { + return discoverProjectWithResolver(ctx, request, resolve) +} + +func discoverProjectWithResolver(ctx context.Context, request *model.Request, evaluate discoveryResolver) (*model.Resolution, error) { + if err := ctx.Err(); err != nil { + return nil, err + } root, err := canonicalPath(request.BasePath) if err != nil { return nil, err } - stacks, units, err := discoveryPaths(root, request.IncludedPaths, request.ExcludedPaths) + scan, err := discoveryPaths(root, request.IncludedPaths, request.ExcludedPaths) if err != nil { return nil, err } d := &projectDiscovery{ - ctx: ctx, request: request, root: root, + ctx: ctx, request: request, root: root, evaluate: evaluate, overlay: afero.NewCopyOnWriteFs(afero.NewReadOnlyFs(afero.NewOsFs()), afero.NewMemMapFs()), - result: &model.Resolution{}, + result: &model.Resolution{Warnings: scan.Warnings}, templates: map[string]bool{}, generated: map[string]bool{}, } - if err := d.discoverStacks(stacks); err != nil { + if err := d.discoverStacks(scan.Stacks); err != nil { return nil, err } - if err := d.discoverStandaloneUnits(units); err != nil { + if err := d.discoverStandaloneUnits(scan.Units); err != nil { return nil, err } slices.SortFunc(d.result.Inventory, func(a, b model.DiscoveredUnit) int { return strings.Compare(a.Stack+"/"+a.Name, b.Stack+"/"+b.Name) }) - if err := checkIdentityCollisions(d.result.Inventory); err != nil { + if err := checkIdentityCollisions(d.root, d.result.Inventory); err != nil { return nil, err } return d.result, nil } +func isDiscoveryCancellation(err error) bool { + return errors.Is(err, context.Canceled) || errors.Is(err, context.DeadlineExceeded) +} + // skipped reports whether path lies inside a catalog template directory. func (d *projectDiscovery) skipped(path string) bool { for dir := range d.templates { @@ -151,11 +149,14 @@ func (d *projectDiscovery) parse(path string, generate bool) (*model.Resolution, if generate { req.Command = "generate" } - resolved, err := resolve(d.ctx, &req, d.overlay) - if err != nil { - return nil, discoveryError(d.root, path, "EVALUATION_FAILED", err) + if err := d.ctx.Err(); err != nil { + return nil, err } - return resolved, nil + resolved, err := d.evaluate(d.ctx, &req, d.overlay) + if contextErr := d.ctx.Err(); contextErr != nil { + return nil, contextErr + } + return resolved, err } // discoverStacks visits live stacks. Catalog stack templates may require values @@ -167,21 +168,26 @@ func (d *projectDiscovery) discoverStacks(stacks []string) error { if d.skipped(path) { continue } - if _, err := d.parse(path, true); err != nil { - failures[path] = err - continue - } - if err := d.visit(path, &unitPlacement{stack: d.nameFor(filepath.Dir(path)), generationRoot: path}, 0); err != nil { + if err := d.discoverStack(path); err != nil { + if isDiscoveryCancellation(err) { + return err + } failures[path] = err } } for _, path := range stacks { if err := failures[path]; err != nil && !d.skipped(path) { - return fmt.Errorf("discover %s: %w", path, err) + // The components of a stack that cannot be generated are unknown, so the stack file itself is the unit. + d.result.Failures = append(d.result.Failures, model.UnitFailure{Stack: d.nameFor(filepath.Dir(path)), Name: filepath.Base(path), Err: problemFor(err, path)}) } } // A template that parsed independently is still a template, not another live stack. d.result.Inventory = slices.DeleteFunc(d.result.Inventory, func(unit model.DiscoveredUnit) bool { return d.skipped(unit.GenerationRoot) }) + for _, child := range d.childFailures { + if !d.skipped(child.generationRoot) { + d.result.Failures = append(d.result.Failures, child.failure) + } + } return nil } @@ -198,8 +204,12 @@ func (d *projectDiscovery) discoverStandaloneUnits(units []string) error { if dir == d.root { stack, name = "default", "root" } + // One failing unit is recorded and reported on its own; the other units stay usable. if err := d.visit(path, &unitPlacement{stack: stack, name: name}, 0); err != nil { - return fmt.Errorf("discover %s: %w", path, err) + if isDiscoveryCancellation(err) { + return err + } + d.result.Failures = append(d.result.Failures, model.UnitFailure{Stack: stack, Name: filepath.ToSlash(name), Err: problemFor(err, path)}) } } return nil @@ -220,7 +230,7 @@ func (p *unitPlacement) child(name, source string, files []string) *unitPlacemen func (d *projectDiscovery) visit(path string, placement *unitPlacement, depth int) error { if depth > maxStackDepth { - return fmt.Errorf("%w %s", errDiscoveryDepth, path) + return &model.Problem{Code: model.CodeEvaluationError, Summary: "Terragrunt stacks are nested too deeply", Detail: fmt.Sprintf("Nested `terragrunt.stack.hcl` generation exceeded %d levels, which usually means a stack includes itself.", maxStackDepth), Hints: []string{"Remove the circular `stack` reference."}, File: path, Safe: true} } resolved, err := d.parse(path, false) if err != nil { @@ -237,7 +247,7 @@ func (d *projectDiscovery) visitStack(resolved *model.Resolution, placement *uni for _, child := range slices.Sorted(maps.Keys(resolved.StackConfigs)) { source := resolved.StackSources[child] d.templates[source] = true - if err := d.visit(resolved.StackConfigs[child], placement.child(resolved.Stacks[child], source, files), depth+1); err != nil { + if err := d.visitChild(resolved.StackConfigs[child], placement.child(resolved.Stacks[child], source, files), depth); err != nil { return err } } @@ -247,16 +257,26 @@ func (d *projectDiscovery) visitStack(resolved *model.Resolution, placement *uni if physical, err := canonicalPath(resolved.UnitConfigs[child]); err == nil { d.generated[physical] = true } - if err := d.visit(resolved.UnitConfigs[child], placement.child(child, source, files), depth+1); err != nil { + if err := d.visitChild(resolved.UnitConfigs[child], placement.child(child, source, files), depth); err != nil { return err } } return nil } +// visitChild visits a generated child. Only cancellation stops the stack; any other failure is +// recorded at the child's own address so its siblings stay usable. +func (d *projectDiscovery) visitChild(path string, placement *unitPlacement, depth int) error { + err := d.visit(path, placement, depth+1) + if err == nil || isDiscoveryCancellation(err) { + return err + } + d.childFailures = append(d.childFailures, childFailure{placement.generationRoot, model.UnitFailure{Stack: placement.stack, Name: filepath.ToSlash(placement.name), Err: problemFor(err, path)}}) + return nil +} + func (d *projectDiscovery) visitUnit(path string, resolved *model.Resolution, placement *unitPlacement) error { if resolved.Source == "" && len(resolved.Folders) == 0 { - // Shared include, not a runnable unit. return d.checkInlineModule(path) } resolved.Files = append(resolved.Files, placement.files...) @@ -268,6 +288,8 @@ func (d *projectDiscovery) visitUnit(path string, resolved *model.Resolution, pl return nil } +// checkInlineModule rejects Terraform files without a source and reports a shared include, +// which is not a runnable local-source unit and so is not listed. func (d *projectDiscovery) checkInlineModule(path string) error { entries, err := afero.ReadDir(d.overlay, filepath.Dir(path)) if err != nil { @@ -275,19 +297,27 @@ func (d *projectDiscovery) checkInlineModule(path string) error { } for _, entry := range entries { if !entry.IsDir() && (strings.HasSuffix(entry.Name(), ".tf") || strings.HasSuffix(entry.Name(), ".tf.json")) { - return discoveryError(d.root, path, "MISSING_TERRAFORM_SOURCE", errInlineModuleSource) + return missingSourceProblem(path) } } + rel, _ := filepath.Rel(d.root, path) + d.result.Warnings = append(d.result.Warnings, model.Warning{ + Code: model.WarningUnitWithoutSource, Path: filepath.ToSlash(rel), + Message: fmt.Sprintf("%s has no terraform { source } block and no Terraform files next to it, so it is treated as a shared configuration and is not listed as a component", filepath.ToSlash(rel)), + }) return nil } -func checkIdentityCollisions(inventory []model.DiscoveredUnit) error { +func checkIdentityCollisions(root string, inventory []model.DiscoveredUnit) error { identities := map[string]string{} for i := range inventory { unit := &inventory[i] key := unit.Stack + "\x00" + unit.Name if previous, exists := identities[key]; exists { - return fmt.Errorf("%w %s/%s: %s and %s", errIdentityCollision, unit.Stack, unit.Name, previous, unit.Config.Path) + return &model.Problem{ + Code: model.CodeEvaluationError, Summary: "Two Terragrunt units resolve to the same component", Detail: fmt.Sprintf("%s/%s is produced by both %s and %s.", unit.Stack, unit.Name, filepath.ToSlash(mustRel(root, previous)), filepath.ToSlash(mustRel(root, unit.Config.Path))), Safe: true, + Hints: []string{"Remove one of the units, a symlink that repeats a unit, or an overlapping `included_paths` entry."}, + } } identities[key] = unit.Config.Path } @@ -322,7 +352,7 @@ func readStack(ctx context.Context, logger log.Logger, environment *atmosbridge. // it declares. Units are generated by their declaring stack file. func (e *unitEvaluator) generateStack(ctx context.Context, path string, depth int) error { if depth > maxStackDepth { - return fmt.Errorf("%w %s", errGenerationDepth, path) + return fmt.Errorf("%w at %s", errStackDepth, path) } if err := atmosbridge.GenerateStack(ctx, e.logger, e.environment, path, e.request.Tofu); err != nil { return err @@ -371,7 +401,7 @@ func stackResolution(pctx *config.ParsingContext, stack *config.StackConfig, pat return nil, err } if _, exists := result.Units[id]; exists { - return nil, fmt.Errorf("%w %s in %s", errDuplicateUnit, id, path) + return nil, fmt.Errorf("%w unit instance %s in %s", errDuplicateInstance, id, path) } result.Units[id] = unit.Path result.UnitSources[id] = localPath(dir, unit.Source) @@ -387,7 +417,7 @@ func stackResolution(pctx *config.ParsingContext, stack *config.StackConfig, pat return nil, err } if _, exists := result.Stacks[id]; exists { - return nil, fmt.Errorf("%w %s in %s", errDuplicateStack, id, path) + return nil, fmt.Errorf("%w stack instance %s in %s", errDuplicateInstance, id, path) } result.Stacks[id] = child.Path result.StackSources[id] = localPath(dir, child.Source) @@ -396,34 +426,71 @@ func stackResolution(pctx *config.ParsingContext, stack *config.StackConfig, pat return result, nil } +// inventorySource records the module a unit names. A local module must exist, so a bulk run stops +// before any unit runs, and a remote source must use a supported transport. +func inventorySource(environment *atmosbridge.Venv, cfg *config.TerragruntConfig, path string, result *model.Resolution) error { + if cfg.Terraform == nil || cfg.Terraform.Source == nil { + return nil + } + result.Source = *cfg.Terraform.Source + if !plainLocalSource(result.Source) { + // A local `dir//subdir` source must exist, like a plain local source. + if localSubdirSource(result.Source) { + if problem := localSubdirSourceProblem(environment.FS, path, result.Source); problem != nil { + return problem + } + } + // An unsupported remote transport fails discovery too, before any unit runs. + return validateRemoteSource(result.Source) + } + result.Source = localPath(filepath.Dir(path), result.Source) + // The active filesystem also sees modules that a generated stack unit names in the overlay. + if problem := localSourceProblem(environment.FS, path, *cfg.Terraform.Source, result.Source); problem != nil { + return problem + } + result.Folders = []string{result.Source} + return nil +} + // resolveUnitInventory reads only the source and dependency blocks of a unit. func (e *unitEvaluator) resolveUnitInventory(ctx context.Context) (*model.Resolution, error) { cfg, err := config.PartialParseConfigFile(ctx, e.logger, e.environment, - e.pctx.WithDecodeList(config.DependenciesBlock, config.DependencyBlock, config.TerraformSource), e.path, nil) + e.pctx.WithDecodeList(config.DependenciesBlock, config.DependencyBlock, config.TerraformSource, config.TerragruntFlags, config.TerragruntVersionConstraints, config.FeatureFlagsBlock, config.ExcludeBlock, config.ErrorsBlock), e.path, nil) if err != nil { return nil, err } + // Reject unsupported settings now, so every selected unit is validated before any + // infrastructure command runs. + if err := rejectUnsupported(cfg, e.path); err != nil { + return nil, err + } + if err := checkExecutionSyntax(e.environment.FS, cfg, e.path); err != nil { + return nil, err + } result := &model.Resolution{Path: e.path, Files: append(e.pctx.FilesRead.Paths(), e.path)} - if unsupportedDependencyWorkspace(e.request, cfg) { - return nil, fmt.Errorf("%w: %s", errDependencyWorkspace, e.path) + if err := inventoryWorkspaceProblem(e.request, cfg, e.path); err != nil { + return nil, err } - if cfg.Terraform != nil && cfg.Terraform.Source != nil { - result.Source = *cfg.Terraform.Source - if plainLocalSource(result.Source) { - result.Source = localPath(filepath.Dir(e.path), result.Source) - result.Folders = []string{result.Source} - } + if err := inventorySource(e.environment, cfg, e.path, result); err != nil { + return nil, err } - if result.OutputDependencies, err = outputDependencies(cfg, e.path); err != nil { + if result.OutputDependencies, err = outputDependencies(e.environment, cfg, e.path); err != nil { return nil, err } - if result.Dependencies, err = orderDependencies(e.environment, cfg, e.path); err != nil { + // Output reads also require a scheduling edge before preparation. + order, err := orderDependencies(e.environment, cfg, e.path) + if err != nil { return nil, err } + for _, dependency := range append(slices.Clone(result.OutputDependencies), order...) { + if !slices.Contains(result.Dependencies, dependency) { + result.Dependencies = append(result.Dependencies, dependency) + } + } return result, nil } -func outputDependencies(cfg *config.TerragruntConfig, path string) ([]string, error) { +func outputDependencies(environment *atmosbridge.Venv, cfg *config.TerragruntConfig, path string) ([]string, error) { var result []string for _, dep := range cfg.TerragruntDependencies { if (dep.Enabled != nil && !*dep.Enabled) || (dep.SkipOutputs != nil && *dep.SkipOutputs) { @@ -432,15 +499,13 @@ func outputDependencies(cfg *config.TerragruntConfig, path string) ([]string, er if !dep.ConfigPath.IsKnown() || dep.ConfigPath.IsNull() { continue } - dependency := localPath(filepath.Dir(path), dep.ConfigPath.AsString()) - if filepath.Ext(dependency) != ".hcl" { - dependency = filepath.Join(dependency, unitFile) - } - physical, err := canonicalPath(dependency) + physical, err := outputDependencyPath(environment, path, dep.ConfigPath.AsString()) if err != nil { return nil, err } - result = append(result, physical) + if !slices.Contains(result, physical) { + result = append(result, physical) + } } return result, nil } @@ -456,16 +521,54 @@ func orderDependencies(environment *atmosbridge.Venv, cfg *config.TerragruntConf } info, err := environment.FS.Stat(dependency) if err != nil { - return nil, err + return nil, dependencyProblem(path, dependency, err) } if info.IsDir() { dependency = filepath.Join(dependency, unitFile) } - physical, err := canonicalPath(dependency) + physical, err := logicalPath(dependency) if err != nil { - return nil, fmt.Errorf("native adapter dependency must target a unit: %w", err) + return nil, dependencyProblem(path, dependency, err) + } + if !slices.Contains(result, physical) { + result = append(result, physical) } - result = append(result, physical) } return result, nil } + +func (d *projectDiscovery) discoverStack(path string) error { + if _, err := d.parse(path, true); err != nil { + return err + } + return d.visit(path, &unitPlacement{stack: d.nameFor(filepath.Dir(path)), generationRoot: path}, 0) +} + +var ( + errStackDepth = errors.New("nested-stack generation depth exceeded") + errExpansionKey = errors.New("unsupported") + errDuplicateInstance = errors.New("duplicate") +) + +func outputDependencyPath(environment *atmosbridge.Venv, path, configured string) (string, error) { + dependency := localPath(filepath.Dir(path), configured) + if filepath.Ext(dependency) != ".hcl" { + dependency = filepath.Join(dependency, unitFile) + } + physical, err := logicalPath(dependency) + if err != nil { + return "", dependencyProblem(path, dependency, err) + } + if _, err := environment.FS.Stat(physical); err != nil { + return "", dependencyProblem(path, dependency, err) + } + return physical, nil +} + +func inventoryWorkspaceProblem(request *model.Request, cfg *config.TerragruntConfig, path string) error { + if unsupportedDependencyWorkspace(request, cfg) { + workspace := request.Environment["TF_WORKSPACE"] + return workspaceDependencyProblem(path, workspace) + } + return nil +} diff --git a/plugins/terragrunt/internal/engine/discovery_paths.go b/plugins/terragrunt/internal/engine/discovery_paths.go index 67c5c4dda90..80eaf3ab72a 100644 --- a/plugins/terragrunt/internal/engine/discovery_paths.go +++ b/plugins/terragrunt/internal/engine/discovery_paths.go @@ -10,54 +10,71 @@ import ( "strings" "github.com/bmatcuk/doublestar/v4" -) -var ( - errDiscoveryPattern = errors.New("invalid discovery path pattern") - errDiscoveryTraversal = errors.New("discovery paths must stay within the Atmos base path") - errDiscoveryRoot = errors.New("discovery root is outside the Atmos base path or unavailable") + "github.com/cloudposse/atmos/plugins/terragrunt/internal/model" ) -// discoveryPaths walks only the literal prefixes of included patterns and prunes -// excluded directories. Paths select entry points; HCL references are resolved later. -func discoveryPaths(root string, includes, excludes []string) ([]string, []string, error) { - if len(includes) == 0 { - includes = []string{"**"} +// pathScan is what a walk of the discovery scope found. Stacks and Units are absolute +// paths of terragrunt.stack.hcl and terragrunt.hcl files. +type pathScan struct { + Stacks, Units []string + Warnings []model.Warning +} + +// normalizePattern rewrites a configured pattern into one canonical, slash-separated form +// so included_paths and excluded_paths treat `live`, `live/`, `./live` and `live/**` alike. +func normalizePattern(pattern string) string { + for strings.HasPrefix(pattern, "./") { + pattern = strings.TrimPrefix(pattern, "./") } - if err := validateDiscoveryPatterns(append(slices.Clone(includes), excludes...)); err != nil { - return nil, nil, err + pattern = strings.TrimRight(pattern, "/") + if pattern == "" || pattern == "." { + return "**" } - walk := discoveryPathWalk{root: root, includes: includes, excludes: excludes, seen: map[string]bool{}} - for _, pattern := range includes { - start, err := discoveryStart(root, pattern) - if err != nil { - return nil, nil, err - } - walk.start = start - if err := filepath.WalkDir(start, walk.visit); err != nil { - return nil, nil, err - } + return pattern +} + +// selects reports whether a pattern selects a project-relative path. A pattern selects +// what it matches and everything below it, so a directory name needs no `/**` suffix. +func selects(pattern, rel string) bool { + if ok, _ := doublestar.Match(pattern, rel); ok { + return true } - slices.Sort(walk.stacks) - slices.Sort(walk.units) - return walk.stacks, walk.units, nil + ok, _ := doublestar.Match(pattern+"/**", rel) + return ok } -func validateDiscoveryPatterns(patterns []string) error { - for _, pattern := range patterns { - if pattern == "" || filepath.IsAbs(pattern) || strings.Contains(pattern, "\\") || !doublestar.ValidatePattern(pattern) { - return fmt.Errorf("%w: %q", errDiscoveryPattern, pattern) - } - for _, part := range strings.Split(pattern, "/") { - if part == ".." { - return fmt.Errorf("%w: %q", errDiscoveryTraversal, pattern) +func selectsAny(patterns []string, rel string) bool { + return slices.ContainsFunc(patterns, func(pattern string) bool { return selects(pattern, rel) }) +} + +// validatePatterns normalizes and checks both lists, naming the offending key and pattern. +func validatePatterns(includes, excludes []string) ([]string, []string, error) { + normalize := func(key string, patterns []string) ([]string, error) { + result := make([]string, 0, len(patterns)) + for _, pattern := range patterns { + if pattern == "" || filepath.IsAbs(pattern) || strings.Contains(pattern, "\\") || !doublestar.ValidatePattern(pattern) { + return nil, fmt.Errorf("%w: %s pattern %q", errInvalidDiscoveryPattern, key, pattern) } + for _, part := range strings.Split(pattern, "/") { + if part == ".." { + return nil, fmt.Errorf("%w: %s patterns must stay within the Atmos base path: %q", errInvalidDiscoveryPattern, key, pattern) + } + } + result = append(result, normalizePattern(pattern)) } + return result, nil } - return nil + includes, err := normalize("included_paths", includes) + if err != nil { + return nil, nil, err + } + excludes, err = normalize("excluded_paths", excludes) + return includes, excludes, err } -func discoveryStart(root, pattern string) (string, error) { +// walkStart is the directory or file a pattern's walk begins at: its literal prefix. +func walkStart(root, pattern string) string { prefix := pattern if index := strings.IndexAny(pattern, "*?[{"); index >= 0 { prefix = pattern[:index] @@ -65,70 +82,139 @@ func discoveryStart(root, pattern string) (string, error) { prefix = filepath.Dir(prefix) } } - start := filepath.Join(root, filepath.FromSlash(prefix)) - // Do not follow a configured root symlink outside the project. - resolved, err := canonicalPath(start) - if err != nil || !within(resolved, root) { - return "", fmt.Errorf("%w: %s", errDiscoveryRoot, start) + return filepath.Join(root, filepath.FromSlash(prefix)) +} + +// discoveryPaths walks only the literal prefixes of included patterns and prunes +// excluded directories. Paths select entry points; HCL references are resolved later. +// +// Like upstream Terragrunt, directory symlinks are not followed and a symlinked +// terragrunt.hcl is a unit at the place the link lives. A skipped link is reported. +func discoveryPaths(root string, includes, excludes []string) (pathScan, error) { + if len(includes) == 0 { + includes = []string{"**"} + } + includes, excludes, err := validatePatterns(includes, excludes) + if err != nil { + return pathScan{}, err + } + walker := &discoveryWalker{root: root, includes: includes, excludes: excludes, seen: map[string]bool{}, matched: map[string]int{}} + for _, pattern := range includes { + if err := walker.walk(pattern); err != nil { + return pathScan{}, err + } } - return start, nil + scan := walker.scan + matched := walker.matched + slices.Sort(scan.Stacks) + slices.Sort(scan.Units) + scan.Warnings = append(scan.Warnings, unmatchedWarnings(includes, matched)...) + slices.SortFunc(scan.Warnings, func(a, b model.Warning) int { return strings.Compare(a.Code+a.Path+a.Message, b.Code+b.Path+b.Message) }) + scan.Warnings = slices.Compact(scan.Warnings) + return scan, nil +} + +// isTerragruntFile reports whether a file name is a Terragrunt unit or stack configuration. +func isTerragruntFile(name string) bool { + return name == "terragrunt.hcl" || name == "terragrunt.stack.hcl" } -func matchesDiscoveryPaths(patterns []string, path string) bool { - for _, pattern := range patterns { - if ok, _ := doublestar.Match(pattern, path); ok { - return true +// unmatchedWarnings names included_paths patterns that selected no Terragrunt file. +// Counting happens per configuration file, so a pattern that matches only directories, +// or a name that differs in case, is reported instead of silently producing no units. +func unmatchedWarnings(includes []string, matched map[string]int) []model.Warning { + var warnings []model.Warning + for _, pattern := range slices.Compact(slices.Sorted(slices.Values(includes))) { + if matched[pattern] == 0 { + warnings = append(warnings, model.Warning{ + Code: model.WarningPatternUnmatched, Path: pattern, + Message: fmt.Sprintf("included_paths pattern %q matched no terragrunt.hcl or terragrunt.stack.hcl (patterns are relative to the Atmos base path and select the path and everything below it)", pattern), + }) } } - return false + return warnings } -// discoveryPathWalk shares duplicate tracking across independently scoped roots. -type discoveryPathWalk struct { - root, start string +func symlinkWarning(rel string) model.Warning { + return model.Warning{ + Code: model.WarningSymlinkedDirectory, Path: rel, + Message: fmt.Sprintf("symlinked directory %s was not followed, so units below it are not discovered; like Terragrunt, Atmos follows only a symlinked terragrunt.hcl or terragrunt.stack.hcl file", rel), + } +} + +var ( + errInvalidDiscoveryPattern = errors.New("invalid discovery pattern") + errDiscoveryRoot = errors.New("discovery root is outside the Atmos base path or unavailable") +) + +type discoveryWalker struct { + root string includes, excludes []string seen map[string]bool - stacks, units []string + matched map[string]int + scan pathScan } -func (w *discoveryPathWalk) visit(path string, entry fs.DirEntry, walkErr error) error { - if os.IsNotExist(walkErr) && path == w.start { - return nil - } - if walkErr != nil { - return walkErr +func (w *discoveryWalker) walk(pattern string) error { + start := walkStart(w.root, pattern) + resolved, err := canonicalPath(start) + if err != nil || !within(resolved, w.root) { + return fmt.Errorf("%w: %s", errDiscoveryRoot, start) } + return filepath.WalkDir(start, func(path string, entry fs.DirEntry, walkErr error) error { + if os.IsNotExist(walkErr) && path == start { + return nil + } + if walkErr != nil { + return walkErr + } + return w.visit(path, entry) + }) +} + +func (w *discoveryWalker) visit(path string, entry fs.DirEntry) error { rel, _ := filepath.Rel(w.root, path) rel = filepath.ToSlash(rel) - if matchesDiscoveryPaths(w.excludes, rel) { + if selectsAny(w.excludes, rel) { if entry.IsDir() { return filepath.SkipDir } return nil } - if entry.IsDir() { - return w.visitDirectory(path, entry) + if entry.Type()&fs.ModeSymlink != 0 { + if info, err := os.Stat(path); err == nil && info.IsDir() { + w.scan.Warnings = append(w.scan.Warnings, symlinkWarning(rel)) + return nil + } } - if w.seen[path] || !matchesDiscoveryPaths(w.includes, rel) { + if entry.IsDir() { + if path != w.root && (strings.HasPrefix(entry.Name(), ".") || entry.Name() == "node_modules") { + return filepath.SkipDir + } return nil } - w.record(path, entry.Name()) + w.record(path, rel, entry.Name()) return nil } -func (w *discoveryPathWalk) record(path, name string) { +func (w *discoveryWalker) record(path, rel, name string) { + // Only configuration files count toward matched-pattern warnings. + if !isTerragruntFile(name) || !selectsAny(w.includes, rel) { + return + } + for _, candidate := range w.includes { + if selects(candidate, rel) { + w.matched[candidate]++ + } + } + if w.seen[path] { + return + } w.seen[path] = true switch name { - case stackFile: - w.stacks = append(w.stacks, path) - case unitFile: - w.units = append(w.units, path) + case "terragrunt.stack.hcl": + w.scan.Stacks = append(w.scan.Stacks, path) + case "terragrunt.hcl": + w.scan.Units = append(w.scan.Units, path) } } - -func (w *discoveryPathWalk) visitDirectory(path string, entry fs.DirEntry) error { - if path != w.root && (strings.HasPrefix(entry.Name(), ".") || entry.Name() == "node_modules") { - return filepath.SkipDir - } - return nil -} diff --git a/plugins/terragrunt/internal/engine/discovery_paths_test.go b/plugins/terragrunt/internal/engine/discovery_paths_test.go index c71a5bf4f63..652d31ea08f 100644 --- a/plugins/terragrunt/internal/engine/discovery_paths_test.go +++ b/plugins/terragrunt/internal/engine/discovery_paths_test.go @@ -1,11 +1,16 @@ package engine import ( + "context" "encoding/json" + "errors" + "fmt" "os" "path/filepath" + "slices" "testing" + "github.com/spf13/afero" "github.com/stretchr/testify/require" "github.com/cloudposse/atmos/plugins/terragrunt/internal/model" @@ -19,19 +24,23 @@ func TestDiscoveryScopeAndPruning(t *testing.T) { require.NoError(t, os.MkdirAll(filepath.Dir(path), 0o755)) require.NoError(t, os.WriteFile(path, nil, 0o644)) } - stacks, units, err := discoveryPaths(root, []string{"live/**", "live/unit/**"}, []string{"live/excluded/**"}) + scan, err := discoveryPaths(root, []string{"live/**", "live/unit/**"}, []string{"live/excluded/**"}) require.NoError(t, err) - require.Equal(t, []string{filepath.Join(root, "live", "terragrunt.stack.hcl")}, stacks) - require.Equal(t, []string{filepath.Join(root, "live", "unit", "terragrunt.hcl")}, units) - _, _, err = discoveryPaths(root, []string{"absent/**"}, nil) + require.Equal(t, []string{filepath.Join(root, "live", "terragrunt.stack.hcl")}, scan.Stacks) + require.Equal(t, []string{filepath.Join(root, "live", "unit", "terragrunt.hcl")}, scan.Units) + scan, err = discoveryPaths(root, []string{"absent/**"}, nil) require.NoError(t, err) + require.Len(t, scan.Warnings, 1) + require.Equal(t, model.WarningPatternUnmatched, scan.Warnings[0].Code) for _, pattern := range []string{"", "[", "../**", `live\**`, filepath.Join(root, "**")} { - _, _, err := discoveryPaths(root, []string{pattern}, nil) + _, err := discoveryPaths(root, []string{pattern}, nil) require.Error(t, err) } - _, units, err = discoveryPaths(root, nil, nil) + _, err = discoveryPaths(root, nil, []string{"../x"}) + require.ErrorContains(t, err, "excluded_paths") + scan, err = discoveryPaths(root, nil, nil) require.NoError(t, err) - require.Contains(t, units, filepath.Join(root, "catalog", "terragrunt.hcl")) + require.Contains(t, scan.Units, filepath.Join(root, "catalog", "terragrunt.hcl")) } func TestDiscoveryRejectsOutsideSymlink(t *testing.T) { @@ -40,7 +49,7 @@ func TestDiscoveryRejectsOutsideSymlink(t *testing.T) { if err := os.Symlink(outside, filepath.Join(root, "external")); err != nil { t.Skipf("symlink unavailable: %v", err) } - _, _, err := discoveryPaths(root, []string{"external/**"}, nil) + _, err := discoveryPaths(root, []string{"external/**"}, nil) require.ErrorContains(t, err, "outside the Atmos base path") } @@ -52,9 +61,14 @@ func TestDiscoveryDistinguishesIncludesFromInlineModules(t *testing.T) { result, err := Resolve(t.Context(), request) require.NoError(t, err) require.Empty(t, result.Inventory) + require.Len(t, result.Warnings, 1, "a shared configuration without terraform.source is reported, not silently dropped") + require.Equal(t, model.WarningUnitWithoutSource, result.Warnings[0].Code) + require.Equal(t, "terragrunt.hcl", result.Warnings[0].Path) require.NoError(t, os.WriteFile(filepath.Join(root, "main.tf"), []byte("terraform {}"), 0o644)) - _, err = Resolve(t.Context(), request) - require.ErrorContains(t, err, "inline Terraform modules") + result, err = Resolve(t.Context(), request) + require.NoError(t, err) + require.Len(t, result.Failures, 1) + require.ErrorContains(t, result.Failures[0].Err, "terraform.source") } func TestDiscoveryReportsBrokenStacksAndMissingDependencies(t *testing.T) { @@ -80,10 +94,15 @@ func TestDiscoveryReportsBrokenStacksAndMissingDependencies(t *testing.T) { req.Environment["TF_WORKSPACE"] = "stage" } req.Phase = "inventory" - _, err := Resolve(t.Context(), req) - require.Error(t, err) + // A failing unit or stack is reported on its own instead of failing the project. + result, err := Resolve(t.Context(), req) + require.NoError(t, err) + require.Len(t, result.Failures, 1) + var problem *model.Problem + require.ErrorAs(t, result.Failures[0].Err, &problem) + require.NotEmpty(t, problem.Hints) if name == "named workspace outputs" { - require.ErrorContains(t, err, "non-default workspaces") + require.Equal(t, model.CodeWorkspace, problem.Code) } }) } @@ -114,6 +133,168 @@ func TestPathHelpersAcceptMixedSeparators(t *testing.T) { require.Equal(t, nativeUnit, localPath(filepath.Join(root, "live"), "../catalog/units/app")) } +// scanTree writes empty unit files and returns the project root. +func scanTree(t *testing.T, files ...string) string { + t.Helper() + root, err := filepath.EvalSymlinks(t.TempDir()) + require.NoError(t, err) + for _, name := range files { + path := filepath.Join(root, filepath.FromSlash(name)) + require.NoError(t, os.MkdirAll(filepath.Dir(path), 0o755)) + require.NoError(t, os.WriteFile(path, nil, 0o644)) + } + return root +} + +func relativeUnits(t *testing.T, root string, scan pathScan) []string { + t.Helper() + var result []string + for _, path := range scan.Units { + rel, err := filepath.Rel(root, path) + require.NoError(t, err) + result = append(result, filepath.ToSlash(rel)) + } + return result +} + +// included_paths and excluded_paths share one pattern language: a pattern selects what it +// matches and everything below it, with or without a trailing slash, `./`, or `/**`. +func TestIncludedAndExcludedPathsShareOnePatternLanguage(t *testing.T) { + root := scanTree(t, "live/dev/vpc/terragrunt.hcl", "live/dev/app/terragrunt.hcl", "live/prod/vpc/terragrunt.hcl") + all := []string{"live/dev/app/terragrunt.hcl", "live/dev/vpc/terragrunt.hcl", "live/prod/vpc/terragrunt.hcl"} + for _, test := range []struct { + pattern string + want []string + }{ + {"live", all}, + {"live/", all}, + {"./live", all}, + {"./**", all}, + {".", all}, + {"live/**", all}, + {"live/dev", all[:2]}, + {"live/dev/", all[:2]}, + {"live/dev/*", all[:2]}, + {"live/dev/vpc", []string{"live/dev/vpc/terragrunt.hcl"}}, + {"live/dev/vpc/terragrunt.hcl", []string{"live/dev/vpc/terragrunt.hcl"}}, + {"**/vpc", []string{"live/dev/vpc/terragrunt.hcl", "live/prod/vpc/terragrunt.hcl"}}, + {"live/*/vpc", []string{"live/dev/vpc/terragrunt.hcl", "live/prod/vpc/terragrunt.hcl"}}, + } { + t.Run("include "+test.pattern, func(t *testing.T) { + scan, err := discoveryPaths(root, []string{test.pattern}, nil) + require.NoError(t, err) + require.Equal(t, test.want, relativeUnits(t, root, scan)) + require.Empty(t, scan.Warnings) + }) + t.Run("exclude "+test.pattern, func(t *testing.T) { + scan, err := discoveryPaths(root, nil, []string{test.pattern}) + require.NoError(t, err) + var kept []string + for _, unit := range all { + if !slices.Contains(test.want, unit) { + kept = append(kept, unit) + } + } + require.Equal(t, kept, relativeUnits(t, root, scan)) + }) + } +} + +func TestUnmatchedIncludePatternIsReported(t *testing.T) { + root := scanTree(t, "live/dev/vpc/terragrunt.hcl") + scan, err := discoveryPaths(root, []string{"live/dev", "lives", "live/none/*"}, nil) + require.NoError(t, err) + require.Equal(t, []string{"live/dev/vpc/terragrunt.hcl"}, relativeUnits(t, root, scan)) + var patterns []string + for _, warning := range scan.Warnings { + require.Equal(t, model.WarningPatternUnmatched, warning.Code) + patterns = append(patterns, warning.Path) + } + require.Contains(t, patterns, "live/none/*") + require.Len(t, patterns, 2, "the matching pattern is not reported") +} + +// Upstream Terragrunt does not follow directory symlinks (without its symlinks experiment) and +// treats a symlinked terragrunt.hcl as a unit where the link lives. A skipped link is reported. +func TestSymlinksFollowUpstreamTerragruntAndAreReported(t *testing.T) { + root := scanTree(t, "live/prod/vpc/terragrunt.hcl", "live/dev/vpc/terragrunt.hcl") + if err := os.Symlink("prod", filepath.Join(root, "live", "linked")); err != nil { + t.Skipf("symlink unavailable: %v", err) + } + require.NoError(t, os.MkdirAll(filepath.Join(root, "live", "dev", "copy"), 0o755)) + require.NoError(t, os.Symlink(filepath.Join("..", "vpc", "terragrunt.hcl"), filepath.Join(root, "live", "dev", "copy", "terragrunt.hcl"))) + scan, err := discoveryPaths(root, []string{"live/**", "live"}, nil) + require.NoError(t, err) + require.Equal(t, []string{"live/dev/copy/terragrunt.hcl", "live/dev/vpc/terragrunt.hcl", "live/prod/vpc/terragrunt.hcl"}, relativeUnits(t, root, scan), "the symlinked file is a unit at its own location; the linked directory adds none") + require.Len(t, scan.Warnings, 1, "overlapping patterns report a skipped link once") + require.Equal(t, model.WarningSymlinkedDirectory, scan.Warnings[0].Code) + require.Equal(t, "live/linked", scan.Warnings[0].Path) + require.Contains(t, scan.Warnings[0].Message, "live/linked") + again, err := discoveryPaths(root, []string{"live/**"}, nil) + require.NoError(t, err) + require.Equal(t, scan, again, "discovery is deterministic") +} + +func TestLogicalPathKeepsASymlinkedConfigurationFile(t *testing.T) { + root := scanTree(t, "a/terragrunt.hcl") + require.NoError(t, os.MkdirAll(filepath.Join(root, "b"), 0o755)) + if err := os.Symlink(filepath.Join("..", "a", "terragrunt.hcl"), filepath.Join(root, "b", "terragrunt.hcl")); err != nil { + t.Skipf("symlink unavailable: %v", err) + } + got, err := logicalPath(filepath.Join(root, "b", "terragrunt.hcl")) + require.NoError(t, err) + require.Equal(t, filepath.Join(root, "b", "terragrunt.hcl"), got) + canonical, err := canonicalPath(filepath.Join(root, "b", "terragrunt.hcl")) + require.NoError(t, err) + require.Equal(t, filepath.Join(root, "a", "terragrunt.hcl"), canonical) +} + +// A pattern that selects only ordinary files, such as a docs directory, is still reported as +// matching no Terragrunt configuration. +func TestIncludePatternMatchingOnlyOrdinaryFilesIsReported(t *testing.T) { + root := scanTree(t, "live/dev/vpc/terragrunt.hcl", "docs/README.md", "docs/notes/main.tf") + scan, err := discoveryPaths(root, []string{"live/dev", "docs"}, nil) + require.NoError(t, err) + require.Equal(t, []string{"live/dev/vpc/terragrunt.hcl"}, relativeUnits(t, root, scan)) + require.Len(t, scan.Warnings, 1) + require.Equal(t, model.WarningPatternUnmatched, scan.Warnings[0].Code) + require.Equal(t, "docs", scan.Warnings[0].Path) +} + +// A generated child that fails is reported at its own address; its siblings are still discovered. +func TestDiscoveryKeepsSiblingsOfAFailedGeneratedUnit(t *testing.T) { + root, err := filepath.EvalSymlinks(t.TempDir()) + require.NoError(t, err) + write := func(name, content string) { + path := filepath.Join(root, filepath.FromSlash(name)) + require.NoError(t, os.MkdirAll(filepath.Dir(path), 0o755)) + require.NoError(t, os.WriteFile(path, []byte(content), 0o644)) + } + write("modules/unit/main.tf", "") + write("catalog/good/terragrunt.hcl", "terraform { source = \"${get_repo_root()}/modules/unit\" }\n") + write("catalog/broken/terragrunt.hcl", "terraform { source = \"${get_repo_root()}/modules/unit\" }\nterraform_binary = \"tofu\"\n") + write(".git/HEAD", "ref: refs/heads/main\n") + write("live/terragrunt.stack.hcl", ` +unit "a_broken" { + source = "../catalog/broken" + path = "a_broken" +} +unit "z_good" { + source = "../catalog/good" + path = "z_good" +} +`) + result, err := Resolve(t.Context(), model.Request{BasePath: root, Phase: "inventory", Command: "plan", Tofu: "must-not-execute", Environment: map[string]string{}, IncludedPaths: []string{"live"}}) + require.NoError(t, err) + require.Len(t, result.Inventory, 1) + require.Equal(t, "z_good", result.Inventory[0].Name) + require.Len(t, result.Failures, 1) + require.Equal(t, "live", result.Failures[0].Stack) + require.Equal(t, "a_broken", result.Failures[0].Name) + problem := asProblem(t, result.Failures[0].Err) + require.Equal(t, "terraform_binary", problem.Key) +} + // stackProject writes a module, a catalog unit pointing at it, and the given files. func stackProject(t *testing.T, files map[string]string) string { t.Helper() @@ -214,8 +395,11 @@ unit "app" { require.NoDirExists(t, filepath.Join(root, "live", ".terragrunt-stack"), "discovery generates only into its overlay") request.ExcludedPaths = nil - _, err = Resolve(t.Context(), request) - require.Error(t, err, "an included broken stack is still reported") + result, err = Resolve(t.Context(), request) + require.NoError(t, err, "a broken stack is reported as a failure of its own, not of the whole discovery") + require.Equal(t, []string{"live/app"}, inventoryNames(t, result)) + require.Len(t, result.Failures, 1, "an included broken stack is still reported") + require.Equal(t, "live/other", result.Failures[0].Stack) } // A unit that a stack generates beside its stack file belongs to that stack; a @@ -246,12 +430,79 @@ unit "app" { require.Empty(t, result.Inventory[1].GenerationRoot) } -func TestDiscoveryParsePreservesCallerRequest(t *testing.T) { - request := nativeUnit(t, "") - request.Phase = "inventory" - original := request - discovery := projectDiscovery{ctx: t.Context(), request: &request, root: request.BasePath} - _, err := discovery.parse(filepath.Join(request.BasePath, "missing", stackFile), true) - require.Error(t, err) - require.Equal(t, original, request) +// Inject errors during evaluation, after discovery has begun, without global +// parser hooks or timing-dependent cancellation goroutines. +func TestDiscoveryEvaluationErrors(t *testing.T) { + for _, stage := range []string{"generation", "stack", "unit", "child stack", "child unit"} { + for _, cause := range []error{context.Canceled, context.DeadlineExceeded, errors.New("invalid unit")} { + t.Run(stage+"/"+cause.Error(), func(t *testing.T) { + root, err := filepath.EvalSymlinks(t.TempDir()) + require.NoError(t, err) + first := filepath.Join(root, "a", "terragrunt.stack.hcl") + if stage == "unit" { + first = filepath.Join(root, "a", "terragrunt.hcl") + } + later := filepath.Join(root, "z", "terragrunt.hcl") + for _, path := range []string{first, later} { + require.NoError(t, os.MkdirAll(filepath.Dir(path), 0o755)) + require.NoError(t, os.WriteFile(path, nil, 0o644)) + } + child := filepath.Join(root, "generated", "terragrunt.hcl") + if stage == "child stack" { + child = filepath.Join(root, "generated", "terragrunt.stack.hcl") + } + failed, continued := false, false + evaluate := func(_ context.Context, req *model.Request, _ afero.Fs) (*model.Resolution, error) { + if req.Path == later { + continued = true + return &model.Resolution{Path: later, Source: "module"}, nil + } + target := req.Path == child || (req.Path == first && (stage == "unit" || (stage == "generation" && req.Command == "generate") || (stage == "stack" && req.Command != "generate"))) + if target { + failed = true + return nil, fmt.Errorf("evaluating %s: %w", req.Path, cause) + } + result := &model.Resolution{} + if req.Command != "generate" { + switch stage { + case "child stack": + result.StackConfigs = map[string]string{"child": child} + result.StackSources = map[string]string{"child": filepath.Join(root, "catalog")} + result.Stacks = map[string]string{"child": "child"} + case "child unit": + result.UnitConfigs = map[string]string{"child": child} + result.UnitSources = map[string]string{"child": filepath.Join(root, "catalog")} + } + } + return result, nil + } + result, err := discoverProjectWithResolver(t.Context(), &model.Request{BasePath: root}, evaluate) + require.True(t, failed, "must reach the intended evaluation failure") + if errors.Is(cause, context.Canceled) || errors.Is(cause, context.DeadlineExceeded) { + require.ErrorIs(t, err, cause) + require.Nil(t, result, "canceled discovery must not return partial inventory") + require.False(t, continued, "must stop before evaluating subsequent units") + } else { + require.NoError(t, err) + require.True(t, continued) + require.Len(t, result.Failures, 1) + require.Len(t, result.Inventory, 1) + require.Equal(t, later, result.Inventory[0].Config.Path) + } + }) + } + } +} + +func TestDiscoveryContextCanceledDuringEvaluation(t *testing.T) { + root := t.TempDir() + require.NoError(t, os.WriteFile(filepath.Join(root, "terragrunt.hcl"), nil, 0o644)) + ctx, cancel := context.WithCancel(t.Context()) + defer cancel() + result, err := discoverProjectWithResolver(ctx, &model.Request{BasePath: root}, func(context.Context, *model.Request, afero.Fs) (*model.Resolution, error) { + cancel() + return &model.Resolution{}, nil + }) + require.ErrorIs(t, err, context.Canceled) + require.Nil(t, result) } diff --git a/plugins/terragrunt/internal/engine/environment_test.go b/plugins/terragrunt/internal/engine/environment_test.go index 142b7d563e7..93504bf4eff 100644 --- a/plugins/terragrunt/internal/engine/environment_test.go +++ b/plugins/terragrunt/internal/engine/environment_test.go @@ -21,6 +21,19 @@ func TestMain(m *testing.M) { _, _ = os.Stdout.WriteString(os.Getenv("ATMOS_TERRAGRUNT_VERSION_CHILD")) os.Exit(0) } + // The effect child records its argument in ./effects and prints it, so tests can observe when + // run_cmd executes without depending on a platform shell. + if os.Getenv(effectChildEnv) == "1" && len(os.Args) > 1 { + text := os.Args[len(os.Args)-1] + file, err := os.OpenFile("effects", os.O_APPEND|os.O_CREATE|os.O_WRONLY, 0o600) + if err != nil { + os.Exit(1) + } + _, _ = file.WriteString(text + "\n") + _ = file.Close() + _, _ = os.Stdout.WriteString(text) + os.Exit(0) + } if os.Getenv("ATMOS_TERRAGRUNT_ENV_CHILD") == "1" { cwd, _ := os.Getwd() _ = json.NewEncoder(os.Stdout).Encode(map[string]string{"cwd": cwd, "profile": os.Getenv("AWS_PROFILE"), "token": os.Getenv("AWS_WEB_IDENTITY_TOKEN_FILE"), "custom": os.Getenv("CUSTOM_VALUE"), "path": os.Getenv("PATH")}) diff --git a/plugins/terragrunt/internal/engine/execution.go b/plugins/terragrunt/internal/engine/execution.go index 5b928eabaf7..09cc8656af0 100644 --- a/plugins/terragrunt/internal/engine/execution.go +++ b/plugins/terragrunt/internal/engine/execution.go @@ -5,11 +5,17 @@ import ( "errors" "fmt" "maps" + "path/filepath" "reflect" "regexp" "slices" + "strings" "github.com/gruntwork-io/terragrunt/pkg/config" + "github.com/hashicorp/hcl/v2" + "github.com/hashicorp/hcl/v2/gohcl" + "github.com/hashicorp/hcl/v2/hclparse" + "github.com/hashicorp/hcl/v2/hclsyntax" "github.com/spf13/afero" "github.com/cloudposse/atmos/plugins/terragrunt/internal/model" @@ -190,7 +196,8 @@ func errorHooksFromConfig(source []config.ErrorHook) ([]model.Hook, error) { } for _, pattern := range converted.OnErrors { if _, err := regexp.Compile(pattern); err != nil { - return nil, fmt.Errorf("%w: error_hook %q on_errors is not a valid regular expression", ErrUnsupportedExecution, converted.Name) + return nil, unsupportedHook(fmt.Sprintf("error_hook %q", converted.Name), fmt.Sprintf("Invalid `on_errors` pattern in error_hook %q", converted.Name), + "An `on_errors` entry is not a valid regular expression.", "Fix the pattern; Atmos matches `on_errors` entries as Go regular expressions.") } } result = append(result, converted) @@ -201,18 +208,226 @@ func errorHooksFromConfig(source []config.ErrorHook) ([]model.Hook, error) { func validateHook(kind string, hook *model.Hook) error { for _, command := range hook.Commands { if reason, unsupported := unsupportedLifecycles[command]; unsupported { - return fmt.Errorf("%w: %s %q lists the %q lifecycle command, which %s; it is not implemented", ErrUnsupportedExecution, kind, hook.Name, command, reason) + return unsupportedHook(fmt.Sprintf("%s %q", kind, hook.Name), fmt.Sprintf("Unsupported hook command `%s` in %s %q", command, kind, hook.Name), + fmt.Sprintf("The `%s` lifecycle command %s, which Atmos does not implement, so the hook would never run.", command, reason), + "Remove the lifecycle command from `commands`. Hooks can name Terraform commands such as `plan`, `apply`, and `init`.") } } if len(hook.Execute) == 0 || hook.Execute[0] == "" { - return fmt.Errorf("%w: %s %q must set a non-empty execute list", ErrUnsupportedExecution, kind, hook.Name) + return unsupportedHook(fmt.Sprintf("%s %q", kind, hook.Name), fmt.Sprintf("%s %q has no command to run", kind, hook.Name), + "The hook sets no `execute` list or its first entry is empty.", "Set `execute = [\"program\", \"argument\"]` on the hook.") } if _, builtin := builtinHookExecutables[hook.Execute[0]]; builtin { - return fmt.Errorf("%w: %s %q runs the built-in %q integration, which is not implemented", ErrUnsupportedExecution, kind, hook.Name, hook.Execute[0]) + return unsupportedHook(fmt.Sprintf("%s %q", kind, hook.Name), fmt.Sprintf("Unsupported built-in `%s` hook in %s %q", hook.Execute[0], kind, hook.Name), + fmt.Sprintf("Terragrunt intercepts a hook that executes `%s` and runs its own integration, which Atmos does not implement.", hook.Execute[0]), + "Run the tool through a wrapper script, for example `execute = [\"./run-tflint.sh\"]`, or run it as a separate workflow step before the Terraform command.") } return nil } +// checkExecutionSyntax validates declaration shapes and literal hook restrictions without +// a Terragrunt evaluation context. Dynamic values (including dependency outputs and +// run_cmd) are validated by executionFromConfig during preparation. +func checkExecutionSyntax(filesystem afero.Fs, cfg *config.TerragruntConfig, path string) error { + paths, err := includedExecutionPaths(filesystem, cfg, path) + if err != nil { + return err + } + effective, err := effectiveExecutionBlocks(filesystem, paths) + if err != nil { + return err + } + // A declaration that a later file replaces never runs, so only the effective ones are checked. + for _, declaration := range effective { + if err := checkExecutionBlock(declaration.block); err != nil { + return hookProblem(err, declaration.path) + } + } + return nil +} + +// includedExecutionPaths lists the files whose execution blocks apply to a unit, in merge order. +// Included files come first, in the order the unit declares its includes, and the unit comes last, +// so a later same-named declaration replaces an earlier one, as Terragrunt merges hooks and +// extra_arguments by name. +func includedExecutionPaths(filesystem afero.Fs, cfg *config.TerragruntConfig, path string) ([]string, error) { + var paths []string + for _, name := range includeDeclarationOrder(filesystem, path, cfg.ProcessedIncludes) { + include := cfg.ProcessedIncludes[name] + strategy, err := include.GetMergeStrategy() + if err != nil { + return nil, err + } + if strategy == config.NoMerge { + continue + } + included := include.Path + if !filepath.IsAbs(included) { + included = filepath.Join(filepath.Dir(path), included) + } + paths = append(paths, included) + } + return append(paths, path), nil +} + +// includeDeclarationOrder returns the names of the processed includes in the order the unit declares +// them. The processed map has no order, and a later include takes precedence over an earlier one. +// An include the unit file does not declare by name keeps a deterministic place after the others. +func includeDeclarationOrder(filesystem afero.Fs, path string, processed config.IncludeConfigsMap) []string { + var ordered []string + for _, name := range declaredIncludeNames(filesystem, path) { + if _, ok := processed[name]; ok && !slices.Contains(ordered, name) { + ordered = append(ordered, name) + } + } + for _, name := range slices.Sorted(maps.Keys(processed)) { + if !slices.Contains(ordered, name) { + ordered = append(ordered, name) + } + } + return ordered +} + +// declaredIncludeNames reads the `include` blocks of a unit file in declaration order. A bare +// `include` has the empty name. An unreadable or unparseable file yields none, because the +// execution blocks of the same file are parsed and reported afterwards. +func declaredIncludeNames(filesystem afero.Fs, path string) []string { + data, err := afero.ReadFile(filesystem, path) + if err != nil { + return nil + } + parser := hclparse.NewParser() + var file *hcl.File + var diagnostics hcl.Diagnostics + if strings.HasSuffix(path, ".json") { + file, diagnostics = parser.ParseJSON(data, path) + } else { + file, diagnostics = parser.ParseHCL(data, path) + } + if diagnostics.HasErrors() { + return nil + } + var names []string + if body, ok := file.Body.(*hclsyntax.Body); ok { + for _, block := range body.Blocks { + if block.Type == "include" { + names = append(names, strings.Join(block.Labels, "")) + } + } + return names + } + // JSON bodies name every include, so the labelled schema reads them in order. + content, _, diagnostics := file.Body.PartialContent(&hcl.BodySchema{Blocks: []hcl.BlockHeaderSchema{{Type: "include", LabelNames: []string{"name"}}}}) + if diagnostics.HasErrors() { + return nil + } + for _, block := range content.Blocks { + names = append(names, block.Labels[0]) + } + return names +} + +// executionDeclaration is one execution block and the file that declares it. +type executionDeclaration struct { + block *hcl.Block + path string +} + +// effectiveExecutionBlocks parses every file and keeps, for each block type and name, the last +// declaration in file order. Unparseable files are still reported. +func effectiveExecutionBlocks(filesystem afero.Fs, paths []string) ([]executionDeclaration, error) { + var order []string + effective := map[string]executionDeclaration{} + for _, path := range paths { + blocks, err := executionDeclarations(filesystem, path) + if err != nil { + return nil, err + } + for _, block := range blocks { + key := block.Type + "\x00" + strings.Join(block.Labels, "\x00") + if _, seen := effective[key]; !seen { + order = append(order, key) + } + effective[key] = executionDeclaration{block: block, path: path} + } + } + result := make([]executionDeclaration, 0, len(order)) + for _, key := range order { + result = append(result, effective[key]) + } + return result, nil +} + +func checkExecutionBlock(block *hcl.Block) error { + target := executionBlockTarget(block.Type) + if target == nil { + return nil + } + schema, _ := gohcl.ImpliedBodySchema(target) + content, diagnostics := block.Body.Content(schema) + if diagnostics.HasErrors() { + return diagnostics + } + if block.Type == "extra_arguments" || !hookMayRun(content) { + return nil + } + hook := model.Hook{Name: block.Labels[0], Execute: []string{"deferred"}} + if commands, ok := literalStringList(content, "commands"); ok { + hook.Commands = commands + } + if execute, ok := literalStringList(content, "execute"); ok { + hook.Execute = execute + } + if err := validateHook(block.Type, &hook); err != nil { + return err + } + if patterns, ok := literalStringList(content, "on_errors"); ok { + _, err := errorHooksFromConfig([]config.ErrorHook{{Name: hook.Name, Commands: hook.Commands, Execute: hook.Execute, OnErrors: patterns}}) + return err + } + return nil +} + +// executionBlockTarget returns the configuration type that declares a block, or nil for a block that +// holds no execution settings. +func executionBlockTarget(blockType string) any { + switch blockType { + case "extra_arguments": + return &config.TerraformExtraArguments{} + case "before_hook", "after_hook": + return &config.Hook{} + case "error_hook": + return &config.ErrorHook{} + } + return nil +} + +// hookMayRun reports false when a literal `if = false` disables the hook. A dynamic condition may +// disable it too, but that is decided only during preparation. +func hookMayRun(content *hcl.BodyContent) bool { + condition, ok := content.Attributes["if"] + if !ok { + return true + } + var enabled bool + diagnostics := gohcl.DecodeExpression(condition.Expr, nil, &enabled) + return !diagnostics.HasErrors() && enabled +} + +// literalStringList decodes an attribute that is a literal list of strings. It reports false when +// the attribute is absent or dynamic, which preparation validates once values are known. +func literalStringList(content *hcl.BodyContent, name string) ([]string, bool) { + attribute, ok := content.Attributes[name] + if !ok { + return nil, false + } + var values []string + if diagnostics := gohcl.DecodeExpression(attribute.Expr, nil, &values); diagnostics.HasErrors() { + return nil, false + } + return values, true +} + // rejectArgumentDependentExecution fails closed until saved apply arguments can be bound. func rejectArgumentDependentExecution(filesystem afero.Fs, paths []string) error { for _, path := range paths { @@ -223,3 +438,35 @@ func rejectArgumentDependentExecution(filesystem afero.Fs, paths []string) error } return nil } + +func executionDeclarations(filesystem afero.Fs, path string) ([]*hcl.Block, error) { + data, err := afero.ReadFile(filesystem, path) + if err != nil { + return nil, err + } + parser := hclparse.NewParser() + var file *hcl.File + var diagnostics hcl.Diagnostics + if strings.HasSuffix(path, ".json") { + file, diagnostics = parser.ParseJSON(data, path) + } else { + file, diagnostics = parser.ParseHCL(data, path) + } + if diagnostics.HasErrors() { + return nil, diagnostics + } + content, _, diagnostics := file.Body.PartialContent(&hcl.BodySchema{Blocks: []hcl.BlockHeaderSchema{{Type: "terraform"}}}) + if diagnostics.HasErrors() { + return nil, diagnostics + } + var blocks []*hcl.Block + schema, _ := gohcl.ImpliedBodySchema(&config.TerraformConfig{}) + for _, terraform := range content.Blocks { + declarations, diagnostics := terraform.Body.Content(schema) + if diagnostics.HasErrors() { + return nil, diagnostics + } + blocks = append(blocks, declarations.Blocks...) + } + return blocks, nil +} diff --git a/plugins/terragrunt/internal/engine/execution_test.go b/plugins/terragrunt/internal/engine/execution_test.go index 57b4949041c..a347288f1f0 100644 --- a/plugins/terragrunt/internal/engine/execution_test.go +++ b/plugins/terragrunt/internal/engine/execution_test.go @@ -194,12 +194,121 @@ func TestValidateHookRequiresRunnableExecuteList(t *testing.T) { t.Run(name, func(t *testing.T) { err := validateHook("before_hook", &model.Hook{Name: "h", Commands: []string{"plan"}, Execute: execute}) require.ErrorIs(t, err, ErrUnsupportedExecution) - require.ErrorContains(t, err, "non-empty execute list") + require.ErrorContains(t, err, "no command to run") }) } require.NoError(t, validateHook("before_hook", &model.Hook{Name: "h", Commands: []string{"plan"}, Execute: []string{"tool"}})) } +// effectChildEnv makes the test binary act as a cross-platform effect command (see TestMain). +const effectChildEnv = "ATMOS_TERRAGRUNT_EFFECT_CHILD" + +// effectCommand returns a run_cmd call that appends text to ./effects and prints it, using the +// test binary itself instead of a platform shell. +func effectCommand(t *testing.T, text string) string { + t.Helper() + executable, err := os.Executable() + require.NoError(t, err) + quoted, err := json.Marshal(executable) + require.NoError(t, err) + return "run_cmd(" + string(quoted) + ", \"" + text + "\")" +} + +// effectUnit is executionUnit with the effect child enabled for run_cmd. +func effectUnit(t *testing.T, body string) model.Request { + t.Helper() + request := executionUnit(t, body) + request.Environment[effectChildEnv] = "1" + return request +} + +func TestDiscoveryDefersExecutionValues(t *testing.T) { + request := effectUnit(t, ` + before_hook "dynamic" { + commands = ["plan"] + execute = ["echo", dependency.base.outputs.value, `+effectCommand(t, "hook")+`] + } + extra_arguments "dynamic" { + commands = ["plan"] + arguments = [dependency.base.outputs.value, `+effectCommand(t, "args")+`] + } + `) + body, err := os.ReadFile(request.Path) + require.NoError(t, err) + require.NoError(t, os.MkdirAll(filepath.Join(request.BasePath, "base"), 0o755)) + require.NoError(t, os.WriteFile(filepath.Join(request.BasePath, "base", "terragrunt.hcl"), []byte("inputs = {}"), 0o644)) + body = append(body, []byte(` + dependency "base" { + config_path = "./base" + skip_outputs = true + mock_outputs = { value = "from-dependency" } + } + `)...) + require.NoError(t, os.WriteFile(request.Path, body, 0o644)) + request.Phase = "discover" + discovered, err := Resolve(t.Context(), request) + require.NoError(t, err) + require.Nil(t, discovered.Execution) + require.NoFileExists(t, filepath.Join(request.BasePath, "effects")) + request.Phase = "" + prepared, err := Resolve(t.Context(), request) + require.NoError(t, err) + require.Equal(t, []string{"echo", "from-dependency", "hook"}, prepared.Execution.BeforeHooks[0].Execute) + require.Equal(t, []string{"from-dependency", "args"}, prepared.Execution.ExtraArguments[0].Arguments) + effects, err := os.ReadFile(filepath.Join(request.BasePath, "effects")) + require.NoError(t, err) + require.Contains(t, string(effects), "hook") + require.Contains(t, string(effects), "args") +} + +func TestDiscoveryValidatesExecutionSyntaxWithoutEffects(t *testing.T) { + for _, test := range []struct { + name, body, want string + }{ + {"lifecycle", `before_hook "hook" { + commands = ["read-config"] + execute = [run_cmd("must-not-execute")] +}`, "Unsupported hook command"}, + {"builtin", `before_hook "hook" { + commands = [run_cmd("must-not-execute")] + execute = ["tflint"] +}`, "Unsupported built-in"}, + {"missing execute", `before_hook "hook" { + commands = ["plan"] +}`, "execute"}, + {"unknown argument setting", `extra_arguments "args" { + commands = ["plan"] + unexpected = run_cmd("must-not-execute") +}`, "unexpected"}, + {"invalid regex", `error_hook "hook" { + commands = ["plan"] + execute = [run_cmd("must-not-execute")] + on_errors = ["["] +}`, "Invalid `on_errors`"}, + {"disabled", `before_hook "hook" { + if = false + commands = ["read-config"] + execute = [run_cmd("must-not-execute")] +}`, ""}, + {"dynamic condition", `before_hook "hook" { + if = run_cmd("must-not-execute") == "enabled" + commands = ["read-config"] + execute = [run_cmd("must-not-execute")] +}`, ""}, + } { + t.Run(test.name, func(t *testing.T) { + request := executionUnit(t, test.body) + request.Phase = "discover" + _, err := Resolve(t.Context(), request) + if test.want == "" { + require.NoError(t, err) + } else { + require.ErrorContains(t, err, test.want) + } + }) + } +} + // A saved plan is applied from its plan-time preparation, but Terragrunt evaluates the // configuration again for apply. Hooks that depend on the command are recorded for apply. func TestPlanPreparationRecordsApplyTimeExecution(t *testing.T) { @@ -315,6 +424,39 @@ func TestPlanPreparationOmitsApplyWhenCommandIsNotObserved(t *testing.T) { } } +func TestDiscoveryDefersInheritedExecutionValues(t *testing.T) { + request := effectUnit(t, `before_hook "parent" { + commands = ["plan"] + execute = ["echo", `+effectCommand(t, "inherited")+`] +}`) + parent := filepath.Join(request.BasePath, "parent.hcl") + require.NoError(t, os.Rename(request.Path, parent)) + require.NoError(t, os.WriteFile(request.Path, []byte("include \"parent\" { path = \"./parent.hcl\" }"), 0o644)) + request.Phase = "discover" + _, err := Resolve(t.Context(), request) + require.NoError(t, err) + require.NoFileExists(t, filepath.Join(request.BasePath, "effects")) + request.Phase = "" + result, err := Resolve(t.Context(), request) + require.NoError(t, err) + require.Equal(t, []string{"echo", "inherited"}, result.Execution.BeforeHooks[0].Execute) + require.FileExists(t, filepath.Join(request.BasePath, "effects")) +} + +func TestPreparationValidatesDynamicHookSettings(t *testing.T) { + request := executionUnit(t, `before_hook "dynamic" { + commands = [get_env("HOOK_COMMAND")] + execute = ["echo"] +}`) + request.Environment["HOOK_COMMAND"] = "read-config" + request.Phase = "discover" + _, err := Resolve(t.Context(), request) + require.NoError(t, err) + request.Phase = "" + _, err = Resolve(t.Context(), request) + require.ErrorContains(t, err, "Unsupported hook command") +} + func TestObservesCommandAssumesUnreadableFilesDo(t *testing.T) { require.True(t, observesCommand(afero.NewMemMapFs(), []string{"missing.hcl"})) } @@ -327,6 +469,74 @@ func hookNames(hooks []model.Hook) []string { return names } +// A parent hook that a unit replaces by name never runs, so its unsupported built-in executable +// does not block the unit. The same parent hook is still rejected when nothing replaces it. +func TestDiscoveryValidatesEffectiveHooksNotOverriddenParents(t *testing.T) { + for _, test := range []struct { + name, child, want string + }{ + {"overridden by name", `before_hook "lint" { + commands = ["plan"] + execute = ["./wrapper.sh"] +}`, ""}, + {"inherited unchanged", "", "Unsupported built-in"}, + } { + t.Run(test.name, func(t *testing.T) { + request := executionUnit(t, test.child) + parent := filepath.Join(request.BasePath, "parent.hcl") + require.NoError(t, os.WriteFile(parent, []byte(`terraform { + before_hook "lint" { + commands = ["plan"] + execute = ["tflint"] + } +} +`), 0o644)) + body, err := os.ReadFile(request.Path) + require.NoError(t, err) + require.NoError(t, os.WriteFile(request.Path, append([]byte("include \"parent\" { path = \"./parent.hcl\" }\n"), body...), 0o644)) + request.Phase = "discover" + _, err = Resolve(t.Context(), request) + if test.want == "" { + require.NoError(t, err) + return + } + require.ErrorContains(t, err, test.want) + }) + } +} + +// Includes merge in declaration order, so a later declaration replaces an earlier one whatever the +// include names sort to. Here the unsupported built-in hook is replaced only when its include is first. +func TestDiscoveryUsesIncludeDeclarationOrderForHookOverrides(t *testing.T) { + for _, test := range []struct { + name, first, second, want string + }{ + {"later declaration replaces an earlier one", "zeta", "alpha", ""}, + {"earlier declaration is replaced by the later one", "alpha", "zeta", "Unsupported built-in"}, + } { + t.Run(test.name, func(t *testing.T) { + request := executionUnit(t, "") + hook := func(execute string) []byte { + return []byte("terraform {\n before_hook \"lint\" {\n commands = [\"plan\"]\n execute = [" + execute + "]\n }\n}\n") + } + // The built-in executable is declared by the "zeta" include, the wrapper by "alpha". + require.NoError(t, os.WriteFile(filepath.Join(request.BasePath, "zeta.hcl"), hook(`"tflint"`), 0o644)) + require.NoError(t, os.WriteFile(filepath.Join(request.BasePath, "alpha.hcl"), hook(`"./wrapper.sh"`), 0o644)) + body, err := os.ReadFile(request.Path) + require.NoError(t, err) + includes := "include \"" + test.first + "\" { path = \"./" + test.first + ".hcl\" }\ninclude \"" + test.second + "\" { path = \"./" + test.second + ".hcl\" }\n" + require.NoError(t, os.WriteFile(request.Path, append([]byte(includes), body...), 0o644)) + request.Phase = "discover" + _, err = Resolve(t.Context(), request) + if test.want == "" { + require.NoError(t, err) + return + } + require.ErrorContains(t, err, test.want) + }) + } +} + func TestPlanRejectsArgumentDependentSavedHooks(t *testing.T) { request := executionUnit(t, `before_hook "args" { commands = ["apply"] diff --git a/plugins/terragrunt/internal/engine/generated_inputs.go b/plugins/terragrunt/internal/engine/generated_inputs.go new file mode 100644 index 00000000000..4001a9dadf9 --- /dev/null +++ b/plugins/terragrunt/internal/engine/generated_inputs.go @@ -0,0 +1,101 @@ +package engine + +import ( + "encoding/json" + "errors" + "fmt" + "io/fs" + "os" + "path/filepath" + "strings" + + "github.com/hashicorp/hcl/v2" + "github.com/hashicorp/hcl/v2/hclsyntax" + + "github.com/cloudposse/atmos/plugins/terragrunt/internal/model" +) + +// unitLockFile hands the host the lock file kept beside terragrunt.hcl, so the provider versions +// and hashes it pins take effect in the working directory. The host rewrites it only when this +// copy changes, so what `init` records in the working directory is not undone on every run. +// Atmos does not write lock file changes back to the unit directory. +func unitLockFile(result *model.Resolution) error { + if _, exists := result.GeneratedPolicies[lockFileName]; exists { + // An explicit generate block for the lock file wins over the copy. + return nil + } + content, err := os.ReadFile(filepath.Join(filepath.Dir(result.Path), lockFileName)) + if errors.Is(err, fs.ErrNotExist) { + return nil + } + if err != nil { + return fmt.Errorf("read the lock file beside %s: %w", filepath.Base(result.Path), err) + } + result.Generated[lockFileName] = string(content) + result.GeneratedPolicies[lockFileName] = "overwrite" + result.UnitFileCopies[lockFileName] = true + return nil +} + +// declaresBackend reports whether generated Terraform configuration holds a backend or cloud block. +// Content that cannot be parsed is not a backend: Terraform reports its own syntax error later. +func declaresBackend(path, content string) bool { + lower := strings.ToLower(path) + switch { + case strings.HasSuffix(lower, ".tf.json"): + return jsonDeclaresBackend([]byte(content)) + case strings.HasSuffix(lower, ".tf"): + return hclDeclaresBackend([]byte(content), path) + default: + return false + } +} + +func hclDeclaresBackend(content []byte, path string) bool { + file, diagnostics := hclsyntax.ParseConfig(content, path, hcl.InitialPos) + if diagnostics.HasErrors() { + return false + } + body, ok := file.Body.(*hclsyntax.Body) + if !ok { + return false + } + for _, block := range body.Blocks { + if block.Type != "terraform" { + continue + } + for _, nested := range block.Body.Blocks { + if nested.Type == "backend" || nested.Type == "cloud" { + return true + } + } + } + return false +} + +func jsonDeclaresBackend(content []byte) bool { + var document struct { + Terraform json.RawMessage `json:"terraform"` + } + if json.Unmarshal(content, &document) != nil || len(document.Terraform) == 0 { + return false + } + // "terraform" is an object, or a list of objects in the alternate JSON syntax. + var blocks []map[string]json.RawMessage + if json.Unmarshal(document.Terraform, &blocks) != nil { + var single map[string]json.RawMessage + if json.Unmarshal(document.Terraform, &single) != nil { + return false + } + blocks = []map[string]json.RawMessage{single} + } + for _, block := range blocks { + if _, ok := block["backend"]; ok { + return true + } + if _, ok := block["cloud"]; ok { + return true + } + } + return false +} diff --git a/plugins/terragrunt/internal/engine/generation.go b/plugins/terragrunt/internal/engine/generation.go index 79b6a3b124d..b7f700c7c77 100644 --- a/plugins/terragrunt/internal/engine/generation.go +++ b/plugins/terragrunt/internal/engine/generation.go @@ -2,7 +2,6 @@ package engine import ( "context" - "errors" "fmt" "path/filepath" @@ -13,28 +12,17 @@ import ( "github.com/cloudposse/atmos/plugins/terragrunt/internal/model" ) -const generatedPathErrorFormat = "%w: %s" - -var ( - errGeneratedPath = errors.New("unsupported generated path") - errDuplicateGeneratedPath = errors.New("duplicate generated path") - errBackendGeneratedPath = errors.New("unsupported backend generated path") - errDuplicateBackendGeneratedPath = errors.New("duplicate backend generated path") -) - func generatedFiles(ctx context.Context, logger log.Logger, cfg *config.TerragruntConfig, result *model.Resolution) error { result.GeneratedPolicies = make(map[string]string) + result.UnitFileCopies = make(map[string]bool) for _, generated := range cfg.GenerateConfigs { - if generated.Disable && (generated.IfDisabledStr == "" || generated.IfDisabledStr == "skip") { + if skipDisabledGeneration(generated.Disable, generated.IfDisabledStr) { continue } - if !filepath.IsLocal(generated.Path) || generated.Path == "." { - return fmt.Errorf(generatedPathErrorFormat, errGeneratedPath, generated.Path) + if err := validateGeneratedPath(generated.Path, result); err != nil { + return err } path := filepath.Clean(generated.Path) - if _, exists := result.GeneratedPolicies[path]; exists { - return fmt.Errorf(generatedPathErrorFormat, errDuplicateGeneratedPath, generated.Path) - } policy := generated.IfExistsStr if generated.Disable { policy = generated.IfDisabledStr @@ -45,23 +33,60 @@ func generatedFiles(ctx context.Context, logger log.Logger, cfg *config.Terragru return err } result.Generated[path] = string(content) + // Any generate block can interpolate credentials, regardless of its target. + markSensitive(result, path) } result.GeneratedPolicies[path] = policy + if !generated.Disable && declaresBackend(path, result.Generated[path]) { + // Arbitrary Terraform text has no verified state-reader identity. + return unresolvedGeneratedBackend(result.Path) + } + } + if err := unitLockFile(result); err != nil { + return err } return generatedBackend(ctx, logger, cfg, result) } +// skipDisabledGeneration omits declarations whose disabled policy leaves the file untouched. +func skipDisabledGeneration(disabled bool, policy string) bool { + return disabled && (policy == "" || policy == "skip") +} + +// validateGeneratedPath rejects escaping and duplicate targets before rendering their bytes. +func validateGeneratedPath(path string, result *model.Resolution) error { + if !filepath.IsLocal(path) || path == "." { + return &model.Problem{ + Code: model.CodeInvalidGenerate, Summary: "A `generate` block writes outside the unit's working directory", Detail: fmt.Sprintf("The `generate` path %q is absolute or escapes the working directory.", path), + Hints: []string{"Use a relative `path` without `..`, for example `path = \"provider.tf\"`."}, File: result.Path, Key: "generate", Safe: true, Err: ErrUnsupportedExecution, + } + } + if _, exists := result.GeneratedPolicies[filepath.Clean(path)]; exists { + return &model.Problem{ + Code: model.CodeInvalidGenerate, Summary: "Two `generate` blocks write the same file", Detail: fmt.Sprintf("More than one `generate` block writes %q.", path), + Hints: []string{"Give each `generate` block a distinct `path`."}, File: result.Path, Key: "generate", Safe: true, Err: ErrUnsupportedExecution, + } + } + return nil +} + func generatedBackend(ctx context.Context, logger log.Logger, cfg *config.TerragruntConfig, result *model.Resolution) error { if cfg.RemoteState == nil || cfg.RemoteState.Generate == nil { return nil } generated := cfg.RemoteState.Generate if !filepath.IsLocal(generated.Path) || generated.Path == "." { - return fmt.Errorf(generatedPathErrorFormat, errBackendGeneratedPath, generated.Path) + return &model.Problem{ + Code: model.CodeInvalidGenerate, Summary: "The `remote_state` generate block writes outside the unit's working directory", Detail: fmt.Sprintf("The `remote_state` generate path %q is absolute or escapes the working directory.", generated.Path), + Hints: []string{"Use a relative path without `..`, for example `path = \"backend.tf\"`."}, File: result.Path, Key: "remote_state.generate", Safe: true, Err: ErrUnsupportedExecution, + } } path := filepath.Clean(generated.Path) if _, exists := result.GeneratedPolicies[path]; exists { - return fmt.Errorf(generatedPathErrorFormat, errDuplicateBackendGeneratedPath, generated.Path) + return &model.Problem{ + Code: model.CodeInvalidGenerate, Summary: "A `generate` block and `remote_state` write the same file", Detail: fmt.Sprintf("A `generate` block already writes %q.", generated.Path), + Hints: []string{"Give the `remote_state` generate block a different `path`."}, File: result.Path, Key: "remote_state.generate", Safe: true, Err: ErrUnsupportedExecution, + } } content, err := atmosbridge.RenderBackend(ctx, logger, cfg.RemoteState.Config, result.Backend) if err != nil { @@ -70,5 +95,14 @@ func generatedBackend(ctx context.Context, logger log.Logger, cfg *config.Terrag result.Generated[path] = string(content) result.GeneratedPolicies[path] = generated.IfExists result.BackendGenerated = true + markSensitive(result, path) return nil } + +// markSensitive records a generated file whose bytes can hold credentials. +func markSensitive(result *model.Resolution, path string) { + if result.SensitiveGenerated == nil { + result.SensitiveGenerated = map[string]bool{} + } + result.SensitiveGenerated[path] = true +} diff --git a/plugins/terragrunt/internal/engine/generation_test.go b/plugins/terragrunt/internal/engine/generation_test.go index 1502d34d211..6230b0c14ce 100644 --- a/plugins/terragrunt/internal/engine/generation_test.go +++ b/plugins/terragrunt/internal/engine/generation_test.go @@ -1,6 +1,7 @@ package engine import ( + "fmt" "os" "path/filepath" "strings" @@ -36,8 +37,11 @@ generate "json" { require.Equal(t, "overwrite_terragrunt", result.GeneratedPolicies[filepath.Join("nested", "provider.tf")]) require.Equal(t, "remove_terragrunt", result.GeneratedPolicies["obsolete.tf"]) require.Empty(t, result.Generated["obsolete.tf"]) + require.False(t, result.SensitiveGenerated["obsolete.tf"], "removal policies have no bytes to restore") require.Equal(t, `{"a":true}`, result.Generated["data.json"]) require.Equal(t, "skip", result.GeneratedPolicies["data.json"]) + require.True(t, result.SensitiveGenerated["data.json"]) + require.True(t, result.SensitiveGenerated[filepath.Join("nested", "provider.tf")]) require.NoDirExists(t, filepath.Join(filepath.Dir(request.Path), "nested")) } @@ -123,6 +127,36 @@ func TestPreventDestroyGuardFailsClosedWhenConfigCannotBeParsed(t *testing.T) { require.NotContains(t, err.Error(), "protected by prevent_destroy", "the failure is a parse error, not a protection verdict") } +func TestPlainGenerateBlockRejectsUnverifiedBackendIdentity(t *testing.T) { + for name, test := range map[string]struct { + path, contents string + want bool + }{ + "hcl backend": {"backend.tf", "terraform {\n backend \"local\" {\n path = \"x\"\n }\n}\n", true}, + "hcl cloud block": {"cloud.tf", "terraform {\n cloud {\n organization = \"o\"\n }\n}\n", true}, + "json backend": {"backend.tf.json", `{"terraform":{"backend":{"local":{"path":"x"}}}}`, true}, + "json list form": {"backend.tf.json", `{"terraform":[{"backend":{"local":{}}}]}`, true}, + "required version": {"versions.tf", "terraform {\n required_version = \">= 1.0\"\n}\n", false}, + "provider only": {"provider.tf", "provider \"aws\" {}\n", false}, + "not terraform": {"notes.txt", "terraform { backend \"local\" {} }", false}, + "invalid hcl": {"broken.tf", "terraform { backend", false}, + "json provider": {"provider.tf.json", `{"provider":{"aws":{}}}`, false}, + } { + t.Run(name, func(t *testing.T) { + request := nativeUnit(t, "generate \"x\" {\n path = \""+test.path+"\"\n if_exists = \"overwrite\"\n disable_signature = true\n contents = <<-EOT\n"+test.contents+"\nEOT\n}\n") + result, err := Resolve(t.Context(), request) + if test.want { + require.ErrorContains(t, err, "Cannot verify the backend identity") + require.Nil(t, result) + return + } + require.NoError(t, err) + require.False(t, result.BackendGenerated) + require.True(t, result.SensitiveGenerated[test.path], "all enabled generate blocks may contain credentials") + }) + } +} + func TestPreventDestroyGuardWithCommandLocals(t *testing.T) { for _, test := range []struct { name string @@ -149,6 +183,68 @@ func TestPreventDestroyGuardWithCommandLocals(t *testing.T) { } } +func TestDisabledGenerateBlockIsNotABackend(t *testing.T) { + request := nativeUnit(t, "generate \"x\" {\n path = \"backend.tf\"\n if_exists = \"overwrite\"\n disable = true\n if_disabled = \"remove\"\n contents = \"terraform { backend \\\"local\\\" {} }\"\n}\n") + result, err := Resolve(t.Context(), request) + require.NoError(t, err) + require.False(t, result.BackendGenerated) +} + +func TestLockFileBesideTheUnitIsHandedToTheHost(t *testing.T) { + request := nativeUnit(t, "") + lock := filepath.Join(filepath.Dir(request.Path), ".terraform.lock.hcl") + t.Run("absent", func(t *testing.T) { + result, err := Resolve(t.Context(), request) + require.NoError(t, err) + require.NotContains(t, result.Generated, ".terraform.lock.hcl") + require.Empty(t, result.UnitFileCopies) + }) + t.Run("present", func(t *testing.T) { + require.NoError(t, os.WriteFile(lock, []byte("# pinned\n"), 0o600)) + result, err := Resolve(t.Context(), request) + require.NoError(t, err) + require.Equal(t, "# pinned\n", result.Generated[".terraform.lock.hcl"]) + require.Equal(t, "overwrite", result.GeneratedPolicies[".terraform.lock.hcl"]) + require.True(t, result.UnitFileCopies[".terraform.lock.hcl"]) + }) + for _, policy := range []string{"skip", "overwrite", "error", "overwrite_terragrunt"} { + t.Run("explicit generate wins with "+policy, func(t *testing.T) { + explicit := nativeUnit(t, "generate \"lock\" {\n path = \".terraform.lock.hcl\"\n if_exists = \""+policy+"\"\n disable_signature = true\n contents = \"# generated\\n\"\n}\n") + require.NoError(t, os.WriteFile(filepath.Join(filepath.Dir(explicit.Path), ".terraform.lock.hcl"), []byte("# pinned\n"), 0o600)) + result, err := Resolve(t.Context(), explicit) + require.NoError(t, err) + require.Equal(t, "# generated\n", result.Generated[".terraform.lock.hcl"]) + require.Equal(t, policy, result.GeneratedPolicies[".terraform.lock.hcl"]) + require.False(t, result.UnitFileCopies[".terraform.lock.hcl"]) + }) + } +} + +func TestProviderGenerateCredentialsAreSensitiveAcrossPolicies(t *testing.T) { + for _, policy := range []string{"overwrite", "overwrite_terragrunt", "skip", "error"} { + t.Run(policy, func(t *testing.T) { + request := nativeUnit(t, fmt.Sprintf(`locals { + credential = "provider-test-secret" +} +generate "provider" { + path = "provider.tf" + if_exists = %q + contents = <<-EOT +provider "aws" { + secret_key = "${local.credential}" +} +EOT +}`, policy)) + result, err := Resolve(t.Context(), request) + require.NoError(t, err) + require.Contains(t, result.Generated["provider.tf"], "provider-test-secret") + require.True(t, result.SensitiveGenerated["provider.tf"]) + require.False(t, result.BackendGenerated) + require.Equal(t, policy, result.GeneratedPolicies["provider.tf"]) + }) + } +} + func TestPreventDestroyGuardStaysClosedWhenSyntaxCannotDecide(t *testing.T) { for name, extra := range map[string]string{ "expression depends on a command local": "locals { flag = run_cmd(\"must-not-execute\") }\nprevent_destroy = local.flag == \"x\"", diff --git a/plugins/terragrunt/internal/engine/policies.go b/plugins/terragrunt/internal/engine/policies.go index 7569c7ed858..2dc25a5c876 100644 --- a/plugins/terragrunt/internal/engine/policies.go +++ b/plugins/terragrunt/internal/engine/policies.go @@ -19,13 +19,6 @@ import ( "github.com/cloudposse/atmos/plugins/terragrunt/internal/model" ) -var ( - errModuleSourceTransport = errors.New("native adapter does not support this module source transport") - errTerraformSource = errors.New("native adapter requires terraform.source") - errTerraformBinary = errors.New("native adaptation of terraform_binary is not implemented") - errUnsupportedSettings = errors.New("native adaptation of errors, exclude, feature flags, or IAM settings is not implemented") -) - // ErrVersionRejected reports a terraform or terragrunt version that violates a // declared version constraint. var ErrVersionRejected = errors.New("version constraint rejected") @@ -90,23 +83,23 @@ func checkVersions(ctx context.Context, request *model.Request, terraform, terra // checkUnitPolicies rejects Terragrunt semantics that the native adapter cannot // represent faithfully. Backend reads skip the version probe, which may execute // the selected binary. -func checkUnitPolicies(ctx context.Context, request *model.Request, cfg *config.TerragruntConfig) error { +func checkUnitPolicies(ctx context.Context, request *model.Request, cfg *config.TerragruntConfig, path string) error { if missingTerraformSource(cfg) { - return errTerraformSource + return missingSourceProblem(path) } if request.Phase != "backend" { if err := checkVersions(ctx, request, cfg.TerraformVersionConstraint, cfg.TerragruntVersionConstraint); err != nil { return err } } - if cfg.TerraformBinary != "" { - return errTerraformBinary + if err := rejectUnsupported(cfg, path); err != nil { + return err } // Hooks and extra arguments are adapted as execution data by unitExecution. if err := checkPreventDestroy(request, cfg); err != nil { return err } - return checkUnsupportedSettings(cfg) + return nil } // checkPreventDestroy evaluates the same effective arguments as the engine. @@ -124,25 +117,35 @@ func checkPreventDestroy(request *model.Request, cfg *config.TerragruntConfig) e return nil } -// checkUnsupportedSettings rejects error handling, exclusion, feature flag and -// IAM settings, which have no native equivalent. -func checkUnsupportedSettings(cfg *config.TerragruntConfig) error { - if cfg.Errors != nil || cfg.Exclude != nil || len(cfg.FeatureFlags) > 0 || cfg.IamRole != "" || cfg.IamWebIdentityToken != "" { - return errUnsupportedSettings - } - return nil +// localSubdirSource reports a local module path that selects a subdirectory with `//`, such as +// `../modules/sub//app`. Terragrunt copies the tree before `//`, so it is no remote transport. Only an +// explicit local spelling counts: a bare `host/org/repo//dir` is a remote shorthand. +func localSubdirSource(source string) bool { + if _, ok := copiedSourceRoot("", source); !ok { + return false + } + return filepath.IsAbs(source) || strings.HasPrefix(source, ".") } // validateRemoteSource preserves the getter URI for Atmos's source provisioner. // Registry sources need Terragrunt version selection and are not interchangeable // with direct archive or Git sources. func validateRemoteSource(source string) error { + if localSubdirSource(source) { + return nil + } for _, prefix := range []string{"git::https://", "git::http://", "git::ssh://", "git::file://", "https://", "http://"} { if strings.HasPrefix(source, prefix) { return nil } } - return errModuleSourceTransport + scheme, _, found := strings.Cut(source, "://") + return &model.Problem{ + Code: model.CodeInvalidSource, Summary: "Unsupported module source transport", + Detail: fmt.Sprintf("The module source scheme %q is not supported. Registry sources such as `tfr://` need Terragrunt's version selection and are not interchangeable with Git or archive sources.", scheme), + Hints: []string{"Use a local path, a `git::https://`, `git::ssh://`, or `https://` source, or a Git source pinned with `?ref=`."}, + Key: "terraform.source", Safe: found, Err: ErrUnsupportedExecution, + } } // effectivePolicyArguments matches engine insertion order: command environment, diff --git a/plugins/terragrunt/internal/engine/problems.go b/plugins/terragrunt/internal/engine/problems.go new file mode 100644 index 00000000000..0ba29f487a3 --- /dev/null +++ b/plugins/terragrunt/internal/engine/problems.go @@ -0,0 +1,240 @@ +package engine + +import ( + "context" + "errors" + "fmt" + "os" + "path/filepath" + "strings" + + "github.com/gruntwork-io/terragrunt/pkg/config" + "github.com/hashicorp/hcl/v2" + "github.com/spf13/afero" + + "github.com/cloudposse/atmos/plugins/terragrunt/internal/model" +) + +// readOnlyDenial is the text of the guard that stops discovery from running commands, decrypting, +// or using the network. +const readOnlyDenial = "read-only discovery cannot" + +// unsupportedSettingHints maps each setting the native adaptation rejects to its remedy. +var unsupportedSettingHints = map[string]string{ + "terraform_binary": "Select the executable with `components.terraform.command` in atmos.yaml, for example `tofu`, and remove `terraform_binary` from the unit.", + "errors": "Remove the `errors` block. Retry failing steps with a workflow step `retry:` setting instead.", + "exclude": "Remove the `exclude` block and leave the unit out with `discovery.excluded_paths` in atmos.yaml.", + "iam_role": "Remove the IAM setting and authenticate with an Atmos identity, for example `atmos terraform plan --identity `.", + "iam_web_identity_token": "Remove `iam_web_identity_token` and authenticate with an Atmos identity, for example `atmos terraform plan --identity `.", + "feature": "Remove the `feature` blocks and pass the values as Atmos stack variables instead.", +} + +// unsupportedSetting reports a Terragrunt setting the native adaptation does not implement. +func unsupportedSetting(key, file string) *model.Problem { + hint := unsupportedSettingHints[key] + problem := &model.Problem{ + Code: model.CodeUnsupportedSetting, Summary: fmt.Sprintf("Unsupported Terragrunt setting `%s`", key), + Detail: fmt.Sprintf("The native Terragrunt adaptation does not implement `%s`, so this unit cannot be planned or applied.", key), + File: file, Key: key, Safe: true, Err: ErrUnsupportedExecution, + } + if hint != "" { + problem.Hints = []string{hint} + } + return problem +} + +// problemFor classifies an evaluation error. A Problem passes through with its file filled in. +// An HCL diagnostic becomes a syntax or evaluation problem at the diagnostic's position. +func problemFor(err error, file string) error { + if err == nil { + return nil + } + var problem *model.Problem + if errors.As(err, &problem) { + if problem.File == "" { + problem.File = file + } + return problem + } + var diagnostics hcl.Diagnostics + if errors.As(err, &diagnostics) && len(diagnostics) > 0 { + return diagnosticProblem(diagnostics, file, err) + } + if errors.Is(err, context.Canceled) || errors.Is(err, context.DeadlineExceeded) { + return err + } + // Other evaluator errors can contain evaluated values, so their text stays out of the safe fields. + return &model.Problem{ + Code: model.CodeEvaluationError, Summary: "Terragrunt could not evaluate the configuration", Detail: err.Error(), File: file, Err: err, + Hints: []string{"Check the unit's configuration."}, + } +} + +// diagnosticProblem describes the first error diagnostic. A diagnostic that points at a source range +// and carries neither an expression nor an evaluation context comes from the parser or the block +// schema, so it holds no evaluated values and its detail is safe. A conversion diagnostic can lack +// an expression while it quotes an evaluated value, so a missing expression alone proves nothing. +func diagnosticProblem(diagnostics hcl.Diagnostics, file string, cause error) *model.Problem { + diagnostic := diagnostics[0] + for _, candidate := range diagnostics { + if candidate.Severity == hcl.DiagError { + diagnostic = candidate + break + } + } + problem := &model.Problem{Code: model.CodeEvaluationError, Summary: "Terragrunt configuration error: " + diagnostic.Summary, File: file, Detail: diagnostic.Detail, Safe: parserDiagnostic(diagnostic), Err: cause} + if diagnostic.Subject != nil { + if diagnostic.Subject.Filename != "" { + problem.File = diagnostic.Subject.Filename + } + problem.Line, problem.Column = diagnostic.Subject.Start.Line, diagnostic.Subject.Start.Column + } + switch { + case strings.Contains(diagnostic.Detail, readOnlyDenial): + problem.Code, problem.Safe = model.CodeEffectDenied, false + problem.Summary = "Terragrunt configuration runs a command, reads a secret, or uses the network during discovery" + problem.Hints = []string{"Discovery only reads files. Keep `run_cmd`, `sops_decrypt_file`, and other effectful functions out of the values that `terraform.source` and `dependency` blocks use; they can stay in `inputs`, which are evaluated when the unit runs."} + case problem.Safe: + problem.Code = model.CodeSyntaxError + problem.Summary = "Invalid Terragrunt configuration: " + diagnostic.Summary + problem.Hints = []string{"Fix the HCL at the reported position, then run the command again."} + default: + problem.Hints = []string{"Check the expression at the reported position."} + } + return problem +} + +// parserDiagnostic reports whether a diagnostic originates from parsing or schema decoding. Those +// diagnostics always carry the source range they describe and never an expression or evaluation +// context, which is how a diagnostic about an evaluated value is attached. +func parserDiagnostic(diagnostic *hcl.Diagnostic) bool { + return diagnostic.Expression == nil && diagnostic.EvalContext == nil && diagnostic.Subject != nil && diagnostic.Subject.Filename != "" +} + +// unsupportedHook names a rejected hook, with a custom summary and remedy. +func unsupportedHook(key, summary, detail string, hints ...string) *model.Problem { + return &model.Problem{Code: model.CodeUnsupportedHook, Summary: summary, Detail: detail, Hints: hints, Key: key, Safe: true, Err: ErrUnsupportedExecution} +} + +// rejectUnsupported fails a unit that uses a setting the native adaptation does not implement. It +// checks everything a partial or full parse decodes, in a fixed order, so the first rejection is +// stable. Hook and extra_arguments semantics are checked by executionFromConfig. +func rejectUnsupported(cfg *config.TerragruntConfig, path string) error { + switch { + case cfg.TerraformBinary != "": + return unsupportedSetting("terraform_binary", path) + case cfg.Errors != nil: + return unsupportedSetting("errors", path) + case cfg.Exclude != nil: + return unsupportedSetting("exclude", path) + case len(cfg.FeatureFlags) > 0: + return unsupportedSetting("feature", path) + case cfg.IamRole != "": + return unsupportedSetting("iam_role", path) + case cfg.IamWebIdentityToken != "": + return unsupportedSetting("iam_web_identity_token", path) + } + if cfg.Terraform != nil { + if _, err := executionFromConfig(cfg.Terraform); err != nil { + return hookProblem(err, path) + } + } + return nil +} + +func workspaceDependencyProblem(path, workspace string) *model.Problem { + return &model.Problem{ + Code: model.CodeWorkspace, Summary: "Dependency outputs are not supported with a non-default Terraform workspace", + Detail: fmt.Sprintf("TF_WORKSPACE is `%s` and this unit declares a `dependency` block. Reading dependency outputs from a non-default workspace needs a dependency backend adapter that does not exist yet.", workspace), + Hints: []string{"Unset `TF_WORKSPACE` for this command, or run the unit's dependencies and this unit in the default workspace."}, + File: path, Key: "dependency", Safe: true, Err: ErrUnsupportedExecution, + } +} + +func missingSourceProblem(path string) *model.Problem { + return &model.Problem{ + Code: model.CodeMissingSource, Summary: "The unit has no `terraform.source`", + Detail: "Native Terragrunt support runs units that name a module with `terraform { source = ... }`. A unit with inline Terraform files is not supported.", + Hints: []string{"Add `terraform { source = \"...\" }` to the unit, or move the inline Terraform files into a module directory and reference it."}, + File: path, Key: "terraform.source", Safe: true, Err: ErrUnsupportedExecution, + } +} + +// sourceDirectoryProblem keeps the evaluated source out of the shown detail, because it can hold a +// value that is not registered for masking. The cause still carries it for local debugging. +func sourceDirectoryProblem(path, source string) *model.Problem { + return &model.Problem{ + Code: model.CodeInvalidSource, Summary: "The module `source` is not an existing local directory", + Detail: "`terraform.source` resolves to a local path that does not exist or is not a directory.", + Hints: []string{"Point `terraform.source` at a module directory relative to the unit, or use a Git or HTTPS module source."}, + File: path, Key: "terraform.source", Safe: true, Err: fmt.Errorf("%w: local source %q is not a directory", ErrUnsupportedExecution, source), + } +} + +// hookProblem keeps the hook diagnostics from executionFromConfig, which name hooks but no values. +func hookProblem(err error, path string) error { + var problem *model.Problem + if errors.As(err, &problem) { + if problem.File == "" { + problem.File = path + } + return problem + } + return err +} + +func mustRel(root, path string) string { + if rel, err := filepath.Rel(root, path); err == nil { + return rel + } + return path +} + +// localSourceProblem checks that a local module source is an existing directory. A missing path or +// a file is reported as such; any other failure, such as a permission error, is reported as an +// unreadable source and keeps its cause. +func localSourceProblem(filesystem afero.Fs, path, source, resolved string) *model.Problem { + info, err := filesystem.Stat(resolved) + switch { + case errors.Is(err, os.ErrNotExist), err == nil && !info.IsDir(): + return sourceDirectoryProblem(path, source) + case err != nil: + return &model.Problem{ + Code: model.CodeInvalidSource, Summary: "The module `source` directory cannot be read", + Detail: "Checking the local `terraform.source` directory failed with a filesystem error other than a missing path.", + Hints: []string{"Check the permissions of the module directory and its parent directories."}, + File: path, Key: "terraform.source", Safe: true, Err: err, + } + } + return nil +} + +// dependencyProblem keeps evaluated target paths out of safe details; the cause remains available +// for classification and local debugging. +func dependencyProblem(file, _ string, err error) *model.Problem { + summary := "A dependency points at a unit that cannot be read" + hint := "Create the target unit, or fix `config_path` in the `dependency` or `dependencies` block." + if errors.Is(err, os.ErrNotExist) { + summary = "A dependency points at a path that does not exist" + hint = "The target may have been deleted or renamed. Fix `config_path` in the `dependency` or `dependencies` block, or remove the dependency." + } + return &model.Problem{Code: model.CodeDependency, Summary: summary, Detail: "The dependency target could not be read.", Hints: []string{hint}, File: file, Key: "dependency", Safe: true, Err: err} +} + +// localSubdirSourceProblem checks a local `dir//subdir` module source. Terragrunt copies the tree +// before `//` and runs the subdirectory, so both must exist; each is reported like a plain local +// source. A source without a local copied tree has nothing to check. +func localSubdirSourceProblem(filesystem afero.Fs, path, source string) *model.Problem { + root, ok := copiedSourceRoot(filepath.Dir(path), source) + if !ok { + return nil + } + if problem := localSourceProblem(filesystem, path, source, root); problem != nil { + return problem + } + _, subdirectory, _ := strings.Cut(filepath.ToSlash(source), "//") + if subdirectory == "" { + return nil + } + return localSourceProblem(filesystem, path, source, filepath.Join(root, filepath.FromSlash(subdirectory))) +} diff --git a/plugins/terragrunt/internal/engine/problems_test.go b/plugins/terragrunt/internal/engine/problems_test.go new file mode 100644 index 00000000000..f12cbd741ab --- /dev/null +++ b/plugins/terragrunt/internal/engine/problems_test.go @@ -0,0 +1,229 @@ +package engine + +import ( + "context" + "encoding/json" + "errors" + "io/fs" + "os" + "path/filepath" + "strings" + "testing" + + "github.com/hashicorp/hcl/v2" + "github.com/spf13/afero" + "github.com/stretchr/testify/require" + + "github.com/cloudposse/atmos/plugins/terragrunt/internal/model" +) + +func asProblem(t *testing.T, err error) *model.Problem { + t.Helper() + var problem *model.Problem + require.ErrorAs(t, err, &problem) + return problem +} + +// Every unsupported setting is rejected during discovery, so a bulk run can validate each selected +// unit before any infrastructure command runs. +func TestDiscoveryRejectsUnsupportedSettingsNamingTheKey(t *testing.T) { + for name, test := range map[string]struct{ extra, key string }{ + "terraform_binary": {`terraform_binary = "tofu"`, "terraform_binary"}, + "errors": {"errors {\n retry \"x\" {\n retryable_errors = [\".*\"]\n max_attempts = 2\n sleep_interval_sec = 1\n }\n}", "errors"}, + "exclude": {"exclude {\n if = true\n actions = [\"plan\"]\n}", "exclude"}, + "iam": {`iam_role = "arn:aws:iam::1:role/x"`, "iam_role"}, + "web identity": {`iam_web_identity_token = "token"`, "iam_web_identity_token"}, + "feature": {"feature \"flag\" {\n default = true\n}", "feature"}, + } { + t.Run(name, func(t *testing.T) { + request := nativeUnit(t, test.extra) + request.Phase = "discover" + _, err := Resolve(t.Context(), request) + problem := asProblem(t, err) + require.Equal(t, model.CodeUnsupportedSetting, problem.Code) + require.Equal(t, test.key, problem.Key) + require.Equal(t, "terragrunt.hcl", filepath.Base(problem.File)) + require.True(t, problem.Safe) + require.NotEmpty(t, problem.Hints) + }) + } +} + +func TestDiscoveryRejectsUnsupportedHooks(t *testing.T) { + request := nativeUnit(t, "") + body, err := os.ReadFile(request.Path) + require.NoError(t, err) + hooked := strings.Replace(string(body), "{ source", "{\n source", 1) + hooked = strings.Replace(hooked, " }\n", "\n before_hook \"lint\" {\n commands = [\"plan\"]\n execute = [\"tflint\"]\n }\n}\n", 1) + require.NoError(t, os.WriteFile(request.Path, []byte(hooked), 0o644)) + request.Phase = "discover" + _, err = Resolve(t.Context(), request) + problem := asProblem(t, err) + require.Equal(t, model.CodeUnsupportedHook, problem.Code) + require.Contains(t, problem.Key, "lint") + require.Equal(t, "terragrunt.hcl", filepath.Base(problem.File)) +} + +func TestSyntaxErrorNamesFileLineAndColumnAndStaysSafe(t *testing.T) { + request := nativeUnit(t, "inputs = {\n a = find_in_parent_folders(\"x\"\n}\n") + request.Phase = "discover" + _, err := Resolve(t.Context(), request) + problem := asProblem(t, err) + require.Equal(t, model.CodeSyntaxError, problem.Code) + require.Equal(t, "terragrunt.hcl", filepath.Base(problem.File)) + require.Positive(t, problem.Line) + require.Positive(t, problem.Column) + require.True(t, problem.Safe, "parser diagnostics hold no evaluated values") + require.Contains(t, problem.Summary, "Missing argument separator") +} + +func TestEvaluationErrorKeepsEvaluatedTextHidden(t *testing.T) { + request := nativeUnit(t, "inputs = {\n a = tonumber(\"hunter2\")\n}\n") + _, err := Resolve(t.Context(), request) + problem := asProblem(t, err) + require.False(t, problem.Safe) + require.NotContains(t, problem.Summary, "hunter2") + for _, hint := range problem.Hints { + require.NotContains(t, hint, "hunter2") + } + require.Positive(t, problem.Line) +} + +func TestMissingDependencyTargetStaysHidden(t *testing.T) { + request := nativeUnit(t, "dependency \"gone\" {\n config_path = get_env(\"DEPENDENCY_TARGET\")\n}\n") + request.Phase = "discover" + request.Environment["DEPENDENCY_TARGET"] = "./secret-dependency-target" + _, err := Resolve(t.Context(), request) + problem := asProblem(t, err) + require.Equal(t, model.CodeDependency, problem.Code) + require.True(t, problem.Safe) + require.NotContains(t, problem.Error(), "secret-dependency-target") + expectedFile, err := filepath.EvalSymlinks(request.Path) + require.NoError(t, err) + require.Equal(t, expectedFile, problem.File) + require.NotEmpty(t, problem.Hints) + require.ErrorIs(t, problem, os.ErrNotExist) + require.Equal(t, "dependency", problem.Key) +} + +func TestProblemForPassesCancellationAndWrapsUnknownErrors(t *testing.T) { + wrapped := problemFor(errors.New("secret detail"), filepath.Join("a", "b.hcl")) + problem := asProblem(t, wrapped) + require.False(t, problem.Safe) + require.Equal(t, model.CodeEvaluationError, problem.Code) + require.NoError(t, problemFor(nil, "f")) + require.ErrorIs(t, problemFor(errContextCanceled(), "f"), errContextCanceled()) +} + +func errContextCanceled() error { return context.Canceled } + +// A missing local module is rejected during discovery, so bulk preflight stops before any unit +// runs, and the shown detail never repeats the evaluated source. +func TestMissingLocalSourceFailsDiscoveryWithoutShowingTheSource(t *testing.T) { + for _, phase := range []string{"discover", ""} { + t.Run("phase="+phase, func(t *testing.T) { + request := nativeUnit(t, "") + missing := filepath.Join(filepath.Dir(request.Path), "secret-token-value") + source, err := json.Marshal(filepath.ToSlash(missing)) + require.NoError(t, err) + require.NoError(t, os.WriteFile(request.Path, []byte("terraform { source = "+string(source)+" }\n"), 0o644)) + request.Phase = phase + _, err = Resolve(t.Context(), request) + problem := asProblem(t, err) + require.Equal(t, model.CodeInvalidSource, problem.Code) + require.Equal(t, "terraform.source", problem.Key) + require.True(t, problem.Safe) + require.NotContains(t, problem.Detail, "secret-token-value") + require.NotContains(t, problem.Summary, "secret-token-value") + require.ErrorIs(t, err, ErrUnsupportedExecution) + }) + } +} + +// statFailureFs fails every Stat with a fixed error, as a directory without search permission would. +type statFailureFs struct { + afero.Fs + err error +} + +func (f statFailureFs) Stat(string) (os.FileInfo, error) { return nil, f.err } + +// A local module source that is missing or a file is an invalid source; any other filesystem failure +// is reported as unreadable and keeps its cause instead of being mistaken for a missing directory. +func TestLocalSourceProblemDistinguishesFilesystemFailures(t *testing.T) { + memory := afero.NewMemMapFs() + require.NoError(t, memory.MkdirAll("/project/module", 0o755)) + require.NoError(t, afero.WriteFile(memory, "/project/file.tf", nil, 0o644)) + require.Nil(t, localSourceProblem(memory, "/project/terragrunt.hcl", "./module", "/project/module")) + + for _, resolved := range []string{"/project/absent", "/project/file.tf"} { + problem := localSourceProblem(memory, "/project/terragrunt.hcl", "./x", resolved) + require.NotNil(t, problem, resolved) + require.Equal(t, "The module `source` is not an existing local directory", problem.Summary, resolved) + } + + problem := localSourceProblem(statFailureFs{Fs: memory, err: fs.ErrPermission}, "/project/terragrunt.hcl", "./module", "/project/module") + require.NotNil(t, problem) + require.Equal(t, model.CodeInvalidSource, problem.Code) + require.Equal(t, "The module `source` directory cannot be read", problem.Summary) + require.ErrorIs(t, problem, fs.ErrPermission, "the filesystem cause is preserved") + require.True(t, problem.Safe) + require.NotContains(t, problem.Detail, "/project", "the safe detail names no path") +} + +// A diagnostic without an expression is safe only when it points at a source range, as parser and +// schema diagnostics do. A conversion diagnostic can lack an expression and still quote a value. +func TestDiagnosticProblemTrustsOnlyParserAndSchemaDiagnostics(t *testing.T) { + located := &hcl.Range{Filename: "terragrunt.hcl", Start: hcl.Pos{Line: 3, Column: 5}} + for _, test := range []struct { + name string + diagnostic *hcl.Diagnostic + safe bool + }{ + {"parser or schema diagnostic", &hcl.Diagnostic{Severity: hcl.DiagError, Summary: "Unsupported argument", Detail: "An argument named \"x\" is not expected here.", Subject: located}, true}, + {"conversion diagnostic without an expression", &hcl.Diagnostic{Severity: hcl.DiagError, Summary: "Unsuitable value", Detail: "Unsuitable value: hunter2", Subject: located, EvalContext: &hcl.EvalContext{}}, false}, + {"diagnostic without a source range", &hcl.Diagnostic{Severity: hcl.DiagError, Summary: "Unsuitable value", Detail: "Unsuitable value: hunter2"}, false}, + } { + t.Run(test.name, func(t *testing.T) { + problem := diagnosticProblem(hcl.Diagnostics{test.diagnostic}, "terragrunt.hcl", test.diagnostic) + require.Equal(t, test.safe, problem.Safe) + if !test.safe { + require.NotEqual(t, model.CodeSyntaxError, problem.Code) + } + }) + } +} + +// A local `dir//subdir` module source is checked during discovery like a plain local source: both the +// copied tree and the requested subdirectory must exist, and an existing pair passes. +func TestLocalSubdirectorySourceMustExistDuringDiscovery(t *testing.T) { + for _, test := range []struct { + name string + source string + valid bool + }{ + {"missing copied tree", "./absent//app", false}, + {"missing subdirectory", "./modules//absent", false}, + {"subdirectory is a file", "./modules//file.tf", false}, + {"existing subdirectory", "./modules//app", true}, + } { + t.Run(test.name, func(t *testing.T) { + request := nativeUnit(t, "") + root := filepath.Dir(request.Path) + require.NoError(t, os.MkdirAll(filepath.Join(root, "modules", "app"), 0o755)) + require.NoError(t, os.WriteFile(filepath.Join(root, "modules", "file.tf"), nil, 0o644)) + require.NoError(t, os.WriteFile(request.Path, []byte("terraform { source = \""+test.source+"\" }\n"), 0o644)) + request.Phase = "discover" + _, err := Resolve(t.Context(), request) + if test.valid { + require.NoError(t, err) + return + } + problem := asProblem(t, err) + require.Equal(t, model.CodeInvalidSource, problem.Code) + require.Equal(t, "The module `source` is not an existing local directory", problem.Summary) + require.True(t, problem.Safe) + require.ErrorIs(t, err, ErrUnsupportedExecution) + }) + } +} diff --git a/plugins/terragrunt/internal/engine/resolve.go b/plugins/terragrunt/internal/engine/resolve.go index 54710922458..09adf866763 100644 --- a/plugins/terragrunt/internal/engine/resolve.go +++ b/plugins/terragrunt/internal/engine/resolve.go @@ -2,9 +2,9 @@ package engine import ( + "bytes" "context" "errors" - "fmt" "io" "os" "path/filepath" @@ -25,8 +25,6 @@ import ( "github.com/cloudposse/atmos/plugins/terragrunt/internal/model" ) -var errSourceDirectory = errors.New("native adapter source must be an existing local directory") - // Resolve evaluates HCL using the pinned Terragrunt parser. // // The public value API isolates invocation fields; this SDK-only plugin has no host tracker dependency. @@ -39,7 +37,16 @@ func Resolve(ctx context.Context, request model.Request) (*model.Resolution, err // ErrPreventDestroy rejects a destroy operation on a unit that sets prevent_destroy. var ErrPreventDestroy = errors.New("unit is protected by prevent_destroy") +// resolve preserves the structured L6 error boundary around invocation-local evaluation. func resolve(ctx context.Context, request *model.Request, filesystem afero.Fs) (*model.Resolution, error) { + result, err := resolveRequest(ctx, request, filesystem) + if err != nil { + return nil, problemFor(err, request.Path) + } + return result, nil +} + +func resolveRequest(ctx context.Context, request *model.Request, filesystem afero.Fs) (*model.Resolution, error) { if request.Phase == "inventory" { return discoverProject(ctx, request) } @@ -91,7 +98,7 @@ func (e *unitEvaluator) resolveUnit(ctx context.Context) (*model.Resolution, err if err != nil { return nil, err } - if err := checkUnitPolicies(ctx, e.request, cfg); err != nil { + if err := checkUnitPolicies(ctx, e.request, cfg, e.path); err != nil { return nil, err } execution, applyFiles, err := e.unitExecution(ctx, cfg) @@ -148,12 +155,13 @@ func backendGenerationOverlay(ctx context.Context, request *model.Request, files // requestPath uses one physical path identity for parsing and CLI execution. On // macOS, mixing /var and /private/var breaks upstream diamond dependency reads. +// Like upstream Terragrunt, a symlinked terragrunt.hcl stays a unit where the link lives. func requestPath(path string) (string, error) { path, err := filepath.Abs(strings.TrimPrefix(path, "terragrunt://")) if err != nil { return "", err } - return canonicalPath(path) + return logicalPath(path) } func evaluationLogger(level string) (log.Logger, error) { @@ -314,7 +322,11 @@ func (e *unitEvaluator) parseUnit(ctx context.Context) (*config.TerragruntConfig if err != nil { return nil, err } - if e.request.Environment[downloadDirEnv] == "" { + // Reject unverifiable backend generation before the full download_dir probe. + if err := checkBackendGeneration(environment.FS, append(e.pctx.FilesRead.Paths(), e.path)); err != nil { + return nil, err + } + if e.request.Environment[downloadDirEnv] == "" && mentionsDownloadDir(environment.FS, append(e.pctx.FilesRead.Paths(), e.path)) { cfg.DownloadDir, err = downloadDir(ctx, e.logger, environment, e.pctx) } return cfg, err @@ -341,6 +353,18 @@ func downloadDir(ctx context.Context, logger log.Logger, environment *atmosbridg return full.DownloadDir, nil } +// mentionsDownloadDir reports whether any parsed configuration file names download_dir. The probe +// in downloadDir parses the whole unit, so a unit that never sets it skips the probe and its +// inputs stay unevaluated during backend reads. +func mentionsDownloadDir(filesystem afero.Fs, paths []string) bool { + for _, path := range paths { + if content, err := afero.ReadFile(filesystem, path); err == nil && bytes.Contains(content, []byte("download_dir")) { + return true + } + } + return false +} + // withoutInputsFS projects inputs to the keys referenced by the configuration. // References are shared across reads so read_terragrunt_config can supply inputs // needed by download_dir without evaluating unrelated input expressions. @@ -545,8 +569,8 @@ func (e *unitEvaluator) unitResolution(cfg *config.TerragruntConfig) (*model.Res result := &model.Resolution{Path: e.path, Inputs: cfg.Inputs, Source: *cfg.Terraform.Source, Generated: map[string]string{}, Files: append(e.pctx.FilesRead.Paths(), e.path)} if plainLocalSource(result.Source) { result.Source = localPath(filepath.Dir(e.path), result.Source) - if info, err := os.Stat(result.Source); err != nil || !info.IsDir() { - return nil, fmt.Errorf("%w: %s", errSourceDirectory, result.Source) + if problem := localSourceProblem(e.environment.FS, e.path, *cfg.Terraform.Source, result.Source); problem != nil { + return nil, problem } result.Folders = []string{result.Source} } else if err := validateRemoteSource(result.Source); err != nil { diff --git a/plugins/terragrunt/internal/engine/resolve_test.go b/plugins/terragrunt/internal/engine/resolve_test.go index 40be1de2d6e..25ff65c84af 100644 --- a/plugins/terragrunt/internal/engine/resolve_test.go +++ b/plugins/terragrunt/internal/engine/resolve_test.go @@ -19,7 +19,12 @@ import ( func nativeUnit(t *testing.T, extra string) model.Request { t.Helper() - root := t.TempDir() + return nativeUnitAt(t, t.TempDir(), extra) +} + +// nativeUnitAt writes the native unit fixture below root, which the caller owns. +func nativeUnitAt(t *testing.T, root, extra string) model.Request { + t.Helper() module := filepath.Join(root, "module") require.NoError(t, os.MkdirAll(module, 0o755)) source, err := json.Marshal(filepath.ToSlash(module)) @@ -53,13 +58,13 @@ inputs = { large = 9007199254740993, nested = { nullable = null, enabled = false func TestNativeRejectsUnsupportedSemantics(t *testing.T) { for _, test := range []struct{ name, extra, message string }{ {"binary override", `terraform_binary = "another-tool"`, "terraform_binary"}, - {"IAM role", `iam_role = "unsupported"`, "IAM settings"}, + {"IAM role", `iam_role = "unsupported"`, "iam_role"}, {"generation escape", `generate "x" { path = "../x.tf" if_exists = "overwrite" disable_signature = true contents = "" -}`, "unsupported generated path"}, +}`, "escapes the working directory"}, {"version constraint", `terragrunt_version_constraint = "> 99.0.0"`, "rejects version"}, } { t.Run(test.name, func(t *testing.T) { @@ -101,7 +106,11 @@ inputs = { value = local.value } `) request.Phase = "discover" _, err := Resolve(t.Context(), request) - require.ErrorContains(t, err, "read-only discovery cannot execute") + var problem *model.Problem + require.ErrorAs(t, err, &problem) + require.Equal(t, model.CodeEffectDenied, problem.Code) + require.False(t, problem.Safe) + require.NotEmpty(t, problem.Hints) } func TestVersionConstraintsUseSelectedExecutable(t *testing.T) { @@ -160,7 +169,7 @@ generate "disabled" { require.Error(t, err) require.NoError(t, os.WriteFile(request.Path, []byte("inputs = {}"), 0o644)) _, err = Resolve(t.Context(), request) - require.ErrorContains(t, err, "requires terraform.source") + require.ErrorContains(t, err, "terraform.source") for _, source := range []string{"tfr://example.invalid/repo", "./absent"} { encoded, err := json.Marshal(source) require.NoError(t, err) @@ -170,6 +179,46 @@ generate "disabled" { } } +// Discovery validates the module source transport, so an unsupported remote source fails before any +// infrastructure command runs rather than only when the unit is prepared. A local source that +// selects a subdirectory with `//` is no remote transport and must not be rejected as one. +func TestDiscoveryValidatesRemoteModuleSources(t *testing.T) { + for _, test := range []struct { + source string + invalid bool + }{ + {source: "tfr://example.invalid/repo", invalid: true}, + {source: "git::https://example.invalid/modules.git//vpc?ref=v1.0.0"}, + {source: "https://example.invalid/module.zip"}, + // A local module with a `//` subdirectory is not a remote source: Terragrunt copies the tree before it. + {source: "../../modules/sub//app"}, + // A bare host path with `//` stays a remote shorthand that discovery cannot interpret. + {source: "example.invalid/org/modules//app", invalid: true}, + } { + t.Run(test.source, func(t *testing.T) { + // The unit sits two levels below the test's own root, so the relative local source resolves + // inside that root, and the module it names exists there. + top := t.TempDir() + unit := filepath.Join(top, "stack", "unit") + require.NoError(t, os.MkdirAll(unit, 0o755)) + require.NoError(t, os.MkdirAll(filepath.Join(top, "modules", "sub", "app"), 0o755)) + request := nativeUnitAt(t, unit, "") + encoded, err := json.Marshal(test.source) + require.NoError(t, err) + require.NoError(t, os.WriteFile(request.Path, []byte("terraform { source = "+string(encoded)+" }"), 0o644)) + request.Phase = "discover" + _, err = Resolve(t.Context(), request) + if !test.invalid { + require.NoError(t, err) + return + } + var problem *model.Problem + require.ErrorAs(t, err, &problem) + require.Equal(t, model.CodeInvalidSource, problem.Code) + }) + } +} + func TestResolvePreservesUnitOverlay(t *testing.T) { request := nativeUnit(t, "") directory := filepath.Dir(request.Path) @@ -192,6 +241,25 @@ func TestResolvePreservesUnitOverlay(t *testing.T) { require.NotContains(t, result.Overlay, "override.tf") } +// The unit path is the root of copy-rule globs, so a glob metacharacter in it (or in the name of +// any test that owns a temp directory) must not widen the overlay read to the parent directory. +func TestResolveUnitInDirectoryWithGlobMetacharacters(t *testing.T) { + base := t.TempDir() + locked := filepath.Join(base, "locked") + root := filepath.Join(base, "unit{with-brace") + require.NoError(t, os.MkdirAll(root, 0o755)) + require.NoError(t, os.MkdirAll(locked, 0o755)) + request := nativeUnitAt(t, root, "") + require.NoError(t, os.WriteFile(filepath.Join(root, ".tflint.hcl"), []byte("# lint"), 0o644)) + if runtime.GOOS != "windows" && os.Geteuid() != 0 { + require.NoError(t, os.Chmod(locked, 0)) + t.Cleanup(func() { _ = os.Chmod(locked, 0o755) }) + } + result, err := Resolve(t.Context(), request) + require.NoError(t, err) + require.Equal(t, []byte("# lint"), result.Overlay[".tflint.hcl"]) +} + func TestRejectRemoteStateEncryption(t *testing.T) { request := nativeUnit(t, `remote_state { backend = "local" @@ -267,24 +335,26 @@ remote_state { } func TestOverlayRejectsLockFilesAndCarriesModes(t *testing.T) { - t.Run("committed dependency lock file", func(t *testing.T) { + t.Run("lock file beside the unit is a unit file copy, not an overlay file", func(t *testing.T) { request := nativeUnit(t, "") require.NoError(t, os.WriteFile(filepath.Join(filepath.Dir(request.Path), ".terraform.lock.hcl"), []byte("# lock"), 0o644)) + result, err := Resolve(t.Context(), request) + require.NoError(t, err) + require.NotContains(t, result.Overlay, ".terraform.lock.hcl") + require.Equal(t, "# lock", result.Generated[".terraform.lock.hcl"]) + require.True(t, result.UnitFileCopies[".terraform.lock.hcl"]) + }) + t.Run("nested dependency lock file", func(t *testing.T) { + request := nativeUnit(t, "") + nested := filepath.Join(filepath.Dir(request.Path), "sub") + require.NoError(t, os.MkdirAll(nested, 0o755)) + require.NoError(t, os.WriteFile(filepath.Join(nested, ".terraform.lock.hcl"), []byte("# lock"), 0o644)) _, err := Resolve(t.Context(), request) require.ErrorIs(t, err, atmosbridge.ErrUnsupportedOverlay) require.ErrorContains(t, err, ".terraform.lock.hcl") request.Phase = "backend" _, err = Resolve(t.Context(), request) require.NoError(t, err, "backend reads never export the overlay") - request.Phase = "" - data, err := os.ReadFile(request.Path) - require.NoError(t, err) - excluded := strings.Replace(string(data), "terraform { source =", "terraform {\n exclude_from_copy = [\".terraform.lock.hcl\"]\n source =", 1) - excluded = strings.Replace(excluded, " }", "\n}", 1) - require.NoError(t, os.WriteFile(request.Path, []byte(excluded), 0o644)) - result, err := Resolve(t.Context(), request) - require.NoError(t, err) - require.NotContains(t, result.Overlay, ".terraform.lock.hcl") }) t.Run("executable file keeps its mode", func(t *testing.T) { if runtime.GOOS == "windows" { diff --git a/plugins/terragrunt/internal/engine/source_watches.go b/plugins/terragrunt/internal/engine/source_watches.go index 56eabce36ae..73084aa6257 100644 --- a/plugins/terragrunt/internal/engine/source_watches.go +++ b/plugins/terragrunt/internal/engine/source_watches.go @@ -1,13 +1,14 @@ package engine import ( - "errors" + "cmp" "fmt" - "os" + "maps" "path/filepath" "slices" "strings" + "github.com/gruntwork-io/terragrunt/pkg/config" "github.com/hashicorp/hcl/v2" "github.com/hashicorp/hcl/v2/hclsyntax" "github.com/spf13/afero" @@ -16,89 +17,539 @@ import ( "github.com/cloudposse/atmos/plugins/terragrunt/internal/model" ) -var errTrackSources = errors.New("cannot track configuration sources in") +// lockFileName is the provider lock file Terragrunt keeps beside a unit. +const lockFileName = ".terraform.lock.hcl" + +const hclExtension = ".hcl" + +// fileScan is the evaluation scope of one HCL file being inspected. +type fileScan struct { + // base resolves relative paths named by file functions in this file. + base string + // unit is the directory get_terragrunt_dir() reports while this file is evaluated. + unit string + ctx *hcl.EvalContext +} + +// queuedFile is an HCL file to inspect. Files the unit evaluated must exist; +// files discovered through read_terragrunt_config are best effort. +type queuedFile struct { + path string + required bool + // unit overrides the evaluation directory of an optional file. A configuration included by a + // file read through read_terragrunt_config evaluates in the including file's scope. + unit string +} + +// watchTracker collects everything a unit reads that partial discovery does not +// evaluate: input expressions, generate blocks, extra arguments, and hooks. +type watchTracker struct { + watchEvaluator + result *model.Resolution + files map[string]struct{} + folders map[string]struct{} + visited map[[2]string]struct{} + queue []queuedFile +} // trackInputSources inspects syntax, not values. Partial discovery intentionally // leaves inputs unevaluated; syntax watches include conditional and optional reads -// without reading state, decrypting secrets, or invoking user commands. -// -// Configuration that inputs read (read_terragrunt_config) is scanned the same way, transitively, -// so a change to its own inputs' sources affects the unit. Each file is scanned once. +// without reading state, decrypting secrets, or invoking user commands. Paths are +// resolved with a pure expression subset (see watchEvaluator) so the common +// get_terragrunt_dir() idioms name exact files; only genuinely unresolvable +// expressions watch the whole project. func trackInputSources(fs afero.Fs, root string, result *model.Resolution) error { - evaluated := len(result.Files) - scanned := map[string]bool{} - for i := 0; i < len(result.Files); i++ { - path := result.Files[i] - if scanned[path] || !strings.HasSuffix(path, ".hcl") { - continue + tracker := newWatchTracker(fs, root, result) + if err := tracker.scan(); err != nil { + return err + } + tracker.addStaticWatches() + tracker.merge() + return nil +} + +func newWatchTracker(fs afero.Fs, root string, result *model.Resolution) *watchTracker { + tracker := &watchTracker{ + watchEvaluator: watchEvaluator{fs: fs, root: root, unit: unitDirectory(result), original: unitDirectory(result)}, + result: result, + files: map[string]struct{}{}, + folders: map[string]struct{}{}, + visited: map[[2]string]struct{}{}, + } + if contents, err := afero.ReadFile(fs, tracker.unitFile()); err == nil { + if file, diagnostics := hclsyntax.ParseConfig(contents, tracker.unitFile(), hcl.InitialPos); !diagnostics.HasErrors() { + tracker.parent = tracker.includeParent(file.Body.(*hclsyntax.Body)) } - scanned[path] = true - contents, err := afero.ReadFile(fs, path) - if err != nil { - // A file found only by this scan may be an optional read that does not exist yet; - // it stays a watch. Every evaluated file must be readable. - if i >= evaluated && errors.Is(err, os.ErrNotExist) { - continue - } - return fmt.Errorf("track configuration sources %s: %w", path, err) + } + return tracker +} + +// unitDirectory is the directory get_terragrunt_dir() reports. +func unitDirectory(result *model.Resolution) string { + if result.Path != "" { + return filepath.Dir(result.Path) + } + for _, path := range result.Files { + if strings.HasSuffix(path, hclExtension) { + return filepath.Dir(path) } - if err := scanInputSources(root, path, contents, result); err != nil { + } + return "." +} + +func (t *watchTracker) unitFile() string { + if t.result.Path != "" { + return t.result.Path + } + return filepath.Join(t.unit, "terragrunt.hcl") +} + +func (t *watchTracker) scan() error { + for _, path := range t.result.Files { + // Lock files are watched, never inspected: they hold no expressions. + if strings.HasSuffix(path, hclExtension) && filepath.Base(path) != lockFileName { + t.queue = append(t.queue, queuedFile{path: path, required: true}) + } + } + for len(t.queue) > 0 { + next := t.queue[0] + t.queue = t.queue[1:] + scope := t.unit + if !next.required { + scope = cmp.Or(next.unit, filepath.Dir(next.path)) + } + key := [2]string{next.path, scope} + if _, seen := t.visited[key]; seen { + continue + } + t.visited[key] = struct{}{} + if err := t.scanFile(next); err != nil { return err } } - result.Files = slices.Compact(slices.Sorted(slices.Values(result.Files))) - result.Folders = slices.Compact(slices.Sorted(slices.Values(result.Folders))) return nil } -// scanInputSources adds syntax-derived watches while retaining non-HCL data files. -func scanInputSources(root, path string, contents []byte, result *model.Resolution) error { - file, diagnostics := hclsyntax.ParseConfig(contents, path, hcl.InitialPos) +func (t *watchTracker) scanFile(next queuedFile) error { + contents, err := afero.ReadFile(t.fs, next.path) + if err != nil { + return optionalFailure(next, fmt.Errorf("track configuration sources %s: %w", next.path, err)) + } + file, diagnostics := hclsyntax.ParseConfig(contents, next.path, hcl.InitialPos) if diagnostics.HasErrors() { // Files also lists data files that functions read, and a data file may be named // *.hcl without being HCL. Every configuration file already parsed during // evaluation, so one that does not parse is data: it stays a file watch. return nil } - inputs := file.Body.(*hclsyntax.Body).Attributes["inputs"] - if inputs == nil { + body := file.Body.(*hclsyntax.Body) + // Terragrunt evaluates a file read with read_terragrunt_config as that file's own + // configuration, so get_terragrunt_dir() there names the read file's directory. + // The unit's own files and includes keep the unit's directory. + evaluator := t.watchEvaluator + if !next.required { + evaluator.unit = cmp.Or(next.unit, filepath.Dir(next.path)) + evaluator.parent = evaluator.includeParent(body) + } + scan := fileScan{base: filepath.Dir(next.path), unit: evaluator.unit, ctx: evaluator.context(evaluator.locals(body))} + if inputs := body.Attributes["inputs"]; inputs != nil { + t.visit(scan, inputs.Expr) + } + for _, block := range body.Blocks { + t.scanBlock(scan, block, next.required) + } + return nil +} + +// optionalFailure ignores problems in files that were only discovered through +// read_terragrunt_config; the unit's own files must be inspectable. +func optionalFailure(next queuedFile, err error) error { + if !next.required { return nil } - walk := hclsyntax.VisitAll(inputs.Expr, func(node hclsyntax.Node) hcl.Diagnostics { + return err +} + +func (t *watchTracker) scanBlock(scan fileScan, block *hclsyntax.Block, required bool) { + switch block.Type { + case "generate", "remote_state": + t.visit(scan, block) + case "terraform": + t.scanTerraform(scan, block) + case "include": + // Discovery already followed the unit's own includes; the includes of a file reached + // through read_terragrunt_config are followed here. + if !required { + t.followInclude(scan, block) + } + case "locals": + // Discovery already evaluated the unit's own locals; files reached through + // read_terragrunt_config are only inspected here. + if !required { + t.visit(scan, block) + } + } +} + +func (t *watchTracker) visit(scan fileScan, node hclsyntax.Node) { + _ = hclsyntax.VisitAll(node, func(node hclsyntax.Node) hcl.Diagnostics { if call, ok := node.(*hclsyntax.FunctionCallExpr); ok { - trackSourceCall(root, filepath.Dir(path), call, result) + t.trackCall(scan, call) } return nil }) - if walk.HasErrors() { - return fmt.Errorf("%w %s", errTrackSources, path) - } - return nil } -func trackSourceCall(root, directory string, call *hclsyntax.FunctionCallExpr, result *model.Resolution) { +// trackCall names the files one function call reads. +func (t *watchTracker) trackCall(scan fileScan, call *hclsyntax.FunctionCallExpr) { switch call.Name { case "file", "fileexists", "filebase64", "filemd5", "filesha1", "filesha256", "filesha512", "filebase64sha256", "filebase64sha512", "templatefile", "read_tfvars_file", "read_terragrunt_config", - "sops_decrypt_file", "mark_as_read", "fileset", "find_in_parent_folders": - default: + "sops_decrypt_file", "mark_as_read": + t.trackFile(scan, call) + case "fileset": + if path, ok := t.firstPath(scan, call); ok { + t.add(t.folders, path) + } else { + t.fallback() + } + case "find_in_parent_folders": + value, diagnostics := call.Value(scan.ctx) + if diagnostics.HasErrors() || value.IsNull() || !value.IsKnown() || value.Type() != cty.String { + t.fallback() + return + } + t.addDetected(value.AsString()) + } +} + +// trackFile watches the file a read function names and follows configuration files it reads. +func (t *watchTracker) trackFile(scan fileScan, call *hclsyntax.FunctionCallExpr) { + path, ok := t.firstPath(scan, call) + if !ok { + t.fallback() return } - // Unknown paths can refer anywhere in the project. A root directory watch is - // deliberately conservative and catches new files as well as changed files. - if len(call.Args) == 0 || call.Name == "find_in_parent_folders" { - result.Folders = append(result.Folders, root) + if call.Name == "read_terragrunt_config" && t.followConfigurationDirectory(path) { return } - value, diagnostics := call.Args[0].Value(nil) - if diagnostics.HasErrors() || !value.IsKnown() || value.IsNull() || value.Type() != cty.String { - result.Folders = append(result.Folders, root) + t.add(t.files, path) + if call.Name != "read_terragrunt_config" { return } - path := localPath(directory, value.AsString()) - if call.Name == "fileset" { - result.Folders = append(result.Folders, path) - } else { - result.Files = append(result.Files, path) + switch { + case strings.HasSuffix(path, hclExtension): + t.queue = append(t.queue, queuedFile{path: path}) + case strings.HasSuffix(path, ".json"): + // A JSON configuration holds its expressions in strings the HCL syntax walk cannot + // inspect, so any file it reads could be anywhere in the project. + t.fallback() + } +} + +// followConfigurationDirectory handles a directory passed to read_terragrunt_config. Terragrunt +// reads the default configuration inside it, so the directory is watched, which also covers a +// configuration created later, and the default configuration is inspected when it exists. +func (t *watchTracker) followConfigurationDirectory(path string) bool { + if info, err := t.fs.Stat(path); err != nil || !info.IsDir() { + return false + } + t.add(t.folders, path) + // Match Terragrunt's lookup order, including JSON precedence when both exist. + for _, name := range config.DefaultTerragruntConfigPaths { + selected := filepath.Join(path, name) + if exists, _ := afero.Exists(t.fs, selected); !exists { + continue + } + if strings.HasSuffix(selected, ".json") { + t.fallback() + } else { + t.queue = append(t.queue, queuedFile{path: selected}) + } + break + } + return true +} + +// followInclude watches the configuration an include block names and inspects it in the scope of +// the file that includes it. An include whose path cannot be resolved may name any file, so the +// watch widens to the project. +func (t *watchTracker) followInclude(scan fileScan, block *hclsyntax.Block) { + attribute := block.Body.Attributes["path"] + if attribute == nil { + t.fallback() + return + } + value, ok := stringOf(attribute.Expr, scan.ctx) + if !ok { + t.fallback() + return + } + path := localPath(scan.unit, value) + t.add(t.files, path) + switch { + case strings.HasSuffix(path, hclExtension): + t.queue = append(t.queue, queuedFile{path: path, unit: scan.unit}) + case strings.HasSuffix(path, ".json"): + t.fallback() + } +} + +// firstPath evaluates the first argument against the file's directory. +func (t *watchTracker) firstPath(scan fileScan, call *hclsyntax.FunctionCallExpr) (string, bool) { + if len(call.Args) == 0 { + return "", false + } + path, ok := stringOf(call.Args[0], scan.ctx) + if !ok { + return "", false + } + return localPath(scan.base, path), true +} + +// scanTerraform watches var files and hook scripts. Hooks and extra arguments +// run later under the host; only the files they name matter for selection. +func (t *watchTracker) scanTerraform(scan fileScan, block *hclsyntax.Block) { + t.visit(scan, block) + for _, child := range block.Body.Blocks { + switch child.Type { + case "extra_arguments": + for _, name := range []string{"required_var_files", "optional_var_files"} { + if attribute := child.Body.Attributes[name]; attribute != nil { + t.watchList(scan, attribute.Expr) + } + } + if attribute := child.Body.Attributes["arguments"]; attribute != nil { + t.watchVarFileArguments(scan, attribute.Expr) + } + case "before_hook", "after_hook", "error_hook": + t.watchHook(scan, child.Body) + } + } +} + +// watchVarFileArguments finds explicit Terraform var files in extra_arguments. +// An unresolved word could itself be a var-file flag or its path, so it must +// conservatively watch the project even when other arguments are known. +func (t *watchTracker) watchVarFileArguments(scan fileScan, expression hclsyntax.Expression) { + words, resolved := hookWords(scan, expression) + if !resolved { + t.fallback() + } + for index := 0; index < len(words); index++ { + word := words[index] + if word == "-var-file" { + if index+1 == len(words) || words[index+1] == "" { + t.fallback() + continue + } + index++ + t.add(t.files, localPath(t.unit, words[index])) + } else if path, found := strings.CutPrefix(word, "-var-file="); found { + if path == "" { + t.fallback() + continue + } + t.add(t.files, localPath(t.unit, path)) + } + } +} + +// listElements returns the element expressions of a literal list. A computed +// list is one element: its whole value is evaluated at once. +func listElements(expression hclsyntax.Expression) []hclsyntax.Expression { + if tuple, ok := expression.(*hclsyntax.TupleConsExpr); ok { + return tuple.Exprs + } + return []hclsyntax.Expression{expression} +} + +// watchList watches each string in a list. Elements that cannot be resolved +// widen the watch to the project root. +func (t *watchTracker) watchList(scan fileScan, expression hclsyntax.Expression) { + for _, element := range listElements(expression) { + if path, ok := stringOf(element, scan.ctx); ok { + t.add(t.files, localPath(t.unit, path)) + continue + } + if values, diagnostics := element.Value(scan.ctx); !diagnostics.HasErrors() && values.IsWhollyKnown() && !values.IsNull() && values.CanIterateElements() { + for _, value := range values.AsValueSlice() { + if value.Type() == cty.String && !value.IsNull() { + t.add(t.files, localPath(t.unit, value.AsString())) + } + } + continue + } + t.fallback() + } +} + +// watchHook watches the hook program (and script arguments) when they live in +// the project, plus the hook working directory. Resolution is best effort: a +// program found on PATH is not a project file. +func (t *watchTracker) watchHook(scan fileScan, body *hclsyntax.Body) { + // Relative script paths resolve against the hook's working directory when it names one. + // The whole directory is watched, which also covers slashless script arguments in it. + base := t.unit + if attribute := body.Attributes["working_dir"]; attribute != nil { + directory, ok := stringOf(attribute.Expr, scan.ctx) + if !ok { + // A script may live anywhere the unresolved directory points, so widen the watch. + t.fallback() + return + } + base = localPath(t.unit, directory) + t.add(t.folders, base) + } + execute := body.Attributes["execute"] + if execute == nil { + return + } + words, resolved := hookWords(scan, execute.Expr) + if !resolved { + // An unresolved word may name a project script, so widen the watch. + t.fallback() + } + for index, word := range words { + if !strings.ContainsAny(word, `/\`) { + continue + } + path := localPath(base, word) + if info, err := t.fs.Stat(path); index == 0 || (err == nil && !info.IsDir()) { + t.add(t.files, path) + } + } +} + +// hookWords returns the words of a hook's `execute` list, evaluating a computed list such as +// `execute = local.command` as a whole. It reports false when any word cannot be resolved. +func hookWords(scan fileScan, expression hclsyntax.Expression) ([]string, bool) { + var words []string + resolved := true + for _, element := range listElements(expression) { + if word, ok := stringOf(element, scan.ctx); ok { + words = append(words, word) + continue + } + values, diagnostics := element.Value(scan.ctx) + if diagnostics.HasErrors() || !values.IsWhollyKnown() || values.IsNull() || !values.CanIterateElements() { + resolved = false + continue + } + for _, value := range values.AsValueSlice() { + if value.Type() == cty.String && !value.IsNull() { + words = append(words, value.AsString()) + } + } + } + return words, resolved +} + +// add records a watch inside the project; paths outside it cannot appear in the +// project's change set and would make the whole inventory unreportable, unless a symlink leads back in. +func (t *watchTracker) add(set map[string]struct{}, path string) { + if within(path, t.root) { + set[path] = struct{}{} + return + } + // A link outside the project can lead into it. Git reports changes by the target's path, so the + // canonical target is the watch. + if resolved, err := canonicalPath(path); err == nil && resolved != path && within(resolved, t.root) { + set[resolved] = struct{}{} + } +} + +func (t *watchTracker) addDetected(path string) { + if info, err := t.fs.Stat(path); err == nil && info.IsDir() { + t.add(t.folders, path) + return + } + t.add(t.files, path) +} + +// fallback watches the whole project for expressions that cannot be resolved. +// Unknown paths can refer anywhere; a root directory watch is deliberately +// conservative and catches new files as well as changed files. +func (t *watchTracker) fallback() { + t.folders[t.root] = struct{}{} +} + +// addStaticWatches adds inputs to every unit that appear in no expression: the +// lock file beside the unit, and the whole directory Terragrunt copies when the +// source selects a subdirectory with `//`. +func (t *watchTracker) addStaticWatches() { + t.add(t.files, filepath.Join(t.unit, lockFileName)) + if source := t.result.Source; source != "" && !plainLocalSource(source) { + if copied, ok := copiedSourceRoot(t.unit, source); ok { + t.add(t.folders, copied) + } else if repository, ok := gitFileRepository(source); ok { + t.add(t.folders, repository) + } + } +} + +// gitFileRepository returns the local repository a `git::file://` source clones. When it lies in +// the project it is watched whole, so the module and the relative var files copied with it select +// the unit. The add method drops a repository outside the project, whose changes the project cannot contain. +func gitFileRepository(source string) (string, bool) { + rest, found := strings.CutPrefix(source, "git::file://") + if !found { + return "", false + } + rest, _, _ = strings.Cut(rest, "?") + if index := strings.Index(rest, "//"); index > 0 { + rest = rest[:index] + } + // A file URL can name a Windows volume after the leading slash. + if len(rest) > 2 && rest[0] == '/' && rest[2] == ':' { + rest = rest[1:] + } + if rest == "" { + return "", false + } + return filepath.Clean(filepath.FromSlash(rest)), true +} + +// copiedSourceRoot returns the directory a local `dir//subdir` source copies: the +// whole tree before `//`, so sibling modules referenced by relative path count. +func copiedSourceRoot(unit, source string) (string, bool) { + volume := filepath.VolumeName(source) + rest := source[len(volume):] + if strings.ContainsAny(rest, "?:") { + return "", false + } + index := strings.Index(filepath.ToSlash(rest), "//") + if index <= 0 { + return "", false + } + return localPath(unit, volume+rest[:index]), true +} + +// merge folds the collected watches into the resolution. Symlinked paths watch +// the resolved target as well as the link, because Git reports changes by the +// target's path. Files already covered by a watched folder are dropped: they add +// nothing, and they differ between checkouts when a folder is a submodule that +// one revision has not populated. +func (t *watchTracker) merge() { + files := slices.Concat(t.result.Files, slices.Collect(maps.Keys(t.files))) + folders := slices.Concat(t.result.Folders, slices.Collect(maps.Keys(t.folders))) + files = append(files, t.symlinkTargets(files)...) + folders = append(folders, t.symlinkTargets(folders)...) + folders = slices.Compact(slices.Sorted(slices.Values(folders))) + files = slices.DeleteFunc(files, func(path string) bool { + return path != t.result.Path && slices.ContainsFunc(folders, func(folder string) bool { return within(path, folder) }) + }) + t.result.Files = slices.Compact(slices.Sorted(slices.Values(files))) + t.result.Folders = folders +} + +func (t *watchTracker) symlinkTargets(paths []string) []string { + var targets []string + for _, path := range paths { + resolved, err := canonicalPath(path) + if err == nil && resolved != path && within(resolved, t.root) { + targets = append(targets, resolved) + } } + return targets } diff --git a/plugins/terragrunt/internal/engine/source_watches_test.go b/plugins/terragrunt/internal/engine/source_watches_test.go index 3d204ff2d62..b381d582424 100644 --- a/plugins/terragrunt/internal/engine/source_watches_test.go +++ b/plugins/terragrunt/internal/engine/source_watches_test.go @@ -1,6 +1,8 @@ package engine import ( + "fmt" + "os" "path/filepath" "testing" @@ -39,7 +41,8 @@ inputs = { } func TestInputWatchesConservativelyTrackDynamicPaths(t *testing.T) { - for _, expression := range []string{`file(local.path)`, `file("${get_terragrunt_dir()}/config.json")`, `fileset(local.root, "*.yaml")`, `find_in_parent_folders("optional.hcl")`, `file(null)`, `file(123)`, `file()`} { + // Expressions that need the environment, outputs, or missing locals cannot be resolved. + for _, expression := range []string{`file(local.path)`, `file("${get_env("DIR")}/config.json")`, `file(dependency.net.outputs.path)`, `fileset(local.root, "*.yaml")`, `find_in_parent_folders("optional.hcl")`, `file(null)`, `file(123)`, `file()`} { t.Run(expression, func(t *testing.T) { root := t.TempDir() path := filepath.Join(root, "terragrunt.hcl") @@ -64,7 +67,8 @@ func TestInputWatchesFailuresAndIncludedInputs(t *testing.T) { // failing the whole inventory. require.NoError(t, afero.WriteFile(fs, path, []byte("inputs = {"), 0o600)) require.NoError(t, trackInputSources(fs, root, result)) - require.Equal(t, []string{path}, result.Files) + // The lock file beside the unit is always watched as well. + require.Equal(t, []string{filepath.Join(root, lockFileName), path}, result.Files) require.Empty(t, result.Folders) require.NoError(t, afero.WriteFile(fs, path, []byte(`locals { unrelated = "constant" }`), 0o600)) require.NoError(t, trackInputSources(fs, root, result)) @@ -101,3 +105,175 @@ func TestInputWatchesFollowNestedConfigurationReads(t *testing.T) { } require.Empty(t, result.Folders) } + +// A JSON configuration read by read_terragrunt_config cannot be inspected for nested reads, so the +// read widens the watch to the project instead of silently missing them. +func TestInputWatchesWidenForJSONConfigurationReads(t *testing.T) { + root := t.TempDir() + fs := afero.NewMemMapFs() + unit := filepath.Join(root, "terragrunt.hcl") + require.NoError(t, fs.MkdirAll(root, 0o700)) + require.NoError(t, afero.WriteFile(fs, unit, []byte(`inputs = { common = read_terragrunt_config("shared.hcl.json") }`), 0o600)) + result := &model.Resolution{Files: []string{unit}} + require.NoError(t, trackInputSources(fs, root, result)) + require.Equal(t, []string{root}, result.Folders) + require.NotContains(t, result.Files, filepath.Join(root, "data.json")) + + require.NoError(t, afero.WriteFile(fs, unit, []byte(`inputs = { common = read_terragrunt_config("shared.hcl") }`), 0o600)) + result = &model.Resolution{Files: []string{unit}} + require.NoError(t, trackInputSources(fs, root, result)) + require.Empty(t, result.Folders, "an HCL configuration is still inspected rather than widened") +} + +// A path outside the project that is a symlink into it is watched at its canonical target. +func TestInputWatchesKeepSymlinkTargetsInsideTheProject(t *testing.T) { + root, err := filepath.EvalSymlinks(t.TempDir()) + require.NoError(t, err) + outside, err := filepath.EvalSymlinks(t.TempDir()) + require.NoError(t, err) + target := filepath.Join(root, "shared", "data.json") + require.NoError(t, os.MkdirAll(filepath.Dir(target), 0o700)) + require.NoError(t, os.WriteFile(target, []byte("{}"), 0o600)) + link := filepath.Join(outside, "data.json") + if err := os.Symlink(target, link); err != nil { + t.Skipf("symlink unavailable: %v", err) + } + unit := filepath.Join(root, "terragrunt.hcl") + outsideOnly := filepath.Join(outside, "elsewhere.json") + require.NoError(t, os.WriteFile(unit, []byte(`inputs = { + linked = file("`+filepath.ToSlash(link)+`") + other = file("`+filepath.ToSlash(outsideOnly)+`") +}`), 0o600)) + result := &model.Resolution{Files: []string{unit}} + require.NoError(t, trackInputSources(afero.NewOsFs(), root, result)) + require.Contains(t, result.Files, target) + require.NotContains(t, result.Files, link) + require.NotContains(t, result.Files, outsideOnly) +} + +// A directory passed to read_terragrunt_config selects its default configuration, so the directory +// is watched rather than recorded as a file, while a file path stays a file watch. +func TestInputWatchesResolveDirectoryConfigurationReads(t *testing.T) { + root := t.TempDir() + fs := afero.NewMemMapFs() + require.NoError(t, fs.MkdirAll(filepath.Join(root, "shared"), 0o700)) + unit := filepath.Join(root, "terragrunt.hcl") + require.NoError(t, afero.WriteFile(fs, unit, []byte(`inputs = { common = read_terragrunt_config("shared") }`), 0o600)) + require.NoError(t, afero.WriteFile(fs, filepath.Join(root, "shared", "terragrunt.hcl"), []byte(`inputs = { nested = read_terragrunt_config("../other.hcl") }`), 0o600)) + result := &model.Resolution{Files: []string{unit}} + require.NoError(t, trackInputSources(fs, root, result)) + require.Equal(t, []string{filepath.Join(root, "shared")}, result.Folders) + require.NotContains(t, result.Files, filepath.Join(root, "shared"), "a directory is not a file watch") + require.Contains(t, result.Files, filepath.Join(root, "other.hcl"), "the default configuration in the directory is followed") + + // A path that is not a directory stays a file watch without widening to a folder. + require.NoError(t, afero.WriteFile(fs, unit, []byte(`inputs = { common = read_terragrunt_config("shared/terragrunt.hcl") }`), 0o600)) + result = &model.Resolution{Files: []string{unit}} + require.NoError(t, trackInputSources(fs, root, result)) + require.Empty(t, result.Folders) + require.Contains(t, result.Files, filepath.Join(root, "shared", "terragrunt.hcl")) +} + +// Includes in a configuration read through inputs are followed in the including file's scope, and an +// include that cannot be resolved widens the watch instead of hiding included reads. +func TestInputWatchesFollowIncludesOfConfigurationReads(t *testing.T) { + root := t.TempDir() + fs := afero.NewMemMapFs() + require.NoError(t, fs.MkdirAll(filepath.Join(root, "shared", "base"), 0o700)) + unit := filepath.Join(root, "terragrunt.hcl") + write := func(path, contents string) { + t.Helper() + require.NoError(t, afero.WriteFile(fs, path, []byte(contents), 0o600)) + } + write(unit, `inputs = { common = read_terragrunt_config("shared/common.hcl") }`) + write(filepath.Join(root, "shared", "common.hcl"), `include "base" { path = "base/base.hcl" }`) + write(filepath.Join(root, "shared", "base", "base.hcl"), `inputs = { + local = file("local.json") + scoped = file("${get_terragrunt_dir()}/scoped.json") +}`) + result := &model.Resolution{Files: []string{unit}} + require.NoError(t, trackInputSources(fs, root, result)) + for _, path := range []string{ + filepath.Join(root, "shared", "base", "base.hcl"), + filepath.Join(root, "shared", "base", "local.json"), + filepath.Join(root, "shared", "scoped.json"), + } { + require.Contains(t, result.Files, path) + } + require.NotContains(t, result.Files, filepath.Join(root, "shared", "base", "scoped.json"), "the included file evaluates in the including scope") + require.Empty(t, result.Folders, "a resolvable include does not widen the watch") + + for name, include := range map[string]string{ + "unresolved path": `include "base" { path = get_env("BASE") }`, + "missing path": `include "base" {}`, + "json include": `include "base" { path = "base.hcl.json" }`, + } { + t.Run(name, func(t *testing.T) { + write(filepath.Join(root, "shared", "common.hcl"), include) + result := &model.Resolution{Files: []string{unit}} + require.NoError(t, trackInputSources(fs, root, result)) + require.Equal(t, []string{root}, result.Folders) + }) + } +} + +func TestInputWatchesDirectoryJSONDefault(t *testing.T) { + for _, withHCL := range []bool{false, true} { + t.Run(fmt.Sprint(withHCL), func(t *testing.T) { + root := t.TempDir() + fs := afero.NewMemMapFs() + shared := filepath.Join(root, "shared") + require.NoError(t, fs.MkdirAll(shared, 0o700)) + unit := filepath.Join(root, "terragrunt.hcl") + require.NoError(t, afero.WriteFile(fs, unit, []byte(`inputs = { shared = read_terragrunt_config("shared") }`), 0o600)) + require.NoError(t, afero.WriteFile(fs, filepath.Join(shared, "terragrunt.hcl.json"), []byte(`{"inputs":{"data":"${file(\"../data.json\")}"}}`), 0o600)) + if withHCL { + require.NoError(t, afero.WriteFile(fs, filepath.Join(shared, "terragrunt.hcl"), []byte(`inputs = {}`), 0o600)) + } + result := &model.Resolution{Files: []string{unit}} + require.NoError(t, trackInputSources(fs, root, result)) + require.Contains(t, result.Folders, root) + }) + } +} + +func TestInputWatchesRepeatedIncludeScopes(t *testing.T) { + root := t.TempDir() + fs := afero.NewMemMapFs() + write := func(path, contents string) { + t.Helper() + path = filepath.Join(root, path) + require.NoError(t, fs.MkdirAll(filepath.Dir(path), 0o700)) + require.NoError(t, afero.WriteFile(fs, path, []byte(contents), 0o600)) + } + write("terragrunt.hcl", `inputs = { a = read_terragrunt_config("a/env.hcl"), b = read_terragrunt_config("b/env.hcl") }`) + for _, dir := range []string{"a", "b"} { + write(dir+"/env.hcl", `include "base" { path = "../base.hcl" }`) + } + write("base.hcl", ` +include "cycle" { path = "../base.hcl" } +inputs = { data = file("${get_terragrunt_dir()}/data.json") } +`) + result := &model.Resolution{Files: []string{filepath.Join(root, "terragrunt.hcl")}} + require.NoError(t, trackInputSources(fs, root, result)) + for _, dir := range []string{"a", "b"} { + require.Contains(t, result.Files, filepath.Join(root, dir, "data.json")) + } + require.Empty(t, result.Folders) +} + +func TestInputWatchesOriginalCallerInNestedReads(t *testing.T) { + root := t.TempDir() + fs := afero.NewMemMapFs() + unit := filepath.Join(root, "app", "terragrunt.hcl") + require.NoError(t, fs.MkdirAll(filepath.Dir(unit), 0o700)) + require.NoError(t, fs.MkdirAll(filepath.Join(root, "shared"), 0o700)) + require.NoError(t, afero.WriteFile(fs, unit, []byte(`inputs = { common = read_terragrunt_config("../shared/common.hcl") }`), 0o600)) + require.NoError(t, afero.WriteFile(fs, filepath.Join(root, "shared", "common.hcl"), []byte(`inputs = { original = file("${get_original_terragrunt_dir()}/data.json"), current = file("${get_terragrunt_dir()}/current.json") }`), 0o600)) + result := &model.Resolution{Files: []string{unit}} + require.NoError(t, trackInputSources(fs, root, result)) + require.Contains(t, result.Files, filepath.Join(root, "app", "data.json")) + require.Contains(t, result.Files, filepath.Join(root, "shared", "current.json")) + require.NotContains(t, result.Files, filepath.Join(root, "shared", "data.json")) + require.Empty(t, result.Folders) +} diff --git a/plugins/terragrunt/internal/engine/watch_expressions.go b/plugins/terragrunt/internal/engine/watch_expressions.go new file mode 100644 index 00000000000..b1fb1ba6f2c --- /dev/null +++ b/plugins/terragrunt/internal/engine/watch_expressions.go @@ -0,0 +1,212 @@ +package engine + +import ( + "cmp" + "errors" + "fmt" + "maps" + "path/filepath" + "slices" + + "github.com/hashicorp/hcl/v2" + "github.com/hashicorp/hcl/v2/hclsyntax" + "github.com/spf13/afero" + "github.com/zclconf/go-cty/cty" + "github.com/zclconf/go-cty/cty/function" + "github.com/zclconf/go-cty/cty/function/stdlib" +) + +// errWatchUnresolvable reports an expression that needs state, the environment, +// or another evaluation input that inventory deliberately never reads. +var errWatchUnresolvable = errors.New("expression cannot be resolved without evaluating the unit") + +// watchEvaluator evaluates the pure subset of Terragrunt expressions that name +// files: path helpers, string functions, and local values. Inventory runs without +// state, secrets, or user commands, so nothing here can read outputs, run +// commands, or consult the environment. Anything outside the subset is an error +// and the caller falls back to a conservative watch. +type watchEvaluator struct { + fs afero.Fs + // root is the project root. Searches and watches never leave it. + root string + // unit is the directory of the unit configuration (get_terragrunt_dir). + unit string + // original is the caller directory retained across nested configuration reads. + original string + // parent is the directory of the first include; empty when not statically known. + parent string +} + +// context builds an evaluation context whose local values are already resolved. +func (e *watchEvaluator) context(locals cty.Value) *hcl.EvalContext { + return &hcl.EvalContext{Functions: e.functions(), Variables: map[string]cty.Value{"local": locals}} +} + +// locals resolves every local value that only depends on the pure subset. Locals +// that need anything else are left out, so references to them become unresolvable. +func (e *watchEvaluator) locals(body *hclsyntax.Body) cty.Value { + pending := map[string]hclsyntax.Expression{} + for _, block := range body.Blocks { + if block.Type != "locals" { + continue + } + for name, attribute := range block.Body.Attributes { + pending[name] = attribute.Expr + } + } + values := map[string]cty.Value{} + ctx := e.context(cty.EmptyObjectVal) + for len(pending) > 0 { + progress := false + for _, name := range slices.Sorted(maps.Keys(pending)) { + ctx.Variables["local"] = cty.ObjectVal(values) + value, diagnostics := pending[name].Value(ctx) + if diagnostics.HasErrors() || !value.IsWhollyKnown() || value.IsNull() { + continue + } + values[name] = value + delete(pending, name) + progress = true + } + if !progress { + break + } + } + return cty.ObjectVal(values) +} + +// stringOf evaluates an expression that must produce a known, non-null string. +func stringOf(expression hclsyntax.Expression, ctx *hcl.EvalContext) (string, bool) { + value, diagnostics := expression.Value(ctx) + if diagnostics.HasErrors() || !value.IsWhollyKnown() || value.IsNull() || value.Type() != cty.String { + return "", false + } + return value.AsString(), true +} + +func (e *watchEvaluator) functions() map[string]function.Function { + functions := map[string]function.Function{ + "format": stdlib.FormatFunc, + "join": stdlib.JoinFunc, + "replace": stdlib.ReplaceFunc, + "lower": stdlib.LowerFunc, + "upper": stdlib.UpperFunc, + "concat": stdlib.ConcatFunc, + "trimprefix": stdlib.TrimPrefixFunc, + "trimsuffix": stdlib.TrimSuffixFunc, + "dirname": pathFunction(filepath.Dir), + "basename": pathFunction(filepath.Base), + "abspath": pathFunction(func(path string) string { + if filepath.IsAbs(path) { + return filepath.Clean(path) + } + return filepath.Join(e.unit, path) + }), + "get_terragrunt_dir": constantFunction(func() (string, error) { return e.unit, nil }), + "get_original_terragrunt_dir": constantFunction(func() (string, error) { return cmp.Or(e.original, e.unit), nil }), + "get_parent_terragrunt_dir": constantFunction(e.parentDirectory), + "path_relative_to_include": constantFunction(func() (string, error) { return e.relativeToParent(false) }), + "path_relative_from_include": constantFunction(func() (string, error) { return e.relativeToParent(true) }), + "get_repo_root": constantFunction(e.repositoryRoot), + } + functions["find_in_parent_folders"] = function.New(&function.Spec{ + VarParam: &function.Parameter{Name: "args", Type: cty.String}, + Type: function.StaticReturnType(cty.String), + Impl: func(args []cty.Value, _ cty.Type) (cty.Value, error) { + found, err := e.findInParents(args) + return cty.StringVal(found), err + }, + }) + return functions +} + +func pathFunction(transform func(string) string) function.Function { + return function.New(&function.Spec{ + Params: []function.Parameter{{Name: "path", Type: cty.String}}, + Type: function.StaticReturnType(cty.String), + Impl: func(args []cty.Value, _ cty.Type) (cty.Value, error) { + return cty.StringVal(transform(args[0].AsString())), nil + }, + }) +} + +func constantFunction(compute func() (string, error)) function.Function { + return function.New(&function.Spec{ + Type: function.StaticReturnType(cty.String), + Impl: func([]cty.Value, cty.Type) (cty.Value, error) { + value, err := compute() + return cty.StringVal(value), err + }, + }) +} + +func (e *watchEvaluator) parentDirectory() (string, error) { + if e.parent == "" { + return "", fmt.Errorf("%w: no statically known include", errWatchUnresolvable) + } + return e.parent, nil +} + +func (e *watchEvaluator) relativeToParent(fromParent bool) (string, error) { + parent, err := e.parentDirectory() + if err != nil { + return "", err + } + if fromParent { + return filepath.Rel(e.unit, parent) + } + return filepath.Rel(parent, e.unit) +} + +// repositoryRoot finds the nearest ancestor holding a .git entry. Linked +// worktrees and submodules use a .git file, so existence is enough. +func (e *watchEvaluator) repositoryRoot() (string, error) { + for dir := e.unit; ; dir = filepath.Dir(dir) { + if _, err := e.fs.Stat(filepath.Join(dir, ".git")); err == nil { + return dir, nil + } + if filepath.Dir(dir) == dir { + return "", fmt.Errorf("%w: no Git repository", errWatchUnresolvable) + } + } +} + +// findInParents mirrors find_in_parent_folders: the search starts above the unit +// directory and stops at the project root, then falls back to the second argument. +func (e *watchEvaluator) findInParents(args []cty.Value) (string, error) { + if len(args) > 2 { + return "", fmt.Errorf("%w: too many arguments", errWatchUnresolvable) + } + name := "terragrunt.hcl" + if len(args) > 0 { + name = args[0].AsString() + } + for dir := filepath.Dir(e.unit); within(dir, e.root); dir = filepath.Dir(dir) { + candidate := filepath.Join(dir, name) + if _, err := e.fs.Stat(candidate); err == nil { + return candidate, nil + } + if dir == e.root || filepath.Dir(dir) == dir { + break + } + } + if len(args) == 2 { + return args[1].AsString(), nil + } + return "", fmt.Errorf("%w: %s was not found in a parent folder", errWatchUnresolvable, name) +} + +// includeParent returns the directory of the first include whose path can be +// resolved without a parent directory. +func (e *watchEvaluator) includeParent(body *hclsyntax.Body) string { + ctx := e.context(e.locals(body)) + for _, block := range body.Blocks { + if block.Type != "include" || block.Body.Attributes["path"] == nil { + continue + } + if path, ok := stringOf(block.Body.Attributes["path"].Expr, ctx); ok { + return filepath.Dir(localPath(e.unit, path)) + } + } + return "" +} diff --git a/plugins/terragrunt/internal/engine/watch_expressions_test.go b/plugins/terragrunt/internal/engine/watch_expressions_test.go new file mode 100644 index 00000000000..f92b77f4969 --- /dev/null +++ b/plugins/terragrunt/internal/engine/watch_expressions_test.go @@ -0,0 +1,397 @@ +package engine + +import ( + "os" + "path/filepath" + "testing" + + "github.com/spf13/afero" + "github.com/stretchr/testify/require" + + "github.com/cloudposse/atmos/plugins/terragrunt/internal/model" +) + +// watchProject lays out live/root.hcl, a unit that includes it, and the files the +// unit's expressions can name. Paths are canonical so symlink expansion is a no-op. +type watchProject struct { + t *testing.T + fs afero.Fs + root string + unit string +} + +func newWatchProject(t *testing.T) *watchProject { + t.Helper() + root, err := filepath.EvalSymlinks(t.TempDir()) + require.NoError(t, err) + project := &watchProject{t: t, fs: afero.NewMemMapFs(), root: root, unit: filepath.Join(root, "live", "prod", "app")} + project.write(filepath.Join(root, "live", "root.hcl"), `locals { region = "east" }`) + project.write(filepath.Join(root, ".git"), "gitdir: elsewhere") + return project +} + +func (p *watchProject) write(path, contents string) { + p.t.Helper() + require.NoError(p.t, p.fs.MkdirAll(filepath.Dir(path), 0o700)) + require.NoError(p.t, afero.WriteFile(p.fs, path, []byte(contents), 0o600)) +} + +// track writes the unit configuration and returns the watches inventory computes. +func (p *watchProject) track(configuration string) *model.Resolution { + p.t.Helper() + path := filepath.Join(p.unit, "terragrunt.hcl") + p.write(path, `include "root" { path = find_in_parent_folders("root.hcl") } +`+configuration) + result := &model.Resolution{Path: path, Files: []string{path, filepath.Join(p.root, "live", "root.hcl")}} + require.NoError(p.t, trackInputSources(p.fs, p.root, result)) + return result +} + +func TestWatchesResolveCommonPathIdioms(t *testing.T) { + project := newWatchProject(t) + project.write(filepath.Join(project.root, "live", "common.json"), "{}") + live := filepath.Join(project.root, "live") + tests := []struct { + name string + inputs string + file string + wantFolder bool + }{ + {name: "terragrunt dir", inputs: `file("${get_terragrunt_dir()}/x.json")`, file: filepath.Join(project.unit, "x.json")}, + {name: "parent dir", inputs: `templatefile("${get_parent_terragrunt_dir()}/t.tpl", {})`, file: filepath.Join(live, "t.tpl")}, + {name: "relative to include", inputs: `file("${get_parent_terragrunt_dir()}/${path_relative_to_include()}/x")`, file: filepath.Join(project.unit, "x")}, + {name: "relative from include", inputs: `file("${get_terragrunt_dir()}/${path_relative_from_include()}/x")`, file: filepath.Join(project.unit, "..", "..", "x")}, + {name: "abspath", inputs: `file(abspath("rel.json"))`, file: filepath.Join(project.unit, "rel.json")}, + {name: "format with literal", inputs: `file(format("%s/%s.json", get_terragrunt_dir(), "svc"))`, file: filepath.Join(project.unit, "svc.json")}, + {name: "dirname and join", inputs: `file(join("/", [dirname(get_terragrunt_dir()), "sibling.json"]))`, file: filepath.Join(project.root, "live", "prod", "sibling.json")}, + {name: "find in parent folders", inputs: `file(find_in_parent_folders("common.json"))`, file: filepath.Join(live, "common.json")}, + {name: "repo root", inputs: `file("${get_repo_root()}/shared/x.yaml")`, file: filepath.Join(project.root, "shared", "x.yaml")}, + {name: "relative literal", inputs: `file("input.json")`, file: filepath.Join(project.unit, "input.json")}, + } + for _, test := range tests { + t.Run(test.name, func(t *testing.T) { + result := project.track("inputs = { v = " + test.inputs + " }\n") + require.Contains(t, result.Files, filepath.Clean(test.file)) + require.Empty(t, result.Folders, "a resolvable path must not watch the project") + }) + } +} + +func TestWatchesResolveLocals(t *testing.T) { + project := newWatchProject(t) + result := project.track(` +locals { + name = "svc" + path = "${get_terragrunt_dir()}/${local.name}.json" + unknown = get_env("MISSING") +} +inputs = { v = file(local.path) } +`) + require.Contains(t, result.Files, filepath.Join(project.unit, "svc.json")) + require.Empty(t, result.Folders) + + // A local that needs the environment makes only the expressions using it conservative. + result = project.track(` +locals { unknown = get_env("MISSING") } +inputs = { v = file(local.unknown) } +`) + require.Equal(t, []string{project.root}, result.Folders) +} + +func TestWatchesFallBackForUnresolvableOnly(t *testing.T) { + project := newWatchProject(t) + // One resolvable and one unresolvable read: the unresolvable one widens the watch. + result := project.track(`inputs = { a = file("${get_terragrunt_dir()}/a.json"), b = file("${get_env("DIR")}/b.json") }`) + require.Equal(t, []string{project.root}, result.Folders) + require.NotContains(t, result.Files, filepath.Join(project.unit, "a.json"), "covered by the project watch") + + // Reads outside the project cannot appear in a change set; they must not fail inventory. + result = project.track(`inputs = { a = file("/etc/hostname") }`) + require.Empty(t, result.Folders) + require.NotContains(t, result.Files, "/etc/hostname") +} + +func TestWatchesCoverGenerateExtraArgumentsAndHooks(t *testing.T) { + project := newWatchProject(t) + project.write(filepath.Join(project.root, "scripts", "check.sh"), "echo") + result := project.track(` +terraform { + source = "../../../modules/m" + extra_arguments "vars" { + commands = ["plan"] + required_var_files = ["${get_terragrunt_dir()}/required.tfvars", "${get_parent_terragrunt_dir()}/common.tfvars"] + optional_var_files = ["${get_terragrunt_dir()}/optional.tfvars"] + } + before_hook "script" { + commands = ["plan"] + execute = ["../../../scripts/hook.sh"] + } + after_hook "arg" { + commands = ["plan"] + execute = ["bash", "${get_repo_root()}/scripts/check.sh", "plain-argument", "${get_repo_root()}/missing.sh"] + working_dir = "${get_repo_root()}/tools" + } + error_hook "path" { + commands = ["plan"] + execute = ["tofu", "version"] + } +} +generate "provider" { + path = "provider.tf" + if_exists = "overwrite" + contents = file("${get_terragrunt_dir()}/provider.tpl") +} +remote_state { + backend = "local" + generate = { path = "backend.tf", if_exists = "overwrite" } + config = { path = file("${get_terragrunt_dir()}/state-path.txt") } +} +`) + for _, want := range []string{ + filepath.Join(project.unit, "required.tfvars"), + filepath.Join(project.root, "live", "common.tfvars"), + filepath.Join(project.unit, "optional.tfvars"), + filepath.Join(project.root, "scripts", "hook.sh"), + filepath.Join(project.root, "scripts", "check.sh"), + filepath.Join(project.unit, "provider.tpl"), + filepath.Join(project.unit, "state-path.txt"), + filepath.Join(project.unit, lockFileName), + } { + require.Contains(t, result.Files, want) + } + // Plain words and missing arguments are not project files. + require.NotContains(t, result.Files, filepath.Join(project.root, "missing.sh")) + require.NotContains(t, result.Files, filepath.Join(project.unit, "plain-argument")) + require.Equal(t, []string{filepath.Join(project.root, "tools")}, result.Folders) +} + +func TestWatchesVarFilesFallBackWhenDynamic(t *testing.T) { + project := newWatchProject(t) + result := project.track(` +terraform { + extra_arguments "vars" { + commands = ["plan"] + optional_var_files = ["${get_terragrunt_dir()}/${get_env("ENV", "dev")}.tfvars"] + } +} +`) + require.Equal(t, []string{project.root}, result.Folders) +} + +func TestWatchesExtraArgumentVarFiles(t *testing.T) { + project := newWatchProject(t) + result := project.track(` +locals { flags = ["-var-file=${get_terragrunt_dir()}/../../shared/computed.tfvars", "-lock=false"] } +terraform { + extra_arguments "literal" { + commands = ["plan"] + arguments = ["-var-file=${get_terragrunt_dir()}/../../shared/arg.tfvars", "-var-file", "relative.tfvars", "-lock=false"] + } + extra_arguments "computed" { + commands = ["apply"] + arguments = local.flags + } +} +`) + for _, path := range []string{ + filepath.Join(project.root, "live", "shared", "arg.tfvars"), + filepath.Join(project.root, "live", "shared", "computed.tfvars"), + filepath.Join(project.unit, "relative.tfvars"), + } { + require.Contains(t, result.Files, path) + } + require.Empty(t, result.Folders) + + result = project.track(` +terraform { + extra_arguments "dynamic" { + commands = ["plan"] + arguments = ["-var-file=${get_terragrunt_dir()}/${get_env("ENV")}.tfvars"] + } +} +`) + require.Equal(t, []string{project.root}, result.Folders) +} + +func TestWatchesFollowReadConfigurationFiles(t *testing.T) { + project := newWatchProject(t) + project.write(filepath.Join(project.unit, "extra.hcl"), ` +locals { body = file("${get_terragrunt_dir()}/nested.txt") } +inputs = { v = file("${get_terragrunt_dir()}/inputs.txt") } +`) + result := project.track(` +generate "x" { + path = "x.tf" + if_exists = "overwrite" + contents = read_terragrunt_config("${get_terragrunt_dir()}/extra.hcl").locals.body +} +`) + require.Contains(t, result.Files, filepath.Join(project.unit, "extra.hcl")) + require.Contains(t, result.Files, filepath.Join(project.unit, "nested.txt")) + // Only the read configuration's locals are inspected; its inputs are not evaluated for the reader. + require.Contains(t, result.Files, filepath.Join(project.unit, "inputs.txt")) + require.Empty(t, result.Folders) +} + +func TestWatchesCoverWholeCopiedSourceRoot(t *testing.T) { + project := newWatchProject(t) + for _, test := range []struct { + source string + want []string + }{ + {source: "../../../modules/sub//app", want: []string{filepath.Join(project.root, "modules", "sub")}}, + {source: filepath.Join(project.root, "modules", "sub") + "//app", want: []string{filepath.Join(project.root, "modules", "sub")}}, + {source: "git::https://example.invalid/repo.git//sub?ref=v1"}, + // An in-project Git repository is watched whole; one outside the project cannot change in it. + {source: "git::file://" + filepath.ToSlash(filepath.Join(project.root, "modules", "repo")) + "//app?ref=main", want: []string{filepath.Join(project.root, "modules", "repo")}}, + {source: "git::file://" + filepath.ToSlash(filepath.Join(project.root, "modules", "repo")) + "?ref=main", want: []string{filepath.Join(project.root, "modules", "repo")}}, + {source: "git::file://" + filepath.ToSlash(filepath.Dir(project.root)) + "//app?ref=main"}, + {source: "https://example.invalid/archive.zip//sub"}, + {source: "../../../modules/plain"}, + } { + t.Run(test.source, func(t *testing.T) { + path := filepath.Join(project.unit, "terragrunt.hcl") + project.write(path, "") + result := &model.Resolution{Path: path, Files: []string{path}, Source: test.source} + require.NoError(t, trackInputSources(project.fs, project.root, result)) + require.Equal(t, test.want, result.Folders) + }) + } +} + +func TestWatchesDropFilesCoveredByFolders(t *testing.T) { + // A submodule module is empty in revisions that predate it. Its files add nothing to the + // folder watch and must not make the dependency list differ between revisions. + project := newWatchProject(t) + module := filepath.Join(project.root, "modules", "sm") + path := filepath.Join(project.unit, "terragrunt.hcl") + project.write(path, "") + files := []string{path, filepath.Join(module, "main.tf")} + result := &model.Resolution{Path: path, Files: slicesClone(files), Folders: []string{module}} + require.NoError(t, trackInputSources(project.fs, project.root, result)) + require.NotContains(t, result.Files, filepath.Join(module, "main.tf")) + require.Contains(t, result.Files, path) + require.Equal(t, []string{module}, result.Folders) + + empty := &model.Resolution{Path: path, Files: []string{path}, Folders: []string{module}} + require.NoError(t, trackInputSources(project.fs, project.root, empty)) + require.Equal(t, result.Files, empty.Files) + require.Equal(t, result.Folders, empty.Folders) +} + +func slicesClone(values []string) []string { return append([]string(nil), values...) } + +func TestWatchesIncludeSymlinkTargets(t *testing.T) { + root, err := filepath.EvalSymlinks(t.TempDir()) + require.NoError(t, err) + target := filepath.Join(root, "modules", "other") + require.NoError(t, os.MkdirAll(target, 0o700)) + link := filepath.Join(root, "modules", "linked") + if err := os.Symlink("other", link); err != nil { + t.Skipf("symlinks unavailable: %v", err) + } + unit := filepath.Join(root, "live", "link") + require.NoError(t, os.MkdirAll(unit, 0o700)) + path := filepath.Join(unit, "terragrunt.hcl") + require.NoError(t, os.WriteFile(path, []byte(`inputs = { v = file("${get_terragrunt_dir()}/../../modules/linked/data.txt") }`), 0o600)) + + result := &model.Resolution{Path: path, Files: []string{path}, Folders: []string{link}} + require.NoError(t, trackInputSources(afero.NewOsFs(), root, result)) + // Git reports edits under the target's path, so both the link and the target are watched. + require.Equal(t, []string{link, target}, result.Folders) + + // A file reached through a linked directory is watched under its target path too. + file := &model.Resolution{Path: path, Files: []string{path}} + require.NoError(t, trackInputSources(afero.NewOsFs(), root, file)) + require.Contains(t, file.Files, filepath.Join(target, "data.txt")) + require.Contains(t, file.Files, filepath.Join(link, "data.txt")) +} + +func TestWatchesIgnoreSymlinkTargetsOutsideProject(t *testing.T) { + root, err := filepath.EvalSymlinks(t.TempDir()) + require.NoError(t, err) + outside, err := filepath.EvalSymlinks(t.TempDir()) + require.NoError(t, err) + link := filepath.Join(root, "modules", "external") + require.NoError(t, os.MkdirAll(filepath.Dir(link), 0o700)) + if err := os.Symlink(outside, link); err != nil { + t.Skipf("symlinks unavailable: %v", err) + } + path := filepath.Join(root, "terragrunt.hcl") + require.NoError(t, os.WriteFile(path, nil, 0o600)) + result := &model.Resolution{Path: path, Files: []string{path}, Folders: []string{link}} + require.NoError(t, trackInputSources(afero.NewOsFs(), root, result)) + require.Equal(t, []string{link}, result.Folders) +} + +// A computed hook command list is evaluated as a whole, so the script it names is watched; a +// command that cannot be resolved widens the watch instead of silently watching nothing. +func TestWatchesResolveComputedHookCommands(t *testing.T) { + project := newWatchProject(t) + project.write(filepath.Join(project.root, "scripts", "computed.sh"), "echo") + result := project.track(` +locals { command = ["bash", "${get_repo_root()}/scripts/computed.sh"] } +terraform { + source = "../../../modules/m" + before_hook "computed" { + commands = ["plan"] + execute = local.command + } +} +`) + require.Contains(t, result.Files, filepath.Join(project.root, "scripts", "computed.sh")) + require.Empty(t, result.Folders) + + result = project.track(` +terraform { + source = "../../../modules/m" + before_hook "unresolved" { + commands = ["plan"] + execute = ["bash", "${get_env("HOOK_DIR")}/run.sh"] + } +} +`) + require.Equal(t, []string{project.root}, result.Folders) +} + +// A file read with read_terragrunt_config resolves get_terragrunt_dir() against its own directory. +func TestWatchesResolveReadConfigurationFromItsOwnDirectory(t *testing.T) { + project := newWatchProject(t) + shared := filepath.Join(project.root, "live", "shared") + project.write(filepath.Join(shared, "extra.hcl"), `locals { body = file("${get_terragrunt_dir()}/data.json") }`) + result := project.track(` +inputs = { v = read_terragrunt_config("${get_terragrunt_dir()}/../../shared/extra.hcl").locals.body } +`) + require.Contains(t, result.Files, filepath.Join(shared, "data.json")) + require.NotContains(t, result.Files, filepath.Join(project.unit, "data.json")) + require.Empty(t, result.Folders) +} + +// Hook scripts resolve against the hook's working directory, and an unresolved directory widens the watch. +func TestWatchesResolveHookScriptsFromWorkingDirectory(t *testing.T) { + project := newWatchProject(t) + project.write(filepath.Join(project.root, "scripts", "run.sh"), "echo") + result := project.track(` +terraform { + before_hook "relative" { + commands = ["plan"] + execute = ["../scripts/run.sh", "check.sh"] + working_dir = "${get_repo_root()}/tools" + } +} +`) + // The working directory is watched whole, covering slashless arguments such as check.sh. + require.Contains(t, result.Files, filepath.Join(project.root, "scripts", "run.sh")) + require.Equal(t, []string{filepath.Join(project.root, "tools")}, result.Folders) + + result = project.track(` +terraform { + before_hook "dynamic" { + commands = ["plan"] + execute = ["bash", "check.sh"] + working_dir = get_env("HOOK_DIR") + } +} +`) + require.Equal(t, []string{project.root}, result.Folders) +} diff --git a/plugins/terragrunt/internal/model/model.go b/plugins/terragrunt/internal/model/model.go index fb24b89bd02..dd2393206a6 100644 --- a/plugins/terragrunt/internal/model/model.go +++ b/plugins/terragrunt/internal/model/model.go @@ -40,10 +40,42 @@ type Resolution struct { Backend map[string]any `json:"backend"` Generated map[string]string `json:"generated"` GeneratedPolicies map[string]string `json:"generated_policies,omitempty"` + // UnitFileCopies identifies literal files copied from beside the unit. + UnitFileCopies map[string]bool `json:"unit_file_copies,omitempty"` + // SensitiveGenerated names generated files whose contents can hold + // credentials. The host masks their bytes and keeps them out of saved-plan records. + SensitiveGenerated map[string]bool `json:"sensitive_generated,omitempty"` Units map[string]string `json:"units,omitempty"` Stacks map[string]string `json:"stacks,omitempty"` + // Failures are units that could not be evaluated. A failing unit never hides healthy ones. + Failures []UnitFailure `json:"failures,omitempty"` + // Warnings are non-fatal facts about the scan that the user should see. + Warnings []Warning `json:"warnings,omitempty"` } +// UnitFailure is a unit that discovery could not evaluate. Stack and Name are the Atmos +// coordinates the unit would have had; Err is a Problem or a plain error. +type UnitFailure struct { + Stack string + Name string + Err error +} + +// Warning is a non-fatal discovery finding. Code is stable and machine readable; Path is +// relative to the project root and uses forward slashes. +type Warning struct { + Code string `json:"code"` + Path string `json:"path,omitempty"` + Message string `json:"message"` +} + +// Warning codes reported by discovery. +const ( + WarningSymlinkedDirectory = "symlinked_directory_not_followed" + WarningPatternUnmatched = "discovery_pattern_unmatched" + WarningUnitWithoutSource = "unit_without_terraform_source" +) + // DiscoveredUnit carries source identity independently from its Atmos name. type DiscoveredUnit struct { Stack string `json:"stack"` @@ -86,3 +118,51 @@ type Hook struct { SuppressStdout bool `json:"suppress_stdout,omitempty"` OnErrors []string `json:"on_errors,omitempty"` } + +// Problem codes are stable identifiers shared with the host and with tests. +const ( + CodeUnsupportedSetting = "unsupported_setting" + CodeSyntaxError = "syntax_error" + CodeEvaluationError = "evaluation_failed" + CodeEffectDenied = "effect_denied" + CodeDependency = "dependency_error" + CodeInvalidSource = "invalid_source" + CodeInvalidGenerate = "invalid_generate_path" + CodeMissingSource = "missing_terraform_source" + CodeUnsupportedHook = "unsupported_hook" + CodeWorkspace = "workspace_unsupported" + CodeInternal = "plugin_error" +) + +// Problem is a rejection the user can act on. Summary, Hints, File, Line, Column, and Key never +// contain evaluated values and are always shown. Detail is shown only when Safe is true. +type Problem struct { + Code string + Summary string + Detail string + Hints []string + // File is an absolute path; the provider makes it project relative. + File string + Line, Column int + // Key names the setting, block, or lifecycle that caused the problem. + Key string + Safe bool + // Err is the underlying cause. It is never shown to the user. + Err error +} + +// Error returns the headline and detail for local diagnostics and tests. The provider never sends it +// to the host: it sends only the structured fields, honoring Safe. +// +//nolint:lintroller // Standalone plugin diagnostic formatting; importing host perf would break the SDK-only boundary. +func (p *Problem) Error() string { + if p.Detail == "" { + return p.Summary + } + return p.Summary + ": " + p.Detail +} + +// Unwrap exposes the cause for classification. +// +//nolint:lintroller // Pure cause delegation in a standalone plugin; no host tracker dependency. +func (p *Problem) Unwrap() error { return p.Err } diff --git a/plugins/terragrunt/internal/provider/boundaries_test.go b/plugins/terragrunt/internal/provider/boundaries_test.go index 18826595680..da5841b67e5 100644 --- a/plugins/terragrunt/internal/provider/boundaries_test.go +++ b/plugins/terragrunt/internal/provider/boundaries_test.go @@ -70,12 +70,36 @@ func TestDiscoveryRejectsUnrepresentableInventory(t *testing.T) { return &model.Resolution{Inventory: []model.DiscoveredUnit{unit}}, nil }) result, err := p.Discover(t.Context(), &pluginv1.DiscoverComponentsRequest{RootDirectory: base.RootDirectory, Provider: base.Provider, ProviderConfiguration: base.ProviderConfiguration, Operation: base.Operation, IncludePaths: []string{"live/**"}}) - require.Nil(t, result) if name == "evaluator failure" { + require.Nil(t, result) require.Equal(t, codes.Canceled, status.Code(err)) - } else { - require.Equal(t, codes.Unimplemented, status.Code(err)) + return } + // A unit that cannot be represented is reported on its own and never hides healthy units. + require.NoError(t, err) + if name == "dependency outside scope" { + // The unit is kept with its out-of-scope dependency; the host fails it, naming the target. + require.Empty(t, result.Diagnostics) + require.Len(t, result.Components, 1) + component := result.Components[0] + require.Equal(t, "live", component.Address.Stack) + require.Equal(t, "app", component.Address.Component) + require.Len(t, component.Dependencies, 1) + target := component.Dependencies[0].Target + require.Equal(t, "default", target.Stack) + require.Equal(t, "other", target.Component) + require.Equal(t, "terraform", target.ComponentType) + require.Equal(t, "terraform", component.Address.ComponentType) + require.Equal(t, pluginv1.ComponentDependency_ORDER, component.Dependencies[0].Kind) + return + } + require.Empty(t, result.Components) + require.Len(t, result.Diagnostics, 1) + detail := result.Diagnostics[0].Detail + require.Equal(t, "app", detail.Unit) + require.Equal(t, "live", detail.Stack) + require.Equal(t, "discover", detail.Operation) + require.NotEqual(t, pluginv1.Severity_SEVERITY_WARNING, detail.Severity) }) } } diff --git a/plugins/terragrunt/internal/provider/discovery.go b/plugins/terragrunt/internal/provider/discovery.go index aef82453a44..01d41aa4105 100644 --- a/plugins/terragrunt/internal/provider/discovery.go +++ b/plugins/terragrunt/internal/provider/discovery.go @@ -37,14 +37,20 @@ func (p *Provider) Discover(ctx context.Context, request *pluginv1.DiscoverCompo addresses[unit.Config.Path] = &pluginv1.ComponentAddress{Stack: unit.Stack, Component: unit.Name, ComponentType: "terraform"} } response := &pluginv1.DiscoverComponentsResponse{} + for _, failure := range result.Failures { + response.Diagnostics = append(response.Diagnostics, unitDiagnostic(req.BasePath, failure)) + } for i := range result.Inventory { unit := &result.Inventory[i] discovered, err := discoveredUnit(&req, unit, addresses) if err != nil { - return nil, rpcError(err) + // One unit that cannot be described must not hide the others. + response.Diagnostics = append(response.Diagnostics, unitDiagnostic(req.BasePath, model.UnitFailure{Stack: unit.Stack, Name: unit.Name, Err: problemFor(err, unit.Config.Path)})) + continue } response.Components = append(response.Components, discovered) } + response.Diagnostics = append(response.Diagnostics, warningDiagnostics(result.Warnings)...) return response, nil } @@ -86,7 +92,7 @@ func discoveredUnit(req *model.Request, unit *model.DiscoveredUnit, addresses ma return nil, err } instance := &pluginv1.DiscoveredComponent{Address: addresses[unit.Config.Path], Provider: "terragrunt", Engine: engineName(req.Tofu), Source: source, Configuration: config, Sources: watched} - instance.Dependencies, err = discoveredDependencies(&unit.Config, addresses) + instance.Dependencies, err = discoveredDependencies(req.BasePath, &unit.Config, addresses) if err != nil { return nil, err } @@ -114,12 +120,17 @@ func discoveredModule(root string, cfg *model.Resolution) (string, error) { } // discoveredDependencies binds evaluated dependency paths to discovered addresses. -func discoveredDependencies(cfg *model.Resolution, addresses map[string]*pluginv1.ComponentAddress) ([]*pluginv1.ComponentDependency, error) { +func discoveredDependencies(root string, cfg *model.Resolution, addresses map[string]*pluginv1.ComponentAddress) ([]*pluginv1.ComponentDependency, error) { var dependencies []*pluginv1.ComponentDependency for _, path := range cfg.Dependencies { target := addresses[path] if target == nil { - return nil, ErrUnsupported + // The target is outside this evaluation (not discovered, or imported on its own). + // Name it the way discovery would, so the host can decide whether it exists. + var err error + if target, err = derivedAddress(root, path); err != nil { + return nil, err + } } kind := pluginv1.ComponentDependency_ORDER if slices.Contains(cfg.OutputDependencies, path) { @@ -168,3 +179,20 @@ func directoryPattern(path string) string { } return clean + "/**" } + +// derivedAddress names a unit that was not part of the evaluation: the component is its +// directory name and the stack is the directory above it, relative to the project. +func derivedAddress(base, path string) (*pluginv1.ComponentAddress, error) { + dir, err := relative(base, filepath.Dir(path)) + if err != nil { + return nil, err + } + stack, name := "default", "root" + if dir != "." { + name = filepath.Base(dir) + if parent := filepath.ToSlash(filepath.Dir(dir)); parent != "." { + stack = parent + } + } + return &pluginv1.ComponentAddress{Stack: stack, Component: name, ComponentType: "terraform"}, nil +} diff --git a/plugins/terragrunt/internal/provider/errors.go b/plugins/terragrunt/internal/provider/errors.go new file mode 100644 index 00000000000..d1e7d44ceb6 --- /dev/null +++ b/plugins/terragrunt/internal/provider/errors.go @@ -0,0 +1,182 @@ +package provider + +import ( + "context" + "errors" + "fmt" + "path/filepath" + + "google.golang.org/grpc/codes" + "google.golang.org/grpc/status" + + "github.com/cloudposse/atmos/plugins/terragrunt/internal/engine" + "github.com/cloudposse/atmos/plugins/terragrunt/internal/model" + "github.com/cloudposse/atmos/sdk/component" + pluginv1 "github.com/cloudposse/atmos/sdk/proto/atmos/plugin/v1" +) + +// rpcFailure is a plugin error that already carries the structured description the host renders. +// It becomes a gRPC status whose details hold the description. +type rpcFailure struct { + failure *component.Failure + code codes.Code +} + +//nolint:lintroller // Pure structured-summary access in a standalone plugin; no host tracker dependency. +func (f *rpcFailure) Error() string { return f.failure.Proto().Summary } + +// GRPCStatus implements the interface the gRPC server uses to build the response status. +// +//nolint:lintroller // SDK-to-gRPC error conversion stays independent of the host performance tracker. +func (f *rpcFailure) GRPCStatus() *status.Status { + return status.Convert(f.failure.Err(f.code)) +} + +// rpcError converts an evaluation error to an RPC error. Evaluator payloads are never copied into +// a message the host prints: HCL diagnostics can contain secrets, so only the structured fields +// that carry no evaluated values are shown, and detail text appears only when it is marked safe. +func rpcError(err error) error { + if errors.Is(err, context.Canceled) { + return status.Error(codes.Canceled, "Terragrunt evaluation canceled") + } + if errors.Is(err, context.DeadlineExceeded) { + return status.Error(codes.DeadlineExceeded, "Terragrunt evaluation timed out") + } + var failure *rpcFailure + if errors.As(err, &failure) { + return failure + } + return &rpcFailure{failure: failureFor(err), code: codeFor(err)} +} + +func codeFor(err error) codes.Code { + var problem *model.Problem + if errors.As(err, &problem) { + switch problem.Code { + case model.CodeUnsupportedSetting, model.CodeUnsupportedHook, model.CodeWorkspace: + return codes.Unimplemented + } + return codes.InvalidArgument + } + if errors.Is(err, ErrUnsupported) || errors.Is(err, engine.ErrUnsupportedExecution) { + return codes.Unimplemented + } + return codes.InvalidArgument +} + +// failureFor describes err. A model.Problem keeps its own fields; anything else becomes a generic +// failure whose message stays hidden. +func failureFor(err error) *component.Failure { + var problem *model.Problem + if errors.As(err, &problem) { + failure := component.NewFailure(problem.Code, problem.Summary).Key(problem.Key) + if problem.Safe { + failure.SafeDetail(problem.Detail) + } else if problem.Detail != "" { + failure.Detail(problem.Detail) + } + for _, hint := range problem.Hints { + failure.Hint(hint) + } + if problem.File != "" { + failure.At(problem.File, uint32(max(problem.Line, 0)), uint32(max(problem.Column, 0))) //nolint:gosec // Line and column are small non-negative positions. + } + return failure + } + if errors.Is(err, ErrUnsupported) { + return component.NewFailure(model.CodeUnsupportedSetting, "Unsupported native Terragrunt configuration"). + SafeDetail("The request uses a mode, option, or path the native Terragrunt adaptation does not support."). + Hint("Check `components.terraform.terragrunt` in atmos.yaml: only `mode: native` is supported.") + } + return component.NewFailure(model.CodeInternal, "Terragrunt evaluation failed").Detail(err.Error()) +} + +// scoped names the operation, stack, and component on a failure and makes its file project relative. +func scoped(err error, operation, root string, address *pluginv1.ComponentAddress) error { + var failure *rpcFailure + if !errors.As(err, &failure) { + return err + } + detail := failure.failure.Proto() + updated := component.NewFailure(detail.Code, detail.Summary).Operation(operation).Key(detail.ConfigKey) + if detail.UserSafe { + updated.SafeDetail(detail.Detail) + } else if detail.Detail != "" { + updated.Detail(detail.Detail) + } + for _, hint := range detail.Hints { + updated.Hint(hint) + } + if detail.Location != nil { + updated.At(projectRelative(root, detail.Location.File), detail.Location.Line, detail.Location.Column) + } + if address != nil { + updated.Unit(address.Stack, address.Component) + } + return &rpcFailure{failure: updated, code: failure.code} +} + +// projectRelative returns file relative to the project root, or file unchanged when it lies elsewhere. +func projectRelative(root, file string) string { + if root == "" || file == "" || !filepath.IsAbs(file) { + return file + } + if resolved, err := filepath.EvalSymlinks(root); err == nil { + root = resolved + } + rel, err := filepath.Rel(root, file) + if err != nil || !filepath.IsLocal(rel) { + return file + } + return filepath.ToSlash(rel) +} + +// unitDiagnostic reports one failed unit as a per-unit diagnostic that does not fail discovery. +func unitDiagnostic(root string, failure model.UnitFailure) *pluginv1.Diagnostic { + address := &pluginv1.ComponentAddress{Stack: failure.Stack, Component: failure.Name, ComponentType: "terraform"} + var rpc *rpcFailure + if errors.As(scoped(rpcError(failure.Err), "discover", root, address), &rpc) { + diagnostic := rpc.failure.Diagnostic() + // Detail the host may not print can hold evaluated values, so a diagnostic never carries it. + if !diagnostic.Detail.UserSafe { + diagnostic.Detail.Detail = "" + } + return diagnostic + } + return component.NewFailure(model.CodeInternal, fmt.Sprintf("Terragrunt evaluation failed for %s", failure.Name)).Unit(failure.Stack, failure.Name).Operation("discover").Diagnostic() +} + +// settingProblem names an atmos.yaml setting under components.terraform.terragrunt and how to fix it. +func settingProblem(key, summary, hint string) *model.Problem { + return &model.Problem{Code: model.CodeUnsupportedSetting, Summary: summary, Hints: []string{hint}, File: "atmos.yaml", Key: "components.terraform.terragrunt." + key, Safe: true, Err: ErrUnsupported} +} + +// problemFor gives an error without a description a file, so the user can find the unit. +func problemFor(err error, file string) error { + var problem *model.Problem + if errors.As(err, &problem) || file == "" { + return err + } + return &model.Problem{ + Code: model.CodeInvalidSource, Summary: "The unit cannot be described to Atmos", File: file, Safe: false, Detail: err.Error(), Err: err, + Hints: []string{"Check that the unit's `terraform.source` and its dependencies are inside the project root."}, + } +} + +// warningDiagnostics turns discovery findings into diagnostics the host prints without +// failing the command. The messages never contain evaluated values: only paths and patterns. +func warningDiagnostics(warnings []model.Warning) []*pluginv1.Diagnostic { + diagnostics := make([]*pluginv1.Diagnostic, 0, len(warnings)) + for _, warning := range warnings { + diagnostics = append(diagnostics, &pluginv1.Diagnostic{ + Field: warning.Path, + Message: warning.Message, + Detail: &pluginv1.ErrorDetail{ + Code: warning.Code, Summary: warning.Message, UserSafe: true, + Severity: pluginv1.Severity_SEVERITY_WARNING, Operation: "discover", + Location: &pluginv1.SourceLocation{File: warning.Path}, + }, + }) + } + return diagnostics +} diff --git a/plugins/terragrunt/internal/provider/errors_test.go b/plugins/terragrunt/internal/provider/errors_test.go new file mode 100644 index 00000000000..5ea00ab081e --- /dev/null +++ b/plugins/terragrunt/internal/provider/errors_test.go @@ -0,0 +1,177 @@ +package provider + +import ( + "context" + "errors" + "os" + "path/filepath" + "testing" + + "github.com/stretchr/testify/require" + "google.golang.org/grpc/codes" + "google.golang.org/grpc/status" + "google.golang.org/protobuf/encoding/protojson" + + "github.com/cloudposse/atmos/plugins/terragrunt/internal/engine" + "github.com/cloudposse/atmos/plugins/terragrunt/internal/model" + "github.com/cloudposse/atmos/sdk/component" + pluginv1 "github.com/cloudposse/atmos/sdk/proto/atmos/plugin/v1" +) + +func TestUnknownEvaluatorErrorKeepsItsTextHidden(t *testing.T) { + err := rpcError(errors.New("password=hunter2")) + detail, ok := component.FailureDetail(err) + require.True(t, ok) + require.False(t, detail.UserSafe) + require.NotContains(t, status.Convert(err).Message(), "hunter2") + require.Empty(t, detail.Hints, "the host adds the debug hint for withheld detail") + require.Equal(t, codes.InvalidArgument, status.Code(err)) +} + +func TestProblemKeepsItsFieldsAndRelativizesTheFile(t *testing.T) { + root, err := filepath.EvalSymlinks(t.TempDir()) + require.NoError(t, err) + problem := &model.Problem{Code: model.CodeSyntaxError, Summary: "Invalid", File: filepath.Join(root, "live", "dev", "terragrunt.hcl"), Line: 4, Column: 9, Safe: true, Detail: "comma", Hints: []string{"fix it"}} + err = scoped(rpcError(problem), "discover", root, &pluginv1.ComponentAddress{Stack: "live/dev", Component: "vpc"}) + detail, ok := component.FailureDetail(err) + require.True(t, ok) + require.Equal(t, "live/dev/terragrunt.hcl", detail.Location.File) + require.Equal(t, uint32(4), detail.Location.Line) + require.Equal(t, uint32(9), detail.Location.Column) + require.Equal(t, "vpc", detail.Unit) + require.Equal(t, "live/dev", detail.Stack) + require.Equal(t, "discover", detail.Operation) + require.True(t, detail.UserSafe) + require.Equal(t, []string{"fix it"}, detail.Hints) +} + +func TestCancellationAndOutsideFilesAreNotRewritten(t *testing.T) { + require.Equal(t, codes.Canceled, status.Code(rpcError(context.Canceled))) + require.Equal(t, codes.DeadlineExceeded, status.Code(rpcError(context.DeadlineExceeded))) + require.Equal(t, filepath.Join(string(filepath.Separator), "elsewhere", "x.hcl"), projectRelative(t.TempDir(), filepath.Join(string(filepath.Separator), "elsewhere", "x.hcl"))) + require.Equal(t, "atmos.yaml", projectRelative(t.TempDir(), "atmos.yaml")) + plain := errors.New("plain") + require.Equal(t, plain, scoped(plain, "prepare", "", nil)) +} + +func TestSettingsNameTheOffendingKey(t *testing.T) { + for name, test := range map[string]struct { + config map[string]any + key string + }{ + "misspelled key": {map[string]any{"discoverey": map[string]any{}}, "components.terraform.terragrunt.discoverey"}, + "mode": {map[string]any{"mode": "nativ"}, "components.terraform.terragrunt.mode"}, + } { + t.Run(name, func(t *testing.T) { + value, err := encode(test.config) + require.NoError(t, err) + err = settings(value) + var problem *model.Problem + require.ErrorAs(t, err, &problem) + require.Equal(t, test.key, problem.Key) + require.NotEmpty(t, problem.Hints) + detail, ok := component.FailureDetail(rpcError(err)) + require.True(t, ok) + require.Equal(t, test.key, detail.ConfigKey) + }) + } +} + +func TestUnitDiagnosticNamesTheUnit(t *testing.T) { + root := t.TempDir() + diagnostic := unitDiagnostic(root, model.UnitFailure{Stack: "live/dev", Name: "vpc", Err: &model.Problem{Code: model.CodeDependency, Summary: "Missing", Safe: true}}) + require.Equal(t, "vpc", diagnostic.Detail.Unit) + require.Equal(t, "live/dev", diagnostic.Detail.Stack) + require.Equal(t, "discover", diagnostic.Detail.Operation) + require.Equal(t, pluginv1.Severity_SEVERITY_ERROR, diagnostic.Detail.Severity) + require.Equal(t, "Missing", diagnostic.Message) +} + +// Detail text that is not marked safe can hold evaluated values, so a discovery diagnostic omits it +// while it keeps the headline, hints, and location. +func TestUnitDiagnosticOmitsUnsafeDetail(t *testing.T) { + root, err := filepath.EvalSymlinks(t.TempDir()) + require.NoError(t, err) + for _, test := range []struct { + name string + safe bool + want string + }{ + {"unsafe detail is omitted", false, ""}, + {"safe detail is kept", true, "evaluated value hunter2"}, + } { + t.Run(test.name, func(t *testing.T) { + diagnostic := unitDiagnostic(root, model.UnitFailure{Stack: "live", Name: "vpc", Err: &model.Problem{ + Code: model.CodeEvaluationError, Summary: "Could not evaluate", Detail: "evaluated value hunter2", Safe: test.safe, + File: filepath.Join(root, "live", "vpc", "terragrunt.hcl"), Line: 4, Column: 2, Hints: []string{"Check the unit."}, + }}) + require.Equal(t, test.want, diagnostic.Detail.Detail) + require.Equal(t, test.safe, diagnostic.Detail.UserSafe) + require.Equal(t, "Could not evaluate", diagnostic.Message) + require.Equal(t, "live/vpc/terragrunt.hcl", diagnostic.Detail.Location.File) + require.Equal(t, uint32(4), diagnostic.Detail.Location.Line) + require.Equal(t, []string{"Check the unit."}, diagnostic.Detail.Hints) + if !test.safe { + wire, err := protojson.Marshal(diagnostic) + require.NoError(t, err) + require.NotContains(t, string(wire), "hunter2") + } + }) + } +} + +func TestDiscoveryReportsAFailedUnitAndKeepsTheHealthyOnes(t *testing.T) { + req := componentRequest(t) + healthy := model.DiscoveredUnit{Stack: "live", Name: "ok", Config: model.Resolution{Path: filepath.Join(req.RootDirectory, "live", "ok", "terragrunt.hcl"), Folders: []string{filepath.Join(req.RootDirectory, "module")}}} + p := New(func(context.Context, model.Request) (*model.Resolution, error) { + return &model.Resolution{ + Inventory: []model.DiscoveredUnit{healthy}, + Failures: []model.UnitFailure{{Stack: "live", Name: "broken", Err: &model.Problem{Code: model.CodeSyntaxError, Summary: "Invalid", Safe: true}}}, + }, nil + }) + response, err := p.Discover(t.Context(), &pluginv1.DiscoverComponentsRequest{RootDirectory: req.RootDirectory, Provider: req.Provider, ProviderConfiguration: req.ProviderConfiguration, Operation: req.Operation, IncludePaths: []string{"live/**"}}) + require.NoError(t, err) + require.Len(t, response.Components, 1) + require.Equal(t, "ok", response.Components[0].Address.Component) + require.Len(t, response.Diagnostics, 1) + require.Equal(t, "broken", response.Diagnostics[0].Detail.Unit) +} + +// Exercise the evaluator and both provider output paths with an environment-derived target. +func TestDependencyDiagnosticsKeepEvaluatedTargetsHidden(t *testing.T) { + for _, block := range []string{ + "dependency \"upstream\" { config_path = get_env(\"DEPENDENCY_TARGET\") }", + "dependencies { paths = [get_env(\"DEPENDENCY_TARGET\")] }", + } { + t.Run(block, func(t *testing.T) { + root, err := filepath.EvalSymlinks(t.TempDir()) + require.NoError(t, err) + file := filepath.Join(root, "terragrunt.hcl") + require.NoError(t, os.WriteFile(file, []byte(block), 0o600)) + const secret = "secret-dependency-target" + _, err = engine.Resolve(t.Context(), model.Request{ + Path: file, BasePath: root, Phase: "discover", Command: "plan", Tofu: "must-not-execute", + Environment: map[string]string{"DEPENDENCY_TARGET": "./" + secret}, + }) + var problem *model.Problem + require.ErrorAs(t, err, &problem) + require.Equal(t, model.CodeDependency, problem.Code) + require.ErrorIs(t, err, os.ErrNotExist) + require.Contains(t, problem.Err.Error(), secret, "the cause remains available locally") + rpc := scoped(rpcError(err), "discover", root, &pluginv1.ComponentAddress{Stack: "live", Component: "unit"}) + require.NotContains(t, rpc.Error(), secret) + wire, marshalErr := protojson.Marshal(status.Convert(rpc).Proto()) + require.NoError(t, marshalErr) + require.NotContains(t, string(wire), secret) + diagnostic := unitDiagnostic(root, model.UnitFailure{Stack: "live", Name: "unit", Err: err}) + wire, marshalErr = protojson.Marshal(diagnostic) + require.NoError(t, marshalErr) + require.NotContains(t, string(wire), secret) + require.True(t, diagnostic.Detail.UserSafe) + require.Equal(t, "terragrunt.hcl", diagnostic.Detail.Location.File) + require.Equal(t, "dependency", diagnostic.Detail.ConfigKey) + require.Equal(t, "unit", diagnostic.Detail.Unit) + require.NotEmpty(t, diagnostic.Detail.Hints) + }) + } +} diff --git a/plugins/terragrunt/internal/provider/fixtures_test.go b/plugins/terragrunt/internal/provider/fixtures_test.go index c3cec2d24d2..7e1a798787f 100644 --- a/plugins/terragrunt/internal/provider/fixtures_test.go +++ b/plugins/terragrunt/internal/provider/fixtures_test.go @@ -113,6 +113,7 @@ func TestExplicitStackStateAndPreparation(t *testing.T) { require.Len(t, prepared.Files, 3) require.Equal(t, "backend.tf", prepared.Files[0].Path) require.Contains(t, string(prepared.Files[0].Content), `backend "local"`) + require.True(t, prepared.Files[0].Sensitive, "the host keeps generated backend bytes out of saved-plan records") require.Equal(t, "terragrunt.hcl", prepared.Files[1].Path) require.Equal(t, "terragrunt.values.hcl", prepared.Files[2].Path) require.NotNil(t, prepared.Inputs) @@ -172,3 +173,31 @@ func TestRemoteModuleSourceIsNotAFileWatch(t *testing.T) { require.Len(t, instance.Sources, 1) require.Equal(t, "live/app/terragrunt.hcl", instance.Sources[0].Path) } + +func TestDerivedAddressNamesAnUnscopedUnit(t *testing.T) { + root, err := filepath.EvalSymlinks(t.TempDir()) + require.NoError(t, err) + for path, want := range map[string]*pluginv1.ComponentAddress{ + filepath.Join(root, "live", "dev", "vpc", "terragrunt.hcl"): {Stack: "live/dev", Component: "vpc", ComponentType: "terraform"}, + filepath.Join(root, "vpc", "terragrunt.hcl"): {Stack: "default", Component: "vpc", ComponentType: "terraform"}, + filepath.Join(root, "terragrunt.hcl"): {Stack: "default", Component: "root", ComponentType: "terraform"}, + } { + got, err := derivedAddress(root, path) + require.NoError(t, err) + require.Equal(t, want.Stack, got.Stack) + require.Equal(t, want.Component, got.Component) + require.Equal(t, want.ComponentType, got.ComponentType) + } + _, err = derivedAddress(root, filepath.Join(filepath.Dir(root), "elsewhere", "terragrunt.hcl")) + require.ErrorIs(t, err, ErrUnsupported) +} + +func TestWarningDiagnosticsNeverFailTheCommand(t *testing.T) { + diagnostics := warningDiagnostics([]model.Warning{{Code: model.WarningSymlinkedDirectory, Path: "live/linked", Message: "symlinked directory live/linked was not followed"}}) + require.Len(t, diagnostics, 1) + require.Equal(t, pluginv1.Severity_SEVERITY_WARNING, diagnostics[0].Detail.Severity) + require.Equal(t, model.WarningSymlinkedDirectory, diagnostics[0].Detail.Code) + require.True(t, diagnostics[0].Detail.UserSafe) + require.Equal(t, "live/linked", diagnostics[0].Detail.Location.File) + require.Empty(t, warningDiagnostics(nil)) +} diff --git a/plugins/terragrunt/internal/provider/preparation.go b/plugins/terragrunt/internal/provider/preparation.go index e28beb0c30a..550585a2c7f 100644 --- a/plugins/terragrunt/internal/provider/preparation.go +++ b/plugins/terragrunt/internal/provider/preparation.go @@ -52,6 +52,11 @@ func (p *Provider) componentRequest(ctx context.Context, request *pluginv1.Compo // //nolint:lintroller // Standalone plugin code cannot import the host performance tracker. func (p *Provider) Prepare(ctx context.Context, request *pluginv1.ComponentRequest) (*pluginv1.PrepareComponentResponse, error) { + response, err := p.prepare(ctx, request) + return response, scoped(err, "prepare", request.GetRootDirectory(), request.GetAddress()) +} + +func (p *Provider) prepare(ctx context.Context, request *pluginv1.ComponentRequest) (*pluginv1.PrepareComponentResponse, error) { req, err := p.componentRequest(ctx, request) if err != nil { return nil, rpcError(err) @@ -170,7 +175,7 @@ func preparedFiles(result *model.Resolution) []*pluginv1.ComponentGeneratedFile if _, generated := result.Generated[name]; generated { mode = 0 } - filesOut = append(filesOut, &pluginv1.ComponentGeneratedFile{Path: name, Content: files[name], IfExists: policy, Mode: mode}) + filesOut = append(filesOut, &pluginv1.ComponentGeneratedFile{Path: name, Content: files[name], IfExists: policy, Sensitive: result.SensitiveGenerated[name], UnitFileCopy: result.UnitFileCopies[name], Mode: mode}) } return filesOut } @@ -199,6 +204,11 @@ func stateIdentity(req *model.Request, result *model.Resolution) (*pluginv1.Comp // //nolint:lintroller // Standalone plugin code cannot import the host performance tracker. func (p *Provider) Resolve(ctx context.Context, request *pluginv1.ComponentRequest) (*pluginv1.ResolveComponentStateResponse, error) { + response, err := p.resolveState(ctx, request) + return response, scoped(err, "resolve state", request.GetRootDirectory(), request.GetAddress()) +} + +func (p *Provider) resolveState(ctx context.Context, request *pluginv1.ComponentRequest) (*pluginv1.ResolveComponentStateResponse, error) { req, err := p.componentRequest(ctx, request) if err != nil { return nil, rpcError(err) diff --git a/plugins/terragrunt/internal/provider/provider.go b/plugins/terragrunt/internal/provider/provider.go index 4d2f1994a09..07478c501a0 100644 --- a/plugins/terragrunt/internal/provider/provider.go +++ b/plugins/terragrunt/internal/provider/provider.go @@ -5,15 +5,13 @@ import ( "context" "encoding/json" "errors" + "fmt" "maps" "path/filepath" "slices" "strings" ctyjson "github.com/zclconf/go-cty/cty/json" - "google.golang.org/genproto/googleapis/rpc/errdetails" - "google.golang.org/grpc/codes" - "google.golang.org/grpc/status" "github.com/cloudposse/atmos/plugins/terragrunt/internal/engine" "github.com/cloudposse/atmos/plugins/terragrunt/internal/model" @@ -84,15 +82,15 @@ func settings(value *pluginv1.ComponentValue) error { if err := decode(value, &config); err != nil { return err } - for key := range config { + for _, key := range slices.Sorted(maps.Keys(config)) { switch key { case "enabled", "provider", "mode", "discovery": default: - return ErrUnsupported + return settingProblem(key, fmt.Sprintf("Unknown setting `components.terraform.terragrunt.%s`", key), "Valid keys are `enabled`, `provider`, `mode`, and `discovery`. Check the spelling.") } } if mode, _ := config["mode"].(string); mode != "" && mode != "native" { - return ErrUnsupported + return settingProblem("mode", "Unsupported value for `components.terraform.terragrunt.mode`", "The only supported mode is `native`. Remove `mode` or set `mode: native`.") } return nil } @@ -156,45 +154,12 @@ func sources(root string, resolved *model.Resolution) ([]*pluginv1.ComponentSour return result, nil } -func rpcError(err error) error { - if errors.Is(err, context.Canceled) { - return status.Error(codes.Canceled, "Terragrunt evaluation canceled") - } - if errors.Is(err, context.DeadlineExceeded) { - return status.Error(codes.DeadlineExceeded, "Terragrunt evaluation timed out") - } - // HCL diagnostics can contain secrets. Do not copy evaluator payloads into RPC errors. - if errors.Is(err, ErrUnsupported) { - return status.Error(codes.Unimplemented, ErrUnsupported.Error()) - } - // Unsupported hook semantics name only hook identifiers and commands, so the reason is safe to show. - if errors.Is(err, engine.ErrUnsupportedExecution) { - return status.Error(codes.Unimplemented, err.Error()) - } - if errors.Is(err, engine.ErrPreventDestroy) { - return status.Error(codes.FailedPrecondition, engine.ErrPreventDestroy.Error()) - } - result := status.New(codes.InvalidArgument, "Terragrunt evaluation failed") - var diagnostic *engine.DiscoveryError - if errors.As(err, &diagnostic) { - // Never serialize the wrapped evaluator error or any evaluated HCL values. - detailed, detailErr := result.WithDetails(&errdetails.ErrorInfo{ - Domain: "atmos.terragrunt.discovery.v1", Reason: diagnostic.Category, - Metadata: map[string]string{"file": diagnostic.File}, - }) - if detailErr == nil { - return detailed.Err() - } - } - return result.Err() -} - // Validate rejects unsupported operation and provider settings without evaluating HCL. // //nolint:lintroller // Standalone plugin code cannot import the host performance tracker. func (p *Provider) Validate(ctx context.Context, request *pluginv1.ComponentRequest) (*pluginv1.ValidateResponse, error) { if err := validate(ctx, request); err != nil { - return nil, rpcError(err) + return nil, scoped(rpcError(err), "validate", request.GetRootDirectory(), request.GetAddress()) } return &pluginv1.ValidateResponse{}, nil } diff --git a/plugins/terragrunt/internal/provider/provider_test.go b/plugins/terragrunt/internal/provider/provider_test.go index 91e6ed82105..1777dc05d31 100644 --- a/plugins/terragrunt/internal/provider/provider_test.go +++ b/plugins/terragrunt/internal/provider/provider_test.go @@ -11,12 +11,14 @@ import ( "time" "github.com/stretchr/testify/require" - "google.golang.org/genproto/googleapis/rpc/errdetails" "google.golang.org/grpc/codes" "google.golang.org/grpc/status" + "google.golang.org/protobuf/encoding/prototext" + "google.golang.org/protobuf/proto" "github.com/cloudposse/atmos/plugins/terragrunt/internal/engine" "github.com/cloudposse/atmos/plugins/terragrunt/internal/model" + "github.com/cloudposse/atmos/sdk/component" pluginv1 "github.com/cloudposse/atmos/sdk/proto/atmos/plugin/v1" ) @@ -199,12 +201,45 @@ func TestPrepareOmitsExecutionWithoutHooks(t *testing.T) { // An unsupported hook must reach the user with its specific reason, unlike generic evaluator // failures whose payloads can hold secrets. func TestUnsupportedExecutionIsReportedSpecifically(t *testing.T) { - cause := fmt.Errorf("%w: before_hook %q lists the %q lifecycle command", engine.ErrUnsupportedExecution, "setup", "init-from-module") + cause := &model.Problem{Code: model.CodeUnsupportedHook, Summary: "Unsupported hook command `init-from-module` in before_hook \"setup\"", Key: "before_hook \"setup\"", Safe: true, Detail: "never runs", Hints: []string{"Remove it."}, File: "live/unit/terragrunt.hcl", Err: engine.ErrUnsupportedExecution} p := New(func(context.Context, model.Request) (*model.Resolution, error) { return nil, cause }) - _, err := p.Prepare(t.Context(), componentRequest(t)) + request := componentRequest(t) + request.Address = &pluginv1.ComponentAddress{Stack: "live", Component: "app", ComponentType: "terraform"} + _, err := p.Prepare(t.Context(), request) require.Equal(t, codes.Unimplemented, status.Code(err)) require.Contains(t, err.Error(), "init-from-module") require.Contains(t, err.Error(), "setup") + detail, ok := component.FailureDetail(err) + require.True(t, ok) + require.Equal(t, "prepare", detail.Operation) + require.Equal(t, "live/unit/terragrunt.hcl", detail.Location.File) + require.True(t, detail.UserSafe) + require.Equal(t, []string{"Remove it."}, detail.Hints) + require.NotEmpty(t, detail.Unit) +} + +func TestPreparationTransmitsUnitFileCopyProvenance(t *testing.T) { + for _, copied := range []bool{false, true} { + t.Run(map[bool]string{false: "explicit generation", true: "adjacent copy"}[copied], func(t *testing.T) { + request := componentRequest(t) + p := New(func(_ context.Context, _ model.Request) (*model.Resolution, error) { + return &model.Resolution{ + Generated: map[string]string{".terraform.lock.hcl": "# pinned\n"}, + GeneratedPolicies: map[string]string{".terraform.lock.hcl": "overwrite"}, + UnitFileCopies: map[string]bool{".terraform.lock.hcl": copied}, + }, nil + }) + result, err := p.Prepare(t.Context(), request) + require.NoError(t, err) + require.Len(t, result.Files, 1) + wire, err := proto.Marshal(result) + require.NoError(t, err) + decoded := &pluginv1.PrepareComponentResponse{} + require.NoError(t, proto.Unmarshal(wire, decoded)) + require.Equal(t, copied, decoded.Files[0].UnitFileCopy) + require.Equal(t, "overwrite", result.Files[0].IfExists) + }) + } } func TestPreparationsSerializeGenerationAndParsing(t *testing.T) { @@ -244,36 +279,94 @@ func TestPreparationsSerializeGenerationAndParsing(t *testing.T) { } } +// A unit that fails discovery is reported as a diagnostic for that unit; neither evaluated +// values nor the absolute project path reach the response. func TestDiscoveryDiagnosticsRedactPayloads(t *testing.T) { - for _, test := range []struct{ name, content, reason string }{ - {"inline", "locals { password = \"secret-evaluated-value\" }", "MISSING_TERRAFORM_SOURCE"}, - {"parse", "locals { password = \"secret-evaluated-value\"", "EVALUATION_FAILED"}, + for _, test := range []struct{ name, content string }{ + {"inline", "locals { password = \"secret-evaluated-value\" }"}, + {"parse", "locals { password = \"secret-evaluated-value\""}, } { t.Run(test.name, func(t *testing.T) { request := componentRequest(t) require.NoError(t, os.WriteFile(filepath.Join(request.RootDirectory, "terragrunt.hcl"), []byte(test.content), 0o600)) require.NoError(t, os.WriteFile(filepath.Join(request.RootDirectory, "main.tf"), []byte("terraform {}"), 0o600)) - _, err := New(nil).Discover(t.Context(), &pluginv1.DiscoverComponentsRequest{ + response, err := New(nil).Discover(t.Context(), &pluginv1.DiscoverComponentsRequest{ RootDirectory: request.RootDirectory, Provider: request.Provider, ProviderConfiguration: request.ProviderConfiguration, Operation: request.Operation, IncludePaths: []string{"**"}, }) - require.Equal(t, codes.InvalidArgument, status.Code(err)) - details := status.Convert(err).Details() - require.Len(t, details, 1) - info := details[0].(*errdetails.ErrorInfo) - require.Equal(t, "atmos.terragrunt.discovery.v1", info.Domain) - require.Equal(t, test.reason, info.Reason) - require.Equal(t, map[string]string{"file": "terragrunt.hcl"}, info.Metadata) - require.NotContains(t, status.Convert(err).Proto().String(), "secret-evaluated-value") - require.NotContains(t, status.Convert(err).Proto().String(), request.RootDirectory) + require.NoError(t, err) + require.Empty(t, response.GetComponents()) + require.NotEmpty(t, response.GetDiagnostics(), "the failed unit is reported") + serialized := prototext.Format(response) + require.NotContains(t, serialized, "secret-evaluated-value") + require.NotContains(t, serialized, request.RootDirectory) + }) + } +} + +// Scheduling edges must be available from discovery, before preparation can read outputs. +func TestDiscoveryOutputDependencyOrdering(t *testing.T) { + for _, test := range []struct { + name, blocks string + kinds []pluginv1.ComponentDependency_Kind + }{ + {"output only", `dependency "network" { config_path = "../network" }`, []pluginv1.ComponentDependency_Kind{pluginv1.ComponentDependency_OUTPUT}}, + {"output and ordering", `dependency "network" { config_path = "../network" } +dependencies { paths = ["../network", "../network"] }`, []pluginv1.ComponentDependency_Kind{pluginv1.ComponentDependency_OUTPUT}}, + {"ordering only", `dependencies { paths = ["../network"] }`, []pluginv1.ComponentDependency_Kind{pluginv1.ComponentDependency_ORDER}}, + {"skip outputs", `dependency "network" { + config_path = "../network" + skip_outputs = true +}`, []pluginv1.ComponentDependency_Kind{pluginv1.ComponentDependency_ORDER}}, + {"disabled", `dependency "network" { + config_path = "../network" + enabled = false +}`, nil}, + } { + t.Run(test.name, func(t *testing.T) { + root, err := filepath.EvalSymlinks(t.TempDir()) + require.NoError(t, err) + for _, dir := range []string{"module", "live/app", "live/network"} { + require.NoError(t, os.MkdirAll(filepath.Join(root, dir), 0o755)) + } + config := "terraform { source = \"../../module\" }\n" + require.NoError(t, os.WriteFile(filepath.Join(root, "live", "network", "terragrunt.hcl"), []byte(config), 0o600)) + require.NoError(t, os.WriteFile(filepath.Join(root, "live", "app", "terragrunt.hcl"), []byte(config+test.blocks+"\ninputs = { deferred = run_cmd(\"must-not-execute\") }\n"), 0o600)) + opts, err := encode(map[string]any{"enabled": true, "mode": "native", "discovery": map[string]any{"enabled": true}}) + require.NoError(t, err) + var phases []string + p := New(func(ctx context.Context, req model.Request) (*model.Resolution, error) { + phases = append(phases, req.Phase) + return engine.Resolve(ctx, req) + }) + response, err := p.Discover(t.Context(), &pluginv1.DiscoverComponentsRequest{ + RootDirectory: root, Provider: "terragrunt", ProviderConfiguration: opts, + IncludePaths: []string{"live/**"}, Operation: &pluginv1.ComponentOperation{Mode: "native", Command: "plan"}, + Invocation: &pluginv1.Invocation{Environment: map[string]string{"ATMOS_TERRAFORM_BINARY": "must-not-execute"}}, + }) + require.NoError(t, err) + require.Empty(t, response.Diagnostics) + require.Equal(t, []string{"inventory"}, phases) + require.Len(t, response.Components, 2) + app := response.Components[0] + network := response.Components[1] + require.Equal(t, "app", app.Address.Component) + require.Equal(t, "network", network.Address.Component) + require.Len(t, app.Dependencies, len(test.kinds)) + for i, kind := range test.kinds { + require.Equal(t, kind, app.Dependencies[i].Kind) + require.Equal(t, network.Address, app.Dependencies[i].Target) + } + require.Empty(t, network.Dependencies) + require.NoDirExists(t, filepath.Join(root, "live", "app", ".terragrunt-cache")) }) } } func TestRPCErrorPreservesDestroyProtection(t *testing.T) { err := rpcError(fmt.Errorf("private evaluator context: %w", engine.ErrPreventDestroy)) - require.Equal(t, codes.FailedPrecondition, status.Code(err)) - require.Equal(t, engine.ErrPreventDestroy.Error(), status.Convert(err).Message()) + require.Equal(t, codes.InvalidArgument, status.Code(err)) + require.NotContains(t, status.Convert(err).Message(), "private evaluator context") } func TestPreparationExportsOverlayModes(t *testing.T) { diff --git a/plugins/terragrunt/plugin.tests.yaml b/plugins/terragrunt/plugin.tests.yaml index ee00aa57c8e..99f79bf95ad 100644 --- a/plugins/terragrunt/plugin.tests.yaml +++ b/plugins/terragrunt/plugin.tests.yaml @@ -52,3 +52,69 @@ integration: script: | lifecycle = module.load("scripts/lifecycle_extended.star") lifecycle.local_source_refresh() + - name: imports-and-discovery + type: script + interpreter: starlark + script: | + imports = module.load("scripts/imports.star") + imports.imports_and_discovery() + - name: stack-overrides + type: script + interpreter: starlark + script: | + imports = module.load("scripts/imports.star") + imports.stack_overrides() + - name: stack-pinned-tools + type: script + interpreter: starlark + script: | + imports = module.load("scripts/imports.star") + imports.stack_pinned_tools() + - name: diagnostics + type: script + interpreter: starlark + script: | + diagnostics = module.load("scripts/diagnostics.star") + diagnostics.diagnostics() + - name: settings-repair + type: script + interpreter: starlark + script: | + diagnostics = module.load("scripts/diagnostics.star") + diagnostics.settings_repair() + - name: saved-plan-lifecycle + type: script + interpreter: starlark + script: | + hardening = module.load("scripts/hardening.star") + hardening.saved_plan_lifecycle() + - name: local-source-stability + type: script + interpreter: starlark + script: | + hardening = module.load("scripts/hardening.star") + hardening.local_source_stability() + - name: generation-lifecycle + type: script + interpreter: starlark + script: | + hardening = module.load("scripts/hardening.star") + hardening.generation_lifecycle() + - name: state-identity-warning + type: script + interpreter: starlark + script: | + hardening = module.load("scripts/hardening.star") + hardening.cache_state_warning() + - name: hook-semantics + type: script + interpreter: starlark + script: | + hardening = module.load("scripts/hardening.star") + hardening.hook_semantics() + - name: pinned-tools + type: script + interpreter: starlark + script: | + hardening = module.load("scripts/hardening.star") + hardening.pinned_tools() diff --git a/plugins/terragrunt/schemas/config.json b/plugins/terragrunt/schemas/config.json index b86b276a57c..bc02cf66490 100644 --- a/plugins/terragrunt/schemas/config.json +++ b/plugins/terragrunt/schemas/config.json @@ -22,7 +22,7 @@ "items": { "type": "string" }, - "minItems": 1 + "description": "Project-relative patterns that scope automatic discovery. Defaults to every unit below the project root (`**`) when discovery is enabled." }, "excluded_paths": { "type": "array", @@ -32,15 +32,13 @@ } }, "required": [ - "enabled", - "included_paths" + "enabled" ], "additionalProperties": false } }, "required": [ - "enabled", - "discovery" + "enabled" ], "additionalProperties": false } diff --git a/plugins/terragrunt/scripts/affected.star b/plugins/terragrunt/scripts/affected.star index 5bf56ec0a09..b7e7248c67e 100644 --- a/plugins/terragrunt/scripts/affected.star +++ b/plugins/terragrunt/scripts/affected.star @@ -185,13 +185,22 @@ inputs = { fixtures.commit(root, "delete glob member") expect_components(affected(root, base), ["app"]) - # Unknown path expressions watch project sources conservatively, never caches. + # Locals that resolve to literals name an exact file, so unrelated project files select nothing. fixtures.write(root, "live/prod/app/terragrunt.hcl", unit("network") + ''' locals { input_path = "config.json" } inputs = { value = file(local.input_path) } ''') - base = fixtures.commit(root, "declare dynamic file input") + base = fixtures.commit(root, "declare literal-local file input") fixtures.write(root, "new-directory/config.json", "{}") + fixtures.commit(root, "create unrelated file with the same name") + expect_components(affected(root, base), []) + + # Unknown path expressions (the environment is never read) watch project sources conservatively, never caches. + fixtures.write(root, "live/prod/app/terragrunt.hcl", unit("network") + ''' +inputs = { value = file("${get_env("INPUT_DIR", "")}/config.json") } +''') + base = fixtures.commit(root, "declare dynamic file input") + fixtures.write(root, "new-directory/config.json", '{"possible": "input"}') fixtures.commit(root, "create possible dynamic input") expect_components(affected(root, base), ["app"]) base = fixtures.git(root, "rev-parse", "HEAD") @@ -200,6 +209,130 @@ inputs = { value = file(local.input_path) } expect_components(affected(root, base), []) +def selected_after(root, label, *edits): + # Commit the edits on top of the current HEAD and report which units the change selects. + base = fixtures.git(root, "rev-parse", "HEAD") + for path, content in edits: + fixtures.write(root, path, content) + fixtures.commit(root, label) + return sorted([row["component"] for row in affected(root, base)]) + + +def expect_selected(root, label, expected, *edits): + actual = selected_after(root, label, *edits) + fixtures.check(actual == sorted(expected), label + ": expected " + str(expected) + ", selected " + str(actual)) + + +HOOKS_UNIT = ''' +terraform { + source = "../../../modules/unit" + extra_arguments "vars" { + commands = ["plan"] + required_var_files = ["${get_terragrunt_dir()}/extra.tfvars"] + optional_var_files = ["${get_terragrunt_dir()}/optional.tfvars"] + } + before_hook "check" { + commands = ["plan"] + execute = ["../../../scripts/hook.sh"] + } +} +generate "extra" { + path = "extra.tf" + if_exists = "overwrite" + contents = file("gen.tpl") +} +''' + + +def test_unevaluated_blocks(root): + # Files named only by extra_arguments, generate, and hooks are not evaluated during + # discovery, yet an edit to any of them changes what the unit plans. + seed_units(root) + fixtures.write(root, "live/prod/hooks/terragrunt.hcl", HOOKS_UNIT) + fixtures.write(root, "live/prod/hooks/extra.tfvars", "name = \"one\"\n") + fixtures.write(root, "live/prod/hooks/gen.tpl", "# generated one\n") + fixtures.write(root, "scripts/hook.sh", "echo one\n") + fixtures.write(root, "docs/guide.md", "guide\n") + fixtures.commit(root, "unit with var files, generate, and hook") + expect_selected(root, "required var file", ["hooks"], ("live/prod/hooks/extra.tfvars", "name = \"two\"\n")) + expect_selected(root, "optional var file created", ["hooks"], ("live/prod/hooks/optional.tfvars", "name = \"three\"\n")) + expect_selected(root, "generate template", ["hooks"], ("live/prod/hooks/gen.tpl", "# generated two\n")) + expect_selected(root, "hook script", ["hooks"], ("scripts/hook.sh", "echo two\n")) + expect_selected(root, "lock file beside the unit", ["hooks"], ("live/prod/hooks/.terraform.lock.hcl", "# lock\n")) + expect_selected(root, "unrelated documentation", [], ("docs/guide.md", "guide two\n")) + + +def test_copied_source_root(root): + # Terragrunt copies everything before `//`, so a sibling of the selected subdirectory counts. + seed_units(root) + fixtures.write(root, "modules/sub/app/main.tf", 'module "shared" { source = "../shared" }\n') + fixtures.write(root, "modules/sub/shared/main.tf", 'output "v" { value = 1 }\n') + fixtures.write(root, "live/prod/subdir/terragrunt.hcl", 'terraform { source = "../../../modules/sub//app" }\n') + fixtures.commit(root, "unit selecting a module subdirectory") + expect_selected(root, "sibling of the selected subdirectory", ["subdir"], ("modules/sub/shared/main.tf", 'output "v" { value = 2 }\n')) + expect_selected(root, "selected subdirectory", ["subdir"], ("modules/sub/app/main.tf", '# edited\nmodule "shared" { source = "../shared" }\n')) + expect_selected(root, "unrelated module", [], ("modules/unused/main.tf", "# unused\n"), ("README.md", "readme\n")) + + +def test_symlinked_module(root): + # Git reports edits under the symlink target, not the path the unit names. + seed_units(root) + fixtures.write(root, "modules/other/main.tf", 'output "vpc_id" { value = "other" }\n') + fixtures.write(root, "scratch/link-target", "other") + blob = fixtures.git(root, "hash-object", "-w", "scratch/link-target") + fs.remove(root + "/scratch", recursive=True) + fixtures.git(root, "update-index", "--add", "--cacheinfo", "120000," + blob + ",modules/linked") + fixtures.git(root, "checkout-index", "-f", "modules/linked") + if not fs.stat(root + "/modules/linked", follow_symlinks=False).is_symlink: + ui.info("Symbolic links are unavailable on this host; skipping the symlinked module scenario") + return + fixtures.write(root, "live/prod/linked/terragrunt.hcl", 'terraform { source = "../../../modules/linked" }\n') + fixtures.write(root, "live/prod/direct/terragrunt.hcl", 'terraform { source = "../../../modules/other" }\n') + fixtures.commit(root, "units using a symlinked module and its target") + expect_selected(root, "edit through the symlink target", ["direct", "linked"], ("modules/other/main.tf", 'output "vpc_id" { value = "edited" }\n')) + expect_selected(root, "unrelated module", [], ("modules/unused/main.tf", "# unused\n")) + + +def test_dynamic_path_precision(root): + # get_terragrunt_dir() style paths resolve to exact files; only unresolvable expressions + # (here: the environment) keep the conservative project-wide watch. + seed_units(root) + fixtures.write(root, "live/prod/exact/terragrunt.hcl", unit() + ''' +locals { name = "settings" } +inputs = { + a = file("${get_terragrunt_dir()}/a.json") + b = file(format("%s/%s.json", get_terragrunt_dir(), local.name)) + c = file(find_in_parent_folders("shared.json")) +} +''') + fixtures.write(root, "live/prod/exact/a.json", "{}") + fixtures.write(root, "live/prod/exact/settings.json", "{}") + fixtures.write(root, "live/shared.json", "{}") + fixtures.write(root, "live/prod/env/terragrunt.hcl", unit() + 'inputs = { a = file("${get_env("DYNAMIC_DIR", "")}/a.json") }\n') + fixtures.commit(root, "units with dynamic file inputs") + expect_selected(root, "unrelated files", ["env"], ("README.md", "readme\n"), ("docs/guide.md", "guide\n")) + expect_selected(root, "get_terragrunt_dir file", ["env", "exact"], ("live/prod/exact/a.json", '{"a": 1}')) + expect_selected(root, "format and local file", ["env", "exact"], ("live/prod/exact/settings.json", '{"b": 1}')) + expect_selected(root, "find_in_parent_folders file", ["env", "exact"], ("live/shared.json", '{"c": 1}')) + + +def test_submodule_module_source(root): + # A submodule is empty in the BASE checkout; comparing a revision with itself selects nothing. + seed_units(root) + repository = fs.mkdtemp(prefix="atmos-terragrunt-submodule-") + defer(fs.remove, repository, recursive=True) + fixtures.git(repository, "init", "-b", "main") + fixtures.git(repository, "config", "user.name", "Atmos fixture") + fixtures.git(repository, "config", "user.email", "fixture@example.invalid") + fixtures.write(repository, "main.tf", 'output "vpc_id" { value = "submodule" }\n') + fixtures.commit(repository, "submodule module") + fixtures.git(root, "-c", "protocol.file.allow=always", "submodule", "add", repository, "modules/submodule") + fixtures.write(root, "live/prod/submodule/terragrunt.hcl", 'terraform { source = "../../../modules/submodule" }\n') + fixtures.commit(root, "unit using a submodule module") + fixtures.check(not affected(root, "HEAD"), "A submodule module made an unchanged unit affected") + expect_selected(root, "unrelated change", [], ("README.md", "readme\n")) + + def test_explicit_imports(root, config): # Explicit imports opt into roots without enabling automatic filesystem discovery. seed_units(root) @@ -224,6 +357,11 @@ def main(): ("deleted-unit", test_deleted_unit), ("file-watches-and-rename", test_file_watches_and_rename), ("input-watches", test_input_watches), + ("unevaluated-blocks", test_unevaluated_blocks), + ("copied-source-root", test_copied_source_root), + ("symlinked-module", test_symlinked_module), + ("dynamic-path-precision", test_dynamic_path_precision), + ("submodule-module-source", test_submodule_module_source), ]: root = work + "/" + name fixtures.project(root, package) diff --git a/plugins/terragrunt/scripts/diagnostics.star b/plugins/terragrunt/scripts/diagnostics.star new file mode 100644 index 00000000000..a6b3a56b064 --- /dev/null +++ b/plugins/terragrunt/scripts/diagnostics.star @@ -0,0 +1,185 @@ +# Opt-in CLI acceptance tests for plugin diagnostics: every rejection names the unit, the file, and +# the setting; one bad unit never breaks the project; bulk runs validate every selected unit before +# any infrastructure command runs; and `atmos config set` can always repair plugin settings. +# Rendered messages drop Markdown backticks, so expectations name plain text. +# Fixtures are generated here and use only built-in terraform_data resources. +fixtures = module.load("fixtures.star") + +STACK = "live/dev" + +MODULE = """ +resource "terraform_data" "unit" { + input = "ok" +} +""" + +BACKEND = """ +remote_state { + backend = "local" + config = { path = "${get_terragrunt_dir()}/state/terraform.tfstate" } + generate = { path = "backend.tf", if_exists = "overwrite" } +} +""" + +GOOD = 'terraform {\n source = "../../../modules/unit"\n}\n' + BACKEND + + +def unit(source_block): + return 'terraform {\n source = "../../../modules/unit"\n' + source_block + "}\n" + BACKEND + + +def project(prefix, binary, units): + root = fixtures.workspace(prefix) + config = fixtures.configuration(fs.resolve("."), binary=binary) + config["stacks"] = {"base_path": "stacks", "included_paths": ["**/*.yaml"]} + fixtures.write(root, "atmos.yaml", json.encode(config)) + fixtures.write(root, "modules/unit/main.tf", MODULE) + # An ordinary YAML stack proves non-Terragrunt components keep working beside a broken unit. + fixtures.write(root, "stacks/plain.yaml", "components:\n terraform:\n plain:\n vars: {}\n") + for name in units: + fixtures.write(root, "live/dev/" + name + "/terragrunt.hcl", units[name]) + return root + + +def output_of(result): + return result.stdout + result.stderr + + +def expect(output, fragments, context): + for fragment in fragments: + fixtures.check(fragment in output, context + " is missing '" + fragment + "':\n" + output) + + +def forbid(output, fragments, context): + for fragment in fragments: + fixtures.check(fragment not in output, context + " must not contain '" + fragment + "':\n" + output) + + +def plan(root, name): + return output_of(fixtures.run_failing(root, ["terraform", "plan", name, "-s", STACK, "--", "-no-color"])) + + +def verify_rejections_name_file_unit_and_setting(root): + # An HCL syntax error reports the file with line and column. + syntax = plan(root, "syntax") + expect(syntax, ["Missing argument separator", "live/dev/syntax/terragrunt.hcl", "Component syntax in stack live/dev"], "syntax error") + fixtures.check(syntax.count("live/dev/syntax/terragrunt.hcl:") > 0, "syntax error has no line and column:\n" + syntax) + for name, setting, hint in [ + ("binary", "terraform_binary", "components.terraform.command"), + ("errors", "errors", "retry"), + ("exclude", "exclude", "excluded_paths"), + ("iam", "iam_role", "identity"), + ("feature", "feature", "stack variables"), + ("lint", "tflint", "wrapper script"), + ("lifecycle", "init-from-module", "Remove the lifecycle command"), + ("escape", "generate", "relative"), + ]: + output = plan(root, name) + expect(output, ["live/dev/" + name + "/terragrunt.hcl", "Component " + name + " in stack live/dev", setting, hint], name) + forbid(output, ["plugin API operation failed", "unsupported plugin extension"], name) + + +def verify_secrets_stay_out_of_errors(root): + output = plan(root, "secret") + expect(output, ["Component secret in stack live/dev", "ATMOS_LOGS_LEVEL=Debug"], "secret-bearing evaluation error") + forbid(output, ["hunter2"], "secret-bearing evaluation error") + + +def verify_one_bad_unit_does_not_break_the_project(root): + instances = output_of(fixtures.run(root, ["list", "instances", "--format=json"])) + expect(instances, ["good", "syntax", "plain"], "list instances") + stacks = output_of(fixtures.run(root, ["list", "stacks"])) + expect(stacks, ["live/dev", "plain"], "list stacks") + fixtures.run(root, ["describe", "stacks", "--format=json"]) + # The healthy unit and the YAML component still work. + fixtures.run(root, ["terraform", "plan", "good", "-s", STACK, "--", "-no-color"]) + + +def probe(work, real): + # The project's executable is a probe that records every invocation in invocations.log beside + # itself and then runs the real tool, so a test can prove that no infrastructure command ran. + suffix = exec.run(["go", "env", "GOEXE"], output="capture").stdout.strip() + name = "tofu" if "tofu" in real.replace("\\", "/").rsplit("/", 1)[-1] else "terraform" + directory = work + "/probe" + fs.mkdir(directory, parents=True) + exec.run(["go", "build", "-o", directory + "/" + name + suffix, "./fixtures/toolprobe"], working_directory=fs.resolve("."), output="capture") + fs.write_file(directory + "/real-tool.txt", real) + return directory + "/" + name + suffix, directory + "/invocations.log" + + +def verify_bulk_runs_validate_before_running(root, invocations): + # Earlier scenarios ran the healthy unit; only what the bulk runs invoke counts here. + if fs.exists(invocations): + fs.remove(invocations) + for operation in ["plan", "apply", "destroy"]: + arguments = ["terraform", operation, "--all", "-s", STACK, "--"] + (["-auto-approve"] if operation != "plan" else []) + ["-no-color"] + output = output_of(fixtures.run_failing(root, arguments)) + expect(output, ["no infrastructure command was run", "Component syntax", "Component binary"], operation + " --all") + ran = fs.read_file(invocations) if fs.exists(invocations) else "" + fixtures.check(ran.strip() == "", operation + " --all ran the engine before rejecting the broken units:\n" + ran) + fixtures.check(not fs.exists(root + "/live/dev/good/state/terraform.tfstate"), operation + " --all ran a healthy unit before rejecting the broken ones") + + +def diagnostics(): + binary = exec.which("tofu") or exec.which("terraform") + fixtures.check(binary != None, "Install OpenTofu or Terraform for integration tests") + units = { + "good": GOOD, + "syntax": 'terraform {\n source = "../../../modules/unit"\n}\ninputs = {\n a = find_in_parent_folders("x"\n}\n', + "binary": unit("") + 'terraform_binary = "terraform"\n', + "errors": unit("") + 'errors {\n retry "x" {\n retryable_errors = [".*"]\n max_attempts = 2\n sleep_interval_sec = 1\n }\n}\n', + "exclude": unit("") + 'exclude {\n if = true\n actions = ["plan"]\n}\n', + "iam": unit("") + 'iam_role = "arn:aws:iam::123456789012:role/x"\n', + "feature": unit("") + 'feature "flag" {\n default = true\n}\n', + "lint": unit(' before_hook "lint" {\n commands = ["plan"]\n execute = ["tflint"]\n }\n'), + "lifecycle": unit(' before_hook "setup" {\n commands = ["init-from-module"]\n execute = ["echo", "x"]\n }\n'), + "escape": unit("") + 'generate "x" {\n path = "../escape.tf"\n if_exists = "overwrite"\n contents = ""\n}\n', + "secret": unit("") + 'inputs = {\n a = tonumber("hunter2")\n}\n', + } + work = fixtures.workspace("atmos-terragrunt-diagnostics-probe-") + probed, invocations = probe(work, binary) + root = project("atmos-terragrunt-diagnostics-", probed, units) + verify_rejections_name_file_unit_and_setting(root) + verify_secrets_stay_out_of_errors(root) + verify_one_bad_unit_does_not_break_the_project(root) + fixtures.check(fs.exists(invocations), "the healthy plan did not run through the probe, so its absence later proves nothing") + verify_bulk_runs_validate_before_running(root, invocations) + ui.success("Diagnostics: rich unit errors, redaction, isolation, bulk pre-flight") + + +def config_set(root, key, value, kind=None): + arguments = ["config", "set"] + (["--type=" + kind] if kind else []) + [key, value] + return fixtures.run(root, arguments) + + +def settings_repair(): + binary = exec.which("tofu") or exec.which("terraform") + fixtures.check(binary != None, "Install OpenTofu or Terraform for integration tests") + root = fixtures.workspace("atmos-terragrunt-settings-") + config = fixtures.configuration(fs.resolve("."), binary=binary) + config["components"]["terraform"]["terragrunt"] = {"enabled": True} + fixtures.write(root, "atmos.yaml", json.encode(config)) + fixtures.write(root, "modules/unit/main.tf", MODULE) + fixtures.write(root, "live/dev/good/terragrunt.hcl", GOOD) + + # Enabling discovery before included_paths must neither lock the project out nor stop `config set`. + config_set(root, "components.terraform.terragrunt.discovery.enabled", "true") + instances = output_of(fixtures.run(root, ["list", "instances"])) + expect(instances, ["good"], "discovery with the default scope") + config_set(root, "components.terraform.terragrunt.discovery.included_paths", '["live/**"]', kind="yaml") + fixtures.run(root, ["list", "instances"]) + + # A misspelled key and an unsupported mode fail loudly, name the key, and can be repaired. + config_set(root, "components.terraform.terragrunt.discoverey.enabled", "true") + output = output_of(fixtures.run_failing(root, ["list", "stacks"])) + expect(output, ["discoverey"], "misspelled key") + fixtures.run(root, ["config", "delete", "components.terraform.terragrunt.discoverey"]) + stacks = output_of(fixtures.run(root, ["list", "stacks"])) + expect(stacks, [STACK], "repaired misspelled key") + config_set(root, "components.terraform.terragrunt.mode", "nativ") + output = output_of(fixtures.run_failing(root, ["list", "stacks"])) + expect(output, ["components.terraform.terragrunt.mode"], "invalid mode") + config_set(root, "components.terraform.terragrunt.mode", "native") + stacks = output_of(fixtures.run(root, ["list", "stacks"])) + expect(stacks, [STACK], "repaired mode") + ui.success("Settings: default scope, config set repair, key-naming validation") diff --git a/plugins/terragrunt/scripts/hardening.star b/plugins/terragrunt/scripts/hardening.star new file mode 100644 index 00000000000..075dae44eb4 --- /dev/null +++ b/plugins/terragrunt/scripts/hardening.star @@ -0,0 +1,358 @@ +# Opt-in CLI acceptance tests for the hardening of native Terragrunt execution: the lifecycle of +# saved plans, local module refresh, generated files and backends, hook semantics, and pinned +# tools. Every fixture is generated here, uses only built-in terraform_data resources, and never +# touches a cloud. Importing this module has no effects; each integration step calls one scenario. +fixtures = module.load("fixtures.star") +lifecycle = module.load("lifecycle_extended.star") + +STACK = lifecycle.STACK + + +def engine(): + binary = exec.which("tofu") or exec.which("terraform") + fixtures.check(binary != None, "Install OpenTofu or Terraform for integration tests") + return binary + + +def new_project(prefix): + return lifecycle.project(fixtures.workspace(prefix), "project", fs.resolve("."), engine()) + + +def output_of(result): + return result.stdout + result.stderr + + +def expect(result, text, context): + fixtures.check(text in output_of(result), "Expected '" + text + "' in " + context + ":\n" + output_of(result)) + + +def expect_absent(result, text, context): + fixtures.check(text not in output_of(result), "Did not expect '" + text + "' in " + context + ":\n" + output_of(result)) + + +def workdir_files(root, unit, name): + return fs.glob(root + "/.workdir/terraform/" + STACK + "-" + unit + "-*/" + name) + + +SECRET_TAG = "binding-secret-tag" + + +def saved_plan_lifecycle(): + root = new_project("atmos-terragrunt-planlife-") + unit = lifecycle.PLAN_UNIT.replace("@GREETING@", "life") + 'inputs = { tag = "' + SECRET_TAG + '" }\n' + lifecycle.write_unit(root, "planned", unit) + + # Commands that only read or check the unit must leave the reviewed plan in place. + lifecycle.terraform(root, "plan", "planned") + planfile = lifecycle.planfiles(root)[0] + binding = planfile + ".atmos.json" + for verb in ["validate", "init"]: + lifecycle.terraform(root, verb, "planned") + fixtures.check(fs.exists(planfile), "'" + verb + "' deleted the saved plan") + fixtures.check(fs.exists(binding), "'" + verb + "' deleted the plan binding") + + # The binding proves what was reviewed without holding input values. + record = fs.read_file(binding) + fixtures.check(SECRET_TAG not in record, "The plan binding stores an input value:\n" + record) + + # An explicit absolute path verifies exactly like the canonical location. + applied = lifecycle.terraform(root, "apply", "planned", ["-auto-approve"], flags=["--from-plan=" + planfile]) + fixtures.check("Apply complete! Resources: 1 added" in applied.stdout, applied.stdout) + fixtures.check(not fs.exists(planfile), "A consumed plan file was kept") + fixtures.check(not fs.exists(binding), "A consumed plan binding was kept") + + # Applying the same plan again says what happened. + again = lifecycle.terraform(root, "apply", "planned", ["-auto-approve"], flags=["--from-plan"], failing=True) + expect(again, "has no saved plan", "second apply of a consumed plan") + expect_absent(again, "working directory changed", "second apply of a consumed plan") + + # A direct apply makes an earlier plan stale and removes it. + lifecycle.terraform(root, "plan", "planned") + fixtures.check(fs.exists(planfile), "The plan was not recreated") + lifecycle.terraform(root, "apply", "planned", ["-auto-approve"]) + fixtures.check(not fs.exists(planfile), "A direct apply left a stale saved plan") + stale = lifecycle.terraform(root, "apply", "planned", ["-auto-approve"], flags=["--from-plan"], failing=True) + expect(stale, "has no saved plan", "apply of a plan made stale by a direct apply") + + # A unit that was never planned has no plan to apply. + lifecycle.write_unit(root, "fresh", lifecycle.PLAN_UNIT.replace("@GREETING@", "fresh")) + never = lifecycle.terraform(root, "apply", "fresh", ["-auto-approve"], flags=["--from-plan"], failing=True) + expect(never, "has no saved plan", "apply of a never planned unit") + expect_absent(never, "binding is missing", "apply of a never planned unit") + ui.success("Saved plans: survive read-only commands, explicit paths, consumed on apply, clear errors, no input values stored") + + +def local_source_stability(): + root = new_project("atmos-terragrunt-localstable-") + fixtures.write(root, "modules/local/main.tf", lifecycle.versioned_module("local-v1")) + lifecycle.write_unit(root, "local", lifecycle.source_unit("../../../modules/local")) + lifecycle.terraform(root, "plan", "local") + + # Nothing changed: the module is neither copied again nor announced as expired. + quiet = lifecycle.terraform(root, "plan", "local") + expect_absent(quiet, "Source cache expired", "plan of an unchanged local module") + expect_absent(quiet, "Local source changed", "plan of an unchanged local module") + expect_absent(quiet, "Auto-provisioned source", "plan of an unchanged local module") + + # An edit is detected and reaches the plan. + fixtures.write(root, "modules/local/main.tf", lifecycle.versioned_module("local-v2")) + edited = lifecycle.terraform(root, "plan", "local") + expect(edited, "Local source changed", "plan after a module edit") + expect(edited, "local-v2", "plan after a module edit") + quiet = lifecycle.terraform(root, "plan", "local") + expect_absent(quiet, "Local source changed", "second plan after the edit") + ui.success("Local module: unchanged source is not re-copied, an edit is picked up once") + + +GENERATE_BLOCK = """ +generate "extra" { + path = "extra.tf" + if_exists = "overwrite_terragrunt" + contents = "variable \\"stale_marker\\" {\\n default = \\"x\\"\\n}\\n" +} +""" + +ERROR_POLICY_BLOCK = """ +generate "provider" { + path = "provider.tf" + if_exists = "error" + contents = "# generated provider file\\n" +} +""" + +PLAIN_BACKEND_UNIT = """ +terraform { + source = "../../../modules/unit" +} + +generate "backend" { + path = "backend.tf" + if_exists = "overwrite_terragrunt" + disable_signature = false + contents = <<-EOT + terraform { + backend "local" { + path = "@STATE@" + } + } + EOT +} +""" + +CACHE_STATE_UNIT = """ +terraform { + source = "../../../modules/unit" +} +""" + + +def generation_lifecycle(): + root = new_project("atmos-terragrunt-generation-") + base = lifecycle.PLAN_UNIT.replace("@GREETING@", "gen") + + # A generated file exists while its block does, and stops applying once the block is removed. + lifecycle.write_unit(root, "gen", base + GENERATE_BLOCK) + lifecycle.terraform(root, "plan", "gen") + fixtures.check(len(workdir_files(root, "gen", "extra.tf")) == 1, "The generated file was not written") + lifecycle.write_unit(root, "gen", base) + lifecycle.terraform(root, "plan", "gen") + fixtures.check(len(workdir_files(root, "gen", "extra.tf")) == 0, "A removed generate block still applies its file") + + # if_exists = "error" accepts the file this unit generated on every rerun. + lifecycle.write_unit(root, "errpolicy", base + ERROR_POLICY_BLOCK) + for _ in range(3): + lifecycle.terraform(root, "plan", "errpolicy") + + # A plain generate block that writes a backend replaces the backend Atmos would write. + state = lifecycle.hcl_path(root) + "/live/dev/plainbackend/state/terraform.tfstate" + fixtures.write(root, "live/dev/plainbackend/terragrunt.hcl", PLAIN_BACKEND_UNIT.replace("@STATE@", state)) + plain = lifecycle.terraform(root, "plan", "plainbackend") + expect_absent(plain, "Duplicate backend configuration", "plan with a generated backend block") + fixtures.check(len(workdir_files(root, "plainbackend", "backend.tf.json")) == 0, "Atmos wrote a second backend file") + + # A lock file kept beside terragrunt.hcl is copied into the working directory. + lifecycle.write_unit(root, "locked", base) + fixtures.write(root, "live/dev/locked/.terraform.lock.hcl", "# pinned beside the unit\n") + lifecycle.terraform(root, "plan", "locked") + locks = workdir_files(root, "locked", ".terraform.lock.hcl") + fixtures.check(len(locks) == 1, "The lock file beside the unit was not copied") + fixtures.check("pinned beside the unit" in fs.read_file(locks[0]), "The copied lock file lost its content") + ui.success("Generated files: removed blocks stop applying, error policy reruns, generated backend, unit lock file") + + +def cache_state_warning(): + root = new_project("atmos-terragrunt-cachestate-") + fixtures.write(root, "live/dev/cache/terragrunt.hcl", CACHE_STATE_UNIT) + fixtures.write(root, "modules/unit/main.tf", lifecycle.UNIT_MODULE) + warned = lifecycle.terraform(root, "plan", "cache") + expect(warned, "keeps its local state", "plan of a unit without remote_state") + expect(warned, ".terragrunt-cache", "plan of a unit without remote_state") + expect(warned, "remote_state", "plan of a unit without remote_state") + + # A stable explicit path is not warned about. + lifecycle.write_unit(root, "stable", lifecycle.PLAN_UNIT.replace("@GREETING@", "stable")) + quiet = lifecycle.terraform(root, "plan", "stable") + expect_absent(quiet, "keeps its local state", "plan of a unit with a stable local backend") + ui.success("State identity: a unit whose state lives in .terragrunt-cache is warned about") + + +DETAILED_UNIT = """ +terraform { + source = "../../../modules/unit" + + before_hook "state-hook" { + commands = ["state"] + execute = ["@TOOL@", "version"] + } + after_hook "after" { + commands = ["plan"] + execute = ["@TOOL@", "version"] + } + error_hook "alert" { + commands = ["plan"] + execute = ["@TOOL@", "version"] + on_errors = [".*"] + } +} +""" + +BAD_DIRECTORY_UNIT = """ +terraform { + source = "../../../modules/unit" + + before_hook "bad-directory" { + commands = ["plan"] + execute = ["@TOOL@", "version"] + working_dir = "${get_terragrunt_dir()}/nodir" + } +} +""" + + +def hook_semantics(): + binary = engine() + root = lifecycle.project(fixtures.workspace("atmos-terragrunt-hooksem-"), "project", fs.resolve("."), binary) + tool = lifecycle.tool_of(binary) + + # plan -detailed-exitcode returns 2 when changes are present. That is success for hooks. + lifecycle.write_unit(root, "dx", DETAILED_UNIT.replace("@TOOL@", tool)) + changed = lifecycle.terraform(root, "plan", "dx", ["-detailed-exitcode"], failing=True) + fixtures.check(changed.exit_code == 2, "Expected exit status 2 for a plan with changes, got " + str(changed.exit_code) + "\n" + output_of(changed)) + expect(changed, "Executing after hook: after", "plan with changes") + expect_absent(changed, "Executing error hook", "plan with changes") + # Hook lines name the unit, so bulk output can be attributed. + expect(changed, "(dx in " + STACK + ")", "plan with changes") + + # A hook that lists "state" runs for every state subcommand. + lifecycle.terraform(root, "apply", "dx", ["-auto-approve"]) + listed = fixtures.run(root, ["terraform", "state", "list", "dx", "-s", STACK]) + expect(listed, "Executing before hook: state-hook", "state list") + + # A hook working directory that does not exist is reported as such. + lifecycle.write_unit(root, "baddir", BAD_DIRECTORY_UNIT.replace("@TOOL@", tool)) + failed = lifecycle.terraform(root, "plan", "baddir", failing=True) + expect(failed, "nodir", "hook with a missing working_dir") + expect(failed, "does not exist", "hook with a missing working_dir") + ui.success("Hooks: detailed exit code is not a failure, command matching, unit in announce lines, missing directory named") + + +CONSUMER_UNIT = """ +terraform { + source = "../../../modules/unit" + + before_hook "pinned" { + commands = ["plan"] + execute = ["tofu", "version"] + } +} + +dependency "producer" { + config_path = "../producer" +} + +inputs = { + tag = dependency.producer.outputs.value + suffix = split("\\n", run_cmd("--terragrunt-quiet", "tofu", "version"))[0] +} +""" + + +def pinned_invoke(root, arguments, path): + # The ambient PATH holds no OpenTofu, so only the pinned copy can satisfy a command. + return atmos.run( + arguments, + working_directory=root, + env={"PATH": path, "NO_COLOR": "1", "ATMOS_TELEMETRY_ENABLED": "false", "TF_IN_AUTOMATION": "true"}, + output="capture", + check=False, + ) + + +ORDINARY_STACK = """ +dependencies: + tools: + opentofu: "9.9.8" +components: + terraform: + ordinary: + command: tofu +""" + + +def install_probe(tools, version, real, suffix): + directory = tools + "/bin/opentofu/opentofu/" + version + fs.mkdir(directory, parents=True) + exec.run(["go", "build", "-o", directory + "/tofu" + suffix, "./fixtures/toolprobe"], working_directory=fs.resolve("."), output="capture") + fs.write_file(directory + "/real-tool.txt", real) + return directory + + +def pinned_tools(): + real = engine() + if "tofu" not in real.replace("\\", "/").rsplit("/", 1)[-1]: + ui.warning("Pinned tool scenario needs OpenTofu; skipped") + return + work = fixtures.workspace("atmos-terragrunt-pinned-") + suffix = exec.run(["go", "env", "GOEXE"], output="capture").stdout.strip() + tools = work + "/tools" + pinned = install_probe(tools, "9.9.9", real, suffix) + stack_pinned = install_probe(tools, "9.9.8", real, suffix) + + # The project pins OpenTofu in .tool-versions, the baseline for units (which have no stack + # manifest); nothing is downloaded because the pinned copy is already installed. + config = fixtures.configuration(fs.resolve("."), binary="tofu") + config["toolchain"] = {"install": "never", "install_path": tools} + config["stacks"] = {"base_path": "stacks", "included_paths": ["**/*.yaml"]} + root = work + "/project" + fixtures.write(root, "atmos.yaml", json.encode(config)) + fixtures.write(root, ".tool-versions", "opentofu 9.9.9\n") + fixtures.write(root, "modules/unit/main.tf", lifecycle.UNIT_MODULE) + fixtures.write(root, "components/terraform/ordinary/main.tf", lifecycle.UNIT_MODULE) + fixtures.write(root, "stacks/ordinary.yaml", ORDINARY_STACK) + lifecycle.write_unit(root, "producer", lifecycle.PLAN_UNIT.replace("@GREETING@", "from-producer")) + lifecycle.write_unit(root, "consumer", CONSUMER_UNIT) + ambient = work + "/empty-path" + fs.mkdir(ambient, parents=True) + + applied = pinned_invoke(root, ["terraform", "apply", "producer", "-s", STACK, "--", "-auto-approve", "-no-color"], ambient) + fixtures.check(applied.exit_code == 0, "apply with a pinned tool failed:\n" + output_of(applied)) + planned = pinned_invoke(root, ["terraform", "plan", "consumer", "-s", STACK, "--", "-no-color"], ambient) + fixtures.check(planned.exit_code == 0, "plan with a pinned tool failed:\n" + output_of(planned)) + expect(planned, "Executing before hook: pinned", "plan of the consumer") + + # Every consumer of the tool went through the pinned copy: the Terraform commands, the + # dependency output read, run_cmd in HCL, and the hook. + log = fs.read_file(pinned + "/invocations.log").splitlines() + commands = [line.split(" ")[0] for line in log] + for command in ["init", "apply", "plan", "output"]: + fixtures.check(command in commands, "The pinned tool never ran '" + command + "': " + str(log)) + versions = len([command for command in commands if command == "version"]) + fixtures.check(versions >= 2, "run_cmd and the hook should both run the pinned tool: " + str(log)) + + # An ordinary component takes its pin from dependencies.tools in the stack manifest, which wins + # over the project's .tool-versions. A top-level dependencies key in atmos.yaml is not read. + ordinary = pinned_invoke(root, ["terraform", "plan", "ordinary", "-s", "ordinary", "--", "-no-color"], ambient) + fixtures.check(ordinary.exit_code == 0, "plan of an ordinary component with a stack pin failed:\n" + output_of(ordinary)) + stack_log = fs.read_file(stack_pinned + "/invocations.log").splitlines() + fixtures.check("plan" in [line.split(" ")[0] for line in stack_log], "The stack manifest pin was not used: " + str(stack_log)) + ui.success("Pinned tools: terraform commands, dependency output reads, run_cmd, and hooks use the pinned copy without an ambient PATH entry") diff --git a/plugins/terragrunt/scripts/imports.star b/plugins/terragrunt/scripts/imports.star new file mode 100644 index 00000000000..eac399041ad --- /dev/null +++ b/plugins/terragrunt/scripts/imports.star @@ -0,0 +1,456 @@ +# Opt-in CLI acceptance tests for terragrunt:// imports, discovery patterns, and symlinks. +# Every fixture is generated here, uses only built-in terraform_data resources, and never +# touches a cloud. Importing this module has no effects. +fixtures = module.load("fixtures.star") +hardening = module.load("hardening.star") +lifecycle = module.load("lifecycle_extended.star") + +MODULE = """ +variable "name" { + type = string + default = "unset" +} +resource "terraform_data" "this" { + input = var.name +} +output "name" { + value = var.name +} +""" + + +def unit(name, dependency=None): + text = 'terraform { source = "../../../modules/m" }\ninputs = { name = "' + name + '" }\n' + if dependency: + text += 'dependency "dep" {\n config_path = "../' + dependency + '"\n mock_outputs = { name = "mock" }\n}\n' + return text + + +def project(work, name, package, binary, discovery=True, scope="live/**", excluded=None): + root = work + "/" + name + fixtures.project(root, package) + config = fixtures.configuration(package, scope=scope, binary=binary) + settings = config["components"]["terraform"]["terragrunt"]["discovery"] + settings["enabled"] = discovery + if excluded: + settings["excluded_paths"] = excluded + config["stacks"] = {"base_path": "stacks", "included_paths": ["*.yaml"], "excluded_paths": ["mixins/**"]} + fixtures.write(root, "atmos.yaml", json.encode(config)) + fixtures.write(root, "modules/m/main.tf", MODULE) + for path, text in [("dev/vpc", unit("dev/vpc")), ("dev/app", unit("dev/app", "vpc")), ("prod/vpc", unit("prod/vpc")), ("prod/db", unit("prod/db"))]: + fixtures.write(root, "live/" + path + "/terragrunt.hcl", text) + return root + + +def instances(root, *flags): + rows = json.decode(fixtures.run(root, ["list", "instances", "--format=json"] + list(flags)).stdout) + return sorted([row["Stack"] + "/" + row["Component"] for row in rows]) + + +def stack_file(root, name, imports, extra=""): + text = "import:\n" + "".join([" - " + item + "\n" for item in imports]) + extra + fixtures.write(root, "stacks/" + name + ".yaml", text) + + +def test_import_joins_the_importing_stack(work, package, binary): + root = project(work, "joins", package, binary, discovery=False) + # A mixin imported by one stack, a direct directory import, and a single-file import. + stack_file(root, "mixins/dev-units", ["terragrunt://live/dev"]) + stack_file(root, "dev", ["mixins/dev-units"]) + stack_file(root, "staging", ["mixins/dev-units", "terragrunt://live/prod/db/terragrunt.hcl"]) + expected = ["dev/app", "dev/vpc", "staging/app", "staging/db", "staging/vpc"] + fixtures.check(instances(root) == expected, "Unexpected instances: " + str(instances(root))) + fixtures.check(instances(root, "--stack", "staging") == ["staging/app", "staging/db", "staging/vpc"], "Unexpected staging instances") + + # No stack is derived from the unit's directory: the importing stacks are the only ones. + stacks = fixtures.run(root, ["list", "stacks"]).stdout.split() + fixtures.check(sorted(stacks) == ["dev", "staging"], "Imports created stacks: " + str(stacks)) + described = json.decode(fixtures.run(root, ["describe", "stacks", "--format=json"]).stdout) + fixtures.check(sorted(described.keys()) == ["dev", "staging"], "describe stacks created stacks: " + str(described.keys())) + app = described["staging"]["components"]["terraform"]["app"] + fixtures.check(app["dependencies"]["components"][0]["name"] == "vpc", "A unit depends on its sibling in the importing stack") + + # The unit runs in whichever stack imported it. + for stack in ["dev", "staging"]: + planned = fixtures.run(root, ["terraform", "plan", "vpc", "-s", stack, "--", "-no-color"]) + fixtures.check("terraform_data.this" in planned.stdout, "plan vpc -s " + stack + ":\n" + planned.stdout) + component = json.decode(fixtures.run(root, ["describe", "component", "vpc", "-s", "dev", "--format=json"]).stdout) + fixtures.check(component["settings"]["configuration"]["source"] == "live/dev/vpc/terragrunt.hcl", "Unexpected unit: " + str(component["settings"])) + return root + + +def test_import_with_discovery(work, package, binary): + root = project(work, "mixed", package, binary, excluded=["live/dev/app"]) + stack_file(root, "dev", ["terragrunt://live/dev"]) + # live/dev is imported (even the unit excluded from discovery); live/prod is discovered into + # its own derived stacks, and nothing is both. + expected = ["dev/app", "dev/vpc", "live/prod/db", "live/prod/vpc"] + fixtures.check(instances(root) == expected, "Unexpected instances: " + str(instances(root))) + + +def test_conflicts(work, package, binary): + root = project(work, "conflicts", package, binary, discovery=False) + # Two units with one name in one stack. + stack_file(root, "both", ["terragrunt://live/dev/vpc", "terragrunt://live/prod/vpc"]) + result = fixtures.run_failing(root, ["list", "instances"]) + output = result.stdout + result.stderr + fixtures.check("live/dev/vpc/terragrunt.hcl" in output and "live/prod/vpc/terragrunt.hcl" in output, "Both units must be named:\n" + output) + fs.remove(root + "/stacks/both.yaml") + + # A YAML component with the name of an imported unit is an override, not a conflict. + stack_file(root, "clash", ["terragrunt://live/dev/vpc"], "components:\n terraform:\n vpc:\n vars:\n a: 1\n") + fixtures.check(instances(root, "--stack", "clash") == ["clash/vpc"], "A YAML component named like a unit must merge: " + str(instances(root))) + fs.remove(root + "/stacks/clash.yaml") + + # An import that selects no unit is an error, not an empty result. + fixtures.write(root, "catalog/terragrunt.hcl", "locals { shared = true }\n") + stack_file(root, "empty", ["terragrunt://catalog"]) + result = fixtures.run_failing(root, ["list", "instances"]) + fixtures.check("matched no components" in result.stdout + result.stderr, "Empty import must be explained:\n" + result.stdout + result.stderr) + + +def test_patterns_select_alike(work, package, binary): + expected = ["live/dev/app", "live/dev/vpc"] + for scope in ["live/dev", "live/dev/", "./live/dev", "live/dev/*", "live/dev/**"]: + root = project(work, "include-" + scope.replace("/", "_").replace("*", "S").replace(".", "D"), package, binary, scope=scope) + fixtures.check(instances(root) == expected, scope + " selected " + str(instances(root))) + for excluded in ["live/prod", "live/prod/", "./live/prod", "live/prod/*", "live/prod/**"]: + root = project(work, "exclude-" + excluded.replace("/", "_").replace("*", "S").replace(".", "D"), package, binary, scope="./**", excluded=[excluded]) + fixtures.check(instances(root) == expected, excluded + " left " + str(instances(root))) + root = project(work, "warns", package, binary, scope="live/nothing") + result = fixtures.run(root, ["list", "instances"]) + fixtures.check("live/nothing" in result.stderr and "matched no" in result.stderr, "An unmatched pattern must be reported:\n" + result.stderr) + + +def symlink(root, path, target): + # Starlark cannot create links, so git materializes them (mode 120000) in the disposable repo. + fixtures.write(root, "scratch/link-target", target) + blob = fixtures.git(root, "hash-object", "-w", "scratch/link-target") + fs.remove(root + "/scratch", recursive=True) + fixtures.git(root, "update-index", "--add", "--cacheinfo", "120000," + blob + "," + path) + fixtures.git(root, "checkout-index", "-f", path) + return fs.stat(root + "/" + path).is_symlink + + +def test_symlinks_follow_upstream(work, package, binary): + root = project(work, "links", package, binary) + if not symlink(root, "live/linked", "prod") or not symlink(root, "live/dev/copy/terragrunt.hcl", "../vpc/terragrunt.hcl"): + ui.info("Symbolic links are unavailable on this host; skipping the symlink scenario") + return + found = instances(root) + fixtures.check("live/dev/copy" in found and "live/dev/vpc" in found, "A symlinked terragrunt.hcl is a unit at its own location: " + str(found)) + fixtures.check(not any(["linked" in item for item in found]), "A symlinked directory is not followed: " + str(found)) + result = fixtures.run(root, ["list", "instances"]) + fixtures.check("live/linked" in result.stderr and "not followed" in result.stderr, "The skipped link must be reported:\n" + result.stderr) + fixtures.check(instances(root) == found, "Discovery must be deterministic") + + +OVERRIDE_MODULE = """ +variable "name" { + type = string + default = "unset" +} +variable "tags" { + type = map(string) + default = {} +} +variable "extra" { + type = string + default = "module-default" +} +variable "zones" { + type = list(string) + default = [] +} +resource "terraform_data" "this" { + input = var.name +} +output "zones" { + value = var.zones +} +output "name" { + value = var.name +} +output "tags" { + value = var.tags +} +output "extra" { + value = var.extra +} +output "workspace" { + value = terraform.workspace +} +""" + +OVERRIDE_VPC = """ +terraform { source = "../../../modules/o" } +inputs = { + name = "from-hcl" + extra = "from-hcl" + tags = { team = "hcl", env = "dev" } + zones = ["a", "b"] +} +""" + +# The unit sets TF_VAR_extra through extra_arguments and gives no input for it. +OVERRIDE_ENVU = """ +terraform { + source = "../../../modules/o" + extra_arguments "e" { + commands = ["plan", "apply"] + env_vars = { TF_VAR_extra = "unit-env" } + } +} +inputs = { name = "envu" } +""" + +UNIT_IMPORTS = ["terragrunt://live/over/vpc", "terragrunt://live/over/envu"] + + +def override_project(work, package, binary): + root = work + "/override" + fixtures.project(root, package) + config = fixtures.configuration(package, binary=binary) + config["components"]["terraform"]["terragrunt"]["discovery"]["enabled"] = False + config["stacks"] = {"base_path": "stacks", "included_paths": ["*.yaml"], "excluded_paths": ["mixins/**"]} + fixtures.write(root, "atmos.yaml", json.encode(config)) + fixtures.write(root, "modules/o/main.tf", OVERRIDE_MODULE) + fixtures.write(root, "live/over/vpc/terragrunt.hcl", OVERRIDE_VPC) + fixtures.write(root, "live/over/envu/terragrunt.hcl", OVERRIDE_ENVU) + return root + + +def apply_outputs(root, unit, stack): + result = fixtures.run(root, ["terraform", "apply", unit, "-s", stack, "--", "-auto-approve", "-no-color"]) + outputs = json.decode(fixtures.run(root, ["terraform", "output", unit, "-s", stack, "--", "-json"]).stdout) + return {"values": {key: outputs[key]["value"] for key in outputs}, "result": result} + + +def check_rejected(root, stack, unit, expected): + result = fixtures.run_failing(root, ["terraform", "plan", unit, "-s", stack, "--", "-no-color"]) + output = result.stdout + result.stderr + for text in expected: + fixtures.check(text in output, "Expected '" + text + "' in:\n" + output) + + +def test_stack_overrides(work, package, binary): + root = override_project(work, package, binary) + fixtures.write(root, "stacks/plain.yaml", "import:\n - " + "\n - ".join(UNIT_IMPORTS) + "\n") + fixtures.write(root, "stacks/override.yaml", "import:\n - " + "\n - ".join(UNIT_IMPORTS) + """ +components: + terraform: + vpc: + metadata: + terraform_workspace: blue + vars: + name: from-stack + zones: [c] + tags: + team: stack + settings: + owner: platform + envu: + env: + TF_VAR_extra: stack-env + providers: + terraform: {} + hooks: + announce: + kind: step + type: log + events: [after.terraform.apply] + with: + level: warn + content: stack-hook-ran +""") + + # Without a YAML component the unit's own inputs and extra_arguments environment apply. + plain = apply_outputs(root, "vpc", "plain")["values"] + fixtures.check(plain["name"] == "from-hcl" and plain["extra"] == "from-hcl" and plain["tags"] == {"team": "hcl", "env": "dev"}, "Unexpected unit values: " + str(plain)) + fixtures.check(plain["workspace"] == "default", "Unexpected workspace: " + str(plain)) + fixtures.check(apply_outputs(root, "envu", "plain")["values"]["extra"] == "unit-env", "The unit's environment was not applied") + + # The YAML component deep-merges over the unit: a scalar replaces, a map merges key by key, + # untouched inputs stay, and metadata, settings, env, hooks, and providers all take effect. + over = apply_outputs(root, "vpc", "override")["values"] + fixtures.check(over["name"] == "from-stack", "A stack var did not override the unit input: " + str(over)) + fixtures.check(over["tags"] == {"team": "stack", "env": "dev"}, "Tags must merge deeply: " + str(over)) + fixtures.check(over["extra"] == "from-hcl", "An untouched unit input was lost: " + str(over)) + fixtures.check(over["zones"] == ["c"], "A list is replaced under the default list merge strategy: " + str(over)) + fixtures.check(over["workspace"] == "blue", "metadata.terraform_workspace was ignored: " + str(over)) + again = apply_outputs(root, "vpc", "plain")["values"] + fixtures.check(again["name"] == "from-hcl" and again["workspace"] == "default", "Another stack imports the same unit without the override: " + str(again)) + varfiles = fs.glob(root + "/.workdir/terraform/override-vpc-*/atmos-*.terraform.tfvars.json") + fixtures.check(len(varfiles) == 1, "Expected one generated var file: " + str(varfiles)) + fixtures.check(json.decode(fs.read_file(varfiles[0])) == {"name": "from-stack", "extra": "from-hcl", "zones": ["c"], "tags": {"team": "stack", "env": "dev"}}, "The generated var file is not the merged result: " + fs.read_file(varfiles[0])) + envu = apply_outputs(root, "envu", "override") + fixtures.check(envu["values"]["extra"] == "stack-env", "Stack env must win over the unit's environment: " + str(envu["values"])) + fixtures.check("stack-hook-ran" in envu["result"].stderr + envu["result"].stdout, "The stack's Atmos hook did not run for the unit:\n" + envu["result"].stderr) + fixtures.check(len(fs.glob(root + "/.workdir/terraform/override-envu-*/providers_override.tf.json")) == 1, "The stack's providers section was not generated") + component = json.decode(fixtures.run(root, ["describe", "component", "vpc", "-s", "override", "--format=json"]).stdout) + fixtures.check(component["vars"] == {"name": "from-stack", "zones": ["c"], "tags": {"team": "stack"}}, "describe shows the stack layer: " + str(component["vars"])) + fixtures.check(component["settings"]["owner"] == "platform" and component["settings"]["configuration"]["source"] == "live/over/vpc/terragrunt.hcl", "Settings must merge with the unit marker: " + str(component["settings"])) + + # Abstract components feed a unit-named component like they feed any other; !unset removes a + # value of the stack layers, so the unit's own input shows again. + fixtures.write(root, "stacks/inherits.yaml", "import:\n - terragrunt://live/over/vpc\n" + """ +components: + terraform: + base: + metadata: + type: abstract + vars: + name: from-base + extra: from-base + tags: + base: b + team: base + vpc: + metadata: + inherits: [base] + terraform_workspace: inherited + vars: + extra: !unset + tags: + added: c +""") + inherited = apply_outputs(root, "vpc", "inherits")["values"] + fixtures.check(inherited["name"] == "from-base", "An abstract base must override the unit input: " + str(inherited)) + fixtures.check(inherited["extra"] == "from-hcl", "!unset removes the base value, the unit input remains: " + str(inherited)) + fixtures.check(inherited["tags"] == {"base": "b", "team": "base", "added": "c", "env": "dev"}, "Unexpected inherited tags: " + str(inherited)) + + # A stack map merges over the unit input map, and null replaces a unit input with null. + fixtures.write(root, "stacks/replaced.yaml", "import:\n - terragrunt://live/over/vpc\n" + """ +components: + terraform: + base: + metadata: + type: abstract + vars: + tags: + base: b + team: base + vpc: + metadata: + inherits: [base] + terraform_workspace: replaced + vars: + extra: null + tags: + only: stack +""") + replaced = apply_outputs(root, "vpc", "replaced")["values"] + fixtures.check(replaced["tags"] == {"only": "stack", "base": "b", "team": "base", "env": "dev"}, "Stack layers merge over the unit input: " + str(replaced)) + fixtures.check("extra" not in replaced, "null must replace the unit input: " + str(replaced)) + + # A YAML component in an import that comes before the unit's import still overrides the unit. + fixtures.write(root, "stacks/mixins/catalog.yaml", "components:\n terraform:\n vpc:\n metadata:\n terraform_workspace: early\n vars:\n name: catalog-name\n") + fixtures.write(root, "stacks/early.yaml", "import:\n - mixins/catalog\n - terragrunt://live/over/vpc\n") + early = apply_outputs(root, "vpc", "early")["values"] + fixtures.check(early["name"] == "catalog-name" and early["workspace"] == "early", "An earlier import must override the unit: " + str(early)) + + # Locking a unit from the stack blocks changes, like any component. + fixtures.write(root, "stacks/locked.yaml", "import:\n - terragrunt://live/over/vpc\ncomponents:\n terraform:\n vpc:\n metadata:\n locked: true\n") + locked = fixtures.run_failing(root, ["terraform", "apply", "vpc", "-s", "locked", "--", "-auto-approve", "-no-color"]) + fixtures.check("locked" in locked.stderr + locked.stdout, "A locked unit must refuse apply:\n" + locked.stderr) + + # Settings the unit owns cannot be overridden, and the error says why. + fixtures.write(root, "stacks/backend.yaml", "import:\n - terragrunt://live/over/vpc\ncomponents:\n terraform:\n vpc:\n backend_type: s3\n backend:\n s3:\n bucket: other\n") + check_rejected(root, "backend", "vpc", ["backend_type", "remote_state", "owned"]) + fixtures.write(root, "stacks/module.yaml", "import:\n - terragrunt://live/over/vpc\ncomponents:\n terraform:\n vpc:\n metadata:\n component: other\n") + check_rejected(root, "module", "vpc", ["metadata.component", "other", "owned"]) + fixtures.write(root, "stacks/source.yaml", "import:\n - terragrunt://live/over/vpc\ncomponents:\n terraform:\n vpc:\n source:\n uri: ./modules/other\n") + check_rejected(root, "source", "vpc", ["source.uri", "modules/other", "terragrunt.hcl"]) + # A global backend in the stack is not a component override: the unit keeps its own state. + fixtures.write(root, "stacks/global-backend.yaml", "import:\n - terragrunt://live/over/vpc\nterraform:\n backend_type: s3\n backend:\n s3:\n bucket: global\ncomponents:\n terraform:\n vpc:\n metadata:\n terraform_workspace: shielded\n") + shielded = apply_outputs(root, "vpc", "global-backend")["values"] + fixtures.check(shielded["workspace"] == "shielded", "A global backend must not capture a unit: " + str(shielded)) + ui.success("Stack overrides: vars, env, settings, metadata, hooks, providers, inherits, !unset, null, lists, earlier imports; owned settings rejected") + + +def stack_pin(version): + return "dependencies:\n tools:\n opentofu: \"" + version + "\"\n" + + +def test_stack_pinned_tools(work, package, binary): + if "tofu" not in binary.replace("\\", "/").rsplit("/", 1)[-1]: + ui.warning("Stack pinned tool scenario needs OpenTofu; skipped") + return + suffix = exec.run(["go", "env", "GOEXE"], output="capture").stdout.strip() + tools = work + "/pin-tools" + versions = ["9.9.9", "9.9.8", "9.9.7", "9.9.6"] + probes = {version: hardening.install_probe(tools, version, binary, suffix) for version in versions} + config = fixtures.configuration(package, binary="tofu") + config["toolchain"] = {"install": "never", "install_path": tools} + config["stacks"] = {"base_path": "stacks", "included_paths": ["*.yaml"]} + root = work + "/pin" + fixtures.write(root, "atmos.yaml", json.encode(config)) + fixtures.write(root, ".tool-versions", "opentofu 9.9.9\n") + fixtures.write(root, "modules/unit/main.tf", lifecycle.UNIT_MODULE) + lifecycle.write_unit(root, "producer", lifecycle.PLAN_UNIT.replace("@GREETING@", "from-producer")) + lifecycle.write_unit(root, "consumer", hardening.CONSUMER_UNIT) + fixtures.write(root, "live/solo/lone/terragrunt.hcl", lifecycle.PLAN_UNIT.replace("@GREETING@", "lone") + lifecycle.BACKEND) + imports = "import:\n - terragrunt://live/dev\n" + terraform = "terraform:\n dependencies:\n tools:\n opentofu: \"9.9.7\"\n" + component = "components:\n terraform:\n" + "".join([" " + unit + ":\n dependencies:\n tools:\n opentofu: \"9.9.6\"\n" for unit in ["producer", "consumer"]]) + fixtures.write(root, "stacks/pin-global.yaml", imports + stack_pin("9.9.8")) + fixtures.write(root, "stacks/pin-terraform.yaml", imports + stack_pin("9.9.8") + terraform) + fixtures.write(root, "stacks/pin-component.yaml", imports + stack_pin("9.9.8") + terraform + component) + ambient = work + "/pin-empty-path" + fs.mkdir(ambient, parents=True) + + def used(version): + log = probes[version] + "/invocations.log" + return fs.read_file(log).splitlines() if fs.exists(log) else [] + + # The pin from the stack manifest reaches an imported unit at global, terraform, and component + # level, nearest wins, and every consumer of the tool uses it with no tool on the ambient PATH. + for stack, version in [("pin-global", "9.9.8"), ("pin-terraform", "9.9.7"), ("pin-component", "9.9.6")]: + for probe in versions: + if fs.exists(probes[probe] + "/invocations.log"): + fs.remove(probes[probe] + "/invocations.log") + applied = hardening.pinned_invoke(root, ["terraform", "apply", "producer", "-s", stack, "--", "-auto-approve", "-no-color"], ambient) + fixtures.check(applied.exit_code == 0, stack + ": apply failed:\n" + hardening.output_of(applied)) + planned = hardening.pinned_invoke(root, ["terraform", "plan", "consumer", "-s", stack, "--", "-no-color"], ambient) + fixtures.check(planned.exit_code == 0, stack + ": plan failed:\n" + hardening.output_of(planned)) + commands = [line.split(" ")[0] for line in used(version)] + for command in ["init", "apply", "plan", "output"]: + fixtures.check(command in commands, stack + ": " + version + " never ran '" + command + "': " + str(commands)) + fixtures.check(len([command for command in commands if command == "version"]) >= 2, stack + ": run_cmd and the hook should use " + version + ": " + str(commands)) + for other in versions: + if other != version: + fixtures.check(used(other) == [], stack + ": " + other + " must not run: " + str(used(other))) + + # A unit that is only discovered has no stack manifest and takes .tool-versions. + for probe in versions: + if fs.exists(probes[probe] + "/invocations.log"): + fs.remove(probes[probe] + "/invocations.log") + lone = hardening.pinned_invoke(root, ["terraform", "plan", "lone", "-s", "live/solo", "--", "-no-color"], ambient) + fixtures.check(lone.exit_code == 0, "plan of a discovered unit failed:\n" + hardening.output_of(lone)) + fixtures.check("plan" in [line.split(" ")[0] for line in used("9.9.9")], "A discovered unit must use .tool-versions: " + str(used("9.9.9"))) + fixtures.check(used("9.9.8") == [] and used("9.9.7") == [] and used("9.9.6") == [], "A discovered unit must not use stack pins") + ui.success("Stack pinned tools: global, terraform, and component pins reach imported units; discovered units use .tool-versions") + + +def imports_and_discovery(): + package = fs.resolve(".") + binary = exec.which("tofu") or exec.which("terraform") + fixtures.check(binary != None, "Install OpenTofu or Terraform for integration tests") + work = fixtures.workspace("atmos-terragrunt-imports-") + test_import_joins_the_importing_stack(work, package, binary) + test_import_with_discovery(work, package, binary) + test_conflicts(work, package, binary) + test_patterns_select_alike(work, package, binary) + test_symlinks_follow_upstream(work, package, binary) + + +def stack_overrides(): + binary = exec.which("tofu") or exec.which("terraform") + fixtures.check(binary != None, "Install OpenTofu or Terraform for integration tests") + test_stack_overrides(fixtures.workspace("atmos-terragrunt-overrides-"), fs.resolve("."), binary) + + +def stack_pinned_tools(): + binary = exec.which("tofu") or exec.which("terraform") + fixtures.check(binary != None, "Install OpenTofu or Terraform for integration tests") + test_stack_pinned_tools(fixtures.workspace("atmos-terragrunt-stack-pins-"), fs.resolve("."), binary) diff --git a/plugins/terragrunt/scripts/lifecycle_extended.star b/plugins/terragrunt/scripts/lifecycle_extended.star index e73e278df84..6db1a284c57 100644 --- a/plugins/terragrunt/scripts/lifecycle_extended.star +++ b/plugins/terragrunt/scripts/lifecycle_extended.star @@ -491,6 +491,25 @@ def git_source_refresh(): fixtures.check(applied_version(root, "remote") == "module-v2", "A changed Git ref kept the previously provisioned module") ui.success("Local Git module source: fetched at v1, refreshed at v2") + # A branch moves, so its fetched copy is reused until refreshed. The run says so and + # names the command that refreshes it; a pinned tag stays silent. + write_unit(root, "remote", source_unit(source + "main")) + terraform(root, "apply", "remote", ["-auto-approve"]) + repeated = terraform(root, "apply", "remote", ["-auto-approve"]) + fixtures.check("floating ref" in repeated.stderr and "source pull remote" in repeated.stderr, "A reused branch source did not explain how to refresh it:\n" + repeated.stderr) + repository = work + "/module-repository" + fixtures.write(repository, "module/main.tf", versioned_module("module-v3")) + fixtures.commit(repository, "module v3") + terraform(root, "apply", "remote", ["-auto-approve"]) + fixtures.check(applied_version(root, "remote") == "module-v2", "A branch source refreshed without being asked") + fixtures.run(root, ["terraform", "source", "pull", "remote", "-s", STACK, "--force"]) + terraform(root, "apply", "remote", ["-auto-approve"]) + fixtures.check(applied_version(root, "remote") == "module-v3", "Pulling a branch source with --force did not fetch the new commit") + write_unit(root, "remote", source_unit(source + "v2")) + pinned = terraform(root, "apply", "remote", ["-auto-approve"]) + fixtures.check("floating ref" not in pinned.stderr, "A pinned tag was reported as floating:\n" + pinned.stderr) + ui.success("Branch module source: reuse is explained and --force refreshes it") + def local_source_refresh(): package = fs.resolve(".") diff --git a/proto/atmos/plugin/v1/plugin.proto b/proto/atmos/plugin/v1/plugin.proto index 103fd955044..0b261ef8e5a 100644 --- a/proto/atmos/plugin/v1/plugin.proto +++ b/proto/atmos/plugin/v1/plugin.proto @@ -74,7 +74,63 @@ message AuthenticateEvent { Credentials result = 3; } } -message Diagnostic { string field = 1; string message = 2; } +// SourceLocation points at a position in a user-authored file. Paths are +// relative to the project root when the file lives below it; line and column +// are 1-based and zero means unknown. +message SourceLocation { + string file = 1; + uint32 line = 2; + uint32 column = 3; +} + +// ErrorDetail is the structured, user-facing description of a plugin failure. +// A plugin returns it as a gRPC status detail (google.rpc.Status.details) on a +// failed call, or inside Diagnostic.detail for a per-item failure that must not +// fail the whole call. The host renders it with the Atmos error builder. +// +// Secret safety contract: code, summary, hints, location, unit, stack, +// config_key, and operation are always shown and MUST NOT contain evaluated +// values. detail is shown only when user_safe is true. When user_safe is false +// the host omits detail from the error, names the unit and operation, hints at +// the debug log level, and records the masked detail in the debug log only. +// The host also applies secret masking to every field before display. +message ErrorDetail { + // Stable machine-readable identifier such as "unsupported_setting" or "syntax_error". + string code = 1; + // One-line headline naming what failed, for example "Unsupported Terragrunt setting". + string summary = 2; + // Longer explanation. Shown only when user_safe is true. + string detail = 3; + // Actionable remedies, shown as hints in order. + repeated string hints = 4; + // Where the problem is authored. + SourceLocation location = 5; + // Component (unit) name, in Atmos terms. + string unit = 6; + // Stack the unit belongs to. + string stack = 7; + // Configuration key, setting, or block that caused the failure, for example "terraform_binary". + string config_key = 8; + // True when detail contains no evaluated values and is safe to print. + bool user_safe = 9; + // The operation that failed, such as "discover", "prepare", or "validate". + string operation = 10; + // Severity of the problem. Unset means ERROR. + Severity severity = 11; +} + +// Severity classifies an ErrorDetail. A WARNING never fails a command: the host +// prints it on the UI channel and continues. An ERROR fails the commands that +// target the named unit, or the whole command when no unit is named. +enum Severity { + SEVERITY_UNSPECIFIED = 0; + SEVERITY_ERROR = 1; + SEVERITY_WARNING = 2; +} + +// Diagnostic describes one problem. field and message keep their original meaning; +// detail adds the structured form and takes precedence when present. +message Diagnostic { string field = 1; string message = 2; ErrorDetail detail = 3; } message ValidateResponse { repeated Diagnostic diagnostics = 1; } message Unsupported { string operation = 1; string reason = 2; } message RefreshRequest { ProviderRequest provider = 1; Credentials current = 2; } @@ -213,7 +269,11 @@ message DiscoverComponentsRequest { ComponentValue provider_configuration = 10; } message DiscoverComponentsResponse { + // Healthy components. One failing component must not hide the others. repeated DiscoveredComponent components = 1; + // Per-component failures. A diagnostic whose detail names a unit and stack is + // reported only to commands that target that component; others keep working. + // A diagnostic without a unit is a project-level failure. repeated Diagnostic diagnostics = 2; } service ComponentDiscovery { @@ -256,6 +316,9 @@ message ComponentGeneratedFile { // for sensitive files). The host keeps only owner/group/other read and // execute bits plus owner write, and ignores the field on Windows. uint32 mode = 5; + // True only for a literal file copied from beside the configuration unit, + // never for an explicit generate block. Absent means normal policy handling. + bool unit_file_copy = 6; } // Evaluated command metadata; the host executes it with the invocation's // environment, toolchain, cancellation, and I/O. Lists retain declaration order. @@ -303,6 +366,62 @@ message PrepareComponentResponse { // emit its default backend file. bool backend_generated = 10; ComponentExecutionConfiguration execution = 11; + // Maps each input name to the authored file, line, and column that produced + // its value, for provenance reporting. Missing entries mean unknown origin. + map input_origins = 12; + + // Fields 13-18 are reserved for Terragrunt parity. The host does not read + // them yet; a plugin that cannot honor a setting must reject it with an + // ErrorDetail instead of returning it, and must leave these fields unset. + // Engine executable selected by terraform_binary. + string terraform_binary = 13; + // errors block: retry and ignore policies. + ComponentErrorsConfiguration errors = 14; + // exclude block: conditional unit exclusion. + ComponentExclude exclude = 15; + // iam_role, iam_assume_role_session_name, iam_assume_role_duration, iam_web_identity_token. + ComponentIam iam = 16; + // encryption block as opaque, provider-defined data. + ComponentValue encryption = 17; + // feature flag name to its evaluated value rendered as a string. + map feature_flags = 18; +} + +// ComponentErrorsConfiguration mirrors the Terragrunt errors block. Reserved; not implemented. +message ComponentErrorsConfiguration { + repeated ComponentRetryPolicy retry = 1; + repeated ComponentIgnorePolicy ignore = 2; +} +message ComponentRetryPolicy { + string name = 1; + // Regular expressions matched against command errors. + repeated string retryable_errors = 2; + uint32 max_attempts = 3; + uint32 sleep_interval_seconds = 4; +} +message ComponentIgnorePolicy { + string name = 1; + // Regular expressions matched against command errors. + repeated string ignorable_errors = 2; + string message = 3; + // Extra signals written for the caller, keyed by name. + map signals = 4; +} +// ComponentExclude mirrors the Terragrunt exclude block. Reserved; not implemented. +message ComponentExclude { + // Evaluated "if" condition. + bool condition = 1; + // Commands the exclusion applies to; "all" and "all_except_output" are plugin vocabulary. + repeated string actions = 2; + bool exclude_dependencies = 3; +} +// ComponentIam mirrors the Terragrunt IAM settings. Reserved; not implemented. +// The web identity token is a secret and must be carried as a sensitive value. +message ComponentIam { + string role = 1; + string session_name = 2; + uint32 duration_seconds = 3; + ComponentValue web_identity_token = 4; } // Reevaluate immediately before the selected operation, after its dependencies. // Preparing configuration does not authorize executing infrastructure commands. diff --git a/sdk/component/failure.go b/sdk/component/failure.go new file mode 100644 index 00000000000..26734d2db4d --- /dev/null +++ b/sdk/component/failure.go @@ -0,0 +1,114 @@ +package component + +import ( + "errors" + + "google.golang.org/grpc/codes" + "google.golang.org/grpc/status" + "google.golang.org/protobuf/proto" + + pluginv1 "github.com/cloudposse/atmos/sdk/proto/atmos/plugin/v1" +) + +// Failure builds the structured, user-facing description of a plugin problem. +// +// The host renders a Failure with its error builder: a title, an explanation, hints, the +// authored file and position, and the unit and stack. Everything except the detail text is +// always shown, so it must never contain evaluated values. Detail text is shown only after +// SafeDetail; Detail marks it as possibly secret and the host then keeps it in the masked +// debug log. +// +//nolint:lintroller // Pure SDK helper; the standalone SDK cannot import host perf tracking. +type Failure struct { + detail *pluginv1.ErrorDetail +} + +// NewFailure starts a failure with a stable machine-readable code and a one-line headline. +func NewFailure(code, summary string) *Failure { + return &Failure{detail: &pluginv1.ErrorDetail{Code: code, Summary: summary, Severity: pluginv1.Severity_SEVERITY_ERROR}} +} + +// Detail sets explanatory text that may contain evaluated values. The host does not print it. +func (f *Failure) Detail(text string) *Failure { + f.detail.Detail, f.detail.UserSafe = text, false + return f +} + +// SafeDetail sets explanatory text that contains no evaluated values, so the host prints it. +func (f *Failure) SafeDetail(text string) *Failure { + f.detail.Detail, f.detail.UserSafe = text, true + return f +} + +// Hint appends an actionable remedy. Hints are shown in order. +func (f *Failure) Hint(text string) *Failure { + f.detail.Hints = append(f.detail.Hints, text) + return f +} + +// At records the authored position; line and column are 1-based and zero means unknown. +func (f *Failure) At(file string, line, column uint32) *Failure { + f.detail.Location = &pluginv1.SourceLocation{File: file, Line: line, Column: column} + return f +} + +// Unit names the component and the stack it belongs to, in Atmos terms. +func (f *Failure) Unit(stack, name string) *Failure { + f.detail.Stack, f.detail.Unit = stack, name + return f +} + +// Key names the setting or block that caused the failure. +func (f *Failure) Key(key string) *Failure { + f.detail.ConfigKey = key + return f +} + +// Operation names the failing step, such as "discover" or "prepare". +func (f *Failure) Operation(operation string) *Failure { + f.detail.Operation = operation + return f +} + +// Warning downgrades the failure so the host prints it without failing the command. +func (f *Failure) Warning() *Failure { + f.detail.Severity = pluginv1.Severity_SEVERITY_WARNING + return f +} + +// Proto returns an independent copy of the wire form. +func (f *Failure) Proto() *pluginv1.ErrorDetail { + return proto.Clone(f.detail).(*pluginv1.ErrorDetail) +} + +// Diagnostic returns the failure as a per-item diagnostic that does not fail the call. +func (f *Failure) Diagnostic() *pluginv1.Diagnostic { + return &pluginv1.Diagnostic{Field: f.detail.ConfigKey, Message: f.detail.Summary, Detail: f.Proto()} +} + +// Err returns a gRPC status error carrying the failure as a status detail. +func (f *Failure) Err(code codes.Code) error { + withDetail, err := status.New(code, f.detail.Summary).WithDetails(f.Proto()) + if err != nil { + // A detail that cannot be attached must still fail the call with its headline. + return status.Error(code, f.detail.Summary) + } + return withDetail.Err() +} + +// FailureDetail extracts the structured failure from an error, looking through wrapping. +func FailureDetail(err error) (*pluginv1.ErrorDetail, bool) { + if err == nil { + return nil, false + } + var carrier interface{ GRPCStatus() *status.Status } + if !errors.As(err, &carrier) { + return nil, false + } + for _, item := range carrier.GRPCStatus().Details() { + if detail, ok := item.(*pluginv1.ErrorDetail); ok { + return detail, true + } + } + return nil, false +} diff --git a/sdk/component/failure_test.go b/sdk/component/failure_test.go new file mode 100644 index 00000000000..96e402ebf54 --- /dev/null +++ b/sdk/component/failure_test.go @@ -0,0 +1,83 @@ +package component_test + +import ( + "fmt" + "reflect" + "testing" + + "google.golang.org/grpc/codes" + "google.golang.org/grpc/status" + + "github.com/cloudposse/atmos/sdk/component" + pluginv1 "github.com/cloudposse/atmos/sdk/proto/atmos/plugin/v1" +) + +func TestFailureRoundTripsThroughStatus(t *testing.T) { + t.Parallel() + err := component.NewFailure("unsupported_setting", "Unsupported Terragrunt setting"). + Operation("prepare").Unit("live/dev", "vpc").Key("terraform_binary").At("live/dev/vpc/terragrunt.hcl", 7, 3). + Hint("Remove terraform_binary.").Hint("Set components.terraform.command instead."). + SafeDetail("terraform_binary is not supported.").Err(codes.Unimplemented) + if status.Code(err) != codes.Unimplemented || status.Convert(err).Message() != "Unsupported Terragrunt setting" { + t.Fatalf("unexpected status: %v", err) + } + detail, ok := component.FailureDetail(err) + if !ok { + t.Fatal("detail missing") + } + checks := map[string][2]any{ + "code": {detail.Code, "unsupported_setting"}, "unit": {detail.Unit, "vpc"}, "stack": {detail.Stack, "live/dev"}, + "key": {detail.ConfigKey, "terraform_binary"}, "operation": {detail.Operation, "prepare"}, "safe": {detail.UserSafe, true}, + "line": {detail.Location.Line, uint32(7)}, "column": {detail.Location.Column, uint32(3)}, "file": {detail.Location.File, "live/dev/vpc/terragrunt.hcl"}, + "hints": {detail.Hints, []string{"Remove terraform_binary.", "Set components.terraform.command instead."}}, + } + for name, pair := range checks { + if !reflect.DeepEqual(pair[0], pair[1]) { + t.Errorf("%s: got %v want %v", name, pair[0], pair[1]) + } + } +} + +func TestFailureDefaultsToUnsafeError(t *testing.T) { + t.Parallel() + detail := component.NewFailure("c", "s").Detail("value was hunter2").Diagnostic().Detail + if detail.UserSafe || detail.Severity != pluginv1.Severity_SEVERITY_ERROR { + t.Fatalf("unexpected defaults: %v", detail) + } +} + +func TestFailureWarningDiagnostic(t *testing.T) { + t.Parallel() + diagnostic := component.NewFailure("skipped", "Directory skipped").Warning().Diagnostic() + if diagnostic.Detail.Severity != pluginv1.Severity_SEVERITY_WARNING || diagnostic.Message != "Directory skipped" { + t.Fatalf("unexpected diagnostic: %v", diagnostic) + } +} + +func TestFailureDetailMissing(t *testing.T) { + t.Parallel() + for name, err := range map[string]error{"plain": fmt.Errorf("plain"), "nodetails": status.Error(codes.Internal, "x"), "nil": nil} { + if _, ok := component.FailureDetail(err); ok { + t.Errorf("%s: unexpected detail", name) + } + } +} + +func TestFailureDetailThroughWrapping(t *testing.T) { + t.Parallel() + err := component.NewFailure("c", "s").Err(codes.InvalidArgument) + if _, ok := component.FailureDetail(fmt.Errorf("wrapped: %w", err)); !ok { + t.Fatal("detail lost through wrapping") + } +} + +func TestFailureProtoIsIndependent(t *testing.T) { + t.Parallel() + failure := component.NewFailure("c", "s").Hint("a").At("f", 1, 2) + first := failure.Proto() + first.Hints[0], first.Location.File = "mutated", "mutated" + second := failure.Proto() + if second.Hints[0] != "a" || second.Location.File != "f" { + t.Fatalf("copy aliases the builder: %v", second) + } +} diff --git a/sdk/proto/atmos/plugin/v1/plugin.pb.go b/sdk/proto/atmos/plugin/v1/plugin.pb.go index 26aa21b2b58..57f75fe9c17 100644 --- a/sdk/proto/atmos/plugin/v1/plugin.pb.go +++ b/sdk/proto/atmos/plugin/v1/plugin.pb.go @@ -25,6 +25,58 @@ const ( _ = protoimpl.EnforceVersion(protoimpl.MaxVersion - 20) ) +// Severity classifies an ErrorDetail. A WARNING never fails a command: the host +// prints it on the UI channel and continues. An ERROR fails the commands that +// target the named unit, or the whole command when no unit is named. +type Severity int32 + +const ( + Severity_SEVERITY_UNSPECIFIED Severity = 0 + Severity_SEVERITY_ERROR Severity = 1 + Severity_SEVERITY_WARNING Severity = 2 +) + +// Enum value maps for Severity. +var ( + Severity_name = map[int32]string{ + 0: "SEVERITY_UNSPECIFIED", + 1: "SEVERITY_ERROR", + 2: "SEVERITY_WARNING", + } + Severity_value = map[string]int32{ + "SEVERITY_UNSPECIFIED": 0, + "SEVERITY_ERROR": 1, + "SEVERITY_WARNING": 2, + } +) + +func (x Severity) Enum() *Severity { + p := new(Severity) + *p = x + return p +} + +func (x Severity) String() string { + return protoimpl.X.EnumStringOf(x.Descriptor(), protoreflect.EnumNumber(x)) +} + +func (Severity) Descriptor() protoreflect.EnumDescriptor { + return file_atmos_plugin_v1_plugin_proto_enumTypes[0].Descriptor() +} + +func (Severity) Type() protoreflect.EnumType { + return &file_atmos_plugin_v1_plugin_proto_enumTypes[0] +} + +func (x Severity) Number() protoreflect.EnumNumber { + return protoreflect.EnumNumber(x) +} + +// Deprecated: Use Severity.Descriptor instead. +func (Severity) EnumDescriptor() ([]byte, []int) { + return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{0} +} + type ComponentSource_Kind int32 const ( @@ -61,11 +113,11 @@ func (x ComponentSource_Kind) String() string { } func (ComponentSource_Kind) Descriptor() protoreflect.EnumDescriptor { - return file_atmos_plugin_v1_plugin_proto_enumTypes[0].Descriptor() + return file_atmos_plugin_v1_plugin_proto_enumTypes[1].Descriptor() } func (ComponentSource_Kind) Type() protoreflect.EnumType { - return &file_atmos_plugin_v1_plugin_proto_enumTypes[0] + return &file_atmos_plugin_v1_plugin_proto_enumTypes[1] } func (x ComponentSource_Kind) Number() protoreflect.EnumNumber { @@ -74,7 +126,7 @@ func (x ComponentSource_Kind) Number() protoreflect.EnumNumber { // Deprecated: Use ComponentSource_Kind.Descriptor instead. func (ComponentSource_Kind) EnumDescriptor() ([]byte, []int) { - return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{30, 0} + return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{32, 0} } type ComponentDependency_Kind int32 @@ -110,11 +162,11 @@ func (x ComponentDependency_Kind) String() string { } func (ComponentDependency_Kind) Descriptor() protoreflect.EnumDescriptor { - return file_atmos_plugin_v1_plugin_proto_enumTypes[1].Descriptor() + return file_atmos_plugin_v1_plugin_proto_enumTypes[2].Descriptor() } func (ComponentDependency_Kind) Type() protoreflect.EnumType { - return &file_atmos_plugin_v1_plugin_proto_enumTypes[1] + return &file_atmos_plugin_v1_plugin_proto_enumTypes[2] } func (x ComponentDependency_Kind) Number() protoreflect.EnumNumber { @@ -123,7 +175,7 @@ func (x ComponentDependency_Kind) Number() protoreflect.EnumNumber { // Deprecated: Use ComponentDependency_Kind.Descriptor instead. func (ComponentDependency_Kind) EnumDescriptor() ([]byte, []int) { - return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{31, 0} + return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{33, 0} } // Invocation identifies the live call authorized to use the Atmos API broker. @@ -981,17 +1033,229 @@ func (*AuthenticateEvent_Interaction) isAuthenticateEvent_Event() {} func (*AuthenticateEvent_Result) isAuthenticateEvent_Event() {} +// SourceLocation points at a position in a user-authored file. Paths are +// relative to the project root when the file lives below it; line and column +// are 1-based and zero means unknown. +type SourceLocation struct { + state protoimpl.MessageState `protogen:"open.v1"` + File string `protobuf:"bytes,1,opt,name=file,proto3" json:"file,omitempty"` + Line uint32 `protobuf:"varint,2,opt,name=line,proto3" json:"line,omitempty"` + Column uint32 `protobuf:"varint,3,opt,name=column,proto3" json:"column,omitempty"` + unknownFields protoimpl.UnknownFields + sizeCache protoimpl.SizeCache +} + +func (x *SourceLocation) Reset() { + *x = SourceLocation{} + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[12] + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + ms.StoreMessageInfo(mi) +} + +func (x *SourceLocation) String() string { + return protoimpl.X.MessageStringOf(x) +} + +func (*SourceLocation) ProtoMessage() {} + +func (x *SourceLocation) ProtoReflect() protoreflect.Message { + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[12] + if x != nil { + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + if ms.LoadMessageInfo() == nil { + ms.StoreMessageInfo(mi) + } + return ms + } + return mi.MessageOf(x) +} + +// Deprecated: Use SourceLocation.ProtoReflect.Descriptor instead. +func (*SourceLocation) Descriptor() ([]byte, []int) { + return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{12} +} + +func (x *SourceLocation) GetFile() string { + if x != nil { + return x.File + } + return "" +} + +func (x *SourceLocation) GetLine() uint32 { + if x != nil { + return x.Line + } + return 0 +} + +func (x *SourceLocation) GetColumn() uint32 { + if x != nil { + return x.Column + } + return 0 +} + +// ErrorDetail is the structured, user-facing description of a plugin failure. +// A plugin returns it as a gRPC status detail (google.rpc.Status.details) on a +// failed call, or inside Diagnostic.detail for a per-item failure that must not +// fail the whole call. The host renders it with the Atmos error builder. +// +// Secret safety contract: code, summary, hints, location, unit, stack, +// config_key, and operation are always shown and MUST NOT contain evaluated +// values. detail is shown only when user_safe is true. When user_safe is false +// the host omits detail from the error, names the unit and operation, hints at +// the debug log level, and records the masked detail in the debug log only. +// The host also applies secret masking to every field before display. +type ErrorDetail struct { + state protoimpl.MessageState `protogen:"open.v1"` + // Stable machine-readable identifier such as "unsupported_setting" or "syntax_error". + Code string `protobuf:"bytes,1,opt,name=code,proto3" json:"code,omitempty"` + // One-line headline naming what failed, for example "Unsupported Terragrunt setting". + Summary string `protobuf:"bytes,2,opt,name=summary,proto3" json:"summary,omitempty"` + // Longer explanation. Shown only when user_safe is true. + Detail string `protobuf:"bytes,3,opt,name=detail,proto3" json:"detail,omitempty"` + // Actionable remedies, shown as hints in order. + Hints []string `protobuf:"bytes,4,rep,name=hints,proto3" json:"hints,omitempty"` + // Where the problem is authored. + Location *SourceLocation `protobuf:"bytes,5,opt,name=location,proto3" json:"location,omitempty"` + // Component (unit) name, in Atmos terms. + Unit string `protobuf:"bytes,6,opt,name=unit,proto3" json:"unit,omitempty"` + // Stack the unit belongs to. + Stack string `protobuf:"bytes,7,opt,name=stack,proto3" json:"stack,omitempty"` + // Configuration key, setting, or block that caused the failure, for example "terraform_binary". + ConfigKey string `protobuf:"bytes,8,opt,name=config_key,json=configKey,proto3" json:"config_key,omitempty"` + // True when detail contains no evaluated values and is safe to print. + UserSafe bool `protobuf:"varint,9,opt,name=user_safe,json=userSafe,proto3" json:"user_safe,omitempty"` + // The operation that failed, such as "discover", "prepare", or "validate". + Operation string `protobuf:"bytes,10,opt,name=operation,proto3" json:"operation,omitempty"` + // Severity of the problem. Unset means ERROR. + Severity Severity `protobuf:"varint,11,opt,name=severity,proto3,enum=atmos.plugin.v1.Severity" json:"severity,omitempty"` + unknownFields protoimpl.UnknownFields + sizeCache protoimpl.SizeCache +} + +func (x *ErrorDetail) Reset() { + *x = ErrorDetail{} + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[13] + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + ms.StoreMessageInfo(mi) +} + +func (x *ErrorDetail) String() string { + return protoimpl.X.MessageStringOf(x) +} + +func (*ErrorDetail) ProtoMessage() {} + +func (x *ErrorDetail) ProtoReflect() protoreflect.Message { + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[13] + if x != nil { + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + if ms.LoadMessageInfo() == nil { + ms.StoreMessageInfo(mi) + } + return ms + } + return mi.MessageOf(x) +} + +// Deprecated: Use ErrorDetail.ProtoReflect.Descriptor instead. +func (*ErrorDetail) Descriptor() ([]byte, []int) { + return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{13} +} + +func (x *ErrorDetail) GetCode() string { + if x != nil { + return x.Code + } + return "" +} + +func (x *ErrorDetail) GetSummary() string { + if x != nil { + return x.Summary + } + return "" +} + +func (x *ErrorDetail) GetDetail() string { + if x != nil { + return x.Detail + } + return "" +} + +func (x *ErrorDetail) GetHints() []string { + if x != nil { + return x.Hints + } + return nil +} + +func (x *ErrorDetail) GetLocation() *SourceLocation { + if x != nil { + return x.Location + } + return nil +} + +func (x *ErrorDetail) GetUnit() string { + if x != nil { + return x.Unit + } + return "" +} + +func (x *ErrorDetail) GetStack() string { + if x != nil { + return x.Stack + } + return "" +} + +func (x *ErrorDetail) GetConfigKey() string { + if x != nil { + return x.ConfigKey + } + return "" +} + +func (x *ErrorDetail) GetUserSafe() bool { + if x != nil { + return x.UserSafe + } + return false +} + +func (x *ErrorDetail) GetOperation() string { + if x != nil { + return x.Operation + } + return "" +} + +func (x *ErrorDetail) GetSeverity() Severity { + if x != nil { + return x.Severity + } + return Severity_SEVERITY_UNSPECIFIED +} + +// Diagnostic describes one problem. field and message keep their original meaning; +// detail adds the structured form and takes precedence when present. type Diagnostic struct { state protoimpl.MessageState `protogen:"open.v1"` Field string `protobuf:"bytes,1,opt,name=field,proto3" json:"field,omitempty"` Message string `protobuf:"bytes,2,opt,name=message,proto3" json:"message,omitempty"` + Detail *ErrorDetail `protobuf:"bytes,3,opt,name=detail,proto3" json:"detail,omitempty"` unknownFields protoimpl.UnknownFields sizeCache protoimpl.SizeCache } func (x *Diagnostic) Reset() { *x = Diagnostic{} - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[12] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[14] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -1003,7 +1267,7 @@ func (x *Diagnostic) String() string { func (*Diagnostic) ProtoMessage() {} func (x *Diagnostic) ProtoReflect() protoreflect.Message { - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[12] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[14] if x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -1016,7 +1280,7 @@ func (x *Diagnostic) ProtoReflect() protoreflect.Message { // Deprecated: Use Diagnostic.ProtoReflect.Descriptor instead. func (*Diagnostic) Descriptor() ([]byte, []int) { - return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{12} + return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{14} } func (x *Diagnostic) GetField() string { @@ -1033,6 +1297,13 @@ func (x *Diagnostic) GetMessage() string { return "" } +func (x *Diagnostic) GetDetail() *ErrorDetail { + if x != nil { + return x.Detail + } + return nil +} + type ValidateResponse struct { state protoimpl.MessageState `protogen:"open.v1"` Diagnostics []*Diagnostic `protobuf:"bytes,1,rep,name=diagnostics,proto3" json:"diagnostics,omitempty"` @@ -1042,7 +1313,7 @@ type ValidateResponse struct { func (x *ValidateResponse) Reset() { *x = ValidateResponse{} - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[13] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[15] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -1054,7 +1325,7 @@ func (x *ValidateResponse) String() string { func (*ValidateResponse) ProtoMessage() {} func (x *ValidateResponse) ProtoReflect() protoreflect.Message { - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[13] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[15] if x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -1067,7 +1338,7 @@ func (x *ValidateResponse) ProtoReflect() protoreflect.Message { // Deprecated: Use ValidateResponse.ProtoReflect.Descriptor instead. func (*ValidateResponse) Descriptor() ([]byte, []int) { - return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{13} + return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{15} } func (x *ValidateResponse) GetDiagnostics() []*Diagnostic { @@ -1087,7 +1358,7 @@ type Unsupported struct { func (x *Unsupported) Reset() { *x = Unsupported{} - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[14] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[16] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -1099,7 +1370,7 @@ func (x *Unsupported) String() string { func (*Unsupported) ProtoMessage() {} func (x *Unsupported) ProtoReflect() protoreflect.Message { - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[14] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[16] if x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -1112,7 +1383,7 @@ func (x *Unsupported) ProtoReflect() protoreflect.Message { // Deprecated: Use Unsupported.ProtoReflect.Descriptor instead. func (*Unsupported) Descriptor() ([]byte, []int) { - return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{14} + return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{16} } func (x *Unsupported) GetOperation() string { @@ -1139,7 +1410,7 @@ type RefreshRequest struct { func (x *RefreshRequest) Reset() { *x = RefreshRequest{} - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[15] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[17] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -1151,7 +1422,7 @@ func (x *RefreshRequest) String() string { func (*RefreshRequest) ProtoMessage() {} func (x *RefreshRequest) ProtoReflect() protoreflect.Message { - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[15] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[17] if x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -1164,7 +1435,7 @@ func (x *RefreshRequest) ProtoReflect() protoreflect.Message { // Deprecated: Use RefreshRequest.ProtoReflect.Descriptor instead. func (*RefreshRequest) Descriptor() ([]byte, []int) { - return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{15} + return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{17} } func (x *RefreshRequest) GetProvider() *ProviderRequest { @@ -1194,7 +1465,7 @@ type RefreshResponse struct { func (x *RefreshResponse) Reset() { *x = RefreshResponse{} - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[16] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[18] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -1206,7 +1477,7 @@ func (x *RefreshResponse) String() string { func (*RefreshResponse) ProtoMessage() {} func (x *RefreshResponse) ProtoReflect() protoreflect.Message { - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[16] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[18] if x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -1219,7 +1490,7 @@ func (x *RefreshResponse) ProtoReflect() protoreflect.Message { // Deprecated: Use RefreshResponse.ProtoReflect.Descriptor instead. func (*RefreshResponse) Descriptor() ([]byte, []int) { - return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{16} + return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{18} } func (x *RefreshResponse) GetResult() isRefreshResponse_Result { @@ -1272,7 +1543,7 @@ type EnvironmentResponse struct { func (x *EnvironmentResponse) Reset() { *x = EnvironmentResponse{} - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[17] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[19] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -1284,7 +1555,7 @@ func (x *EnvironmentResponse) String() string { func (*EnvironmentResponse) ProtoMessage() {} func (x *EnvironmentResponse) ProtoReflect() protoreflect.Message { - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[17] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[19] if x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -1297,7 +1568,7 @@ func (x *EnvironmentResponse) ProtoReflect() protoreflect.Message { // Deprecated: Use EnvironmentResponse.ProtoReflect.Descriptor instead. func (*EnvironmentResponse) Descriptor() ([]byte, []int) { - return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{17} + return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{19} } func (x *EnvironmentResponse) GetEnvironment() map[string]string { @@ -1317,7 +1588,7 @@ type PrepareEnvironmentRequest struct { func (x *PrepareEnvironmentRequest) Reset() { *x = PrepareEnvironmentRequest{} - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[18] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[20] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -1329,7 +1600,7 @@ func (x *PrepareEnvironmentRequest) String() string { func (*PrepareEnvironmentRequest) ProtoMessage() {} func (x *PrepareEnvironmentRequest) ProtoReflect() protoreflect.Message { - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[18] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[20] if x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -1342,7 +1613,7 @@ func (x *PrepareEnvironmentRequest) ProtoReflect() protoreflect.Message { // Deprecated: Use PrepareEnvironmentRequest.ProtoReflect.Descriptor instead. func (*PrepareEnvironmentRequest) Descriptor() ([]byte, []int) { - return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{18} + return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{20} } func (x *PrepareEnvironmentRequest) GetProvider() *ProviderRequest { @@ -1371,7 +1642,7 @@ type CredentialPath struct { func (x *CredentialPath) Reset() { *x = CredentialPath{} - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[19] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[21] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -1383,7 +1654,7 @@ func (x *CredentialPath) String() string { func (*CredentialPath) ProtoMessage() {} func (x *CredentialPath) ProtoReflect() protoreflect.Message { - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[19] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[21] if x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -1396,7 +1667,7 @@ func (x *CredentialPath) ProtoReflect() protoreflect.Message { // Deprecated: Use CredentialPath.ProtoReflect.Descriptor instead. func (*CredentialPath) Descriptor() ([]byte, []int) { - return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{19} + return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{21} } func (x *CredentialPath) GetLocation() string { @@ -1436,7 +1707,7 @@ type PathsResponse struct { func (x *PathsResponse) Reset() { *x = PathsResponse{} - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[20] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[22] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -1448,7 +1719,7 @@ func (x *PathsResponse) String() string { func (*PathsResponse) ProtoMessage() {} func (x *PathsResponse) ProtoReflect() protoreflect.Message { - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[20] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[22] if x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -1461,7 +1732,7 @@ func (x *PathsResponse) ProtoReflect() protoreflect.Message { // Deprecated: Use PathsResponse.ProtoReflect.Descriptor instead. func (*PathsResponse) Descriptor() ([]byte, []int) { - return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{20} + return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{22} } func (x *PathsResponse) GetPaths() []*CredentialPath { @@ -1481,7 +1752,7 @@ type IdentityAuthenticateRequest struct { func (x *IdentityAuthenticateRequest) Reset() { *x = IdentityAuthenticateRequest{} - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[21] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[23] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -1493,7 +1764,7 @@ func (x *IdentityAuthenticateRequest) String() string { func (*IdentityAuthenticateRequest) ProtoMessage() {} func (x *IdentityAuthenticateRequest) ProtoReflect() protoreflect.Message { - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[21] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[23] if x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -1506,7 +1777,7 @@ func (x *IdentityAuthenticateRequest) ProtoReflect() protoreflect.Message { // Deprecated: Use IdentityAuthenticateRequest.ProtoReflect.Descriptor instead. func (*IdentityAuthenticateRequest) Descriptor() ([]byte, []int) { - return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{21} + return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{23} } func (x *IdentityAuthenticateRequest) GetIdentity() *IdentityRequest { @@ -1533,7 +1804,7 @@ type IdentityPrepareRequest struct { func (x *IdentityPrepareRequest) Reset() { *x = IdentityPrepareRequest{} - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[22] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[24] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -1545,7 +1816,7 @@ func (x *IdentityPrepareRequest) String() string { func (*IdentityPrepareRequest) ProtoMessage() {} func (x *IdentityPrepareRequest) ProtoReflect() protoreflect.Message { - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[22] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[24] if x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -1558,7 +1829,7 @@ func (x *IdentityPrepareRequest) ProtoReflect() protoreflect.Message { // Deprecated: Use IdentityPrepareRequest.ProtoReflect.Descriptor instead. func (*IdentityPrepareRequest) Descriptor() ([]byte, []int) { - return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{22} + return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{24} } func (x *IdentityPrepareRequest) GetIdentity() *IdentityRequest { @@ -1588,7 +1859,7 @@ type CommandRequest struct { func (x *CommandRequest) Reset() { *x = CommandRequest{} - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[23] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[25] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -1600,7 +1871,7 @@ func (x *CommandRequest) String() string { func (*CommandRequest) ProtoMessage() {} func (x *CommandRequest) ProtoReflect() protoreflect.Message { - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[23] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[25] if x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -1613,7 +1884,7 @@ func (x *CommandRequest) ProtoReflect() protoreflect.Message { // Deprecated: Use CommandRequest.ProtoReflect.Descriptor instead. func (*CommandRequest) Descriptor() ([]byte, []int) { - return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{23} + return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{25} } func (x *CommandRequest) GetInvocation() *Invocation { @@ -1653,7 +1924,7 @@ type CommandExit struct { func (x *CommandExit) Reset() { *x = CommandExit{} - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[24] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[26] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -1665,7 +1936,7 @@ func (x *CommandExit) String() string { func (*CommandExit) ProtoMessage() {} func (x *CommandExit) ProtoReflect() protoreflect.Message { - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[24] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[26] if x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -1678,7 +1949,7 @@ func (x *CommandExit) ProtoReflect() protoreflect.Message { // Deprecated: Use CommandExit.ProtoReflect.Descriptor instead. func (*CommandExit) Descriptor() ([]byte, []int) { - return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{24} + return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{26} } func (x *CommandExit) GetCode() uint32 { @@ -1702,7 +1973,7 @@ type CommandEvent struct { func (x *CommandEvent) Reset() { *x = CommandEvent{} - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[25] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[27] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -1714,7 +1985,7 @@ func (x *CommandEvent) String() string { func (*CommandEvent) ProtoMessage() {} func (x *CommandEvent) ProtoReflect() protoreflect.Message { - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[25] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[27] if x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -1727,7 +1998,7 @@ func (x *CommandEvent) ProtoReflect() protoreflect.Message { // Deprecated: Use CommandEvent.ProtoReflect.Descriptor instead. func (*CommandEvent) Descriptor() ([]byte, []int) { - return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{25} + return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{27} } func (x *CommandEvent) GetEvent() isCommandEvent_Event { @@ -1801,7 +2072,7 @@ type ComponentValue struct { func (x *ComponentValue) Reset() { *x = ComponentValue{} - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[26] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[28] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -1813,7 +2084,7 @@ func (x *ComponentValue) String() string { func (*ComponentValue) ProtoMessage() {} func (x *ComponentValue) ProtoReflect() protoreflect.Message { - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[26] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[28] if x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -1826,7 +2097,7 @@ func (x *ComponentValue) ProtoReflect() protoreflect.Message { // Deprecated: Use ComponentValue.ProtoReflect.Descriptor instead. func (*ComponentValue) Descriptor() ([]byte, []int) { - return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{26} + return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{28} } func (x *ComponentValue) GetTypeJson() []byte { @@ -1859,7 +2130,7 @@ type ComponentValuePath struct { func (x *ComponentValuePath) Reset() { *x = ComponentValuePath{} - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[27] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[29] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -1871,7 +2142,7 @@ func (x *ComponentValuePath) String() string { func (*ComponentValuePath) ProtoMessage() {} func (x *ComponentValuePath) ProtoReflect() protoreflect.Message { - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[27] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[29] if x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -1884,7 +2155,7 @@ func (x *ComponentValuePath) ProtoReflect() protoreflect.Message { // Deprecated: Use ComponentValuePath.ProtoReflect.Descriptor instead. func (*ComponentValuePath) Descriptor() ([]byte, []int) { - return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{27} + return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{29} } func (x *ComponentValuePath) GetSteps() []*ComponentValueStep { @@ -1907,7 +2178,7 @@ type ComponentValueStep struct { func (x *ComponentValueStep) Reset() { *x = ComponentValueStep{} - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[28] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[30] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -1919,7 +2190,7 @@ func (x *ComponentValueStep) String() string { func (*ComponentValueStep) ProtoMessage() {} func (x *ComponentValueStep) ProtoReflect() protoreflect.Message { - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[28] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[30] if x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -1932,7 +2203,7 @@ func (x *ComponentValueStep) ProtoReflect() protoreflect.Message { // Deprecated: Use ComponentValueStep.ProtoReflect.Descriptor instead. func (*ComponentValueStep) Descriptor() ([]byte, []int) { - return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{28} + return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{30} } func (x *ComponentValueStep) GetSelector() isComponentValueStep_Selector { @@ -1989,7 +2260,7 @@ type ComponentAddress struct { func (x *ComponentAddress) Reset() { *x = ComponentAddress{} - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[29] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[31] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -2001,7 +2272,7 @@ func (x *ComponentAddress) String() string { func (*ComponentAddress) ProtoMessage() {} func (x *ComponentAddress) ProtoReflect() protoreflect.Message { - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[29] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[31] if x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -2014,7 +2285,7 @@ func (x *ComponentAddress) ProtoReflect() protoreflect.Message { // Deprecated: Use ComponentAddress.ProtoReflect.Descriptor instead. func (*ComponentAddress) Descriptor() ([]byte, []int) { - return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{29} + return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{31} } func (x *ComponentAddress) GetStack() string { @@ -2051,7 +2322,7 @@ type ComponentSource struct { func (x *ComponentSource) Reset() { *x = ComponentSource{} - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[30] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[32] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -2063,7 +2334,7 @@ func (x *ComponentSource) String() string { func (*ComponentSource) ProtoMessage() {} func (x *ComponentSource) ProtoReflect() protoreflect.Message { - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[30] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[32] if x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -2076,7 +2347,7 @@ func (x *ComponentSource) ProtoReflect() protoreflect.Message { // Deprecated: Use ComponentSource.ProtoReflect.Descriptor instead. func (*ComponentSource) Descriptor() ([]byte, []int) { - return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{30} + return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{32} } func (x *ComponentSource) GetPath() string { @@ -2110,7 +2381,7 @@ type ComponentDependency struct { func (x *ComponentDependency) Reset() { *x = ComponentDependency{} - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[31] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[33] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -2122,7 +2393,7 @@ func (x *ComponentDependency) String() string { func (*ComponentDependency) ProtoMessage() {} func (x *ComponentDependency) ProtoReflect() protoreflect.Message { - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[31] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[33] if x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -2135,7 +2406,7 @@ func (x *ComponentDependency) ProtoReflect() protoreflect.Message { // Deprecated: Use ComponentDependency.ProtoReflect.Descriptor instead. func (*ComponentDependency) Descriptor() ([]byte, []int) { - return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{31} + return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{33} } func (x *ComponentDependency) GetTarget() *ComponentAddress { @@ -2167,7 +2438,7 @@ type DiscoveredComponent struct { func (x *DiscoveredComponent) Reset() { *x = DiscoveredComponent{} - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[32] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[34] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -2179,7 +2450,7 @@ func (x *DiscoveredComponent) String() string { func (*DiscoveredComponent) ProtoMessage() {} func (x *DiscoveredComponent) ProtoReflect() protoreflect.Message { - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[32] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[34] if x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -2192,7 +2463,7 @@ func (x *DiscoveredComponent) ProtoReflect() protoreflect.Message { // Deprecated: Use DiscoveredComponent.ProtoReflect.Descriptor instead. func (*DiscoveredComponent) Descriptor() ([]byte, []int) { - return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{32} + return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{34} } func (x *DiscoveredComponent) GetAddress() *ComponentAddress { @@ -2259,7 +2530,7 @@ type ComponentEffects struct { func (x *ComponentEffects) Reset() { *x = ComponentEffects{} - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[33] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[35] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -2271,7 +2542,7 @@ func (x *ComponentEffects) String() string { func (*ComponentEffects) ProtoMessage() {} func (x *ComponentEffects) ProtoReflect() protoreflect.Message { - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[33] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[35] if x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -2284,7 +2555,7 @@ func (x *ComponentEffects) ProtoReflect() protoreflect.Message { // Deprecated: Use ComponentEffects.ProtoReflect.Descriptor instead. func (*ComponentEffects) Descriptor() ([]byte, []int) { - return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{33} + return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{35} } func (x *ComponentEffects) GetCommands() bool { @@ -2339,7 +2610,7 @@ type ComponentOperation struct { func (x *ComponentOperation) Reset() { *x = ComponentOperation{} - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[34] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[36] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -2351,7 +2622,7 @@ func (x *ComponentOperation) String() string { func (*ComponentOperation) ProtoMessage() {} func (x *ComponentOperation) ProtoReflect() protoreflect.Message { - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[34] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[36] if x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -2364,7 +2635,7 @@ func (x *ComponentOperation) ProtoReflect() protoreflect.Message { // Deprecated: Use ComponentOperation.ProtoReflect.Descriptor instead. func (*ComponentOperation) Descriptor() ([]byte, []int) { - return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{34} + return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{36} } func (x *ComponentOperation) GetCommand() string { @@ -2415,7 +2686,7 @@ type DiscoverComponentsRequest struct { func (x *DiscoverComponentsRequest) Reset() { *x = DiscoverComponentsRequest{} - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[35] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[37] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -2427,7 +2698,7 @@ func (x *DiscoverComponentsRequest) String() string { func (*DiscoverComponentsRequest) ProtoMessage() {} func (x *DiscoverComponentsRequest) ProtoReflect() protoreflect.Message { - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[35] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[37] if x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -2440,7 +2711,7 @@ func (x *DiscoverComponentsRequest) ProtoReflect() protoreflect.Message { // Deprecated: Use DiscoverComponentsRequest.ProtoReflect.Descriptor instead. func (*DiscoverComponentsRequest) Descriptor() ([]byte, []int) { - return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{35} + return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{37} } func (x *DiscoverComponentsRequest) GetInvocation() *Invocation { @@ -2514,16 +2785,20 @@ func (x *DiscoverComponentsRequest) GetProviderConfiguration() *ComponentValue { } type DiscoverComponentsResponse struct { - state protoimpl.MessageState `protogen:"open.v1"` - Components []*DiscoveredComponent `protobuf:"bytes,1,rep,name=components,proto3" json:"components,omitempty"` - Diagnostics []*Diagnostic `protobuf:"bytes,2,rep,name=diagnostics,proto3" json:"diagnostics,omitempty"` + state protoimpl.MessageState `protogen:"open.v1"` + // Healthy components. One failing component must not hide the others. + Components []*DiscoveredComponent `protobuf:"bytes,1,rep,name=components,proto3" json:"components,omitempty"` + // Per-component failures. A diagnostic whose detail names a unit and stack is + // reported only to commands that target that component; others keep working. + // A diagnostic without a unit is a project-level failure. + Diagnostics []*Diagnostic `protobuf:"bytes,2,rep,name=diagnostics,proto3" json:"diagnostics,omitempty"` unknownFields protoimpl.UnknownFields sizeCache protoimpl.SizeCache } func (x *DiscoverComponentsResponse) Reset() { *x = DiscoverComponentsResponse{} - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[36] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[38] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -2535,7 +2810,7 @@ func (x *DiscoverComponentsResponse) String() string { func (*DiscoverComponentsResponse) ProtoMessage() {} func (x *DiscoverComponentsResponse) ProtoReflect() protoreflect.Message { - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[36] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[38] if x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -2548,7 +2823,7 @@ func (x *DiscoverComponentsResponse) ProtoReflect() protoreflect.Message { // Deprecated: Use DiscoverComponentsResponse.ProtoReflect.Descriptor instead. func (*DiscoverComponentsResponse) Descriptor() ([]byte, []int) { - return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{36} + return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{38} } func (x *DiscoverComponentsResponse) GetComponents() []*DiscoveredComponent { @@ -2582,7 +2857,7 @@ type ComponentRequest struct { func (x *ComponentRequest) Reset() { *x = ComponentRequest{} - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[37] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[39] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -2594,7 +2869,7 @@ func (x *ComponentRequest) String() string { func (*ComponentRequest) ProtoMessage() {} func (x *ComponentRequest) ProtoReflect() protoreflect.Message { - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[37] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[39] if x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -2607,7 +2882,7 @@ func (x *ComponentRequest) ProtoReflect() protoreflect.Message { // Deprecated: Use ComponentRequest.ProtoReflect.Descriptor instead. func (*ComponentRequest) Descriptor() ([]byte, []int) { - return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{37} + return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{39} } func (x *ComponentRequest) GetInvocation() *Invocation { @@ -2687,7 +2962,7 @@ type ComponentStateIdentity struct { func (x *ComponentStateIdentity) Reset() { *x = ComponentStateIdentity{} - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[38] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[40] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -2699,7 +2974,7 @@ func (x *ComponentStateIdentity) String() string { func (*ComponentStateIdentity) ProtoMessage() {} func (x *ComponentStateIdentity) ProtoReflect() protoreflect.Message { - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[38] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[40] if x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -2712,7 +2987,7 @@ func (x *ComponentStateIdentity) ProtoReflect() protoreflect.Message { // Deprecated: Use ComponentStateIdentity.ProtoReflect.Descriptor instead. func (*ComponentStateIdentity) Descriptor() ([]byte, []int) { - return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{38} + return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{40} } func (x *ComponentStateIdentity) GetBackendType() string { @@ -2754,14 +3029,17 @@ type ComponentGeneratedFile struct { // Optional POSIX permission bits. Zero means the host default (0644, or 0600 // for sensitive files). The host keeps only owner/group/other read and // execute bits plus owner write, and ignores the field on Windows. - Mode uint32 `protobuf:"varint,5,opt,name=mode,proto3" json:"mode,omitempty"` + Mode uint32 `protobuf:"varint,5,opt,name=mode,proto3" json:"mode,omitempty"` + // True only for a literal file copied from beside the configuration unit, + // never for an explicit generate block. Absent means normal policy handling. + UnitFileCopy bool `protobuf:"varint,6,opt,name=unit_file_copy,json=unitFileCopy,proto3" json:"unit_file_copy,omitempty"` unknownFields protoimpl.UnknownFields sizeCache protoimpl.SizeCache } func (x *ComponentGeneratedFile) Reset() { *x = ComponentGeneratedFile{} - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[39] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[41] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -2773,7 +3051,7 @@ func (x *ComponentGeneratedFile) String() string { func (*ComponentGeneratedFile) ProtoMessage() {} func (x *ComponentGeneratedFile) ProtoReflect() protoreflect.Message { - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[39] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[41] if x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -2786,7 +3064,7 @@ func (x *ComponentGeneratedFile) ProtoReflect() protoreflect.Message { // Deprecated: Use ComponentGeneratedFile.ProtoReflect.Descriptor instead. func (*ComponentGeneratedFile) Descriptor() ([]byte, []int) { - return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{39} + return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{41} } func (x *ComponentGeneratedFile) GetPath() string { @@ -2824,6 +3102,13 @@ func (x *ComponentGeneratedFile) GetMode() uint32 { return 0 } +func (x *ComponentGeneratedFile) GetUnitFileCopy() bool { + if x != nil { + return x.UnitFileCopy + } + return false +} + // Evaluated command metadata; the host executes it with the invocation's // environment, toolchain, cancellation, and I/O. Lists retain declaration order. type ComponentExecutionConfiguration struct { @@ -2844,7 +3129,7 @@ type ComponentExecutionConfiguration struct { func (x *ComponentExecutionConfiguration) Reset() { *x = ComponentExecutionConfiguration{} - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[40] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[42] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -2856,7 +3141,7 @@ func (x *ComponentExecutionConfiguration) String() string { func (*ComponentExecutionConfiguration) ProtoMessage() {} func (x *ComponentExecutionConfiguration) ProtoReflect() protoreflect.Message { - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[40] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[42] if x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -2869,7 +3154,7 @@ func (x *ComponentExecutionConfiguration) ProtoReflect() protoreflect.Message { // Deprecated: Use ComponentExecutionConfiguration.ProtoReflect.Descriptor instead. func (*ComponentExecutionConfiguration) Descriptor() ([]byte, []int) { - return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{40} + return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{42} } func (x *ComponentExecutionConfiguration) GetExtraArguments() []*ComponentExtraArguments { @@ -2920,7 +3205,7 @@ type ComponentExtraArguments struct { func (x *ComponentExtraArguments) Reset() { *x = ComponentExtraArguments{} - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[41] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[43] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -2932,7 +3217,7 @@ func (x *ComponentExtraArguments) String() string { func (*ComponentExtraArguments) ProtoMessage() {} func (x *ComponentExtraArguments) ProtoReflect() protoreflect.Message { - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[41] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[43] if x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -2945,7 +3230,7 @@ func (x *ComponentExtraArguments) ProtoReflect() protoreflect.Message { // Deprecated: Use ComponentExtraArguments.ProtoReflect.Descriptor instead. func (*ComponentExtraArguments) Descriptor() ([]byte, []int) { - return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{41} + return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{43} } func (x *ComponentExtraArguments) GetCommands() []string { @@ -3000,7 +3285,7 @@ type ComponentHook struct { func (x *ComponentHook) Reset() { *x = ComponentHook{} - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[42] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[44] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -3012,7 +3297,7 @@ func (x *ComponentHook) String() string { func (*ComponentHook) ProtoMessage() {} func (x *ComponentHook) ProtoReflect() protoreflect.Message { - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[42] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[44] if x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -3025,7 +3310,7 @@ func (x *ComponentHook) ProtoReflect() protoreflect.Message { // Deprecated: Use ComponentHook.ProtoReflect.Descriptor instead. func (*ComponentHook) Descriptor() ([]byte, []int) { - return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{42} + return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{44} } func (x *ComponentHook) GetName() string { @@ -3092,13 +3377,31 @@ type PrepareComponentResponse struct { // emit its default backend file. BackendGenerated bool `protobuf:"varint,10,opt,name=backend_generated,json=backendGenerated,proto3" json:"backend_generated,omitempty"` Execution *ComponentExecutionConfiguration `protobuf:"bytes,11,opt,name=execution,proto3" json:"execution,omitempty"` - unknownFields protoimpl.UnknownFields - sizeCache protoimpl.SizeCache + // Maps each input name to the authored file, line, and column that produced + // its value, for provenance reporting. Missing entries mean unknown origin. + InputOrigins map[string]*SourceLocation `protobuf:"bytes,12,rep,name=input_origins,json=inputOrigins,proto3" json:"input_origins,omitempty" protobuf_key:"bytes,1,opt,name=key" protobuf_val:"bytes,2,opt,name=value"` + // Fields 13-18 are reserved for Terragrunt parity. The host does not read + // them yet; a plugin that cannot honor a setting must reject it with an + // ErrorDetail instead of returning it, and must leave these fields unset. + // Engine executable selected by terraform_binary. + TerraformBinary string `protobuf:"bytes,13,opt,name=terraform_binary,json=terraformBinary,proto3" json:"terraform_binary,omitempty"` + // errors block: retry and ignore policies. + Errors *ComponentErrorsConfiguration `protobuf:"bytes,14,opt,name=errors,proto3" json:"errors,omitempty"` + // exclude block: conditional unit exclusion. + Exclude *ComponentExclude `protobuf:"bytes,15,opt,name=exclude,proto3" json:"exclude,omitempty"` + // iam_role, iam_assume_role_session_name, iam_assume_role_duration, iam_web_identity_token. + Iam *ComponentIam `protobuf:"bytes,16,opt,name=iam,proto3" json:"iam,omitempty"` + // encryption block as opaque, provider-defined data. + Encryption *ComponentValue `protobuf:"bytes,17,opt,name=encryption,proto3" json:"encryption,omitempty"` + // feature flag name to its evaluated value rendered as a string. + FeatureFlags map[string]string `protobuf:"bytes,18,rep,name=feature_flags,json=featureFlags,proto3" json:"feature_flags,omitempty" protobuf_key:"bytes,1,opt,name=key" protobuf_val:"bytes,2,opt,name=value"` + unknownFields protoimpl.UnknownFields + sizeCache protoimpl.SizeCache } func (x *PrepareComponentResponse) Reset() { *x = PrepareComponentResponse{} - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[43] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[45] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -3110,7 +3413,7 @@ func (x *PrepareComponentResponse) String() string { func (*PrepareComponentResponse) ProtoMessage() {} func (x *PrepareComponentResponse) ProtoReflect() protoreflect.Message { - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[43] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[45] if x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -3123,7 +3426,7 @@ func (x *PrepareComponentResponse) ProtoReflect() protoreflect.Message { // Deprecated: Use PrepareComponentResponse.ProtoReflect.Descriptor instead. func (*PrepareComponentResponse) Descriptor() ([]byte, []int) { - return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{43} + return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{45} } func (x *PrepareComponentResponse) GetEngine() string { @@ -3203,6 +3506,380 @@ func (x *PrepareComponentResponse) GetExecution() *ComponentExecutionConfigurati return nil } +func (x *PrepareComponentResponse) GetInputOrigins() map[string]*SourceLocation { + if x != nil { + return x.InputOrigins + } + return nil +} + +func (x *PrepareComponentResponse) GetTerraformBinary() string { + if x != nil { + return x.TerraformBinary + } + return "" +} + +func (x *PrepareComponentResponse) GetErrors() *ComponentErrorsConfiguration { + if x != nil { + return x.Errors + } + return nil +} + +func (x *PrepareComponentResponse) GetExclude() *ComponentExclude { + if x != nil { + return x.Exclude + } + return nil +} + +func (x *PrepareComponentResponse) GetIam() *ComponentIam { + if x != nil { + return x.Iam + } + return nil +} + +func (x *PrepareComponentResponse) GetEncryption() *ComponentValue { + if x != nil { + return x.Encryption + } + return nil +} + +func (x *PrepareComponentResponse) GetFeatureFlags() map[string]string { + if x != nil { + return x.FeatureFlags + } + return nil +} + +// ComponentErrorsConfiguration mirrors the Terragrunt errors block. Reserved; not implemented. +type ComponentErrorsConfiguration struct { + state protoimpl.MessageState `protogen:"open.v1"` + Retry []*ComponentRetryPolicy `protobuf:"bytes,1,rep,name=retry,proto3" json:"retry,omitempty"` + Ignore []*ComponentIgnorePolicy `protobuf:"bytes,2,rep,name=ignore,proto3" json:"ignore,omitempty"` + unknownFields protoimpl.UnknownFields + sizeCache protoimpl.SizeCache +} + +func (x *ComponentErrorsConfiguration) Reset() { + *x = ComponentErrorsConfiguration{} + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[46] + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + ms.StoreMessageInfo(mi) +} + +func (x *ComponentErrorsConfiguration) String() string { + return protoimpl.X.MessageStringOf(x) +} + +func (*ComponentErrorsConfiguration) ProtoMessage() {} + +func (x *ComponentErrorsConfiguration) ProtoReflect() protoreflect.Message { + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[46] + if x != nil { + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + if ms.LoadMessageInfo() == nil { + ms.StoreMessageInfo(mi) + } + return ms + } + return mi.MessageOf(x) +} + +// Deprecated: Use ComponentErrorsConfiguration.ProtoReflect.Descriptor instead. +func (*ComponentErrorsConfiguration) Descriptor() ([]byte, []int) { + return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{46} +} + +func (x *ComponentErrorsConfiguration) GetRetry() []*ComponentRetryPolicy { + if x != nil { + return x.Retry + } + return nil +} + +func (x *ComponentErrorsConfiguration) GetIgnore() []*ComponentIgnorePolicy { + if x != nil { + return x.Ignore + } + return nil +} + +type ComponentRetryPolicy struct { + state protoimpl.MessageState `protogen:"open.v1"` + Name string `protobuf:"bytes,1,opt,name=name,proto3" json:"name,omitempty"` + // Regular expressions matched against command errors. + RetryableErrors []string `protobuf:"bytes,2,rep,name=retryable_errors,json=retryableErrors,proto3" json:"retryable_errors,omitempty"` + MaxAttempts uint32 `protobuf:"varint,3,opt,name=max_attempts,json=maxAttempts,proto3" json:"max_attempts,omitempty"` + SleepIntervalSeconds uint32 `protobuf:"varint,4,opt,name=sleep_interval_seconds,json=sleepIntervalSeconds,proto3" json:"sleep_interval_seconds,omitempty"` + unknownFields protoimpl.UnknownFields + sizeCache protoimpl.SizeCache +} + +func (x *ComponentRetryPolicy) Reset() { + *x = ComponentRetryPolicy{} + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[47] + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + ms.StoreMessageInfo(mi) +} + +func (x *ComponentRetryPolicy) String() string { + return protoimpl.X.MessageStringOf(x) +} + +func (*ComponentRetryPolicy) ProtoMessage() {} + +func (x *ComponentRetryPolicy) ProtoReflect() protoreflect.Message { + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[47] + if x != nil { + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + if ms.LoadMessageInfo() == nil { + ms.StoreMessageInfo(mi) + } + return ms + } + return mi.MessageOf(x) +} + +// Deprecated: Use ComponentRetryPolicy.ProtoReflect.Descriptor instead. +func (*ComponentRetryPolicy) Descriptor() ([]byte, []int) { + return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{47} +} + +func (x *ComponentRetryPolicy) GetName() string { + if x != nil { + return x.Name + } + return "" +} + +func (x *ComponentRetryPolicy) GetRetryableErrors() []string { + if x != nil { + return x.RetryableErrors + } + return nil +} + +func (x *ComponentRetryPolicy) GetMaxAttempts() uint32 { + if x != nil { + return x.MaxAttempts + } + return 0 +} + +func (x *ComponentRetryPolicy) GetSleepIntervalSeconds() uint32 { + if x != nil { + return x.SleepIntervalSeconds + } + return 0 +} + +type ComponentIgnorePolicy struct { + state protoimpl.MessageState `protogen:"open.v1"` + Name string `protobuf:"bytes,1,opt,name=name,proto3" json:"name,omitempty"` + // Regular expressions matched against command errors. + IgnorableErrors []string `protobuf:"bytes,2,rep,name=ignorable_errors,json=ignorableErrors,proto3" json:"ignorable_errors,omitempty"` + Message string `protobuf:"bytes,3,opt,name=message,proto3" json:"message,omitempty"` + // Extra signals written for the caller, keyed by name. + Signals map[string]*ComponentValue `protobuf:"bytes,4,rep,name=signals,proto3" json:"signals,omitempty" protobuf_key:"bytes,1,opt,name=key" protobuf_val:"bytes,2,opt,name=value"` + unknownFields protoimpl.UnknownFields + sizeCache protoimpl.SizeCache +} + +func (x *ComponentIgnorePolicy) Reset() { + *x = ComponentIgnorePolicy{} + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[48] + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + ms.StoreMessageInfo(mi) +} + +func (x *ComponentIgnorePolicy) String() string { + return protoimpl.X.MessageStringOf(x) +} + +func (*ComponentIgnorePolicy) ProtoMessage() {} + +func (x *ComponentIgnorePolicy) ProtoReflect() protoreflect.Message { + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[48] + if x != nil { + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + if ms.LoadMessageInfo() == nil { + ms.StoreMessageInfo(mi) + } + return ms + } + return mi.MessageOf(x) +} + +// Deprecated: Use ComponentIgnorePolicy.ProtoReflect.Descriptor instead. +func (*ComponentIgnorePolicy) Descriptor() ([]byte, []int) { + return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{48} +} + +func (x *ComponentIgnorePolicy) GetName() string { + if x != nil { + return x.Name + } + return "" +} + +func (x *ComponentIgnorePolicy) GetIgnorableErrors() []string { + if x != nil { + return x.IgnorableErrors + } + return nil +} + +func (x *ComponentIgnorePolicy) GetMessage() string { + if x != nil { + return x.Message + } + return "" +} + +func (x *ComponentIgnorePolicy) GetSignals() map[string]*ComponentValue { + if x != nil { + return x.Signals + } + return nil +} + +// ComponentExclude mirrors the Terragrunt exclude block. Reserved; not implemented. +type ComponentExclude struct { + state protoimpl.MessageState `protogen:"open.v1"` + // Evaluated "if" condition. + Condition bool `protobuf:"varint,1,opt,name=condition,proto3" json:"condition,omitempty"` + // Commands the exclusion applies to; "all" and "all_except_output" are plugin vocabulary. + Actions []string `protobuf:"bytes,2,rep,name=actions,proto3" json:"actions,omitempty"` + ExcludeDependencies bool `protobuf:"varint,3,opt,name=exclude_dependencies,json=excludeDependencies,proto3" json:"exclude_dependencies,omitempty"` + unknownFields protoimpl.UnknownFields + sizeCache protoimpl.SizeCache +} + +func (x *ComponentExclude) Reset() { + *x = ComponentExclude{} + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[49] + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + ms.StoreMessageInfo(mi) +} + +func (x *ComponentExclude) String() string { + return protoimpl.X.MessageStringOf(x) +} + +func (*ComponentExclude) ProtoMessage() {} + +func (x *ComponentExclude) ProtoReflect() protoreflect.Message { + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[49] + if x != nil { + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + if ms.LoadMessageInfo() == nil { + ms.StoreMessageInfo(mi) + } + return ms + } + return mi.MessageOf(x) +} + +// Deprecated: Use ComponentExclude.ProtoReflect.Descriptor instead. +func (*ComponentExclude) Descriptor() ([]byte, []int) { + return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{49} +} + +func (x *ComponentExclude) GetCondition() bool { + if x != nil { + return x.Condition + } + return false +} + +func (x *ComponentExclude) GetActions() []string { + if x != nil { + return x.Actions + } + return nil +} + +func (x *ComponentExclude) GetExcludeDependencies() bool { + if x != nil { + return x.ExcludeDependencies + } + return false +} + +// ComponentIam mirrors the Terragrunt IAM settings. Reserved; not implemented. +// The web identity token is a secret and must be carried as a sensitive value. +type ComponentIam struct { + state protoimpl.MessageState `protogen:"open.v1"` + Role string `protobuf:"bytes,1,opt,name=role,proto3" json:"role,omitempty"` + SessionName string `protobuf:"bytes,2,opt,name=session_name,json=sessionName,proto3" json:"session_name,omitempty"` + DurationSeconds uint32 `protobuf:"varint,3,opt,name=duration_seconds,json=durationSeconds,proto3" json:"duration_seconds,omitempty"` + WebIdentityToken *ComponentValue `protobuf:"bytes,4,opt,name=web_identity_token,json=webIdentityToken,proto3" json:"web_identity_token,omitempty"` + unknownFields protoimpl.UnknownFields + sizeCache protoimpl.SizeCache +} + +func (x *ComponentIam) Reset() { + *x = ComponentIam{} + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[50] + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + ms.StoreMessageInfo(mi) +} + +func (x *ComponentIam) String() string { + return protoimpl.X.MessageStringOf(x) +} + +func (*ComponentIam) ProtoMessage() {} + +func (x *ComponentIam) ProtoReflect() protoreflect.Message { + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[50] + if x != nil { + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + if ms.LoadMessageInfo() == nil { + ms.StoreMessageInfo(mi) + } + return ms + } + return mi.MessageOf(x) +} + +// Deprecated: Use ComponentIam.ProtoReflect.Descriptor instead. +func (*ComponentIam) Descriptor() ([]byte, []int) { + return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{50} +} + +func (x *ComponentIam) GetRole() string { + if x != nil { + return x.Role + } + return "" +} + +func (x *ComponentIam) GetSessionName() string { + if x != nil { + return x.SessionName + } + return "" +} + +func (x *ComponentIam) GetDurationSeconds() uint32 { + if x != nil { + return x.DurationSeconds + } + return 0 +} + +func (x *ComponentIam) GetWebIdentityToken() *ComponentValue { + if x != nil { + return x.WebIdentityToken + } + return nil +} + type ResolveComponentStateResponse struct { state protoimpl.MessageState `protogen:"open.v1"` State *ComponentStateIdentity `protobuf:"bytes,1,opt,name=state,proto3" json:"state,omitempty"` @@ -3214,7 +3891,7 @@ type ResolveComponentStateResponse struct { func (x *ResolveComponentStateResponse) Reset() { *x = ResolveComponentStateResponse{} - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[44] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[51] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -3226,7 +3903,7 @@ func (x *ResolveComponentStateResponse) String() string { func (*ResolveComponentStateResponse) ProtoMessage() {} func (x *ResolveComponentStateResponse) ProtoReflect() protoreflect.Message { - mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[44] + mi := &file_atmos_plugin_v1_plugin_proto_msgTypes[51] if x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -3239,7 +3916,7 @@ func (x *ResolveComponentStateResponse) ProtoReflect() protoreflect.Message { // Deprecated: Use ResolveComponentStateResponse.ProtoReflect.Descriptor instead. func (*ResolveComponentStateResponse) Descriptor() ([]byte, []int) { - return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{44} + return file_atmos_plugin_v1_plugin_proto_rawDescGZIP(), []int{51} } func (x *ResolveComponentStateResponse) GetState() *ComponentStateIdentity { @@ -3346,11 +4023,30 @@ const file_atmos_plugin_v1_plugin_proto_rawDesc = "" + "\bprogress\x18\x01 \x01(\v2\x19.atmos.plugin.v1.ProgressH\x00R\bprogress\x12@\n" + "\vinteraction\x18\x02 \x01(\v2\x1c.atmos.plugin.v1.InteractionH\x00R\vinteraction\x126\n" + "\x06result\x18\x03 \x01(\v2\x1c.atmos.plugin.v1.CredentialsH\x00R\x06resultB\a\n" + - "\x05event\"<\n" + + "\x05event\"P\n" + + "\x0eSourceLocation\x12\x12\n" + + "\x04file\x18\x01 \x01(\tR\x04file\x12\x12\n" + + "\x04line\x18\x02 \x01(\rR\x04line\x12\x16\n" + + "\x06column\x18\x03 \x01(\rR\x06column\"\xe1\x02\n" + + "\vErrorDetail\x12\x12\n" + + "\x04code\x18\x01 \x01(\tR\x04code\x12\x18\n" + + "\asummary\x18\x02 \x01(\tR\asummary\x12\x16\n" + + "\x06detail\x18\x03 \x01(\tR\x06detail\x12\x14\n" + + "\x05hints\x18\x04 \x03(\tR\x05hints\x12;\n" + + "\blocation\x18\x05 \x01(\v2\x1f.atmos.plugin.v1.SourceLocationR\blocation\x12\x12\n" + + "\x04unit\x18\x06 \x01(\tR\x04unit\x12\x14\n" + + "\x05stack\x18\a \x01(\tR\x05stack\x12\x1d\n" + + "\n" + + "config_key\x18\b \x01(\tR\tconfigKey\x12\x1b\n" + + "\tuser_safe\x18\t \x01(\bR\buserSafe\x12\x1c\n" + + "\toperation\x18\n" + + " \x01(\tR\toperation\x125\n" + + "\bseverity\x18\v \x01(\x0e2\x19.atmos.plugin.v1.SeverityR\bseverity\"r\n" + "\n" + "Diagnostic\x12\x14\n" + "\x05field\x18\x01 \x01(\tR\x05field\x12\x18\n" + - "\amessage\x18\x02 \x01(\tR\amessage\"Q\n" + + "\amessage\x18\x02 \x01(\tR\amessage\x124\n" + + "\x06detail\x18\x03 \x01(\v2\x1c.atmos.plugin.v1.ErrorDetailR\x06detail\"Q\n" + "\x10ValidateResponse\x12=\n" + "\vdiagnostics\x18\x01 \x03(\v2\x1b.atmos.plugin.v1.DiagnosticR\vdiagnostics\"C\n" + "\vUnsupported\x12\x1c\n" + @@ -3494,13 +4190,14 @@ const file_atmos_plugin_v1_plugin_proto_rawDesc = "" + "\fbackend_type\x18\x01 \x01(\tR\vbackendType\x129\n" + "\abackend\x18\x02 \x01(\v2\x1f.atmos.plugin.v1.ComponentValueR\abackend\x12\x1c\n" + "\tworkspace\x18\x03 \x01(\tR\tworkspace\x12+\n" + - "\x11working_directory\x18\x04 \x01(\tR\x10workingDirectory\"\x95\x01\n" + + "\x11working_directory\x18\x04 \x01(\tR\x10workingDirectory\"\xbb\x01\n" + "\x16ComponentGeneratedFile\x12\x12\n" + "\x04path\x18\x01 \x01(\tR\x04path\x12\x18\n" + "\acontent\x18\x02 \x01(\fR\acontent\x12\x1b\n" + "\tif_exists\x18\x03 \x01(\tR\bifExists\x12\x1c\n" + "\tsensitive\x18\x04 \x01(\bR\tsensitive\x12\x12\n" + - "\x04mode\x18\x05 \x01(\rR\x04mode\"\x81\x03\n" + + "\x04mode\x18\x05 \x01(\rR\x04mode\x12$\n" + + "\x0eunit_file_copy\x18\x06 \x01(\bR\funitFileCopy\"\x81\x03\n" + "\x1fComponentExecutionConfiguration\x12Q\n" + "\x0fextra_arguments\x18\x01 \x03(\v2(.atmos.plugin.v1.ComponentExtraArgumentsR\x0eextraArguments\x12A\n" + "\fbefore_hooks\x18\x02 \x03(\v2\x1e.atmos.plugin.v1.ComponentHookR\vbeforeHooks\x12?\n" + @@ -3526,7 +4223,8 @@ const file_atmos_plugin_v1_plugin_proto_rawDesc = "" + "\frun_on_error\x18\x05 \x01(\bR\n" + "runOnError\x12'\n" + "\x0fsuppress_stdout\x18\x06 \x01(\bR\x0esuppressStdout\x12\x1b\n" + - "\ton_errors\x18\a \x03(\tR\bonErrors\"\xfd\x04\n" + + "\ton_errors\x18\a \x03(\tR\bonErrors\"\x85\n" + + "\n" + "\x18PrepareComponentResponse\x12\x16\n" + "\x06engine\x18\x01 \x01(\tR\x06engine\x12+\n" + "\x11working_directory\x18\x02 \x01(\tR\x10workingDirectory\x12#\n" + @@ -3539,11 +4237,55 @@ const file_atmos_plugin_v1_plugin_proto_rawDesc = "" + "\vdiagnostics\x18\t \x03(\v2\x1b.atmos.plugin.v1.DiagnosticR\vdiagnostics\x12+\n" + "\x11backend_generated\x18\n" + " \x01(\bR\x10backendGenerated\x12N\n" + - "\texecution\x18\v \x01(\v20.atmos.plugin.v1.ComponentExecutionConfigurationR\texecution\"\xd9\x01\n" + + "\texecution\x18\v \x01(\v20.atmos.plugin.v1.ComponentExecutionConfigurationR\texecution\x12`\n" + + "\rinput_origins\x18\f \x03(\v2;.atmos.plugin.v1.PrepareComponentResponse.InputOriginsEntryR\finputOrigins\x12)\n" + + "\x10terraform_binary\x18\r \x01(\tR\x0fterraformBinary\x12E\n" + + "\x06errors\x18\x0e \x01(\v2-.atmos.plugin.v1.ComponentErrorsConfigurationR\x06errors\x12;\n" + + "\aexclude\x18\x0f \x01(\v2!.atmos.plugin.v1.ComponentExcludeR\aexclude\x12/\n" + + "\x03iam\x18\x10 \x01(\v2\x1d.atmos.plugin.v1.ComponentIamR\x03iam\x12?\n" + + "\n" + + "encryption\x18\x11 \x01(\v2\x1f.atmos.plugin.v1.ComponentValueR\n" + + "encryption\x12`\n" + + "\rfeature_flags\x18\x12 \x03(\v2;.atmos.plugin.v1.PrepareComponentResponse.FeatureFlagsEntryR\ffeatureFlags\x1a`\n" + + "\x11InputOriginsEntry\x12\x10\n" + + "\x03key\x18\x01 \x01(\tR\x03key\x125\n" + + "\x05value\x18\x02 \x01(\v2\x1f.atmos.plugin.v1.SourceLocationR\x05value:\x028\x01\x1a?\n" + + "\x11FeatureFlagsEntry\x12\x10\n" + + "\x03key\x18\x01 \x01(\tR\x03key\x12\x14\n" + + "\x05value\x18\x02 \x01(\tR\x05value:\x028\x01\"\x9b\x01\n" + + "\x1cComponentErrorsConfiguration\x12;\n" + + "\x05retry\x18\x01 \x03(\v2%.atmos.plugin.v1.ComponentRetryPolicyR\x05retry\x12>\n" + + "\x06ignore\x18\x02 \x03(\v2&.atmos.plugin.v1.ComponentIgnorePolicyR\x06ignore\"\xae\x01\n" + + "\x14ComponentRetryPolicy\x12\x12\n" + + "\x04name\x18\x01 \x01(\tR\x04name\x12)\n" + + "\x10retryable_errors\x18\x02 \x03(\tR\x0fretryableErrors\x12!\n" + + "\fmax_attempts\x18\x03 \x01(\rR\vmaxAttempts\x124\n" + + "\x16sleep_interval_seconds\x18\x04 \x01(\rR\x14sleepIntervalSeconds\"\x9c\x02\n" + + "\x15ComponentIgnorePolicy\x12\x12\n" + + "\x04name\x18\x01 \x01(\tR\x04name\x12)\n" + + "\x10ignorable_errors\x18\x02 \x03(\tR\x0fignorableErrors\x12\x18\n" + + "\amessage\x18\x03 \x01(\tR\amessage\x12M\n" + + "\asignals\x18\x04 \x03(\v23.atmos.plugin.v1.ComponentIgnorePolicy.SignalsEntryR\asignals\x1a[\n" + + "\fSignalsEntry\x12\x10\n" + + "\x03key\x18\x01 \x01(\tR\x03key\x125\n" + + "\x05value\x18\x02 \x01(\v2\x1f.atmos.plugin.v1.ComponentValueR\x05value:\x028\x01\"}\n" + + "\x10ComponentExclude\x12\x1c\n" + + "\tcondition\x18\x01 \x01(\bR\tcondition\x12\x18\n" + + "\aactions\x18\x02 \x03(\tR\aactions\x121\n" + + "\x14exclude_dependencies\x18\x03 \x01(\bR\x13excludeDependencies\"\xbf\x01\n" + + "\fComponentIam\x12\x12\n" + + "\x04role\x18\x01 \x01(\tR\x04role\x12!\n" + + "\fsession_name\x18\x02 \x01(\tR\vsessionName\x12)\n" + + "\x10duration_seconds\x18\x03 \x01(\rR\x0fdurationSeconds\x12M\n" + + "\x12web_identity_token\x18\x04 \x01(\v2\x1f.atmos.plugin.v1.ComponentValueR\x10webIdentityToken\"\xd9\x01\n" + "\x1dResolveComponentStateResponse\x12=\n" + "\x05state\x18\x01 \x01(\v2'.atmos.plugin.v1.ComponentStateIdentityR\x05state\x12:\n" + "\asources\x18\x02 \x03(\v2 .atmos.plugin.v1.ComponentSourceR\asources\x12=\n" + - "\vdiagnostics\x18\x03 \x03(\v2\x1b.atmos.plugin.v1.DiagnosticR\vdiagnostics2\x9f\x01\n" + + "\vdiagnostics\x18\x03 \x03(\v2\x1b.atmos.plugin.v1.DiagnosticR\vdiagnostics*N\n" + + "\bSeverity\x12\x18\n" + + "\x14SEVERITY_UNSPECIFIED\x10\x00\x12\x12\n" + + "\x0eSEVERITY_ERROR\x10\x01\x12\x14\n" + + "\x10SEVERITY_WARNING\x10\x022\x9f\x01\n" + "\x04Info\x12O\n" + "\bDescribe\x12 .atmos.plugin.v1.DescribeRequest\x1a!.atmos.plugin.v1.DescribeResponse\x12F\n" + "\tConfigure\x12!.atmos.plugin.v1.ConfigureRequest\x1a\x16.google.protobuf.Empty2\xd3\x04\n" + @@ -3586,191 +4328,217 @@ func file_atmos_plugin_v1_plugin_proto_rawDescGZIP() []byte { } var ( - file_atmos_plugin_v1_plugin_proto_enumTypes = make([]protoimpl.EnumInfo, 2) - file_atmos_plugin_v1_plugin_proto_msgTypes = make([]protoimpl.MessageInfo, 53) + file_atmos_plugin_v1_plugin_proto_enumTypes = make([]protoimpl.EnumInfo, 3) + file_atmos_plugin_v1_plugin_proto_msgTypes = make([]protoimpl.MessageInfo, 63) file_atmos_plugin_v1_plugin_proto_goTypes = []any{ - ComponentSource_Kind(0), // 0: atmos.plugin.v1.ComponentSource.Kind - ComponentDependency_Kind(0), // 1: atmos.plugin.v1.ComponentDependency.Kind - (*Invocation)(nil), // 2: atmos.plugin.v1.Invocation - (*Credentials)(nil), // 3: atmos.plugin.v1.Credentials - (*ServiceVersionRange)(nil), // 4: atmos.plugin.v1.ServiceVersionRange - (*Export)(nil), // 5: atmos.plugin.v1.Export - (*DescribeRequest)(nil), // 6: atmos.plugin.v1.DescribeRequest - (*DescribeResponse)(nil), // 7: atmos.plugin.v1.DescribeResponse - (*ConfigureRequest)(nil), // 8: atmos.plugin.v1.ConfigureRequest - (*ProviderRequest)(nil), // 9: atmos.plugin.v1.ProviderRequest - (*IdentityRequest)(nil), // 10: atmos.plugin.v1.IdentityRequest - (*Progress)(nil), // 11: atmos.plugin.v1.Progress - (*Interaction)(nil), // 12: atmos.plugin.v1.Interaction - (*AuthenticateEvent)(nil), // 13: atmos.plugin.v1.AuthenticateEvent - (*Diagnostic)(nil), // 14: atmos.plugin.v1.Diagnostic - (*ValidateResponse)(nil), // 15: atmos.plugin.v1.ValidateResponse - (*Unsupported)(nil), // 16: atmos.plugin.v1.Unsupported - (*RefreshRequest)(nil), // 17: atmos.plugin.v1.RefreshRequest - (*RefreshResponse)(nil), // 18: atmos.plugin.v1.RefreshResponse - (*EnvironmentResponse)(nil), // 19: atmos.plugin.v1.EnvironmentResponse - (*PrepareEnvironmentRequest)(nil), // 20: atmos.plugin.v1.PrepareEnvironmentRequest - (*CredentialPath)(nil), // 21: atmos.plugin.v1.CredentialPath - (*PathsResponse)(nil), // 22: atmos.plugin.v1.PathsResponse - (*IdentityAuthenticateRequest)(nil), // 23: atmos.plugin.v1.IdentityAuthenticateRequest - (*IdentityPrepareRequest)(nil), // 24: atmos.plugin.v1.IdentityPrepareRequest - (*CommandRequest)(nil), // 25: atmos.plugin.v1.CommandRequest - (*CommandExit)(nil), // 26: atmos.plugin.v1.CommandExit - (*CommandEvent)(nil), // 27: atmos.plugin.v1.CommandEvent - (*ComponentValue)(nil), // 28: atmos.plugin.v1.ComponentValue - (*ComponentValuePath)(nil), // 29: atmos.plugin.v1.ComponentValuePath - (*ComponentValueStep)(nil), // 30: atmos.plugin.v1.ComponentValueStep - (*ComponentAddress)(nil), // 31: atmos.plugin.v1.ComponentAddress - (*ComponentSource)(nil), // 32: atmos.plugin.v1.ComponentSource - (*ComponentDependency)(nil), // 33: atmos.plugin.v1.ComponentDependency - (*DiscoveredComponent)(nil), // 34: atmos.plugin.v1.DiscoveredComponent - (*ComponentEffects)(nil), // 35: atmos.plugin.v1.ComponentEffects - (*ComponentOperation)(nil), // 36: atmos.plugin.v1.ComponentOperation - (*DiscoverComponentsRequest)(nil), // 37: atmos.plugin.v1.DiscoverComponentsRequest - (*DiscoverComponentsResponse)(nil), // 38: atmos.plugin.v1.DiscoverComponentsResponse - (*ComponentRequest)(nil), // 39: atmos.plugin.v1.ComponentRequest - (*ComponentStateIdentity)(nil), // 40: atmos.plugin.v1.ComponentStateIdentity - (*ComponentGeneratedFile)(nil), // 41: atmos.plugin.v1.ComponentGeneratedFile - (*ComponentExecutionConfiguration)(nil), // 42: atmos.plugin.v1.ComponentExecutionConfiguration - (*ComponentExtraArguments)(nil), // 43: atmos.plugin.v1.ComponentExtraArguments - (*ComponentHook)(nil), // 44: atmos.plugin.v1.ComponentHook - (*PrepareComponentResponse)(nil), // 45: atmos.plugin.v1.PrepareComponentResponse - (*ResolveComponentStateResponse)(nil), // 46: atmos.plugin.v1.ResolveComponentStateResponse - nil, // 47: atmos.plugin.v1.Invocation.EnvironmentEntry - nil, // 48: atmos.plugin.v1.ConfigureRequest.BrokerIdsEntry - nil, // 49: atmos.plugin.v1.Interaction.FieldsEntry - nil, // 50: atmos.plugin.v1.EnvironmentResponse.EnvironmentEntry - nil, // 51: atmos.plugin.v1.PrepareEnvironmentRequest.EnvironmentEntry - nil, // 52: atmos.plugin.v1.IdentityPrepareRequest.EnvironmentEntry - nil, // 53: atmos.plugin.v1.ComponentOperation.OptionsEntry - nil, // 54: atmos.plugin.v1.ComponentExtraArguments.EnvironmentEntry - (*timestamppb.Timestamp)(nil), // 55: google.protobuf.Timestamp - (*structpb.Struct)(nil), // 56: google.protobuf.Struct - (*emptypb.Empty)(nil), // 57: google.protobuf.Empty + Severity(0), // 0: atmos.plugin.v1.Severity + ComponentSource_Kind(0), // 1: atmos.plugin.v1.ComponentSource.Kind + ComponentDependency_Kind(0), // 2: atmos.plugin.v1.ComponentDependency.Kind + (*Invocation)(nil), // 3: atmos.plugin.v1.Invocation + (*Credentials)(nil), // 4: atmos.plugin.v1.Credentials + (*ServiceVersionRange)(nil), // 5: atmos.plugin.v1.ServiceVersionRange + (*Export)(nil), // 6: atmos.plugin.v1.Export + (*DescribeRequest)(nil), // 7: atmos.plugin.v1.DescribeRequest + (*DescribeResponse)(nil), // 8: atmos.plugin.v1.DescribeResponse + (*ConfigureRequest)(nil), // 9: atmos.plugin.v1.ConfigureRequest + (*ProviderRequest)(nil), // 10: atmos.plugin.v1.ProviderRequest + (*IdentityRequest)(nil), // 11: atmos.plugin.v1.IdentityRequest + (*Progress)(nil), // 12: atmos.plugin.v1.Progress + (*Interaction)(nil), // 13: atmos.plugin.v1.Interaction + (*AuthenticateEvent)(nil), // 14: atmos.plugin.v1.AuthenticateEvent + (*SourceLocation)(nil), // 15: atmos.plugin.v1.SourceLocation + (*ErrorDetail)(nil), // 16: atmos.plugin.v1.ErrorDetail + (*Diagnostic)(nil), // 17: atmos.plugin.v1.Diagnostic + (*ValidateResponse)(nil), // 18: atmos.plugin.v1.ValidateResponse + (*Unsupported)(nil), // 19: atmos.plugin.v1.Unsupported + (*RefreshRequest)(nil), // 20: atmos.plugin.v1.RefreshRequest + (*RefreshResponse)(nil), // 21: atmos.plugin.v1.RefreshResponse + (*EnvironmentResponse)(nil), // 22: atmos.plugin.v1.EnvironmentResponse + (*PrepareEnvironmentRequest)(nil), // 23: atmos.plugin.v1.PrepareEnvironmentRequest + (*CredentialPath)(nil), // 24: atmos.plugin.v1.CredentialPath + (*PathsResponse)(nil), // 25: atmos.plugin.v1.PathsResponse + (*IdentityAuthenticateRequest)(nil), // 26: atmos.plugin.v1.IdentityAuthenticateRequest + (*IdentityPrepareRequest)(nil), // 27: atmos.plugin.v1.IdentityPrepareRequest + (*CommandRequest)(nil), // 28: atmos.plugin.v1.CommandRequest + (*CommandExit)(nil), // 29: atmos.plugin.v1.CommandExit + (*CommandEvent)(nil), // 30: atmos.plugin.v1.CommandEvent + (*ComponentValue)(nil), // 31: atmos.plugin.v1.ComponentValue + (*ComponentValuePath)(nil), // 32: atmos.plugin.v1.ComponentValuePath + (*ComponentValueStep)(nil), // 33: atmos.plugin.v1.ComponentValueStep + (*ComponentAddress)(nil), // 34: atmos.plugin.v1.ComponentAddress + (*ComponentSource)(nil), // 35: atmos.plugin.v1.ComponentSource + (*ComponentDependency)(nil), // 36: atmos.plugin.v1.ComponentDependency + (*DiscoveredComponent)(nil), // 37: atmos.plugin.v1.DiscoveredComponent + (*ComponentEffects)(nil), // 38: atmos.plugin.v1.ComponentEffects + (*ComponentOperation)(nil), // 39: atmos.plugin.v1.ComponentOperation + (*DiscoverComponentsRequest)(nil), // 40: atmos.plugin.v1.DiscoverComponentsRequest + (*DiscoverComponentsResponse)(nil), // 41: atmos.plugin.v1.DiscoverComponentsResponse + (*ComponentRequest)(nil), // 42: atmos.plugin.v1.ComponentRequest + (*ComponentStateIdentity)(nil), // 43: atmos.plugin.v1.ComponentStateIdentity + (*ComponentGeneratedFile)(nil), // 44: atmos.plugin.v1.ComponentGeneratedFile + (*ComponentExecutionConfiguration)(nil), // 45: atmos.plugin.v1.ComponentExecutionConfiguration + (*ComponentExtraArguments)(nil), // 46: atmos.plugin.v1.ComponentExtraArguments + (*ComponentHook)(nil), // 47: atmos.plugin.v1.ComponentHook + (*PrepareComponentResponse)(nil), // 48: atmos.plugin.v1.PrepareComponentResponse + (*ComponentErrorsConfiguration)(nil), // 49: atmos.plugin.v1.ComponentErrorsConfiguration + (*ComponentRetryPolicy)(nil), // 50: atmos.plugin.v1.ComponentRetryPolicy + (*ComponentIgnorePolicy)(nil), // 51: atmos.plugin.v1.ComponentIgnorePolicy + (*ComponentExclude)(nil), // 52: atmos.plugin.v1.ComponentExclude + (*ComponentIam)(nil), // 53: atmos.plugin.v1.ComponentIam + (*ResolveComponentStateResponse)(nil), // 54: atmos.plugin.v1.ResolveComponentStateResponse + nil, // 55: atmos.plugin.v1.Invocation.EnvironmentEntry + nil, // 56: atmos.plugin.v1.ConfigureRequest.BrokerIdsEntry + nil, // 57: atmos.plugin.v1.Interaction.FieldsEntry + nil, // 58: atmos.plugin.v1.EnvironmentResponse.EnvironmentEntry + nil, // 59: atmos.plugin.v1.PrepareEnvironmentRequest.EnvironmentEntry + nil, // 60: atmos.plugin.v1.IdentityPrepareRequest.EnvironmentEntry + nil, // 61: atmos.plugin.v1.ComponentOperation.OptionsEntry + nil, // 62: atmos.plugin.v1.ComponentExtraArguments.EnvironmentEntry + nil, // 63: atmos.plugin.v1.PrepareComponentResponse.InputOriginsEntry + nil, // 64: atmos.plugin.v1.PrepareComponentResponse.FeatureFlagsEntry + nil, // 65: atmos.plugin.v1.ComponentIgnorePolicy.SignalsEntry + (*timestamppb.Timestamp)(nil), // 66: google.protobuf.Timestamp + (*structpb.Struct)(nil), // 67: google.protobuf.Struct + (*emptypb.Empty)(nil), // 68: google.protobuf.Empty } ) var file_atmos_plugin_v1_plugin_proto_depIdxs = []int32{ - 55, // 0: atmos.plugin.v1.Invocation.deadline:type_name -> google.protobuf.Timestamp - 47, // 1: atmos.plugin.v1.Invocation.environment:type_name -> atmos.plugin.v1.Invocation.EnvironmentEntry - 55, // 2: atmos.plugin.v1.Credentials.expires_at:type_name -> google.protobuf.Timestamp - 56, // 3: atmos.plugin.v1.Credentials.data:type_name -> google.protobuf.Struct - 4, // 4: atmos.plugin.v1.DescribeRequest.host_supported:type_name -> atmos.plugin.v1.ServiceVersionRange - 4, // 5: atmos.plugin.v1.DescribeResponse.services:type_name -> atmos.plugin.v1.ServiceVersionRange - 5, // 6: atmos.plugin.v1.DescribeResponse.exports:type_name -> atmos.plugin.v1.Export - 48, // 7: atmos.plugin.v1.ConfigureRequest.broker_ids:type_name -> atmos.plugin.v1.ConfigureRequest.BrokerIdsEntry - 56, // 8: atmos.plugin.v1.ConfigureRequest.settings:type_name -> google.protobuf.Struct - 2, // 9: atmos.plugin.v1.ProviderRequest.invocation:type_name -> atmos.plugin.v1.Invocation - 56, // 10: atmos.plugin.v1.ProviderRequest.spec:type_name -> google.protobuf.Struct - 2, // 11: atmos.plugin.v1.IdentityRequest.invocation:type_name -> atmos.plugin.v1.Invocation - 56, // 12: atmos.plugin.v1.IdentityRequest.spec:type_name -> google.protobuf.Struct - 56, // 13: atmos.plugin.v1.IdentityRequest.principal:type_name -> google.protobuf.Struct - 49, // 14: atmos.plugin.v1.Interaction.fields:type_name -> atmos.plugin.v1.Interaction.FieldsEntry - 11, // 15: atmos.plugin.v1.AuthenticateEvent.progress:type_name -> atmos.plugin.v1.Progress - 12, // 16: atmos.plugin.v1.AuthenticateEvent.interaction:type_name -> atmos.plugin.v1.Interaction - 3, // 17: atmos.plugin.v1.AuthenticateEvent.result:type_name -> atmos.plugin.v1.Credentials - 14, // 18: atmos.plugin.v1.ValidateResponse.diagnostics:type_name -> atmos.plugin.v1.Diagnostic - 9, // 19: atmos.plugin.v1.RefreshRequest.provider:type_name -> atmos.plugin.v1.ProviderRequest - 3, // 20: atmos.plugin.v1.RefreshRequest.current:type_name -> atmos.plugin.v1.Credentials - 3, // 21: atmos.plugin.v1.RefreshResponse.credentials:type_name -> atmos.plugin.v1.Credentials - 16, // 22: atmos.plugin.v1.RefreshResponse.unsupported:type_name -> atmos.plugin.v1.Unsupported - 50, // 23: atmos.plugin.v1.EnvironmentResponse.environment:type_name -> atmos.plugin.v1.EnvironmentResponse.EnvironmentEntry - 9, // 24: atmos.plugin.v1.PrepareEnvironmentRequest.provider:type_name -> atmos.plugin.v1.ProviderRequest - 51, // 25: atmos.plugin.v1.PrepareEnvironmentRequest.environment:type_name -> atmos.plugin.v1.PrepareEnvironmentRequest.EnvironmentEntry - 21, // 26: atmos.plugin.v1.PathsResponse.paths:type_name -> atmos.plugin.v1.CredentialPath - 10, // 27: atmos.plugin.v1.IdentityAuthenticateRequest.identity:type_name -> atmos.plugin.v1.IdentityRequest - 3, // 28: atmos.plugin.v1.IdentityAuthenticateRequest.upstream:type_name -> atmos.plugin.v1.Credentials - 10, // 29: atmos.plugin.v1.IdentityPrepareRequest.identity:type_name -> atmos.plugin.v1.IdentityRequest - 52, // 30: atmos.plugin.v1.IdentityPrepareRequest.environment:type_name -> atmos.plugin.v1.IdentityPrepareRequest.EnvironmentEntry - 2, // 31: atmos.plugin.v1.CommandRequest.invocation:type_name -> atmos.plugin.v1.Invocation - 56, // 32: atmos.plugin.v1.CommandRequest.settings:type_name -> google.protobuf.Struct - 26, // 33: atmos.plugin.v1.CommandEvent.exit:type_name -> atmos.plugin.v1.CommandExit - 29, // 34: atmos.plugin.v1.ComponentValue.sensitive_paths:type_name -> atmos.plugin.v1.ComponentValuePath - 30, // 35: atmos.plugin.v1.ComponentValuePath.steps:type_name -> atmos.plugin.v1.ComponentValueStep - 28, // 36: atmos.plugin.v1.ComponentValueStep.index:type_name -> atmos.plugin.v1.ComponentValue - 0, // 37: atmos.plugin.v1.ComponentSource.kind:type_name -> atmos.plugin.v1.ComponentSource.Kind - 31, // 38: atmos.plugin.v1.ComponentDependency.target:type_name -> atmos.plugin.v1.ComponentAddress - 1, // 39: atmos.plugin.v1.ComponentDependency.kind:type_name -> atmos.plugin.v1.ComponentDependency.Kind - 31, // 40: atmos.plugin.v1.DiscoveredComponent.address:type_name -> atmos.plugin.v1.ComponentAddress - 28, // 41: atmos.plugin.v1.DiscoveredComponent.configuration:type_name -> atmos.plugin.v1.ComponentValue - 32, // 42: atmos.plugin.v1.DiscoveredComponent.sources:type_name -> atmos.plugin.v1.ComponentSource - 33, // 43: atmos.plugin.v1.DiscoveredComponent.dependencies:type_name -> atmos.plugin.v1.ComponentDependency - 53, // 44: atmos.plugin.v1.ComponentOperation.options:type_name -> atmos.plugin.v1.ComponentOperation.OptionsEntry - 2, // 45: atmos.plugin.v1.DiscoverComponentsRequest.invocation:type_name -> atmos.plugin.v1.Invocation - 36, // 46: atmos.plugin.v1.DiscoverComponentsRequest.operation:type_name -> atmos.plugin.v1.ComponentOperation - 35, // 47: atmos.plugin.v1.DiscoverComponentsRequest.effects:type_name -> atmos.plugin.v1.ComponentEffects - 28, // 48: atmos.plugin.v1.DiscoverComponentsRequest.provider_configuration:type_name -> atmos.plugin.v1.ComponentValue - 34, // 49: atmos.plugin.v1.DiscoverComponentsResponse.components:type_name -> atmos.plugin.v1.DiscoveredComponent - 14, // 50: atmos.plugin.v1.DiscoverComponentsResponse.diagnostics:type_name -> atmos.plugin.v1.Diagnostic - 2, // 51: atmos.plugin.v1.ComponentRequest.invocation:type_name -> atmos.plugin.v1.Invocation - 31, // 52: atmos.plugin.v1.ComponentRequest.address:type_name -> atmos.plugin.v1.ComponentAddress - 36, // 53: atmos.plugin.v1.ComponentRequest.operation:type_name -> atmos.plugin.v1.ComponentOperation - 35, // 54: atmos.plugin.v1.ComponentRequest.effects:type_name -> atmos.plugin.v1.ComponentEffects - 28, // 55: atmos.plugin.v1.ComponentRequest.configuration:type_name -> atmos.plugin.v1.ComponentValue - 28, // 56: atmos.plugin.v1.ComponentRequest.provider_configuration:type_name -> atmos.plugin.v1.ComponentValue - 28, // 57: atmos.plugin.v1.ComponentStateIdentity.backend:type_name -> atmos.plugin.v1.ComponentValue - 43, // 58: atmos.plugin.v1.ComponentExecutionConfiguration.extra_arguments:type_name -> atmos.plugin.v1.ComponentExtraArguments - 44, // 59: atmos.plugin.v1.ComponentExecutionConfiguration.before_hooks:type_name -> atmos.plugin.v1.ComponentHook - 44, // 60: atmos.plugin.v1.ComponentExecutionConfiguration.after_hooks:type_name -> atmos.plugin.v1.ComponentHook - 44, // 61: atmos.plugin.v1.ComponentExecutionConfiguration.error_hooks:type_name -> atmos.plugin.v1.ComponentHook - 42, // 62: atmos.plugin.v1.ComponentExecutionConfiguration.apply:type_name -> atmos.plugin.v1.ComponentExecutionConfiguration - 54, // 63: atmos.plugin.v1.ComponentExtraArguments.environment:type_name -> atmos.plugin.v1.ComponentExtraArguments.EnvironmentEntry - 28, // 64: atmos.plugin.v1.PrepareComponentResponse.inputs:type_name -> atmos.plugin.v1.ComponentValue - 40, // 65: atmos.plugin.v1.PrepareComponentResponse.state:type_name -> atmos.plugin.v1.ComponentStateIdentity - 41, // 66: atmos.plugin.v1.PrepareComponentResponse.files:type_name -> atmos.plugin.v1.ComponentGeneratedFile - 33, // 67: atmos.plugin.v1.PrepareComponentResponse.dependencies:type_name -> atmos.plugin.v1.ComponentDependency - 32, // 68: atmos.plugin.v1.PrepareComponentResponse.sources:type_name -> atmos.plugin.v1.ComponentSource - 14, // 69: atmos.plugin.v1.PrepareComponentResponse.diagnostics:type_name -> atmos.plugin.v1.Diagnostic - 42, // 70: atmos.plugin.v1.PrepareComponentResponse.execution:type_name -> atmos.plugin.v1.ComponentExecutionConfiguration - 40, // 71: atmos.plugin.v1.ResolveComponentStateResponse.state:type_name -> atmos.plugin.v1.ComponentStateIdentity - 32, // 72: atmos.plugin.v1.ResolveComponentStateResponse.sources:type_name -> atmos.plugin.v1.ComponentSource - 14, // 73: atmos.plugin.v1.ResolveComponentStateResponse.diagnostics:type_name -> atmos.plugin.v1.Diagnostic - 28, // 74: atmos.plugin.v1.ComponentOperation.OptionsEntry.value:type_name -> atmos.plugin.v1.ComponentValue - 6, // 75: atmos.plugin.v1.Info.Describe:input_type -> atmos.plugin.v1.DescribeRequest - 8, // 76: atmos.plugin.v1.Info.Configure:input_type -> atmos.plugin.v1.ConfigureRequest - 9, // 77: atmos.plugin.v1.AuthProvider.Validate:input_type -> atmos.plugin.v1.ProviderRequest - 9, // 78: atmos.plugin.v1.AuthProvider.Authenticate:input_type -> atmos.plugin.v1.ProviderRequest - 17, // 79: atmos.plugin.v1.AuthProvider.Refresh:input_type -> atmos.plugin.v1.RefreshRequest - 9, // 80: atmos.plugin.v1.AuthProvider.Logout:input_type -> atmos.plugin.v1.ProviderRequest - 9, // 81: atmos.plugin.v1.AuthProvider.Environment:input_type -> atmos.plugin.v1.ProviderRequest - 9, // 82: atmos.plugin.v1.AuthProvider.Paths:input_type -> atmos.plugin.v1.ProviderRequest - 20, // 83: atmos.plugin.v1.AuthProvider.PrepareEnvironment:input_type -> atmos.plugin.v1.PrepareEnvironmentRequest - 10, // 84: atmos.plugin.v1.AuthIdentity.Validate:input_type -> atmos.plugin.v1.IdentityRequest - 23, // 85: atmos.plugin.v1.AuthIdentity.Authenticate:input_type -> atmos.plugin.v1.IdentityAuthenticateRequest - 10, // 86: atmos.plugin.v1.AuthIdentity.Logout:input_type -> atmos.plugin.v1.IdentityRequest - 10, // 87: atmos.plugin.v1.AuthIdentity.Environment:input_type -> atmos.plugin.v1.IdentityRequest - 10, // 88: atmos.plugin.v1.AuthIdentity.Paths:input_type -> atmos.plugin.v1.IdentityRequest - 24, // 89: atmos.plugin.v1.AuthIdentity.PrepareEnvironment:input_type -> atmos.plugin.v1.IdentityPrepareRequest - 25, // 90: atmos.plugin.v1.Commands.Run:input_type -> atmos.plugin.v1.CommandRequest - 37, // 91: atmos.plugin.v1.ComponentDiscovery.Discover:input_type -> atmos.plugin.v1.DiscoverComponentsRequest - 39, // 92: atmos.plugin.v1.ComponentValidation.Validate:input_type -> atmos.plugin.v1.ComponentRequest - 39, // 93: atmos.plugin.v1.ComponentPreparation.Prepare:input_type -> atmos.plugin.v1.ComponentRequest - 39, // 94: atmos.plugin.v1.ComponentState.Resolve:input_type -> atmos.plugin.v1.ComponentRequest - 7, // 95: atmos.plugin.v1.Info.Describe:output_type -> atmos.plugin.v1.DescribeResponse - 57, // 96: atmos.plugin.v1.Info.Configure:output_type -> google.protobuf.Empty - 15, // 97: atmos.plugin.v1.AuthProvider.Validate:output_type -> atmos.plugin.v1.ValidateResponse - 13, // 98: atmos.plugin.v1.AuthProvider.Authenticate:output_type -> atmos.plugin.v1.AuthenticateEvent - 18, // 99: atmos.plugin.v1.AuthProvider.Refresh:output_type -> atmos.plugin.v1.RefreshResponse - 57, // 100: atmos.plugin.v1.AuthProvider.Logout:output_type -> google.protobuf.Empty - 19, // 101: atmos.plugin.v1.AuthProvider.Environment:output_type -> atmos.plugin.v1.EnvironmentResponse - 22, // 102: atmos.plugin.v1.AuthProvider.Paths:output_type -> atmos.plugin.v1.PathsResponse - 19, // 103: atmos.plugin.v1.AuthProvider.PrepareEnvironment:output_type -> atmos.plugin.v1.EnvironmentResponse - 15, // 104: atmos.plugin.v1.AuthIdentity.Validate:output_type -> atmos.plugin.v1.ValidateResponse - 13, // 105: atmos.plugin.v1.AuthIdentity.Authenticate:output_type -> atmos.plugin.v1.AuthenticateEvent - 57, // 106: atmos.plugin.v1.AuthIdentity.Logout:output_type -> google.protobuf.Empty - 19, // 107: atmos.plugin.v1.AuthIdentity.Environment:output_type -> atmos.plugin.v1.EnvironmentResponse - 22, // 108: atmos.plugin.v1.AuthIdentity.Paths:output_type -> atmos.plugin.v1.PathsResponse - 19, // 109: atmos.plugin.v1.AuthIdentity.PrepareEnvironment:output_type -> atmos.plugin.v1.EnvironmentResponse - 27, // 110: atmos.plugin.v1.Commands.Run:output_type -> atmos.plugin.v1.CommandEvent - 38, // 111: atmos.plugin.v1.ComponentDiscovery.Discover:output_type -> atmos.plugin.v1.DiscoverComponentsResponse - 15, // 112: atmos.plugin.v1.ComponentValidation.Validate:output_type -> atmos.plugin.v1.ValidateResponse - 45, // 113: atmos.plugin.v1.ComponentPreparation.Prepare:output_type -> atmos.plugin.v1.PrepareComponentResponse - 46, // 114: atmos.plugin.v1.ComponentState.Resolve:output_type -> atmos.plugin.v1.ResolveComponentStateResponse - 95, // [95:115] is the sub-list for method output_type - 75, // [75:95] is the sub-list for method input_type - 75, // [75:75] is the sub-list for extension type_name - 75, // [75:75] is the sub-list for extension extendee - 0, // [0:75] is the sub-list for field type_name + 66, // 0: atmos.plugin.v1.Invocation.deadline:type_name -> google.protobuf.Timestamp + 55, // 1: atmos.plugin.v1.Invocation.environment:type_name -> atmos.plugin.v1.Invocation.EnvironmentEntry + 66, // 2: atmos.plugin.v1.Credentials.expires_at:type_name -> google.protobuf.Timestamp + 67, // 3: atmos.plugin.v1.Credentials.data:type_name -> google.protobuf.Struct + 5, // 4: atmos.plugin.v1.DescribeRequest.host_supported:type_name -> atmos.plugin.v1.ServiceVersionRange + 5, // 5: atmos.plugin.v1.DescribeResponse.services:type_name -> atmos.plugin.v1.ServiceVersionRange + 6, // 6: atmos.plugin.v1.DescribeResponse.exports:type_name -> atmos.plugin.v1.Export + 56, // 7: atmos.plugin.v1.ConfigureRequest.broker_ids:type_name -> atmos.plugin.v1.ConfigureRequest.BrokerIdsEntry + 67, // 8: atmos.plugin.v1.ConfigureRequest.settings:type_name -> google.protobuf.Struct + 3, // 9: atmos.plugin.v1.ProviderRequest.invocation:type_name -> atmos.plugin.v1.Invocation + 67, // 10: atmos.plugin.v1.ProviderRequest.spec:type_name -> google.protobuf.Struct + 3, // 11: atmos.plugin.v1.IdentityRequest.invocation:type_name -> atmos.plugin.v1.Invocation + 67, // 12: atmos.plugin.v1.IdentityRequest.spec:type_name -> google.protobuf.Struct + 67, // 13: atmos.plugin.v1.IdentityRequest.principal:type_name -> google.protobuf.Struct + 57, // 14: atmos.plugin.v1.Interaction.fields:type_name -> atmos.plugin.v1.Interaction.FieldsEntry + 12, // 15: atmos.plugin.v1.AuthenticateEvent.progress:type_name -> atmos.plugin.v1.Progress + 13, // 16: atmos.plugin.v1.AuthenticateEvent.interaction:type_name -> atmos.plugin.v1.Interaction + 4, // 17: atmos.plugin.v1.AuthenticateEvent.result:type_name -> atmos.plugin.v1.Credentials + 15, // 18: atmos.plugin.v1.ErrorDetail.location:type_name -> atmos.plugin.v1.SourceLocation + 0, // 19: atmos.plugin.v1.ErrorDetail.severity:type_name -> atmos.plugin.v1.Severity + 16, // 20: atmos.plugin.v1.Diagnostic.detail:type_name -> atmos.plugin.v1.ErrorDetail + 17, // 21: atmos.plugin.v1.ValidateResponse.diagnostics:type_name -> atmos.plugin.v1.Diagnostic + 10, // 22: atmos.plugin.v1.RefreshRequest.provider:type_name -> atmos.plugin.v1.ProviderRequest + 4, // 23: atmos.plugin.v1.RefreshRequest.current:type_name -> atmos.plugin.v1.Credentials + 4, // 24: atmos.plugin.v1.RefreshResponse.credentials:type_name -> atmos.plugin.v1.Credentials + 19, // 25: atmos.plugin.v1.RefreshResponse.unsupported:type_name -> atmos.plugin.v1.Unsupported + 58, // 26: atmos.plugin.v1.EnvironmentResponse.environment:type_name -> atmos.plugin.v1.EnvironmentResponse.EnvironmentEntry + 10, // 27: atmos.plugin.v1.PrepareEnvironmentRequest.provider:type_name -> atmos.plugin.v1.ProviderRequest + 59, // 28: atmos.plugin.v1.PrepareEnvironmentRequest.environment:type_name -> atmos.plugin.v1.PrepareEnvironmentRequest.EnvironmentEntry + 24, // 29: atmos.plugin.v1.PathsResponse.paths:type_name -> atmos.plugin.v1.CredentialPath + 11, // 30: atmos.plugin.v1.IdentityAuthenticateRequest.identity:type_name -> atmos.plugin.v1.IdentityRequest + 4, // 31: atmos.plugin.v1.IdentityAuthenticateRequest.upstream:type_name -> atmos.plugin.v1.Credentials + 11, // 32: atmos.plugin.v1.IdentityPrepareRequest.identity:type_name -> atmos.plugin.v1.IdentityRequest + 60, // 33: atmos.plugin.v1.IdentityPrepareRequest.environment:type_name -> atmos.plugin.v1.IdentityPrepareRequest.EnvironmentEntry + 3, // 34: atmos.plugin.v1.CommandRequest.invocation:type_name -> atmos.plugin.v1.Invocation + 67, // 35: atmos.plugin.v1.CommandRequest.settings:type_name -> google.protobuf.Struct + 29, // 36: atmos.plugin.v1.CommandEvent.exit:type_name -> atmos.plugin.v1.CommandExit + 32, // 37: atmos.plugin.v1.ComponentValue.sensitive_paths:type_name -> atmos.plugin.v1.ComponentValuePath + 33, // 38: atmos.plugin.v1.ComponentValuePath.steps:type_name -> atmos.plugin.v1.ComponentValueStep + 31, // 39: atmos.plugin.v1.ComponentValueStep.index:type_name -> atmos.plugin.v1.ComponentValue + 1, // 40: atmos.plugin.v1.ComponentSource.kind:type_name -> atmos.plugin.v1.ComponentSource.Kind + 34, // 41: atmos.plugin.v1.ComponentDependency.target:type_name -> atmos.plugin.v1.ComponentAddress + 2, // 42: atmos.plugin.v1.ComponentDependency.kind:type_name -> atmos.plugin.v1.ComponentDependency.Kind + 34, // 43: atmos.plugin.v1.DiscoveredComponent.address:type_name -> atmos.plugin.v1.ComponentAddress + 31, // 44: atmos.plugin.v1.DiscoveredComponent.configuration:type_name -> atmos.plugin.v1.ComponentValue + 35, // 45: atmos.plugin.v1.DiscoveredComponent.sources:type_name -> atmos.plugin.v1.ComponentSource + 36, // 46: atmos.plugin.v1.DiscoveredComponent.dependencies:type_name -> atmos.plugin.v1.ComponentDependency + 61, // 47: atmos.plugin.v1.ComponentOperation.options:type_name -> atmos.plugin.v1.ComponentOperation.OptionsEntry + 3, // 48: atmos.plugin.v1.DiscoverComponentsRequest.invocation:type_name -> atmos.plugin.v1.Invocation + 39, // 49: atmos.plugin.v1.DiscoverComponentsRequest.operation:type_name -> atmos.plugin.v1.ComponentOperation + 38, // 50: atmos.plugin.v1.DiscoverComponentsRequest.effects:type_name -> atmos.plugin.v1.ComponentEffects + 31, // 51: atmos.plugin.v1.DiscoverComponentsRequest.provider_configuration:type_name -> atmos.plugin.v1.ComponentValue + 37, // 52: atmos.plugin.v1.DiscoverComponentsResponse.components:type_name -> atmos.plugin.v1.DiscoveredComponent + 17, // 53: atmos.plugin.v1.DiscoverComponentsResponse.diagnostics:type_name -> atmos.plugin.v1.Diagnostic + 3, // 54: atmos.plugin.v1.ComponentRequest.invocation:type_name -> atmos.plugin.v1.Invocation + 34, // 55: atmos.plugin.v1.ComponentRequest.address:type_name -> atmos.plugin.v1.ComponentAddress + 39, // 56: atmos.plugin.v1.ComponentRequest.operation:type_name -> atmos.plugin.v1.ComponentOperation + 38, // 57: atmos.plugin.v1.ComponentRequest.effects:type_name -> atmos.plugin.v1.ComponentEffects + 31, // 58: atmos.plugin.v1.ComponentRequest.configuration:type_name -> atmos.plugin.v1.ComponentValue + 31, // 59: atmos.plugin.v1.ComponentRequest.provider_configuration:type_name -> atmos.plugin.v1.ComponentValue + 31, // 60: atmos.plugin.v1.ComponentStateIdentity.backend:type_name -> atmos.plugin.v1.ComponentValue + 46, // 61: atmos.plugin.v1.ComponentExecutionConfiguration.extra_arguments:type_name -> atmos.plugin.v1.ComponentExtraArguments + 47, // 62: atmos.plugin.v1.ComponentExecutionConfiguration.before_hooks:type_name -> atmos.plugin.v1.ComponentHook + 47, // 63: atmos.plugin.v1.ComponentExecutionConfiguration.after_hooks:type_name -> atmos.plugin.v1.ComponentHook + 47, // 64: atmos.plugin.v1.ComponentExecutionConfiguration.error_hooks:type_name -> atmos.plugin.v1.ComponentHook + 45, // 65: atmos.plugin.v1.ComponentExecutionConfiguration.apply:type_name -> atmos.plugin.v1.ComponentExecutionConfiguration + 62, // 66: atmos.plugin.v1.ComponentExtraArguments.environment:type_name -> atmos.plugin.v1.ComponentExtraArguments.EnvironmentEntry + 31, // 67: atmos.plugin.v1.PrepareComponentResponse.inputs:type_name -> atmos.plugin.v1.ComponentValue + 43, // 68: atmos.plugin.v1.PrepareComponentResponse.state:type_name -> atmos.plugin.v1.ComponentStateIdentity + 44, // 69: atmos.plugin.v1.PrepareComponentResponse.files:type_name -> atmos.plugin.v1.ComponentGeneratedFile + 36, // 70: atmos.plugin.v1.PrepareComponentResponse.dependencies:type_name -> atmos.plugin.v1.ComponentDependency + 35, // 71: atmos.plugin.v1.PrepareComponentResponse.sources:type_name -> atmos.plugin.v1.ComponentSource + 17, // 72: atmos.plugin.v1.PrepareComponentResponse.diagnostics:type_name -> atmos.plugin.v1.Diagnostic + 45, // 73: atmos.plugin.v1.PrepareComponentResponse.execution:type_name -> atmos.plugin.v1.ComponentExecutionConfiguration + 63, // 74: atmos.plugin.v1.PrepareComponentResponse.input_origins:type_name -> atmos.plugin.v1.PrepareComponentResponse.InputOriginsEntry + 49, // 75: atmos.plugin.v1.PrepareComponentResponse.errors:type_name -> atmos.plugin.v1.ComponentErrorsConfiguration + 52, // 76: atmos.plugin.v1.PrepareComponentResponse.exclude:type_name -> atmos.plugin.v1.ComponentExclude + 53, // 77: atmos.plugin.v1.PrepareComponentResponse.iam:type_name -> atmos.plugin.v1.ComponentIam + 31, // 78: atmos.plugin.v1.PrepareComponentResponse.encryption:type_name -> atmos.plugin.v1.ComponentValue + 64, // 79: atmos.plugin.v1.PrepareComponentResponse.feature_flags:type_name -> atmos.plugin.v1.PrepareComponentResponse.FeatureFlagsEntry + 50, // 80: atmos.plugin.v1.ComponentErrorsConfiguration.retry:type_name -> atmos.plugin.v1.ComponentRetryPolicy + 51, // 81: atmos.plugin.v1.ComponentErrorsConfiguration.ignore:type_name -> atmos.plugin.v1.ComponentIgnorePolicy + 65, // 82: atmos.plugin.v1.ComponentIgnorePolicy.signals:type_name -> atmos.plugin.v1.ComponentIgnorePolicy.SignalsEntry + 31, // 83: atmos.plugin.v1.ComponentIam.web_identity_token:type_name -> atmos.plugin.v1.ComponentValue + 43, // 84: atmos.plugin.v1.ResolveComponentStateResponse.state:type_name -> atmos.plugin.v1.ComponentStateIdentity + 35, // 85: atmos.plugin.v1.ResolveComponentStateResponse.sources:type_name -> atmos.plugin.v1.ComponentSource + 17, // 86: atmos.plugin.v1.ResolveComponentStateResponse.diagnostics:type_name -> atmos.plugin.v1.Diagnostic + 31, // 87: atmos.plugin.v1.ComponentOperation.OptionsEntry.value:type_name -> atmos.plugin.v1.ComponentValue + 15, // 88: atmos.plugin.v1.PrepareComponentResponse.InputOriginsEntry.value:type_name -> atmos.plugin.v1.SourceLocation + 31, // 89: atmos.plugin.v1.ComponentIgnorePolicy.SignalsEntry.value:type_name -> atmos.plugin.v1.ComponentValue + 7, // 90: atmos.plugin.v1.Info.Describe:input_type -> atmos.plugin.v1.DescribeRequest + 9, // 91: atmos.plugin.v1.Info.Configure:input_type -> atmos.plugin.v1.ConfigureRequest + 10, // 92: atmos.plugin.v1.AuthProvider.Validate:input_type -> atmos.plugin.v1.ProviderRequest + 10, // 93: atmos.plugin.v1.AuthProvider.Authenticate:input_type -> atmos.plugin.v1.ProviderRequest + 20, // 94: atmos.plugin.v1.AuthProvider.Refresh:input_type -> atmos.plugin.v1.RefreshRequest + 10, // 95: atmos.plugin.v1.AuthProvider.Logout:input_type -> atmos.plugin.v1.ProviderRequest + 10, // 96: atmos.plugin.v1.AuthProvider.Environment:input_type -> atmos.plugin.v1.ProviderRequest + 10, // 97: atmos.plugin.v1.AuthProvider.Paths:input_type -> atmos.plugin.v1.ProviderRequest + 23, // 98: atmos.plugin.v1.AuthProvider.PrepareEnvironment:input_type -> atmos.plugin.v1.PrepareEnvironmentRequest + 11, // 99: atmos.plugin.v1.AuthIdentity.Validate:input_type -> atmos.plugin.v1.IdentityRequest + 26, // 100: atmos.plugin.v1.AuthIdentity.Authenticate:input_type -> atmos.plugin.v1.IdentityAuthenticateRequest + 11, // 101: atmos.plugin.v1.AuthIdentity.Logout:input_type -> atmos.plugin.v1.IdentityRequest + 11, // 102: atmos.plugin.v1.AuthIdentity.Environment:input_type -> atmos.plugin.v1.IdentityRequest + 11, // 103: atmos.plugin.v1.AuthIdentity.Paths:input_type -> atmos.plugin.v1.IdentityRequest + 27, // 104: atmos.plugin.v1.AuthIdentity.PrepareEnvironment:input_type -> atmos.plugin.v1.IdentityPrepareRequest + 28, // 105: atmos.plugin.v1.Commands.Run:input_type -> atmos.plugin.v1.CommandRequest + 40, // 106: atmos.plugin.v1.ComponentDiscovery.Discover:input_type -> atmos.plugin.v1.DiscoverComponentsRequest + 42, // 107: atmos.plugin.v1.ComponentValidation.Validate:input_type -> atmos.plugin.v1.ComponentRequest + 42, // 108: atmos.plugin.v1.ComponentPreparation.Prepare:input_type -> atmos.plugin.v1.ComponentRequest + 42, // 109: atmos.plugin.v1.ComponentState.Resolve:input_type -> atmos.plugin.v1.ComponentRequest + 8, // 110: atmos.plugin.v1.Info.Describe:output_type -> atmos.plugin.v1.DescribeResponse + 68, // 111: atmos.plugin.v1.Info.Configure:output_type -> google.protobuf.Empty + 18, // 112: atmos.plugin.v1.AuthProvider.Validate:output_type -> atmos.plugin.v1.ValidateResponse + 14, // 113: atmos.plugin.v1.AuthProvider.Authenticate:output_type -> atmos.plugin.v1.AuthenticateEvent + 21, // 114: atmos.plugin.v1.AuthProvider.Refresh:output_type -> atmos.plugin.v1.RefreshResponse + 68, // 115: atmos.plugin.v1.AuthProvider.Logout:output_type -> google.protobuf.Empty + 22, // 116: atmos.plugin.v1.AuthProvider.Environment:output_type -> atmos.plugin.v1.EnvironmentResponse + 25, // 117: atmos.plugin.v1.AuthProvider.Paths:output_type -> atmos.plugin.v1.PathsResponse + 22, // 118: atmos.plugin.v1.AuthProvider.PrepareEnvironment:output_type -> atmos.plugin.v1.EnvironmentResponse + 18, // 119: atmos.plugin.v1.AuthIdentity.Validate:output_type -> atmos.plugin.v1.ValidateResponse + 14, // 120: atmos.plugin.v1.AuthIdentity.Authenticate:output_type -> atmos.plugin.v1.AuthenticateEvent + 68, // 121: atmos.plugin.v1.AuthIdentity.Logout:output_type -> google.protobuf.Empty + 22, // 122: atmos.plugin.v1.AuthIdentity.Environment:output_type -> atmos.plugin.v1.EnvironmentResponse + 25, // 123: atmos.plugin.v1.AuthIdentity.Paths:output_type -> atmos.plugin.v1.PathsResponse + 22, // 124: atmos.plugin.v1.AuthIdentity.PrepareEnvironment:output_type -> atmos.plugin.v1.EnvironmentResponse + 30, // 125: atmos.plugin.v1.Commands.Run:output_type -> atmos.plugin.v1.CommandEvent + 41, // 126: atmos.plugin.v1.ComponentDiscovery.Discover:output_type -> atmos.plugin.v1.DiscoverComponentsResponse + 18, // 127: atmos.plugin.v1.ComponentValidation.Validate:output_type -> atmos.plugin.v1.ValidateResponse + 48, // 128: atmos.plugin.v1.ComponentPreparation.Prepare:output_type -> atmos.plugin.v1.PrepareComponentResponse + 54, // 129: atmos.plugin.v1.ComponentState.Resolve:output_type -> atmos.plugin.v1.ResolveComponentStateResponse + 110, // [110:130] is the sub-list for method output_type + 90, // [90:110] is the sub-list for method input_type + 90, // [90:90] is the sub-list for extension type_name + 90, // [90:90] is the sub-list for extension extendee + 0, // [0:90] is the sub-list for field type_name } func init() { file_atmos_plugin_v1_plugin_proto_init() } @@ -3783,16 +4551,16 @@ func file_atmos_plugin_v1_plugin_proto_init() { (*AuthenticateEvent_Interaction)(nil), (*AuthenticateEvent_Result)(nil), } - file_atmos_plugin_v1_plugin_proto_msgTypes[16].OneofWrappers = []any{ + file_atmos_plugin_v1_plugin_proto_msgTypes[18].OneofWrappers = []any{ (*RefreshResponse_Credentials)(nil), (*RefreshResponse_Unsupported)(nil), } - file_atmos_plugin_v1_plugin_proto_msgTypes[25].OneofWrappers = []any{ + file_atmos_plugin_v1_plugin_proto_msgTypes[27].OneofWrappers = []any{ (*CommandEvent_Stdout)(nil), (*CommandEvent_Stderr)(nil), (*CommandEvent_Exit)(nil), } - file_atmos_plugin_v1_plugin_proto_msgTypes[28].OneofWrappers = []any{ + file_atmos_plugin_v1_plugin_proto_msgTypes[30].OneofWrappers = []any{ (*ComponentValueStep_Attribute)(nil), (*ComponentValueStep_Index)(nil), } @@ -3801,8 +4569,8 @@ func file_atmos_plugin_v1_plugin_proto_init() { File: protoimpl.DescBuilder{ GoPackagePath: reflect.TypeOf(x{}).PkgPath(), RawDescriptor: unsafe.Slice(unsafe.StringData(file_atmos_plugin_v1_plugin_proto_rawDesc), len(file_atmos_plugin_v1_plugin_proto_rawDesc)), - NumEnums: 2, - NumMessages: 53, + NumEnums: 3, + NumMessages: 63, NumExtensions: 0, NumServices: 8, }, diff --git a/website/blog/2026-10-29-native-terragrunt-compatibility.mdx b/website/blog/2026-10-29-native-terragrunt-compatibility.mdx index 514727c715a..1d3240d8054 100644 --- a/website/blog/2026-10-29-native-terragrunt-compatibility.mdx +++ b/website/blog/2026-10-29-native-terragrunt-compatibility.mdx @@ -30,8 +30,10 @@ go build -o build/atmos . ./build/atmos plugin build plugins/terragrunt export PATH="$PWD/build:$PATH" export ATMOS_TERRAGRUNT_PLUGIN_PATH="$PWD/plugins/terragrunt" -atmos init ./examples/terragrunt-native ./terragrunt-native --copy --no-git +atmos init ./examples/terragrunt-native ./terragrunt-native \ + --copy --no-git cd terragrunt-native +export ATMOS_PROFILE=dev ``` Install OpenTofu before planning. The recording starts after the host and plugin have been built and the package path configured. @@ -39,10 +41,10 @@ Install OpenTofu before planning. The recording starts after the host and plugin -Set the package path to your built plugin, then inspect and plan the discovered unit: +The example's `dev` profile reads the package path from `ATMOS_TERRAGRUNT_PLUGIN_PATH`. Select it, then inspect and plan the discovered unit: ```shell -atmos config set plugins.packages.terragrunt.path "$ATMOS_TERRAGRUNT_PLUGIN_PATH" +export ATMOS_PROFILE=dev atmos list stacks atmos list instances atmos terraform plan demo -s live @@ -57,17 +59,34 @@ The example already enables native discovery. It stores state locally if you cho From your project's root, register the built plugin and opt into native discovery with [config set](/cli/commands/config/config-set): ```shell -atmos config set plugins.packages.terragrunt.path "$ATMOS_TERRAGRUNT_PLUGIN_PATH" +atmos config set plugins.packages.terragrunt.path \ + "$ATMOS_TERRAGRUNT_PLUGIN_PATH" +atmos config set components.terraform.base_path \ + components/terraform atmos config set components.terraform.command tofu atmos config set components.terraform.terragrunt.enabled true atmos config set components.terraform.terragrunt.mode native -atmos config set --type=yaml components.terraform.terragrunt.discovery.included_paths '["live/**"]' -atmos config set components.terraform.terragrunt.discovery.enabled true +atmos config set --type=yaml \ + components.terraform.terragrunt.discovery.included_paths \ + '["live/**"]' +atmos config set \ + components.terraform.terragrunt.discovery.enabled true atmos list stacks atmos list instances ``` -Adjust `live/**` to your Terragrunt directory layout. Keep your existing Terraform command setting if you use Terraform instead of OpenTofu. Discovery does not require per-unit Atmos YAML. Use the component and stack names from the inventory when running `atmos terraform plan -s `. +Adjust `live/**` to your Terragrunt directory layout. Atmos requires `components.terraform.base_path` even though Terragrunt units do not live there, and the directory does not need to exist. Keep your existing Terraform command setting if you use Terraform instead of OpenTofu. Discovery does not require per-unit Atmos YAML. Use the component and stack names from the inventory when running `atmos terraform plan -s `. + +## Put units in the stacks you already have + +Discovery names a unit's stack after the directory above it. To make a unit a component of one of your own stacks instead, import it from that stack's file: + +```yaml +import: + - terragrunt://live/dev +``` + +The units below `live/dev` become components of the importing stack, and no extra stack appears. Import the same unit from a shared mixin and every stack that uses the mixin gets it. If an imported unit has the same component name as a YAML component in that stack, Atmos names both and stops. Discovery paths, patterns, and symlink rules are described in the [migration guide](/migration/terragrunt#discovery-paths). ## Inspect the executor diff --git a/website/docs/cli/commands/describe/describe-affected.mdx b/website/docs/cli/commands/describe/describe-affected.mdx index 3394d9c30ed..08b6fe76346 100644 --- a/website/docs/cli/commands/describe/describe-affected.mdx +++ b/website/docs/cli/commands/describe/describe-affected.mdx @@ -88,7 +88,7 @@ The command performs the following: - The command compares each stack manifest section of the stack configurations from both sources looking for differences - And finally, the command outputs a JSON or YAML document consisting of a list of the affected components and stacks - and what caused it to be affected + and what caused it to be affected. Results are sorted by stack. Within a stack, components of each type are sorted by name. Since Atmos first checks the component folders for changes, if it finds any affected files, it will mark all related components and stacks as affected. Atmos will then skip evaluating the stacks for differences since it already diff --git a/website/docs/migration/terragrunt.mdx b/website/docs/migration/terragrunt.mdx index 9a6ec552bd1..2b387288127 100644 --- a/website/docs/migration/terragrunt.mdx +++ b/website/docs/migration/terragrunt.mdx @@ -916,34 +916,145 @@ atmos terraform apply --all atmos describe affected --base origin/main ``` -To add a unit to the inventory from a stack file instead of discovering it automatically, import it: +Units found this way get their names from where they live on disk, so no stack YAML is needed. To place a unit in a stack you already have, import it instead (see [Import units into a stack](#import-units-into-a-stack)). +### Component and stack names + +Discovery derives names from paths relative to the Atmos base path. Stack name templates and patterns (`name_template`, `name_pattern`) do not apply to discovered units, and `atmos list instances` is the authoritative list of names. + +| Unit | Component | Stack | +| --- | --- | --- | +| `live/dev/vpc/terragrunt.hcl` | `vpc` (the unit directory name) | `live/dev` (the directory above the unit) | +| `live/vpc/terragrunt.hcl` | `vpc` | `live` | +| `live/terragrunt.hcl` | `live` | `default` | +| `terragrunt.hcl` in the base path | `root` | `default` | +| `live/terragrunt.stack.hcl` generating `dev/app` | `dev/app` (the unit's path inside the stack) | `live` (the directory of the stack file) | + +Two units with the same name in the same stack are an error that names both files. Directory names with spaces, dots, or Unicode work as they are. + +A unit that is imported into a stack keeps its component name, but its stack is the stack that imports it. A YAML component with the same name is not a second definition: it is [an override of the unit](#override-a-unit-from-stack-configuration). + +### Import units into a stack + +Import a unit, a directory of units, or a `terragrunt.stack.hcl` to make the units components of the stack that imports them, next to the components defined in that stack's YAML: + + ```yaml import: - - terragrunt://live/dev/vpc/terragrunt.hcl + - terragrunt://live/dev +vars: + stage: dev ``` - -The import adds the unit to the shared inventory. It does not move the unit into the importing stack: -the unit keeps the coordinates discovery gives it, its directory (`live/dev`) as the stack and its -unit name (`vpc`) as the component. Select it with those coordinates, whatever the importing file is named: + ```shell -atmos terraform plan vpc -s live/dev +atmos list instances --stack dev +atmos terraform plan vpc -s dev ``` +The path after `terragrunt://` is relative to the Atmos base path, not to the stack file. It must be a project-relative file named `terragrunt.hcl` or `terragrunt.stack.hcl`, or a directory. Globs, `..`, absolute paths, `?ref=`, and `#` fragments are rejected. A path may not contain any of the characters `\`, `?`, `#`, `*`, `[`, or `{`, even in a literal directory name, so rename such a directory or import its parent. The `context`, `skip_if_missing`, `ttl`, and `nested_imports` options are not supported for these imports. + +- **An import never creates a stack.** The units become components of the importing stack. Only units found by automatic discovery and not imported anywhere get a stack name from their directory. A unit that is imported is not also listed under its discovered stack. +- **Mixins and catalogs work like YAML imports.** An import inside a mixin applies to every stack that imports the mixin, and the same unit can be imported by several stacks, each getting its own component. All of them share the unit's backend configuration, and therefore its state. Import a unit into more than one stack only when sharing is intended, or vary the backend key per stack in HCL, for example with `get_env()`. +- **Imports ignore `discovery.included_paths` and `discovery.excluded_paths`.** An import is an explicit request, so a unit excluded from automatic discovery can still be imported. Discovery does not need to be enabled for imports to work. +- **A unit name is unique within a stack.** If an import brings in two different units that share a component name in the same stack, Atmos fails and names both files. A YAML component with the name of an imported unit is not a conflict; it deep-merges over the unit (see [Override a unit from stack configuration](#override-a-unit-from-stack-configuration)). +- **An import must select a unit.** An import that matches no unit with a `terraform` `source` is an error, not an empty result. +- **Dependencies stay inside the stack.** A unit that depends on another unit gets a dependency on the component of the same name in the same stack. Import the dependency into that stack too, in the same import or a separate one; a bulk run (`--all`) fails with a missing-dependency error naming both components otherwise. + +### Override a unit from stack configuration + +An imported unit is the base of a component in the importing stack. Stack configuration for the same component name deep-merges over it with the usual rules, so you can change any setting of a unit without editing its HCL. The YAML can sit in the importing manifest or in any other import, before or after the unit's import. + + +```yaml +import: + - terragrunt://live/dev +components: + terraform: + vpc: + metadata: + inherits: [defaults] + vars: + name: dev-vpc # replaces the HCL input + tags: + team: platform # merged into the HCL tags map + env: + TF_LOG: ERROR # wins over env_vars from the unit's extra_arguments + settings: + owner: platform +``` + + +Precedence, from lowest to highest: + +1. The unit's own evaluation. Inputs, `extra_arguments` environment, and hooks come from the HCL, evaluated again for every operation because inputs can depend on dependency outputs. +2. Sections of abstract components named in `metadata.inherits`. +3. Imports, in order, then the importing manifest, then the component's own block, then `overrides`. + +Per section: + +| Section | Behavior | +| --- | --- | +| `vars` | Merged over the unit's inputs when the command runs: maps merge key by key, scalars and lists are replaced (lists follow `settings.list_merge_strategy`). Set a variable to `null` to remove a unit input. `!unset` removes a value that came from stack layers (a catalog or base component), and then the unit's own input applies. Stack-wide `vars` reach the unit like they reach every component. | +| `env` | Wins over the environment of the unit's `extra_arguments`. It is also visible to `get_env()` while the HCL is evaluated. | +| `settings`, `metadata` (`enabled`, `locked`, `terraform_workspace`, `inherits`), `hooks`, `providers`, `auth`, `dependencies` | Ordinary stack configuration. `terraform_workspace` selects the workspace the unit runs in. A `dependencies.tools` entry at the stack, `terraform`, or component level pins the tool for every command, `run_cmd`, and hook of the unit. | +| `command` | Selects the executable, for example a pinned `tofu`. It must be the same engine the unit was discovered with. Switching between OpenTofu and Terraform is an error. | +| `backend`, `backend_type` | Not overridable. The unit's `remote_state` decides where its state lives, and dependency reads and saved plans rely on the same location. A component that sets `backend_type` to anything but the default, or backend options of its own, fails with an error that points to `remote_state`. A backend set in a global section is ignored for units. | +| `metadata.component`, `source.uri` | Not overridable. The module comes from the unit's `terraform { source }`. Other `source` settings such as `version` merge normally. | + +`atmos describe component` shows the stack layer: the unit's HCL inputs are evaluated only when a command runs, so they are not listed. The merged variables are written to the generated var file in the component's working directory. + +Units cannot carry `metadata.labels` or `metadata.tags` today. + +### Discovery paths + +`included_paths` and `excluded_paths` use the same pattern language. Patterns are [doublestar](https://github.com/bmatcuk/doublestar) globs relative to the Atmos base path, written with forward slashes on every platform. A pattern selects the path it matches and everything below it, so a directory name needs no `/**`. A leading `./` and a trailing `/` are ignored. Absolute patterns, `..`, and backslashes are rejected. + +| Pattern | Selects | +| --- | --- | +| `live`, `live/`, `./live`, `live/**` | every unit below `live` | +| `live/dev/*` | every unit below any direct child of `live/dev` (a unit directory, or a folder of them) | +| `live/dev/vpc` | the unit in that directory | +| `live/dev/vpc/terragrunt.hcl` | exactly that file | +| `**/vpc`, `live/*/vpc` | every unit directory named `vpc` | +| `.`, `./**`, `**` | everything (the default when `included_paths` is not set) | + +A pattern is compared in the case the filesystem reports, so `Live` and `live` can behave differently between macOS and Linux. Discovery skips hidden directories, `node_modules`, `.terragrunt-cache`, and `.terragrunt-stack`. + +Discovery reports what it cannot use instead of silently returning less: + +- An `included_paths` pattern that selects no `terragrunt.hcl` or `terragrunt.stack.hcl` produces a warning that names the pattern. +- A `terragrunt.hcl` without a `terraform { source }` block (and without Terraform files beside it) is treated as shared configuration, such as an `include` target, and a warning names it. +- A dependency on a unit outside the discovery scope fails that unit's commands and names both units. Add the dependency to `included_paths` or import it. + +### Symlinks + +Atmos follows the same rules as Terragrunt v1.2.1, whose symlink support is an opt-in experiment. Symlinks are best avoided. + +- A symlinked directory is not followed, so units below it are not discovered. A warning names each skipped link. Import the real directory instead, or list the target in `included_paths`. +- A symlinked `terragrunt.hcl` or `terragrunt.stack.hcl` is a unit at the place the link lives, named from that location, and its content is read through the link. `find_in_parent_folders()` and relative paths resolve from the link's location. +- A symlink inside a watched module or the unit's configuration that resolves outside the project root is rejected before anything runs. The error names the file and where it leads. Replace the link with a copy, or point it inside the project. + +### Provenance and inspection + +`atmos describe component` for a unit shows the unit's evaluated source and dependencies. It does not point at unrelated YAML: a discovered unit has no stack manifest, and an imported unit is attributed to the manifest that imported it. Per-input provenance is not available yet, and `vars` and `backend` show the stack's values until the unit is prepared for a command. + ### What Atmos runs for you - `dependency` and `dependencies` blocks become ordering and output edges, and dependency outputs are refreshed in the same apply. - `extra_arguments` and `before_hook`, `after_hook`, and `error_hook` blocks run with the same environment, identity, and toolchain as the Terraform command. A hook that lists `plan` does not run for `apply`. The implicit `init` matches hooks for `init`, not for the requested command. - Dry runs do not evaluate Terragrunt configuration. Their command previews may omit dynamically computed extra arguments, required var files, and environment variables. Real runs include these values. - Generated files and backend configuration keep their Terragrunt identity, so existing state stays addressable. -- `atmos terraform plan` followed by `atmos terraform apply --from-plan` applies exactly the reviewed plan. Atmos rejects the apply if the plan file, the unit's configuration, or the working directory changed after the plan. The original working directory must still exist. -- Remote Git and HTTPS module sources are fetched by Atmos. Local module sources are re-copied on every run so edits take effect. +- `atmos terraform plan` followed by `atmos terraform apply --from-plan` applies exactly the reviewed plan. Atmos rejects the apply if the plan file, the unit's configuration, or the working directory changed after the plan. The original working directory must still exist. Commands such as `output`, `validate`, and `init` leave the saved plan in place, and a successful apply removes it, so applying the same plan twice reports that no saved plan exists. +- Remote Git and HTTPS module sources are fetched by Atmos. A source that follows a branch (`?ref=main`) is fetched once and reused, with a notice; run `atmos terraform source pull -s --force` to refresh it. A local module is copied again only when its files change. +- Removing a `generate` block removes the file it generated, and a `generate` block that writes a `backend` block replaces the backend file Atmos would write. Atmos copies the `.terraform.lock.hcl` beside `terragrunt.hcl` into the working directory but does not copy changes back; copy it yourself after `init -upgrade`. +- Tool versions come from `.tool-versions` and from `dependencies.tools`, which Atmos reads from stack manifests and component sections, not `atmos.yaml`. An imported unit takes `dependencies.tools` from the importing stack's configuration, and those pins take precedence over `.tool-versions`. A discovered unit has no stack manifest, so it takes its pins from `.tool-versions`. +- A unit without a stable `remote_state` path keeps local state under `.terragrunt-cache`, which moves with the checkout path and disappears when the cache is deleted. Atmos keeps that location so existing state stays addressable, and warns when it applies. - `!terraform.output` and `!terraform.state` can read a Terragrunt unit like any other component. ### What it does not support yet -Unsupported settings fail before any infrastructure command runs. Atmos does not fall back to the Terragrunt executable. Currently rejected: +Unsupported settings fail before any infrastructure command runs, with an error that names the unit, the file with line and column, and the setting. With automatic discovery, a broken unit does not affect other units or YAML components: it stays listed, and only commands that target it fail. An explicit import is stricter: it fails when any unit it selects reports an error, so that import contributes no units until the error is fixed. `apply`, `plan`, and `destroy` with `--all` validate every selected unit first and report "no infrastructure command was run". Evaluation errors can contain secrets, so Atmos shows the location and a hint but not the evaluator's text; run with `ATMOS_LOGS_LEVEL=Debug` to log the full message with secrets masked. Atmos does not fall back to the Terragrunt executable. Currently rejected: - `terraform_binary`, `errors`, `exclude`, feature flags, and the `iam_role` and web-identity settings. - `remote_state` with `disable_init` or encryption. Atmos does not create backend resources.