Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .github/workflows/terraform-test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -95,5 +95,8 @@ jobs:
- name: Validate Terraform root
run: terraform -chdir="${{ matrix.working_directory }}" validate

- name: Run Terraform native tests
run: terraform -chdir="${{ matrix.working_directory }}" test

- name: Lint Terraform root
run: tflint --chdir="${{ matrix.working_directory }}"
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ The MVP application source is present: a FastAPI daily-entry API and a warm Next
- [Backend MVP](backend/README.md): email/password MVP auth, user-scoped daily-entry API, operational endpoints, PostgreSQL persistence, Alembic migration, tests, linting, and local configuration example.
- [Frontend MVP](frontend/README.md): sign-in/sign-up UI, three-small-happy-things bedtime UI, private history calendar, backend API integration, operational endpoints, tests, linting, and local configuration example.
- [Terraform foundation](infra/terraform/README.md): version constraints, remote state, and applied sandbox network, RDS data, ECR/ECS platform, edge, and service roots.
- [Terraform workflow controls](.github/workflows/terraform-plan.yml): backend-free PR validation plus manual target-selected plan, apply, and destroy controls.
- [Terraform workflow controls](.github/workflows/terraform-test.yml): backend-free PR format, validate, native test, and TFLint checks plus manual target-selected plan, apply, and destroy controls.
- [Application CI](.github/workflows/application-ci.yml): changed-component backend and frontend linting, unit tests, and frontend build checks.
- [Image publication](.github/workflows/image-publish.yml): changed-component `main` publication plus a main-only manual initial publication, Trivy gate, immutable ECR publication, and digest handoff artifact.
- [Service bootstrap](.github/workflows/service-bootstrap.yml): manually creates one selected initial service only after repository-specific digest and immutable source-commit-tag verification.
Expand Down
4 changes: 2 additions & 2 deletions docs/cicd.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,11 +10,11 @@ Terraform apply is workflow-dispatch only: at dispatch it resolves and records t

The implemented Terraform workflow set is intentionally separate: `terraform-test.yml`, `terraform-plan.yml`, `terraform-apply.yml`, and `terraform-destroy.yml`. They have different triggers and IAM permissions; no reusable `_terraform-operation.yml` workflow or composite Terraform-init action is part of the baseline. The applied roots are `foundations/network`, `foundations/platform`, `foundations/edge`, `stacks/data`, `stacks/backend-service`, and `stacks/frontend-service`.

`terraform-test.yml` runs backend-free format, validate and TFLint checks for pull requests that change Terraform or Terraform workflow files. It detects the affected canonical roots and validates only implemented roots that changed. A shared module or Terraform workflow change conservatively validates every implemented root. `terraform-plan.yml`, `terraform-apply.yml`, and `terraform-destroy.yml` are separate `workflow_dispatch` controls. Each accepts a canonical `target`, resolves and logs the immutable `origin/main` SHA at dispatch, then resolves the target to its fixed root directory. Plan uses `environment: sandbox` and the read-only `happy-post-sandbox-terraform-plan` role, creates a plan, and uploads its rendered artifact. Apply uses the separate `happy-post-sandbox-terraform-apply` role, creates a fresh exact plan, and applies it. Apply confirmation is `apply-<target>`; destroy confirmation is `destroy-<target>`. A target that is defined by the workflow but absent from the resolved commit fails before credentials are configured. Pull requests, including forks, never receive AWS credentials. The current implemented targets are `network`, `data`, `platform`, `edge`, `backend-service`, and `frontend-service`.
`terraform-test.yml` runs backend-free format, validate, native Terraform tests, and TFLint checks for pull requests that change Terraform or Terraform workflow files. It detects the affected canonical roots and checks only implemented roots that changed. A shared module or Terraform workflow change conservatively checks every implemented root. Native Terraform tests use mock providers and remote-state overrides where needed, so they validate static configuration, naming, variable validation, digest constraints, and security assumptions without creating AWS resources. Terraform tests do not report coverage to SonarQube; SonarQube coverage is limited to backend and frontend application tests. `terraform-plan.yml`, `terraform-apply.yml`, and `terraform-destroy.yml` are separate `workflow_dispatch` controls. Each accepts a canonical `target`, resolves and logs the immutable `origin/main` SHA at dispatch, then resolves the target to its fixed root directory. Plan uses `environment: sandbox` and the read-only `happy-post-sandbox-terraform-plan` role, creates a plan, and uploads its rendered artifact. Apply uses the separate `happy-post-sandbox-terraform-apply` role, creates a fresh exact plan, and applies it. Apply confirmation is `apply-<target>`; destroy confirmation is `destroy-<target>`. A target that is defined by the workflow but absent from the resolved commit fails before credentials are configured. Pull requests, including forks, never receive AWS credentials. The current implemented targets are `network`, `data`, `platform`, `edge`, `backend-service`, and `frontend-service`.

`application-ci.yml` is separate from Terraform controls and uses no AWS credentials. It detects changed backend and frontend paths in every pull request. A backend change runs locked dependency installation, Ruff, and Pytest; a frontend change runs locked dependency installation, ESLint, Vitest, and the Next.js production build. A Compose or application-CI workflow change runs both component jobs.

`security-ci.yml` is independent of application tests and deployment. Trivy scans every pull request and `main` push for secrets, HIGH/CRITICAL dependency vulnerabilities, and HIGH/CRITICAL IaC misconfigurations. Snyk and SonarQube Cloud run only for same-repository pull requests and `main` pushes because they require repository secrets; fork pull requests never receive those secrets. Snyk blocks HIGH/CRITICAL dependency and IaC findings. SonarQube Cloud scans the combined backend/frontend project and waits for its quality gate.
`security-ci.yml` is independent of application tests and deployment. Trivy scans every pull request and `main` push for secrets, HIGH/CRITICAL dependency vulnerabilities, and HIGH/CRITICAL IaC misconfigurations. Snyk and SonarQube Cloud run only for same-repository pull requests and `main` pushes because they require repository secrets; fork pull requests never receive those secrets. Snyk blocks HIGH/CRITICAL dependency and IaC findings. SonarQube Cloud scans the combined backend/frontend project, consumes only backend/frontend application coverage reports, and waits for its quality gate.

`image-publish.yml` runs after a `main` push that changes `backend/**` or `frontend/**`, and can be manually dispatched from `main` to publish `backend`, `frontend`, or `all` for initial publication. It builds each selected image locally, blocks HIGH/CRITICAL Trivy image findings before AWS credentials are assumed, pushes only that component's immutable ECR repository, then uploads a seven-day artifact containing ECR's confirmed SHA-256 digest and source commit. `service-bootstrap.yml` is a separate `workflow_dispatch` control. It accepts `component`, `image_digest`, `source_commit`, and `bootstrap-<component>` confirmation, resolves immutable `main`, rejects malformed/all-zero digests and malformed source SHAs, verifies the digest exists in the selected repository with its matching immutable `sha-<source_commit>` tag, and applies only the selected service root through the sandbox Terraform-apply role. It is initial-service-only.

Expand Down
6 changes: 3 additions & 3 deletions docs/implementation-backlog.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ The documentation baseline and application MVPs are complete. The remaining work

## P5 — CI/CD and operational validation

- [x] Implement changed-root backend-free Terraform format/validate/TFLint on pull requests, plus workflow-dispatch-only plan/apply/destroy controls with a fixed canonical-root target allow-list.
- [x] Implement changed-root backend-free Terraform format, validate, native test, and TFLint checks on pull requests, plus workflow-dispatch-only plan/apply/destroy controls with a fixed canonical-root target allow-list.
- [x] Implement changed-component backend and frontend CI checks: locked dependencies, linting, unit tests, and frontend production build, without AWS credentials.
- [x] Implement Trivy, Snyk, and SonarQube Cloud security scanning with explicit scanner gates and fork-safe secret boundaries.
- [x] Remediate the Starlette HIGH findings and record the two approved, time-bound Trivy infrastructure exceptions for baseline sandbox trade-offs.
Expand All @@ -64,8 +64,8 @@ The documentation baseline and application MVPs are complete. The remaining work
- [x] Update local Compose to run PostgreSQL and apply migrations before backend startup.
- [x] Add frontend loading, validation, success, error, and simple monthly history states.
- [x] Publish new immutable frontend and backend images from merged P6 `main`.
- [ ] Merge the controlled database migration workflow, update the CloudFormation bootstrap stack, then run `alembic upgrade head` as a one-off private ECS task from the verified backend image digest.
- [ ] Deploy the P6 backend, deploy the P6 frontend, then verify entry persistence through RDS.
- [x] Merge the controlled database migration workflow, update the CloudFormation bootstrap stack, then run `alembic upgrade head` as a one-off private ECS task from the verified backend image digest.
- [x] Deploy the P6 backend, deploy the P6 frontend, then verify entry persistence through RDS.
- [ ] Re-run the restart persistence demo: create an entry, restart backend, and confirm the happy things remain.
- [ ] For real users, replace MVP app-level auth with managed identity such as Amazon Cognito, Auth0, or Clerk.

Expand Down
2 changes: 1 addition & 1 deletion docs/security-decisions.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ Security CI has no AWS credentials or GitHub environment access. Trivy scans sec

One GitHub environment does not mean one AWS role. Separate IAM roles preserve operational boundaries for Terraform planning, Terraform apply/destroy, image publishing, and ECS deployment. The plan role is restricted to the approved repository's `pull_request` claim; Terraform apply/destroy and ECS deployment roles are restricted to its `environment:sandbox` claim; the ECR publish role is restricted to its `ref:refs/heads/main` claim. The Terraform apply role has `ecr:DescribeImages` only for the two Happy Post repositories so the manual service-bootstrap workflow can confirm a supplied digest belongs to the selected repository; it has no ECR image-push permission. The ECS deployment role can read image metadata only from those two repositories, pass only the four named Happy Post task/execution roles to `ecs-tasks.amazonaws.com`, register task definitions, update only the two named Happy Post services, and run only the backend migration task-definition family for controlled database migrations. `ecs:DescribeTaskDefinition` has its own `Resource: "*"` statement because AWS does not support resource-level scope for that read action; task-definition tag reads remain limited to the Happy Post task-definition ARN prefix.

The Terraform test workflow maps changed files to a fixed allow-list of canonical roots and runs backend-free validation only; a shared module or Terraform workflow change validates every implemented root. The manual plan workflow uses `happy-post-sandbox-terraform-plan` only from `environment:sandbox`, resolves and logs immutable `origin/main`, and accepts a fixed canonical-root `target` mapped to a fixed directory. Apply and destroy use the separate `happy-post-sandbox-terraform-apply` role from the same environment; apply creates and applies an exact fresh plan in one job. A target missing from the resolved commit fails before credentials are configured. The workflow source contains role ARNs, which are non-secret account configuration; it contains no AWS access keys or secret values.
The Terraform test workflow maps changed files to a fixed allow-list of canonical roots and runs backend-free validation only; a shared module or Terraform workflow change validates every implemented root. Backend-free validation includes native Terraform tests that use mock providers and remote-state overrides where needed, so pull requests can check static naming, variable validation, digest constraints, private-network assumptions, and least-privilege boundaries without AWS credentials or resource creation. The manual plan workflow uses `happy-post-sandbox-terraform-plan` only from `environment:sandbox`, resolves and logs immutable `origin/main`, and accepts a fixed canonical-root `target` mapped to a fixed directory. Apply and destroy use the separate `happy-post-sandbox-terraform-apply` role from the same environment; apply creates and applies an exact fresh plan in one job. A target missing from the resolved commit fails before credentials are configured. The workflow source contains role ARNs, which are non-secret account configuration; it contains no AWS access keys or secret values.

## Terraform state-backend protection

Expand Down
13 changes: 13 additions & 0 deletions infra/terraform/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -91,18 +91,31 @@ From the repository root, run backend-free validation first:
terraform -chdir=infra/terraform/foundations/network init -backend=false
terraform fmt -check -recursive infra/terraform
terraform -chdir=infra/terraform/foundations/network validate
terraform -chdir=infra/terraform/foundations/network test
terraform -chdir=infra/terraform/stacks/data init -backend=false
terraform -chdir=infra/terraform/stacks/data validate
terraform -chdir=infra/terraform/stacks/data test
terraform -chdir=infra/terraform/foundations/platform init -backend=false
terraform -chdir=infra/terraform/foundations/platform validate
terraform -chdir=infra/terraform/foundations/platform test
terraform -chdir=infra/terraform/foundations/edge init -backend=false
terraform -chdir=infra/terraform/foundations/edge validate
terraform -chdir=infra/terraform/foundations/edge test
terraform -chdir=infra/terraform/stacks/backend-service init -backend=false
terraform -chdir=infra/terraform/stacks/backend-service validate
terraform -chdir=infra/terraform/stacks/backend-service test
terraform -chdir=infra/terraform/stacks/frontend-service init -backend=false
terraform -chdir=infra/terraform/stacks/frontend-service validate
terraform -chdir=infra/terraform/stacks/frontend-service test
```

Native Terraform tests are backend-free and use provider mocks plus
remote-state overrides where needed. They check static configuration, variable
validation, naming, digest constraints, private-network assumptions, and
least-privilege boundaries without creating AWS resources. They are not
application coverage inputs; SonarQube coverage applies only to backend and
frontend application tests.

For an authorised read-only review of the real state and AWS configuration, use
the approved short-lived AWS profile and initialise the declared backend:

Expand Down
54 changes: 54 additions & 0 deletions infra/terraform/foundations/edge/edge.tftest.hcl
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
mock_provider "aws" {}

override_data {
target = data.terraform_remote_state.network

values = {
outputs = {
vpc_id = "vpc-0123456789abcdef0"
public_subnet_ids = {
"0" = "subnet-0123456789abcdef0"
"1" = "subnet-0123456789abcdef1"
}
security_group_ids = {
alb = "sg-0123456789abcdef0"
}
}
}
}

run "edge_rejects_non_sandbox_environment" {
command = plan

variables {
environment = "production"
}

expect_failures = [
var.environment,
]
}

run "edge_rejects_unapproved_application_domain" {
command = plan

variables {
application_domain = "example.com"
}

expect_failures = [
var.application_domain,
]
}

run "edge_rejects_unapproved_hosted_zone_id" {
command = plan

variables {
route53_hosted_zone_id = "Z11111111111111111111"
}

expect_failures = [
var.route53_hosted_zone_id,
]
}
77 changes: 77 additions & 0 deletions infra/terraform/foundations/network/network.tftest.hcl
Original file line number Diff line number Diff line change
@@ -0,0 +1,77 @@
mock_provider "aws" {
mock_data "aws_availability_zones" {
defaults = {
names = [
"ap-southeast-2a",
"ap-southeast-2b",
"ap-southeast-2c",
]
}
}
}

run "network_static_security_boundaries" {
command = apply

assert {
condition = aws_vpc.this.cidr_block == "10.42.0.0/16"
error_message = "The sandbox VPC must keep the approved 10.42.0.0/16 CIDR."
}

assert {
condition = aws_vpc.this.enable_dns_hostnames && aws_vpc.this.enable_dns_support
error_message = "The sandbox VPC must support DNS for ECS, ALB, and RDS integration."
}

assert {
condition = length(aws_subnet.public) == 2 && length(aws_subnet.application) == 2 && length(aws_subnet.database) == 2
error_message = "Public, private application, and private database subnet tiers must each span two Availability Zones."
}

assert {
condition = aws_subnet.public["0"].map_public_ip_on_launch == false && aws_subnet.public["1"].map_public_ip_on_launch == false
error_message = "Public subnets must not auto-assign public IPs to launched resources."
}

assert {
condition = length([for route in aws_route_table.application.route : route if route.cidr_block == "0.0.0.0/0" && route.nat_gateway_id == aws_nat_gateway.this.id]) == 1
error_message = "Private application subnets must use the single sandbox NAT Gateway for outbound access."
}

assert {
condition = length(aws_route_table.database.route) == 0
error_message = "Private database subnets must not have internet or NAT routes."
}

assert {
condition = aws_vpc_security_group_ingress_rule.alb_https.from_port == 443 && aws_vpc_security_group_ingress_rule.alb_https.cidr_ipv4 == "0.0.0.0/0"
error_message = "The ALB security group must allow public HTTPS ingress."
}

assert {
condition = aws_vpc_security_group_ingress_rule.alb_http.from_port == 80 && aws_vpc_security_group_ingress_rule.alb_http.cidr_ipv4 == "0.0.0.0/0"
error_message = "The ALB security group must allow public HTTP ingress only for HTTP-to-HTTPS redirect."
}

assert {
condition = aws_vpc_security_group_ingress_rule.backend_from_alb.referenced_security_group_id == aws_security_group.alb.id && aws_vpc_security_group_ingress_rule.backend_from_alb.from_port == 8000
error_message = "Backend service ingress must come only from the ALB security group on the backend container port."
}

assert {
condition = aws_vpc_security_group_egress_rule.backend_database.referenced_security_group_id == aws_security_group.database.id && aws_vpc_security_group_egress_rule.backend_database.from_port == 5432
error_message = "Backend egress to PostgreSQL must target only the database security group on TCP 5432."
}
}

run "network_rejects_non_sandbox_environment" {
command = plan

variables {
environment = "production"
}

expect_failures = [
var.environment,
]
}
Loading
Loading