Skip to content

Add Lambda VPC proxy for MBE-to-OpenEMS B2B access#75

Open
amalbet wants to merge 9 commits into
mainfrom
feature/lambda-proxy
Open

Add Lambda VPC proxy for MBE-to-OpenEMS B2B access#75
amalbet wants to merge 9 commits into
mainfrom
feature/lambda-proxy

Conversation

@amalbet

@amalbet amalbet commented Apr 21, 2026

Copy link
Copy Markdown

Summary

Adds a Lambda VPC proxy to iac/dev/ that allows MBE (on Vercel) to securely access the OpenEMS B2B REST API without exposing the backend to the public internet. This is Option 3 from the architecture discussion in PR #70.

  • Lambda sits inside the VPC, forwards JSON-RPC requests to the backend via private IP
  • Function URL with IAM auth (SigV4) — stronger than API keys, logged in CloudTrail
  • Stateless passthrough (~15 lines) — no business logic, no dependencies, deploy once
  • Cost: ~$0/mo (Lambda free tier covers MBE's ~100-500 req/month)
  • Backend config uses partial backend pattern — no infrastructure metadata in the public repo

Closes #73

Changes

New files:

  • iac/dev/lambda/index.mjs — Node.js 20 proxy function (handles errors, base64 encoding)
  • iac/dev/lambda.tf — Lambda function, IAM roles, security group, Function URL, CloudWatch log group, Vercel invoker IAM user + access key
  • iac/dev/backend.tfvars.example — template for backend config (bucket + table names provided at init time, not committed)

Modified files:

  • iac/dev/backend.tf — switched to partial backend pattern (no hardcoded bucket/table names)
  • iac/dev/security.tf — added EC2 SG ingress rule from Lambda SG on port 8082 (SG-to-SG)
  • iac/dev/outputs.tf — renamed b2b_url → b2b_url_direct, added b2b_url_lambda, IAM key outputs
  • iac/dev/variables.tf — added openems_b2b_creds (sensitive)
  • iac/dev/terraform.tfvars.example — documented new variable
  • iac/dev/.gitignore — added backend.tfvars, lambda/proxy.zip
  • iac/dev/README.md — updated init instructions for backend config

Prerequisites

Before terraform apply:

  • Cloud admin provides S3 bucket and DynamoDB table names (goes in local backend.tfvars, gitignored)
  • Lambda permissions added to the deploying IAM user (AWSLambda_FullAccess or scoped)
  • openems_b2b_creds set in terraform.tfvars (base64-encoded user:password)

Test plan

  • terraform init -backend-config=backend.tfvars succeeds
  • terraform plan shows Lambda + IAM + SG resources to create (no changes to existing EC2/VPC)
  • terraform apply succeeds
  • Lambda appears in AWS Console, attached to the dev VPC
  • Manual test with awscurl: POST to the Function URL with SigV4 → valid JSON-RPC response from backend
  • CloudWatch logs show the invocation
  • EC2 SG shows inbound rule from Lambda SG on port 8082
  • terraform destroy cleans up all Lambda resources + log group
  • No infrastructure metadata (bucket names, table names, account IDs) in committed code

🤖 Generated with Claude Code

amalbet and others added 6 commits April 16, 2026 15:39
@amalbet @claude
Add iac/dev/ for single-EC2 dev environment
6b77f95 
Provisions an isolated dev environment on AWS: VPC 10.100.0.0/16 with a
public subnet, t3.large Ubuntu 22.04 instance running the full
docker-compose stack via setup.sh, security group scoped to an
allowed_ips variable (no 0.0.0.0/0 exposure), and an IAM role for SSM
Session Manager access (no SSH key required).

State is stored in the existing openems-deployment-tf-state-file S3
bucket under a separate key (iac/dev/terraform.tfstate) so it does not
collide with the production ECS deployment in iac/.

User-data clones the local-deployment branch and runs setup.sh --edges 2
on first boot. Bootstrap takes ~10 min on a fresh instance.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Signed-off-by: Alejandro Malbet <amalbet@gmail.com>
@aidan-barnes-axm
Update Environment tag from "dev" to "aidev"
c492c39 
Signed-off-by: Aidan Barnes <66229298+aidan-barnes-axm@users.noreply.github.com>
@amalbet @claude
Update backend.tf and provider.tf for EIOT dev account
1ad3cfe 
- backend.tf: point to Aidan's state bucket
  (docker-openems-feature-dev-iac) and lock table
  (docker-openems-feature-dev-iac-state-lock) in the dev account
- provider.tf: change Environment tag from "dev" to "aidev" to match
  the IAM policies Aidan configured for the dev account

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Signed-off-by: Alejandro Malbet <amalbet@gmail.com>
@amalbet @claude
Add Lambda VPC proxy for MBE-to-OpenEMS B2B access
4a2f583 
Implements ticket #73. Adds a Node.js 20 Lambda function inside the dev VPC
that proxies JSON-RPC requests from Vercel (MBE) to the OpenEMS Backend on
its private IP via port 8082. Auth is IAM SigV4 via Function URL.

- iac/dev/lambda/index.mjs: passthrough proxy with error handling, native
  fetch (no npm deps), handles isBase64Encoded, returns 502 on unreachable
- iac/dev/lambda.tf: all Lambda resources — archive_file, IAM role +
  AWSLambdaVPCAccessExecutionRole, Lambda SG (egress 8082 to EC2 SG),
  aws_lambda_function (128MB/30s/VPC), Function URL (AWS_IAM/BUFFERED),
  CloudWatch log group (14-day retention), MBE invoker IAM user + access key
  + user policy (lambda:InvokeFunctionUrl on this Lambda ARN only)
- iac/dev/security.tf: aws_security_group_rule ingress 8082 from Lambda SG
  to EC2 SG (SG-to-SG reference, no CIDR)
- iac/dev/outputs.tf: rename b2b_url → b2b_url_direct, add b2b_url_lambda,
  lambda_invoker_access_key_id, lambda_invoker_secret_key (sensitive)
- iac/dev/variables.tf: add openems_b2b_creds (sensitive, no default)
- iac/dev/terraform.tfvars.example: document openems_b2b_creds format

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Signed-off-by: Alejandro Malbet <amalbet@gmail.com>
@amalbet @claude
Fix Lambda SG egress block and gitignore proxy.zip
9dc3e0d 
- lambda.tf: use `security_groups` (list) instead of invalid
  `source_security_group_id` for inline egress block
- .gitignore: add lambda/proxy.zip (created by archive_file at plan time)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Signed-off-by: Alejandro Malbet <amalbet@gmail.com>
@amalbet @claude
Move backend config to partial backend pattern
a7005fa 
S3 bucket and DynamoDB table names are now provided at init time
via backend.tfvars (gitignored), not hardcoded in backend.tf.
This keeps infrastructure metadata out of the public repo.

Usage: terraform init -backend-config=backend.tfvars

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Signed-off-by: Alejandro Malbet <amalbet@gmail.com>
@amalbet amalbet mentioned this pull request Apr 21, 2026
Open
3 tasks
amalbet added 2 commits April 21, 2026 15:16
@amalbet
fix(iac/dev): B2B REST port 8082 → 8075
3bebb29 
OpenEMS runs the B2B REST (JSON-RPC) endpoint on 8075 per the
`Backend2Backend.Rest` config. Port 8082 is the UI ↔ Backend WebSocket
(the browser connects there from the UI at :4200). PR #75 was using
8082 everywhere for "B2B REST" — the Lambda proxy would have connected
to the UI websocket instead of the REST endpoint.

Verified against docker-openems local-deployment:
- docker-compose.yml maps 8075:8075 with comment "B2B REST API"
- openems-backend/config.d/Backend2Backend/Rest/*.config → port=8075
- openems-backend/config.d/Ui/Websocket.config             → port=8082
- metering-billing-engine/src/lib/openems/__tests__/client.test.ts:33
  uses http://localhost:8075 as the B2B base URL

Changes:
- security.tf: keep 8082 ingress (relabeled "UI Backend WebSocket"),
  add new 8075 ingress for B2B REST, change Lambda→EC2 SG rule to 8075
- lambda.tf: Lambda SG egress 8082 → 8075
- lambda/index.mjs: /jsonrpc endpoint 8082 → 8075
- outputs.tf: b2b_url_direct 8082 → 8075; new ui_backend_ws_url output
- README.md: port table updated; 8082 now "UI ↔ Backend WebSocket",
  8075 is "B2B REST"

Found during 2026-04-21 end-to-end validation of the dev stack.

Signed-off-by: Alejandro Malbet <amalbet@gmail.com>
@amalbet
fix(iac/dev): use IMDSv2 token-based metadata fetch in EC2 bootstrap
6df1657 
The EC2 instance is provisioned with http_tokens = "required"
(IMDSv2 enforced) in ec2.tf:25-28. The bootstrap log echoes used
plain `curl http://169.254.169.254/...` which silently fails under
IMDSv2 — the final 3 lines of the bootstrap log would print empty
URLs instead of the instance's public IP.

Replaces the 3 separate `curl` calls with a single token fetch, then
reuses the token for the metadata query. Also corrects the B2B port
in the bootstrap echo from 8082 to 8075 (per the previous commit).

Impact is cosmetic only (setup.sh itself doesn't hit metadata), but
the pattern is flagged in mbe-docs/docs/learnings.md as a recurring
bug worth fixing at the source.

Signed-off-by: Alejandro Malbet <amalbet@gmail.com>
@amalbet

amalbet commented Apr 21, 2026

Copy link
Copy Markdown
Author

Update (2026-04-21): validated end-to-end + 2 bug fixes

Validated this PR end-to-end against the dev AWS account today:

  • terraform apply created 17/20 resources (the 3 failures are IAM user creation for mbe_invoker — separate iam:CreateUser permission ask in Slack)
  • EC2 bootstraps with 2 simulated edges (setup.sh --edges 2)
  • Lambda Function URL invocation via SigV4 (signed with my own profile, since mbe_invoker can't be provisioned yet) → 14KB proxied response with full edge config from edge0
  • Stack endpoints all responding: UI (4200), Odoo (10016), B2B REST (8075), and the Lambda Function URL

Found 2 bugs in the process and pushed fixes to this branch — both small, labeled, and self-contained. No destructive changes:

3bebb29 — fix(iac/dev): B2B REST port 8082 → 8075
PR #75 used 8082 everywhere for "B2B REST", but 8082 is actually the UI↔Backend WebSocket on local-deployment. The real B2B REST (JSON-RPC) is on 8075. Applying as-is would have the Lambda connect to the UI websocket and fail. Verified against openems-backend/config.d/Backend2Backend/Rest/*.config, docker-compose.yml, and the MBE client test at src/lib/openems/__tests__/client.test.ts:33.

6df1657 — fix(iac/dev): IMDSv2 token-based metadata fetch
EC2 is provisioned with http_tokens = "required" (IMDSv2) per ec2.tf:25-28. The bootstrap log echoes used plain curl http://169.254.169.254/... which silently fails under IMDSv2. Cosmetic only (setup.sh itself doesn't hit metadata), but fixing at source since it's a pattern flagged in our learnings log.

@aidan-barnes-axm — please review when you have a chance. Separately, I've also flagged a couple of non-Terraform bugs in docker-compose.yml and setup.sh on local-deployment (stale addons mount path + multi-edge override doesn't pre-build images). Those will go into a follow-up PR against local-deployment since they're separate from the IaC.

@amalbet amalbet mentioned this pull request Apr 21, 2026
Merged
5 tasks
@tushabe

tushabe commented Apr 21, 2026

Copy link
Copy Markdown

Review findings

[P1] Proxy targets the UI websocket port instead of B2B REST

iac/dev/lambda/index.mjs:8

The Lambda forwards JSON-RPC to port 8082, but the local-deployment branch configures Backend2Backend.Rest on port 8075; 8082 is Ui.Websocket. As written, the MBE proxy will hit the wrong OpenEMS service or fail, and the Lambda security-group rule also only permits 8082. Please update the Lambda target, security group, direct output, and docs to 8075, or change the backend config consistently.

[P1] Function URL invoker lacks required Lambda permission

iac/dev/lambda.tf:145

The Vercel IAM user is granted only lambda:InvokeFunctionUrl. Current AWS Lambda Function URL IAM auth requires both lambda:InvokeFunctionUrl and lambda:InvokeFunction for invocation, so this user can receive 403 responses even with valid SigV4 signing. Please add lambda:InvokeFunction, ideally constrained with lambda:InvokedViaFunctionUrl=true.

[P3] Bootstrap logs use IMDSv1 despite requiring IMDSv2

iac/dev/user-data.sh.tpl:45-47

The EC2 resource sets http_tokens = "required", but these metadata curl calls do not fetch and pass an IMDSv2 token. The stack may still finish, but the final bootstrap log URLs will be blank or fail with 401. Please fetch a token once and pass X-aws-ec2-metadata-token for these metadata calls.

@amalbet
fix(iac/dev): mbe_invoker policy needs lambda:InvokeFunction + tighte…
bea6fbb 
…r condition

Lambda Function URLs with AuthType=AWS_IAM require BOTH
lambda:InvokeFunctionUrl AND lambda:InvokeFunction on the caller's
identity policy. The previous policy granted only
lambda:InvokeFunctionUrl, so mbe_invoker would have hit a 403 the
first time it tried to invoke the Function URL — even with valid
SigV4 signing.

Not caught earlier because end-to-end testing today used my own
profile (AWSLambda_FullAccess), which has both permissions. The
scoped service user wouldn't.

Also adds the lambda:InvokedViaFunctionUrl=true condition to ensure
the user can ONLY invoke via the Function URL — not via the direct
Lambda Invoke API. Tighter security posture matching the original
intent of the scoped policy.

Per @tushabe review on PR #75 — thanks Aaron.

Signed-off-by: Alejandro Malbet <amalbet@gmail.com>
@amalbet

amalbet commented Apr 22, 2026

Copy link
Copy Markdown
Author

@tushabe — really useful review, thank you. Status on each finding:

P1 #1 (port 8082 → 8075) — already addressed

Same bug I caught and fixed earlier today in commit 3bebb29. Your independent catch validates the fix though — exact same diagnosis (8082 is Ui.Websocket, 8075 is Backend2Backend.Rest, MBE client tests confirm). Reviewing the latest branch should show all 8082 references for B2B replaced with 8075 across security.tf, lambda.tf, lambda/index.mjs, outputs.tf, and README.md.

P1 #2 (lambda:InvokeFunction missing) — real new catch, fixed in commit bea6fbb

You're right — this would have failed the first time mbe_invoker actually invoked the Function URL. I missed it because my end-to-end test today used my own profile (AWSLambda_FullAccess), which has both permissions; the scoped service user wouldn't.

Pushed your suggested fix:

  • Added lambda:InvokeFunction alongside lambda:InvokeFunctionUrl
  • Added the lambda:InvokedViaFunctionUrl=true condition so the user can ONLY invoke via the Function URL, not the direct Invoke API — even tighter than what was there before

Will be properly tested once Aidan grants the iam:CreateUser perms that are blocking mbe_invoker provisioning (separate Slack thread).

P3 (IMDSv2 token fetch) — already addressed

Fixed in commit 6df1657. Token-based metadata fetch, also corrected the B2B port in the bootstrap echo.

Branch state after this update:

  • 3bebb29 — port 8082 → 8075 (5 files)
  • 6df1657 — IMDSv2 token-based metadata fetch
  • bea6fbb — lambda:InvokeFunction + InvokedViaFunctionUrl condition (this one)

CI green, PR mergeable. Ready when you and @aidan-barnes-axm sign off.

@amalbet amalbet requested review from aidan-barnes-axm and axmsoftware and removed request for aidan-barnes-axm April 22, 2026 01:46
@tushabe

tushabe commented Apr 22, 2026

Copy link
Copy Markdown

@amalbet unfortunately I don't have write access to this repo yet so i can only provide review feedback, I can't approve PRs. Over to @aidan-barnes-axm and @axmsoftware

@tushabe tushabe closed this Apr 22, 2026
@tushabe tushabe reopened this Apr 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@axmsoftware axmsoftware Awaiting requested review from axmsoftware
@aidan-barnes-axm aidan-barnes-axm Awaiting requested review from aidan-barnes-axm

At least 1 approving review is required to merge this pull request.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Add Lambda VPC proxy for MBE-to-OpenEMS B2B access

3 participants

@amalbet @tushabe @aidan-barnes-axm