Skip to content

Commit ccae4b3

Browse files
simotasimota
committed
chore(dynamodb): add autoloop and e2e scripts
1 parent 2812ef0 commit ccae4b3

9 files changed

Lines changed: 1425 additions & 0 deletions

File tree

‎.agents/orbit.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,4 @@
11
| Date | Agent | Action | Files | Outcome |
22
| --- | --- | --- | --- | --- |
33
| 2026-04-30 | Orbit | generated S3 Codex autoloop script set | `scripts/s3-autoloop/`, `.gitignore`, `README.md` | ready for bounded S3 implementation loop with foundation/full gates |
4+
| 2026-05-01 | Orbit | generated DynamoDB Codex autoloop script set | `scripts/dynamodb-autoloop/`, `.gitignore` | ready for bounded DynamoDB implementation loop with staged verification gates |

‎.gitignore‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -25,6 +25,16 @@ scripts/gcs-autoloop/progress.md
2525
scripts/gcs-autoloop/done.md
2626
scripts/gcs-autoloop/iteration-*.out
2727
scripts/gcs-autoloop/prompt-*.*
28+
scripts/dynamodb-autoloop/.run-loop.lock
29+
scripts/dynamodb-autoloop/.circuit-state
30+
scripts/dynamodb-autoloop/runner.jsonl
31+
scripts/dynamodb-autoloop/runner.log
32+
scripts/dynamodb-autoloop/state.env
33+
scripts/dynamodb-autoloop/state.env.sha256
34+
scripts/dynamodb-autoloop/progress.md
35+
scripts/dynamodb-autoloop/done.md
36+
scripts/dynamodb-autoloop/iteration-*.out
37+
scripts/dynamodb-autoloop/prompt-*.*
2838
scripts/dashboard-design-renewal-autoloop/.run-loop.lock
2939
scripts/dashboard-design-renewal-autoloop/.circuit-state
3040
scripts/dashboard-design-renewal-autoloop/runner.jsonl

‎scripts/dynamodb-autoloop/README.md‎

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
# DynamoDB Server Autoloop
2+
3+
Codex-driven autonomous loop for completing the `devcloud` Amazon DynamoDB compatible local server.
4+
5+
## Usage
6+
7+
```bash
8+
bash scripts/dynamodb-autoloop/bootstrap.sh
9+
bash scripts/dynamodb-autoloop/run-loop.sh
10+
```
11+
12+
Useful options:
13+
14+
```bash
15+
MAX_ITERATIONS=40 bash scripts/dynamodb-autoloop/run-loop.sh
16+
AUTOCOMMIT=true bash scripts/dynamodb-autoloop/run-loop.sh
17+
CODEX_ARGS="--full-auto" bash scripts/dynamodb-autoloop/run-loop.sh
18+
VERIFY_STAGE=foundation bash scripts/dynamodb-autoloop/run-loop.sh
19+
DONE_VERIFY_STAGE=full bash scripts/dynamodb-autoloop/run-loop.sh
20+
VERIFY_STAGE=full bash scripts/dynamodb-autoloop/verify.sh
21+
```
22+
23+
## Verification Stages
24+
25+
- `foundation`: checks DynamoDB design contract, repository build, CLI help, and current Go test suite.
26+
- `config`: checks DynamoDB config, daemon wiring, and workspace initialization once implemented.
27+
- `protocol`: checks JSON 1.0 endpoint dispatch, `CreateTable`, `DescribeTable`, and `ListTables`.
28+
- `item-core`: checks `PutItem`, `GetItem`, `UpdateItem`, `DeleteItem`, and conditional write behavior.
29+
- `query-index`: checks `Query`, `Scan`, pagination, and GSI/LSI-oriented behavior.
30+
- `dashboard`: checks dashboard service registry and `/api/dynamodb/*` visibility.
31+
- `full`: final DynamoDB compatibility acceptance gate, including AWS CLI smoke when `aws` is installed.
32+
33+
Runtime files are ignored by git: `progress.md`, `state.env`, `runner.log`, `runner.jsonl`, `iteration-*.out`, `done.md`, and lock/circuit files.

‎scripts/dynamodb-autoloop/bootstrap.sh‎

Lines changed: 53 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,53 @@
1+
#!/usr/bin/env bash
2+
set -euo pipefail
3+
4+
LOOP_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
5+
ROOT_DIR="$(cd "${LOOP_DIR}/../.." && pwd)"
6+
STATE_FILE="${LOOP_DIR}/state.env"
7+
PROGRESS_FILE="${LOOP_DIR}/progress.md"
8+
9+
cd "${ROOT_DIR}"
10+
11+
if [[ ! -f "${LOOP_DIR}/goal.md" ]]; then
12+
echo "[FAIL] Missing ${LOOP_DIR}/goal.md" >&2
13+
exit 1
14+
fi
15+
16+
if [[ ! -f "${PROGRESS_FILE}" ]]; then
17+
tmp_progress="$(mktemp "${PROGRESS_FILE}.XXXXXX")"
18+
{
19+
echo "# DynamoDB Autoloop Progress"
20+
echo ""
21+
echo "Created at: $(date -u +"%Y-%m-%dT%H:%M:%SZ")"
22+
echo ""
23+
echo "NEXUS_LOOP_STATUS: READY"
24+
echo "NEXUS_LOOP_SUMMARY: DynamoDB implementation loop initialized."
25+
} > "${tmp_progress}"
26+
mv "${tmp_progress}" "${PROGRESS_FILE}"
27+
fi
28+
29+
tmp_state="$(mktemp "${STATE_FILE}.XXXXXX")"
30+
cat > "${tmp_state}" <<EOF
31+
CONTRACT_VERSION=1.1.0
32+
NEXT_ITERATION=1
33+
LAST_STATUS=READY
34+
LAST_UPDATED_AT=$(date -u +"%Y-%m-%dT%H:%M:%SZ")
35+
TOTAL_TOKENS=0
36+
TOTAL_API_CALLS=0
37+
ESTIMATED_COST_USD=0
38+
ITER_TOKENS=0
39+
ITER_API_CALLS=0
40+
VERIFY_STAGE=foundation
41+
DONE_VERIFY_STAGE=full
42+
CIRCUIT_STATE=CLOSED
43+
CIRCUIT_FAIL_COUNT=0
44+
EOF
45+
mv "${tmp_state}" "${STATE_FILE}"
46+
shasum -a 256 "${STATE_FILE}" | awk '{print $1}' > "${STATE_FILE}.sha256"
47+
48+
bash -n "${LOOP_DIR}/run-loop.sh"
49+
bash -n "${LOOP_DIR}/recover.sh"
50+
bash -n "${LOOP_DIR}/verify.sh"
51+
52+
echo "[OK] DynamoDB autoloop bootstrap complete"
53+
echo "Run: bash scripts/dynamodb-autoloop/run-loop.sh"

‎scripts/dynamodb-autoloop/goal.md‎

Lines changed: 46 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,46 @@
1+
# DynamoDB Server Autoloop Goal
2+
3+
## Goal
4+
5+
Implement an Amazon DynamoDB compatible local server for `devcloud`, following `docs/design-dynamodb-compat.md`.
6+
7+
## Why
8+
9+
`devcloud` already has local Mail, S3, GCS, and dashboard foundations. DynamoDB compatibility should add a local AWS SDK / CLI target for table, item, expression, index, stream, transaction, and PartiQL workflows without reaching AWS or depending on DynamoDB Local.
10+
11+
## Acceptance Criteria
12+
13+
1. `devcloud up` starts a DynamoDB JSON API endpoint on the configured local port, defaulting to `127.0.0.1:8000`.
14+
2. DynamoDB low-level JSON protocol works with `POST /`, `Content-Type: application/x-amz-json-1.0`, and `X-Amz-Target: DynamoDB_20120810.{Operation}`.
15+
3. Table operations work: `CreateTable`, `DescribeTable`, `ListTables`, `UpdateTable` for supported metadata, and `DeleteTable`.
16+
4. Item operations work: `PutItem`, `GetItem`, `UpdateItem`, `DeleteItem`, `BatchGetItem`, and `BatchWriteItem`.
17+
5. `AttributeValue` supports `S`, `N`, `B`, `BOOL`, `NULL`, `M`, `L`, `SS`, `NS`, and `BS` with DynamoDB-compatible JSON response shapes.
18+
6. Expressions work for the documented MVP: `ConditionExpression`, `UpdateExpression`, `ProjectionExpression`, `FilterExpression`, and `KeyConditionExpression`.
19+
7. `Query` and `Scan` support `Limit`, `ExclusiveStartKey`, `LastEvaluatedKey`, sort ordering, filters, and projection for tested fixtures.
20+
8. GSI and LSI metadata, projection updates, and index `Query` / `Scan` work for tested fixtures.
21+
9. Strict SigV4 mode validates signed AWS SDK / CLI requests without logging credentials, signatures, canonical requests, or payloads.
22+
10. Existing Mail, S3, GCS, and dashboard behavior remains compatible; `go test ./...` passes.
23+
11. Dashboard service registry exposes DynamoDB, and `/api/dynamodb/*` can inspect tables, indexes, items, streams, and TTL state.
24+
12. `VERIFY_STAGE=full bash scripts/dynamodb-autoloop/verify.sh` passes.
25+
26+
## Out of Scope for This Loop
27+
28+
- Real AWS IAM, STS, CloudWatch, CloudTrail, KMS, or external AWS service calls.
29+
- DynamoDB Local, LocalStack, or other external emulator dependency.
30+
- DAX protocol.
31+
- Multi-region global table replication.
32+
- Real provisioned billing, adaptive capacity, and AWS-managed auto scaling.
33+
- Full Contributor Insights, Kinesis streaming destination, and S3 import/export behavior.
34+
35+
## Implementation Guidance
36+
37+
- Preserve existing Mail, S3, GCS, and dashboard behavior before broad storage refactors.
38+
- Prefer small vertical slices with tests.
39+
- Keep runtime data under `.devcloud/`.
40+
- Use Go standard library unless a dependency is clearly justified.
41+
- Keep DynamoDB protocol, service logic, expression parsing, and storage boundaries separate.
42+
- Do not log credentials, Authorization headers, signatures, canonical requests, item payloads, or other sensitive request bodies.
43+
- Treat `docs/design-dynamodb-compat.md` as the implementation contract.
44+
45+
NEXUS_LOOP_STATUS: READY
46+
NEXUS_LOOP_SUMMARY: DynamoDB compatibility loop contract is ready.

‎scripts/dynamodb-autoloop/recover.sh‎

Lines changed: 66 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,66 @@
1+
#!/usr/bin/env bash
2+
set -euo pipefail
3+
4+
LOOP_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
5+
PROGRESS_FILE="${LOOP_DIR}/progress.md"
6+
STATE_FILE="${LOOP_DIR}/state.env"
7+
8+
echo "=== DynamoDB Autoloop Recovery ==="
9+
10+
if [[ -f "${LOOP_DIR}/.run-loop.lock" ]]; then
11+
LOCK_PID="$(cat "${LOOP_DIR}/.run-loop.lock" 2>/dev/null || true)"
12+
if [[ -n "${LOCK_PID}" ]] && kill -0 "${LOCK_PID}" 2>/dev/null; then
13+
echo "[BLOCKED] Active runner lock: PID ${LOCK_PID}"
14+
exit 1
15+
fi
16+
rm -f "${LOOP_DIR}/.run-loop.lock"
17+
echo "[OK] Removed stale runner lock"
18+
fi
19+
20+
LATEST_ITER="$(grep -oE 'Iteration [0-9]+' "${PROGRESS_FILE}" 2>/dev/null | awk '{print $2}' | tail -1 || true)"
21+
if [[ -z "${LATEST_ITER}" ]]; then
22+
LATEST_ITER=0
23+
fi
24+
NEXT_ITER=$((LATEST_ITER + 1))
25+
26+
TAIL_CONTENT="$(tail -40 "${PROGRESS_FILE}" 2>/dev/null || true)"
27+
if echo "${TAIL_CONTENT}" | grep -q 'NEXUS_LOOP_STATUS: DONE'; then
28+
STATUS="DONE"
29+
else
30+
STATUS="CONTINUE"
31+
fi
32+
33+
TMP_STATE="$(mktemp "${STATE_FILE}.XXXXXX")"
34+
cat > "${TMP_STATE}" <<EOF
35+
CONTRACT_VERSION=1.1.0
36+
NEXT_ITERATION=${NEXT_ITER}
37+
LAST_STATUS=${STATUS}
38+
LAST_UPDATED_AT=$(date -u +"%Y-%m-%dT%H:%M:%SZ")
39+
RECOVERED_FROM=progress_evidence
40+
TOTAL_TOKENS=0
41+
TOTAL_API_CALLS=0
42+
ESTIMATED_COST_USD=0
43+
ITER_TOKENS=0
44+
ITER_API_CALLS=0
45+
VERIFY_STAGE=foundation
46+
DONE_VERIFY_STAGE=full
47+
CIRCUIT_STATE=CLOSED
48+
CIRCUIT_FAIL_COUNT=0
49+
EOF
50+
mv "${TMP_STATE}" "${STATE_FILE}"
51+
shasum -a 256 "${STATE_FILE}" | awk '{print $1}' > "${STATE_FILE}.sha256"
52+
53+
{
54+
echo ""
55+
echo "## Recovery — $(date -u +"%Y-%m-%dT%H:%M:%SZ")"
56+
echo "- Latest iteration found: ${LATEST_ITER}"
57+
echo "- Rebuilt state.env with NEXT_ITERATION=${NEXT_ITER}"
58+
echo "- Cleared circuit state"
59+
echo "- Decision: CONTINUE"
60+
echo ""
61+
echo "NEXUS_LOOP_STATUS: CONTINUE"
62+
echo "NEXUS_LOOP_SUMMARY: Recovery completed; loop can resume from iteration ${NEXT_ITER}."
63+
} >> "${PROGRESS_FILE}"
64+
65+
rm -f "${LOOP_DIR}/.circuit-state"
66+
echo "[OK] Recovery complete; next iteration ${NEXT_ITER}"

0 commit comments

Comments
 (0)