Skip to content

sre-lab: teach the Azure commands before the wrapper - #49

Merged
hellices merged 2 commits into
mainfrom
fix/sre-agent-review-rounds
Aug 15, 2026
Merged

hellices merged 2 commits into
mainfrom
fix/sre-agent-review-rounds

Conversation

@hellices

Copy link
Copy Markdown
Owner

Why

lab.sh run s1 hid the operation it performs, so the lab taught a wrapper instead of the Azure change. Each scenario guide now leads with the real commands.

What changed

  • scenario guides open with a manual path: real az containerapp update, az role assignment delete/create, loadgen.py, alert queries, state recording, and capture
  • lab.sh run/capture moved to a labeled shortcut section at the end of each guide
  • the manual path gates every destructive step on resolved azd values and a started run, waits for the new revision, verifies recovery before recording it, and skips capture without a valid alert id or https endpoint
  • README describes the manual-first flow and the shortcut

Verification

  • doc suites: 109 passed
  • extracted 28 guide bash blocks: bash -n clean, no set -e/exit/trap
  • simulated azd failure: no Azure call issued, run not started
  • simulated injection failure: no load, no recovery mutation, run recorded as failed

Lead each scenario guide with the real az, loadgen, state and capture commands so a reader learns the operation instead of lab.sh. The manual path gates every destructive step on resolved azd values and a started run, verifies recovery before recording it, and demotes the scripts to a labeled shortcut.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates the SRE Agent event lab documentation to teach the underlying Azure operations first (manual az/tooling commands) and present lab.sh as an optional shortcut, with tests enforcing the new guide structure.

Changes:

  • Reworked scenario guides (S1–S3) to lead with “수동 실행” steps that show the real az/RBAC changes and evidence recording flow, followed by a “지름길” section using lab.sh.
  • Updated the lab README to describe the manual-first flow and the shortcut.
  • Added/updated guide validation tests to require section order and to assert key manual commands are present in the manual sections.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
monitor/sre-agent-event-lab/scripts/tests/test_lab_guides.py Adds assertions that scenario guides lead with manual Azure commands and keep manual steps runnable, plus updated required section order.
monitor/sre-agent-event-lab/README.md Documents the new manual-first approach and positions lab.sh as a shortcut.
monitor/sre-agent-event-lab/guides/02-scenario-s1.md Replaces wrapper-first instructions with a detailed manual run flow, then a shortcut section.
monitor/sre-agent-event-lab/guides/03-scenario-s2.md Same manual-first restructuring and runnable steps as S1, with updated shortcut placement.
monitor/sre-agent-event-lab/guides/04-scenario-s3.md Introduces a manual RBAC-focused flow first, then the lab.sh shortcut.
Suppressed comments (1)

monitor/sre-agent-event-lab/guides/02-scenario-s1.md:304

  • This paragraph also states that the manual run does not go through the gate, but the manual steps use begin-run which enforces the same gate. Consider rewording to distinguish the real difference (manual steps can be interrupted and do not guarantee automatic recovery) rather than implying the gate is skipped.
지름길은 순서 게이트도 적용합니다. 어떤 시나리오든 실행이 `running`이나 `failed`로 남아 있으면 세 시나리오 모두 새 실행이 거부됩니다. 세 시나리오는 같은 Container App 하나를 쓰고, 끝나지 않은 실행은 장애가 아직 살아 있을 수 있는 상태이기 때문입니다. `failed`는 그 시나리오를 다시 실행하면 풀리고, `running`은 실행이 끝나기를 기다리거나 `python3 scripts/lab_state.py mark-failed s1`처럼 끝난 방식을 기록해야 풀립니다. 수동 실행은 이 게이트를 거치지 않으므로 시나리오를 겹쳐 돌리지 않는 책임은 운영자에게 있습니다.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread monitor/sre-agent-event-lab/guides/02-scenario-s1.md Outdated
Comment thread monitor/sre-agent-event-lab/guides/03-scenario-s2.md Outdated
Comment thread monitor/sre-agent-event-lab/README.md Outdated
The manual path calls lab_state.py begin-run, so it is subject to the same ordering and duplicate-run gate. State that the real difference is recovery: the shortcut reverts through its exit trap, the manual path relies on the operator.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated no new comments.

Suppressed comments (1)

monitor/sre-agent-event-lab/guides/04-scenario-s3.md:246

  • 이 문장은 수동 실행에 “배포 출력 확인”과 “전파 대기 5분 제한” 같은 확인이 없다고 말하지만, 위 수동 실행 블록에서도 azd 값 검증(빈 값이면 단계 전체 skip)과 300초 전파 대기 제한을 이미 수행합니다. 차이는 지름길이 종료 트랩으로 복구를 보장한다는 점이므로, 문구를 수동 실행의 실제 동작(자동 복구 부재)과 일치시키는 게 좋겠습니다.
`run`은 위 순서에 더해 필요한 배포 출력이 비어 있지 않은지 삭제 전에 확인하고, 전파 대기를 최대 5분으로 제한하며, 종료 시 역할 복구를 보장합니다. 수동 실행에서는 이 확인들이 없으므로 삭제 후 복구 명령까지 반드시 직접 완료해야 합니다.

@hellices
hellices merged commit 410dc1c into main Aug 15, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants