Skip to content

docs: v2 launch-prep shakedown report - #281

Closed
kjgbot wants to merge 3 commits into
mainfrom
shakedown/v2-launch-0910
Closed

kjgbot wants to merge 3 commits into
mainfrom
shakedown/v2-launch-0910

Conversation

@kjgbot

@kjgbot kjgbot commented Sep 10, 2026

Copy link
Copy Markdown
Contributor

The six-flow launch shakedown found two blockers for the requested flagship demo: local LLM execution is unavailable in both YAML and TypeScript, and YAML lacks upstream-output bindings. The report records the exact failures alongside passing deterministic/deep-cwd runs and a real Codex TypeScript agent run.

Findings are filed as #273#278; small help/summary and model-registry documentation fixes are in #279 and #280. The latest #269 observer/cloud-login follow-up was exercised: token minting works with the canonical API override, but the printed dashboard URL returns HTTP 404. Its test suite also fails 4/38 when composed with #268; that evidence was posted to #264 for the existing owner.

Validation and literal output are included: kernel release and SDK builds pass; the initial composed TypeScript/Rust checks pass; the CLI fix passes TypeScript, Rust, and all 63 existing CLI tests. The late observer suite failure is recorded explicitly. This draft PR changes only documentation, captured evidence, and the completed publication handoff; no in-flight feature branch was modified. Launch-ready verdict remains NO.

Session-Id: 01a08ba4-4473-7903-b7c3-e778edb9b523
Session-Id: 01a08ba4-4473-7903-b7c3-e778edb9b523
@coderabbitai

coderabbitai Bot commented Sep 10, 2026

Copy link
Copy Markdown

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Session-Id: 01a08ba4-4473-7903-b7c3-e778edb9b523

Session-Id: 01a08ba4-4473-7903-b7c3-e778edb9b523
@miyaontherelay
miyaontherelay force-pushed the shakedown/v2-launch-0910 branch from 1bff4fe to 2cd5164 Compare September 10, 2026 14:59
@github-actions

Copy link
Copy Markdown

Review swarm: maintainability

No fresh transcript was produced for run ade3458c-9cf6-4eab-8348-5692e8171527 (MISSING).

@github-actions

Copy link
Copy Markdown

Review swarm: history

No fresh transcript was produced for run ade3458c-9cf6-4eab-8348-5692e8171527 (MISSING).

@github-actions

Copy link
Copy Markdown

Review swarm: structure

No fresh transcript was produced for run ade3458c-9cf6-4eab-8348-5692e8171527 (MISSING).

@github-actions

Copy link
Copy Markdown

Review swarm: FAILED

  • maintainability: MISSING
  • history: MISSING
  • structure: MISSING

Cloud run: ade3458c-9cf6-4eab-8348-5692e8171527

@kjgbot

kjgbot commented Sep 10, 2026

Copy link
Copy Markdown
Contributor Author

maintainability lens — FAIL

Maintainability review — PR #281

Blockers

Latent footgun: .shakedown-actions/ACTIONS.txt + README.md split. The ACTIONS.txt file lists eight shell-style commands (flows OPEN-PR ..., flows FILE-ISSUE ...) as if executable. A one-line README.md sibling says "COMPLETED: … Do not re-run ACTIONS.txt." That's a warning in a different file than the actions themselves. Six months from now, a contributor who greps .shakedown-actions/*.txt for a "how did we file shakedown issues?" template will find eight commands with no in-band warning. Fold the disclaimer into a top-of-file comment inside ACTIONS.txt, or delete it — the outcome is already recorded in evidence/shakedown-0910/publication.json. Also note that flows OPEN-PR / flows FILE-ISSUE are not real CLI verbs (the shakedown itself proved the true CLI surface has no such verbs); nothing labels them as pseudo-instructions to a coordinator.

Two-source duplication with no canonical designation. Files under .shakedown-actions/issues/*.md, help-pr-body.md, docs-pr-body.md, report-pr-body.md are byte-identical to their counterparts under evidence/shakedown-0910/issues/, evidence/shakedown-0910/help-pr-body.md, etc. No file states which tree is source-of-truth. If a follow-up edit lands in only one place, the drift is silent.

Concerns

  • SHAKEDOWN-REPORT.md at repo root has no date in its name. The content is dated 2026-09-10, but the filename suggests a canonical rolling artifact. The next shakedown will either overwrite it or accumulate stale content. Prefer evidence/shakedown-0910/REPORT.md, or embed -0910 in the root filename.
  • Line-number citations decay. SHAKEDOWN-REPORT.md references preflight.ts:180 and observer-link.test.ts:74 (the latter in an unmerged branch). Both go stale on the next refactor with no signal.
  • evidence/shakedown-0910/publication.json has no schema. Keys mix REST forms (issues/273, pulls/279, issues/comments/5620663794). If anything programmatic reads this, the contract is implicit. A one-line comment or sibling .schema.json would fix it.
  • Failure-handling silence in captured HTTP 401 evidence. evidence/shakedown-0910/delivery-*.txt preserves failed gh invocations; DELIVERY.md explains the actions succeeded elsewhere. A contributor copying these as a template for the next shakedown will paste an HTTP 401 into their own report.

Notes

  • Live GitHub URLs (#273#281) are treated as citations. They will silently rot if issues are renumbered/deleted; consider adding "state as of 2026-09-10" beside them.
  • Absolute host paths (/Users/khaliqgant/flows-shakedown-0910/...) are unavoidable in captured evidence — fine as history, worth flagging in the report intro so a reader doesn't try to reproduce verbatim.
  • The PARKED [run_parked] and REFUSED [invalid_invocation] transcripts are exemplary — typed, unambiguous, actionable. Keep this style.

Nothing here breaks correctness; the blockers are about a stranger safely finding and changing this evidence in six months.

REVIEW_FAILED

@kjgbot

kjgbot commented Sep 10, 2026

Copy link
Copy Markdown
Contributor Author

history lens — PASS

Blockers: none. PR #281 passes the HISTORY lens.

I ran git log --oneline -40, read the repository guidance and RFC-0001, and examined the drive log’s relevant corrections, NEXT, DIRECTIVES, and all three report commit messages.

  • Repeated mistakes: I found no reintroduction of deliberately removed behavior. The report addresses the drive log’s recurring evidence-reporting mistakes: SHAKEDOWN-REPORT.md:19–21 separates earlier successful checks from the later observer failures, while evidence/shakedown-0910/followup-observer-tests.txt:83–89 preserves the command’s failing result. The “fixed and pushed” statements reference captured local/remote SHA comparisons in help-push-verification.txt:1–10 and docs-push-verification.txt:1–10.
  • Settled decisions: No new contradiction found. The observer issue explicitly requires observer failures to remain nonfatal (evidence/shakedown-0910/issues/observer-origins.md:47–53), consistent with decision flow/drive f59e279 08271341 #7’s projection boundary. The LLM follow-up preserves the distinct step type and typed verification (issues/llm-local.md:49–59). These documents introduce no runtime implementation or gate changes.
  • Commit truthfulness: The subjects of 20ee1fad, 5962d5d, and 2cd51644 accurately describe adding findings, control captures, and publication links. None claims that this PR implements the separately reported fixes or makes the launch ready.

Concerns: The missing execution/binding capabilities and composed observer-suite failures remain substantive follow-up work (SHAKEDOWN-REPORT.md:41–47). They are explicitly disclosed existing limitations, not regressions introduced by this report.

Notes: The experimental snippets are labeled evidence fixtures rather than supported examples (evidence/shakedown-0910/flow-sources.md:1–4). The publication handoff explicitly prohibits rerunning its actions (.shakedown-actions/README.md:1). This review assesses the supplied diff and recorded evidence; I did not rerun the historical product tests.

REVIEW_PASSED

@kjgbot

kjgbot commented Sep 10, 2026

Copy link
Copy Markdown
Contributor Author

structure lens — MISSING

@kjgbot

kjgbot commented Sep 10, 2026

Copy link
Copy Markdown
Contributor Author

🎯 review-swarm: FAILED (M:fail H:pass S:missing)

Lens transcripts posted as sibling comments above.

@kjgbot

kjgbot commented Sep 10, 2026

Copy link
Copy Markdown
Contributor Author

closed: shakedown report served its purpose for the 2026-09-11 launch. Branch and evidence preserved at shakedown/v2-launch-0910; all filed findings tracked as issues #273#278.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants