Skip to content

ci(ios): keep every step's request log in the Settings replay smoke - #3374

Merged
thymikee merged 1 commit into
mainfrom
ci/ios-settings-smoke-debug-logs
Oct 10, 2026
Merged

thymikee merged 1 commit into
mainfrom
ci/ios-settings-smoke-debug-logs

Conversation

@thymikee

@thymikee thymikee commented Oct 10, 2026 •

Copy link
Copy Markdown
Member

Summary

Adds --debug to the iOS Settings replay smoke on CI.

Without it, a successful step's request log is never written to disk. In a failed 01-settings.ad attempt, the artifacts keep only the failing wait's log. The click before it, whose tap went to the stale point, leaves no record of its pre-tap capture or its post-open settle. #3367 needs that record to tell "the layout moved after the capture" apart from "the capture read a layout still settling".

With --debug, every step's request log is kept in the uploaded ios-artifacts. That includes post_gesture_snapshot_stabilized and post_gesture_snapshot_stabilization_timeout. Only this one replay step changes.

Trade-off: --debug also streams the daemon log into the job output, so this step's console output gets longer.

Refs #3367.

Validation

  • pnpm check:affected --run passed.
  • The uploaded artifacts already include the session requests/ folder (checked in run 37815426076), so the extra logs will be uploaded with no other change.

🤖 Generated with Claude Code

View guided diff Turn on auto-fix

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 1 file

View guided diff | Turn on auto-fix | Re-trigger cubic

@github-actions

Copy link
Copy Markdown
Contributor

Size Report

Metric Base Current Diff
Installed (including dependencies) 5.16 MB 5.16 MB 0 B
Package (unpacked) 5.16 MB 5.16 MB 0 B
Package (download) 1.55 MB 1.55 MB +5 B

Startup median (7 runs, lower is better):

Scenario Base Current Diff
CLI --version 27.5 ms 27.0 ms -0.5 ms
CLI --help 84.6 ms 82.6 ms -2.0 ms

@thymikee

Copy link
Copy Markdown
Member Author

Thanks for the change. I reviewed 84623cc and found no code problems in the diff, but I can't call it ready yet because the PR doesn't show that --debug on test writes a request log for every step, passing ones included, under --artifacts-dir. Please show one iOS Settings replay smoke run, or a local test ... --debug --artifacts-dir run, where the artifact directory holds a request log for each step, and confirm the upload step at .github/workflows/ios.yml#L175 collects that path (https://github.com/callstack/agent-device/blob/84623cc/.github/workflows/ios.yml#L175). Without that, the flag may only raise verbosity while logs stay limited to failing steps, and #3367 would still be open.

The failing check is the Android Smoke Tests job, which fails on the helper-transport assertion expecting 'persistent-session'. It is likely unrelated, because this diff only touches the iOS Settings replay step in .github/workflows/ios.yml. I judged that from the diff scope and the log excerpt, and I did not read the iOS upload step or any iOS run output. There are no conflicts. The remaining step before merge is that run output showing per-step request logs for passing steps.

Not blocking: the same evidence would also confirm what --debug does in the test command, which I did not check in source, and you can take or leave that.

@thymikee

Copy link
Copy Markdown
Member Author

Shown on this PR's own iOS run: 38062260581 at 84623cce9.

  • Upload path. "Upload iOS artifacts" uses upload-agent-device-artifacts with agent-state-dir: $AGENT_DEVICE_STATE_DIR. That action uploads ${agent-state-dir}/sessions/** (.github/actions/upload-agent-device-artifacts/action.yml:28). Request logs live there, not under --artifacts-dir.
  • Per-step logs. 01-settings.ad passed on attempt 1. The attempt's request log (sessions/…_1-01-settings_attempt-1/requests/<requestId>_attempt_1.ndjson) has 96 entries, with a request_start for each step: open, appstate, click, wait, is, find, back. All 7 steps passed. By command: open 28, click 20, wait 15, find 15, appstate 8, back 7, is 3.
  • Before this PR. On main, a failed attempt's log kept only the failing wait (3 entries in run 37815426076), and a passing attempt kept nothing.
  • Why. --debug sets the diagnostics scope's debug. emitDiagnostic then writes each event live to the request log (packages/host-kit/src/internal/diagnostics.ts:155-160). Without it, flushDiagnosticsToSessionFile drops the events of a successful request (:218).

This run's open also reported postOpenObservation: "probe-failed", which #3376 relies on. The failing Android Smoke job doesn't touch this diff.

@thymikee

Copy link
Copy Markdown
Member Author

Thanks, that settles it for 84623cc. The iOS run 38062260581 passed at this commit, and the upload step collects sessions/** from the state dir, which is where the request logs live. The source matches your trace: with --debug, each event is written to the request log as it happens, so passing steps keep their logs too. The failing Android Smoke Tests job does not touch this diff. I'm marking this ready for human review.

@thymikee thymikee added the ready-for-human Valid work that needs human implementation, judgment, or maintainer merge label Oct 10, 2026
@thymikee
thymikee merged commit 91cbba9 into main Oct 10, 2026
18 of 19 checks passed
@thymikee
thymikee deleted the ci/ios-settings-smoke-debug-logs branch October 10, 2026 16:59
@github-actions

Copy link
Copy Markdown
Contributor
PR Preview Action v1.8.1
Preview removed because the pull request was closed.
2026-10-10 17:01 UTC

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready-for-human Valid work that needs human implementation, judgment, or maintainer merge

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant