Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 30 additions & 0 deletions .github/actions/build-agent-device/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
name: Build Agent Device
description: Build an immutable upstream revision for manual Apple validation
inputs:
sha:
description: Full commit SHA in callstack/agent-device
required: true
runs:
using: composite
steps:
- name: Require an immutable revision
shell: bash
env:
AGENT_DEVICE_REVISION: ${{ inputs.sha }}
run: '[[ "$AGENT_DEVICE_REVISION" =~ ^[0-9a-f]{40}$ ]]'

- uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
with:
repository: callstack/agent-device
ref: ${{ inputs.sha }}
path: .agent-device-validation
persist-credentials: false

- name: Build CLI and use its matching runner source
shell: bash
working-directory: .agent-device-validation
run: |
corepack pnpm install --frozen-lockfile
corepack pnpm build
echo "AGENT_DEVICE_BIN=$PWD/bin/agent-device.mjs" >> "$GITHUB_ENV"
echo "Agent Device source: $(git rev-parse HEAD)" >> "$GITHUB_STEP_SUMMARY"
10 changes: 10 additions & 0 deletions .github/workflows/exact-package-candidate.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,10 @@ on:
- cron: '17 3 * * *'
workflow_dispatch:
inputs:
agent_device_sha:
description: Optional full agent-device commit SHA to build for Apple validation
required: false
type: string
channel:
description: Validation channel recorded with the candidate
required: true
Expand Down Expand Up @@ -488,6 +492,12 @@ jobs:
- name: Setup
uses: ./.github/actions/setup

- name: Build the requested Agent Device revision
if: runner.os == 'macOS' && inputs.agent_device_sha != ''
uses: ./.github/actions/build-agent-device
with:
sha: ${{ inputs.agent_device_sha }}

- name: Download inspected candidate
uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4
with:
Expand Down
127 changes: 127 additions & 0 deletions .github/workflows/repro-cold-simulator-touch.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,127 @@
name: Repro cold-simulator lost touch stream

# Reproduction for the agent-device upstream report tracked in issue #84: the
# first `gesture drag` synthesized into a freshly erased, cold-booted iOS 26.5
# simulator on a hosted macos-26 runner reports ok without the expected app
# outcome. Every job erases and cold-boots the simulator several times and
# records what the app observed. Manual validation can require prompt delivery
# or reproduction; observation runs retain results without enforcing a verdict.

on:
workflow_dispatch:
inputs:
agent_device_sha:
description: Optional full agent-device commit SHA to build for Apple validation
required: false
type: string
expectation:
description: Required delivery result (observe only records evidence)
type: choice
default: observe
options: [observe, prompt, reproduced]
iterations:
description: Cold boot iterations per job
default: '3'
required: false
samples:
description: Parallel jobs per configuration
default: '4'
required: false
push:
branches:
- docs/issue-84-agent-device-upstream-handoff
paths:
- .github/workflows/repro-cold-simulator-touch.yml
- scripts/repro-cold-simulator-touch.mjs

concurrency:
group: repro-cold-simulator-touch-${{ github.ref }}-${{ inputs.agent_device_sha || 'release' }}
cancel-in-progress: false

permissions:
contents: read

jobs:
plan:
name: Plan matrix
runs-on: ubuntu-latest
outputs:
samples: ${{ steps.plan.outputs.samples }}
steps:
- id: plan
run: |
count='${{ inputs.samples || '4' }}'
echo "samples=$(node -e "console.log(JSON.stringify(Array.from({length: Number('$count')}, (_, i) => i + 1)))")" >> "$GITHUB_OUTPUT"

repro:
name: ${{ matrix.label }} / sample ${{ matrix.sample }}
needs: plan
runs-on: ${{ matrix.runner }}
timeout-minutes: 120
strategy:
fail-fast: false
matrix:
sample: ${{ fromJSON(needs.plan.outputs.samples) }}
include:
- label: ios26-macos-26
runner: macos-26
xcode: '26'
runtime: iOS-26-5
runtime_display: iOS 26.5
exclude: []
env:
RCT_USE_RN_DEP: 1
RCT_USE_PREBUILT_RNCORE: 1
steps:
- name: Checkout
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0

- name: Setup
uses: ./.github/actions/setup

- name: Build the requested Agent Device revision
if: runner.os == 'macOS' && inputs.agent_device_sha != ''
uses: ./.github/actions/build-agent-device
with:
sha: ${{ inputs.agent_device_sha }}

- name: Use appropriate Xcode version
if: matrix.xcode != ''
uses: maxim-lobanov/setup-xcode@60606e260d2fc5762a71e64e74b2174e8ea3c8bd # v1.6.0
with:
xcode-version: ${{ matrix.xcode }}

- name: Verify simulator runtime
run: xcrun simctl list runtimes | grep -F '${{ matrix.runtime_display }}'

- name: Cache cocoapods
uses: actions/cache@5a3ec84eff668545956fd18022155c47e93e2684 # v4.2.3
with:
path: example/ios/Pods
key: ${{ runner.os }}-contract-cocoapods-${{ hashFiles('example/ios/Podfile.lock') }}
restore-keys: |
${{ runner.os }}-contract-cocoapods-

- name: Install cocoapods
run: |
cd example
bundle install
bundle exec pod install --project-directory=ios

- name: Build the Scenario Lab (same Release build as the device-contract job)
run: |
cd example/ios
xcodebuild -workspace ReorderableExample.xcworkspace -scheme ReorderableExample -configuration Release -sdk iphonesimulator -derivedDataPath build CODE_SIGNING_ALLOWED=NO | tail -n 40
test -d build/Build/Products/Release-iphonesimulator/ReorderableExample.app

- name: Reproduce
run: node scripts/repro-cold-simulator-touch.mjs --runtime '${{ matrix.runtime }}' --iterations '${{ inputs.iterations || '3' }}' --out artifacts/repro-cold-touch --expect '${{ inputs.expectation || 'observe' }}'

- name: Upload evidence
if: always()
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4
with:
name: repro-cold-touch-${{ matrix.label }}-sample-${{ matrix.sample }}
if-no-files-found: warn
retention-days: 30
path: artifacts/repro-cold-touch/
4 changes: 4 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -108,3 +108,7 @@ artifacts/candidate/

# Local agent worktrees
.claude/worktrees/

# Source builds and cold-touch validation evidence
.agent-device-validation/
artifacts/repro-cold-touch*/
12 changes: 12 additions & 0 deletions docs/adr/0006-automated-publication-confidence.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,3 +33,15 @@ Amended 2026-09-04 ([#65](https://github.com/thiagobrez/react-native-reorderable
| `android.fallback` | Detox |

The iOS 26 cell moves to Detox because the first pointer gesture the Agent Device XCTest runner synthesizes into a freshly erased, cold-booted iOS 26.5 simulator on hosted runners arrives seconds late or never while injection reports success ([#54](https://github.com/thiagobrez/react-native-reorderable/issues/54)) — the pipeline's dominant failure, and one no widened gesture timing or scenario retry may paper over, because a silent input drop is indistinguishable from a broken contract. That cell's unique responsibility is proving automatic engine selection picks the fallback engine on a runtime without native SwiftUI reordering, and Detox pointer synthesis proves that equally: scenario definitions, terminal public-outcome assertions, and the four-engine parity comparison are unchanged. Agent Device is retained on iOS 27, where its selector-targeted center drag is the only harness that expresses insertion against native SwiftUI rows.

Amended 2026-09-06: restore Agent Device for all five `ios26.auto-fallback`
pointer scenarios, using the same semantic target plans as `ios27.fallback`.
The restoration is being validated with the cold-start synthesis fix in
[callstack/agent-device#2362](https://github.com/callstack/agent-device/pull/2362).
Detox continues to own accessibility actions and in-flight lifecycle mutations,
Android pointer scenarios, and native iOS multi-selection. The package-candidate
workflow still owns the four-engine contract and parity checks; PR gate placement,
exact-tarball validation, render backstops, and the single retained infrastructure
retry (including runner-cache cleanup) retain their existing responsibilities.
Cold-start delivery and the normal contract must pass before this restoration is
ready to merge; a successful warm-simulator run alone is insufficient.
22 changes: 22 additions & 0 deletions docs/handoffs/agent-device-upstream-reports/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
# Agent Device cold-simulator investigation (issue #84)

The active report is [callstack/agent-device#2362](https://github.com/callstack/agent-device/pull/2362). [Downstream PR #101](https://github.com/thiagobrez/react-native-reorderable/pull/101) restores Agent Device for the five iOS 26 pointer scenarios and allows validation against an immutable upstream source revision.

Read the [current proposal and evidence](./fix-proposal.md) and [reproduction rules](./reproduction.md) before reusing earlier claims. The original handoff overstated both the cause and the fix. Missing app outcomes are established in retained runs; a persistent digitizer-attachment defect and a successful warm-up cure are not.

## Current status

- The maintainer's structural and regression-test requests are implemented. The PR remains draft pending a controlled cold-start comparison showing improvement.
- iOS 27 fallback and native lanes passed the latest completed full matrix; iOS 26 failed before gestures during toolchain discovery. Earlier iOS 26 runs include both successful gestures and missing outcomes. See the [explicit environment comparison](https://github.com/callstack/agent-device/pull/2362#issuecomment-5581718543).
- The separate Android smoke-positioning correction [#2369](https://github.com/callstack/agent-device/pull/2369) passed its checks and was merged by the maintainer.
- The second proposed report (`dyld` / `libcurl`) was dropped as unconfirmed. Do not reopen it from the original handoff.

## Evidence and historical material

The [original report draft](./report-1-ios26-cold-simulator-lost-touch-stream.md) is historical material, not a current paste-ready report. Its transport explanation, public-XCTest control claim and video-derived latency estimates are superseded by the reassessment.

`stage-evidence.sh` retrieves the original recordings and logs from runs 33488489650 and 33355396042. Their original artifact expiry dates are September 15 and September 14, 2026. Retain downloaded copies before expiry. New reproduction runs retain their evidence for 30 days and are linked from the upstream PR.

Video frames establish visible outcomes. Similar recording and command durations do not establish clock synchronization: do not subtract a command wall timestamp from video presentation time to estimate physical touch latency.

Issue [#80](https://github.com/thiagobrez/react-native-reorderable/issues/80) tracks adopting a subsequent release. Maintainer approval and passing downstream validation are still required; this handoff does not authorize merging or releasing.
41 changes: 41 additions & 0 deletions docs/handoffs/agent-device-upstream-reports/fix-proposal.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
# Proposed agent-device synthesized-input preparation

[Upstream PR #2362](https://github.com/callstack/agent-device/pull/2362) is a proposed fix, not a shipped or confirmed fix. [CI restoration PR #101](https://github.com/thiagobrez/react-native-reorderable/pull/101) restores Agent Device for the five iOS 26 pointer scenarios while preserving the exact-candidate gate, existing whole-job retry, render backstop, parity checks and Pods cache fix.

## Current implementation

At upstream `211f232ee85df5742e3be093a008b46f80a6e1a7`, the shared iOS synthesized-event constructor attempts one empty event record before constructing the first real input record. Preparation adds no pointer paths or app-delivering contact, uses the already-resolved orientation and process ID, and adds no accessibility, screenshot or frame lookup. Only successful preparation sets the process-wide flag; a failure does not suppress the real action and allows a later attempt.

The maintainer requested this shared, contact-free boundary in place of the original gesture-only warm-up tap. The native regression exercises the actual bridge across gesture, scroll, drag, tap, swipe and sequence entry points. It observes an initial preparation failure, a later successful preparation and no further empty records. The real event record is created after preparation so its timestamps are fresh.

## What the evidence establishes

- The native ordering regression fails without the change and passes with it. This establishes ordering and retry semantics, not a cure for missing input.
- Local runs of the same Objective-C implementation passed three cold boots, nine gestures and three selector presses. A subsequent one-boot check of the corrected measurement harness also passed.
- [Hosted XCTest nightly](https://github.com/callstack/agent-device/actions/runs/34060558728) passes at `211f232ee`, including the isolated regression and the source-derived full-suite count of 226. [iOS smoke](https://github.com/callstack/agent-device/actions/runs/34058548288/attempts/2) and [CI/coverage](https://github.com/callstack/agent-device/actions/runs/34058548253) pass at that SHA.
- All downstream device, consumer, runtime and parity gates passed in [34056536337](https://github.com/thiagobrez/react-native-reorderable/actions/runs/34056536337), using `1cf7e5f1cfa7ff96ac5dd781b8af90fc6037457e` (identical Objective-C implementation). iOS 26 used its existing whole-job retry after an initial scenario-readiness failure. The workflow's optional final npm dry-run failed because version 1.0.0 was already published; the entire run is therefore not green.
- The later complete matrix [34060592647](https://github.com/thiagobrez/react-native-reorderable/actions/runs/34060592647) at `211f232ee` failed: iOS 27 fallback/native and Android passed, but both iOS 26 preparation attempts hit five-second Xcode toolchain-query timeouts. No iOS 26 gestures were measured in that run. Parity therefore could not pass.

Comparable hosted cold-start evidence and maintainer approval remain required. The fresh same-harness [base](https://github.com/thiagobrez/react-native-reorderable/actions/runs/34060459388) and [candidate](https://github.com/thiagobrez/react-native-reorderable/actions/runs/34060458003) both failed validation. Base first gestures were lost/error/prompt; candidate first gestures were error/error/prompt. The candidate also had an unobserved post-relaunch outcome after successful preparation. These are not a usable green comparison. [Detailed results](https://github.com/callstack/agent-device/pull/2362#issuecomment-5562425331) retain the setup and observation limitations.

## Withdrawn claims

In the candidate's unobserved post-relaunch gesture, UIKit logged touch dispatches after an initial delay of about 2.3 seconds from virtual-digitizer attachment. Passing sampled gestures reached initial app dispatch in roughly 10–33 ms. This contradicts complete non-delivery in that sample and suggests delayed/coalesced delivery may disrupt gesture recognition; touch-phase evidence is still needed. See the [native dispatch comparison](https://github.com/callstack/agent-device/pull/2362#issuecomment-5562449851).

The original persistent-digitizer explanation is contradicted by traces: digitizers attach and detach per real gesture, and empty preparation does not attach one. Preparation may affect other XCTest state, but that mechanism remains unproven.

The later `press id="engine-fallback"` probe uses private synthesis in both release 0.20.10 and the candidate (`fallbackAttempted=false`). It is not an independent public-XCTest control. A later successful input also cannot exclude an earlier simulator stall.

Post-command observation waits are not physical touch-delivery latency. A viewport or runner error cannot establish a missing app outcome. Video presentation timestamps must not be aligned with command wall time without an independently verified synchronization point.

See the [current measurement rules](./reproduction.md#measurement-limits-and-current-validation), [trace correction](https://github.com/callstack/agent-device/pull/2362#issuecomment-5561886720), and [historical proposal](https://github.com/thiagobrez/react-native-reorderable/blob/afdde2e0867beae448ec9784a641bbdd6cbdbde4/docs/handoffs/agent-device-upstream-reports/fix-proposal.md). The historical contact-based proposal and causal claims are superseded.

## Independent Android smoke blocker

Upstream Android smoke failed before input because a fixed scroll moved the press canary above the viewport. [PR #2369](https://github.com/callstack/agent-device/pull/2369) isolates a bounded visibility-based test-positioning correction. It changes no runtime code; [hosted Android smoke](https://github.com/callstack/agent-device/actions/runs/34061497846) and all other checks pass at `6211bfc3f15789fbb8cf5fd5907d399a7dd58d54`. The maintainer approved and merged it on September 6, 2026.

## iOS 26 versus iOS 27

The closest control is iOS 27 **fallback**: it explicitly selects the same engine that iOS 26 selects automatically, with the same app, scenarios, semantic gesture plans and Agent Device revision. The environment combinations differ: iOS 26.5 uses `macos-26` with Xcode 26; iOS 27.0 uses `xcode-27` with Xcode 27. The completed jobs also report different host macOS versions. This is an observed reliability difference between those environments, not proof that the simulator OS alone causes it.

The subsequent [touch-phase diagnostic run](https://github.com/thiagobrez/react-native-reorderable/actions/runs/34062782494) measured no gestures: two preparation errors and an app setup timeout. Its observation-only green status is not fix validation.
Loading
Loading