Skip to content

fix(factory): stop desktop beta lane hanging on ASC processing #641

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
JakubAnderwald merged 2 commits into from
Sep 18, 2026
Merged

fix(factory): stop desktop beta lane hanging on ASC processing #641

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
88904f0
fix(factory): stop desktop beta lane hanging on ASC processing
JakubAnderwald Sep 18, 2026
45e3e19
fix(factory): address review on lane cap and desktop notes
JakubAnderwald Sep 18, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 14 additions & 4 deletions apps/desktop/fastlane/Fastfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,10 +31,15 @@ end

load_local_secrets

def post_release_notes(max_chars:)
# `build` is the macOS build number just uploaded. post-release-notes.mjs needs
# it to target THAT build: macOS and iOS share one App Store Connect app, and iOS
# builds also carry the macOS-only fields the script used to match on.
def post_release_notes(max_chars:, build:)
Dir.chdir(PROJECT_ROOT) do
notes = sh("bash", "scripts/generate-release-notes.sh", "--max-chars", max_chars.to_s).strip
sh("node", "scripts/post-release-notes.mjs", "--platform", "macos", "--notes", notes) unless notes.empty?
next if notes.empty?

sh("node", "scripts/post-release-notes.mjs", "--platform", "macos", "--notes", notes, "--build", build.to_s)
end
rescue StandardError => e
UI.important("Release notes posting failed (non-fatal): #{e.message}")
Expand Down Expand Up @@ -291,7 +296,12 @@ platform :mac do
upload_to_testflight(
api_key: api_key,
pkg: pkg_path,
skip_waiting_for_build_processing: false
# Don't block on Apple's processing (matches mobile). Waiting hung the
# #623 pre-merge lane for 3 days: build 56 went VALID in App Store
# Connect, but fastlane's poll never saw it, so the lane never exited
# and the factory never announced the build. post-release-notes.mjs
# polls for the exact build itself.
skip_waiting_for_build_processing: true
)
else
upload_to_app_store(
Expand All @@ -304,7 +314,7 @@ platform :mac do
end

# Post release notes
post_release_notes(max_chars: 4000)
post_release_notes(max_chars: 4000, build: new_build_number)

# Phase G: comment "Now live" on closed support issues for this build.
# TestFlight (beta) and the App Store (production) ship the same macOS
Expand Down
149 changes: 112 additions & 37 deletions apps/desktop/scripts/post-release-notes.mjs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,13 @@
* Post release notes to App Store Connect (TestFlight) for the macOS desktop app.
*
* Usage:
* node post-release-notes.mjs --platform macos --notes "Release notes text"
* node post-release-notes.mjs --platform macos --notes "Release notes text" --build N
*
* --build is the macOS build number just uploaded. It is REQUIRED: macOS and iOS
* ship under the same App Store Connect app (eu.drafto.mobile), and iOS builds
* ALSO carry computedMinMacOsVersion (iPhone apps run on Apple Silicon Macs), so
* "the newest build with macOS fields" resolved to iOS build 42 and overwrote
* its notes. Match the exact number on the MAC_OS preReleaseVersion instead.
*
* Environment variables:
* ASC_API_KEY_ID - App Store Connect API Key ID
Expand All @@ -14,14 +20,17 @@

import { createSign } from "node:crypto";
import { readFileSync } from "node:fs";
import { pathToFileURL } from "node:url";

const args = process.argv.slice(2);
const notesIdx = args.indexOf("--notes");
const notes = notesIdx !== -1 ? args[notesIdx + 1] : "";

if (!notes) {
console.error("Error: --notes is required");
process.exit(1);
export function parseArgs(argv) {
// Return the token after `flag`, but treat a missing value or the next flag as
// absent — otherwise `--notes --build 29` would swallow `--build` as the notes.
const valueAfter = (flag) => {
const index = argv.indexOf(flag);
const value = index === -1 ? undefined : argv[index + 1];
return value && !value.startsWith("--") ? value : "";
};
return { notes: valueAfter("--notes"), build: valueAfter("--build") };
Comment thread
coderabbitai[bot] marked this conversation as resolved.
}

// --- App Store Connect ---
Expand Down Expand Up @@ -98,7 +107,52 @@ export const ascFetch = async (url, options = {}) => {
throw lastError;
};

async function postTestFlightNotes(releaseNotes) {
// CFBundleVersion: one to three period-separated integers. Checked up front so a
// typo fails fast instead of polling App Store Connect for 5 minutes.
export const isValidBuildNumber = (build) => /^\d+(\.\d+){0,2}$/.test(String(build));

/**
* Flatten an App Store Connect `/builds` response (data + included) into
* `{ id, version, uploadedDate, platform }` records. `platform` comes ONLY from
* the build's preReleaseVersion ("IOS" | "MAC_OS"). The macOS-only build fields
* are not a usable fallback here: iOS builds report computedMinMacOsVersion too.
*/
export function normalizeBuilds(buildsResponse) {
const preReleaseById = new Map(
(buildsResponse.included || [])
.filter((item) => item.type === "preReleaseVersions")
.map((item) => [item.id, item.attributes]),
);
return (buildsResponse.data || []).map((build) => {
const preReleaseId = build.relationships?.preReleaseVersion?.data?.id;
return {
id: build.id,
version: build.attributes?.version,
uploadedDate: build.attributes?.uploadedDate,
platform: preReleaseId ? preReleaseById.get(preReleaseId)?.platform : undefined,
};
});
}

/**
* Pick the macOS build with exactly `buildNumber`. Build numbers are not unique
* across platforms (iOS and macOS count independently), so filter to MAC_OS
* first; among survivors the most recently uploaded wins.
*/
export function selectMacBuild(builds, { buildNumber }) {
const candidates = builds.filter(
(build) => build.platform === "MAC_OS" && String(build.version) === String(buildNumber),
);
if (candidates.length === 0) {
return null;
}
const uploadedAt = (build) => (build.uploadedDate ? Date.parse(build.uploadedDate) : 0);
return candidates.reduce((newest, build) =>
uploadedAt(build) > uploadedAt(newest) ? build : newest,
);
}

async function postTestFlightNotes(releaseNotes, buildNumber) {
const keyId = process.env.ASC_API_KEY_ID;
const issuerId = process.env.ASC_API_ISSUER_ID;
const privateKeyP8 =
Expand Down Expand Up @@ -132,34 +186,41 @@ async function postTestFlightNotes(releaseNotes) {
"Content-Type": "application/json",
};

// 1. Find the latest macOS build (filter by platform-identifying fields for multi-platform app).
// Uses the top-level /v1/builds endpoint because the relationship endpoint /v1/apps/{id}/builds
// no longer accepts the `sort` query parameter.
const buildsRes = await ascFetch(
`${baseUrl}/builds?filter[app]=${appId}&sort=-uploadedDate&limit=10&fields[builds]=version,processingState,computedMinMacOsVersion,lsMinimumSystemVersion`,
{ headers },
);
if (!buildsRes.ok) {
throw new Error(`List builds failed: ${buildsRes.status} ${await buildsRes.text()}`);
}
const buildsData = await buildsRes.json();

if (!buildsData.data || buildsData.data.length === 0) {
console.error("Skipping TestFlight: no builds found");
return;
// 1. Find the exact macOS build. The lane uploads with
// skip_waiting_for_build_processing, so a fresh build may not be indexed yet —
// poll until it appears. filter[version] rather than sort-by-uploadedDate: a
// still-processing build has a null uploadedDate and sorts last.
// `preReleaseVersion` MUST be in fields[builds] or ASC omits the relationship
// linkage and the platform can't be resolved from the include.
const buildsUrl =
`${baseUrl}/builds?filter[app]=${appId}&filter[version]=${encodeURIComponent(buildNumber)}` +
`&fields[builds]=version,uploadedDate,processingState,preReleaseVersion` +
`&include=preReleaseVersion&fields[preReleaseVersions]=platform`;
const maxAttempts = 15;
let target = null;
for (let attempt = 1; attempt <= maxAttempts; attempt++) {
const buildsRes = await ascFetch(buildsUrl, { headers });
if (!buildsRes.ok) {
throw new Error(`List builds failed: ${buildsRes.status} ${await buildsRes.text()}`);
}
target = selectMacBuild(normalizeBuilds(await buildsRes.json()), { buildNumber });
if (target || attempt === maxAttempts) {
break;
}
console.log(
`TestFlight: macOS build ${buildNumber} not indexed yet (attempt ${attempt}/${maxAttempts}); retrying in 20s…`,
);
await sleep(20_000);
}

// Filter to macOS builds only (have computedMinMacOsVersion or lsMinimumSystemVersion)
const macosBuild = buildsData.data.find(
(b) => b.attributes.computedMinMacOsVersion || b.attributes.lsMinimumSystemVersion,
);

if (!macosBuild) {
console.error("Skipping TestFlight: no macOS builds found (only iOS builds present)");
if (!target) {
console.error(
`Skipping TestFlight: macOS build ${buildNumber} not found for app ${appId} after ${maxAttempts} attempts`,
);
return;
}

const buildId = macosBuild.id;
const buildId = target.id;

// 2. Check if a betaBuildLocalization already exists for en-US
const locRes = await ascFetch(
Expand Down Expand Up @@ -208,20 +269,34 @@ async function postTestFlightNotes(releaseNotes) {
}
}

console.log(
`TestFlight: "What to Test" updated for macOS build ${macosBuild.attributes.version}`,
);
console.log(`TestFlight: "What to Test" updated for macOS build ${target.version}`);
}

// --- Main ---

async function main() {
const { notes, build } = parseArgs(process.argv.slice(2));
if (!notes) {
console.error("Error: --notes is required");
process.exit(1);
}
if (!build) {
console.error("Error: --build is required so notes target THAT macOS build");
process.exit(1);
}
if (!isValidBuildNumber(build)) {
console.error(`Error: --build "${build}" is not a CFBundleVersion (e.g. 56 or 1.2.3)`);
process.exit(1);
}
try {
await postTestFlightNotes(notes);
await postTestFlightNotes(notes, build);
} catch (err) {
console.error(`TestFlight error: ${err.message}`);
process.exit(1);
}
}

main();
// Only run when invoked directly (not when imported by tests).
if (process.argv[1] && import.meta.url === pathToFileURL(process.argv[1]).href) {
main();
}
2 changes: 2 additions & 0 deletions docs/operations/factory-runbook.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -164,6 +164,7 @@ A card in **In Test** needs a build a human can actually install. `--watch` ther
| `FACTORY_INTEST_BETA_DESKTOP` | `0` | Additionally dispatch the **macOS** lane. Only turn on after the fossil validation below. |
| `FACTORY_INTEST_TIMEOUT_SEC` | `600` | Wall-clock cap for the In Test scenario writer (read-only stage). |
| `FACTORY_LANE_STALE_MIN` | `120` | A lane silent this long with no exit code is declared dead and retried. |
| `FACTORY_LANE_MAX_MIN` | `240` | A lane still running this long after ITS dispatch is killed (whole process group) and retried. |
| `FACTORY_LANE_MAX_ATTEMPTS` | `3` | Retry budget per lane per commit; a new commit resets it. |
| `DRAFTO_BETA_MOBILE_ROOT` | `/Users/jakub/code/drafto-beta-mobile` | Dedicated mobile build root. |
| `DRAFTO_DESKTOP_BUILD_ROOT` | `/Users/jakub/code/drafto-beta-desktop` | Dedicated macOS build root (clonefile replica of the fossil). |
Expand Down Expand Up @@ -233,6 +234,7 @@ cat logs/factory/beta-lane-mobile-463-b243d8196fa2.log.exit # its exit cod
| absent, log written recently | still building | left alone |
| absent, log silent > `FACTORY_LANE_STALE_MIN` | wrapper killed before it could record | treated as failed, retried |
| absent, **no log at all**, dispatched > `FACTORY_LANE_STALE_MIN` | killed before it could even open its log | treated as failed, retried (judged against `intestBetaAt`) |
| absent, log active, dispatched > `FACTORY_LANE_MAX_MIN` | hung but chatty (e.g. a stuck poll loop) | the lane's process group is killed (TERM, then KILL); treated as failed, retried |
| present but empty / non-numeric | caught mid-write | left alone; the staleness rows above are the backstop |
| `0` | succeeded | stays suppressed; if a failure was announced for this lane+commit, a retraction is posted once (`<!-- drafto-factory-beta-recovered:<lane>:<sha12> -->`) |
| non-zero, attempt < `FACTORY_LANE_MAX_ATTEMPTS` | failed, budget remaining | dropped from `intestBetaLanes`, retried next tick, reported once (`<!-- drafto-factory-beta-failed:<lane>:<sha12> -->`) |
Expand Down
Loading
Loading