Repository navigation
feat(orcad): items 2–7 — degradation, natives, daemon, ops, deploy #16398
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
11 commits
Select commit
Hold shift + click to select a range
c90442d
fix(ports): stop joining an undefined resourcesPath on a non-Electron…
nwparker 0a2933a
feat(orcad): prove node-pty loads before anything requires it
nwparker 01a42af
test(orcad): pin the node-pty precondition to ground truth, not a pre…
nwparker 0c98311
feat(orcad): run the terminal daemon, and the ops contract around it
nwparker 35c99ea
feat(orcad): deploy, activate and roll back a versioned orcad install
nwparker fa89133
test(orcad): assert the precondition is self-consistent, not environm…
nwparker c3561b0
fix(orcad): drop an eslint-disable the rule no longer needs
nwparker 200f297
test(orcad): separate slot placement from the load verdict
nwparker a092599
test(orcad): gate the load-dependent cases on a real load, not on the…
nwparker b2ff8cc
test(orcad): gate on the whole premise — binding AND spawn-helper
nwparker 46954d9
fix(orcad): preserve degradation types after rebase
nwparker File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,212 @@ | ||
| #!/usr/bin/env node | ||
| /** | ||
| * Build one node-pty prebuilt for the CURRENT platform/arch/libc and file it in orcad's | ||
| * prebuilds matrix, so a deployment target needs no C/C++ toolchain. | ||
| * | ||
| * node-pty is the only ABI-sensitive native module orcad requires. It is also PATCHED in | ||
| * this repo (config/patches/node-pty@1.1.0.patch), and that patch is the glibc-floor fix: | ||
| * `.symver` pins on openpty/forkpty/pthread_sigmask plus the `--no-as-needed` ldflags that | ||
| * keep libutil/libpthread in DT_NEEDED. An upstream prebuilt has none of it and reproduces | ||
| * #9902. So the matrix is compiled from patched sources here, and this script refuses to | ||
| * run if the patch is not in the tree it is about to compile. | ||
| * | ||
| * orcad pins its own Node runtime, so the ABI dimension is fixed and the matrix varies | ||
| * only platform/arch/libc: | ||
| * linux-x64-glibc, linux-arm64-glibc, linux-x64-musl, linux-arm64-musl, | ||
| * darwin-x64, darwin-arm64 | ||
| * | ||
| * CI runs this once per slot, each inside the container that owns that libc/arch, and | ||
| * merges the resulting `out/orcad/prebuilds` trees. `--slot=<name>` forces the label so | ||
| * the glibc/musl distinction is recorded from the container rather than detected. | ||
| * | ||
| * Usage: | ||
| * node config/scripts/build-orcad-prebuilds.mjs [--slot=linux-x64-musl] | ||
| * node config/scripts/build-orcad-prebuilds.mjs --require-slots # release gate | ||
| */ | ||
| import { copyFileSync, existsSync, mkdirSync, readFileSync, writeFileSync } from 'node:fs' | ||
| import { createRequire } from 'node:module' | ||
| import { dirname, join } from 'node:path' | ||
| import process from 'node:process' | ||
| import { spawnSync } from 'node:child_process' | ||
|
|
||
| const require = createRequire(import.meta.url) | ||
| const ROOT = join(import.meta.dirname, '..', '..') | ||
| const PREBUILDS_DIR = join(ROOT, 'out', 'orcad', 'prebuilds') | ||
|
|
||
| /** Every slot a shipped matrix must fill. The single source of truth for the matrix. */ | ||
| export const MATRIX_SLOTS = [ | ||
| 'linux-x64-glibc', | ||
| 'linux-arm64-glibc', | ||
| 'linux-x64-musl', | ||
| 'linux-arm64-musl', | ||
| 'darwin-x64', | ||
| 'darwin-arm64' | ||
| ] | ||
|
|
||
| /** | ||
| * Why the report header and not `ldd`: `glibcVersionRuntime` is present only when the | ||
| * process is linked against glibc, and musl images have no `ldd` worth parsing. | ||
| */ | ||
| export function detectLibc(platform = process.platform, header = readReportHeader()) { | ||
| if (platform !== 'linux') { | ||
| return 'none' | ||
| } | ||
| return header && typeof header === 'object' && 'glibcVersionRuntime' in header ? 'glibc' : 'musl' | ||
| } | ||
|
|
||
| function readReportHeader() { | ||
| try { | ||
| return process.report?.getReport?.()?.header | ||
| } catch { | ||
| return undefined | ||
| } | ||
| } | ||
|
|
||
| export function slotName(argv = process.argv, platform = process.platform, arch = process.arch) { | ||
| const forced = argv.find((arg) => arg.startsWith('--slot=')) | ||
| if (forced) { | ||
| return forced.slice('--slot='.length) | ||
| } | ||
| const libc = detectLibc(platform) | ||
| return libc === 'none' ? `${platform}-${arch}` : `${platform}-${arch}-${libc}` | ||
| } | ||
|
|
||
| /** | ||
| * The patch is what holds the Ubuntu 20.04 floor. Compiling without it produces a binary | ||
| * that loads fine on the build host and dies on the target — the exact failure the matrix | ||
| * exists to prevent, now baked into a shipped artifact instead of a first-connect error. | ||
| */ | ||
| export function assertNodePtyPatchApplied(nodePtyDir) { | ||
| const bindingGyp = readFileSync(join(nodePtyDir, 'binding.gyp'), 'utf8') | ||
| const ptySource = readFileSync(join(nodePtyDir, 'src', 'unix', 'pty.cc'), 'utf8') | ||
| const missing = [] | ||
| if (!bindingGyp.includes('--no-as-needed,-l:libutil.so.1')) { | ||
| missing.push("binding.gyp is missing the '--no-as-needed,-l:libutil.so.1' ldflag") | ||
| } | ||
| if (!ptySource.includes('.symver openpty,openpty@')) { | ||
| missing.push('src/unix/pty.cc is missing the .symver glibc pins') | ||
| } | ||
| if (missing.length > 0) { | ||
| throw new Error( | ||
| [ | ||
| '[orcad-prebuilds] refusing to build: config/patches/node-pty@1.1.0.patch is not applied.', | ||
| ...missing.map((line) => ` - ${line}`), | ||
| 'A prebuilt compiled without it will not load on Ubuntu 20.04 (see', | ||
| 'docs/reference/linux-glibc-compatibility.md and #9902). Run `pnpm install` to apply patches.' | ||
| ].join('\n') | ||
| ) | ||
| } | ||
| } | ||
|
|
||
| export function readManifest(prebuildsDir) { | ||
| try { | ||
| return JSON.parse(readFileSync(join(prebuildsDir, 'manifest.json'), 'utf8')) | ||
| } catch { | ||
| return null | ||
| } | ||
| } | ||
|
|
||
| /** | ||
| * Why merge rather than overwrite: CI builds one slot per container and merges the trees. | ||
| * A manifest that records only the last slot would erase every other container's record, | ||
| * and `--require-slots` would then reject a complete matrix. | ||
| */ | ||
| export function mergeManifest(existing, next) { | ||
| const slots = new Set([...(existing?.slots ?? []), next.slot]) | ||
| return { | ||
| module: 'node-pty', | ||
| version: next.version, | ||
| nodeAbi: next.nodeAbi, | ||
| slots: [...slots].sort() | ||
| } | ||
| } | ||
|
|
||
| function nodePtyDir() { | ||
| return dirname(require.resolve('node-pty/package.json')) | ||
| } | ||
|
|
||
| function compileNodePty(dir) { | ||
| const built = join(dir, 'build', 'Release', 'pty.node') | ||
| if (existsSync(built)) { | ||
| console.log(`[orcad-prebuilds] reusing existing build at ${built}`) | ||
| return built | ||
| } | ||
| console.log('[orcad-prebuilds] compiling node-pty from patched source ...') | ||
| const result = spawnSync(process.platform === 'win32' ? 'npx.cmd' : 'npx', ['node-gyp', 'rebuild'], { | ||
| cwd: dir, | ||
| stdio: 'inherit', | ||
| env: process.env, | ||
| windowsHide: true | ||
| }) | ||
| if (result.status !== 0) { | ||
| throw new Error(`[orcad-prebuilds] node-gyp rebuild failed (status ${result.status})`) | ||
| } | ||
| if (!existsSync(built)) { | ||
| throw new Error(`[orcad-prebuilds] node-gyp succeeded but ${built} is missing`) | ||
| } | ||
| return built | ||
| } | ||
|
|
||
| function requireSlots() { | ||
| const manifest = readManifest(PREBUILDS_DIR) | ||
| const have = new Set(manifest?.slots ?? []) | ||
| const missing = MATRIX_SLOTS.filter((slot) => !have.has(slot)) | ||
| if (missing.length > 0) { | ||
| console.error( | ||
| `[orcad-prebuilds] matrix incomplete — missing ${missing.join(', ')}. ` + | ||
| 'Hosts on those slots fall back to a source build and need a C/C++ toolchain.' | ||
| ) | ||
| process.exitCode = 1 | ||
| return | ||
| } | ||
| console.log(`[orcad-prebuilds] matrix complete — ${MATRIX_SLOTS.length} slots`) | ||
| } | ||
|
|
||
| function build() { | ||
| const dir = nodePtyDir() | ||
| assertNodePtyPatchApplied(dir) | ||
| const slot = slotName() | ||
| const slotDir = join(PREBUILDS_DIR, slot) | ||
| mkdirSync(slotDir, { recursive: true }) | ||
|
|
||
| const builtBinary = compileNodePty(dir) | ||
| copyFileSync(builtBinary, join(slotDir, 'pty.node')) | ||
| console.log(`[orcad-prebuilds] stored ${slot}/pty.node`) | ||
|
|
||
| // Why spawn-helper ships too: on Unix node-pty posix_spawns build/Release/spawn-helper, | ||
| // so a slot without it installs cleanly and then fails ENOENT the first time a user | ||
| // opens a terminal. Windows has no spawn-helper. | ||
| if (process.platform !== 'win32') { | ||
| const helperSource = join(dirname(builtBinary), 'spawn-helper') | ||
| if (!existsSync(helperSource)) { | ||
| throw new Error(`[orcad-prebuilds] spawn-helper missing at ${helperSource}`) | ||
| } | ||
| copyFileSync(helperSource, join(slotDir, 'spawn-helper')) | ||
| console.log(`[orcad-prebuilds] stored ${slot}/spawn-helper`) | ||
| } | ||
|
|
||
| // The static floor gate, applied to the artifact we are about to ship rather than only | ||
| // to the packaged desktop app. objdump is Linux-only, which is where the floor lives. | ||
| if (process.platform === 'linux') { | ||
| const { verifyLinuxGlibcFloor } = require('./verify-linux-glibc-floor.cjs') | ||
| verifyLinuxGlibcFloor(slotDir) | ||
| } | ||
|
|
||
| const manifest = mergeManifest(readManifest(PREBUILDS_DIR), { | ||
| slot, | ||
| version: require('node-pty/package.json').version, | ||
| nodeAbi: process.versions.modules | ||
| }) | ||
| writeFileSync(join(PREBUILDS_DIR, 'manifest.json'), `${JSON.stringify(manifest, null, 2)}\n`) | ||
| console.log( | ||
| `[orcad-prebuilds] manifest: node-pty ${manifest.version}, ABI ${manifest.nodeAbi}, slots ${manifest.slots.join(', ')}` | ||
| ) | ||
| } | ||
|
|
||
| if (process.argv[1] && process.argv[1].endsWith('build-orcad-prebuilds.mjs')) { | ||
| if (process.argv.includes('--require-slots')) { | ||
| requireSlots() | ||
| } else { | ||
| build() | ||
| } | ||
| } | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,121 @@ | ||
| import { mkdirSync, mkdtempSync, rmSync, writeFileSync } from 'node:fs' | ||
| import { tmpdir } from 'node:os' | ||
| import { join } from 'node:path' | ||
| import { afterEach, describe, expect, it } from 'vitest' | ||
| import { | ||
| assertNodePtyPatchApplied, | ||
| detectLibc, | ||
| MATRIX_SLOTS, | ||
| mergeManifest, | ||
| readManifest, | ||
| slotName | ||
| } from './build-orcad-prebuilds.mjs' | ||
|
|
||
| const PATCHED_BINDING_GYP = "'ldflags': ['-Wl,--no-as-needed,-l:libutil.so.1,-l:libpthread.so.0,--as-needed']" | ||
| const PATCHED_PTY_CC = '__asm__(".symver openpty,openpty@" ORCA_GLIBC_COMPAT_VERSION);' | ||
|
|
||
| const dirs = [] | ||
| const stage = (bindingGyp, ptyCc) => { | ||
| const dir = mkdtempSync(join(tmpdir(), 'orcad-prebuild-src-')) | ||
| dirs.push(dir) | ||
| mkdirSync(join(dir, 'src', 'unix'), { recursive: true }) | ||
| writeFileSync(join(dir, 'binding.gyp'), bindingGyp) | ||
| writeFileSync(join(dir, 'src', 'unix', 'pty.cc'), ptyCc) | ||
| return dir | ||
| } | ||
| afterEach(() => { | ||
| for (const dir of dirs.splice(0)) { | ||
| rmSync(dir, { recursive: true, force: true }) | ||
| } | ||
| }) | ||
|
|
||
| describe('assertNodePtyPatchApplied', () => { | ||
| it('accepts a tree with both halves of the glibc-floor fix', () => { | ||
| expect(() => | ||
| assertNodePtyPatchApplied(stage(PATCHED_BINDING_GYP, PATCHED_PTY_CC)) | ||
| ).not.toThrow() | ||
| }) | ||
|
|
||
| it('refuses to build when the ldflags half is missing', () => { | ||
| // The .symver pins alone let gcc's --as-needed drop libutil/libpthread from | ||
| // DT_NEEDED, which loads on the build host and fails on Ubuntu 20.04 — #9902 again, | ||
| // this time baked into a shipped prebuilt. | ||
| expect(() => assertNodePtyPatchApplied(stage("'ldflags': []", PATCHED_PTY_CC))).toThrow( | ||
| /--no-as-needed,-l:libutil\.so\.1/ | ||
| ) | ||
| }) | ||
|
|
||
| it('refuses to build when the .symver pins are missing', () => { | ||
| expect(() => assertNodePtyPatchApplied(stage(PATCHED_BINDING_GYP, '// upstream'))).toThrow( | ||
| /\.symver glibc pins/ | ||
| ) | ||
| }) | ||
|
|
||
| it('names the patch and the doc so the fix is findable', () => { | ||
| expect(() => assertNodePtyPatchApplied(stage("'ldflags': []", '// upstream'))).toThrow( | ||
| /config\/patches\/node-pty@1\.1\.0\.patch/ | ||
| ) | ||
| }) | ||
| }) | ||
|
|
||
| describe('slot naming', () => { | ||
| it('covers every platform orcad ships to', () => { | ||
| expect([...MATRIX_SLOTS].sort()).toEqual([ | ||
| 'darwin-arm64', | ||
| 'darwin-x64', | ||
| 'linux-arm64-glibc', | ||
| 'linux-arm64-musl', | ||
| 'linux-x64-glibc', | ||
| 'linux-x64-musl' | ||
| ]) | ||
| }) | ||
|
|
||
| it('lets CI force the label so the container decides glibc vs musl', () => { | ||
| // Detection inside a container that happens to run a differently-linked Node would | ||
| // file the build under the wrong slot. The forced label must beat detection outright, | ||
| // so assert against one detection could never produce for this platform/arch. | ||
| expect(slotName(['node', 'x', '--slot=linux-x64-glibc'], 'linux', 'arm64')).toBe( | ||
| 'linux-x64-glibc' | ||
| ) | ||
| }) | ||
|
|
||
| it('omits the libc dimension off Linux', () => { | ||
| expect(slotName([], 'darwin', 'arm64')).toBe('darwin-arm64') | ||
| }) | ||
|
|
||
| it('reads glibc from the report header and musl from its absence', () => { | ||
| expect(detectLibc('linux', { glibcVersionRuntime: '2.31' })).toBe('glibc') | ||
| expect(detectLibc('linux', {})).toBe('musl') | ||
| expect(detectLibc('darwin', { glibcVersionRuntime: '2.31' })).toBe('none') | ||
| }) | ||
| }) | ||
|
|
||
| describe('mergeManifest', () => { | ||
| it('accumulates slots across the per-container CI runs that build them', () => { | ||
| // Overwriting would erase every other container's record, and the release gate would | ||
| // then reject a matrix that is actually complete. | ||
| const first = mergeManifest(null, { slot: 'linux-x64-glibc', version: '1.1.0', nodeAbi: '127' }) | ||
| const second = mergeManifest(first, { | ||
| slot: 'linux-arm64-musl', | ||
| version: '1.1.0', | ||
| nodeAbi: '127' | ||
| }) | ||
|
|
||
| expect(second.slots).toEqual(['linux-arm64-musl', 'linux-x64-glibc']) | ||
| expect(second).toMatchObject({ module: 'node-pty', version: '1.1.0', nodeAbi: '127' }) | ||
| }) | ||
|
|
||
| it('does not duplicate a slot rebuilt twice', () => { | ||
| const once = mergeManifest(null, { slot: 'darwin-arm64', version: '1.1.0', nodeAbi: '127' }) | ||
| expect(mergeManifest(once, { slot: 'darwin-arm64', version: '1.1.0', nodeAbi: '127' }).slots) | ||
| .toEqual(['darwin-arm64']) | ||
| }) | ||
| }) | ||
|
|
||
| describe('readManifest', () => { | ||
| it('returns null instead of throwing when no matrix has been built', () => { | ||
| const dir = mkdtempSync(join(tmpdir(), 'orcad-prebuild-manifest-')) | ||
| dirs.push(dir) | ||
| expect(readManifest(dir)).toBeNull() | ||
| }) | ||
| }) |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🗄️ Data Integrity & Integration | 🟠 Major | 🏗️ Heavy lift
🔎 Supported by static analysis
🏁 Script executed:
Repository: stablyai/orca
Length of output: 7251
🏁 Script executed:
Repository: stablyai/orca
Length of output: 50370
🏁 Script executed:
Repository: stablyai/orca
Length of output: 16823
🏁 Script executed:
Repository: stablyai/orca
Length of output: 50369
🏁 Script executed:
Repository: stablyai/orca
Length of output: 50372
Validate forced slot metadata before merging.
slotName()accepts a complete arbitrary slot label, so an arm64 build can populatelinux-x64-glibc. A later x64 host can copy that binary because the slot and Node ABI checks pass, then fail to load it. Derive platform and architecture from the build host, allow only the libc override, and rejectmergeManifest()inputs with differentnodeAbiornode-ptyversions.