Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 6 additions & 2 deletions FORK.md
Original file line number Diff line number Diff line change
Expand Up @@ -59,14 +59,18 @@ Releases keep the `<upstream core>-preview.<UTC date>.<n>` naming, whatever the
1. Fetches the target (the newest stable tag; see above). If the integration branch already contains it, or is ahead of every stable tag, it verifies that the integration tree is published and retries an unfinished publication.
2. Merges a newer tag on a `sync/upstream-<UTC stamp>` branch, or re-checks an open `sync/upstream-*` PR (one a failed run opened, or an agent's conflict fix). `agentContextBudget.json` conflicts are taken from upstream and re-measured.
3. Runs format, lint, typecheck and the server tests. Every test must pass; pre-existing failures still block publication.
4. Builds the release in a container (`scripts/fork/build-release.sh`, the recipe every `v0.1.9-preview.*` release used) and smoke-tests it, then runs the built CLI's `__service-preflight` read-only against the live database for launcher protocols 3 and 4. The same output reports each configured Codex instance's shadow home; an instance whose shadow the new build would refuse to set up (`not-ready`) or could not inspect (`unknown`) stops the release, while `auto-settle` ones, which the build moves aside into a dated backup beside the shadow, do not.
4. Builds the release in a container (`scripts/fork/build-release.sh`, the recipe every `v0.1.9-preview.*` release used) and smoke-tests it, then runs the built CLI's `__service-preflight` read-only against the live database for launcher protocols 3 and 4. The release gate clears caller-provided `CODEX_HOME` and `CODEX_SQLITE_HOME` so an agent shell cannot redirect the check; per-instance environment from T3 settings still applies. The same output reports each configured Codex instance's shadow home; an instance whose shadow the new build would refuse to set up (`not-ready`) or could not inspect (`unknown`) stops the release. `auto-settle` conflicts are safe for setup: a legacy `memories_v2` link is removed without changing its shared target, while the shadow's own SQLite databases are moved to a dated backup beside the shadow.
5. When all of that and the hosted checks pass, it merges the PR and publishes `v<upstream nightly core>-preview.<UTC date>.<n>` with the tarball and `SHA256SUMS` (`scripts/fork/publish-release.sh`). The host's update guard then installs it when no threads are running.

A conflict opens (or updates) an issue labelled `upstream-sync`, or comments on the open sync PR. A failed check opens a PR that says what failed. Neither publishes anything. The scheduled agent watches a pending PR and resumes on CI/review completion, so publication need not wait until the next hourly check. An existing candidate is completed before newer upstream changes are added, preventing repeated CI resets. Exit codes: 0 done or nothing to do, 3 conflict, 4 failed check, 2 environment problem.

To release by hand from a clean, merged integration commit: `scripts/fork/build-release.sh <version>`, then `scripts/fork/publish-release.sh <version> <artifacts-dir> <merge-commit> "<notes>"`. Versions must sort above every earlier fork release under `sort -V`, because the guard installs the newest one.

The host update guard that installs these releases lives in `scripts/fork/host/t3-update-guard/` (see its README). It waits up to 30 min for an idle server, then forces the update after a bounded wait for pending approvals and in-flight tool calls. It never stops the service while the kernel shows the exit hang, and it reports which interrupted threads continued. Change it here, run `test-guard.sh`, then install it.
The host update guard that installs these releases lives in `scripts/fork/host/t3-update-guard/` (see its README). It waits up to 30 min for an idle server, then forces the update after a bounded wait for pending approvals and in-flight tool calls. It never stops the service while the kernel shows the exit hang, and it reports which interrupted threads continued. It prefers builds with a positive Codex shadow preflight, but keeps a database-compatible build running if no safer fallback is known; a shadow conflict disables only that Codex instance. Change it here, run `test-guard.sh`, then install it.

### Codex shadow compatibility

New releases no longer link the shared `memories_v2` directory into Codex shadow homes. If a prior release has already created that link, the new release removes the link and keeps the shared contents; any real, instance-local `memories_v2` directory is left in place. After Codex creates local memories, an older release may treat that directory as an unsupported shadow conflict and refuse to start that Codex instance. The server can still serve other work, and the update guard keeps a database-compatible build running. Upgrade to a release that recognizes local `memories_v2` before expecting that provider instance to start; do not remove its local memories to make an older build run.

The service's `T3CODE_RELEASE_BASE_URL` selects the fork feed for both metadata and
self-update targets. An upstream desktop's exact nightly target resolves to the
Expand Down
156 changes: 156 additions & 0 deletions apps/server/src/cloud/codexShadowPreflight.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ import * as FileSystem from "effect/FileSystem";
import * as Path from "effect/Path";

import { symlinksSupported } from "@t3tools/shared/testing/symlinks";
import * as HostProcess from "@t3tools/shared/HostProcess";
import { unmanagedProviders } from "../policy/managedProviders.ts";
import { unmanagedSettings } from "../policy/managedSettings.ts";
import { inspectConfiguredCodexShadowHomes } from "./codexShadowPreflight.ts";
Expand Down Expand Up @@ -255,6 +256,161 @@ it.layer(NodeServices.layer)("inspectConfiguredCodexShadowHomes", (it) => {
}),
);

it.effect.skipIf(!symlinksSupported)(
"uses CODEX_HOME when the shadow path is configured without homePath",
() =>
Effect.gen(function* () {
const fileSystem = yield* FileSystem.FileSystem;
const path = yield* Path.Path;
const stateDir = yield* fileSystem.makeTempDirectoryScoped({ prefix: "t3code-state-" });
const home = yield* fileSystem.makeTempDirectoryScoped({ prefix: "t3code-home-" });
const sharedHome = yield* fileSystem.makeTempDirectoryScoped({
prefix: "t3code-codex-shared-",
});
const shadowHome = yield* fileSystem.makeTempDirectoryScoped({
prefix: "t3code-codex-shadow-",
});
yield* fileSystem.writeFileString(path.join(sharedHome, "config.toml"), "shared");
yield* fileSystem.writeFileString(path.join(shadowHome, "config.toml"), "local");
yield* fileSystem.writeFileString(
path.join(stateDir, "settings.json"),
JSON.stringify({
providerInstances: {
codex_work: {
driver: "codex",
config: { shadowHomePath: shadowHome },
environment: [{ name: "CODEX_HOME", value: sharedHome }],
},
},
}),
);

const report = yield* inspectConfiguredCodexShadowHomes(stateDir).pipe(
Effect.provideService(HostProcess.HomeDirectory, home),
);

expect(report.status).toBe("not-ready");
expect(report.instances).toEqual([
{
instanceId: "codex_work",
shadowHomePath: shadowHome,
status: "not-ready",
conflicts: [
{
entryName: "config.toml",
path: path.join(shadowHome, "config.toml"),
autoSettle: false,
},
],
},
]);
}),
);

it.effect.skipIf(!symlinksSupported)(
"uses CODEX_HOME when homePath contains only whitespace",
() =>
Effect.gen(function* () {
const fileSystem = yield* FileSystem.FileSystem;
const path = yield* Path.Path;
const stateDir = yield* fileSystem.makeTempDirectoryScoped({ prefix: "t3code-state-" });
const home = yield* fileSystem.makeTempDirectoryScoped({ prefix: "t3code-home-" });
const sharedHome = yield* fileSystem.makeTempDirectoryScoped({
prefix: "t3code-codex-shared-",
});
const shadowHome = yield* fileSystem.makeTempDirectoryScoped({
prefix: "t3code-codex-shadow-",
});
yield* fileSystem.writeFileString(path.join(sharedHome, "config.toml"), "shared");
yield* fileSystem.writeFileString(path.join(shadowHome, "config.toml"), "local");
yield* fileSystem.writeFileString(
path.join(stateDir, "settings.json"),
JSON.stringify({
providerInstances: {
codex_work: {
driver: "codex",
config: { homePath: " ", shadowHomePath: shadowHome },
environment: [{ name: "CODEX_HOME", value: sharedHome }],
},
},
}),
);

const report = yield* inspectConfiguredCodexShadowHomes(stateDir).pipe(
Effect.provideService(HostProcess.HomeDirectory, home),
);

expect(report.status).toBe("not-ready");
expect(report.instances[0]).toMatchObject({
instanceId: "codex_work",
shadowHomePath: shadowHome,
status: "not-ready",
conflicts: [
{
entryName: "config.toml",
path: path.join(shadowHome, "config.toml"),
autoSettle: false,
},
],
});
}),
);

it.effect.skipIf(
!symlinksSupported ||
HostProcess.Platform.defaultValue() === "win32" ||
HostProcess.Username.defaultValue() === "root",
)("marks a shared memories_v2 link not-ready when its shadow directory is read-only", () =>
Effect.gen(function* () {
const fileSystem = yield* FileSystem.FileSystem;
const path = yield* Path.Path;
const stateDir = yield* fileSystem.makeTempDirectoryScoped({ prefix: "t3code-state-" });
const sharedHome = yield* fileSystem.makeTempDirectoryScoped({
prefix: "t3code-codex-shared-",
});
const shadowHome = yield* fileSystem.makeTempDirectoryScoped({
prefix: "t3code-codex-shadow-",
});
const sharedMemories = path.join(sharedHome, "memories_v2");
const shadowMemories = path.join(shadowHome, "memories_v2");
yield* fileSystem.makeDirectory(sharedMemories);
yield* fileSystem.symlink(sharedMemories, shadowMemories);
yield* fileSystem.writeFileString(
path.join(stateDir, "settings.json"),
JSON.stringify({
providerInstances: {
codex_work: {
driver: "codex",
config: { homePath: sharedHome, shadowHomePath: shadowHome },
},
},
}),
);
yield* fileSystem.chmod(shadowHome, 0o555);

const report = yield* inspectConfiguredCodexShadowHomes(stateDir).pipe(
Effect.ensuring(fileSystem.chmod(shadowHome, 0o755).pipe(Effect.orDie)),
);

expect(report.status).toBe("not-ready");
expect(report.instances).toEqual([
{
instanceId: "codex_work",
shadowHomePath: shadowHome,
status: "not-ready",
conflicts: [
{
entryName: "memories_v2",
path: shadowMemories,
autoSettle: false,
},
],
},
]);
expect(yield* fileSystem.readLink(shadowMemories)).toBe(sharedMemories);
}),
);

it.effect("is ready when there is no settings file", () =>
Effect.gen(function* () {
const fileSystem = yield* FileSystem.FileSystem;
Expand Down
4 changes: 2 additions & 2 deletions apps/server/src/cloud/codexShadowPreflight.ts
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ import { decodeSettingsFileContents } from "../serverSettings.ts";
export interface CodexShadowHomeReport {
readonly instanceId: string;
readonly shadowHomePath: string;
/** `auto-settle`: setup will move the shadow's own databases aside; `not-ready`: setup will refuse. */
/** `auto-settle`: setup can remove a shared link or move the shadow's own databases aside. */
readonly status: "ready" | "auto-settle" | "not-ready" | "unknown";
readonly conflicts: ReadonlyArray<{
readonly entryName: string;
Expand Down Expand Up @@ -129,7 +129,7 @@ export const inspectConfiguredCodexShadowHomes = Effect.fn("inspectConfiguredCod
config,
)
: yield* resolveCodexHomeLayout(
!config.homePath && !config.shadowHomePath && environment.CODEX_HOME?.trim()
!config.homePath?.trim() && environment.CODEX_HOME?.trim()
? { ...config, homePath: environment.CODEX_HOME }
: config,
);
Expand Down
86 changes: 86 additions & 0 deletions apps/server/src/provider/Drivers/CodexHomeLayout.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -459,11 +459,19 @@ it.layer(NodeServices.layer)("CodexHomeLayout", (it) => {

yield* fileSystem.makeDirectory(path.join(sharedHome, "log"));
yield* fileSystem.makeDirectory(path.join(sharedHome, "memories"));
yield* fileSystem.makeDirectory(path.join(sharedHome, "memories_v2"));
yield* fileSystem.makeDirectory(path.join(sharedHome, "tmp"));
yield* writeTextFile(path.join(sharedHome, "config.toml"), 'model = "gpt-5-codex"\n');
yield* writeTextFile(path.join(shadowHome, "auth.json"), '{"shadow":true}\n');
yield* fileSystem.makeDirectory(path.join(shadowHome, "log"), { recursive: true });
yield* fileSystem.makeDirectory(path.join(shadowHome, "memories"), { recursive: true });
yield* fileSystem.makeDirectory(path.join(shadowHome, "memories_v2"), {
recursive: true,
});
yield* writeTextFile(
path.join(shadowHome, "memories_v2", "memory_summary.md"),
"shadow memory",
);
yield* fileSystem.makeDirectory(path.join(shadowHome, "tmp"), { recursive: true });

const layout = yield* resolveCodexHomeLayout(
Expand All @@ -482,17 +490,68 @@ it.layer(NodeServices.layer)("CodexHomeLayout", (it) => {
const memoriesLinkResult = yield* fileSystem
.readLink(path.join(shadowHome, "memories"))
.pipe(Effect.result);
const memoriesV2LinkResult = yield* fileSystem
.readLink(path.join(shadowHome, "memories_v2"))
.pipe(Effect.result);
const tmpLinkResult = yield* fileSystem
.readLink(path.join(shadowHome, "tmp"))
.pipe(Effect.result);
const memoriesV2Content = yield* fileSystem.readFileString(
path.join(shadowHome, "memories_v2", "memory_summary.md"),
);

expect(configTarget).toBe(path.join(sharedHome, "config.toml"));
expect(logLinkResult._tag).toBe("Failure");
expect(memoriesLinkResult._tag).toBe("Failure");
expect(memoriesV2LinkResult._tag).toBe("Failure");
expect(memoriesV2Content).toBe("shadow memory");
expect(tmpLinkResult._tag).toBe("Failure");
}),
);

it.effect.skipIf(!symlinksSupported)(
"unlinks legacy memories_v2 links to the shared home without changing shared memory",
() =>
Effect.gen(function* () {
const fileSystem = yield* FileSystem.FileSystem;
const path = yield* Path.Path;
const sharedHome = yield* makeTempDir("t3code-codex-shared-");
const shadowRoot = yield* makeTempDir("t3code-codex-shadow-root-");
const shadowHome = path.join(shadowRoot, "shadow");
const sharedMemoriesV2 = path.join(sharedHome, "memories_v2");
const shadowMemoriesV2 = path.join(shadowHome, "memories_v2");
const sharedSummary = path.join(sharedMemoriesV2, "memory_summary.md");

yield* fileSystem.makeDirectory(sharedMemoriesV2, { recursive: true });
yield* fileSystem.makeDirectory(shadowHome, { recursive: true });
yield* writeTextFile(sharedSummary, "shared memory");
yield* fileSystem.symlink(sharedMemoriesV2, shadowMemoriesV2);

const layout = yield* resolveCodexHomeLayout(
decodeCodexSettings({
homePath: sharedHome,
shadowHomePath: shadowHome,
}),
);

const conflicts = yield* inspectCodexShadowHome(layout, noSqliteHome);
expect(conflicts).toContainEqual({
entryName: "memories_v2",
path: shadowMemoriesV2,
sqliteFiles: undefined,
autoSettle: true,
});
expect(yield* fileSystem.readLink(shadowMemoriesV2)).toBe(sharedMemoriesV2);

yield* materializeCodexShadowHome(layout, noSqliteHome);

expect((yield* fileSystem.readLink(shadowMemoriesV2).pipe(Effect.result))._tag).toBe(
"Failure",
);
expect(yield* fileSystem.readFileString(sharedSummary)).toBe("shared memory");
}),
);

it.effect("rejects shadow homes that point at the shared home", () =>
Effect.gen(function* () {
const sharedHome = yield* makeTempDir("t3code-codex-shared-");
Expand Down Expand Up @@ -550,6 +609,33 @@ it.layer(NodeServices.layer)("CodexHomeLayout", (it) => {
}),
);

it.effect.skipIf(!symlinksSupported)(
"leaves legacy shared memories_v2 links in place when another conflict refuses setup",
() =>
Effect.gen(function* () {
const fileSystem = yield* FileSystem.FileSystem;
const path = yield* Path.Path;
const sharedHome = yield* makeTempDir("t3code-codex-shared-");
const shadowRoot = yield* makeTempDir("t3code-codex-shadow-root-");
const shadowHome = path.join(shadowRoot, "shadow");
const sharedMemories = path.join(sharedHome, "memories_v2");
const shadowMemories = path.join(shadowHome, "memories_v2");
yield* writeTextFile(path.join(sharedHome, "config.toml"), 'model = "shared"\n');
yield* writeTextFile(path.join(shadowHome, "config.toml"), 'model = "local"\n');
yield* fileSystem.makeDirectory(sharedMemories, { recursive: true });
yield* fileSystem.makeDirectory(shadowHome, { recursive: true });
yield* fileSystem.symlink(sharedMemories, shadowMemories);

const layout = yield* resolveCodexHomeLayout(
decodeCodexSettings({ homePath: sharedHome, shadowHomePath: shadowHome }),
);
const error = yield* materializeCodexShadowHome(layout, noSqliteHome).pipe(Effect.flip);

expect(error).toBeInstanceOf(CodexShadowHomeEntryConflictError);
expect(yield* fileSystem.readLink(shadowMemories)).toBe(sharedMemories);
}),
);

// Codex wrote memories_v2_1.sqlite into the shadow before CODEX_SQLITE_HOME, and the
// shared home gained its own copy later; the next setup failed every session.
const ownDatabaseFiles = [
Expand Down
Loading
Loading