From 3e37a15e95e04029d052b0820b800e8f0200678b Mon Sep 17 00:00:00 2001 From: z2z23n0 Date: Sat, 18 Jul 2026 19:43:56 +0800 Subject: [PATCH] feat: add Codex profile migration --- README.md | 43 ++ README.zh-CN.md | 37 ++ cmd/capsule/main.go | 5 +- cmd/capsule/main_test.go | 6 + cmd/capsule/profile.go | 229 +++++++++ internal/profile/clone.go | 119 +++++ internal/profile/discover.go | 97 ++++ internal/profile/export.go | 548 ++++++++++++++++++++++ internal/profile/fetch.go | 167 +++++++ internal/profile/fs.go | 312 +++++++++++++ internal/profile/import.go | 780 +++++++++++++++++++++++++++++++ internal/profile/profile_test.go | 434 +++++++++++++++++ internal/profile/schedule.go | 157 +++++++ internal/profile/serve.go | 94 ++++ internal/profile/types.go | 208 +++++++++ internal/profile/verify.go | 169 +++++++ skills/agent-capsule/SKILL.md | 133 +++++- 17 files changed, 3536 insertions(+), 2 deletions(-) create mode 100644 cmd/capsule/profile.go create mode 100644 internal/profile/clone.go create mode 100644 internal/profile/discover.go create mode 100644 internal/profile/export.go create mode 100644 internal/profile/fetch.go create mode 100644 internal/profile/fs.go create mode 100644 internal/profile/import.go create mode 100644 internal/profile/profile_test.go create mode 100644 internal/profile/schedule.go create mode 100644 internal/profile/serve.go create mode 100644 internal/profile/types.go create mode 100644 internal/profile/verify.go diff --git a/README.md b/README.md index b03e0a4..f94cc6e 100644 --- a/README.md +++ b/README.md @@ -29,6 +29,11 @@ restore it into their own agent and keep working. Agent Capsule currently supports Codex and Claude Code export/import, including cross-agent artifact imports through share links and zip capsules. +It also supports controlled Codex profile migration between Macs: selected +projects and their native threads, user configuration, skills, memories, and +automations can be moved while preserving the target Mac's authentication and +device identity. + Codex image uploads referenced by a session are preserved. Agent Capsule does not package arbitrary non-image files yet. @@ -126,6 +131,44 @@ capsule verify --target codex --home ~/.codex --thread --target- capsule verify --target claude --home ~/.claude --thread --target-cwd . ``` +## Codex profile migration + +Profile migration is separate from session handoff. It preserves selected +native thread ids and uses controlled overwrite semantics for a newly installed +target Codex. Project working trees are recreated with Git; uncommitted and +untracked files are not copied. + +```bash +capsule profile export \ + --target-home /Users//.codex \ + --target-workspace /Users//workspace \ + --project /path/to/project-a \ + --git-bundle-fallback \ + --out ~/.codex/profile-migrations/ +capsule profile serve ~/.codex/profile-migrations/ --listen :8765 +``` + +On the target Mac: + +```bash +capsule profile fetch --out ~/.codex/profile-migrations/ +capsule profile clone ~/.codex/profile-migrations/ --execute +capsule profile import ~/.codex/profile-migrations/ --home ~/.codex +capsule profile schedule-import ~/.codex/profile-migrations/ --home ~/.codex --execute +``` + +After Codex reopens: + +```bash +capsule profile verify ~/.codex/profile-migrations/ --home ~/.codex +capsule profile unschedule ~/.codex/profile-migrations/ --home ~/.codex --execute +``` + +The one-shot LaunchAgent uses `KeepAlive=false`, removes its plist after the +attempt, and writes `import-status.json`. The profile allowlist excludes auth, +provider tokens, installation/device ids, cookies, Keychain data, managed +plugins, caches, logs, worktrees, and `skills/.system`. + ## Privacy commitments For link sharing, Agent Capsule encrypts the capsule locally before upload. The diff --git a/README.zh-CN.md b/README.zh-CN.md index ba6e26c..589d165 100644 --- a/README.zh-CN.md +++ b/README.zh-CN.md @@ -20,6 +20,8 @@ Agent Capsule 会把这段会话打包成一个可以一键导入的胶囊。接 Agent Capsule 目前支持 Codex 和 Claude Code 的导出/导入，也支持通过分享链接和 zip 胶囊进行跨 agent artifact 导入。 +它也支持在两台 Mac 之间做受控的 Codex profile 迁移：迁移选定项目及其原生任务、用户配置、skills、memories 和 automations，同时保留目标 Mac 自己的登录态和设备身份。 + Codex 会话里引用的图片上传会被保留。Agent Capsule 目前还不会打包任意非图片文件。 同源导入会创建新的原生 thread/session。跨 agent 导入会保留可见对话、工具证据、工作上下文，并把源 agent 的 raw transcript 写入 sidecar 供后续深挖；它不迁移 provider credential、登录态、云端状态、文件系统 checkpoint 或 agent 私有加密状态。 @@ -98,6 +100,41 @@ capsule verify --target codex --home ~/.codex --thread --target- capsule verify --target claude --home ~/.claude --thread --target-cwd . ``` +## Codex Profile 迁移 + +Profile 迁移和单任务交接是两套语义。它会保留选中任务原来的 thread id，并对新装目标 Codex 的受控白名单做覆盖。项目工作树由 Git 在目标机器重建，不复制未提交和未跟踪文件。 + +源 Mac： + +```bash +capsule profile discover +capsule profile export \ + --target-home /Users//.codex \ + --target-workspace /Users//workspace \ + --project /path/to/project-a \ + --git-bundle-fallback \ + --out ~/.codex/profile-migrations/ +capsule profile serve ~/.codex/profile-migrations/ --listen :8765 +``` + +目标 Mac： + +```bash +capsule profile fetch <带令牌的源地址> --out ~/.codex/profile-migrations/ +capsule profile clone ~/.codex/profile-migrations/ --execute +capsule profile import ~/.codex/profile-migrations/ --home ~/.codex +capsule profile schedule-import ~/.codex/profile-migrations/ --home ~/.codex --execute +``` + +Codex 重新打开后： + +```bash +capsule profile verify ~/.codex/profile-migrations/ --home ~/.codex +capsule profile unschedule ~/.codex/profile-migrations/ --home ~/.codex --execute +``` + +一次性 LaunchAgent 使用 `KeepAlive=false`，执行后会删除自己的 plist，并写入 `import-status.json`。迁移白名单明确排除认证、provider token、安装/设备 id、Cookie、Keychain、托管插件、缓存、日志、worktree 和 `skills/.system`。 + ## 隐私承诺 链接分享时，Agent Capsule 会先在本机加密胶囊再上传。托管服务、Worker、R2 bucket 或 S3 兼容 bucket 只会收到加密后的胶囊字节和加密后的预览 payload。没有 `#k=...` fragment key，这些服务无法解密会话内容。 diff --git a/cmd/capsule/main.go b/cmd/capsule/main.go index 3be7e96..2e0ea16 100644 --- a/cmd/capsule/main.go +++ b/cmd/capsule/main.go @@ -37,6 +37,8 @@ func run(args []string) error { return runImport(args[1:]) case "verify": return runVerify(args[1:]) + case "profile": + return runProfile(args[1:]) case "help", "-h", "--help": usage() return nil @@ -216,5 +218,6 @@ Usage: capsule import session.capsule.zip --target claude --target-cwd . --execute capsule import "https://example.workers.dev/s/share-id#k=..." --target codex --target-cwd . --execute capsule verify --target codex --home ~/.codex --thread --target-cwd . - capsule verify --target claude --home ~/.claude --thread --target-cwd .`) + capsule verify --target claude --home ~/.claude --thread --target-cwd . + capsule profile help`) } diff --git a/cmd/capsule/main_test.go b/cmd/capsule/main_test.go index 69f8ca2..920aa50 100644 --- a/cmd/capsule/main_test.go +++ b/cmd/capsule/main_test.go @@ -24,6 +24,12 @@ func TestHandoffCommandRemoved(t *testing.T) { } } +func TestProfileHelpCommand(t *testing.T) { + if err := run([]string{"profile", "help"}); err != nil { + t.Fatal(err) + } +} + func TestImportCommandOpensRestoredCodexThread(t *testing.T) { sourceHome, threadID := createFakeCodexHome(t) out := filepath.Join(t.TempDir(), "session.capsule.zip") diff --git a/cmd/capsule/profile.go b/cmd/capsule/profile.go new file mode 100644 index 0000000..47ebb11 --- /dev/null +++ b/cmd/capsule/profile.go @@ -0,0 +1,229 @@ +package main + +import ( + "flag" + "fmt" + "os" + "path/filepath" + "strings" + + "github.com/z2z23n0/agent-capsule/internal/profile" +) + +type stringList []string + +func (values *stringList) String() string { return strings.Join(*values, ",") } +func (values *stringList) Set(value string) error { + *values = append(*values, value) + return nil +} + +func runProfile(args []string) error { + if len(args) == 0 { + profileUsage() + return flag.ErrHelp + } + switch args[0] { + case "discover": + return runProfileDiscover(args[1:]) + case "export": + return runProfileExport(args[1:]) + case "fetch": + return runProfileFetch(args[1:]) + case "clone": + return runProfileClone(args[1:]) + case "import": + return runProfileImport(args[1:]) + case "verify": + return runProfileVerify(args[1:]) + case "schedule-import": + return runProfileSchedule(args[1:]) + case "unschedule": + return runProfileUnschedule(args[1:]) + case "serve": + return runProfileServe(args[1:]) + case "help", "-h", "--help": + profileUsage() + return nil + default: + return fmt.Errorf("unknown profile command %q", args[0]) + } +} + +func runProfileDiscover(args []string) error { + fs := flag.NewFlagSet("profile discover", flag.ContinueOnError) + home := fs.String("home", "", "source Codex home") + if err := fs.Parse(args); err != nil { + return err + } + result, err := profile.Discover(profile.DiscoverOptions{Home: *home}) + if err != nil { + return err + } + return printJSON(result) +} + +func runProfileExport(args []string) error { + fs := flag.NewFlagSet("profile export", flag.ContinueOnError) + home := fs.String("home", "", "source Codex home") + targetHome := fs.String("target-home", "", "target Codex home") + targetWorkspace := fs.String("target-workspace", "", "target project workspace") + out := fs.String("out", "", "output profile bundle directory") + unsafe := fs.Bool("unsafe-include-secrets", false, "allow high-confidence secrets in allowlisted profile files") + gitBundles := fs.Bool("git-bundle-fallback", false, "prepare committed Git history as a private-clone fallback") + var projects stringList + fs.Var(&projects, "project", "project root to migrate; repeat for each project") + if err := fs.Parse(args); err != nil { + return err + } + result, err := profile.Export(profile.ExportOptions{Home: *home, TargetHome: *targetHome, TargetWorkspace: *targetWorkspace, Projects: projects, Out: *out, UnsafeIncludeSecrets: *unsafe, GitBundleFallback: *gitBundles}) + if err != nil { + return err + } + return printJSON(result) +} + +func positionalFirst(name string, args []string, configure func(*flag.FlagSet), run func(string, *flag.FlagSet) error) error { + fs := flag.NewFlagSet(name, flag.ContinueOnError) + configure(fs) + positional := "" + parseArgs := args + if len(args) > 0 && !strings.HasPrefix(args[0], "-") { + positional = args[0] + parseArgs = args[1:] + } + if err := fs.Parse(parseArgs); err != nil { + return err + } + if positional == "" && fs.NArg() == 1 { + positional = fs.Arg(0) + } else if fs.NArg() != 0 { + return fmt.Errorf("usage: capsule %s [flags]", name) + } + if positional == "" { + return fmt.Errorf("usage: capsule %s [flags]", name) + } + return run(positional, fs) +} + +func runProfileFetch(args []string) error { + var out *string + var includeGitBundles *bool + return positionalFirst("profile fetch", args, func(fs *flag.FlagSet) { + out = fs.String("out", "", "target profile bundle directory") + includeGitBundles = fs.Bool("include-git-bundles", false, "fetch private-clone fallback bundles") + }, func(source string, _ *flag.FlagSet) error { + result, err := profile.Fetch(profile.FetchOptions{Source: source, Out: *out, IncludeGitBundles: *includeGitBundles}) + if err != nil { + return err + } + return printJSON(result) + }) +} + +func runProfileClone(args []string) error { + var execute *bool + return positionalFirst("profile clone", args, func(fs *flag.FlagSet) { + execute = fs.Bool("execute", false, "clone project repositories and checkout exported commits") + }, func(bundle string, _ *flag.FlagSet) error { + result, err := profile.CloneProjects(profile.CloneOptions{BundleDir: bundle, Execute: *execute}) + if err != nil { + return err + } + return printJSON(result) + }) +} + +func runProfileImport(args []string) error { + var home *string + var execute *bool + return positionalFirst("profile import", args, func(fs *flag.FlagSet) { + home = fs.String("home", "", "target Codex home") + execute = fs.Bool("execute", false, "apply the controlled overwrite") + }, func(bundle string, _ *flag.FlagSet) error { + result, err := profile.Import(profile.ImportOptions{BundleDir: bundle, Home: *home, Execute: *execute, RequireStopped: *execute}) + if err != nil { + return err + } + return printJSON(result) + }) +} + +func runProfileVerify(args []string) error { + var home *string + return positionalFirst("profile verify", args, func(fs *flag.FlagSet) { + home = fs.String("home", "", "target Codex home") + }, func(bundle string, _ *flag.FlagSet) error { + result, err := profile.Verify(profile.VerifyOptions{BundleDir: bundle, Home: *home}) + if err != nil { + return err + } + if err := printJSON(result); err != nil { + return err + } + if result.Status != "ok" { + return fmt.Errorf("profile verification failed") + } + return nil + }) +} + +func runProfileSchedule(args []string) error { + var home, cli *string + var execute *bool + return positionalFirst("profile schedule-import", args, func(fs *flag.FlagSet) { + home = fs.String("home", "", "target Codex home") + cli = fs.String("cli", "", "capsule executable path") + execute = fs.Bool("execute", false, "bootstrap the one-shot LaunchAgent") + }, func(bundle string, _ *flag.FlagSet) error { + result, err := profile.ScheduleImport(profile.ScheduleOptions{BundleDir: bundle, Home: *home, CLIPath: *cli, Submit: *execute}) + if err != nil { + return err + } + return printJSON(result) + }) +} + +func runProfileUnschedule(args []string) error { + var home *string + var execute *bool + return positionalFirst("profile unschedule", args, func(fs *flag.FlagSet) { + home = fs.String("home", "", "target Codex home") + execute = fs.Bool("execute", false, "unload and remove the LaunchAgent plist") + }, func(bundle string, _ *flag.FlagSet) error { + result, err := profile.Unschedule(profile.UnscheduleOptions{BundleDir: bundle, Home: *home, Submit: *execute}) + if err != nil { + return err + } + return printJSON(result) + }) +} + +func runProfileServe(args []string) error { + var listen *string + return positionalFirst("profile serve", args, func(fs *flag.FlagSet) { + listen = fs.String("listen", ":8765", "listen address") + }, func(bundle string, _ *flag.FlagSet) error { + server, err := profile.NewServer(bundle, *listen) + if err != nil { + return err + } + if err := printJSON(map[string]any{"status": "serving", "bundle_dir": filepath.Clean(bundle), "urls": server.URLs, "pid": os.Getpid()}); err != nil { + return err + } + return server.Serve() + }) +} + +func profileUsage() { + fmt.Fprintln(os.Stderr, `Codex profile migration commands: + capsule profile discover [--home ~/.codex] + capsule profile export --project ... --target-home /Users//.codex --target-workspace /Users//workspace --out + capsule profile serve --listen :8765 + capsule profile fetch --out ~/.codex/profile-migrations/ + capsule profile clone [--execute] + capsule profile import --home ~/.codex [--execute] + capsule profile schedule-import --home ~/.codex [--execute] + capsule profile verify --home ~/.codex + capsule profile unschedule [--execute]`) +} diff --git a/internal/profile/clone.go b/internal/profile/clone.go new file mode 100644 index 0000000..f380184 --- /dev/null +++ b/internal/profile/clone.go @@ -0,0 +1,119 @@ +package profile + +import ( + "fmt" + "os" + "os/exec" + "path/filepath" +) + +type CloneOptions struct { + BundleDir string + Execute bool +} + +type CloneResult struct { + Status string `json:"status"` + DryRun bool `json:"dry_run"` + Actions []CloneAction `json:"actions"` + Cloned int `json:"cloned"` + Reused int `json:"reused"` +} + +func CloneProjects(opts CloneOptions) (*CloneResult, error) { + manifest, err := readManifest(opts.BundleDir) + if err != nil { + return nil, err + } + actions := BuildClonePlan(manifest) + result := &CloneResult{Status: "planned", DryRun: !opts.Execute, Actions: actions} + if !opts.Execute { + return result, nil + } + for _, action := range actions { + if action.NeedsBundle { + return nil, fmt.Errorf("cannot clone %s: %s", action.TargetPath, action.BundleReason) + } + repo := repoForTarget(manifest, action.TargetPath) + created := false + if isGitRepo(action.TargetPath) { + result.Reused++ + if action.Commit != "" && !gitHasCommit(action.TargetPath, action.Commit) && action.Remote != "" { + _ = exec.Command("git", "-C", action.TargetPath, "fetch", "--all", "--tags").Run() + } + } else { + if _, err := os.Stat(action.TargetPath); err == nil { + return nil, fmt.Errorf("target exists but is not a Git repository: %s", action.TargetPath) + } + if err := os.MkdirAll(filepath.Dir(action.TargetPath), 0o755); err != nil { + return nil, err + } + cloneSource := action.Remote + if cloneSource == "" { + cloneSource = localBundlePath(opts.BundleDir, repo) + } + output, cloneErr := exec.Command("git", "clone", cloneSource, action.TargetPath).CombinedOutput() + if cloneErr != nil && action.Remote != "" && repo.BundlePath != "" { + _ = os.RemoveAll(action.TargetPath) + bundle := localBundlePath(opts.BundleDir, repo) + if !fileMatches(bundle, repo.BundleSHA256, repo.BundleBytes) { + return nil, fmt.Errorf("git clone %s failed: %w: %s; rerun profile fetch with --include-git-bundles", action.Remote, cloneErr, output) + } + cloneSource = bundle + output, cloneErr = exec.Command("git", "clone", cloneSource, action.TargetPath).CombinedOutput() + } + if cloneErr != nil { + if repo.BundlePath != "" && !fileMatches(localBundlePath(opts.BundleDir, repo), repo.BundleSHA256, repo.BundleBytes) { + return nil, fmt.Errorf("Git fallback is not present for %s; rerun profile fetch with --include-git-bundles", action.TargetPath) + } + return nil, fmt.Errorf("git clone %s: %w: %s", cloneSource, cloneErr, output) + } + result.Cloned++ + created = true + } + if created && action.Commit != "" && !gitHasCommit(action.TargetPath, action.Commit) && repo.BundlePath != "" { + bundle := localBundlePath(opts.BundleDir, repo) + if !fileMatches(bundle, repo.BundleSHA256, repo.BundleBytes) { + return nil, fmt.Errorf("commit %s is not available from %s; rerun profile fetch with --include-git-bundles", action.Commit, action.Remote) + } + if err := os.RemoveAll(action.TargetPath); err != nil { + return nil, err + } + if output, err := exec.Command("git", "clone", bundle, action.TargetPath).CombinedOutput(); err != nil { + return nil, fmt.Errorf("git clone fallback bundle: %w: %s", err, output) + } + } + if action.Commit != "" { + if output, err := exec.Command("git", "-C", action.TargetPath, "checkout", action.Commit).CombinedOutput(); err != nil { + return nil, fmt.Errorf("checkout %s in %s: %w: %s", action.Commit, action.TargetPath, err, output) + } + } + } + result.Status = "ok" + return result, nil +} + +func gitHasCommit(repo, commit string) bool { + return exec.Command("git", "-C", repo, "cat-file", "-e", commit+"^{commit}").Run() == nil +} + +func repoForTarget(manifest *Manifest, target string) GitRepo { + for _, project := range manifest.Projects { + for _, repo := range project.Repos { + if repo.TargetPath == target { + return repo + } + } + } + return GitRepo{} +} + +func localBundlePath(bundleDir string, repo GitRepo) string { + path, _ := safeJoin(bundleDir, repo.BundlePath) + return path +} + +func isGitRepo(path string) bool { + command := exec.Command("git", "-C", path, "rev-parse", "--is-inside-work-tree") + return command.Run() == nil +} diff --git a/internal/profile/discover.go b/internal/profile/discover.go new file mode 100644 index 0000000..a9f40ed --- /dev/null +++ b/internal/profile/discover.go @@ -0,0 +1,97 @@ +package profile + +import ( + "database/sql" + "encoding/json" + "fmt" + "os" + "path/filepath" + "sort" + + _ "modernc.org/sqlite" +) + +type DiscoverOptions struct { + Home string +} + +type ProjectCandidate struct { + ID string `json:"id"` + Name string `json:"name"` + Path string `json:"path"` + Threads int `json:"threads"` + GitRepos int `json:"git_repos"` + AllClean bool `json:"all_clean"` + Available bool `json:"available"` +} + +type DiscoverResult struct { + Status string `json:"status"` + Home string `json:"home"` + Projects []ProjectCandidate `json:"projects"` +} + +func Discover(opts DiscoverOptions) (*DiscoverResult, error) { + home, err := resolveHome(opts.Home) + if err != nil { + return nil, err + } + data, err := os.ReadFile(filepath.Join(home, ".codex-global-state.json")) + if err != nil { + return nil, err + } + var state map[string]any + if err := json.Unmarshal(data, &state); err != nil { + return nil, err + } + localProjects := mapValue(state["local-projects"]) + result := &DiscoverResult{Status: "ok", Home: home} + for id, raw := range localProjects { + project := mapValue(raw) + name := stringValue(project["name"]) + for _, root := range stringSlice(project["rootPaths"]) { + candidate := ProjectCandidate{ID: id, Name: name, Path: root} + if candidate.Name == "" { + candidate.Name = filepath.Base(root) + } + if info, statErr := os.Stat(root); statErr == nil && info.IsDir() { + candidate.Available = true + described, describeErr := describeProjects([]string{root}, filepath.Dir(root)) + if describeErr == nil && len(described) == 1 { + candidate.GitRepos = len(described[0].Repos) + candidate.AllClean = candidate.GitRepos > 0 + for _, repo := range described[0].Repos { + candidate.AllClean = candidate.AllClean && repo.Clean + } + } + } + result.Projects = append(result.Projects, candidate) + } + } + if err := addThreadCounts(filepath.Join(home, "state_5.sqlite"), result.Projects); err != nil { + return nil, err + } + sort.Slice(result.Projects, func(i, j int) bool { + if result.Projects[i].Name == result.Projects[j].Name { + return result.Projects[i].Path < result.Projects[j].Path + } + return result.Projects[i].Name < result.Projects[j].Name + }) + return result, nil +} + +func addThreadCounts(dbPath string, projects []ProjectCandidate) error { + db, err := sql.Open("sqlite", dbPath) + if err != nil { + return err + } + defer db.Close() + for index := range projects { + path := filepath.Clean(projects[index].Path) + prefix := path + string(filepath.Separator) + if err := db.QueryRow("select count(*) from threads where cwd = ? or substr(cwd, 1, ?) = ?", path, len(prefix), prefix).Scan(&projects[index].Threads); err != nil { + return fmt.Errorf("count threads for %s: %w", path, err) + } + } + return nil +} diff --git a/internal/profile/export.go b/internal/profile/export.go new file mode 100644 index 0000000..452f8eb --- /dev/null +++ b/internal/profile/export.go @@ -0,0 +1,548 @@ +package profile + +import ( + "database/sql" + "encoding/json" + "errors" + "fmt" + "io/fs" + "os" + "os/exec" + "path/filepath" + "sort" + "strings" + "time" + + "github.com/google/uuid" + _ "modernc.org/sqlite" +) + +func Export(opts ExportOptions) (*ExportResult, error) { + home, err := resolveHome(opts.Home) + if err != nil { + return nil, err + } + if len(opts.Projects) == 0 { + return nil, errors.New("at least one --project is required; migration scope must be explicit") + } + if opts.TargetWorkspace == "" { + return nil, errors.New("missing --target-workspace") + } + targetWorkspace, err := filepath.Abs(opts.TargetWorkspace) + if err != nil { + return nil, err + } + targetHome := opts.TargetHome + if targetHome == "" { + targetHome = filepath.Join(filepath.Dir(targetWorkspace), ".codex") + } + projects, err := describeProjects(opts.Projects, targetWorkspace) + if err != nil { + return nil, err + } + id := uuid.NewString() + out := opts.Out + if out == "" { + out = filepath.Join(home, "profile-migrations", id) + } + out, err = filepath.Abs(out) + if err != nil { + return nil, err + } + if entries, readErr := os.ReadDir(out); readErr == nil && len(entries) > 0 { + return nil, fmt.Errorf("output directory is not empty: %s", out) + } else if readErr != nil && !isNotExist(readErr) { + return nil, readErr + } + if err := os.MkdirAll(out, 0o700); err != nil { + return nil, err + } + profileFiles, err := exportProfileFiles(home, out, opts.UnsafeIncludeSecrets) + if err != nil { + return nil, err + } + threads, err := exportThreads(home, out, projects) + if err != nil { + return nil, err + } + if opts.GitBundleFallback { + if err := exportGitBundles(out, projects); err != nil { + return nil, err + } + } + manifest := &Manifest{ + Schema: Schema, + ID: id, + CreatedAt: time.Now().UTC(), + SourceHome: home, + TargetHome: targetHome, + SourceUserHome: filepath.Dir(home), + TargetUserHome: filepath.Dir(targetHome), + SourceWorkspace: commonProjectParent(projects), + TargetWorkspace: targetWorkspace, + Projects: projects, + Threads: threads, + ProfileFiles: profileFiles, + Exclusions: append([]string(nil), DefaultExclusions...), + } + manifest.Stats.Projects = len(projects) + manifest.Stats.Threads = len(threads) + manifest.Stats.ProfileFiles = len(profileFiles) + for _, project := range projects { + manifest.Stats.GitRepos += len(project.Repos) + for _, repo := range project.Repos { + manifest.Stats.Bytes += repo.BundleBytes + } + } + for _, file := range profileFiles { + manifest.Stats.Bytes += file.Bytes + } + for _, thread := range threads { + manifest.Stats.Bytes += thread.Bytes + } + manifestPath := filepath.Join(out, "manifest.json") + if err := writeJSON(manifestPath, manifest); err != nil { + return nil, err + } + return &ExportResult{ + Status: "ok", + BundleDir: out, + ManifestPath: manifestPath, + Manifest: manifest, + ClonePlan: BuildClonePlan(manifest), + }, nil +} + +func exportGitBundles(out string, projects []Project) error { + for projectIndex := range projects { + project := &projects[projectIndex] + for repoIndex := range project.Repos { + repo := &project.Repos[repoIndex] + name := project.ID + if repo.RelativePath != "" { + name += "-" + strings.NewReplacer("/", "-", "\\", "-").Replace(repo.RelativePath) + } + relative := filepath.ToSlash(filepath.Join("git-bundles", name+".bundle")) + target, err := safeJoin(out, relative) + if err != nil { + return err + } + if err := os.MkdirAll(filepath.Dir(target), 0o755); err != nil { + return err + } + repoRoot := filepath.Join(project.SourcePath, filepath.FromSlash(repo.RelativePath)) + if output, err := exec.Command("git", "-C", repoRoot, "bundle", "create", target, "--all").CombinedOutput(); err != nil { + return fmt.Errorf("create Git bundle for %s: %w: %s", project.Name, err, output) + } + hash, size, err := fileHash(target) + if err != nil { + return err + } + repo.BundlePath = relative + repo.BundleSHA256 = hash + repo.BundleBytes = size + } + } + return nil +} + +func describeProjects(paths []string, targetWorkspace string) ([]Project, error) { + seen := map[string]bool{} + seenTargets := map[string]bool{} + projects := make([]Project, 0, len(paths)) + for _, path := range paths { + absolute, err := filepath.Abs(path) + if err != nil { + return nil, err + } + info, err := os.Stat(absolute) + if err != nil { + return nil, fmt.Errorf("project %s: %w", absolute, err) + } + if !info.IsDir() { + return nil, fmt.Errorf("project is not a directory: %s", absolute) + } + if seen[absolute] { + continue + } + seen[absolute] = true + name := filepath.Base(absolute) + targetPath := filepath.Join(targetWorkspace, name) + if seenTargets[targetPath] { + return nil, fmt.Errorf("multiple project roots map to the same target path: %s", targetPath) + } + seenTargets[targetPath] = true + project := Project{ + ID: name, + Name: name, + SourcePath: absolute, + TargetPath: targetPath, + } + project.Repos, err = discoverGitRepos(project) + if err != nil { + return nil, err + } + projects = append(projects, project) + } + sort.Slice(projects, func(i, j int) bool { return projects[i].SourcePath < projects[j].SourcePath }) + return projects, nil +} + +func discoverGitRepos(project Project) ([]GitRepo, error) { + var roots []string + err := filepath.WalkDir(project.SourcePath, func(path string, entry fs.DirEntry, walkErr error) error { + if walkErr != nil { + return walkErr + } + if !entry.IsDir() { + return nil + } + rel, err := filepath.Rel(project.SourcePath, path) + if err != nil { + return err + } + depth := 0 + if rel != "." { + depth = len(strings.Split(filepath.ToSlash(rel), "/")) + } + if depth > 2 { + return filepath.SkipDir + } + if entry.Name() == ".git" { + roots = append(roots, filepath.Dir(path)) + return filepath.SkipDir + } + if entry.Name() == "node_modules" || entry.Name() == "vendor" || entry.Name() == ".cache" { + return filepath.SkipDir + } + return nil + }) + if err != nil { + return nil, err + } + var repos []GitRepo + for _, root := range roots { + rel, _ := filepath.Rel(project.SourcePath, root) + if rel == "." { + rel = "" + } + repo := GitRepo{ + RelativePath: filepath.ToSlash(rel), + TargetPath: filepath.Join(project.TargetPath, rel), + Branch: gitOutput(root, "branch", "--show-current"), + Commit: gitOutput(root, "rev-parse", "HEAD"), + Remotes: map[string]string{}, + Clean: gitOutput(root, "status", "--porcelain") == "", + } + remoteLines := strings.Split(gitOutput(root, "remote", "-v"), "\n") + for _, line := range remoteLines { + fields := strings.Fields(line) + if len(fields) >= 3 && fields[2] == "(fetch)" { + repo.Remotes[fields[0]] = fields[1] + } + } + repos = append(repos, repo) + } + sort.Slice(repos, func(i, j int) bool { return repos[i].RelativePath < repos[j].RelativePath }) + return repos, nil +} + +func gitOutput(root string, args ...string) string { + command := exec.Command("git", append([]string{"-C", root}, args...)...) + data, err := command.Output() + if err != nil { + return "" + } + return strings.TrimSpace(string(data)) +} + +func exportProfileFiles(home, out string, unsafe bool) ([]File, error) { + var result []File + for _, relative := range DefaultProfileDatabases { + source := filepath.Join(home, relative) + if _, err := os.Stat(source); isNotExist(err) { + continue + } else if err != nil { + return nil, err + } + bundleRel := filepath.ToSlash(filepath.Join("profile", relative)) + target, err := safeJoin(out, bundleRel) + if err != nil { + return nil, err + } + if err := snapshotSQLite(source, target); err != nil { + return nil, fmt.Errorf("snapshot %s: %w", relative, err) + } + hash, size, err := fileHash(target) + if err != nil { + return nil, err + } + result = append(result, File{TargetRelativePath: relative, BundlePath: bundleRel, SHA256: hash, Bytes: size, Mode: 0o600}) + } + for _, relative := range DefaultProfilePaths { + source := filepath.Join(home, relative) + info, err := os.Lstat(source) + if isNotExist(err) { + continue + } + if err != nil { + return nil, err + } + if info.Mode()&os.ModeSymlink != 0 { + linkTarget, err := os.Readlink(source) + if err != nil { + return nil, err + } + result = append(result, File{TargetRelativePath: relative, Mode: uint32(os.ModeSymlink), LinkTarget: linkTarget}) + continue + } + if info.Mode().IsRegular() { + if !unsafe { + if err := scanFileForSecrets(source); err != nil { + return nil, err + } + } + bundleRel := filepath.ToSlash(filepath.Join("profile", relative)) + target, err := safeJoin(out, bundleRel) + if err != nil { + return nil, err + } + hash, size, err := copyAndHash(source, target, info.Mode()) + if err != nil { + return nil, err + } + result = append(result, File{TargetRelativePath: relative, BundlePath: bundleRel, SHA256: hash, Bytes: size, Mode: uint32(info.Mode().Perm())}) + continue + } + files, err := copyTreeFiles(source, out, relative, unsafe) + if err != nil { + return nil, err + } + result = append(result, files...) + } + sort.Slice(result, func(i, j int) bool { return result[i].TargetRelativePath < result[j].TargetRelativePath }) + return result, nil +} + +func snapshotSQLite(source, target string) error { + if err := os.MkdirAll(filepath.Dir(target), 0o755); err != nil { + return err + } + _ = os.Remove(target) + db, err := sql.Open("sqlite", source) + if err != nil { + return err + } + defer db.Close() + escaped := strings.ReplaceAll(target, "'", "''") + _, err = db.Exec("VACUUM INTO '" + escaped + "'") + return err +} + +func exportThreads(home, out string, projects []Project) ([]Thread, error) { + dbPath := filepath.Join(home, "state_5.sqlite") + db, err := sql.Open("sqlite", dbPath) + if err != nil { + return nil, err + } + defer db.Close() + rows, err := db.Query("select * from threads") + if err != nil { + return nil, fmt.Errorf("read Codex threads: %w", err) + } + defer rows.Close() + columns, err := rows.Columns() + if err != nil { + return nil, err + } + index, err := readSessionIndex(filepath.Join(home, "session_index.jsonl")) + if err != nil && !isNotExist(err) { + return nil, err + } + paths, err := mapSessionPaths(home) + if err != nil { + return nil, err + } + var threads []Thread + for rows.Next() { + row, err := scanRow(rows, columns) + if err != nil { + return nil, err + } + cwd := stringValue(row["cwd"]) + project, ok := pathWithin(cwd, projects) + if !ok { + continue + } + id := stringValue(row["id"]) + sourceSession := stringValue(row["rollout_path"]) + if sourceSession == "" { + sourceSession = paths[id] + } + if sourceSession == "" { + return nil, fmt.Errorf("session file not found for selected thread %s", id) + } + rel, err := filepath.Rel(home, sourceSession) + if err != nil || strings.HasPrefix(rel, "..") { + return nil, fmt.Errorf("thread %s session is outside Codex home: %s", id, sourceSession) + } + bundleRel := filepath.ToSlash(filepath.Join("sessions", id+".jsonl")) + bundlePath, err := safeJoin(out, bundleRel) + if err != nil { + return nil, err + } + info, err := os.Stat(sourceSession) + if err != nil { + return nil, err + } + hash, size, err := copyAndHash(sourceSession, bundlePath, info.Mode()) + if err != nil { + return nil, err + } + targetCWD := replaceRoot(cwd, project.SourcePath, project.TargetPath) + targetRel := filepath.ToSlash(rel) + row["cwd"] = targetCWD + row["rollout_path"] = filepath.Join(home, filepath.FromSlash(targetRel)) + entry := index[id] + threads = append(threads, Thread{ + ID: id, + Title: firstNonEmpty(stringValue(row["title"]), stringValue(entry["thread_name"])), + SourceCWD: cwd, + TargetCWD: targetCWD, + SourceSessionPath: sourceSession, + TargetSessionRelative: targetRel, + BundlePath: bundleRel, + SHA256: hash, + Bytes: size, + Row: row, + IndexEntry: entry, + }) + } + if err := rows.Err(); err != nil { + return nil, err + } + sort.Slice(threads, func(i, j int) bool { return threads[i].ID < threads[j].ID }) + return threads, nil +} + +func scanRow(rows *sql.Rows, columns []string) (map[string]any, error) { + values := make([]any, len(columns)) + pointers := make([]any, len(columns)) + for i := range values { + pointers[i] = &values[i] + } + if err := rows.Scan(pointers...); err != nil { + return nil, err + } + result := make(map[string]any, len(columns)) + for i, column := range columns { + if bytes, ok := values[i].([]byte); ok { + result[column] = string(bytes) + } else { + result[column] = values[i] + } + } + return result, nil +} + +func readSessionIndex(path string) (map[string]map[string]any, error) { + file, err := os.Open(path) + if err != nil { + return nil, err + } + defer file.Close() + decoder := json.NewDecoder(file) + result := map[string]map[string]any{} + for decoder.More() { + var entry map[string]any + if err := decoder.Decode(&entry); err != nil { + return nil, err + } + if id := stringValue(entry["id"]); id != "" { + result[id] = entry + } + } + return result, nil +} + +func mapSessionPaths(home string) (map[string]string, error) { + result := map[string]string{} + for _, name := range []string{"sessions", "archived_sessions"} { + root := filepath.Join(home, name) + if _, err := os.Stat(root); isNotExist(err) { + continue + } + err := filepath.WalkDir(root, func(path string, entry fs.DirEntry, walkErr error) error { + if walkErr != nil { + return walkErr + } + if entry.IsDir() || !strings.HasSuffix(entry.Name(), ".jsonl") { + return nil + } + base := strings.TrimSuffix(entry.Name(), ".jsonl") + if len(base) >= 36 { + candidate := base[len(base)-36:] + if _, err := uuid.Parse(candidate); err == nil { + result[candidate] = path + } + } + return nil + }) + if err != nil { + return nil, err + } + } + return result, nil +} + +func commonProjectParent(projects []Project) string { + if len(projects) == 0 { + return "" + } + parent := filepath.Dir(projects[0].SourcePath) + for _, project := range projects[1:] { + for parent != string(filepath.Separator) && project.SourcePath != parent && !strings.HasPrefix(project.SourcePath, parent+string(filepath.Separator)) { + parent = filepath.Dir(parent) + } + } + return parent +} + +func BuildClonePlan(manifest *Manifest) []CloneAction { + var actions []CloneAction + for _, project := range manifest.Projects { + if len(project.Repos) == 0 { + actions = append(actions, CloneAction{Project: project.Name, TargetPath: project.TargetPath, NeedsBundle: true, BundleReason: "project root has no Git repository"}) + continue + } + for _, repo := range project.Repos { + remote := repo.Remotes["origin"] + if remote == "" { + for _, value := range repo.Remotes { + remote = value + break + } + } + action := CloneAction{Project: project.Name, TargetPath: repo.TargetPath, Remote: remote, Branch: repo.Branch, Commit: repo.Commit} + if remote == "" && repo.BundlePath == "" { + action.NeedsBundle = true + action.BundleReason = "repository has no fetch remote" + } else if remote != "" { + action.Command = fmt.Sprintf("git clone %s %s && git -C %s checkout %s", quoteShell(remote), quoteShell(repo.TargetPath), quoteShell(repo.TargetPath), quoteShell(repo.Commit)) + } else { + action.Command = fmt.Sprintf("git clone %s %s && git -C %s checkout %s", quoteShell(repo.BundlePath), quoteShell(repo.TargetPath), quoteShell(repo.TargetPath), quoteShell(repo.Commit)) + } + actions = append(actions, action) + } + } + return actions +} + +func firstNonEmpty(values ...string) string { + for _, value := range values { + if value != "" { + return value + } + } + return "" +} diff --git a/internal/profile/fetch.go b/internal/profile/fetch.go new file mode 100644 index 0000000..6f68ae1 --- /dev/null +++ b/internal/profile/fetch.go @@ -0,0 +1,167 @@ +package profile + +import ( + "encoding/json" + "fmt" + "io" + "net/http" + "net/url" + "os" + "path/filepath" + "strings" + "time" +) + +func Fetch(opts FetchOptions) (*FetchResult, error) { + if strings.TrimSpace(opts.Source) == "" { + return nil, fmt.Errorf("missing profile source") + } + if strings.TrimSpace(opts.Out) == "" { + return nil, fmt.Errorf("missing --out") + } + out, err := filepath.Abs(opts.Out) + if err != nil { + return nil, err + } + if parsed, parseErr := url.Parse(opts.Source); parseErr == nil && (parsed.Scheme == "http" || parsed.Scheme == "https") { + return fetchHTTP(opts.Source, out, opts.IncludeGitBundles) + } + source, err := filepath.Abs(opts.Source) + if err != nil { + return nil, err + } + return fetchLocal(source, out, opts.IncludeGitBundles) +} + +func fetchLocal(source, out string, includeGitBundles bool) (*FetchResult, error) { + manifest, err := readManifest(source) + if err != nil { + return nil, err + } + result := &FetchResult{Status: "ok", BundleDir: out} + if err := syncBundleFiles(manifest, out, includeGitBundles, func(relative, target string) error { + sourcePath, err := safeJoin(source, relative) + if err != nil { + return err + } + info, err := os.Stat(sourcePath) + if err != nil { + return err + } + return copyFile(sourcePath, target, info.Mode()) + }, result); err != nil { + return nil, err + } + if err := writeJSON(filepath.Join(out, "manifest.json"), manifest); err != nil { + return nil, err + } + return result, nil +} + +func fetchHTTP(source, out string, includeGitBundles bool) (*FetchResult, error) { + base := strings.TrimRight(source, "/") + if strings.HasSuffix(base, "/manifest.json") { + base = strings.TrimSuffix(base, "/manifest.json") + } + client := &http.Client{Timeout: 0} + response, err := client.Get(base + "/manifest.json") + if err != nil { + return nil, err + } + defer response.Body.Close() + if response.StatusCode != http.StatusOK { + return nil, fmt.Errorf("fetch manifest: HTTP %s", response.Status) + } + var manifest Manifest + if err := json.NewDecoder(io.LimitReader(response.Body, 16*1024*1024)).Decode(&manifest); err != nil { + return nil, err + } + if manifest.Schema != Schema { + return nil, fmt.Errorf("unsupported profile schema %q", manifest.Schema) + } + result := &FetchResult{Status: "ok", BundleDir: out} + if err := syncBundleFiles(&manifest, out, includeGitBundles, func(relative, target string) error { + segments := strings.Split(filepath.ToSlash(relative), "/") + for index := range segments { + segments[index] = url.PathEscape(segments[index]) + } + requestURL := base + "/" + strings.Join(segments, "/") + response, err := client.Get(requestURL) + if err != nil { + return err + } + defer response.Body.Close() + if response.StatusCode != http.StatusOK { + return fmt.Errorf("fetch %s: HTTP %s", relative, response.Status) + } + if err := os.MkdirAll(filepath.Dir(target), 0o755); err != nil { + return err + } + tmp := target + fmt.Sprintf(".tmp-%d", time.Now().UnixNano()) + file, err := os.OpenFile(tmp, os.O_CREATE|os.O_TRUNC|os.O_WRONLY, 0o600) + if err != nil { + return err + } + _, copyErr := io.Copy(file, response.Body) + closeErr := file.Close() + if copyErr != nil { + return copyErr + } + if closeErr != nil { + return closeErr + } + return os.Rename(tmp, target) + }, result); err != nil { + return nil, err + } + if err := writeJSON(filepath.Join(out, "manifest.json"), &manifest); err != nil { + return nil, err + } + return result, nil +} + +func syncBundleFiles(manifest *Manifest, out string, includeGitBundles bool, download func(relative, target string) error, result *FetchResult) error { + type item struct { + path string + hash string + size int64 + } + items := make([]item, 0, len(manifest.ProfileFiles)+len(manifest.Threads)) + for _, file := range manifest.ProfileFiles { + if file.LinkTarget != "" { + continue + } + items = append(items, item{path: file.BundlePath, hash: file.SHA256, size: file.Bytes}) + } + for _, thread := range manifest.Threads { + items = append(items, item{path: thread.BundlePath, hash: thread.SHA256, size: thread.Bytes}) + } + if includeGitBundles { + for _, project := range manifest.Projects { + for _, repo := range project.Repos { + if repo.BundlePath != "" { + items = append(items, item{path: repo.BundlePath, hash: repo.BundleSHA256, size: repo.BundleBytes}) + } + } + } + } + for _, entry := range items { + target, err := safeJoin(out, entry.path) + if err != nil { + return err + } + if fileMatches(target, entry.hash, entry.size) { + result.ReusedFiles++ + continue + } + if err := download(entry.path, target); err != nil { + return err + } + if !fileMatches(target, entry.hash, entry.size) { + return fmt.Errorf("checksum mismatch after fetching %s", entry.path) + } + result.DownloadedFiles++ + result.DownloadedBytes += entry.size + } + return nil +} diff --git a/internal/profile/fs.go b/internal/profile/fs.go new file mode 100644 index 0000000..184e337 --- /dev/null +++ b/internal/profile/fs.go @@ -0,0 +1,312 @@ +package profile + +import ( + "bufio" + "crypto/sha256" + "encoding/hex" + "encoding/json" + "errors" + "fmt" + "io" + "io/fs" + "os" + "path/filepath" + "regexp" + "sort" + "strings" +) + +var highConfidenceSecrets = []*regexp.Regexp{ + regexp.MustCompile(`(?i)(?:api[_-]?key|access[_-]?token|secret[_-]?key)\s*[=:]\s*["']?[A-Za-z0-9_./+=-]{20,}`), + regexp.MustCompile(`\bsk-[A-Za-z0-9_-]{20,}\b`), + regexp.MustCompile(`\bgh[opsu]_[A-Za-z0-9]{20,}\b`), + regexp.MustCompile(`-----BEGIN (?:RSA |EC |OPENSSH )?PRIVATE KEY-----`), +} + +func resolveHome(path string) (string, error) { + if path == "" { + if value := os.Getenv("CODEX_HOME"); value != "" { + path = value + } else { + home, err := os.UserHomeDir() + if err != nil { + return "", err + } + path = filepath.Join(home, ".codex") + } + } + if path == "~" || strings.HasPrefix(path, "~/") { + home, err := os.UserHomeDir() + if err != nil { + return "", err + } + path = filepath.Join(home, strings.TrimPrefix(path, "~/")) + } + return filepath.Abs(path) +} + +func safeJoin(root, relative string) (string, error) { + return safeJoinWithFinalSymlink(root, relative, false) +} + +func safeJoinAllowFinalSymlink(root, relative string) (string, error) { + return safeJoinWithFinalSymlink(root, relative, true) +} + +func safeJoinWithFinalSymlink(root, relative string, allowFinalSymlink bool) (string, error) { + relative = filepath.FromSlash(relative) + if relative == "" || filepath.IsAbs(relative) { + return "", fmt.Errorf("unsafe relative path %q", relative) + } + clean := filepath.Clean(relative) + if clean == "." || clean == ".." || strings.HasPrefix(clean, ".."+string(filepath.Separator)) { + return "", fmt.Errorf("unsafe relative path %q", relative) + } + target := filepath.Join(root, clean) + current := root + parts := strings.Split(clean, string(filepath.Separator)) + for index, part := range parts { + current = filepath.Join(current, part) + info, err := os.Lstat(current) + if isNotExist(err) { + break + } + if err != nil { + return "", err + } + if info.Mode()&os.ModeSymlink != 0 { + if allowFinalSymlink && index == len(parts)-1 { + continue + } + return "", fmt.Errorf("refusing path through symlink: %s", current) + } + } + return target, nil +} + +func writeJSON(path string, value any) error { + if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil { + return err + } + tmp := path + ".tmp" + file, err := os.OpenFile(tmp, os.O_CREATE|os.O_TRUNC|os.O_WRONLY, 0o600) + if err != nil { + return err + } + enc := json.NewEncoder(file) + enc.SetEscapeHTML(false) + enc.SetIndent("", " ") + if err := enc.Encode(value); err != nil { + file.Close() + return err + } + if err := file.Close(); err != nil { + return err + } + return os.Rename(tmp, path) +} + +func readManifest(bundleDir string) (*Manifest, error) { + data, err := os.ReadFile(filepath.Join(bundleDir, "manifest.json")) + if err != nil { + return nil, err + } + var manifest Manifest + if err := json.Unmarshal(data, &manifest); err != nil { + return nil, fmt.Errorf("parse profile manifest: %w", err) + } + if manifest.Schema != Schema { + return nil, fmt.Errorf("unsupported profile schema %q", manifest.Schema) + } + return &manifest, nil +} + +func copyAndHash(source, target string, mode fs.FileMode) (string, int64, error) { + in, err := os.Open(source) + if err != nil { + return "", 0, err + } + defer in.Close() + if err := os.MkdirAll(filepath.Dir(target), 0o755); err != nil { + return "", 0, err + } + tmp := target + ".tmp" + out, err := os.OpenFile(tmp, os.O_CREATE|os.O_TRUNC|os.O_WRONLY, mode.Perm()) + if err != nil { + return "", 0, err + } + hash := sha256.New() + n, copyErr := io.Copy(io.MultiWriter(out, hash), in) + closeErr := out.Close() + if copyErr != nil { + return "", n, copyErr + } + if closeErr != nil { + return "", n, closeErr + } + if err := os.Rename(tmp, target); err != nil { + return "", n, err + } + return hex.EncodeToString(hash.Sum(nil)), n, nil +} + +func fileHash(path string) (string, int64, error) { + file, err := os.Open(path) + if err != nil { + return "", 0, err + } + defer file.Close() + hash := sha256.New() + n, err := io.Copy(hash, file) + if err != nil { + return "", n, err + } + return hex.EncodeToString(hash.Sum(nil)), n, nil +} + +func fileMatches(path, expected string, size int64) bool { + info, err := os.Stat(path) + if err != nil || !info.Mode().IsRegular() || info.Size() != size { + return false + } + actual, _, err := fileHash(path) + return err == nil && actual == expected +} + +func copyTreeFiles(sourceRoot, bundleRoot, targetPrefix string, unsafe bool) ([]File, error) { + var files []File + err := filepath.WalkDir(sourceRoot, func(path string, entry fs.DirEntry, walkErr error) error { + if walkErr != nil { + return walkErr + } + rel, err := filepath.Rel(sourceRoot, path) + if err != nil { + return err + } + if rel == "." { + return nil + } + relSlash := filepath.ToSlash(rel) + if relSlash == ".system" || strings.HasPrefix(relSlash, ".system/") { + if entry.IsDir() { + return filepath.SkipDir + } + return nil + } + if entry.Type()&os.ModeSymlink != 0 { + linkTarget, err := os.Readlink(path) + if err != nil { + return err + } + targetRel := filepath.ToSlash(filepath.Join(targetPrefix, rel)) + files = append(files, File{TargetRelativePath: targetRel, Mode: uint32(os.ModeSymlink), LinkTarget: linkTarget}) + return nil + } + info, err := entry.Info() + if err != nil { + return err + } + if !info.Mode().IsRegular() { + return nil + } + targetRel := filepath.ToSlash(filepath.Join(targetPrefix, rel)) + bundleRel := filepath.ToSlash(filepath.Join("profile", targetRel)) + if !unsafe { + if err := scanFileForSecrets(path); err != nil { + return err + } + } + bundlePath, err := safeJoin(bundleRoot, bundleRel) + if err != nil { + return err + } + hash, size, err := copyAndHash(path, bundlePath, info.Mode()) + if err != nil { + return err + } + files = append(files, File{TargetRelativePath: targetRel, BundlePath: bundleRel, SHA256: hash, Bytes: size, Mode: uint32(info.Mode().Perm())}) + return nil + }) + sort.Slice(files, func(i, j int) bool { return files[i].TargetRelativePath < files[j].TargetRelativePath }) + return files, err +} + +func scanFileForSecrets(path string) error { + info, err := os.Stat(path) + if err != nil { + return err + } + if info.Size() > 8*1024*1024 { + return nil + } + file, err := os.Open(path) + if err != nil { + return err + } + defer file.Close() + scanner := bufio.NewScanner(file) + scanner.Buffer(make([]byte, 64*1024), 1024*1024) + line := 0 + for scanner.Scan() { + line++ + text := scanner.Text() + for _, pattern := range highConfidenceSecrets { + if pattern.MatchString(text) { + return fmt.Errorf("secret scan blocked %s:%d; review it or rerun with --unsafe-include-secrets", path, line) + } + } + } + return scanner.Err() +} + +func copyFile(source, target string, mode fs.FileMode) error { + _, _, err := copyAndHash(source, target, mode) + return err +} + +func stringValue(value any) string { + switch item := value.(type) { + case string: + return item + case []byte: + return string(item) + case nil: + return "" + default: + return fmt.Sprint(item) + } +} + +func pathWithin(path string, roots []Project) (Project, bool) { + clean := filepath.Clean(path) + var match Project + for _, project := range roots { + root := filepath.Clean(project.SourcePath) + if clean == root || strings.HasPrefix(clean, root+string(filepath.Separator)) { + if len(root) > len(match.SourcePath) { + match = project + } + } + } + return match, match.SourcePath != "" +} + +func replaceRoot(path, source, target string) string { + clean := filepath.Clean(path) + source = filepath.Clean(source) + if clean == source { + return target + } + if strings.HasPrefix(clean, source+string(filepath.Separator)) { + return filepath.Join(target, strings.TrimPrefix(clean, source+string(filepath.Separator))) + } + return path +} + +func quoteShell(value string) string { + return "'" + strings.ReplaceAll(value, "'", "'\\''") + "'" +} + +func isNotExist(err error) bool { + return errors.Is(err, os.ErrNotExist) +} diff --git a/internal/profile/import.go b/internal/profile/import.go new file mode 100644 index 0000000..55f9db0 --- /dev/null +++ b/internal/profile/import.go @@ -0,0 +1,780 @@ +package profile + +import ( + "bufio" + "crypto/sha256" + "database/sql" + "encoding/json" + "errors" + "fmt" + "os" + "os/exec" + "path/filepath" + "sort" + "strings" + "time" + "unicode/utf8" + + _ "modernc.org/sqlite" +) + +func Import(opts ImportOptions) (*ImportResult, error) { + bundleDir, err := filepath.Abs(opts.BundleDir) + if err != nil { + return nil, err + } + manifest, err := readManifest(bundleDir) + if err != nil { + return nil, err + } + home := opts.Home + if home == "" { + home = manifest.TargetHome + } + home, err = resolveHome(home) + if err != nil { + return nil, err + } + missing := missingProjectRepos(manifest) + result := &ImportResult{ + Status: "planned", + DryRun: !opts.Execute, + TargetHome: home, + Projects: len(manifest.Projects), + Threads: len(manifest.Threads), + ProfileFiles: len(manifest.ProfileFiles), + MissingProjects: missing, + PreservedIdentity: true, + } + for _, file := range manifest.ProfileFiles { + join := safeJoin + if file.LinkTarget != "" { + join = safeJoinAllowFinalSymlink + } + target, joinErr := join(home, file.TargetRelativePath) + if joinErr != nil { + return nil, joinErr + } + result.Writes = append(result.Writes, target) + } + for _, thread := range manifest.Threads { + target, joinErr := safeJoin(home, thread.TargetSessionRelative) + if joinErr != nil { + return nil, joinErr + } + result.Writes = append(result.Writes, target) + } + result.Writes = append(result.Writes, + filepath.Join(home, "state_5.sqlite"), + filepath.Join(home, "session_index.jsonl"), + filepath.Join(home, ".codex-global-state.json"), + ) + sort.Strings(result.Writes) + if !opts.Execute { + return result, nil + } + if len(missing) > 0 { + return nil, fmt.Errorf("project repositories must be cloned before import: %s", strings.Join(missing, ", ")) + } + if opts.RequireStopped { + if running, err := codexAppRunning(); err != nil { + return nil, err + } else if running { + return nil, errors.New("Codex App is running; quit it before profile import or use profile schedule-import") + } + } + if _, err := os.Stat(filepath.Join(home, "state_5.sqlite")); err != nil { + return nil, fmt.Errorf("target Codex database is missing; open Codex once before importing: %w", err) + } + if err := validateBundleFiles(bundleDir, manifest); err != nil { + return nil, err + } + if err := checkpointSQLite(filepath.Join(home, "state_5.sqlite")); err != nil { + return nil, err + } + backupDir, err := backupTarget(home, manifest) + if err != nil { + return nil, err + } + result.BackupDir = backupDir + if err := importProfileFiles(bundleDir, home, manifest); err != nil { + return nil, err + } + if err := importSessions(bundleDir, home, manifest); err != nil { + return nil, err + } + if err := upsertThreads(filepath.Join(home, "state_5.sqlite"), manifest.Threads, home); err != nil { + return nil, err + } + if err := mergeSessionIndex(home, manifest.Threads); err != nil { + return nil, err + } + if err := mergeGlobalState(home, manifest); err != nil { + return nil, err + } + if err := checkpointSQLite(filepath.Join(home, "state_5.sqlite")); err != nil { + return nil, err + } + result.Status = "ok" + return result, nil +} + +func validateBundleFiles(bundleDir string, manifest *Manifest) error { + for _, file := range manifest.ProfileFiles { + if file.LinkTarget != "" { + continue + } + path, err := safeJoin(bundleDir, file.BundlePath) + if err != nil { + return err + } + if !fileMatches(path, file.SHA256, file.Bytes) { + return fmt.Errorf("profile bundle file is missing or corrupt: %s", file.BundlePath) + } + } + for _, thread := range manifest.Threads { + path, err := safeJoin(bundleDir, thread.BundlePath) + if err != nil { + return err + } + if !fileMatches(path, thread.SHA256, thread.Bytes) { + return fmt.Errorf("profile session is missing or corrupt: %s", thread.BundlePath) + } + } + return nil +} + +func missingProjectRepos(manifest *Manifest) []string { + var missing []string + for _, project := range manifest.Projects { + if len(project.Repos) == 0 { + missing = append(missing, project.TargetPath) + continue + } + for _, repo := range project.Repos { + if !isGitRepo(repo.TargetPath) { + missing = append(missing, repo.TargetPath) + } + } + } + sort.Strings(missing) + return missing +} + +func codexAppRunning() (bool, error) { + command := exec.Command("pgrep", "-x", "ChatGPT") + err := command.Run() + if err == nil { + return true, nil + } + var exitErr *exec.ExitError + if errors.As(err, &exitErr) && exitErr.ExitCode() == 1 { + return false, nil + } + return false, err +} + +func checkpointSQLite(path string) error { + db, err := sql.Open("sqlite", path) + if err != nil { + return err + } + defer db.Close() + var busy, logFrames, checkpointed int + if err := db.QueryRow("PRAGMA wal_checkpoint(TRUNCATE)").Scan(&busy, &logFrames, &checkpointed); err != nil { + return fmt.Errorf("checkpoint Codex database: %w", err) + } + if busy != 0 { + return errors.New("Codex database remained busy during WAL checkpoint") + } + return nil +} + +func backupTarget(home string, manifest *Manifest) (string, error) { + backup := filepath.Join(home, "backups_state", "agent-capsule-profile", time.Now().UTC().Format("20060102T150405Z")+"-"+manifest.ID) + if err := os.MkdirAll(backup, 0o700); err != nil { + return "", err + } + seen := map[string]bool{} + backupPath := func(relative string) error { + if seen[relative] { + return nil + } + seen[relative] = true + source, err := safeJoin(home, relative) + if err != nil { + return err + } + info, err := os.Stat(source) + if isNotExist(err) { + return nil + } + if err != nil { + return err + } + if !info.Mode().IsRegular() { + return nil + } + target, err := safeJoin(backup, relative) + if err != nil { + return err + } + return copyFile(source, target, info.Mode()) + } + for _, relative := range []string{"state_5.sqlite", "state_5.sqlite-wal", "state_5.sqlite-shm", "session_index.jsonl", ".codex-global-state.json"} { + if err := backupPath(relative); err != nil { + return "", err + } + } + for _, file := range manifest.ProfileFiles { + if file.LinkTarget == "" { + if err := backupPath(file.TargetRelativePath); err != nil { + return "", err + } + } + if strings.HasSuffix(file.TargetRelativePath, ".sqlite") { + if err := backupPath(file.TargetRelativePath + "-wal"); err != nil { + return "", err + } + if err := backupPath(file.TargetRelativePath + "-shm"); err != nil { + return "", err + } + } + } + if err := backupManagedProfileTrees(home, backup); err != nil { + return "", err + } + for _, thread := range manifest.Threads { + if err := backupPath(thread.TargetSessionRelative); err != nil { + return "", err + } + } + return backup, nil +} + +func importProfileFiles(bundleDir, home string, manifest *Manifest) error { + if err := pruneManagedProfileTrees(home, manifest); err != nil { + return err + } + for _, file := range manifest.ProfileFiles { + target, err := safeJoin(home, file.TargetRelativePath) + if err != nil { + return err + } + if file.LinkTarget != "" { + if err := os.MkdirAll(filepath.Dir(target), 0o755); err != nil { + return err + } + linkTarget := string(rewriteProfileText([]byte(file.LinkTarget), manifest, home)) + if err := os.Symlink(linkTarget, target); err != nil { + return err + } + continue + } + source, err := safeJoin(bundleDir, file.BundlePath) + if err != nil { + return err + } + data, err := os.ReadFile(source) + if err != nil { + return err + } + if utf8.Valid(data) { + data = rewriteProfileText(data, manifest, home) + } + if err := writeBytes(target, data, os.FileMode(file.Mode)); err != nil { + return err + } + if strings.HasSuffix(file.TargetRelativePath, ".sqlite") { + _ = os.Remove(target + "-wal") + _ = os.Remove(target + "-shm") + } + } + return nil +} + +func backupManagedProfileTrees(home, backup string) error { + for _, root := range managedProfileDirs() { + source := filepath.Join(home, root) + if _, err := os.Stat(source); isNotExist(err) { + continue + } else if err != nil { + return err + } + err := filepath.WalkDir(source, func(path string, entry os.DirEntry, walkErr error) error { + if walkErr != nil { + return walkErr + } + relHome, err := filepath.Rel(home, path) + if err != nil { + return err + } + if filepath.ToSlash(relHome) == "skills/.system" && entry.IsDir() { + return filepath.SkipDir + } + if entry.IsDir() { + return nil + } + target, err := safeJoin(backup, filepath.ToSlash(relHome)) + if err != nil { + return err + } + if entry.Type()&os.ModeSymlink != 0 { + linkTarget, err := os.Readlink(path) + if err != nil { + return err + } + if err := os.MkdirAll(filepath.Dir(target), 0o755); err != nil { + return err + } + return os.Symlink(linkTarget, target) + } + info, err := entry.Info() + if err != nil { + return err + } + if !info.Mode().IsRegular() { + return nil + } + return copyFile(path, target, info.Mode()) + }) + if err != nil { + return err + } + } + return nil +} + +func pruneManagedProfileTrees(home string, manifest *Manifest) error { + desired := map[string]bool{} + desiredLinks := map[string]bool{} + for _, file := range manifest.ProfileFiles { + relative := filepath.Clean(filepath.FromSlash(file.TargetRelativePath)) + desired[relative] = true + if file.LinkTarget != "" { + desiredLinks[relative] = true + } + } + for _, root := range managedProfileDirs() { + rootPath := filepath.Join(home, root) + if _, err := os.Stat(rootPath); isNotExist(err) { + continue + } else if err != nil { + return err + } + var paths []string + err := filepath.WalkDir(rootPath, func(path string, entry os.DirEntry, walkErr error) error { + if walkErr != nil { + return walkErr + } + rel, err := filepath.Rel(home, path) + if err != nil { + return err + } + if filepath.ToSlash(rel) == "skills/.system" && entry.IsDir() { + return filepath.SkipDir + } + if path != rootPath { + paths = append(paths, path) + } + return nil + }) + if err != nil { + return err + } + sort.Slice(paths, func(i, j int) bool { return len(paths[i]) > len(paths[j]) }) + for _, path := range paths { + rel, _ := filepath.Rel(home, path) + info, err := os.Lstat(path) + if isNotExist(err) { + continue + } + if err != nil { + return err + } + if info.IsDir() { + _ = os.Remove(path) + continue + } + cleanRel := filepath.Clean(rel) + if desiredLinks[cleanRel] || !desired[cleanRel] { + if err := os.Remove(path); err != nil { + return err + } + } + } + } + return nil +} + +func managedProfileDirs() []string { + return []string{"rules", "skills", "memories", "data", "automations"} +} + +func rewriteProfileText(data []byte, manifest *Manifest, targetHome string) []byte { + text := string(data) + targetUserHome := manifest.TargetUserHome + if targetUserHome == "" { + targetUserHome = filepath.Dir(targetHome) + } + replacements := [][2]string{{manifest.SourceHome, targetHome}} + if manifest.SourceUserHome != "" { + replacements = append(replacements, [2]string{manifest.SourceUserHome, targetUserHome}) + } + if manifest.SourceWorkspace != "" && manifest.TargetWorkspace != "" { + replacements = append(replacements, [2]string{manifest.SourceWorkspace, manifest.TargetWorkspace}) + } + for _, project := range manifest.Projects { + replacements = append(replacements, [2]string{project.SourcePath, project.TargetPath}) + } + sort.SliceStable(replacements, func(i, j int) bool { return len(replacements[i][0]) > len(replacements[j][0]) }) + for _, replacement := range replacements { + text = strings.ReplaceAll(text, replacement[0], replacement[1]) + } + return []byte(text) +} + +func importSessions(bundleDir, home string, manifest *Manifest) error { + for _, thread := range manifest.Threads { + source, err := safeJoin(bundleDir, thread.BundlePath) + if err != nil { + return err + } + target, err := safeJoin(home, thread.TargetSessionRelative) + if err != nil { + return err + } + if err := rewriteSession(source, target, manifest, home); err != nil { + return fmt.Errorf("rewrite session %s: %w", thread.ID, err) + } + } + return nil +} + +func rewriteSession(source, target string, manifest *Manifest, targetHome string) error { + in, err := os.Open(source) + if err != nil { + return err + } + defer in.Close() + if err := os.MkdirAll(filepath.Dir(target), 0o755); err != nil { + return err + } + tmp := target + ".tmp" + out, err := os.OpenFile(tmp, os.O_CREATE|os.O_TRUNC|os.O_WRONLY, 0o600) + if err != nil { + return err + } + writer := bufio.NewWriter(out) + scanner := bufio.NewScanner(in) + scanner.Buffer(make([]byte, 64*1024), 64*1024*1024) + for scanner.Scan() { + line := scanner.Bytes() + var event map[string]any + if err := json.Unmarshal(line, &event); err != nil { + out.Close() + return err + } + typeName := stringValue(event["type"]) + if typeName == "session_meta" || typeName == "turn_context" { + rewriteJSONStrings(event["payload"], manifest, targetHome) + } else { + if _, err := writer.Write(append(append([]byte(nil), line...), '\n')); err != nil { + out.Close() + return err + } + continue + } + encoded, err := json.Marshal(event) + if err != nil { + out.Close() + return err + } + if _, err := writer.Write(append(encoded, '\n')); err != nil { + out.Close() + return err + } + } + if err := scanner.Err(); err != nil { + out.Close() + return err + } + if err := writer.Flush(); err != nil { + out.Close() + return err + } + if err := out.Close(); err != nil { + return err + } + return os.Rename(tmp, target) +} + +func rewriteJSONStrings(value any, manifest *Manifest, targetHome string) { + switch item := value.(type) { + case map[string]any: + for key, child := range item { + if text, ok := child.(string); ok { + item[key] = string(rewriteProfileText([]byte(text), manifest, targetHome)) + } else { + rewriteJSONStrings(child, manifest, targetHome) + } + } + case []any: + for index, child := range item { + if text, ok := child.(string); ok { + item[index] = string(rewriteProfileText([]byte(text), manifest, targetHome)) + } else { + rewriteJSONStrings(child, manifest, targetHome) + } + } + } +} + +func upsertThreads(dbPath string, threads []Thread, home string) error { + db, err := sql.Open("sqlite", dbPath) + if err != nil { + return err + } + defer db.Close() + columns, err := tableColumns(db, "threads") + if err != nil { + return err + } + tx, err := db.Begin() + if err != nil { + return err + } + defer tx.Rollback() + for _, thread := range threads { + row := cloneMap(thread.Row) + row["id"] = thread.ID + row["cwd"] = thread.TargetCWD + targetPath, err := safeJoin(home, thread.TargetSessionRelative) + if err != nil { + return err + } + row["rollout_path"] = targetPath + var names []string + for _, column := range columns { + if _, ok := row[column]; ok { + names = append(names, column) + } + } + if len(names) == 0 { + return errors.New("target threads table has no compatible columns") + } + placeholders := make([]string, len(names)) + updates := make([]string, 0, len(names)-1) + values := make([]any, len(names)) + for i, name := range names { + placeholders[i] = "?" + values[i] = row[name] + if name != "id" { + updates = append(updates, fmt.Sprintf("%s=excluded.%s", quoteIdentifier(name), quoteIdentifier(name))) + } + } + query := fmt.Sprintf("insert into threads (%s) values (%s) on conflict(id) do update set %s", joinIdentifiers(names), strings.Join(placeholders, ","), strings.Join(updates, ",")) + if _, err := tx.Exec(query, values...); err != nil { + return fmt.Errorf("upsert thread %s: %w", thread.ID, err) + } + } + return tx.Commit() +} + +func tableColumns(db *sql.DB, table string) ([]string, error) { + rows, err := db.Query("pragma table_info(" + quoteIdentifier(table) + ")") + if err != nil { + return nil, err + } + defer rows.Close() + var result []string + for rows.Next() { + var cid, notNull, pk int + var name, typ string + var defaultValue any + if err := rows.Scan(&cid, &name, &typ, ¬Null, &defaultValue, &pk); err != nil { + return nil, err + } + result = append(result, name) + } + return result, rows.Err() +} + +func quoteIdentifier(value string) string { + return `"` + strings.ReplaceAll(value, `"`, `""`) + `"` +} + +func joinIdentifiers(values []string) string { + quoted := make([]string, len(values)) + for index, value := range values { + quoted[index] = quoteIdentifier(value) + } + return strings.Join(quoted, ",") +} + +func mergeSessionIndex(home string, threads []Thread) error { + path := filepath.Join(home, "session_index.jsonl") + entries, err := readSessionIndex(path) + if err != nil && !isNotExist(err) { + return err + } + if entries == nil { + entries = map[string]map[string]any{} + } + for _, thread := range threads { + entry := cloneMap(thread.IndexEntry) + entry["id"] = thread.ID + if stringValue(entry["thread_name"]) == "" { + entry["thread_name"] = thread.Title + } + entries[thread.ID] = entry + } + ids := make([]string, 0, len(entries)) + for id := range entries { + ids = append(ids, id) + } + sort.Strings(ids) + if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil { + return err + } + tmp := path + ".tmp" + file, err := os.OpenFile(tmp, os.O_CREATE|os.O_TRUNC|os.O_WRONLY, 0o600) + if err != nil { + return err + } + encoder := json.NewEncoder(file) + encoder.SetEscapeHTML(false) + for _, id := range ids { + if err := encoder.Encode(entries[id]); err != nil { + file.Close() + return err + } + } + if err := file.Close(); err != nil { + return err + } + return os.Rename(tmp, path) +} + +func mergeGlobalState(home string, manifest *Manifest) error { + path := filepath.Join(home, ".codex-global-state.json") + state := map[string]any{} + if data, err := os.ReadFile(path); err == nil { + if err := json.Unmarshal(data, &state); err != nil { + return err + } + } else if !isNotExist(err) { + return err + } + localProjects := mapValue(state["local-projects"]) + assignments := mapValue(state["thread-project-assignments"]) + hints := mapValue(state["thread-workspace-root-hints"]) + projectOrder := stringSlice(state["project-order"]) + savedRoots := stringSlice(state["electron-saved-workspace-roots"]) + projectless := stringSlice(state["projectless-thread-ids"]) + projectlessSet := make(map[string]bool, len(projectless)) + for _, id := range projectless { + projectlessSet[id] = true + } + projectIDs := map[string]string{} + now := time.Now().UnixMilli() + for _, project := range manifest.Projects { + id := deterministicProjectID(project.TargetPath) + projectIDs[project.ID] = id + localProjects[id] = map[string]any{"id": id, "name": project.Name, "rootPaths": []string{project.TargetPath}, "createdAt": now, "updatedAt": now} + projectOrder = appendUnique(projectOrder, id) + savedRoots = appendUnique(savedRoots, project.TargetPath) + } + for _, thread := range manifest.Threads { + project := targetProjectForPath(thread.TargetCWD, manifest.Projects) + if project.ID == "" { + continue + } + assignments[thread.ID] = map[string]any{"projectKind": "local", "projectId": projectIDs[project.ID], "path": project.TargetPath, "cwd": thread.TargetCWD, "pendingCoreUpdate": false} + hints[thread.ID] = project.TargetPath + delete(projectlessSet, thread.ID) + } + projectless = projectless[:0] + for id := range projectlessSet { + projectless = append(projectless, id) + } + sort.Strings(projectless) + state["local-projects"] = localProjects + state["project-order"] = projectOrder + state["electron-saved-workspace-roots"] = savedRoots + state["thread-project-assignments"] = assignments + state["thread-workspace-root-hints"] = hints + state["projectless-thread-ids"] = projectless + return writeJSON(path, state) +} + +func deterministicProjectID(path string) string { + hash := fmt.Sprintf("%x", sha256.Sum256([]byte(filepath.Clean(path)))) + return "local-" + hash[:32] +} + +func targetProjectForPath(path string, projects []Project) Project { + clean := filepath.Clean(path) + var match Project + for _, project := range projects { + root := filepath.Clean(project.TargetPath) + if clean == root || strings.HasPrefix(clean, root+string(filepath.Separator)) { + if len(root) > len(match.TargetPath) { + match = project + } + } + } + return match +} + +func mapValue(value any) map[string]any { + if result, ok := value.(map[string]any); ok { + return result + } + return map[string]any{} +} + +func stringSlice(value any) []string { + items, ok := value.([]any) + if !ok { + if strings, ok := value.([]string); ok { + return append([]string(nil), strings...) + } + return nil + } + result := make([]string, 0, len(items)) + for _, item := range items { + if text, ok := item.(string); ok { + result = append(result, text) + } + } + return result +} + +func appendUnique(values []string, value string) []string { + for _, existing := range values { + if existing == value { + return values + } + } + return append(values, value) +} + +func cloneMap(source map[string]any) map[string]any { + result := make(map[string]any, len(source)) + for key, value := range source { + result[key] = value + } + return result +} + +func writeBytes(path string, data []byte, mode os.FileMode) error { + if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil { + return err + } + tmp := path + ".tmp" + if err := os.WriteFile(tmp, data, mode.Perm()); err != nil { + return err + } + return os.Rename(tmp, path) +} diff --git a/internal/profile/profile_test.go b/internal/profile/profile_test.go new file mode 100644 index 0000000..6a1895b --- /dev/null +++ b/internal/profile/profile_test.go @@ -0,0 +1,434 @@ +package profile + +import ( + "context" + "database/sql" + "encoding/json" + "fmt" + "os" + "os/exec" + "path/filepath" + "strings" + "testing" + "time" + + _ "modernc.org/sqlite" +) + +func TestProfileExportImportAndVerify(t *testing.T) { + env := newMigrationTestEnv(t) + result, err := Export(ExportOptions{ + Home: env.sourceHome, + TargetHome: env.targetHome, + TargetWorkspace: env.targetWorkspace, + Projects: []string{env.sourceProject}, + Out: env.bundle, + }) + if err != nil { + t.Fatal(err) + } + if result.Manifest.Stats.Threads != 1 { + t.Fatalf("threads = %d, want 1", result.Manifest.Stats.Threads) + } + assertManifestExcludes(t, result.Manifest, "auth.json") + assertManifestExcludes(t, result.Manifest, "skills/.system/managed.md") + assertManifestIncludes(t, result.Manifest, "skills/user-skill/SKILL.md") + assertManifestIncludes(t, result.Manifest, "skills/linked-skill") + assertManifestIncludes(t, result.Manifest, "memories_1.sqlite") + + cloneRepo(t, env.sourceProject, env.targetProject) + dryRun, err := Import(ImportOptions{BundleDir: env.bundle, Home: env.targetHome}) + if err != nil { + t.Fatal(err) + } + if !dryRun.DryRun || dryRun.Status != "planned" { + t.Fatalf("unexpected dry run: %#v", dryRun) + } + imported, err := Import(ImportOptions{BundleDir: env.bundle, Home: env.targetHome, Execute: true}) + if err != nil { + t.Fatal(err) + } + if imported.Status != "ok" || imported.BackupDir == "" || !imported.PreservedIdentity { + t.Fatalf("unexpected import: %#v", imported) + } + if got := readFile(t, filepath.Join(env.targetHome, "auth.json")); got != "target-auth" { + t.Fatalf("target auth changed: %q", got) + } + if got := readFile(t, filepath.Join(env.targetHome, "skills", ".system", "managed.md")); got != "target-managed" { + t.Fatalf("managed skill changed: %q", got) + } + if _, err := os.Stat(filepath.Join(env.targetHome, "skills", "stale", "SKILL.md")); !os.IsNotExist(err) { + t.Fatalf("stale user skill was not pruned: %v", err) + } + if got := readFile(t, filepath.Join(imported.BackupDir, "skills", "stale", "SKILL.md")); got != "stale" { + t.Fatalf("stale skill was not backed up: %q", got) + } + if target, err := os.Readlink(filepath.Join(imported.BackupDir, "skills", "linked-skill")); err != nil || target != "/stale/linked-skill" { + t.Fatalf("replaced symlink backup = %q, %v", target, err) + } + if got := readKVDB(t, filepath.Join(env.targetHome, "memories_1.sqlite")); got != "source-memory" { + t.Fatalf("memory database was not migrated: %q", got) + } + config := readFile(t, filepath.Join(env.targetHome, "config.toml")) + if strings.Contains(config, filepath.Dir(env.sourceHome)) || !strings.Contains(config, env.targetProject) || !strings.Contains(config, filepath.Join(filepath.Dir(env.targetHome), ".local", "bin", "tool")) { + t.Fatalf("config paths were not rewritten: %s", config) + } + linkedSkill := filepath.Join(env.targetHome, "skills", "linked-skill") + if target, err := os.Readlink(linkedSkill); err != nil || target != filepath.Join(env.targetProject, "skills", "linked-skill") { + t.Fatalf("linked skill target = %q, %v", target, err) + } + if got := readFile(t, filepath.Join(linkedSkill, "SKILL.md")); got != "linked skill\n" { + t.Fatalf("linked skill content = %q", got) + } + state := readJSONMap(t, filepath.Join(env.targetHome, ".codex-global-state.json")) + if state["electron-local-remote-control-installation-id"] != "target-installation" { + t.Fatalf("target machine identity changed: %#v", state) + } + if state["electron-remote-control-client-enrollments"] == nil { + t.Fatalf("target remote enrollment was removed: %#v", state) + } + verify, err := Verify(VerifyOptions{BundleDir: env.bundle, Home: env.targetHome}) + if err != nil { + t.Fatal(err) + } + if verify.Status != "ok" { + t.Fatalf("verification failed: %#v", verify) + } +} + +func TestProfileExportBlocksHighConfidenceSecret(t *testing.T) { + env := newMigrationTestEnv(t) + writeFile(t, filepath.Join(env.sourceHome, "config.toml"), `api_key = "sk-abcdefghijklmnopqrstuvwxyz123456"`) + _, err := Export(ExportOptions{Home: env.sourceHome, TargetHome: env.targetHome, TargetWorkspace: env.targetWorkspace, Projects: []string{env.sourceProject}, Out: env.bundle}) + if err == nil || !strings.Contains(err.Error(), "secret scan blocked") { + t.Fatalf("expected secret scan failure, got %v", err) + } +} + +func TestDiscoverReadsCodexProjectState(t *testing.T) { + env := newMigrationTestEnv(t) + result, err := Discover(DiscoverOptions{Home: env.sourceHome}) + if err != nil { + t.Fatal(err) + } + if len(result.Projects) != 1 { + t.Fatalf("projects = %#v", result.Projects) + } + project := result.Projects[0] + if project.Name != "project-a" || project.Path != env.sourceProject || project.Threads != 1 || project.GitRepos != 1 || !project.AllClean { + t.Fatalf("unexpected candidate: %#v", project) + } +} + +func TestFetchIsIncremental(t *testing.T) { + env := newMigrationTestEnv(t) + if _, err := Export(ExportOptions{Home: env.sourceHome, TargetHome: env.targetHome, TargetWorkspace: env.targetWorkspace, Projects: []string{env.sourceProject}, Out: env.bundle}); err != nil { + t.Fatal(err) + } + server, err := NewServer(env.bundle, "127.0.0.1:0") + if err != nil { + t.Fatal(err) + } + go func() { _ = server.Serve() }() + defer server.Close(context.Background()) + target := filepath.Join(t.TempDir(), "fetched") + first, err := Fetch(FetchOptions{Source: server.URLs[0], Out: target}) + if err != nil { + t.Fatal(err) + } + if first.DownloadedFiles == 0 || first.ReusedFiles != 0 { + t.Fatalf("unexpected first fetch: %#v", first) + } + manifest, err := readManifest(target) + if err != nil { + t.Fatal(err) + } + missing, err := safeJoin(target, manifest.Threads[0].BundlePath) + if err != nil { + t.Fatal(err) + } + if err := os.Remove(missing); err != nil { + t.Fatal(err) + } + second, err := Fetch(FetchOptions{Source: server.URLs[0], Out: target}) + if err != nil { + t.Fatal(err) + } + if second.DownloadedFiles != 1 || second.ReusedFiles != first.DownloadedFiles-1 { + t.Fatalf("fetch was not incremental: first=%#v second=%#v", first, second) + } +} + +func TestScheduleImportUsesOneShotLaunchAgent(t *testing.T) { + env := newMigrationTestEnv(t) + if _, err := Export(ExportOptions{Home: env.sourceHome, TargetHome: env.targetHome, TargetWorkspace: env.targetWorkspace, Projects: []string{env.sourceProject}, Out: env.bundle}); err != nil { + t.Fatal(err) + } + result, err := ScheduleImport(ScheduleOptions{BundleDir: env.bundle, Home: env.targetHome, CLIPath: "/usr/local/bin/capsule"}) + if err != nil { + t.Fatal(err) + } + plist := readFile(t, result.PlistPath) + if !strings.Contains(plist, "KeepAlive") { + t.Fatalf("LaunchAgent is not one-shot:\n%s", plist) + } + if strings.Contains(result.StagingDir, "Downloads") || !strings.HasPrefix(result.StagingDir, env.targetHome+string(filepath.Separator)) { + t.Fatalf("unsafe staging dir: %s", result.StagingDir) + } + runner := readFile(t, result.RunnerPath) + if !strings.Contains(runner, "profile verify") || !strings.Contains(runner, "tell application \"ChatGPT\" to quit") || !strings.Contains(runner, "PLIST_PATH="+quoteShell(result.PlistPath)) || !strings.Contains(runner, "trap cleanup EXIT") { + t.Fatalf("runner is incomplete:\n%s", runner) + } + if output, err := exec.Command("bash", "-n", result.RunnerPath).CombinedOutput(); err != nil { + t.Fatalf("runner shell syntax: %v: %s", err, output) + } +} + +func TestGitBundleFallbackIsFetchedLazily(t *testing.T) { + env := newMigrationTestEnv(t) + if _, err := Export(ExportOptions{Home: env.sourceHome, TargetHome: env.targetHome, TargetWorkspace: env.targetWorkspace, Projects: []string{env.sourceProject}, Out: env.bundle, GitBundleFallback: true}); err != nil { + t.Fatal(err) + } + fetched := filepath.Join(t.TempDir(), "fetched") + withoutBundles, err := Fetch(FetchOptions{Source: env.bundle, Out: fetched}) + if err != nil { + t.Fatal(err) + } + manifest, err := readManifest(fetched) + if err != nil { + t.Fatal(err) + } + repo := manifest.Projects[0].Repos[0] + bundlePath, err := safeJoin(fetched, repo.BundlePath) + if err != nil { + t.Fatal(err) + } + if _, err := os.Stat(bundlePath); !os.IsNotExist(err) { + t.Fatalf("Git bundle should not be fetched by default: %v", err) + } + withBundles, err := Fetch(FetchOptions{Source: env.bundle, Out: fetched, IncludeGitBundles: true}) + if err != nil { + t.Fatal(err) + } + if withBundles.DownloadedFiles != 1 || withBundles.ReusedFiles != withoutBundles.DownloadedFiles { + t.Fatalf("unexpected fallback fetch: before=%#v after=%#v", withoutBundles, withBundles) + } + if _, err := CloneProjects(CloneOptions{BundleDir: fetched, Execute: true}); err != nil { + t.Fatal(err) + } + if !isGitRepo(env.targetProject) { + t.Fatalf("target was not cloned from fallback bundle") + } +} + +type migrationTestEnv struct { + sourceHome string + targetHome string + sourceProject string + targetProject string + targetWorkspace string + bundle string + threadID string +} + +func newMigrationTestEnv(t *testing.T) migrationTestEnv { + t.Helper() + root := t.TempDir() + env := migrationTestEnv{ + sourceHome: filepath.Join(root, "source", ".codex"), + targetHome: filepath.Join(root, "target", ".codex"), + sourceProject: filepath.Join(root, "source", "workspace", "project-a"), + targetWorkspace: filepath.Join(root, "target", "workspace"), + bundle: filepath.Join(root, "bundle"), + threadID: "019f0000-0000-7000-8000-000000000001", + } + env.targetProject = filepath.Join(env.targetWorkspace, "project-a") + for _, dir := range []string{env.sourceHome, env.targetHome, env.sourceProject, env.targetWorkspace} { + if err := os.MkdirAll(dir, 0o755); err != nil { + t.Fatal(err) + } + } + initRepo(t, env.sourceProject) + writeFile(t, filepath.Join(env.sourceProject, "skills", "linked-skill", "SKILL.md"), "linked skill\n") + runCommand(t, "git", "-C", env.sourceProject, "add", "skills/linked-skill/SKILL.md") + runCommand(t, "git", "-C", env.sourceProject, "commit", "-m", "add linked skill") + sourceSession := filepath.Join(env.sourceHome, "sessions", "2026", "07", "18", "rollout-2026-07-18T00-00-00-"+env.threadID+".jsonl") + writeFile(t, sourceSession, strings.Join([]string{ + fmt.Sprintf(`{"timestamp":"2026-07-18T00:00:00Z","type":"session_meta","payload":{"id":%q,"cwd":%q}}`, env.threadID, env.sourceProject), + fmt.Sprintf(`{"timestamp":"2026-07-18T00:00:01Z","type":"turn_context","payload":{"cwd":%q}}`, env.sourceProject), + `{"timestamp":"2026-07-18T00:00:02Z","type":"response_item","payload":{"type":"message","role":"user","content":[{"type":"input_text","text":"keep source path in visible conversation"}]}}`, + }, "\n")+"\n") + createThreadsDB(t, filepath.Join(env.sourceHome, "state_5.sqlite"), env.threadID, env.sourceProject, sourceSession) + createThreadsDB(t, filepath.Join(env.targetHome, "state_5.sqlite"), "target-empty-thread", filepath.Join(root, "target", "empty"), filepath.Join(env.targetHome, "sessions", "empty.jsonl")) + writeJSONLine(t, filepath.Join(env.sourceHome, "session_index.jsonl"), map[string]any{"id": env.threadID, "thread_name": "Migrated thread", "updated_at": "2026-07-18T00:00:02Z"}) + writeJSONLine(t, filepath.Join(env.targetHome, "session_index.jsonl"), map[string]any{"id": "target-empty-thread", "thread_name": "Empty", "updated_at": "2026-07-18T00:00:00Z"}) + writeFile(t, filepath.Join(env.sourceHome, "config.toml"), "project = \""+env.sourceProject+"\"\nruntime = \""+filepath.Join(filepath.Dir(env.sourceHome), ".local", "bin", "tool")+"\"\n") + writeFile(t, filepath.Join(env.sourceHome, "AGENTS.md"), "source instructions\n") + writeFile(t, filepath.Join(env.sourceHome, "skills", "user-skill", "SKILL.md"), "user skill\n") + if err := os.Symlink(filepath.Join(env.sourceProject, "skills", "linked-skill"), filepath.Join(env.sourceHome, "skills", "linked-skill")); err != nil { + t.Fatal(err) + } + writeFile(t, filepath.Join(env.sourceHome, "skills", ".system", "managed.md"), "source managed\n") + writeFile(t, filepath.Join(env.sourceHome, "auth.json"), "source-auth") + createKVDB(t, filepath.Join(env.sourceHome, "memories_1.sqlite"), "source-memory") + writeJSONTest(t, filepath.Join(env.sourceHome, ".codex-global-state.json"), map[string]any{ + "local-projects": map[string]any{ + "local-source-project": map[string]any{"id": "local-source-project", "name": "project-a", "rootPaths": []string{env.sourceProject}}, + }, + }) + writeFile(t, filepath.Join(env.targetHome, "auth.json"), "target-auth") + writeFile(t, filepath.Join(env.targetHome, "skills", ".system", "managed.md"), "target-managed") + writeFile(t, filepath.Join(env.targetHome, "skills", "stale", "SKILL.md"), "stale") + if err := os.Symlink("/stale/linked-skill", filepath.Join(env.targetHome, "skills", "linked-skill")); err != nil { + t.Fatal(err) + } + writeJSONTest(t, filepath.Join(env.targetHome, ".codex-global-state.json"), map[string]any{ + "electron-local-remote-control-installation-id": "target-installation", + "electron-remote-control-client-enrollments": []any{map[string]any{"id": "source-controller"}}, + "local-projects": map[string]any{}, + "thread-project-assignments": map[string]any{}, + "thread-workspace-root-hints": map[string]any{}, + "project-order": []any{}, + "electron-saved-workspace-roots": []any{}, + "projectless-thread-ids": []any{env.threadID}, + }) + return env +} + +func initRepo(t *testing.T, path string) { + t.Helper() + runCommand(t, "git", "-C", path, "init") + runCommand(t, "git", "-C", path, "config", "user.email", "test@example.com") + runCommand(t, "git", "-C", path, "config", "user.name", "Test") + writeFile(t, filepath.Join(path, "README.md"), "test\n") + runCommand(t, "git", "-C", path, "add", "README.md") + runCommand(t, "git", "-C", path, "commit", "-m", "initial") +} + +func cloneRepo(t *testing.T, source, target string) { + t.Helper() + if err := os.MkdirAll(filepath.Dir(target), 0o755); err != nil { + t.Fatal(err) + } + runCommand(t, "git", "clone", source, target) +} + +func runCommand(t *testing.T, name string, args ...string) { + t.Helper() + if output, err := exec.Command(name, args...).CombinedOutput(); err != nil { + t.Fatalf("%s %v: %v: %s", name, args, err, output) + } +} + +func createThreadsDB(t *testing.T, path, id, cwd, rollout string) { + t.Helper() + if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil { + t.Fatal(err) + } + db, err := sql.Open("sqlite", path) + if err != nil { + t.Fatal(err) + } + defer db.Close() + if _, err := db.Exec(`create table threads (id text primary key, cwd text not null, title text not null, rollout_path text not null, updated_at integer not null default 0)`); err != nil { + t.Fatal(err) + } + if _, err := db.Exec(`insert into threads (id, cwd, title, rollout_path, updated_at) values (?, ?, ?, ?, ?)`, id, cwd, "Migrated thread", rollout, time.Now().UnixMilli()); err != nil { + t.Fatal(err) + } +} + +func createKVDB(t *testing.T, path, value string) { + t.Helper() + db, err := sql.Open("sqlite", path) + if err != nil { + t.Fatal(err) + } + defer db.Close() + if _, err := db.Exec(`create table kv (value text); insert into kv values (?)`, value); err != nil { + t.Fatal(err) + } +} + +func readKVDB(t *testing.T, path string) string { + t.Helper() + db, err := sql.Open("sqlite", path) + if err != nil { + t.Fatal(err) + } + defer db.Close() + var value string + if err := db.QueryRow(`select value from kv limit 1`).Scan(&value); err != nil { + t.Fatal(err) + } + return value +} + +func assertManifestIncludes(t *testing.T, manifest *Manifest, relative string) { + t.Helper() + for _, file := range manifest.ProfileFiles { + if file.TargetRelativePath == relative { + return + } + } + t.Fatalf("manifest does not include %s", relative) +} + +func assertManifestExcludes(t *testing.T, manifest *Manifest, relative string) { + t.Helper() + for _, file := range manifest.ProfileFiles { + if file.TargetRelativePath == relative { + t.Fatalf("manifest unexpectedly includes %s", relative) + } + } +} + +func writeFile(t *testing.T, path, value string) { + t.Helper() + if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil { + t.Fatal(err) + } + if err := os.WriteFile(path, []byte(value), 0o600); err != nil { + t.Fatal(err) + } +} + +func writeJSONLine(t *testing.T, path string, value any) { + t.Helper() + data, err := json.Marshal(value) + if err != nil { + t.Fatal(err) + } + writeFile(t, path, string(data)+"\n") +} + +func writeJSONTest(t *testing.T, path string, value any) { + t.Helper() + data, err := json.Marshal(value) + if err != nil { + t.Fatal(err) + } + writeFile(t, path, string(data)+"\n") +} + +func readFile(t *testing.T, path string) string { + t.Helper() + data, err := os.ReadFile(path) + if err != nil { + t.Fatal(err) + } + return string(data) +} + +func readJSONMap(t *testing.T, path string) map[string]any { + t.Helper() + data, err := os.ReadFile(path) + if err != nil { + t.Fatal(err) + } + var value map[string]any + if err := json.Unmarshal(data, &value); err != nil { + t.Fatal(err) + } + return value +} diff --git a/internal/profile/schedule.go b/internal/profile/schedule.go new file mode 100644 index 0000000..6b60d85 --- /dev/null +++ b/internal/profile/schedule.go @@ -0,0 +1,157 @@ +package profile + +import ( + "fmt" + "os" + "os/exec" + "path/filepath" + "runtime" + "strconv" + "strings" +) + +func ScheduleImport(opts ScheduleOptions) (*ScheduleResult, error) { + if opts.Submit && runtime.GOOS != "darwin" { + return nil, fmt.Errorf("profile schedule-import is supported only on macOS") + } + manifest, err := readManifest(opts.BundleDir) + if err != nil { + return nil, err + } + home := opts.Home + if home == "" { + home = manifest.TargetHome + } + home, err = resolveHome(home) + if err != nil { + return nil, err + } + cliPath := opts.CLIPath + if cliPath == "" { + cliPath, err = os.Executable() + if err != nil { + return nil, err + } + } + cliPath, err = filepath.Abs(cliPath) + if err != nil { + return nil, err + } + staging := filepath.Join(home, "profile-migrations", manifest.ID) + if filepath.Clean(opts.BundleDir) != filepath.Clean(staging) { + if _, err := Fetch(FetchOptions{Source: opts.BundleDir, Out: staging, IncludeGitBundles: true}); err != nil { + return nil, err + } + } + label := "com.agent-capsule.profile-import." + strings.ReplaceAll(manifest.ID, "_", "-") + runner := filepath.Join(staging, "run-import.sh") + statusPath := filepath.Join(staging, "import-status.json") + logPath := filepath.Join(staging, "import.log") + runnerText := fmt.Sprintf(`#!/bin/bash +set -euo pipefail +PLIST_PATH=%s +cleanup() { /bin/rm -f "$PLIST_PATH"; } +trap cleanup EXIT +sleep 5 +/usr/bin/osascript -e 'tell application "ChatGPT" to quit' >/dev/null 2>&1 || true +attempt=0 +while [ "$attempt" -lt 30 ]; do + if ! /usr/bin/pgrep -x ChatGPT >/dev/null 2>&1; then break; fi + sleep 1 + attempt=$((attempt + 1)) +done +if /usr/bin/pgrep -x ChatGPT >/dev/null 2>&1; then + /usr/bin/pkill -TERM -x ChatGPT || true + sleep 3 +fi +if /usr/bin/pgrep -x ChatGPT >/dev/null 2>&1; then + printf '{"status":"failed","error":"Codex App did not stop"}\n' > %s + exit 1 +fi +if %s profile import %s --home %s --execute; then + if %s profile verify %s --home %s; then + printf '{"status":"ok"}\n' > %s + /usr/bin/open -a ChatGPT + exit 0 + fi +fi +printf '{"status":"failed","error":"import or verification failed","log":%s}\n' > %s +/usr/bin/open -a ChatGPT +exit 1 +`, quoteShell(plistPathFor(home, label)), quoteShell(statusPath), quoteShell(cliPath), quoteShell(staging), quoteShell(home), quoteShell(cliPath), quoteShell(staging), quoteShell(home), quoteShell(statusPath), strconv.Quote(logPath), quoteShell(statusPath)) + if err := os.WriteFile(runner, []byte(runnerText), 0o700); err != nil { + return nil, err + } + launchDir := filepath.Join(filepath.Dir(home), "Library", "LaunchAgents") + if err := os.MkdirAll(launchDir, 0o755); err != nil { + return nil, err + } + plistPath := filepath.Join(launchDir, label+".plist") + plist := fmt.Sprintf(` + + + + Label%s + ProgramArguments + /bin/bash%s + RunAtLoad + KeepAlive + ProcessTypeBackground + StandardOutPath%s + StandardErrorPath%s + + +`, xmlEscape(label), xmlEscape(runner), xmlEscape(logPath), xmlEscape(logPath)) + if err := os.WriteFile(plistPath, []byte(plist), 0o600); err != nil { + return nil, err + } + result := &ScheduleResult{Status: "prepared", Label: label, StagingDir: staging, PlistPath: plistPath, RunnerPath: runner, StatusPath: statusPath, LogPath: logPath} + if !opts.Submit { + return result, nil + } + uid := os.Getuid() + domain := fmt.Sprintf("gui/%d", uid) + _ = exec.Command("launchctl", "bootout", domain+"/"+label).Run() + if output, err := exec.Command("launchctl", "bootstrap", domain, plistPath).CombinedOutput(); err != nil { + return nil, fmt.Errorf("launchctl bootstrap: %w: %s", err, output) + } + result.Status = "scheduled" + return result, nil +} + +func plistPathFor(home, label string) string { + return filepath.Join(filepath.Dir(home), "Library", "LaunchAgents", label+".plist") +} + +func Unschedule(opts UnscheduleOptions) (*UnscheduleResult, error) { + manifest, err := readManifest(opts.BundleDir) + if err != nil { + return nil, err + } + label := "com.agent-capsule.profile-import." + strings.ReplaceAll(manifest.ID, "_", "-") + home := opts.Home + if home == "" { + home = manifest.TargetHome + } + home, err = resolveHome(home) + if err != nil { + return nil, err + } + plistPath := filepath.Join(filepath.Dir(home), "Library", "LaunchAgents", label+".plist") + result := &UnscheduleResult{Status: "planned", Label: label, PlistPath: plistPath} + if !opts.Submit { + return result, nil + } + domain := fmt.Sprintf("gui/%d", os.Getuid()) + _ = exec.Command("launchctl", "bootout", domain+"/"+label).Run() + if err := os.Remove(plistPath); err != nil && !isNotExist(err) { + return nil, err + } + result.Status = "ok" + return result, nil +} + +func xmlEscape(value string) string { + replacer := strings.NewReplacer("&", "&", "<", "<", ">", ">", `"`, """, "'", "'") + return replacer.Replace(value) +} diff --git a/internal/profile/serve.go b/internal/profile/serve.go new file mode 100644 index 0000000..e8de787 --- /dev/null +++ b/internal/profile/serve.go @@ -0,0 +1,94 @@ +package profile + +import ( + "context" + "crypto/rand" + "encoding/hex" + "fmt" + "net" + "net/http" + "path/filepath" + "strings" +) + +type Server struct { + Listener net.Listener + HTTP *http.Server + URLs []string +} + +func NewServer(bundleDir, listen string) (*Server, error) { + bundleDir, err := filepath.Abs(bundleDir) + if err != nil { + return nil, err + } + if _, err := readManifest(bundleDir); err != nil { + return nil, err + } + if listen == "" { + listen = ":8765" + } + listener, err := net.Listen("tcp", listen) + if err != nil { + return nil, err + } + tokenBytes := make([]byte, 16) + if _, err := rand.Read(tokenBytes); err != nil { + listener.Close() + return nil, err + } + token := hex.EncodeToString(tokenBytes) + prefix := "/" + token + "/" + files := http.StripPrefix(prefix, http.FileServer(http.Dir(bundleDir))) + handler := http.HandlerFunc(func(writer http.ResponseWriter, request *http.Request) { + if !strings.HasPrefix(request.URL.Path, prefix) { + http.NotFound(writer, request) + return + } + writer.Header().Set("Cache-Control", "no-store") + files.ServeHTTP(writer, request) + }) + server := &Server{Listener: listener, HTTP: &http.Server{Handler: handler}} + port := listener.Addr().(*net.TCPAddr).Port + server.URLs = serverURLs(port, token) + return server, nil +} + +func (server *Server) Serve() error { + err := server.HTTP.Serve(server.Listener) + if err == http.ErrServerClosed { + return nil + } + return err +} + +func (server *Server) Close(ctx context.Context) error { + return server.HTTP.Shutdown(ctx) +} + +func serverURLs(port int, token string) []string { + seen := map[string]bool{} + var urls []string + add := func(host string) { + value := fmt.Sprintf("http://%s:%d/%s", host, port, token) + if !seen[value] { + seen[value] = true + urls = append(urls, value) + } + } + add("127.0.0.1") + interfaces, _ := net.Interfaces() + for _, item := range interfaces { + if item.Flags&net.FlagUp == 0 || item.Flags&net.FlagLoopback != 0 { + continue + } + addresses, _ := item.Addrs() + for _, address := range addresses { + ip, _, err := net.ParseCIDR(address.String()) + if err == nil && ip.To4() != nil { + add(ip.String()) + } + } + } + return urls +} diff --git a/internal/profile/types.go b/internal/profile/types.go new file mode 100644 index 0000000..946c032 --- /dev/null +++ b/internal/profile/types.go @@ -0,0 +1,208 @@ +package profile + +import "time" + +const Schema = "agent-capsule.codex-profile.v1" + +var DefaultProfilePaths = []string{ + "AGENTS.md", + "config.toml", + "hooks.json", + "history.jsonl", + "rules", + "skills", + "memories", + "data", + "automations", +} + +var DefaultProfileDatabases = []string{ + "memories_1.sqlite", + "goals_1.sqlite", +} + +var DefaultExclusions = []string{ + "auth.json and provider credentials", + "installation and machine identifiers", + "Keychain, cookies, and browser profiles", + "plugins/cache, logs, tmp, and shell snapshots", + "worktrees and project working trees", + "skills/.system (managed by the target Codex installation)", + "uncommitted and untracked project files", +} + +type Manifest struct { + Schema string `json:"schema"` + ID string `json:"id"` + CreatedAt time.Time `json:"created_at"` + SourceHome string `json:"source_home"` + TargetHome string `json:"target_home"` + SourceUserHome string `json:"source_user_home,omitempty"` + TargetUserHome string `json:"target_user_home,omitempty"` + SourceWorkspace string `json:"source_workspace,omitempty"` + TargetWorkspace string `json:"target_workspace,omitempty"` + Projects []Project `json:"projects"` + Threads []Thread `json:"threads"` + ProfileFiles []File `json:"profile_files"` + Exclusions []string `json:"exclusions"` + Stats ManifestStats `json:"stats"` +} + +type ManifestStats struct { + Projects int `json:"projects"` + GitRepos int `json:"git_repos"` + Threads int `json:"threads"` + ProfileFiles int `json:"profile_files"` + Bytes int64 `json:"bytes"` +} + +type Project struct { + ID string `json:"id"` + Name string `json:"name"` + SourcePath string `json:"source_path"` + TargetPath string `json:"target_path"` + Repos []GitRepo `json:"repos"` +} + +type GitRepo struct { + RelativePath string `json:"relative_path"` + TargetPath string `json:"target_path"` + Branch string `json:"branch,omitempty"` + Commit string `json:"commit,omitempty"` + Remotes map[string]string `json:"remotes,omitempty"` + Clean bool `json:"clean"` + BundlePath string `json:"bundle_path,omitempty"` + BundleSHA256 string `json:"bundle_sha256,omitempty"` + BundleBytes int64 `json:"bundle_bytes,omitempty"` +} + +type Thread struct { + ID string `json:"id"` + Title string `json:"title,omitempty"` + SourceCWD string `json:"source_cwd"` + TargetCWD string `json:"target_cwd"` + SourceSessionPath string `json:"source_session_path"` + TargetSessionRelative string `json:"target_session_relative_path"` + BundlePath string `json:"bundle_path"` + SHA256 string `json:"sha256"` + Bytes int64 `json:"bytes"` + Row map[string]any `json:"row,omitempty"` + IndexEntry map[string]any `json:"index_entry,omitempty"` +} + +type File struct { + TargetRelativePath string `json:"target_relative_path"` + BundlePath string `json:"bundle_path"` + SHA256 string `json:"sha256"` + Bytes int64 `json:"bytes"` + Mode uint32 `json:"mode"` + LinkTarget string `json:"link_target,omitempty"` +} + +type ExportOptions struct { + Home string + TargetHome string + TargetWorkspace string + Projects []string + Out string + UnsafeIncludeSecrets bool + GitBundleFallback bool +} + +type ExportResult struct { + Status string `json:"status"` + BundleDir string `json:"bundle_dir"` + ManifestPath string `json:"manifest_path"` + Manifest *Manifest `json:"manifest"` + ClonePlan []CloneAction `json:"clone_plan"` +} + +type CloneAction struct { + Project string `json:"project"` + TargetPath string `json:"target_path"` + Remote string `json:"remote,omitempty"` + Branch string `json:"branch,omitempty"` + Commit string `json:"commit,omitempty"` + Command string `json:"command,omitempty"` + NeedsBundle bool `json:"needs_bundle"` + BundleReason string `json:"bundle_reason,omitempty"` +} + +type FetchOptions struct { + Source string + Out string + IncludeGitBundles bool +} + +type FetchResult struct { + Status string `json:"status"` + BundleDir string `json:"bundle_dir"` + DownloadedFiles int `json:"downloaded_files"` + ReusedFiles int `json:"reused_files"` + DownloadedBytes int64 `json:"downloaded_bytes"` +} + +type ImportOptions struct { + BundleDir string + Home string + Execute bool + RequireStopped bool +} + +type ImportResult struct { + Status string `json:"status"` + DryRun bool `json:"dry_run"` + TargetHome string `json:"target_home"` + BackupDir string `json:"backup_dir,omitempty"` + Projects int `json:"projects"` + Threads int `json:"threads"` + ProfileFiles int `json:"profile_files"` + MissingProjects []string `json:"missing_projects,omitempty"` + Writes []string `json:"writes"` + PreservedIdentity bool `json:"preserved_target_identity"` +} + +type VerifyOptions struct { + BundleDir string + Home string +} + +type VerifyResult struct { + Status string `json:"status"` + TargetHome string `json:"target_home"` + Projects int `json:"projects"` + Threads int `json:"threads"` + ProfileFiles int `json:"profile_files"` + DatabaseCheck string `json:"database_check,omitempty"` + Failures []string `json:"failures,omitempty"` + Warnings []string `json:"warnings,omitempty"` +} + +type ScheduleOptions struct { + BundleDir string + Home string + CLIPath string + Submit bool +} + +type ScheduleResult struct { + Status string `json:"status"` + Label string `json:"label"` + StagingDir string `json:"staging_dir"` + PlistPath string `json:"plist_path"` + RunnerPath string `json:"runner_path"` + StatusPath string `json:"status_path"` + LogPath string `json:"log_path"` +} + +type UnscheduleOptions struct { + BundleDir string + Home string + Submit bool +} + +type UnscheduleResult struct { + Status string `json:"status"` + Label string `json:"label"` + PlistPath string `json:"plist_path"` +} diff --git a/internal/profile/verify.go b/internal/profile/verify.go new file mode 100644 index 0000000..6533277 --- /dev/null +++ b/internal/profile/verify.go @@ -0,0 +1,169 @@ +package profile + +import ( + "bufio" + "database/sql" + "encoding/json" + "fmt" + "os" + "path/filepath" + "strings" + + _ "modernc.org/sqlite" +) + +func Verify(opts VerifyOptions) (*VerifyResult, error) { + manifest, err := readManifest(opts.BundleDir) + if err != nil { + return nil, err + } + home := opts.Home + if home == "" { + home = manifest.TargetHome + } + home, err = resolveHome(home) + if err != nil { + return nil, err + } + result := &VerifyResult{Status: "ok", TargetHome: home, Projects: len(manifest.Projects), Threads: len(manifest.Threads), ProfileFiles: len(manifest.ProfileFiles)} + for _, project := range manifest.Projects { + for _, repo := range project.Repos { + if !isGitRepo(repo.TargetPath) { + result.Failures = append(result.Failures, "missing Git repository: "+repo.TargetPath) + } + } + } + dbPath := filepath.Join(home, "state_5.sqlite") + db, err := sql.Open("sqlite", dbPath) + if err != nil { + return nil, err + } + defer db.Close() + if err := db.QueryRow("PRAGMA integrity_check").Scan(&result.DatabaseCheck); err != nil { + result.Failures = append(result.Failures, "database integrity check failed: "+err.Error()) + } else if result.DatabaseCheck != "ok" { + result.Failures = append(result.Failures, "database integrity check: "+result.DatabaseCheck) + } + for _, thread := range manifest.Threads { + path, err := safeJoin(home, thread.TargetSessionRelative) + if err != nil { + result.Failures = append(result.Failures, err.Error()) + continue + } + if _, err := os.Stat(path); err != nil { + result.Failures = append(result.Failures, "missing session "+thread.ID) + continue + } + var cwd, rollout string + if err := db.QueryRow("select cwd, rollout_path from threads where id = ?", thread.ID).Scan(&cwd, &rollout); err != nil { + result.Failures = append(result.Failures, "missing database row "+thread.ID) + } else { + if cwd != thread.TargetCWD { + result.Failures = append(result.Failures, fmt.Sprintf("thread %s cwd is %q, want %q", thread.ID, cwd, thread.TargetCWD)) + } + if rollout != path { + result.Failures = append(result.Failures, fmt.Sprintf("thread %s rollout path is %q, want %q", thread.ID, rollout, path)) + } + } + if err := verifySessionPaths(path, manifest, home); err != nil { + result.Failures = append(result.Failures, fmt.Sprintf("thread %s: %v", thread.ID, err)) + } + } + for _, file := range manifest.ProfileFiles { + join := safeJoin + if file.LinkTarget != "" { + join = safeJoinAllowFinalSymlink + } + path, err := join(home, file.TargetRelativePath) + if err != nil { + result.Failures = append(result.Failures, err.Error()) + continue + } + if file.LinkTarget != "" { + actual, err := os.Readlink(path) + if err != nil { + result.Failures = append(result.Failures, "missing profile symlink: "+file.TargetRelativePath) + continue + } + expected := string(rewriteProfileText([]byte(file.LinkTarget), manifest, home)) + if actual != expected { + result.Failures = append(result.Failures, fmt.Sprintf("profile symlink %s points to %q, want %q", file.TargetRelativePath, actual, expected)) + } + if _, err := os.Stat(path); err != nil { + result.Failures = append(result.Failures, "broken profile symlink: "+file.TargetRelativePath) + } + continue + } + if _, err := os.Stat(path); err != nil { + result.Failures = append(result.Failures, "missing profile file: "+file.TargetRelativePath) + } + } + if err := verifyGlobalState(home, manifest); err != nil { + result.Failures = append(result.Failures, err.Error()) + } + if len(result.Failures) > 0 { + result.Status = "failed" + } + return result, nil +} + +func verifySessionPaths(path string, manifest *Manifest, home string) error { + file, err := os.Open(path) + if err != nil { + return err + } + defer file.Close() + scanner := bufio.NewScanner(file) + scanner.Buffer(make([]byte, 64*1024), 64*1024*1024) + foundTarget := false + for scanner.Scan() { + var event map[string]any + if err := json.Unmarshal(scanner.Bytes(), &event); err != nil { + return err + } + typeName := stringValue(event["type"]) + if typeName != "session_meta" && typeName != "turn_context" { + continue + } + data, _ := json.Marshal(event["payload"]) + text := string(data) + if strings.Contains(text, manifest.SourceHome) || (manifest.SourceUserHome != "" && strings.Contains(text, manifest.SourceUserHome)) || (manifest.SourceWorkspace != "" && strings.Contains(text, manifest.SourceWorkspace)) { + return fmt.Errorf("source path remains in session metadata") + } + if strings.Contains(text, home) || strings.Contains(text, manifest.TargetWorkspace) { + foundTarget = true + } + } + if err := scanner.Err(); err != nil { + return err + } + if !foundTarget { + return fmt.Errorf("target paths were not found in session metadata") + } + return nil +} + +func verifyGlobalState(home string, manifest *Manifest) error { + data, err := os.ReadFile(filepath.Join(home, ".codex-global-state.json")) + if err != nil { + return err + } + var state map[string]any + if err := json.Unmarshal(data, &state); err != nil { + return err + } + projects := mapValue(state["local-projects"]) + for _, project := range manifest.Projects { + id := deterministicProjectID(project.TargetPath) + if _, ok := projects[id]; !ok { + return fmt.Errorf("global state is missing project %s", project.Name) + } + } + assignments := mapValue(state["thread-project-assignments"]) + for _, thread := range manifest.Threads { + if _, ok := assignments[thread.ID]; !ok { + return fmt.Errorf("global state is missing thread assignment %s", thread.ID) + } + } + return nil +} diff --git a/skills/agent-capsule/SKILL.md b/skills/agent-capsule/SKILL.md index aea5001..1ce267c 100644 --- a/skills/agent-capsule/SKILL.md +++ b/skills/agent-capsule/SKILL.md @@ -1,6 +1,6 @@ --- name: agent-capsule -description: Use when an agent needs to install or use Agent Capsule to export, share, inspect, import, restore, or verify Codex and Claude Code session capsules, `.capsule.zip` files, or encrypted Agent Capsule links. Covers CLI setup, sender workflows, receiver bootstrap, approved import into local agent history, import-as-new semantics, secret-scan handling, and verification. +description: Use when an agent needs to install or use Agent Capsule to export, share, inspect, import, restore, or verify Codex and Claude Code sessions, or to migrate a Codex profile between Macs. Covers session capsules, encrypted links, target-device orchestration, project selection and cloning, controlled profile overwrite, one-shot offline import, and verification. --- # Agent Capsule @@ -14,6 +14,13 @@ prompt. The CLI is the source of truth. A `.capsule.zip` or share link must remain self-bootstrapping for agents that do not have this skill installed. +There are two separate workflows: + +- Session handoff imports one conversation as a new thread/session. +- Codex profile migration preserves selected native thread ids and overwrites a controlled profile allowlist on a disposable target installation. + +Never use session import semantics to approximate a profile migration. + ## CLI Setup Check whether the CLI is available: @@ -23,6 +30,9 @@ command -v capsule capsule help ``` +For a profile migration, also require `capsule profile help` to succeed. If the +installed binary predates the profile command group, update it before starting. + If the user asked to export, share, import, restore, inspect, or verify a capsule and the CLI is missing, install the latest released binary: ```bash @@ -67,6 +77,127 @@ If artifact export fails with a secret-scan warning, stop and tell the user what Treat a full share URL containing `#k=...` as sensitive. The URL fragment is the decryption key. +## Codex Mac-to-Mac Profile Migration + +Trigger this workflow when the user asks to migrate, move, or copy their Codex +setup from this Mac to another Mac. If the user says to use Agent Capsule and +names a connected target device, treat that as approval to orchestrate the +source and receiver steps end to end. Use Codex Remote Connections and a +target-owned Codex task when available; this is task coordination, not remote +GUI control. + +### Confirm The Boundary + +Before writing anything, establish: + +- the target Mac/host and target username; +- target `~/.codex` and project workspace paths; +- that controlled overwrite is acceptable; +- included or excluded project names. + +List candidate project names when the project boundary is not already explicit. +Use `capsule profile discover --home "${CODEX_HOME:-$HOME/.codex}"` as the +source of truth for project roots and selected-thread counts. +Project migration includes committed Git state only. Uncommitted and untracked +files are excluded unless the user separately asks for them. Never include a +generic documents directory such as `~/Documents/Codex` merely because tasks +have used it as a cwd. + +The target must have opened Codex at least once, be signed in, and have no local +data the user wants to keep. Do not migrate authentication, provider tokens, +Keychain data, installation ids, device enrollment, cookies, browser state, +managed plugins, caches, logs, worktrees, or `skills/.system`. + +### Export On The Source Mac + +Pass every approved project root explicitly. Prepare committed Git bundles as a +fallback for private remotes or local commits; they are not transferred unless +needed. + +```bash +capsule profile export \ + --target-home /Users//.codex \ + --target-workspace /Users//workspace \ + --project /path/to/project-a \ + --project /path/to/project-b \ + --git-bundle-fallback \ + --out ~/.codex/profile-migrations/ +``` + +Review the JSON clone plan and counts before continuing. Then start the +tokenized LAN server in a long-running terminal session: + +```bash +capsule profile serve ~/.codex/profile-migrations/ --listen :8765 +``` + +The current hosted Worker/R2 link API is for individual session capsules, not +multi-GiB profile directories. Use this LAN streaming path unless a future +profile-specific multipart backend is explicitly available. + +### Receive Through The Target Codex Task + +Send the reachable tokenized URL to the target-owned Codex task and run: + +```bash +capsule profile fetch --out ~/.codex/profile-migrations/ +capsule profile clone ~/.codex/profile-migrations/ +capsule profile clone ~/.codex/profile-migrations/ --execute +``` + +The first clone command is a dry run. If a remote clone fails because the target +lacks access or the exported commit was not pushed, fetch only the Git fallback +objects and retry: + +```bash +capsule profile fetch \ + --out ~/.codex/profile-migrations/ \ + --include-git-bundles +capsule profile clone ~/.codex/profile-migrations/ --execute +``` + +Do not copy project working trees over HTTP. Install or configure Git on the +target through its Codex task when needed. + +### Offline Import And Restart + +Run a dry import before scheduling the write: + +```bash +capsule profile import ~/.codex/profile-migrations/ --home ~/.codex +capsule profile schedule-import ~/.codex/profile-migrations/ \ + --home ~/.codex --execute +``` + +The schedule command stages under `~/.codex`, creates a LaunchAgent with +`RunAtLoad=true` and `KeepAlive=false`, waits briefly, quits Codex, checkpoints +the target SQLite WAL, backs up the target state, imports, verifies, writes +`import-status.json`, reopens Codex, and removes its plist. Never replace this +with `launchctl submit`, a KeepAlive job, a Downloads-hosted script, or a child +process tied to the running Codex process. + +The import preserves target authentication and remote-device identity. It +overwrites only the exported config/skill/memory/automation allowlist, keeps +`skills/.system`, rewrites source home/project paths, merges selected thread +rows, and rebuilds project/sidebar assignments. + +### Verify And Clean Up + +After the target Codex reconnects, inspect the status and run verification from +a target-owned task: + +```bash +cat ~/.codex/profile-migrations//import-status.json +capsule profile verify ~/.codex/profile-migrations/ --home ~/.codex +capsule profile unschedule ~/.codex/profile-migrations/ \ + --home ~/.codex --execute +``` + +Confirm database integrity, all selected session files and rows, rewritten cwd +and rollout paths, project Git roots, sidebar project assignments, user skills, +and MCP configuration. Report any MCP or GitHub CLI login still required on the +target. Stop the source `profile serve` process after verification. + ## Artifact Import Or Restore Accept either a local `.capsule.zip` path or an encrypted share URL. If a link is missing `#k=...`, ask for the full link before importing.