From 0b2dfc6bd4927ee0b925659d57fbd351ca485497 Mon Sep 17 00:00:00 2001 From: mega123-art Date: Mon, 13 Jul 2026 11:15:47 +0530 Subject: [PATCH 1/4] =?UTF-8?q?debug(#115):=20candidate=20fix=20for=20git?= =?UTF-8?q?=20object-write=20loss=20=E2=80=94=20core.createObject=3Drename?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Teed up on top of Zo's root-cause hunt (link() false-success on .git/objects under untrusted_app). Verified against git v2.43.0 object-file.c: with core.createObject=rename git skips link() entirely and uses rename(), which survives proot (probe6). So the interim fix is one line of git config — no shim, no proot rebuild. - probes/probe10.sh: reproduces the loss in link mode, shows it gone in rename mode, then a real network clone with the fix. Run right after probe9 confirms. - shims/liblinkfix.c: broader LD_PRELOAD option (EXDEV on object-path link/ linkat → caller's rename fallback) for non-git tools that lack the config knob. - README: fix ladder — (1) core.createObject=rename config, (2) liblinkfix shim, (3) basement proot linkat patch. All gated on probe9 naming link as the culprit. Co-Authored-By: Claude Opus 4.8 --- debug/issue-115/README.md | 24 +++++++++++++++++ debug/issue-115/probes/probe10.sh | 41 ++++++++++++++++++++++++++++++ debug/issue-115/shims/liblinkfix.c | 35 +++++++++++++++++++++++++ 3 files changed, 100 insertions(+) create mode 100644 debug/issue-115/probes/probe10.sh create mode 100644 debug/issue-115/shims/liblinkfix.c diff --git a/debug/issue-115/README.md b/debug/issue-115/README.md index fa36ad8..e080caa 100644 --- a/debug/issue-115/README.md +++ b/debug/issue-115/README.md @@ -38,6 +38,30 @@ Decision table is in the handoff comment (link at top). Short version: if the lo `link ret=0 ... target_exists_after=0`, proot's `link` is a false success — that is the root cause. +## Candidate fix — run probe10 right after probe9 confirms link + +git 2.43 `finalize_object_file()` (object-file.c, verified against v2.43.0): with +`core.createObject=rename` (`OBJECT_CREATION_USES_RENAMES`) it **skips `link()` entirely** and +uses `rename()`, which works under proot (probe6). So the interim fix is **one line of git +config — no shim, no proot rebuild**: + +``` +run sh /root/probe10.sh and save the full output to /root/probe10_result.txt +``` +probe10 reproduces the loss in link mode, shows it gone in rename mode, then does a real +network clone with the fix. Verdict: fixed iff rename-mode survivors=50 AND clone exit=0. + +**Fix ladder (apply in order once probe9 names `link`):** +1. **Config, git-only, zero code** — `core.createObject=rename` in the guest (system gitconfig + or `ServerManager.buildGuestEnv` via `GIT_CONFIG_*`). Fixes the shipping bug (git is the only + proven-affected tool). Ship this. +2. **`shims/liblinkfix.c`, all tools** — LD_PRELOAD that returns EXDEV on object-path `link`/ + `linkat`, forcing the caller's own rename fallback. Use only if a non-git tool (npm/cargo) + hits the same link false-success and has no config knob. +3. **Basement, proot `linkat` patch** — fix the false-success in our own proot build (we own it + from the Seeker process_vm fix; GPLv2 → publish the patch). The true fix: covers every tool, + no per-tool workaround. This is the long-term "fix the basement" answer. + ## How to stage these on a device ```bash diff --git a/debug/issue-115/probes/probe10.sh b/debug/issue-115/probes/probe10.sh new file mode 100644 index 0000000..acf8d3f --- /dev/null +++ b/debug/issue-115/probes/probe10.sh @@ -0,0 +1,41 @@ +# issue #115 — CANDIDATE FIX test. Gated on probe9 first confirming `link ret=0 +# target_exists_after=0` for .git/objects (proot's link() is a false success). +# +# git 2.43 finalize_object_file() (object-file.c): if core.createObject=rename +# (OBJECT_CREATION_USES_RENAMES) it `goto try_rename` and SKIPS link() entirely, using +# rename() — which works under proot (probe6: 50 renames survive, 50 link-writes vanish). +# So the interim fix is one line of git config: no LD_PRELOAD shim, no proot rebuild. +# probe10 proves it end to end: reproduce the loss in link mode, show it gone in rename mode, +# then a real network clone with the fix. +# +# Run as a TRUE app child (paste into the in-app agent chat), never run-as: +# run sh /root/probe10.sh and save the full output to /root/probe10_result.txt +# then: adb shell "run-as com.iqlabs.agentnet cat files/rootfs/root/probe10_result.txt" +set -u + +echo "=== baseline: link mode (git default) — expect objects to VANISH ===" +rm -rf /root/o_link; git init -q /root/o_link; cd /root/o_link +ok=0; for i in $(seq 1 50); do git hash-object -w --stdin </dev/null 2>&1 && ok=$((ok+1)) +obj-$i +EOF +done +echo "link mode : hash-object reported ok=$ok/50 survivors_on_disk=$(find .git/objects -type f | wc -l)" + +echo "=== fix: core.createObject=rename — expect all 50 to SURVIVE ===" +rm -rf /root/o_rename; git init -q /root/o_rename; cd /root/o_rename +git config core.createObject rename +ok=0; for i in $(seq 1 50); do git hash-object -w --stdin </dev/null 2>&1 && ok=$((ok+1)) +obj-$i +EOF +done +echo "rename mode: hash-object reported ok=$ok/50 survivors_on_disk=$(find .git/objects -type f | wc -l)" + +echo "=== real clone WITH the fix (global) — expect exit 0 + clean fsck ===" +git config --global core.createObject rename +cd /root; rm -rf exq +git clone -q https://github.com/expressjs/express.git exq 2>&1 | tail -3; echo "clone exit=$?" +( cd exq 2>/dev/null && git fsck 2>&1 | tail -3 ) +git config --global --unset core.createObject 2>/dev/null + +echo "=== VERDICT: fix works iff rename-mode survivors=50 AND clone exit=0 AND fsck clean ===" +echo "=== DONE ===" diff --git a/debug/issue-115/shims/liblinkfix.c b/debug/issue-115/shims/liblinkfix.c new file mode 100644 index 0000000..b49ee8d --- /dev/null +++ b/debug/issue-115/shims/liblinkfix.c @@ -0,0 +1,35 @@ +/* issue #115 — CANDIDATE FIX (broad). Gated on probe9 confirming proot's link() is a false + * success on .git/objects paths (link ret=0, target_exists_after=0). + * + * PREFER the config fix first (probe10): `git config core.createObject rename` skips link() + * for git with zero native code. Use THIS shim only if a NON-git tool (npm/cargo/…) also hits + * the same link false-success and has no equivalent config — it covers every tool at once. + * + * Mechanism: git 2.43 finalize_object_file() falls back to rename() on ANY link error except + * EEXIST (object-file.c, verified). rename() works under proot (probe6). So for object-path + * links we DON'T call the broken proot link — we return EXDEV, forcing the caller's own tested + * rename fallback. Scoped to "/objects/" so real hardlinks elsewhere are untouched. Covers both + * link() and linkat() in case git/the tool issues the *at form. + * + * Stage like the other shims (guest is aarch64 GLIBC, no NDK): + * zig cc -target aarch64-linux-gnu -shared -fPIC -O2 -o liblinkfix.so liblinkfix.c + * then LD_PRELOAD=/root/liblinkfix.so . + */ +#define _GNU_SOURCE +#include +#include +#include +#include +#include + +static int obj(const char *p) { return p && strstr(p, "/objects/") != 0; } + +int link(const char *a, const char *b) { + if (obj(a) || obj(b)) { errno = EXDEV; return -1; } /* force the caller's rename fallback */ + return syscall(SYS_linkat, AT_FDCWD, a, AT_FDCWD, b, 0); +} + +int linkat(int fda, const char *a, int fdb, const char *b, int flags) { + if (obj(a) || obj(b)) { errno = EXDEV; return -1; } + return syscall(SYS_linkat, fda, a, fdb, b, flags); +} From 44b34210a266051a9b0f3165d2bc14f21b486b35 Mon Sep 17 00:00:00 2001 From: mega123-art Date: Mon, 13 Jul 2026 11:53:55 +0530 Subject: [PATCH 2/4] =?UTF-8?q?fix(android):=20git=20core.createObject=3Dr?= =?UTF-8?q?ename=20in=20guest=20=E2=80=94=20end=20#115=20object=20loss?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit proot's link() is a false success under untrusted_app: it returns 0 but the target file never appears (proven on-device SM-A356E: `link ret=0 target_exists_after=0`, 0/50 loose objects survive). git's default finalize (mkstemp -> link -> unlink) trusts the 0 and reports success, so clone/commit/ fetch silently lose objects ("remote did not send all necessary objects"). Write a system /etc/gitconfig at install with core.createObject=rename, which makes git skip link() and use rename() (which proot handles correctly). Covers every git — the node server's and the user's interactive shell — not just clone. Verified end-to-end via the shipping path (system gitconfig, plain `git clone`, no per-command flag): express.git clones exit 0, 53322 objects in-pack, clean checkout, 0 fsck errors. Supersedes the clone-only dulwich shim (#114); the all-tools basement fix (proot linkat patch) stays tracked in #115. Co-Authored-By: Claude Opus 4.8 --- .../src/main/java/com/iqlabs/agentnet/Installer.kt | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/surfaces/android/app/src/main/java/com/iqlabs/agentnet/Installer.kt b/surfaces/android/app/src/main/java/com/iqlabs/agentnet/Installer.kt index 68dd622..038ac24 100644 --- a/surfaces/android/app/src/main/java/com/iqlabs/agentnet/Installer.kt +++ b/surfaces/android/app/src/main/java/com/iqlabs/agentnet/Installer.kt @@ -163,6 +163,17 @@ class Installer(private val ctx: Context) { // first, then write a plain file. Same for /etc/hosts. writeFresh(File(p.rootfs, "etc/resolv.conf"), "nameserver 8.8.8.8\nnameserver 8.8.4.4\n") writeFresh(File(p.rootfs, "etc/hosts"), "127.0.0.1 localhost\n::1 localhost\n") + // #115: under untrusted_app, proot's link() is a FALSE success — it returns 0 but the + // target file never appears (proven on-device: `link ret=0 target_exists_after=0`). git's + // default loose-object finalize is mkstemp -> link(tmp,final) -> unlink(tmp); it trusts + // link's 0 and reports the object written, so clone/commit/fetch silently lose objects + // ("remote did not send all necessary objects" / "bad object"). core.createObject=rename + // makes git skip link() entirely and use rename(), which proot handles correctly (verified: + // 0/50 loose objects survive with link, 50/50 with rename, real clone then succeeds). System + // gitconfig so EVERY git — the server's and the user's interactive shell — gets it. This + // supersedes the clone-only dulwich shim (#114). The basement fix (patching proot's linkat + // for all tools) is tracked separately in #115. + writeFresh(File(p.rootfs, "etc/gitconfig"), "[core]\n\tcreateObject = rename\n") val tmp = File(p.rootfs, "tmp").apply { mkdirs() } runCatching { android.system.Os.chmod(tmp.absolutePath, 0b001_111_111_111) } // 1777 } From f10dffa9bf643ae0b03b8c2d6419dc9160ff0d06 Mon Sep 17 00:00:00 2001 From: mega123-art Date: Mon, 13 Jul 2026 12:30:33 +0530 Subject: [PATCH 3/4] =?UTF-8?q?fix(android):=20reach=20existing=20installs?= =?UTF-8?q?=20=E2=80=94=20write=20guest=20gitconfig=20every=20launch?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit configureGuest only runs on a fresh MARKER, so the #115 git fix (previous commit) would miss devices that installed before it: an APK update hits the isInstalled early return and never rewrites /etc. Extract the write into writeGuestGitConfig() and also call it on the already-installed path — idempotent 30-byte write, far cheaper than a MARKER bump + rootfs re-extract. Verified on-device (SM-A356E): app already installed ("already installed" log, no re-extract) → delete /etc/gitconfig, relaunch → recreated; fresh install writes it via configureGuest as before. Co-Authored-By: Claude Opus 4.8 --- .../java/com/iqlabs/agentnet/Installer.kt | 31 ++++++++++++------- 1 file changed, 20 insertions(+), 11 deletions(-) diff --git a/surfaces/android/app/src/main/java/com/iqlabs/agentnet/Installer.kt b/surfaces/android/app/src/main/java/com/iqlabs/agentnet/Installer.kt index 038ac24..86683b1 100644 --- a/surfaces/android/app/src/main/java/com/iqlabs/agentnet/Installer.kt +++ b/surfaces/android/app/src/main/java/com/iqlabs/agentnet/Installer.kt @@ -82,6 +82,11 @@ class Installer(private val ctx: Context) { val p = Paths.layout(ctx) val serverCrc = assetCrc("agentnet-server.tar") if (isInstalled()) { + // #115: reach devices that installed BEFORE this fix. configureGuest (below) only + // runs on a fresh marker, so an APK update alone would leave existing rootfs without + // /etc/gitconfig and git would stay broken. This write is idempotent + tiny, so do it + // on every launch — far cheaper than a MARKER bump + full rootfs re-extract. + writeGuestGitConfig(p) // Heavy artifacts (proot + rootfs) are in place. But the server bundle changes // every build — refresh it if this APK shipped a different one. if (serverUpToDate(serverCrc)) { @@ -163,21 +168,25 @@ class Installer(private val ctx: Context) { // first, then write a plain file. Same for /etc/hosts. writeFresh(File(p.rootfs, "etc/resolv.conf"), "nameserver 8.8.8.8\nnameserver 8.8.4.4\n") writeFresh(File(p.rootfs, "etc/hosts"), "127.0.0.1 localhost\n::1 localhost\n") - // #115: under untrusted_app, proot's link() is a FALSE success — it returns 0 but the - // target file never appears (proven on-device: `link ret=0 target_exists_after=0`). git's - // default loose-object finalize is mkstemp -> link(tmp,final) -> unlink(tmp); it trusts - // link's 0 and reports the object written, so clone/commit/fetch silently lose objects - // ("remote did not send all necessary objects" / "bad object"). core.createObject=rename - // makes git skip link() entirely and use rename(), which proot handles correctly (verified: - // 0/50 loose objects survive with link, 50/50 with rename, real clone then succeeds). System - // gitconfig so EVERY git — the server's and the user's interactive shell — gets it. This - // supersedes the clone-only dulwich shim (#114). The basement fix (patching proot's linkat - // for all tools) is tracked separately in #115. - writeFresh(File(p.rootfs, "etc/gitconfig"), "[core]\n\tcreateObject = rename\n") + writeGuestGitConfig(p) val tmp = File(p.rootfs, "tmp").apply { mkdirs() } runCatching { android.system.Os.chmod(tmp.absolutePath, 0b001_111_111_111) } // 1777 } + // #115: under untrusted_app, proot's link() is a FALSE success — returns 0 but the target + // file never appears (proven on-device: `link ret=0 target_exists_after=0`). git's default + // loose-object finalize is mkstemp -> link(tmp,final) -> unlink(tmp); git trusts link's 0 and + // reports the object written, so clone/commit/fetch silently lose objects ("remote did not + // send all necessary objects" / "bad object"). core.createObject=rename makes git skip link() + // and use rename(), which proot handles correctly (verified: 0/50 loose objects survive with + // link, 50/50 with rename, real clone succeeds). System gitconfig so EVERY git — the server's + // and the user's interactive shell — gets it. Supersedes the clone-only dulwich shim (#114); + // the all-tools basement fix (patch proot's linkat) is tracked separately in #115. + private fun writeGuestGitConfig(p: Paths.Layout) { + runCatching { writeFresh(File(p.rootfs, "etc/gitconfig"), "[core]\n\tcreateObject = rename\n") } + .onFailure { Log.w(TAG, "could not write guest /etc/gitconfig (#115 fix)", it) } + } + // Write `text` to `file`, first removing any existing symlink/file at that path so // we never follow a dangling symlink (e.g. Ubuntu's /etc/resolv.conf link). private fun writeFresh(file: File, text: String) { From cd712e2c8dbe4ba856ac703dbd2923a3dcf597db Mon Sep 17 00:00:00 2001 From: mega123-art Date: Mon, 13 Jul 2026 13:32:34 +0530 Subject: [PATCH 4/4] =?UTF-8?q?fix(android):=20drop=20--link2symlink=20?= =?UTF-8?q?=E2=80=94=20the=20real=20#115=20root=20cause=20(all=20tools)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Root cause, proven on-device (SM-A356E, untrusted_app): the kernel denies hardlinks in the untrusted_app domain (`ln a b` -> EACCES). --link2symlink catches that and FAKES success (pokes syscall result 0) via a symlink-refcount scheme whose links dangle in the guest namespace. The fake defeats callers' error handling: git's finalize_object_file falls back to rename, and pnpm's store-linker falls back to copy, ONLY when link FAILS — never when it lies with a 0. Hence silent git object loss and broken pnpm installs. Removing --link2symlink makes linkat fail honestly with EACCES, so every tool with a link fallback recovers on its own. Verified on-device with l2s off: - git (forced link mode): 50/50 loose objects survive (rename fallback) - pnpm (default hardlink): 589/589 files, require OK (copy fallback) - node server still boots: server ready (HTTP 200) — no regression This is the basement fix: one flag removed fixes git + pnpm + any link()-using tool, with no proot rebuild and no per-tool config. The core.createObject=rename gitconfig (prior commits) stays as belt-and-suspenders (skips the doomed link attempt for git). Supersedes the clone-only dulwich shim (#114). Co-Authored-By: Claude Opus 4.8 --- .../java/com/iqlabs/agentnet/DirectProotExec.kt | 13 ++++++++++++- 1 file changed, 12 insertions(+), 1 deletion(-) diff --git a/surfaces/android/app/src/main/java/com/iqlabs/agentnet/DirectProotExec.kt b/surfaces/android/app/src/main/java/com/iqlabs/agentnet/DirectProotExec.kt index 0dee758..c8f4c8a 100644 --- a/surfaces/android/app/src/main/java/com/iqlabs/agentnet/DirectProotExec.kt +++ b/surfaces/android/app/src/main/java/com/iqlabs/agentnet/DirectProotExec.kt @@ -95,7 +95,18 @@ class DirectProotExec(private val layout: Paths.Layout) : GuestExec { val guestArgv = listOf( layout.proot, "--kill-on-exit", - "--link2symlink", // app storage has no hardlinks; proot fakes them + // #115 ROOT CAUSE + fix: DO NOT add --link2symlink. In the untrusted_app domain the + // kernel denies hardlinks (verified on-device: `ln a b` -> EACCES). --link2symlink + // "helpfully" catches that and FAKES success (pokes syscall result 0) by swapping in a + // symlink-refcount scheme — but the fake defeats callers' own error handling: git's + // finalize_object_file and pnpm's store-linker both fall back to rename/copy when link + // FAILS, but never when it lies with a 0. Result: silent object loss ("remote did not + // send all necessary objects") and broken pnpm installs. With l2s OFF, linkat fails + // honestly with EACCES and both tools fall back correctly (verified on-device: git + // 50/50 objects survive, pnpm 589/589 files, real clone clean). One flag removed fixes + // every link()-using tool with a fallback — no proot rebuild, no per-tool config. + // (The core.createObject=rename gitconfig from #116 is now belt-and-suspenders: it just + // skips the doomed link attempt for git specifically.) // NOTE: kept to flags the Termux proot build supports. --sysvipc is dropped // (node doesn't need SysV IPC). -L and --kernel-release are likewise omitted as // non-essential (add back only if a specific build is confirmed to accept them).