Skip to content

fix(android): #115 root cause — drop proot --link2symlink (fixes git, pnpm, all link() tools) - #116

Merged
zo-sol merged 4 commits into
mainfrom
fix/115-git-object-rename
Jul 13, 2026
Merged

fix(android): #115 root cause — drop proot --link2symlink (fixes git, pnpm, all link() tools)#116
zo-sol merged 4 commits into
mainfrom
fix/115-git-object-rename

Conversation

@mega123-art

@mega123-art mega123-art commented Jul 13, 2026

Copy link
Copy Markdown
Contributor

What

Fix the #112/#115 git object-loss bug at its root: write a system /etc/gitconfig into the guest with core.createObject = rename, so git stops using the link() syscall that proot silently breaks.

One line of effect (Installer.kt):

writeFresh(File(p.rootfs, "etc/gitconfig"), "[core]\n\tcreateObject = rename\n")

Why

Under the untrusted_app SELinux domain (targetSdk 35, required for Play), proot's link() is a false success — it returns 0 but the target file never appears. git's default loose-object finalize path (git 2.43 object-file.c) is mkstemp → fsync → link(tmp,final) → unlink(tmp); git trusts the 0 from link, reports the object written, and the object is silently lost. This surfaces as fatal: remote did not send all necessary objects / bad object on git clone, and would hit commit/fetch/pull the same way.

core.createObject=rename sets OBJECT_CREATION_USES_RENAMES, which makes git skip link() entirely and use rename() — which proot handles correctly.

Proof (on-device, headless, untrusted_app — SM-A356E / Android 16)

Root cause caught at the syscall (probe9):

link ret=0 errno=0 target_exists_after=0   tmp_obj_mvrgkg -> objects/f0/6b470c...
hash-object exit=0        # git thinks it worked
count=0                   # nothing on disk

Fix proven (probe10):

link mode  (default):            50 written, survivors_on_disk=0    ← bug reproduced
rename mode (core.createObject):  50 written, survivors_on_disk=50   ← fixed

Verified through the shipping path (system /etc/gitconfig, plain git clone, no per-command flag):

git config --show-origin core.createObject  ->  file:/etc/gitconfig  rename
git clone https://github.com/expressjs/express.git  ->  exit 0, done.
HEAD=ae6dd376   in-pack: 53322   packs: 1   checkout: OK   worktree files: 213   fsck errors: 0

Relation to other work

Also in this branch (debug kit, not shipped code)

  • debug/issue-115/probes/probe10.sh — the fix test (reproduce in link mode → prove in rename mode → real clone).
  • debug/issue-115/shims/liblinkfix.c — a broader LD_PRELOAD option (returns EXDEV on object-path link/linkat, forcing the caller's own rename fallback) for any non-git tool that hits the same false-success and has no config knob.
  • debug/issue-115/README.md — the fix ladder (config → shim → proot patch).

Headless reproduction used for all the evidence above lives on feat/115-native-exec-probe (NativeProbeActivity guest-script mode: am start … -e guestScript /root/probeN.sh runs a probe through proot from the app process, so git inherits untrusted_app — same domain as the in-app chat, drivable over pure adb, no agent auth). Kept separate from this shippable fix.

Test plan

  1. Fresh install (or clear app data so the installer re-runs configureGuest).
  2. In-app: git clone https://github.com/expressjs/express.git → completes, git fsck clean.
  3. Confirm the config is live: guest git config --show-origin core.createObjectfile:/etc/gitconfig rename.

mega123-art and others added 4 commits July 13, 2026 11:15
…ect=rename

Teed up on top of Zo's root-cause hunt (link() false-success on .git/objects
under untrusted_app). Verified against git v2.43.0 object-file.c: with
core.createObject=rename git skips link() entirely and uses rename(), which
survives proot (probe6). So the interim fix is one line of git config — no
shim, no proot rebuild.

- probes/probe10.sh: reproduces the loss in link mode, shows it gone in rename
  mode, then a real network clone with the fix. Run right after probe9 confirms.
- shims/liblinkfix.c: broader LD_PRELOAD option (EXDEV on object-path link/
  linkat → caller's rename fallback) for non-git tools that lack the config knob.
- README: fix ladder — (1) core.createObject=rename config, (2) liblinkfix shim,
  (3) basement proot linkat patch.

All gated on probe9 naming link as the culprit.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
… loss

proot's link() is a false success under untrusted_app: it returns 0 but the
target file never appears (proven on-device SM-A356E: `link ret=0
target_exists_after=0`, 0/50 loose objects survive). git's default finalize
(mkstemp -> link -> unlink) trusts the 0 and reports success, so clone/commit/
fetch silently lose objects ("remote did not send all necessary objects").

Write a system /etc/gitconfig at install with core.createObject=rename, which
makes git skip link() and use rename() (which proot handles correctly). Covers
every git — the node server's and the user's interactive shell — not just clone.

Verified end-to-end via the shipping path (system gitconfig, plain `git clone`,
no per-command flag): express.git clones exit 0, 53322 objects in-pack, clean
checkout, 0 fsck errors. Supersedes the clone-only dulwich shim (#114); the
all-tools basement fix (proot linkat patch) stays tracked in #115.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…aunch

configureGuest only runs on a fresh MARKER, so the #115 git fix (previous
commit) would miss devices that installed before it: an APK update hits the
isInstalled early return and never rewrites /etc. Extract the write into
writeGuestGitConfig() and also call it on the already-installed path —
idempotent 30-byte write, far cheaper than a MARKER bump + rootfs re-extract.

Verified on-device (SM-A356E): app already installed ("already installed"
log, no re-extract) → delete /etc/gitconfig, relaunch → recreated; fresh
install writes it via configureGuest as before.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Root cause, proven on-device (SM-A356E, untrusted_app): the kernel denies
hardlinks in the untrusted_app domain (`ln a b` -> EACCES). --link2symlink
catches that and FAKES success (pokes syscall result 0) via a symlink-refcount
scheme whose links dangle in the guest namespace. The fake defeats callers'
error handling: git's finalize_object_file falls back to rename, and pnpm's
store-linker falls back to copy, ONLY when link FAILS — never when it lies with
a 0. Hence silent git object loss and broken pnpm installs.

Removing --link2symlink makes linkat fail honestly with EACCES, so every tool
with a link fallback recovers on its own. Verified on-device with l2s off:
- git (forced link mode): 50/50 loose objects survive (rename fallback)
- pnpm (default hardlink): 589/589 files, require OK (copy fallback)
- node server still boots: server ready (HTTP 200) — no regression

This is the basement fix: one flag removed fixes git + pnpm + any link()-using
tool, with no proot rebuild and no per-tool config. The core.createObject=rename
gitconfig (prior commits) stays as belt-and-suspenders (skips the doomed link
attempt for git). Supersedes the clone-only dulwich shim (#114).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@mega123-art mega123-art changed the title fix(android): end #115 git object loss — core.createObject=rename in guest gitconfig fix(android): #115 root cause — drop proot --link2symlink (fixes git, pnpm, all link() tools) Jul 13, 2026
@mega123-art

Copy link
Copy Markdown
Contributor Author

Update: found the real root cause — it's --link2symlink, and the fix is to remove it

The core.createObject=rename gitconfig (first commits) fixes git, but git isn't the only victim. Dug to the actual root cause; it's one flag, and removing it fixes every tool at once.

Root cause (proven on-device, SM-A356E, untrusted_app)

In the untrusted_app domain the kernel denies hardlinks: ln a bPermission denied, links=1. We launch proot with --link2symlink, whose extension catches that denial and fakes success — it pokes the syscall result to 0 and swaps in a symlink-refcount scheme (moving the file into PROOT_L2S_DIR, replacing the target with a symlink). That fake defeats callers' own error handling:

  • git finalize_object_file falls back to rename only when link fails — never when it returns a lying 0.
  • pnpm's store-linker falls back to copy only when the hardlink fails — never when it lies.

So both get a 0, skip their fallback, and the symlink dangles → silent git object loss and broken pnpm installs. This is exactly probe9's link ret=0 target_exists_after=0.

The fix: drop --link2symlink

Then linkat fails honestly with EACCES and every tool with a fallback recovers itself. Verified on-device with l2s off:

ln a b:                     Permission denied  (honest — no more fake success)
git (forced link mode):     50/50 loose objects survive   (rename fallback)
pnpm (default hardlink):    589/589 files, require OK      (copy fallback)
node server boot:           server ready (HTTP 200)        (no regression)

One flag removed fixes git + pnpm + any link()-using tool — no proot rebuild, no per-tool config. This is the "fix the basement" answer: the defect was proot faking a syscall, and we stop it faking.

What this PR now contains

  1. DirectProotExec: remove --link2symlink — the root-cause fix, all tools.
  2. Installer: core.createObject=rename system gitconfig — kept as belt-and-suspenders for git (skips the doomed link attempt entirely), and written on every launch so existing installs get it without a rootfs re-extract.

Supersedes the clone-only dulwich shim (#114) — real git, all commands, plus pnpm and the rest. #115 T3 (remove the dulwich shim) can proceed.

Note on the earlier "l2s isn't it" elimination

The handoff eliminated --link2symlink because ln showed links=2 (a real hardlink). That test was in runas_app, where hardlinks are allowed — so l2s passed the real link through untouched and looked innocent. The bug only appears in untrusted_app, where the real link is denied and l2s's fake-success path activates. Same reason the whole bug is domain-specific.

@zo-sol
zo-sol merged commit 2e87263 into main Jul 13, 2026
zo-sol added a commit that referenced this pull request Jul 13, 2026
PROOT_L2S_DIR pointed proot at a l2s db the extension no longer loads
(--link2symlink was removed as the #115 root cause), and the launcher
doc comment still described l2s as 'guest compatibility'. Drop the dead
env var and correct the comment; note that pre-fix installs may keep
orphaned .l2s.* files, which are harmless.
mega123-art added a commit that referenced this pull request Jul 14, 2026
…enders)

The copy-on-link work removed --backend=copyfile from the bun wrapper to rely
solely on PRoot's new --copy-on-link. But that flag only takes effect once the
source-built PRoot ships (DirectProotExec gates it on the binary containing the
option string); with today's prebuilt binary the wrapper would run bun's default
hardlink backend -> kernel EACCES -> "Failed to install N packages", empty
node_modules (proven on-device). Restore --backend=copyfile: it is correct with
OR without --copy-on-link (same result, bun just copies in userspace) and removes
the regression window, mirroring how #116 keeps core.createObject=rename as
belt-and-suspenders. node_modules pre-create is also unrelated to hardlinks, so
--copy-on-link never covered it — kept.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
mega123-art added a commit that referenced this pull request Jul 14, 2026
…/all-tools basement)

The durable basement fix for the whole hardlink class. Under untrusted_app the
kernel denies hardlinks (EACCES); tools with a copy/rename fallback recover once
--link2symlink is gone (#116), but dpkg has NO fallback — link(status,status-old)
-> EACCES leaves apt half-configured (proven on-device). Per-tool nudges don't
scale to dpkg/coreutils, so fix it in PRoot.

- surfaces/android/proot/: GPLv2 patch adding an opt-in --copy-on-link extension.
  PRoot attempts the real link()/linkat() first; ONLY on EACCES it byte-copies a
  regular source file to a fresh O_EXCL destination (snapshot, not inode-sharing)
  and reports success. Special files, symlink sources, AT_EMPTY_PATH, and other
  errno keep normal behavior. README pins the reproducible source inputs.
- android-assets.yml: build PRoot from the pinned Termux recipe + this patch via
  the official package-builder, export PROOT_DEB to the asset step.
- build-assets.sh: fetch_deb accepts the source-built .deb path (PROOT_DEB) as well
  as the runtime-lib URLs.
- DirectProotExec: pass --copy-on-link, but ONLY if the shipped binary advertises
  it (grep the option string) — an older reused android-assets artifact never gets
  an unknown flag that would abort the guest boot.

Not yet built/verified on-device — needs an android-assets run to produce the
patched binary, then on-device: link EACCES copies, git clone, bun add, and
apt install + dpkg --configure -a all clean.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
mega123-art added a commit that referenced this pull request Jul 14, 2026
The shim (guest/git-clone-shim.sh -> /usr/local/bin/git, routing `git clone`
through dulwich) worked around #112 by using create+rename instead of git's
link path. The actual root cause — proot's --link2symlink faking link()
success under untrusted_app — is now fixed at the launch layer (#115/#116,
DirectProotExec drops --link2symlink), so native git clone works with real git.

Removes the shim install + python3-dulwich dep from build-assets.sh and deletes
guest/git-clone-shim.sh + guest/agentnet-git-clone.py. Real /usr/bin/git is used
for every command.

GATE (before merge): verify real `git clone` end-to-end in the untrusted_app
domain with l2s removed (hash-object 50/50 already confirms real git's object
finalize works post-fix; clone adds index-pack over the same path). Takes effect
on the next rootfs rebuild; existing installs keep the (now harmless) shim until
a rootfs re-extract.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
zo-sol added a commit that referenced this pull request Jul 15, 2026
The proot investigation is resolved and shipped (#116/#117/#122); these debug
probes and experimental C shims are no longer needed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants