Fix #112: git clone in the proot guest under targetSdk-35

[mega123-art]

Fixes #112.

Summary

git clone fails for every repo inside the app's proot guest on Android — fatal: remote did not send all necessary objects. This PR fixes it by routing git clone through dulwich (pure-Python git) in the guest, which clones in a single process and sidesteps the proot failure. Verified working on-device (Samsung Galaxy A35, Android 15, targetSdk 35).

Root cause (verified on-device, not theory)

The trigger is the targetSdk 28 → 35 bump we made preparing for Play Store — not the Ubuntu 24.04 / glibc / rseq theory from the issue thread.

targetSdk selects the app's SELinux domain: 28 → the grandfathered-looser untrusted_app_28, 35 → the strict untrusted_app. Under the strict domain, proot corrupts git's multi-process clone path specifically:

• the smart-HTTP pack fetch (git → git-remote-https → index-pack, concurrent + piped) truncates the packfile right after the PACK header → empty object store → "did not send all necessary objects";
• freshly written objects are briefly invisible across sibling git subprocesses (an index-pack output that a following git can't read in the same proot session).

Everything else is fine in-app: curl (GET and the raw git-upload-pack POST — full pack), node/the server, git ls-remote, and single git commands all work. Only git's multi-process object machinery breaks.

What was ruled out (empirically, on the device)

Hypothesis	Result
Ubuntu 24.04 / glibc / rseq (the issue's theory)	❌ 22.04 rootfs fails identically
HTTP/2 vs HTTP/1.1, protocol.version, pack.threads	❌ no change
Filesystem / link2symlink corruption	❌ repo skeleton + pack write fine; problem is the fetch
PROOT_NO_SECCOMP (full-ptrace mode)	❌ node boots, but clone still fails
Socket / pipe transport	❌ 10 MB download ✓, 200 MB pipe ✓, POST pack ✓

The clean A/B: same code, same rootfs — targetSdk 28 clones successfully, targetSdk 35 fails. Since Play requires targetSdk 35 for new apps, a downgrade isn't an option, and there's no proot flag that fixes multi-process tracing under untrusted_app (this is the same wall Termux hit — why it stays targetSdk 28 and off Play).

The fix

A git shim installed ahead of /usr/bin/git on the guest PATH:

• git clone … → dulwich porcelain.clone — one process, plain file I/O (the path that works under proot here). No git-remote-https, no concurrent index-pack, no cross-subprocess handoff.
• every other git subcommand → passes through to the real git unchanged.

Changes

• build-assets.sh: install python3-dulwich; install the shim as /usr/local/bin/git.
• guest/git-clone-shim.sh, guest/agentnet-git-clone.py: the shim + dulwich clone.
• Installer: bump rootfs marker v4 → v5 so existing installs re-extract the rootfs carrying the fix.
• .gitignore: ignore the generated agentnet-rootfs.env manifest.

Verification

On-device (Galaxy A35, targetSdk-35 build), git clone https://github.com/octocat/Hello-World.git:

• HEAD resolves to the correct commit, packfile present, working tree checked out, git fsck clean.

Notes / follow-ups

• Ships to devices only after android-assets re-runs — the shim + dulwich live in the ~1 GB rootfs tar, which is rebuilt by that workflow; android-apk then bundles it and marker v5 re-extracts on update.
• Scope: this shims clone. git fetch / pull / push still use native git and likely hit the same targetSdk-35 proot bug — out of scope for git clone always fails in proot (Android) guest — "remote did not send all necessary objects" (suspected Ubuntu 24.04 rseq regression) #112 but the same root cause; dulwich covers them too if we want a follow-up.
• The branch is named …rootfs-rollback from the original (disproven) theory; the fix here is unrelated to the rollback.

[zo-sol]

I ran a controlled failure-isolation matrix on the Seeker (the device #112 was filed from) that bears directly on this PR's root-cause claim, plus a code review of the change set.

Measurement: the corruption is network-timing-dependent

Setup: the app's exact proot invocation (same binary, flags, env, run under the app UID via run-as), cloning real GitHub over a USB-reversed CONNECT proxy on the host so link quality is a controlled variable. The proxy's byte integrity is checksum-verified (identical sha256 through-proxy vs direct), and the same adb-reverse path passes 8/8 on a clean link — so neither the proxy nor the USB transport corrupts data.

Config	rseq disabled (GLIBC_TUNABLES)	--link2symlink	Link	Result
Clean-link sweep (file://, local smart-HTTP, throttled local, real GitHub, concurrent app proot)	both states	both states	clean/fast	48/48 pass
D (pre-fix state)	no (rseq active)	on	flaky (16KB chunks, 10% chance of 200-900ms stalls)	2 fails / 6 clones — curl 56 GnuTLS recv error (-9): Error decoding the received TLS packet
A (current main)	yes	on	same flaky	8/8 pass (P ≈ 4% of that happening by chance at D's failure rate)
C (l2s isolation)	no (rseq active)	off	same flaky	fail on clone 2, same GnuTLS error (run then truncated by a USB drop; the fail is the datum)

Reading:

1. The failure requires a slow/stalling network. Nothing fails in any config on a clean link. That explains the maddening reproducibility pattern ("fails 20 minutes straight" one morning, unreproducible later) — the variable was Wi-Fi quality, and it was uncontrolled in both of our earlier tests.
2. The corrupted layer is below git. GnuTLS receives a byte stream that fails TLS record decoding. TCP and the (checksum-verified) proxy deliver intact bytes to the kernel, so the stream is being corrupted between the kernel socket buffer and the traced process's read() — i.e., in proot's syscall translation under load, exactly where a kernel-rseq-vs-ptrace conflict would bite. A slow link fragments the transfer into many more translated reads, multiplying exposure. This also subsumes the "truncated right after the PACK header" observation: same corrupted-read family, seen one layer higher.
3. --link2symlink is not the mechanism (C still fails without it), and disabling rseq is protective on this device (A: 8/8).

This does not invalidate your Galaxy A35 targetSdk 28-vs-35 A/B — two devices can genuinely differ here. But it does mean the Seeker evidence no longer supports "the rseq theory is disproven", and the network confound was live in the A/B unless link quality was pinned. Two asks before this PR's root-cause narrative lands:

• Re-run the A35 comparison with a controlled flaky link (harness below — it needs only adb and a debug build).
• Confirm at runtime whether the failing targetSdk-35 build actually had the tunable in the guest env: cat /proc/<node pid>/environ | tr '\0' '\n' | grep GLIBC. If A35 still fails with rseq disabled under a controlled link, that is a real second mechanism and the single-process fallback earns its place.

On the change set itself (independent of root cause)

1. Internal contradiction. The PR body concludes the Ubuntu version is irrelevant ("22.04 fails identically"), yet the change pins CI to 22.04 with hard-fail guards calling 24.04 "known-bad", and the new ServerManager comment says #112 "is fixed by shipping the 22.04 rootfs again". If the shim is the fix, the rollback and guards are vestigial theory and add a CI trap (see 5). One story, please.
2. Shim arg matching. The sh shim matches the word clone anywhere in argv, so git commit -m clone or git checkout clone misroute to dulwich. Match only the first non-option argument.
3. Flag parsing. -b <name> makes <name> positional (it becomes the URL — confusing hard failure), and --branch=x is silently dropped (clones the default branch — silently wrong, worse than failing). Jammy's dulwich 0.20.31 supports porcelain.clone(..., branch=..., depth=...) (checked against the sdist), so this is a small fix. Same bug class for any value-taking flag (-o, -c, --origin).
4. Installer details. The comment says "v4 -> v5" but main is at v3; the backup dir is named .guest-home-before-v4 while the marker is v5. Also, if the process dies between the rootfs delete and restoreGuestHome, the next install finds no home, preserveGuestHome returns null, and the backup is orphaned — restore should also look for a leftover backup dir.
5. Merge-ordering trap. Once merged, android-apk/android-release hard-fail until android-assets is manually re-run (current artifacts carry no manifest). Either sequence the merge with an assets re-run, or make the guard warn instead of fail when the manifest is absent.
6. Repo convention: commits here don't carry AI co-author attribution — a squash-merge can strip it.

Happy to help land a fixed shim as a belt-and-suspenders layer if the controlled A35 re-test still shows a targetSdk-dependent failure with rseq disabled — but the rootfs rollback and the 22.04 CI pin should come out either way.

Harness (host wrapper + guest A/B + flaky proxy)

Host-side (Mac/Linux), flaky CONNECT proxy + reverse tunnel:

adb reverse tcp:8899 tcp:8899
node flaky-proxy.js   # below

// flaky-proxy.js — forwards CONNECT tunnels; server->phone dribbled in 16KB
// chunks, 10% chance of a 200-900ms stall per chunk. Byte-exact (checksum it).
const net=require('net'),http=require('http');
function flakyPipe(src,dst){
  src.on('data',d=>{src.pause();let off=0;
    (function step(){
      if(off>=d.length){src.resume();return;}
      const c=d.slice(off,off+16384);off+=16384;
      const wait=Math.random()<0.10?200+Math.random()*700:55;
      dst.write(c,()=>setTimeout(step,wait));})();});
  src.on('end',()=>dst.end());src.on('error',()=>dst.destroy());
}
http.createServer((q,s)=>{s.writeHead(502);s.end();})
.on('connect',(req,sock,head)=>{
  const i=req.url.lastIndexOf(':');
  const up=net.connect(+req.url.slice(i+1)||443,req.url.slice(0,i),()=>{
    sock.write('HTTP/1.1 200 Connection Established\r\n\r\n');
    if(head&&head.length)up.write(head);
    flakyPipe(up,sock);sock.pipe(up);});
  up.on('error',()=>sock.destroy());sock.on('error',()=>up.destroy());
}).listen(8899,'127.0.0.1');

Device host wrapper (run-as <appId> sh abtest-host.sh l2s|nol2s [norseqfix]) — replicate your app's DirectProotExec invocation exactly (loader env, soname shim, L2S dir, same flags), toggling --link2symlink via arg 1 and GLIBC_TUNABLES=glibc.pthread.rseq=0 vs an inert tunable via arg 2, then inside the guest run N clones of a real repo through https_proxy=http://127.0.0.1:8899 while node spins 4 worker threads beside git under the same tracer. Judge each clone by exit code + git fsck + a .pack existing. (Full scripts in the #112 thread if useful.)

[zo-sol]

Correction — the root-cause section of my review is superseded. You were right about the mechanism.

I now have the decisive on-device datapoint from the Seeker, and it flips my verdict:

• The in-app agent ran git clone https://github.com/octocat/Hello-World.git today at 16:08 KST, on a healthy Wi-Fi link, on the current build. It fails consistently with bad object 7fd1a60b… / remote did not send all necessary objects.
• The rseq fix IS active in that path — verified live on the running process tree (libproot.so 28953 → node 28961, /proc/28961/environ contains GLIBC_TUNABLES=glibc.pthread.rseq=0), so the failure survives the tunable. The device agent's own earlier traces agree (rseq env applied in-guest, still truncated; single-thread, protocol v0, HTTP/1.1 all fail; pack cut right after the PACK header on the sideband).
• Why my flaky-link matrix missed it: my entire harness ran under adb run-as, and I just checked the SELinux contexts — the app's proot/node run in u:r:untrusted_app, while run-as children run in u:r:runas_app. I measured a different domain. In runas_app, everything passes on a clean link and only a flaky link + active rseq corrupts (that part of the data stands, but it's a different, milder failure mode than the app's).

So the reconciled picture: the app-path breakage is domain-gated (untrusted_app, i.e. the targetSdk-35 domain), exactly as this PR claims. The rseq tunable is not sufficient there; keep it as the low-cost guard it is.

What still stands from my review (all code-level, none of it root-cause):

1. the 22.04 rollback + CI hard-fail pin contradicts the PR's own finding that the Ubuntu version is irrelevant — should come out;
2. shim clone arg matching (git commit -m clone misroutes);
3. -b <name> / --branch=x parsing (dulwich 0.20.31 supports branch=/depth= kwargs, small fix);
4. Installer marker comment (main is at v3) + orphaned-backup edge in restoreGuestHome;
5. merge ordering (android-assets must re-run before android-apk passes its new guard).

With 1–3 addressed I'm happy to approve — the single-process clone path is the right call for the untrusted_app wall, and a follow-up covering fetch/pull (same mechanism) is worth an issue. Apologies for the detour, and nice diagnosis.

[zo-sol]

Pushed the review fixes directly to this branch (0b6ea27) so we can land the shim today — the mechanism is confirmed and users are blocked:

• shim routes on the first non-option arg only; value-taking flags are consumed, -b/--branch honored (bytes for dulwich 0.20.31), --depth kept
• dropped the 22.04 rollback + manifest + CI hard-fail guards (vestige of the disproven first theory; the guards would also break every APK build until android-assets re-ran)
• Installer: marker comment corrected (main was at v3), crashed-install backup adoption
• comment accuracy: rseq-off stays as a separately-measured guard, not the git clone always fails in proot (Android) guest — "remote did not send all necessary objects" (suspected Ubuntu 24.04 rseq regression) #112 fix

Verified locally: sh/py syntax, 6 shim routing cases, 3 flag-parsing cases, Kotlin compiles.

Landing sequence (ordering matters because the fix lives in the rootfs tar): android-assets is building from this branch now → merge on green → the push-triggered android-apk picks up the fresh assets artifact → marker v5 re-extracts on devices → on-device in-app clone verification on the Seeker.

@mega123-art please shout if any of this misrepresents your change — happy to follow up. The fetch/pull/push gap is tracked in the #112 fix-ladder comment.