[macOS] Docker VMM virtiofs returns wrong symlink metadata starting in Docker Desktop 4.87

[andrew-meter]

Bug report

Summary

On Docker Desktop 4.87.0 for Mac with Docker VMM (libkrun) virtualization,
lstat() on symbolic links inside host bind mounts returns wrong, unstable
metadata. Freshly created symlinks can report st_size=0 (POSIX requires a
symlink's st_size to equal the length of its target path), longer-lived
symlinks decay to st_size=0 and st_nlink=0, and the values served for a
single unchanged path flap between correct and zeroed across consecutive
observations. readlink() usually returns the correct target, but on
symlinks in the decayed st_nlink=0 state a process can also observe
zero-length readlink content while another process reads the correct
target moments later, so data reads flap as well.

git caches lstat metadata (including st_size) per index entry to detect
modifications, so any repository bind-mounted from the host shows phantom
symlink modifications:

• Right after container start or a branch checkout, git status lists
  every symlink as modified although nothing touched them.
• git restore . appears to do nothing: the first git status afterwards
  still lists the symlinks, and only the second run shows a clean tree,
  regardless of how much time passes between the two runs.
• git pull --rebase fails on "local edits" to files that git status
  did not list; repeated git rebase --continue gradually "heals" it.

This is a regression from Docker Desktop 4.86 and is specific to Docker
VMM. The same host, image, volume, and workflow behave correctly on 4.86
with VMM, and on 4.87 with Apple Virtualization Framework using either
VirtioFS or gRPC FUSE.

Environment

• Host: macOS 26.6, Apple Silicon (arm64)
• Docker Desktop: 4.87.0 (236836), client/engine 29.7.2, API 1.55,
  containerd v2.2.5, runc 1.3.6
• Virtualization: Docker VMM ("UseLibkrun": true)
• Guest kernel (4.87): 7.0.12-linuxkit #1 SMP PREEMPT Wed Aug 12 20:18:49 UTC 2026 aarch64
• Guest kernel (4.86, not affected): 6.12.76-linuxkit #1 SMP Mon Jul 27 16:56:11 UTC 2026 aarch64
• Bind mount inside the container, from /proc/mounts:
  host /simpletest virtiofs rw,nosuid,nodev,relatime,ignore_atime,no_xattr 0 0
• git inside the container: 2.54.0

Backend matrix

Captured by toggling the Docker Desktop UI and reading
~/Library/Group Containers/group.com.docker/settings-store.json:

Configuration (Docker Desktop 4.87 unless noted)	Result
4.86, Docker VMM (UseLibkrun=true)	OK
Docker VMM (UseLibkrun=true, VirtioFS keys present)	broken
Docker VMM (UseLibkrun=true, gRPC FUSE keys present)	broken
AVF + VirtioFS (UseVirtualizationFramework=true, UseVirtualizationFrameworkVirtioFS=true)	OK
AVF + gRPC FUSE (UseVirtualizationFramework=true, UseGrpcfuse=true)	OK

In VMM mode the file-sharing selector is not shown in the UI; the
UseGrpcfuse / UseVirtualizationFrameworkVirtioFS keys remain in the
settings file but appear to be ignored, and both recorded VMM combinations
misbehave identically. Since AVF + VirtioFS on 4.87 is healthy, the guest
virtiofs client alone does not explain it; the defect is in the VMM
backend's virtiofs device (or its interaction with the 7.0.12 guest).

Expected result

A symlink's st_size equals the length of its target path and stays
stable, on every filesystem:

$ ln -s 0123456789abcde /mnt/probe_link && stat -c 'st_size=%s st_nlink=%h' /mnt/probe_link
st_size=15 st_nlink=1

The same probe against a non-virtiofs filesystem in the same container
(/tmp, overlayfs) returns exactly that.

Actual result

On the VMM virtiofs bind mount, observed from inside the container:

$ python3 -c 'import os; os.symlink("0123456789abcde", "link_a")'
$ # lstat sampled at t=0, 0.3s, 1s, 2s, 4s, 8s, 15s, 25s after creation:
t=0    size=0  nlink=1  ino=5371  readlink_len=15
t=25   size=0  nlink=1  ino=5371  readlink_len=15   (all samples identical)

readlink() is correct (15 bytes) while st_size stays 0. Other symlinks
in the same mount show the opposite transition, from correct to zeroed and
back, with no guest-side activity on those paths in between:

$ ls -la GEMINI.md                       # observation 1
lrwxrwxrwx 0 root root 0 Aug 19 16:22 GEMINI.md -> AGENTS.md   (size 0, nlink 0)
$ stat -c 'size=%s nlink=%h' GEMINI.md   # ~1 minute later
size=9 nlink=1                                                  (healed)
$ # minutes later, git status reports GEMINI.md as modified again (decayed)

A two-day-old symlink whose target is a directory stayed zeroed across
every observation over several minutes (.agents/skills/bar, target
../../.claude/skills/bar, real length 29):

$ stat -c 'size=%s nlink=%h' .agents/skills/bar
size=0 nlink=0            # st_nlink=0 for an existing, listable path

For a symlink in that decayed state, git's own readlink() observed zero
bytes of content while a shell readlink on the same path immediately
afterwards returned the correct 9-byte target:

$ stat -c '%n size=%s nlink=%h' CLAUDE.md && readlink CLAUDE.md
CLAUDE.md size=0 nlink=0
AGENTS.md
$ git diff CLAUDE.md      # rendered from a zero-length readlink by git
diff --git a/CLAUDE.md b/CLAUDE.md
--- a/CLAUDE.md
+++ b/CLAUDE.md
@@ -1 +0,0 @@
-AGENTS.md
\ No newline at end of file

One-shot reproducer

Prerequisite: Docker Desktop 4.87.x on macOS with Virtual Machine Manager
selected as the virtualization backend. Save as reproduce-vmm-symlink.sh
and run from the host. The metadata corruption is timing-dependent; if a
run comes out clean, re-run it (in our environment it fires nearly every
time, in one of the two probes).

#!/usr/bin/env bash
set -euo pipefail

workdir="$(mktemp -d "$HOME/vmm-symlink-repro.XXXXXX")"
trap 'rm -rf "$workdir"' EXIT

echo "== probe 1: symlink lstat metadata (every st_size should be 15) =="
docker run --rm -v "$workdir":/mnt alpine:3.22 sh -c '
  for i in 1 2 3 4 5; do ln -s 0123456789abcde /mnt/link$i; done
  stat -c "%n st_size=%s st_nlink=%h" /mnt/link*
  sleep 3
  echo "--- after 3s:"
  stat -c "%n st_size=%s st_nlink=%h" /mnt/link*
'

echo "== probe 2: git phantom symlink modification (both runs should be empty) =="
docker run --rm -v "$workdir":/mnt alpine:3.22 sh -c '
  apk add --quiet git
  cd /mnt && git init -q repo && cd repo
  git config user.email repro@example.com && git config user.name repro
  echo hello > tgt.txt && ln -s tgt.txt link1
  git add -A && git commit -qm init
  rm link1 && git checkout -- link1
  sleep 3
  echo "--- git status run 1:"; git status --porcelain
  echo "--- git status run 2:"; git status --porcelain
'

Observed output for probe 2 on 4.87 VMM (a from-scratch repository):

--- git status run 1:
 M link1
--- git status run 2:

The same script under Docker Desktop 4.86, or under 4.87 with Apple
Virtualization Framework (VirtioFS or gRPC FUSE), prints st_size=15 for
every link and two empty status runs.

Diagnostics

Two different inode numbering ranges for one path

Deleting and recreating a symlink at the same path moved it between two
distinct inode-number ranges with different metadata quality, suggesting
two attribute sources inside the backend:

$ ln -s tgt.txt link1 && git add -A && git commit -qm init
$ stat -c 'size=%s nlink=%h ino=%i' link1
size=7 nlink=1 ino=47438504          # correct size, "high" inode range
$ rm link1 && git restore .
$ stat -c 'size=%s nlink=%h ino=%i' link1
size=0 nlink=1 ino=5437              # zeroed size, "low" inode range

Why git needs exactly two git status runs

git stores each index entry's last-seen lstat data (size, mtime, ctime,
inode). For a symlink the blob content is the target string, so a correct
st_size equals the target length. Two code paths matter:

1. If the current st_size differs from a nonzero cached size, git
   reports the entry as modified without reading the link at all.
2. If only timestamps/inode changed, or the cached size is 0, git does a
   full readlink() compare; on a match it silently rewrites the index
   entry with the freshly observed stat data ("racily clean" repair).

Each git status therefore re-rolls the dice on whatever metadata the
virtiofs device serves at that instant: one run records flapped values
into the index or reports a phantom diff, and the next run, observing a
different flap state, repairs it. Because readlink() content can itself
come back empty in the decayed state (see above), even git's full content
re-check can confirm a phantom modification instead of clearing it. Captured on the from-scratch repository
above (git ls-files --debug): the index recorded size: 7 at commit
time, size: 0 after git restore ., run 1 printed M link1, run 2
was clean. Waiting 15+ seconds between the runs does not change the
pattern, so this is not a fixed-interval cache timeout.

The same mechanism explains the git pull --rebase failures on files that
git status never listed: each rebase step re-stats the tree and can
observe a different flap state than the status run that preceded it.

Impact on a real repository

On a bind-mounted repository whose index was written while metadata was
correct, consecutive git status runs seconds apart, with zero writes in
between, disagree and even grow:

--- status run 1: 12 phantom entries
 M .agents/skills/foo-analysis
 M .agents/skills/foo-debug
 ...           (12 symlinks, all actually unmodified)
--- status run 2: 14 phantom entries
 M .agents/skills/...          (same 12)
 M CLAUDE.md                   (symlink, healed earlier, decayed again)
 M GEMINI.md                   (symlink, healed earlier, decayed again)

git ls-files --debug confirms every phantom entry is a symlink whose
index-recorded size is the true target length while live lstat() returns
0 (e.g. .agents/skills/bar: index size: 29, live size=0 nlink=0).

Workaround

Switch the virtualization backend from Docker VMM to Apple Virtualization
Framework with VirtioFS (or gRPC FUSE):

"UseGrpcfuse": false,
"UseLibkrun": false,
"UseVirtualizationFramework": true,
"UseVirtualizationFrameworkRosetta": false,
"UseVirtualizationFrameworkVirtioFS": true

Reverting to Docker Desktop 4.86 with VMM also avoids the problem.

Platform

macOS

Version information

Client:
 Version:           29.7.2
 API version:       1.55
 Go version:        go1.26.5
 Git commit:        a7dcaa6
 Built:             Wed Aug  5 18:27:50 2026
 OS/Arch:           darwin/arm64
 Context:           desktop-linux

Server: Docker Desktop 4.88.1 (237512)
 Engine:
  Version:          29.7.2
  API version:      1.55 (minimum version 1.40)
  Go version:       go1.26.5
  Git commit:       6a43e3d
  Built:            Wed Aug  5 18:28:35 2026
  OS/Arch:          linux/arm64
  Experimental:     false
 containerd:
  Version:          v2.3.3
  GitCommit:        aad11006b869517fcd3009450b6f82da282e1a9b
 runc:
  Version:          1.4.3
  GitCommit:        v1.4.3-0-gbb14dabe
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0
Client:
 Version:    29.7.2
 Context:    desktop-linux
 Debug Mode: false
 Plugins:
  agent: Docker AI Agent Runner (Docker Inc.)
    Version:  v1.124.0
    Path:     /Users/username/.docker/cli-plugins/docker-agent
  ai: Docker AI Agent - Ask Gordon (Docker Inc.)
    Version:  v1.30.0
    Path:     /Users/username/.docker/cli-plugins/docker-ai
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.36.1-desktop.1
    Path:     /Users/username/.docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v5.4.0
    Path:     /Users/username/.docker/cli-plugins/docker-compose
  debug: Get a shell into any image or container (Docker Inc.)
    Version:  0.0.47
    Path:     /Users/username/.docker/cli-plugins/docker-debug
  desktop: Docker Desktop commands (Docker Inc.)
    Version:  v0.4.3
    Path:     /Users/username/.docker/cli-plugins/docker-desktop
  dhi: CLI for managing Docker Hardened Images (Docker Inc.)
    Version:  v0.0.7
    Path:     /Users/username/.docker/cli-plugins/docker-dhi
  extension: Manages Docker extensions (Docker Inc.)
    Version:  v0.2.31
    Path:     /Users/username/.docker/cli-plugins/docker-extension
  init: Creates Docker-related starter files for your project (Docker Inc.)
    Version:  v1.4.0
    Path:     /Users/username/.docker/cli-plugins/docker-init
  mcp: Docker MCP Plugin (Docker Inc.)
    Version:  v0.43.3
    Path:     /Users/username/.docker/cli-plugins/docker-mcp
  model: Docker Model Runner (Docker Inc.)
    Version:  v1.2.6
    Path:     /Users/username/.docker/cli-plugins/docker-model
  offload: Docker Offload (Docker Inc.)
    Version:  v0.6.13
    Path:     /Users/username/.docker/cli-plugins/docker-offload
  pass: Docker Pass Secrets Manager Plugin (beta) (Docker Inc.)
    Version:  v0.2.1
    Path:     /Users/username/.docker/cli-plugins/docker-pass
  sandbox: "docker sandbox" is deprecated, use Docker Sandboxes instead (Docker Inc.)
    Version:  v0.13.0
    Path:     /Users/username/.docker/cli-plugins/docker-sandbox
  scout: Docker Scout (Docker Inc.)
    Version:  v1.24.0
    Path:     /Users/username/.docker/cli-plugins/docker-scout

Server:
 Containers: 2
  Running: 2
  Paused: 0
  Stopped: 0
 Images: 2
 Server Version: 29.7.2
 Storage Driver: overlayfs
  driver-type: io.containerd.snapshotter.v1
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
 CDI spec directories:
  /etc/cdi
  /var/run/cdi
 Discovered Devices:
  cdi: docker.com/gpu=webgpu
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: aad11006b869517fcd3009450b6f82da282e1a9b
 runc version: v1.4.3-0-gbb14dabe
 init version: de40ad0
 Security Options:
  seccomp
   Profile: builtin
  cgroupns
 Kernel Version: 7.0.12-linuxkit
 Operating System: Docker Desktop
 OSType: linux
 Architecture: aarch64
 CPUs: 12
 Total Memory: 11.67GiB
 Name: docker-desktop
 ID: 69d91972-450c-4f06-91ad-3ef9b1314e5a
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 HTTP Proxy: http.docker.internal:3128
 HTTPS Proxy: http.docker.internal:3128
 No Proxy: hubproxy.docker.internal
 Labels:
  com.docker.desktop.address=unix:///Users/username/Library/Containers/com.docker.docker/Data/docker-cli.sock
 Experimental: false
 Insecure Registries:
  hubproxy.docker.internal:5555
  127.0.0.0/8
  ::1/128
 Live Restore Enabled: false
 Firewall Backend: iptables

Diagnostics ID

E11D28AD-A7A2-4E9B-8B5A-BDC6145BCEF6

[andrew-meter]

I wouldn't be surprised if there's a connection or common root cause with:
#628

[andrew-meter]

I bet this is related too:
#649

[bsousaa]

Fix in progress to be released with Desktop 4.91