Fix: [LinkSafeFileSystem] stop links from shadowing real folders in EnumerateFiles

[NeuralFault]

LinkSafeFileSystem.EnumerateFiles had a regression where a directory link (symlink/junction) pointing at a real folder in the same tree could drop that real folder's entire subtree from enumeration. This left models indexed only under an alias path whose folder name parses to SharedFolderType.Unknown, so the UI showed bare filenames with no preview, version line, or base-model chip, and sidecar metadata was ignored.

Problem

1. Link and real folder collided on the visited key

EnumerateFiles visited each physical directory at most once and keyed identity by resolved path. A link pointing at a second folder in the same tree produced the same key as that folder. Real-world case: a diffusion_models symlink sitting beside the real DiffusionModels folder (some packages require that folder name).

GetRealPath("...\diffusion_models") resolves to ...\DiffusionModels, so both entries mapped to one key and one of them had its subtree skipped.

2. The loser was decided by reversed push order

The key was claimed inside the reversed push loop (for i = count-1; i >= 0; i--) while the stack popped in enumeration order, so the last enumerated entry won the key. On NTFS, _ (0x5F) sorts above Z (0x5A), so diffusion_models always beat DiffusionModels and the real folder was dropped.

Solution

LinkSafeFileSystem.EnumerateFiles (rewritten)

Real directories are now keyed ordinally, so two folders whose names differ only in case are never conflated:

var visitedRealDirsExact = new Dictionary<string, string>(StringComparer.Ordinal);

Identity is claimed when a directory is popped, not when it is queued, so scan order decides which spelling wins instead of reversed push order:

while (realDirs.Count > 0 || linkedDirs.Count > 0)
{
    var dir = realDirs.Count > 0 ? realDirs.Pop() : linkedDirs.Pop();

    if (dir.IsLink) { /* claim or skip */ }
    else            { /* claim or skip */ }
}

Real subdirectories are drained fully before any link is considered via two stacks, so a link can never take the visited key from the folder it points at, including a deeper link pointing at a top-level folder:

// Real directories are drained to completion before any link is considered...
var realDirs = new Stack<(string Path, string RealPath, int Depth, bool IsLink)>();
var linkedDirs = new Stack<(string Path, string RealPath, int Depth, bool IsLink)>();

Link-resolved targets are compared with platform case sensitivity (PathComparer), keeping the loop guard correct per OS:

var visitedRealDirsForLinkTargets = new Dictionary<string, string>(PathComparer);
var visitedLinkTargets = new Dictionary<string, string>(PathComparer);

• Windows/macOS (PathComparer = OrdinalIgnoreCase): a junction whose stored target is spelled differently from the scan root still matches and is skipped, so the loop guard holds.
• Linux (PathComparer = Ordinal): a link to foo is not conflated with a real Foo on a case-sensitive filesystem.

Skip logging now distinguishes the benign case (a link shadowed by a real folder, logged at Info) from data loss (a real folder shadowed, logged at Warn), and names the earlier scanned path:

Logger.Info("Skipping {Path}: the same directory was already scanned as {ClaimedBy}", ...); // link branch
Logger.Warn("Skipping {Path}: the same directory was already scanned as {ClaimedBy}", ...); // real branch

LinkSafeFileSystemTests (added)

• EnumerateFiles_RealFolderShadowedByLink_KeepsRealFolderPaths: [DataTestMethod] covering both a sibling link (diffusion_models) and a deeper link (sub/alias) pointing at DiffusionModels, asserting only the real-folder paths are yielded.
• EnumerateFiles_JunctionTargetCaseMismatch_KeepsRealFolderPaths: Windows-only (Compat.IsWindows guard, Assert.Inconclusive elsewhere) covering a junction whose stored target casing differs from the scan root.

Verification

• Windows: junction case-mismatch test passes; manual smoke test confirmed the diffusion_models alias no longer hides DiffusionModels.
• Linux: LinkSafeFileSystemTests gives 15 passed, 0 failed (1 skipped: the Windows-only junction test). Could not confirm issue on Linux as particular issue with symlink did not appear. Indexing of DiffusionModels resolved properly with diffusion_models symlink present.
• dotnet build for StabilityMatrix.Core succeeds with 0 errors.

Issue diagnosed with assistance from Hermes Agent / DeepSeek V4.1 Flash using local repo reference and logs provided by reporting Discord user.
Reported issue reproduced, and the fix confirmed post-change.

Authored with assistance from DeepSeek V4 Pro

[mohnjiles]

Nice catch, and the fix is right — claim-on-pop + draining real dirs before links closes the shadowing properly. All 16 tests pass on Windows here too.

One regression I could reproduce: the non-link branch checks visitedRealDirsExact but never visitedLinkTargets, so a real dir reached through a link gets rescanned if another link already claimed it as its target.

Repro: root holds a_inner → ext/X/Y and b_outer → ext/X. y.json is yielded twice (a_inner\y.json and b_outer\Y\y.json). Swap the names (b_inner / a_outer) and it's yielded once, so it's enumeration-order dependent. The old single visited set handled this; for the indexer it means duplicate entries for one physical file.

One condition fixes it (all tests + the repro pass with it):

if (
    visitedRealDirsExact.TryGetValue(dir.RealPath, out var realClaimer)
    || visitedLinkTargets.TryGetValue(dir.RealPath, out realClaimer)
)

Real dirs under the root are all claimed before any link pops, so this can't reintroduce the shadowing. Would be great to add that nested-targets case as a test (both name orders as DataRows).

Non-blocking nits:

• IsLink in the tuple duplicates which stack the entry came from — could drop the field.
• The <summary> now describes log levels and has a "skipped… which are skipped" sentence; could trim back to what a caller needs, the inline comments already carry the why.

🤖 Reviewed with Claude Code on mohnjiles' behalf

[NeuralFault]

Yep, reproduced. The real dir branch was only checking visitedRealDirsExact, so once the inner link had claimed ext/X/Y, the Y reached through the outer link got scanned a second time. Applied your condition as-is.

The name-order test you suggested only decides the walk order on NTFS, and the suite runs on Linux here on my end where it doesn't, so both rows passed even before the fix. I pinned the order with depth instead, link to ext/X/Y one level deeper so it's always popped first, which fails on the old code regardless of enumeration order. Kept that case too, gated to Windows since it genuinely depends on NTFS ordering. Whole class on Windows: 20 pass, none skipped.

Nits done, IsLink is gone from the tuples and the summary is back to the caller-facing part.

Also ran it on Windows against the real indexer, not just the unit tests. Then put four junctions in that models dir. The aliased diffusion_models pointing at a real DiffusionModels. A link to ext\X\Y sitting alongside a link to ext\X, which is the arrangement from your report. Two links pointing at the same folder. And one pointing back at the models root for the loop case.

All four deduped and logged the skip, each naming the path that got there first. The nested-target fixture is the only one of the four that needed the new check, the other three were already handled by existing logic. Index came out with 3 entries for the 3 files.