fix: make StorOps actually work on macOS -- packaging, memory, and APFS duplicate counting

.github/workflows/test.yml

@@ -12,13 +12,17 @@ jobs:
       fail-fast: false
       matrix:
         os: [ubuntu-latest, macos-latest, windows-latest]
+        # 3.9 is the declared floor (a stock macOS `python3` and RHEL 9 both
+        # ship it) and is tested so it stays real; 3.13 catches deprecations
+        # early. The middle of the range is left to those two to bracket.
+        python-version: ["3.9", "3.13"]
     runs-on: ${{ matrix.os }}
     steps:
       - uses: actions/checkout@v4
 
       - uses: actions/setup-python@v5
         with:
-          python-version: "3.11"
+          python-version: ${{ matrix.python-version }}
 
       - name: Install gdu (Linux/macOS only -- the preferred scan backend, du is the fallback)
         if: runner.os != 'Windows'
@@ -34,3 +38,66 @@ jobs:
 
       - name: Run unit + integration tests
         run: pytest tests/unit tests/integration -q
+
+  # The backend a real Mac actually uses. gdu is a `brew install` away, so
+  # the job above never exercises BSD `du` -- which is where macOS' own
+  # quirks live (its -a and -d are mutually exclusive, so an unguarded
+  # file-level scan is unbounded in depth; a `storops scan ~` cost ~2.6GB
+  # peak RSS before that was caught). Without this job those regressions
+  # are invisible on every runner.
+  test-macos-without-gdu:
+    runs-on: macos-latest
+    steps:
+      - uses: actions/checkout@v4
+
+      - uses: actions/setup-python@v5
+        with:
+          python-version: "3.13"
+
+      - name: Confirm gdu is absent, so the du backend is the one under test
+        run: |
+          ! command -v gdu || { echo "gdu unexpectedly on PATH"; exit 1; }
+
+      - name: Install storops (editable) + dev dependencies
+        run: pip install -e ".[dev]"
+
+      - name: Run unit + integration tests
+        run: pytest tests/unit tests/integration -q
+
+      - name: The du backend really is what got selected
+        run: |
+          python - <<'PY'
+          from storops import platform
+          backend = platform.get_scan_backend()
+          assert backend.name == "Du", backend.name
+          print("scan backend:", backend.name)
+          PY
+
+  # `pip install -e .` leaves the repo-root rules/ directory reachable, so it
+  # cannot tell whether the built distribution actually ships the rule files.
+  # It did not: every command in an installed storops failed with "could not
+  # locate the rules/ directory" until this was caught.
+  test-installed-wheel:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+
+      - uses: actions/setup-python@v5
+        with:
+          python-version: "3.9"
+
+      - name: Build the wheel
+        run: |
+          pip install build
+          python -m build --wheel
+
+      - name: Install it somewhere the source tree cannot be reached
+        run: |
+          python -m venv /tmp/wheel-venv
+          /tmp/wheel-venv/bin/pip install dist/*.whl
+
+      - name: The installed package can identify a path (i.e. it found its rules)
+        run: |
+          cd /tmp
+          /tmp/wheel-venv/bin/storops identify /tmp --json
+          /tmp/wheel-venv/bin/storops scan /tmp --json > /dev/null

README.md

@@ -51,8 +51,10 @@ coverage.
 
 ## Requirements
 
-- **Python 3.11+** — the only implementation (`src/storops/`); `python3`/
-  `python` needs to be on `PATH`. No `pip install` is required for the
+- **Python 3.9+** — the only implementation (`src/storops/`); `python3`/
+  `python` needs to be on `PATH`. 3.9 is the floor deliberately: it is what
+  a stock macOS ships as `python3`, and StorOps is most useful on a machine
+  nobody has set a modern toolchain up on yet. No `pip install` is required for the
   common "cloned into a skills directory" install path — `python -m storops`
   works straight out of the checkout.
 - **Windows**: NTFS volumes. [WizTree](https://diskanalyzer.com/) is
@@ -79,7 +81,7 @@ StorOps is a plain agent skill: a directory with a `SKILL.md` at its root,
 discovered by name and description rather than invoked as a slash command. No
 build step and no `pip install` required — the agent reads `SKILL.md` to
 decide when to use the skill, then invokes `python -m storops <verb>`
-directly. The only runtime requirements are Python 3.11+ and, on Windows,
+directly. The only runtime requirements are Python 3.9+ and, on Windows,
 WizTree — see [Requirements](#requirements) above.
 
 ### Ask your agent to install it (recommended)

README.zh-CN.md

@@ -45,8 +45,9 @@ Windows token;只有关键系统路径规则(`rules/windows.yaml`/`linux.yaml`/
 
 ## 环境要求
 
-- **Python 3.11+**——唯一的实现(`src/storops/`);`python3`/`python`
-  需要在 `PATH` 上。最常见的"克隆进 skills 目录"安装方式不需要
+- **Python 3.9+**——唯一的实现(`src/storops/`);`python3`/`python`
+  需要在 `PATH` 上。下限定在 3.9 是刻意的:macOS 自带的 `python3` 就是
+  3.9,而 StorOps 最该派上用场的,恰恰是还没配好现代工具链的机器。最常见的"克隆进 skills 目录"安装方式不需要
   `pip install`——从 checkout 目录直接运行 `python -m storops` 即可。
 - **Windows**:NTFS 卷。[WizTree](https://diskanalyzer.com/) 是可选的——
   StorOps 自带的原生扫描(`os.scandir`,以工作队列在整棵扫描树内并行;
@@ -67,7 +68,7 @@ Windows token;只有关键系统路径规则(`rules/windows.yaml`/`linux.yaml`/
 StorOps 是一个标准的 agent skill:一个根目录带有 `SKILL.md` 的目录,agent 依据
 其 name/description 自动发现并调用,而非以 slash command 的形式手动触发。无需
 构建步骤,也无需 `pip install`——agent 会读取 `SKILL.md` 来判断何时使用该
-skill,然后直接调用 `python -m storops <verb>`。运行时依赖是 Python 3.11+,
+skill,然后直接调用 `python -m storops <verb>`。运行时依赖是 Python 3.9+,
 以及在 Windows 上的 WizTree,详见上方[环境要求](#环境要求)。
 
 ### 直接让 Agent 帮你安装(推荐)

SKILL.md

@@ -79,11 +79,24 @@ commands ad hoc; the user should not need to know command names.
     (e.g. "by the way, installing gdu would make these scans noticeably
     faster") -- don't repeat it on every single command, and don't mention it
     at all on Windows or when it's `null`.
+14. On macOS, a scan's per-directory sizes will not add up to the volume's
+    used space, and you must not present them as if they should. Two
+    separate reasons: StorOps already prunes APFS firmlinks (`/Users` and
+    `/System/Volumes/Data/Users` are literally the same directory, and an
+    unpruned `du /` counts the user's whole home twice), but APFS *block
+    sharing* -- file clones, and Time Machine local snapshots -- remains,
+    and no per-path size can attribute shared blocks to one path. Report
+    the ranking and the individual sizes, which are sound; do not compute
+    "everything else" by subtracting the total from the drive's used
+    figure, and do not tell the user their disk is lying to them.
 
 ## Workflow: "why is my drive full?"
 
 1. `storops scan C:\` (or the drive the user mentioned) for top-level
-   consumers and free space.
+   consumers and free space. On macOS scan `/` -- not `/System/Volumes/Data`:
+   StorOps prunes the Data volume's firmlinked duplicates when it is handed
+   the real root, and scanning the Data volume directly reports the same
+   content under its uglier internal paths.
 2. To see what's inside several large entries at once, prefer one `storops
    search <path> --folders --max-depth 2` (bump to `3` if two levels isn't
    enough) over `storops inspect`-ing each one individually. `inspect`

pyproject.toml

@@ -7,11 +7,18 @@ name = "storops"
 version = "2.0.0a0"
 description = "Storage Operations for AI Agents -- cross-platform disk/storage diagnosis, cleanup, and migration CLI."
 readme = "README.md"
-requires-python = ">=3.11"
+# 3.9 is the floor because that is what a stock macOS ships as
+# `python3` (Xcode CLT) and what RHEL 9 ships -- StorOps is most useful on
+# a machine the user has NOT already set up a modern toolchain on, so
+# demanding 3.11 there turned `pip install storops` into a hard stop for
+# no reason: the whole test suite passes unmodified on 3.9, and CI now
+# pins the floor so it stays that way.
+requires-python = ">=3.9"
 license = { text = "MIT" }
 authors = [{ name = "tzzs" }]
 classifiers = [
     "Programming Language :: Python :: 3",
+    "Programming Language :: Python :: 3.9",
     "Operating System :: OS Independent",
     "Environment :: Console",
 ]
@@ -23,11 +30,30 @@ dev = ["pytest>=8"]
 [project.scripts]
 storops = "storops.cli:main"
 
-[tool.setuptools.packages.find]
-where = ["src"]
+# `rules/` deliberately lives at the repo root, not under src/storops/: it
+# is agent-facing content that SKILL.md/README.md point at by that path,
+# and rules/README.md documents it as the place to add rules. Mapping it
+# in as the `storops.rules` package is what gets those YAML files into the
+# wheel -- without this, `pip install storops` produced a package whose
+# every command died with "could not locate the rules/ directory", since
+# package-data patterns can only ever match files that are already inside
+# a package directory. An explicit `packages` list is required because
+# `packages.find` cannot discover a package that lives outside `where`;
+# tests/unit/test_packaging.py keeps the list from drifting.
+[tool.setuptools]
+package-dir = { "" = "src", "storops.rules" = "rules" }
+packages = [
+    "storops",
+    "storops.core",
+    "storops.output",
+    "storops.platform",
+    "storops.platform.backends",
+    "storops.platform.windows",
+    "storops.rules",
+]
 
 [tool.setuptools.package-data]
-storops = ["rules/*.yaml"]
+"storops.rules" = ["*.yaml"]
 
 [tool.pytest.ini_options]
 testpaths = ["tests"]

rules/README.md

@@ -8,9 +8,9 @@ path is from its name alone (see [`docs/DESIGN.md`](../docs/DESIGN.md) §3.3).
 
 | File | Covers |
 |---|---|
-| `ai-models.yaml` | AI/ML model weights and inference-tool caches (LM Studio, Ollama, Hugging Face, ComfyUI/Stable Diffusion, PyTorch/CUDA) -- `path_patterns` are currently Windows-token-only |
-| `applications.yaml` | Dev tooling (npm, pnpm, pip, uv, conda, Git, VS Code, JetBrains, Visual Studio, Docker, WSL) and general consumer apps (Steam, Chrome, Edge, Discord, Adobe) -- `path_patterns` are currently Windows-token-only |
-| `caches.yaml` | Generic OS/browser/temp caches not owned by one specific application above -- `path_patterns` are currently Windows-token-only |
+| `ai-models.yaml` | AI/ML model weights and inference-tool caches (LM Studio, Ollama, Hugging Face, ComfyUI/Stable Diffusion, PyTorch/CUDA) |
+| `applications.yaml` | Dev tooling (npm, pnpm, pip, uv, conda, Git, VS Code, JetBrains, Visual Studio, Docker, WSL, Gradle, Cargo, Homebrew, the Xcode/CoreSimulator toolchain) and general consumer apps (Steam, Chrome, Edge, Discord, Adobe) |
+| `caches.yaml` | Generic OS/browser/temp caches not owned by one specific application above |
 | `windows.yaml` | Windows system paths StorOps must never classify as safe to touch |
 | `linux.yaml` | Linux system paths StorOps must never classify as safe to touch |
 | `macos.yaml` | macOS system paths StorOps must never classify as safe to touch |

rules/applications.yaml

@@ -357,3 +357,167 @@
   notes: >
     No Linux pattern here by design -- the Adobe Creative Cloud desktop
     suite does not ship a Linux build, so there is no real path to match.
+
+# --- Apple developer toolchain (macOS) --------------------------------------
+# The largest reclaimable consumers on a Mac used for development, and all
+# previously "unknown"/critical -- which meant StorOps could see tens of GB
+# in a scan and had nothing to say about any of it.
+
+- id: xcode-derived-data
+  application: Xcode
+  category: ide-cache
+  path_patterns:
+    - "%HOME%/Library/Developer/Xcode/DerivedData/*"
+  confidence: 0.95
+  owner: user
+  purpose: >
+    Per-project build intermediates, module caches, and indexes that Xcode
+    regenerates from the project source on the next build.
+  deletable: true
+  migratable: false
+  cleanup_risk: low
+  cleanup_consequence: >
+    The next build of each affected project is a full rebuild (slower
+    once); nothing that is not reproducible from source is lost.
+  notes: >
+    Routinely the single largest directory under ~/Library/Developer on a
+    machine with more than a couple of Xcode projects.
+
+- id: xcode-device-support
+  application: Xcode
+  category: ide-cache
+  path_patterns:
+    - "%HOME%/Library/Developer/Xcode/iOS DeviceSupport/*"
+    - "%HOME%/Library/Developer/Xcode/watchOS DeviceSupport/*"
+    - "%HOME%/Library/Developer/Xcode/tvOS DeviceSupport/*"
+  confidence: 0.95
+  owner: user
+  purpose: >
+    Debug symbols copied off each physical device the first time it is
+    attached, kept per OS build -- so one entry accumulates per iOS version
+    ever connected, including long-obsolete ones.
+  deletable: true
+  migratable: false
+  cleanup_risk: medium
+  cleanup_consequence: >
+    Re-copied off the device (a few minutes) the next time that exact OS
+    build is attached for debugging; symbolication of already-captured
+    crash logs from that build is lost until then.
+
+- id: xcode-archives
+  application: Xcode
+  category: build-output
+  path_patterns:
+    - "%HOME%/Library/Developer/Xcode/Archives/*"
+  confidence: 0.95
+  owner: user
+  purpose: >
+    Archived builds with their dSYMs -- what App Store submissions were cut
+    from, and the only way to symbolicate crash reports from a shipped build.
+  deletable: false
+  migratable: true
+  migration_method: manual
+  migration_config_hint: >
+    Move the Archives directory to external storage by hand; Xcode's
+    Organizer reads whatever is present at this path.
+  cleanup_risk: high
+  cleanup_consequence: >
+    Not reproducible: deleting an archive permanently loses the ability to
+    symbolicate crash reports from the build it came from.
+  notes: >
+    Deliberately never offered for deletion even though it sits beside
+    DerivedData and looks similar -- these are shipping artifacts.
+
+- id: coresimulator-caches
+  application: Xcode Simulator
+  category: ide-cache
+  path_patterns:
+    - "%HOME%/Library/Developer/CoreSimulator/Caches/*"
+  confidence: 0.95
+  owner: user
+  purpose: Downloaded simulator runtime disk images and their extraction scratch space.
+  deletable: true
+  migratable: false
+  cleanup_risk: medium
+  cleanup_consequence: >
+    Simulator runtimes are re-downloaded from Apple (several GB each) the
+    next time one is needed.
+
+- id: coresimulator-devices
+  application: Xcode Simulator
+  category: application-data
+  path_patterns:
+    - "%HOME%/Library/Developer/CoreSimulator/Devices/*"
+  confidence: 0.95
+  owner: user
+  purpose: >
+    One full disk image per created simulator device, including every app
+    installed into it and its data.
+  deletable: true
+  migratable: false
+  cleanup_risk: high
+  cleanup_consequence: >
+    Deleting these directories by hand leaves CoreSimulator's own device
+    index pointing at devices that no longer exist.
+  notes: >
+    Reclaim this through Xcode itself -- `xcrun simctl delete unavailable`
+    removes devices for uninstalled runtimes and keeps the index
+    consistent. Classified high on purpose so `cleanup plan` identifies it
+    but never proposes deleting it.
+
+- id: homebrew-cache
+  application: Homebrew
+  category: package-manager-cache
+  path_patterns:
+    - "%CACHES%/Homebrew/*"
+    - "%HOME%/Library/Caches/Homebrew/*"
+    - "%XDG_CACHE_HOME%/Homebrew/*"
+  confidence: 0.95
+  owner: user
+  purpose: Downloaded bottles, source tarballs, and their partial downloads.
+  deletable: true
+  migratable: false
+  cleanup_risk: low
+  cleanup_consequence: >
+    Re-downloaded on the next install or upgrade of the affected formula;
+    installed software is unaffected.
+  notes: "`brew cleanup -s` does the same thing through Homebrew itself."
+
+# --- Cross-platform build caches missing from the rule base entirely --------
+
+- id: gradle-cache
+  application: Gradle
+  category: package-manager-cache
+  path_patterns:
+    - "%HOME%/.gradle/caches/*"
+    - "%USERPROFILE%\\.gradle\\caches\\*"
+  confidence: 0.9
+  owner: user
+  purpose: Resolved dependency jars, build-script compilation output, and transformed artifacts.
+  deletable: true
+  migratable: true
+  migration_method: app-config
+  migration_config_hint: >
+    Set GRADLE_USER_HOME to the new location (Gradle puts caches/ beneath it).
+  cleanup_risk: medium
+  cleanup_consequence: >
+    Every dependency is re-downloaded and every build script recompiled on
+    the next build; offline builds stop working until then.
+
+- id: cargo-registry
+  application: Cargo (Rust)
+  category: package-manager-cache
+  path_patterns:
+    - "%HOME%/.cargo/registry/*"
+    - "%USERPROFILE%\\.cargo\\registry\\*"
+  confidence: 0.9
+  owner: user
+  purpose: Downloaded crate sources and the registry index.
+  deletable: true
+  migratable: true
+  migration_method: app-config
+  migration_config_hint: "Set CARGO_HOME to the new location."
+  cleanup_risk: medium
+  cleanup_consequence: >
+    Every dependency is re-downloaded from crates.io on the next build;
+    offline builds stop working until then.