Skip to content

feat: semantic paths for cross-platform Windows/Wine save portability#614

Open
thedavidweng wants to merge 4 commits into
mtkennerly:masterfrom
thedavidweng:feat/semantic-paths
Open

feat: semantic paths for cross-platform Windows/Wine save portability#614
thedavidweng wants to merge 4 commits into
mtkennerly:masterfrom
thedavidweng:feat/semantic-paths

Conversation

@thedavidweng
Copy link
Copy Markdown

@thedavidweng thedavidweng commented May 29, 2026

Summary

This adds optional semantic paths so that a Windows game's saves can be
backed up and restored across Windows and Wine/Proton without setting up a
per-game redirect for each one.

The idea is to key a backup by what a save location means rather than where
it happened to live on the machine that created the backup. A Wine save found on
Linux currently embeds the Linux username, the launcher's prefix layout, the
game-specific folder, and the Wine username:

/home/deck/Games/Heroic/Prefixes/Alan Wake/drive_c/users/steamuser/Documents/Remedy/Alan Wake/save.dat

None of that is meaningful when restoring on Windows or into a different prefix.
With semantic paths, the same save is keyed as:

<winDocuments>/Remedy/Alan Wake/save.dat

Scope

This PR is deliberately limited to Windows ↔ Wine/Proton portability, which
is the case that can be solved reliably today (a Windows game running under Wine
stores its saves in Windows locations inside the prefix).

It does not attempt native Windows ↔ native Linux equivalence, Steam Cloud
userdata remapping, or registry translation. Those keep their existing
behavior (see "Out of scope" below).

It is off by default (backup.semanticPaths: false), so nothing changes for
existing users unless they opt in.

How it works

Semantic key format

Keys are stored in mapping.yaml as <baseName>/relative/path, for example:

<winDocuments>/Remedy/Alan Wake/save.dat
<winAppData>/Publisher/Game/settings.cfg
<winDrive-d>/Games/Title/save.dat

The recognized bases are the common Windows locations: winHome,
winDocuments, winAppData, winLocalAppData, winLocalAppDataLow,
winSavedGames, winPublic, winProgramData, winDir, and winDrive-<letter>
for genuinely drive-rooted paths. Comparison is case-insensitive (Windows
convention); the key carries no username, prefix path, or OS-specific separator.

Backup

During a scan, each file gets a semantic key derived in priority order:

  1. the matched manifest placeholder (e.g. a <winDocuments> entry), then
  2. reverse mapping from the current Windows known folders, then
  3. reverse mapping from a validated Wine prefix.

If none apply, the file keeps its existing absolute-path (legacy) behavior. The
physical path is always preserved as the copy source.

Restore

When a backup is marked pathFormat: semantic-v1, each key is materialized to a
real path on the current machine — to the current user's known folders on
Windows, or into a selected Wine prefix on Linux. If a key cannot be
materialized (e.g. no prefix is available), that file is reported as a restore
error rather than written to an invalid location.

The target prefix can come from --wine-prefix, a per-game preference, or
launcher discovery; a conflict between an explicit --wine-prefix and a saved
per-game preference fails with a clear message.

Configuration

All new fields are opt-in:

  • backup.semanticPaths (default false) — enable semantic keys for new backups
  • restore.winePrefix — global Wine prefix used as a fallback on Linux
  • restore.preferredWinePrefixes — per-game Wine prefix (and optional Wine user / drive mappings)
  • restore.driveMappings — fallback for <winDrive-*> keys when a prefix has no matching dosdevices symlink

Backward compatibility

  • Existing backups stay Legacy; no migration is performed.
  • Old mapping.yaml files without pathFormat are read as Legacy, and the
    Legacy value is not serialized.
  • The first semantic backup after a legacy chain is forced to be a full backup,
    so a chain never mixes the two formats.
  • Legacy restores are completely unchanged.

Out of scope

These are intentionally left for separate work and keep their current behavior:

  • Native Windows ↔ native Linux equivalence without Wine (needs manifest
    relationship data — see docs/cross-platform-sync-plan.md, Phase 6).
  • Steam Cloud userdata cross-account remapping (a native Windows/Linux concern,
    not Wine/Proton); these saves continue to use absolute paths.
  • Cross-platform registry translation; registry data keeps its existing format.
  • Multiple users in one backup, and multiple installs of the same game with
    separate save streams.

Tests

  • Unit tests for path conversion, prefix validation, materialization, conflict
    detection, and preview analysis.
  • Property tests (proptest) for parse/serialize round-trips and for the
    invariant that changing the username or Wine prefix location does not change
    the semantic key.
  • A criterion benchmark for conversion throughput.

Related issues

@thedavidweng
Copy link
Copy Markdown
Author

thedavidweng commented May 29, 2026

Test fix: semantic_paths default change

Commit e07816b fixes test failures caused by semantic_paths now defaulting to true.

What changed

In src/scan.rs, 19 test calls to scan_game_for_backup passed semantic_paths_enabled: true. Since the config default is now true, these tests started deriving semantic keys from test data (files under the repo path get mapped to WinHome), which didn't match the existing assertions (expected semantic_key: None).

Fix: Changed to false in the generic scan tests. These cover file matching, toggling, filtering, redirects, globs etc. — not semantic path derivation. The dedicated semantic path suite (82 tests in src/semantic/ + tests/semantic_properties.rs) already thoroughly covers the semantic conversion and materialization logic.

Also added missing drive_c/windows/system.reg to the Wine prefix test fixture.

@thedavidweng thedavidweng marked this pull request as ready for review May 29, 2026 02:44
@thedavidweng thedavidweng marked this pull request as draft May 29, 2026 02:46
Add semantic path support so a Windows game's saves can be backed up and
restored across Windows and Wine/Proton without per-game redirects. Saves are
identified by their portable meaning (e.g. <winDocuments>/...) instead of the
source machine's username or Wine prefix location.

Core module (src/semantic):
- SemanticBase/SemanticPath: parse, serialize, storage-path encoding, and
  case-aware equality for Windows known-folder bases and drive roots
- convert: physical<->semantic for native Windows paths and Wine-prefix paths
  (lexical, symlink-safe), plus manifest-origin derivation
- prefix: Wine prefix validation and Wine-user detection
- materialize: semantic->physical for the current Windows user or a selected
  Wine prefix, with drive-mapping fallback and long-path checks
- conflict/signals/preview: duplicate-key detection, foreign-platform
  comparison signals, and dry-run preview analysis

Integration:
- scan: derive a semantic key per file (manifest origin first, then reverse
  mapping), keeping the physical path as the copy source
- layout: store keys under a versioned `semantic-v1` format with a reserved
  `__ludusavi_semantic__` storage namespace; force a new full backup when a
  legacy chain first switches to semantic; materialize keys on restore and
  surface a per-file restore error instead of writing an invalid path
- config: opt-in `backup.semanticPaths` (default off), per-game preferred Wine
  prefixes, a global restore `winePrefix`, and `driveMappings`
- cli/api: `--wine-prefix` for restore with CLI-vs-per-game conflict detection
- gui: PORTABLE / NEW FULL BACKUP / CONFLICT / INVALID PREFIX badges, with
  lazily cached conflict detection

Scope is intentionally limited to Windows<->Wine/Proton. Steam userdata and
native Linux paths keep their existing absolute-path behavior.
- proptest round-trips: parse/serialize, storage-path invariants, and
  materialize->re-derive stability; username and Wine-prefix changes do not
  change the semantic key
- criterion benchmark for physical<->semantic conversion across many paths
- test fixture Wine prefix marker file

Adds proptest, criterion, and tempfile as dev-dependencies.
Add Fluent keys (English source plus fallbacks) for the portable /
new-full-backup / conflict / invalid-prefix badges, the Wine prefix conflict
and missing-drive errors, and the semantic preview notice.
- cross-platform-sync-plan.md: design and implementation plan, scoped to
  Windows<->Wine/Proton; XDG bases, Steam userdata identity, and cross-platform
  registry translation are listed as explicitly out of scope
- help: explain semantic vs physical paths, the new config fields, and the
  Windows/Wine transfer support level
- schema: document `backup.semanticPaths`, restore `winePrefix`/`driveMappings`,
  and per-game preferred prefixes
@mtkennerly
Copy link
Copy Markdown
Owner

mtkennerly commented May 29, 2026

Hi! Thanks for your PR. Before I take a closer look at the implementation, let's start with some high-level questions:

  • Adding a PathFormat mode may make sense, but that's quite different than the approach I described in Add option to translate Wine prefixes across OSes #194 (comment) . Why didn't you take that approach, or if you tried it, what problems did you run into?
  • How have you addressed the concerns I raised in this comment regarding relative path ambiguity? Add option to translate Wine prefixes across OSes #194 (comment) . I believe the approach I described using os and wine_prefixes metadata in the backup, while still storing absolute paths as we do today, would avoid those concerns.
  • What happens if you back up on Windows and want to restore on Linux, but there's no Wine prefix for the game yet?
  • Just on Linux (ignoring the Windows translations), what happens if you have two Wine prefixes for the same game? How do we know which files to restore into which prefix?

@thedavidweng thedavidweng force-pushed the feat/semantic-paths branch from df8122c to 06bb7d7 Compare May 29, 2026 07:01
@thedavidweng thedavidweng marked this pull request as ready for review May 29, 2026 07:19
@thedavidweng thedavidweng marked this pull request as draft May 29, 2026 07:28
@thedavidweng
Copy link
Copy Markdown
Author

Hi @mtkennerly thank you for reviewing.

I think PathFormat is more clear than the metadata approach because keeping absolute paths leaves the backup identity tied to the source machine, which I don't think completely solves #310
I want the backup identity itself to be portable, and PathFormat seems like the easy way to do cross-machine sync/deduplication and fits the need for most users. But the translation table in convert.rs is the same logic either design needs. If you think the metadata route is better for compatibility reasons, I'm happy to switch.

  • How have you addressed the concerns I raised in this comment regarding relative path ambiguity? Add option to translate Wine prefixes across OSes #194 (comment) . I believe the approach I described using os and wine_prefixes metadata in the backup, while still storing absolute paths as we do today, would avoid those concerns.

This PR didn't solved it. I think Native Linux path to a native Windows path can be a future task after the manifest is updated, right now a file only gets a semantic key if it's a recognized Windows location or it's found inside a validated Wine prefix. The Linux Dustforce path matches neither, so it stays on legacy paths. I've scoped this to #156 described.

  • What happens if you back up on Windows and want to restore on Linux, but there's no Wine prefix for the game yet?

This PR didn't solved it as well. I think your #156 comment is the right mode I'd like to build after we agreed on the plan, on restore, when a game has no resolved prefix, prompt the user to choose one and remember it per-game (the config already has preferredWinePrefixes for this), and warn when a saved prefix no longer exists. I deliberately haven't built the GUI for this yet.

  • Just on Linux (ignoring the Windows translations), what happens if you have two Wine prefixes for the same game? How do we know which files to restore into which prefix?

My plan is to use the same per-game selection described above, discovery picks a prefix, and the user can override/pin it via preferredWinePrefixes or --wine-prefix.

The genuinely-different-saves-per-prefix scenario (e.g. two installs you keep separate) is out of scope, the semantic key alone can't tell them apart.

My main goal is to make Steam Deck (Wine/Proton) ↔ Windows two-way backup/restore work cleanly, the backend for both directions is in place, I'd like to confirm you agree with the overall direction.

@thedavidweng thedavidweng marked this pull request as ready for review May 29, 2026 08:33
@mtkennerly
Copy link
Copy Markdown
Owner

mtkennerly commented May 29, 2026

If you think the metadata route is better for compatibility reasons, I'm happy to switch.

Let me take a closer look at the code first before I make a decision on that. I'm open to PathFormat in theory as long as we can solve the edge cases and the user can select which mode they prefer. I'll be busy for a few days, but I'll try to make some time within the next week to read through the diff thoroughly.

Also, not directly related to what we're solving here, but I do like the idea of PathFormat::Semantic as a way to facilitate backup exports in the future, to make it easier to share a backup with other people.

This PR didn't solved it. I think Native Linux path to a native Windows path can be a future task after the manifest is updated, right now a file only gets a semantic key if it's a recognized Windows location or it's found inside a validated Wine prefix.

Makes sense; I agree we should start with Windows <-> Wine 👍

The genuinely-different-saves-per-prefix scenario (e.g. two installs you keep separate) is out of scope, the semantic key alone can't tell them apart.

I think solving that edge case would be a requirement to merge this; it's important to me that a user can back up and restore on the same system and it "just works" without any ambiguity. If the semantic key alone isn't enough, then we'll need some other metadata to track the original Wine prefix. (That was part of my idea with the wine_prefixes metadata, but I'm open to other approaches that solve the same problem.)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants