Skip to content

Extra effect folders (FOSFORA_TRAMA_DIRS, FOSFORA_EFFECT_DIRS) - #300

Merged
kevinraymond merged 5 commits into
kevinraymond:mainfrom
BuddhiLW:feat/effect-folders
Nov 16, 2026
Merged

kevinraymond merged 5 commits into
kevinraymond:mainfrom
BuddhiLW:feat/effect-folders

Conversation

@BuddhiLW

Copy link
Copy Markdown
Contributor

Split out of #299, as asked: this PR holds only the extra effect folders.

FOSFORA_TRAMA_DIRS and FOSFORA_EFFECT_DIRS list directories (split like PATH, a leading ~ is home) whose trama .wgsl and .pfx effects load after the built-ins (built-in indices do not move) and hot-reload when an external generator writes to them. A relative shader in an extra .pfx resolves against that file's own directory.

Changes since #299:

  • Symlinked folders. read_pfx now localizes the shader paths against the path as listed, then canonicalizes it for source_path. Before, it canonicalized first, so a folder in FOSFORA_EFFECT_DIRS reached through a symlink failed is_under at startup and the shader stayed relative to assets/shaders. The new test a_symlinked_extra_dir_still_localizes_shaders covers it. It goes through a small read_pfx_in(path, dirs) so it does not touch the environment.
  • recognised is now recognized in trama/dirs.rs.
  • The CHANGELOG entry is the one from Live REPL, extra effect folders, opt-in hardware video decode #299.

Checked locally: cargo fmt --check, cargo test -p fosfora-app --lib -- dirs, cargo clippy -p fosfora-app --all-targets -- -D warnings, scripts/check_changelog.py.

kevinraymond added a commit that referenced this pull request Nov 12, 2026
The attribution section told every agent to commit as Kevin Raymond and to
set git config to that if it differed. The file is tracked, so an agent in
a contributor's clone followed it too: PRs #300 and #301 arrived with two
commits authored as Kevin and all five committed as him. Agents now keep
the identity that is configured and never change it.
@kevinraymond

Copy link
Copy Markdown
Owner

Two of the commits carry my name and email instead of yours, and so does the committer on the others. I think the repo's CLAUDE.md told your agent to set that. I've fixed the file. Please set your own user.name and user.email in your clone, rewrite both branches with git rebase origin/main --exec 'git commit --amend --no-edit --reset-author', and force-push.

After that, this PR looks good and I will get it merged.

FOSFORA_TRAMA_DIRS lists directories (split like PATH, leading ~ expanded)
whose *.wgsl files are trama effects exactly like assets/trama/effects/:
loaded at startup, a new file registers live, a change hot-swaps, a broken
save keeps the last good version. An external generator can write straight
into one; the first user is tessera, whose desargues emitter writes field
loops to tessera/out/layers/.

Load order is the built-in directory, then each extra directory; an id
defined twice takes the last definition, which is also what a hot reload
does (the changed file wins). The watcher recognises the built-in directory
by name (trama/effects) and an extra one by being listed, watched
non-recursively and seeded like the built-in one so startup reads are not
reported as changes. A shader-library change reloads every directory.

Tested end to end through the real watcher: a file written into an extra
directory arrives on the trama route, not the layer-shader one.
FOSFORA_EFFECT_DIRS lists directories (same rules as FOSFORA_TRAMA_DIRS)
whose *.pfx files are layer effects like assets/effects/: they appear in the
catalog (under "Yours" unless authored "Fosfora"), load onto layers, save in
presets by name, and hot-reload when the .pfx or one of its shaders changes.
Each directory is watched recursively, so a .pfx and its .wgsl can sit side by
side. The first user is tessera, whose loops get a catalog entry each.

A relative `shader` in an extra directory's .pfx resolves against that .pfx's
own directory (effect::dirs::localize), at scan and again on every hot reload,
so the reload path cannot point it back at assets/shaders. Built-in effects
and absolute paths are untouched. Extra effects are appended after the
built-ins, so built-in indices do not move.

The built-in scan now reads each file through the same read_pfx helper, which
keeps the interaction_3d fixup in one place.
A folder in FOSFORA_EFFECT_DIRS reached through a symlink failed is_under
at startup, because read_pfx canonicalized the .pfx path while the listed
folders are only made absolute; the shader then stayed relative to
assets/shaders. Localize against the path as listed, canonicalize after.
read_pfx_in takes the folders so a test covers a symlinked one.

Also: US spelling (recognized) in trama/dirs.rs, and the CHANGELOG entry.
@BuddhiLW
BuddhiLW force-pushed the feat/effect-folders branch from c1f3daa to 0ae3e0f Compare November 12, 2026 14:10
@BuddhiLW

Copy link
Copy Markdown
Contributor Author

Sorry about that. The clone had a repo-local user.name / user.email override with your identity, set from the old CLAUDE.md. I removed it, re-applied the three commits on origin/main with the author reset, and force-pushed. Author and committer are now mine on all of them, and the tree is the same as before apart from the CLAUDE.md change that came in from main.

@BuddhiLW

BuddhiLW commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor Author

To be clear, the wrong author was my mistake. Your assessment is right: my agent followed the old CLAUDE.md, and I did not check the result. I will pay attention to it next time.

@kevinraymond

Copy link
Copy Markdown
Owner

Thanks for sorting out the authorship. That also let CI run on this PR for the first time, and Windows is failing on two of the new tests in effect/dirs.rs. Linux and macOS are green. Both are in the tests, not in the feature.

extra_dir_effects_read_shaders_beside_their_pfx

left:  "graphene.wgsl"
right: "/srv/tessera/out/effects/graphene.wgsl"

/srv/... has no drive on Windows, so std::path::absolute turns the .pfx path into D:\srv\... and it no longer starts with the listed directory. The app doesn't hit this, because parse makes every listed directory absolute first. Building the paths from a root that is absolute everywhere fixes it:

let root = std::env::temp_dir().join("tessera");
let dirs = vec![root.clone()];
let other = std::env::temp_dir().join("elsewhere").join("b.wgsl");
let mut e = effect("graphene.wgsl", &["sub/a.wgsl", &other.display().to_string()]);
localize_in(&mut e, &root.join("graphene.pfx"), &dirs);
assert_eq!(Path::new(&e.shader), root.join("graphene.wgsl"));
assert_eq!(Path::new(&e.passes[0].shader), root.join("sub/a.wgsl"));
assert_eq!(Path::new(&e.passes[1].shader), other, "absolute paths are kept");

built_in_effects_keep_assets_relative_shaders passes on Windows today, but only because of the same mismatch, so it should get the same root.

pfx_files_are_found_recursively_and_sorted

left:  ["a.pfx", "b.pfx", "sub\\c.pfx"]
right: ["a.pfx", "b.pfx", "sub/c.pfx"]

The function is right; the test compares display() strings. Comparing paths works on both:

let found: Vec<PathBuf> = pfx_files(&root)
    .iter()
    .map(|p| p.strip_prefix(&root).unwrap().to_path_buf())
    .collect();
assert_eq!(
    found,
    [PathBuf::from("a.pfx"), PathBuf::from("b.pfx"), Path::new("sub").join("c.pfx")]
);

I did not run these on Windows, so I won't promise they work as is!

One more thing I noticed, which doesn't need to hold this PR up. ~ is expanded against HOME, and Windows normally doesn't set it, so ~/effects stays literal there. Falling back to USERPROFILE would cover it.

Once Windows is green I'll merge.

…OFILE

The tests built paths from /srv/..., which has no drive on Windows, so
std::path::absolute moved them to D:\srv\... and they no longer started with
the listed directory; build them from temp_dir instead. Compare pfx_files
results as paths, not display() strings. ~ now falls back to USERPROFILE
when HOME is unset or empty (home_from, tested without the process env).
@BuddhiLW

Copy link
Copy Markdown
Contributor Author

Thanks for the diagnosis. Pushed 830fb1b:

  • The three tests in effect/dirs.rs now build their roots from std::env::temp_dir(), as you suggested. pfx_files_are_found_recursively_and_sorted now compares PathBufs, not display() strings.
  • ~ now falls back to USERPROFILE when HOME is unset or empty. A unit test covers this without touching the process environment.
  • I merged main to resolve the CHANGELOG conflict with Opt-in hardware video decode for media layers (FOSFORA_HWACCEL) #301. The branch is mergeable again.

fmt, clippy -D warnings and the full test suite pass locally with --features video (Linux). I haven't run Windows myself, so the Windows CI leg is the real check.

@kevinraymond

Copy link
Copy Markdown
Owner

Windows is green, so I'm merging this now.

One thing for a follow-up PR ... localize_in rewrites shader and each pass's shader, but not the compute shaders for a particle effect (particles.compute_shader, particles.reaction_diffusion.compute_shader and particles.trail_field.compute_shader).

So a particle effect in an extra folder still looks for its sim in assets/shaders. That's a big part of the format (25 of the 60 built-in effects use a compute shader), and none of them can ship as a self-contained folder yet. Adding those three fields tolocalize_in, with a test along the lines of extra_dir_effects_read_shaders_beside_their_pfx, should cover it.

While you're in there, the CHANGELOG entry reads the folders hot-reload "when an external tool writes to them". That holds for trama .wgsl files, which register live, but a new .pfx still needs a restart. Please reword the existing entry rather than add a new one, since both changes ship in the same release.

Thanks again!

@kevinraymond
kevinraymond merged commit aea7f42 into kevinraymond:main Nov 16, 2026
13 checks passed
@BuddhiLW

Copy link
Copy Markdown
Contributor Author

I'm the one who thanks taking up the PR. Great software. I have a lot of fun using it, on my own.

Regards,
Pedro

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants