Cache template sources instead of composed templates

[RobAndrewHurst]

This changes template caching to work by src rather than by template.

Multiple templates can reference the same source. The first request stores the source-loading promise in a Map, so concurrent requests reuse that promise and the source is fetched only once.

Each template still receives its own copy of the source response, allowing it to be composed independently without modifying cached data.

Recursive Sources

Source responses may contain additional src properties.

The workspace cache discovers these recursively:

1. Find all source references in the workspace.
2. Start loading each unique source.
3. Inspect the fetched responses for more sources.
4. Repeat until no new sources are found.

---

---

Build-Time Assets

The workspace cache command now treats filesystem and remote sources differently:

• Existing file: sources remain unchanged.
• Remote sources are fetched and written to a local assets directory.
• Remote references are rewritten to file: references.
• Nested remote sources are handled recursively.
• Stable hashed filenames ensure repeated references use the same asset.

This allows a generated workspace to use normal filesystem loading without embedding every source response into one large workspace file.

---

---

Testing

Added coverage for:

• Concurrent requests sharing one source promise.
• Templates sharing a source while remaining independent.
• Recursive and duplicate source discovery.
• Build-time remote asset generation.
• Circular remote source references.
• Existing file: sources remaining unchanged.

[RobAndrewHurst]

The scopes endpoint was expensive because it generated the complete scope list at request time by composing every:

• Top-level locale
• Nested locale
• Layer
• Referenced template
• Recursively referenced source

The srcMap reduced duplicate provider reads, but the endpoint previously called:

workspaceCache(true)

This replaced the workspace object—and therefore its workspace-keyed srcMap—on every scopes request.

The solution now has two levels:

1. Build-time precomputation through workspace:cache
2. Runtime promise caching as a fallback

When a generated workspace is deployed, the expensive traversal no longer occurs in the scopes request.

---

Previous request flow

Previously, each admin request to:

GET /api/workspace/scopes

performed the following:

1. Force-refresh the workspace.
2. Create a new workspace object and source map.
3. Iterate every top-level locale.
4. Resolve its templates and layers.
5. recursively resolve nested locales.
6. Populate workspace.scopes through composeObj().
7. Convert scope arrays to dot-notation strings.
8. Deduplicate and sort them.
9. Return the result.

The forced refresh was especially problematic because srcMap is held in a WeakMap keyed by workspace identity:

WeakMap<Workspace, Map<src, Promise>>

A new workspace object means a new source cache. Repeated requests therefore could not benefit from earlier source fetches.

---

New design

1. Build-time generation

Running:

pnpm workspace:cache -- \
  --workspace=file:./workspace.json \
  --output=./workspace.cached.json

now performs the full expensive operation ahead of deployment.

Scope generation

The new mod/workspace/generateScopes.js then:

1. Composes every workspace locale.
2. Loads every locale layer.
3. Recursively processes nested locales.
4. Allows composeObj() to populate the runtime workspace.scopes set.
5. Converts scope arrays into dot-notation strings.
6. Removes empty scopes.
7. Deduplicates them.
8. Sorts the final list.

The result is stored in the generated workspace:

{
  "cachedScopes": [
    "germany",
    "germany.globalvista",
    "uk",
    "uk.coremarkets",
    "uk.coremarkets.brand_a"
  ]
}

The runtime workspace.scopes value is a JavaScript Set, which cannot be represented correctly in JSON, so it is deleted before serialization.

2. Runtime fast path

The scopes endpoint first performs the existing admin check.

It then checks the workspace for the generated array:

if (Array.isArray(cachedWorkspace.cachedScopes)) {
  res.send([...cachedWorkspace.cachedScopes]);
  return;
}

For a generated workspace, the endpoint therefore does not:

• Compose locales
• Resolve nested locales
• Load layers
• Process templates
• Read static source assets
• Generate or sort scopes

It only:

1. Authenticates the admin.
2. Reads the cached workspace.
3. Clones cachedScopes.
4. Sends the response.

The array is cloned to prevent response handling or callers from modifying the cached value.

---

3. Runtime fallback

A generated workspace is not mandatory. Normal source workspaces still work.

When cachedScopes is absent, the endpoint uses:

WeakMap<Workspace, Promise<Array<string>>>

The diagram covers the build-time cache pipeline, runtime fast path, WeakMap fallback, and srcMap provider deduplication.