[Personalize] Support dynamic token replacement from decision tables

[bransbury]

• This PR follows the Contribution Guide
• Changelog updated
• Upgrade guide entry added

Description / Motivation

This PR adds support for dynamic token replacement from Sitecore Personalize decision table responses across the JSS personalization pipeline.

Problem

When Sitecore Personalize decision tables return tokens (e.g. { firstName: "Bob", city: "Portland" }), there is currently no mechanism in JSS to apply these values to layout data. Content authors can configure {{firstName}} placeholders in their content, but the values are never substituted at render time.

Solution

New token-replace utility (sitecore-jss/src/personalize/token-replace.ts):

• replaceTokens(): substitutes {{key}} and {{key|fallback}} mustache-style tokens in strings
• replaceTokensInObject(): recursively walks layout data and applies token replacement to all string values
• Supports fallback values ({{name|Valued Customer}}), whitespace-trimmed keys, removeUnmatched option, and onUnmatched callback
• Includes prototype pollution guards (__proto__, constructor, prototype keys are skipped)
• ReDoS-safe regex design (no polynomial backtracking)

Next.js middleware (personalize-middleware.ts):

• Collects tokens from each personalize execution's decision table response
• Encodes tokens as base64 JSON in the x-sc-personalize-tokens response header
• Tokens are keyed by variant friendly ID for multi-execution scenarios

Next.js page-props plugin (personalize.ts template):

• Reads the x-sc-personalize-tokens header from the incoming request
• Merges per-variant token maps with collision detection/logging
• Applies replaceTokensInObject() to layout data after personalizeLayout() runs
• Validates all token values are strings; guards against prototype pollution

Proxy PersonalizeHelper (sitecore-jss-proxy):

• getVariantIds() now returns { variantIds, tokens } to pass tokens through
• Applies token replacement in personalizeLayoutData() after layout personalization

🛠 Breaking Changes

PersonalizeHelper.getVariantIds() in sitecore-jss-proxy changed its return type from Promise<string[]> to Promise<{ variantIds: string[]; tokens: Record<string, string> }>. This is a protected method — only subclasses that override it need to update their signature. This is unlikely to affect most consumers since the proxy package is used by non-Next.js rendering hosts and this method is not part of the public API surface.

Testing Details

• Unit Test Added
• Manual Test/Other (Please elaborate)

New test coverage:

• token-replace.test.ts: 25+ test cases covering: single/multiple token replacement, fallback values, whitespace trimming, removeUnmatched, onUnmatched callback, nested object traversal, array handling, null/undefined safety, prototype pollution guard, ReDoS guard, HTML/URL content, special regex characters in values
• personalize-middleware.test.ts: updated tests for token forwarding via header, multi-execution token aggregation, base64 encoding
• PersonalizeHelper.test.ts: updated tests for token application to layout data, empty tokens handling, collision detection

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to change)

Packages affected

Package	Changes
sitecore-jss	New token-replace.ts utility, exported from personalize subpath
sitecore-jss-nextjs	Middleware collects & forwards tokens; re-exports PERSONALIZE_TOKENS_HEADER
sitecore-jss-proxy	PersonalizeHelper applies tokens to layout data (⚠️ breaking: getVariantIds return type)
create-sitecore-jss	Updated XM Cloud template personalize.ts plugin to consume tokens

[bransbury]

Looks like this was committed erroneously in the past so I've removed and added to .gitignore. This is unrelated to the main changes