Skip to content

Reduce the repeated validation and allocation - #11

Merged
xinz merged 9 commits into
mainfrom
opti/validate
Sep 11, 2026
Merged

xinz merged 9 commits into
mainfrom
opti/validate

Conversation

@xinz

@xinz xinz commented Sep 10, 2026 •

Copy link
Copy Markdown
Owner

Reduce string-length counting allocations to minLength and maxLength.

Direct predicate pass cases

Median µs/op, long inputs:

Input Code points minLength before → after maxLength before → after
ASCII 100,000 527.667 → 242.787 (0.460×) 514.472 → 236.729 (0.460×)
Two-byte 100,000 695.769 → 440.267 (0.633×) 677.444 → 436.630 (0.645×)
Three-byte 100,000 896.143 → 464.500 (0.518×) 891.429 → 472.043 (0.530×)
Four-byte 100,000 1,022.611 → 404.000 (0.395×) 961.550 → 419.020 (0.436×)
Combining 200,000 1,524.615 → 636.968 (0.418×) 1,491.571 → 639.484 (0.429×)

Both predicates improve materially for long strings: roughly 1.55×–2.53× faster, depending on UTF-8 width.

Public validation with both keywords

Input Code points Both pass: before → after Both fail: before → after
ASCII 100,000 1,056.875 → 488.459 (0.462×) 1,057.050 → 482.047 (0.456×)
Two-byte 100,000 1,396.500 → 891.087 (0.638×) 1,382.214 → 881.304 (0.638×)
Three-byte 100,000 1,825.909 → 943.500 (0.517×) 1,698.400 → 936.190 (0.551×)
Four-byte 100,000 2,006.111 → 810.458 (0.404×) 2,010.500 → 839.864 (0.418×)
Combining 200,000 2,938.667 → 1,300.333 (0.442×) 2,900.833 → 1,300.444 (0.448×)

The combined minLength + maxLength validation path is about 1.57×–2.48× faster for these long-input workloads.

Share regex compilation between sibling object keywords

  • When one schema node contains both patternProperties and additionalProperties, each successful pattern regex is compiled once and reused.
  • The cache is local to that keyword pass—no ETS, process dictionary, global state, or public schema field.
  • Schemas without both sibling keywords retain the original reducer and handlers, avoiding added per-keyword overhead on the common path.
  • $defs ordering, vocabulary gates, regex-error paths, child-schema error paths, rule order, and compiled rule shapes are preserved.
Patterns Pattern-only control: before → after additionalProperties: false: before → after After / before additionalProperties: true: before → after After / before
16 0.0615 → 0.0623 0.1271 → 0.0731 0.575× 0.1289 → 0.0689 0.535×
32 0.1281 → 0.1311 0.2633 → 0.1422 0.540× 0.2589 → 0.1436 0.555×
128 0.5164 → 0.5420 1.0591 → 0.5536 0.523× 1.0652 → 0.5727 0.538×
512 2.2074 → 2.2090 4.4497 → 2.3621 0.531× 4.4787 → 2.4099 0.538×

Avoid repeated runtime pattern classification

  • The cache:
    • is scoped to one validate_entry/5 invocation;
    • stores structural regex-match facts only, never annotations, errors, child-validation results, or reference state;
    • supports both sibling execution orders;
    • preserves all overlapping patternProperties validations;
    • preserves the distinction between structural matches and evaluated-property annotations;
    • does not run regexes before preceding rules such as $ref / loaders;
    • exits the cache-aware dispatcher immediately after both sibling rules have executed.
  • Added a no-allocation bypass for the one-pattern/one-key case, where cache setup costs more than the one duplicated regex check.

For 512 unmatched keys and 512 patterns:

Rule order Reductions before → after
patternProperties → additionalProperties 4,468,985 → 2,108,016 (~53% less)
additionalProperties → patternProperties 4,469,734 → 3,160,295 (~29% less)

The reverse order deliberately retains checks for later overlapping patterns. Pattern-only controls remain effectively unchanged; all-declared-property inputs have no duplicated classification to remove, so no speedup is claimed there.

Reduce evaluated-key set/list round trips when validate anyOf

  • Direct merging is considered only for an anyOf with at least four child schemas and an incoming evaluated-key set of at least eight entries. This keeps ordinary, small, and empty-parent anyOf calls on their prior reducer.
  • Every specialized child call still receives the identical original evaluated set. Successful branches are all visited, including later $ref branches; evaluated keys from earlier branches are never supplied to a later child.
  • The specialized reducer initially retains successful child evaluated-key %MapSet{} values and tracks their total cardinality. It uses balanced pairwise MapSet.union/2 only when the known shared incoming evaluated keys make up at least two thirds of successful child evaluated-key memberships and at least four children succeed.
  • If any successful prefix falls below that two-thirds ratio, the reducer immediately converts its reverse %MapSet{} accumulator into the exact list-accumulator shape used before P8 and resumes the original reducer. A later branch is deliberately not allowed to restore direct merging: this conservative cutoff avoids retaining a losing raw-set buffer.
  • The specialized reducer is called only by validate_anyOf/5; other applicator paths retain their existing result-handling path.
  • Direct MapSet.union/2 would preserve arbitrary list-valued Elixir keys, whereas the historic list-rebuild path recursively flattens them. After a candidate direct merge, the deduplicated evaluated-key set is checked once; if it contains a list value, the code reruns the historic bulk merge exactly. JSON object property names remain strings, and no non-JSON legacy behavior changes.
  • No schema/rule/context fields, process state, ETS table, global cache, loader behavior, dynamic scope, error accumulator, or public result shape changed. Failed branches remain suppressed after an anyOf success; all-failed error order remains source order.

These are synthetic anyOf validation workloads, not general application speedup guarantees.

Workload Width Before (ms) After (ms) Time change Reductions before → after Reduction change
anyOf, parent=16, one new evaluated key/branch 4 0.0093 0.0036 -61.3% 444 → 408 -8.1%
anyOf, parent=16, one new evaluated key/branch 16 0.0402 0.0130 -67.7% 1,396 → 946 -32.2%
anyOf, parent=16, one new evaluated key/branch 64 0.0770 0.0298 -61.3% 4,923 → 3,085 -37.3%
anyOf, parent=16, one new evaluated key/branch 128 0.1433 0.0618 -56.9% 9,732 → 5,931 -39.1%
Empty-parent, disjoint anyOf bypass 128 0.0293 0.0296 +1.0% 4,838 → 4,839 +0.0%
Parent=1, 16 evaluated keys/branch bypass 128 0.4296 0.4312 +0.4% 19,974 → 20,045 +0.4%
Candidate that immediately resumes bulk merging 128 0.5494 0.5574 +1.5% 26,313 → 24,582 -6.6%
Candidate that resumes bulk merging after 127 retained sets 128 0.4212 0.4466 +6.0% 18,337 → 19,300 +5.3%

The parent-dominated target shape eliminates repeated conversion of the same incoming evaluated keys. It reduced reductions by 8.1% at width 4 and by 32.2%–39.1% at widths 16–128; wall time was 56.9%–67.7% lower in this run. The two bypass controls remain within run variation, so no broad anyOf speedup is claimed. An immediate fallback also has no claimed speedup. The deliberately conservative late-fallback case pays 5.3% more reductions to convert its retained successful-branch prefix and then resume bulk merging; its 6.0% wall time increase is reported as a workload-specific cost, not a semantic change.

Reduce repeated error-list flattening call validation

  • Already-flat %Error{} lists now pass through unchanged. Nested error groups and legacy tuple entries are flattened and formatted in one traversal.
  • Internal keyword/rule error grouping and unevaluated* rewriting remain unchanged; a broader internal error-tree mode is deferred.
  • Fix a private recursive validation_error_list type that validate_entry/5 already returns.
Workload Size Before (ms) After (ms) Time change Reductions before → after Reduction change
Flat item errors 128 0.0097 0.0071 -26.8% 2,587 → 1,944 -24.9%
Flat item errors 1,024 0.1483 0.1343 -9.4% 20,507 → 15,384 -25.0%
Flat item errors 8,192 1.3570 1.1811 -13.0% 176,082 → 133,009 -24.5%
All-failed allOf 128 branches 0.0073 0.0047 -35.6% 2,207 → 1,564 -29.1%
All-failed anyOf 128 branches 0.0069 0.0051 -26.1% 2,212 → 1,569 -29.1%
All-failed oneOf 128 branches 0.0074 0.0051 -31.1% 2,207 → 1,564 -29.1%
False unevaluatedProperties 1,024 properties 0.1928 0.1450 -24.8% 32,782 → 26,682 -18.6%
Nested properties, 256 errors depth 1 0.0211 0.0167 -20.9% 7,595 → 6,312 -16.9%
Nested properties, 256 errors depth 32 0.0886 0.0868 -2.0% 23,963 → 22,680 -5.4%
Nested properties, 256 errors depth 128 0.3055 0.3085 +1.0% 74,651 → 73,368 -1.7%

Wide flat errors consistently remove roughly one quarter of reductions, while 128-branch failed applicators remove about 29%. False-schema rewriting also benefits because its already-rewritten flat list no longer gets rebuilt at the
public boundary. The valid primitive control remains at 14 reductions. At deep property depth 128, unchanged internal flattening dominates and wall time is within variation; no speedup is claimed for that shape.

Remove redundant input-key presence checks when validating required

N/A

Avoid rescanning built-in Draft 2020-12 resources after compilation, and reduce scope-scanner traversal

  • The compiler-generated registry already contains every canonical resource and #meta identity, so built-in family construction no longer repeats the same raw-resource scope scan after each compilation.
  • ScopeScanner now reduces scope-bearing children directly. The existing Map.values/1 order remains explicit for map-valued keywords, preserving duplicate-identity and anchor overwrite precedence.

The traversal benchmark validates production and benchmark-only reducer outputs
with === before measurement. Each row reports median milliseconds per complete
ScopeScanner.scan/1 operation from five adaptive batches targeting 20 ms;
result verification, input/source construction, warmup, and explicit GC are outside
measurement. Before reloads schema_traversal.ex and scope_scanner.ex from
8bef311 (combined_sha256=edbd1e34600a82044f14332fc9e5f27904cb312211b10cf79004be8c2ec7ce5b);
after uses the working tree
(combined_sha256=c85cb18db066cc54e092a7e1f7aa7c8421224ee6f2125bc8d7efb11cfc6b2fc8).

Scanner input Before (ms) After (ms) Change Reductions before → after
Built-in family, 9 resources 0.0789 0.0600 24.0% faster 26,388.8 → 20,042.2
Wide properties, 1,024 schemas 7.6620 6.8230 11.0% faster 545,973.5 → 479,825.5
Nested properties, depth 1,024 6.3843 4.5358 29.0% faster 407,000.7 → 346,931.5
4,096 inactive extension values 0.0012 0.0009 tiny/noisy 262.8 → 200.2

| Fixed cutoff | Mean direct-counter time | Relative to best |
|---|---:|---:|
| unrolled only | 3,484.12 ns/op | +1.31% |
| <= 64 bytes | 3,475.09 ns/op | +1.04% |
| <= 128 bytes | 3,457.94 ns/op | +0.55% |
| **<= 256 bytes** | **3,439.16 ns/op** | **best** |
| <= 384 bytes | 3,441.56 ns/op | +0.07% |
| <= 512 bytes | 3,466.82 ns/op | +0.80% |
@xinz
xinz merged commit 3edf5f8 into main Sep 11, 2026
1 check passed
@xinz
xinz deleted the opti/validate branch September 11, 2026 14:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant