Proposal for fixing schema validation rejects documents containing $schema - #620
Closed
kehoecj wants to merge 121 commits into
Closed
Proposal for fixing schema validation rejects documents containing $schema#620kehoecj wants to merge 121 commits into
kehoecj wants to merge 121 commits into
Conversation
- Rename binary from `validator` to `cfv` — breaking change, no compat shim - Add subcommand router: `cfv check` (validation), `cfv format` (stub), `cfv version`, `cfv help [subcommand]` - `cfv check` is functionally identical to the old `validator` binary - Bare `cfv [flags] [paths]` dispatches to check (no subcommand = check) - Reserve `--fix` and `--unsafe` flags on check (no-op until Phase 4) - Update module path: v2 → v3 - Update version string: 'validator version' → 'cfv version' - Migrate all txtar integration tests to cmd/cfv/testdata/ - Add /cfv to .gitignore Migration: `validator .` → `cfv check .` Coverage: 93.9% total, cmd/cfv 96.6%
- Replace `validator [flags]` with `cfv check [flags]` throughout - Update go install path: v2/cmd/validator → v3/cmd/cfv - Update build commands: cmd/validator/validator.go → cmd/cfv/cfv.go - Update binary name in install/verify instructions - Update CLI reference to show subcommand structure - Remove validator --version in favor of cfv version subcommand
Spec files were written to resolve ambiguities before implementation. Now that decisions are settled, inline them into the plan and delete the separate files. One document, no dead links.
1. cfv --version now works (global FlagSet parses --version before subcommand dispatch) 2. Package-level flagSet var removed. fs is threaded explicitly or accessed via cfg.fs/cfg.isFlagSet(). No more shared mutable state. 3. Error messages use fmt.Fprintf(os.Stderr, "cfv: %v") instead of log.Printf. No timestamps, consistent prefix. 4. Stdin duplicate validation: 'cfv check - -' and 'cfv check - file' both produce clear errors instead of confusing filesystem walk errors. Tests added: version.txtar, stdin_duplicate.txtar, --version in Test_subcommandRouter.
Formatting output must be visually identical across all formatters. Same symbols, same line format, same message structure as check. One tool, not a patchwork.
The formatting engine's public contract: - Formatter.Format(src, opts) returns canonically formatted bytes - Options uses zero-value = format-specific defaults pattern - IsFormatted helper: Format(src) == src (byte equality) - Result type carries pipeline state (changed, error) Constraints documented in package doc: - Stateless (concurrent-safe) - Idempotent (Format(Format(x)) == Format(x)) - Comment-preserving (non-negotiable) No formatters yet — interface only. The JSON formatter (next commit) will be the reference implementation that proves this interface works.
Breaking change to the Report type (v3 module path):
Old: IsValid bool, ValidationError error, ValidationErrors []string,
Warnings []string, ErrorType string, StartLine/StartColumn int,
ErrorLines/ErrorColumns []int
New: Status (Pass/Fail/Unformatted), Issues []Issue (Type, Message,
Line, Column), Notes []string
This enables the formatter to produce Reports with StatusUnformatted
and IssueTypeFormat — same struct, same reporters, consistent output.
Updated:
- pkg/reporter/reporter.go — new types
- pkg/reporter/stdout_reporter.go — renders ✓/×/~ based on Status
- pkg/reporter/json_reporter.go — status string field
- pkg/reporter/junit_reporter.go — uses Issues for failures
- pkg/reporter/sarif_reporter.go — maps Issues to SARIF results
- pkg/reporter/github_reporter.go — emits ::error or ::warning by Status
- pkg/cli/cli.go — validate() builds Issues from errors
- pkg/cli/group_output.go — groups by Issue type
- All reporter and CLI tests updated
- schema_map.txtar: warning → note (schema warnings are now Notes)
- pkg/formatter/formatter.go: Formatter interface, Options, DefaultFormatOptions() - pkg/formatter/jsonfmt/: JSON formatter using tidwall/pretty - 11 fixture tests, idempotency tests, option tests, fuzz seed corpus - sort keys, indent width/style, final newline, CRLF line endings - pkg/filetype/formatters.go: registers formatters on FileTypes slice - pkg/filetype/file_type.go: Formatter field added to FileType struct - pkg/cli/format.go: Format() method on CLI, parallel worker pool, atomic file writes (temp + rename), WithFix option - cmd/cfv/cfv.go: runFormat() wired to real pipeline (not stub) parseFormatFlags() with same finder infrastructure as check - cmd/cfv/testdata/format.txtar: cfv format . reports unformatted files - cmd/cfv/testdata/format_fix.txtar: cfv format --fix . rewrites files cfv format . now works for JSON files. Other formats: nil Formatter, silently skipped until their formatter is registered.
Added 'Current State — Resume Here' section at top with: - What's done (Phase 1 complete, Phase 2 JSON done) - What's next (YAML formatter, Task 5) - Exact commands to resume - Pipeline state Updated Phase 2 task list with ✅/🔲 status markers.
Phase 2 Tasks 4.5–6: Full formatting vertical slice. Formatters: - JSON: tidwall/pretty, indent/tabs/sort-keys/line-width/CRLF - YAML: goccy/go-yaml token-preserving AST, indent/sort-keys/quote-style - HCL: hclwrite.Format canonical style wrapper Config system: - .cfv.toml [format] + [format.<type>] sections with JSON Schema validation - CLI flags: --indent, --use-tabs, --sort-keys, --no-sort-keys, --line-ending, --max-line-width, --quote-style, --diff - Resolution cascade: CLI > per-format config > global config > defaults - Strict validation: unknown keys error, invalid values error, range checks Architecture: - FormatOptionsFunc for per-format option resolution - FileSystem interface on writeFileAtomic for testability - Fail-fast on unsupported YAML constructs (clear error, not corruption) - AddColumn-based depth tracking for indent normalization Library swap: - Replaced gopkg.in/yaml.v3 with github.com/goccy/go-yaml - Eliminates: !!merge injection, folded joining, emoji quoting, comment instability - Both formatter and validator use single YAML library Testing: - 94.3% coverage (up from ~91%) - 17M fuzz executions (60s × 2 targets) - Exhaustive cascade txtar tests (46 commands) - CLI flag validation tests (14 assertions) - Mock filesystem tests for writeFileAtomic error paths - Real-world smoke test (1830 files) Bug fixes: - Parse errors no longer reported as pass - StatusFail now triggers exit 1 in format mode - Multi-doc duplicate keys caught in all documents - quote-style only affects values, never keys - Flow-style nodes skip reindent (prevents panic) - CLI flags validated (range, enum, mutual exclusion)
Phase 2 Task 7+8: Five new formatters extending cfv format coverage. Formatters added: - XML: helium DOM-based (replaces go-xmlfmt, -1 dep). Mixed content detected and skipped with ErrSkipped notification to user. - TOML: line-oriented with pelletier/go-toml/v2 validation. Comment-preserving (inline and standalone). - INI: line-oriented with gopkg.in/ini.v1 validation. IgnoreContinuation enabled (backslashes are literal). - Properties: line-oriented with magiconair/properties validation. Multiline values preserved verbatim. Escape sequences untouched. - ENV: custom line-oriented, zero external deps. Also includes: - Task 4.5.9: resolveConfig split into resolveBaseConfig + resolveCheckConfig + resolveFormatConfig - ErrSkipped sentinel type in pkg/formatter for skippable files - HCL empty input guard (returns newline instead of nil) - INI/Properties re-validation fallback for edge cases - FormatConfig.Properties field added to .cfv.toml schema Stress tested: 57M+ fuzz executions across all formatters. Pipeline: 93.2% coverage, 0 lint issues, all tests pass. Net dependency change: -1 (removed go-xmlfmt/xmlfmt).
Phase 4: Fix Engine with 3 safe fix rules, proven under 14.8M fuzz executions. Rules: - json-trailing-comma: removes trailing commas before ] and } - schema-string-to-int: coerces "8080" to 8080 per schema - schema-string-to-bool: coerces "true"/"false" to true/false per schema Architecture: - Rule interface (ID + Detect) produces byte-range Fix structs - Fixer applies fixes left-to-right, drops overlapping conflicts - CLI verifies fixed output passes validation before writing - Atomic file writes (temp + rename) Testing: - Fixpoint harness: output matches expected + schema passes + idempotent - No-regression: valid files unchanged - Fuzz: 7.2M executions (trailing comma) + 7.6M (schema fixes) = 0 failures - Integration: 2 txtar tests prove end-to-end CLI behavior Pipeline: 91.8% coverage, 0 lint issues, all tests pass.
CST-based JSONC formatter that handles JSON with Comments and trailing commas. Uses tailscale/hujson (already in go.mod) for lossless parsing and serialization. Features: - Comment preservation (line // and block /* */) - Configurable indent (spaces or tabs) - SortKeys (recursive alphabetical sort) - Standard JSON passthrough (no trailing commas added to pure JSON) - Short primitive arrays kept inline (<80 chars) - Trailing commas on multiline JSONC structures Architecture: - Custom CST walker (formatState) for indentation — no use of hujson Format() which has alignment behavior incompatible with configurable indent width. - Idempotent by construction: same tree always produces same output. - Zero bail-outs, zero re-validation checks. Known files now formatted: tsconfig.json, .eslintrc.json, .babelrc, devcontainer.json, jsconfig.json, and all .jsonc extension files. Also adds XML fuzz corpus entry documenting helium StripBlanks bug (upstream issue filed). Stress tested: 10M+ fuzz executions, zero failures. Pipeline: 91.6% coverage, 0 lint issues, all tests pass.
Decision analysis: - pelletier unstable: rejected (API stability label) - WASM taplo: rejected (50% binary bloat, build complexity) - Fork go-toml: rejected (maintenance burden) - Custom tokenizer: accepted (format-only, simpler than full parser) Competition is taplo. Bar: no one misses taplo because cfv cannot handle their file. 7-8 day estimate.
Based on analysis of taplo source code (lexer, parser, formatter). Covers: token types, lexer design (string boundary handling), grouper, printer, edge cases, test strategy, and 8-day implementation schedule. Decisions: match taplo defaults (auto-expand arrays at 80 cols, trailing commas on multiline arrays, expand inline tables, max 2 blank lines).
Replace line-oriented TOML formatting with a tokenize→group→print CST
pipeline. Eliminates all bail-outs and enables full formatting features.
Architecture:
- Tokenizer: lossless lexer, all 4 string types, multiline boundary
detection (ported from taplo). 9M+ fuzz executions clean.
- Grouper: token stream → logical groups. Tracks bracket/brace depth
for multiline values. Merges consecutive blank lines.
- Printer: normalized key=value spacing, configurable indentation,
SortKeys with comment attachment, blank line collapsing, array
auto-expand at 80 columns, trailing commas, inline table normalization.
Value normalization:
- Inline tables: { key = "val", key2 = "val2" }
- Arrays auto-expand when line exceeds 80 columns (with key prefix)
- Trailing commas on multiline array elements
- Correct indent depth for expanded array elements
- Values containing comments: preserved verbatim (matches taplo behavior
where comment-containing arrays cannot be collapsed)
Validation: pelletier/go-toml/v2 Unmarshal (no unstable package usage).
Tested:
- 9 fixture pairs (basic, comments, sections, multiline, already_formatted,
simple, inline_table, array_expand, sort_keys)
- 9.3M fuzz executions, zero failures
- Real-world: Cargo.toml (ripgrep), pyproject.toml (black), hugo.toml
- Code review: 3 bugs found and fixed (comment attachment, table header
spacing, multi-line comment indent)
- golangci-lint: 0 issues
The stress test exercises the real cfv binary end-to-end: cfv format --fix → cfv check → cfv format (idempotent) Covers every format with intentionally messy input files. Must be automated, not manual QA.
Replace line-oriented properties formatting with tokenize→print pipeline. Eliminates all bail-outs (4 previously), removes re-validation check and idempotency check. Architecture: - Tokenizer: handles keys (with escape sequences), separators, values (with continuation lines), comments (# and !), form feed, bare \r. - Printer: groups entries with attached comments, normalizes separator to " = ", preserves continuations verbatim, SortKeys with comment attachment. Validation: magiconair/properties (unchanged). Proven: - 9M fuzz executions, zero failures - All existing fixtures pass unchanged - CLI: cfv format --fix on Spring Boot application.properties with continuations, both comment styles, messy spacing - Idempotent, validates after formatting - golangci-lint: 0 issues
- cfv format --help now shows all 9 registered formatters dynamically instead of hardcoded "json" - Added JSONC to FormatConfig struct and per-format config resolution so .cfv.toml [format.jsonc] overrides work Users can now configure: [format.jsonc] indent = 4 sort-keys = true
Phase 6A.1-6A.3 (partial): Custom YAML tokenizer for format-preserving formatting. Every byte of input is exactly one token. Losslessness proven via 50 unit tests and 9.3M fuzz executions (zero failures). Tokenizer handles: indent, newlines, comments, block scalars (opaque), flow collections (opaque), key/colon/value splitting, sequence entries, document markers, directives, anchors, aliases, tags. Printer implements: indent normalization (depth-based reindent) and SortKeys (entry grouping by depth, alphabetical sort within scope). Also in this commit: - propfmt: Fix #1 (blanks travel with entries during sort) and Fix #2 (sep token assignment). New fixtures: continuation, escaped_keys, sorted_comments. - inifmt: Fix 4 lint issues (identical-switch-branches, unhandled-error, tagged switch, gofmt alignment). New fixtures: escaped_keys, sort_escaped. New edge case tests (CR, CRLF, whitespace-only). - yamlfmt: ErrSkipped for empty input, error for IndentTabs, fix hasFormattableRoot nil-body edge case. - PLAN.md: Phases 2, 3, 6 marked complete. Phase 6A plan added. The YAML tokenizer/printer are additive — the existing goccy-based Format function is unchanged. Task 6A.4 (swap Format to use CST) is next.
Replace goccy/go-yaml with gopkg.in/yaml.v3 across the entire project. goccy was too permissive as a validator — it accepted invalid YAML that the spec (and prettier, and yaml.v3) correctly rejects. This caused formatter instability on pathological inputs. Changes: - pkg/validator/yaml.go: Rewrote using v2 code as reference. Decoder loop for multi-doc, yaml.Node for position map, regex error parsing. - pkg/formatter/yamlfmt/yaml.go: Clean 98-line file. yaml.v3 Decoder loop validation, bare scalar rejection, CST pipeline. All dead goccy AST code removed (~200 lines). - pkg/formatter/yamlfmt/yaml_test.go: Removed goccy/parser import, removed t.Skip workarounds for goccy bugs. - internal/generate/knownfiles/main.go: Swapped import. - go.mod: goccy removed, yaml.v3 promoted to direct. Also fixes 18 golangci-lint issues in printer.go and tokenizer.go (early-return patterns, identical branches, unused function, renamed 'close' builtin shadow, tagged switches). Pipeline: go vet clean, gofmt clean, 0 lint issues, 91.2% coverage. Fuzz testing reveals remaining edge cases in reindent logic for pathological inputs (sibling sequence entries at same indent). These are now correctly rejected by yaml.v3 in real usage but still trigger in fuzz. Tracked for follow-up.
…ejection Fixes from fuzz testing: - Block scalar content lines now shift by same delta as parent indent (plan requirement that wasn't implemented) - TokDash no longer swallows trailing newlines (was causing sibling sequence entries to get different depths) - Nil root documents rejected (prevents binary garbage that yaml.v3 parses as nil from reaching the tokenizer) - SortKeys: entries get newline separation after reorder - SortKeys: trailing whitespace excluded from last entry range - Trailing whitespace stripped from all output lines - Multi-line quoted values detected correctly (isFullyQuotedSingle/Double) - Inline comment space normalization only after content, not at line start - lineHasStructure flag gates block scalar detection Remaining: continuation value reindent needs structural knowledge from yaml.v3 Node tree (planned in PLAN-reindent-fix.md). Fuzz still finds edge cases with plain scalar continuations.
Use yaml.v3's Node tree to distinguish structural lines (keys, sequence items) from continuation lines (multi-line scalar values). Only structural lines get independently renormalized; continuations shift by parent delta. Also: validate newline-appended form of input to catch parser inconsistencies. Fix findEntryStart to not grab inline comments as leading comments. Skip blank-line indents in depth computation. Thread structuralLines through sortKeys path. Remaining issue: after SortKeys reorders entries, the line-number-based structural map becomes stale. Need to attach structural classification to tokens directly rather than using line numbers. Tracked for next fix.
Replace the spaghetti of external maps (structuralLines, lineNums, depths) threaded through every function with metadata on the Token struct itself: Structural bool and Depth int. Architecture: 1. tokenize(src) → tokens with Kind + Raw 2. annotate(tokens, src) → sets Structural + Depth using yaml.v3 Node tree 3. reindentTokens(tokens, width) → modifies Raw on indent tokens 4. sortKeys(tokens) → reorders; metadata travels with tokens 5. serialize → concatenate Raw fields Key insight: after sort reorders tokens, line-number-based maps become stale. Token-level metadata survives reordering. Fuzz results (both 45s, zero failures): FuzzYAMLFormatter: 634K executions FuzzYAMLFormatterWithOptions: 1.2M executions Pipeline: 0 lint issues, all tests pass across entire project.
… content Replace helium Writer-based serialization with custom tokenizer + printer. Same architecture as YAML: tokenize → annotate → format → serialize. Key achievements: - Mixed content handled correctly (preserved verbatim, no ErrSkipped) - The helium entity-encoding bug (StripBlanks) is eliminated - The helium Writer.Format mixed-content bug is eliminated - helium is now used ONLY for validation (Parse), not serialization New features: - XMLWhitespaceSensitivity option: ignore (default, pretty-print) / preserve - XMLSelfClosingSpace option: control space before /> - Both modes work: 'ignore' inserts formatting, 'preserve' only reindents Tokenizer: handles XMLDecl, Doctype, ProcInst, Comment, CDATA, OpenTag, CloseTag, SelfClose, Text, Indent, Newline. 6.4M fuzz executions on tokenizer losslessness, zero failures. All existing fixtures pass. The previously-failing fuzz corpus entry (545d7bc9e35711a4) now passes — the helium bug no longer affects us. STATUS: 9/9 formats with zero bail-outs. - JSON ✅ JSONC ✅ YAML ✅ TOML ✅ XML ✅ - INI ✅ Properties ✅ ENV ✅ HCL ✅
…s, fuzz fixes Phase 9 code review fixes (35 findings): - YAML SortKeys anchor/alias safety (skip sort when dependencies exist) - XML mixed content detection (inline during formatting, not annotation) - XML PI classification fix (xml-stylesheet not misclassified as XMLDecl) - TOML comment duplication in arrays fixed - Properties \uXXXX decoding in sort keys - INI/XML/JSONC documentation and defensive fixes - Removed dead Result struct from formatter package Phase 7B formatter hardening: - YAML |+/|- block scalar chomping preservation (serializeWithStrip) - XML semantic equivalence verification (real DOM comparison) - propfmt EOF continuation fix (break at EOF, 8 explicit test cases) - Real-world corpus test (11 repo config files) - 102-case ephemeral stress test with semantic equivalence checking - Fuzz targets with option cycling for all 6 configurable formatters Phase 7C architectural fixes: - YAML: replaced indent-based depth heuristic with AST-derived depth (single source of truth from yaml.v3 Node tree) - Deleted computeDepths/recomputeDepths/AfterDash (the heuristic) - New: buildASTMetadata, collectMetadata, assignASTMetadata, computeNewIndent - Formula: indent = depth*tw + seqOffset*2 + 2 if (inSeq && !hasDash) - Sort uses ASTDepth (position-invariant, no recomputation after reorder) - Matches prettier output for sequence mappings at all indent widths - TOML extractKey: skip leading whitespace (SortKeys + indent fix) Pipeline: vet, fmt, golangci-lint (0 issues), build, 22 packages pass. Coverage: 91%+.
Adds TestIndentSequencesDisabled covering single-level, nested, deeply nested, comments, and top-level sequences. Confirms that compact mode aligns dashes at the parent key's column (yamlfmt-compatible behavior). The column 0 output for root-level keys is correct — it matches Google's yamlfmt with indentless_arrays=true. The default mode (indent-sequences=true) matches prettier.
Refactor in progress (stashed). Object collapsing works but is too aggressive — parity regressed from 79% to 66%. Root cause likely missing key-name length in line-width calculation. All unit tests pass. Next session: diagnose with real file, fix, verify no regression.
JSON/JSONC:
- Drop tidwall/pretty, use hujson-only format engine
- Fix object collapsing regression (preserve source multiline state)
- Add keyPrefixLen for accurate line-width calculation
- Fix empty object spacing ({} not { })
- JSONC MaxLineWidth default resolution
TOML:
- Array expansion inside inline tables when line > 80
- Nested array recursive expansion (inExpandedArray flag)
- Sub-table blank line suppression (no blank between siblings)
- Comment alignment from formatted widths (not source columns)
- Array-with-comments no longer emitted verbatim
YAML:
- Sequence indentation under mapping keys (zero-width TokIndent)
- Flow quote normalization (convertFlowQuotes)
- Block scalar content indent normalization
- Block scalar trailing blank line trimming (clip chomping)
- Blank line collapse (max 1 between elements)
- Flow bracket spacing normalization (no space after [, before ])
- Comment spacing after colon (key: # → key: #)
- Document marker indent fix (always column 0)
- Blank-line indent fix (stay at 0, prevent delta accumulation)
- Block scalar internal blank line whitespace stripping
Parity suite:
- Skip invalid JSON files (JSON.parse check)
All 22 test packages pass. golangci-lint: 0 issues.
TOML: - Fix estimateInlineTableWidth to account for normalized spacing - Sub-table/sibling blank line suppression - Include trailing comment width in array expansion line-width calc - Trailing comments in arrays stay inline (splitByComma fix) - Comment alignment within expanded arrays (pre-compute max width) YAML: - Fix blank-line indent accumulation (stay at 0) - Document markers always at column 0 - Strip whitespace from blank lines inside block scalars TOML 98.7%, YAML 98.7%, JSON 97.2%, Overall 97.8% (405/414).
Implements prettier's printNumber rules (verified against source): - Lowercase scientific notation (E → e) - Remove unnecessary + and leading zeros in exponent (E+034 → e34) - Remove trailing zeros after decimal point (1.10 → 1.1, 1.0 stays) Also fixes parity suite to pass config:false to prettier and disable all external config in cfv (--no-prettier-config, --no-taplo-config, --no-yamlfmt-config) for fair defaults-only comparison.
Taplo's spec: allowed_blank_lines controls PRESERVATION of source blank lines, not insertion. Previously cfv added blank lines before every table header via ensureBlankLine(). Now it only preserves blank lines that were in the source (represented as GroupBlank in the group sequence). Source: taplo-0.14.0/src/formatter/mod.rs — Options.newlines() caps at allowed_blank_lines+1 but never adds beyond source count. Also removes now-unused ensureBlankLine, isTable, nextNonCommentKind, extractTableKey, and tableParent functions.
Implements taplo's comment alignment algorithm from source: - Align across ALL entries in a group (not just commented ones) - Blank lines break alignment groups - Multiline values disable alignment for the entire group - Predict array expansion (same width logic) to detect future multiline Source: taplo-0.14.0/src/formatter/mod.rs format_rows(), can_align check TOML parity: 148/149. Overall: 99.0% (408/412).
Per prettier source (flow-mapping-sequence.js): flow sequences that exceed printWidth get expanded to multi-line flow format with each element on its own line, indented by tabWidth, with trailing comma. Adds expandLongFlowSequences pass + splitFlowElements parser. Also adds MaxLineWidth (80) to YAML DefaultOptions + resolveOptions. Source: prettier src/language-yaml/print/flow-mapping-sequence.js
…ents spec Implement taplo's format_rows alignment algorithm for trailing comments inside expanded arrays (mod.rs lines 1017-1019, 1229-1295): - Split array elements into alignment sub-groups at standalone comments and blank lines (matching taplo's add_values flush behavior) - Compute max value width across ALL entries in each sub-group - Position trailing comments at max_width + 1 space separator - Active even with a single trailing comment (align_single_comments: true) Previous behavior used a hardcoded 2-space gap when fewer than 2 trailing comments existed, and did not split alignment groups at standalone comments. Resolves #586. TOML parity: 149/149 (100%).
Implement prettier-compatible fill layout for all-numeric arrays: - isConciseArray: detects arrays where all elements are numeric literals - formatArrayFill: packs items per line up to printWidth (fill layout) - Preserves source blank lines between elements (forces expansion) - Short concise arrays with no blank lines still collapse inline Add missing number normalization rule from prettier's printNumber: - Rule 2b: remove unnecessary scientific notation when exponent is 0 (1e00 → 1, 2e+00 → 2, 2e-00 → 2) Source traceability: - prettier src/language-js/print/array.js:122 (isConciselyPrintedArray) - prettier src/language-js/print/array.js:168 (printArrayElementsConcisely) - prettier src/document/printer/printer.js:343 (fill layout engine) - prettier src/utilities/print-number.js (rule 2b regex) Parity: JSON 108/108 (100%), overall 410/412 (99.5%)
P3: Fix root-level flow expansion indent — brackets at col 0, elements at
indentWidth. No spurious leading blank line.
P4: TOML blank lines inside arrays preserved — printArrayMultiline now emits
source blank lines between element groups (capped at allowed_blank_lines=2).
Leading blank lines in TOML files also preserved (matching taplo).
P2: YAML comment indentation loss — rewrote Pass 3 annotation to distinguish
end comments from leading comments using source indent comparison. End comments
inherit from previous context; leading comments from next.
P1: Flow mapping expansion — expandLongFlowSequences now handles both { } and
[ ] flow collections. Splits on comma at depth 0 for both bracket types.
Validation:
- Prettier YAML test suite: 155→165 pass (of 359)
- Taplo test suite: 2→4 pass (of 4 default-option tests)
- All 22 test packages pass, golangci-lint 0 issues
- All fixes verified idempotent
… fix Phase 3 (formatter fixes): - P5: Already working (no code change needed) - P6: Strip blank lines between mapping key colon and first child value - P7: Deferred to 3.1 (whack-a-mole pattern, zero real-world impact) Phase 4 (config resolution redesign): - Two-tier model: .cfv.toml exists → sole authority; no .cfv.toml → per-format tool config ownership (prettier for JSON/JSONC/YAML, yamlfmt overrides prettier for YAML if found, taplo for TOML). Editorconfig always base layer in Tier 2. - Removed flags: --no-prettier-config, --no-taplo-config, --no-yamlfmt-config - Changed: --no-config now disables ALL config (matches prettier semantics) - Kept: --no-editorconfig (industry standard exception) Phase 6 (formal rule specification tests): - 56 rule property tests across 8 formats (Test_<FORMAT>_<RuleName>) - 10 new golden fixtures filling coverage gaps - -update flag on all 9 formatter TestFixtures (go test -args -update) - 8 new fuzz targets (all formats now covered) Alignment fix: - formatDefaults() now delegates to each package's DefaultOptions() - Single source of truth — eliminates drift between cfv.go and formatter packages - Added DefaultOptions() to hclfmt, added LineEnding to jsonfmt.DefaultOptions() Bug fix: - trailingComma:es5 correctly maps to TrailingCommasAll (not Preserve) Parity: 261/263 (99.2%) — no regression Coverage: 92.1% (above 90% threshold) golangci-lint: 0 issues
- Rewrite README with new tagline and capability table - Update introduction, quick-start, formatting guide for unified cfv check - Remove --no-prettier-config, --no-taplo-config, --no-yamlfmt-config from all docs - Update --no-config to reflect new semantics (disables ALL config) - Add --fix and --unsafe to check subcommand flags reference - Document exit code 1 now covers formatting issues - Simplify CI/CD guide to single cfv check command - Update pre-commit hook to use cfv check --fix - Add new guide: Using cfv with Existing Tools - Add new reference: Formatting Differences - Remove redundant Supported Formats table from README
- Add format checking to cfv check (gated on !report.HasErrors()) - Add --fix support for formatting in check subcommand - Fix hujson.Standardize mutation with bytes.Clone in JSONC validator - Format all existing txtar fixture files for check format compatibility - Add 7 unit tests in pkg/cli/check_format_test.go - Add 3 integration tests (check_format, check_format_fix_all, check_format_reporters) - Add 6 config pipeline txtar tests - Update CHANGELOG Phase 5 implementation complete. All 22 packages pass, 0 lint issues, 92.1% coverage.
removeInsignificantWhitespace() was stripping ALL TokNewline and TokIndent tokens unconditionally, including ones inside text-only elements. This caused multi-line text like paragraphs to concatenate without separators: <description>Line one.Line two.</description> The fix introduces context-aware whitespace classification: - markContentWhitespace() classifies each whitespace token as structural (between tags, safe to remove) or content (adjacent to text, must keep) - removeInsignificantWhitespace() respects the classification - insertFormattingWhitespace() emits preserved content whitespace with correct reindentation and tracks multiline state for close tag placement Also fixes a pre-existing bug in preserve mode where annotate() assigned interior depth to indents before close tags, causing them to be indented one level too deep. Fixed with a second pass that decrements depth for indents preceding close tags. Validated against 807 godot XML files: zero data corruption, fully idempotent, zero text content differences between original and formatted. Closes: XML text-content collapse bug
When cfv borrows a tool's config for formatting settings (Tier 2, no .cfv.toml), it now also reads that tool's ignore/exclude mechanism: - .prettierignore: gitignore-style patterns, skips JSON/JSONC/YAML files - taplo.toml exclude: glob array, skips TOML files - .yamlfmt exclude + .yamlfmtignore: both mechanisms, skips YAML files Ignore files only affect format checking — not syntax or schema validation. A file in .prettierignore still gets its syntax validated; it just doesn't get a ~ for formatting. Design follows the two-tier model: - Tier 1 (.cfv.toml exists): external ignores NOT read - Tier 2 (no .cfv.toml): ignores read alongside tool configs - --no-config: all config+ignores disabled Validated on next.js: 212 fewer false ~ marks (739 → 527) with .prettierignore respected. Syntax error count unchanged. New files: - pkg/formatter/formatignore.go: FormatIgnores type, pattern matching - pkg/formatter/formatignore_test.go: 16 unit tests - cmd/cfv/testdata/check_format_ignore_prettier.txtar: integration test - cmd/cfv/testdata/check_format_ignore_taplo.txtar: integration test
- CHANGELOG: XML text-content fix (Fixed), format-ignore support (Added) - existing-tools.md: new 'Ignore files' section documenting .prettierignore, taplo exclude, and .yamlfmtignore behavior
… formatters
Bug 1: YAML flow bracket indent destroyed on re-format
- addFlowMappingPadding stripped indentation after newline when removing
bracketSpacing. Now checks for preceding newline before stripping.
Bug 2: YAML comment indent oscillation (2 defects)
- Defect 1: replaced source-indent classification with AST-depth-based
classification. AST depth is stable across passes; source indent is not.
- Defect 2: added lineHasDash guard to prevent AtSeqItem misassignment
when prev is a key inside a sequence, not a dash line.
Bug 3: YAML block scalar indent oscillation
- Extra spaces between colon and tag/block-scalar caused mis-tokenization.
Now advances past extra spaces so content characters dispatch correctly.
Bug 4: XML comment in text content blank line growth
- insertFormattingWhitespace unconditionally added newline before comments,
even when content whitespace already positioned them. Now checks if
previous token is TokIndent before inserting.
Bug 5: TOML comment alignment oscillation (array collapse)
- entryHasMultilineValue and entryKeyValueWidth now use estimateSingleLineArray
for arrays, giving stable width predictions across passes.
Bug 6: XML self-closing tag indent decrement
- applySelfClosingSpace detected indentation before /> (newline + whitespace)
and skips modification to preserve multi-line tag formatting.
Bug 7: TOML comment alignment after inline table expansion
- Extended Bug 5 fix to handle BraceOpen (inline tables). Both
entryHasMultilineValue and entryKeyValueWidth now use
estimateInlineTableWidth for inline tables.
Bug 8: XML self-closing multi-space stripped one-at-a-time
- applySelfClosingSpace now strips ALL spaces before /> in one pass using
a trimEnd loop, and ensures exactly one space when wantSpace=true.
Harness validation: 49/50 repos idempotent (was 22/50 before).
1 deferred: spring-framework XML inline comment in text content (Bug 9,
pathological test fixture, converges in 2 passes, not oscillating).
Pipeline: go vet, gofmt, golangci-lint (0 issues), all tests pass,
coverage 92.2%.
…/TOON (#618) Schema resolution for JSON, JSONC, TOML, and TOON is now external-only. The $schema property in documents is treated as data, not a cfv directive. Use --schema-map, .cfv.toml [schema-map], or --schemastore for schema validation. Breaking changes: - JSON/JSONC/TOML/TOON no longer implement SchemaValidator - MarshalToJSON no longer strips $schema from documents - --require-schema now fails these formats when no external schema source is configured (schema-map or schemastore) Retained (unchanged): - YAML # yaml-language-server comment schema detection - XML xsi:noNamespaceSchemaLocation - SARIF built-in schema by version field Additional fixes: - YAML MarshalToJSON now returns a clear error for .inf/.nan values instead of crashing json.Marshal - Removed dead code (buildJSONPositionMap) Pipeline: vet ✓, gofmt ✓, golangci-lint 0 issues, build ✓, all 22 packages pass, coverage 92.2%
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
| if raw[i] == 'u' && i+4 < len(raw) { | ||
| hex := string(raw[i+1 : i+5]) | ||
| if r, err := strconv.ParseUint(hex, 16, 32); err == nil { | ||
| b.WriteRune(rune(r)) //nolint:gosec // r is bounded to 32 bits by ParseUint |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
13 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #618 by removing the inline schema mechanism. This was never the right call for a validator to check a schema by validating a schema.