Repository navigation
feat(parser): implement short cluster and attached value parsing - #29
Merged
Merged
Conversation
- Token::text now stores the bare option name (without prefix) for option tokens; matched_prefix carries the prefix separately. This is cleaner since dispatch always stripped the prefix anyway, and it avoids any allocation when expanding clusters. - tokenize() recognises short-prefix tokens with body length > 1 that have no exact spec_map match and expands them as a cluster (-xvf → -x -v -f) or attached value (-ofile → -o file), using views into the original arg string. - Tests updated for the new Token::text semantics; cluster/attached-value tests added to test_tokenize.cc and test_parse.cc. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Introduces conversion::negatable<Prefix> which converts a string token
to std::pair<std::string, bool> by stripping a compile-time negation
prefix: "lto" -> {"lto", true}, "no-lto" -> {"lto", false}.
Exposes pack::insert_or_assign (was implemented but commented out) as
a terminal action that stores pairs into std::map<K, V>.
Adds examples/compiler.cc demonstrating -Werror/-Wno-unused style
warning flags and --feature=lto/--feature=no-pgo style feature toggles
using short-cluster parsing combined with the new pipeline actions.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Documents conversion::negatable and pack::insert_or_assign with an interactive playground showing -W/-Wno- style warning flags and --feature=<name>/--feature=no-<name> feature toggles. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add sidebar_position frontmatter to each example page so order is preserved. sidebars.ts no longer needs manual updates when new example pages are added. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Contributor
There was a problem hiding this comment.
Pull request overview
Implements GNU-style short option cluster (-xvf) and attached-value (-ofile, -O2) parsing in the tokenizer/parser, and updates the public token semantics to separate the matched prefix from the bare option name. The PR also expands the action pipeline with a negatable conversion + insert_or_assign pack action, and adds a new “compiler wrapper” example with docs sidebar reorganization.
Changes:
- Change
Token::textsemantics for option tokens (bare name only) and propagate through dispatch + tests. - Extend
tokenize()to expand short clusters and parse attached values without copying argument substrings. - Add
conversion::negatableand enablepack::insert_or_assign, plus a new compiler-wrapper example and docs sidebar autogeneration.
Reviewed changes
Copilot reviewed 16 out of 16 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| TODO.md | Marks short cluster / attached value work item as completed. |
| tests/test_tokenize.cc | Updates token expectations for new Token::text semantics; adds cluster/attached-value tokenizer tests. |
| tests/test_parse.cc | Adds end-to-end parse tests for clusters and attached values. |
| include/cli/parser.hh | Core implementation: new token semantics + cluster/attached-value tokenization + dispatch update. |
| include/cli/action.hh | Adds conversion::negatable, fixes InsertOrAssign storage typing, and enables pack::insert_or_assign. |
| examples/compiler.cc | New example demonstrating clusters, attached values, negatable key/value options, and map storage. |
| docs/sidebars.ts | Switches examples sidebar to autogenerated structure. |
| docs/docs/examples/*.mdx | Adds sidebar ordering metadata and documents the new compiler-wrapper example. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Fix tokenizer attached-value branch to greedily consume additional tokens when nargs.max > 1 (e.g. -ofile1 file2 with nargs=2) - Add tokenize tests: ValueOptionBodyIsValue, AttachedValueNargs2MissingSecond, GreedyStopsBeforeNextKnownOption, SingleDashIsPositional, NegativeNumberConsumedAsOptionValue, ShortOptionInlineSeparator, ShortOptionInlineEmptyValue - Add parse tests: ClusterNargs2 attached and flags-then-values cases Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…ster loop Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Comment on lines
+264
to
+270
| // Consume additional tokens for nargs > 1. | ||
| int count = 1; | ||
| while (i < args.size()) { | ||
| const std::string_view next = args[i]; | ||
| if (next == cfg.end_of_options_separator) break; | ||
| if (command_names.contains(std::string(next))) break; | ||
| if (nargs.max != -1 && count >= nargs.max) break; |
…egatable
- Add NegatableResult{name, enabled} struct to cli::conversion
- Add tuple protocol (tuple_size, tuple_element, get<>) at end of file
so pack::insert_or_assign continues to work via PairLike
- Add conversion tests: no-prefix, prefixed, prefix-only, mid-string
- Fix docs sidebar: move link to _category_.json to avoid duplicate
examples/index entry from autogenerated items
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Token::textnow stores the bare option name (without prefix) for option tokens;matched_prefixcarries the prefix. Dispatch already stripped the prefix, so this is cleaner and avoids allocation during cluster expansion.tokenize()recognises short-prefix tokens with body > 1 char and expands them as a cluster (-xvf→-x -v -f) or attached value (-ofile→-o file), using views into the original arg string — no heap allocation needed.Token::textsemantics; 7 new tokenizer tests and 4 new end-to-end parse tests added.TODO.md: mark short cluster / attached value as done.🤖 Generated with Claude Code