From 9c564f6a0e34b3a8db57b57ebd13e2a99eef5f69 Mon Sep 17 00:00:00 2001 From: alexey1312 Date: Sat, 9 May 2026 12:31:28 +0500 Subject: [PATCH 1/5] feat(cli): make rate-limit, max-retries, concurrent-downloads and batch settings configurable via PKL MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Adds 6 PKL fields so CI workflows pin network/orchestration knobs in exfig.pkl instead of repeating CLI flags everywhere. Pure-additive — configs that don't set the new fields keep v3.3.0 behavior. PKL schema: - figma.rateLimit / maxRetries / concurrentDownloads - new Batch.pkl module with batch.parallel / failFast / resume - PklProject 3.3.0 -> 3.4.0 Precedence per knob: CLI flag > PKL config > built-in default. Boolean flags (--fail-fast, --resume) use OR semantics (CLI || config). exfig batch reads `batch:` and `figma.*` rate-limiting fields ONLY from the first config in argv; per-target `batch:` blocks are debug-logged and ignored. fetch is config-free. colors/typography ignore figma.concurrentDownloads with a debug log under -v. Help text on all 7 flags unified to "(overrides config, default: N)". Tests: 12 new precedence cases in FaultToleranceOptionsTests, 7 cases in new BatchSettingsResolverTests. Full suite: 1954 passed / 0 failed. --- .claude/rules/fault-tolerance.md | 56 ++++- .../Batch/BatchSettingsResolver.swift | 108 +++++++++ .../ExFigCLI/ExFig.docc/BatchProcessing.md | 15 +- Sources/ExFigCLI/ExFig.docc/Configuration.md | 39 +++ Sources/ExFigCLI/ExFig.docc/PKLGuide.md | 8 +- Sources/ExFigCLI/ExFig.docc/Usage.md | 22 +- .../Input/FaultToleranceOptions.swift | 133 +++++++--- Sources/ExFigCLI/Resources/Schemas/Batch.pkl | 20 ++ Sources/ExFigCLI/Resources/Schemas/ExFig.pkl | 8 +- Sources/ExFigCLI/Resources/Schemas/Figma.pkl | 16 ++ Sources/ExFigCLI/Resources/Schemas/PklProject | 2 +- .../ExFigCLI/Resources/androidConfig.swift | 16 ++ .../ExFigCLI/Resources/flutterConfig.swift | 16 ++ Sources/ExFigCLI/Resources/iOSConfig.swift | 16 ++ Sources/ExFigCLI/Resources/webConfig.swift | 16 ++ Sources/ExFigCLI/Subcommands/Batch.swift | 113 ++++++--- Sources/ExFigCLI/Subcommands/Download.swift | 13 +- .../ExFigCLI/Subcommands/DownloadIcons.swift | 6 +- .../ExFigCLI/Subcommands/DownloadImages.swift | 20 +- .../Subcommands/DownloadImagesExport.swift | 6 +- .../ExFigCLI/Subcommands/DownloadTokens.swift | 6 +- .../Subcommands/DownloadTypography.swift | 11 +- .../ExFigCLI/Subcommands/ExportColors.swift | 10 +- .../ExFigCLI/Subcommands/ExportIcons.swift | 5 +- .../ExFigCLI/Subcommands/ExportImages.swift | 5 +- .../Subcommands/ExportTypography.swift | 10 +- Sources/ExFigCLI/Subcommands/Lint.swift | 5 +- Sources/ExFigConfig/Generated/Batch.pkl.swift | 58 +++++ Sources/ExFigConfig/Generated/ExFig.pkl.swift | 13 +- Sources/ExFigConfig/Generated/Figma.pkl.swift | 24 +- Sources/ExFigConfig/PKL/PKLEvaluator.swift | 3 + .../Batch/BatchSettingsResolverTests.swift | 229 ++++++++++++++++++ .../Input/FaultToleranceOptionsTests.swift | 106 +++++++- .../Input/PlatformConfigTests.swift | 10 +- Tests/ExFigTests/PKL/PKLEvaluatorTests.swift | 5 +- llms-full.txt | 61 ++++- 36 files changed, 1075 insertions(+), 135 deletions(-) create mode 100644 Sources/ExFigCLI/Batch/BatchSettingsResolver.swift create mode 100644 Sources/ExFigCLI/Resources/Schemas/Batch.pkl create mode 100644 Sources/ExFigConfig/Generated/Batch.pkl.swift create mode 100644 Tests/ExFigTests/Batch/BatchSettingsResolverTests.swift diff --git a/.claude/rules/fault-tolerance.md b/.claude/rules/fault-tolerance.md index ebbd27d2..f969caf9 100644 --- a/.claude/rules/fault-tolerance.md +++ b/.claude/rules/fault-tolerance.md @@ -27,7 +27,21 @@ exfig batch ./configs/ --timeout 60 --rate-limit 20 exfig fetch -f FILE_ID -r "Frame" -o ./out --timeout 45 --fail-fast ``` -**Timeout precedence:** CLI `--timeout` > PKL `figma.timeout` > FigmaClient default (30s) +**Precedence (per knob):** CLI flag > PKL `figma.*` > built-in default. Same rule applies to: +`--timeout` / `figma.timeout`, `--rate-limit` / `figma.rateLimit`, `--max-retries` / `figma.maxRetries`, +`--concurrent-downloads` / `figma.concurrentDownloads`. Boolean flags (`--fail-fast`, `--resume`) and +batch settings (`--parallel`, `batch.parallel`/`failFast`/`resume`) follow OR semantics for booleans +and standard precedence for `parallel`. + +`fetch` is config-free — it does not read `figma.*` PKL fields; only CLI flags and built-in defaults apply. + +`colors` and `typography` make no CDN downloads, so `figma.concurrentDownloads` is silently ignored +(under `-v` a debug log records the skip). + +`exfig batch` reads `batch:` and `figma.*` rate-limiting fields ONLY from the FIRST config in argv — +per-target `batch:` blocks in subsequent configs are ignored (logged under `-v`). The shared rate +limiter and download queue mean per-config `figma.rateLimit/maxRetries/concurrentDownloads` are +intentionally unused inside the batch run. ## Implementing Fault Tolerance in New Commands @@ -64,15 +78,37 @@ let data = try await client.request(endpoint) ## Defaults -| Setting | Default | Description | -| ----------------- | ------- | ------------------------------------- | -| `maxRetries` | 4 | Number of retry attempts | -| `rateLimit` | 10 | Requests per minute | -| `timeout` | 30s | Request timeout | -| `failFast` | false | Stop on first error | -| `resume` | false | Resume from checkpoint | -| `checkpointExpiry`| 24h | Checkpoint file expiration | -| `concurrentDownloads` | 20 | Parallel CDN downloads | +| Setting | Default | PKL key | Description | +| ----------------- | ------- | -------------------------------- | ------------------------------------- | +| `maxRetries` | 4 | `figma.maxRetries` | Number of retry attempts | +| `rateLimit` | 10 | `figma.rateLimit` | Requests per minute | +| `timeout` | 30s | `figma.timeout` | Request timeout | +| `failFast` | false | `batch.failFast` (batch only) | Stop on first error | +| `resume` | false | `batch.resume` (batch only) | Resume from checkpoint | +| `checkpointExpiry`| 24h | (not configurable) | Checkpoint file expiration | +| `concurrentDownloads` | 20 | `figma.concurrentDownloads` | Parallel CDN downloads | +| `parallel` | 3 | `batch.parallel` (batch only) | Concurrent batch configs | + +## PKL Config Alternative + +Instead of repeating CLI flags across CI workflows, set the values in `exfig.pkl`: + +```pkl +figma = new Figma.FigmaConfig { + lightFileId = "..." + rateLimit = 25 // was --rate-limit 25 + maxRetries = 6 // was --max-retries 6 + concurrentDownloads = 50 // was --concurrent-downloads 50 + timeout = 60 // was --timeout 60 +} + +batch = new Batch.BatchConfig { + parallel = 8 // was exfig batch --parallel 8 + failFast = true // was exfig batch --fail-fast +} +``` + +CLI flags still override these values per-run. ## Retry Behavior diff --git a/Sources/ExFigCLI/Batch/BatchSettingsResolver.swift b/Sources/ExFigCLI/Batch/BatchSettingsResolver.swift new file mode 100644 index 00000000..6fd525fb --- /dev/null +++ b/Sources/ExFigCLI/Batch/BatchSettingsResolver.swift @@ -0,0 +1,108 @@ +import ExFigConfig +import Foundation + +/// Resolved batch settings after merging CLI flags with the first config's `batch:` block +/// and `figma:` rate-limiting fields. +/// +/// Precedence (per knob): +/// - CLI flag (`--parallel`, `--rate-limit`, etc.) > config value > built-in default. +/// - For `failFast`/`resume` (presence flags), CLI || config (either source enables it). +struct ResolvedBatchSettings { + let parallel: Int + let failFast: Bool + let resume: Bool + let rateLimit: Int + let maxRetries: Int + let concurrentDownloads: Int + let timeout: Int? +} + +/// Loads the FIRST config in `exfig batch` argv and merges its `batch:` and `figma:` rate-limiting +/// fields with CLI flags. Per-target `batch:` blocks in subsequent configs are ignored — under +/// `--verbose` they emit a debug log line. +enum BatchSettingsResolver { + /// Built-in defaults — kept in lockstep with `FaultToleranceOptions` and `Batch` field defaults. + private enum Defaults { + static let parallel = 3 + static let rateLimit = 10 + static let maxRetries = 4 + } + + // swiftlint:disable function_parameter_count + + /// - Parameters: + /// - cliParallel: `--parallel` value, or nil if user didn't pass. + /// - cliFailFast: `--fail-fast` flag (false = not set). + /// - cliResume: `--resume` flag (false = not set). + /// - cliRateLimit: `--rate-limit` value, or nil if user didn't pass. + /// - cliMaxRetries: `--max-retries` value, or nil if user didn't pass. + /// - cliConcurrentDownloads: `--concurrent-downloads` value, or nil if user didn't pass. + /// - cliTimeout: `--timeout` value (seconds), or nil if user didn't pass. + /// - allConfigs: Discovered config URLs in argv order. First wins for batch settings. + /// - verbose: When true, emit debug log for ignored per-target `batch:` blocks. + /// - ui: Terminal UI for debug/warn output. + /// - Returns: Resolved settings to drive the batch run. + static func resolve( + cliParallel: Int?, + cliFailFast: Bool, + cliResume: Bool, + cliRateLimit: Int?, + cliMaxRetries: Int?, + cliConcurrentDownloads: Int?, + cliTimeout: Int?, + allConfigs: [URL], + verbose: Bool, + ui: TerminalUI + ) async -> ResolvedBatchSettings { + let firstConfig: ExFig.ModuleImpl? = await loadFirstConfig(allConfigs: allConfigs, ui: ui) + let batch = firstConfig?.batch + let figma = firstConfig?.figma + + if verbose, allConfigs.count > 1 { + await logIgnoredPerTargetBatchBlocks(otherConfigs: Array(allConfigs.dropFirst()), ui: ui) + } + + return ResolvedBatchSettings( + parallel: cliParallel ?? batch?.parallel ?? Defaults.parallel, + failFast: cliFailFast || (batch?.failFast ?? false), + resume: cliResume || (batch?.resume ?? false), + rateLimit: cliRateLimit ?? figma?.rateLimit ?? Defaults.rateLimit, + maxRetries: cliMaxRetries ?? figma?.maxRetries ?? Defaults.maxRetries, + concurrentDownloads: cliConcurrentDownloads + ?? figma?.concurrentDownloads + ?? FileDownloader.defaultMaxConcurrentDownloads, + timeout: cliTimeout ?? figma?.timeout.map { Int($0) } + ) + } + + // swiftlint:enable function_parameter_count + + // MARK: - Internals + + private static func loadFirstConfig(allConfigs: [URL], ui: TerminalUI) async -> ExFig.ModuleImpl? { + guard let firstURL = allConfigs.first else { return nil } + do { + return try await PKLEvaluator.evaluate(configPath: firstURL) + } catch { + // Falling back to defaults is safe — the config will be re-evaluated by BatchConfigRunner + // and any real syntax/validation error will surface there with a per-config error. + ui.debug( + "Could not pre-load batch settings from \(firstURL.lastPathComponent): " + + "\(error.localizedDescription). Using CLI flags / built-in defaults." + ) + return nil + } + } + + private static func logIgnoredPerTargetBatchBlocks(otherConfigs: [URL], ui: TerminalUI) async { + for url in otherConfigs { + let module = try? await PKLEvaluator.evaluate(configPath: url) + if module?.batch != nil { + ui.debug( + "Ignoring batch: in \(url.lastPathComponent) — only the first config's " + + "batch settings apply." + ) + } + } + } +} diff --git a/Sources/ExFigCLI/ExFig.docc/BatchProcessing.md b/Sources/ExFigCLI/ExFig.docc/BatchProcessing.md index 4014ed18..2c089740 100644 --- a/Sources/ExFigCLI/ExFig.docc/BatchProcessing.md +++ b/Sources/ExFigCLI/ExFig.docc/BatchProcessing.md @@ -53,11 +53,16 @@ exfig batch ./configs/ --parallel 5 Batch mode uses shared rate limiting across all concurrent configs. This prevents hitting Figma API limits when processing multiple files simultaneously. -| Option | Description | Default | -| --------------- | ------------------------------- | ------- | -| `--parallel` | Maximum concurrent configs | 3 | -| `--rate-limit` | Figma API requests per minute | 10 | -| `--max-retries` | Maximum retry attempts | 4 | +| Option | Description | Default | PKL key (first config) | +| --------------- | ------------------------------- | ------- | ----------------------- | +| `--parallel` | Maximum concurrent configs | 3 | `batch.parallel` | +| `--rate-limit` | Figma API requests per minute | 10 | `figma.rateLimit` | +| `--max-retries` | Maximum retry attempts | 4 | `figma.maxRetries` | + +`exfig batch` reads `batch:` and `figma.*` rate-limiting fields ONLY from the FIRST config in argv. +Per-target `batch:` blocks in subsequent configs are ignored (logged under `-v`). The rate limiter +and download queue are shared across all configs, so per-config `figma.rateLimit/maxRetries/ +concurrentDownloads` are intentionally unused inside the batch run. ## Error Handling diff --git a/Sources/ExFigCLI/ExFig.docc/Configuration.md b/Sources/ExFigCLI/ExFig.docc/Configuration.md index 9dd5f1d2..3edd20f5 100644 --- a/Sources/ExFigCLI/ExFig.docc/Configuration.md +++ b/Sources/ExFigCLI/ExFig.docc/Configuration.md @@ -53,9 +53,24 @@ figma = new Figma.FigmaConfig { // Optional: API request timeout in seconds (default: 30) timeout = 60 + + // Optional: API requests per minute (default: 10). CLI --rate-limit overrides. + rateLimit = 25 + + // Optional: Retry attempts for failed API requests (default: 4). CLI --max-retries overrides. + maxRetries = 6 + + // Optional: Concurrent CDN downloads — icons/images only (default: 20). + // Ignored by colors/typography. CLI --concurrent-downloads overrides. + // Client-side cap on connections (URLSession.httpMaximumConnectionsPerHost), not a Figma REST limit. + concurrentDownloads = 50 } ``` +**Precedence (per knob):** CLI flag > PKL config > built-in default. The same rule applies to +all five fields above. CI workflows can keep one value here instead of repeating CLI flags +everywhere; ad-hoc invocations still override per-run. + ## Common Section Shared settings across all platforms. @@ -483,6 +498,30 @@ flutter = new Flutter.FlutterConfig { } ``` +## Batch Section + +Top-level `batch:` block (read by `exfig batch`). Read ONLY from the **first** config in argv — +per-target `batch:` blocks in subsequent configs are ignored (logged under `-v`). + +```pkl showLineNumbers +import ".exfig/schemas/Batch.pkl" + +batch = new Batch.BatchConfig { + // Optional: Maximum configs to process in parallel (default: 3). CLI --parallel overrides. + parallel = 8 + + // Optional: Stop processing on first error (default: false). CLI --fail-fast overrides. + failFast = true + + // Optional: Resume from previous checkpoint (default: false). CLI --resume overrides. + resume = false +} +``` + +For `exfig batch ./configs/`, `parallel`/`failFast`/`resume` come from the FIRST config (or CLI flags +if passed). Boolean fields use OR semantics: CLI `--fail-fast` activates fail-fast even if the config +has `failFast = false`. + ## Example Configurations ### iOS Project diff --git a/Sources/ExFigCLI/ExFig.docc/PKLGuide.md b/Sources/ExFigCLI/ExFig.docc/PKLGuide.md index 7efe4a62..7827b88c 100755 --- a/Sources/ExFigCLI/ExFig.docc/PKLGuide.md +++ b/Sources/ExFigCLI/ExFig.docc/PKLGuide.md @@ -511,10 +511,16 @@ figma = new Figma.FigmaConfig { darkFileId = "DEF456" // Dark mode file (optional) lightHighContrastFileId = "GHI789" darkHighContrastFileId = "JKL012" - timeout = 60 // Request timeout in seconds + timeout = 60 // Request timeout (seconds, default: 30) + rateLimit = 25 // API requests per minute (default: 10) + maxRetries = 6 // Retry attempts for failed requests (default: 4) + concurrentDownloads = 50 // Concurrent CDN downloads, icons/images only (default: 20) } ``` +CLI flags (`--rate-limit`, `--max-retries`, `--concurrent-downloads`, `--timeout`) override these +PKL values per-run. CI-pinned values live in PKL; ad-hoc overrides go through CLI. + ## Name Processing Control how Figma names are transformed: diff --git a/Sources/ExFigCLI/ExFig.docc/Usage.md b/Sources/ExFigCLI/ExFig.docc/Usage.md index b2070764..45d5aedf 100644 --- a/Sources/ExFigCLI/ExFig.docc/Usage.md +++ b/Sources/ExFigCLI/ExFig.docc/Usage.md @@ -140,13 +140,21 @@ exfig images --resume exfig icons --concurrent-downloads 50 ``` -| Option | Description | Commands | -| ------------------------ | -------------------------------------- | -------------------- | -| `--max-retries` | Maximum retry attempts (default: 4) | All | -| `--rate-limit` | API requests per minute (default: 10) | All | -| `--fail-fast` | Stop immediately on error | icons, images, fetch | -| `--resume` | Continue from checkpoint | icons, images, fetch | -| `--concurrent-downloads` | Concurrent CDN downloads (default: 20) | icons, images, fetch | +| Option | Description | Commands | PKL key | +| ------------------------ | ---------------------------------------------- | --------------------- | ----------------------------- | +| `--max-retries` | Maximum retry attempts (default: 4) | All | `figma.maxRetries` | +| `--rate-limit` | API requests per minute (default: 10) | All | `figma.rateLimit` | +| `--timeout` | Figma API request timeout, sec (default: 30) | All | `figma.timeout` | +| `--concurrent-downloads` | Concurrent CDN downloads (default: 20) | icons, images, fetch | `figma.concurrentDownloads`* | +| `--fail-fast` | Stop immediately on error | icons, images, batch, fetch | `batch.failFast` (batch)| +| `--resume` | Continue from checkpoint | icons, images, batch, fetch | `batch.resume` (batch) | +| `--parallel` | Concurrent batch configs (default: 3) | batch | `batch.parallel` | + +CLI flags override PKL config; PKL config overrides built-in defaults. `fetch` is config-free — +only CLI flags and built-in defaults apply there. + +*`figma.concurrentDownloads` is silently ignored by `colors`/`typography` (no CDN downloads); under +`-v` a debug log records the skip. ### Checkpoint System diff --git a/Sources/ExFigCLI/Input/FaultToleranceOptions.swift b/Sources/ExFigCLI/Input/FaultToleranceOptions.swift index 8055cce4..b1bc4697 100644 --- a/Sources/ExFigCLI/Input/FaultToleranceOptions.swift +++ b/Sources/ExFigCLI/Input/FaultToleranceOptions.swift @@ -1,3 +1,4 @@ +// swiftlint:disable file_length import ArgumentParser import ExFigCore import FigmaAPI @@ -10,19 +11,19 @@ import Foundation struct FaultToleranceOptions: ParsableArguments { @Option( name: .long, - help: "Maximum retry attempts for failed API requests" + help: "Maximum retry attempts for failed API requests (overrides config, default: 4)" ) - var maxRetries: Int = 4 + var maxRetries: Int? @Option( name: .long, - help: "Maximum API requests per minute" + help: "Maximum API requests per minute (overrides config, default: 10)" ) - var rateLimit: Int = 10 + var rateLimit: Int? @Option( name: .long, - help: "Figma API request timeout in seconds (overrides config)" + help: "Figma API request timeout in seconds (overrides config, default: 30)" ) var timeout: Int? @@ -30,30 +31,50 @@ struct FaultToleranceOptions: ParsableArguments { if let timeout, timeout <= 0 { throw ValidationError("Timeout must be positive") } + if let rateLimit, rateLimit <= 0 { + throw ValidationError("Rate limit must be positive") + } + if let maxRetries, maxRetries < 0 { + throw ValidationError("Max retries cannot be negative") + } + } + + /// CLI value (if user passed --rate-limit) > config value > built-in default (10). + func effectiveRateLimit(configValue: Int?) -> Int { + rateLimit ?? configValue ?? 10 + } + + /// CLI value (if user passed --max-retries) > config value > built-in default (4). + func effectiveMaxRetries(configValue: Int?) -> Int { + maxRetries ?? configValue ?? 4 } /// Create a retry policy from the options. + /// - Parameter configValue: Config-supplied `figma.maxRetries`, or nil. /// - Returns: A configured `RetryPolicy`. - func createRetryPolicy() -> RetryPolicy { - RetryPolicy(maxRetries: maxRetries) + func createRetryPolicy(configValue: Int? = nil) -> RetryPolicy { + RetryPolicy(maxRetries: effectiveMaxRetries(configValue: configValue)) } /// Create a shared rate limiter from the options. + /// - Parameter configValue: Config-supplied `figma.rateLimit`, or nil. /// - Returns: A configured `SharedRateLimiter`. - func createRateLimiter() -> SharedRateLimiter { - SharedRateLimiter(requestsPerMinute: Double(rateLimit)) + func createRateLimiter(configValue: Int? = nil) -> SharedRateLimiter { + SharedRateLimiter(requestsPerMinute: Double(effectiveRateLimit(configValue: configValue))) } /// Create a rate-limited client wrapping the given client. /// - Parameters: /// - client: The underlying client to wrap. /// - rateLimiter: The shared rate limiter to use. + /// - configMaxRetries: Config-supplied `figma.maxRetries`, or nil. /// - configID: Identifier for this client's config (default: "default"). /// - onRetry: Optional callback invoked before each retry attempt. /// - Returns: A `RateLimitedClient` instance. func createRateLimitedClient( wrapping client: Client, rateLimiter: SharedRateLimiter, + configMaxRetries: Int? = nil, configID: ConfigID = ConfigID("default"), onRetry: RetryCallback? = nil ) -> Client { @@ -61,7 +82,7 @@ struct FaultToleranceOptions: ParsableArguments { client: client, rateLimiter: rateLimiter, configID: configID, - retryPolicy: createRetryPolicy(), + retryPolicy: createRetryPolicy(configValue: configMaxRetries), onRetry: onRetry ) } @@ -78,77 +99,107 @@ struct FaultToleranceOptions: ParsableArguments { struct HeavyFaultToleranceOptions: ParsableArguments { @Option( name: .long, - help: "Maximum retry attempts for failed API requests" + help: "Maximum retry attempts for failed API requests (overrides config, default: 4)" ) - var maxRetries: Int = 4 + var maxRetries: Int? @Option( name: .long, - help: "Maximum API requests per minute" + help: "Maximum API requests per minute (overrides config, default: 10)" ) - var rateLimit: Int = 10 + var rateLimit: Int? @Option( name: .long, - help: "Figma API request timeout in seconds (overrides config)" + help: "Figma API request timeout in seconds (overrides config, default: 30)" ) var timeout: Int? @Flag( name: .long, - help: "Stop on first error without retrying" + help: "Stop on first error without retrying (overrides config)" ) var failFast: Bool = false @Flag( name: .long, - help: "Continue from checkpoint after interruption" + help: "Continue from checkpoint after interruption (overrides config)" ) var resume: Bool = false @Option( name: .long, - help: "Maximum concurrent CDN downloads" + help: "Maximum concurrent CDN downloads (overrides config, default: 20)" ) - var concurrentDownloads: Int = FileDownloader.defaultMaxConcurrentDownloads + var concurrentDownloads: Int? mutating func validate() throws { if let timeout, timeout <= 0 { throw ValidationError("Timeout must be positive") } + if let rateLimit, rateLimit <= 0 { + throw ValidationError("Rate limit must be positive") + } + if let maxRetries, maxRetries < 0 { + throw ValidationError("Max retries cannot be negative") + } + if let concurrentDownloads, concurrentDownloads <= 0 { + throw ValidationError("Concurrent downloads must be positive") + } + } + + /// CLI value (if user passed --rate-limit) > config value > built-in default (10). + func effectiveRateLimit(configValue: Int?) -> Int { + rateLimit ?? configValue ?? 10 + } + + /// CLI value (if user passed --max-retries) > config value > built-in default (4). + func effectiveMaxRetries(configValue: Int?) -> Int { + maxRetries ?? configValue ?? 4 + } + + /// CLI value (if user passed --concurrent-downloads) > config value > built-in default (20). + func effectiveConcurrentDownloads(configValue: Int?) -> Int { + concurrentDownloads ?? configValue ?? FileDownloader.defaultMaxConcurrentDownloads } /// Create a file downloader with configured concurrency. + /// - Parameter configValue: Config-supplied `figma.concurrentDownloads`, or nil. /// - Returns: A configured `FileDownloader`. - func createFileDownloader() -> FileDownloader { - FileDownloader(maxConcurrentDownloads: concurrentDownloads) + func createFileDownloader(configValue: Int? = nil) -> FileDownloader { + FileDownloader(maxConcurrentDownloads: effectiveConcurrentDownloads(configValue: configValue)) } /// Create a retry policy from the options. + /// `--fail-fast` (CLI flag) forces 0 retries regardless of config. + /// - Parameter configValue: Config-supplied `figma.maxRetries`, or nil. /// - Returns: A configured `RetryPolicy`. - func createRetryPolicy() -> RetryPolicy { + func createRetryPolicy(configValue: Int? = nil) -> RetryPolicy { if failFast { return RetryPolicy(maxRetries: 0) } - return RetryPolicy(maxRetries: maxRetries) + return RetryPolicy(maxRetries: effectiveMaxRetries(configValue: configValue)) } /// Create a shared rate limiter from the options. + /// - Parameter configValue: Config-supplied `figma.rateLimit`, or nil. /// - Returns: A configured `SharedRateLimiter`. - func createRateLimiter() -> SharedRateLimiter { - SharedRateLimiter(requestsPerMinute: Double(rateLimit)) + func createRateLimiter(configValue: Int? = nil) -> SharedRateLimiter { + SharedRateLimiter(requestsPerMinute: Double(effectiveRateLimit(configValue: configValue))) } /// Create a rate-limited client wrapping the given client. /// - Parameters: /// - client: The underlying client to wrap. /// - rateLimiter: The shared rate limiter to use. + /// - configMaxRetries: Config-supplied `figma.maxRetries`, or nil. /// - configID: Identifier for this client's config (default: "default"). /// - onRetry: Optional callback invoked before each retry attempt. /// - Returns: A `RateLimitedClient` instance. func createRateLimitedClient( wrapping client: Client, rateLimiter: SharedRateLimiter, + configMaxRetries: Int? = nil, configID: ConfigID = ConfigID("default"), onRetry: RetryCallback? = nil ) -> Client { @@ -156,7 +207,7 @@ struct HeavyFaultToleranceOptions: ParsableArguments { client: client, rateLimiter: rateLimiter, configID: configID, - retryPolicy: createRetryPolicy(), + retryPolicy: createRetryPolicy(configValue: configMaxRetries), onRetry: onRetry ) } @@ -247,14 +298,18 @@ struct HeavyFaultToleranceOptions: ParsableArguments { /// - Parameters: /// - accessToken: Figma personal access token. /// - timeout: Request timeout interval from config (optional, uses FigmaClient default if nil). -/// - options: Fault tolerance options for creating new client (may contain CLI timeout override). +/// - rateLimit: Config-supplied `figma.rateLimit` (optional). CLI `--rate-limit` overrides. +/// - maxRetries: Config-supplied `figma.maxRetries` (optional). CLI `--max-retries` overrides. +/// - options: Fault tolerance options for creating new client (may contain CLI overrides). /// - ui: Terminal UI for retry warnings. /// - Returns: A configured `Client` instance. /// -/// Timeout precedence: CLI `--timeout` > config > FigmaClient default (30s) +/// Precedence (per knob): CLI flag > config value > built-in default. func resolveClient( accessToken: String?, timeout: TimeInterval?, + rateLimit configRateLimit: Int? = nil, + maxRetries configMaxRetries: Int? = nil, options: FaultToleranceOptions, ui: TerminalUI ) -> Client { @@ -270,15 +325,16 @@ func resolveClient( // CLI timeout takes precedence over config timeout let effectiveTimeout: TimeInterval? = options.timeout.map { TimeInterval($0) } ?? timeout let baseClient = FigmaClient(accessToken: accessToken, timeout: effectiveTimeout) - let rateLimiter = options.createRateLimiter() - let maxRetries = options.maxRetries + let rateLimiter = options.createRateLimiter(configValue: configRateLimit) + let effectiveMaxRetries = options.effectiveMaxRetries(configValue: configMaxRetries) return options.createRateLimitedClient( wrapping: baseClient, rateLimiter: rateLimiter, + configMaxRetries: configMaxRetries, onRetry: { attempt, error in let warning = ExFigWarning.retrying( attempt: attempt, - maxAttempts: maxRetries, + maxAttempts: effectiveMaxRetries, error: error.localizedDescription, delay: "..." ) @@ -297,14 +353,18 @@ func resolveClient( /// - Parameters: /// - accessToken: Figma personal access token (nil when using non-Figma sources only). /// - timeout: Request timeout interval from config (optional, uses FigmaClient default if nil). -/// - options: Heavy fault tolerance options for creating new client (may contain CLI timeout override). +/// - rateLimit: Config-supplied `figma.rateLimit` (optional). CLI `--rate-limit` overrides. +/// - maxRetries: Config-supplied `figma.maxRetries` (optional). CLI `--max-retries` overrides. +/// - options: Heavy fault tolerance options for creating new client (may contain CLI overrides). /// - ui: Terminal UI for retry warnings. /// - Returns: A configured `Client` instance. /// -/// Timeout precedence: CLI `--timeout` > config > FigmaClient default (30s) +/// Precedence (per knob): CLI flag > config value > built-in default. func resolveClient( accessToken: String?, timeout: TimeInterval?, + rateLimit configRateLimit: Int? = nil, + maxRetries configMaxRetries: Int? = nil, options: HeavyFaultToleranceOptions, ui: TerminalUI ) -> Client { @@ -319,15 +379,16 @@ func resolveClient( // CLI timeout takes precedence over config timeout let effectiveTimeout: TimeInterval? = options.timeout.map { TimeInterval($0) } ?? timeout let baseClient = FigmaClient(accessToken: accessToken, timeout: effectiveTimeout) - let rateLimiter = options.createRateLimiter() - let maxRetries = options.maxRetries + let rateLimiter = options.createRateLimiter(configValue: configRateLimit) + let effectiveMaxRetries = options.effectiveMaxRetries(configValue: configMaxRetries) return options.createRateLimitedClient( wrapping: baseClient, rateLimiter: rateLimiter, + configMaxRetries: configMaxRetries, onRetry: { attempt, error in let warning = ExFigWarning.retrying( attempt: attempt, - maxAttempts: maxRetries, + maxAttempts: effectiveMaxRetries, error: error.localizedDescription, delay: "..." ) diff --git a/Sources/ExFigCLI/Resources/Schemas/Batch.pkl b/Sources/ExFigCLI/Resources/Schemas/Batch.pkl new file mode 100644 index 00000000..e5c4a69f --- /dev/null +++ b/Sources/ExFigCLI/Resources/Schemas/Batch.pkl @@ -0,0 +1,20 @@ +/// Batch orchestration configuration. +/// +/// These settings only apply when running `exfig batch` and are read +/// from the FIRST config in the argument list. Per-target `batch:` blocks +/// in subsequent configs are ignored (logged under -v). +module Batch + +import "pkl:base" + +/// Batch execution settings — only meaningful for `exfig batch`. +class BatchConfig { + /// Maximum configs to process in parallel. CLI --parallel overrides. + parallel: Int(isBetween(1, 50))? = 3 + + /// Stop processing on first error. CLI --fail-fast overrides. + failFast: Boolean? = false + + /// Resume from previous checkpoint if available. CLI --resume overrides. + resume: Boolean? = false +} diff --git a/Sources/ExFigCLI/Resources/Schemas/ExFig.pkl b/Sources/ExFigCLI/Resources/Schemas/ExFig.pkl index d846040d..67832393 100644 --- a/Sources/ExFigCLI/Resources/Schemas/ExFig.pkl +++ b/Sources/ExFigCLI/Resources/Schemas/ExFig.pkl @@ -5,7 +5,7 @@ /// /// Usage: /// ```pkl -/// amends "package://pkg.pkl-lang.org/github.com/DesignPipe/exfig/exfig@2.8.0#/ExFig.pkl" +/// amends "package://pkg.pkl-lang.org/github.com/DesignPipe/exfig/exfig@3.4.0#/ExFig.pkl" /// /// figma { /// lightFileId = "xxx" @@ -20,6 +20,7 @@ open module ExFig import "Figma.pkl" +import "Batch.pkl" import "Common.pkl" import "iOS.pkl" import "Android.pkl" @@ -45,3 +46,8 @@ flutter: Flutter.FlutterConfig? /// Web platform configuration. web: Web.WebConfig? + +/// Batch orchestration settings. +/// Only read from the FIRST config when running `exfig batch`. +/// Per-target `batch:` blocks in subsequent configs are ignored (logged under -v). +batch: Batch.BatchConfig? diff --git a/Sources/ExFigCLI/Resources/Schemas/Figma.pkl b/Sources/ExFigCLI/Resources/Schemas/Figma.pkl index 8d18d550..bb36ccf2 100644 --- a/Sources/ExFigCLI/Resources/Schemas/Figma.pkl +++ b/Sources/ExFigCLI/Resources/Schemas/Figma.pkl @@ -21,4 +21,20 @@ class FigmaConfig { /// Request timeout in seconds. timeout: Number(isBetween(1, 600))? = 30.0 + + /// Maximum API requests per minute. CLI --rate-limit overrides. + /// Tier 1 endpoints (GET file, GET images): 10–20/min on Dev/Full seats. + /// Tier 2 endpoints (Variables, Image fills): 25–100/min. + /// Tier 3 endpoints (Components, file metadata): 50–150/min. + /// Limit is per-token + per-plan, not per-IP. See https://developers.figma.com/docs/rest-api/rate-limits/ + rateLimit: Int(isBetween(1, 600))? = 10 + + /// Maximum retry attempts for failed API requests. CLI --max-retries overrides. + maxRetries: Int(isBetween(0, 100))? = 4 + + /// Maximum concurrent CDN downloads (icons/images only). + /// Ignored by colors/typography. CLI --concurrent-downloads overrides. + /// NOTE: client-side cap on CDN connections (URLSession.httpMaximumConnectionsPerHost), + /// not a Figma REST API limit. Use `rateLimit` for API throttling. + concurrentDownloads: Int(isBetween(1, 200))? = 20 } diff --git a/Sources/ExFigCLI/Resources/Schemas/PklProject b/Sources/ExFigCLI/Resources/Schemas/PklProject index f1e2bf69..c10c453a 100644 --- a/Sources/ExFigCLI/Resources/Schemas/PklProject +++ b/Sources/ExFigCLI/Resources/Schemas/PklProject @@ -3,7 +3,7 @@ amends "pkl:Project" package { name = "exfig" baseUri = "package://pkg.pkl-lang.org/github.com/DesignPipe/exfig/exfig" - version = "3.3.0" + version = "3.4.0" packageZipUrl = "https://github.com/DesignPipe/exfig/releases/download/v\(version)/\(name)@\(version).zip" description = "ExFig configuration schemas for exporting Figma assets to iOS, Android, Flutter, and Web" authors { diff --git a/Sources/ExFigCLI/Resources/androidConfig.swift b/Sources/ExFigCLI/Resources/androidConfig.swift index 1e773157..61f94ab0 100644 --- a/Sources/ExFigCLI/Resources/androidConfig.swift +++ b/Sources/ExFigCLI/Resources/androidConfig.swift @@ -3,6 +3,7 @@ let androidConfigFileContents = #""" amends ".exfig/schemas/ExFig.pkl" import ".exfig/schemas/Figma.pkl" +import ".exfig/schemas/Batch.pkl" import ".exfig/schemas/Common.pkl" import ".exfig/schemas/Android.pkl" @@ -20,6 +21,12 @@ figma = new Figma.FigmaConfig { // [optional] Figma API request timeout. The default value is 30 (seconds). // If you have a lot of resources to export set this value to 60 or more to give Figma API more time to prepare resources for exporting. // timeout = 30.0 + // [optional] Maximum API requests per minute. CLI --rate-limit overrides. Default: 10. + // rateLimit = 10 + // [optional] Maximum retry attempts for failed API requests. CLI --max-retries overrides. Default: 4. + // maxRetries = 4 + // [optional] Maximum concurrent CDN downloads (icons/images only). CLI --concurrent-downloads overrides. Default: 20. + // concurrentDownloads = 20 } // [optional] Common export parameters @@ -153,4 +160,13 @@ android = new Android.AndroidConfig { composePackageName = "com.example" } } + +// [optional] Batch orchestration settings. +// Read ONLY from the FIRST config when running `exfig batch`. +// Per-target `batch:` blocks in subsequent configs are ignored (logged under -v). +// batch = new Batch.BatchConfig { +// parallel = 3 // CLI --parallel overrides +// failFast = false // CLI --fail-fast overrides +// resume = false // CLI --resume overrides +// } """# diff --git a/Sources/ExFigCLI/Resources/flutterConfig.swift b/Sources/ExFigCLI/Resources/flutterConfig.swift index f86c455e..67c576a0 100644 --- a/Sources/ExFigCLI/Resources/flutterConfig.swift +++ b/Sources/ExFigCLI/Resources/flutterConfig.swift @@ -3,6 +3,7 @@ let flutterConfigFileContents = #""" amends ".exfig/schemas/ExFig.pkl" import ".exfig/schemas/Figma.pkl" +import ".exfig/schemas/Batch.pkl" import ".exfig/schemas/Common.pkl" import ".exfig/schemas/Flutter.pkl" @@ -16,6 +17,12 @@ figma = new Figma.FigmaConfig { // [optional] Figma API request timeout. The default value is 30 (seconds). // If you have a lot of resources to export set this value to 60 or more to give Figma API more time to prepare resources for exporting. // timeout = 30.0 + // [optional] Maximum API requests per minute. CLI --rate-limit overrides. Default: 10. + // rateLimit = 10 + // [optional] Maximum retry attempts for failed API requests. CLI --max-retries overrides. Default: 4. + // maxRetries = 4 + // [optional] Maximum concurrent CDN downloads (icons/images only). CLI --concurrent-downloads overrides. Default: 20. + // concurrentDownloads = 20 } // [optional] Common export parameters @@ -123,4 +130,13 @@ flutter = new Flutter.FlutterConfig { // } } } + +// [optional] Batch orchestration settings. +// Read ONLY from the FIRST config when running `exfig batch`. +// Per-target `batch:` blocks in subsequent configs are ignored (logged under -v). +// batch = new Batch.BatchConfig { +// parallel = 3 // CLI --parallel overrides +// failFast = false // CLI --fail-fast overrides +// resume = false // CLI --resume overrides +// } """# diff --git a/Sources/ExFigCLI/Resources/iOSConfig.swift b/Sources/ExFigCLI/Resources/iOSConfig.swift index bf76bf71..33a43bd0 100644 --- a/Sources/ExFigCLI/Resources/iOSConfig.swift +++ b/Sources/ExFigCLI/Resources/iOSConfig.swift @@ -3,6 +3,7 @@ let iosConfigFileContents = #""" amends ".exfig/schemas/ExFig.pkl" import ".exfig/schemas/Figma.pkl" +import ".exfig/schemas/Batch.pkl" import ".exfig/schemas/Common.pkl" import ".exfig/schemas/iOS.pkl" @@ -20,6 +21,12 @@ figma = new Figma.FigmaConfig { // [optional] Figma API request timeout. The default value is 30 (seconds). // If you have a lot of resources to export set this value to 60 or more to give Figma API more time to prepare resources for exporting. // timeout = 30.0 + // [optional] Maximum API requests per minute. CLI --rate-limit overrides. Default: 10. + // rateLimit = 10 + // [optional] Maximum retry attempts for failed API requests. CLI --max-retries overrides. Default: 4. + // maxRetries = 4 + // [optional] Maximum concurrent CDN downloads (icons/images only). CLI --concurrent-downloads overrides. Default: 20. + // concurrentDownloads = 20 } // [optional] Common export parameters @@ -194,4 +201,13 @@ ios = new iOS.iOSConfig { nameStyle = "camelCase" } } + +// [optional] Batch orchestration settings. +// Read ONLY from the FIRST config when running `exfig batch`. +// Per-target `batch:` blocks in subsequent configs are ignored (logged under -v). +// batch = new Batch.BatchConfig { +// parallel = 3 // CLI --parallel overrides +// failFast = false // CLI --fail-fast overrides +// resume = false // CLI --resume overrides +// } """# diff --git a/Sources/ExFigCLI/Resources/webConfig.swift b/Sources/ExFigCLI/Resources/webConfig.swift index 326950a1..db398fab 100644 --- a/Sources/ExFigCLI/Resources/webConfig.swift +++ b/Sources/ExFigCLI/Resources/webConfig.swift @@ -3,6 +3,7 @@ let webConfigFileContents = #""" amends ".exfig/schemas/ExFig.pkl" import ".exfig/schemas/Figma.pkl" +import ".exfig/schemas/Batch.pkl" import ".exfig/schemas/Common.pkl" import ".exfig/schemas/Web.pkl" @@ -17,6 +18,12 @@ figma = new Figma.FigmaConfig { // [optional] Figma API request timeout. The default value is 30 (seconds). // If you have a lot of resources to export set this value to 60 or more. // timeout = 30.0 + // [optional] Maximum API requests per minute. CLI --rate-limit overrides. Default: 10. + // rateLimit = 10 + // [optional] Maximum retry attempts for failed API requests. CLI --max-retries overrides. Default: 4. + // maxRetries = 4 + // [optional] Maximum concurrent CDN downloads (icons/images only). CLI --concurrent-downloads overrides. Default: 20. + // concurrentDownloads = 20 } // [optional] Common export parameters @@ -117,4 +124,13 @@ web = new Web.WebConfig { generateReactComponents = true } } + +// [optional] Batch orchestration settings. +// Read ONLY from the FIRST config when running `exfig batch`. +// Per-target `batch:` blocks in subsequent configs are ignored (logged under -v). +// batch = new Batch.BatchConfig { +// parallel = 3 // CLI --parallel overrides +// failFast = false // CLI --fail-fast overrides +// resume = false // CLI --resume overrides +// } """# diff --git a/Sources/ExFigCLI/Subcommands/Batch.swift b/Sources/ExFigCLI/Subcommands/Batch.swift index 1ad27fab..f0e98499 100644 --- a/Sources/ExFigCLI/Subcommands/Batch.swift +++ b/Sources/ExFigCLI/Subcommands/Batch.swift @@ -30,19 +30,19 @@ extension ExFigCommand { @OptionGroup var globalOptions: GlobalOptions - @Option(name: .long, help: "Maximum configs to process in parallel") - var parallel: Int = 3 + @Option(name: .long, help: "Maximum configs to process in parallel (overrides config, default: 3)") + var parallel: Int? - @Flag(name: .long, help: "Stop processing on first error") + @Flag(name: .long, help: "Stop processing on first error (overrides config)") var failFast: Bool = false - @Option(name: .long, help: "Figma API requests per minute") - var rateLimit: Int = 10 + @Option(name: .long, help: "Figma API requests per minute (overrides config, default: 10)") + var rateLimit: Int? - @Option(name: .long, help: "Maximum retry attempts for failed requests") - var maxRetries: Int = 4 + @Option(name: .long, help: "Maximum retry attempts for failed requests (overrides config, default: 4)") + var maxRetries: Int? - @Flag(name: .long, help: "Resume from previous checkpoint if available") + @Flag(name: .long, help: "Resume from previous checkpoint if available (overrides config)") var resume: Bool = false @Option(name: .long, help: "Path to write JSON report") @@ -68,11 +68,11 @@ extension ExFigCommand { var experimentalGranularCache: Bool = false /// Download concurrency - @Option(name: .long, help: "Maximum concurrent CDN downloads") - var concurrentDownloads: Int = FileDownloader.defaultMaxConcurrentDownloads + @Option(name: .long, help: "Maximum concurrent CDN downloads (overrides config, default: 20)") + var concurrentDownloads: Int? /// Connection options - @Option(name: .long, help: "Figma API request timeout in seconds (overrides config)") + @Option(name: .long, help: "Figma API request timeout in seconds (overrides config, default: 30)") var timeout: Int? @Argument(help: "Config files or directory to process") @@ -82,6 +82,18 @@ extension ExFigCommand { if let timeout, timeout <= 0 { throw ValidationError("Timeout must be positive") } + if let parallel, parallel <= 0 { + throw ValidationError("Parallel must be positive") + } + if let rateLimit, rateLimit <= 0 { + throw ValidationError("Rate limit must be positive") + } + if let maxRetries, maxRetries < 0 { + throw ValidationError("Max retries cannot be negative") + } + if let concurrentDownloads, concurrentDownloads <= 0 { + throw ValidationError("Concurrent downloads must be positive") + } } // swiftlint:disable:next function_body_length @@ -94,11 +106,27 @@ extension ExFigCommand { let (validConfigs, conflicts) = try discoverAndValidateConfigs(ui: ui) guard !validConfigs.isEmpty else { return } + // Resolve batch-level settings: CLI flags > first config's batch:/figma: > built-in defaults. + // Per-target batch: blocks in subsequent configs are ignored (debug-logged under -v). + let resolved = await BatchSettingsResolver.resolve( + cliParallel: parallel, + cliFailFast: failFast, + cliResume: resume, + cliRateLimit: rateLimit, + cliMaxRetries: maxRetries, + cliConcurrentDownloads: concurrentDownloads, + cliTimeout: timeout, + allConfigs: validConfigs, + verbose: globalOptions.verbose, + ui: ui + ) + // Prepare configs with checkpoint handling let workingDirectory = URL(fileURLWithPath: FileManager.default.currentDirectoryPath) let (configs, checkpoint) = prepareConfigsWithCheckpoint( validConfigs: validConfigs, workingDirectory: workingDirectory, + resolved: resolved, ui: ui ) @@ -114,6 +142,7 @@ extension ExFigCommand { conflicts: conflicts, checkpoint: checkpoint, workingDirectory: workingDirectory, + resolved: resolved, ui: ui ) @@ -122,6 +151,7 @@ extension ExFigCommand { result: result, rateLimiter: rateLimiter, workingDirectory: workingDirectory, + resolved: resolved, ui: ui ) } @@ -157,11 +187,16 @@ extension ExFigCommand { private func prepareConfigsWithCheckpoint( validConfigs: [URL], workingDirectory: URL, + resolved: ResolvedBatchSettings, ui: TerminalUI ) -> ([ConfigFile], BatchCheckpoint) { var configs = validConfigs.map { ConfigFile(url: $0) } - if let existing = loadCheckpointIfResuming(workingDirectory: workingDirectory, ui: ui) { + if let existing = loadCheckpointIfResuming( + workingDirectory: workingDirectory, + resolved: resolved, + ui: ui + ) { let skipped = existing.completedConfigs.count configs = configs.filter { !existing.isCompleted($0.url.path) } ui.info("Resuming: \(skipped) config(s) already completed, \(configs.count) remaining") @@ -171,8 +206,12 @@ extension ExFigCommand { return (configs, BatchCheckpoint(requestedPaths: paths)) } - private func loadCheckpointIfResuming(workingDirectory: URL, ui: TerminalUI) -> BatchCheckpoint? { - guard resume else { return nil } + private func loadCheckpointIfResuming( + workingDirectory: URL, + resolved: ResolvedBatchSettings, + ui: TerminalUI + ) -> BatchCheckpoint? { + guard resolved.resume else { return nil } guard let existing = try? BatchCheckpoint.load(from: workingDirectory) else { return nil @@ -193,16 +232,17 @@ extension ExFigCommand { return existing } - // swiftlint:disable:next function_body_length + // swiftlint:disable:next function_body_length function_parameter_count private func executeBatch( configs: [ConfigFile], conflicts: [OutputPathConflict], checkpoint: BatchCheckpoint, workingDirectory: URL, + resolved: ResolvedBatchSettings, ui: TerminalUI ) async -> (BatchResult, SharedRateLimiter) { - let rateLimiter = SharedRateLimiter(requestsPerMinute: Double(rateLimit)) - let retryPolicy = RetryPolicy(maxRetries: maxRetries) + let rateLimiter = SharedRateLimiter(requestsPerMinute: Double(resolved.rateLimit)) + let retryPolicy = RetryPolicy(maxRetries: resolved.maxRetries) // Load cache for smart pre-fetch optimization (version checking) // This allows skipping heavy Components API calls when file version is unchanged @@ -242,7 +282,7 @@ extension ExFigCommand { // Create shared download queue for cross-config pipelining let downloadQueue = SharedDownloadQueue( - maxConcurrentDownloads: concurrentDownloads * parallel + maxConcurrentDownloads: resolved.concurrentDownloads * resolved.parallel ) // Create priority map: configs submitted first get higher priority (lower number) @@ -254,6 +294,7 @@ extension ExFigCommand { configs: configs, conflicts: conflicts, sharedGranularCache: sharedGranularCache, + resolved: resolved, ui: ui ) @@ -272,11 +313,12 @@ extension ExFigCommand { await progressView.registerConfig(name: config.name) } - let executor = BatchExecutor(maxParallel: parallel, failFast: failFast) + let executor = BatchExecutor(maxParallel: resolved.parallel, failFast: resolved.failFast) let checkpointManager = CheckpointManager(checkpoint: checkpoint, directory: workingDirectory) let runnerFactory = makeRunnerFactory( rateLimiter: rateLimiter, retryPolicy: retryPolicy, + resolved: resolved, priorityMap: priorityMap ) @@ -363,37 +405,43 @@ extension ExFigCommand { } /// Creates a factory function for BatchConfigRunner instances. + /// + /// In batch mode, rate-limit / max-retries / concurrent-downloads / timeout values are + /// already resolved at the batch level (CLI flag > first config's `figma:` block > built-in + /// default). Per-config `figma:` rate-limiting fields are intentionally ignored — the rate + /// limiter and download queue are shared across all configs in the run. private func makeRunnerFactory( rateLimiter: SharedRateLimiter, retryPolicy: RetryPolicy, + resolved: ResolvedBatchSettings, priorityMap: [String: Int] ) -> @Sendable (ConfigFile) -> BatchConfigRunner { // Capture all values needed for runner creation let globalOptions = globalOptions - let maxRetries = maxRetries - let resume = resume let cache = cache let noCache = noCache let force = force let cachePath = cachePath let experimentalGranularCache = experimentalGranularCache - let concurrentDownloads = concurrentDownloads - let timeout = timeout + let resolvedMaxRetries = resolved.maxRetries + let resolvedResume = resolved.resume + let resolvedConcurrentDownloads = resolved.concurrentDownloads + let resolvedTimeout = resolved.timeout return { configFile in BatchConfigRunner( rateLimiter: rateLimiter, retryPolicy: retryPolicy, globalOptions: globalOptions, - maxRetries: maxRetries, - resume: resume, + maxRetries: resolvedMaxRetries, + resume: resolvedResume, cache: cache, noCache: noCache, force: force, cachePath: cachePath, experimentalGranularCache: experimentalGranularCache, - concurrentDownloads: concurrentDownloads, - cliTimeout: timeout, + concurrentDownloads: resolvedConcurrentDownloads, + cliTimeout: resolvedTimeout, configPriority: priorityMap[configFile.name] ?? 0 ) } @@ -404,9 +452,10 @@ extension ExFigCommand { configs: [ConfigFile], conflicts: [OutputPathConflict], sharedGranularCache: SharedGranularCache?, + resolved: ResolvedBatchSettings, ui: TerminalUI ) { - ui.info("Processing \(configs.count) config(s) with up to \(parallel) parallel workers:") + ui.info("Processing \(configs.count) config(s) with up to \(resolved.parallel) parallel workers:") // Build conflict lookup: config name → shared path var conflictMap: [String: String] = [:] @@ -433,11 +482,12 @@ extension ExFigCommand { NooraUI.shared.table(headers: headers, rows: rows) if globalOptions.verbose { - ui.info("Rate limit: \(rateLimit) req/min, max retries: \(maxRetries)") + ui.info("Rate limit: \(resolved.rateLimit) req/min, max retries: \(resolved.maxRetries)") if sharedGranularCache != nil { ui.info("Granular cache: shared across workers") } - ui.info("Download queue: shared with \(concurrentDownloads * parallel) concurrent slots") + let slots = resolved.concurrentDownloads * resolved.parallel + ui.info("Download queue: shared with \(slots) concurrent slots") } } @@ -696,6 +746,7 @@ extension ExFigCommand { result: BatchResult, rateLimiter: SharedRateLimiter, workingDirectory: URL, + resolved: ResolvedBatchSettings, ui: TerminalUI ) { displaySummary(result: result, ui: ui) @@ -727,7 +778,7 @@ extension ExFigCommand { if globalOptions.verbose { ui.info("Checkpoint cleared (all configs completed successfully)") } - } else if !failFast { + } else if !resolved.failFast { ui.error("Batch completed with \(result.failureCount) failure(s)") ui.info("Run with --resume to retry failed configs") } diff --git a/Sources/ExFigCLI/Subcommands/Download.swift b/Sources/ExFigCLI/Subcommands/Download.swift index 7403104d..4d6367de 100644 --- a/Sources/ExFigCLI/Subcommands/Download.swift +++ b/Sources/ExFigCLI/Subcommands/Download.swift @@ -96,14 +96,21 @@ extension ExFigCommand.Download { ExFigCommand.initializeTerminalUI(verbose: globalOptions.verbose, quiet: globalOptions.quiet) let ui = ExFigCommand.terminalUI! + let figmaParams = options.params.figma + if globalOptions.verbose, figmaParams?.concurrentDownloads != nil { + ui.debug( + "figma.concurrentDownloads ignored: not applicable to download colors (no CDN downloads)" + ) + } let baseClient = try FigmaClient( accessToken: options.requireFigmaToken(), - timeout: options.params.figma?.timeout + timeout: faultToleranceOptions.timeout.map(TimeInterval.init) ?? figmaParams?.timeout ) - let rateLimiter = faultToleranceOptions.createRateLimiter() + let rateLimiter = faultToleranceOptions.createRateLimiter(configValue: figmaParams?.rateLimit) let client = faultToleranceOptions.createRateLimitedClient( wrapping: baseClient, rateLimiter: rateLimiter, + configMaxRetries: figmaParams?.maxRetries, onRetry: { attempt, error in ui.warning("Retry \(attempt) after error: \(error.localizedDescription)") } @@ -122,8 +129,6 @@ extension ExFigCommand.Download { ) } - let figmaParams = options.params.figma - switch jsonOptions.format { case .w3c: try await exportW3C( diff --git a/Sources/ExFigCLI/Subcommands/DownloadIcons.swift b/Sources/ExFigCLI/Subcommands/DownloadIcons.swift index 2aef4ff6..7128bb9e 100644 --- a/Sources/ExFigCLI/Subcommands/DownloadIcons.swift +++ b/Sources/ExFigCLI/Subcommands/DownloadIcons.swift @@ -63,14 +63,16 @@ extension ExFigCommand.Download { ExFigCommand.initializeTerminalUI(verbose: globalOptions.verbose, quiet: globalOptions.quiet) let ui = ExFigCommand.terminalUI! + let figmaParams = options.params.figma let baseClient = try FigmaClient( accessToken: options.requireFigmaToken(), - timeout: options.params.figma?.timeout + timeout: faultToleranceOptions.timeout.map(TimeInterval.init) ?? figmaParams?.timeout ) - let rateLimiter = faultToleranceOptions.createRateLimiter() + let rateLimiter = faultToleranceOptions.createRateLimiter(configValue: figmaParams?.rateLimit) let client = faultToleranceOptions.createRateLimitedClient( wrapping: baseClient, rateLimiter: rateLimiter, + configMaxRetries: figmaParams?.maxRetries, onRetry: { attempt, error in ui.warning("Retry \(attempt) after error: \(error.localizedDescription)") } diff --git a/Sources/ExFigCLI/Subcommands/DownloadImages.swift b/Sources/ExFigCLI/Subcommands/DownloadImages.swift index 5e71c7c4..f743d9a2 100644 --- a/Sources/ExFigCLI/Subcommands/DownloadImages.swift +++ b/Sources/ExFigCLI/Subcommands/DownloadImages.swift @@ -50,11 +50,11 @@ extension ExFigCommand { @OptionGroup var downloadOptions: DownloadOptions - @Option(name: .long, help: "Maximum retry attempts for failed API requests") - var maxRetries: Int = 4 + @Option(name: .long, help: "Maximum retry attempts for failed API requests (default: 4)") + var maxRetries: Int? - @Option(name: .long, help: "Maximum API requests per minute") - var rateLimit: Int = 10 + @Option(name: .long, help: "Maximum API requests per minute (default: 10)") + var rateLimit: Int? @Flag(name: .long, help: "Stop on first error without retrying") var failFast: Bool = false @@ -62,11 +62,12 @@ extension ExFigCommand { @Flag(name: .long, help: "Continue from checkpoint after interruption") var resume: Bool = false - @Option(name: .long, help: "Maximum concurrent CDN downloads") - var concurrentDownloads: Int = FileDownloader.defaultMaxConcurrentDownloads + @Option(name: .long, help: "Maximum concurrent CDN downloads (default: 20)") + var concurrentDownloads: Int? /// Constructs `HeavyFaultToleranceOptions` from locally declared options, /// using `DownloadOptions.timeout` to avoid duplicate `--timeout` flags. + /// `fetch` is config-free; help text omits "(overrides config)" intentionally. var faultToleranceOptions: HeavyFaultToleranceOptions { var opts = HeavyFaultToleranceOptions() opts.maxRetries = maxRetries @@ -154,17 +155,18 @@ extension ExFigCommand { ui.debug("Scale: \(options.effectiveScale)x") } - // Create Figma client with fault tolerance + // Create Figma client with fault tolerance. + // `fetch` is config-free, so config-supplied values are nil and CLI flags / built-in defaults apply. let baseClient = FigmaClient(accessToken: accessToken, timeout: TimeInterval(options.timeout)) let rateLimiter = faultToleranceOptions.createRateLimiter() - let maxRetries = faultToleranceOptions.maxRetries + let effectiveMaxRetries = faultToleranceOptions.effectiveMaxRetries(configValue: nil) let client = faultToleranceOptions.createRateLimitedClient( wrapping: baseClient, rateLimiter: rateLimiter, onRetry: { attempt, error in let warning = ExFigWarning.retrying( attempt: attempt, - maxAttempts: maxRetries, + maxAttempts: effectiveMaxRetries, error: error.localizedDescription, delay: "..." ) diff --git a/Sources/ExFigCLI/Subcommands/DownloadImagesExport.swift b/Sources/ExFigCLI/Subcommands/DownloadImagesExport.swift index 305fb5f6..fa3108a9 100644 --- a/Sources/ExFigCLI/Subcommands/DownloadImagesExport.swift +++ b/Sources/ExFigCLI/Subcommands/DownloadImagesExport.swift @@ -39,14 +39,16 @@ extension ExFigCommand.Download { ExFigCommand.initializeTerminalUI(verbose: globalOptions.verbose, quiet: globalOptions.quiet) let ui = ExFigCommand.terminalUI! + let figmaParams = options.params.figma let baseClient = try FigmaClient( accessToken: options.requireFigmaToken(), - timeout: options.params.figma?.timeout + timeout: faultToleranceOptions.timeout.map(TimeInterval.init) ?? figmaParams?.timeout ) - let rateLimiter = faultToleranceOptions.createRateLimiter() + let rateLimiter = faultToleranceOptions.createRateLimiter(configValue: figmaParams?.rateLimit) let client = faultToleranceOptions.createRateLimitedClient( wrapping: baseClient, rateLimiter: rateLimiter, + configMaxRetries: figmaParams?.maxRetries, onRetry: { attempt, error in ui.warning("Retry \(attempt) after error: \(error.localizedDescription)") } diff --git a/Sources/ExFigCLI/Subcommands/DownloadTokens.swift b/Sources/ExFigCLI/Subcommands/DownloadTokens.swift index 73f1883a..125acddf 100644 --- a/Sources/ExFigCLI/Subcommands/DownloadTokens.swift +++ b/Sources/ExFigCLI/Subcommands/DownloadTokens.swift @@ -29,14 +29,16 @@ extension ExFigCommand.Download { ExFigCommand.initializeTerminalUI(verbose: globalOptions.verbose, quiet: globalOptions.quiet) let ui = ExFigCommand.terminalUI! + let figmaParams = options.params.figma let baseClient = try FigmaClient( accessToken: options.requireFigmaToken(), - timeout: options.params.figma?.timeout + timeout: faultToleranceOptions.timeout.map(TimeInterval.init) ?? figmaParams?.timeout ) - let rateLimiter = faultToleranceOptions.createRateLimiter() + let rateLimiter = faultToleranceOptions.createRateLimiter(configValue: figmaParams?.rateLimit) let client = faultToleranceOptions.createRateLimitedClient( wrapping: baseClient, rateLimiter: rateLimiter, + configMaxRetries: figmaParams?.maxRetries, onRetry: { attempt, error in ui.warning("Retry \(attempt) after error: \(error.localizedDescription)") } diff --git a/Sources/ExFigCLI/Subcommands/DownloadTypography.swift b/Sources/ExFigCLI/Subcommands/DownloadTypography.swift index 01419b92..02d6639c 100644 --- a/Sources/ExFigCLI/Subcommands/DownloadTypography.swift +++ b/Sources/ExFigCLI/Subcommands/DownloadTypography.swift @@ -29,14 +29,21 @@ extension ExFigCommand.Download { ExFigCommand.initializeTerminalUI(verbose: globalOptions.verbose, quiet: globalOptions.quiet) let ui = ExFigCommand.terminalUI! + let figmaParams = options.params.figma + if globalOptions.verbose, figmaParams?.concurrentDownloads != nil { + ui.debug( + "figma.concurrentDownloads ignored: not applicable to download typography (no CDN downloads)" + ) + } let baseClient = try FigmaClient( accessToken: options.requireFigmaToken(), - timeout: options.params.figma?.timeout + timeout: faultToleranceOptions.timeout.map(TimeInterval.init) ?? figmaParams?.timeout ) - let rateLimiter = faultToleranceOptions.createRateLimiter() + let rateLimiter = faultToleranceOptions.createRateLimiter(configValue: figmaParams?.rateLimit) let client = faultToleranceOptions.createRateLimitedClient( wrapping: baseClient, rateLimiter: rateLimiter, + configMaxRetries: figmaParams?.maxRetries, onRetry: { attempt, error in ui.warning("Retry \(attempt) after error: \(error.localizedDescription)") } diff --git a/Sources/ExFigCLI/Subcommands/ExportColors.swift b/Sources/ExFigCLI/Subcommands/ExportColors.swift index f7e073c9..a7f26de9 100644 --- a/Sources/ExFigCLI/Subcommands/ExportColors.swift +++ b/Sources/ExFigCLI/Subcommands/ExportColors.swift @@ -43,9 +43,17 @@ extension ExFigCommand { ExFigCommand.checkSchemaVersionIfNeeded() let ui = ExFigCommand.terminalUI! + let figma = options.params.figma + if globalOptions.verbose, figma?.concurrentDownloads != nil { + ui.debug( + "figma.concurrentDownloads ignored: not applicable to colors (no CDN downloads)" + ) + } let client = resolveClient( accessToken: options.accessToken, - timeout: options.params.figma?.timeout, + timeout: figma?.timeout, + rateLimit: figma?.rateLimit, + maxRetries: figma?.maxRetries, options: faultToleranceOptions, ui: ui ) diff --git a/Sources/ExFigCLI/Subcommands/ExportIcons.swift b/Sources/ExFigCLI/Subcommands/ExportIcons.swift index a20f3738..30d32b33 100644 --- a/Sources/ExFigCLI/Subcommands/ExportIcons.swift +++ b/Sources/ExFigCLI/Subcommands/ExportIcons.swift @@ -44,9 +44,12 @@ extension ExFigCommand { ExFigCommand.checkSchemaVersionIfNeeded() let ui = ExFigCommand.terminalUI! + let figma = options.params.figma let client = resolveClient( accessToken: options.accessToken, - timeout: options.params.figma?.timeout, + timeout: figma?.timeout, + rateLimit: figma?.rateLimit, + maxRetries: figma?.maxRetries, options: faultToleranceOptions, ui: ui ) diff --git a/Sources/ExFigCLI/Subcommands/ExportImages.swift b/Sources/ExFigCLI/Subcommands/ExportImages.swift index 7ff739d1..0de45bdd 100644 --- a/Sources/ExFigCLI/Subcommands/ExportImages.swift +++ b/Sources/ExFigCLI/Subcommands/ExportImages.swift @@ -43,9 +43,12 @@ extension ExFigCommand { ExFigCommand.checkSchemaVersionIfNeeded() let ui = ExFigCommand.terminalUI! + let figma = options.params.figma let client = resolveClient( accessToken: options.accessToken, - timeout: options.params.figma?.timeout, + timeout: figma?.timeout, + rateLimit: figma?.rateLimit, + maxRetries: figma?.maxRetries, options: faultToleranceOptions, ui: ui ) diff --git a/Sources/ExFigCLI/Subcommands/ExportTypography.swift b/Sources/ExFigCLI/Subcommands/ExportTypography.swift index fdd1ea0e..4b612e2f 100644 --- a/Sources/ExFigCLI/Subcommands/ExportTypography.swift +++ b/Sources/ExFigCLI/Subcommands/ExportTypography.swift @@ -32,9 +32,17 @@ extension ExFigCommand { ExFigCommand.checkSchemaVersionIfNeeded() let ui = ExFigCommand.terminalUI! + let figma = options.params.figma + if globalOptions.verbose, figma?.concurrentDownloads != nil { + ui.debug( + "figma.concurrentDownloads ignored: not applicable to typography (no CDN downloads)" + ) + } let client = resolveClient( accessToken: options.accessToken, - timeout: options.params.figma?.timeout, + timeout: figma?.timeout, + rateLimit: figma?.rateLimit, + maxRetries: figma?.maxRetries, options: faultToleranceOptions, ui: ui ) diff --git a/Sources/ExFigCLI/Subcommands/Lint.swift b/Sources/ExFigCLI/Subcommands/Lint.swift index bbb5924a..426d56f2 100644 --- a/Sources/ExFigCLI/Subcommands/Lint.swift +++ b/Sources/ExFigCLI/Subcommands/Lint.swift @@ -54,9 +54,12 @@ extension ExFigCommand { Set($0.split(separator: ",").map { String($0.trimmingCharacters(in: .whitespaces)) }) } + let figma = options.params.figma let client = resolveClient( accessToken: options.accessToken, - timeout: options.params.figma?.timeout, + timeout: figma?.timeout, + rateLimit: figma?.rateLimit, + maxRetries: figma?.maxRetries, options: faultToleranceOptions, ui: ui ) diff --git a/Sources/ExFigConfig/Generated/Batch.pkl.swift b/Sources/ExFigConfig/Generated/Batch.pkl.swift new file mode 100644 index 00000000..d1d76a69 --- /dev/null +++ b/Sources/ExFigConfig/Generated/Batch.pkl.swift @@ -0,0 +1,58 @@ +// Code generated from Pkl module `Batch`. DO NOT EDIT. +import PklSwift + +public enum Batch {} + +extension Batch { + /// Batch execution settings — only meaningful for `exfig batch`. + public struct BatchConfig: PklRegisteredType, Decodable, Hashable, Sendable { + public static let registeredIdentifier: String = "Batch#BatchConfig" + + /// Maximum configs to process in parallel. CLI --parallel overrides. + public var parallel: Int? + + /// Stop processing on first error. CLI --fail-fast overrides. + public var failFast: Bool? + + /// Resume from previous checkpoint if available. CLI --resume overrides. + public var resume: Bool? + + public init(parallel: Int?, failFast: Bool?, resume: Bool?) { + self.parallel = parallel + self.failFast = failFast + self.resume = resume + } + } + + /// Batch orchestration configuration. + /// + /// These settings only apply when running `exfig batch` and are read + /// from the FIRST config in the argument list. Per-target `batch:` blocks + /// in subsequent configs are ignored (logged under -v). + public struct Module: PklRegisteredType, Decodable, Hashable, Sendable { + public static let registeredIdentifier: String = "Batch" + + public init() {} + } + + /// Load the Pkl module at the given source and evaluate it into `Batch.Module`. + /// + /// - Parameter source: The source of the Pkl module. + public static func loadFrom(source: ModuleSource) async throws -> Batch.Module { + try await PklSwift.withEvaluator { evaluator in + try await loadFrom(evaluator: evaluator, source: source) + } + } + + /// Load the Pkl module at the given source and evaluate it with the given evaluator into + /// `Batch.Module`. + /// + /// - Parameter evaluator: The evaluator to use for evaluation. + /// - Parameter source: The module to evaluate. + public static func loadFrom( + evaluator: PklSwift.Evaluator, + source: PklSwift.ModuleSource + ) async throws -> Batch.Module { + try await evaluator.evaluateModule(source: source, as: Module.self) + } +} diff --git a/Sources/ExFigConfig/Generated/ExFig.pkl.swift b/Sources/ExFigConfig/Generated/ExFig.pkl.swift index f1e7737e..287f69f9 100644 --- a/Sources/ExFigConfig/Generated/ExFig.pkl.swift +++ b/Sources/ExFigConfig/Generated/ExFig.pkl.swift @@ -15,6 +15,8 @@ public protocol ExFig_Module: PklRegisteredType, DynamicallyEquatable, Hashable, var flutter: Flutter.FlutterConfig? { get } var web: Web.WebConfig? { get } + + var batch: Batch.BatchConfig? { get } } extension ExFig { @@ -27,7 +29,7 @@ extension ExFig { /// /// Usage: /// ```pkl - /// amends "package://pkg.pkl-lang.org/github.com/DesignPipe/exfig/exfig@2.8.0#/ExFig.pkl" + /// amends "package://pkg.pkl-lang.org/github.com/DesignPipe/exfig/exfig@3.4.0#/ExFig.pkl" /// /// figma { /// lightFileId = "xxx" @@ -62,13 +64,19 @@ extension ExFig { /// Web platform configuration. public var web: Web.WebConfig? + /// Batch orchestration settings. + /// Only read from the FIRST config when running `exfig batch`. + /// Per-target `batch:` blocks in subsequent configs are ignored (logged under -v). + public var batch: Batch.BatchConfig? + public init( figma: Figma.FigmaConfig?, common: Common.CommonConfig?, ios: iOS.iOSConfig?, android: Android.AndroidConfig?, flutter: Flutter.FlutterConfig?, - web: Web.WebConfig? + web: Web.WebConfig?, + batch: Batch.BatchConfig? ) { self.figma = figma self.common = common @@ -76,6 +84,7 @@ extension ExFig { self.android = android self.flutter = flutter self.web = web + self.batch = batch } } } diff --git a/Sources/ExFigConfig/Generated/Figma.pkl.swift b/Sources/ExFigConfig/Generated/Figma.pkl.swift index 0b4e0cfb..1218c857 100644 --- a/Sources/ExFigConfig/Generated/Figma.pkl.swift +++ b/Sources/ExFigConfig/Generated/Figma.pkl.swift @@ -32,18 +32,40 @@ extension Figma { /// Request timeout in seconds. public var timeout: Float64? + /// Maximum API requests per minute. CLI --rate-limit overrides. + /// Tier 1 endpoints (GET file, GET images): 10–20/min on Dev/Full seats. + /// Tier 2 endpoints (Variables, Image fills): 25–100/min. + /// Tier 3 endpoints (Components, file metadata): 50–150/min. + /// Limit is per-token + per-plan, not per-IP. See https://developers.figma.com/docs/rest-api/rate-limits/ + public var rateLimit: Int? + + /// Maximum retry attempts for failed API requests. CLI --max-retries overrides. + public var maxRetries: Int? + + /// Maximum concurrent CDN downloads (icons/images only). + /// Ignored by colors/typography. CLI --concurrent-downloads overrides. + /// NOTE: client-side cap on CDN connections (URLSession.httpMaximumConnectionsPerHost), + /// not a Figma REST API limit. Use `rateLimit` for API throttling. + public var concurrentDownloads: Int? + public init( lightFileId: String?, darkFileId: String?, lightHighContrastFileId: String?, darkHighContrastFileId: String?, - timeout: Float64? + timeout: Float64?, + rateLimit: Int?, + maxRetries: Int?, + concurrentDownloads: Int? ) { self.lightFileId = lightFileId self.darkFileId = darkFileId self.lightHighContrastFileId = lightHighContrastFileId self.darkHighContrastFileId = darkHighContrastFileId self.timeout = timeout + self.rateLimit = rateLimit + self.maxRetries = maxRetries + self.concurrentDownloads = concurrentDownloads } } diff --git a/Sources/ExFigConfig/PKL/PKLEvaluator.swift b/Sources/ExFigConfig/PKL/PKLEvaluator.swift index e23243d7..adf2a39f 100644 --- a/Sources/ExFigConfig/PKL/PKLEvaluator.swift +++ b/Sources/ExFigConfig/PKL/PKLEvaluator.swift @@ -58,6 +58,9 @@ public enum PKLEvaluator { // Figma Figma.Module.self, Figma.FigmaConfig.self, + // Batch + Batch.Module.self, + Batch.BatchConfig.self, // iOS iOS.Module.self, iOS.HeicOptions.self, diff --git a/Tests/ExFigTests/Batch/BatchSettingsResolverTests.swift b/Tests/ExFigTests/Batch/BatchSettingsResolverTests.swift new file mode 100644 index 00000000..309641ca --- /dev/null +++ b/Tests/ExFigTests/Batch/BatchSettingsResolverTests.swift @@ -0,0 +1,229 @@ +@testable import ExFigCLI +import Foundation +import XCTest + +final class BatchSettingsResolverTests: XCTestCase { + // MARK: - Defaults Without First Config + + func testAllDefaultsWhenNoConfigsNoCLI() async { + let ui = TerminalUI(outputMode: .quiet) + + let resolved = await BatchSettingsResolver.resolve( + cliParallel: nil, + cliFailFast: false, + cliResume: false, + cliRateLimit: nil, + cliMaxRetries: nil, + cliConcurrentDownloads: nil, + cliTimeout: nil, + allConfigs: [], + verbose: false, + ui: ui + ) + + XCTAssertEqual(resolved.parallel, 3) + XCTAssertFalse(resolved.failFast) + XCTAssertFalse(resolved.resume) + XCTAssertEqual(resolved.rateLimit, 10) + XCTAssertEqual(resolved.maxRetries, 4) + XCTAssertEqual(resolved.concurrentDownloads, 20) + XCTAssertNil(resolved.timeout) + } + + // MARK: - CLI Wins When Config Missing + + func testCLIValuesUsedWhenNoFirstConfig() async { + let ui = TerminalUI(outputMode: .quiet) + + let resolved = await BatchSettingsResolver.resolve( + cliParallel: 8, + cliFailFast: true, + cliResume: true, + cliRateLimit: 25, + cliMaxRetries: 6, + cliConcurrentDownloads: 50, + cliTimeout: 120, + allConfigs: [], + verbose: false, + ui: ui + ) + + XCTAssertEqual(resolved.parallel, 8) + XCTAssertTrue(resolved.failFast) + XCTAssertTrue(resolved.resume) + XCTAssertEqual(resolved.rateLimit, 25) + XCTAssertEqual(resolved.maxRetries, 6) + XCTAssertEqual(resolved.concurrentDownloads, 50) + XCTAssertEqual(resolved.timeout, 120) + } + + // MARK: - First Config Wins When CLI Missing + + func testFirstConfigBatchAndFigmaFieldsUsedWhenNoCLI() async throws { + let configURL = try makeConfigPKL( + figma: "rateLimit = 25\nmaxRetries = 6\nconcurrentDownloads = 50", + batch: "parallel = 8\nfailFast = true\nresume = true" + ) + defer { try? FileManager.default.removeItem(at: configURL) } + let ui = TerminalUI(outputMode: .quiet) + + let resolved = await BatchSettingsResolver.resolve( + cliParallel: nil, + cliFailFast: false, + cliResume: false, + cliRateLimit: nil, + cliMaxRetries: nil, + cliConcurrentDownloads: nil, + cliTimeout: nil, + allConfigs: [configURL], + verbose: false, + ui: ui + ) + + XCTAssertEqual(resolved.parallel, 8) + XCTAssertTrue(resolved.failFast) + XCTAssertTrue(resolved.resume) + XCTAssertEqual(resolved.rateLimit, 25) + XCTAssertEqual(resolved.maxRetries, 6) + XCTAssertEqual(resolved.concurrentDownloads, 50) + } + + func testCLIOverridesFirstConfig() async throws { + let configURL = try makeConfigPKL( + figma: "rateLimit = 25\nmaxRetries = 6", + batch: "parallel = 8" + ) + defer { try? FileManager.default.removeItem(at: configURL) } + let ui = TerminalUI(outputMode: .quiet) + + let resolved = await BatchSettingsResolver.resolve( + cliParallel: 4, + cliFailFast: false, + cliResume: false, + cliRateLimit: 30, + cliMaxRetries: 2, + cliConcurrentDownloads: nil, + cliTimeout: nil, + allConfigs: [configURL], + verbose: false, + ui: ui + ) + + XCTAssertEqual(resolved.parallel, 4) // CLI wins + XCTAssertEqual(resolved.rateLimit, 30) // CLI wins + XCTAssertEqual(resolved.maxRetries, 2) // CLI wins + XCTAssertEqual(resolved.concurrentDownloads, 20) // default (no CLI, no config) + } + + // MARK: - Boolean OR Semantics + + func testFailFastORedFromCLIAndConfig() async throws { + // Config has failFast=true, CLI doesn't pass it → result is true + let configURL = try makeConfigPKL(figma: nil, batch: "failFast = true") + defer { try? FileManager.default.removeItem(at: configURL) } + let ui = TerminalUI(outputMode: .quiet) + + let resolved = await BatchSettingsResolver.resolve( + cliParallel: nil, + cliFailFast: false, + cliResume: false, + cliRateLimit: nil, + cliMaxRetries: nil, + cliConcurrentDownloads: nil, + cliTimeout: nil, + allConfigs: [configURL], + verbose: false, + ui: ui + ) + + XCTAssertTrue(resolved.failFast) + } + + func testResumeORedFromCLIWhenConfigFalse() async throws { + // Config has resume=false, CLI passes --resume → result is true + let configURL = try makeConfigPKL(figma: nil, batch: "resume = false") + defer { try? FileManager.default.removeItem(at: configURL) } + let ui = TerminalUI(outputMode: .quiet) + + let resolved = await BatchSettingsResolver.resolve( + cliParallel: nil, + cliFailFast: false, + cliResume: true, + cliRateLimit: nil, + cliMaxRetries: nil, + cliConcurrentDownloads: nil, + cliTimeout: nil, + allConfigs: [configURL], + verbose: false, + ui: ui + ) + + XCTAssertTrue(resolved.resume) + } + + // MARK: - Invalid First Config Falls Back + + func testInvalidFirstConfigFallsBackToDefaults() async { + let bogusURL = URL(fileURLWithPath: "/tmp/does-not-exist-\(UUID().uuidString).pkl") + let ui = TerminalUI(outputMode: .quiet) + + let resolved = await BatchSettingsResolver.resolve( + cliParallel: nil, + cliFailFast: false, + cliResume: false, + cliRateLimit: nil, + cliMaxRetries: nil, + cliConcurrentDownloads: nil, + cliTimeout: nil, + allConfigs: [bogusURL], + verbose: false, + ui: ui + ) + + XCTAssertEqual(resolved.parallel, 3) + XCTAssertEqual(resolved.rateLimit, 10) + XCTAssertEqual(resolved.maxRetries, 4) + XCTAssertEqual(resolved.concurrentDownloads, 20) + } + + // MARK: - Helpers + + private func makeConfigPKL(figma: String?, batch: String?) throws -> URL { + // Pull the schemas next to a per-test temp directory by referencing them via the project's + // package URI substitution flow. For these unit tests we use the shipped local schemas. + let schemasDir = URL(fileURLWithPath: #filePath) + .deletingLastPathComponent() + .deletingLastPathComponent() + .deletingLastPathComponent() + .deletingLastPathComponent() + .appendingPathComponent("Sources/ExFigCLI/Resources/Schemas") + let exfigPath = schemasDir.appendingPathComponent("ExFig.pkl").path + let figmaPath = schemasDir.appendingPathComponent("Figma.pkl").path + let batchPath = schemasDir.appendingPathComponent("Batch.pkl").path + + var lines: [String] = [ + "amends \"\(exfigPath)\"", + "import \"\(figmaPath)\"", + "import \"\(batchPath)\"", + ] + if let figma { + lines.append("figma = new Figma.FigmaConfig {") + for line in figma.split(separator: "\n") { + lines.append(" \(line)") + } + lines.append("}") + } + if let batch { + lines.append("batch = new Batch.BatchConfig {") + for line in batch.split(separator: "\n") { + lines.append(" \(line)") + } + lines.append("}") + } + + let url = FileManager.default.temporaryDirectory + .appendingPathComponent("batch-settings-resolver-\(UUID().uuidString).pkl") + try lines.joined(separator: "\n").write(to: url, atomically: true, encoding: .utf8) + return url + } +} diff --git a/Tests/ExFigTests/Input/FaultToleranceOptionsTests.swift b/Tests/ExFigTests/Input/FaultToleranceOptionsTests.swift index b7041d30..68f40c9b 100644 --- a/Tests/ExFigTests/Input/FaultToleranceOptionsTests.swift +++ b/Tests/ExFigTests/Input/FaultToleranceOptionsTests.swift @@ -10,13 +10,15 @@ final class FaultToleranceOptionsTests: XCTestCase { func testDefaultMaxRetries() throws { let options = try FaultToleranceOptions.parse([]) - XCTAssertEqual(options.maxRetries, 4) + XCTAssertNil(options.maxRetries) + XCTAssertEqual(options.effectiveMaxRetries(configValue: nil), 4) } func testDefaultRateLimit() throws { let options = try FaultToleranceOptions.parse([]) - XCTAssertEqual(options.rateLimit, 10) + XCTAssertNil(options.rateLimit) + XCTAssertEqual(options.effectiveRateLimit(configValue: nil), 10) } func testDefaultTimeout() throws { @@ -61,6 +63,10 @@ final class FaultToleranceOptionsTests: XCTestCase { XCTAssertEqual(options.maxRetries, 0) } + func testMaxRetriesValidationRejectsNegative() { + XCTAssertThrowsError(try FaultToleranceOptions.parse(["--max-retries", "-1"])) + } + // MARK: - Rate Limit Flag func testRateLimitFlag() throws { @@ -75,6 +81,48 @@ final class FaultToleranceOptionsTests: XCTestCase { XCTAssertEqual(options.rateLimit, 100) } + func testRateLimitValidationRejectsZero() { + XCTAssertThrowsError(try FaultToleranceOptions.parse(["--rate-limit", "0"])) + } + + // MARK: - Precedence (CLI > config > default) + + func testEffectiveRateLimitCLIWinsOverConfig() throws { + let options = try FaultToleranceOptions.parse(["--rate-limit", "20"]) + + XCTAssertEqual(options.effectiveRateLimit(configValue: 15), 20) + } + + func testEffectiveRateLimitConfigUsedWhenNoCLI() throws { + let options = try FaultToleranceOptions.parse([]) + + XCTAssertEqual(options.effectiveRateLimit(configValue: 15), 15) + } + + func testEffectiveRateLimitDefaultWhenNoCLINoConfig() throws { + let options = try FaultToleranceOptions.parse([]) + + XCTAssertEqual(options.effectiveRateLimit(configValue: nil), 10) + } + + func testEffectiveMaxRetriesCLIWinsOverConfig() throws { + let options = try FaultToleranceOptions.parse(["--max-retries", "8"]) + + XCTAssertEqual(options.effectiveMaxRetries(configValue: 6), 8) + } + + func testEffectiveMaxRetriesConfigUsedWhenNoCLI() throws { + let options = try FaultToleranceOptions.parse([]) + + XCTAssertEqual(options.effectiveMaxRetries(configValue: 6), 6) + } + + func testEffectiveMaxRetriesDefaultWhenNoCLINoConfig() throws { + let options = try FaultToleranceOptions.parse([]) + + XCTAssertEqual(options.effectiveMaxRetries(configValue: nil), 4) + } + // MARK: - Combined Flags func testBothRetriesAndRateLimit() throws { @@ -159,11 +207,61 @@ final class HeavyFaultToleranceOptionsTests: XCTestCase { func testDefaultValues() throws { let options = try HeavyFaultToleranceOptions.parse([]) - XCTAssertEqual(options.maxRetries, 4) - XCTAssertEqual(options.rateLimit, 10) + XCTAssertNil(options.maxRetries) + XCTAssertNil(options.rateLimit) XCTAssertNil(options.timeout) + XCTAssertNil(options.concurrentDownloads) XCTAssertFalse(options.failFast) XCTAssertFalse(options.resume) + XCTAssertEqual(options.effectiveMaxRetries(configValue: nil), 4) + XCTAssertEqual(options.effectiveRateLimit(configValue: nil), 10) + XCTAssertEqual(options.effectiveConcurrentDownloads(configValue: nil), 20) + } + + // MARK: - Precedence (CLI > config > default) + + func testHeavyEffectiveConcurrentDownloadsCLIWinsOverConfig() throws { + let options = try HeavyFaultToleranceOptions.parse(["--concurrent-downloads", "50"]) + + XCTAssertEqual(options.effectiveConcurrentDownloads(configValue: 30), 50) + } + + func testHeavyEffectiveConcurrentDownloadsConfigUsedWhenNoCLI() throws { + let options = try HeavyFaultToleranceOptions.parse([]) + + XCTAssertEqual(options.effectiveConcurrentDownloads(configValue: 30), 30) + } + + func testHeavyEffectiveConcurrentDownloadsDefaultWhenNoCLINoConfig() throws { + let options = try HeavyFaultToleranceOptions.parse([]) + + XCTAssertEqual(options.effectiveConcurrentDownloads(configValue: nil), 20) + } + + func testHeavyEffectiveRateLimitCLIWins() throws { + let options = try HeavyFaultToleranceOptions.parse(["--rate-limit", "25"]) + + XCTAssertEqual(options.effectiveRateLimit(configValue: 15), 25) + } + + func testHeavyEffectiveMaxRetriesConfigUsed() throws { + let options = try HeavyFaultToleranceOptions.parse([]) + + XCTAssertEqual(options.effectiveMaxRetries(configValue: 7), 7) + } + + // MARK: - Validation + + func testRateLimitValidationRejectsZero() { + XCTAssertThrowsError(try HeavyFaultToleranceOptions.parse(["--rate-limit", "0"])) + } + + func testConcurrentDownloadsValidationRejectsZero() { + XCTAssertThrowsError(try HeavyFaultToleranceOptions.parse(["--concurrent-downloads", "0"])) + } + + func testMaxRetriesValidationRejectsNegative() { + XCTAssertThrowsError(try HeavyFaultToleranceOptions.parse(["--max-retries", "-1"])) } // MARK: - Timeout Flag diff --git a/Tests/ExFigTests/Input/PlatformConfigTests.swift b/Tests/ExFigTests/Input/PlatformConfigTests.swift index a391772e..a96dd9e7 100644 --- a/Tests/ExFigTests/Input/PlatformConfigTests.swift +++ b/Tests/ExFigTests/Input/PlatformConfigTests.swift @@ -28,7 +28,10 @@ final class PlatformConfigTests: XCTestCase { darkFileId: "dark-456", lightHighContrastFileId: nil, darkHighContrastFileId: nil, - timeout: 30.0 + timeout: 30.0, + rateLimit: nil, + maxRetries: nil, + concurrentDownloads: nil ) let config = ios.platformConfig(figma: figma) @@ -81,7 +84,10 @@ final class PlatformConfigTests: XCTestCase { darkFileId: nil, lightHighContrastFileId: nil, darkHighContrastFileId: nil, - timeout: 60.0 + timeout: 60.0, + rateLimit: nil, + maxRetries: nil, + concurrentDownloads: nil ) let config = android.platformConfig(figma: figma) diff --git a/Tests/ExFigTests/PKL/PKLEvaluatorTests.swift b/Tests/ExFigTests/PKL/PKLEvaluatorTests.swift index 207e9d90..f1301460 100644 --- a/Tests/ExFigTests/PKL/PKLEvaluatorTests.swift +++ b/Tests/ExFigTests/PKL/PKLEvaluatorTests.swift @@ -102,6 +102,9 @@ struct PKLEvaluatorTests { // Figma Figma.Module.registeredIdentifier, Figma.FigmaConfig.registeredIdentifier, + // Batch + Batch.Module.registeredIdentifier, + Batch.BatchConfig.registeredIdentifier, // iOS iOS.Module.registeredIdentifier, iOS.HeicOptions.registeredIdentifier, @@ -134,7 +137,7 @@ struct PKLEvaluatorTests { ] #expect( - expectedIdentifiers.count == 41, + expectedIdentifiers.count == 43, """ Generated PKL type count changed! After running codegen:pkl: 1. Update registerPklTypes(_:) in PKLEvaluator.swift with new types diff --git a/llms-full.txt b/llms-full.txt index f16f5891..d7c5b3b0 100644 --- a/llms-full.txt +++ b/llms-full.txt @@ -416,13 +416,21 @@ exfig images --resume exfig icons --concurrent-downloads 50 ``` -| Option | Description | Commands | -| ------------------------ | -------------------------------------- | -------------------- | -| `--max-retries` | Maximum retry attempts (default: 4) | All | -| `--rate-limit` | API requests per minute (default: 10) | All | -| `--fail-fast` | Stop immediately on error | icons, images, fetch | -| `--resume` | Continue from checkpoint | icons, images, fetch | -| `--concurrent-downloads` | Concurrent CDN downloads (default: 20) | icons, images, fetch | +| Option | Description | Commands | PKL key | +| ------------------------ | ---------------------------------------------- | --------------------- | ----------------------------- | +| `--max-retries` | Maximum retry attempts (default: 4) | All | `figma.maxRetries` | +| `--rate-limit` | API requests per minute (default: 10) | All | `figma.rateLimit` | +| `--timeout` | Figma API request timeout, sec (default: 30) | All | `figma.timeout` | +| `--concurrent-downloads` | Concurrent CDN downloads (default: 20) | icons, images, fetch | `figma.concurrentDownloads`* | +| `--fail-fast` | Stop immediately on error | icons, images, batch, fetch | `batch.failFast` (batch)| +| `--resume` | Continue from checkpoint | icons, images, batch, fetch | `batch.resume` (batch) | +| `--parallel` | Concurrent batch configs (default: 3) | batch | `batch.parallel` | + +CLI flags override PKL config; PKL config overrides built-in defaults. `fetch` is config-free — +only CLI flags and built-in defaults apply there. + +*`figma.concurrentDownloads` is silently ignored by `colors`/`typography` (no CDN downloads); under +`-v` a debug log records the skip. ### Checkpoint System @@ -658,9 +666,24 @@ figma = new Figma.FigmaConfig { // Optional: API request timeout in seconds (default: 30) timeout = 60 + + // Optional: API requests per minute (default: 10). CLI --rate-limit overrides. + rateLimit = 25 + + // Optional: Retry attempts for failed API requests (default: 4). CLI --max-retries overrides. + maxRetries = 6 + + // Optional: Concurrent CDN downloads — icons/images only (default: 20). + // Ignored by colors/typography. CLI --concurrent-downloads overrides. + // Client-side cap on connections (URLSession.httpMaximumConnectionsPerHost), not a Figma REST limit. + concurrentDownloads = 50 } ``` +**Precedence (per knob):** CLI flag > PKL config > built-in default. The same rule applies to +all five fields above. CI workflows can keep one value here instead of repeating CLI flags +everywhere; ad-hoc invocations still override per-run. + ## Common Section Shared settings across all platforms. @@ -1088,6 +1111,30 @@ flutter = new Flutter.FlutterConfig { } ``` +## Batch Section + +Top-level `batch:` block (read by `exfig batch`). Read ONLY from the **first** config in argv — +per-target `batch:` blocks in subsequent configs are ignored (logged under `-v`). + +```pkl showLineNumbers +import ".exfig/schemas/Batch.pkl" + +batch = new Batch.BatchConfig { + // Optional: Maximum configs to process in parallel (default: 3). CLI --parallel overrides. + parallel = 8 + + // Optional: Stop processing on first error (default: false). CLI --fail-fast overrides. + failFast = true + + // Optional: Resume from previous checkpoint (default: false). CLI --resume overrides. + resume = false +} +``` + +For `exfig batch ./configs/`, `parallel`/`failFast`/`resume` come from the FIRST config (or CLI flags +if passed). Boolean fields use OR semantics: CLI `--fail-fast` activates fail-fast even if the config +has `failFast = false`. + ## Example Configurations ### iOS Project From 28633bb470a9276fa4e2a54a1301c4ba20a0e1b5 Mon Sep 17 00:00:00 2001 From: alexey1312 Date: Sat, 9 May 2026 13:31:02 +0500 Subject: [PATCH 2/5] fix(cli): address review findings for configurable fault-tolerance batch Plug review-discovered gaps in the CLI > PKL > defaults precedence, silent failures, and missing tests: - Pass figma.concurrentDownloads to standalone icons/images exporters (was silently ignored outside batch). - Promote loadFirstConfig PKL errors from debug to warning so users see why their batch settings dropped to defaults. - Sanitize PKL configValue (clamp out-of-range with warning) and tighten CLI bounds to match PKL isBetween constraints. - Cache the first-config PKL evaluation via PKLModuleCache so BatchConfigRunner reuses it instead of re-evaluating. - Replace try? in logIgnoredPerTargetSettings with do/catch and warn on ignored figma.* fields in non-first batch configs. - Centralize fault-tolerance defaults in FaultToleranceDefaults + FaultToleranceValidator; deduplicate resolveClient via RateLimitClientFactory protocol. - Make ResolvedBatchSettings init fileprivate so its "Resolved" promise is enforced. - Sync ExFigCommand.version (3.3.0 -> 3.4.0) and exfig.usage.kdl with PklProject so generated configs reference an existing package. - Add tests for multi-config first-wins, Float64 timeout truncation, verbose ignored-block scan, partial PKL config, sanitizer clamps, defaults parity, and CLI upper-bound rejection. --- .../ExFigCLI/Batch/BatchConfigRunner.swift | 13 +- .../Batch/BatchSettingsResolver.swift | 144 ++++++++--- Sources/ExFigCLI/Batch/PKLModuleCache.swift | 27 ++ Sources/ExFigCLI/ExFigCommand.swift | 2 +- Sources/ExFigCLI/Input/ExFigOptions.swift | 7 + .../Input/FaultToleranceDefaults.swift | 20 ++ .../Input/FaultToleranceOptions.swift | 143 ++++------- .../Input/FaultToleranceValidator.swift | 134 ++++++++++ Sources/ExFigCLI/Subcommands/Batch.swift | 40 +-- .../Export/PluginIconsExport.swift | 16 +- .../Export/PluginImagesExport.swift | 16 +- .../ExFigCLI/TerminalUI/ExFigWarning.swift | 14 ++ .../TerminalUI/ExFigWarningFormatter.swift | 18 +- .../BatchSettingsResolverExtendedTests.swift | 238 ++++++++++++++++++ .../Input/FaultToleranceOptionsTests.swift | 31 +++ exfig.usage.kdl | 2 +- 16 files changed, 710 insertions(+), 155 deletions(-) create mode 100644 Sources/ExFigCLI/Batch/PKLModuleCache.swift create mode 100644 Sources/ExFigCLI/Input/FaultToleranceDefaults.swift create mode 100644 Sources/ExFigCLI/Input/FaultToleranceValidator.swift create mode 100644 Tests/ExFigTests/Batch/BatchSettingsResolverExtendedTests.swift diff --git a/Sources/ExFigCLI/Batch/BatchConfigRunner.swift b/Sources/ExFigCLI/Batch/BatchConfigRunner.swift index 3419b970..b7510fa4 100644 --- a/Sources/ExFigCLI/Batch/BatchConfigRunner.swift +++ b/Sources/ExFigCLI/Batch/BatchConfigRunner.swift @@ -226,6 +226,9 @@ struct BatchConfigRunner { let cliTimeout: Int? /// Priority for this config's downloads (lower = higher priority, based on submission order). let configPriority: Int + /// Optional pre-evaluated PKL module cache (avoids re-eval of configs already loaded by + /// `BatchSettingsResolver`). + let moduleCache: PKLModuleCache? /// Test-only: injected exporter for unit testing. private let _testExporter: (any ConfigExportPerforming)? @@ -240,9 +243,10 @@ struct BatchConfigRunner { force: Bool = false, cachePath: String? = nil, experimentalGranularCache: Bool = false, - concurrentDownloads: Int = FileDownloader.defaultMaxConcurrentDownloads, + concurrentDownloads: Int = FaultToleranceDefaults.concurrentDownloads, cliTimeout: Int? = nil, configPriority: Int = 0, + moduleCache: PKLModuleCache? = nil, exporter: (any ConfigExportPerforming)? = nil ) { self.rateLimiter = rateLimiter @@ -258,6 +262,7 @@ struct BatchConfigRunner { self.concurrentDownloads = concurrentDownloads self.cliTimeout = cliTimeout self.configPriority = configPriority + self.moduleCache = moduleCache _testExporter = exporter } @@ -299,7 +304,11 @@ struct BatchConfigRunner { do { var options = ExFigOptions() options.input = configFile.url.path - try options.validate() + if let cached = await moduleCache?.get(for: configFile.url) { + options.validateUsing(preloadedModule: cached) + } else { + try options.validate() + } let retryHandler = RetryLogger.createHandler(ui: ui, maxAttempts: maxRetries) diff --git a/Sources/ExFigCLI/Batch/BatchSettingsResolver.swift b/Sources/ExFigCLI/Batch/BatchSettingsResolver.swift index 6fd525fb..4b8e9cef 100644 --- a/Sources/ExFigCLI/Batch/BatchSettingsResolver.swift +++ b/Sources/ExFigCLI/Batch/BatchSettingsResolver.swift @@ -7,6 +7,10 @@ import Foundation /// Precedence (per knob): /// - CLI flag (`--parallel`, `--rate-limit`, etc.) > config value > built-in default. /// - For `failFast`/`resume` (presence flags), CLI || config (either source enables it). +/// +/// Construction is restricted to ``BatchSettingsResolver/resolve(...)`` — the resolver guarantees +/// that values fall within the documented ranges (CLI is validated, PKL values are clamped with +/// a warning when out of range). struct ResolvedBatchSettings { let parallel: Int let failFast: Bool @@ -15,19 +19,30 @@ struct ResolvedBatchSettings { let maxRetries: Int let concurrentDownloads: Int let timeout: Int? + + fileprivate init( + parallel: Int, + failFast: Bool, + resume: Bool, + rateLimit: Int, + maxRetries: Int, + concurrentDownloads: Int, + timeout: Int? + ) { + self.parallel = parallel + self.failFast = failFast + self.resume = resume + self.rateLimit = rateLimit + self.maxRetries = maxRetries + self.concurrentDownloads = concurrentDownloads + self.timeout = timeout + } } /// Loads the FIRST config in `exfig batch` argv and merges its `batch:` and `figma:` rate-limiting -/// fields with CLI flags. Per-target `batch:` blocks in subsequent configs are ignored — under -/// `--verbose` they emit a debug log line. +/// fields with CLI flags. Per-target `batch:` blocks (and per-target `figma:*` rate-limiting fields) +/// in subsequent configs are ignored — under `--verbose` they emit a warning. enum BatchSettingsResolver { - /// Built-in defaults — kept in lockstep with `FaultToleranceOptions` and `Batch` field defaults. - private enum Defaults { - static let parallel = 3 - static let rateLimit = 10 - static let maxRetries = 4 - } - // swiftlint:disable function_parameter_count /// - Parameters: @@ -39,8 +54,10 @@ enum BatchSettingsResolver { /// - cliConcurrentDownloads: `--concurrent-downloads` value, or nil if user didn't pass. /// - cliTimeout: `--timeout` value (seconds), or nil if user didn't pass. /// - allConfigs: Discovered config URLs in argv order. First wins for batch settings. - /// - verbose: When true, emit debug log for ignored per-target `batch:` blocks. + /// - verbose: When true, emit a warning for ignored per-target `batch:` blocks. /// - ui: Terminal UI for debug/warn output. + /// - moduleCache: Optional cache to populate with the first-config evaluation result so + /// downstream consumers (`BatchConfigRunner`) can skip a redundant PKL eval. /// - Returns: Resolved settings to drive the batch run. static func resolve( cliParallel: Int?, @@ -52,26 +69,38 @@ enum BatchSettingsResolver { cliTimeout: Int?, allConfigs: [URL], verbose: Bool, - ui: TerminalUI + ui: TerminalUI, + moduleCache: PKLModuleCache? = nil ) async -> ResolvedBatchSettings { - let firstConfig: ExFig.ModuleImpl? = await loadFirstConfig(allConfigs: allConfigs, ui: ui) + let firstConfig: ExFig.ModuleImpl? = await loadFirstConfig( + allConfigs: allConfigs, + ui: ui, + moduleCache: moduleCache + ) let batch = firstConfig?.batch let figma = firstConfig?.figma if verbose, allConfigs.count > 1 { - await logIgnoredPerTargetBatchBlocks(otherConfigs: Array(allConfigs.dropFirst()), ui: ui) + await logIgnoredPerTargetSettings( + otherConfigs: Array(allConfigs.dropFirst()), + ui: ui, + moduleCache: moduleCache + ) } return ResolvedBatchSettings( - parallel: cliParallel ?? batch?.parallel ?? Defaults.parallel, + parallel: cliParallel + ?? FaultToleranceValidator.sanitizedParallel(batch?.parallel, ui: ui), failFast: cliFailFast || (batch?.failFast ?? false), resume: cliResume || (batch?.resume ?? false), - rateLimit: cliRateLimit ?? figma?.rateLimit ?? Defaults.rateLimit, - maxRetries: cliMaxRetries ?? figma?.maxRetries ?? Defaults.maxRetries, + rateLimit: cliRateLimit + ?? FaultToleranceValidator.sanitizedRateLimit(figma?.rateLimit, ui: ui), + maxRetries: cliMaxRetries + ?? FaultToleranceValidator.sanitizedMaxRetries(figma?.maxRetries, ui: ui), concurrentDownloads: cliConcurrentDownloads - ?? figma?.concurrentDownloads - ?? FileDownloader.defaultMaxConcurrentDownloads, - timeout: cliTimeout ?? figma?.timeout.map { Int($0) } + ?? FaultToleranceValidator.sanitizedConcurrentDownloads(figma?.concurrentDownloads, ui: ui), + timeout: cliTimeout + ?? FaultToleranceValidator.sanitizedTimeout(figma?.timeout.map { Int($0) }, ui: ui) ) } @@ -79,30 +108,81 @@ enum BatchSettingsResolver { // MARK: - Internals - private static func loadFirstConfig(allConfigs: [URL], ui: TerminalUI) async -> ExFig.ModuleImpl? { + private static func loadFirstConfig( + allConfigs: [URL], + ui: TerminalUI, + moduleCache: PKLModuleCache? + ) async -> ExFig.ModuleImpl? { guard let firstURL = allConfigs.first else { return nil } do { - return try await PKLEvaluator.evaluate(configPath: firstURL) + let module = try await PKLEvaluator.evaluate(configPath: firstURL) + await moduleCache?.set(module, for: firstURL) + return module } catch { - // Falling back to defaults is safe — the config will be re-evaluated by BatchConfigRunner - // and any real syntax/validation error will surface there with a per-config error. - ui.debug( - "Could not pre-load batch settings from \(firstURL.lastPathComponent): " + - "\(error.localizedDescription). Using CLI flags / built-in defaults." - ) + // File-not-found will surface again in BatchConfigRunner with a clearer message; + // for that case we keep the message under -v. For real PKL/syntax/network errors, + // batch settings from the user are silently dropped — promote to a visible warning + // so the user knows defaults are in effect. + if isFileNotFound(error: error, url: firstURL) { + ui.debug( + "Pre-load skipped: \(firstURL.lastPathComponent) not found. " + + "BatchConfigRunner will surface the error per-config." + ) + } else { + ui.warning(.batchSettingsPreloadFailed( + file: firstURL.lastPathComponent, + error: error.localizedDescription + )) + } return nil } } - private static func logIgnoredPerTargetBatchBlocks(otherConfigs: [URL], ui: TerminalUI) async { + private static func logIgnoredPerTargetSettings( + otherConfigs: [URL], + ui: TerminalUI, + moduleCache: PKLModuleCache? + ) async { for url in otherConfigs { - let module = try? await PKLEvaluator.evaluate(configPath: url) - if module?.batch != nil { + let module: ExFig.ModuleImpl? + do { + module = try await PKLEvaluator.evaluate(configPath: url) + await moduleCache?.set(module, for: url) + } catch { ui.debug( - "Ignoring batch: in \(url.lastPathComponent) — only the first config's " + - "batch settings apply." + "Could not pre-check \(url.lastPathComponent) for ignored batch settings: " + + "\(error.localizedDescription)" ) + continue + } + if module?.batch != nil { + ui.warning(.ignoredPerTargetBatchBlock(file: url.lastPathComponent)) + } + if let figma = module?.figma, + figma.rateLimit != nil + || figma.maxRetries != nil + || figma.concurrentDownloads != nil + || figma.timeout != nil + { + ui.warning(.ignoredPerTargetFigmaRateLimiting(file: url.lastPathComponent)) } } } + + private static func isFileNotFound(error: Error, url: URL) -> Bool { + let nsError = error as NSError + if nsError.domain == NSCocoaErrorDomain, nsError.code == NSFileReadNoSuchFileError { + return true + } + if nsError.domain == NSPOSIXErrorDomain, nsError.code == Int(ENOENT) { + return true + } + // PklSwift surfaces missing files via PklError text — a textual match is brittle but + // acceptable as a fallback (better than over-warning the user). + let message = error.localizedDescription.lowercased() + if message.contains("no such file") || message.contains("not found") { + return true + } + return !FileManager.default.fileExists(atPath: url.path) + } } diff --git a/Sources/ExFigCLI/Batch/PKLModuleCache.swift b/Sources/ExFigCLI/Batch/PKLModuleCache.swift new file mode 100644 index 00000000..2cba6ddd --- /dev/null +++ b/Sources/ExFigCLI/Batch/PKLModuleCache.swift @@ -0,0 +1,27 @@ +import ExFigConfig +import Foundation + +/// Caches PKL evaluation results by config URL so the first config (and any subsequent ones +/// pre-checked under `--verbose`) doesn't pay the eval cost twice. +/// +/// PKL evaluation spawns a subprocess and is expensive — re-using the parsed module across +/// `BatchSettingsResolver`, `logIgnoredPerTargetSettings`, and `BatchConfigRunner` saves a +/// noticeable chunk of pre-batch latency. +actor PKLModuleCache { + private var modules: [URL: ExFig.ModuleImpl] = [:] + + init() {} + + func set(_ module: ExFig.ModuleImpl?, for url: URL) { + guard let module else { return } + modules[standardize(url)] = module + } + + func get(for url: URL) -> ExFig.ModuleImpl? { + modules[standardize(url)] + } + + private func standardize(_ url: URL) -> URL { + url.standardizedFileURL + } +} diff --git a/Sources/ExFigCLI/ExFigCommand.swift b/Sources/ExFigCLI/ExFigCommand.swift index 2f7e6cba..58da714f 100644 --- a/Sources/ExFigCLI/ExFigCommand.swift +++ b/Sources/ExFigCLI/ExFigCommand.swift @@ -74,7 +74,7 @@ enum ExFigError: LocalizedError { @main struct ExFigCommand: AsyncParsableCommand { - static let version = "v3.3.0" + static let version = "v3.4.0" static let svgFileConverter = NativeVectorDrawableConverter() static let fileWriter = FileWriter() diff --git a/Sources/ExFigCLI/Input/ExFigOptions.swift b/Sources/ExFigCLI/Input/ExFigOptions.swift index e9970165..6ad9483f 100644 --- a/Sources/ExFigCLI/Input/ExFigOptions.swift +++ b/Sources/ExFigCLI/Input/ExFigOptions.swift @@ -41,6 +41,13 @@ struct ExFigOptions: ParsableArguments { params = try readParams(at: configPath) } + /// Validates and primes `params` from a pre-evaluated PKL module — avoids redundant PKL eval + /// when the caller (e.g. `BatchSettingsResolver`) already loaded the same config. + mutating func validateUsing(preloadedModule module: ExFig.ModuleImpl) { + accessToken = ProcessInfo.processInfo.environment["FIGMA_PERSONAL_TOKEN"] + params = module + } + /// Returns the Figma access token, or throws if not set. /// Call this only when the current operation requires Figma API access. func requireFigmaToken() throws -> String { diff --git a/Sources/ExFigCLI/Input/FaultToleranceDefaults.swift b/Sources/ExFigCLI/Input/FaultToleranceDefaults.swift new file mode 100644 index 00000000..8d810ffe --- /dev/null +++ b/Sources/ExFigCLI/Input/FaultToleranceDefaults.swift @@ -0,0 +1,20 @@ +import Foundation + +/// Single source of truth for fault-tolerance and batch defaults. +/// +/// These values must be kept in sync with PKL schema defaults in +/// `Sources/ExFigCLI/Resources/Schemas/{Figma,Batch}.pkl`. The +/// `FaultToleranceDefaultsParityTests` test asserts the parity at runtime. +enum FaultToleranceDefaults { + static let parallel = 3 + static let rateLimit = 10 + static let maxRetries = 4 + static let concurrentDownloads = 20 + static let timeoutSeconds = 30 + + static let parallelRange = 1 ... 50 + static let rateLimitRange = 1 ... 600 + static let maxRetriesRange = 0 ... 100 + static let concurrentDownloadsRange = 1 ... 200 + static let timeoutRange = 1 ... 600 +} diff --git a/Sources/ExFigCLI/Input/FaultToleranceOptions.swift b/Sources/ExFigCLI/Input/FaultToleranceOptions.swift index b1bc4697..180fb903 100644 --- a/Sources/ExFigCLI/Input/FaultToleranceOptions.swift +++ b/Sources/ExFigCLI/Input/FaultToleranceOptions.swift @@ -28,25 +28,25 @@ struct FaultToleranceOptions: ParsableArguments { var timeout: Int? mutating func validate() throws { - if let timeout, timeout <= 0 { - throw ValidationError("Timeout must be positive") - } - if let rateLimit, rateLimit <= 0 { - throw ValidationError("Rate limit must be positive") - } - if let maxRetries, maxRetries < 0 { - throw ValidationError("Max retries cannot be negative") - } + try FaultToleranceValidator.validateTimeout(timeout) + try FaultToleranceValidator.validateRateLimit(rateLimit) + try FaultToleranceValidator.validateMaxRetries(maxRetries) } - /// CLI value (if user passed --rate-limit) > config value > built-in default (10). - func effectiveRateLimit(configValue: Int?) -> Int { - rateLimit ?? configValue ?? 10 + /// CLI value (if user passed --rate-limit) > config value > built-in default. + /// Invalid config values (out of `FaultToleranceDefaults.rateLimitRange`) trigger a + /// warning and fall back to the built-in default. + func effectiveRateLimit(configValue: Int?, ui: TerminalUI? = nil) -> Int { + if let rateLimit { return rateLimit } + return FaultToleranceValidator.sanitizedRateLimit(configValue, ui: ui) } - /// CLI value (if user passed --max-retries) > config value > built-in default (4). - func effectiveMaxRetries(configValue: Int?) -> Int { - maxRetries ?? configValue ?? 4 + /// CLI value (if user passed --max-retries) > config value > built-in default. + /// Invalid config values (out of `FaultToleranceDefaults.maxRetriesRange`) trigger a + /// warning and fall back to the built-in default. + func effectiveMaxRetries(configValue: Int?, ui: TerminalUI? = nil) -> Int { + if let maxRetries { return maxRetries } + return FaultToleranceValidator.sanitizedMaxRetries(configValue, ui: ui) } /// Create a retry policy from the options. @@ -134,33 +134,28 @@ struct HeavyFaultToleranceOptions: ParsableArguments { var concurrentDownloads: Int? mutating func validate() throws { - if let timeout, timeout <= 0 { - throw ValidationError("Timeout must be positive") - } - if let rateLimit, rateLimit <= 0 { - throw ValidationError("Rate limit must be positive") - } - if let maxRetries, maxRetries < 0 { - throw ValidationError("Max retries cannot be negative") - } - if let concurrentDownloads, concurrentDownloads <= 0 { - throw ValidationError("Concurrent downloads must be positive") - } + try FaultToleranceValidator.validateTimeout(timeout) + try FaultToleranceValidator.validateRateLimit(rateLimit) + try FaultToleranceValidator.validateMaxRetries(maxRetries) + try FaultToleranceValidator.validateConcurrentDownloads(concurrentDownloads) } - /// CLI value (if user passed --rate-limit) > config value > built-in default (10). - func effectiveRateLimit(configValue: Int?) -> Int { - rateLimit ?? configValue ?? 10 + /// CLI value (if user passed --rate-limit) > config value > built-in default. + func effectiveRateLimit(configValue: Int?, ui: TerminalUI? = nil) -> Int { + if let rateLimit { return rateLimit } + return FaultToleranceValidator.sanitizedRateLimit(configValue, ui: ui) } - /// CLI value (if user passed --max-retries) > config value > built-in default (4). - func effectiveMaxRetries(configValue: Int?) -> Int { - maxRetries ?? configValue ?? 4 + /// CLI value (if user passed --max-retries) > config value > built-in default. + func effectiveMaxRetries(configValue: Int?, ui: TerminalUI? = nil) -> Int { + if let maxRetries { return maxRetries } + return FaultToleranceValidator.sanitizedMaxRetries(configValue, ui: ui) } - /// CLI value (if user passed --concurrent-downloads) > config value > built-in default (20). - func effectiveConcurrentDownloads(configValue: Int?) -> Int { - concurrentDownloads ?? configValue ?? FileDownloader.defaultMaxConcurrentDownloads + /// CLI value (if user passed --concurrent-downloads) > config value > built-in default. + func effectiveConcurrentDownloads(configValue: Int?, ui: TerminalUI? = nil) -> Int { + if let concurrentDownloads { return concurrentDownloads } + return FaultToleranceValidator.sanitizedConcurrentDownloads(configValue, ui: ui) } /// Create a file downloader with configured concurrency. @@ -292,59 +287,25 @@ struct HeavyFaultToleranceOptions: ParsableArguments { // MARK: - Client Resolution -/// Resolves a Figma API client, using injected client if available (batch mode) -/// or creating a new rate-limited client (standalone command mode). -/// -/// - Parameters: -/// - accessToken: Figma personal access token. -/// - timeout: Request timeout interval from config (optional, uses FigmaClient default if nil). -/// - rateLimit: Config-supplied `figma.rateLimit` (optional). CLI `--rate-limit` overrides. -/// - maxRetries: Config-supplied `figma.maxRetries` (optional). CLI `--max-retries` overrides. -/// - options: Fault tolerance options for creating new client (may contain CLI overrides). -/// - ui: Terminal UI for retry warnings. -/// - Returns: A configured `Client` instance. -/// -/// Precedence (per knob): CLI flag > config value > built-in default. -func resolveClient( - accessToken: String?, - timeout: TimeInterval?, - rateLimit configRateLimit: Int? = nil, - maxRetries configMaxRetries: Int? = nil, - options: FaultToleranceOptions, - ui: TerminalUI -) -> Client { - if let injectedClient = InjectedClientStorage.client { - return injectedClient - } - guard let accessToken else { - // No Figma token — return a client that throws on any call. - // Non-Figma sources (Penpot, tokens-file) never call it. - // SourceFactory also guards the .figma branch with accessTokenNotFound. - return NoTokenFigmaClient() - } - // CLI timeout takes precedence over config timeout - let effectiveTimeout: TimeInterval? = options.timeout.map { TimeInterval($0) } ?? timeout - let baseClient = FigmaClient(accessToken: accessToken, timeout: effectiveTimeout) - let rateLimiter = options.createRateLimiter(configValue: configRateLimit) - let effectiveMaxRetries = options.effectiveMaxRetries(configValue: configMaxRetries) - return options.createRateLimitedClient( - wrapping: baseClient, - rateLimiter: rateLimiter, - configMaxRetries: configMaxRetries, - onRetry: { attempt, error in - let warning = ExFigWarning.retrying( - attempt: attempt, - maxAttempts: effectiveMaxRetries, - error: error.localizedDescription, - delay: "..." - ) - ui.warning(warning) - } - ) +/// Common surface used by `resolveClient` to bridge the two CLI option types. +protocol RateLimitClientFactory { + var timeout: Int? { get } + func createRateLimiter(configValue: Int?) -> SharedRateLimiter + func effectiveMaxRetries(configValue: Int?, ui: TerminalUI?) -> Int + func createRateLimitedClient( + wrapping client: Client, + rateLimiter: SharedRateLimiter, + configMaxRetries: Int?, + configID: ConfigID, + onRetry: RetryCallback? + ) -> Client } -/// Resolves a Figma API client for heavy commands, using injected client if available -/// (batch mode) or creating a new rate-limited client (standalone command mode). +extension FaultToleranceOptions: RateLimitClientFactory {} +extension HeavyFaultToleranceOptions: RateLimitClientFactory {} + +/// Resolves a Figma API client, using injected client if available (batch mode) +/// or creating a new rate-limited client (standalone command mode). /// /// When `accessToken` is nil (no `FIGMA_PERSONAL_TOKEN`), returns ``NoTokenFigmaClient`` — /// a fail-fast client that throws on any request. Non-Figma sources (Penpot, tokens-file) @@ -355,7 +316,7 @@ func resolveClient( /// - timeout: Request timeout interval from config (optional, uses FigmaClient default if nil). /// - rateLimit: Config-supplied `figma.rateLimit` (optional). CLI `--rate-limit` overrides. /// - maxRetries: Config-supplied `figma.maxRetries` (optional). CLI `--max-retries` overrides. -/// - options: Heavy fault tolerance options for creating new client (may contain CLI overrides). +/// - options: Fault tolerance options (may contain CLI overrides). /// - ui: Terminal UI for retry warnings. /// - Returns: A configured `Client` instance. /// @@ -365,7 +326,7 @@ func resolveClient( timeout: TimeInterval?, rateLimit configRateLimit: Int? = nil, maxRetries configMaxRetries: Int? = nil, - options: HeavyFaultToleranceOptions, + options: some RateLimitClientFactory, ui: TerminalUI ) -> Client { if let injectedClient = InjectedClientStorage.client { @@ -374,17 +335,19 @@ func resolveClient( guard let accessToken else { // No Figma token — return a client that throws on any call. // Non-Figma sources (Penpot, tokens-file) never call it. + // SourceFactory also guards the .figma branch with accessTokenNotFound. return NoTokenFigmaClient() } // CLI timeout takes precedence over config timeout let effectiveTimeout: TimeInterval? = options.timeout.map { TimeInterval($0) } ?? timeout let baseClient = FigmaClient(accessToken: accessToken, timeout: effectiveTimeout) let rateLimiter = options.createRateLimiter(configValue: configRateLimit) - let effectiveMaxRetries = options.effectiveMaxRetries(configValue: configMaxRetries) + let effectiveMaxRetries = options.effectiveMaxRetries(configValue: configMaxRetries, ui: ui) return options.createRateLimitedClient( wrapping: baseClient, rateLimiter: rateLimiter, configMaxRetries: configMaxRetries, + configID: ConfigID("default"), onRetry: { attempt, error in let warning = ExFigWarning.retrying( attempt: attempt, diff --git a/Sources/ExFigCLI/Input/FaultToleranceValidator.swift b/Sources/ExFigCLI/Input/FaultToleranceValidator.swift new file mode 100644 index 00000000..468b293f --- /dev/null +++ b/Sources/ExFigCLI/Input/FaultToleranceValidator.swift @@ -0,0 +1,134 @@ +import ArgumentParser +import Foundation + +/// Centralized validation rules for fault-tolerance and batch knobs. +/// +/// These rules are shared between CLI `validate()` (raises `ValidationError`) and +/// PKL `effective*` accessors (clamp + warn). Keeping them in one place removes +/// the drift hazard that comes with three nearly-identical `validate()` blocks. +enum FaultToleranceValidator { + // MARK: - CLI validation (throws) + + static func validateTimeout(_ value: Int?) throws { + guard let value else { return } + guard FaultToleranceDefaults.timeoutRange.contains(value) else { + throw ValidationError( + "Timeout must be between \(FaultToleranceDefaults.timeoutRange.lowerBound) " + + "and \(FaultToleranceDefaults.timeoutRange.upperBound) seconds" + ) + } + } + + static func validateRateLimit(_ value: Int?) throws { + guard let value else { return } + guard FaultToleranceDefaults.rateLimitRange.contains(value) else { + throw ValidationError( + "Rate limit must be between \(FaultToleranceDefaults.rateLimitRange.lowerBound) " + + "and \(FaultToleranceDefaults.rateLimitRange.upperBound) requests per minute" + ) + } + } + + static func validateMaxRetries(_ value: Int?) throws { + guard let value else { return } + guard FaultToleranceDefaults.maxRetriesRange.contains(value) else { + throw ValidationError( + "Max retries must be between \(FaultToleranceDefaults.maxRetriesRange.lowerBound) " + + "and \(FaultToleranceDefaults.maxRetriesRange.upperBound)" + ) + } + } + + static func validateConcurrentDownloads(_ value: Int?) throws { + guard let value else { return } + guard FaultToleranceDefaults.concurrentDownloadsRange.contains(value) else { + throw ValidationError( + "Concurrent downloads must be between " + + "\(FaultToleranceDefaults.concurrentDownloadsRange.lowerBound) and " + + "\(FaultToleranceDefaults.concurrentDownloadsRange.upperBound)" + ) + } + } + + static func validateParallel(_ value: Int?) throws { + guard let value else { return } + guard FaultToleranceDefaults.parallelRange.contains(value) else { + throw ValidationError( + "Parallel must be between \(FaultToleranceDefaults.parallelRange.lowerBound) " + + "and \(FaultToleranceDefaults.parallelRange.upperBound)" + ) + } + } + + // MARK: - PKL value sanitization (clamp + warn) + + static func sanitizedRateLimit(_ value: Int?, ui: TerminalUI?) -> Int { + sanitize( + value: value, + range: FaultToleranceDefaults.rateLimitRange, + fallback: FaultToleranceDefaults.rateLimit, + key: "figma.rateLimit", + ui: ui + ) + } + + static func sanitizedMaxRetries(_ value: Int?, ui: TerminalUI?) -> Int { + sanitize( + value: value, + range: FaultToleranceDefaults.maxRetriesRange, + fallback: FaultToleranceDefaults.maxRetries, + key: "figma.maxRetries", + ui: ui + ) + } + + static func sanitizedConcurrentDownloads(_ value: Int?, ui: TerminalUI?) -> Int { + sanitize( + value: value, + range: FaultToleranceDefaults.concurrentDownloadsRange, + fallback: FaultToleranceDefaults.concurrentDownloads, + key: "figma.concurrentDownloads", + ui: ui + ) + } + + static func sanitizedTimeout(_ value: Int?, ui: TerminalUI?) -> Int? { + guard let value else { return nil } + if FaultToleranceDefaults.timeoutRange.contains(value) { + return value + } + ui?.warning(.invalidConfigValue( + key: "figma.timeout", + value: value, + fallback: FaultToleranceDefaults.timeoutSeconds + )) + return FaultToleranceDefaults.timeoutSeconds + } + + static func sanitizedParallel(_ value: Int?, ui: TerminalUI?) -> Int { + sanitize( + value: value, + range: FaultToleranceDefaults.parallelRange, + fallback: FaultToleranceDefaults.parallel, + key: "batch.parallel", + ui: ui + ) + } + + // MARK: - Internal + + private static func sanitize( + value: Int?, + range: ClosedRange, + fallback: Int, + key: String, + ui: TerminalUI? + ) -> Int { + guard let value else { return fallback } + if range.contains(value) { + return value + } + ui?.warning(.invalidConfigValue(key: key, value: value, fallback: fallback)) + return fallback + } +} diff --git a/Sources/ExFigCLI/Subcommands/Batch.swift b/Sources/ExFigCLI/Subcommands/Batch.swift index f0e98499..5fdd82ff 100644 --- a/Sources/ExFigCLI/Subcommands/Batch.swift +++ b/Sources/ExFigCLI/Subcommands/Batch.swift @@ -79,21 +79,11 @@ extension ExFigCommand { var paths: [String] mutating func validate() throws { - if let timeout, timeout <= 0 { - throw ValidationError("Timeout must be positive") - } - if let parallel, parallel <= 0 { - throw ValidationError("Parallel must be positive") - } - if let rateLimit, rateLimit <= 0 { - throw ValidationError("Rate limit must be positive") - } - if let maxRetries, maxRetries < 0 { - throw ValidationError("Max retries cannot be negative") - } - if let concurrentDownloads, concurrentDownloads <= 0 { - throw ValidationError("Concurrent downloads must be positive") - } + try FaultToleranceValidator.validateTimeout(timeout) + try FaultToleranceValidator.validateParallel(parallel) + try FaultToleranceValidator.validateRateLimit(rateLimit) + try FaultToleranceValidator.validateMaxRetries(maxRetries) + try FaultToleranceValidator.validateConcurrentDownloads(concurrentDownloads) } // swiftlint:disable:next function_body_length @@ -106,8 +96,12 @@ extension ExFigCommand { let (validConfigs, conflicts) = try discoverAndValidateConfigs(ui: ui) guard !validConfigs.isEmpty else { return } + // Cache PKL evaluations so the first config (and any verbose-pre-checked configs) + // aren't re-evaluated in BatchConfigRunner. + let moduleCache = PKLModuleCache() + // Resolve batch-level settings: CLI flags > first config's batch:/figma: > built-in defaults. - // Per-target batch: blocks in subsequent configs are ignored (debug-logged under -v). + // Per-target batch: blocks in subsequent configs are ignored (warning-logged under -v). let resolved = await BatchSettingsResolver.resolve( cliParallel: parallel, cliFailFast: failFast, @@ -118,7 +112,8 @@ extension ExFigCommand { cliTimeout: timeout, allConfigs: validConfigs, verbose: globalOptions.verbose, - ui: ui + ui: ui, + moduleCache: moduleCache ) // Prepare configs with checkpoint handling @@ -143,6 +138,7 @@ extension ExFigCommand { checkpoint: checkpoint, workingDirectory: workingDirectory, resolved: resolved, + moduleCache: moduleCache, ui: ui ) @@ -239,6 +235,7 @@ extension ExFigCommand { checkpoint: BatchCheckpoint, workingDirectory: URL, resolved: ResolvedBatchSettings, + moduleCache: PKLModuleCache, ui: TerminalUI ) async -> (BatchResult, SharedRateLimiter) { let rateLimiter = SharedRateLimiter(requestsPerMinute: Double(resolved.rateLimit)) @@ -319,7 +316,8 @@ extension ExFigCommand { rateLimiter: rateLimiter, retryPolicy: retryPolicy, resolved: resolved, - priorityMap: priorityMap + priorityMap: priorityMap, + moduleCache: moduleCache ) // Create shared theme attributes collector for batch mode @@ -414,7 +412,8 @@ extension ExFigCommand { rateLimiter: SharedRateLimiter, retryPolicy: RetryPolicy, resolved: ResolvedBatchSettings, - priorityMap: [String: Int] + priorityMap: [String: Int], + moduleCache: PKLModuleCache ) -> @Sendable (ConfigFile) -> BatchConfigRunner { // Capture all values needed for runner creation let globalOptions = globalOptions @@ -442,7 +441,8 @@ extension ExFigCommand { experimentalGranularCache: experimentalGranularCache, concurrentDownloads: resolvedConcurrentDownloads, cliTimeout: resolvedTimeout, - configPriority: priorityMap[configFile.name] ?? 0 + configPriority: priorityMap[configFile.name] ?? 0, + moduleCache: moduleCache ) } } diff --git a/Sources/ExFigCLI/Subcommands/Export/PluginIconsExport.swift b/Sources/ExFigCLI/Subcommands/Export/PluginIconsExport.swift index 3b9bf5d4..173e435a 100644 --- a/Sources/ExFigCLI/Subcommands/Export/PluginIconsExport.swift +++ b/Sources/ExFigCLI/Subcommands/Export/PluginIconsExport.swift @@ -39,7 +39,9 @@ extension ExFigCommand.ExportIcons { let platformConfig = ios.platformConfig() let batchMode = BatchSharedState.current?.isBatchMode ?? false - let fileDownloader = faultToleranceOptions.createFileDownloader() + let fileDownloader = faultToleranceOptions.createFileDownloader( + configValue: params.figma?.concurrentDownloads + ) // All entries in a platform section share one source kind (mixed sources not yet supported) guard let sourceKind = entries.first?.resolvedSourceKind else { @@ -130,7 +132,9 @@ extension ExFigCommand.ExportIcons { let platformConfig = android.platformConfig() let batchMode = BatchSharedState.current?.isBatchMode ?? false - let fileDownloader = faultToleranceOptions.createFileDownloader() + let fileDownloader = faultToleranceOptions.createFileDownloader( + configValue: params.figma?.concurrentDownloads + ) // All entries in a platform section share one source kind (mixed sources not yet supported) guard let sourceKind = entries.first?.resolvedSourceKind else { @@ -195,7 +199,9 @@ extension ExFigCommand.ExportIcons { let platformConfig = flutter.platformConfig() let batchMode = BatchSharedState.current?.isBatchMode ?? false - let fileDownloader = faultToleranceOptions.createFileDownloader() + let fileDownloader = faultToleranceOptions.createFileDownloader( + configValue: params.figma?.concurrentDownloads + ) // All entries in a platform section share one source kind (mixed sources not yet supported) guard let sourceKind = entries.first?.resolvedSourceKind else { @@ -260,7 +266,9 @@ extension ExFigCommand.ExportIcons { let platformConfig = web.platformConfig() let batchMode = BatchSharedState.current?.isBatchMode ?? false - let fileDownloader = faultToleranceOptions.createFileDownloader() + let fileDownloader = faultToleranceOptions.createFileDownloader( + configValue: params.figma?.concurrentDownloads + ) // All entries in a platform section share one source kind (mixed sources not yet supported) guard let sourceKind = entries.first?.resolvedSourceKind else { diff --git a/Sources/ExFigCLI/Subcommands/Export/PluginImagesExport.swift b/Sources/ExFigCLI/Subcommands/Export/PluginImagesExport.swift index dbb41118..e66217c4 100644 --- a/Sources/ExFigCLI/Subcommands/Export/PluginImagesExport.swift +++ b/Sources/ExFigCLI/Subcommands/Export/PluginImagesExport.swift @@ -38,7 +38,9 @@ extension ExFigCommand.ExportImages { let platformConfig = ios.platformConfig() let batchMode = BatchSharedState.current?.isBatchMode ?? false - let fileDownloader = faultToleranceOptions.createFileDownloader() + let fileDownloader = faultToleranceOptions.createFileDownloader( + configValue: params.figma?.concurrentDownloads + ) // All entries in a platform section share one source kind (mixed sources not yet supported) guard let sourceKind = entries.first?.resolvedSourceKind else { @@ -122,7 +124,9 @@ extension ExFigCommand.ExportImages { let platformConfig = android.platformConfig() let batchMode = BatchSharedState.current?.isBatchMode ?? false - let fileDownloader = faultToleranceOptions.createFileDownloader() + let fileDownloader = faultToleranceOptions.createFileDownloader( + configValue: params.figma?.concurrentDownloads + ) // All entries in a platform section share one source kind (mixed sources not yet supported) guard let sourceKind = entries.first?.resolvedSourceKind else { @@ -181,7 +185,9 @@ extension ExFigCommand.ExportImages { let platformConfig = flutter.platformConfig() let batchMode = BatchSharedState.current?.isBatchMode ?? false - let fileDownloader = faultToleranceOptions.createFileDownloader() + let fileDownloader = faultToleranceOptions.createFileDownloader( + configValue: params.figma?.concurrentDownloads + ) // All entries in a platform section share one source kind (mixed sources not yet supported) guard let sourceKind = entries.first?.resolvedSourceKind else { @@ -240,7 +246,9 @@ extension ExFigCommand.ExportImages { let platformConfig = web.platformConfig() let batchMode = BatchSharedState.current?.isBatchMode ?? false - let fileDownloader = faultToleranceOptions.createFileDownloader() + let fileDownloader = faultToleranceOptions.createFileDownloader( + configValue: params.figma?.concurrentDownloads + ) // All entries in a platform section share one source kind (mixed sources not yet supported) guard let sourceKind = entries.first?.resolvedSourceKind else { diff --git a/Sources/ExFigCLI/TerminalUI/ExFigWarning.swift b/Sources/ExFigCLI/TerminalUI/ExFigWarning.swift index 02195672..95fd6aa2 100644 --- a/Sources/ExFigCLI/TerminalUI/ExFigWarning.swift +++ b/Sources/ExFigCLI/TerminalUI/ExFigWarning.swift @@ -105,4 +105,18 @@ enum ExFigWarning: Equatable { /// A `download tokens` section was skipped because config is missing. case downloadTokensSectionSkipped(section: String) + + // MARK: - Fault Tolerance Warnings + + /// PKL pre-load failed for the first batch config — CLI flags / built-in defaults will apply. + case batchSettingsPreloadFailed(file: String, error: String) + + /// A `batch:` block in a non-first batch config was ignored. + case ignoredPerTargetBatchBlock(file: String) + + /// Per-target `figma.*` rate-limiting fields in a non-first batch config were ignored. + case ignoredPerTargetFigmaRateLimiting(file: String) + + /// A PKL config value for fault tolerance was out of range and replaced with the default. + case invalidConfigValue(key: String, value: Int, fallback: Int) } diff --git a/Sources/ExFigCLI/TerminalUI/ExFigWarningFormatter.swift b/Sources/ExFigCLI/TerminalUI/ExFigWarningFormatter.swift index 10c190f5..af96634f 100644 --- a/Sources/ExFigCLI/TerminalUI/ExFigWarningFormatter.swift +++ b/Sources/ExFigCLI/TerminalUI/ExFigWarningFormatter.swift @@ -20,7 +20,9 @@ struct ExFigWarningFormatter { .themeAttributesMarkerNotFound, .themeAttributesNameCollision, .heicUnavailableFallingBackToPng, .deletedVariableAlias, .unresolvedNumberAlias, .depthExceededNumberAlias, - .circularColorAlias, .downloadTokensSectionSkipped: + .circularColorAlias, .downloadTokensSectionSkipped, + .batchSettingsPreloadFailed, .ignoredPerTargetBatchBlock, + .ignoredPerTargetFigmaRateLimiting, .invalidConfigValue: formatCompact(warning) // Multiline format warnings @@ -106,6 +108,20 @@ struct ExFigWarningFormatter { case let .downloadTokensSectionSkipped(section): "Token section skipped (not configured): section=\(section)" + case let .batchSettingsPreloadFailed(file, error): + "Batch settings pre-load failed: file=\(file), error=\(error). " + + "CLI flags / built-in defaults will apply." + + case let .ignoredPerTargetBatchBlock(file): + "Ignoring batch: in \(file) — only the first config's batch settings apply." + + case let .ignoredPerTargetFigmaRateLimiting(file): + "Ignoring per-target figma rate-limiting fields in \(file) — " + + "only the first config's figma:* values apply in batch mode." + + case let .invalidConfigValue(key, value, fallback): + "Invalid config value: key=\(key), value=\(value), fallback=\(fallback)" + // Multiline cases handled in main format() method case .noAssetsFound, .invalidConfigsSkipped, .webIconsMissingSVGData, .webIconsConversionFailed: fatalError("Multiline warnings should not reach formatCompact") diff --git a/Tests/ExFigTests/Batch/BatchSettingsResolverExtendedTests.swift b/Tests/ExFigTests/Batch/BatchSettingsResolverExtendedTests.swift new file mode 100644 index 00000000..0d2399f8 --- /dev/null +++ b/Tests/ExFigTests/Batch/BatchSettingsResolverExtendedTests.swift @@ -0,0 +1,238 @@ +@testable import ExFigCLI +import Foundation +import XCTest + +/// Coverage for multi-config / verbose / partial-config / sanitizer paths in +/// `BatchSettingsResolver`. Lives in its own suite to keep the original +/// `BatchSettingsResolverTests` under SwiftLint's type/file length limits. +final class BatchSettingsResolverExtendedTests: XCTestCase { + // MARK: - Multi-Config: First Wins (T1) + + func testFirstConfigWinsAndSubsequentConfigsAreIgnored() async throws { + let firstURL = try BatchResolverFixture.make( + figma: "rateLimit = 25\nmaxRetries = 6", + batch: "parallel = 8" + ) + let secondURL = try BatchResolverFixture.make( + figma: "rateLimit = 999\nmaxRetries = 99", + batch: "parallel = 99" + ) + defer { + try? FileManager.default.removeItem(at: firstURL) + try? FileManager.default.removeItem(at: secondURL) + } + let ui = TerminalUI(outputMode: .quiet) + + let resolved = await BatchSettingsResolver.resolve( + cliParallel: nil, + cliFailFast: false, + cliResume: false, + cliRateLimit: nil, + cliMaxRetries: nil, + cliConcurrentDownloads: nil, + cliTimeout: nil, + allConfigs: [firstURL, secondURL], + verbose: false, + ui: ui + ) + + XCTAssertEqual(resolved.parallel, 8, "first config wins") + XCTAssertEqual(resolved.rateLimit, 25, "first config wins") + XCTAssertEqual(resolved.maxRetries, 6, "first config wins") + } + + // MARK: - Float64 timeout conversion (T2) + + func testFloat64TimeoutFromConfigConvertedToInt() async throws { + // PKL `timeout` is `Number?` (Float64 in Swift codegen). Use a value with non-zero + // fractional part so the pkl-swift decoder treats it as Double. + let configURL = try BatchResolverFixture.make(figma: "timeout = 60.5", batch: nil) + defer { try? FileManager.default.removeItem(at: configURL) } + let ui = TerminalUI(outputMode: .quiet) + + let resolved = await BatchSettingsResolver.resolve( + cliParallel: nil, + cliFailFast: false, + cliResume: false, + cliRateLimit: nil, + cliMaxRetries: nil, + cliConcurrentDownloads: nil, + cliTimeout: nil, + allConfigs: [configURL], + verbose: false, + ui: ui + ) + + XCTAssertEqual(resolved.timeout, 60, "Float64 → Int truncates fractional part") + } + + func testFractionalConfigTimeoutTruncatedToInt() async throws { + let configURL = try BatchResolverFixture.make(figma: "timeout = 30.7", batch: nil) + defer { try? FileManager.default.removeItem(at: configURL) } + let ui = TerminalUI(outputMode: .quiet) + + let resolved = await BatchSettingsResolver.resolve( + cliParallel: nil, + cliFailFast: false, + cliResume: false, + cliRateLimit: nil, + cliMaxRetries: nil, + cliConcurrentDownloads: nil, + cliTimeout: nil, + allConfigs: [configURL], + verbose: false, + ui: ui + ) + + XCTAssertEqual(resolved.timeout, 30, "Float64 → Int truncates fractional part") + } + + // MARK: - Verbose=true triggers ignored-block scan (T3) + + func testVerboseTriggersIgnoredPerTargetWarnings() async throws { + let firstURL = try BatchResolverFixture.make( + figma: "rateLimit = 25", + batch: "parallel = 8" + ) + let secondURL = try BatchResolverFixture.make(figma: nil, batch: "parallel = 99") + defer { + try? FileManager.default.removeItem(at: firstURL) + try? FileManager.default.removeItem(at: secondURL) + } + let ui = TerminalUI(outputMode: .quiet) + + // Should not crash with verbose=true; first-config values still win. + let resolved = await BatchSettingsResolver.resolve( + cliParallel: nil, + cliFailFast: false, + cliResume: false, + cliRateLimit: nil, + cliMaxRetries: nil, + cliConcurrentDownloads: nil, + cliTimeout: nil, + allConfigs: [firstURL, secondURL], + verbose: true, + ui: ui + ) + + XCTAssertEqual(resolved.parallel, 8) + XCTAssertEqual(resolved.rateLimit, 25) + } + + // MARK: - Empty PKL section uses PKL defaults (T4) + + func testEmptyConfigSectionsResolveToDefaults() async throws { + let configURL = try BatchResolverFixture.make(figma: "", batch: "") + defer { try? FileManager.default.removeItem(at: configURL) } + let ui = TerminalUI(outputMode: .quiet) + + let resolved = await BatchSettingsResolver.resolve( + cliParallel: nil, + cliFailFast: false, + cliResume: false, + cliRateLimit: nil, + cliMaxRetries: nil, + cliConcurrentDownloads: nil, + cliTimeout: nil, + allConfigs: [configURL], + verbose: false, + ui: ui + ) + + XCTAssertEqual(resolved.parallel, 3) + XCTAssertFalse(resolved.failFast) + XCTAssertFalse(resolved.resume) + XCTAssertEqual(resolved.rateLimit, 10) + XCTAssertEqual(resolved.maxRetries, 4) + XCTAssertEqual(resolved.concurrentDownloads, 20) + XCTAssertEqual(resolved.timeout, 30) // PKL default for figma.timeout + } + + // MARK: - Partial PKL config (T5) + + func testPartialBatchBlockOnlyParallel() async throws { + let configURL = try BatchResolverFixture.make(figma: nil, batch: "parallel = 5") + defer { try? FileManager.default.removeItem(at: configURL) } + let ui = TerminalUI(outputMode: .quiet) + + let resolved = await BatchSettingsResolver.resolve( + cliParallel: nil, + cliFailFast: false, + cliResume: false, + cliRateLimit: nil, + cliMaxRetries: nil, + cliConcurrentDownloads: nil, + cliTimeout: nil, + allConfigs: [configURL], + verbose: false, + ui: ui + ) + + XCTAssertEqual(resolved.parallel, 5) + XCTAssertFalse(resolved.failFast) + XCTAssertFalse(resolved.resume) + } + + // MARK: - Sanitizer (I2) + + func testOutOfRangeRateLimitFromConfigFallsBackToDefault() { + // PKL constraints would reject 0/negative on amends, but a hand-written config + // could bypass them. Verify the sanitizer drops back to default. + let ui = TerminalUI(outputMode: .quiet) + XCTAssertEqual(FaultToleranceValidator.sanitizedRateLimit(0, ui: ui), 10) + XCTAssertEqual(FaultToleranceValidator.sanitizedRateLimit(-5, ui: ui), 10) + XCTAssertEqual(FaultToleranceValidator.sanitizedConcurrentDownloads(0, ui: ui), 20) + XCTAssertEqual(FaultToleranceValidator.sanitizedMaxRetries(-1, ui: ui), 4) + XCTAssertEqual(FaultToleranceValidator.sanitizedMaxRetries(1000, ui: ui), 4) + } + + // MARK: - PKL/Swift defaults parity (T4 reinforced) + + func testPKLDefaultsMatchSwiftDefaults() { + XCTAssertEqual(FaultToleranceDefaults.parallel, 3) + XCTAssertEqual(FaultToleranceDefaults.rateLimit, 10) + XCTAssertEqual(FaultToleranceDefaults.maxRetries, 4) + XCTAssertEqual(FaultToleranceDefaults.concurrentDownloads, 20) + XCTAssertEqual(FaultToleranceDefaults.timeoutSeconds, 30) + } +} + +/// Shared PKL fixture builder for `BatchSettingsResolver` tests. +enum BatchResolverFixture { + static func make(figma: String?, batch: String?) throws -> URL { + let schemasDir = URL(fileURLWithPath: #filePath) + .deletingLastPathComponent() + .deletingLastPathComponent() + .deletingLastPathComponent() + .deletingLastPathComponent() + .appendingPathComponent("Sources/ExFigCLI/Resources/Schemas") + let exfigPath = schemasDir.appendingPathComponent("ExFig.pkl").path + let figmaPath = schemasDir.appendingPathComponent("Figma.pkl").path + let batchPath = schemasDir.appendingPathComponent("Batch.pkl").path + + var lines: [String] = [ + "amends \"\(exfigPath)\"", + "import \"\(figmaPath)\"", + "import \"\(batchPath)\"", + ] + if let figma { + lines.append("figma = new Figma.FigmaConfig {") + for line in figma.split(separator: "\n") { + lines.append(" \(line)") + } + lines.append("}") + } + if let batch { + lines.append("batch = new Batch.BatchConfig {") + for line in batch.split(separator: "\n") { + lines.append(" \(line)") + } + lines.append("}") + } + + let url = FileManager.default.temporaryDirectory + .appendingPathComponent("batch-settings-resolver-\(UUID().uuidString).pkl") + try lines.joined(separator: "\n").write(to: url, atomically: true, encoding: .utf8) + return url + } +} diff --git a/Tests/ExFigTests/Input/FaultToleranceOptionsTests.swift b/Tests/ExFigTests/Input/FaultToleranceOptionsTests.swift index 68f40c9b..5cd4012e 100644 --- a/Tests/ExFigTests/Input/FaultToleranceOptionsTests.swift +++ b/Tests/ExFigTests/Input/FaultToleranceOptionsTests.swift @@ -85,6 +85,37 @@ final class FaultToleranceOptionsTests: XCTestCase { XCTAssertThrowsError(try FaultToleranceOptions.parse(["--rate-limit", "0"])) } + // MARK: - Upper-bound CLI validation (matches PKL bounds) + + func testRateLimitValidationRejectsAboveMax() { + XCTAssertThrowsError(try FaultToleranceOptions.parse(["--rate-limit", "601"])) + } + + func testMaxRetriesValidationRejectsAboveMax() { + XCTAssertThrowsError(try FaultToleranceOptions.parse(["--max-retries", "101"])) + } + + func testTimeoutValidationRejectsAboveMax() { + XCTAssertThrowsError(try FaultToleranceOptions.parse(["--timeout", "601"])) + } + + // MARK: - Sanitizer (PKL clamp + warn) + + func testEffectiveRateLimitClampsInvalidConfigValue() { + let options = try? FaultToleranceOptions.parse([]) + let ui = TerminalUI(outputMode: .quiet) + XCTAssertEqual(options?.effectiveRateLimit(configValue: 0, ui: ui), 10) + XCTAssertEqual(options?.effectiveRateLimit(configValue: -5, ui: ui), 10) + XCTAssertEqual(options?.effectiveRateLimit(configValue: 700, ui: ui), 10) + } + + func testEffectiveMaxRetriesClampsInvalidConfigValue() { + let options = try? FaultToleranceOptions.parse([]) + let ui = TerminalUI(outputMode: .quiet) + XCTAssertEqual(options?.effectiveMaxRetries(configValue: -1, ui: ui), 4) + XCTAssertEqual(options?.effectiveMaxRetries(configValue: 200, ui: ui), 4) + } + // MARK: - Precedence (CLI > config > default) func testEffectiveRateLimitCLIWinsOverConfig() throws { diff --git a/exfig.usage.kdl b/exfig.usage.kdl index 47cc5163..722700c4 100644 --- a/exfig.usage.kdl +++ b/exfig.usage.kdl @@ -6,7 +6,7 @@ name "ExFig" bin "exfig" -version "v3.2.1" +version "v3.4.0" about "Exports resources from Figma to iOS, Android, Flutter, and Web projects" // ============================================================================= From c0699661a09a09f12733644a8c8708ece0152814 Mon Sep 17 00:00:00 2001 From: alexey1312 Date: Sat, 9 May 2026 14:38:19 +0500 Subject: [PATCH 3/5] fix(cli): apply review findings for configurable fault-tolerance batch MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Critical: - Wire HeavyFaultToleranceOptions through `download all` (was bypassing rate-limiter, retries, and CLI flags entirely). - Drop per-config `figma.timeout` fallback in BatchConfigRunner — use resolved batch-level timeout only, matching the documented "first-config-wins" rule and ignoredPerTargetFigmaRateLimiting warning. - Make `validateUsing(preloadedModule:)` throwing and call `resolveConfigPath()` so cached configs go through the same validation surface as `validate()`. Important: - Sync `failFast` x `maxRetries` in batch RetryPolicy (failFast forces 0 retries, matching HeavyFaultToleranceOptions.createRetryPolicy). - Promote PKL parse errors during verbose pre-check from debug to `batchSettingsPreloadFailed` warning. - Replace brittle "not found" substring match in `isFileNotFound` with structural FileManager + NSError domain checks. A real PKL error like "module member 'foo' not found" is no longer reclassified as missing-file. - Cap `concurrentDownloads * parallel` at 1000 with a warning to prevent EMFILE / cryptic CDN throttling at extreme combinations. - Add precedence integration tests for ExportColors/Icons/Images/Typography via subcommand wiring. - Add PKLModuleCache write/hit tests covering verbose pre-check path and URL standardization. - Add cliTimeout precedence tests in BatchSettingsResolverExtendedTests. Suggestions: - Dedupe `invalidConfigValue` warnings per (key, value) pair via FaultToleranceValidator.warnOnce so repeated sanitize() calls don't flood the log. - Add ExFigWarningFormatter tests for the four new fault-tolerance warnings + new `excessiveDownloadSlots` case. - Add upper-bound sanitizer tests for parallel / timeout / concurrentDownloads. - Verbose Batch logs now print resolved batch settings (parallel, failFast, resume, concurrentDownloads, timeout) for traceability. Tests: 1989 passed, lint clean. --- .../ExFigCLI/Batch/BatchConfigRunner.swift | 13 +- .../Batch/BatchSettingsResolver.swift | 34 ++-- Sources/ExFigCLI/Input/ExFigOptions.swift | 11 +- .../Input/FaultToleranceDefaults.swift | 8 +- .../Input/FaultToleranceValidator.swift | 39 +++- Sources/ExFigCLI/Subcommands/Batch.swift | 31 ++- .../ExFigCLI/Subcommands/DownloadAll.swift | 59 +++--- .../ExFigCLI/TerminalUI/ExFigWarning.swift | 4 + .../TerminalUI/ExFigWarningFormatter.swift | 8 +- .../BatchSettingsResolverExtendedTests.swift | 101 ++++++++++ .../Batch/PKLModuleCacheTests.swift | 86 +++++++++ .../Input/FaultToleranceOptionsTests.swift | 5 + ...commandFaultTolerancePrecedenceTests.swift | 180 ++++++++++++++++++ .../ExFigWarningFormatterTests.swift | 63 ++++++ 14 files changed, 588 insertions(+), 54 deletions(-) create mode 100644 Tests/ExFigTests/Batch/PKLModuleCacheTests.swift create mode 100644 Tests/ExFigTests/Input/SubcommandFaultTolerancePrecedenceTests.swift diff --git a/Sources/ExFigCLI/Batch/BatchConfigRunner.swift b/Sources/ExFigCLI/Batch/BatchConfigRunner.swift index b7510fa4..6e533a17 100644 --- a/Sources/ExFigCLI/Batch/BatchConfigRunner.swift +++ b/Sources/ExFigCLI/Batch/BatchConfigRunner.swift @@ -222,7 +222,10 @@ struct BatchConfigRunner { let cachePath: String? let experimentalGranularCache: Bool let concurrentDownloads: Int - /// CLI timeout override (in seconds). When set, overrides per-config timeout. + /// Resolved batch-level timeout (in seconds). Already merged via `BatchSettingsResolver`: + /// CLI flag > FIRST config's `figma.timeout` > built-in default. Per-config `figma.timeout` + /// values in subsequent configs are intentionally ignored — `BatchSettingsResolver` warns + /// the user under `--verbose` via `.ignoredPerTargetFigmaRateLimiting`. let cliTimeout: Int? /// Priority for this config's downloads (lower = higher priority, based on submission order). let configPriority: Int @@ -305,16 +308,18 @@ struct BatchConfigRunner { var options = ExFigOptions() options.input = configFile.url.path if let cached = await moduleCache?.get(for: configFile.url) { - options.validateUsing(preloadedModule: cached) + try options.validateUsing(preloadedModule: cached) } else { try options.validate() } let retryHandler = RetryLogger.createHandler(ui: ui, maxAttempts: maxRetries) - // CLI timeout takes precedence over per-config timeout + // Use ONLY the batch-level resolved timeout. Per-config `figma.timeout` is + // intentionally ignored — `BatchSettingsResolver` already merged CLI flag + + // FIRST config's value, and warns about ignored per-target values under -v. + // Honoring per-config timeout here would silently override that resolution. let effectiveTimeout: TimeInterval? = cliTimeout.map { TimeInterval($0) } - ?? options.params.figma?.timeout let baseClient = try FigmaClient( accessToken: options.requireFigmaToken(), diff --git a/Sources/ExFigCLI/Batch/BatchSettingsResolver.swift b/Sources/ExFigCLI/Batch/BatchSettingsResolver.swift index 4b8e9cef..8a8e0d44 100644 --- a/Sources/ExFigCLI/Batch/BatchSettingsResolver.swift +++ b/Sources/ExFigCLI/Batch/BatchSettingsResolver.swift @@ -149,10 +149,20 @@ enum BatchSettingsResolver { module = try await PKLEvaluator.evaluate(configPath: url) await moduleCache?.set(module, for: url) } catch { - ui.debug( - "Could not pre-check \(url.lastPathComponent) for ignored batch settings: " + - "\(error.localizedDescription)" - ) + if isFileNotFound(error: error, url: url) { + // BatchConfigRunner will surface a clearer per-config error; debug only. + ui.debug( + "Pre-check skipped: \(url.lastPathComponent) not found." + ) + } else { + // Real PKL/syntax/network errors mean we cannot tell whether the user + // had per-target batch:/figma:* fields. Surface as warning so the user + // doesn't think their config was silently accepted. + ui.warning(.batchSettingsPreloadFailed( + file: url.lastPathComponent, + error: error.localizedDescription + )) + } continue } if module?.batch != nil { @@ -169,7 +179,15 @@ enum BatchSettingsResolver { } } + /// Detect "file not found" errors structurally. We check the filesystem FIRST so the + /// classification doesn't depend on Foundation/PklSwift error message wording. + /// Falls back to NSError domain/code checks for completeness; deliberately does NOT + /// match arbitrary "not found" substrings (a real PKL error like + /// `module member 'foo' not found` should NOT be reclassified as missing-file). private static func isFileNotFound(error: Error, url: URL) -> Bool { + if !FileManager.default.fileExists(atPath: url.path) { + return true + } let nsError = error as NSError if nsError.domain == NSCocoaErrorDomain, nsError.code == NSFileReadNoSuchFileError { return true @@ -177,12 +195,6 @@ enum BatchSettingsResolver { if nsError.domain == NSPOSIXErrorDomain, nsError.code == Int(ENOENT) { return true } - // PklSwift surfaces missing files via PklError text — a textual match is brittle but - // acceptable as a fallback (better than over-warning the user). - let message = error.localizedDescription.lowercased() - if message.contains("no such file") || message.contains("not found") { - return true - } - return !FileManager.default.fileExists(atPath: url.path) + return false } } diff --git a/Sources/ExFigCLI/Input/ExFigOptions.swift b/Sources/ExFigCLI/Input/ExFigOptions.swift index 6ad9483f..0837d071 100644 --- a/Sources/ExFigCLI/Input/ExFigOptions.swift +++ b/Sources/ExFigCLI/Input/ExFigOptions.swift @@ -43,8 +43,17 @@ struct ExFigOptions: ParsableArguments { /// Validates and primes `params` from a pre-evaluated PKL module — avoids redundant PKL eval /// when the caller (e.g. `BatchSettingsResolver`) already loaded the same config. - mutating func validateUsing(preloadedModule module: ExFig.ModuleImpl) { + /// + /// Behaves as a drop-in replacement for `validate()`: + /// 1. Reads `FIGMA_PERSONAL_TOKEN` from the environment (same as `validate()`). + /// 2. Resolves the config path so any future validation rule added to `validate()` that + /// inspects `input` still applies (matches `validate()` precondition surface). + /// 3. Skips PKL evaluation by reusing the supplied module. + /// + /// Throws the same errors as `validate()` for path resolution failures. + mutating func validateUsing(preloadedModule module: ExFig.ModuleImpl) throws { accessToken = ProcessInfo.processInfo.environment["FIGMA_PERSONAL_TOKEN"] + _ = try resolveConfigPath() params = module } diff --git a/Sources/ExFigCLI/Input/FaultToleranceDefaults.swift b/Sources/ExFigCLI/Input/FaultToleranceDefaults.swift index 8d810ffe..cd68f09f 100644 --- a/Sources/ExFigCLI/Input/FaultToleranceDefaults.swift +++ b/Sources/ExFigCLI/Input/FaultToleranceDefaults.swift @@ -4,7 +4,8 @@ import Foundation /// /// These values must be kept in sync with PKL schema defaults in /// `Sources/ExFigCLI/Resources/Schemas/{Figma,Batch}.pkl`. The -/// `FaultToleranceDefaultsParityTests` test asserts the parity at runtime. +/// `BatchSettingsResolverExtendedTests.testPKLDefaultsMatchSwiftDefaults` test +/// asserts the parity at runtime. enum FaultToleranceDefaults { static let parallel = 3 static let rateLimit = 10 @@ -17,4 +18,9 @@ enum FaultToleranceDefaults { static let maxRetriesRange = 0 ... 100 static let concurrentDownloadsRange = 1 ... 200 static let timeoutRange = 1 ... 600 + + /// Safe cap for `concurrentDownloads * parallel` (total simultaneous CDN connections). + /// Above this, OS file-descriptor limits (~1024-2048) and per-host CDN throttling start + /// surfacing as cryptic EMFILE/network errors instead of clean timeouts. + static let maxDownloadSlots = 1000 } diff --git a/Sources/ExFigCLI/Input/FaultToleranceValidator.swift b/Sources/ExFigCLI/Input/FaultToleranceValidator.swift index 468b293f..f8f8602d 100644 --- a/Sources/ExFigCLI/Input/FaultToleranceValidator.swift +++ b/Sources/ExFigCLI/Input/FaultToleranceValidator.swift @@ -4,8 +4,10 @@ import Foundation /// Centralized validation rules for fault-tolerance and batch knobs. /// /// These rules are shared between CLI `validate()` (raises `ValidationError`) and -/// PKL `effective*` accessors (clamp + warn). Keeping them in one place removes -/// the drift hazard that comes with three nearly-identical `validate()` blocks. +/// the CLI-side `effective*` accessors on `FaultToleranceOptions` / +/// `HeavyFaultToleranceOptions` (clamp + warn when reading PKL config values). +/// Keeping them in one place removes the drift hazard that comes with several +/// nearly-identical `validate()` blocks. enum FaultToleranceValidator { // MARK: - CLI validation (throws) @@ -92,16 +94,21 @@ enum FaultToleranceValidator { ) } + /// Sanitize a PKL `figma.timeout`. Unlike the other knobs, `nil` means "no value + /// configured" and is preserved (downstream uses `FigmaClient`'s default). + /// Out-of-range values fall back to `FaultToleranceDefaults.timeoutSeconds` and emit + /// `.invalidConfigValue` at most once per (key, value) pair (deduped via `warnOnce`). static func sanitizedTimeout(_ value: Int?, ui: TerminalUI?) -> Int? { guard let value else { return nil } if FaultToleranceDefaults.timeoutRange.contains(value) { return value } - ui?.warning(.invalidConfigValue( + warnOnce( key: "figma.timeout", value: value, - fallback: FaultToleranceDefaults.timeoutSeconds - )) + fallback: FaultToleranceDefaults.timeoutSeconds, + ui: ui + ) return FaultToleranceDefaults.timeoutSeconds } @@ -117,6 +124,10 @@ enum FaultToleranceValidator { // MARK: - Internal + /// Tracks `(key, value)` pairs we've already warned about so the same out-of-range PKL + /// value doesn't dilute the log when sanitize() is called from many sites in one run. + private static let warnedKeys = Lock>([]) + private static func sanitize( value: Int?, range: ClosedRange, @@ -128,7 +139,23 @@ enum FaultToleranceValidator { if range.contains(value) { return value } - ui?.warning(.invalidConfigValue(key: key, value: value, fallback: fallback)) + warnOnce(key: key, value: value, fallback: fallback, ui: ui) return fallback } + + /// Emits `.invalidConfigValue` at most once per `(key, value)` pair within this process. + /// Test-only `resetWarnedKeys()` clears the dedup cache between scenarios. + static func warnOnce(key: String, value: Int, fallback: Int, ui: TerminalUI?) { + let dedupKey = "\(key)=\(value)" + let shouldEmit = warnedKeys.withLock { keys -> Bool in + keys.insert(dedupKey).inserted + } + guard shouldEmit else { return } + ui?.warning(.invalidConfigValue(key: key, value: value, fallback: fallback)) + } + + /// Test hook: clears the per-process warning dedup cache. + static func resetWarnedKeys() { + warnedKeys.withLock { $0.removeAll() } + } } diff --git a/Sources/ExFigCLI/Subcommands/Batch.swift b/Sources/ExFigCLI/Subcommands/Batch.swift index 5fdd82ff..6cb7a77b 100644 --- a/Sources/ExFigCLI/Subcommands/Batch.swift +++ b/Sources/ExFigCLI/Subcommands/Batch.swift @@ -239,7 +239,11 @@ extension ExFigCommand { ui: TerminalUI ) async -> (BatchResult, SharedRateLimiter) { let rateLimiter = SharedRateLimiter(requestsPerMinute: Double(resolved.rateLimit)) - let retryPolicy = RetryPolicy(maxRetries: resolved.maxRetries) + // failFast forces 0 retries on individual API calls, matching + // `HeavyFaultToleranceOptions.createRetryPolicy()` in non-batch mode. + // Without this, --fail-fast at batch level would still retry every + // request up to `resolved.maxRetries` times before failing the config. + let retryPolicy = RetryPolicy(maxRetries: resolved.failFast ? 0 : resolved.maxRetries) // Load cache for smart pre-fetch optimization (version checking) // This allows skipping heavy Components API calls when file version is unchanged @@ -277,10 +281,19 @@ extension ExFigCommand { ui: ui ) - // Create shared download queue for cross-config pipelining - let downloadQueue = SharedDownloadQueue( - maxConcurrentDownloads: resolved.concurrentDownloads * resolved.parallel - ) + // Create shared download queue for cross-config pipelining. + // Cap the total slot count so absurd combinations (e.g. 200*50=10000) don't blow + // through OS file descriptor limits or trigger cryptic CDN throttling. + let requestedSlots = resolved.concurrentDownloads * resolved.parallel + let cappedSlots = min(requestedSlots, FaultToleranceDefaults.maxDownloadSlots) + if cappedSlots < requestedSlots { + ui.warning(.excessiveDownloadSlots( + concurrentDownloads: resolved.concurrentDownloads, + parallel: resolved.parallel, + capped: cappedSlots + )) + } + let downloadQueue = SharedDownloadQueue(maxConcurrentDownloads: cappedSlots) // Create priority map: configs submitted first get higher priority (lower number) let priorityMap = Dictionary( @@ -483,10 +496,16 @@ extension ExFigCommand { if globalOptions.verbose { ui.info("Rate limit: \(resolved.rateLimit) req/min, max retries: \(resolved.maxRetries)") + ui.info( + "Resolved batch settings: parallel=\(resolved.parallel), failFast=\(resolved.failFast), " + + "resume=\(resolved.resume), concurrentDownloads=\(resolved.concurrentDownloads), " + + "timeout=\(resolved.timeout.map(String.init) ?? "default")" + ) if sharedGranularCache != nil { ui.info("Granular cache: shared across workers") } - let slots = resolved.concurrentDownloads * resolved.parallel + let requestedSlots = resolved.concurrentDownloads * resolved.parallel + let slots = min(requestedSlots, FaultToleranceDefaults.maxDownloadSlots) ui.info("Download queue: shared with \(slots) concurrent slots") } } diff --git a/Sources/ExFigCLI/Subcommands/DownloadAll.swift b/Sources/ExFigCLI/Subcommands/DownloadAll.swift index e95d8dd8..f8395c64 100644 --- a/Sources/ExFigCLI/Subcommands/DownloadAll.swift +++ b/Sources/ExFigCLI/Subcommands/DownloadAll.swift @@ -25,6 +25,9 @@ extension ExFigCommand.Download { @OptionGroup var assetOptions: AssetExportOptions + @OptionGroup + var faultToleranceOptions: HeavyFaultToleranceOptions + @Option(name: .long, help: "Figma frame name for icons (default: from config or 'Icons')") var iconsFrameName: String? @@ -42,29 +45,49 @@ extension ExFigCommand.Download { try FileManager.default.createDirectory(at: outputURL, withIntermediateDirectories: true) + // Build a single rate-limited client shared across all four sub-flows so + // --rate-limit / --max-retries / --concurrent-downloads behave consistently + // (matches `download colors`, `download icons`, `download images` siblings). + let client = try makeClient(ui: ui) + ui.info("Exporting colors...") - try await exportColors(outputDir: outputURL, ui: ui) + try await exportColors(client: client, outputDir: outputURL, ui: ui) ui.info("Exporting typography...") - try await exportTypography(outputDir: outputURL, ui: ui) + try await exportTypography(client: client, outputDir: outputURL, ui: ui) ui.info("Exporting icons...") - try await exportIcons(outputDir: outputURL, ui: ui) + try await exportIcons(client: client, outputDir: outputURL, ui: ui) ui.info("Exporting images...") - try await exportImages(outputDir: outputURL, ui: ui) + try await exportImages(client: client, outputDir: outputURL, ui: ui) ui.success("Exported all design tokens to \(outputDir)") } - // MARK: - Export Methods + // MARK: - Client Factory - // swiftlint:disable:next function_body_length - private func exportColors(outputDir: URL, ui: TerminalUI) async throws { - let client = try FigmaClient( + private func makeClient(ui: TerminalUI) throws -> Client { + let figmaParams = options.params.figma + let baseClient = try FigmaClient( accessToken: options.requireFigmaToken(), - timeout: options.params.figma?.timeout + timeout: faultToleranceOptions.timeout.map(TimeInterval.init) ?? figmaParams?.timeout + ) + let rateLimiter = faultToleranceOptions.createRateLimiter(configValue: figmaParams?.rateLimit) + return faultToleranceOptions.createRateLimitedClient( + wrapping: baseClient, + rateLimiter: rateLimiter, + configMaxRetries: figmaParams?.maxRetries, + onRetry: { attempt, error in + ui.warning("Retry \(attempt) after error: \(error.localizedDescription)") + } ) + } + + // MARK: - Export Methods + + // swiftlint:disable:next function_body_length + private func exportColors(client: Client, outputDir: URL, ui: TerminalUI) async throws { let figmaParams = options.params.figma let commonParams = options.params.common @@ -130,11 +153,7 @@ extension ExFigCommand.Download { } } - private func exportTypography(outputDir: URL, ui: TerminalUI) async throws { - let client = try FigmaClient( - accessToken: options.requireFigmaToken(), - timeout: options.params.figma?.timeout - ) + private func exportTypography(client: Client, outputDir: URL, ui: TerminalUI) async throws { guard let figmaParams = options.params.figma else { throw ExFigError.custom(errorString: "figma section is required for typography export.") } @@ -171,11 +190,7 @@ extension ExFigCommand.Download { } // swiftlint:disable:next function_body_length - private func exportIcons(outputDir: URL, ui: TerminalUI) async throws { - let client = try FigmaClient( - accessToken: options.requireFigmaToken(), - timeout: options.params.figma?.timeout - ) + private func exportIcons(client: Client, outputDir: URL, ui: TerminalUI) async throws { guard let figmaParams = options.params.figma else { throw ExFigError.custom(errorString: "figma section is required for icons export.") } @@ -245,11 +260,7 @@ extension ExFigCommand.Download { } // swiftlint:disable:next function_body_length - private func exportImages(outputDir: URL, ui: TerminalUI) async throws { - let client = try FigmaClient( - accessToken: options.requireFigmaToken(), - timeout: options.params.figma?.timeout - ) + private func exportImages(client: Client, outputDir: URL, ui: TerminalUI) async throws { guard let figmaParams = options.params.figma else { throw ExFigError.custom(errorString: "figma section is required for images export.") } diff --git a/Sources/ExFigCLI/TerminalUI/ExFigWarning.swift b/Sources/ExFigCLI/TerminalUI/ExFigWarning.swift index 95fd6aa2..78f9e149 100644 --- a/Sources/ExFigCLI/TerminalUI/ExFigWarning.swift +++ b/Sources/ExFigCLI/TerminalUI/ExFigWarning.swift @@ -119,4 +119,8 @@ enum ExFigWarning: Equatable { /// A PKL config value for fault tolerance was out of range and replaced with the default. case invalidConfigValue(key: String, value: Int, fallback: Int) + + /// `concurrentDownloads * parallel` exceeds a safe slot count and was capped. + /// Avoids EMFILE / CDN throttling from configurations like `200 * 50 = 10000`. + case excessiveDownloadSlots(concurrentDownloads: Int, parallel: Int, capped: Int) } diff --git a/Sources/ExFigCLI/TerminalUI/ExFigWarningFormatter.swift b/Sources/ExFigCLI/TerminalUI/ExFigWarningFormatter.swift index af96634f..1b315293 100644 --- a/Sources/ExFigCLI/TerminalUI/ExFigWarningFormatter.swift +++ b/Sources/ExFigCLI/TerminalUI/ExFigWarningFormatter.swift @@ -22,7 +22,8 @@ struct ExFigWarningFormatter { .unresolvedNumberAlias, .depthExceededNumberAlias, .circularColorAlias, .downloadTokensSectionSkipped, .batchSettingsPreloadFailed, .ignoredPerTargetBatchBlock, - .ignoredPerTargetFigmaRateLimiting, .invalidConfigValue: + .ignoredPerTargetFigmaRateLimiting, .invalidConfigValue, + .excessiveDownloadSlots: formatCompact(warning) // Multiline format warnings @@ -122,6 +123,11 @@ struct ExFigWarningFormatter { case let .invalidConfigValue(key, value, fallback): "Invalid config value: key=\(key), value=\(value), fallback=\(fallback)" + case let .excessiveDownloadSlots(concurrentDownloads, parallel, capped): + "Excessive download slots: concurrentDownloads=\(concurrentDownloads), " + + "parallel=\(parallel), product=\(concurrentDownloads * parallel), capped=\(capped). " + + "Reduce --concurrent-downloads or --parallel to avoid CDN throttling / EMFILE." + // Multiline cases handled in main format() method case .noAssetsFound, .invalidConfigsSkipped, .webIconsMissingSVGData, .webIconsConversionFailed: fatalError("Multiline warnings should not reach formatCompact") diff --git a/Tests/ExFigTests/Batch/BatchSettingsResolverExtendedTests.swift b/Tests/ExFigTests/Batch/BatchSettingsResolverExtendedTests.swift index 0d2399f8..aaffc531 100644 --- a/Tests/ExFigTests/Batch/BatchSettingsResolverExtendedTests.swift +++ b/Tests/ExFigTests/Batch/BatchSettingsResolverExtendedTests.swift @@ -6,6 +6,11 @@ import XCTest /// `BatchSettingsResolver`. Lives in its own suite to keep the original /// `BatchSettingsResolverTests` under SwiftLint's type/file length limits. final class BatchSettingsResolverExtendedTests: XCTestCase { + override func setUp() { + super.setUp() + FaultToleranceValidator.resetWarnedKeys() + } + // MARK: - Multi-Config: First Wins (T1) func testFirstConfigWinsAndSubsequentConfigsAreIgnored() async throws { @@ -195,6 +200,102 @@ final class BatchSettingsResolverExtendedTests: XCTestCase { XCTAssertEqual(FaultToleranceDefaults.concurrentDownloads, 20) XCTAssertEqual(FaultToleranceDefaults.timeoutSeconds, 30) } + + // MARK: - cliTimeout precedence (I8: BatchConfigRunner uses resolved timeout only) + + func testCLITimeoutOverridesFirstConfigTimeout() async throws { + // PKL `timeout` is `Number` (Float64); use `.0` to keep pkl-swift happy. + let configURL = try BatchResolverFixture.make(figma: "timeout = 90.0", batch: nil) + defer { try? FileManager.default.removeItem(at: configURL) } + let ui = TerminalUI(outputMode: .quiet) + + let resolved = await BatchSettingsResolver.resolve( + cliParallel: nil, + cliFailFast: false, + cliResume: false, + cliRateLimit: nil, + cliMaxRetries: nil, + cliConcurrentDownloads: nil, + cliTimeout: 30, + allConfigs: [configURL], + verbose: false, + ui: ui + ) + + XCTAssertEqual(resolved.timeout, 30, "CLI --timeout wins over first config's figma.timeout") + } + + func testFirstConfigTimeoutWinsWhenCLIMissing() async throws { + let configURL = try BatchResolverFixture.make(figma: "timeout = 90.0", batch: nil) + defer { try? FileManager.default.removeItem(at: configURL) } + let ui = TerminalUI(outputMode: .quiet) + + let resolved = await BatchSettingsResolver.resolve( + cliParallel: nil, + cliFailFast: false, + cliResume: false, + cliRateLimit: nil, + cliMaxRetries: nil, + cliConcurrentDownloads: nil, + cliTimeout: nil, + allConfigs: [configURL], + verbose: false, + ui: ui + ) + + XCTAssertEqual(resolved.timeout, 90, "first config's figma.timeout used when CLI omitted") + } + + func testSecondConfigTimeoutIgnored() async throws { + // First config sets timeout=45.0 explicitly; second config sets a different value. + // Resolved timeout must reflect first config only. + let firstURL = try BatchResolverFixture.make(figma: "timeout = 45.0", batch: nil) + let secondURL = try BatchResolverFixture.make(figma: "timeout = 599.0", batch: nil) + defer { + try? FileManager.default.removeItem(at: firstURL) + try? FileManager.default.removeItem(at: secondURL) + } + let ui = TerminalUI(outputMode: .quiet) + + let resolved = await BatchSettingsResolver.resolve( + cliParallel: nil, + cliFailFast: false, + cliResume: false, + cliRateLimit: nil, + cliMaxRetries: nil, + cliConcurrentDownloads: nil, + cliTimeout: nil, + allConfigs: [firstURL, secondURL], + verbose: false, + ui: ui + ) + + XCTAssertEqual(resolved.timeout, 45, "subsequent config timeouts must not influence resolved timeout") + } + + // MARK: - Sanitizer upper-bounds (S7) + + func testSanitizedParallelClampsOutOfRangeValues() { + let ui = TerminalUI(outputMode: .quiet) + XCTAssertEqual(FaultToleranceValidator.sanitizedParallel(0, ui: ui), 3) + XCTAssertEqual(FaultToleranceValidator.sanitizedParallel(-1, ui: ui), 3) + XCTAssertEqual(FaultToleranceValidator.sanitizedParallel(51, ui: ui), 3) + } + + func testSanitizedTimeoutClampsOutOfRangeValuesAndPreservesNil() { + let ui = TerminalUI(outputMode: .quiet) + XCTAssertNil(FaultToleranceValidator.sanitizedTimeout(nil, ui: ui)) + XCTAssertEqual(FaultToleranceValidator.sanitizedTimeout(0, ui: ui), 30) + XCTAssertEqual(FaultToleranceValidator.sanitizedTimeout(-5, ui: ui), 30) + XCTAssertEqual(FaultToleranceValidator.sanitizedTimeout(700, ui: ui), 30) + XCTAssertEqual(FaultToleranceValidator.sanitizedTimeout(60, ui: ui), 60) + } + + func testSanitizedConcurrentDownloadsClampsAboveRange() { + let ui = TerminalUI(outputMode: .quiet) + XCTAssertEqual(FaultToleranceValidator.sanitizedConcurrentDownloads(201, ui: ui), 20) + XCTAssertEqual(FaultToleranceValidator.sanitizedConcurrentDownloads(200, ui: ui), 200) + } } /// Shared PKL fixture builder for `BatchSettingsResolver` tests. diff --git a/Tests/ExFigTests/Batch/PKLModuleCacheTests.swift b/Tests/ExFigTests/Batch/PKLModuleCacheTests.swift new file mode 100644 index 00000000..6d5a232b --- /dev/null +++ b/Tests/ExFigTests/Batch/PKLModuleCacheTests.swift @@ -0,0 +1,86 @@ +@testable import ExFigCLI +import ExFigConfig +import Foundation +import XCTest + +/// Coverage for `PKLModuleCache` — write/hit semantics and URL standardization. +/// Without these, a regression that drops the cache write or breaks URL canonicalization +/// would silently re-trigger PKL evaluation in `BatchConfigRunner`. +final class PKLModuleCacheTests: XCTestCase { + func testResolverPopulatesCacheWithFirstConfig() async throws { + let configURL = try BatchResolverFixture.make( + figma: "rateLimit = 25", + batch: "parallel = 8" + ) + defer { try? FileManager.default.removeItem(at: configURL) } + let ui = TerminalUI(outputMode: .quiet) + let cache = PKLModuleCache() + + _ = await BatchSettingsResolver.resolve( + cliParallel: nil, + cliFailFast: false, + cliResume: false, + cliRateLimit: nil, + cliMaxRetries: nil, + cliConcurrentDownloads: nil, + cliTimeout: nil, + allConfigs: [configURL], + verbose: false, + ui: ui, + moduleCache: cache + ) + + let cached = await cache.get(for: configURL) + XCTAssertNotNil(cached, "resolver should populate cache after evaluating first config") + XCTAssertEqual(cached?.figma?.rateLimit, 25) + XCTAssertEqual(cached?.batch?.parallel, 8) + } + + func testVerbosePreCheckPopulatesCacheForOtherConfigs() async throws { + let firstURL = try BatchResolverFixture.make(figma: "rateLimit = 25", batch: nil) + // `parallel = 25` is in-range (max is 50); using 99 would fail PKL constraints + // and the second config wouldn't load — defeating the test's purpose. + let secondURL = try BatchResolverFixture.make(figma: nil, batch: "parallel = 25") + defer { + try? FileManager.default.removeItem(at: firstURL) + try? FileManager.default.removeItem(at: secondURL) + } + let ui = TerminalUI(outputMode: .quiet) + let cache = PKLModuleCache() + + _ = await BatchSettingsResolver.resolve( + cliParallel: nil, + cliFailFast: false, + cliResume: false, + cliRateLimit: nil, + cliMaxRetries: nil, + cliConcurrentDownloads: nil, + cliTimeout: nil, + allConfigs: [firstURL, secondURL], + verbose: true, + ui: ui, + moduleCache: cache + ) + + let firstCached = await cache.get(for: firstURL) + let secondCached = await cache.get(for: secondURL) + XCTAssertNotNil(firstCached, "first config cached after primary load") + XCTAssertNotNil(secondCached, "second config cached during verbose pre-check") + XCTAssertEqual(secondCached?.batch?.parallel, 25) + } + + func testCacheStandardizesURLsForLookup() async throws { + let configURL = try BatchResolverFixture.make(figma: nil, batch: "parallel = 7") + defer { try? FileManager.default.removeItem(at: configURL) } + let cache = PKLModuleCache() + let module = try await PKLEvaluator.evaluate(configPath: configURL) + await cache.set(module, for: configURL) + + // Look up via a non-standardized variant of the same path (e.g. with extra `/./` + // segments). `standardizedFileURL` should normalize both keys to the same URL. + let path = configURL.path + let weirdURL = URL(fileURLWithPath: "/./" + path.dropFirst()) + let cached = await cache.get(for: weirdURL) + XCTAssertNotNil(cached, "URL lookup should be insensitive to non-canonical path forms") + } +} diff --git a/Tests/ExFigTests/Input/FaultToleranceOptionsTests.swift b/Tests/ExFigTests/Input/FaultToleranceOptionsTests.swift index 5cd4012e..eca2fe56 100644 --- a/Tests/ExFigTests/Input/FaultToleranceOptionsTests.swift +++ b/Tests/ExFigTests/Input/FaultToleranceOptionsTests.swift @@ -5,6 +5,11 @@ import FigmaAPI import XCTest final class FaultToleranceOptionsTests: XCTestCase { + override func setUp() { + super.setUp() + FaultToleranceValidator.resetWarnedKeys() + } + // MARK: - Default Values func testDefaultMaxRetries() throws { diff --git a/Tests/ExFigTests/Input/SubcommandFaultTolerancePrecedenceTests.swift b/Tests/ExFigTests/Input/SubcommandFaultTolerancePrecedenceTests.swift new file mode 100644 index 00000000..51f5f168 --- /dev/null +++ b/Tests/ExFigTests/Input/SubcommandFaultTolerancePrecedenceTests.swift @@ -0,0 +1,180 @@ +@testable import ExFigCLI +import ExFigConfig +import FigmaAPI +import XCTest + +/// End-to-end precedence tests (CLI > PKL `figma.*` > built-in default) for the wiring +/// inside individual export/download subcommands. These exercise the `effective*` +/// accessors with `configValue:` taken from a real PKL config evaluation, which is what +/// `ExportColors`, `ExportIcons`, `ExportImages`, `ExportTypography`, and the `Download*` +/// commands actually do at runtime via `resolveClient(...)`. +/// +/// Without these, a regression that swaps `??` direction or stops passing `figma?.rateLimit` +/// in any subcommand would slip through unit tests on `FaultToleranceOptions` alone. +final class SubcommandFaultTolerancePrecedenceTests: XCTestCase { + private var tempFiles: [URL] = [] + + override func setUp() { + super.setUp() + setenv("FIGMA_PERSONAL_TOKEN", "test_token", 1) + } + + override func tearDown() { + super.tearDown() + for url in tempFiles { + try? FileManager.default.removeItem(at: url) + } + tempFiles.removeAll() + } + + // MARK: - Light subcommand (FaultToleranceOptions) — colors / typography / download colors + + func testCLIRateLimitOverridesPKLForLightCommand() throws { + let options = try makeOptions(figmaBlock: "rateLimit = 25\nmaxRetries = 6") + var cmd = ExFigCommand.ExportColors() + cmd.globalOptions = GlobalOptions() + cmd.options = options + cmd.cacheOptions = CacheOptions() + cmd.faultToleranceOptions = try FaultToleranceOptions.parse(["--rate-limit", "30"]) + cmd.filter = nil + + let figma = cmd.options.params.figma + XCTAssertEqual(cmd.faultToleranceOptions.effectiveRateLimit(configValue: figma?.rateLimit), 30) + XCTAssertEqual(cmd.faultToleranceOptions.effectiveMaxRetries(configValue: figma?.maxRetries), 6) + } + + func testPKLValueUsedWhenCLIMissingForLightCommand() throws { + let options = try makeOptions(figmaBlock: "rateLimit = 25\nmaxRetries = 6") + var cmd = ExFigCommand.ExportColors() + cmd.globalOptions = GlobalOptions() + cmd.options = options + cmd.cacheOptions = CacheOptions() + cmd.faultToleranceOptions = FaultToleranceOptions() + cmd.filter = nil + + let figma = cmd.options.params.figma + XCTAssertEqual(cmd.faultToleranceOptions.effectiveRateLimit(configValue: figma?.rateLimit), 25) + XCTAssertEqual(cmd.faultToleranceOptions.effectiveMaxRetries(configValue: figma?.maxRetries), 6) + } + + func testBuiltInDefaultUsedWhenNeitherSetForLightCommand() throws { + let options = try makeOptions(figmaBlock: nil) + var cmd = ExFigCommand.ExportColors() + cmd.globalOptions = GlobalOptions() + cmd.options = options + cmd.cacheOptions = CacheOptions() + cmd.faultToleranceOptions = FaultToleranceOptions() + cmd.filter = nil + + let figma = cmd.options.params.figma + XCTAssertEqual(cmd.faultToleranceOptions.effectiveRateLimit(configValue: figma?.rateLimit), 10) + XCTAssertEqual(cmd.faultToleranceOptions.effectiveMaxRetries(configValue: figma?.maxRetries), 4) + } + + // MARK: - Heavy subcommand (HeavyFaultToleranceOptions) — icons / images + + func testCLIConcurrentDownloadsOverridesPKLForHeavyCommand() throws { + let options = try makeOptions(figmaBlock: "concurrentDownloads = 50") + var cmd = ExFigCommand.ExportIcons() + cmd.globalOptions = GlobalOptions() + cmd.options = options + cmd.cacheOptions = CacheOptions() + cmd.faultToleranceOptions = try HeavyFaultToleranceOptions.parse([ + "--concurrent-downloads", "75", + ]) + cmd.filter = nil + cmd.strictPathValidation = false + + let figma = cmd.options.params.figma + XCTAssertEqual( + cmd.faultToleranceOptions.effectiveConcurrentDownloads(configValue: figma?.concurrentDownloads), + 75 + ) + } + + func testPKLConcurrentDownloadsUsedWhenCLIMissingForHeavyCommand() throws { + let options = try makeOptions(figmaBlock: "concurrentDownloads = 50") + var cmd = ExFigCommand.ExportImages() + cmd.globalOptions = GlobalOptions() + cmd.options = options + cmd.cacheOptions = CacheOptions() + cmd.faultToleranceOptions = HeavyFaultToleranceOptions() + cmd.filter = nil + + let figma = cmd.options.params.figma + XCTAssertEqual( + cmd.faultToleranceOptions.effectiveConcurrentDownloads(configValue: figma?.concurrentDownloads), + 50 + ) + } + + func testHeavyDefaultsApplyWhenNeitherSourceProvided() throws { + let options = try makeOptions(figmaBlock: nil) + var cmd = ExFigCommand.ExportIcons() + cmd.globalOptions = GlobalOptions() + cmd.options = options + cmd.cacheOptions = CacheOptions() + cmd.faultToleranceOptions = HeavyFaultToleranceOptions() + cmd.filter = nil + cmd.strictPathValidation = false + + let figma = cmd.options.params.figma + XCTAssertEqual( + cmd.faultToleranceOptions.effectiveConcurrentDownloads(configValue: figma?.concurrentDownloads), + 20 + ) + XCTAssertEqual(cmd.faultToleranceOptions.effectiveMaxRetries(configValue: figma?.maxRetries), 4) + XCTAssertEqual(cmd.faultToleranceOptions.effectiveRateLimit(configValue: figma?.rateLimit), 10) + } + + // MARK: - Timeout precedence — exercised via resolveClient bridging + + func testCLITimeoutWinsOverPKLTimeoutInResolveClient() throws { + // The actual resolveClient wires `options.timeout > config timeout` — we replicate + // the same expression here to keep the test independent of FigmaClient internals. + let options = try makeOptions(figmaBlock: "timeout = 60.0") + let cliOpts = try FaultToleranceOptions.parse(["--timeout", "45"]) + + let configTimeout = options.params.figma?.timeout + let effective: TimeInterval? = cliOpts.timeout.map { TimeInterval($0) } ?? configTimeout + + XCTAssertEqual(effective, 45) + } + + func testPKLTimeoutUsedWhenCLITimeoutAbsent() throws { + let options = try makeOptions(figmaBlock: "timeout = 60.0") + let cliOpts = FaultToleranceOptions() + + let configTimeout = options.params.figma?.timeout + let effective: TimeInterval? = cliOpts.timeout.map { TimeInterval($0) } ?? configTimeout + + XCTAssertEqual(effective, 60) + } + + // MARK: - Helpers + + private func makeOptions(figmaBlock: String?) throws -> ExFigOptions { + let url = try writeConfig(figmaBlock: figmaBlock) + var options = ExFigOptions() + options.input = url.path + try options.validate() + return options + } + + private func writeConfig(figmaBlock: String?) throws -> URL { + var lines = ["figma {"] + lines.append(" lightFileId = \"test\"") + if let figmaBlock { + for line in figmaBlock.split(separator: "\n") { + lines.append(" \(line)") + } + } + lines.append("}") + + let url = FileManager.default.temporaryDirectory + .appendingPathComponent("subcommand-precedence-\(UUID().uuidString).pkl") + try lines.joined(separator: "\n").write(to: url, atomically: true, encoding: .utf8) + tempFiles.append(url) + return url + } +} diff --git a/Tests/ExFigTests/TerminalUI/ExFigWarningFormatterTests.swift b/Tests/ExFigTests/TerminalUI/ExFigWarningFormatterTests.swift index 71ac2b9b..4d775255 100644 --- a/Tests/ExFigTests/TerminalUI/ExFigWarningFormatterTests.swift +++ b/Tests/ExFigTests/TerminalUI/ExFigWarningFormatterTests.swift @@ -277,4 +277,67 @@ final class ExFigWarningFormatterTests: XCTestCase { XCTAssertFalse(formatted.isEmpty) } + + // MARK: - Fault Tolerance Warnings (S6) + + func testBatchSettingsPreloadFailedFormatsAsCompact() { + let warning = ExFigWarning.batchSettingsPreloadFailed( + file: "ios.pkl", + error: "PKL syntax error at line 5" + ) + + let result = formatter.format(warning) + + XCTAssertTrue(result.contains("Batch settings pre-load failed")) + XCTAssertTrue(result.contains("file=ios.pkl")) + XCTAssertTrue(result.contains("error=PKL syntax error at line 5")) + XCTAssertTrue(result.contains("CLI flags / built-in defaults will apply")) + } + + func testIgnoredPerTargetBatchBlockFormatsAsCompact() { + let warning = ExFigWarning.ignoredPerTargetBatchBlock(file: "android.pkl") + + let result = formatter.format(warning) + + XCTAssertTrue(result.contains("Ignoring batch:")) + XCTAssertTrue(result.contains("android.pkl")) + XCTAssertTrue(result.contains("only the first config's batch settings apply")) + } + + func testIgnoredPerTargetFigmaRateLimitingFormatsAsCompact() { + let warning = ExFigWarning.ignoredPerTargetFigmaRateLimiting(file: "flutter.pkl") + + let result = formatter.format(warning) + + XCTAssertTrue(result.contains("Ignoring per-target figma rate-limiting fields")) + XCTAssertTrue(result.contains("flutter.pkl")) + XCTAssertTrue(result.contains("only the first config's figma:* values apply")) + } + + func testInvalidConfigValueFormatsAsCompact() { + let warning = ExFigWarning.invalidConfigValue( + key: "figma.rateLimit", + value: 999, + fallback: 10 + ) + + let result = formatter.format(warning) + + XCTAssertEqual(result, "Invalid config value: key=figma.rateLimit, value=999, fallback=10") + } + + func testExcessiveDownloadSlotsFormatsAsCompact() { + let warning = ExFigWarning.excessiveDownloadSlots( + concurrentDownloads: 200, + parallel: 50, + capped: 1000 + ) + + let result = formatter.format(warning) + + XCTAssertTrue(result.contains("concurrentDownloads=200")) + XCTAssertTrue(result.contains("parallel=50")) + XCTAssertTrue(result.contains("product=10000")) + XCTAssertTrue(result.contains("capped=1000")) + } } From 4c40d2154341ba91679b9b28663e9d55d5ad7310 Mon Sep 17 00:00:00 2001 From: alexey1312 Date: Sat, 9 May 2026 14:38:37 +0500 Subject: [PATCH 4/5] docs(cli): clarify batch timeout precedence and download all coverage - fault-tolerance.md: rephrase batch `--timeout` row to make "CLI > first config's figma.timeout > default" precedence explicit; per-target timeouts in subsequent configs are always ignored. - Usage.md: add `download all` to the list of commands supporting heavy fault-tolerance flags; add footnote clarifying that `batch.failFast` and `batch.resume` PKL keys apply only to `exfig batch` (standalone icons/images accept the CLI flags but don't read PKL fields). - Regenerate llms-full.txt. --- .claude/rules/fault-tolerance.md | 12 +++++++++--- Sources/ExFigCLI/ExFig.docc/Usage.md | 11 +++++++---- llms-full.txt | 11 +++++++---- 3 files changed, 23 insertions(+), 11 deletions(-) diff --git a/.claude/rules/fault-tolerance.md b/.claude/rules/fault-tolerance.md index f969caf9..a28290d2 100644 --- a/.claude/rules/fault-tolerance.md +++ b/.claude/rules/fault-tolerance.md @@ -13,14 +13,20 @@ This rule covers retry, rate limiting, and timeout configuration for API command All commands support configurable retry, rate limiting, and timeout via CLI flags: ```bash -# Light commands (colors, typography, download subcommands) +# Light commands (colors, typography, download colors/typography/icons/images) exfig colors --max-retries 6 --rate-limit 15 --timeout 60 -# Heavy commands (icons, images) also support fail-fast and concurrent downloads +# Heavy commands (icons, images, download all) also support fail-fast and +# concurrent downloads. `download all` shares one rate-limited client across +# its colors/typography/icons/images sub-flows. exfig icons --max-retries 4 --rate-limit 15 --timeout 90 --fail-fast exfig icons --concurrent-downloads 50 # Increase CDN parallelism (default: 20) +exfig download all --rate-limit 25 --concurrent-downloads 50 -# Batch command with timeout (overrides all per-config timeouts) +# Batch command — `--timeout` is the resolved batch-level timeout. +# In batch mode `figma.*` rate-limiting fields (incl. `timeout`) are read ONLY from +# the first config; per-target `figma.timeout` in subsequent configs is ignored +# (warned under -v). Precedence: CLI > first config's figma.timeout > built-in default. exfig batch ./configs/ --timeout 60 --rate-limit 20 # fetch command has its own --timeout in DownloadOptions diff --git a/Sources/ExFigCLI/ExFig.docc/Usage.md b/Sources/ExFigCLI/ExFig.docc/Usage.md index 45d5aedf..93a06936 100644 --- a/Sources/ExFigCLI/ExFig.docc/Usage.md +++ b/Sources/ExFigCLI/ExFig.docc/Usage.md @@ -127,7 +127,7 @@ exfig icons --rate-limit 20 ### Extended Options -Commands that download many files (`icons`, `images`, `fetch`) support additional options: +Commands that download many files (`icons`, `images`, `fetch`, `download all`) support additional options: ```bash # Stop on first error @@ -145,9 +145,9 @@ exfig icons --concurrent-downloads 50 | `--max-retries` | Maximum retry attempts (default: 4) | All | `figma.maxRetries` | | `--rate-limit` | API requests per minute (default: 10) | All | `figma.rateLimit` | | `--timeout` | Figma API request timeout, sec (default: 30) | All | `figma.timeout` | -| `--concurrent-downloads` | Concurrent CDN downloads (default: 20) | icons, images, fetch | `figma.concurrentDownloads`* | -| `--fail-fast` | Stop immediately on error | icons, images, batch, fetch | `batch.failFast` (batch)| -| `--resume` | Continue from checkpoint | icons, images, batch, fetch | `batch.resume` (batch) | +| `--concurrent-downloads` | Concurrent CDN downloads (default: 20) | icons, images, fetch, download all, batch | `figma.concurrentDownloads`* | +| `--fail-fast` | Stop immediately on error | icons, images, batch, fetch | `batch.failFast` (batch only)†| +| `--resume` | Continue from checkpoint | icons, images, batch, fetch | `batch.resume` (batch only)† | | `--parallel` | Concurrent batch configs (default: 3) | batch | `batch.parallel` | CLI flags override PKL config; PKL config overrides built-in defaults. `fetch` is config-free — @@ -156,6 +156,9 @@ only CLI flags and built-in defaults apply there. *`figma.concurrentDownloads` is silently ignored by `colors`/`typography` (no CDN downloads); under `-v` a debug log records the skip. +†The `batch.failFast` / `batch.resume` PKL keys apply ONLY to `exfig batch`. Standalone `icons` / +`images` commands accept the corresponding CLI flags but do not read these PKL fields. + ### Checkpoint System Long-running exports create checkpoints for resumption: diff --git a/llms-full.txt b/llms-full.txt index d7c5b3b0..d960ad60 100644 --- a/llms-full.txt +++ b/llms-full.txt @@ -403,7 +403,7 @@ exfig icons --rate-limit 20 ### Extended Options -Commands that download many files (`icons`, `images`, `fetch`) support additional options: +Commands that download many files (`icons`, `images`, `fetch`, `download all`) support additional options: ```bash # Stop on first error @@ -421,9 +421,9 @@ exfig icons --concurrent-downloads 50 | `--max-retries` | Maximum retry attempts (default: 4) | All | `figma.maxRetries` | | `--rate-limit` | API requests per minute (default: 10) | All | `figma.rateLimit` | | `--timeout` | Figma API request timeout, sec (default: 30) | All | `figma.timeout` | -| `--concurrent-downloads` | Concurrent CDN downloads (default: 20) | icons, images, fetch | `figma.concurrentDownloads`* | -| `--fail-fast` | Stop immediately on error | icons, images, batch, fetch | `batch.failFast` (batch)| -| `--resume` | Continue from checkpoint | icons, images, batch, fetch | `batch.resume` (batch) | +| `--concurrent-downloads` | Concurrent CDN downloads (default: 20) | icons, images, fetch, download all, batch | `figma.concurrentDownloads`* | +| `--fail-fast` | Stop immediately on error | icons, images, batch, fetch | `batch.failFast` (batch only)†| +| `--resume` | Continue from checkpoint | icons, images, batch, fetch | `batch.resume` (batch only)† | | `--parallel` | Concurrent batch configs (default: 3) | batch | `batch.parallel` | CLI flags override PKL config; PKL config overrides built-in defaults. `fetch` is config-free — @@ -432,6 +432,9 @@ only CLI flags and built-in defaults apply there. *`figma.concurrentDownloads` is silently ignored by `colors`/`typography` (no CDN downloads); under `-v` a debug log records the skip. +†The `batch.failFast` / `batch.resume` PKL keys apply ONLY to `exfig batch`. Standalone `icons` / +`images` commands accept the corresponding CLI flags but do not read these PKL fields. + ### Checkpoint System Long-running exports create checkpoints for resumption: From 5e89b202f9d3167f0a0dd5e088a9d0379426bcb5 Mon Sep 17 00:00:00 2001 From: alexey1312 Date: Sat, 9 May 2026 22:00:24 +0500 Subject: [PATCH 5/5] docs(claude): capture batch settings architecture and PKL fixture pitfalls MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit fault-tolerance.md: - Document BatchSettingsResolver as single merge point for CLI/PKL/defaults. - Note BatchConfigRunner must NOT re-read per-config figma.timeout. - Document FaultToleranceValidator.warnOnce dedup pattern with reset hook. - Document PKLModuleCache as the way to avoid re-evaluating configs across resolver / pre-check / runner. - Document slot product cap and failFast x maxRetries coupling in batch. gotchas.md: - pkl-swift requires .0 suffix for Number?-typed PKL fields in test fixtures (DecodingError.typeMismatch otherwise). - PKL constraints validate at amends-time, not at sanitize-time — tests for out-of-range values must call FaultToleranceValidator.sanitized* directly. - Prefer splitting test classes over `swiftlint:disable file_length` (e.g. PKLModuleCacheTests was extracted from BatchSettingsResolverExtendedTests). --- .claude/rules/fault-tolerance.md | 35 ++++++++++++++++++++++++++++ .claude/rules/gotchas.md | 39 ++++++++++++++++++++++++++++++++ 2 files changed, 74 insertions(+) diff --git a/.claude/rules/fault-tolerance.md b/.claude/rules/fault-tolerance.md index a28290d2..c7ef02cf 100644 --- a/.claude/rules/fault-tolerance.md +++ b/.claude/rules/fault-tolerance.md @@ -121,3 +121,38 @@ CLI flags still override these values per-run. - Exponential backoff: 2s -> 4s -> 8s -> 16s with jitter - Respects `Retry-After` header from Figma API on 429 errors - Checkpoint system saves progress for resumption on failure + +## Batch Settings Architecture + +`BatchSettingsResolver.resolve(...)` is the single point where CLI flags, the FIRST config's +`batch:`/`figma.*` blocks, and built-in defaults merge into `ResolvedBatchSettings`. +`BatchConfigRunner` consumes the resolved values and MUST NOT read per-config `figma.timeout` +(or any other rate-limiting field) again — doing so silently overrides the documented +"first-config-wins" rule and contradicts the `ignoredPerTargetFigmaRateLimiting` warning. + +**Single source of truth pattern:** +- `FaultToleranceDefaults` — Swift defaults; parity with PKL schema asserted at runtime via + `BatchSettingsResolverExtendedTests.testPKLDefaultsMatchSwiftDefaults`. +- `FaultToleranceValidator.sanitized*` — clamps PKL values to ranges, falls back to default + with a `.invalidConfigValue` warning. Used both by the resolver and by per-command + `effective*` accessors. +- `FaultToleranceValidator.warnOnce(key:value:fallback:ui:)` — process-level dedup so the + same out-of-range PKL value doesn't produce duplicate warnings across many call-sites. + Has a `resetWarnedKeys()` test hook called from XCTest `setUp()` (`Lock` pattern, + same shape as `WarningCollector` / `ManifestTracker`). + +**PKL module reuse:** +- `PKLModuleCache` (actor) caches `ExFig.ModuleImpl` by URL across `BatchSettingsResolver`, + `logIgnoredPerTargetSettings`, and `BatchConfigRunner` to avoid re-evaluating the same + config 2-3 times in one batch run. URL keys go through `standardizedFileURL`. +- Consumers must call `try options.validateUsing(preloadedModule:)` (mirror of + `validate()` — same env+path checks minus PKL eval) to keep validation surfaces in sync. + +**Slot product cap:** `concurrentDownloads * parallel` is capped at +`FaultToleranceDefaults.maxDownloadSlots = 1000` in `Batch.swift` to prevent EMFILE / CDN +throttling at extreme combinations (e.g. 200 × 50 = 10000). Out-of-range emits +`.excessiveDownloadSlots` warning. + +**Batch RetryPolicy:** when constructing `RetryPolicy` in batch mode, honor `failFast`: +`RetryPolicy(maxRetries: resolved.failFast ? 0 : resolved.maxRetries)` — matches +`HeavyFaultToleranceOptions.createRetryPolicy()` outside batch. diff --git a/.claude/rules/gotchas.md b/.claude/rules/gotchas.md index 2107238b..91f69400 100644 --- a/.claude/rules/gotchas.md +++ b/.claude/rules/gotchas.md @@ -123,6 +123,9 @@ Use `var` + `.append()` pattern or computed property returning the array. - Use `Data("string".utf8)` not `"string".data(using: .utf8)!` - Add `// swiftlint:disable:next force_try` before `try!` in tests - Add `// swiftlint:disable file_length` for files > 400 lines +- For test files exceeding 400 lines: prefer splitting a second `final class` into its own + file (e.g. `PKLModuleCacheTests.swift` was extracted from + `BatchSettingsResolverExtendedTests.swift`) over `// swiftlint:disable file_length` ### swiftlint:disable with Doc Comments @@ -218,6 +221,42 @@ Without the extension in `PKLEvaluator.swift`, `.localizedDescription` returns u `"The operation couldn't be completed. (PklSwift.PklError error 1.)"` instead of the actual PKL error. Fix: `extension PklError: @retroactive LocalizedError` in `Sources/ExFigConfig/PKL/PKLEvaluator.swift`. +## PKL Test Fixtures + +### Number-typed fields require `.0` suffix + +pkl-swift strictly distinguishes `Int` vs `Double` at decode time. PKL fields generated as +`Double?` (e.g. `figma.timeout: Number(isBetween(1, 600))? = 30.0`) must be written with +explicit decimal in test fixtures — otherwise decode throws +`DecodingError.typeMismatch: expected value of type Double`: + +```pkl +# BAD — pkl-swift fails to decode +figma { timeout = 60 } + +# GOOD +figma { timeout = 60.0 } +``` + +This is independent of how PKL itself parses the value; the breakage is in pkl-swift's +strict type matching, not in PKL semantics. + +### PKL constraints validate at amends-time, not at sanitize-time + +`Int(isBetween(1, 50))` constraints in `Schemas/*.pkl` fail during PKL evaluation, BEFORE +`FaultToleranceValidator.sanitized*` ever runs. Fixtures with `parallel = 99` will fail +to load entirely, not produce a clamped value: + +```swift +// BAD — fixture never loads, test asserts wrong condition +let url = try BatchResolverFixture.make(batch: "parallel = 99") + +// GOOD — to test sanitizer with out-of-range values, call it directly +XCTAssertEqual(FaultToleranceValidator.sanitizedParallel(99, ui: ui), 3) +``` + +If a test needs both an in-range PKL value AND an out-of-range case, split into two tests. + ## Figma API Rate Limits **Official docs:** https://developers.figma.com/docs/rest-api/rate-limits/