diff --git a/CLAUDE.md b/CLAUDE.md index 7c550ea2..4ab43b81 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -32,7 +32,7 @@ xl-core/ → Pure domain model (Cell, Sheet, Workbook, Patch, Style), ma xl-ooxml/ → Pure OOXML mapping (XlsxReader, XlsxWriter, SharedStrings, Styles) xl-cats-effect/ → IO interpreters and streaming (Excel[F], ExcelIO, SAX-based streaming) xl-benchmarks/ → JMH performance benchmarks -xl-evaluator/ → Formula parser/evaluator (TExpr GADT, 82 functions, dependency graphs) +xl-evaluator/ → Formula parser/evaluator (TExpr GADT, 88 functions, dependency graphs) xl-testkit/ → Test laws, generators, helpers [future] xl-agent/ → AI agent benchmark runner (Anthropic API, skill comparison) ``` @@ -92,7 +92,7 @@ excel.read(path).flatMap(wb => excel.write(wb, outPath)) ```bash ./mill __.compile # Compile all -./mill __.test # Run all tests (731+) +./mill __.test # Run all tests (1080+) ./mill xl-core.test # Test specific module ./mill __.reformat # Format (Scalafmt 3.10.1) ./mill __.checkFormat # CI check @@ -355,7 +355,7 @@ sheet.evaluateFormula("=SUM(A1:A10)") // XLResult[CellValue] sheet.evaluateWithDependencyCheck() // Safe eval with cycle detection ``` -**82 Functions**: SUM, SUMIF, SUMIFS, SUMPRODUCT, COUNT, COUNTA, COUNTBLANK, COUNTIF, COUNTIFS, AVERAGE, AVERAGEIF, AVERAGEIFS, MEDIAN, STDEV, STDEVP, VAR, VARP, MIN, MAX, IF, AND, OR, NOT, ISNUMBER, ISTEXT, ISBLANK, ISERR, ISERROR, CONCATENATE, LEFT, RIGHT, MID, LEN, UPPER, LOWER, TRIM, SUBSTITUTE, TEXT, VALUE, TODAY, NOW, DATE, YEAR, MONTH, DAY, HOUR, MINUTE, SECOND, EOMONTH, ABS, ROUND, ROUNDUP, ROUNDDOWN, INT, MOD, POWER, SQRT, LOG, LN, EXP, FLOOR, CEILING, TRUNC, SIGN, PMT, FV, PV, RATE, NPER, NPV, IRR, VLOOKUP, XLOOKUP, PI, ROW, COLUMN, ROWS, COLUMNS, ADDRESS, TRANSPOSE +**88 Functions**: SUM, SUMIF, SUMIFS, SUMPRODUCT, COUNT, COUNTA, COUNTBLANK, COUNTIF, COUNTIFS, AVERAGE, AVERAGEIF, AVERAGEIFS, MEDIAN, STDEV, STDEVP, VAR, VARP, MIN, MAX, IF, IFERROR, AND, OR, NOT, ISNUMBER, ISTEXT, ISBLANK, ISERR, ISERROR, CONCATENATE, LEFT, RIGHT, MID, LEN, UPPER, LOWER, TRIM, FIND, SUBSTITUTE, TEXT, VALUE, TODAY, NOW, DATE, YEAR, MONTH, DAY, EOMONTH, EDATE, DATEDIF, NETWORKDAYS, WORKDAY, YEARFRAC, ABS, ROUND, ROUNDUP, ROUNDDOWN, INT, MOD, POWER, SQRT, LOG, LN, EXP, FLOOR, CEILING, TRUNC, SIGN, PMT, FV, PV, RATE, NPER, NPV, IRR, XNPV, XIRR, VLOOKUP, XLOOKUP, INDEX, MATCH, PI, ROW, COLUMN, ROWS, COLUMNS, ADDRESS, TRANSPOSE ### Rich Text ```scala @@ -387,12 +387,12 @@ Styles deduplicated by `CellStyle.canonicalKey`. Build style index before emitti **Framework**: MUnit + ScalaCheck | **Generators**: `xl-core/test/src/com/tjclp/xl/Generators.scala` -**980+ tests**: addressing (17), patch (21), style (60), datetime (8), codec (42), batch (46), syntax (18), optics (34), OOXML (24), streaming (18), RichText (5), formula (51+), v0.3.0 regressions (36), CLI (100+) +**1080+ tests**: addressing (17), patch (21), style (60), datetime (8), codec (42), batch (46), syntax (18), optics (34), OOXML (24), streaming (18), RichText (5), formula (51+), v0.3.0 regressions (36), CLI (100+) ## Documentation - **Roadmap**: `docs/plan/roadmap.md` (single source of truth for work scheduling) -- **Status**: `docs/STATUS.md` (current capabilities, 980+ tests) +- **Status**: `docs/STATUS.md` (current capabilities, 1080+ tests) - **Design**: `docs/design/*.md` (architecture, purity charter, domain model) - **Reference**: `docs/reference/*.md` (examples, scaffolds, performance guide) diff --git a/docs/LIMITATIONS.md b/docs/LIMITATIONS.md index d5a2e534..eff46817 100644 --- a/docs/LIMITATIONS.md +++ b/docs/LIMITATIONS.md @@ -1,7 +1,7 @@ # XL Current Limitations and Future Roadmap **Last Updated**: 2025-12-27 (Docs Cleanup) -**Current Phase**: Core domain + OOXML + streaming I/O complete; formula system complete (**81 functions** + cross-sheet support); tables + benchmarks complete; row/column serialization complete; **security hardening complete** (ZIP bomb detection, XXE prevention, formula injection guards in both in-memory and streaming writes). +**Current Phase**: Core domain + OOXML + streaming I/O complete; formula system complete (**88 functions** + cross-sheet support); tables + benchmarks complete; row/column serialization complete; **security hardening complete** (ZIP bomb detection, XXE prevention, formula injection guards in both in-memory and streaming writes). This document provides a comprehensive overview of what XL can and cannot do today, with clear links to future implementation plans. @@ -113,7 +113,7 @@ This document provides a comprehensive overview of what XL can and cannot do tod #### 6. Formula System ✅ **PRODUCTION READY** **Status**: Complete (WI-07, WI-08, WI-09a-h + TJC-351 cross-sheet formulas) -**Features**: Parser, evaluator, **81 functions** (including SUMIF, COUNTIF, SUMIFS, COUNTIFS, XLOOKUP, INDEX, MATCH, XIRR, XNPV), dependency graph, cycle detection, cross-sheet references +**Features**: Parser, evaluator, **88 functions** (including SUMIF, COUNTIF, SUMIFS, COUNTIFS, XLOOKUP, INDEX, MATCH, XIRR, XNPV), dependency graph, cycle detection, cross-sheet references **Plan**: [Formula System](plan/formula-system.md) **Phase**: WI-07, WI-08, WI-09a/b/c/d Complete + Financial Functions + Cross-Sheet Formulas @@ -700,7 +700,7 @@ See: [plan/23-security.md](plan/23-security.md) | **Streaming Read** | ✅ | ✅ | XL: 55k rows/s, POI: ~40k rows/s | | **Multi-sheet** | ✅ | ✅ | XL: Arbitrary, POI: Sequential | | **Styles** | ✅ | ✅ | XL: Full in-memory; streaming uses minimal default styles | -| **Formulas (eval)** | ✅ | ✅ | XL: 81 functions, dependency graph, cycle detection | +| **Formulas (eval)** | ✅ | ✅ | XL: 88 functions, dependency graph, cycle detection | | **Tables** | ✅ | ✅ | XL: Full table support with AutoFilter, structured refs | | **Charts** | ❌ | ✅ | POI: Full support | | **Drawings** | ❌ | ✅ | POI: Images/shapes | @@ -757,7 +757,7 @@ SAX parsing is inherently synchronous - the `parser.parse()` call blocks until t - Multi-sheet workbooks - Core cell types and rich text - Styling in in-memory workflows (full styles supported) -- Formula evaluation (81 functions, dependency graph, cycle detection) +- Formula evaluation (88 functions, dependency graph, cycle detection) - Excel Tables (structured data with AutoFilter, headers, styling) - Performance-critical workloads (benchmarked vs POI) diff --git a/docs/STATUS.md b/docs/STATUS.md index 4588850a..95fda51b 100644 --- a/docs/STATUS.md +++ b/docs/STATUS.md @@ -42,7 +42,7 @@ - ✅ HTML export: `sheet.toHtml(range"A1:B10")` - ✅ **Formula Parsing** (WI-07 complete): TExpr GADT, FormulaParser, FormulaPrinter with round-trip verification and scientific notation - ✅ **Formula Evaluation** (WI-08 complete): Pure functional evaluator with total error handling, short-circuit semantics, and Excel-compatible behavior -- ✅ **Function Library** (WI-09a-h complete): **81 built-in functions** (aggregate, conditional, logical, text, date, financial, lookup, math), extensible type class parser, evaluation API +- ✅ **Function Library** (WI-09a-h + TJC-1055 complete): **88 built-in functions** (aggregate, conditional, logical, text, date, financial, lookup, math), extensible type class parser, evaluation API. Text functions include TRIM, MID, FIND, SUBSTITUTE, VALUE, TEXT (added in TJC-1055 / GH-116). - ✅ **Dependency Graph** (WI-09d complete): Circular reference detection (Tarjan's SCC), topological sort (Kahn's algorithm), safe evaluation with cycle detection - ✅ **Cross-Sheet Formula References** (TJC-351): Single cell refs (`=Sales!A1`), range refs (`=SUM(Sales!A1:A10)`), arithmetic with cross-sheet refs, workbook-level cycle detection (`DependencyGraph.fromWorkbook`) @@ -78,7 +78,7 @@ ### Test Coverage -**980+ tests across 6 modules** (includes P7+P8 string interpolation + WI-07/08/09/09d formula system + TJC-351 cross-sheet formulas + WI-10 table support + WI-15 benchmarks + WI-17 SAX streaming write + v0.3.0 regressions): +**1080+ tests across 6 modules** (includes P7+P8 string interpolation + WI-07/08/09/09d formula system + TJC-351 cross-sheet formulas + WI-10 table support + WI-15 benchmarks + WI-17 SAX streaming write + v0.3.0 regressions + TJC-1055 text functions): - **xl-core**: ~500+ tests - 17 addressing (Column, Row, ARef, CellRange laws) - 21 patch (Monoid laws, application semantics) @@ -102,7 +102,7 @@ - **xl-cats-effect**: ~30+ tests - True streaming I/O with fs2-data-xml (constant memory, 100k+ rows) - Memory tests (O(1) verification, concurrent streams) -- **xl-evaluator**: ~280 tests (parser, evaluator, function library, evaluation API, dependency graph, cross-sheet formulas, integration) +- **xl-evaluator**: ~338 tests (parser, evaluator, function library, evaluation API, dependency graph, cross-sheet formulas, integration) - **Parser (WI-07)**: 57 tests - 7 property-based round-trip tests (parse ∘ print = id) - 26 parser unit tests (literals, operators, functions, edge cases) @@ -130,16 +130,17 @@ **Formula System** (WI-07, WI-08, WI-09a/b/c/d - Production Ready): - ✅ **Parsing** (WI-07): Typed AST (TExpr GADT), FormulaParser, FormulaPrinter, round-trip verification, 57 tests - ✅ **Evaluation** (WI-08): Pure functional evaluator, total error handling, short-circuit semantics, 58 tests -- ✅ **Function Library** (WI-09a-h complete): **81 built-in functions**, extensible type class parser, evaluation API, 174 tests - - **Aggregate** (9): SUM, COUNT, COUNTA, COUNTBLANK, AVERAGE, MEDIAN, MIN, MAX, STDEV, STDEVP, VAR, VARP - - **Conditional** (6): SUMIF, COUNTIF, SUMIFS, COUNTIFS, AVERAGEIF, AVERAGEIFS, SUMPRODUCT - - **Logical** (8): IF, AND, OR, NOT, ISNUMBER, ISTEXT, ISBLANK, ISERR, ISERROR - - **Text** (12): CONCATENATE, LEFT, RIGHT, MID, LEN, UPPER, LOWER, TRIM, SUBSTITUTE, TEXT, VALUE - - **Date** (13): TODAY, NOW, DATE, YEAR, MONTH, DAY, HOUR, MINUTE, SECOND, EOMONTH, EDATE, DATEDIF, NETWORKDAYS, WORKDAY, YEARFRAC +- ✅ **Function Library** (WI-09a-h + TJC-1055 complete): **88 built-in functions**, extensible type class parser, evaluation API, 236 tests + - **Aggregate** (12): SUM, COUNT, COUNTA, COUNTBLANK, AVERAGE, MEDIAN, MIN, MAX, STDEV, STDEVP, VAR, VARP + - **Conditional** (7): SUMIF, COUNTIF, SUMIFS, COUNTIFS, AVERAGEIF, AVERAGEIFS, SUMPRODUCT + - **Logical** (9): IF, IFERROR, AND, OR, NOT, ISNUMBER, ISTEXT, ISBLANK, ISERR, ISERROR + - **Text** (12): CONCATENATE, LEFT, RIGHT, MID, LEN, UPPER, LOWER, TRIM, FIND, SUBSTITUTE, TEXT, VALUE + - **Date** (12): TODAY, NOW, DATE, YEAR, MONTH, DAY, EOMONTH, EDATE, DATEDIF, NETWORKDAYS, WORKDAY, YEARFRAC - **Math** (16): ABS, ROUND, ROUNDUP, ROUNDDOWN, INT, MOD, POWER, SQRT, LOG, LN, EXP, FLOOR, CEILING, TRUNC, SIGN, PI - - **Financial** (7): NPV, IRR, XNPV, XIRR, PMT, FV, PV, RATE, NPER + - **Financial** (9): NPV, IRR, XNPV, XIRR, PMT, FV, PV, RATE, NPER - **Lookup** (4): VLOOKUP, XLOOKUP, INDEX, MATCH - - **Info** (4): ROW, COLUMN, ROWS, COLUMNS, ADDRESS + - **Info** (5): ROW, COLUMN, ROWS, COLUMNS, ADDRESS + - **Array** (1): TRANSPOSE - FunctionSpec registry: macro-collected specs with extensible registry - APIs: sheet.evaluateFormula(), sheet.evaluateCell(), sheet.evaluateAllFormulas() - Clock trait for pure date/time functions (deterministic testing) @@ -210,7 +211,7 @@ - ✅ P7: String interpolation Phase 1 (runtime validation for all macros) - ✅ P8: String interpolation Phase 2 (compile-time optimization) - ✅ P31: Optics, RichText, HTML export, enhanced ergonomics -- ✅ **Formula System** (WI-07/08/09): Parser, evaluator, 81 functions, dependency graph, cycle detection +- ✅ **Formula System** (WI-07/08/09): Parser, evaluator, 88 functions, dependency graph, cycle detection - ✅ **Excel Tables** (WI-10): Structured data with headers, AutoFilter, styling - ✅ **Benchmarks** (WI-15): JMH performance suite (XL vs POI) - ✅ **SAX Write** (WI-17): Fast SAX/StAX streaming write path @@ -304,7 +305,7 @@ xl-cats-effect/src/com/tjclp/xl/io/ ``` ### Completed Modules (Additional) -- `xl-evaluator/` ✅ **Complete** (WI-07/08/09 - formula parsing, evaluation, 81 functions, dependency graph) +- `xl-evaluator/` ✅ **Complete** (WI-07/08/09 - formula parsing, evaluation, 88 functions, dependency graph) - `xl-benchmarks/` ✅ **Complete** (WI-15 - JMH performance benchmarks) ### Not Started (Future Phases) diff --git a/docs/design/architecture.md b/docs/design/architecture.md index 1c7c22c1..34e1c97d 100644 --- a/docs/design/architecture.md +++ b/docs/design/architecture.md @@ -184,6 +184,6 @@ The evaluator implements: `Evaluator.eval: TExpr[A] => Sheet => Either[EvalError - Topological sort for evaluation order (Kahn's algorithm) - Short-circuit evaluation for And/Or - Division by zero handling (returns `CellError.Div0`) -- 81 Excel functions: SUM, AVERAGE, IF, VLOOKUP, XLOOKUP, SUMIF, COUNTIF, NPV, IRR, and more +- 88 Excel functions: SUM, AVERAGE, IF, VLOOKUP, XLOOKUP, SUMIF, COUNTIF, NPV, IRR, and more See `docs/STATUS.md` for the complete function list. diff --git a/docs/plan/roadmap.md b/docs/plan/roadmap.md index e8a7732d..2a36b619 100644 --- a/docs/plan/roadmap.md +++ b/docs/plan/roadmap.md @@ -8,7 +8,7 @@ ## TL;DR -**Current Status**: Production-ready with **81 formula functions**, SAX streaming (36% faster than POI), Excel tables, and full OOXML round-trip. 733+ tests passing. +**Current Status**: Production-ready with **88 formula functions**, SAX streaming (36% faster than POI), Excel tables, and full OOXML round-trip. 1080+ tests passing. **Current Version**: 0.6.1 @@ -73,7 +73,8 @@ CLI expansion with 7 new commands and evaluator fixes: All completed phases are documented in git history. Key milestones: - **P0-P8**: Foundation, OOXML, streaming, codecs, macros -- **WI-07/08/09**: Formula parser, evaluator, 81 functions +- **WI-07/08/09**: Formula parser, evaluator, 88 functions +- **TJC-1055** (closes GH-116): Text functions — TRIM, MID, FIND, SUBSTITUTE, VALUE, TEXT (88 functions total) - **WI-10**: Excel table support - **WI-17**: SAX streaming write (36% faster than POI) - **WI-19**: Row/column property serialization diff --git a/docs/reference/examples.md b/docs/reference/examples.md index 67b4e086..cc13b82c 100644 --- a/docs/reference/examples.md +++ b/docs/reference/examples.md @@ -252,6 +252,7 @@ scala-cli examples/dependency-analysis.sc scala-cli examples/data-validation.sc scala-cli examples/sales-pipeline.sc scala-cli examples/evaluator-demo.sc +scala-cli examples/text_functions_demo.sc # TRIM, MID, FIND, SUBSTITUTE, VALUE, TEXT ``` ## 4) Chart spec (Future - WI-11) diff --git a/examples/text_functions_demo.sc b/examples/text_functions_demo.sc new file mode 100644 index 00000000..1170e8aa --- /dev/null +++ b/examples/text_functions_demo.sc @@ -0,0 +1,117 @@ +#!/usr/bin/env -S scala-cli shebang +//> using file project.scala + + +// Demonstrates the 6 text functions added in TJC-1055 / GH-116: +// TRIM, MID, FIND, SUBSTITUTE, VALUE, TEXT +// +// Each section: build a small workbook, apply realistic formulas, and print +// "formula = result (expected: ...)" so any divergence pops out visually. +// +// Run with: +// 1. Publish locally: ./mill xl.publishLocal +// 2. Run script: scala-cli run examples/text_functions_demo.sc + +import com.tjclp.xl.{*, given} +import com.tjclp.xl.cells.CellValue + +println("=== XL Text Functions Demo (TJC-1055 / GH-116) ===\n") + +/** Evaluate a formula on the given sheet and stringify the result. */ +def eval(formula: String, sheet: Sheet): String = + sheet.evaluateFormula(formula) match + case Right(CellValue.Text(s)) => s"\"$s\"" + case Right(CellValue.Number(n)) => n.toString + case Right(CellValue.Bool(b)) => b.toString + case Right(other) => other.toString + case Left(err) => s"" + +/** Print a formula result alongside the expected value. Mismatches visually pop. */ +def show(formula: String, sheet: Sheet, expected: String): Unit = + val got = eval(formula, sheet) + val mark = if got == expected then "✓" else "✗" + println(f" $mark%s $formula%-50s = $got%-30s (expected: $expected)") + + +// ===================================================================== +// 1. TRIM + SUBSTITUTE — clean messy CSV-imported data +// ===================================================================== +println("\n--- 1. Cleanup pipeline (TRIM, SUBSTITUTE) ---") + +val cleanup = Sheet("Cleanup") + .put(ref"A1", CellValue.Text(" alice@example.com ")) + .put(ref"A2", CellValue.Text("Name: Bob; Age: 42")) + .put(ref"A3", CellValue.Text("a,b,,c,,,d")) + +show("=TRIM(A1)", cleanup, "\"alice@example.com\"") +show("=SUBSTITUTE(A2, \"; \", \" | \")", cleanup, "\"Name: Bob | Age: 42\"") +show("=SUBSTITUTE(A3, \",,\", \",\")", cleanup, "\"a,b,c,,d\"") +show("=SUBSTITUTE(SUBSTITUTE(A3, \",,\", \",\"), \",,\", \",\")", cleanup, "\"a,b,c,d\"") + + +// ===================================================================== +// 2. VALUE — parse currency / percent / accounting strings +// ===================================================================== +println("\n--- 2. Numeric parsing (VALUE) ---") + +val parsing = Sheet("Parsing") + .put(ref"A1", CellValue.Text("$1,234.56")) + .put(ref"A2", CellValue.Text("(500)")) + .put(ref"A3", CellValue.Text("45.5%")) + .put(ref"A4", CellValue.Text(" $-1,000 ")) + +show("=VALUE(A1)", parsing, "1234.56") +show("=VALUE(A2)", parsing, "-500") +show("=VALUE(A3)", parsing, "0.455") +show("=VALUE(A4)", parsing, "-1000") + + +// ===================================================================== +// 3. TEXT — format numbers / dates for display +// ===================================================================== +println("\n--- 3. Display formatting (TEXT) ---") + +val formatting = Sheet("Formatting") + .put(ref"A1", CellValue.Number(BigDecimal("1234567.89"))) + .put(ref"A2", CellValue.Number(BigDecimal("0.075"))) + .put(ref"A3", CellValue.Number(BigDecimal("-1234.5"))) + +show("=TEXT(A1, \"#,##0.00\")", formatting, "\"1,234,567.89\"") +show("=TEXT(A2, \"0.00%\")", formatting, "\"7.50%\"") +show("=TEXT(A3, \"#,##0.00;-#,##0.00\")", formatting, "\"-1,234.50\"") +show("=TEXT(A1, \"0\")", formatting, "\"1234568\"") + + +// ===================================================================== +// 4. FIND + MID — extract email domain (function composition) +// ===================================================================== +println("\n--- 4. Extract email domain (FIND + MID) ---") + +val emails = Sheet("Emails") + .put(ref"A1", CellValue.Text("alice@example.com")) + .put(ref"A2", CellValue.Text("bob@tjclp.com")) + .put(ref"A3", CellValue.Text("charlie+filter@gmail.co.uk")) + +// =MID(A1, FIND("@", A1) + 1, 100) — MID handles overflow by clamping +show("=MID(A1, FIND(\"@\", A1) + 1, 100)", emails, "\"example.com\"") +show("=MID(A2, FIND(\"@\", A2) + 1, 100)", emails, "\"tjclp.com\"") +show("=MID(A3, FIND(\"@\", A3) + 1, 100)", emails, "\"gmail.co.uk\"") + + +// ===================================================================== +// 5. Round-trip: TEXT(VALUE(s)) — normalize messy currency input +// ===================================================================== +println("\n--- 5. Round-trip: messy → number → canonical (TEXT(VALUE(...))) ---") + +val roundtrip = Sheet("Roundtrip") + .put(ref"A1", CellValue.Text("$1,234.56")) + .put(ref"A2", CellValue.Text("(2,500)")) + .put(ref"A3", CellValue.Text("78.9%")) + +show("=TEXT(VALUE(A1), \"#,##0.00\")", roundtrip, "\"1,234.56\"") +show("=TEXT(VALUE(A2), \"#,##0.00;-#,##0.00\")", roundtrip, "\"-2,500.00\"") +show("=TEXT(VALUE(A3), \"0.00%\")", roundtrip, "\"78.90%\"") + + +println("\n=== Demo Complete ===") +println("Tip: change a formula above and re-run to explore behavior.") diff --git a/plugin/skills/xl-cli/SKILL.md b/plugin/skills/xl-cli/SKILL.md index ccbbded8..6984f922 100644 --- a/plugin/skills/xl-cli/SKILL.md +++ b/plugin/skills/xl-cli/SKILL.md @@ -60,7 +60,7 @@ Ensure `~/.local/bin` is in your PATH: `export PATH="$HOME/.local/bin:$PATH"` ### Info Commands (no file required) ```bash -xl functions # List all 82 supported functions +xl functions # List all 88 supported functions xl rasterizers # Check SVG-to-raster backends ``` @@ -392,7 +392,7 @@ xl -f data.xlsx -s Sheet1 eval "=SUM(A1:A10)" --with "A1=500" # What-if xl -f data.xlsx -s Sheet1 eval "=SUM(A1:A5)" --with "A1=0,A5=0" # Multiple overrides (comma-separated) ``` -See [reference/FORMULAS.md](reference/FORMULAS.md) for 82 supported functions. +See [reference/FORMULAS.md](reference/FORMULAS.md) for 88 supported functions. ### Create Formatted Report @@ -534,7 +534,7 @@ xl -f huge.xlsx --max-size 500 cell A1 # Custom 500MB limit | Command | Description | |---------|-------------| -| `functions` | List all 82 supported Excel functions | +| `functions` | List all 88 supported Excel functions | | `rasterizers` | List SVG-to-raster backends with status | ### Workbook Commands @@ -619,6 +619,6 @@ Run `xl sort --help` for sorting details. ## Links - `xl --help` for detailed usage and examples -- [reference/FORMULAS.md](reference/FORMULAS.md) for 82 supported functions +- [reference/FORMULAS.md](reference/FORMULAS.md) for 88 supported functions - [reference/COLORS.md](reference/COLORS.md) for color names - [reference/OUTPUT-FORMATS.md](reference/OUTPUT-FORMATS.md) for format specs diff --git a/plugin/skills/xl-cli/reference/FORMULAS.md b/plugin/skills/xl-cli/reference/FORMULAS.md index 469be4c6..e2765594 100644 --- a/plugin/skills/xl-cli/reference/FORMULAS.md +++ b/plugin/skills/xl-cli/reference/FORMULAS.md @@ -1,6 +1,6 @@ # Supported Formula Functions -The `eval` command supports 81 Excel functions. +The `eval` command supports 88 Excel functions. ## Math Functions @@ -61,6 +61,12 @@ The `eval` command supports 81 Excel functions. | LEN | `=LEN(text)` | `=LEN(A1)` | | UPPER | `=UPPER(text)` | `=UPPER(A1)` | | LOWER | `=LOWER(text)` | `=LOWER(A1)` | +| TRIM | `=TRIM(text)` | `=TRIM(A1)` → strips ASCII spaces, collapses internal runs | +| MID | `=MID(text, start, length)` | `=MID(A1, 2, 3)` → 1-indexed substring | +| FIND | `=FIND(find_text, within_text, [start])` | `=FIND("@", A1)` → 1-indexed position, case-sensitive | +| SUBSTITUTE | `=SUBSTITUTE(text, old, new, [instance])` | `=SUBSTITUTE(A1,"old","new")` → all by default, or Nth | +| VALUE | `=VALUE(text)` | `=VALUE("$1,234.56")` → 1234.56; handles currency, percent, accounting parens | +| TEXT | `=TEXT(value, format_code)` | `=TEXT(A1,"#,##0.00")` → formatted string; supports `0`, `#`, `,`, `%`, `$`, date tokens | ## Date Functions diff --git a/xl-core/src/com/tjclp/xl/display/FormatCodeParser.scala b/xl-core/src/com/tjclp/xl/display/FormatCodeParser.scala index e20ae977..f2eacafb 100644 --- a/xl-core/src/com/tjclp/xl/display/FormatCodeParser.scala +++ b/xl-core/src/com/tjclp/xl/display/FormatCodeParser.scala @@ -429,7 +429,10 @@ object FormatCodeParser: val color = section.condition.collect { case Condition.Color(c) => c } val formatted = applyPattern(effectiveValue, section.pattern) - (formatted, color) + val withDefaultSign = + if value < 0 && format.negative.isEmpty && !formatted.startsWith("-") then s"-$formatted" + else formatted + (withDefaultSign, color) /** * Apply a format pattern to a number. diff --git a/xl-core/test/src/com/tjclp/xl/display/FormatCodeParserSpec.scala b/xl-core/test/src/com/tjclp/xl/display/FormatCodeParserSpec.scala index 6b7d1e84..74ec9b03 100644 --- a/xl-core/test/src/com/tjclp/xl/display/FormatCodeParserSpec.scala +++ b/xl-core/test/src/com/tjclp/xl/display/FormatCodeParserSpec.scala @@ -175,6 +175,12 @@ class FormatCodeParserSpec extends FunSuite: assertEquals(formatted, "1,234.50") } + test("applyFormat: negative with single section preserves default minus sign") { + val code = FormatCodeParser.parse("#,##0.00").toOption.get + val (formatted, _) = FormatCodeParser.applyFormat(BigDecimal("-1234.5"), code) + assertEquals(formatted, "-1,234.50") + } + test("applyFormat: percent") { val code = FormatCodeParser.parse("0%").toOption.get val (formatted, _) = FormatCodeParser.applyFormat(BigDecimal("0.15"), code) diff --git a/xl-evaluator/src/com/tjclp/xl/formula/ast/TExprCoercions.scala b/xl-evaluator/src/com/tjclp/xl/formula/ast/TExprCoercions.scala index 3303933b..7ed4a85a 100644 --- a/xl-evaluator/src/com/tjclp/xl/formula/ast/TExprCoercions.scala +++ b/xl-evaluator/src/com/tjclp/xl/formula/ast/TExprCoercions.scala @@ -19,6 +19,11 @@ trait TExprCoercions: def asStringExpr(expr: TExpr[?]): TExpr[String] = expr match case PolyRef(at, anchor) => Ref(at, anchor, decodeAsString) case SheetPolyRef(sheet, at, anchor) => SheetRef(sheet, at, anchor, decodeAsString) + case TExpr.Lit(value: String) => TExpr.Lit(value) + case TExpr.Lit(value: BigDecimal) => TExpr.Lit(value.toString) + case TExpr.Lit(value: Boolean) => TExpr.Lit(if value then "TRUE" else "FALSE") + case TExpr.Lit(value: java.time.LocalDate) => TExpr.Lit(value.toString) + case TExpr.Lit(value: java.time.LocalDateTime) => TExpr.Lit(value.toString) case other => other.asInstanceOf[TExpr[String]] // Safe: non-PolyRef already has correct type /** @@ -44,15 +49,16 @@ trait TExprCoercions: case PolyRef(at, anchor) => Ref(at, anchor, decodeAsInt) case SheetPolyRef(sheet, at, anchor) => SheetRef(sheet, at, anchor, decodeAsInt) case TExpr.Lit(bd: BigDecimal) if bd.isValidInt => TExpr.Lit(bd.toInt) - // Convert BigDecimal expressions to Int (YEAR/MONTH/DAY/LEN return BigDecimal) - case call: TExpr.Call[?] if call.spec == FunctionSpecs.year => - ToInt(call.asInstanceOf[TExpr[BigDecimal]]) - case call: TExpr.Call[?] if call.spec == FunctionSpecs.month => - ToInt(call.asInstanceOf[TExpr[BigDecimal]]) - case call: TExpr.Call[?] if call.spec == FunctionSpecs.day => - ToInt(call.asInstanceOf[TExpr[BigDecimal]]) - case call: TExpr.Call[?] if call.spec == FunctionSpecs.len => + // Any function call returning BigDecimal (flagged via returnsNumeric) — wrap in ToInt. + // Covers SUM, COUNT, AVERAGE, ROUND, ABS, MOD, ROW, COLUMN, MATCH, PMT, FIND, LEN, + // YEAR/MONTH/DAY, etc. — every numeric-returning function in the registry. + case call: TExpr.Call[?] if call.spec.flags.returnsNumeric => ToInt(call.asInstanceOf[TExpr[BigDecimal]]) + // Arithmetic expressions return BigDecimal — wrap in ToInt to avoid + // a runtime ClassCastException when used in Int-arg positions + // (e.g. =MID(A1, FIND("@", A1) + 1, 100)). + case _: TExpr.Add | _: TExpr.Sub | _: TExpr.Mul | _: TExpr.Div | _: TExpr.Pow => + ToInt(expr.asInstanceOf[TExpr[BigDecimal]]) case other => other.asInstanceOf[TExpr[Int]] // Safe: non-PolyRef already has correct type /** diff --git a/xl-evaluator/src/com/tjclp/xl/formula/ast/TExprTextOps.scala b/xl-evaluator/src/com/tjclp/xl/formula/ast/TExprTextOps.scala index 7a293079..1d1cf34d 100644 --- a/xl-evaluator/src/com/tjclp/xl/formula/ast/TExprTextOps.scala +++ b/xl-evaluator/src/com/tjclp/xl/formula/ast/TExprTextOps.scala @@ -56,3 +56,36 @@ trait TExprTextOps: */ def lower(text: TExpr[String]): TExpr[String] = Call(FunctionSpecs.lower, text) + + /** TRIM whitespace per Excel rules (collapses ASCII space runs; preserves nbsp/tab). */ + def trim(text: TExpr[String]): TExpr[String] = + Call(FunctionSpecs.trim, text) + + /** MID(text, start, len) — 1-indexed substring extraction. */ + def mid(text: TExpr[String], start: TExpr[Int], length: TExpr[Int]): TExpr[String] = + Call(FunctionSpecs.mid, (text, start, length)) + + /** FIND(find_text, within_text, [start_num]) — case-sensitive 1-indexed search. */ + def find( + findText: TExpr[String], + withinText: TExpr[String], + start: Option[TExpr[Int]] = None + ): TExpr[BigDecimal] = + Call(FunctionSpecs.find, (findText, withinText, start)) + + /** SUBSTITUTE(text, old, new, [instance]) — match-by-content text replacement. */ + def substitute( + text: TExpr[String], + oldText: TExpr[String], + newText: TExpr[String], + instance: Option[TExpr[Int]] = None + ): TExpr[String] = + Call(FunctionSpecs.substitute, (text, oldText, newText, instance)) + + /** VALUE(text) — parse text as numeric (handles currency, percent, whitespace). */ + def value(text: TExpr[String]): TExpr[BigDecimal] = + Call(FunctionSpecs.value, text) + + /** TEXT(value, format) — format numeric/date/text using Excel format codes. */ + def text(v: TExpr[Any], format: TExpr[String]): TExpr[String] = + Call(FunctionSpecs.text, (v, format)) diff --git a/xl-evaluator/src/com/tjclp/xl/formula/functions/FunctionSpec.scala b/xl-evaluator/src/com/tjclp/xl/formula/functions/FunctionSpec.scala index 64d4a00b..29067872 100644 --- a/xl-evaluator/src/com/tjclp/xl/formula/functions/FunctionSpec.scala +++ b/xl-evaluator/src/com/tjclp/xl/formula/functions/FunctionSpec.scala @@ -16,7 +16,13 @@ import com.tjclp.xl.workbooks.Workbook */ final case class FunctionFlags( returnsDate: Boolean = false, - returnsTime: Boolean = false + returnsTime: Boolean = false, + /** + * True for functions returning `BigDecimal`. Lets coercion code (e.g. `asIntExpr`) wrap any + * numeric-returning Call in `ToInt` without listing each function by name. Mutually exclusive + * with `returnsDate` / `returnsTime` — a function returns one type. + */ + returnsNumeric: Boolean = false ) final case class ArgPrinter( diff --git a/xl-evaluator/src/com/tjclp/xl/formula/functions/FunctionSpecsAggregate.scala b/xl-evaluator/src/com/tjclp/xl/formula/functions/FunctionSpecsAggregate.scala index 9a706a13..483aeff9 100644 --- a/xl-evaluator/src/com/tjclp/xl/formula/functions/FunctionSpecsAggregate.scala +++ b/xl-evaluator/src/com/tjclp/xl/formula/functions/FunctionSpecsAggregate.scala @@ -141,7 +141,11 @@ trait FunctionSpecsAggregate extends FunctionSpecsBase: private def variadicAggregateSpec( name: String ): FunctionSpec[BigDecimal] { type Args = List[NumericArg] } = - FunctionSpec.simple[BigDecimal, List[NumericArg]](name, Arity.atLeastOne) { (args, ctx) => + FunctionSpec.simple[BigDecimal, List[NumericArg]]( + name, + Arity.atLeastOne, + flags = FunctionFlags(returnsNumeric = true) + ) { (args, ctx) => Aggregator.lookup(name) match case None => Left(EvalError.EvalFailed(s"Unknown aggregator: $name", None)) @@ -264,7 +268,11 @@ trait FunctionSpecsAggregate extends FunctionSpecsBase: variadicAggregateSpec("VARP") val sumif: FunctionSpec[BigDecimal] { type Args = SumIfArgs } = - FunctionSpec.simple[BigDecimal, SumIfArgs]("SUMIF", Arity.Range(2, 3)) { (args, ctx) => + FunctionSpec.simple[BigDecimal, SumIfArgs]( + "SUMIF", + Arity.Range(2, 3), + flags = FunctionFlags(returnsNumeric = true) + ) { (args, ctx) => val (rangeLocation, criteria, sumRangeLocationOpt) = args evalValue(ctx, criteria).flatMap { criteriaValue => val criterion = CriteriaMatcher.parse(criteriaValue) @@ -319,7 +327,11 @@ trait FunctionSpecsAggregate extends FunctionSpecsBase: } val countif: FunctionSpec[BigDecimal] { type Args = CountIfArgs } = - FunctionSpec.simple[BigDecimal, CountIfArgs]("COUNTIF", Arity.two) { (args, ctx) => + FunctionSpec.simple[BigDecimal, CountIfArgs]( + "COUNTIF", + Arity.two, + flags = FunctionFlags(returnsNumeric = true) + ) { (args, ctx) => val (rangeLocation, criteria) = args evalValue(ctx, criteria).flatMap { criteriaValue => val criterion = CriteriaMatcher.parse(criteriaValue) @@ -345,7 +357,11 @@ trait FunctionSpecsAggregate extends FunctionSpecsBase: } val sumifs: FunctionSpec[BigDecimal] { type Args = SumIfsArgs } = - FunctionSpec.simple[BigDecimal, SumIfsArgs]("SUMIFS", Arity.AtLeast(3)) { (args, ctx) => + FunctionSpec.simple[BigDecimal, SumIfsArgs]( + "SUMIFS", + Arity.AtLeast(3), + flags = FunctionFlags(returnsNumeric = true) + ) { (args, ctx) => val (sumRangeLocation, conditions) = args evalCriteriaValues(ctx, conditions) .flatMap { criteriaValues => @@ -441,100 +457,106 @@ trait FunctionSpecsAggregate extends FunctionSpecsBase: } val countifs: FunctionSpec[BigDecimal] { type Args = CountIfsArgs } = - FunctionSpec.simple[BigDecimal, CountIfsArgs]("COUNTIFS", Arity.AtLeast(2)) { - (conditions, ctx) => - evalCriteriaValues(ctx, conditions) - .flatMap { criteriaValues => - val parsedConditions = parseConditions(conditions, criteriaValues) - - parsedConditions match - case Nil => Right(BigDecimal(0)) - case (firstLoc, _) :: rest => - val dimensionError = rest.collectFirst { - case (loc, _) - if loc.range.width != firstLoc.range.width || - loc.range.height != firstLoc.range.height => - EvalError.EvalFailed( - s"COUNTIFS: all ranges must have same dimensions (first is ${firstLoc.range.height}×${firstLoc.range.width}, this is ${loc.range.height}×${loc.range.width})", - Some(s"COUNTIFS(...)") - ) - } + FunctionSpec.simple[BigDecimal, CountIfsArgs]( + "COUNTIFS", + Arity.AtLeast(2), + flags = FunctionFlags(returnsNumeric = true) + ) { (conditions, ctx) => + evalCriteriaValues(ctx, conditions) + .flatMap { criteriaValues => + val parsedConditions = parseConditions(conditions, criteriaValues) + + parsedConditions match + case Nil => Right(BigDecimal(0)) + case (firstLoc, _) :: rest => + val dimensionError = rest.collectFirst { + case (loc, _) + if loc.range.width != firstLoc.range.width || + loc.range.height != firstLoc.range.height => + EvalError.EvalFailed( + s"COUNTIFS: all ranges must have same dimensions (first is ${firstLoc.range.height}×${firstLoc.range.width}, this is ${loc.range.height}×${loc.range.width})", + Some(s"COUNTIFS(...)") + ) + } - dimensionError match - case Some(err) => Left(err) - case None => - // GH-192: Resolve all criteria ranges to their target sheets FIRST - val resolvedConditions: Either[ + dimensionError match + case Some(err) => Left(err) + case None => + // GH-192: Resolve all criteria ranges to their target sheets FIRST + val resolvedConditions: Either[ + EvalError, + List[ + (com.tjclp.xl.sheets.Sheet, TExpr.RangeLocation, CriteriaMatcher.Criterion) + ] + ] = + parsedConditions.foldLeft[Either[ EvalError, List[ - (com.tjclp.xl.sheets.Sheet, TExpr.RangeLocation, CriteriaMatcher.Criterion) + ( + com.tjclp.xl.sheets.Sheet, + TExpr.RangeLocation, + CriteriaMatcher.Criterion + ) ] - ] = - parsedConditions.foldLeft[Either[ - EvalError, - List[ - ( - com.tjclp.xl.sheets.Sheet, - TExpr.RangeLocation, - CriteriaMatcher.Criterion - ) - ] - ]](Right(List.empty)) { - case (Left(err), _) => Left(err) - case (Right(acc), (loc, criterion)) => - Evaluator.resolveRangeLocation(loc, ctx.sheet, ctx.workbook).map { - sheet => - acc :+ (sheet, loc, criterion) - } - } + ]](Right(List.empty)) { + case (Left(err), _) => Left(err) + case (Right(acc), (loc, criterion)) => + Evaluator.resolveRangeLocation(loc, ctx.sheet, ctx.workbook).map { sheet => + acc :+ (sheet, loc, criterion) + } + } - resolvedConditions.flatMap { resolved => - val bounds = computeBounds(resolved.map { case (sheet, loc, _) => - (loc.range, sheet) - }) - // GH-192: Constrain full-column/row ranges to shared bounds - val constrainedConditions = resolved.map { case (sheet, loc, criterion) => - (sheet, constrainRange(loc.range, bounds), criterion) + resolvedConditions.flatMap { resolved => + val bounds = computeBounds(resolved.map { case (sheet, loc, _) => + (loc.range, sheet) + }) + // GH-192: Constrain full-column/row ranges to shared bounds + val constrainedConditions = resolved.map { case (sheet, loc, criterion) => + (sheet, constrainRange(loc.range, bounds), criterion) + } + + // GH-192: Use iterator-based folding with index tracking + val criteriaCells = + constrainedConditions.map { case (sheet, range, criterion) => + (sheet, range.cells.toVector, criterion) } + val refCount = criteriaCells.headOption.map(_._2.length).getOrElse(0) - // GH-192: Use iterator-based folding with index tracking - val criteriaCells = - constrainedConditions.map { case (sheet, range, criterion) => - (sheet, range.cells.toVector, criterion) - } - val refCount = criteriaCells.headOption.map(_._2.length).getOrElse(0) - - (0 until refCount) - .foldLeft[Either[EvalError, Int]](Right(0)) { - case (Left(err), _) => Left(err) - case (Right(count), idx) => - // Check all conditions - val matchResult = - criteriaCells.foldLeft[Either[EvalError, Boolean]](Right(true)) { - case (Left(err), _) => Left(err) - case (Right(false), _) => Right(false) // Short-circuit - case (Right(true), (criteriaSheet, cells, criterion)) => - val testRef = cells(idx) - evalCellValueForMatch( - criteriaSheet(testRef).value, - criteriaSheet, - ctx - ) - .map { testValue => - CriteriaMatcher.matches(testValue, criterion) - } - } - matchResult.map { allMatch => - if allMatch then count + 1 else count + (0 until refCount) + .foldLeft[Either[EvalError, Int]](Right(0)) { + case (Left(err), _) => Left(err) + case (Right(count), idx) => + // Check all conditions + val matchResult = + criteriaCells.foldLeft[Either[EvalError, Boolean]](Right(true)) { + case (Left(err), _) => Left(err) + case (Right(false), _) => Right(false) // Short-circuit + case (Right(true), (criteriaSheet, cells, criterion)) => + val testRef = cells(idx) + evalCellValueForMatch( + criteriaSheet(testRef).value, + criteriaSheet, + ctx + ) + .map { testValue => + CriteriaMatcher.matches(testValue, criterion) + } } - } - .map(BigDecimal(_)) - } - } + matchResult.map { allMatch => + if allMatch then count + 1 else count + } + } + .map(BigDecimal(_)) + } + } } val averageif: FunctionSpec[BigDecimal] { type Args = AverageIfArgs } = - FunctionSpec.simple[BigDecimal, AverageIfArgs]("AVERAGEIF", Arity.Range(2, 3)) { (args, ctx) => + FunctionSpec.simple[BigDecimal, AverageIfArgs]( + "AVERAGEIF", + Arity.Range(2, 3), + flags = FunctionFlags(returnsNumeric = true) + ) { (args, ctx) => val (rangeLocation, criteria, avgRangeLocationOpt) = args evalValue(ctx, criteria).flatMap { criteriaValue => val criterion = CriteriaMatcher.parse(criteriaValue) @@ -594,7 +616,11 @@ trait FunctionSpecsAggregate extends FunctionSpecsBase: } val averageifs: FunctionSpec[BigDecimal] { type Args = AverageIfsArgs } = - FunctionSpec.simple[BigDecimal, AverageIfsArgs]("AVERAGEIFS", Arity.AtLeast(3)) { (args, ctx) => + FunctionSpec.simple[BigDecimal, AverageIfsArgs]( + "AVERAGEIFS", + Arity.AtLeast(3), + flags = FunctionFlags(returnsNumeric = true) + ) { (args, ctx) => val (avgRangeLocation, conditions) = args evalCriteriaValues(ctx, conditions) .flatMap { criteriaValues => @@ -731,7 +757,11 @@ trait FunctionSpecsAggregate extends FunctionSpecsBase: else Right(matrix(row)(col)) val sumproduct: FunctionSpec[BigDecimal] { type Args = SumProductArgs } = - FunctionSpec.simple[BigDecimal, SumProductArgs]("SUMPRODUCT", Arity.atLeastOne) { (args, ctx) => + FunctionSpec.simple[BigDecimal, SumProductArgs]( + "SUMPRODUCT", + Arity.atLeastOne, + flags = FunctionFlags(returnsNumeric = true) + ) { (args, ctx) => import com.tjclp.xl.formula.ast.TExpr args match diff --git a/xl-evaluator/src/com/tjclp/xl/formula/functions/FunctionSpecsBase.scala b/xl-evaluator/src/com/tjclp/xl/formula/functions/FunctionSpecsBase.scala index 98f487ed..88c35b64 100644 --- a/xl-evaluator/src/com/tjclp/xl/formula/functions/FunctionSpecsBase.scala +++ b/xl-evaluator/src/com/tjclp/xl/formula/functions/FunctionSpecsBase.scala @@ -50,6 +50,10 @@ trait FunctionSpecsBase: type BinaryNumericOpt = (TExpr[BigDecimal], Option[TExpr[BigDecimal]]) type UnaryText = TExpr[String] type BinaryTextInt = (TExpr[String], TExpr[Int]) + type TextIntInt = (TExpr[String], TExpr[Int], TExpr[Int]) + type FindArgs = (TExpr[String], TExpr[String], Option[TExpr[Int]]) + type SubstituteArgs = (TExpr[String], TExpr[String], TExpr[String], Option[TExpr[Int]]) + type TextArgs = (TExpr[Any], TExpr[String]) type TextList = List[TExpr[String]] type UnaryBoolean = TExpr[Boolean] type BooleanList = List[TExpr[Boolean]] diff --git a/xl-evaluator/src/com/tjclp/xl/formula/functions/FunctionSpecsDateTime.scala b/xl-evaluator/src/com/tjclp/xl/formula/functions/FunctionSpecsDateTime.scala index 8d946daa..79cf46ca 100644 --- a/xl-evaluator/src/com/tjclp/xl/formula/functions/FunctionSpecsDateTime.scala +++ b/xl-evaluator/src/com/tjclp/xl/formula/functions/FunctionSpecsDateTime.scala @@ -93,17 +93,29 @@ trait FunctionSpecsDateTime extends FunctionSpecsBase: } val year: FunctionSpec[BigDecimal] { type Args = UnaryDate } = - FunctionSpec.simple[BigDecimal, UnaryDate]("YEAR", Arity.one) { (expr, ctx) => + FunctionSpec.simple[BigDecimal, UnaryDate]( + "YEAR", + Arity.one, + flags = FunctionFlags(returnsNumeric = true) + ) { (expr, ctx) => ctx.evalExpr(expr).map(date => BigDecimal(date.getYear)) } val month: FunctionSpec[BigDecimal] { type Args = UnaryDate } = - FunctionSpec.simple[BigDecimal, UnaryDate]("MONTH", Arity.one) { (expr, ctx) => + FunctionSpec.simple[BigDecimal, UnaryDate]( + "MONTH", + Arity.one, + flags = FunctionFlags(returnsNumeric = true) + ) { (expr, ctx) => ctx.evalExpr(expr).map(date => BigDecimal(date.getMonthValue)) } val day: FunctionSpec[BigDecimal] { type Args = UnaryDate } = - FunctionSpec.simple[BigDecimal, UnaryDate]("DAY", Arity.one) { (expr, ctx) => + FunctionSpec.simple[BigDecimal, UnaryDate]( + "DAY", + Arity.one, + flags = FunctionFlags(returnsNumeric = true) + ) { (expr, ctx) => ctx.evalExpr(expr).map(date => BigDecimal(date.getDayOfMonth)) } @@ -137,7 +149,11 @@ trait FunctionSpecsDateTime extends FunctionSpecsBase: } val datedif: FunctionSpec[BigDecimal] { type Args = DatePairUnit } = - FunctionSpec.simple[BigDecimal, DatePairUnit]("DATEDIF", Arity.three) { (args, ctx) => + FunctionSpec.simple[BigDecimal, DatePairUnit]( + "DATEDIF", + Arity.three, + flags = FunctionFlags(returnsNumeric = true) + ) { (args, ctx) => val (startDateExpr, endDateExpr, unitExpr) = args for start <- ctx.evalExpr(startDateExpr) @@ -184,18 +200,21 @@ trait FunctionSpecsDateTime extends FunctionSpecsBase: } val networkdays: FunctionSpec[BigDecimal] { type Args = DatePairOptRange } = - FunctionSpec.simple[BigDecimal, DatePairOptRange]("NETWORKDAYS", Arity.Range(2, 3)) { - (args, ctx) => - val (startDateExpr, endDateExpr, holidaysOpt) = args - for - start <- ctx.evalExpr(startDateExpr) - end <- ctx.evalExpr(endDateExpr) - yield - val holidays = holidaySet(holidaysOpt, ctx) + FunctionSpec.simple[BigDecimal, DatePairOptRange]( + "NETWORKDAYS", + Arity.Range(2, 3), + flags = FunctionFlags(returnsNumeric = true) + ) { (args, ctx) => + val (startDateExpr, endDateExpr, holidaysOpt) = args + for + start <- ctx.evalExpr(startDateExpr) + end <- ctx.evalExpr(endDateExpr) + yield + val holidays = holidaySet(holidaysOpt, ctx) - val (earlier, later) = if start.isBefore(end) then (start, end) else (end, start) - val count = countWorkingDays(earlier, later, holidays) - BigDecimal(if start.isBefore(end) || start.isEqual(end) then count else -count) + val (earlier, later) = if start.isBefore(end) then (start, end) else (end, start) + val count = countWorkingDays(earlier, later, holidays) + BigDecimal(if start.isBefore(end) || start.isEqual(end) then count else -count) } val workday: FunctionSpec[LocalDate] { type Args = DateIntOptRange } = @@ -232,7 +251,8 @@ trait FunctionSpecsDateTime extends FunctionSpecsBase: printer.expr(basisExpr) ) s"YEARFRAC(${rendered.mkString(", ")})" - } + }, + flags = FunctionFlags(returnsNumeric = true) ) { (args, ctx) => val (startDateExpr, endDateExpr, basisOpt) = args val basisValueEither = basisOpt match diff --git a/xl-evaluator/src/com/tjclp/xl/formula/functions/FunctionSpecsFinancialCashflow.scala b/xl-evaluator/src/com/tjclp/xl/formula/functions/FunctionSpecsFinancialCashflow.scala index bad6cd5b..47888773 100644 --- a/xl-evaluator/src/com/tjclp/xl/formula/functions/FunctionSpecsFinancialCashflow.scala +++ b/xl-evaluator/src/com/tjclp/xl/formula/functions/FunctionSpecsFinancialCashflow.scala @@ -23,7 +23,11 @@ trait FunctionSpecsFinancialCashflow extends FunctionSpecsBase: .toList val npv: FunctionSpec[BigDecimal] { type Args = NpvArgs } = - FunctionSpec.simple[BigDecimal, NpvArgs]("NPV", Arity.two) { (args, ctx) => + FunctionSpec.simple[BigDecimal, NpvArgs]( + "NPV", + Arity.two, + flags = FunctionFlags(returnsNumeric = true) + ) { (args, ctx) => val (rateExpr, range) = args ctx.evalExpr(rateExpr).flatMap { rate => val onePlusR = BigDecimal(1) + rate @@ -47,7 +51,11 @@ trait FunctionSpecsFinancialCashflow extends FunctionSpecsBase: } val irr: FunctionSpec[BigDecimal] { type Args = IrrArgs } = - FunctionSpec.simple[BigDecimal, IrrArgs]("IRR", Arity.Range(1, 2)) { (args, ctx) => + FunctionSpec.simple[BigDecimal, IrrArgs]( + "IRR", + Arity.Range(1, 2), + flags = FunctionFlags(returnsNumeric = true) + ) { (args, ctx) => val (range, guessOpt) = args val cashFlows = numericValues(range, ctx) @@ -112,7 +120,11 @@ trait FunctionSpecsFinancialCashflow extends FunctionSpecsBase: } val xnpv: FunctionSpec[BigDecimal] { type Args = XnpvArgs } = - FunctionSpec.simple[BigDecimal, XnpvArgs]("XNPV", Arity.three) { (args, ctx) => + FunctionSpec.simple[BigDecimal, XnpvArgs]( + "XNPV", + Arity.three, + flags = FunctionFlags(returnsNumeric = true) + ) { (args, ctx) => val (rateExpr, valuesRange, datesRange) = args for rate <- ctx.evalExpr(rateExpr) @@ -152,7 +164,11 @@ trait FunctionSpecsFinancialCashflow extends FunctionSpecsBase: } val xirr: FunctionSpec[BigDecimal] { type Args = XirrArgs } = - FunctionSpec.simple[BigDecimal, XirrArgs]("XIRR", Arity.Range(2, 3)) { (args, ctx) => + FunctionSpec.simple[BigDecimal, XirrArgs]( + "XIRR", + Arity.Range(2, 3), + flags = FunctionFlags(returnsNumeric = true) + ) { (args, ctx) => val (valuesRange, datesRange, guessOpt) = args val values = numericValues(valuesRange, ctx) val dates = dateValues(datesRange, ctx) diff --git a/xl-evaluator/src/com/tjclp/xl/formula/functions/FunctionSpecsFinancialTvm.scala b/xl-evaluator/src/com/tjclp/xl/formula/functions/FunctionSpecsFinancialTvm.scala index a7e12107..20997260 100644 --- a/xl-evaluator/src/com/tjclp/xl/formula/functions/FunctionSpecsFinancialTvm.scala +++ b/xl-evaluator/src/com/tjclp/xl/formula/functions/FunctionSpecsFinancialTvm.scala @@ -7,7 +7,11 @@ import com.tjclp.xl.formula.{Clock, Arity} trait FunctionSpecsFinancialTvm extends FunctionSpecsBase: val pmt: FunctionSpec[BigDecimal] { type Args = TvmArgs } = - FunctionSpec.simple[BigDecimal, TvmArgs]("PMT", Arity.Range(3, 5)) { (args, ctx) => + FunctionSpec.simple[BigDecimal, TvmArgs]( + "PMT", + Arity.Range(3, 5), + flags = FunctionFlags(returnsNumeric = true) + ) { (args, ctx) => val (rateExpr, nperExpr, pvExpr, fvOpt, typeOpt) = args for rate <- ctx.evalExpr(rateExpr).map(_.toDouble) @@ -29,7 +33,11 @@ trait FunctionSpecsFinancialTvm extends FunctionSpecsBase: } val fv: FunctionSpec[BigDecimal] { type Args = TvmArgs } = - FunctionSpec.simple[BigDecimal, TvmArgs]("FV", Arity.Range(3, 5)) { (args, ctx) => + FunctionSpec.simple[BigDecimal, TvmArgs]( + "FV", + Arity.Range(3, 5), + flags = FunctionFlags(returnsNumeric = true) + ) { (args, ctx) => val (rateExpr, nperExpr, pmtExpr, pvOpt, typeOpt) = args for rate <- ctx.evalExpr(rateExpr).map(_.toDouble) @@ -50,7 +58,11 @@ trait FunctionSpecsFinancialTvm extends FunctionSpecsBase: } val pv: FunctionSpec[BigDecimal] { type Args = TvmArgs } = - FunctionSpec.simple[BigDecimal, TvmArgs]("PV", Arity.Range(3, 5)) { (args, ctx) => + FunctionSpec.simple[BigDecimal, TvmArgs]( + "PV", + Arity.Range(3, 5), + flags = FunctionFlags(returnsNumeric = true) + ) { (args, ctx) => val (rateExpr, nperExpr, pmtExpr, fvOpt, typeOpt) = args for rate <- ctx.evalExpr(rateExpr).map(_.toDouble) @@ -71,7 +83,11 @@ trait FunctionSpecsFinancialTvm extends FunctionSpecsBase: } val nper: FunctionSpec[BigDecimal] { type Args = TvmArgs } = - FunctionSpec.simple[BigDecimal, TvmArgs]("NPER", Arity.Range(3, 5)) { (args, ctx) => + FunctionSpec.simple[BigDecimal, TvmArgs]( + "NPER", + Arity.Range(3, 5), + flags = FunctionFlags(returnsNumeric = true) + ) { (args, ctx) => val (rateExpr, pmtExpr, pvExpr, fvOpt, typeOpt) = args for rate <- ctx.evalExpr(rateExpr).map(_.toDouble) @@ -95,7 +111,11 @@ trait FunctionSpecsFinancialTvm extends FunctionSpecsBase: } val rate: FunctionSpec[BigDecimal] { type Args = RateArgs } = - FunctionSpec.simple[BigDecimal, RateArgs]("RATE", Arity.Range(3, 6)) { (args, ctx) => + FunctionSpec.simple[BigDecimal, RateArgs]( + "RATE", + Arity.Range(3, 6), + flags = FunctionFlags(returnsNumeric = true) + ) { (args, ctx) => val (nperExpr, pmtExpr, pvExpr, fvOpt, typeOpt, guessOpt) = args for nper <- ctx.evalExpr(nperExpr).map(_.toDouble) diff --git a/xl-evaluator/src/com/tjclp/xl/formula/functions/FunctionSpecsLookupIndex.scala b/xl-evaluator/src/com/tjclp/xl/formula/functions/FunctionSpecsLookupIndex.scala index d499eb7c..a8e91834 100644 --- a/xl-evaluator/src/com/tjclp/xl/formula/functions/FunctionSpecsLookupIndex.scala +++ b/xl-evaluator/src/com/tjclp/xl/formula/functions/FunctionSpecsLookupIndex.scala @@ -88,7 +88,11 @@ trait FunctionSpecsLookupIndex extends FunctionSpecsBase: } val matchFn: FunctionSpec[BigDecimal] { type Args = MatchArgs } = - FunctionSpec.simple[BigDecimal, MatchArgs]("MATCH", Arity.Range(2, 3)) { (args, ctx) => + FunctionSpec.simple[BigDecimal, MatchArgs]( + "MATCH", + Arity.Range(2, 3), + flags = FunctionFlags(returnsNumeric = true) + ) { (args, ctx) => val (lookupValue, lookupArray, matchTypeOpt) = args val matchTypeExpr = matchTypeOpt.getOrElse(TExpr.Lit(BigDecimal(1))) for diff --git a/xl-evaluator/src/com/tjclp/xl/formula/functions/FunctionSpecsMath.scala b/xl-evaluator/src/com/tjclp/xl/formula/functions/FunctionSpecsMath.scala index b282af30..ab5cc879 100644 --- a/xl-evaluator/src/com/tjclp/xl/formula/functions/FunctionSpecsMath.scala +++ b/xl-evaluator/src/com/tjclp/xl/formula/functions/FunctionSpecsMath.scala @@ -19,12 +19,20 @@ trait FunctionSpecsMath extends FunctionSpecsBase: rounded * scale val abs: FunctionSpec[BigDecimal] { type Args = UnaryNumeric } = - FunctionSpec.simple[BigDecimal, UnaryNumeric]("ABS", Arity.one) { (expr, ctx) => + FunctionSpec.simple[BigDecimal, UnaryNumeric]( + "ABS", + Arity.one, + flags = FunctionFlags(returnsNumeric = true) + ) { (expr, ctx) => ctx.evalExpr(expr).map(_.abs) } val sqrt: FunctionSpec[BigDecimal] { type Args = UnaryNumeric } = - FunctionSpec.simple[BigDecimal, UnaryNumeric]("SQRT", Arity.one) { (expr, ctx) => + FunctionSpec.simple[BigDecimal, UnaryNumeric]( + "SQRT", + Arity.one, + flags = FunctionFlags(returnsNumeric = true) + ) { (expr, ctx) => ctx.evalExpr(expr).flatMap { value => if value < 0 then Left( @@ -38,7 +46,11 @@ trait FunctionSpecsMath extends FunctionSpecsBase: } val round: FunctionSpec[BigDecimal] { type Args = BinaryNumeric } = - FunctionSpec.simple[BigDecimal, BinaryNumeric]("ROUND", Arity.two) { (args, ctx) => + FunctionSpec.simple[BigDecimal, BinaryNumeric]( + "ROUND", + Arity.two, + flags = FunctionFlags(returnsNumeric = true) + ) { (args, ctx) => val (valueExpr, numDigitsExpr) = args for value <- ctx.evalExpr(valueExpr) @@ -49,7 +61,8 @@ trait FunctionSpecsMath extends FunctionSpecsBase: val roundUp: FunctionSpec[BigDecimal] { type Args = BinaryNumeric } = FunctionSpec.simple[BigDecimal, BinaryNumeric]( "ROUNDUP", - Arity.two + Arity.two, + flags = FunctionFlags(returnsNumeric = true) ) { (args, ctx) => val (valueExpr, numDigitsExpr) = args for @@ -65,7 +78,8 @@ trait FunctionSpecsMath extends FunctionSpecsBase: val roundDown: FunctionSpec[BigDecimal] { type Args = BinaryNumeric } = FunctionSpec.simple[BigDecimal, BinaryNumeric]( "ROUNDDOWN", - Arity.two + Arity.two, + flags = FunctionFlags(returnsNumeric = true) ) { (args, ctx) => val (valueExpr, numDigitsExpr) = args for @@ -79,7 +93,11 @@ trait FunctionSpecsMath extends FunctionSpecsBase: } val mod: FunctionSpec[BigDecimal] { type Args = BinaryNumeric } = - FunctionSpec.simple[BigDecimal, BinaryNumeric]("MOD", Arity.two) { (args, ctx) => + FunctionSpec.simple[BigDecimal, BinaryNumeric]( + "MOD", + Arity.two, + flags = FunctionFlags(returnsNumeric = true) + ) { (args, ctx) => val (numberExpr, divisorExpr) = args for number <- ctx.evalExpr(numberExpr) @@ -96,7 +114,8 @@ trait FunctionSpecsMath extends FunctionSpecsBase: val power: FunctionSpec[BigDecimal] { type Args = BinaryNumeric } = FunctionSpec.simple[BigDecimal, BinaryNumeric]( "POWER", - Arity.two + Arity.two, + flags = FunctionFlags(returnsNumeric = true) ) { (args, ctx) => val (numberExpr, powerExpr) = args for @@ -108,7 +127,8 @@ trait FunctionSpecsMath extends FunctionSpecsBase: val log: FunctionSpec[BigDecimal] { type Args = BinaryNumericOpt } = FunctionSpec.simple[BigDecimal, BinaryNumericOpt]( "LOG", - Arity.Range(1, 2) + Arity.Range(1, 2), + flags = FunctionFlags(returnsNumeric = true) ) { (args, ctx) => val (numberExpr, baseExprOpt) = args val baseExpr = baseExprOpt.getOrElse(TExpr.Lit(BigDecimal(10))) @@ -137,7 +157,11 @@ trait FunctionSpecsMath extends FunctionSpecsBase: } val ln: FunctionSpec[BigDecimal] { type Args = UnaryNumeric } = - FunctionSpec.simple[BigDecimal, UnaryNumeric]("LN", Arity.one) { (expr, ctx) => + FunctionSpec.simple[BigDecimal, UnaryNumeric]( + "LN", + Arity.one, + flags = FunctionFlags(returnsNumeric = true) + ) { (expr, ctx) => ctx.evalExpr(expr).flatMap { value => if value <= 0 then Left( @@ -148,14 +172,22 @@ trait FunctionSpecsMath extends FunctionSpecsBase: } val exp: FunctionSpec[BigDecimal] { type Args = UnaryNumeric } = - FunctionSpec.simple[BigDecimal, UnaryNumeric]("EXP", Arity.one) { (expr, ctx) => + FunctionSpec.simple[BigDecimal, UnaryNumeric]( + "EXP", + Arity.one, + flags = FunctionFlags(returnsNumeric = true) + ) { (expr, ctx) => ctx.evalExpr(expr).map { value => BigDecimal(Math.exp(value.toDouble)) } } val floor: FunctionSpec[BigDecimal] { type Args = BinaryNumeric } = - FunctionSpec.simple[BigDecimal, BinaryNumeric]("FLOOR", Arity.two) { (args, ctx) => + FunctionSpec.simple[BigDecimal, BinaryNumeric]( + "FLOOR", + Arity.two, + flags = FunctionFlags(returnsNumeric = true) + ) { (args, ctx) => val (numberExpr, significanceExpr) = args for number <- ctx.evalExpr(numberExpr) @@ -184,7 +216,8 @@ trait FunctionSpecsMath extends FunctionSpecsBase: val ceiling: FunctionSpec[BigDecimal] { type Args = BinaryNumeric } = FunctionSpec.simple[BigDecimal, BinaryNumeric]( "CEILING", - Arity.two + Arity.two, + flags = FunctionFlags(returnsNumeric = true) ) { (args, ctx) => val (numberExpr, significanceExpr) = args for @@ -215,7 +248,8 @@ trait FunctionSpecsMath extends FunctionSpecsBase: val trunc: FunctionSpec[BigDecimal] { type Args = BinaryNumericOpt } = FunctionSpec.simple[BigDecimal, BinaryNumericOpt]( "TRUNC", - Arity.Range(1, 2) + Arity.Range(1, 2), + flags = FunctionFlags(returnsNumeric = true) ) { (args, ctx) => val (numberExpr, numDigitsExprOpt) = args val numDigitsExpr = numDigitsExprOpt.getOrElse(TExpr.Lit(BigDecimal(0))) @@ -226,7 +260,11 @@ trait FunctionSpecsMath extends FunctionSpecsBase: } val sign: FunctionSpec[BigDecimal] { type Args = UnaryNumeric } = - FunctionSpec.simple[BigDecimal, UnaryNumeric]("SIGN", Arity.one) { (expr, ctx) => + FunctionSpec.simple[BigDecimal, UnaryNumeric]( + "SIGN", + Arity.one, + flags = FunctionFlags(returnsNumeric = true) + ) { (expr, ctx) => ctx.evalExpr(expr).map { value => if value > 0 then BigDecimal(1) else if value < 0 then BigDecimal(-1) @@ -235,11 +273,19 @@ trait FunctionSpecsMath extends FunctionSpecsBase: } val int: FunctionSpec[BigDecimal] { type Args = UnaryNumeric } = - FunctionSpec.simple[BigDecimal, UnaryNumeric]("INT", Arity.one) { (expr, ctx) => + FunctionSpec.simple[BigDecimal, UnaryNumeric]( + "INT", + Arity.one, + flags = FunctionFlags(returnsNumeric = true) + ) { (expr, ctx) => ctx.evalExpr(expr).map(_.setScale(0, BigDecimal.RoundingMode.FLOOR)) } val pi: FunctionSpec[BigDecimal] { type Args = NoArgs } = - FunctionSpec.simple[BigDecimal, NoArgs]("PI", Arity.none) { (_, _) => + FunctionSpec.simple[BigDecimal, NoArgs]( + "PI", + Arity.none, + flags = FunctionFlags(returnsNumeric = true) + ) { (_, _) => Right(BigDecimal(Math.PI)) } diff --git a/xl-evaluator/src/com/tjclp/xl/formula/functions/FunctionSpecsReference.scala b/xl-evaluator/src/com/tjclp/xl/formula/functions/FunctionSpecsReference.scala index b2262cc6..bf142ea7 100644 --- a/xl-evaluator/src/com/tjclp/xl/formula/functions/FunctionSpecsReference.scala +++ b/xl-evaluator/src/com/tjclp/xl/formula/functions/FunctionSpecsReference.scala @@ -34,7 +34,11 @@ trait FunctionSpecsReference extends FunctionSpecsBase: else columnToLetter(quotient, letter.toString + acc) val row: FunctionSpec[BigDecimal] { type Args = Option[AnyExpr] } = - FunctionSpec.simple[BigDecimal, Option[AnyExpr]]("ROW", Arity.Range(0, 1)) { (exprOpt, ctx) => + FunctionSpec.simple[BigDecimal, Option[AnyExpr]]( + "ROW", + Arity.Range(0, 1), + flags = FunctionFlags(returnsNumeric = true) + ) { (exprOpt, ctx) => exprOpt match case Some(expr) => extractARef(expr) match @@ -60,34 +64,41 @@ trait FunctionSpecsReference extends FunctionSpecsBase: } val column: FunctionSpec[BigDecimal] { type Args = Option[AnyExpr] } = - FunctionSpec.simple[BigDecimal, Option[AnyExpr]]("COLUMN", Arity.Range(0, 1)) { - (exprOpt, ctx) => - exprOpt match - case Some(expr) => - extractARef(expr) match - case Some(aref) => Right(BigDecimal(aref.col.index0 + 1)) - case None => - Left( - EvalError.EvalFailed( - "COLUMN requires a cell reference", - Some(s"COLUMN($expr)") - ) + FunctionSpec.simple[BigDecimal, Option[AnyExpr]]( + "COLUMN", + Arity.Range(0, 1), + flags = FunctionFlags(returnsNumeric = true) + ) { (exprOpt, ctx) => + exprOpt match + case Some(expr) => + extractARef(expr) match + case Some(aref) => Right(BigDecimal(aref.col.index0 + 1)) + case None => + Left( + EvalError.EvalFailed( + "COLUMN requires a cell reference", + Some(s"COLUMN($expr)") ) - case None => - // Zero-argument form: COLUMN() returns column of current cell - ctx.currentCell match - case Some(ref) => Right(BigDecimal(ref.col.index0 + 1)) - case None => - Left( - EvalError.EvalFailed( - "COLUMN() with no arguments requires a cell context", - Some("COLUMN()") - ) + ) + case None => + // Zero-argument form: COLUMN() returns column of current cell + ctx.currentCell match + case Some(ref) => Right(BigDecimal(ref.col.index0 + 1)) + case None => + Left( + EvalError.EvalFailed( + "COLUMN() with no arguments requires a cell context", + Some("COLUMN()") ) + ) } val rows: FunctionSpec[BigDecimal] { type Args = AnyExpr } = - FunctionSpec.simple[BigDecimal, AnyExpr]("ROWS", Arity.one) { (expr, ctx) => + FunctionSpec.simple[BigDecimal, AnyExpr]( + "ROWS", + Arity.one, + flags = FunctionFlags(returnsNumeric = true) + ) { (expr, ctx) => extractCellRange(expr) match case Some(range) => val rowCount = range.rowEnd.index0 - range.rowStart.index0 + 1 @@ -102,7 +113,11 @@ trait FunctionSpecsReference extends FunctionSpecsBase: } val columns: FunctionSpec[BigDecimal] { type Args = AnyExpr } = - FunctionSpec.simple[BigDecimal, AnyExpr]("COLUMNS", Arity.one) { (expr, ctx) => + FunctionSpec.simple[BigDecimal, AnyExpr]( + "COLUMNS", + Arity.one, + flags = FunctionFlags(returnsNumeric = true) + ) { (expr, ctx) => extractCellRange(expr) match case Some(range) => val colCount = range.colEnd.index0 - range.colStart.index0 + 1 diff --git a/xl-evaluator/src/com/tjclp/xl/formula/functions/FunctionSpecsText.scala b/xl-evaluator/src/com/tjclp/xl/formula/functions/FunctionSpecsText.scala index 1da57633..5693dedd 100644 --- a/xl-evaluator/src/com/tjclp/xl/formula/functions/FunctionSpecsText.scala +++ b/xl-evaluator/src/com/tjclp/xl/formula/functions/FunctionSpecsText.scala @@ -1,9 +1,12 @@ package com.tjclp.xl.formula.functions +import com.tjclp.xl.cells.CellValue +import com.tjclp.xl.display.NumFmtFormatter import com.tjclp.xl.formula.ast.{TExpr, ExprValue} import com.tjclp.xl.formula.eval.{EvalError, Evaluator} import com.tjclp.xl.formula.parser.ParseError import com.tjclp.xl.formula.{Clock, Arity} +import com.tjclp.xl.styles.numfmt.NumFmt trait FunctionSpecsText extends FunctionSpecsBase: val concatenate: FunctionSpec[String] { type Args = TextList } = @@ -45,7 +48,11 @@ trait FunctionSpecsText extends FunctionSpecsBase: } val len: FunctionSpec[BigDecimal] { type Args = UnaryText } = - FunctionSpec.simple[BigDecimal, UnaryText]("LEN", Arity.one) { (expr, ctx) => + FunctionSpec.simple[BigDecimal, UnaryText]( + "LEN", + Arity.one, + flags = FunctionFlags(returnsNumeric = true) + ) { (expr, ctx) => ctx.evalExpr(expr).map(text => BigDecimal(text.length)) } @@ -58,3 +65,168 @@ trait FunctionSpecsText extends FunctionSpecsBase: FunctionSpec.simple[String, UnaryText]("LOWER", Arity.one) { (expr, ctx) => ctx.evalExpr(expr).map(_.toLowerCase) } + + val trim: FunctionSpec[String] { type Args = UnaryText } = + FunctionSpec.simple[String, UnaryText]("TRIM", Arity.one) { (textExpr, ctx) => + ctx.evalExpr(textExpr).map(trimAsciiSpaces) + } + + /** + * Excel TRIM rule: collapse runs of ASCII space (0x20) to a single space and strip leading / + * trailing 0x20s. All other whitespace (tab, newline, nbsp, BOM, ZWSP) is preserved verbatim. + */ + private def trimAsciiSpaces(s: String): String = + s.split(' ').iterator.filter(_.nonEmpty).mkString(" ") + + val mid: FunctionSpec[String] { type Args = TextIntInt } = + FunctionSpec.simple[String, TextIntInt]("MID", Arity.three) { (args, ctx) => + val (textExpr, startExpr, lengthExpr) = args + for + text <- ctx.evalExpr(textExpr) + start <- ctx.evalExpr(startExpr) + length <- ctx.evalExpr(lengthExpr) + result <- + if start < 1 then Left(EvalError.EvalFailed(s"MID: start must be >= 1, got $start")) + else if length < 0 then + Left(EvalError.EvalFailed(s"MID: length must be >= 0, got $length")) + else if start > text.length then Right("") + else + val from = start - 1 + val to = math.min(from.toLong + length.toLong, text.length.toLong).toInt + Right(text.substring(from, to)) + yield result + } + + val find: FunctionSpec[BigDecimal] { type Args = FindArgs } = + FunctionSpec.simple[BigDecimal, FindArgs]( + "FIND", + Arity.Range(2, 3), + flags = FunctionFlags(returnsNumeric = true) + ) { (args, ctx) => + val (findExpr, withinExpr, startOpt) = args + for + needle <- ctx.evalExpr(findExpr) + haystack <- ctx.evalExpr(withinExpr) + start <- startOpt.fold[Either[EvalError, Int]](Right(1))(e => ctx.evalExpr(e)) + result <- + if start < 1 then Left(EvalError.EvalFailed(s"FIND: start must be >= 1, got $start")) + else if start > haystack.length then + // Excel: "If start_num is greater than the length of within_text, + // FIND returns the #VALUE! error value." This applies to both empty + // and non-empty needles — start past length is invalid regardless. + Left(EvalError.EvalFailed(s"FIND: start ($start) is past end of text")) + else if needle.isEmpty then Right(BigDecimal(start)) + else + val idx = haystack.indexOf(needle, start - 1) + if idx < 0 then Left(EvalError.EvalFailed(s"FIND: '$needle' not found in '$haystack'")) + else Right(BigDecimal(idx + 1)) + yield result + } + + val substitute: FunctionSpec[String] { type Args = SubstituteArgs } = + FunctionSpec.simple[String, SubstituteArgs]("SUBSTITUTE", Arity.Range(3, 4)) { (args, ctx) => + val (textExpr, oldExpr, newExpr, instExpr) = args + for + text <- ctx.evalExpr(textExpr) + oldS <- ctx.evalExpr(oldExpr) + newS <- ctx.evalExpr(newExpr) + instOpt <- + instExpr.fold[Either[EvalError, Option[Int]]](Right(None))(e => + ctx.evalExpr(e).map(Some(_)) + ) + result <- substituteImpl(text, oldS, newS, instOpt) + yield result + } + + private def substituteImpl( + text: String, + oldS: String, + newS: String, + instOpt: Option[Int] + ): Either[EvalError, String] = + instOpt match + case Some(n) if n < 1 => + Left(EvalError.EvalFailed(s"SUBSTITUTE: instance must be >= 1, got $n")) + case _ if oldS.isEmpty => Right(text) + case Some(n) => Right(replaceNthOccurrence(text, oldS, newS, n)) + case None => Right(text.replace(oldS, newS)) + + /** Replace only the nth (1-indexed) forward, non-overlapping occurrence. */ + private def replaceNthOccurrence(s: String, oldS: String, newS: String, n: Int): String = + @annotation.tailrec + def findNth(idx: Int, count: Int): Int = + val next = s.indexOf(oldS, idx) + if next < 0 then -1 + else if count == n then next + else findNth(next + oldS.length, count + 1) + val pos = findNth(0, 1) + if pos < 0 then s + else s.substring(0, pos) + newS + s.substring(pos + oldS.length) + + val value: FunctionSpec[BigDecimal] { type Args = UnaryText } = + FunctionSpec.simple[BigDecimal, UnaryText]( + "VALUE", + Arity.one, + flags = FunctionFlags(returnsNumeric = true) + ) { (textExpr, ctx) => + ctx.evalExpr(textExpr).flatMap(parseExcelNumber) + } + + /** + * Parse an Excel-style numeric string. Handles currency ($), thousands commas, percent suffix + * (×1/100), accounting parentheses (negative), scientific notation, sign, and whitespace. + * + * Date and time strings are rejected (deferred per TJC-1055 scope decision). + */ + private def parseExcelNumber(input: String): Either[EvalError, BigDecimal] = + val trimmed = input.trim + if trimmed.isEmpty then Right(BigDecimal(0)) + else + val (negFromParens, afterParens) = + if trimmed.startsWith("(") && trimmed.endsWith(")") then + (true, trimmed.substring(1, trimmed.length - 1)) + else (false, trimmed) + // Reject "(-N)" / "(+N)": accounting parens combined with an inner sign + // double-negates. Excel returns #VALUE! for this pattern. + val innerStartsWithSign = + val inner = afterParens.trim + val afterCurrency = if inner.startsWith("$") then inner.drop(1).trim else inner + val innerHasLeadingSign = inner.startsWith("-") || inner.startsWith("+") + val currencyStrippedHasLeadingSign = + afterCurrency.startsWith("-") || afterCurrency.startsWith("+") + innerHasLeadingSign || currencyStrippedHasLeadingSign + if negFromParens && innerStartsWithSign then + Left(EvalError.EvalFailed(s"VALUE: cannot parse '$input'")) + else + val (isPercent, afterPercent) = + if afterParens.endsWith("%") then (true, afterParens.substring(0, afterParens.length - 1)) + else (false, afterParens) + val cleaned = afterPercent.replace(",", "").replace("$", "").trim + scala.util.Try(BigDecimal(cleaned)).toEither match + case Right(n) => + val signed = if negFromParens then -n else n + Right(if isPercent then signed / 100 else signed) + case Left(_) => + Left(EvalError.EvalFailed(s"VALUE: cannot parse '$input'")) + + val text: FunctionSpec[String] { type Args = TextArgs } = + FunctionSpec.simple[String, TextArgs]("TEXT", Arity.two) { (args, ctx) => + val (valueExpr, formatExpr) = args + for + formatStr <- ctx.evalExpr(formatExpr) + exprValue <- evalValue(ctx, valueExpr) + yield + if formatStr.isEmpty then "" + else + val cv = exprValueForTextFn(exprValue) + NumFmtFormatter.formatValue(cv, NumFmt.Custom(formatStr)) + } + + /** + * Coerce ExprValue → CellValue for TEXT. Empty cells are treated as Number(0) per Excel + * convention; other types pass through the standard toCellValue path. + */ + private def exprValueForTextFn(ev: ExprValue): CellValue = + ev match + case ExprValue.Cell(CellValue.Empty) => CellValue.Number(BigDecimal(0)) + case other => toCellValue(other) diff --git a/xl-evaluator/test/src/com/tjclp/xl/formula/FormulaParserSpec.scala b/xl-evaluator/test/src/com/tjclp/xl/formula/FormulaParserSpec.scala index ae6fdb38..046069c1 100644 --- a/xl-evaluator/test/src/com/tjclp/xl/formula/FormulaParserSpec.scala +++ b/xl-evaluator/test/src/com/tjclp/xl/formula/FormulaParserSpec.scala @@ -1114,7 +1114,7 @@ class FormulaParserSpec extends ScalaCheckSuite: } } - test("Known functions include all 81 functions") { + test("Known functions include all 88 functions") { val functions = FunctionRegistry.allNames assert(functions.contains("SUM")) assert(functions.contains("MIN")) @@ -1199,7 +1199,14 @@ class FormulaParserSpec extends ScalaCheckSuite: assert(functions.contains("NPER")) // Array functions assert(functions.contains("TRANSPOSE")) - assertEquals(functions.length, 82) + // TJC-1055 text functions + assert(functions.contains("TRIM")) + assert(functions.contains("MID")) + assert(functions.contains("FIND")) + assert(functions.contains("SUBSTITUTE")) + assert(functions.contains("VALUE")) + assert(functions.contains("TEXT")) + assertEquals(functions.length, 88) } test("FunctionRegistry.lookup finds spec-based functions") { diff --git a/xl-evaluator/test/src/com/tjclp/xl/formula/TextFunctionsSpec.scala b/xl-evaluator/test/src/com/tjclp/xl/formula/TextFunctionsSpec.scala new file mode 100644 index 00000000..fbc5d2db --- /dev/null +++ b/xl-evaluator/test/src/com/tjclp/xl/formula/TextFunctionsSpec.scala @@ -0,0 +1,597 @@ +package com.tjclp.xl.formula + +import com.tjclp.xl.* +import com.tjclp.xl.cells.{CellError, CellValue} +import com.tjclp.xl.addressing.SheetName +import com.tjclp.xl.formula.ast.TExpr +import com.tjclp.xl.formula.eval.{EvalError, Evaluator} +import com.tjclp.xl.sheets.Sheet +import munit.ScalaCheckSuite +import org.scalacheck.Prop.* +import org.scalacheck.Gen + +import java.time.LocalDate + +/** + * Tests for the 6 text functions added in TJC-1055 / GH-116. + * + * Functions: TRIM, MID, FIND, SUBSTITUTE, VALUE, TEXT. + * + * Each remaining test kills a specific bug class — redundant boundary cases and overlapping + * properties were dropped to bring this spec in line with the repo's per-category density + * (~10 tests / function). Pinning decisions: + * - Type coercion: text functions accept Number / Bool via Excel-style coercion (TRIM(123) == + * "123", TRIM(true) == "TRUE"). + * - Negative TEXT formatting preserves the default minus sign for single-section formats and also + * supports explicit two-section negative formats. + */ +class TextFunctionsSpec extends ScalaCheckSuite: + private val emptySheet = new Sheet(name = SheetName.unsafe("Test")) + private val evaluator = Evaluator.instance + + private def sheetWith(cells: (ARef, CellValue)*): Sheet = + cells.foldLeft(emptySheet) { case (s, (r, v)) => s.put(r, v) } + + /** Number of non-overlapping occurrences of `needle` in `haystack`, scanning forward. */ + private def countOccurrences(haystack: String, needle: String): Int = + if needle.isEmpty then 0 + else + @annotation.tailrec + def loop(idx: Int, acc: Int): Int = + val next = haystack.indexOf(needle, idx) + if next < 0 then acc else loop(next + needle.length, acc + 1) + loop(0, 0) + + /** Small alphanumeric string generator — keeps shrinks readable. */ + private val smallString: Gen[String] = + Gen.choose(0, 20).flatMap(n => Gen.listOfN(n, Gen.alphaNumChar).map(_.mkString)) + + /** Non-empty single-or-multi char needle. */ + private val smallNeedle: Gen[String] = + Gen.choose(1, 4).flatMap(n => Gen.listOfN(n, Gen.alphaNumChar).map(_.mkString)) + + /** + * Generate (haystack, needle) pairs where needle is guaranteed to be a substring of haystack. + * + * Avoids the high-discard-rate trap of `forAll(s, x){ s.contains(x) ==> ... }` where random + * alphanumeric pairs almost never satisfy the precondition. + */ + private val haystackWithNeedle: Gen[(String, String)] = + for + s <- smallString.suchThat(_.nonEmpty) + start <- Gen.choose(0, s.length - 1) + len <- Gen.choose(1, s.length - start) + yield (s, s.substring(start, start + len)) + + // ============================================================ + // §1. TRIM scalars + // ============================================================ + + test("TRIM: collapses internal runs and strips leading/trailing ASCII spaces") { + val expr = TExpr.trim(TExpr.Lit(" hello world ")) + assertEquals(evaluator.eval(expr, emptySheet), Right("hello world")) + } + + test("TRIM: whitespace-only ASCII spaces collapse to empty string") { + val expr = TExpr.trim(TExpr.Lit(" ")) + assertEquals(evaluator.eval(expr, emptySheet), Right("")) + } + + test("TRIM: scalar literals coerce to text") { + assertEquals(emptySheet.evaluateFormula("=TRIM(123)"), Right(CellValue.Text("123"))) + assertEquals(emptySheet.evaluateFormula("=TRIM(TRUE)"), Right(CellValue.Text("TRUE"))) + } + + test("TRIM: collapses ASCII-space runs around a tab; tab is preserved") { + val expr = TExpr.trim(TExpr.Lit("a \t b")) + assertEquals(evaluator.eval(expr, emptySheet), Right("a \t b")) + } + + test("TRIM: internal nbsp run is not collapsed") { + val expr = TExpr.trim(TExpr.Lit("a  b")) + assertEquals(evaluator.eval(expr, emptySheet), Right("a  b")) + } + + test("TRIM: zero-width space (U+200B) is not stripped") { + val expr = TExpr.trim(TExpr.Lit("​hello")) + assertEquals(evaluator.eval(expr, emptySheet), Right("​hello")) + } + + test("TRIM: BOM (U+FEFF) is not stripped") { + val expr = TExpr.trim(TExpr.Lit("hello")) + assertEquals(evaluator.eval(expr, emptySheet), Right("hello")) + } + + // ============================================================ + // §2. MID scalars + // ============================================================ + + test("MID: extracts middle substring (issue golden)") { + val expr = TExpr.mid(TExpr.Lit("Hello"), TExpr.Lit(2), TExpr.Lit(3)) + assertEquals(evaluator.eval(expr, emptySheet), Right("ell")) + } + + test("MID: start one past end returns empty (boundary)") { + val expr = TExpr.mid(TExpr.Lit("Hello"), TExpr.Lit(6), TExpr.Lit(1)) + assertEquals(evaluator.eval(expr, emptySheet), Right("")) + } + + test("MID: start+len beyond length clamps to remainder") { + val expr = TExpr.mid(TExpr.Lit("Hello"), TExpr.Lit(4), TExpr.Lit(100)) + assertEquals(evaluator.eval(expr, emptySheet), Right("lo")) + } + + test("MID: start=0 returns EvalFailed naming MID and start") { + val expr = TExpr.mid(TExpr.Lit("Hello"), TExpr.Lit(0), TExpr.Lit(3)) + val result = evaluator.eval(expr, emptySheet) + assert(result.isLeft, s"start=0 must fail; got $result") + val msg = result.left.toOption.map(_.toString).getOrElse("") + assert(msg.contains("MID") && msg.toLowerCase.contains("start"), s"Error msg: $msg") + } + + test("MID: negative length returns EvalFailed naming MID and length") { + val expr = TExpr.mid(TExpr.Lit("Hello"), TExpr.Lit(1), TExpr.Lit(-1)) + val result = evaluator.eval(expr, emptySheet) + assert(result.isLeft, s"len=-1 must fail; got $result") + val msg = result.left.toOption.map(_.toString).getOrElse("") + assert(msg.contains("MID") && msg.toLowerCase.contains("length"), s"Error msg: $msg") + } + + test("MID: start=Int.MaxValue does not overflow (returns empty)") { + val expr = TExpr.mid(TExpr.Lit("Hello"), TExpr.Lit(Int.MaxValue), TExpr.Lit(3)) + assertEquals(evaluator.eval(expr, emptySheet), Right("")) + } + + // ============================================================ + // §3. FIND scalars + // ============================================================ + + test("FIND: locates first occurrence (1-indexed, issue golden)") { + val expr = TExpr.find(TExpr.Lit("l"), TExpr.Lit("Hello")) + assertEquals(evaluator.eval(expr, emptySheet), Right(BigDecimal(3))) + } + + test("FIND: start parameter advances past first match") { + val expr = TExpr.find(TExpr.Lit("l"), TExpr.Lit("Hello"), Some(TExpr.Lit(4))) + assertEquals(evaluator.eval(expr, emptySheet), Right(BigDecimal(4))) + } + + test("FIND: case-sensitive — lowercase 'h' not found in 'Hello'") { + val expr = TExpr.find(TExpr.Lit("h"), TExpr.Lit("Hello")) + val result = evaluator.eval(expr, emptySheet) + assert(result.isLeft, s"case-sensitive miss must fail; got $result") + } + + test("FIND: regex metachars treated literally (no regex injection)") { + val expr = TExpr.find(TExpr.Lit("."), TExpr.Lit("a.b")) + assertEquals(evaluator.eval(expr, emptySheet), Right(BigDecimal(2))) + } + + test("FIND: empty needle returns start position (Excel quirk)") { + val r1 = evaluator.eval(TExpr.find(TExpr.Lit(""), TExpr.Lit("Hello")), emptySheet) + assertEquals(r1, Right(BigDecimal(1))) + val r2 = evaluator.eval( + TExpr.find(TExpr.Lit(""), TExpr.Lit("Hello"), Some(TExpr.Lit(3))), + emptySheet + ) + assertEquals(r2, Right(BigDecimal(3))) + } + + test("FIND: start=0 fails (Excel min start is 1)") { + val expr = TExpr.find(TExpr.Lit("l"), TExpr.Lit("Hello"), Some(TExpr.Lit(0))) + assert(evaluator.eval(expr, emptySheet).isLeft) + } + + test("FIND: start past end of text fails — Excel: start_num > length → #VALUE!") { + // Empty-needle case is the trap: without the strict bound, =FIND("", "abc", 4) + // would silently succeed and return 4. Excel returns #VALUE! per docs. + val emptyNeedle = + TExpr.find(TExpr.Lit(""), TExpr.Lit("abc"), Some(TExpr.Lit(4))) + assert(evaluator.eval(emptyNeedle, emptySheet).isLeft, "empty-needle past-end must fail") + + val nonEmptyNeedle = + TExpr.find(TExpr.Lit("a"), TExpr.Lit("abc"), Some(TExpr.Lit(4))) + assert(evaluator.eval(nonEmptyNeedle, emptySheet).isLeft, "non-empty-needle past-end must fail") + } + + // ============================================================ + // §4. SUBSTITUTE scalars + // ============================================================ + + test("SUBSTITUTE: replaces all occurrences when instance omitted") { + val expr = TExpr.substitute(TExpr.Lit("Hello"), TExpr.Lit("l"), TExpr.Lit("L")) + assertEquals(evaluator.eval(expr, emptySheet), Right("HeLLo")) + } + + test("SUBSTITUTE: instance=2 replaces only the second occurrence (1-indexed)") { + val expr = + TExpr.substitute(TExpr.Lit("Hello"), TExpr.Lit("l"), TExpr.Lit("L"), Some(TExpr.Lit(2))) + assertEquals(evaluator.eval(expr, emptySheet), Right("HelLo")) + } + + test("SUBSTITUTE: regex metachars in old_text treated literally") { + val expr = TExpr.substitute(TExpr.Lit("a.b.c"), TExpr.Lit("."), TExpr.Lit("X")) + assertEquals(evaluator.eval(expr, emptySheet), Right("aXbXc")) + } + + test("SUBSTITUTE: replacement longer than match — no infinite loop") { + val expr = TExpr.substitute(TExpr.Lit("Hello"), TExpr.Lit("l"), TExpr.Lit("ll")) + assertEquals(evaluator.eval(expr, emptySheet), Right("Hellllo")) + } + + test("SUBSTITUTE: empty old_text returns text unchanged (Excel quirk)") { + val expr = TExpr.substitute(TExpr.Lit("Hello"), TExpr.Lit(""), TExpr.Lit("X")) + assertEquals(evaluator.eval(expr, emptySheet), Right("Hello")) + } + + test("SUBSTITUTE: instance=0 returns EvalFailed") { + val expr = + TExpr.substitute(TExpr.Lit("Hello"), TExpr.Lit("l"), TExpr.Lit("L"), Some(TExpr.Lit(0))) + assert(evaluator.eval(expr, emptySheet).isLeft) + } + + test("SUBSTITUTE: empty old_text with instance<1 still errors (instance validates first)") { + val expr = + TExpr.substitute(TExpr.Lit("Hello"), TExpr.Lit(""), TExpr.Lit("X"), Some(TExpr.Lit(0))) + assert(evaluator.eval(expr, emptySheet).isLeft) + } + + // ============================================================ + // §5. VALUE scalars + // ============================================================ + + test("VALUE: parses decimal with exact precision (kills Double impl)") { + val expr = TExpr.value(TExpr.Lit("123.45")) + assertEquals(evaluator.eval(expr, emptySheet), Right(BigDecimal("123.45"))) + } + + test("VALUE: strips currency symbol and thousands commas") { + val expr = TExpr.value(TExpr.Lit("$1,234.56")) + assertEquals(evaluator.eval(expr, emptySheet), Right(BigDecimal("1234.56"))) + } + + test("VALUE: percent string divided by 100") { + val expr = TExpr.value(TExpr.Lit("45.5%")) + assertEquals(evaluator.eval(expr, emptySheet), Right(BigDecimal("0.455"))) + } + + test("VALUE: accounting parentheses denote negative") { + val expr = TExpr.value(TExpr.Lit("(1,234)")) + assertEquals(evaluator.eval(expr, emptySheet), Right(BigDecimal(-1234))) + } + + test("VALUE: sign + currency interaction ($-1,234.56)") { + val expr = TExpr.value(TExpr.Lit("$-1,234.56")) + assertEquals(evaluator.eval(expr, emptySheet), Right(BigDecimal("-1234.56"))) + } + + test("VALUE: empty string returns 0 (Excel quirk)") { + val expr = TExpr.value(TExpr.Lit("")) + assertEquals(evaluator.eval(expr, emptySheet), Right(BigDecimal(0))) + } + + test("VALUE: accounting parens with inner negative sign is rejected (no double-negate)") { + val expr = TExpr.value(TExpr.Lit("(-5)")) + assert(evaluator.eval(expr, emptySheet).isLeft) + } + + test("VALUE: accounting parens with inner positive sign is rejected") { + val expr = TExpr.value(TExpr.Lit("(+5)")) + assert(evaluator.eval(expr, emptySheet).isLeft) + } + + test("VALUE: accounting parens with inner signed currency is rejected") { + val beforeCurrency = TExpr.value(TExpr.Lit("(-$1,234.56)")) + assert(evaluator.eval(beforeCurrency, emptySheet).isLeft) + + val afterCurrencyNegative = TExpr.value(TExpr.Lit("($-5)")) + assert(evaluator.eval(afterCurrencyNegative, emptySheet).isLeft) + + val afterCurrencyPositive = TExpr.value(TExpr.Lit("($+5)")) + assert(evaluator.eval(afterCurrencyPositive, emptySheet).isLeft) + } + + // ============================================================ + // §6. TEXT scalars + // ============================================================ + + test("TEXT: basic decimal with rounding (issue golden)") { + val expr = TExpr.text(TExpr.Lit(BigDecimal("1234.567")), TExpr.Lit("0.00")) + assertEquals(evaluator.eval(expr, emptySheet), Right("1234.57")) + } + + test("TEXT: half-up rounding (1.555 → '1.56')") { + val expr = TExpr.text(TExpr.Lit(BigDecimal("1.555")), TExpr.Lit("0.00")) + assertEquals(evaluator.eval(expr, emptySheet), Right("1.56")) + } + + test("TEXT: percent format multiplies by 100") { + val expr = TExpr.text(TExpr.Lit(BigDecimal("0.5")), TExpr.Lit("0%")) + assertEquals(evaluator.eval(expr, emptySheet), Right("50%")) + } + + test("TEXT: negative number with single-section format preserves minus sign") { + val expr = TExpr.text(TExpr.Lit(BigDecimal("-1234.5")), TExpr.Lit("0.00")) + assertEquals(evaluator.eval(expr, emptySheet), Right("-1234.50")) + } + + test("TEXT: negative currency via explicit two-section format ('-$1,234.50')") { + // Explicit negative sections use the absolute value and place the sign/currency + // exactly as the format code specifies. + val expr = TExpr.text(TExpr.Lit(BigDecimal("-1234.5")), TExpr.Lit("$#,##0.00;-$#,##0.00")) + assertEquals(evaluator.eval(expr, emptySheet), Right("-$1,234.50")) + } + + test("TEXT: empty format string returns empty (Excel quirk)") { + val expr = TExpr.text(TExpr.Lit(BigDecimal(1234)), TExpr.Lit("")) + assertEquals(evaluator.eval(expr, emptySheet), Right("")) + } + + test("TEXT: text input passes through unchanged") { + val expr = TExpr.text(TExpr.Lit("hello"), TExpr.Lit("0.00")) + assertEquals(evaluator.eval(expr, emptySheet), Right("hello")) + } + + test("TEXT: date input supports date tokens") { + val expr = TExpr.text(TExpr.Lit(LocalDate.of(2025, 1, 15)), TExpr.Lit("yyyy-mm-dd")) + assertEquals(evaluator.eval(expr, emptySheet), Right("2025-01-15")) + } + + // ============================================================ + // §7. Property-based laws (highest-leverage four) + // ============================================================ + + property("TRIM is idempotent: trim(trim(s)) == trim(s)") { + forAll(smallString) { s => + val once = evaluator.eval(TExpr.trim(TExpr.Lit(s)), emptySheet) + val twice = once.flatMap(t => evaluator.eval(TExpr.trim(TExpr.Lit(t)), emptySheet)) + once == twice + } + } + + property("FIND/MID/LEN coupling: MID(s, FIND(x,s), LEN(x)) == x when x ⊆ s") { + forAll(haystackWithNeedle) { case (s, x) => + val findR = evaluator.eval(TExpr.find(TExpr.Lit(x), TExpr.Lit(s)), emptySheet) + val lenR = evaluator.eval(TExpr.len(TExpr.Lit(x)), emptySheet) + val combined = for + k <- findR + n <- lenR + got <- evaluator.eval( + TExpr.mid(TExpr.Lit(s), TExpr.Lit(k.toInt), TExpr.Lit(n.toInt)), + emptySheet + ) + yield got + combined == Right(x) + } + } + + property( + "SUBSTITUTE accounting law: len(sub(s,x,y)) - len(s) == count(x in s)*(len(y)-len(x))" + ) { + forAll(smallString, smallNeedle, smallString) { (s, x, y) => + x.nonEmpty ==> { + val r = evaluator.eval( + TExpr.substitute(TExpr.Lit(s), TExpr.Lit(x), TExpr.Lit(y)), + emptySheet + ) + val c = countOccurrences(s, x) + r.fold(_ => false, t => t.length - s.length == c * (y.length - x.length)) + } + } + } + + property("VALUE/TEXT round-trip: value(text(n, '0.0000;-0.0000')) == n for 4-decimal n") { + // Use unscaled-int construction so generator yields exact 4-decimal BigDecimals, + // and forAllNoShrink so shrinking can't escape the generator's invariants. + val gen = Gen.choose(-10000000L, 10000000L).map(u => BigDecimal(BigInt(u), 4)) + forAllNoShrink(gen) { n => + val r = for + s <- evaluator.eval(TExpr.text(TExpr.Lit(n), TExpr.Lit("0.0000;-0.0000")), emptySheet) + back <- evaluator.eval(TExpr.value(TExpr.Lit(s)), emptySheet) + yield back == n + r.getOrElse(false) + } + } + + // ============================================================ + // §8. Cell-value type matrix — TRIM exercises every CellValue case + // ============================================================ + + test("§8.1 TRIM(A1) where A1 is Empty cell returns ''") { + val sheet = emptySheet + assertEquals(sheet.evaluateFormula("=TRIM(A1)"), Right(CellValue.Text(""))) + } + + test("§8.2 TRIM(A1) where A1 is Number coerces to plain string '123'") { + val sheet = sheetWith(ref"A1" -> CellValue.Number(BigDecimal(123))) + assertEquals(sheet.evaluateFormula("=TRIM(A1)"), Right(CellValue.Text("123"))) + } + + test("§8.3 TRIM(A1) where A1 is Bool coerces to 'TRUE'") { + val sheet = sheetWith(ref"A1" -> CellValue.Bool(true)) + assertEquals(sheet.evaluateFormula("=TRIM(A1)"), Right(CellValue.Text("TRUE"))) + } + + test("§8.4 TRIM(A1) where A1 is Error propagates the error variant") { + val sheet = sheetWith(ref"A1" -> CellValue.Error(CellError.Ref)) + val result = sheet.evaluateFormula("=TRIM(A1)") + assert(result.isLeft, s"error must propagate; got $result") + } + + // ============================================================ + // §9. Composability / nesting + // ============================================================ + + test("§9.1 LEN(TRIM(' hello ')) == 5") { + val sheet = emptySheet + assertEquals( + sheet.evaluateFormula("""=LEN(TRIM(" hello "))"""), + Right(CellValue.Number(BigDecimal(5))) + ) + } + + test("§9.2 ISERROR(FIND('x','abc')) == TRUE — daily safe-search idiom") { + val sheet = emptySheet + assertEquals( + sheet.evaluateFormula("""=ISERROR(FIND("x", "abc"))"""), + Right(CellValue.Bool(true)) + ) + } + + test("§9.3 TEXT(VALUE('$1,234'), '#,##0') pipeline through formula engine") { + val sheet = emptySheet + assertEquals( + sheet.evaluateFormula("""=TEXT(VALUE("$1,234"), "#,##0")"""), + Right(CellValue.Text("1,234")) + ) + } + + test("§9.4 numeric-returning calls work as Int args (returnsNumeric flag)") { + // Pre-flag, only FIND + arithmetic were wrapped; SUM/ROUND/ABS in Int-arg + // positions silently crashed at runtime via the asInstanceOf catch-all. + // After flagging every BigDecimal-returning spec, any of them composes safely. + val sheet = sheetWith( + ref"A1" -> CellValue.Text("Hello, World"), + ref"B1" -> CellValue.Number(BigDecimal(2)), + ref"B2" -> CellValue.Number(BigDecimal(1)), + ref"B3" -> CellValue.Number(BigDecimal(2)) + ) + // =MID(A1, SUM(B1:B3), 5) → start=5 (sum of 2+1+2), substring "o, Wo" + assertEquals( + sheet.evaluateFormula("=MID(A1, SUM(B1:B3), 5)"), + Right(CellValue.Text("o, Wo")) + ) + // =LEFT(A1, ROUND(B1 + 0.4, 0)) → ROUND(2.4, 0) = 2 → "He" + assertEquals( + sheet.evaluateFormula("=LEFT(A1, ROUND(B1 + 0.4, 0))"), + Right(CellValue.Text("He")) + ) + // =MID(A1, ABS(-3), 5) → start=3 → "llo, " + assertEquals( + sheet.evaluateFormula("=MID(A1, ABS(-3), 5)"), + Right(CellValue.Text("llo, ")) + ) + } + + // ============================================================ + // §10. End-to-end via sheet.evaluateFormula (one per function — wiring smoke) + // ============================================================ + + test("§10.1 e2e: =TRIM(A1)") { + val sheet = sheetWith(ref"A1" -> CellValue.Text(" hello world ")) + assertEquals(sheet.evaluateFormula("=TRIM(A1)"), Right(CellValue.Text("hello world"))) + } + + test("§10.2 e2e: =MID(A1, 2, 3)") { + val sheet = sheetWith(ref"A1" -> CellValue.Text("Hello")) + assertEquals(sheet.evaluateFormula("=MID(A1, 2, 3)"), Right(CellValue.Text("ell"))) + } + + test("§10.3 e2e: =FIND(\"o\", A1)") { + val sheet = sheetWith(ref"A1" -> CellValue.Text("Hello")) + assertEquals( + sheet.evaluateFormula("""=FIND("o", A1)"""), + Right(CellValue.Number(BigDecimal(5))) + ) + } + + test("§10.4 e2e: =SUBSTITUTE(A1, \"l\", \"L\")") { + val sheet = sheetWith(ref"A1" -> CellValue.Text("Hello")) + assertEquals( + sheet.evaluateFormula("""=SUBSTITUTE(A1, "l", "L")"""), + Right(CellValue.Text("HeLLo")) + ) + } + + test("§10.5 e2e: =VALUE(A1)") { + val sheet = sheetWith(ref"A1" -> CellValue.Text("$1,234.56")) + assertEquals( + sheet.evaluateFormula("=VALUE(A1)"), + Right(CellValue.Number(BigDecimal("1234.56"))) + ) + } + + test("§10.6 e2e: =TEXT(A1, \"0.00\")") { + val sheet = sheetWith(ref"A1" -> CellValue.Number(BigDecimal("1234.567"))) + assertEquals( + sheet.evaluateFormula("""=TEXT(A1, "0.00")"""), + Right(CellValue.Text("1234.57")) + ) + } + + // ============================================================ + // §11. Parser dispatch edges + // ============================================================ + + test("§11.1 Lowercase function name dispatches via case-insensitive registry") { + val sheet = sheetWith(ref"A1" -> CellValue.Text(" hello ")) + assertEquals(sheet.evaluateFormula("=trim(A1)"), Right(CellValue.Text("hello"))) + } + + test("§11.2 SUBSTITUTE with comma inside string literal preserves arg boundaries") { + val sheet = emptySheet + assertEquals( + sheet.evaluateFormula("""=SUBSTITUTE("a,b", ",", ";")"""), + Right(CellValue.Text("a;b")) + ) + } + + // ============================================================ + // §12. Error-variant fidelity + // ============================================================ + + test("§12.1 MID start=0 message names function and constraint") { + val expr = TExpr.mid(TExpr.Lit("Hi"), TExpr.Lit(0), TExpr.Lit(3)) + val result = evaluator.eval(expr, emptySheet) + val msg = result.left.toOption.map(_.toString).getOrElse("") + assert(msg.contains("MID") && msg.toLowerCase.contains("start"), msg) + } + + test("§12.2 VALUE error echoes the offending input string") { + val expr = TExpr.value(TExpr.Lit("not-a-number")) + val result = evaluator.eval(expr, emptySheet) + val msg = result.left.toOption.map(_.toString).getOrElse("") + assert(msg.contains("VALUE") && msg.contains("not-a-number"), msg) + } + + // ============================================================ + // §13. Sheet dependency graph + // ============================================================ + + test("§13.1 Cell with =TRIM(A1) recalculates when A1 changes") { + val sheet1 = sheetWith(ref"A1" -> CellValue.Text(" one ")) + val r1 = sheet1.evaluateFormula("=TRIM(A1)") + assertEquals(r1, Right(CellValue.Text("one"))) + + val sheet2 = sheet1.put(ref"A1", CellValue.Text(" two ")) + val r2 = sheet2.evaluateFormula("=TRIM(A1)") + assertEquals(r2, Right(CellValue.Text("two"))) + + assertNotEquals(r1, r2) + } + + // ============================================================ + // §14. Determinism / printer round-trip + // ============================================================ + + test("§14.1 FormulaPrinter round-trip for each of the 6 functions") { + val cases: List[String] = List( + """=TRIM(A1)""", + """=MID(A1, 2, 3)""", + """=FIND("o", A1)""", + """=FIND("o", A1, 3)""", + """=SUBSTITUTE(A1, "l", "L")""", + """=SUBSTITUTE(A1, "l", "L", 2)""", + """=VALUE(A1)""", + """=TEXT(A1, "0.00")""" + ) + cases.foreach { src => + val parsed = com.tjclp.xl.formula.parser.FormulaParser.parse(src) + assert(parsed.isRight, s"parse failed: $src — $parsed") + val printed = parsed.toOption + .map(com.tjclp.xl.formula.printer.FormulaPrinter.print(_, includeEquals = true)) + .getOrElse("") + val reparsed = com.tjclp.xl.formula.parser.FormulaParser.parse(printed) + assertEquals(reparsed, parsed, s"round-trip drift: $src -> $printed") + } + }