Skip to content

Honour precision for %e/%E formatting (#13)#95

Open
CedricConday wants to merge 1 commit into
anz-bank:masterfrom
CedricConday:fix/exp-format-precision
Open

Honour precision for %e/%E formatting (#13)#95
CedricConday wants to merge 1 commit into
anz-bank:masterfrom
CedricConday:fix/exp-format-precision

Conversation

@CedricConday

Copy link
Copy Markdown

Closes #13 (the %.10e / %.10E checkbox; %.10f / %.10F was already done).

Problem

The e/E verbs ignored the precision argument entirely — they always emitted the natural (shortest) significand:

d := MustParse64("123456789")
d.Text('e', 3)                 // "1.23456789e+8"  (want "1.235e+8")
fmt.Sprintf("%.3e", d)         // "1.23456789e+8"

This was inconsistent with f/F, which already honour prec, and with the package's stated goal to "conform to standard Go formatting for float types" (DefaultFormatContext64 doc comment).

Fix

Round the significand to prec digits after the leading digit, using the context's rounding mode, then emit exactly that many fractional digits:

  • Rounding reuses the existing Rounding.round + roundStatus primitives, so it respects the configured rounding mode.
  • Zero-padding when prec exceeds the stored digits (%.12e of a 9-digit value).
  • Exponent carry when rounding overflows the leading digit — 9.99 → 1.00e+1.
  • A negative precision keeps the existing natural/shortest form.
  • g/G is untouched: it selects the shortest scientific form, so it routes through the natural path unchanged.

The library keeps its own exponent style (no zero-padding, and a zero exponent is omitted for non-zero values); only the mantissa precision changed.

Tests

  • The three existing Append(_, 'e', 0) assertions were asserting the old behaviour (precision ignored). They now use -1, the natural sentinel, so they still document the shortest form.
  • New TestDecimal64ExpPrecision covers rounding, zero-padding, exponent carry (999999999 → 1e+9), half-to-even on exact decimals (9.95 → 1.0e+1), negatives and zero (0.000e+00), via Text, Append and the fmt.Formatter (%.*e) path.
  • Full suite green (go test ./...), go vet and gofmt clean.

🤖 Generated with Claude Code

https://claude.ai/code/session_01DbUHs855FyBt2J7cG2Qmhh

The `e`/`E` verbs ignored the precision argument and always emitted the
natural (shortest) significand, so `Text('e', 3)`, `Append(buf, 'e', 3)`
and `fmt.Sprintf("%.3e", d)` all produced full-width output instead of
the requested number of fractional digits.

Round the significand to `prec` digits after the leading digit using the
context's rounding mode, zero-padding when `prec` exceeds the available
digits and carrying into the exponent when rounding overflows the leading
digit (e.g. 9.99 -> 1.00e+1). This matches Go's `%.*e` formatting, in line
with the package's stated goal to "conform to standard Go formatting for
float types". A negative precision keeps the existing natural form, and
`g`/`G` is unaffected (it selects the shortest scientific form).

The three existing `Append(_, 'e', 0)` assertions were asserting the old
behaviour where precision was ignored; they now use -1 (the natural
sentinel). Adds TestDecimal64ExpPrecision covering rounding, zero-padding,
exponent carry, half-to-even, negatives and zero.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DbUHs855FyBt2J7cG2Qmhh
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Use prec parameter in (Decimal64).Append

1 participant