Extruder smoother fit improvements

[Hannott]

Summary

Four related improvements to the input-shaping numerics, all host-side Python (no MCU firmware change, no C interface change):

1. shaper_defs.py — fix a missing comma that silently truncated the 3hump_ei shaper's damping-ratio expansion (changes actual toolhead motion for 3hump_ei users).
2. extruder_smoother.py — rework the fit that generates the extruder pressure-advance smoothing kernels: analytic velocities, properly constrained least squares, well-conditioned basis, soft non-negativity.
3. shaper_calibrate.py / graph_shaper.py — fix numpy 2.x incompatibilities (np.trapz removal in numpy 2.4, size-1-array scalar conversion becoming a hard error) that break SHAPER_CALIBRATE smoother fitting on newer numpy. Not on the versions currently pinned in scripts/klippy-requirements.txt (where they only warn), but on unpinned installs, Shake&Tune, and future numpy bumps.
4. shaper_calibrate.py — compute the residual-vibration estimates from analytic velocities with exact evaluation at impulse-time kinks, improving calibration scoring accuracy.

Background: what the extruder smoother does

With input shaping active, the toolhead's real velocity differs from the commanded one. To keep pressure advance synchronized, the extruder uses a polynomial kernel w(t) whose convolution with the commanded motion approximates the shaped toolhead velocity. That smoothed velocity is what every PA model consumes (kin_extruder.c: pa_func(position, pa_velocity, params)), so the kernel quality affects linear and non-linear PA (tanh, reciprocal) alike — the model only maps the smoothed velocity to an advance distance. With PA disabled the kernel is inert.

This PR improves how those kernels are fitted, and fixes adjacent correctness/compatibility issues found along the way.

Changes

1. shaper_defs.py — 3hump_ei expansion coefficient typo

The first row of the amplitude expansion coefficients in get_3hump_ei_shaper read 3.29160 - 1.44380 — arithmetic instead of two list elements. Since the expansion loop derives its order from len(a[0]), the cubic damping-ratio terms of both A and T were silently dropped for every impulse. Verification of the fix: with the comma restored, each expansion order sums to ~0 across the five impulses (zeroth order sums to 1), which is the intended structure making sum(A) = 1 independent of damping ratio.

This changes 3hump_ei impulse timings/amplitudes at nonzero damping — the actual shaper behavior, not just its predicted score. Effect on the residual-vibration curve (up to 0.04 in vibration ratio at ζ=0.1) is shown in the figure under change 4.

2. extruder_smoother.py — kernel fit rework

Three compounding issues fixed in the fitting procedure:

• Analytic velocities. The fit target is now computed from the closed-form derivative of the damped step response, v(t) = ω·e^(−ζωt)·sin(ω_d·t)/√(1−ζ²), instead of first-order finite differences of the position response — removing the differencing error, a half-sample time misalignment, and a dt off-by-one.
• Proper constrained least squares. Constraints (unit integral, zero endpoints, zero endpoint derivatives for smooth_*) were previously imposed by overwriting rows of the normal equations, discarding part of the fit information. Now solved exactly as a KKT system.
• Legendre basis + soft non-negativity. The monomial basis made the solve ill-conditioned (condition number up to ~7×10⁹, now ~10⁵). A short penalty iteration keeps the kernel non-negative to a small absolute tolerance (−2×10⁻³ on the unit-integral kernel), which is ~1% of peak for tall, narrow kernels such as zvd at high damping and well under that for the rest — strict non-negativity is unattainable at these polynomial orders without wrecking the fit.

Relative RMS fit error vs the band-averaged target velocity profile (ζ=0.1):

shaper	before	after		shaper	before	after
zv	0.2%	0.2%		smooth_zv	0.7%	0.4%
mzv	4.4%	4.1%		smooth_mzv	0.2%	0.1%
ei	5.8%	5.4%		smooth_ei	0.9%	0.3%
zvd	8.3%	7.5%		smooth_2hump_ei	0.9%	0.2%
2hump_ei	6.5%	5.6%		smooth_zvd_ei	1.1%	0.3%
3hump_ei	17.9%	8.3%		smooth_si	1.4%	0.3%

Top: the old 3hump_ei kernel swings anti-phase to its target around t=0; the new one tracks it. Bottom: fit error for every shaper.

3. numpy 2.x compatibility fixes

numpy is deprecating and removing APIs this code relied on: np.trapz (removed in numpy 2.4 in favor of np.trapezoid) and the implicit conversion of single-element arrays to Python scalars (now a hard error). On the versions currently pinned in scripts/klippy-requirements.txt (2.0.2 / 2.2.2) np.trapz still exists as a deprecated alias and the size-1 scalar conversion only warns, so a fresh pinned install does not crash. On newer numpy the failures are real, though — both verified on 2.5.1:

• SHAPER_CALIBRATE raises AttributeError (module numpy has no attribute trapz) in _get_smoother_smoothing whenever it fits any smooth_* candidate.
• scripts/graph_shaper.py raises only 0-dimensional arrays can be converted to Python scalars in bisect().

This matters for unpinned environments, Shake&Tune (which imports these modules from the running Klipper installation), and future numpy bumps. Fixed with np.trapezoid when available (an np.trapz fallback keeps numpy 1.x installs working — the fix is numerically a no-op there) and explicit scalar extraction in graph_shaper.

4. shaper_calibrate.py — analytic velocities in vibration estimates

estimate_shaper/estimate_smoother had the same finite-difference construction as the old extruder fit. They now share the step_response_velocity helper. Two accuracy details beyond the shared change: the shaped step velocity has V-shaped kinks at the impulse times which a discrete grid straddles — it is now evaluated exactly at the impulse times — and smooth grid minima get parabolic refinement (sufficient on its own for smoothers, whose smoothed velocity is C¹).

Worst-case absolute error of the estimates vs a converged reference drops from 0.002–0.005 to below 0.002 across all shapers and test damping ratios (0.075/0.1/0.15); the estimate for an unshaped response is now exact to ~1e-8 (previously ~1e-5). Impact on SHAPER_CALIBRATE/Shake&Tune scoring is marginal (sub-1%) but systematic.

Left: the kink-straddling mechanism. Right: effect of the 3hump_ei coefficient fix on the shaper's actual residual-vibration curve. Bottom: estimator worst-case error before/after.

Real-motion validation

Simulated end-to-end on realistic motion (commanded velocity → input shaper/smoother → resonating system at 55.8 Hz, ζ=0.1 → extruder kernel convolution), to establish what the rework does and does not change in practice.

Smooth shapers: behaviorally unchanged (no regression). On gyroid-like alternating-direction moves (100 mm/s, 20 Hz direction modulation, 0.3 mm segments), before/after kernels coincide — sync error RMS 2.12→2.13 mm/s (smooth_ei) and 2.02→2.01 mm/s (smooth_si). Discrete shapers on accel-limited moves are likewise within ±3% in their design frequency ranges. The residual sync error is the kernel's finite time window lagging the direction modulation — identical before and after, not addressable by kernel shape.

Where it visibly improves: 3hump_ei at corner junctions. Step-like velocity changes (sharp corners, segmented-arc junctions) exercise exactly what the fit targets. With the printer resonance inside the band a calibrated 3hump_ei operates in (0.54–0.66 × shaper frequency), a 100→20 mm/s corner produces a +8.4 mm/s transient over-extrusion spike with the old kernel vs 2.3 mm/s with the new one (3.7× lower peak, 1.8× lower RMS). The sweep panel shows the new kernel better across the entire design band; above ~68 Hz (resonance ≈ shaper frequency — outside any calibrated 3hump_ei's operating point) the old kernel happens to score better, shown for completeness.

Verification

• All fitted kernels keep unit integral (to 1e-6) and zero endpoints (machine precision); validated across ζ = 0.05/0.1/0.2.
• The input_shaper.py call path (normalize_coeffs=False + smoother offset) produces well-formed coefficients for all shapers; polynomial orders unchanged, so the C-side 12-coefficient cap (integrate.h) and extruder_set_smoothing_params interface are untouched.
• End-to-end find_best_shaper on synthetic resonance data completes on numpy 2.5 (previously crashed); scripts/graph_shaper.py and scripts/get_extruder_smoother.py render correctly.
• ruff check / ruff format pass with both the pinned pre-commit ruff (0.4.5) and the newer ruff that CI resolves from pyproject.toml.

Deployment notes

• Host-side Python only: git pull + restart the Klipper service. No MCU reflash; the chelper C library is unchanged.
• PA model parameters (linear, tanh, reciprocal) are untouched — existing PA tuning remains valid.
• Shake&Tune picks these changes up automatically since it imports shaper_calibrate/shaper_defs from the running installation.
• Figures are hosted on the non-merge branch pr-assets/extruder-smoother-fit.

Signed-off-by: Åsmund Collin aakjaergaard@gmail.com

[Hannott]

Heads-up for reviewers: while auditing the code downstream of this PR's numpy 2.x fix, we found that the smoother-autotune path it unblocks has a further bug — SHAPER_CALIBRATE smoother recommendations are silently not applied (apply_params/save_params use SHAPER_FREQ_/shaper_freq_ names that the smoother params never read), and SAVE_CONFIG then writes a config that fails validation on restart.

That fix is in the companion PR #913 (no file overlap with this one; both bugs also exist independently of this PR on numpy 1.x installs). Reviewing the two together gives a working end-to-end smoother calibration flow.

[rogerlz]

Thanks — this is an unusually well-validated PR, and the core claims held up under independent verification: the 3hump_ei comma fix restores the intended expansion structure (each order's coefficients sum to ~0 across impulses, zeroth to 1), and all fitted kernels keep unit integral to ~1e-7 with zero endpoints at machine precision across zeta 0.05/0.1/0.2.

One correction for the commit message / PR body: on the numpy versions pinned in scripts/klippy-requirements.txt (2.0.2/2.2.2) np.trapz still exists as a deprecated alias and the 1-element-array-to-scalar conversion only warns, so SHAPER_CALIBRATE wasn't crashing on a fresh pinned install. The crashes are real on newer numpy (the scalar conversion becomes a hard error, and 2.4 removes np.trapz entirely — both verified crashing find_best_shaper on 2.5.1), so the fix is absolutely worthwhile — unpinned envs, Shake&Tune, future numpy bumps — but please reword the "crashed on pinned numpy" claim during the rebase.

Needed before merge: Signed-off-by lines on every commit (docs/CONTRIBUTING.md), and please drop the Co-Authored-By attribution trailers in the same rebase. Squashing the ruff-format fixup commit into the commit it fixes would also be nice.

Two minor notes, not blocking:

• The kernel undershoot reaches ~1% of peak at zeta=0.2 (zvd), a bit above the ~0.1% mentioned — the -2e-3 break tolerance is absolute on the fit grid. Might be worth a tweak or a comment adjustment.
• Since this changes actual 3hump_ei motion and regenerates all extruder PA kernels, a short changelog/docs note would help users understand why behavior shifts after updating.

Will look at #913 alongside this for the end-to-end smoother calibration flow.

[Hannott]

@rogerlz All should be fixed now