diff --git a/AGENTS.md b/AGENTS.md
index 2274056..60fe8d4 100644
--- a/AGENTS.md
+++ b/AGENTS.md
@@ -20,7 +20,7 @@ library live in their own repositories.
## Current shipped status
-The tip describes the shipped v1 line. The current release is **v1.20.0**;
+The tip describes the shipped v1 line. The current release is **v1.21.0**;
the exported-function and generated-kernel counts are the machine-checked
ones in README.md and docs/platforms.md, held to the sources by
`internal/tests/docs`. The tier targets are sse2, avx2, avx512, neon, sve2,
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 0ef52a1..590f1ca 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -1,5 +1,50 @@
# Changelog
+## v1.21.0
+
+
+**The kernel guide's worked example is now checked against the tree.**
+`docs/kernels.md` is the answer to "the operation I need isn't in the 403",
+and its centre is one `spec.Kernel` literal whose eight annotated fields a
+reader copies. Nothing looked inside it: the link gate checks its links and
+the test-name gate checks the tests it cites, and neither reads a code block.
+A renamed manifest field would leave the guide teaching a literal that does
+not compile, in generated code the reader did not write. The gate reads the
+field list out of the document rather than restating it, parses `spec.Kernel`
+from source (`tools/` is a separate module and must never become a consumer
+dependency), and also resolves the example's `RefFunc` against
+`internal/ref` and its `Field` against `internal/kernel` -- the two wirings
+that are easiest to forget, one of which once shipped past the whole normal
+lane. Four mutations, all red: a field renamed in the guide, the same field
+renamed in the struct, an unexported `RefFunc`, an unwired `Field`.
+
+**`PolyEval`, `Convolve`, `Correlate` and `MovingAverage` now vectorize.**
+Their kernels accumulate per output, so the innermost loop carried a
+dependence LLVM cannot break without reassociating -- which the numerical
+contract forbids -- and all eight symbols emitted zero packed arithmetic
+instructions on every amd64 tier. Blocking the *outer* loop by 16 makes the
+innermost loop independent while each output still evaluates the same terms
+in the same order, so the results are bit-identical rather than close: 8-40
+packed instructions per symbol now, and loong64 gains `simd_movavg_f32/f64`,
+6,931 -> 6,933 kernels. Verified over 3.29M cases per tier on all nine tiers
+including catastrophic cancellation and mid-accumulation overflow, where any
+regrouping would show. Speed is not measured, and on arm64 the vectorizer
+takes a different shape that costs stack frames -- `docs/wrong.md` entry 77
+records both.
+
+**Complex operations now run the generated assembly.** They never did:
+`complexOps` returned the portable reference set directly, and the
+generator emitted no complex entries into the per-tier dispatch tables, so
+`DotComplex`, `DotComplexConj`, `ScaleComplex`, `SumComplex` and the
+complex arithmetic set ran ordinary Go on every architecture while nine
+tiers of generated complex assembly sat linked into a test-only aggregator.
+Measured on amd64/avx512, scalar against the tier in one binary: 1.9x-12.9x
+depending on operation and length. Results are unchanged -- the numerical
+contract is the same fixed accumulation order, and the reference and every
+tier agree bit for bit. No API change. `docs/wrong.md` records the finding,
+the measurement, and the generator assertion that now refuses to emit a
+dispatch file for a kernel group it cannot route.
+
## v1.20.0
**The columnar codec tier.** **`BitUnpackInto`** gains a width-
@@ -669,6 +714,8 @@ stops paying.
The README's function table claimed every entry had a runnable example. It was
false for 100 of 109 rows. All 109 have one now, and
`TestReadmeTableHasExamples` fails the build if a row appears without one.
+(It is `TestOperationCatalogHasExamples` now; the guarantee did not change,
+the name did, and this entry is left as it was written.)
`TestGuidesNameRealFunctions` does the same for the prose.
Writing them found three documentation bugs that only executing catches: the
diff --git a/CLAUDE.md b/CLAUDE.md
index 0e33741..03f53fa 100644
--- a/CLAUDE.md
+++ b/CLAUDE.md
@@ -67,7 +67,7 @@ live in their own repositories.
## Current shipped status
-The current release is **v1.20.0**; the exported-function and
+The current release is **v1.21.0**; the exported-function and
generated-kernel counts are the machine-checked ones in README.md and
docs/platforms.md. The v1 API is stable — every exported
function keeps its name, signature, and meaning, and so does the numerical
diff --git a/Makefile b/Makefile
index a27e8f1..0913dea 100644
--- a/Makefile
+++ b/Makefile
@@ -71,24 +71,53 @@ test-vec: ## Run the suite with GOEXPERIMENT=simd, which compiles vec.go
test-tiers: ## Run the suite once per instruction-set tier this CPU has
@for t in $(TIERS); do \
echo "--- GOSIMD=$$t"; \
- GOSIMD=$$t $(GO) test $(PKG) || exit 1; \
+ GOSIMD=$$t $(GO) test -count=1 $(PKG) || exit 1; \
done
.PHONY: test-race
test-race: ## Full suite under the race detector
$(GO) test -race $(PKG)
-# Two fuzz targets. FuzzDifferential drives the public API; the one under
-# internal/conformance drives every generated kernel of every tier against the
-# portable implementation, with the fuzzer choosing the bit patterns rather
-# than a table choosing the values. For floating point that distinction is the
-# whole point: the inputs that break a kernel are not large or small, they are
-# a signalling NaN or a denormal at the exact exponent boundary.
+# Every fuzz target in the tree, DISCOVERED rather than listed.
+#
+# FuzzDifferential drives the public API; the one under internal/conformance
+# drives every generated kernel of every tier against the portable
+# implementation, with the fuzzer choosing the bit patterns rather than a table
+# choosing the values. For floating point that distinction is the whole point:
+# the inputs that break a kernel are not large or small, they are a signalling
+# NaN or a denormal at the exact exponent boundary.
+#
+# This comment used to open "Two fuzz targets" and the recipe named two. There
+# are three, and TWO of them had never been fuzzed by this target:
+#
+# FuzzJSONMasksMatchesSeparateCalls never named; only its seed corpus ever
+# ran, under `go test`
+# FuzzDifferential named, and pointed at the ROOT package
+# while the target lives in
+# internal/tests/arrays
+#
+# The second is the dangerous one, because `go test -run '^$$' -fuzz X .` in a
+# package with NO target called X exits 0 and prints `PASS ... 0.002s`. It does
+# not error, so `make fuzz` had been reporting success for a step that fuzzed
+# nothing. A gate that finishes in two milliseconds is not a fast gate.
+#
+# Discovery makes that unreachable: a target is only fuzzed in the package
+# `go test -list` found it in. The counter below fails the target outright if
+# it ends up running nothing at all.
.PHONY: fuzz
-fuzz: ## Fuzz the public API and the tier-vs-reference differential
- $(GO) test -run '^$$' -fuzz FuzzDifferential -fuzztime $(or $(FUZZTIME),60s) .
- $(GO) test -run '^$$' -fuzz FuzzKernelsAgainstReference \
- -fuzztime $(or $(FUZZTIME),60s) ./internal/conformance/
+fuzz: ## Fuzz every target in the tree
+ @set -e; n=0; \
+ for pkg in $$($(GO) list ./...); do \
+ for t in $$($(GO) test -list '^Fuzz' $$pkg 2>/dev/null | grep '^Fuzz' || true); do \
+ echo "fuzz $$pkg $$t"; \
+ $(GO) test -run '^$$$$' -fuzz "^$$t$$" -fuzztime $(or $(FUZZTIME),60s) $$pkg; \
+ n=$$((n+1)); \
+ done; \
+ done; \
+ if [ "$$n" -eq 0 ]; then \
+ echo "make fuzz ran no targets" >&2; exit 1; \
+ fi; \
+ echo "fuzzed $$n target(s)"
# Every architecture with a backend, under emulation.
#
diff --git a/README.md b/README.md
index 38136c0..a1d327d 100644
--- a/README.md
+++ b/README.md
@@ -1,14 +1,13 @@
-# simd.go
+# simd.go — Fast Whole-Slice SIMD Vector Primitives & Text Scanning for Go (AVX-512, AVX2, NEON, SVE2, RVV)
[](https://github.com/sebishogun/simd/releases/latest)
[](https://pkg.go.dev/github.com/sebishogun/simd)
[](https://github.com/sebishogun/simd/actions/workflows/ci-local.yml)
-**Fast slice math and text scanning for Go, using the vector unit your CPU
-already has — without cgo.**
+**Hardware-accelerated SIMD vector math, text scanning, matrix operations, and columnar codecs for Go — using the vector unit your CPU already has, without cgo.**
-`simd` applies runtime-selected SIMD kernels to ordinary Go slices. The v1.20.0
-tree contains 493 exported functions and 6,931 generated kernels for SSE2,
+`simd` applies runtime-selected SIMD kernels to ordinary Go slices. The v1.21.0
+tree contains 493 exported functions and 6,933 generated kernels for SSE2,
AVX2, AVX-512, NEON, SVE2, RVV, VSX, VX, and LASX. A portable Go path covers
every operation and every unsupported target.
@@ -23,7 +22,7 @@ kernels are compiled ahead of time and committed as assembly, so this is an
ordinary Go dependency with one transitive import (`golang.org/x/sys`, for CPU
feature detection). Dispatch is one static table per operation, so the linker
keeps only the operations a program actually calls: a binary using three
-operation families does not retain all 6,931 kernels.
+operation families does not retain all 6,933 kernels.
## Quick start
@@ -389,7 +388,7 @@ cgo.
## Engineering record
-[**docs/wrong.md**](docs/wrong.md) records 79 things that measurement disproved,
+[**docs/wrong.md**](docs/wrong.md) records 87 things that measurement disproved,
including changes that were deleted rather than shipped. Examples:
- Green test lanes had executed no accelerated code for months.
@@ -403,9 +402,26 @@ including changes that were deleted rather than shipped. Examples:
`docs/research/` carries the longer reasoning behind design decisions;
`05-decisions.md` is the decision record.
+## Frequently asked questions
+
+- **Does Go support SIMD vectorization natively without cgo?**
+ Yes. `simd` provides whole-slice SIMD vector acceleration in pure Go without cgo, C toolchains, or custom build tags. Ahead-of-time generated Plan 9 assembly kernels execute directly under Go's internal register ABI (ABIInternal).
+
+- **Which SIMD instruction sets and CPU architectures are supported?**
+ `simd` ships 6,933 generated kernels across 10 ISA tiers: x86-64 (SSE2, AVX2, AVX-512), ARM64 (NEON, SVE2), RISC-V (RVV 1.0), PowerPC (VSX), SystemZ (VX), and LoongArch (LASX), with a portable Go fallback for all unsupported targets.
+
+- **How does `simd.go` compare to `GOEXPERIMENT=simd` intrinsics in Go 1.26 and Go 1.27?**
+ Go 1.26 introduced experimental intrinsics (`simd/archsimd` on `amd64`), and Go 1.27 expanded it to `arm64` NEON and `wasm`. However, `GOEXPERIMENT=simd` requires consumers to set a build-time flag, does not support scalable vector units (ARM SVE2, RISC-V RVV 1.0), and lacks Clang unrolling optimizations ($4.4\times$ slower on whole-slice loops at $N=256$). `simd.go` requires zero build flags, supports 10 ISA tiers, and delivers optimal whole-slice throughput.
+
+- **Does `simd.go` support WebAssembly (WASM)?**
+ On `GOARCH=wasm`, `simd` automatically executes the portable Go fallback path. Go's WASM assembler (`cmd/internal/obj/wasm`) lacks SIMD128 opcodes and raw byte lifting breaks WebAssembly stack validation, so WebAssembly SIMD is left to Go 1.27's internal compiler intrinsics.
+
+- **What is the minimum slice length (crossover threshold) for SIMD in Go?**
+ The assembly call overhead is roughly 0.45 ns to 1.4 ns. For slice sizes under 16 to 64 elements, scalar Go is faster; for batches of hundreds to millions of elements, `simd` delivers up to $109\times$ speedups.
+
## Status
-**v1.20.0.** The API is stable: every exported function keeps its name,
+**v1.21.0.** The API is stable: every exported function keeps its name,
signature and meaning for the life of v1, and so does the numerical contract
above. [CHANGELOG.md](CHANGELOG.md) states exactly what compatibility covers
and what it excludes. [ROADMAP.md](ROADMAP.md) lists what is still open.
diff --git a/complex.go b/complex.go
index 18701d5..09f7dd1 100644
--- a/complex.go
+++ b/complex.go
@@ -28,9 +28,9 @@ func complexOps[C Complex]() *kernel.Complex[C] {
var zero C
switch any(zero).(type) {
case complex64:
- return any(&refBase.C64).(*kernel.Complex[C])
+ return any(cacheC64.get(&refBase.C64, cplxC64ByTier[:])).(*kernel.Complex[C])
case complex128:
- return any(&refBase.C128).(*kernel.Complex[C])
+ return any(cacheC128.get(&refBase.C128, cplxC128ByTier[:])).(*kernel.Complex[C])
}
panic("simd: unsupported complex type")
}
@@ -44,11 +44,11 @@ func complexParts[C Complex, R Float]() *kernel.ComplexParts[C, R] {
var zero C
switch any(zero).(type) {
case complex64:
- if p, ok := any(&refBase.C64Parts).(*kernel.ComplexParts[C, R]); ok {
+ if p, ok := any(cacheC64Parts.get(&refBase.C64Parts, partsC64PartsByTier[:])).(*kernel.ComplexParts[C, R]); ok {
return p
}
case complex128:
- if p, ok := any(&refBase.C128Parts).(*kernel.ComplexParts[C, R]); ok {
+ if p, ok := any(cacheC128Parts.get(&refBase.C128Parts, partsC128PartsByTier[:])).(*kernel.ComplexParts[C, R]); ok {
return p
}
}
diff --git a/csrc/numeric.c b/csrc/numeric.c
index a8784a0..0532c27 100644
--- a/csrc/numeric.c
+++ b/csrc/numeric.c
@@ -69,15 +69,44 @@ void simd_norm_f64(double *__restrict out, const double *__restrict a,
// what vectorizes; the inner loop over coefficients is a fixed dependency
// chain per element, exactly like the transcendentals in math.c.
//
-// The accumulator is per element rather than per coefficient, so the whole
-// polynomial is evaluated for a register's worth of x at a time.
+// BLOCKED over elements, so the INNERMOST loop is the independent one.
+//
+// The obvious nesting -- outer over elements, inner over coefficients -- does
+// not vectorize, and the reason is structural rather than a cost model: LLVM's
+// loop vectorizer only ever vectorizes the innermost loop, and here that is
+// the Horner recurrence, a serial dependency it cannot break without
+// reassociating. Measured: simd_polyeval_f32 contained ten scalar float
+// instructions and no packed arithmetic at sse2, avx2 and avx512 alike, while
+// passing the "contains a vector instruction" check on the strength of a
+// movups and an xorps. See docs/wrong.md entry 76.
+//
+// Blocking swaps which loop is innermost without changing any element's
+// arithmetic: every output still evaluates the same coefficients in the same
+// order, so the result is bit-identical, not merely close. The accumulator is
+// a fixed-size array so nothing is allocated and the frame stays far under the
+// 512-byte kernel budget -- POLY_BLOCK doubles is 128 bytes.
+//
+// The tail runs the original nesting: it is at most POLY_BLOCK-1 elements and
+// vectorizing it is not worth a second code path to get wrong.
+#define POLY_BLOCK 16
#define POLY_EVAL(T) \
isize n = nd < nx ? nd : nx; \
if (nc == 0) { \
for (isize i = 0; i < n; i++) d[i] = 0; \
return; \
} \
- for (isize i = 0; i < n; i++) { \
+ isize i = 0; \
+ for (; i + POLY_BLOCK <= n; i += POLY_BLOCK) { \
+ T acc[POLY_BLOCK]; \
+ for (isize b = 0; b < POLY_BLOCK; b++) acc[b] = c[nc - 1]; \
+ for (isize k = nc - 2; k >= 0; k--) { \
+ T ck = c[k]; \
+ for (isize b = 0; b < POLY_BLOCK; b++) \
+ acc[b] = acc[b] * x[i + b] + ck; \
+ } \
+ for (isize b = 0; b < POLY_BLOCK; b++) d[i + b] = acc[b]; \
+ } \
+ for (; i < n; i++) { \
T xv = x[i]; \
T acc = c[nc - 1]; \
for (isize k = nc - 2; k >= 0; k--) acc = acc * xv + c[k]; \
@@ -104,12 +133,36 @@ void simd_polyeval_f64(double *__restrict d, const double *__restrict x,
//
// An FFT would be asymptotically better and is a different kernel; this is
// the form that wins for the short kernels callers actually convolve with.
+// BLOCKED over outputs, for the reason POLY_EVAL is: the tap loop is the
+// innermost one and it is a serial float reduction, so LLVM -- which
+// vectorizes only innermost loops -- had nothing to work with, and the kernel
+// shipped scalar on every tier while passing the "contains a vector
+// instruction" check. Blocking makes the innermost loop the independent one
+// over outputs.
+//
+// Each output still accumulates its taps in the same order, so this is
+// bit-identical to the direct form rather than close to it. That matters here
+// more than elsewhere: reassociating a dot product is exactly what
+// `#pragma clang loop vectorize(enable)` would have done, and what the
+// numerical contract forbids.
+#define CONV_BLOCK 16
#define CONVOLVE(T, IDX) \
isize m = nk; \
if (m == 0 || n < m) return; \
isize outn = n - m + 1; \
if (nd < outn) outn = nd; \
- for (isize i = 0; i < outn; i++) { \
+ isize i = 0; \
+ for (; i + CONV_BLOCK <= outn; i += CONV_BLOCK) { \
+ T acc[CONV_BLOCK]; \
+ for (isize b = 0; b < CONV_BLOCK; b++) acc[b] = 0; \
+ for (isize j = 0; j < m; j++) { \
+ T kv = k[IDX]; \
+ for (isize b = 0; b < CONV_BLOCK; b++) \
+ acc[b] += s[i + b + j] * kv; \
+ } \
+ for (isize b = 0; b < CONV_BLOCK; b++) d[i + b] = acc[b]; \
+ } \
+ for (; i < outn; i++) { \
T acc = 0; \
for (isize j = 0; j < m; j++) acc += s[i + j] * k[IDX]; \
d[i] = acc; \
@@ -384,11 +437,27 @@ void simd_scatter_u64(unsigned long long *__restrict d, const int *__restrict id
// rounding error than summing the window afresh. The reference sums each
// window, so this does too, and the outer loop over windows is what
// vectorizes.
+// BLOCKED over outputs, exactly as CONVOLVE is and for the same reason: the
+// window loop was innermost and is a serial float sum, so nothing vectorized.
+// Each output sums its window in the same order, so the result is
+// bit-identical -- which rules out the sliding-window form (subtract the
+// element leaving, add the one arriving), which is O(n) instead of O(n*w) and
+// gives a DIFFERENT number, because floating-point addition does not cancel.
+#define MOVAVG_BLOCK 16
#define MOVING_AVERAGE(T) \
if (w <= 0 || na < w) return; \
isize outn = na - w + 1; \
if (nd < outn) outn = nd; \
- for (isize i = 0; i < outn; i++) { \
+ isize i = 0; \
+ for (; i + MOVAVG_BLOCK <= outn; i += MOVAVG_BLOCK) { \
+ T acc[MOVAVG_BLOCK]; \
+ for (isize b = 0; b < MOVAVG_BLOCK; b++) acc[b] = 0; \
+ for (isize j = 0; j < w; j++) { \
+ for (isize b = 0; b < MOVAVG_BLOCK; b++) acc[b] += a[i + b + j]; \
+ } \
+ for (isize b = 0; b < MOVAVG_BLOCK; b++) d[i + b] = acc[b] / (T)w; \
+ } \
+ for (; i < outn; i++) { \
T acc = 0; \
for (isize j = 0; j < w; j++) acc += a[i + j]; \
d[i] = acc / (T)w; \
diff --git a/dispatch.go b/dispatch.go
index bd7fef7..9c15d9c 100644
--- a/dispatch.go
+++ b/dispatch.go
@@ -96,8 +96,49 @@ func (c *opsCache[T]) get(base *kernel.Ops[T], partial []*kernel.Ops[T]) *kernel
return c.tiers[tierIdx]
}
-// overlay copies every non-nil function field of src over dst.
-func overlay[T Number](dst, src *kernel.Ops[T]) {
+// groupCache is opsCache for the two complex groups. They are the same shape
+// -- a struct of function fields, a static per-tier partial, a lazy merge --
+// but kernel.Complex and kernel.ComplexParts are not kernel.Ops, and Go has
+// no constraint that spans the three. One generic type over the struct
+// itself, rather than over the element type, covers both.
+//
+// Until this existed, complexOps returned the reference set directly and
+// every complex operation ran portable Go on every machine while nine tiers
+// of generated complex assembly -- sse2, avx2, avx512, neon, sve2, rvv, vsx,
+// vx, lasx, across six architectures -- sat linked into the test-only
+// aggregator.
+type groupCache[G any] struct {
+ once sync.Once
+ tiers [len(dispatchTiers)]*G
+}
+
+func (c *groupCache[G]) get(base *G, partial []*G) *G {
+ c.once.Do(func() {
+ for i := range c.tiers {
+ s := *base
+ if i < len(partial) && partial[i] != nil {
+ overlayAny(&s, partial[i])
+ }
+ c.tiers[i] = &s
+ }
+ })
+ return c.tiers[tierIdx]
+}
+
+var (
+ cacheC64 groupCache[kernel.Complex[complex64]]
+ cacheC128 groupCache[kernel.Complex[complex128]]
+ cacheC64Parts groupCache[kernel.ComplexParts[complex64, float32]]
+ cacheC128Parts groupCache[kernel.ComplexParts[complex128, float64]]
+)
+
+// overlay copies every non-nil function field of src over dst. It is
+// overlayAny at kernel.Ops[T], kept as a named function because that is what
+// opsCache reads and the constraint documents the intent.
+func overlay[T Number](dst, src *kernel.Ops[T]) { overlayAny(dst, src) }
+
+// overlayAny is overlay for any struct of function fields.
+func overlayAny[G any](dst, src *G) {
d := reflect.ValueOf(dst).Elem()
s := reflect.ValueOf(src).Elem()
for i := 0; i < d.NumField(); i++ {
diff --git a/dispatch_complex_test.go b/dispatch_complex_test.go
new file mode 100644
index 0000000..5b7aaed
--- /dev/null
+++ b/dispatch_complex_test.go
@@ -0,0 +1,155 @@
+package simd
+
+import (
+ "reflect"
+ "testing"
+)
+
+// The complex surface must reach the generated kernels, the same way the
+// numeric surface does.
+//
+// It did not, from v1.14.0 to v1.20.0. complexOps returned &refBase.C64 /
+// &refBase.C128 directly, with no per-tier overlay, so every complex
+// operation ran the portable Go reference on every machine -- while the
+// repository generated, tested and shipped nine tiers of complex assembly
+// for them. The existing coverage did not catch it because
+// TestDeclaredKernelsAreWired checks the runtime tables for exactly three
+// groups (Bytes, Convert, Mask) and routes everything else -- numeric and
+// complex alike -- through archSets(), the test-only whole-set aggregator
+// whose own generated header says consumers never call it. Nothing walked
+// the tables the runtime actually indexes.
+//
+// So this asks the question a caller asks: is the function DotComplex ends
+// up calling the one the tier's table supplies?
+
+// tierPartial returns the per-tier partial for a group at the running tier,
+// or nil when this build has no table for it (the purego and no-backend
+// builds, and slot 0, the reference).
+func tierPartial[G any](byTier []*G) *G {
+ if tierIdx < 0 || tierIdx >= len(byTier) {
+ return nil
+ }
+ return byTier[tierIdx]
+}
+
+// checkGroup is the whole contract, field by field: where the tier's partial
+// supplies a kernel the dispatch must reach exactly that function, and where
+// it does not the dispatch must keep the reference.
+//
+// The second half matters as much as the first. A missing kernel on a tier
+// is documented as slower and never a correctness gap
+// (docs/lld/kernels-and-dispatch.md), and three architectures rely on it
+// today -- s390x has no complex64 Dot or DotConj, ppc64le adds Sum to that
+// list, loong64 is missing Mul, Div and Conj. An earlier version of this
+// test asserted "not the reference" for one hardcoded field and turned that
+// documented fallback into a red cross-compile gate on two architectures.
+func checkGroup[G any](t *testing.T, name string, got, ref, partial *G) {
+ t.Helper()
+ g := reflect.ValueOf(got).Elem()
+ r := reflect.ValueOf(ref).Elem()
+ var p reflect.Value
+ if partial != nil {
+ p = reflect.ValueOf(partial).Elem()
+ }
+ typ := g.Type()
+
+ supplied := 0
+ for i := 0; i < g.NumField(); i++ {
+ if typ.Field(i).Type.Kind() != reflect.Func {
+ continue
+ }
+ field := name + "." + typ.Field(i).Name
+ gp := g.Field(i).Pointer()
+ rp := r.Field(i).Pointer()
+
+ if partial != nil && !p.Field(i).IsNil() {
+ supplied++
+ want := p.Field(i).Pointer()
+ if gp != want {
+ t.Errorf("%s on tier %q: dispatch reaches %#x, the tier's table holds %#x",
+ field, Tier(), gp, want)
+ }
+ if gp == rp && rp != want {
+ t.Errorf("%s on tier %q: dispatch is still the portable reference", field, Tier())
+ }
+ continue
+ }
+ // No kernel on this tier: the reference stands, unchanged.
+ if gp != rp {
+ t.Errorf("%s on tier %q: no kernel is generated, but dispatch is not the reference",
+ field, Tier())
+ }
+ }
+
+ // A table that supplies nothing on an accelerated tier is a table that
+ // was emitted empty -- which is what the defect looked like.
+ if partial != nil && supplied == 0 && Tier() != "scalar" {
+ t.Errorf("%s on tier %q: the per-tier table supplies no kernel at all", name, Tier())
+ }
+}
+
+func TestComplexDispatchReachesGeneratedKernels(t *testing.T) {
+ checkGroup(t, "C64", complexOps[complex64](), &refBase.C64, tierPartial(cplxC64ByTier[:]))
+ checkGroup(t, "C128", complexOps[complex128](), &refBase.C128, tierPartial(cplxC128ByTier[:]))
+}
+
+// The real-scalar half of the split -- Abs, Real, Imag, Scale -- goes through
+// ComplexParts and had the same hole.
+func TestComplexPartsDispatchReachesGeneratedKernels(t *testing.T) {
+ checkGroup(t, "C64Parts", complexParts[complex64, float32](),
+ &refBase.C64Parts, tierPartial(partsC64PartsByTier[:]))
+ checkGroup(t, "C128Parts", complexParts[complex128, float64](),
+ &refBase.C128Parts, tierPartial(partsC128PartsByTier[:]))
+}
+
+// groupCache indexes its partial slice by tierIdx and guards with
+// `i < len(partial)`, so a table shorter than dispatchTiers silently leaves
+// the top tiers on the reference and a longer one silently drops its tail.
+// Both are generator bugs that no other test would notice.
+func TestComplexTierTablesMatchTheTierList(t *testing.T) {
+ for _, c := range []struct {
+ name string
+ n int
+ }{
+ {"cplxC64ByTier", len(cplxC64ByTier)},
+ {"cplxC128ByTier", len(cplxC128ByTier)},
+ {"partsC64PartsByTier", len(partsC64PartsByTier)},
+ {"partsC128PartsByTier", len(partsC128PartsByTier)},
+ } {
+ if c.n != len(dispatchTiers) {
+ t.Errorf("%s has %d slots, dispatchTiers has %d", c.name, c.n, len(dispatchTiers))
+ }
+ }
+}
+
+// Whatever the dispatch routes to, the answer is the reference's answer.
+// The numerical contract is the one every other tier keeps: a fixed
+// accumulation order, identical on every architecture.
+func TestComplexDispatchMatchesTheReference(t *testing.T) {
+ n := 1000
+ a := make([]complex128, n)
+ b := make([]complex128, n)
+ for i := range a {
+ a[i] = complex(float64(i%17)-8, float64(i%23)-11)
+ b[i] = complex(float64(i%13)-6, float64(i%29)-14)
+ }
+ if got, want := DotComplex(a, b), refBase.C128.Dot(a, b); got != want {
+ t.Errorf("DotComplex = %v, reference = %v", got, want)
+ }
+ if got, want := DotComplexConj(a, b), refBase.C128.DotConj(a, b); got != want {
+ t.Errorf("DotComplexConj = %v, reference = %v", got, want)
+ }
+ if got, want := SumComplex(a), refBase.C128.Sum(a); got != want {
+ t.Errorf("SumComplex = %v, reference = %v", got, want)
+ }
+
+ gotS := append([]complex128(nil), a...)
+ wantS := append([]complex128(nil), a...)
+ ScaleComplex(gotS, 1.5)
+ refBase.C128Parts.Scale(wantS, wantS, 1.5)
+ for i := range gotS {
+ if gotS[i] != wantS[i] {
+ t.Fatalf("ScaleComplex[%d] = %v, reference = %v", i, gotS[i], wantS[i])
+ }
+ }
+}
diff --git a/dispatch_inventory_test.go b/dispatch_inventory_test.go
index f5e4572..4199154 100644
--- a/dispatch_inventory_test.go
+++ b/dispatch_inventory_test.go
@@ -32,6 +32,7 @@ package simd
import (
"reflect"
+ "strings"
"testing"
"github.com/sebishogun/simd/internal/backend"
@@ -65,6 +66,30 @@ func TestDeclaredKernelsAreWired(t *testing.T) {
}
continue
}
+ // The struct groups now have real per-tier tables, so walk those --
+ // the runtime indexes them, and archSets() does not. Four whole
+ // complex groups were missing from the runtime tables for seven
+ // releases with this test green, because everything that was not a
+ // flat group went through the test-only aggregator instead.
+ if slots, ok := allGroupTables[d.Group]; ok {
+ for i, partial := range slots {
+ if i == 0 || partial == nil {
+ continue // slot 0 is the reference; no overlay
+ }
+ f := reflect.ValueOf(partial).FieldByName(d.Field)
+ if !f.IsValid() {
+ t.Errorf("%s: per-tier table for %s has no field %q",
+ d.CName, d.Group, d.Field)
+ break
+ }
+ if f.Kind() == reflect.Func && f.IsNil() && present(sets, dispatchTiers[i], d) {
+ t.Errorf("tier %q: %s.%s is nil in the dispatch table, but the "+
+ "kernel is generated for that tier -- it will never run",
+ dispatchTiers[i], d.Group, d.Field)
+ }
+ }
+ continue
+ }
for tier, set := range sets {
g := reflect.ValueOf(set).FieldByName(d.Group)
if !g.IsValid() {
@@ -105,3 +130,91 @@ func TestInventoryCoversEveryGroup(t *testing.T) {
}
}
}
+
+// present reports whether the generator emitted a kernel for d on tier.
+//
+// Comparing against the REFERENCE, not against nil: archSets() builds each
+// tier from ref.Set() and overlays what was generated, so every field is
+// non-nil there and a nil test would call every operation "generated" on
+// every tier. A tier the generator declined keeps the reference by design
+// (docs/lld/kernels-and-dispatch.md), and only a kernel that exists and is
+// missing from the dispatch table is a defect.
+func present(sets map[string]kernel.Set, tier string, d backend.Declared) bool {
+ set, ok := sets[tier]
+ if !ok {
+ return false
+ }
+ g := reflect.ValueOf(set).FieldByName(d.Group)
+ r := reflect.ValueOf(refBase).FieldByName(d.Group)
+ if !g.IsValid() || !r.IsValid() {
+ return false
+ }
+ f, rf := g.FieldByName(d.Field), r.FieldByName(d.Field)
+ if !f.IsValid() || !rf.IsValid() || f.Kind() != reflect.Func || f.IsNil() {
+ return false
+ }
+ return rf.IsNil() || f.Pointer() != rf.Pointer()
+}
+
+// Every declared kernel is wired in the REFERENCE too, not only in the
+// generated tables.
+//
+// TestDeclaredKernelsAreWired above walks backend.Inventory against the
+// dispatch tables and archSets(), which are the GENERATED side. It never looks
+// at ref.Set(), so a forgotten entry in internal/ref's ops table is invisible
+// to it.
+//
+// Measured 2026-08-15: `Sqrt: sqrt[T]` deleted from floatOps in
+// internal/ref/ref.go. TestDeclaredKernelsAreWired PASSED in both the normal
+// and the purego lane, and the WHOLE normal-lane suite -- all fourteen
+// packages -- passed. Only the whole purego suite caught it, through
+// TestInPlaceMatchesInto, a functional test rather than the completeness gate
+// that exists for exactly this. So a forgotten reference wiring shipped unless
+// somebody happened to run -tags purego.
+//
+// The reference is not optional: it is the live fallback on architectures with
+// no backend, in purego builds, and below every kernel's element threshold. A
+// nil there is a nil pointer dereference in the small-n path of an operation
+// that works fine on a big slice.
+//
+// Fast slots are excluded because ref.Set() leaves them nil ON PURPOSE --
+// until a backend has registered, "no Fast kernel here" and "the accurate
+// kernel" cannot be told apart, and dispatch.go fills them with
+// FillFastFallbacks afterwards. That is the same exclusion the doc comment on
+// TestDeclaredKernelsAreWired describes.
+func TestDeclaredKernelsAreWiredInTheReference(t *testing.T) {
+ if len(backend.Inventory) == 0 {
+ t.Fatal("backend.Inventory is empty; run make codegen")
+ }
+ checked := 0
+ for _, d := range backend.Inventory {
+ if strings.HasPrefix(d.Field, "Fast") {
+ continue
+ }
+ g := reflect.ValueOf(refBase).FieldByName(d.Group)
+ if !g.IsValid() {
+ t.Errorf("%s: kernel.Set has no group %q", d.CName, d.Group)
+ continue
+ }
+ f := g.FieldByName(d.Field)
+ if !f.IsValid() {
+ t.Errorf("%s: kernel.Set.%s has no field %q", d.CName, d.Group, d.Field)
+ continue
+ }
+ if f.Kind() != reflect.Func {
+ continue
+ }
+ checked++
+ if f.IsNil() {
+ t.Errorf("%s.%s is nil in the REFERENCE, and the manifest declares %s "+
+ "for that group: internal/ref is the live fallback below the "+
+ "element threshold, in purego builds, and on every architecture "+
+ "with no backend, so this is a nil call rather than a slow one",
+ d.Group, d.Field, d.CName)
+ }
+ }
+ if checked == 0 {
+ t.Fatal("no reference slots were checked, so this gate measures nothing")
+ }
+ t.Logf("%d declared non-Fast kernels checked against the reference", checked)
+}
diff --git a/dispatch_tables_amd64.go b/dispatch_tables_amd64.go
index ed7ad96..1d16221 100644
--- a/dispatch_tables_amd64.go
+++ b/dispatch_tables_amd64.go
@@ -3175,3 +3175,151 @@ var opsU8ByTier = [...]*kernel.Ops[uint8]{
&opsU8AVX2,
&opsU8AVX512,
}
+
+var cplxC128SSE2 = kernel.Complex[complex128]{
+ Add: amd64.CaddComplex128SSE2,
+ Sub: amd64.CsubComplex128SSE2,
+ Mul: amd64.CmulComplex128SSE2,
+ Div: amd64.CdivComplex128SSE2,
+ Neg: amd64.CnegComplex128SSE2,
+ Conj: amd64.CconjComplex128SSE2,
+ Sum: amd64.CsumComplex128SSE2,
+ Dot: amd64.CdotComplex128SSE2,
+ DotConj: amd64.CdotconjComplex128SSE2,
+}
+
+var cplxC128AVX2 = kernel.Complex[complex128]{
+ Add: amd64.CaddComplex128AVX2,
+ Sub: amd64.CsubComplex128AVX2,
+ Mul: amd64.CmulComplex128AVX2,
+ Div: amd64.CdivComplex128AVX2,
+ Neg: amd64.CnegComplex128AVX2,
+ Conj: amd64.CconjComplex128AVX2,
+ Sum: amd64.CsumComplex128AVX2,
+ Dot: amd64.CdotComplex128AVX2,
+ DotConj: amd64.CdotconjComplex128AVX2,
+}
+
+var cplxC128AVX512 = kernel.Complex[complex128]{
+ Add: amd64.CaddComplex128AVX512,
+ Sub: amd64.CsubComplex128AVX512,
+ Mul: amd64.CmulComplex128AVX512,
+ Div: amd64.CdivComplex128AVX512,
+ Neg: amd64.CnegComplex128AVX512,
+ Conj: amd64.CconjComplex128AVX512,
+ Sum: amd64.CsumComplex128AVX512,
+ Dot: amd64.CdotComplex128AVX512,
+ DotConj: amd64.CdotconjComplex128AVX512,
+}
+
+var cplxC128ByTier = [...]*kernel.Complex[complex128]{
+ nil,
+ &cplxC128SSE2,
+ &cplxC128AVX2,
+ &cplxC128AVX512,
+}
+
+var cplxC64SSE2 = kernel.Complex[complex64]{
+ Add: amd64.CaddComplex64SSE2,
+ Sub: amd64.CsubComplex64SSE2,
+ Mul: amd64.CmulComplex64SSE2,
+ Div: amd64.CdivComplex64SSE2,
+ Neg: amd64.CnegComplex64SSE2,
+ Conj: amd64.CconjComplex64SSE2,
+ Sum: amd64.CsumComplex64SSE2,
+ Dot: amd64.CdotComplex64SSE2,
+ DotConj: amd64.CdotconjComplex64SSE2,
+}
+
+var cplxC64AVX2 = kernel.Complex[complex64]{
+ Add: amd64.CaddComplex64AVX2,
+ Sub: amd64.CsubComplex64AVX2,
+ Mul: amd64.CmulComplex64AVX2,
+ Div: amd64.CdivComplex64AVX2,
+ Neg: amd64.CnegComplex64AVX2,
+ Conj: amd64.CconjComplex64AVX2,
+ Sum: amd64.CsumComplex64AVX2,
+ Dot: amd64.CdotComplex64AVX2,
+ DotConj: amd64.CdotconjComplex64AVX2,
+}
+
+var cplxC64AVX512 = kernel.Complex[complex64]{
+ Add: amd64.CaddComplex64AVX512,
+ Sub: amd64.CsubComplex64AVX512,
+ Mul: amd64.CmulComplex64AVX512,
+ Div: amd64.CdivComplex64AVX512,
+ Neg: amd64.CnegComplex64AVX512,
+ Conj: amd64.CconjComplex64AVX512,
+ Sum: amd64.CsumComplex64AVX512,
+ Dot: amd64.CdotComplex64AVX512,
+ DotConj: amd64.CdotconjComplex64AVX512,
+}
+
+var cplxC64ByTier = [...]*kernel.Complex[complex64]{
+ nil,
+ &cplxC64SSE2,
+ &cplxC64AVX2,
+ &cplxC64AVX512,
+}
+
+var partsC128PartsSSE2 = kernel.ComplexParts[complex128, float64]{
+ Abs: amd64.CabsComplex128SSE2,
+ Real: amd64.CrealComplex128SSE2,
+ Imag: amd64.CimagComplex128SSE2,
+ Scale: amd64.CscaleComplex128SSE2,
+ FromParts: amd64.CfromPartsComplex128SSE2,
+}
+
+var partsC128PartsAVX2 = kernel.ComplexParts[complex128, float64]{
+ Abs: amd64.CabsComplex128AVX2,
+ Real: amd64.CrealComplex128AVX2,
+ Imag: amd64.CimagComplex128AVX2,
+ Scale: amd64.CscaleComplex128AVX2,
+ FromParts: amd64.CfromPartsComplex128AVX2,
+}
+
+var partsC128PartsAVX512 = kernel.ComplexParts[complex128, float64]{
+ Abs: amd64.CabsComplex128AVX512,
+ Real: amd64.CrealComplex128AVX512,
+ Imag: amd64.CimagComplex128AVX512,
+ Scale: amd64.CscaleComplex128AVX512,
+ FromParts: amd64.CfromPartsComplex128AVX512,
+}
+
+var partsC128PartsByTier = [...]*kernel.ComplexParts[complex128, float64]{
+ nil,
+ &partsC128PartsSSE2,
+ &partsC128PartsAVX2,
+ &partsC128PartsAVX512,
+}
+
+var partsC64PartsSSE2 = kernel.ComplexParts[complex64, float32]{
+ Abs: amd64.CabsComplex64SSE2,
+ Real: amd64.CrealComplex64SSE2,
+ Imag: amd64.CimagComplex64SSE2,
+ Scale: amd64.CscaleComplex64SSE2,
+ FromParts: amd64.CfromPartsComplex64SSE2,
+}
+
+var partsC64PartsAVX2 = kernel.ComplexParts[complex64, float32]{
+ Abs: amd64.CabsComplex64AVX2,
+ Real: amd64.CrealComplex64AVX2,
+ Imag: amd64.CimagComplex64AVX2,
+ Scale: amd64.CscaleComplex64AVX2,
+ FromParts: amd64.CfromPartsComplex64AVX2,
+}
+
+var partsC64PartsAVX512 = kernel.ComplexParts[complex64, float32]{
+ Abs: amd64.CabsComplex64AVX512,
+ Real: amd64.CrealComplex64AVX512,
+ Imag: amd64.CimagComplex64AVX512,
+ Scale: amd64.CscaleComplex64AVX512,
+ FromParts: amd64.CfromPartsComplex64AVX512,
+}
+
+var partsC64PartsByTier = [...]*kernel.ComplexParts[complex64, float32]{
+ nil,
+ &partsC64PartsSSE2,
+ &partsC64PartsAVX2,
+ &partsC64PartsAVX512,
+}
diff --git a/dispatch_tables_amd64_test.go b/dispatch_tables_amd64_test.go
index 519ecb7..151d2bf 100644
--- a/dispatch_tables_amd64_test.go
+++ b/dispatch_tables_amd64_test.go
@@ -127,5 +127,25 @@ var allFlatTables = map[string][]any{
"Bytes.VarintDecodeU64": {tblBytesVarintDecodeU64[0], tblBytesVarintDecodeU64[1], tblBytesVarintDecodeU64[2], tblBytesVarintDecodeU64[3]},
}
+// allGroupTables enumerates the per-tier partial structs the runtime
+// overlays from, keyed by kernel.Set group. Slot 0 is the reference
+// and is nil; the rest are one per accelerated tier, in tier order.
+var allGroupTables = map[string][]any{
+ "F32": {nil, opsF32SSE2, opsF32AVX2, opsF32AVX512},
+ "F64": {nil, opsF64SSE2, opsF64AVX2, opsF64AVX512},
+ "I16": {nil, opsI16SSE2, opsI16AVX2, opsI16AVX512},
+ "I32": {nil, opsI32SSE2, opsI32AVX2, opsI32AVX512},
+ "I64": {nil, opsI64SSE2, opsI64AVX2, opsI64AVX512},
+ "I8": {nil, opsI8SSE2, opsI8AVX2, opsI8AVX512},
+ "U16": {nil, opsU16SSE2, opsU16AVX2, opsU16AVX512},
+ "U32": {nil, opsU32SSE2, opsU32AVX2, opsU32AVX512},
+ "U64": {nil, opsU64SSE2, opsU64AVX2, opsU64AVX512},
+ "U8": {nil, opsU8SSE2, opsU8AVX2, opsU8AVX512},
+ "C128": {nil, cplxC128SSE2, cplxC128AVX2, cplxC128AVX512},
+ "C64": {nil, cplxC64SSE2, cplxC64AVX2, cplxC64AVX512},
+ "C128Parts": {nil, partsC128PartsSSE2, partsC128PartsAVX2, partsC128PartsAVX512},
+ "C64Parts": {nil, partsC64PartsSSE2, partsC64PartsAVX2, partsC64PartsAVX512},
+}
+
// archSets hands the tests every tier's complete kernel set.
var archSets func() map[string]kernel.Set = amd64.Sets
diff --git a/dispatch_tables_arm64.go b/dispatch_tables_arm64.go
index 99f8cc6..343289f 100644
--- a/dispatch_tables_arm64.go
+++ b/dispatch_tables_arm64.go
@@ -2232,3 +2232,107 @@ var opsU8ByTier = [...]*kernel.Ops[uint8]{
&opsU8NEON,
&opsU8SVE2,
}
+
+var cplxC128NEON = kernel.Complex[complex128]{
+ Add: arm64.CaddComplex128NEON,
+ Sub: arm64.CsubComplex128NEON,
+ Mul: arm64.CmulComplex128NEON,
+ Div: arm64.CdivComplex128NEON,
+ Neg: arm64.CnegComplex128NEON,
+ Conj: arm64.CconjComplex128NEON,
+ Sum: arm64.CsumComplex128NEON,
+ Dot: arm64.CdotComplex128NEON,
+ DotConj: arm64.CdotconjComplex128NEON,
+}
+
+var cplxC128SVE2 = kernel.Complex[complex128]{
+ Add: arm64.CaddComplex128SVE2,
+ Sub: arm64.CsubComplex128SVE2,
+ Mul: arm64.CmulComplex128SVE2,
+ Div: arm64.CdivComplex128SVE2,
+ Neg: arm64.CnegComplex128SVE2,
+ Conj: arm64.CconjComplex128SVE2,
+ Sum: arm64.CsumComplex128SVE2,
+ Dot: arm64.CdotComplex128SVE2,
+ DotConj: arm64.CdotconjComplex128SVE2,
+}
+
+var cplxC128ByTier = [...]*kernel.Complex[complex128]{
+ nil,
+ &cplxC128NEON,
+ &cplxC128SVE2,
+}
+
+var cplxC64NEON = kernel.Complex[complex64]{
+ Add: arm64.CaddComplex64NEON,
+ Sub: arm64.CsubComplex64NEON,
+ Mul: arm64.CmulComplex64NEON,
+ Div: arm64.CdivComplex64NEON,
+ Neg: arm64.CnegComplex64NEON,
+ Conj: arm64.CconjComplex64NEON,
+ Sum: arm64.CsumComplex64NEON,
+ Dot: arm64.CdotComplex64NEON,
+ DotConj: arm64.CdotconjComplex64NEON,
+}
+
+var cplxC64SVE2 = kernel.Complex[complex64]{
+ Add: arm64.CaddComplex64SVE2,
+ Sub: arm64.CsubComplex64SVE2,
+ Mul: arm64.CmulComplex64SVE2,
+ Div: arm64.CdivComplex64SVE2,
+ Neg: arm64.CnegComplex64SVE2,
+ Conj: arm64.CconjComplex64SVE2,
+ Sum: arm64.CsumComplex64SVE2,
+ Dot: arm64.CdotComplex64SVE2,
+ DotConj: arm64.CdotconjComplex64SVE2,
+}
+
+var cplxC64ByTier = [...]*kernel.Complex[complex64]{
+ nil,
+ &cplxC64NEON,
+ &cplxC64SVE2,
+}
+
+var partsC128PartsNEON = kernel.ComplexParts[complex128, float64]{
+ Abs: arm64.CabsComplex128NEON,
+ Real: arm64.CrealComplex128NEON,
+ Imag: arm64.CimagComplex128NEON,
+ Scale: arm64.CscaleComplex128NEON,
+ FromParts: arm64.CfromPartsComplex128NEON,
+}
+
+var partsC128PartsSVE2 = kernel.ComplexParts[complex128, float64]{
+ Abs: arm64.CabsComplex128SVE2,
+ Real: arm64.CrealComplex128SVE2,
+ Imag: arm64.CimagComplex128SVE2,
+ Scale: arm64.CscaleComplex128SVE2,
+ FromParts: arm64.CfromPartsComplex128SVE2,
+}
+
+var partsC128PartsByTier = [...]*kernel.ComplexParts[complex128, float64]{
+ nil,
+ &partsC128PartsNEON,
+ &partsC128PartsSVE2,
+}
+
+var partsC64PartsNEON = kernel.ComplexParts[complex64, float32]{
+ Abs: arm64.CabsComplex64NEON,
+ Real: arm64.CrealComplex64NEON,
+ Imag: arm64.CimagComplex64NEON,
+ Scale: arm64.CscaleComplex64NEON,
+ FromParts: arm64.CfromPartsComplex64NEON,
+}
+
+var partsC64PartsSVE2 = kernel.ComplexParts[complex64, float32]{
+ Abs: arm64.CabsComplex64SVE2,
+ Real: arm64.CrealComplex64SVE2,
+ Imag: arm64.CimagComplex64SVE2,
+ Scale: arm64.CscaleComplex64SVE2,
+ FromParts: arm64.CfromPartsComplex64SVE2,
+}
+
+var partsC64PartsByTier = [...]*kernel.ComplexParts[complex64, float32]{
+ nil,
+ &partsC64PartsNEON,
+ &partsC64PartsSVE2,
+}
diff --git a/dispatch_tables_arm64_test.go b/dispatch_tables_arm64_test.go
index c793529..4fb7aaf 100644
--- a/dispatch_tables_arm64_test.go
+++ b/dispatch_tables_arm64_test.go
@@ -127,5 +127,25 @@ var allFlatTables = map[string][]any{
"Bytes.VarintDecodeU64": {tblBytesVarintDecodeU64[0], tblBytesVarintDecodeU64[1], tblBytesVarintDecodeU64[2]},
}
+// allGroupTables enumerates the per-tier partial structs the runtime
+// overlays from, keyed by kernel.Set group. Slot 0 is the reference
+// and is nil; the rest are one per accelerated tier, in tier order.
+var allGroupTables = map[string][]any{
+ "F32": {nil, opsF32NEON, opsF32SVE2},
+ "F64": {nil, opsF64NEON, opsF64SVE2},
+ "I16": {nil, opsI16NEON, opsI16SVE2},
+ "I32": {nil, opsI32NEON, opsI32SVE2},
+ "I64": {nil, opsI64NEON, opsI64SVE2},
+ "I8": {nil, opsI8NEON, opsI8SVE2},
+ "U16": {nil, opsU16NEON, opsU16SVE2},
+ "U32": {nil, opsU32NEON, opsU32SVE2},
+ "U64": {nil, opsU64NEON, opsU64SVE2},
+ "U8": {nil, opsU8NEON, opsU8SVE2},
+ "C128": {nil, cplxC128NEON, cplxC128SVE2},
+ "C64": {nil, cplxC64NEON, cplxC64SVE2},
+ "C128Parts": {nil, partsC128PartsNEON, partsC128PartsSVE2},
+ "C64Parts": {nil, partsC64PartsNEON, partsC64PartsSVE2},
+}
+
// archSets hands the tests every tier's complete kernel set.
var archSets func() map[string]kernel.Set = arm64.Sets
diff --git a/dispatch_tables_fallback.go b/dispatch_tables_fallback.go
index 889f311..109c129 100644
--- a/dispatch_tables_fallback.go
+++ b/dispatch_tables_fallback.go
@@ -138,3 +138,7 @@ var opsU16ByTier = [...]*kernel.Ops[uint16]{nil}
var opsU32ByTier = [...]*kernel.Ops[uint32]{nil}
var opsU64ByTier = [...]*kernel.Ops[uint64]{nil}
var opsU8ByTier = [...]*kernel.Ops[uint8]{nil}
+var cplxC128ByTier = [...]*kernel.Complex[complex128]{nil}
+var cplxC64ByTier = [...]*kernel.Complex[complex64]{nil}
+var partsC128PartsByTier = [...]*kernel.ComplexParts[complex128, float64]{nil}
+var partsC64PartsByTier = [...]*kernel.ComplexParts[complex64, float32]{nil}
diff --git a/dispatch_tables_fallback_test.go b/dispatch_tables_fallback_test.go
index 375ed65..d3e5972 100644
--- a/dispatch_tables_fallback_test.go
+++ b/dispatch_tables_fallback_test.go
@@ -122,6 +122,10 @@ var allFlatTables = map[string][]any{
"Bytes.VarintDecodeU64": {tblBytesVarintDecodeU64[0]},
}
+// No generated backend here: every group table is the reference slot
+// alone, so the walk has nothing to check.
+var allGroupTables = map[string][]any{}
+
// No generated backend here: the numeric half of the inventory walk
// has nothing to check and skips on nil.
var archSets func() map[string]kernel.Set
diff --git a/dispatch_tables_loong64.go b/dispatch_tables_loong64.go
index 961cdf2..e9f795e 100644
--- a/dispatch_tables_loong64.go
+++ b/dispatch_tables_loong64.go
@@ -650,6 +650,7 @@ var opsF32LASX = kernel.Ops[float32]{
Correlate: loong64.CorrelateFloat32LASX,
Tile: loong64.TileFloat32LASX,
Gather: loong64.GatherFloat32LASX,
+ MovingAverage: loong64.MovingAverageFloat32LASX,
ShiftDiv: loong64.ShiftDivFloat32LASX,
LayerNorm: loong64.LayerNormFloat32LASX,
Random: loong64.RandomF32LASX,
@@ -746,6 +747,7 @@ var opsF64LASX = kernel.Ops[float64]{
Correlate: loong64.CorrelateFloat64LASX,
Tile: loong64.TileFloat64LASX,
Gather: loong64.GatherFloat64LASX,
+ MovingAverage: loong64.MovingAverageFloat64LASX,
ShiftDiv: loong64.ShiftDivFloat64LASX,
LayerNorm: loong64.LayerNormFloat64LASX,
Random: loong64.RandomF64LASX,
@@ -1308,3 +1310,55 @@ var opsU8ByTier = [...]*kernel.Ops[uint8]{
nil,
&opsU8LASX,
}
+
+var cplxC128LASX = kernel.Complex[complex128]{
+ Add: loong64.CaddComplex128LASX,
+ Sub: loong64.CsubComplex128LASX,
+ Neg: loong64.CnegComplex128LASX,
+ Sum: loong64.CsumComplex128LASX,
+ Dot: loong64.CdotComplex128LASX,
+ DotConj: loong64.CdotconjComplex128LASX,
+}
+
+var cplxC128ByTier = [...]*kernel.Complex[complex128]{
+ nil,
+ &cplxC128LASX,
+}
+
+var cplxC64LASX = kernel.Complex[complex64]{
+ Add: loong64.CaddComplex64LASX,
+ Sub: loong64.CsubComplex64LASX,
+ Neg: loong64.CnegComplex64LASX,
+ Sum: loong64.CsumComplex64LASX,
+ Dot: loong64.CdotComplex64LASX,
+ DotConj: loong64.CdotconjComplex64LASX,
+}
+
+var cplxC64ByTier = [...]*kernel.Complex[complex64]{
+ nil,
+ &cplxC64LASX,
+}
+
+var partsC128PartsLASX = kernel.ComplexParts[complex128, float64]{
+ Abs: loong64.CabsComplex128LASX,
+ Real: loong64.CrealComplex128LASX,
+ Imag: loong64.CimagComplex128LASX,
+ Scale: loong64.CscaleComplex128LASX,
+ FromParts: loong64.CfromPartsComplex128LASX,
+}
+
+var partsC128PartsByTier = [...]*kernel.ComplexParts[complex128, float64]{
+ nil,
+ &partsC128PartsLASX,
+}
+
+var partsC64PartsLASX = kernel.ComplexParts[complex64, float32]{
+ Real: loong64.CrealComplex64LASX,
+ Imag: loong64.CimagComplex64LASX,
+ Scale: loong64.CscaleComplex64LASX,
+}
+
+var partsC64PartsByTier = [...]*kernel.ComplexParts[complex64, float32]{
+ nil,
+ &partsC64PartsLASX,
+}
diff --git a/dispatch_tables_loong64_test.go b/dispatch_tables_loong64_test.go
index acfef7c..a6b31d5 100644
--- a/dispatch_tables_loong64_test.go
+++ b/dispatch_tables_loong64_test.go
@@ -127,5 +127,25 @@ var allFlatTables = map[string][]any{
"Bytes.VarintDecodeU64": {tblBytesVarintDecodeU64[0], tblBytesVarintDecodeU64[1]},
}
+// allGroupTables enumerates the per-tier partial structs the runtime
+// overlays from, keyed by kernel.Set group. Slot 0 is the reference
+// and is nil; the rest are one per accelerated tier, in tier order.
+var allGroupTables = map[string][]any{
+ "F32": {nil, opsF32LASX},
+ "F64": {nil, opsF64LASX},
+ "I16": {nil, opsI16LASX},
+ "I32": {nil, opsI32LASX},
+ "I64": {nil, opsI64LASX},
+ "I8": {nil, opsI8LASX},
+ "U16": {nil, opsU16LASX},
+ "U32": {nil, opsU32LASX},
+ "U64": {nil, opsU64LASX},
+ "U8": {nil, opsU8LASX},
+ "C128": {nil, cplxC128LASX},
+ "C64": {nil, cplxC64LASX},
+ "C128Parts": {nil, partsC128PartsLASX},
+ "C64Parts": {nil, partsC64PartsLASX},
+}
+
// archSets hands the tests every tier's complete kernel set.
var archSets func() map[string]kernel.Set = loong64.Sets
diff --git a/dispatch_tables_ppc64le.go b/dispatch_tables_ppc64le.go
index 35129af..0d75800 100644
--- a/dispatch_tables_ppc64le.go
+++ b/dispatch_tables_ppc64le.go
@@ -1201,3 +1201,59 @@ var opsU8ByTier = [...]*kernel.Ops[uint8]{
nil,
&opsU8VSX,
}
+
+var cplxC128VSX = kernel.Complex[complex128]{
+ Add: ppc64le.CaddComplex128VSX,
+ Sub: ppc64le.CsubComplex128VSX,
+ Mul: ppc64le.CmulComplex128VSX,
+ Neg: ppc64le.CnegComplex128VSX,
+ Conj: ppc64le.CconjComplex128VSX,
+ Sum: ppc64le.CsumComplex128VSX,
+ Dot: ppc64le.CdotComplex128VSX,
+ DotConj: ppc64le.CdotconjComplex128VSX,
+}
+
+var cplxC128ByTier = [...]*kernel.Complex[complex128]{
+ nil,
+ &cplxC128VSX,
+}
+
+var cplxC64VSX = kernel.Complex[complex64]{
+ Add: ppc64le.CaddComplex64VSX,
+ Sub: ppc64le.CsubComplex64VSX,
+ Mul: ppc64le.CmulComplex64VSX,
+ Div: ppc64le.CdivComplex64VSX,
+ Neg: ppc64le.CnegComplex64VSX,
+ Conj: ppc64le.CconjComplex64VSX,
+}
+
+var cplxC64ByTier = [...]*kernel.Complex[complex64]{
+ nil,
+ &cplxC64VSX,
+}
+
+var partsC128PartsVSX = kernel.ComplexParts[complex128, float64]{
+ Abs: ppc64le.CabsComplex128VSX,
+ Real: ppc64le.CrealComplex128VSX,
+ Imag: ppc64le.CimagComplex128VSX,
+ Scale: ppc64le.CscaleComplex128VSX,
+ FromParts: ppc64le.CfromPartsComplex128VSX,
+}
+
+var partsC128PartsByTier = [...]*kernel.ComplexParts[complex128, float64]{
+ nil,
+ &partsC128PartsVSX,
+}
+
+var partsC64PartsVSX = kernel.ComplexParts[complex64, float32]{
+ Abs: ppc64le.CabsComplex64VSX,
+ Real: ppc64le.CrealComplex64VSX,
+ Imag: ppc64le.CimagComplex64VSX,
+ Scale: ppc64le.CscaleComplex64VSX,
+ FromParts: ppc64le.CfromPartsComplex64VSX,
+}
+
+var partsC64PartsByTier = [...]*kernel.ComplexParts[complex64, float32]{
+ nil,
+ &partsC64PartsVSX,
+}
diff --git a/dispatch_tables_ppc64le_test.go b/dispatch_tables_ppc64le_test.go
index 5528db9..d2e53ec 100644
--- a/dispatch_tables_ppc64le_test.go
+++ b/dispatch_tables_ppc64le_test.go
@@ -127,5 +127,25 @@ var allFlatTables = map[string][]any{
"Bytes.VarintDecodeU64": {tblBytesVarintDecodeU64[0], tblBytesVarintDecodeU64[1]},
}
+// allGroupTables enumerates the per-tier partial structs the runtime
+// overlays from, keyed by kernel.Set group. Slot 0 is the reference
+// and is nil; the rest are one per accelerated tier, in tier order.
+var allGroupTables = map[string][]any{
+ "F32": {nil, opsF32VSX},
+ "F64": {nil, opsF64VSX},
+ "I16": {nil, opsI16VSX},
+ "I32": {nil, opsI32VSX},
+ "I64": {nil, opsI64VSX},
+ "I8": {nil, opsI8VSX},
+ "U16": {nil, opsU16VSX},
+ "U32": {nil, opsU32VSX},
+ "U64": {nil, opsU64VSX},
+ "U8": {nil, opsU8VSX},
+ "C128": {nil, cplxC128VSX},
+ "C64": {nil, cplxC64VSX},
+ "C128Parts": {nil, partsC128PartsVSX},
+ "C64Parts": {nil, partsC64PartsVSX},
+}
+
// archSets hands the tests every tier's complete kernel set.
var archSets func() map[string]kernel.Set = ppc64le.Sets
diff --git a/dispatch_tables_riscv64.go b/dispatch_tables_riscv64.go
index 24ef710..24be578 100644
--- a/dispatch_tables_riscv64.go
+++ b/dispatch_tables_riscv64.go
@@ -1420,3 +1420,63 @@ var opsU8ByTier = [...]*kernel.Ops[uint8]{
nil,
&opsU8RVV,
}
+
+var cplxC128RVV = kernel.Complex[complex128]{
+ Add: riscv64.CaddComplex128RVV,
+ Sub: riscv64.CsubComplex128RVV,
+ Mul: riscv64.CmulComplex128RVV,
+ Div: riscv64.CdivComplex128RVV,
+ Neg: riscv64.CnegComplex128RVV,
+ Conj: riscv64.CconjComplex128RVV,
+ Sum: riscv64.CsumComplex128RVV,
+ Dot: riscv64.CdotComplex128RVV,
+ DotConj: riscv64.CdotconjComplex128RVV,
+}
+
+var cplxC128ByTier = [...]*kernel.Complex[complex128]{
+ nil,
+ &cplxC128RVV,
+}
+
+var cplxC64RVV = kernel.Complex[complex64]{
+ Add: riscv64.CaddComplex64RVV,
+ Sub: riscv64.CsubComplex64RVV,
+ Mul: riscv64.CmulComplex64RVV,
+ Div: riscv64.CdivComplex64RVV,
+ Neg: riscv64.CnegComplex64RVV,
+ Conj: riscv64.CconjComplex64RVV,
+ Sum: riscv64.CsumComplex64RVV,
+ Dot: riscv64.CdotComplex64RVV,
+ DotConj: riscv64.CdotconjComplex64RVV,
+}
+
+var cplxC64ByTier = [...]*kernel.Complex[complex64]{
+ nil,
+ &cplxC64RVV,
+}
+
+var partsC128PartsRVV = kernel.ComplexParts[complex128, float64]{
+ Abs: riscv64.CabsComplex128RVV,
+ Real: riscv64.CrealComplex128RVV,
+ Imag: riscv64.CimagComplex128RVV,
+ Scale: riscv64.CscaleComplex128RVV,
+ FromParts: riscv64.CfromPartsComplex128RVV,
+}
+
+var partsC128PartsByTier = [...]*kernel.ComplexParts[complex128, float64]{
+ nil,
+ &partsC128PartsRVV,
+}
+
+var partsC64PartsRVV = kernel.ComplexParts[complex64, float32]{
+ Abs: riscv64.CabsComplex64RVV,
+ Real: riscv64.CrealComplex64RVV,
+ Imag: riscv64.CimagComplex64RVV,
+ Scale: riscv64.CscaleComplex64RVV,
+ FromParts: riscv64.CfromPartsComplex64RVV,
+}
+
+var partsC64PartsByTier = [...]*kernel.ComplexParts[complex64, float32]{
+ nil,
+ &partsC64PartsRVV,
+}
diff --git a/dispatch_tables_riscv64_test.go b/dispatch_tables_riscv64_test.go
index 1d02dc7..d549421 100644
--- a/dispatch_tables_riscv64_test.go
+++ b/dispatch_tables_riscv64_test.go
@@ -127,5 +127,25 @@ var allFlatTables = map[string][]any{
"Bytes.VarintDecodeU64": {tblBytesVarintDecodeU64[0], tblBytesVarintDecodeU64[1]},
}
+// allGroupTables enumerates the per-tier partial structs the runtime
+// overlays from, keyed by kernel.Set group. Slot 0 is the reference
+// and is nil; the rest are one per accelerated tier, in tier order.
+var allGroupTables = map[string][]any{
+ "F32": {nil, opsF32RVV},
+ "F64": {nil, opsF64RVV},
+ "I16": {nil, opsI16RVV},
+ "I32": {nil, opsI32RVV},
+ "I64": {nil, opsI64RVV},
+ "I8": {nil, opsI8RVV},
+ "U16": {nil, opsU16RVV},
+ "U32": {nil, opsU32RVV},
+ "U64": {nil, opsU64RVV},
+ "U8": {nil, opsU8RVV},
+ "C128": {nil, cplxC128RVV},
+ "C64": {nil, cplxC64RVV},
+ "C128Parts": {nil, partsC128PartsRVV},
+ "C64Parts": {nil, partsC64PartsRVV},
+}
+
// archSets hands the tests every tier's complete kernel set.
var archSets func() map[string]kernel.Set = riscv64.Sets
diff --git a/dispatch_tables_s390x.go b/dispatch_tables_s390x.go
index ef3d9f8..a99a780 100644
--- a/dispatch_tables_s390x.go
+++ b/dispatch_tables_s390x.go
@@ -1022,3 +1022,58 @@ var opsU8ByTier = [...]*kernel.Ops[uint8]{
nil,
&opsU8VX,
}
+
+var cplxC128VX = kernel.Complex[complex128]{
+ Add: s390x.CaddComplex128VX,
+ Sub: s390x.CsubComplex128VX,
+ Mul: s390x.CmulComplex128VX,
+ Div: s390x.CdivComplex128VX,
+ Neg: s390x.CnegComplex128VX,
+ Sum: s390x.CsumComplex128VX,
+ Dot: s390x.CdotComplex128VX,
+ DotConj: s390x.CdotconjComplex128VX,
+}
+
+var cplxC128ByTier = [...]*kernel.Complex[complex128]{
+ nil,
+ &cplxC128VX,
+}
+
+var cplxC64VX = kernel.Complex[complex64]{
+ Add: s390x.CaddComplex64VX,
+ Sub: s390x.CsubComplex64VX,
+ Mul: s390x.CmulComplex64VX,
+ Div: s390x.CdivComplex64VX,
+ Neg: s390x.CnegComplex64VX,
+ Conj: s390x.CconjComplex64VX,
+ Sum: s390x.CsumComplex64VX,
+}
+
+var cplxC64ByTier = [...]*kernel.Complex[complex64]{
+ nil,
+ &cplxC64VX,
+}
+
+var partsC128PartsVX = kernel.ComplexParts[complex128, float64]{
+ Abs: s390x.CabsComplex128VX,
+ Real: s390x.CrealComplex128VX,
+ Imag: s390x.CimagComplex128VX,
+ Scale: s390x.CscaleComplex128VX,
+}
+
+var partsC128PartsByTier = [...]*kernel.ComplexParts[complex128, float64]{
+ nil,
+ &partsC128PartsVX,
+}
+
+var partsC64PartsVX = kernel.ComplexParts[complex64, float32]{
+ Abs: s390x.CabsComplex64VX,
+ Real: s390x.CrealComplex64VX,
+ Imag: s390x.CimagComplex64VX,
+ Scale: s390x.CscaleComplex64VX,
+}
+
+var partsC64PartsByTier = [...]*kernel.ComplexParts[complex64, float32]{
+ nil,
+ &partsC64PartsVX,
+}
diff --git a/dispatch_tables_s390x_test.go b/dispatch_tables_s390x_test.go
index b1b03e4..bd7ddda 100644
--- a/dispatch_tables_s390x_test.go
+++ b/dispatch_tables_s390x_test.go
@@ -127,5 +127,25 @@ var allFlatTables = map[string][]any{
"Bytes.VarintDecodeU64": {tblBytesVarintDecodeU64[0], tblBytesVarintDecodeU64[1]},
}
+// allGroupTables enumerates the per-tier partial structs the runtime
+// overlays from, keyed by kernel.Set group. Slot 0 is the reference
+// and is nil; the rest are one per accelerated tier, in tier order.
+var allGroupTables = map[string][]any{
+ "F32": {nil, opsF32VX},
+ "F64": {nil, opsF64VX},
+ "I16": {nil, opsI16VX},
+ "I32": {nil, opsI32VX},
+ "I64": {nil, opsI64VX},
+ "I8": {nil, opsI8VX},
+ "U16": {nil, opsU16VX},
+ "U32": {nil, opsU32VX},
+ "U64": {nil, opsU64VX},
+ "U8": {nil, opsU8VX},
+ "C128": {nil, cplxC128VX},
+ "C64": {nil, cplxC64VX},
+ "C128Parts": {nil, partsC128PartsVX},
+ "C64Parts": {nil, partsC64PartsVX},
+}
+
// archSets hands the tests every tier's complete kernel set.
var archSets func() map[string]kernel.Set = s390x.Sets
diff --git a/docs/architecture.md b/docs/architecture.md
index 6735e8e..adcee0d 100644
--- a/docs/architecture.md
+++ b/docs/architecture.md
@@ -63,7 +63,9 @@ caller
reference with a tier's partial `Ops`, built lazily on first use
(`opsCache` in `dispatch.go`). The only type switch lives there, so the
public wrappers stay one-liners and a caller that never touches a type
- never links its kernels.
+ never links its kernels. The complex groups take the same route through
+ `groupCache[G]`, which is generic over the struct because `kernel.Complex`
+ and `kernel.ComplexParts` are not `kernel.Ops`.
- The reference itself (`refBase`) is a few hundred kilobytes of ordinary Go
and is linked into every consumer regardless; the per-tier assembly is what
the tables keep out.
diff --git a/docs/assets/badges/release.svg b/docs/assets/badges/release.svg
index 9eeab0f..731d9a8 100644
--- a/docs/assets/badges/release.svg
+++ b/docs/assets/badges/release.svg
@@ -1 +1 @@
-
\ No newline at end of file
+
\ No newline at end of file
diff --git a/docs/lld/kernels-and-dispatch.md b/docs/lld/kernels-and-dispatch.md
index ea3b3a3..a328d56 100644
--- a/docs/lld/kernels-and-dispatch.md
+++ b/docs/lld/kernels-and-dispatch.md
@@ -44,6 +44,15 @@ iterating `simdinfo -tiers`.
selects the per-element-type cache, which overlays the reference with the
tier's partial `Ops` struct lazily on first use (one `reflect` pass per
element type, never on the hot path).
+- The **complex groups** use the same shape through `complexOps[C]()` and
+ `complexParts[C, R]()`, over `groupCache[G]` rather than `opsCache[T]`:
+ `opsCache` is constrained to `Number`, and neither `kernel.Complex` nor
+ `kernel.ComplexParts` is a `kernel.Ops`, so the cache is generic over the
+ struct instead of over the element type and merges with `overlayAny`. The
+ per-tier tables are `cplxByTier` and `partsByTier`. This
+ route did not exist until 2026-08-14: before it, every complex operation
+ ran the portable reference on every machine while the generated complex
+ assembly sat linked into the test-only aggregator (`docs/wrong.md`).
- A **generated guard** sits between the table and the kernel. It checks the
per-operation element threshold (`KernelThreshold` is the public view;
the tables are generated from the same manifest and a test holds the two
diff --git a/docs/platforms.md b/docs/platforms.md
index 230c239..053667a 100644
--- a/docs/platforms.md
+++ b/docs/platforms.md
@@ -48,7 +48,7 @@ same portable implementations used for per-operation fallback.
## Kernel coverage
The default build exposes 493 exported functions and the repository contains
-6,931 generated kernels across nine accelerated tier targets. Counts below are
+6,933 generated kernels across nine accelerated tier targets. Counts below are
logical callable kernels, not the extra `Body` trampolines required by the
s390x ABI. `make check-emission` regenerates the inventory as a dry run.
@@ -61,7 +61,7 @@ emitted for AVX2 and declined for SSE2 appears once in each column.
| amd64 (sse2/avx2/avx512) | 2609 | 101 | LLVM declined to vectorize for a tier |
| arm64 (neon/sve2) | 1681 | 127 | LLVM declined to vectorize for a tier |
| riscv64 (rvv) | 891 | 15 | LLVM declined to vectorize |
-| loong64 (lasx) | 728 | 174 | `$fp` ownership and unresolved `.L0` references |
+| loong64 (lasx) | 730 | 172 | `$fp` ownership and unresolved `.L0` references |
| ppc64le (vsx) | 602 | 300 | `r30`/`r2`, `r0`, frame writes, then LLVM refusals |
| s390x (vx) | 420 | 482 | `r13`, which the Go runtime owns |
diff --git a/docs/research/07-nary-closure.md b/docs/research/07-nary-closure.md
new file mode 100644
index 0000000..685d8cb
--- /dev/null
+++ b/docs/research/07-nary-closure.md
@@ -0,0 +1,103 @@
+# A general n-ary combinator taking a Go closure
+
+**Status: measured, design fixed, not implemented.** This record exists so
+the implementation step has a decision behind it rather than a preference.
+Nothing in it is shipped; ROADMAP.md's n-ary section is where the open item
+lives.
+
+## The question
+
+`AddAll` and `MulAll` ship, backed by arity-3 and arity-4 kernels with
+folding beyond. The open half of that roadmap item is a general combinator:
+
+```go
+simd.CombineInto(dst, f, a, b, c) // dst[i] = f(a[i], b[i], c[i])
+```
+
+One function that takes an arbitrary Go closure and applies it elementwise.
+The roadmap already states the constraint — a closure call per element
+defeats vectorization — so the only design question worth measuring is how
+much it defeats it, and whether the answer leaves room for a useful API.
+
+## The measurement
+
+A `//go:noinline` loop calling a closure per element, against the identical
+loop with the body written in. `dst[i] = a[i]*2 + b[i]`, float64, on the
+Zen 5. Instructions retired, slope between `-benchtime 2000x` and `10000x`
+at n=65536, so the per-call setup cancels:
+
+| loop | instructions/element |
+|---|---|
+| closure per element | 17.998 |
+| body written in | 7.032 |
+
+**11.0 extra instructions per element, a 2.56x ratio.** Instructions
+retired are layout- and load-independent, which matters here: the machine
+was not quiet.
+
+Wall clock on the same runs, minimum of three at `-benchtime 2000x` — a
+weaker number, quoted only because it is the one a caller feels:
+
+| n | closure | written in | ratio |
+|---|---|---|---|
+| 64 | 76.06 ns | 15.03 ns | 5.06x |
+| 1024 | 1027 ns | 210.4 ns | 4.88x |
+| 65536 | 67391 ns | 14581 ns | 4.62x |
+
+The wall-clock ratio is worse than the instruction ratio because the
+closure loop also serialises: each call is a dependency the scheduler
+cannot hoist, so the extra instructions do not overlap.
+
+The two ratios bracket the real cost. Against a *kernel* rather than
+against a written-in scalar loop the gap is larger still, because the
+kernel is the thing the closure makes impossible.
+
+## What that rules out
+
+A combinator whose contract is "give me any closure and I will make it
+fast" cannot be built. At 11 extra instructions per element it is slower
+than the scalar loop the caller would have written, and far slower than
+chaining the binary calls that already exist. Shipping it would be an API
+whose name promises the opposite of what it measures.
+
+## The design that survives
+
+The same shape `FilterInto` already uses in this library: **dispatch the
+shapes that have kernels, and be explicit that the rest is a scalar loop.**
+
+1. **A closed set of named operations dispatches to real kernels.** Not a
+ closure — a value from an enumerated type, so the compiler and the
+ reader both know the set is finite:
+
+ ```go
+ simd.CombineInto(dst, simd.OpAdd, a, b, c)
+ ```
+
+ `OpAdd` and `OpMul` reach `Add3`/`Add4`/`Mul3`/`Mul4` and the existing
+ fold. This is `AddAll`/`MulAll` with the operation as a parameter, and
+ it is worth having only if a caller genuinely needs to choose at run
+ time; otherwise `AddAll` is clearer at the call site.
+
+2. **The closure form is a separate function with the cost in its name and
+ its doc.** Something like `CombineFuncInto`, documented as a scalar
+ loop, with the measured 11 instructions per element in the comment so
+ nobody has to rediscover it. It exists for the case where the caller has
+ an operation this library does not have and wants one pass over the data
+ rather than materialising intermediates — which is a real want, and is
+ about memory traffic, not about vector width.
+
+3. **No heuristic that silently switches between them.** A combinator that
+ inspected the closure and sometimes used a kernel would be a guess about
+ what the closure does. The two functions do different things and say so.
+
+The measurement gate for step 2: `CombineFuncInto` ships only if it beats
+the binary chain it replaces at a size where memory traffic dominates — the
+same gate `AddAll` passed. If it does not beat it anywhere, it does not
+ship, and the number goes in `docs/wrong.md`.
+
+## What is not decided here
+
+The exact names and signatures. Those follow the API conventions in
+`docs/api.md` at implementation time, and the enumerated-operation form may
+turn out not to be worth its own function at all — in which case only the
+closure form ships, or neither does.
diff --git a/docs/research/08-goexperiment-simd-small-n.md b/docs/research/08-goexperiment-simd-small-n.md
new file mode 100644
index 0000000..b0449ff
--- /dev/null
+++ b/docs/research/08-goexperiment-simd-small-n.md
@@ -0,0 +1,107 @@
+# `GOEXPERIMENT=simd` for the small-n band
+
+**Status: measured favourable in isolation, then measured AGAIN as built and
+found NEGATIVE. See docs/wrong.md entry 75 before using anything below.**
+
+The table in this document is a direct, inlinable loop. Put behind a package
+boundary -- which is what a shared fallback for every generated guard has to be
+-- the same arithmetic costs **+25.6%** instructions at n=8, **+111.6%** at
+n=9, and **+83.7%** for float64 at n=8, because the loop cannot be inlined into
+the guard while the one-line `ref` forward can. The sizes below are all exact
+multiples of the vector width, which hides the tail cost entirely.
+
+What survives: the bit-identity claim, which was re-verified at every length
+0-40 over IEEE specials for add, sub, mul and div, on both lanes.
+
+**Original status: measured, favourable, not implemented.** This record exists so the
+implementation step has a number behind it rather than a table copied from a
+plan. Nothing in it is shipped; the open item lives in ROADMAP.md's tiers
+section.
+
+## The question
+
+Every generated guard falls back to `internal/ref` below its threshold:
+
+```go
+func addF32AVX512Guarded(dst, a, b []float32) {
+ if len(dst) < threshold { ref.AddInto(dst, a, b); return }
+ addF32AVX512(dst, a, b)
+}
+```
+
+`ref` is a plain Go loop, and below the threshold that loop is the whole answer.
+The proposal is to route the guard's fallback through Go's own `simd/archsimd`
+instead — inlinable intrinsics rather than a non-inlinable assembly call, which
+is what makes them usable at n = 8 where a call costs more than the work.
+
+Two things had to be true before any of it is worth writing: the intrinsics have
+to be **bit-identical** to the portable loop, and they have to actually **win**
+at the sizes where the fallback runs. Both are measurable today — Go 1.26.5 ships
+`simd/archsimd` behind `GOEXPERIMENT=simd` — so they were measured before
+anything was built.
+
+## Bit-identity
+
+Float addition is not reassociated by either path (elementwise, one operation
+per lane), so identity is expected rather than hoped for. Checked anyway, over
+n = 0, 1, 3, 7, 8, 9, 15, 16, 17, 31, 32, 33 with values chosen to be
+cancellation-prone (`i*0.1 + 1e-8` against `i*-0.3 + 1e30`), including every
+tail length the partial-load helpers have to handle:
+
+**Identical at every index, every n.** `LoadFloat32x8SlicePart` /
+`StoreSlicePart` handle the tail without reading or writing past the slice, which
+is the part a hand-written version gets wrong.
+
+## The measurement
+
+`f32 dst[i] = a[i] + b[i]`, the scalar loop against an `archsimd` 8-lane loop
+plus a partial-load tail. Instructions retired and cycles, `perf stat -e
+instructions:u,cycles:u`, three interleaved rounds in one session, 2,000,000
+iterations each, minimum reported. Instructions rather than wall clock because
+the machine's load average was above 1 and this repository's wall-clock noise
+floor is 8.3%.
+
+| n | scalar instr | archsimd instr | | scalar cycles | archsimd cycles | |
+|---|---|---|---|---|---|---|
+| 8 | 227,775,443 | 183,973,765 | **−19.2%** | 38,225,341 | 33,269,977 | −13.0% |
+| 16 | 387,771,780 | 284,004,004 | **−26.8%** | 59,050,260 | 46,597,720 | −21.1% |
+| 32 | 708,189,014 | 483,774,777 | **−31.7%** | 103,791,526 | 64,829,258 | −37.5% |
+
+Every range is disjoint: the slowest archsimd round beats the fastest scalar
+round at all three sizes. The gain grows with n across the band, which is what a
+vector width of 8 predicts — at n = 8 one vector operation replaces eight scalar
+ones but the loop scaffolding is a larger share; by n = 32 it is four vectors
+against thirty-two.
+
+## What this does NOT measure
+
+Stated because the number above is the tempting one to quote:
+
+- **It is a direct call, not the guarded one.** The library's fallback is reached
+ through a dispatch guard, and the guard's own compare and branch are not in
+ either column. The gain should carry — the guard is identical on both sides —
+ but the end-to-end figure will be smaller than 19–32%.
+- **One operation.** `AddInto` is the friendliest possible case: elementwise,
+ one instruction per lane, no reassociation question. The reductions are the
+ interesting ones and they are the ones where bit-identity is genuinely at
+ risk, because `kernel.CombineTree` uses a fixed sixteen-accumulator shape that
+ an intrinsics loop would have to reproduce exactly rather than merely
+ correctly.
+- **amd64 only, here.** `archsimd` builds for amd64 and arm64; riscv64, s390x,
+ loong64 and ppc64le keep the `ref` fallback and gain nothing. Any claim made
+ from this table has to say so.
+- **The build tag is viral.** Behind `//go:build goexperiment.simd`, every lane
+ has to run twice — tagged and untagged — or the tagged path becomes the
+ vacuously-green lane `docs/wrong.md` entry 41 warned about. That cost is
+ ongoing and is not in any of these numbers.
+
+## The decision this supports
+
+Build it, elementwise operations first, with bit-identity as a gate rather than
+an expectation: an operation whose intrinsics path is not bit-identical to `ref`
+does not ship for that operation. Reductions are a separate decision needing
+their own measurement.
+
+The reproducer is thirty lines and lives in this document's history rather than
+in the tree, because a benchmark for a package that does not exist yet is a
+maintenance cost with no reader.
diff --git a/docs/verification.md b/docs/verification.md
index 141fa7c..e2c1c5f 100644
--- a/docs/verification.md
+++ b/docs/verification.md
@@ -17,7 +17,7 @@ complete list. Command roles come from the Makefile, the CI workflow
| `make test-vec` | `GOEXPERIMENT=simd go test` | the amd64-only vector type compiles and passes; skipped where the toolchain lacks the experiment |
| `make test-tiers` | one `go test` per tier the CPU supports (`GOSIMD=`) | catches a kernel correct on AVX-512 and wrong on SSE2 |
| `make test-race` | `go test -race` | full suite under the race detector |
-| `make fuzz` | `FuzzDifferential` (public API) and `FuzzKernelsAgainstReference` (every kernel of every tier vs the reference), `FUZZTIME` seconds each | adversarial bit patterns: signalling NaNs, denormals, exponent boundaries |
+| `make fuzz` | every `Fuzz*` in the tree, discovered per package, `FUZZTIME` seconds each — today `FuzzDifferential` (public API), `FuzzKernelsAgainstReference` (every kernel of every tier vs the reference) and `FuzzJSONMasksMatchesSeparateCalls` | adversarial bit patterns: signalling NaNs, denormals, exponent boundaries |
| `make test-cross` | arm64, s390x, ppc64le under docker + qemu, hermetic (`--network=none`, `CGO_ENABLED=0`, `-vet=off`, `-short`), `simdinfo -require-accelerated` first | the correctness half of the non-amd64 backends; three bugs shipped past a green amd64 suite and only appeared here — two memory corruptions (a kernel clobbering the goroutine register, kernels writing into the caller's frame through a save area) and a reference that computed different bits on architectures where Go fuses a multiply into an add |
| `make test-riscv64` / `make test-loong64` | qemu-user lanes (recent emulators, explicit `-cpu`), `-require-accelerated` | the two architectures the docker lane cannot reach |
| `make test-gates` | riscv64 with **no** vector extension, no acceleration assertion | the fallback must still pass; the SIGILL mis-gate case |
diff --git a/docs/wrong.md b/docs/wrong.md
index f6ffe9c..a0fa3af 100644
--- a/docs/wrong.md
+++ b/docs/wrong.md
@@ -3043,3 +3043,621 @@ element-scratch allocation, but duplicate-skew recovery can allocate one mask
for each equal run it extracts. The active API documentation now states that
exception. Historical changelog text remains a record of the implementation at
the time of each release.
+
+## Every complex kernel was generated, tested, and dispatched to by nothing
+
+**Believed.** The complex surface -- `DotComplex`, `DotComplexConj`,
+`ScaleComplex`, `SumComplex`, the arithmetic set, and the ComplexParts half
+that produces real slices -- runs on the generated assembly like the rest of
+the library. nine `internal//complex_*.s` files exist -- sse2, avx2, avx512, neon,
+sve2, rvv, vsx, vx, lasx, across six architectures -- `registerComplex`
+installs them, and `TestInventoryCoversEveryGroup` asserts that `C64`,
+`C128`, `C64Parts` and `C128Parts` all carry declared kernels. All of that
+was true.
+
+**Actually.** None of it was reachable from a consumer. `complexOps[C]()`
+returned `&refBase.C64` / `&refBase.C128` *directly*, with no per-tier
+overlay, and `complexParts` did the same:
+
+```go
+case complex128:
+ return any(&refBase.C128).(*kernel.Complex[C]) // complex.go:33, before
+```
+
+The dispatch tables the runtime indexes carried no complex entry at all --
+`grep -c Complex dispatch_tables_amd64.go` was **0**, against ten
+`opsXXByTier` tables for the real element types -- because
+`tools/simdgen/emit/dispatch_gen.go` had a `numericElem` map and no complex
+equivalent, so `numericGroupsIn` skipped every complex group and emitted
+nothing for them. The generated complex assembly was reachable only from
+`internal//sets_gen_.go`, whose own generated header says "Tests
+call this; consumers never do".
+
+The test that should have caught it is `TestDeclaredKernelsAreWired`, not
+`TestInventoryCoversEveryGroup` -- the first version of this entry named the
+wrong one. `TestDeclaredKernelsAreWired` checks the runtime tables
+(`allFlatTables`) for exactly three groups, `Bytes`, `Convert` and `Mask`,
+and sends everything else -- numeric AND complex -- through `archSets()`,
+the test-only aggregator. So the numeric groups have the same nominal hole;
+they were saved by having real tables, not by being checked. Nothing walked
+the tables the runtime actually uses for a non-flat group.
+
+Confirmed by disassembly, which is the only thing that would have shown it.
+The symbol survives only where the call is not inlined -- the root package's
+own test binary inlines it and prints nothing, so this is the
+`internal/benchmarks` binary:
+
+```
+$ go test -c -o /tmp/bench.test ./internal/benchmarks
+$ go tool objdump -s 'simd\.DotComplex\[' /tmp/bench.test
+ complex.go:31 LEAQ github.com/sebishogun/simd.refBase+13560(SB), AX
+```
+
+Empirically, from a consumer module calling `simd.DotComplex`: 0 complex
+kernel symbols linked before, 24 after. A consumer calling only `simd.Sum`
+links 0 in both, so the per-operation dead-code elimination survives.
+
+**How it surfaced.** Not from this repository. A downstream evaluation
+(simdblas Task 4, complex Level 1 against gonum) measured `Zdotu`/`Zdotc`
+2.2x-3.1x slower than gonum at n <= 100000, 1.65x-1.86x at n = 1e6. Not
+against "a plain Go loop", as the first version of this entry said: gonum's
+unit-increment complex dot calls `c128.DotuUnitary`/`DotcUnitary`, which are
+hand-written AVX assembly on amd64. Losing to it is not by itself proof of
+anything. The tell was that `GOSIMD=scalar` and the detected tier gave
+identical timings.
+
+**Cost, measured.** `GOSIMD=scalar` against `avx512` in one binary --
+one build, so the 8.3% layout floor does not apply; there is no second
+layout to differ. Complete grid, minimum of six alternating passes,
+reproduced twice. **amd64/avx512 only**: no other architecture was measured,
+and the per-tier ratios elsewhere are not these.
+
+The machine was at load 5-12 for this, against the project rule of load
+under 1. The wall-clock column is therefore the weaker evidence; the
+instruction column below it is layout- and load-independent and is what the
+conclusion rests on.
+
+| operation | scalar ns/op | avx512 ns/op | ratio |
+|---|---|---|---|
+| Dot c128 n=1024 | 749.7 | 101.5 | 7.39x |
+| Dot c128 n=65536 | 47156 | 15459 | 3.05x |
+| Dot c128 n=1048576 | 805434 | 408702 | 1.97x |
+| DotConj c128 n=1024 | 890.1 | 102.5 | 8.68x |
+| DotConj c128 n=65536 | 55570 | 15488 | 3.59x |
+| DotConj c64 n=1024 | 1249 | 97.31 | 12.8x |
+| DotConj c64 n=65536 | 83114 | 6614 | 12.6x |
+| DotConj c64 n=1048576 | 1276125 | 158120 | 8.07x |
+| Sum c128 n=1024 | 470.1 | 43.43 | 10.8x |
+| Sum c128 n=65536 | 29002 | 5611 | 5.17x |
+| Sum c128 n=1048576 | 480519 | 115280 | 4.17x |
+| DotConj c128 n=1048576 | 918445 | 420455 | 2.18x |
+
+Instructions retired per element, two-point slope so setup cancels, which
+is the number that does not move with load:
+
+| operation | scalar | avx512 | ratio |
+|---|---|---|---|
+| Sum c128 n=1048576 | 12577259 | 1040671 | 12.1x |
+| Dot c128 n=1048576 | 32070536 | 2627180 | 12.2x |
+| DotConj c64 n=1048576 | 44563734 | 2030498 | 22.0x |
+
+(The three `DotConj/c128` cells have no instruction row: `go test -bench`
+cannot exclude the deeper `impl=naive` sub-benchmark, so selecting them
+also runs it and its instructions scale with `-benchtime` too.)
+
+**When, exactly.** Not "since the kernels were written" -- the first version
+of this entry said that and it is wrong. The kernels landed in `ba09d24` and
+dispatched correctly through `active.C64`/`active.C128`. The break came with
+`02c258a`, "Per-operation dispatch: the linker keeps only what a program
+calls", which replaced `active` with `refBase` plus per-tier tables and
+emitted no complex tables:
+
+```
+$ git show v1.13.0:complex.go | grep 'active\.C64' -> return any(&active.C64)...
+$ git show v1.14.0:complex.go | grep 'refBase\.C64' -> return any(&refBase.C64)...
+$ git tag --contains 02c258a --sort=creatordate | head -1
+v1.14.0
+```
+
+So the affected releases are **v1.14.0 through v1.20.0** -- seven tags. A
+consumer pinned to v1.13.0 or earlier was never affected. The table
+mechanism itself arrived in that commit, so the missing `complexElem` was
+never an omission from an older design; it was a group the new design did
+not carry over.
+
+**The repository's own baseline held the evidence.** `testdata/bench/amd64.txt`
+was recorded at `f6c72f9`, before the regression, with
+`BenchmarkComplexReduce/Dot/c128/n=1024-8` at 101.2-101.9 ns/op -- a
+benchmark that calls the public `simd.DotComplex`. The portable path for
+that shape is 749.7 ns/op, 7x against `make bench-check`'s 25% threshold.
+Whether the gate was run at `02c258a` is not recoverable from here; the
+baseline was never updated, which is why it is still pre-regression and
+still a working detector.
+
+**Fix.** `dispatch_gen.go` gained `complexElem`/`partsElem` and an
+`emitTierTable` that writes `cplxByTier` and `partsByTier`
+alongside the existing `opsByTier`; `dispatch.go` gained a
+`groupCache[G any]` (opsCache is constrained to `Number`, and neither
+`kernel.Complex` nor `kernel.ComplexParts` is `kernel.Ops`) with an
+`overlayAny`; `complexOps`/`complexParts` now route through it. The
+regenerated tables are **additive only** -- 0 deleted lines across all seven
+`dispatch_tables_*.go`, 479 added.
+
+`dispatch_complex_test.go` is the test that would have caught it: it compares
+the function pointer `DotComplex` actually reaches against the reference
+set's, which is the question a caller asks and the inventory walk does not.
+
+## The three-way partition may already be built, and the 34% has not been re-measured
+
+ROADMAP.md records one losing case for `Sort`: few distinct values at 16384,
+by 34% against `slices.Sort`, with "**the fix is a three-way partition**" and a
+note that it "needs a second kernel". `docs/plans/2026-08-13-simd-production.md`
+Task 1 carries the same plan.
+
+Reading `sort.go` before writing the kernel: `extractEqual` already does the
+three-way split. When the split comes back skewed past `sortSkewLimit`, it
+takes the run equal to the pivot out of the recursion entirely — which is what
+a below/equal/above partition buys — and it does so with kernels this package
+already ships (`EqualScalarInto`, `CountTrue`, `NotMask`, `CompressInto`), so
+no second kernel was needed for that part. Its own comment says so.
+
+That does not close the task, and the reason is that nothing here has been
+measured on a quiet machine. Two attempts:
+
+ load 2.2-2.5, min of 3 x 300: simd 28,531 ns slices 23,000 ns 1.24x
+ load 12.0, min of 5 x 400: simd 117,359 ns slices 79,546 ns noise
+
+The second set is unusable — the same benchmark moved by 4x between them, and
+three review agents were running. The first set is not quiet either. What can
+be said: the gap at that shape is real and still there, and 1.24x is not 1.34x.
+What cannot be said: whether it is 24%, whether the difference from 34% is the
+`extractEqual` work landing, or whether either number survives a quiet machine.
+
+So no number in ROADMAP.md was changed. Overwriting a measured claim with a
+worse-conditioned one is not a correction. What the task needs first is the
+measurement under load average below 1, and then a profile — because if the
+residual is the `copy(a, scratch[:len(a)])` after every partition rather than
+the split itself, a three-way kernel does not address it and would be built for
+a cause nobody checked.
+
+## A forgotten reference wiring shipped past the whole normal-lane suite
+
+**Probed, 2026-08-15.** `Sqrt: sqrt[T]` deleted from `floatOps` in
+`internal/ref/ref.go` — one entry, the shape of a wiring somebody forgets when
+adding a kernel.
+
+| lane | result |
+|---|---|
+| `TestDeclaredKernelsAreWired`, normal | **PASS** (verified with `-v` that it ran) |
+| `TestDeclaredKernelsAreWired`, purego | **PASS** |
+| whole suite, normal — all 14 packages | **PASS** |
+| whole suite, purego | FAIL, `TestInPlaceMatchesInto` |
+
+So the completeness gate that exists for exactly this did not see it, and the
+only thing that did was a functional test in a lane nobody runs by default.
+
+The cause is scope. `TestDeclaredKernelsAreWired` walks `backend.Inventory`
+against the dispatch tables and `archSets()` — the GENERATED side. `ref.Set()`
+is never in the subject, and its doc comment says so plainly: "The subject is
+`active` ... and not `ref.Set()`. That is not a shortcut, it is the only
+correct choice", because ref leaves every Fast slot nil until a backend
+registers. That reasoning is right about Fast slots and was taken to cover the
+whole reference.
+
+The reference is not optional. It is the live fallback on architectures with
+no backend, in `purego` builds, and below every kernel's element threshold —
+so a nil there is a nil call in the small-n path of an operation that works on
+a big slice.
+
+`TestDeclaredKernelsAreWiredInTheReference` is the other half: every
+non-`Fast` entry in `backend.Inventory` must be non-nil in `refBase`. 853
+today. The `Fast` exclusion is the same one the older test's comment describes,
+for the same reason.
+
+**Also settled, and against a plan.** `docs/plans/…-simd-production.md`'s item
+1 proposes generating the `internal/ref` ops-table entry and the
+`kernel.Ops` struct field from the manifest, and justifies it as removing "the
+two easiest things to forget — one of which caused the `-tags purego` panic
+that `TestDispatchTableComplete` now guards". Half of that is wrong: a
+forgotten `kernel.Ops` FIELD cannot ship, because the generator emits code
+referencing it and the build fails; a forgotten `ref` WIRING shipped past
+fourteen packages, as above. The gate is the fix; codegen would be
+convenience, and it would move the numerical contract — which lives in the
+`kernel.Ops` field comments — into a manifest.
+
+## `make fuzz` fuzzed one of its three targets and exited 0
+
+Measured 2026-08-16, by sweeping the family for fuzz targets nothing runs.
+
+The recipe named two targets and the comment above it opened "Two fuzz
+targets". There are three, and only ONE of them was being fuzzed:
+
+| target | what happened |
+|---|---|
+| `FuzzKernelsAgainstReference` | fuzzed, correctly |
+| `FuzzDifferential` | named, and run in the ROOT package while the target lives in `internal/tests/arrays` |
+| `FuzzJSONMasksMatchesSeparateCalls` | never named; only its seed corpus ever ran, under `go test` |
+
+The middle row is the one worth the entry:
+
+```
+$ go test -run '^$' -fuzz FuzzDifferential -fuzztime 3s .
+PASS
+ok github.com/sebishogun/simd 0.002s
+EXIT=0
+```
+
+**`go test -fuzz X` in a package with no target called X exits 0.** It does not
+error, it does not warn, it prints `PASS`. So half of `make fuzz` had been
+reporting success for a step that fuzzed nothing, and the only tell was a
+duration of two milliseconds where sixty seconds were asked for.
+
+Targets are discovered per package now — `go test -list '^Fuzz'` asks the
+compiler, and a target can only be fuzzed in the package it was found in, so
+the wrong-package shape is unreachable rather than merely fixed. The recipe
+also fails outright if it ends up running nothing at all, because a discovery
+loop over an empty list is the same silent green one level up.
+
+**The same sweep, across the family.** Ten repositories. `simdlogs` already
+discovered its targets and its workflow says why: "A hand-maintained list is
+how a new target gets written and never run." `simdhttp` hand-listed three of
+four and is fixed (its own entries 15 and 16 — the missing one had been named
+in a verification document since v1.2.0 and never written). `simdcsv` (3
+targets) and `simdparquet` (13) have no fuzz recipe at all; `simdjson` names 11
+of 14. Those are recorded here and not yet fixed.
+
+**The shape.** A gate that cannot fail is this family's most persistent defect,
+and every instance so far has been a test or an assertion. This is the same
+thing at the build level: a Makefile recipe whose command succeeds while doing
+nothing, in the repository the others depend on.
+
+## 75. The archsimd small-n win depends on inlining, and a package cannot inline
+
+`docs/research/08-goexperiment-simd-small-n.md` measured `GOEXPERIMENT=simd`
+intrinsics against the portable loop at −19.2% (n=8), −26.8% (16) and −31.7%
+(32) instructions, and concluded: build it, elementwise first. The plan put the
+implementation in a new `internal/fastpath` package that every generated guard
+would call below its threshold.
+
+Built, bit-identity-gated, and measured. `perf stat -e instructions:u`,
+2,000,000 iterations, interleaved, three rounds at n=8 and two at the rest,
+disjoint ranges throughout:
+
+| | ref | archsimd | |
+|---|---|---|---|
+| f32 n=8 | 124.06M | 155.85M | **+25.6%** |
+| f32 n=9 | 136.12M | 288.08M | **+111.6%** |
+| f32 n=16 | 220.20M | 228.13M | **+3.6%** |
+| f32 n=17 | 232.14M | 360.57M | **+55.3%** |
+| f32 n=32 | 412.03M | 371.91M | −9.7% |
+| f64 n=8 | 124.17M | 228.07M | **+83.7%** |
+| f64 n=16 | 220.05M | 372.34M | **+69.2%** |
+
+It loses across the whole band the guards actually use, and wins only at f32
+n≥32 — above where any guard falls back.
+
+**The cause is inlining, and the research document had already named the risk
+without measuring it.** `-gcflags=-m`:
+
+```
+fastpath_ref.go:16:6: can inline AddFloat32 <- the one-line forward to ref
+fastpath_simd.go:38:36: inlining call to archsimd.LoadFloat32x8Slice
+ (no "can inline AddFloat32") <- the loop cannot
+```
+
+The portable fallback is a one-line forward, so the guard inlines it and pays
+nothing to reach it. The archsimd version is a loop, and Go does not inline a
+function containing one — so it is a real call, and at n=8 that call is most of
+the work. The record's reproducer was a direct inlined loop and said so: *"It is
+a direct call, not the guarded one … the end-to-end figure will be smaller than
+19–32%."* Smaller was the wrong word. It is negative.
+
+**The record's table measured only exact multiples of the vector width, and that
+hid the worst of it.** n=9 and n=17 — one full vector plus a one-element tail,
+which is the shape the guard band mostly sees — cost +111.6% and +55.3%. A table
+of 8/16/32 cannot show a tail cost.
+
+**float64 loses everywhere in the band**, and for a structural reason: 4 lanes
+means n=8 is two vectors and n=16 is four, so the call is amortised over half as
+much arithmetic as f32 at the same length. The research document measured f32
+only.
+
+Reverted. What the next attempt has to do differently is emit the intrinsics
+**inside the generated backend**, in the same package as the guard, where they
+can inline into it — not behind a package boundary. Until something does that,
+the measured answer for the guard band is that `internal/ref` is the faster
+fallback.
+
+The bit-identity gate that came with it did pass, at every length 0–40 over
+IEEE specials, on both lanes: elementwise archsimd IS bit-identical to `ref`,
+which was the other thing that had to be true. That half of the record stands.
+
+## 76. Forced vectorization: the bucket the pragma is for is the bucket the contract forbids
+
+The plan's item 5 proposed trying `#pragma clang loop vectorize(enable)` per
+(kernel, target) and keeping measured winners, over "346 refusals: ~60
+`scatter_*`, ~30 `reverse_*`, 114 float/`fast_*` — the plausible wins".
+Measured, before writing any pragma. Every number in that sentence is wrong,
+and the investigation ends somewhere else.
+
+**The inventory.** `make check-emission`, clang 22.1.8: **336** (kernel, tier)
+refusals over **132** distinct kernels, not 346.
+
+| tier | vsx | neon | sse2 | vx | sve2 | lasx | rvv | avx2 | avx512 |
+|---|---|---|---|---|---|---|---|---|---|
+| refusals | 81 | 80 | 56 | 36 | 33 | 23 | 14 | 10 | 3 |
+
+By operation, as (kernel, tier) pairs: `scatter` 65, `gather` 31, `reverse` 30,
+`tile` 10, `reversebits` 10 — the permute/scatter family is **162 of 336, 48%**,
+and each needs an instruction the target does not have. `gather` at 31 pairs is
+larger than `reverse` and the plan did not mention it.
+
+**There is no float bucket.** `fast_*` refusals: **zero**. Float-typed
+refusals: 59 of 336, and on the three amd64 tiers only **2** — both
+`f8e*_to_f32`, which are integer bit manipulation. The refused
+`prod`/`dot`/`sumsq*`/`sumsqdev`/`sumsqdiff` entries are all `i64`/`u64`:
+64-bit integer reductions, blocked by a missing multiply (SSE2 has no
+`pmullq`), not by anything a pragma reaches.
+
+**The pragma's own bucket is ten loops.** `-Rpass-analysis=loop-vectorize` over
+all of `csrc` at sse2, by reason:
+
+| reason | loops |
+|---|---|
+| instruction return type cannot be vectorized | 1929 |
+| loop control flow is not understood | 186 |
+| value that could not be identified as reduction is used outside the loop | 128 |
+| potentially faulting load | 58 |
+| store instruction cannot be vectorized | 47 |
+| **cannot prove it is safe to reorder floating-point operations** | **10** |
+
+Ten. And clang's own remark names the remedy: *"allow reordering by specifying
+'#pragma clang loop vectorize(enable)' before the loop or by providing the
+compiler option '-ffast-math'"*. The pragma there is not a cost-model override,
+it is **permission to reassociate** — which changes the answer. The numerical
+contract in `internal/kernel/kernel.go` is the reason `-ffp-contract=off` is in
+`commonFlags` at all, and the repository already has the correct home for a
+kernel that trades it: the `simd_fast_*` prefix. So the one bucket the pragma is
+designed for is the one bucket that cannot take it without becoming a different
+operation.
+
+The ten are `csrc/numeric.c:121,127,133,139` (the `CONVOLVE` accumulation, four
+instantiations), `csrc/numeric.c:399,404`, and `csrc/gemm.c:142` (`GEMM`, four).
+
+**And those same kernels ship scalar today.** `simd_convolve_f32`,
+`simd_correlate_f64` and their siblings are refused on **zero** tiers — they
+pass the emission gate — yet:
+
+| tier | simd_convolve_f32 | packed arithmetic | scalar arithmetic |
+|---|---|---|---|
+| sse2 | 119 instrs | **0** | 10 (`mulss`, `addss`) |
+| avx2 | 140 instrs | **0** | 10 (`vmulss`, `vaddss`) |
+| avx512 | 141 instrs | **0** | 10 |
+
+Its only vector instructions on sse2 are 2 `movups` and 2 `xorps` — a move and
+a register zero. `dispatch_tables_amd64.go` carries
+`Convolve: amd64.ConvolveFloat32SSE2` and `ConvolveFloat32AVX2`, so a caller on
+amd64 dispatches into an assembly kernel that multiplies one element at a time,
+paying the dispatch and threshold machinery for it.
+
+**The gate cannot see this, structurally.** `verify.vectorWidth` returns >0 for
+any amd64 instruction whose operands mention `%xmm`/`%ymm`/`%zmm` — and every
+scalar float instruction on x86-64 uses those registers. `vmulss %xmm0, %xmm1,
+%xmm2` counts as vectorized. So `RequireVector`, whose comment says it exists
+because "dispatching to it would run scalar code under a name promising
+otherwise", cannot report a scalar float kernel on amd64 at all. Its 69 amd64
+refusals are integer kernels, necessarily.
+
+**Counted properly, by the generator.** The first pass here was a shell scan
+over every symbol in each object, which included helpers that are not kernels
+and used a cruder mnemonic match; it reported 26 at sse2 and 8 at avx2. Those
+numbers are superseded. `verify.arithKind` now classifies each instruction as
+packed arithmetic, scalar arithmetic or neither — moves and the `xorps
+%xmm0,%xmm0` zeroing idiom are neither, which is the point — and `make
+check-emission` prints `SCALAR-ONLY` for any kernel that does arithmetic with
+none of it in lanes:
+
+| tier | instances |
+|---|---|
+| amd64/avx512 | 9 |
+| amd64/sse2 | 8 |
+| amd64/avx2 | 8 |
+| arm64/sve2 | 8 |
+| arm64/neon | 5 |
+
+**38 instances over 13 distinct kernels**: `polyeval`, `convolve`, `correlate`,
+`movavg`, `minr` (f32 and f64 each), `rolling_min_f64`, `rolling_max_f64`,
+`dtoa_f64`. It is **not an x86 quirk** — arm64 has 13 of the 38, and there the
+gate is blind for the same reason in a different spelling: `fmul s0, s1, s2` is
+scalar and `fmul v0.4s, v1.4s, v2.4s` is packed, and `vectorWidth` reads the
+register file, not the arrangement.
+
+The classifier runs on amd64 and arm64 only, and `Report.ArithKnown` says so:
+on the other four architectures a zero count means "not measured", and
+`ScalarOnly()` returns false there rather than reporting a finding it did not
+make.
+
+**What this leaves.** Not a pragma sweep. Two separate things:
+
+1. `vectorWidth` on amd64 should not count a scalar-form instruction (`*ss`,
+ `*sd`) as vector, and `RequireVector` should ask "does this kernel do
+ arithmetic, and is none of it packed" rather than "is any register wide".
+ A permute kernel does no arithmetic and must still pass.
+2. Fixing the gate DROPS the eight, falling back to `internal/ref`. Whether
+ that is faster than the scalar assembly is unmeasured: both are scalar, and
+ the assembly costs a call the guard would otherwise inline. That is the
+ benchmark this needs, and the machine was not quiet (load average 1.17, two
+ review agents building) when this was written, so no number is offered.
+
+The transferable part is the same one as entry 25: a gate that asks a proxy
+question answers the proxy question. "Contains an instruction that touches a
+vector register" is not "vectorized", and on the one architecture that runs
+natively here the two are never the same for float code.
+
+## 77. Blocking the outer loop is what vectorizes a reduction kernel, and it is bit-identical
+
+Entry 76 found `polyeval`, `convolve`, `correlate` and `movavg` shipping scalar
+on every tier while passing the emission gate, and left the question of what to
+do about it open. The answer is not a pragma and not a fallback: it is the loop
+nest.
+
+**LLVM's loop vectorizer only ever vectorizes the INNERMOST loop.** Every one of
+these kernels is an outer loop over independent outputs wrapped around an inner
+reduction with a runtime trip count:
+
+```c
+for (isize i = 0; i < n; i++) { // independent — but not innermost
+ T acc = c[nc - 1];
+ for (isize k = nc - 2; k >= 0; k--) // innermost — a serial dependency
+ acc = acc * xv + c[k];
+ d[i] = acc;
+}
+```
+
+So the vectorizer looked at the Horner recurrence, could not break it without
+reassociating, and gave up — and `polyeval`'s own comment claimed the opposite:
+"The outer loop over elements is independent, which is what vectorizes."
+
+**Blocking swaps which loop is innermost without touching any element's
+arithmetic.** A fixed-size accumulator array, elements in tiles of 16:
+
+```c
+for (; i + 16 <= n; i += 16) {
+ T acc[16];
+ for (b) acc[b] = c[nc - 1];
+ for (k) { T ck = c[k]; for (b) acc[b] = acc[b] * x[i + b] + ck; }
+ for (b) d[i + b] = acc[b];
+}
+```
+
+Every output still evaluates the same coefficients in the same order, so this is
+**bit-identical**, not close. That distinction is the whole point: reassociating
+is exactly what `#pragma clang loop vectorize(enable)` would have done and what
+the numerical contract forbids. Nothing is allocated — 16 doubles is 128 bytes
+against the 512-byte kernel frame budget — and the tail runs the original nest.
+
+Packed arithmetic instructions per symbol, from `go run ./simdgen -n -v -only
+csrc/numeric.c` in `tools/`, run twice: once against `git show
+c1d08c4:csrc/numeric.c` and once against the blocked file.
+
+| | before, every amd64 tier | sse2 | avx2 | avx512 |
+|---|---|---|---|---|
+| `simd_polyeval_f32` | 0 | 24 | 20 | 18 |
+| `simd_polyeval_f64` | 0 | 16 | 40 | 20 |
+| `simd_convolve_f32` | 0 | 8 | 20 | 12 |
+| `simd_convolve_f64` | 0 | 16 | 24 | 21 |
+| `simd_correlate_f32` | 0 | 8 | 20 | 20 |
+| `simd_correlate_f64` | 0 | 16 | 24 | 21 |
+| `simd_movavg_f32` | 0 | 16 | 20 | 10 |
+| `simd_movavg_f64` | 0 | 16 | 24 | 20 |
+
+The scalar count does **not** move: 10 at sse2 and 18 at avx2/avx512, identical
+before and after on all eight symbols. So those instructions are not a tail the
+blocking introduced — they were already there, and blocking added packed work
+beside them rather than converting scalar work into it. `movavg` at avx512 f32
+gets only 10 packed against 20 at avx2 because a 16-element f32 block is exactly
+one zmm.
+
+A first version of this entry gave an avx2 column that was wrong in all six rows
+it listed and called the scalar remainder a tail. Both came from reading one
+tier's numbers and generalising.
+
+`SCALAR-ONLY` went from **13 kernels to 5** (`dtoa_f64`, `minr_f32/f64`,
+`rolling_min_f64`, `rolling_max_f64`). And loong64 gained two kernels, 728 →
+730: `simd_movavg_f32` and `simd_movavg_f64` were refused there with "LLVM did
+not vectorize it for lasx" and now emit, so the total moved 6,931 → 6,933.
+
+**The bit-identity claim is tested, and the test was checked for being
+vacuous.** `internal/conformance` compares each tier against `ref` at eighteen
+length pairs (`lenPairs`, `internal/conformance/numeric_test.go:23`), eight of
+which have a destination at or above the guard's threshold of 16 — so
+the SIMD path is genuinely reached. Poisoning the blocked path only
+(`acc[b] = acc[b]*x[i+b] + ck + 1`) reddens it at `dst=16 x=16 coeffs=2 i=0`,
+the first blocked case, which is what says the comparison sees the new code
+rather than the fallback.
+
+**Not measured: whether it is faster.** It executes fewer instructions per
+element and the arithmetic is in lanes, but the machine was not quiet (load
+average 2.27, two review agents building) and no wall-clock or counter number
+is offered. What is established is that the kernels now do their arithmetic in
+lanes and produce the same bits, which is the precondition for the speed
+question rather than an answer to it.
+
+**A review sweep put the bit-identity claim far past what those eighteen pairs
+reach.** Every `nd` in 0..200 crossed with 22 other lengths and with `nd±1` and
+`nd+15/16/17`; `nc`/`nk`/`w` in `{-1,0,1,2,3,4,15,16,17,31,33}`; seven value
+generators per precision including catastrophic cancellation (`1e16, 1, -1e16,
+1`), mid-accumulation overflow (`MaxFloat, MaxFloat, -MaxFloat`, where
+left-to-right gives +Inf and any other grouping gives a finite number),
+magnitudes switching at every 16-element block boundary, ±0 and subnormals, and
+NaN payloads. **3,286,542 cases per tier, 3,027,710 of them reaching the kernel
+rather than `ref`, on all nine tiers — sse2/avx2/avx512 native and
+neon/sve2/rvv/lasx/vsx/vx under qemu-user. Zero differing bits.** Perturbing
+`ref` by +1 only at output index ≥ 16 reddens every comparison site
+(197k–291k mismatches per operation), so the sweep is not vacuous. A separate
+`PROT_NONE` guard-page probe placing dst, signal and kernel each so its last
+element ends on the last accessible byte — ~88k kernel calls per tier — faulted
+nowhere, and the mechanism was checked by reading one element past the page.
+
+**arm64 pays for this, and the cost is not measured.** LLVM did not vectorize
+the block loop there; it unrolled over `b` and vectorized over `j` instead, so
+each accumulator gets a packed multiply of 4 taps and then an in-order scalar
+horizontal reduction. That needs 16 live scalar accumulators plus vector
+temporaries, and the tap loop of `simd_correlate_f32` at neon carries 18 spill
+stores and 18 reloads per iteration:
+
+| tier | kernel | stack | scalar float arith |
+|---|---|---|---|
+| neon | `simd_correlate_f32` | 0B → **320B** | 10 → **82** |
+| neon | `simd_convolve_f32` | 0B → **272B** | 10 → **82** |
+| neon | `simd_convolve_f64` | 0B → 80B | 6 → **70** |
+| neon | `simd_correlate_f64` | 0B → 80B | 6 → **70** |
+| sve2 | `simd_movavg_f64` | 0B → 144B | 2 → **34** |
+
+All are inside the 512-byte frame budget so nothing is dropped, and the results
+are bit-identical — the reduction stays strictly in order. But six arm64 kernels
+went from a zero frame to 80–320 bytes with 7–13x the scalar float arithmetic,
+and this repository has no arm64 hardware lane. Whether the packed tap multiply
+pays for the spill traffic is unknown on the only architecture where the
+question is open.
+
+`movavg` also rules out the obvious alternative: a sliding window (subtract the
+element leaving, add the one arriving) is O(n) instead of O(n·w) and gives a
+DIFFERENT number, because floating-point addition does not cancel.
+
+## 78. Generating the ref/kernel wiring is now maintainability, not correctness
+
+The plan's extensibility item proposed generating two of the eight files a new
+kernel touches -- `internal/ref/ref.go`'s ops-table entry and
+`internal/kernel/kernel.go`'s struct field -- on the argument that they are
+"the two easiest things to forget, one of which caused the `-tags purego` panic
+that `TestDispatchTableComplete` now guards".
+
+Both halves of that failure are now caught by a gate, in both lanes. Measured,
+`Sqrt: sqrt[T]` deleted from `floatOps`:
+
+ normal purego
+ before TestDeclaredKernelsAreWiredInTheReference green red
+ (and red only through TestInPlaceMatchesInto,
+ a functional test, not a completeness gate)
+ after RED RED
+
+`TestDeclaredKernelsAreWiredInTheReference` walks `backend.Inventory` and, for
+every declared kernel, asserts `kernel.Set` has the group AND the field AND
+that the field is non-nil in `ref.Set()`. That is exactly the three-way
+relation generation was proposed to enforce: manifest -> struct field ->
+reference wiring. A forgotten entry cannot ship past the ordinary `go test`
+lane any more.
+
+So the item is not dropped, it is REPRICED. What generation still buys is two
+fewer files to edit per kernel and one less place for a rename to go stale;
+what it no longer buys is protection against a class of defect that has
+already shipped once. That matters for ordering: it is a refactor of a stable
+v1 contract across six architectures, needing `make codegen`, `make verify` and
+the whole cross-architecture matrix to land safely, and its remaining value is
+ergonomic. It goes behind anything that fixes an answer.
+
+The mutation is recorded here rather than left implicit because the plan's own
+justification for the item no longer holds, and the next person to read that
+plan would otherwise re-derive the priority from a sentence that was true when
+it was written.
diff --git a/internal/amd64/numeric_avx2_amd64.s b/internal/amd64/numeric_avx2_amd64.s
index 3433906..dbf1ffc 100644
--- a/internal/amd64/numeric_avx2_amd64.s
+++ b/internal/amd64/numeric_avx2_amd64.s
@@ -195,24 +195,102 @@ TEXT ·polyEvalFloat32AVX2(SB), NOSPLIT|NOFRAME, $0-72
MOVQ x_len+32(FP), R8
MOVQ coeffs_len+56(FP), R9
QUAD $0x4dc14c0f4cc1394c
- QUAD $0x000000f7840fc985
- QUAD $0x0002338e0fc0854d
- QUAD $0xfc8a44107aa1c400
- QUAD $0x0f02e88348c8894c
- QUAD $0x894855000000f08c
- QUAD $0x8d41f8e4834853e5
- QUAD $0xe1834107e183ff49
- QUAD $0x2e661cebd2314507
- QUAD $0x0000000000841f0f
- QUAD $0xff499714117aa1c4
- QUAD $0x00c8840fc2394dc2
- QUAD $0x960c107aa1c40000
- QUAD $0x41d028f8c5c38949
- QUAD $0xcb89481d7401f983
- QUAD $0x90d028f8c5c38949
+ QUAD $0xe58948551a74c985
+ QUAD $0xe483485356415741
+ QUAD $0x31217d10f88349f8
+ QUAD $0x854d00000269e9c0
+ QUAD $0x49000004938e0fc0
+ QUAD $0x000159830f04f883
+ QUAD $0x00000473e9c03100
+ QUAD $0x49fc8a44107aa1c4
+ QUAD $0x00011a8e0f01f983
+ QUAD $0x518d45fe498d4900
+ QUAD $0x8341c0187de2c4ff
+ QUAD $0xe38341cb894503e2
+ QUAD $0x3100000010be4103
+ QUAD $0x2e6666666628ebdb
+ QUAD $0x0000000000841f0f
+ QUAD $0x11fcc59f2411fcc5
+ QUAD $0x4810708d4c209f5c
+ QUAD $0xf48f0fc6394dc389
+ QUAD $0xfcc5f0894c000001
+ QUAD $0x9e5410fcc59e0c10
+ QUAD $0xe028fcc5ce894920
+ QUAD $0x01fb8341d828fcc5
+ QUAD $0xce8949d7894d3474
+ QUAD $0xd828fcc5e028fcc5
+ QUAD $0x0000000000841f0f
+ QUAD $0x187da2c4e159dcc5
+ QUAD $0xe4c5e458d4c5b22c
+ QUAD $0xff49db58d4c5da59
+ QUAD $0x8348e275cfff49ce
+ QUAD $0x04c683498c7203f9
+ QUAD $0x0000000000841f0f
+ QUAD $0x187da2c4e159dcc5
+ QUAD $0xc5e458d4c5f0b26c
+ QUAD $0xc5db58d4c5da59e4
+ QUAD $0x6c187da2c4e159dc
+ QUAD $0xe4c5e458d4c5ecb2
+ QUAD $0xdcc5db58d4c5da59
+ QUAD $0xb26c187da2c4e159
+ QUAD $0x59e4c5e458d4c5e8
+ QUAD $0x59dcc5db58d4c5da
+ QUAD $0xe4b26c187da2c4e1
+ QUAD $0xda59e4c5e458d4c5
+ QUAD $0xfcc68349db58d4c5
+ QUAD $0x15e99a7f03fe8349
+ QUAD $0x49f0408d49ffffff
+ QUAD $0xff4904eac149c289
+ QUAD $0x4807e183d18944c2
+ QUAD $0x10ba41287370f883
+ QUAD $0x00c8e9c031000000
+ QUAD $0xffffffe0b9480000
+ QUAD $0x20f883497fffffff
+ QUAD $0xc031000001ec830f
+ QUAD $0xe2834900000226e9
+ QUAD $0xc031c8187de2c4f8
+ QUAD $0x0000000000841f0f
+ QUAD $0x11fcc5870c11fcc5
+ QUAD $0x874c11fcc520874c
+ QUAD $0xc560874c11fcc540
+ QUAD $0x00000080878c11fc
+ QUAD $0x0000a0878c11fcc5
+ QUAD $0x00c0878c11fcc500
+ QUAD $0xe0878c11fcc50000
+ QUAD $0x878c11fcc5000000
+ QUAD $0x8c11fcc500000100
+ QUAD $0x11fcc50000012087
+ QUAD $0xfcc500000140878c
+ QUAD $0xc500000160878c11
+ QUAD $0x00000180878c11fc
+ QUAD $0x0001a0878c11fcc5
+ QUAD $0x01c0878c11fcc500
+ QUAD $0xe0878c11fcc50000
+ QUAD $0x4980e88348000001
+ QUAD $0xffff6f850ff8c283
+ QUAD $0x8d4c3974c98548ff
+ QUAD $0xc1c0187de2c41050
+ QUAD $0x2e666666666604e1
+ QUAD $0x0000000000841f0f
+ QUAD $0x11fcc5870411fcc5
+ QUAD $0x8349d0894c208744
+ QUAD $0xe875f0c1834810c2
+ QUAD $0x4cd0894cf0c28349
+ QUAD $0x028e0fc12948c189
+ QUAD $0x44107aa1c4000002
+ QUAD $0xea8349ca894dfc8a
+ QUAD $0x41000000c88c0f02
+ QUAD $0x834107e183ff498d
+ QUAD $0x00401f0f15eb07e1
+ QUAD $0xc0ff48871411fac5
+ QUAD $0x0001cc840fc0394c
+ QUAD $0x894d860c10fac500
+ QUAD $0xf98341d028f8c5d3
+ QUAD $0x894dcb89481f7401
+ QUAD $0x001f0fd028f8c5d3
QUAD $0x586aa1c4d259f2c5
QUAD $0xcbff48cbff499a14
- QUAD $0xb87207f88348ee75
+ QUAD $0xb87207fa8349ee75
QUAD $0x00401f0f08c38349
QUAD $0x586aa1c4d259f2c5
QUAD $0xc4d259f2c5e09a54
@@ -226,48 +304,51 @@ TEXT ·polyEvalFloat32AVX2(SB), NOSPLIT|NOFRAME, $0-72
QUAD $0x59f2c5c89a54586a
QUAD $0xc49a54586aa1c4d2
QUAD $0x07fb8349f8c38349
- QUAD $0x4dffffff49e99e7f
- QUAD $0x0000013c8e0fc085
- QUAD $0xc0311b7304f88349
- QUAD $0xf8834900000120e9
- QUAD $0x0109e9c031227304
- QUAD $0x5d5bf8658d480000
- QUAD $0xffffffffe0b948c3
- QUAD $0x7320f883497fffff
- QUAD $0xe0b9485eebc0311b
- QUAD $0x497fffffffffffff
- QUAD $0xe9c0317b7320f883
- QUAD $0x48c0894c000000af
- QUAD $0xc057f8c5d231c821
- QUAD $0x0000000000841f0f
+ QUAD $0x48ffffff49e99e7f
+ QUAD $0xc18948087304f983
+ QUAD $0xf983480000010be9
+ QUAD $0x00bce9d231787320
+ QUAD $0xc82148c0894c0000
+ QUAD $0x9066c057f8c5d231
QUAD $0x11fcc5970411fcc5
QUAD $0x974411fcc5209744
QUAD $0x4860974411fcc540
QUAD $0xe075d0394820c283
- QUAD $0x0000b3840fc03949
- QUAD $0x9a840f1cc0f64100
+ QUAD $0x0000f9840fc03949
+ QUAD $0xe0840f1cc0f64100
QUAD $0x8348c28948000000
QUAD $0xc0214cc889481cc1
QUAD $0x2e666666c057f8c5
QUAD $0x0000000000841f0f
QUAD $0xc28348970411f8c5
- QUAD $0x77ebf275d0394804
- QUAD $0xe2c4c82148c0894c
- QUAD $0x001f0fd231c8187d
- QUAD $0x11fcc5970c11fcc5
- QUAD $0x974c11fcc520974c
- QUAD $0x4860974c11fcc540
- QUAD $0xe075d0394820c283
- QUAD $0xc0f6414774c03949
- QUAD $0x8348c2894828741c
- QUAD $0xc0214cc889481cc1
- QUAD $0x001f0fc81879e2c4
+ QUAD $0xbae9f275d0394804
+ QUAD $0x8348ca8948000000
+ QUAD $0x48c8187de2c4e0e2
+ QUAD $0x4560c6834887348d
+ QUAD $0x0000441f0f66c931
+ QUAD $0xc4a08e4c117ca1c4
+ QUAD $0xa1c4c08e4c117ca1
+ QUAD $0x7ca1c4e08e4c117c
+ QUAD $0x4c20c183498e0c11
+ QUAD $0x74d13948dc75ca39
+ QUAD $0x89445d741cc1f654
+ QUAD $0x48f1294803e683c6
+ QUAD $0x48c81879e2c4c101
+ QUAD $0xe08348c0894cc201
+ QUAD $0x00000000801f0ffc
QUAD $0xc28348970c11f8c5
- QUAD $0x3949f275d0394804
- QUAD $0x870411fac51974c0
- QUAD $0x8704c7f1ebc0ff48
- QUAD $0x49c0ff4800000000
- QUAD $0xc377f8c5f175c039
+ QUAD $0x8548f275d0394804
+ QUAD $0x2e666666661a74f6
+ QUAD $0x0000000000841f0f
+ QUAD $0xc1ff488f0411fac5
+ QUAD $0x658d48f375c83949
+ QUAD $0xc55d5f415e415be8
+ QUAD $0x8948d00148c377f8
+ QUAD $0x00008704c7ddebc1
+ QUAD $0xc03949c0ff480000
+ LONG $0xf8c5f175
+ BYTE $0x77
+ BYTE $0xc3
RET
// func convolveFloat32AVX2(dst []float32, sig []float32, ker []float32)
@@ -280,97 +361,193 @@ TEXT ·convolveFloat32AVX2(SB), NOSPLIT|NOFRAME, $0-72
MOVQ ker_len+56(FP), R9
QUAD $0x56415741e5894855
QUAD $0xe483485354415541
- QUAD $0x74894838ec8348f8
- QUAD $0xc0940fc9854d0824
- QUAD $0x40c69c0f40c8294d
- QUAD $0x0000029f850fc608
- QUAD $0x48c1394c01708d49
- QUAD $0x8e0ff68548f14e0f
- QUAD $0x0fc9854d0000028b
- QUAD $0x0c8d4a000001f18e
- QUAD $0x07e08341c889458a
- QUAD $0xffffffffffe0ba49
- QUAD $0x214d18ca83497fff
- QUAD $0x8d4c0824448b48ca
- QUAD $0x83488a048d4a1c58
- QUAD $0x311024448948fcc0
- QUAD $0xfffffffffdbf49db
- QUAD $0xfffffcbc493fffff
- QUAD $0xfbbd493fffffffff
- QUAD $0x483fffffffffffff
- QUAD $0xfffffffffffffab8
- QUAD $0xfffffffff9ba483f
- QUAD $0xfffff8be493fffff
- QUAD $0x7c89483fffffffff
- QUAD $0x4c20247489482824
- QUAD $0x906634eb18244c89
- QUAD $0x5c8b4828247c8b48
- QUAD $0x489f0411fac53024
- QUAD $0x834904c38349c3ff
- QUAD $0x4808244c894c04c1
- QUAD $0x4cf339482024748b
- QUAD $0x01cd840f18244c8b
- QUAD $0xc530245c89480000
- QUAD $0x7308f98349c057f8
- QUAD $0x08244c8b4cdb310f
- QUAD $0x001f0f000000eae9
- QUAD $0x49db3110247c8b48
- QUAD $0xfffffffffffffeb9
- QUAD $0x2e6666666666663f
+ QUAD $0xc9854d38ec8348f8
+ QUAD $0x20244c894cc0940f
+ QUAD $0x41c19c0f41c8294d
+ QUAD $0x00000591850fc108
+ QUAD $0x48c1394c01408d49
+ QUAD $0x1024448948c14e0f
+ QUAD $0x247c894810f88348
+ QUAD $0x8948282474894830
+ QUAD $0xc93145087d182454
+ QUAD $0x4c8b48000002b3e9
+ QUAD $0x8e8e0fc985482024
+ QUAD $0x418a048d4c000001
+ QUAD $0xbb4903e28341ca89
+ QUAD $0x7ffffffffffffff8
+ QUAD $0x48cb214904cb8349
+ QUAD $0x488a048d482c5e8d
+ QUAD $0x0824448948fcc083
+ QUAD $0xf6314500000010ba
+ QUAD $0xfffffffffffebf49
+ QUAD $0xfffffffdbc493fff
+ QUAD $0xfffcbd493fffffff
+ QUAD $0x2feb3fffffffffff
+ QUAD $0x7ca1c430247c8b48
+ QUAD $0x4c117ca1c4b70411
+ QUAD $0xc28348d1894920b7
+ QUAD $0xc6834840c3834810
+ QUAD $0x24543b48ce894d40
+ QUAD $0xc5000002218f0f10
+ QUAD $0x7304f98348c057f8
+ QUAD $0xe9c957f0c5c03117
+ QUAD $0x2e666666000000b3
QUAD $0x0000000000841f0f
- QUAD $0xc4e49b4c107ac1c4
- QUAD $0xf2c5e89b54107ac1
- QUAD $0x8948c158fac50f59
- QUAD $0x0c59eac5ce314cde
- QUAD $0x7ac1c4c158fac5b1
- QUAD $0x4cde8948ec9b4c10
- QUAD $0xc5b10c59f2c5fe31
- QUAD $0x4c107ac1c4c158fa
- QUAD $0xe6314cde8948f09b
- QUAD $0x7ac1c4b10c59f2c5
- QUAD $0x4cde8948f49b5410
- QUAD $0xc5b11459eac5ee31
- QUAD $0xc4c258fac5c158fa
- QUAD $0x8948f89b4c107ac1
- QUAD $0x0c59f2c5c63148de
- QUAD $0x7ac1c4c158fac5b1
- QUAD $0x48de8948fc9b4c10
- QUAD $0xc5b10c59f2c5d631
- QUAD $0x0c107ac1c4c158fa
- QUAD $0xc5f6314cde89489b
- QUAD $0xc158fac5b10c59f2
- QUAD $0xe0c7834808c38348
- QUAD $0xffff47850fda3949
- QUAD $0x244c8b4cc0854dff
- QUAD $0x48fffffec9840f08
- QUAD $0x48000000009d348d
- QUAD $0x49f7294810247c8b
- QUAD $0x001f0ff631991c8d
- QUAD $0x59f2c5b30c10fac5
- QUAD $0xc6ff48c158fac50f
- QUAD $0x75f03949fcc78348
- QUAD $0x8348fffffe92e9e7
- QUAD $0x7bebc031047304fe
- QUAD $0xffffffffffe0b948
- QUAD $0x047320fe83487fff
- QUAD $0x48f089483febc031
- QUAD $0xc057f8c5d231c821
+ QUAD $0xc5c03108247c8b48
+ QUAD $0x0000441f0fc957f0
+ QUAD $0x59ecc517187de2c4
+ QUAD $0xc5c358fcc5d4835c
+ QUAD $0xc18948f4835459ec
+ QUAD $0x1c187dc2c4f9314c
+ QUAD $0xc5d8836459e4c588
+ QUAD $0xc5c458fcc5ca58f4
+ QUAD $0xc18948f8835459e4
+ QUAD $0x1c187dc2c4e1314c
+ QUAD $0xc5dc836459e4c588
+ QUAD $0xc5c458fcc5ca58f4
+ QUAD $0xc18948fc835459e4
+ QUAD $0x1c187dc2c4e9314c
+ QUAD $0xc5e0836459e4c588
+ QUAD $0xc5c458fcc5ca58f4
+ QUAD $0xca58f4c5831459e4
+ QUAD $0xf0c7834804c08348
+ QUAD $0xffff77850fc33949
+ QUAD $0x244c8b48d2854dff
+ QUAD $0x48ffffff09840f20
+ QUAD $0x4c02e0c148863c8d
+ QUAD $0x31c1294908244c8b
+ QUAD $0x00000000801f0fc0
+ QUAD $0x59ecc511187dc2c4
+ QUAD $0xecc5c358fcc5871c
+ QUAD $0xca58f4c520875459
+ QUAD $0x49fcc18349c0ff48
+ QUAD $0xfffec7e9dc75c239
+ QUAD $0x8d481024448b48ff
+ QUAD $0xe8c149c88949f048
+ QUAD $0x83c08944c0ff4904
+ QUAD $0x0e7370f9834807e0
+ QUAD $0x314500000010b841
+ QUAD $0x8349000000c4e9c9
+ QUAD $0x57f8c5c93145f8e0
+ QUAD $0x2e666666666666c0
QUAD $0x0000000000841f0f
- QUAD $0x11fcc5970411fcc5
- QUAD $0x974411fcc5209744
- QUAD $0x4860974411fcc540
- QUAD $0xe075d0394820c283
- QUAD $0xc6f6403a74c63948
- QUAD $0x8348c2894828741c
- QUAD $0xf02148c889481cc1
- QUAD $0x00401f0fc057f8c5
+ QUAD $0xa1c48f04117ca1c4
+ QUAD $0x7ca1c4208f44117c
+ QUAD $0x117ca1c4408f4411
+ QUAD $0x84117ca1c4608f44
+ QUAD $0x7ca1c4000000808f
+ QUAD $0xc4000000a08f8411
+ QUAD $0x0000c08f84117ca1
+ QUAD $0xe08f84117ca1c400
+ QUAD $0x84117ca1c4000000
+ QUAD $0x7ca1c4000001008f
+ QUAD $0xc4000001208f8411
+ QUAD $0x0001408f84117ca1
+ QUAD $0x608f84117ca1c400
+ QUAD $0x84117ca1c4000001
+ QUAD $0x7ca1c4000001808f
+ QUAD $0xc4000001a08f8411
+ QUAD $0x0001c08f84117ca1
+ QUAD $0xe08f84117ca1c400
+ QUAD $0x4980e98349000001
+ QUAD $0xffff5f850ff8c083
+ QUAD $0x8d4d2a74c08548ff
+ QUAD $0x9066c057f8c51041
+ QUAD $0xa1c48f04117ca1c4
+ QUAD $0xc1894d208f44117c
+ QUAD $0x75c8ff4810c08349
+ QUAD $0xc1894df0c08349e7
+ QUAD $0x4c894c1024448b48
+ QUAD $0x548b48c8294c0824
+ QUAD $0x0f1824748b482824
+ QUAD $0x4c8b48000002948e
+ QUAD $0xd38e0fc985482024
+ QUAD $0x418e048d4c000001
+ QUAD $0xbb4907e28341ca89
+ QUAD $0x7ffffffffffffff8
+ QUAD $0x0824448b48cb2149
+ QUAD $0x1cc38348821c8d48
+ QUAD $0x4c8948fc8e4c8d48
+ QUAD $0x894882048d481824
+ QUAD $0xfffffebe49282444
+ QUAD $0xfdbf493fffffffff
+ QUAD $0x493fffffffffffff
+ QUAD $0xfffffffffffffcbc
+ QUAD $0xfffffffffbbd493f
+ QUAD $0xfffffab8483fffff
+ QUAD $0xf9ba483fffffffff
+ QUAD $0x493fffffffffffff
+ QUAD $0xfffffffffffff8b9
+ QUAD $0x0000441f0f31eb3f
+ QUAD $0x748b4830244c8b48
+ QUAD $0x48b10411fac50824
+ QUAD $0x834804c38348c6ff
+ QUAD $0x2474894804282444
+ QUAD $0x840f1024743b4808
+ QUAD $0x247c8348000001d5
+ QUAD $0xc057f8c50c730820
+ QUAD $0x90000000d2e9f631
+ QUAD $0x247c8b48c057f8c5
+ QUAD $0x0000441f0ff63118
+ QUAD $0xfac5e4b34c10fac5
+ QUAD $0x0f59f2c5e8b35410
+ QUAD $0x4cf18948c158fac5
+ QUAD $0x880c596ac1c4f131
+ QUAD $0x4c10fac5c158fac5
+ QUAD $0xf9314cf18948ecb3
+ QUAD $0xfac5880c5972c1c4
+ QUAD $0xf0b34c10fac5c158
+ QUAD $0xc1c4e1314cf18948
+ QUAD $0x5410fac5880c5972
+ QUAD $0xe9314cf18948f4b3
+ QUAD $0xfac58814596ac1c4
+ QUAD $0xfac5c258fac5c158
+ QUAD $0x48f18948f8b34c10
+ QUAD $0x880c5972c1c4c131
+ QUAD $0x4c10fac5c158fac5
+ QUAD $0xd13148f18948fcb3
+ QUAD $0xfac5880c5972c1c4
+ QUAD $0x48b30c10fac5c158
+ QUAD $0x72c1c4c9314cf189
+ QUAD $0x48c158fac5880c59
+ QUAD $0x49e0c7834808c683
+ QUAD $0xffffff48850ff339
+ QUAD $0xfffeef840fd2854d
+ QUAD $0x000000b50c8d48ff
+ QUAD $0x294818247c8b4800
+ QUAD $0x8d4828244c8b48cf
+ QUAD $0x00401f0fc931b134
+ QUAD $0x59f2c58e0c10fac5
+ QUAD $0xc1ff48c158fac50f
+ QUAD $0x75ca3949fcc78348
+ QUAD $0x8348fffffeb2e9e7
+ QUAD $0x730824448b4c04f8
+ QUAD $0x00008fe9c0894c08
+ QUAD $0x31047320f8834800
+ QUAD $0x8348c2894843ebd2
+ QUAD $0x8348870c8d4ae0e2
+ QUAD $0xc057f8c5f63160c1
+ QUAD $0xfcc5a0b14411fcc5
+ QUAD $0x4411fcc5c0b14411
+ QUAD $0x48b10411fcc5e0b1
+ QUAD $0xe075f2394820c683
+ QUAD $0x741ca85c74d03948
+ QUAD $0x8b480824448b4c6a
+ QUAD $0x03e683ce8910244c
+ QUAD $0x014cc0014cf02948
+ QUAD $0x57f8c5fce18348c2
+ QUAD $0x00000000801f0fc0
QUAD $0xc28348970411f8c5
- QUAD $0x3948f275d0394804
- QUAD $0x00008704c70c74c6
- QUAD $0x48efebc0ff480000
- QUAD $0x5d415c415bd8658d
- QUAD $0x77f8c55d5f415e41
- BYTE $0xc3
+ QUAD $0x8548f275d1394804
+ QUAD $0x2e666666661e74f6
+ QUAD $0x0000000000841f0f
+ QUAD $0x48000000008704c7
+ QUAD $0x751024443948c0ff
+ QUAD $0x5c415bd8658d48ef
+ QUAD $0xc55d5f415e415d41
+ QUAD $0x0824448b48c377f8
+ LONG $0xebd00148
+ BYTE $0xd3
RET
// func correlateFloat32AVX2(dst []float32, sig []float32, ker []float32)
@@ -383,71 +560,149 @@ TEXT ·correlateFloat32AVX2(SB), NOSPLIT|NOFRAME, $0-72
MOVQ ker_len+56(FP), R9
QUAD $0x294dc0940fc9854d
QUAD $0xc20841c29c0f41c8
- QUAD $0x8d49000001e9850f
- QUAD $0x4e0f48c1394c0140
- QUAD $0x01d58e0fc08548c1
- QUAD $0x358e0fc9854d0000
- QUAD $0x41e5894855000001
- QUAD $0xf8e4834853564157
- QUAD $0xb84907e183c98944
- QUAD $0x7fffffffffffffe0
- QUAD $0x4cc8214d18c88349
- QUAD $0x1aebdb31451c568d
- QUAD $0xff499f04117aa1c4
- QUAD $0xc6834804c28349c3
- QUAD $0x00e3840fc3394904
- QUAD $0xdb31c057f8c50000
- QUAD $0x00a2820f08f98349
- QUAD $0x0000441f0f660000
- QUAD $0xc4e49a4c107ac1c4
- QUAD $0xf2c5e89a54107ac1
+ QUAD $0x485500000458850f
+ QUAD $0x554156415741e589
+ QUAD $0x49f8e48348535441
+ QUAD $0x0f48c1394c01408d
+ QUAD $0x077d10f88348c14e
+ QUAD $0x4d0000022fe9c931
+ QUAD $0x000001308e0fc985
+ QUAD $0x4903e08341c88945
+ QUAD $0xfffffffffffff8ba
+ QUAD $0xca214d04ca83497f
+ QUAD $0x0010bf412c5e8d4c
+ QUAD $0xf38948f631450000
+ QUAD $0x2e666666666633eb
+ QUAD $0x0000000000841f0f
+ QUAD $0xa1c4b704117ca1c4
+ QUAD $0x798d4c20b74c117c
+ QUAD $0xc3834840c3834910
+ QUAD $0x0fc73949ce894940
+ QUAD $0xf9894c000001c98f
+ QUAD $0xc5ff3145c057f8c5
+ QUAD $0x0f04f98349c957f0
+ QUAD $0x001f0f0000008782
+ QUAD $0x81c4ba14187da2c4
+ QUAD $0x58fcc5d4bb5c596c
+ QUAD $0xf4bb54596c81c4c3
+ QUAD $0x187da2c4ca58f4c5
+ QUAD $0x5c596c81c404ba54
+ QUAD $0x81c4c358fcc5d8bb
+ QUAD $0x58f4c5f8bb54596c
+ QUAD $0x08ba54187da2c4ca
+ QUAD $0xc5dcbb5c596c81c4
+ QUAD $0x54596c81c4c358fc
+ QUAD $0xa2c4ca58f4c5fcbb
+ QUAD $0x6c81c40cba54187d
+ QUAD $0x596c81c4e0bb5c59
+ QUAD $0xf4c5c358fcc5bb14
+ QUAD $0x394d04c78349ca58
+ QUAD $0x840fc0854d8575fa
+ QUAD $0xbb248d4effffff3c
+ QUAD $0x90ed3145ba3c8d4e
+ QUAD $0x81c4af14187d82c4
+ QUAD $0x596c81c4ac1c596c
+ QUAD $0xc5c358fcc520ac54
+ QUAD $0x394dc5ff49ca58f4
+ QUAD $0xffffff08e9dd75e8
+ QUAD $0x49ca8949f0488d48
+ QUAD $0x8945c2ff4904eac1
+ QUAD $0xf9834807e08341d0
+ QUAD $0x000010ba410d7370
+ QUAD $0x000000aae9c93100
+ QUAD $0xf8c5c931f8e28349
+ QUAD $0x0000441f0f66c057
+ QUAD $0x11fcc58f0411fcc5
+ QUAD $0x8f4411fcc5208f44
+ QUAD $0xc5608f4411fcc540
+ QUAD $0x000000808f8411fc
+ QUAD $0x0000a08f8411fcc5
+ QUAD $0x00c08f8411fcc500
+ QUAD $0xe08f8411fcc50000
+ QUAD $0x8f8411fcc5000000
+ QUAD $0x8411fcc500000100
+ QUAD $0x11fcc5000001208f
+ QUAD $0xfcc5000001408f84
+ QUAD $0xc5000001608f8411
+ QUAD $0x000001808f8411fc
+ QUAD $0x0001a08f8411fcc5
+ QUAD $0x01c08f8411fcc500
+ QUAD $0xe08f8411fcc50000
+ QUAD $0x4980e98348000001
+ QUAD $0xffff6f850ff8c283
+ QUAD $0x8d4c2874c0854dff
+ QUAD $0x9066c057f8c51051
+ QUAD $0x11fcc58f0411fcc5
+ QUAD $0x8349d1894c208f44
+ QUAD $0x49e975c8ff4910c2
+ QUAD $0x8949d1894cf0c283
+ QUAD $0x01e68e0fc82949c0
+ QUAD $0x2a8e0fc9854d0000
+ QUAD $0x8341c88945000001
+ QUAD $0xfffffff8ba4907e0
+ QUAD $0x4cca214d7fffffff
+ QUAD $0x481cc383498e1c8d
+ QUAD $0x2e666624eb8e348d
+ QUAD $0x0000000000841f0f
+ QUAD $0xc1ff488f0411fac5
+ QUAD $0x04c6834804c38349
+ QUAD $0x000197840fc13948
+ QUAD $0x49db31c057f8c500
+ QUAD $0x0000a3820f08f983
+ QUAD $0x00000000801f0f00
+ QUAD $0xc4e49b4c107ac1c4
+ QUAD $0xf2c5e89b54107ac1
QUAD $0x9a5459eac59a0c59
QUAD $0x7ac1c4c158fac504
- QUAD $0x4c59f2c5ec9a4c10
+ QUAD $0x4c59f2c5ec9b4c10
QUAD $0xc1c4c258fac5089a
- QUAD $0x59eac5f09a54107a
+ QUAD $0x59eac5f09b54107a
QUAD $0xc4c158fac50c9a54
- QUAD $0xf2c5f49a4c107ac1
+ QUAD $0xf2c5f49b4c107ac1
QUAD $0xc258fac5109a4c59
- QUAD $0xc5f89a54107ac1c4
+ QUAD $0xc5f89b54107ac1c4
QUAD $0x58fac5149a5459ea
- QUAD $0xfc9a4c107ac1c4c1
+ QUAD $0xfc9b4c107ac1c4c1
QUAD $0xfac5189a4c59f2c5
- QUAD $0x9a14107ac1c4c258
+ QUAD $0x9b14107ac1c4c258
QUAD $0xfac51c9a5459eac5
QUAD $0x8348c258fac5c158
- QUAD $0x6d850fd8394908c3
- QUAD $0x840fc98548ffffff
+ QUAD $0x6d850fda394908c3
+ QUAD $0x840fc0854dffffff
QUAD $0x9a348d4cffffff34
QUAD $0x66ff31459e1c8d48
QUAD $0x0000000000841f0f
QUAD $0x81c4bb0c107aa1c4
QUAD $0xc158fac5be0c5972
- QUAD $0xe875f9394cc7ff49
- QUAD $0x658d48ffffff03e9
- QUAD $0xc35d5f415e415be8
- QUAD $0xc931047304f88348
- QUAD $0xffffffe0ba487eeb
- QUAD $0x20f883487fffffff
- QUAD $0x894843ebc9310473
- QUAD $0xf8c5f631d12148c1
- QUAD $0x2e6666666666c057
- QUAD $0x0000000000841f0f
- QUAD $0x11fcc5b70411fcc5
- QUAD $0xb74411fcc520b744
- QUAD $0x4860b74411fcc540
- QUAD $0xe075f1394820c683
- QUAD $0x741ca83a74c83948
- QUAD $0x1cc28348ce894827
- QUAD $0xf8c5c12148d18948
- QUAD $0x0000441f0f66c057
- QUAD $0xc68348b70411f8c5
- QUAD $0x0aebf275f1394804
- QUAD $0x48000000008f04c7
- QUAD $0xc5f175c83948c1ff
- BYTE $0xf8
- BYTE $0x77
- BYTE $0xc3
+ QUAD $0xe875f8394dc7ff49
+ QUAD $0xf88349ffffff03e9
+ QUAD $0x95e9c88949087304
+ QUAD $0x7320f88349000000
+ QUAD $0xc2894c4aebd23104
+ QUAD $0x8f348d48e0e28348
+ QUAD $0xc5c9314560c68348
+ QUAD $0x0000441f0fc057f8
+ QUAD $0xc4a08e44117ca1c4
+ QUAD $0xa1c4c08e44117ca1
+ QUAD $0x7ca1c4e08e44117c
+ QUAD $0x4c20c183498e0411
+ QUAD $0x74d03949dc75ca39
+ QUAD $0x8963741cc0f64157
+ QUAD $0x49f0294903e683c6
+ QUAD $0xc18948ca0148c801
+ QUAD $0xc057f8c5fce18348
+ QUAD $0x0000000000841f0f
+ QUAD $0xc28348970411f8c5
+ QUAD $0x8548f275d1394804
+ QUAD $0x2e666666661d74f6
+ QUAD $0x0000000000841f0f
+ QUAD $0x000000008704c742
+ QUAD $0xf075c0394cc0ff49
+ QUAD $0x415c415bd8658d48
+ QUAD $0xf8c55d5f415e415d
+ QUAD $0xc88949d10148c377
+ BYTE $0xeb
+ BYTE $0xd6
RET
// func normFloat64AVX2(a []float64) float64
@@ -639,24 +894,118 @@ TEXT ·polyEvalFloat64AVX2(SB), NOSPLIT|NOFRAME, $0-72
MOVQ x_len+32(FP), R8
MOVQ coeffs_len+56(FP), R9
QUAD $0x4dc14c0f4cc1394c
- QUAD $0x000000f7840fc985
- QUAD $0x0002348e0fc0854d
- QUAD $0xf8ca44107ba1c400
- QUAD $0x0f02e88348c8894c
- QUAD $0x894855000000f08c
- QUAD $0x8d41f8e4834853e5
- QUAD $0xe1834107e183ff49
- QUAD $0x2e661cebd2314507
- QUAD $0x0000000000841f0f
- QUAD $0xff49d714117ba1c4
- QUAD $0x00c8840fc2394dc2
- QUAD $0xd60c107ba1c40000
- QUAD $0x41d028f9c5c38949
- QUAD $0xcb89481d7401f983
- QUAD $0x90d028f9c5c38949
+ QUAD $0xe58948551a74c985
+ QUAD $0xe483485356415741
+ QUAD $0x31217d10f88349f8
+ QUAD $0x854d000002e5e9c0
+ QUAD $0x49000005148e0fc0
+ QUAD $0x0001d1830f04f883
+ QUAD $0x000004f3e9c03100
+ QUAD $0x49f8ca44107ba1c4
+ QUAD $0x0001928e0f01f983
+ QUAD $0x518d45fe498d4900
+ QUAD $0x8341c0197de2c4ff
+ QUAD $0xe38341cb894503e2
+ QUAD $0x3100000010be4103
+ QUAD $0x2e6666666634ebdb
+ QUAD $0x0000000000841f0f
+ QUAD $0x11fdc5df3411fdc5
+ QUAD $0xdf7c11fdc520df6c
+ QUAD $0x4c60df44117dc540
+ QUAD $0x394dc3894810708d
+ QUAD $0x4c000002648f0fc6
+ QUAD $0xc5de0c10fdc5f089
+ QUAD $0x10fdc520de5410fd
+ QUAD $0xde6410fdc540de5c
+ QUAD $0xf028fdc5ce894960
+ QUAD $0xf828fdc5e828fdc5
+ QUAD $0x01fb8341c0287dc5
+ QUAD $0xce8949d7894d4574
+ QUAD $0xe828fdc5f028fdc5
+ QUAD $0xc0287dc5f828fdc5
+ QUAD $0x197d22c4f159cdc5
+ QUAD $0xd5c5f658b5c5f20c
+ QUAD $0xc5c5ed58b5c5ea59
+ QUAD $0x3dc5ff58b5c5fb59
+ QUAD $0x49c0583541c4c459
+ QUAD $0x48d175cfff49ceff
+ QUAD $0xffff57820f03f983
+ QUAD $0x001f0f04c68349ff
+ QUAD $0x197d22c4f159cdc5
+ QUAD $0xc5f658b5c5e0f24c
+ QUAD $0xc5ed58b5c5ea59d5
+ QUAD $0xc5ff58b5c5fb59c5
+ QUAD $0xc0583541c4c4593d
+ QUAD $0x197d22c4f159cdc5
+ QUAD $0xc5f658b5c5d8f24c
+ QUAD $0xc5ed58b5c5ea59d5
+ QUAD $0xc5ff58b5c5fb59c5
+ QUAD $0xc0583541c4c4593d
+ QUAD $0x197d22c4f159cdc5
+ QUAD $0xc5f658b5c5d0f24c
+ QUAD $0xc5ed58b5c5ea59d5
+ QUAD $0xc5ff58b5c5fb59c5
+ QUAD $0xc0583541c4c4593d
+ QUAD $0x197d22c4f159cdc5
+ QUAD $0xc5f658b5c5c8f24c
+ QUAD $0xc5ed58b5c5ea59d5
+ QUAD $0xc5ff58b5c5fb59c5
+ QUAD $0xc0583541c4c4593d
+ QUAD $0x03fe8349fcc68349
+ QUAD $0x9de9ffffff528f0f
+ QUAD $0x49f0408d49fffffe
+ QUAD $0xff4904eac149c289
+ QUAD $0x4803e183d18944c2
+ QUAD $0x10ba41287330f883
+ QUAD $0x00c0e9c031000000
+ QUAD $0xfffffff0b9480000
+ QUAD $0x10f883497fffffff
+ QUAD $0xc031000001f4830f
+ QUAD $0xe283490000022ee9
+ QUAD $0xc031c8197de2c4fc
+ QUAD $0x11fdc5c70c11fdc5
+ QUAD $0xc74c11fdc520c74c
+ QUAD $0xc560c74c11fdc540
+ QUAD $0x00000080c78c11fd
+ QUAD $0x0000a0c78c11fdc5
+ QUAD $0x00c0c78c11fdc500
+ QUAD $0xe0c78c11fdc50000
+ QUAD $0xc78c11fdc5000000
+ QUAD $0x8c11fdc500000100
+ QUAD $0x11fdc500000120c7
+ QUAD $0xfdc500000140c78c
+ QUAD $0xc500000160c78c11
+ QUAD $0x00000180c78c11fd
+ QUAD $0x0001a0c78c11fdc5
+ QUAD $0x01c0c78c11fdc500
+ QUAD $0xe0c78c11fdc50000
+ QUAD $0x4940c08348000001
+ QUAD $0xffff6f850ffcc283
+ QUAD $0x8d4c4574c98548ff
+ QUAD $0xc1c0197de2c41050
+ QUAD $0x2e666666666604e1
+ QUAD $0x0000000000841f0f
+ QUAD $0x11fdc5c70411fdc5
+ QUAD $0xc74411fdc520c744
+ QUAD $0x4c60c74411fdc540
+ QUAD $0x834810c28349d089
+ QUAD $0xf0c28349dc75f0c1
+ QUAD $0x2948c1894cd0894c
+ QUAD $0xc4000002068e0fc1
+ QUAD $0x894df8ca44107ba1
+ QUAD $0xcc8c0f02ea8349ca
+ QUAD $0x83ff498d41000000
+ QUAD $0x19eb07e1834107e1
+ QUAD $0x0000000000841f0f
+ QUAD $0xc0ff48c71411fbc5
+ QUAD $0x0001cc840fc0394c
+ QUAD $0x894dc60c10fbc500
+ QUAD $0xf98341d028f9c5d3
+ QUAD $0x894dcb89481f7401
+ QUAD $0x001f0fd028f9c5d3
QUAD $0x586ba1c4d259f3c5
QUAD $0xcbff48cbff49da14
- QUAD $0xb87207f88348ee75
+ QUAD $0xb87207fa8349ee75
QUAD $0x00401f0f08c38349
QUAD $0x586ba1c4d259f3c5
QUAD $0xc4d259f3c5c0da54
@@ -670,48 +1019,51 @@ TEXT ·polyEvalFloat64AVX2(SB), NOSPLIT|NOFRAME, $0-72
QUAD $0x59f3c590da54586b
QUAD $0x88da54586ba1c4d2
QUAD $0x07fb8349f8c38349
- QUAD $0x4dffffff49e99e7f
- QUAD $0x0000013d8e0fc085
- QUAD $0xc0311b7304f88349
- QUAD $0xf8834900000120e9
- QUAD $0x0109e9c031227304
- QUAD $0x5d5bf8658d480000
- QUAD $0xfffffffff0b948c3
- QUAD $0x7310f883497fffff
- QUAD $0xf0b9485eebc0311b
- QUAD $0x497fffffffffffff
- QUAD $0xe9c0317b7310f883
- QUAD $0x48c0894c000000af
- QUAD $0xc057f9c5d231c821
- QUAD $0x0000000000841f0f
- QUAD $0x11fdc5d70411fdc5
- QUAD $0xd74411fdc520d744
- QUAD $0x4860d74411fdc540
+ QUAD $0x48ffffff49e99e7f
+ QUAD $0xc18948087304f983
+ QUAD $0xf983480000010be9
+ QUAD $0x00bce9d231787310
+ QUAD $0xc82148c0894c0000
+ QUAD $0x9066c057f8c5d231
+ QUAD $0x11fcc5d70411fcc5
+ QUAD $0xd74411fcc520d744
+ QUAD $0x4860d74411fcc540
QUAD $0xe075d0394810c283
- QUAD $0x0000b4840fc03949
- QUAD $0x9a840f0cc0f64100
+ QUAD $0x0000fa840fc03949
+ QUAD $0xe0840f0cc0f64100
QUAD $0x8348c28948000000
QUAD $0xc0214cc889480cc1
- QUAD $0x2e666666c057f9c5
+ QUAD $0x2e666666c057f8c5
QUAD $0x0000000000841f0f
- QUAD $0xc28348d70411fdc5
- QUAD $0x78ebf275d0394804
- QUAD $0xe2c4c82148c0894c
- QUAD $0x001f0fd231c8197d
- QUAD $0x11fdc5d70c11fdc5
- QUAD $0xd74c11fdc520d74c
- QUAD $0x4860d74c11fdc540
- QUAD $0xe075d0394810c283
- QUAD $0xc0f6414874c03949
- QUAD $0x8348c2894828740c
- QUAD $0xc0214cc889480cc1
- QUAD $0x001f0fc8197de2c4
+ QUAD $0xc28348d70411fcc5
+ QUAD $0xbbe9f275d0394804
+ QUAD $0xe683c68944000000
+ QUAD $0xc4f22948ca89480f
+ QUAD $0x49c1894dc8197de2
+ QUAD $0x9066c28949f0e183
+ QUAD $0xa1c4d70c117da1c4
+ QUAD $0x7da1c420d74c117d
+ QUAD $0x117da1c440d74c11
+ QUAD $0x4d10c2834960d74c
+ QUAD $0x74f68548dc75d139
+ QUAD $0x89445d7204fe8354
+ QUAD $0x48f1294803e683c6
+ QUAD $0x48c8197de2c4c101
+ QUAD $0xe08348c0894cc201
+ QUAD $0x00000000801f0ffc
QUAD $0xc28348d70c11fdc5
- QUAD $0x3949f275d0394804
- QUAD $0xc70411fbc51a74c0
- QUAD $0x04c748f1ebc0ff48
- QUAD $0xc0ff4800000000c7
- QUAD $0x77f8c5f075c03949
+ QUAD $0x8548f275d0394804
+ QUAD $0x2e666666661a74f6
+ QUAD $0x0000000000841f0f
+ QUAD $0xc1ff48cf0411fbc5
+ QUAD $0x658d48f375c83949
+ QUAD $0xc55d5f415e415be8
+ QUAD $0x8948d00148c377f8
+ QUAD $0x00c704c748ddebc1
+ QUAD $0x3949c0ff48000000
+ LONG $0xc5f075c0
+ BYTE $0xf8
+ BYTE $0x77
BYTE $0xc3
RET
@@ -725,98 +1077,214 @@ TEXT ·convolveFloat64AVX2(SB), NOSPLIT|NOFRAME, $0-72
MOVQ ker_len+56(FP), R9
QUAD $0x56415741e5894855
QUAD $0xe483485354415541
- QUAD $0x74894838ec8348f8
- QUAD $0xc0940fc9854d0824
- QUAD $0x40c69c0f40c8294d
- QUAD $0x000002a0850fc608
- QUAD $0x48c1394c01708d49
- QUAD $0x8e0ff68548f14e0f
- QUAD $0x0fc9854d0000028c
- QUAD $0x0c8d4a000001f18e
- QUAD $0x07e08341c88945ca
- QUAD $0xfffffffffff0ba49
- QUAD $0x214d08ca83497fff
- QUAD $0x8d4c0824448b48ca
- QUAD $0x8348ca048d4a3858
- QUAD $0x311024448948f8c0
- QUAD $0xfffffffffdbf49db
- QUAD $0xfffffcbc491fffff
- QUAD $0xfbbd491fffffffff
- QUAD $0x481fffffffffffff
- QUAD $0xfffffffffffffab8
- QUAD $0xfffffffff9ba481f
- QUAD $0xfffff8be491fffff
- QUAD $0x7c89481fffffffff
- QUAD $0x4c20247489482824
- QUAD $0x906634eb18244c89
- QUAD $0x5c8b4828247c8b48
- QUAD $0x48df0411fbc53024
- QUAD $0x834908c38349c3ff
- QUAD $0x4808244c894c08c1
- QUAD $0x4cf339482024748b
- QUAD $0x01ce840f18244c8b
- QUAD $0xc530245c89480000
- QUAD $0x7308f98349c057f9
- QUAD $0x08244c8b4cdb310f
- QUAD $0x001f0f000000eae9
- QUAD $0x49db3110247c8b48
- QUAD $0xfffffffffffffeb9
- QUAD $0x2e6666666666661f
+ QUAD $0x7c894838ec8348f8
+ QUAD $0xc0940fc9854d1024
+ QUAD $0xc8294d18244c894c
+ QUAD $0x0fc70840c79c0f40
+ QUAD $0x408d490000063d85
+ QUAD $0xc14e0f48c1394c01
+ QUAD $0xf883480824448948
+ QUAD $0x032ce9c931077d10
+ QUAD $0x4d18246c8b4c0000
+ QUAD $0x0f1024648b4ced85
+ QUAD $0x048d4e0000015f8e
+ QUAD $0xfffffffff8ba49ea
+ QUAD $0x4d06ca83497fffff
+ QUAD $0x8d4a685e8d4cea21
+ QUAD $0x10bff8c38348ea1c
+ QUAD $0xb848c93145000000
+ QUAD $0x1ffffffffffffffe
+ QUAD $0x001f0f0000007ee9
+ QUAD $0xc5c957f1c5f63145
+ QUAD $0x4adb57e1c5d257e9
+ QUAD $0xf749f7894dce3c8d
+ QUAD $0xc4f824197d82c4d7
+ QUAD $0x5da1c4f72c595da1
+ QUAD $0xc558fdc520f77459
+ QUAD $0x595da1c4ce58f5c5
+ QUAD $0xc4d558edc540f76c
+ QUAD $0xe5c560f764595da1
+ QUAD $0xcc04117d81c4dc58
+ QUAD $0xc420cc4c117d81c4
+ QUAD $0x81c440cc54117d81
+ QUAD $0x798d4860cc5c117d
+ QUAD $0xc9894980eb834910
+ QUAD $0x638f0f08247c3b48
+ QUAD $0xf9c5f98948000002
+ QUAD $0x840f01fd8349c057
+ QUAD $0x45df8948ffffff74
+ QUAD $0xe9c5c957f1c5f631
+ QUAD $0x9066db57e1c5d257
+ QUAD $0x5d81c427197de2c4
+ QUAD $0xc558fdc598f36c59
+ QUAD $0xc5b8f36c595d81c4
+ QUAD $0x6c595d81c4cd58f5
+ QUAD $0x81c4d558edc5d8f3
+ QUAD $0x58e5c5f8f364595d
+ QUAD $0xc4c73149f7894ddc
+ QUAD $0x5d81c4f824197d82
+ QUAD $0xc558fdc5a0f36c59
+ QUAD $0xc4c0f36c595d81c4
+ QUAD $0xf5c5e0f374595d81
+ QUAD $0x81c4d658edc5cd58
+ QUAD $0xdc58e5c5f324595d
+ QUAD $0xf0c7834802c68349
+ QUAD $0xc5f6418b75f2394d
+ QUAD $0xe9fffffef0850f01
+ QUAD $0x24448b48ffffff26
+ QUAD $0xcf8948f0488d4808
+ QUAD $0x89c7ff4804efc148
+ QUAD $0x70f9834807e083f8
+ QUAD $0x3100000010bf0c73
+ QUAD $0x83480000015ce9c9
+ QUAD $0xc057f9c5c931f8e7
QUAD $0x0000000000841f0f
- QUAD $0xc4c8db4c107bc1c4
- QUAD $0xf3c5d0db54107bc1
- QUAD $0x8948c158fbc50f59
- QUAD $0x0c59ebc5ce314cde
- QUAD $0x7bc1c4c158fbc5f1
- QUAD $0x4cde8948d8db4c10
- QUAD $0xc5f10c59f3c5fe31
- QUAD $0x4c107bc1c4c158fb
- QUAD $0xe6314cde8948e0db
- QUAD $0x7bc1c4f10c59f3c5
- QUAD $0x4cde8948e8db5410
- QUAD $0xc5f11459ebc5ee31
- QUAD $0xc4c258fbc5c158fb
- QUAD $0x8948f0db4c107bc1
- QUAD $0x0c59f3c5c63148de
- QUAD $0x7bc1c4c158fbc5f1
- QUAD $0x48de8948f8db4c10
- QUAD $0xc5f10c59f3c5d631
- QUAD $0x0c107bc1c4c158fb
- QUAD $0xc5f6314cde8948db
- QUAD $0xc158fbc5f10c59f3
- QUAD $0xc0c7834808c38348
- QUAD $0xffff47850fda3949
- QUAD $0x244c8b4cc0854dff
- QUAD $0x48fffffec9840f08
- QUAD $0x4800000000dd348d
- QUAD $0x49f7294810247c8b
- QUAD $0x001f0ff631d91c8d
- QUAD $0x59f3c5f30c10fbc5
- QUAD $0xc6ff48c158fbc50f
- QUAD $0x75f03949f8c78348
- QUAD $0x8348fffffe92e9e7
- QUAD $0x7bebc031047304fe
- QUAD $0xfffffffffff0b948
- QUAD $0x047310fe83487fff
- QUAD $0x48f089483febc031
- QUAD $0xc057f9c5d231c821
- QUAD $0x0000000000841f0f
- QUAD $0x11fdc5d70411fdc5
- QUAD $0xd74411fdc520d744
- QUAD $0x4860d74411fdc540
- QUAD $0xe075d0394810c283
- QUAD $0xc6f6403b74c63948
- QUAD $0x8348c2894828740c
- QUAD $0xf02148c889480cc1
- QUAD $0x00401f0fc057f9c5
- QUAD $0xc28348d70411fdc5
- QUAD $0x3948f275d0394804
- QUAD $0x00c704c7480d74c6
- QUAD $0xeeebc0ff48000000
- QUAD $0x415c415bd8658d48
- QUAD $0xf8c55d5f415e415d
- BYTE $0x77
- BYTE $0xc3
+ QUAD $0xc1c4cc04117dc1c4
+ QUAD $0x7dc1c420cc44117d
+ QUAD $0x117dc1c440cc4411
+ QUAD $0x84117dc1c460cc44
+ QUAD $0x7dc1c400000080cc
+ QUAD $0xc4000000a0cc8411
+ QUAD $0x0000c0cc84117dc1
+ QUAD $0xe0cc84117dc1c400
+ QUAD $0x84117dc1c4000000
+ QUAD $0x7dc1c400000100cc
+ QUAD $0xc400000120cc8411
+ QUAD $0x000140cc84117dc1
+ QUAD $0x60cc84117dc1c400
+ QUAD $0x84117dc1c4000001
+ QUAD $0x7dc1c400000180cc
+ QUAD $0xc4000001a0cc8411
+ QUAD $0x0001c0cc84117dc1
+ QUAD $0xe0cc84117dc1c400
+ QUAD $0x84117dc1c4000001
+ QUAD $0x7dc1c400000200cc
+ QUAD $0xc400000220cc8411
+ QUAD $0x000240cc84117dc1
+ QUAD $0x60cc84117dc1c400
+ QUAD $0x84117dc1c4000002
+ QUAD $0x7dc1c400000280cc
+ QUAD $0xc4000002a0cc8411
+ QUAD $0x0002c0cc84117dc1
+ QUAD $0xe0cc84117dc1c400
+ QUAD $0x84117dc1c4000002
+ QUAD $0x7dc1c400000300cc
+ QUAD $0xc400000320cc8411
+ QUAD $0x000340cc84117dc1
+ QUAD $0x60cc84117dc1c400
+ QUAD $0x84117dc1c4000003
+ QUAD $0x7dc1c400000380cc
+ QUAD $0xc4000003a0cc8411
+ QUAD $0x0003c0cc84117dc1
+ QUAD $0xe0cc84117dc1c400
+ QUAD $0x4880e98348000003
+ QUAD $0xfffebf850ff8c783
+ QUAD $0x8d483874c08548ff
+ QUAD $0x9066c057f9c51079
+ QUAD $0xc1c4cc04117dc1c4
+ QUAD $0x7dc1c420cc44117d
+ QUAD $0x117dc1c440cc4411
+ QUAD $0x8348f9894860cc44
+ QUAD $0x48d975c8ff4810c7
+ QUAD $0x8b48f98948f0c783
+ QUAD $0x8e0fc82948082444
+ QUAD $0x247c8b48000002e6
+ QUAD $0x02058e0fff854818
+ QUAD $0x8941fa048d4c0000
+ QUAD $0xf8bb4907e28341fa
+ QUAD $0x497fffffffffffff
+ QUAD $0x8348ce1c8d48fb21
+ QUAD $0x48f8fa448d4838c3
+ QUAD $0xce048d4820244489
+ QUAD $0xfdbf492824448948
+ QUAD $0x491fffffffffffff
+ QUAD $0xfffffffffffffcbc
+ QUAD $0xfffffffffbbd491f
+ QUAD $0xfffffab8481fffff
+ QUAD $0xf9ba481fffffffff
+ QUAD $0x491fffffffffffff
+ QUAD $0xfffffffffffff8b9
+ QUAD $0x0000441f0f27eb1f
+ QUAD $0x11fbc51024748b48
+ QUAD $0xc38348c1ff48ce04
+ QUAD $0x4808282444834808
+ QUAD $0x0240840f08244c3b
+ QUAD $0x0818247c83480000
+ QUAD $0xf631c057f9c51673
+ QUAD $0x2e666600000113e9
+ QUAD $0x0000000000841f0f
+ QUAD $0x57f9c530244c8948
+ QUAD $0xf63120247c8b48c0
+ QUAD $0x8948d18949c9894c
+ QUAD $0x4de5894de8894cc2
+ QUAD $0xfffffffebf49fc89
+ QUAD $0x00401f0f1fffffff
+ QUAD $0xfbc5c8f34c10fbc5
+ QUAD $0x0f59f3c5d0f35410
+ QUAD $0x4df68949c158fbc5
+ QUAD $0xf00c596b81c4fe31
+ QUAD $0x4c10fbc5c158fbc5
+ QUAD $0xe6314df68949d8f3
+ QUAD $0xfbc5f00c597381c4
+ QUAD $0xe0f34c10fbc5c158
+ QUAD $0x81c4ee314df68949
+ QUAD $0x5410fbc5f00c5973
+ QUAD $0xc63149f68949e8f3
+ QUAD $0xfbc5f014596b81c4
+ QUAD $0xfbc5c258fbc5c158
+ QUAD $0x49f68949f0f34c10
+ QUAD $0xf00c597381c4d631
+ QUAD $0x4c10fbc5c158fbc5
+ QUAD $0xce314df68949f8f3
+ QUAD $0xfbc5f00c597381c4
+ QUAD $0x49f30c10fbc5c158
+ QUAD $0x7381c4ce3149f689
+ QUAD $0x48c158fbc5f00c59
+ QUAD $0x49c0c7834808c683
+ QUAD $0xffffff48850ff339
+ QUAD $0x894de7894dd2854d
+ QUAD $0x4cd08948c58949ec
+ QUAD $0x4c8b48c98949ca89
+ QUAD $0xfffffeb8840f3024
+ QUAD $0x00000000f5348d4c
+ QUAD $0xf7294c20247c8b48
+ QUAD $0x348d492824748b4c
+ QUAD $0x2e666666f63145f6
+ QUAD $0x0000000000841f0f
+ QUAD $0xf3c5f60c107ba1c4
+ QUAD $0xff49c158fbc50f59
+ QUAD $0xf2394df8c78348c6
+ QUAD $0x48fffffe71e9e675
+ QUAD $0xc88948087304f883
+ QUAD $0xf88348000000a7e9
+ QUAD $0x047310244c8b4c10
+ QUAD $0x247c8b4857ebd231
+ QUAD $0x89480fe683fe8908
+ QUAD $0xf0e78348f22948c2
+ QUAD $0x66c88949c057f9c5
+ QUAD $0x0000000000841f0f
+ QUAD $0x81c4c104117d81c4
+ QUAD $0x7d81c420c144117d
+ QUAD $0x117d81c440c14411
+ QUAD $0x4c10c0834960c144
+ QUAD $0x74f68548dc75c739
+ QUAD $0x8b4c767204fe8369
+ QUAD $0x08247c8b4810244c
+ QUAD $0xf0294803e683fe89
+ QUAD $0x8948ca0148c80148
+ QUAD $0x57f9c5fce18348f9
+ QUAD $0x2e666666666666c0
+ QUAD $0x0000000000841f0f
+ QUAD $0x8348d104117dc1c4
+ QUAD $0x48f175d1394804c2
+ QUAD $0x244c8b481e74f685
+ QUAD $0x00000000801f0f10
+ QUAD $0x00000000c104c748
+ QUAD $0x0824443948c0ff48
+ QUAD $0x415bd8658d48ee75
+ QUAD $0x5d5f415e415d415c
+ QUAD $0x48d10148c377f8c5
+ LONG $0xc8ebc889
RET
// func correlateFloat64AVX2(dst []float64, sig []float64, ker []float64)
@@ -829,69 +1297,167 @@ TEXT ·correlateFloat64AVX2(SB), NOSPLIT|NOFRAME, $0-72
MOVQ ker_len+56(FP), R9
QUAD $0x294dc0940fc9854d
QUAD $0xc20841c29c0f41c8
- QUAD $0x8d49000001ea850f
- QUAD $0x4e0f48c1394c0140
- QUAD $0x01d68e0fc08548c1
- QUAD $0x358e0fc9854d0000
- QUAD $0x41e5894855000001
- QUAD $0xf8e4834853564157
- QUAD $0xb84907e183c98944
- QUAD $0x7ffffffffffffff0
- QUAD $0x4cc8214d08c88349
- QUAD $0x1aebdb314538568d
- QUAD $0xff49df04117ba1c4
- QUAD $0xc6834808c28349c3
- QUAD $0x00e3840fc3394908
- QUAD $0xdb31c057f9c50000
- QUAD $0x00a2820f08f98349
- QUAD $0x0000441f0f660000
- QUAD $0xc4c8da4c107bc1c4
- QUAD $0xf3c5d0da54107bc1
+ QUAD $0x4855000004e4850f
+ QUAD $0x485356415741e589
+ QUAD $0x4c01408d49f8e483
+ QUAD $0x8348c14e0f48c139
+ QUAD $0xbfe9c931077d10f8
+ QUAD $0x8e0fc9854d000002
+ QUAD $0xfff8b84900000128
+ QUAD $0x83497fffffffffff
+ QUAD $0x568d4cc8214d06c8
+ QUAD $0x314500000010bb68
+ QUAD $0x2e6666666671ebdb
+ QUAD $0x0000000000841f0f
+ QUAD $0x197de2c4de348d4e
+ QUAD $0xde2c595dc1c4da24
+ QUAD $0xc520de74595dc1c4
+ QUAD $0xc4ce58f5c5c558fd
+ QUAD $0xedc540de6c595dc1
+ QUAD $0xde64595dc1c4d558
+ QUAD $0x7da1c4dc58e5c560
+ QUAD $0x4c117da1c4df0411
+ QUAD $0xdf54117da1c420df
+ QUAD $0x60df5c117da1c440
+ QUAD $0x80ea834910598d48
+ QUAD $0x8f0fc33948cb8949
+ QUAD $0xc5d9894800000226
+ QUAD $0x57f1c5db31c057f9
+ QUAD $0x57e1c5d257e9c5c9
+ QUAD $0x90817401f98349db
+ QUAD $0xc1c4da24197de2c4
+ QUAD $0x58fdc598da6c595d
+ QUAD $0xb8da6c595dc1c4c5
+ QUAD $0xc5d8da74595dc1c4
+ QUAD $0xc4d658edc5cd58f5
+ QUAD $0xe5c5f8da64595dc1
+ QUAD $0xda64197de2c4dc58
+ QUAD $0xa0da6c595dc1c408
+ QUAD $0x595dc1c4c558fdc5
+ QUAD $0xc4cd58f5c5c0da6c
+ QUAD $0xc1c4e0da6c595dc1
+ QUAD $0xd558edc5da24595d
+ QUAD $0x02c38348dc58e5c5
+ QUAD $0xc1f6419375d83949
+ QUAD $0xe9ffffff09850f01
+ QUAD $0xf0488d48ffffff39
+ QUAD $0x4904eac149ca8949
+ QUAD $0xe08341d08945c2ff
+ QUAD $0x410d7370f9834807
+ QUAD $0xe9c93100000010ba
+ QUAD $0xf8e2834900000136
+ QUAD $0x9066c057f9c5c931
+ QUAD $0x11fdc5cf0411fdc5
+ QUAD $0xcf4411fdc520cf44
+ QUAD $0xc560cf4411fdc540
+ QUAD $0x00000080cf8411fd
+ QUAD $0x0000a0cf8411fdc5
+ QUAD $0x00c0cf8411fdc500
+ QUAD $0xe0cf8411fdc50000
+ QUAD $0xcf8411fdc5000000
+ QUAD $0x8411fdc500000100
+ QUAD $0x11fdc500000120cf
+ QUAD $0xfdc500000140cf84
+ QUAD $0xc500000160cf8411
+ QUAD $0x00000180cf8411fd
+ QUAD $0x0001a0cf8411fdc5
+ QUAD $0x01c0cf8411fdc500
+ QUAD $0xe0cf8411fdc50000
+ QUAD $0xcf8411fdc5000001
+ QUAD $0x8411fdc500000200
+ QUAD $0x11fdc500000220cf
+ QUAD $0xfdc500000240cf84
+ QUAD $0xc500000260cf8411
+ QUAD $0x00000280cf8411fd
+ QUAD $0x0002a0cf8411fdc5
+ QUAD $0x02c0cf8411fdc500
+ QUAD $0xe0cf8411fdc50000
+ QUAD $0xcf8411fdc5000002
+ QUAD $0x8411fdc500000300
+ QUAD $0x11fdc500000320cf
+ QUAD $0xfdc500000340cf84
+ QUAD $0xc500000360cf8411
+ QUAD $0x00000380cf8411fd
+ QUAD $0x0003a0cf8411fdc5
+ QUAD $0x03c0cf8411fdc500
+ QUAD $0xe0cf8411fdc50000
+ QUAD $0x4980e98348000003
+ QUAD $0xfffedf850ff8c283
+ QUAD $0x8d4c3474c0854dff
+ QUAD $0x9066c057f9c51051
+ QUAD $0x11fdc5cf0411fdc5
+ QUAD $0xcf4411fdc520cf44
+ QUAD $0x4c60cf4411fdc540
+ QUAD $0xff4910c28349d189
+ QUAD $0x4cf0c28349dd75c8
+ QUAD $0xc82949c08949d189
+ QUAD $0x854d000001ea8e0f
+ QUAD $0x450000012e8e0fc9
+ QUAD $0xba4907e08341c889
+ QUAD $0x7ffffffffffffff8
+ QUAD $0x49ce1c8d4cca214d
+ QUAD $0xebce348d4838c383
+ QUAD $0x2e66666666666628
+ QUAD $0x0000000000841f0f
+ QUAD $0xc1ff48cf0411fbc5
+ QUAD $0x08c6834808c38349
+ QUAD $0x000197840fc13948
+ QUAD $0x49db31c057f9c500
+ QUAD $0x0000a3820f08f983
+ QUAD $0x00000000801f0f00
+ QUAD $0xc4c8db4c107bc1c4
+ QUAD $0xf3c5d0db54107bc1
QUAD $0xda5459ebc5da0c59
QUAD $0x7bc1c4c158fbc508
- QUAD $0x4c59f3c5d8da4c10
+ QUAD $0x4c59f3c5d8db4c10
QUAD $0xc1c4c258fbc510da
- QUAD $0x59ebc5e0da54107b
+ QUAD $0x59ebc5e0db54107b
QUAD $0xc4c158fbc518da54
- QUAD $0xf3c5e8da4c107bc1
+ QUAD $0xf3c5e8db4c107bc1
QUAD $0xc258fbc520da4c59
- QUAD $0xc5f0da54107bc1c4
+ QUAD $0xc5f0db54107bc1c4
QUAD $0x58fbc528da5459eb
- QUAD $0xf8da4c107bc1c4c1
+ QUAD $0xf8db4c107bc1c4c1
QUAD $0xfbc530da4c59f3c5
- QUAD $0xda14107bc1c4c258
+ QUAD $0xdb14107bc1c4c258
QUAD $0xfbc538da5459ebc5
QUAD $0x8348c258fbc5c158
- QUAD $0x6d850fd8394908c3
- QUAD $0x840fc98548ffffff
+ QUAD $0x6d850fda394908c3
+ QUAD $0x840fc0854dffffff
QUAD $0xda348d4cffffff34
QUAD $0x66ff3145de1c8d48
QUAD $0x0000000000841f0f
QUAD $0x81c4fb0c107ba1c4
QUAD $0xc158fbc5fe0c5973
- QUAD $0xe875f9394cc7ff49
- QUAD $0x658d48ffffff03e9
- QUAD $0xc35d5f415e415be8
- QUAD $0xc931047304f88348
- QUAD $0xfffffff0ba487eeb
- QUAD $0x10f883487fffffff
- QUAD $0x894843ebc9310473
- QUAD $0xf8c5f631d12148c1
- QUAD $0x2e6666666666c057
- QUAD $0x0000000000841f0f
- QUAD $0x11fcc5f70411fcc5
- QUAD $0xf74411fcc520f744
- QUAD $0x4860f74411fcc540
- QUAD $0xe075f1394810c683
- QUAD $0x740ca83b74c83948
- QUAD $0x0cc28348ce894827
- QUAD $0xf8c5c12148d18948
- QUAD $0x0000441f0f66c057
- QUAD $0xc68348f70411fcc5
- QUAD $0x0bebf275f1394804
- QUAD $0x00000000cf04c748
- QUAD $0xf075c83948c1ff48
- LONG $0xc377f8c5
+ QUAD $0xe875f8394dc7ff49
+ QUAD $0xf88349ffffff03e9
+ QUAD $0x95e9c88949087304
+ QUAD $0x7310f88349000000
+ QUAD $0x83c68949ebd23104
+ QUAD $0xf22948c2894c0fe6
+ QUAD $0xc5f0e18349c18949
+ QUAD $0x9066ca8949c057f9
+ QUAD $0xa1c4d704117da1c4
+ QUAD $0x7da1c420d744117d
+ QUAD $0x117da1c440d74411
+ QUAD $0x4d10c2834960d744
+ QUAD $0x74f68548dc75d139
+ QUAD $0xc689607204fe8357
+ QUAD $0x0149f0294903e683
+ QUAD $0x48c18948ca0148c8
+ QUAD $0x66c057f9c5fce183
+ QUAD $0x0000000000841f0f
+ QUAD $0xc28348d70411fdc5
+ QUAD $0x8548f275d1394804
+ QUAD $0x2e666666661d74f6
+ QUAD $0x0000000000841f0f
+ QUAD $0x00000000c704c74a
+ QUAD $0xf075c0394cc0ff49
+ QUAD $0x415e415be8658d48
+ QUAD $0x0148c377f8c55d5f
+ LONG $0xc88949d1
+ BYTE $0xeb
+ BYTE $0xda
RET
// func tileFloat32AVX2(dst []float32, pattern []float32)
@@ -2967,21 +3533,25 @@ TEXT ·movingAverageFloat32AVX2(SB), NOSPLIT|NOFRAME, $0-56
MOVQ width+48(FP), R8
QUAD $0x294cc09e0fc0854d
QUAD $0xc10841c19c0f41c1
- QUAD $0x8d48000000e3850f
- QUAD $0x4e0f48ca39480141
- QUAD $0x00cf8e0fc08548c2
- QUAD $0x4853e58948550000
- QUAD $0xc02a82c1c4f8e483
- QUAD $0x83c28944ff488d49
- QUAD $0xfffffff8b94907e2
- QUAD $0x4cc1214d7fffffff
- QUAD $0x22ebd231451c468d
- QUAD $0x0000000000841f0f
- QUAD $0x117aa1c4c85ef2c5
- QUAD $0xc08349c2ff49970c
- QUAD $0xc2394904c6834804
- QUAD $0x3145c957f0c57974
- QUAD $0x664e7207f98348db
+ QUAD $0x48550000010b850f
+ QUAD $0x554156415741e589
+ QUAD $0x48f8e48348535441
+ QUAD $0xfffffffffffff8b8
+ QUAD $0xca394801498d4c7f
+ QUAD $0x10f98349ca4e0f4c
+ QUAD $0xc931000000df8d0f
+ QUAD $0x0000c28d0fc9394c
+ QUAD $0x8d49c02a82c1c400
+ QUAD $0xe28341c28945ff50
+ QUAD $0x8e048d4cc0214c07
+ QUAD $0x8e348d481cc08349
+ QUAD $0x2e666666666627eb
+ QUAD $0x0000000000841f0f
+ QUAD $0x0c11fac5c85ef2c5
+ QUAD $0x04c08349c1ff488f
+ QUAD $0x74c9394c04c68348
+ QUAD $0xdb3145c957f0c57a
+ QUAD $0x2e664f7207fa8348
QUAD $0x0000000000841f0f
QUAD $0xc4e4984c587281c4
QUAD $0x81c4e8984c587281
@@ -2990,14 +3560,54 @@ TEXT ·movingAverageFloat32AVX2(SB), NOSPLIT|NOFRAME, $0-56
QUAD $0x4c587281c4f4984c
QUAD $0x984c587281c4f898
QUAD $0x49980c587281c4fc
- QUAD $0xc075d9394d08c383
- QUAD $0x1c8d4e8b74d28548
+ QUAD $0xc075d8394c08c383
+ QUAD $0x1c8d4e8b74d2854d
QUAD $0x0000441f0fdb319e
QUAD $0xff489b0c5872c1c4
- QUAD $0x6de9f275da3948c3
- QUAD $0x5bf8658d48ffffff
- BYTE $0x5d
- BYTE $0xc3
+ QUAD $0x6de9f275da3949c3
+ QUAD $0x5bd8658d48ffffff
+ QUAD $0x5f415e415d415c41
+ QUAD $0x82c1c4c377f8c55d
+ QUAD $0x49c0187de2c4c02a
+ QUAD $0x8341c28945ff508d
+ QUAD $0xc32149c3894d07e2
+ QUAD $0x0010bc413c5e8d48
+ QUAD $0xf68949ff31450000
+ QUAD $0x2e66666666663beb
+ QUAD $0x0000000000841f0f
+ QUAD $0xd05eecc5c85ef4c5
+ QUAD $0xa1c4bf0c117ca1c4
+ QUAD $0x618d4c20bf54117c
+ QUAD $0xc6834940c3834810
+ QUAD $0x0fcc394dcf894940
+ QUAD $0xe1894cfffffebb8f
+ QUAD $0xc5e43145c957f0c5
+ QUAD $0x0f07fa8348d257e8
+ QUAD $0x2e66660000008c82
+ QUAD $0x0000000000841f0f
+ QUAD $0xc4c4a34c5874a1c4
+ QUAD $0xa1c4e4a354586ca1
+ QUAD $0x6ca1c4c8a34c5874
+ QUAD $0x5874a1c4e8a35458
+ QUAD $0x54586ca1c4cca34c
+ QUAD $0xa34c5874a1c4eca3
+ QUAD $0xf0a354586ca1c4d0
+ QUAD $0xc4d4a34c5874a1c4
+ QUAD $0xa1c4f4a354586ca1
+ QUAD $0x6ca1c4d8a34c5874
+ QUAD $0x5874a1c4f8a35458
+ QUAD $0x54586ca1c4dca34c
+ QUAD $0xa34c5874a1c4fca3
+ QUAD $0x49a314586ca1c4e0
+ QUAD $0x8875e3394d08c483
+ QUAD $0xffff2f840fd2854d
+ QUAD $0xed3145a6248d4fff
+ QUAD $0x0000000000841f0f
+ QUAD $0x81c4ac0c587481c4
+ QUAD $0xc5ff4920ac54586c
+ QUAD $0xff06e9eb75ea394d
+ BYTE $0xff
+ BYTE $0xff
RET
// func shiftDivFloat32AVX2(dst []float32, a []float32, shift float32, denom float32)
@@ -3082,21 +3692,25 @@ TEXT ·movingAverageFloat64AVX2(SB), NOSPLIT|NOFRAME, $0-56
MOVQ width+48(FP), R8
QUAD $0x294cc09e0fc0854d
QUAD $0xc10841c19c0f41c1
- QUAD $0x8d48000000e3850f
- QUAD $0x4e0f48ca39480141
- QUAD $0x00cf8e0fc08548c2
- QUAD $0x4853e58948550000
- QUAD $0xc02a83c1c4f8e483
- QUAD $0x83c28944ff488d49
- QUAD $0xfffffff8b94907e2
- QUAD $0x4cc1214d7fffffff
- QUAD $0x22ebd2314538468d
- QUAD $0x0000000000841f0f
- QUAD $0x117ba1c4c85ef3c5
- QUAD $0xc08349c2ff49d70c
- QUAD $0xc2394908c6834808
- QUAD $0x3145c957f1c57974
- QUAD $0x664e7207f98348db
+ QUAD $0x48550000010b850f
+ QUAD $0x554156415741e589
+ QUAD $0x48f8e48348535441
+ QUAD $0xfffffffffffff8b8
+ QUAD $0xca394801498d4c7f
+ QUAD $0x10f98349ca4e0f4c
+ QUAD $0xc931000000df8d0f
+ QUAD $0x0000c28d0fc9394c
+ QUAD $0x8d49c02a83c1c400
+ QUAD $0xe28341c28945ff50
+ QUAD $0xce048d4cc0214c07
+ QUAD $0xce348d4838c08349
+ QUAD $0x2e666666666627eb
+ QUAD $0x0000000000841f0f
+ QUAD $0x0c11fbc5c85ef3c5
+ QUAD $0x08c08349c1ff48cf
+ QUAD $0x74c9394c08c68348
+ QUAD $0xdb3145c957f1c57a
+ QUAD $0x2e664f7207fa8348
QUAD $0x0000000000841f0f
QUAD $0xc4c8d84c587381c4
QUAD $0x81c4d0d84c587381
@@ -3105,14 +3719,58 @@ TEXT ·movingAverageFloat64AVX2(SB), NOSPLIT|NOFRAME, $0-56
QUAD $0x4c587381c4e8d84c
QUAD $0xd84c587381c4f0d8
QUAD $0x49d80c587381c4f8
- QUAD $0xc075d9394d08c383
- QUAD $0x1c8d4e8b74d28548
+ QUAD $0xc075d8394c08c383
+ QUAD $0x1c8d4e8b74d2854d
QUAD $0x0000441f0fdb31de
QUAD $0xff48db0c5873c1c4
- QUAD $0x6de9f275da3948c3
- QUAD $0x5bf8658d48ffffff
- BYTE $0x5d
- BYTE $0xc3
+ QUAD $0x6de9f275da3949c3
+ QUAD $0x5bd8658d48ffffff
+ QUAD $0x5f415e415d415c41
+ QUAD $0x83c1c4c377f8c55d
+ QUAD $0x49c0197de2c4c02a
+ QUAD $0x8341c28945ff508d
+ QUAD $0x214d04588d4c03e2
+ QUAD $0x10bc41785e8d48c3
+ QUAD $0x8949ff3145000000
+ QUAD $0x2e6666666650ebf6
+ QUAD $0x0000000000841f0f
+ QUAD $0xe05eddc5c85ef5c5
+ QUAD $0xa1c4ff0c117da1c4
+ QUAD $0x5eedc520ff64117d
+ QUAD $0x7da1c4d05ee5c5c8
+ QUAD $0x117da1c440ff4c11
+ QUAD $0x4810618d4c60ff54
+ QUAD $0x4980ee834980eb83
+ QUAD $0xa58f0fcc394dcf89
+ QUAD $0xf1c5e1894cfffffe
+ QUAD $0x57d9c5e43145c957
+ QUAD $0x57e1c5d257e9c5e4
+ QUAD $0x8e820f03fa8348db
+ QUAD $0x2e66666666000000
+ QUAD $0x0000000000841f0f
+ QUAD $0xc488e34c5875a1c4
+ QUAD $0xa1c4a8e364585da1
+ QUAD $0x65a1c4c8e354586d
+ QUAD $0x5875a1c4e8e35c58
+ QUAD $0x64585da1c490e34c
+ QUAD $0xe354586da1c4b0e3
+ QUAD $0xf0e35c5865a1c4d0
+ QUAD $0xc498e34c5875a1c4
+ QUAD $0xa1c4b8e364585da1
+ QUAD $0x65a1c4d8e354586d
+ QUAD $0x5875a1c4f8e35c58
+ QUAD $0x64585da1c4a0e34c
+ QUAD $0xe354586da1c4c0e3
+ QUAD $0x49e31c5865a1c4e0
+ QUAD $0x8875e3394d04c483
+ QUAD $0xffff0f840fd2854d
+ QUAD $0xed3145e6248d4fff
+ QUAD $0x0000000000841f0f
+ QUAD $0x81c4ec0c587581c4
+ QUAD $0x6d81c420ec64585d
+ QUAD $0x586581c440ec5458
+ QUAD $0x394dc5ff4960ec5c
+ QUAD $0xfffffed8e9dd75ea
RET
// func shiftDivFloat64AVX2(dst []float64, a []float64, shift float64, denom float64)
diff --git a/internal/amd64/numeric_avx512_amd64.s b/internal/amd64/numeric_avx512_amd64.s
index 9c397b4..0fd9fb6 100644
--- a/internal/amd64/numeric_avx512_amd64.s
+++ b/internal/amd64/numeric_avx512_amd64.s
@@ -189,24 +189,90 @@ TEXT ·polyEvalFloat32AVX512(SB), NOSPLIT|NOFRAME, $0-72
MOVQ x_len+32(FP), R8
MOVQ coeffs_len+56(FP), R9
QUAD $0x4dc14c0f4cc1394c
- QUAD $0x000000f7840fc985
- QUAD $0x0002538e0fc0854d
- QUAD $0xfc8a44107aa1c400
- QUAD $0x0f02e88348c8894c
- QUAD $0x894855000000f08c
- QUAD $0x8d41f8e4834853e5
- QUAD $0xe1834107e183ff49
- QUAD $0x2e661cebd2314507
+ QUAD $0xe58948551a74c985
+ QUAD $0xe483485356415741
+ QUAD $0x31217d10f88349f8
+ QUAD $0x854d00000205e9c0
+ QUAD $0x49000004338e0fc0
+ QUAD $0x00014a830f08f883
+ QUAD $0x00000413e9c03100
+ QUAD $0x49fc8a44107aa1c4
+ QUAD $0x00010e8e0f01f983
+ QUAD $0x518d45fe498d4900
+ QUAD $0x41c018487df262ff
+ QUAD $0x8341cb894507e283
+ QUAD $0x00000010be4107e3
+ QUAD $0x2e66666623ebdb31
+ QUAD $0x0000000000841f0f
+ QUAD $0x4c9f1411487cf162
+ QUAD $0x394dc3894810708d
+ QUAD $0x4c000001948f0fc6
+ QUAD $0x0c10487cf162f089
+ QUAD $0x487cf162ce89499e
+ QUAD $0x257401fb8341d028
+ QUAD $0xf162ce8949d7894d
+ QUAD $0x00401f0fd028487c
+ QUAD $0xb162d159486cf162
+ QUAD $0xceff49b21458586c
+ QUAD $0xf98348eb75cfff49
+ QUAD $0x9008c68349a57207
+ QUAD $0xb162d159486cf162
+ QUAD $0xf162f8b25458586c
+ QUAD $0x586cb162d159486c
+ QUAD $0x486cf162f7b25458
+ QUAD $0x5458586cb162d159
+ QUAD $0xd159486cf162f6b2
+ QUAD $0xf5b25458586cb162
+ QUAD $0xb162d159486cf162
+ QUAD $0xf162f4b25458586c
+ QUAD $0x586cb162d159486c
+ QUAD $0x486cf162f3b25458
+ QUAD $0x5458586cb162d159
+ QUAD $0xd159486cf162f2b2
+ QUAD $0xf1b25458586cb162
+ QUAD $0x07fe8349f8c68349
+ QUAD $0x49ffffff21e9867f
+ QUAD $0xc149c28949f0408d
+ QUAD $0xd18944c2ff4904ea
+ QUAD $0x7370f8834807e183
+ QUAD $0x3100000010ba4125
+ QUAD $0xffffc0b9487febc0
+ QUAD $0xf883497fffffffff
+ QUAD $0x310000019b830f40
+ QUAD $0x8349000001dde9c0
+ QUAD $0xc818487df262f8e2
+ QUAD $0x0000441f0f66c031
+ QUAD $0x62870c11487cf162
+ QUAD $0x6201874c11487cf1
+ QUAD $0x6202874c11487cf1
+ QUAD $0x6203874c11487cf1
+ QUAD $0x6204874c11487cf1
+ QUAD $0x6205874c11487cf1
+ QUAD $0x6206874c11487cf1
+ QUAD $0x4807874c11487cf1
+ QUAD $0x75f8c2834980e883
+ QUAD $0x8d4c2d74c98548b7
+ QUAD $0xc018487df2621050
+ QUAD $0x0000441f0f04e1c1
+ QUAD $0x4c870411487cf162
+ QUAD $0x834810c28349d089
+ QUAD $0xf0c28349ec75f0c1
+ QUAD $0x2948c1894cd0894c
+ QUAD $0xc4000002068e0fc1
+ QUAD $0x894dfc8a44107aa1
+ QUAD $0xcc8c0f02ea8349ca
+ QUAD $0x83ff498d41000000
+ QUAD $0x19eb07e1834107e1
QUAD $0x0000000000841f0f
- QUAD $0xff499714117aa1c4
- QUAD $0x00c8840fc2394dc2
- QUAD $0x960c107aa1c40000
- QUAD $0x41d028f8c5c38949
- QUAD $0xcb89481d7401f983
- QUAD $0x90d028f8c5c38949
+ QUAD $0xc0ff48871411fac5
+ QUAD $0x0001cc840fc0394c
+ QUAD $0x894d860c10fac500
+ QUAD $0xf98341d028f8c5d3
+ QUAD $0x894dcb89481f7401
+ QUAD $0x001f0fd028f8c5d3
QUAD $0x586aa1c4d259f2c5
QUAD $0xcbff48cbff499a14
- QUAD $0xb87207f88348ee75
+ QUAD $0xb87207fa8349ee75
QUAD $0x00401f0f08c38349
QUAD $0x586aa1c4d259f2c5
QUAD $0xc4d259f2c5e09a54
@@ -220,52 +286,51 @@ TEXT ·polyEvalFloat32AVX512(SB), NOSPLIT|NOFRAME, $0-72
QUAD $0x59f2c5c89a54586a
QUAD $0xc49a54586aa1c4d2
QUAD $0x07fb8349f8c38349
- QUAD $0x4dffffff49e99e7f
- QUAD $0x0000015c8e0fc085
- QUAD $0xc0311b7308f88349
- QUAD $0xf8834900000140e9
- QUAD $0x0129e9c031227308
- QUAD $0x5d5bf8658d480000
- QUAD $0xffffffffc0b948c3
- QUAD $0x7340f883497fffff
- QUAD $0xc0b94866ebc0311b
- QUAD $0x497fffffffffffff
- QUAD $0xe9c0317e7340f883
- QUAD $0x48c0894c000000c7
- QUAD $0xc057f8c5d231c821
- QUAD $0x0000000000841f0f
+ QUAD $0x48ffffff49e99e7f
+ QUAD $0xc18948087308f983
+ QUAD $0xf983480000010be9
+ QUAD $0x00c0e9d231787340
+ QUAD $0xc82148c0894c0000
+ QUAD $0x9066c057f8c5d231
QUAD $0x62970411487cf162
QUAD $0x6201974411487cf1
QUAD $0x6202974411487cf1
QUAD $0x4803974411487cf1
QUAD $0xd875d0394840c283
- QUAD $0x0000cb840fc03949
- QUAD $0xb2840f38c0f64100
+ QUAD $0x0000f1840fc03949
+ QUAD $0xd8840f38c0f64100
QUAD $0x8348c28948000000
QUAD $0xc0214cc8894838c1
QUAD $0x00401f0fc057f8c5
QUAD $0xc28348970411fcc5
- QUAD $0x94e9f275d0394808
- QUAD $0x2148c0894c000000
- QUAD $0x31c818487df262c8
- QUAD $0x2e666666666666d2
- QUAD $0x0000000000841f0f
- QUAD $0x62970c11487cf162
- QUAD $0x6201974c11487cf1
- QUAD $0x6202974c11487cf1
- QUAD $0x4803974c11487cf1
- QUAD $0xd875d0394840c283
- QUAD $0xc0f6414f74c03949
- QUAD $0x8348c28948307438
- QUAD $0xc0214cc8894838c1
- QUAD $0x2e6666c8187de2c4
- QUAD $0x0000000000841f0f
+ QUAD $0xbae9f275d0394808
+ QUAD $0x8348ca8948000000
+ QUAD $0xc818487df262c0e2
+ QUAD $0xc0c6814887348d48
+ QUAD $0x9066c93145000000
+ QUAD $0xfd8e4c11487cb162
+ QUAD $0xfe8e4c11487cb162
+ QUAD $0xff8e4c11487cb162
+ QUAD $0x498e0c11487cb162
+ QUAD $0xd875ca394c40c183
+ QUAD $0x38c1f65074d13948
+ QUAD $0x07e683c689445974
+ QUAD $0xe2c4c10148f12948
+ QUAD $0x894cc20148c8187d
+ QUAD $0x001f0ff8e08348c0
QUAD $0xc28348970c11fcc5
- QUAD $0x3949f275d0394808
- QUAD $0x870411fac51974c0
- QUAD $0x8704c7f1ebc0ff48
- QUAD $0x49c0ff4800000000
- QUAD $0xc377f8c5f175c039
+ QUAD $0x8548f275d0394808
+ QUAD $0x2e666666661a74f6
+ QUAD $0x0000000000841f0f
+ QUAD $0xc1ff488f0411fac5
+ QUAD $0x658d48f375c83949
+ QUAD $0xc55d5f415e415be8
+ QUAD $0x8948d00148c377f8
+ QUAD $0x00008704c7ddebc1
+ QUAD $0xc03949c0ff480000
+ LONG $0xf8c5f175
+ BYTE $0x77
+ BYTE $0xc3
RET
// func convolveFloat32AVX512(dst []float32, sig []float32, ker []float32)
@@ -276,101 +341,328 @@ TEXT ·convolveFloat32AVX512(SB), NOSPLIT|NOFRAME, $0-72
MOVQ dst_len+8(FP), CX
MOVQ sig_len+32(FP), R8
MOVQ ker_len+56(FP), R9
+ // Constant pool appended below the body; the displacements
+ // above were patched to reach it and the aligned loads made
+ // unaligned, since nothing promises alignment inside a TEXT.
QUAD $0x56415741e5894855
QUAD $0xe483485354415541
- QUAD $0x74894838ec8348f8
- QUAD $0xc0940fc9854d0824
- QUAD $0x40c69c0f40c8294d
- QUAD $0x000002af850fc608
- QUAD $0x48c1394c01708d49
- QUAD $0x8e0ff68548f14e0f
- QUAD $0x0fc9854d0000029b
- QUAD $0x0c8d4a000001f28e
- QUAD $0x07e08341c889458a
- QUAD $0xfffffffffff8ba49
- QUAD $0x448b48ca214d7fff
- QUAD $0x8d4a1c588d4c0824
- QUAD $0x8948fcc083488a04
- QUAD $0x182444c748102444
- QUAD $0xfffebe4900000000
- QUAD $0xbf493fffffffffff
- QUAD $0x3ffffffffffffffd
- QUAD $0xfffffffffffcbc49
- QUAD $0xfffffffbbd493fff
- QUAD $0xfffab8483fffffff
- QUAD $0xba483fffffffffff
- QUAD $0x3ffffffffffffff9
- QUAD $0xfffffffffff8bb48
- QUAD $0x4830247c89483fff
- QUAD $0x244c894c28247489
- QUAD $0x0000441f0f3aeb20
- QUAD $0x4c8b4c30247c8b48
- QUAD $0x8f04117aa1c41824
- QUAD $0x4804c38349c1ff49
- QUAD $0x748b480408244483
- QUAD $0x4918244c894c2824
- QUAD $0x0f20244c8b4cf139
- QUAD $0x57f8c5000001ca84
- QUAD $0x45117308f98349c0
- QUAD $0x66000000dbe9c931
+ QUAD $0xc9854d38ec8348f8
+ QUAD $0x20244c894cc0940f
+ QUAD $0x41c19c0f41c8294d
+ QUAD $0x0000067b850fc108
+ QUAD $0x4cc1394c01588d4d
+ QUAD $0x4810fb8349d94e0f
+ QUAD $0x245c894c30247c89
+ QUAD $0x8948182474894828
+ QUAD $0xe9d231077d102454
+ QUAD $0x244c8b480000038e
+ QUAD $0x016a8e0fc9854820
+ QUAD $0xc8898a048d4c0000
+ QUAD $0xfffff8b94903e083
+ QUAD $0xc983497fffffffff
+ QUAD $0x0c568d4cc9214904
+ QUAD $0xfcc283488a148d48
+ QUAD $0xed314500000010b9
+ QUAD $0xfffffffffffebb48
+ QUAD $0xfffffffdbe493fff
+ QUAD $0xfffcbf493fffffff
+ QUAD $0x89493fffffffffff
+ QUAD $0x90662bebd68948f4
+ QUAD $0x7cb16230247c8b48
+ QUAD $0x104a8d48af041148
+ QUAD $0x40c4834940c28349
+ QUAD $0x28245c8b4cd58949
+ QUAD $0x0003018f0fd9394c
+ QUAD $0x247c8348ca894800
+ QUAD $0xc057f8c50c730420
+ QUAD $0x9000000094e9c931
+ QUAD $0x31f78948c057f8c5
+ QUAD $0x00000000801f0fc9
+ QUAD $0xd1620f18487df262
+ QUAD $0xfffff48a8c594874
+ QUAD $0x49c158487cf162ff
+ QUAD $0x7d9262db3149cb89
+ QUAD $0x4874d162980c1848
+ QUAD $0x62fffffff88a8c59
+ QUAD $0xcb8949c158487cf1
+ QUAD $0x18487d9262f3314d
+ QUAD $0x8c594874d162980c
+ QUAD $0x7cf162fffffffc8a
+ QUAD $0x314dcb8949c15848
+ QUAD $0x980c18487d9262fb
+ QUAD $0x628a0c594874d162
+ QUAD $0xc18348c158487cf1
+ QUAD $0xc93949f0c7834804
+ QUAD $0x2d840fc085488675
+ QUAD $0x488c3c8d49ffffff
+ QUAD $0x2949f3894902e1c1
+ QUAD $0x2e66666666c931cb
QUAD $0x0000000000841f0f
- QUAD $0xc9314510247c8b48
+ QUAD $0xf1620b18487dd262
+ QUAD $0x7cf1628f0c594874
+ QUAD $0x8349c1ff48c15848
+ QUAD $0xe9e175c83948fcc3
+ QUAD $0xf0438d4dfffffeec
+ QUAD $0x73000000f0f88149
+ QUAD $0xd23100000010b80c
+ QUAD $0xe8c149000001dce9
+ QUAD $0x49c1894dc0ff4904
+ QUAD $0xc148c8894cf0e183
+ QUAD $0x0510487cf16204e0
+ QUAD $0x48c28948000004b8
+ QUAD $0x10487cf16210c083
+ QUAD $0x57e8c5000004e70d
+ QUAD $0x191d5948fdf262d2
+ QUAD $0x9066ca894d000007
+ QUAD $0x497df262c846fdc5
+ QUAD $0x62c846fdc58f14a3
+ QUAD $0xfdc58714a3497df2
+ QUAD $0x54a3497df262c846
+ QUAD $0xf262c846fdc5018f
+ QUAD $0xfdc5018754a3497d
+ QUAD $0x54a3497df262c846
+ QUAD $0xf262c846fdc5028f
+ QUAD $0xfdc5028754a3497d
+ QUAD $0x54a3497df262c846
+ QUAD $0xf262c846fdc5038f
+ QUAD $0xfdc5038754a3497d
+ QUAD $0x54a3497df262c846
+ QUAD $0xf262c846fdc5048f
+ QUAD $0xfdc5048754a3497d
+ QUAD $0x54a3497df262c846
+ QUAD $0xf262c846fdc5058f
+ QUAD $0xfdc5058754a3497d
+ QUAD $0x54a3497df262c846
+ QUAD $0xf262c846fdc5068f
+ QUAD $0xfdc5068754a3497d
+ QUAD $0x54a3497df262c846
+ QUAD $0xf262c846fdc5078f
+ QUAD $0xfdc5078754a3497d
+ QUAD $0x54a3497df262c846
+ QUAD $0xf262c846fdc5088f
+ QUAD $0xfdc5088754a3497d
+ QUAD $0x54a3497df262c846
+ QUAD $0xf262c846fdc5098f
+ QUAD $0xfdc5098754a3497d
+ QUAD $0x54a3497df262c846
+ QUAD $0xf262c846fdc50a8f
+ QUAD $0xfdc50a8754a3497d
+ QUAD $0x54a3497df262c846
+ QUAD $0xf262c846fdc50b8f
+ QUAD $0xfdc50b8754a3497d
+ QUAD $0x54a3497df262c846
+ QUAD $0xf262c846fdc50c8f
+ QUAD $0xfdc50c8754a3497d
+ QUAD $0x54a3497df262c846
+ QUAD $0xf262c846fdc50d8f
+ QUAD $0xfdc50d8754a3497d
+ QUAD $0x54a3497df262c846
+ QUAD $0xf262c846fdc50e8f
+ QUAD $0xfdc50e8754a3497d
+ QUAD $0x54a3497df262c846
+ QUAD $0xf262c846fdc50f8f
+ QUAD $0xf1620f8754a3497d
+ QUAD $0x48fdf162cbd448f5
+ QUAD $0x850ff0c28349c3d4
+ QUAD $0x74c8394dfffffe6c
+ QUAD $0xd18948c057f8c521
+ QUAD $0x488f0411487cf162
+ QUAD $0x394c10c08348c189
+ QUAD $0x48f0c08348ed7ed8
+ QUAD $0x548948d8894cc289
+ QUAD $0x548b48d029480824
+ QUAD $0x0f1024748b481824
+ QUAD $0x4c8b48000002a68e
+ QUAD $0xdb8e0fc985482024
+ QUAD $0x418e048d4c000001
+ QUAD $0xbb4907e28341ca89
+ QUAD $0x7ffffffffffffff8
+ QUAD $0x0824448b48cb2149
+ QUAD $0x1cc38348821c8d48
+ QUAD $0x4c8948fc8e4c8d48
+ QUAD $0x894882048d481024
+ QUAD $0xfffffebe49182444
+ QUAD $0xfdbf493fffffffff
+ QUAD $0x493fffffffffffff
+ QUAD $0xfffffffffffffcbc
+ QUAD $0xfffffffffbbd493f
+ QUAD $0xfffffab8483fffff
+ QUAD $0xf9b9493fffffffff
+ QUAD $0x483fffffffffffff
+ QUAD $0xfffffffffffff8ba
+ QUAD $0x2e6666666639eb3f
QUAD $0x0000000000841f0f
- QUAD $0xc4e48b4c107a81c4
- QUAD $0xf2c5e88b54107a81
- QUAD $0x894cc158fac50f59
- QUAD $0x0c59eac5f6314cce
- QUAD $0x7a81c4c158fac5b1
- QUAD $0x4cce894cec8b4c10
- QUAD $0xc5b10c59f2c5fe31
- QUAD $0x4c107a81c4c158fa
- QUAD $0xe6314cce894cf08b
- QUAD $0x58fac5b10c59f2c5
- QUAD $0xf48b4c107a81c4c1
- QUAD $0xf2c5ee314cce894c
- QUAD $0xc4c158fac5b10c59
- QUAD $0x894cf88b4c107a81
- QUAD $0x0c59f2c5c63148ce
- QUAD $0x7a81c4c158fac5b1
- QUAD $0x48ce894cfc8b4c10
- QUAD $0xc5b10c59f2c5d631
- QUAD $0x0c107a81c4c158fa
- QUAD $0xc5de3148ce894c8b
- QUAD $0xc158fac5b10c59f2
- QUAD $0xe0c7834808c18349
- QUAD $0xffff47850fca394d
- QUAD $0xfede840fc0854dff
- QUAD $0x00008d348d4affff
- QUAD $0x4810247c8b480000
- QUAD $0x4e0824748b48f729
- QUAD $0x001f0ff6318e0c8d
- QUAD $0xf2c5b10c107ac1c4
- QUAD $0xff48c158fac50f59
- QUAD $0xf03949fcc78348c6
- QUAD $0x48fffffea1e9e675
- QUAD $0xe9c031077308fe83
- QUAD $0xfff8b94800000084
- QUAD $0x83487fffffffffff
- QUAD $0x43ebc031047340fe
- QUAD $0x31f02148c8418d48
- QUAD $0x001f0fc057f8c5d2
- QUAD $0x62970411487cf162
- QUAD $0x6201974411487cf1
- QUAD $0x6202974411487cf1
- QUAD $0x4803974411487cf1
- QUAD $0xd875d0394840c283
- QUAD $0xc6f6404274c63948
- QUAD $0xf8c5f121482d7438
- QUAD $0x0000441f0f66c057
- QUAD $0xc08348870411fcc5
- QUAD $0x8948f275c1394808
- QUAD $0x2e661974ce3948c8
+ QUAD $0x748b4830244c8b48
+ QUAD $0x48b10411fac50824
+ QUAD $0x834804c38348c6ff
+ QUAD $0x2474894804182444
+ QUAD $0x840f2824743b4808
+ QUAD $0x247c8348000001df
+ QUAD $0xc057f8c50c730820
+ QUAD $0x90000000d2e9c931
+ QUAD $0x247c8b48c057f8c5
+ QUAD $0x0000441f0fc93110
+ QUAD $0xfac5e48b4c10fac5
+ QUAD $0x0f59f2c5e88b5410
+ QUAD $0x4cce8948c158fac5
+ QUAD $0xb00c596ac1c4f631
+ QUAD $0x4c10fac5c158fac5
+ QUAD $0xfe314cce8948ec8b
+ QUAD $0xfac5b00c5972c1c4
+ QUAD $0xf08b4c10fac5c158
+ QUAD $0xc1c4e6314cce8948
+ QUAD $0xc158fac5b00c5972
+ QUAD $0x8948f48b4c10fac5
+ QUAD $0x5972c1c4ee314cce
+ QUAD $0xfac5c158fac5b00c
+ QUAD $0x48ce8948f88b4c10
+ QUAD $0xb00c5972c1c4c631
+ QUAD $0x4c10fac5c158fac5
+ QUAD $0xce314cce8948fc8b
+ QUAD $0xfac5b00c5972c1c4
+ QUAD $0x488b0c10fac5c158
+ QUAD $0x72c1c4d63148ce89
+ QUAD $0x48c158fac5b00c59
+ QUAD $0x49e0c7834808c183
+ QUAD $0xffffff48850fcb39
+ QUAD $0xfffeef840fd2854d
+ QUAD $0x0000008d348d48ff
+ QUAD $0x294810247c8b4800
+ QUAD $0x8d481824748b48f7
+ QUAD $0x00401f0ff6318e0c
+ QUAD $0x59f2c5b10c10fac5
+ QUAD $0xc6ff48c158fac50f
+ QUAD $0x75f23949fcc78348
+ QUAD $0x8348fffffeb2e9e7
+ QUAD $0x00000098820f08f8
+ QUAD $0x24448b4c40f88348
+ QUAD $0x484febc931047308
+ QUAD $0x8d4ac0e18348c189
+ QUAD $0x0000c0c281488714
+ QUAD $0x90c057f8c5f63100
+ QUAD $0xfdb24411487cf162
+ QUAD $0xfeb24411487cf162
+ QUAD $0xffb24411487cf162
+ QUAD $0x48b20411487cf162
+ QUAD $0xd875f1394840c683
+ QUAD $0x7438a85e74c83948
+ QUAD $0x89480824448b4c6c
+ QUAD $0x348d4af8e28348c2
+ QUAD $0x2444894cd0014987
+ QUAD $0x001f0fc057f8c508
+ QUAD $0xc183488e0411fcc5
+ QUAD $0x3948f275ca394808
+ QUAD $0x2e666666662874d0
QUAD $0x0000000000841f0f
- QUAD $0x48000000008704c7
- QUAD $0x48f175c63948c0ff
- QUAD $0x5d415c415bd8658d
- QUAD $0x77f8c55d5f415e41
- BYTE $0xc3
+ QUAD $0x8704c70824448b48
+ QUAD $0x48c0ff4800000000
+ QUAD $0x2444394808244489
+ QUAD $0x5bd8658d48e57528
+ QUAD $0x5f415e415d415c41
+ QUAD $0x4c0148c377f8c55d
+ QUAD $0x00000080cceb0824
+ QUAD $0x0000009000000000
+ QUAD $0x000000a000000000
+ QUAD $0x000000b000000000
+ QUAD $0x000000c000000000
+ QUAD $0x000000d000000000
+ QUAD $0x000000e000000000
+ QUAD $0x000000f000000000
+ QUAD $0x0000000000000000
+ QUAD $0x0000001000000000
+ QUAD $0x0000002000000000
+ QUAD $0x0000003000000000
+ QUAD $0x0000004000000000
+ QUAD $0x0000005000000000
+ QUAD $0x0000006000000000
+ QUAD $0x0000007000000000
+ QUAD $0x0000000000000000
+ QUAD $0x0000001000000000
+ QUAD $0x0000002000000000
+ QUAD $0x0000003000000000
+ QUAD $0x0000004000000000
+ QUAD $0x0000005000000000
+ QUAD $0x0000006000000000
+ QUAD $0x0000007000000000
+ QUAD $0x0000008000000000
+ QUAD $0x0000009000000000
+ QUAD $0x000000a000000000
+ QUAD $0x000000b000000000
+ QUAD $0x000000c000000000
+ QUAD $0x000000d000000000
+ QUAD $0x000000e000000000
+ QUAD $0x000000f000000000
+ QUAD $0x0000000000000000
+ QUAD $0x0000001000000000
+ QUAD $0x0000002000000000
+ QUAD $0x0000003000000000
+ QUAD $0x0000004000000000
+ QUAD $0x0000005000000000
+ QUAD $0x0000006000000000
+ QUAD $0x0000007000000000
+ QUAD $0x0000000000000000
+ QUAD $0x0000001000000000
+ QUAD $0x0000002000000000
+ QUAD $0x0000003000000000
+ QUAD $0x0000004000000000
+ QUAD $0x0000005000000000
+ QUAD $0x0000006000000000
+ QUAD $0x0000007000000000
+ QUAD $0x0000000000000000
+ QUAD $0x0000000100000000
+ QUAD $0x0000000200000000
+ QUAD $0x0000000300000000
+ QUAD $0x0000000400000000
+ QUAD $0x0000000500000000
+ QUAD $0x0000000600000000
+ QUAD $0x0000000700000000
+ QUAD $0x0000000000000000
+ QUAD $0x0000000100000000
+ QUAD $0x0000000200000000
+ QUAD $0x0000000300000000
+ QUAD $0x0000000400000000
+ QUAD $0x0000000500000000
+ QUAD $0x0000000600000000
+ QUAD $0x0000000700000000
+ QUAD $0x0000000800000000
+ QUAD $0x0000000900000000
+ QUAD $0x0000000a00000000
+ QUAD $0x0000000b00000000
+ QUAD $0x0000000c00000000
+ QUAD $0x0000000d00000000
+ QUAD $0x0000000e00000000
+ QUAD $0x0000000f00000000
+ QUAD $0x0000000000000000
+ QUAD $0x0000000100000000
+ QUAD $0x0000000200000000
+ QUAD $0x0000000300000000
+ QUAD $0x0000000400000000
+ QUAD $0x0000000500000000
+ QUAD $0x0000000600000000
+ QUAD $0x0000000700000000
+ QUAD $0x0000010000000000
+ QUAD $0x0000008000000000
+ QUAD $0x0000010000000000
+ QUAD $0x0000008000000000
+ QUAD $0xfa53e0a800000000
+ QUAD $0xf4a7c150f1bbcdcb
+ QUAD $0xeefba1f8e3779b97
+ QUAD $0x7f4a7c15d5336963
+ QUAD $0x1ce4e5b99e3779b9
+ QUAD $0x133111ebbf58476d
+ QUAD $0x0000002094d049bb
+ QUAD $0x0000000800000000
+ QUAD $0x7f4a7c1500000000
+ QUAD $0x1ce4e5b99e3779b9
+ QUAD $0x133111ebbf58476d
+ QUAD $0x0000000094d049bb
+ QUAD $0x000000083ca00000
+ QUAD $0x7f4a7c1500000000
+ QUAD $0x1ce4e5b99e3779b9
+ QUAD $0x133111ebbf58476d
+ QUAD $0x0000001094d049bb
+ QUAD $0x0000000400000000
+ LONG $0x00000000
RET
// func correlateFloat32AVX512(dst []float32, sig []float32, ker []float32)
@@ -381,75 +673,302 @@ TEXT ·correlateFloat32AVX512(SB), NOSPLIT|NOFRAME, $0-72
MOVQ dst_len+8(FP), CX
MOVQ sig_len+32(FP), R8
MOVQ ker_len+56(FP), R9
+ // Constant pool appended below the body; the displacements
+ // above were patched to reach it and the aligned loads made
+ // unaligned, since nothing promises alignment inside a TEXT.
QUAD $0x294dc0940fc9854d
QUAD $0xc20841c29c0f41c8
- QUAD $0x8d49000001f9850f
- QUAD $0x4e0f48c1394c0140
- QUAD $0x01e58e0fc08548c1
- QUAD $0x358e0fc9854d0000
- QUAD $0x41e5894855000001
- QUAD $0xf8e4834853564157
- QUAD $0xb84907e183c98944
+ QUAD $0x4855000005c7850f
+ QUAD $0x554156415741e589
+ QUAD $0x49f8e48348535441
+ QUAD $0x0f48c1394c01408d
+ QUAD $0x077d10f88348c14e
+ QUAD $0x4d0000039be9c931
+ QUAD $0x000001798e0fc985
+ QUAD $0x4907e08341c88945
+ QUAD $0xfffffffffffff8ba
+ QUAD $0x1c5e8d4cca214d7f
+ QUAD $0x314500000010bf41
+ QUAD $0x906621ebf38948f6
+ QUAD $0x4cb70411487cb162
+ QUAD $0x4840c3834910798d
+ QUAD $0x3949ce894940c383
+ QUAD $0x4c0000034b8f0fc7
+ QUAD $0x3145c057f8c5f989
+ QUAD $0xe6820f08f98349ff
+ QUAD $0x2e66666666000000
+ QUAD $0x0000000000841f0f
+ QUAD $0x62ba0c18487db262
+ QUAD $0xffe4bb8c59487491
+ QUAD $0x5418487db262ffff
+ QUAD $0xc158487cf16201ba
+ QUAD $0xe8bb8c59486c9162
+ QUAD $0x58487cf162ffffff
+ QUAD $0xba4c18487db262c1
+ QUAD $0xbb8c594874916202
+ QUAD $0x487db262ffffffec
+ QUAD $0x486c916203ba5418
+ QUAD $0x62fffffff0bb9459
+ QUAD $0x7cf162c158487cf1
+ QUAD $0x18487db262c25848
+ QUAD $0x594874916204ba4c
+ QUAD $0xf162fffffff4bb8c
+ QUAD $0x487db262c158487c
+ QUAD $0x4874916205ba4c18
+ QUAD $0x62fffffff8bb8c59
+ QUAD $0x7db262c158487cf1
+ QUAD $0x74916206ba4c1848
+ QUAD $0xfffffffcbb8c5948
+ QUAD $0x07ba5418487db262
+ QUAD $0x9162c158487cf162
+ QUAD $0x7cf162bb0c59486c
+ QUAD $0x4d08c78349c15848
+ QUAD $0xffffff30850ffa39
+ QUAD $0xfffee7840fc0854d
+ QUAD $0x3c8d4ebb248d4eff
+ QUAD $0x2e666666ed3145ba
+ QUAD $0x0000000000841f0f
+ QUAD $0x62af0c18487d9262
+ QUAD $0xf162ac0c59487491
+ QUAD $0x4dc5ff49c158487c
+ QUAD $0xfffeafe9e475e839
+ QUAD $0xfa8149f0508d4cff
+ QUAD $0xb8410d73000000f0
+ QUAD $0xdee9c93100000010
+ QUAD $0x4904eac149000001
+ QUAD $0xe38349d3894dc2ff
+ QUAD $0x04e1c148d9894cf0
+ QUAD $0x04ac0510487cf162
+ QUAD $0xf16210418d4c0000
+ QUAD $0x000004de0d10487c
+ QUAD $0x48fdf262d257e8c5
+ QUAD $0x894c000006601d59
+ QUAD $0x00000000801f0fdb
+ QUAD $0x497df262c846fdc5
+ QUAD $0x62c846fdc58f14a3
+ QUAD $0xfdc58714a3497df2
+ QUAD $0x54a3497df262c846
+ QUAD $0xf262c846fdc5018f
+ QUAD $0xfdc5018754a3497d
+ QUAD $0x54a3497df262c846
+ QUAD $0xf262c846fdc5028f
+ QUAD $0xfdc5028754a3497d
+ QUAD $0x54a3497df262c846
+ QUAD $0xf262c846fdc5038f
+ QUAD $0xfdc5038754a3497d
+ QUAD $0x54a3497df262c846
+ QUAD $0xf262c846fdc5048f
+ QUAD $0xfdc5048754a3497d
+ QUAD $0x54a3497df262c846
+ QUAD $0xf262c846fdc5058f
+ QUAD $0xfdc5058754a3497d
+ QUAD $0x54a3497df262c846
+ QUAD $0xf262c846fdc5068f
+ QUAD $0xfdc5068754a3497d
+ QUAD $0x54a3497df262c846
+ QUAD $0xf262c846fdc5078f
+ QUAD $0xfdc5078754a3497d
+ QUAD $0x54a3497df262c846
+ QUAD $0xf262c846fdc5088f
+ QUAD $0xfdc5088754a3497d
+ QUAD $0x54a3497df262c846
+ QUAD $0xf262c846fdc5098f
+ QUAD $0xfdc5098754a3497d
+ QUAD $0x54a3497df262c846
+ QUAD $0xf262c846fdc50a8f
+ QUAD $0xfdc50a8754a3497d
+ QUAD $0x54a3497df262c846
+ QUAD $0xf262c846fdc50b8f
+ QUAD $0xfdc50b8754a3497d
+ QUAD $0x54a3497df262c846
+ QUAD $0xf262c846fdc50c8f
+ QUAD $0xfdc50c8754a3497d
+ QUAD $0x54a3497df262c846
+ QUAD $0xf262c846fdc50d8f
+ QUAD $0xfdc50d8754a3497d
+ QUAD $0x54a3497df262c846
+ QUAD $0xf262c846fdc50e8f
+ QUAD $0xfdc50e8754a3497d
+ QUAD $0x54a3497df262c846
+ QUAD $0xf262c846fdc50f8f
+ QUAD $0xf1620f8754a3497d
+ QUAD $0x48fdf162cbd448f5
+ QUAD $0x850ff0c38348c3d4
+ QUAD $0x74da394dfffffe6c
+ QUAD $0x001f0fc057f8c521
+ QUAD $0x4c8f0411487cf162
+ QUAD $0x394910c08349c189
+ QUAD $0x4cf0c08349ed7ec0
+ QUAD $0xc82949c08949c189
+ QUAD $0x854d000001e98e0f
+ QUAD $0x450000012e8e0fc9
+ QUAD $0xba4907e08341c889
QUAD $0x7ffffffffffffff8
- QUAD $0x451c568d4cc8214d
- QUAD $0x00401f0f1eebdb31
- QUAD $0xff499f04117aa1c4
- QUAD $0xc6834804c28349c3
- QUAD $0x00e3840fc3394904
- QUAD $0xdb31c057f8c50000
- QUAD $0x00a2820f08f98349
- QUAD $0x0000441f0f660000
- QUAD $0xc4e49a4c107ac1c4
- QUAD $0xf2c5e89a54107ac1
+ QUAD $0x498e1c8d4cca214d
+ QUAD $0xeb8e348d481cc383
+ QUAD $0x2e66666666666628
+ QUAD $0x0000000000841f0f
+ QUAD $0xc1ff488f0411fac5
+ QUAD $0x04c6834804c38349
+ QUAD $0x000196840fc13948
+ QUAD $0x49db31c057f8c500
+ QUAD $0x0000a3820f08f983
+ QUAD $0x00000000801f0f00
+ QUAD $0xc4e49b4c107ac1c4
+ QUAD $0xf2c5e89b54107ac1
QUAD $0xc5c158fac59a0c59
QUAD $0x58fac5049a4c59ea
- QUAD $0xec9a4c107ac1c4c1
+ QUAD $0xec9b4c107ac1c4c1
QUAD $0xc1c4089a4c59f2c5
- QUAD $0x59eac5f09a54107a
+ QUAD $0x59eac5f09b54107a
QUAD $0xc5c158fac50c9a54
QUAD $0x4c107ac1c4c258fa
- QUAD $0x109a4c59f2c5f49a
+ QUAD $0x109a4c59f2c5f49b
QUAD $0x107ac1c4c158fac5
- QUAD $0x9a4c59f2c5f89a4c
+ QUAD $0x9a4c59f2c5f89b4c
QUAD $0x7ac1c4c158fac514
- QUAD $0x4c59f2c5fc9a4c10
- QUAD $0x9a14107ac1c4189a
+ QUAD $0x4c59f2c5fc9b4c10
+ QUAD $0x9b14107ac1c4189a
QUAD $0x4c59eac5c158fac5
QUAD $0x8348c158fac51c9a
- QUAD $0x6d850fd8394908c3
- QUAD $0x840fc98548ffffff
+ QUAD $0x6d850fda394908c3
+ QUAD $0x840fc0854dffffff
QUAD $0x9a348d4cffffff34
QUAD $0x66ff31459e1c8d48
QUAD $0x0000000000841f0f
QUAD $0x81c4bb0c107aa1c4
QUAD $0xc158fac5be0c5972
- QUAD $0xe875f9394cc7ff49
- QUAD $0x658d48ffffff03e9
- QUAD $0xc35d5f415e415be8
- QUAD $0xc931077308f88348
- QUAD $0xf8ba480000008be9
- QUAD $0x487fffffffffffff
- QUAD $0xebc931047340f883
- QUAD $0xc12148c84a8d4848
- QUAD $0x2e66c057f8c5f631
- QUAD $0x0000000000841f0f
- QUAD $0x62b70411487cf162
- QUAD $0x6201b74411487cf1
- QUAD $0x6202b74411487cf1
- QUAD $0x4803b74411487cf1
- QUAD $0xd875f1394840c683
- QUAD $0x7438a84274c83948
- QUAD $0xc057f8c5c221482f
+ QUAD $0xe875f8394dc7ff49
+ QUAD $0xf88349ffffff03e9
+ QUAD $0x4900000099820f08
+ QUAD $0xebd231047340f883
+ QUAD $0xc0e28348c2894c52
+ QUAD $0xc0c681488f348d48
+ QUAD $0xf8c5c93145000000
+ QUAD $0x0000441f0f66c057
+ QUAD $0xfd8e4411487cb162
+ QUAD $0xfe8e4411487cb162
+ QUAD $0xff8e4411487cb162
+ QUAD $0x498e0411487cb162
+ QUAD $0xd875ca394c40c183
+ QUAD $0xc0f6415274d03949
+ QUAD $0x8348c6894c5e7438
+ QUAD $0x01488f0c8d4cf8e6
+ QUAD $0x2e6666c057f8c5f1
QUAD $0x0000000000841f0f
- QUAD $0xc183488f0411fcc5
- QUAD $0x8948f275ca394808
- QUAD $0x2e661974d03948d1
+ QUAD $0x83489104117cc1c4
+ QUAD $0x49f175d6394808c2
+ QUAD $0x2e6666661b74f039
QUAD $0x0000000000841f0f
QUAD $0x48000000008f04c7
- QUAD $0xc5f175c83948c1ff
- BYTE $0xf8
- BYTE $0x77
- BYTE $0xc3
+ QUAD $0x48f175c83948c1ff
+ QUAD $0x5d415c415bd8658d
+ QUAD $0x77f8c55d5f415e41
+ QUAD $0x0080daebd10148c3
+ QUAD $0x0090000000000000
+ QUAD $0x00a0000000000000
+ QUAD $0x00b0000000000000
+ QUAD $0x00c0000000000000
+ QUAD $0x00d0000000000000
+ QUAD $0x00e0000000000000
+ QUAD $0x00f0000000000000
+ QUAD $0x0000000000000000
+ QUAD $0x0010000000000000
+ QUAD $0x0020000000000000
+ QUAD $0x0030000000000000
+ QUAD $0x0040000000000000
+ QUAD $0x0050000000000000
+ QUAD $0x0060000000000000
+ QUAD $0x0070000000000000
+ QUAD $0x0000000000000000
+ QUAD $0x0010000000000000
+ QUAD $0x0020000000000000
+ QUAD $0x0030000000000000
+ QUAD $0x0040000000000000
+ QUAD $0x0050000000000000
+ QUAD $0x0060000000000000
+ QUAD $0x0070000000000000
+ QUAD $0x0080000000000000
+ QUAD $0x0090000000000000
+ QUAD $0x00a0000000000000
+ QUAD $0x00b0000000000000
+ QUAD $0x00c0000000000000
+ QUAD $0x00d0000000000000
+ QUAD $0x00e0000000000000
+ QUAD $0x00f0000000000000
+ QUAD $0x0000000000000000
+ QUAD $0x0010000000000000
+ QUAD $0x0020000000000000
+ QUAD $0x0030000000000000
+ QUAD $0x0040000000000000
+ QUAD $0x0050000000000000
+ QUAD $0x0060000000000000
+ QUAD $0x0070000000000000
+ QUAD $0x0000000000000000
+ QUAD $0x0010000000000000
+ QUAD $0x0020000000000000
+ QUAD $0x0030000000000000
+ QUAD $0x0040000000000000
+ QUAD $0x0050000000000000
+ QUAD $0x0060000000000000
+ QUAD $0x0070000000000000
+ QUAD $0x0000000000000000
+ QUAD $0x0001000000000000
+ QUAD $0x0002000000000000
+ QUAD $0x0003000000000000
+ QUAD $0x0004000000000000
+ QUAD $0x0005000000000000
+ QUAD $0x0006000000000000
+ QUAD $0x0007000000000000
+ QUAD $0x0000000000000000
+ QUAD $0x0001000000000000
+ QUAD $0x0002000000000000
+ QUAD $0x0003000000000000
+ QUAD $0x0004000000000000
+ QUAD $0x0005000000000000
+ QUAD $0x0006000000000000
+ QUAD $0x0007000000000000
+ QUAD $0x0008000000000000
+ QUAD $0x0009000000000000
+ QUAD $0x000a000000000000
+ QUAD $0x000b000000000000
+ QUAD $0x000c000000000000
+ QUAD $0x000d000000000000
+ QUAD $0x000e000000000000
+ QUAD $0x000f000000000000
+ QUAD $0x0000000000000000
+ QUAD $0x0001000000000000
+ QUAD $0x0002000000000000
+ QUAD $0x0003000000000000
+ QUAD $0x0004000000000000
+ QUAD $0x0005000000000000
+ QUAD $0x0006000000000000
+ QUAD $0x0007000000000000
+ QUAD $0x0100000000000000
+ QUAD $0x0080000000000000
+ QUAD $0x0100000000000000
+ QUAD $0x0080000000000000
+ QUAD $0xe0a8000000000000
+ QUAD $0xc150f1bbcdcbfa53
+ QUAD $0xa1f8e3779b97f4a7
+ QUAD $0x7c15d5336963eefb
+ QUAD $0xe5b99e3779b97f4a
+ QUAD $0x11ebbf58476d1ce4
+ QUAD $0x002094d049bb1331
+ QUAD $0x0008000000000000
+ QUAD $0x7c15000000000000
+ QUAD $0xe5b99e3779b97f4a
+ QUAD $0x11ebbf58476d1ce4
+ QUAD $0x000094d049bb1331
+ QUAD $0x00083ca000000000
+ QUAD $0x7c15000000000000
+ QUAD $0xe5b99e3779b97f4a
+ QUAD $0x11ebbf58476d1ce4
+ QUAD $0x001094d049bb1331
+ QUAD $0x0004000000000000
+ LONG $0x00000000
+ BYTE $0x00
+ BYTE $0x00
RET
// func normFloat64AVX512(a []float64) float64
@@ -647,24 +1166,110 @@ TEXT ·polyEvalFloat64AVX512(SB), NOSPLIT|NOFRAME, $0-72
MOVQ x_len+32(FP), R8
MOVQ coeffs_len+56(FP), R9
QUAD $0x4dc14c0f4cc1394c
- QUAD $0x000000f7840fc985
- QUAD $0x0002548e0fc0854d
- QUAD $0xf8ca44107ba1c400
- QUAD $0x0f02e88348c8894c
- QUAD $0x894855000000f08c
- QUAD $0x8d41f8e4834853e5
- QUAD $0xe1834107e183ff49
- QUAD $0x2e661cebd2314507
+ QUAD $0xe58948551a74c985
+ QUAD $0xe483485356415741
+ QUAD $0x31217d10f88349f8
+ QUAD $0x854d000002ade9c0
+ QUAD $0x49000004d48e0fc0
+ QUAD $0x0001a1830f04f883
+ QUAD $0x000004b3e9c03100
+ QUAD $0x49f8ca44107ba1c4
+ QUAD $0x0001628e0f01f983
+ QUAD $0x518d45fe498d4900
+ QUAD $0x41c01948fdf262ff
+ QUAD $0x8341cb894503e283
+ QUAD $0x00000010be4103e3
+ QUAD $0x2e6666662bebdb31
QUAD $0x0000000000841f0f
- QUAD $0xff49d714117ba1c4
- QUAD $0x00c8840fc2394dc2
- QUAD $0xd60c107ba1c40000
- QUAD $0x41d028f9c5c38949
- QUAD $0xcb89481d7401f983
- QUAD $0x90d028f9c5c38949
+ QUAD $0x62df1c1148fdf162
+ QUAD $0x4c01df641148fdf1
+ QUAD $0x394dc3894810708d
+ QUAD $0x4c000002348f0fc6
+ QUAD $0x0c1048fdf162f089
+ QUAD $0xde541048fdf162de
+ QUAD $0x48fdf162ce894901
+ QUAD $0xe02848fdf162d828
+ QUAD $0x894d417401fb8341
+ QUAD $0x48fdf162ce8949d7
+ QUAD $0xe02848fdf162d828
+ QUAD $0x0000000000841f0f
+ QUAD $0x62f22c1948fdb262
+ QUAD $0xd5f162d95948e5f1
+ QUAD $0x5948ddf162db5848
+ QUAD $0x49e45848d5f162e2
+ QUAD $0x48d975cfff49ceff
+ QUAD $0xffff6f820f03f983
+ QUAD $0x2e666604c68349ff
+ QUAD $0x0000000000841f0f
+ QUAD $0xfcf26c1948fdb262
+ QUAD $0xf162d95948e5f162
+ QUAD $0x48ddf162db5848d5
+ QUAD $0xe45848d5f162e259
+ QUAD $0xfbf26c1948fdb262
+ QUAD $0xf162d95948e5f162
+ QUAD $0x48ddf162db5848d5
+ QUAD $0xe45848d5f162e259
+ QUAD $0xfaf26c1948fdb262
+ QUAD $0xf162d95948e5f162
+ QUAD $0x48ddf162db5848d5
+ QUAD $0xe45848d5f162e259
+ QUAD $0xf9f26c1948fdb262
+ QUAD $0xf162d95948e5f162
+ QUAD $0x48ddf162db5848d5
+ QUAD $0xe45848d5f162e259
+ QUAD $0x03fe8349fcc68349
+ QUAD $0xcde9ffffff728f0f
+ QUAD $0x49f0408d49fffffe
+ QUAD $0xff4904eac149c289
+ QUAD $0x4807e183d18944c2
+ QUAD $0x10ba41287370f883
+ QUAD $0x00cce9c031000000
+ QUAD $0xffffffe0b9480000
+ QUAD $0x20f883497fffffff
+ QUAD $0xc031000001e4830f
+ QUAD $0xe2834900000226e9
+ QUAD $0x31c81948fdf262f8
+ QUAD $0x2e666666666666c0
+ QUAD $0x0000000000841f0f
+ QUAD $0x62c70c1148fdf162
+ QUAD $0x6201c74c1148fdf1
+ QUAD $0x6202c74c1148fdf1
+ QUAD $0x6203c74c1148fdf1
+ QUAD $0x6204c74c1148fdf1
+ QUAD $0x6205c74c1148fdf1
+ QUAD $0x6206c74c1148fdf1
+ QUAD $0x6207c74c1148fdf1
+ QUAD $0x6208c74c1148fdf1
+ QUAD $0x6209c74c1148fdf1
+ QUAD $0x620ac74c1148fdf1
+ QUAD $0x620bc74c1148fdf1
+ QUAD $0x620cc74c1148fdf1
+ QUAD $0x620dc74c1148fdf1
+ QUAD $0x620ec74c1148fdf1
+ QUAD $0x480fc74c1148fdf1
+ QUAD $0x0ff8c2834980e883
+ QUAD $0xc98548ffffff7385
+ QUAD $0xf26210508d4c3174
+ QUAD $0x9004e1c1c01948fd
+ QUAD $0x62c7041148fdf162
+ QUAD $0x4c01c7441148fdf1
+ QUAD $0x834810c28349d089
+ QUAD $0xf0c28349e475f0c1
+ QUAD $0x2948c1894cd0894c
+ QUAD $0xc4000001fe8e0fc1
+ QUAD $0x894df8ca44107ba1
+ QUAD $0xc48c0f02ea8349ca
+ QUAD $0x83ff498d41000000
+ QUAD $0x11eb07e1834107e1
+ QUAD $0xc0ff48c71411fbc5
+ QUAD $0x0001cc840fc0394c
+ QUAD $0x894dc60c10fbc500
+ QUAD $0xf98341d028f9c5d3
+ QUAD $0x894dcb89481f7401
+ QUAD $0x001f0fd028f9c5d3
QUAD $0x586ba1c4d259f3c5
QUAD $0xcbff48cbff49da14
- QUAD $0xb87207f88348ee75
+ QUAD $0xb87207fa8349ee75
QUAD $0x00401f0f08c38349
QUAD $0x586ba1c4d259f3c5
QUAD $0xc4d259f3c5c0da54
@@ -678,52 +1283,51 @@ TEXT ·polyEvalFloat64AVX512(SB), NOSPLIT|NOFRAME, $0-72
QUAD $0x59f3c590da54586b
QUAD $0x88da54586ba1c4d2
QUAD $0x07fb8349f8c38349
- QUAD $0x4dffffff49e99e7f
- QUAD $0x0000015d8e0fc085
- QUAD $0xc0311b7304f88349
- QUAD $0xf8834900000140e9
- QUAD $0x0129e9c031227304
- QUAD $0x5d5bf8658d480000
- QUAD $0xffffffffe0b948c3
- QUAD $0x7320f883497fffff
- QUAD $0xe0b94866ebc0311b
- QUAD $0x497fffffffffffff
- QUAD $0xe9c0317e7320f883
- QUAD $0x48c0894c000000c7
- QUAD $0xc057f9c5d231c821
- QUAD $0x0000000000841f0f
- QUAD $0x62d7041148fdf162
- QUAD $0x6201d7441148fdf1
- QUAD $0x6202d7441148fdf1
- QUAD $0x4803d7441148fdf1
+ QUAD $0x48ffffff49e99e7f
+ QUAD $0xc18948087304f983
+ QUAD $0xf983480000010be9
+ QUAD $0x00c0e9d231787320
+ QUAD $0xc82148c0894c0000
+ QUAD $0x9066c057f8c5d231
+ QUAD $0x62d70411487cf162
+ QUAD $0x6201d74411487cf1
+ QUAD $0x6202d74411487cf1
+ QUAD $0x4803d74411487cf1
QUAD $0xd875d0394820c283
- QUAD $0x0000cc840fc03949
- QUAD $0xb2840f1cc0f64100
+ QUAD $0x0000f2840fc03949
+ QUAD $0xd8840f1cc0f64100
QUAD $0x8348c28948000000
QUAD $0xc0214cc889481cc1
- QUAD $0x00401f0fc057f9c5
- QUAD $0xc28348d70411fdc5
- QUAD $0x95e9f275d0394804
- QUAD $0x2148c0894c000000
- QUAD $0x31c81948fdf262c8
- QUAD $0x2e666666666666d2
- QUAD $0x0000000000841f0f
- QUAD $0x62d70c1148fdf162
- QUAD $0x6201d74c1148fdf1
- QUAD $0x6202d74c1148fdf1
- QUAD $0x4803d74c1148fdf1
- QUAD $0xd875d0394820c283
- QUAD $0xc0f6415074c03949
- QUAD $0x8348c2894830741c
- QUAD $0xc0214cc889481cc1
- QUAD $0x2e6666c8197de2c4
- QUAD $0x0000000000841f0f
+ QUAD $0x00401f0fc057f8c5
+ QUAD $0xc28348d70411fcc5
+ QUAD $0xbbe9f275d0394804
+ QUAD $0x8348ca8948000000
+ QUAD $0xc81948fdf262e0e2
+ QUAD $0xc0c68148c7348d48
+ QUAD $0x9066c93145000000
+ QUAD $0xfdce4c1148fdb162
+ QUAD $0xfece4c1148fdb162
+ QUAD $0xffce4c1148fdb162
+ QUAD $0x49ce0c1148fdb162
+ QUAD $0xd875ca394c20c183
+ QUAD $0x1cc1f65074d13948
+ QUAD $0x03e683c689445974
+ QUAD $0xe2c4c10148f12948
+ QUAD $0x894cc20148c8197d
+ QUAD $0x001f0ffce08348c0
QUAD $0xc28348d70c11fdc5
- QUAD $0x3949f275d0394804
- QUAD $0xc70411fbc51a74c0
- QUAD $0x04c748f1ebc0ff48
- QUAD $0xc0ff4800000000c7
- QUAD $0x77f8c5f075c03949
+ QUAD $0x8548f275d0394804
+ QUAD $0x2e666666661a74f6
+ QUAD $0x0000000000841f0f
+ QUAD $0xc1ff48cf0411fbc5
+ QUAD $0x658d48f375c83949
+ QUAD $0xc55d5f415e415be8
+ QUAD $0x8948d00148c377f8
+ QUAD $0x00c704c748ddebc1
+ QUAD $0x3949c0ff48000000
+ LONG $0xc5f075c0
+ BYTE $0xf8
+ BYTE $0x77
BYTE $0xc3
RET
@@ -735,102 +1339,318 @@ TEXT ·convolveFloat64AVX512(SB), NOSPLIT|NOFRAME, $0-72
MOVQ dst_len+8(FP), CX
MOVQ sig_len+32(FP), R8
MOVQ ker_len+56(FP), R9
+ // Constant pool appended below the body; the displacements
+ // above were patched to reach it and the aligned loads made
+ // unaligned, since nothing promises alignment inside a TEXT.
QUAD $0x56415741e5894855
QUAD $0xe483485354415541
- QUAD $0x74894838ec8348f8
- QUAD $0xc0940fc9854d0824
- QUAD $0x40c69c0f40c8294d
- QUAD $0x000002b0850fc608
- QUAD $0x48c1394c01708d49
- QUAD $0x8e0ff68548f14e0f
- QUAD $0x0fc9854d0000029c
- QUAD $0x0c8d4a000001f18e
- QUAD $0x07e08341c88945ca
- QUAD $0xffffffffffe0ba49
- QUAD $0x214d18ca83497fff
- QUAD $0x8d4c0824448b48ca
- QUAD $0x8348ca048d4a3858
- QUAD $0x311024448948f8c0
- QUAD $0xfffffffffdbf49db
- QUAD $0xfffffcbc491fffff
- QUAD $0xfbbd491fffffffff
- QUAD $0x481fffffffffffff
- QUAD $0xfffffffffffffab8
- QUAD $0xfffffffff9ba481f
- QUAD $0xfffff8be491fffff
- QUAD $0x7c89481fffffffff
- QUAD $0x4c20247489482824
- QUAD $0x906634eb18244c89
- QUAD $0x5c8b4828247c8b48
- QUAD $0x48df0411fbc53024
- QUAD $0x834908c38349c3ff
- QUAD $0x4808244c894c08c1
- QUAD $0x4cf339482024748b
- QUAD $0x01de840f18244c8b
- QUAD $0xc530245c89480000
- QUAD $0x7308f98349c057f9
- QUAD $0x08244c8b4cdb310f
- QUAD $0x001f0f000000eae9
- QUAD $0x49db3110247c8b48
- QUAD $0xfffffffffffffeb9
- QUAD $0x2e6666666666661f
+ QUAD $0xc9854d38ec8348f8
+ QUAD $0x28244c894cc0940f
+ QUAD $0x41c19c0f41c8294d
+ QUAD $0x0000061a850fc108
+ QUAD $0x48c1394c01408d49
+ QUAD $0x4810f88348c14e0f
+ QUAD $0x2444894830247c89
+ QUAD $0x8948202474894810
+ QUAD $0xf63145087d182454
+ QUAD $0x4c8b4800000325e9
+ QUAD $0xc78e0fc985482824
+ QUAD $0x41ca048d4c000001
+ QUAD $0xbb4903e28341ca89
+ QUAD $0x7ffffffffffffff8
+ QUAD $0x48cb214904cb8349
+ QUAD $0x48ca148d48585e8d
+ QUAD $0x00000010b9f8c283
+ QUAD $0xfffffffebf49c031
+ QUAD $0xfffdbc491fffffff
+ QUAD $0xbd491fffffffffff
+ QUAD $0x1ffffffffffffffc
+ QUAD $0x0000441f0f6634eb
+ QUAD $0xfdf16230247c8b48
+ QUAD $0x48fdf162c7041148
+ QUAD $0x104e8d4901c74c11
+ QUAD $0x80ee834880eb8348
+ QUAD $0x10244c3b48f0894c
+ QUAD $0x8949000002948f0f
+ QUAD $0x730428247c8348ce
+ QUAD $0xc5c931c057f9c517
+ QUAD $0x000000e0e9c957f1
QUAD $0x0000000000841f0f
- QUAD $0xc4c8db4c107bc1c4
- QUAD $0xf3c5d0db54107bc1
- QUAD $0x8948c158fbc50f59
- QUAD $0x0c59ebc5ce314cde
- QUAD $0x7bc1c4c158fbc5f1
- QUAD $0x4cde8948d8db4c10
- QUAD $0xc5f10c59f3c5fe31
- QUAD $0x4c107bc1c4c158fb
- QUAD $0xe6314cde8948e0db
- QUAD $0x58fbc5f10c59f3c5
- QUAD $0xe8db4c107bc1c4c1
- QUAD $0xf3c5ee314cde8948
- QUAD $0xc4c158fbc5f10c59
- QUAD $0x8948f0db4c107bc1
- QUAD $0x0c59f3c5c63148de
- QUAD $0x7bc1c4c158fbc5f1
- QUAD $0x48de8948f8db4c10
- QUAD $0xc5f10c59f3c5d631
- QUAD $0x0c107bc1c4c158fb
- QUAD $0xc5f6314cde8948db
- QUAD $0xc158fbc5f10c59f3
- QUAD $0xc0c7834808c38348
- QUAD $0xffff47850fda3949
- QUAD $0x244c8b4cc0854dff
- QUAD $0x48fffffec9840f08
- QUAD $0x4800000000dd348d
- QUAD $0x49f7294810247c8b
- QUAD $0x001f0ff631d91c8d
- QUAD $0x59f3c5f30c10fbc5
+ QUAD $0x31d78948c057f9c5
+ QUAD $0x001f0fc957f1c5c9
+ QUAD $0xf162171948fdf262
+ QUAD $0xffffa8cb9c5948ed
+ QUAD $0x62c35848fdf162ff
+ QUAD $0xffe8cb945948edf1
+ QUAD $0xca5848f5f162ffff
+ QUAD $0x9262f9314dc98949
+ QUAD $0xedf162c8141948fd
+ QUAD $0xffffffb0cb9c5948
+ QUAD $0xf162c35848fdf162
+ QUAD $0xfffff0cb945948ed
+ QUAD $0x49ca5848f5f162ff
+ QUAD $0xfd9262e1314dc989
+ QUAD $0x48edf162c8141948
+ QUAD $0x62ffffffb8cb9c59
+ QUAD $0xedf162c35848fdf1
+ QUAD $0xfffffff8cb945948
+ QUAD $0x8949ca5848f5f162
+ QUAD $0x48fd9262e9314dc9
+ QUAD $0x5948edf162c81419
+ QUAD $0x5848fdf162ffcb5c
+ QUAD $0xcb145948edf162c3
+ QUAD $0x8348ca5848f5f162
+ QUAD $0x3949e0c7834804c1
+ QUAD $0x4dffffff41850fcb
+ QUAD $0xfffffed8840fd285
+ QUAD $0x03e1c148ce3c8d48
+ QUAD $0xc931c92949d18949
+ QUAD $0x0000000000841f0f
+ QUAD $0xf162111948fdd262
+ QUAD $0xfdf162cf1c5948ed
+ QUAD $0x5948edf162c35848
+ QUAD $0x5848f5f16201cf54
+ QUAD $0xf8c18349c1ff48ca
+ QUAD $0xfe8ee9d375ca3949
+ QUAD $0x8348f0488d48ffff
+ QUAD $0x000010b80d7370f9
+ QUAD $0x00010ee9f6314500
+ QUAD $0xc1ff4804e9c14800
+ QUAD $0x49f8e68348ce8948
+ QUAD $0x8d4904e6c149f689
+ QUAD $0x051048fdf1621046
+ QUAD $0xc957f1c500000477
+ QUAD $0x0671155948fdf262
+ QUAD $0xf78948f889490000
+ QUAD $0x49fdd262c846fdc5
+ QUAD $0x62c846fdc5c00ca3
+ QUAD $0xc501c04ca349fdd2
+ QUAD $0xa349fdd262c846fd
+ QUAD $0x62c846fdc502c04c
+ QUAD $0xc503c04ca349fdd2
+ QUAD $0xa349fdd262c846fd
+ QUAD $0x62c846fdc504c04c
+ QUAD $0xc505c04ca349fdd2
+ QUAD $0xa349fdd262c846fd
+ QUAD $0x62c846fdc506c04c
+ QUAD $0xc507c04ca349fdd2
+ QUAD $0xa349fdd262c846fd
+ QUAD $0x62c846fdc508c04c
+ QUAD $0xc509c04ca349fdd2
+ QUAD $0xa349fdd262c846fd
+ QUAD $0x62c846fdc50ac04c
+ QUAD $0xc50bc04ca349fdd2
+ QUAD $0xa349fdd262c846fd
+ QUAD $0x62c846fdc50cc04c
+ QUAD $0xc50dc04ca349fdd2
+ QUAD $0xa349fdd262c846fd
+ QUAD $0x62c846fdc50ec04c
+ QUAD $0x620fc04ca349fdd2
+ QUAD $0xc78348c2d448fdf1
+ QUAD $0x48ffffff31850ff8
+ QUAD $0xc52b74c7894cf139
+ QUAD $0x1024748b48c057f9
+ QUAD $0x62f7041148fdb162
+ QUAD $0x4901f7441148fdb1
+ QUAD $0x394810c08348c689
+ QUAD $0x49f0c08348e57ef0
+ QUAD $0x481024748b48c689
+ QUAD $0x4c082474894cf089
+ QUAD $0x4c2024548b48f029
+ QUAD $0x02a88e0f18244c8b
+ QUAD $0x4828244c8b480000
+ QUAD $0x000001ce8e0fc985
+ QUAD $0x41ca8941c9048d4d
+ QUAD $0xfffff8bb4907e283
+ QUAD $0xcb21497fffffffff
+ QUAD $0x1c8d480824448b48
+ QUAD $0x4c8d4938c38348c2
+ QUAD $0x4818244c8948f8c9
+ QUAD $0x2024448948c2048d
+ QUAD $0xfffffffffffebe49
+ QUAD $0xfffffffdbf491fff
+ QUAD $0xfffcbc491fffffff
+ QUAD $0xbd491fffffffffff
+ QUAD $0x1ffffffffffffffb
+ QUAD $0xfffffffffffab848
+ QUAD $0xfffffff9ba481fff
+ QUAD $0xfff8b9491fffffff
+ QUAD $0x2ceb1fffffffffff
+ QUAD $0x748b4830244c8b48
+ QUAD $0x48f10411fbc50824
+ QUAD $0x834808c38348c6ff
+ QUAD $0x2474894808202444
+ QUAD $0x840f1024743b4808
+ QUAD $0x247c8348000001ee
+ QUAD $0xc057f9c50c730828
+ QUAD $0x90000000d2e9c931
+ QUAD $0x247c8b48c057f9c5
+ QUAD $0x0000441f0fc93118
+ QUAD $0xfbc5c8cb4c10fbc5
+ QUAD $0x0f59f3c5d0cb5410
+ QUAD $0x4cce8948c158fbc5
+ QUAD $0xf00c596bc1c4f631
+ QUAD $0x4c10fbc5c158fbc5
+ QUAD $0xfe314cce8948d8cb
+ QUAD $0xfbc5f00c5973c1c4
+ QUAD $0xe0cb4c10fbc5c158
+ QUAD $0xc1c4e6314cce8948
+ QUAD $0xc158fbc5f00c5973
+ QUAD $0x8948e8cb4c10fbc5
+ QUAD $0x5973c1c4ee314cce
+ QUAD $0xfbc5c158fbc5f00c
+ QUAD $0x48ce8948f0cb4c10
+ QUAD $0xf00c5973c1c4c631
+ QUAD $0x4c10fbc5c158fbc5
+ QUAD $0xd63148ce8948f8cb
+ QUAD $0xfbc5f00c5973c1c4
+ QUAD $0x48cb0c10fbc5c158
+ QUAD $0x73c1c4ce314cce89
+ QUAD $0x48c158fbc5f00c59
+ QUAD $0x49c0c7834808c183
+ QUAD $0xffffff48850fcb39
+ QUAD $0xfffeef840fd2854d
+ QUAD $0x000000cd348d48ff
+ QUAD $0x294818247c8b4800
+ QUAD $0x8d482024748b48f7
+ QUAD $0x00401f0ff631ce0c
+ QUAD $0x59f3c5f10c10fbc5
QUAD $0xc6ff48c158fbc50f
- QUAD $0x75f03949f8c78348
- QUAD $0x8348fffffe92e9e7
- QUAD $0x88e9c031077304fe
- QUAD $0xffffe0b948000000
- QUAD $0xfe83487fffffffff
- QUAD $0x4844ebc031047320
- QUAD $0xc5d231c82148f089
- QUAD $0x0000441f0fc057f9
- QUAD $0x62d7041148fdf162
- QUAD $0x6201d7441148fdf1
- QUAD $0x6202d7441148fdf1
- QUAD $0x4803d7441148fdf1
- QUAD $0xd875d0394820c283
- QUAD $0xc6f6404374c63948
- QUAD $0x8348c2894830741c
- QUAD $0xf02148c889481cc1
- QUAD $0x2e666666c057f9c5
+ QUAD $0x75f23949f8c78348
+ QUAD $0x8348fffffeb2e9e7
+ QUAD $0x000000a8820f04f8
+ QUAD $0x24448b4c20f88348
+ QUAD $0x4854ebc931047308
+ QUAD $0x8d4ae0e18348c189
+ QUAD $0x0000c0c28148c714
+ QUAD $0x90c057f9c5f63100
+ QUAD $0xfdf2441148fdf162
+ QUAD $0xfef2441148fdf162
+ QUAD $0xfff2441148fdf162
+ QUAD $0x48f2041148fdf162
+ QUAD $0xd875f1394820c683
+ QUAD $0x1024748b48c83948
+ QUAD $0x8b4c76741ca86874
+ QUAD $0x8348c28948082444
+ QUAD $0x0149c7348d4afce2
+ QUAD $0xf9c5082444894cd0
+ QUAD $0x2e6666666666c057
QUAD $0x0000000000841f0f
- QUAD $0xc28348d70411fdc5
- QUAD $0x3948f275d0394804
- QUAD $0x00c704c7480d74c6
- QUAD $0xeeebc0ff48000000
- QUAD $0x415c415bd8658d48
- QUAD $0xf8c55d5f415e415d
- BYTE $0x77
- BYTE $0xc3
+ QUAD $0xc18348ce0411fdc5
+ QUAD $0x3948f275ca394804
+ QUAD $0x22741024748b48d0
+ QUAD $0x0000000000841f0f
+ QUAD $0x04c7480824448b48
+ QUAD $0xc0ff4800000000c7
+ QUAD $0xc639480824448948
+ QUAD $0x415bd8658d48e675
+ QUAD $0x5d5f415e415d415c
+ QUAD $0x244c0148c377f8c5
+ QUAD $0x0000000080cdeb08
+ QUAD $0x0000000090000000
+ QUAD $0x00000000a0000000
+ QUAD $0x00000000b0000000
+ QUAD $0x00000000c0000000
+ QUAD $0x00000000d0000000
+ QUAD $0x00000000e0000000
+ QUAD $0x00000000f0000000
+ QUAD $0x0000000000000000
+ QUAD $0x0000000010000000
+ QUAD $0x0000000020000000
+ QUAD $0x0000000030000000
+ QUAD $0x0000000040000000
+ QUAD $0x0000000050000000
+ QUAD $0x0000000060000000
+ QUAD $0x0000000070000000
+ QUAD $0x0000000000000000
+ QUAD $0x0000000010000000
+ QUAD $0x0000000020000000
+ QUAD $0x0000000030000000
+ QUAD $0x0000000040000000
+ QUAD $0x0000000050000000
+ QUAD $0x0000000060000000
+ QUAD $0x0000000070000000
+ QUAD $0x0000000080000000
+ QUAD $0x0000000090000000
+ QUAD $0x00000000a0000000
+ QUAD $0x00000000b0000000
+ QUAD $0x00000000c0000000
+ QUAD $0x00000000d0000000
+ QUAD $0x00000000e0000000
+ QUAD $0x00000000f0000000
+ QUAD $0x0000000000000000
+ QUAD $0x0000000010000000
+ QUAD $0x0000000020000000
+ QUAD $0x0000000030000000
+ QUAD $0x0000000040000000
+ QUAD $0x0000000050000000
+ QUAD $0x0000000060000000
+ QUAD $0x0000000070000000
+ QUAD $0x0000000000000000
+ QUAD $0x0000000010000000
+ QUAD $0x0000000020000000
+ QUAD $0x0000000030000000
+ QUAD $0x0000000040000000
+ QUAD $0x0000000050000000
+ QUAD $0x0000000060000000
+ QUAD $0x0000000070000000
+ QUAD $0x0000000000000000
+ QUAD $0x0000000001000000
+ QUAD $0x0000000002000000
+ QUAD $0x0000000003000000
+ QUAD $0x0000000004000000
+ QUAD $0x0000000005000000
+ QUAD $0x0000000006000000
+ QUAD $0x0000000007000000
+ QUAD $0x0000000000000000
+ QUAD $0x0000000001000000
+ QUAD $0x0000000002000000
+ QUAD $0x0000000003000000
+ QUAD $0x0000000004000000
+ QUAD $0x0000000005000000
+ QUAD $0x0000000006000000
+ QUAD $0x0000000007000000
+ QUAD $0x0000000008000000
+ QUAD $0x0000000009000000
+ QUAD $0x000000000a000000
+ QUAD $0x000000000b000000
+ QUAD $0x000000000c000000
+ QUAD $0x000000000d000000
+ QUAD $0x000000000e000000
+ QUAD $0x000000000f000000
+ QUAD $0x0000000000000000
+ QUAD $0x0000000001000000
+ QUAD $0x0000000002000000
+ QUAD $0x0000000003000000
+ QUAD $0x0000000004000000
+ QUAD $0x0000000005000000
+ QUAD $0x0000000006000000
+ QUAD $0x0000000007000000
+ QUAD $0x0000000100000000
+ QUAD $0x0000000080000000
+ QUAD $0x0000000100000000
+ QUAD $0x0000000080000000
+ QUAD $0xcbfa53e0a8000000
+ QUAD $0x97f4a7c150f1bbcd
+ QUAD $0x63eefba1f8e3779b
+ QUAD $0xb97f4a7c15d53369
+ QUAD $0x6d1ce4e5b99e3779
+ QUAD $0xbb133111ebbf5847
+ QUAD $0x000000002094d049
+ QUAD $0x0000000008000000
+ QUAD $0xb97f4a7c15000000
+ QUAD $0x6d1ce4e5b99e3779
+ QUAD $0xbb133111ebbf5847
+ QUAD $0x000000000094d049
+ QUAD $0x00000000083ca000
+ QUAD $0xb97f4a7c15000000
+ QUAD $0x6d1ce4e5b99e3779
+ QUAD $0xbb133111ebbf5847
+ QUAD $0x000000001094d049
+ QUAD $0x0000000004000000
+ BYTE $0x00
+ BYTE $0x00
+ BYTE $0x00
RET
// func correlateFloat64AVX512(dst []float64, sig []float64, ker []float64)
@@ -841,73 +1661,277 @@ TEXT ·correlateFloat64AVX512(SB), NOSPLIT|NOFRAME, $0-72
MOVQ dst_len+8(FP), CX
MOVQ sig_len+32(FP), R8
MOVQ ker_len+56(FP), R9
+ // Constant pool appended below the body; the displacements
+ // above were patched to reach it and the aligned loads made
+ // unaligned, since nothing promises alignment inside a TEXT.
QUAD $0x294dc0940fc9854d
QUAD $0xc20841c29c0f41c8
- QUAD $0x8d49000001fa850f
- QUAD $0x4e0f48c1394c0140
- QUAD $0x01e68e0fc08548c1
- QUAD $0x358e0fc9854d0000
- QUAD $0x41e5894855000001
- QUAD $0xf8e4834853564157
- QUAD $0xb84907e183c98944
- QUAD $0x7fffffffffffffe0
- QUAD $0x4cc8214d18c88349
- QUAD $0x1aebdb314538568d
- QUAD $0xff49df04117ba1c4
- QUAD $0xc6834808c28349c3
- QUAD $0x00e3840fc3394908
- QUAD $0xdb31c057f9c50000
- QUAD $0x00a2820f08f98349
- QUAD $0x0000441f0f660000
- QUAD $0xc4c8da4c107bc1c4
- QUAD $0xf3c5d0da54107bc1
+ QUAD $0x4855000004f8850f
+ QUAD $0x554156415741e589
+ QUAD $0x49f8e48348535441
+ QUAD $0x0f48c1394c01408d
+ QUAD $0x077d10f88348c14e
+ QUAD $0x4d000002d3e9c931
+ QUAD $0x000001778e0fc985
+ QUAD $0x4903e08341c88945
+ QUAD $0xfffffffffffff8ba
+ QUAD $0xca214d04ca83497f
+ QUAD $0x0010bf41585e8d4c
+ QUAD $0xf38948f631450000
+ QUAD $0x2e666666666635eb
+ QUAD $0x0000000000841f0f
+ QUAD $0x62f7041148fdb162
+ QUAD $0x4c01f74c1148fdb1
+ QUAD $0x4880eb834910798d
+ QUAD $0x3949ce894980eb83
+ QUAD $0x4c0000026b8f0fc7
+ QUAD $0x3145c057f9c5f989
+ QUAD $0xf98349c957f1c5ff
+ QUAD $0x90000000b7820f04
+ QUAD $0x62fa141948fdb262
+ QUAD $0xffa8fb9c5948ed91
+ QUAD $0xc35848fdf162ffff
+ QUAD $0xe8fb945948ed9162
+ QUAD $0x5848f5f162ffffff
+ QUAD $0xfa541948fdb262ca
+ QUAD $0xfb9c5948ed916201
+ QUAD $0x48fdf162ffffffb0
+ QUAD $0x945948ed9162c358
+ QUAD $0xf5f162fffffff0fb
+ QUAD $0x1948fdb262ca5848
+ QUAD $0x5948ed916202fa54
+ QUAD $0x9162ffffffb8fb9c
+ QUAD $0xfffff8fb945948ed
+ QUAD $0x62c35848fdf162ff
+ QUAD $0xfdb262ca5848f5f1
+ QUAD $0xed916203fa541948
+ QUAD $0xfdf162fffb5c5948
+ QUAD $0x5948ed9162c35848
+ QUAD $0xca5848f5f162fb14
+ QUAD $0x0ffa394d04c78349
+ QUAD $0xc0854dffffff5385
+ QUAD $0x8d4effffff0a840f
+ QUAD $0x3145fa3c8d4efb24
+ QUAD $0x2e666666666666ed
+ QUAD $0x0000000000841f0f
+ QUAD $0x62ef141948fd9262
+ QUAD $0xf162ec1c5948ed91
+ QUAD $0x48ed9162c35848fd
+ QUAD $0x48f5f16201ec5459
+ QUAD $0xe8394dc5ff49ca58
+ QUAD $0x4cfffffec1e9d675
+ QUAD $0x7370fa8349f0508d
+ QUAD $0x3100000010b8410d
+ QUAD $0xc1490000010ee9c9
+ QUAD $0xd3894dc2ff4904ea
+ QUAD $0x48d9894cf8e38349
+ QUAD $0x6210418d4c04e1c1
+ QUAD $0x00045e051048fdf1
+ QUAD $0xfdf262c957f1c500
+ QUAD $0x4c000005a8155948
+ QUAD $0x0000441f0f66db89
+ QUAD $0x49fdf262c846fdc5
+ QUAD $0x62c846fdc5c70ca3
+ QUAD $0xc501c74ca349fdf2
+ QUAD $0xa349fdf262c846fd
+ QUAD $0x62c846fdc502c74c
+ QUAD $0xc503c74ca349fdf2
+ QUAD $0xa349fdf262c846fd
+ QUAD $0x62c846fdc504c74c
+ QUAD $0xc505c74ca349fdf2
+ QUAD $0xa349fdf262c846fd
+ QUAD $0x62c846fdc506c74c
+ QUAD $0xc507c74ca349fdf2
+ QUAD $0xa349fdf262c846fd
+ QUAD $0x62c846fdc508c74c
+ QUAD $0xc509c74ca349fdf2
+ QUAD $0xa349fdf262c846fd
+ QUAD $0x62c846fdc50ac74c
+ QUAD $0xc50bc74ca349fdf2
+ QUAD $0xa349fdf262c846fd
+ QUAD $0x62c846fdc50cc74c
+ QUAD $0xc50dc74ca349fdf2
+ QUAD $0xa349fdf262c846fd
+ QUAD $0x62c846fdc50ec74c
+ QUAD $0x620fc74ca349fdf2
+ QUAD $0xc38348c2d448fdf1
+ QUAD $0x4dffffff31850ff8
+ QUAD $0xc057f9c52e74da39
+ QUAD $0x0000000000841f0f
+ QUAD $0x62cf041148fdf162
+ QUAD $0x4c01cf441148fdf1
+ QUAD $0x394910c08349c189
+ QUAD $0x4cf0c08349e57ec0
+ QUAD $0xc82949c08949c189
+ QUAD $0x854d000001e28e0f
+ QUAD $0x45000001268e0fc9
+ QUAD $0xba4907e08341c889
+ QUAD $0x7ffffffffffffff8
+ QUAD $0x49ce1c8d4cca214d
+ QUAD $0xebce348d4838c383
+ QUAD $0x00000000801f0f20
+ QUAD $0xc1ff48cf0411fbc5
+ QUAD $0x08c6834808c38349
+ QUAD $0x000197840fc13948
+ QUAD $0x49db31c057f9c500
+ QUAD $0x0000a3820f08f983
+ QUAD $0x00000000801f0f00
+ QUAD $0xc4c8db4c107bc1c4
+ QUAD $0xf3c5d0db54107bc1
QUAD $0xc5c158fbc5da0c59
QUAD $0x58fbc508da4c59eb
- QUAD $0xd8da4c107bc1c4c1
+ QUAD $0xd8db4c107bc1c4c1
QUAD $0xc1c410da4c59f3c5
- QUAD $0x59ebc5e0da54107b
+ QUAD $0x59ebc5e0db54107b
QUAD $0xc5c158fbc518da54
QUAD $0x4c107bc1c4c258fb
- QUAD $0x20da4c59f3c5e8da
+ QUAD $0x20da4c59f3c5e8db
QUAD $0x107bc1c4c158fbc5
- QUAD $0xda4c59f3c5f0da4c
+ QUAD $0xda4c59f3c5f0db4c
QUAD $0x7bc1c4c158fbc528
- QUAD $0x4c59f3c5f8da4c10
- QUAD $0xda14107bc1c430da
+ QUAD $0x4c59f3c5f8db4c10
+ QUAD $0xdb14107bc1c430da
QUAD $0x4c59ebc5c158fbc5
QUAD $0x8348c158fbc538da
- QUAD $0x6d850fd8394908c3
- QUAD $0x840fc98548ffffff
+ QUAD $0x6d850fda394908c3
+ QUAD $0x840fc0854dffffff
QUAD $0xda348d4cffffff34
QUAD $0x66ff3145de1c8d48
QUAD $0x0000000000841f0f
QUAD $0x81c4fb0c107ba1c4
QUAD $0xc158fbc5fe0c5973
- QUAD $0xe875f9394cc7ff49
- QUAD $0x658d48ffffff03e9
- QUAD $0xc35d5f415e415be8
- QUAD $0xc931077304f88348
- QUAD $0xe0ba480000008be9
- QUAD $0x487fffffffffffff
- QUAD $0xebc931047320f883
- QUAD $0x31d12148c1894848
- QUAD $0x2e6666c057f8c5f6
+ QUAD $0xe875f8394dc7ff49
+ QUAD $0xf88349ffffff03e9
+ QUAD $0x4900000099820f04
+ QUAD $0xebd231047320f883
+ QUAD $0xe0e28348c2894c52
+ QUAD $0xc0c68148cf348d48
+ QUAD $0xf9c5c93145000000
+ QUAD $0x0000441f0f66c057
+ QUAD $0xfdce441148fdb162
+ QUAD $0xfece441148fdb162
+ QUAD $0xffce441148fdb162
+ QUAD $0x49ce041148fdb162
+ QUAD $0xd875ca394c20c183
+ QUAD $0xc0f6415374d03949
+ QUAD $0x8348c6894c5f741c
+ QUAD $0x0148cf0c8d4cfce6
+ QUAD $0x2e6666c057f9c5f1
QUAD $0x0000000000841f0f
- QUAD $0x62f70411487cf162
- QUAD $0x6201f74411487cf1
- QUAD $0x6202f74411487cf1
- QUAD $0x4803f74411487cf1
- QUAD $0xd875f1394820c683
- QUAD $0x741ca84374c83948
- QUAD $0x1cc28348ce89482f
- QUAD $0xf8c5c12148d18948
- QUAD $0x2e6666666666c057
+ QUAD $0x8348d104117dc1c4
+ QUAD $0x49f175d6394804c2
+ QUAD $0x2e6666661c74f039
QUAD $0x0000000000841f0f
- QUAD $0xc68348f70411fcc5
- QUAD $0x0bebf275f1394804
QUAD $0x00000000cf04c748
QUAD $0xf075c83948c1ff48
- LONG $0xc377f8c5
+ QUAD $0x415c415bd8658d48
+ QUAD $0xf8c55d5f415e415d
+ QUAD $0x80d9ebd10148c377
+ QUAD $0x9000000000000000
+ QUAD $0xa000000000000000
+ QUAD $0xb000000000000000
+ QUAD $0xc000000000000000
+ QUAD $0xd000000000000000
+ QUAD $0xe000000000000000
+ QUAD $0xf000000000000000
+ QUAD $0x0000000000000000
+ QUAD $0x1000000000000000
+ QUAD $0x2000000000000000
+ QUAD $0x3000000000000000
+ QUAD $0x4000000000000000
+ QUAD $0x5000000000000000
+ QUAD $0x6000000000000000
+ QUAD $0x7000000000000000
+ QUAD $0x0000000000000000
+ QUAD $0x1000000000000000
+ QUAD $0x2000000000000000
+ QUAD $0x3000000000000000
+ QUAD $0x4000000000000000
+ QUAD $0x5000000000000000
+ QUAD $0x6000000000000000
+ QUAD $0x7000000000000000
+ QUAD $0x8000000000000000
+ QUAD $0x9000000000000000
+ QUAD $0xa000000000000000
+ QUAD $0xb000000000000000
+ QUAD $0xc000000000000000
+ QUAD $0xd000000000000000
+ QUAD $0xe000000000000000
+ QUAD $0xf000000000000000
+ QUAD $0x0000000000000000
+ QUAD $0x1000000000000000
+ QUAD $0x2000000000000000
+ QUAD $0x3000000000000000
+ QUAD $0x4000000000000000
+ QUAD $0x5000000000000000
+ QUAD $0x6000000000000000
+ QUAD $0x7000000000000000
+ QUAD $0x0000000000000000
+ QUAD $0x1000000000000000
+ QUAD $0x2000000000000000
+ QUAD $0x3000000000000000
+ QUAD $0x4000000000000000
+ QUAD $0x5000000000000000
+ QUAD $0x6000000000000000
+ QUAD $0x7000000000000000
+ QUAD $0x0000000000000000
+ QUAD $0x0100000000000000
+ QUAD $0x0200000000000000
+ QUAD $0x0300000000000000
+ QUAD $0x0400000000000000
+ QUAD $0x0500000000000000
+ QUAD $0x0600000000000000
+ QUAD $0x0700000000000000
+ QUAD $0x0000000000000000
+ QUAD $0x0100000000000000
+ QUAD $0x0200000000000000
+ QUAD $0x0300000000000000
+ QUAD $0x0400000000000000
+ QUAD $0x0500000000000000
+ QUAD $0x0600000000000000
+ QUAD $0x0700000000000000
+ QUAD $0x0800000000000000
+ QUAD $0x0900000000000000
+ QUAD $0x0a00000000000000
+ QUAD $0x0b00000000000000
+ QUAD $0x0c00000000000000
+ QUAD $0x0d00000000000000
+ QUAD $0x0e00000000000000
+ QUAD $0x0f00000000000000
+ QUAD $0x0000000000000000
+ QUAD $0x0100000000000000
+ QUAD $0x0200000000000000
+ QUAD $0x0300000000000000
+ QUAD $0x0400000000000000
+ QUAD $0x0500000000000000
+ QUAD $0x0600000000000000
+ QUAD $0x0700000000000000
+ QUAD $0x0000000000000000
+ QUAD $0x8000000000000001
+ QUAD $0x0000000000000000
+ QUAD $0x8000000000000001
+ QUAD $0xa800000000000000
+ QUAD $0x50f1bbcdcbfa53e0
+ QUAD $0xf8e3779b97f4a7c1
+ QUAD $0x15d5336963eefba1
+ QUAD $0xb99e3779b97f4a7c
+ QUAD $0xebbf58476d1ce4e5
+ QUAD $0x2094d049bb133111
+ QUAD $0x0800000000000000
+ QUAD $0x1500000000000000
+ QUAD $0xb99e3779b97f4a7c
+ QUAD $0xebbf58476d1ce4e5
+ QUAD $0x0094d049bb133111
+ QUAD $0x083ca00000000000
+ QUAD $0x1500000000000000
+ QUAD $0xb99e3779b97f4a7c
+ QUAD $0xebbf58476d1ce4e5
+ QUAD $0x1094d049bb133111
+ QUAD $0x0400000000000000
+ LONG $0x00000000
+ BYTE $0x00
+ BYTE $0x00
+ BYTE $0x00
RET
// func tileFloat32AVX512(dst []float32, pattern []float32)
@@ -3526,21 +4550,25 @@ TEXT ·movingAverageFloat32AVX512(SB), NOSPLIT|NOFRAME, $0-56
MOVQ width+48(FP), R8
QUAD $0x294cc09e0fc0854d
QUAD $0xc10841c19c0f41c1
- QUAD $0x8d48000000e3850f
- QUAD $0x4e0f48ca39480141
- QUAD $0x00cf8e0fc08548c2
- QUAD $0x4853e58948550000
- QUAD $0xc02a82c1c4f8e483
- QUAD $0x83c28944ff488d49
- QUAD $0xfffffff8b94907e2
- QUAD $0x4cc1214d7fffffff
- QUAD $0x22ebd231451c468d
+ QUAD $0x48550000010b850f
+ QUAD $0x554156415741e589
+ QUAD $0x48f8e48348535441
+ QUAD $0xfffffffffffff8b8
+ QUAD $0xca394801498d4c7f
+ QUAD $0x10f98349ca4e0f4c
+ QUAD $0xc931000000df8d0f
+ QUAD $0x0000c28d0fc9394c
+ QUAD $0x8d49c02a82c1c400
+ QUAD $0xe28341c28945ff50
+ QUAD $0x8e048d4cc0214c07
+ QUAD $0x8e348d481cc08349
+ QUAD $0x2e666666666627eb
QUAD $0x0000000000841f0f
- QUAD $0x117aa1c4c85ef2c5
- QUAD $0xc08349c2ff49970c
- QUAD $0xc2394904c6834804
- QUAD $0x3145c957f0c57974
- QUAD $0x664e7207f98348db
+ QUAD $0x0c11fac5c85ef2c5
+ QUAD $0x04c08349c1ff488f
+ QUAD $0x74c9394c04c68348
+ QUAD $0xdb3145c957f0c57a
+ QUAD $0x2e664f7207fa8348
QUAD $0x0000000000841f0f
QUAD $0xc4e4984c587281c4
QUAD $0x81c4e8984c587281
@@ -3549,14 +4577,46 @@ TEXT ·movingAverageFloat32AVX512(SB), NOSPLIT|NOFRAME, $0-56
QUAD $0x4c587281c4f4984c
QUAD $0x984c587281c4f898
QUAD $0x49980c587281c4fc
- QUAD $0xc075d9394d08c383
- QUAD $0x1c8d4e8b74d28548
+ QUAD $0xc075d8394c08c383
+ QUAD $0x1c8d4e8b74d2854d
QUAD $0x0000441f0fdb319e
QUAD $0xff489b0c5872c1c4
- QUAD $0x6de9f275da3948c3
- QUAD $0x5bf8658d48ffffff
- BYTE $0x5d
- BYTE $0xc3
+ QUAD $0x6de9f275da3949c3
+ QUAD $0x5bd8658d48ffffff
+ QUAD $0x5f415e415d415c41
+ QUAD $0x82c1c4c377f8c55d
+ QUAD $0xc018487df262c02a
+ QUAD $0x41c28945ff508d49
+ QUAD $0x2149c3894d07e283
+ QUAD $0x10bc411c5e8d48c3
+ QUAD $0x8949ff3145000000
+ QUAD $0x2e6666666632ebf6
+ QUAD $0x0000000000841f0f
+ QUAD $0xb162c85e4874f162
+ QUAD $0x618d4cbf0c11487c
+ QUAD $0xc6834940c3834810
+ QUAD $0x0fcc394dcf894940
+ QUAD $0xe1894cfffffec38f
+ QUAD $0x48e43145c957f0c5
+ QUAD $0x2e6666717207fa83
+ QUAD $0x0000000000841f0f
+ QUAD $0xe4a38c584874b162
+ QUAD $0x584874b162ffffff
+ QUAD $0xb162ffffffe8a38c
+ QUAD $0xffffeca38c584874
+ QUAD $0xa38c584874b162ff
+ QUAD $0x4874b162fffffff0
+ QUAD $0x62fffffff4a38c58
+ QUAD $0xfff8a38c584874b1
+ QUAD $0x8c584874b162ffff
+ QUAD $0x74b162fffffffca3
+ QUAD $0x08c48349a30c5848
+ QUAD $0xd2854da375e3394d
+ QUAD $0x8d4fffffff5a840f
+ QUAD $0x001f0fed3145a624
+ QUAD $0x49ac0c5848749162
+ QUAD $0xe9f175ea394dc5ff
+ LONG $0xffffff3c
RET
// func shiftDivFloat32AVX512(dst []float32, a []float32, shift float32, denom float32)
@@ -3671,21 +4731,25 @@ TEXT ·movingAverageFloat64AVX512(SB), NOSPLIT|NOFRAME, $0-56
MOVQ width+48(FP), R8
QUAD $0x294cc09e0fc0854d
QUAD $0xc10841c19c0f41c1
- QUAD $0x8d48000000e3850f
- QUAD $0x4e0f48ca39480141
- QUAD $0x00cf8e0fc08548c2
- QUAD $0x4853e58948550000
- QUAD $0xc02a83c1c4f8e483
- QUAD $0x83c28944ff488d49
- QUAD $0xfffffff8b94907e2
- QUAD $0x4cc1214d7fffffff
- QUAD $0x22ebd2314538468d
+ QUAD $0x48550000010b850f
+ QUAD $0x554156415741e589
+ QUAD $0x48f8e48348535441
+ QUAD $0xfffffffffffff8b8
+ QUAD $0xca394801498d4c7f
+ QUAD $0x10f98349ca4e0f4c
+ QUAD $0xc931000000df8d0f
+ QUAD $0x0000c28d0fc9394c
+ QUAD $0x8d49c02a83c1c400
+ QUAD $0xe28341c28945ff50
+ QUAD $0xce048d4cc0214c07
+ QUAD $0xce348d4838c08349
+ QUAD $0x2e666666666627eb
QUAD $0x0000000000841f0f
- QUAD $0x117ba1c4c85ef3c5
- QUAD $0xc08349c2ff49d70c
- QUAD $0xc2394908c6834808
- QUAD $0x3145c957f1c57974
- QUAD $0x664e7207f98348db
+ QUAD $0x0c11fbc5c85ef3c5
+ QUAD $0x08c08349c1ff48cf
+ QUAD $0x74c9394c08c68348
+ QUAD $0xdb3145c957f1c57a
+ QUAD $0x2e664f7207fa8348
QUAD $0x0000000000841f0f
QUAD $0xc4c8d84c587381c4
QUAD $0x81c4d0d84c587381
@@ -3694,14 +4758,61 @@ TEXT ·movingAverageFloat64AVX512(SB), NOSPLIT|NOFRAME, $0-56
QUAD $0x4c587381c4e8d84c
QUAD $0xd84c587381c4f0d8
QUAD $0x49d80c587381c4f8
- QUAD $0xc075d9394d08c383
- QUAD $0x1c8d4e8b74d28548
+ QUAD $0xc075d8394c08c383
+ QUAD $0x1c8d4e8b74d2854d
QUAD $0x0000441f0fdb31de
QUAD $0xff48db0c5873c1c4
- QUAD $0x6de9f275da3948c3
- QUAD $0x5bf8658d48ffffff
- BYTE $0x5d
- BYTE $0xc3
+ QUAD $0x6de9f275da3949c3
+ QUAD $0x5bd8658d48ffffff
+ QUAD $0x5f415e415d415c41
+ QUAD $0x83c1c4c377f8c55d
+ QUAD $0xc01948fdf262c02a
+ QUAD $0x41c28945ff508d49
+ QUAD $0x2149c3894d07e283
+ QUAD $0x10bc41785e8d48c3
+ QUAD $0x8949ff3145000000
+ QUAD $0x2e6666666640ebf6
+ QUAD $0x0000000000841f0f
+ QUAD $0xb162c85e48f5f162
+ QUAD $0xedf162ff0c1148fd
+ QUAD $0x1148fdb162c85e48
+ QUAD $0x4810618d4c01ff4c
+ QUAD $0x4980ee834980eb83
+ QUAD $0xb58f0fcc394dcf89
+ QUAD $0xf1c5e1894cfffffe
+ QUAD $0x57e9c5e43145c957
+ QUAD $0xc4820f07fa8348d2
+ QUAD $0x0000441f0f000000
+ QUAD $0x88e38c5848f5b162
+ QUAD $0x5848edb162ffffff
+ QUAD $0xb162ffffffc8e394
+ QUAD $0xffff90e38c5848f5
+ QUAD $0xe3945848edb162ff
+ QUAD $0x48f5b162ffffffd0
+ QUAD $0x62ffffff98e38c58
+ QUAD $0xffd8e3945848edb1
+ QUAD $0x8c5848f5b162ffff
+ QUAD $0xedb162ffffffa0e3
+ QUAD $0xffffffe0e3945848
+ QUAD $0xa8e38c5848f5b162
+ QUAD $0x5848edb162ffffff
+ QUAD $0xb162ffffffe8e394
+ QUAD $0xffffb0e38c5848f5
+ QUAD $0xe3945848edb162ff
+ QUAD $0x48f5b162fffffff0
+ QUAD $0x62ffffffb8e38c58
+ QUAD $0xfff8e3945848edb1
+ QUAD $0x4c5848f5b162ffff
+ QUAD $0x145848edb162ffe3
+ QUAD $0xe3394d08c48349e3
+ QUAD $0x854dffffff4a850f
+ QUAD $0x4ffffffef1840fd2
+ QUAD $0x2e66ed3145e6248d
+ QUAD $0x0000000000841f0f
+ QUAD $0x62ec0c5848f59162
+ QUAD $0x4901ec545848ed91
+ QUAD $0xe9e975ea394dc5ff
+ LONG $0xfffffec4
RET
// func shiftDivFloat64AVX512(dst []float64, a []float64, shift float64, denom float64)
@@ -3777,17 +4888,17 @@ TEXT ·randomU64AVX512(SB), NOSPLIT|NOFRAME, $0-32
QUAD $0x8348c67c48fdf262
QUAD $0x93e9c031077320fa
QUAD $0xf162d08948000001
- QUAD $0x000002a50dd458fd
- QUAD $0x02a315d458fdf162
+ QUAD $0x000002c50dd458fd
+ QUAD $0x02c315d458fdf162
QUAD $0x1dd458fdf1620000
- QUAD $0x62c82148000002a1
- QUAD $0x000314256f48fef1
+ QUAD $0x62c82148000002c1
+ QUAD $0x0004b4256f48fef1
QUAD $0x48fdf262c0314500
- QUAD $0xf2620000028f2d59
- QUAD $0x0000028d355948fd
- QUAD $0x028b3d5948fdf262
+ QUAD $0xf262000002af2d59
+ QUAD $0x000002ad355948fd
+ QUAD $0x02ab3d5948fdf262
QUAD $0x055948fd72620000
- QUAD $0x2e66666600000289
+ QUAD $0x2e666666000002a9
QUAD $0x0000000000841f0f
QUAD $0x7162cd4048dd7262
QUAD $0x48b57162d0d448b5
@@ -3829,12 +4940,12 @@ TEXT ·randomU64AVX512(SB), NOSPLIT|NOFRAME, $0-32
QUAD $0x8949000000ac840f
QUAD $0xc8894818c18348c0
QUAD $0x7c48fdd262d02148
- QUAD $0x920deb48f5f162c8
- QUAD $0x5948fdf262000001
- QUAD $0xfdf2620000011015
- QUAD $0x620000010e1d5948
- QUAD $0x00010c255948fdf2
- QUAD $0x122d5948fdf26200
+ QUAD $0x320deb48f5f162c8
+ QUAD $0x5948fdf262000003
+ QUAD $0xfdf2620000013015
+ QUAD $0x620000012e1d5948
+ QUAD $0x00012c255948fdf2
+ QUAD $0x322d5948fdf26200
QUAD $0x2e66666666000001
QUAD $0x0000000000841f0f
QUAD $0xf162f24048f5f262
@@ -3862,7 +4973,11 @@ TEXT ·randomU64AVX512(SB), NOSPLIT|NOFRAME, $0-32
QUAD $0x894cda314d1feac1
QUAD $0xce014cc0ff48c714
QUAD $0xec8948cb75c23948
- QUAD $0x53e0a8c377f8c55d
+ QUAD $0x000100c377f8c55d
+ QUAD $0x0000800000000000
+ QUAD $0x0001000000000000
+ QUAD $0x0000800000000000
+ QUAD $0x53e0a80000000000
QUAD $0xa7c150f1bbcdcbfa
QUAD $0xfba1f8e3779b97f4
QUAD $0x4a7c15d5336963ee
@@ -3880,6 +4995,54 @@ TEXT ·randomU64AVX512(SB), NOSPLIT|NOFRAME, $0-32
QUAD $0x3111ebbf58476d1c
QUAD $0x00001094d049bb13
QUAD $0x0000040000000000
+ QUAD $0x0000800000000000
+ QUAD $0x0000900000000000
+ QUAD $0x0000a00000000000
+ QUAD $0x0000b00000000000
+ QUAD $0x0000c00000000000
+ QUAD $0x0000d00000000000
+ QUAD $0x0000e00000000000
+ QUAD $0x0000f00000000000
+ QUAD $0x0000000000000000
+ QUAD $0x0000100000000000
+ QUAD $0x0000200000000000
+ QUAD $0x0000300000000000
+ QUAD $0x0000400000000000
+ QUAD $0x0000500000000000
+ QUAD $0x0000600000000000
+ QUAD $0x0000700000000000
+ QUAD $0x0000000000000000
+ QUAD $0x0000100000000000
+ QUAD $0x0000200000000000
+ QUAD $0x0000300000000000
+ QUAD $0x0000400000000000
+ QUAD $0x0000500000000000
+ QUAD $0x0000600000000000
+ QUAD $0x0000700000000000
+ QUAD $0x0000800000000000
+ QUAD $0x0000900000000000
+ QUAD $0x0000a00000000000
+ QUAD $0x0000b00000000000
+ QUAD $0x0000c00000000000
+ QUAD $0x0000d00000000000
+ QUAD $0x0000e00000000000
+ QUAD $0x0000f00000000000
+ QUAD $0x0000000000000000
+ QUAD $0x0000100000000000
+ QUAD $0x0000200000000000
+ QUAD $0x0000300000000000
+ QUAD $0x0000400000000000
+ QUAD $0x0000500000000000
+ QUAD $0x0000600000000000
+ QUAD $0x0000700000000000
+ QUAD $0x0000000000000000
+ QUAD $0x0000100000000000
+ QUAD $0x0000200000000000
+ QUAD $0x0000300000000000
+ QUAD $0x0000400000000000
+ QUAD $0x0000500000000000
+ QUAD $0x0000600000000000
+ QUAD $0x0000700000000000
QUAD $0x0000000000000000
QUAD $0x0000010000000000
QUAD $0x0000020000000000
@@ -3930,14 +5093,14 @@ TEXT ·randomF64AVX512(SB), NOSPLIT|NOFRAME, $0-32
QUAD $0xb848000000c3e9c0
QUAD $0x7ffffffffffffff8
QUAD $0x6f48fef162d02148
- QUAD $0xfdf2620000017605
+ QUAD $0xfdf262000002f605
QUAD $0xfdf262c931ce7c48
- QUAD $0x6200000264155948
- QUAD $0x0002621d5948fdf2
- QUAD $0x60255948fdf26200
- QUAD $0x1948fdf262000002
- QUAD $0xfdf2620000025e2d
- QUAD $0x900000025c355948
+ QUAD $0x6200000404155948
+ QUAD $0x0004021d5948fdf2
+ QUAD $0x00255948fdf26200
+ QUAD $0x1948fdf262000004
+ QUAD $0xfdf262000003fe2d
+ QUAD $0x90000003fc355948
QUAD $0xf162fa4048fdf262
QUAD $0x48bdf162f9d448c5
QUAD $0xef48bdf1621ed773
@@ -3957,7 +5120,7 @@ TEXT ·randomF64AVX512(SB), NOSPLIT|NOFRAME, $0-32
QUAD $0x1ce4e5b9b948ce29
QUAD $0x11ebb849bf58476d
QUAD $0xfbc594d049bb1331
- QUAD $0xb949000001b50510
+ QUAD $0xb949000003550510
QUAD $0x9e3779b97f4a7c15
QUAD $0x0000000000841f0f
QUAD $0x491eeac149f28949
@@ -3969,7 +5132,55 @@ TEXT ·randomF64AVX512(SB), NOSPLIT|NOFRAME, $0-32
QUAD $0xc70c11fbc5c859f3
QUAD $0x3948ce014cc0ff48
QUAD $0xc55dec8948bd75c2
- QUAD $0x0000000000c377f8
+ QUAD $0x0000000080c377f8
+ QUAD $0x0000000090000000
+ QUAD $0x00000000a0000000
+ QUAD $0x00000000b0000000
+ QUAD $0x00000000c0000000
+ QUAD $0x00000000d0000000
+ QUAD $0x00000000e0000000
+ QUAD $0x00000000f0000000
+ QUAD $0x0000000000000000
+ QUAD $0x0000000010000000
+ QUAD $0x0000000020000000
+ QUAD $0x0000000030000000
+ QUAD $0x0000000040000000
+ QUAD $0x0000000050000000
+ QUAD $0x0000000060000000
+ QUAD $0x0000000070000000
+ QUAD $0x0000000000000000
+ QUAD $0x0000000010000000
+ QUAD $0x0000000020000000
+ QUAD $0x0000000030000000
+ QUAD $0x0000000040000000
+ QUAD $0x0000000050000000
+ QUAD $0x0000000060000000
+ QUAD $0x0000000070000000
+ QUAD $0x0000000080000000
+ QUAD $0x0000000090000000
+ QUAD $0x00000000a0000000
+ QUAD $0x00000000b0000000
+ QUAD $0x00000000c0000000
+ QUAD $0x00000000d0000000
+ QUAD $0x00000000e0000000
+ QUAD $0x00000000f0000000
+ QUAD $0x0000000000000000
+ QUAD $0x0000000010000000
+ QUAD $0x0000000020000000
+ QUAD $0x0000000030000000
+ QUAD $0x0000000040000000
+ QUAD $0x0000000050000000
+ QUAD $0x0000000060000000
+ QUAD $0x0000000070000000
+ QUAD $0x0000000000000000
+ QUAD $0x0000000010000000
+ QUAD $0x0000000020000000
+ QUAD $0x0000000030000000
+ QUAD $0x0000000040000000
+ QUAD $0x0000000050000000
+ QUAD $0x0000000060000000
+ QUAD $0x0000000070000000
+ QUAD $0x0000000000000000
QUAD $0x0000000001000000
QUAD $0x0000000002000000
QUAD $0x0000000003000000
@@ -4001,6 +5212,10 @@ TEXT ·randomF64AVX512(SB), NOSPLIT|NOFRAME, $0-32
QUAD $0x0000000005000000
QUAD $0x0000000006000000
QUAD $0x0000000007000000
+ QUAD $0x0000000100000000
+ QUAD $0x0000000080000000
+ QUAD $0x0000000100000000
+ QUAD $0x0000000080000000
QUAD $0xcbfa53e0a8000000
QUAD $0x97f4a7c150f1bbcd
QUAD $0x63eefba1f8e3779b
@@ -4040,15 +5255,15 @@ TEXT ·randomF32AVX512(SB), NOSPLIT|NOFRAME, $0-32
QUAD $0xc031077310fa8348
QUAD $0xd089480000011ee9
QUAD $0x6f48fef162c82148
- QUAD $0xfef162000002bc05
- QUAD $0x62000002f20d6f48
+ QUAD $0xfef1620000043c05
+ QUAD $0x62000004720d6f48
QUAD $0xc03145d67c48fdf2
- QUAD $0x03871d5948fdf262
+ QUAD $0x05271d5948fdf262
QUAD $0x255948fdf2620000
- QUAD $0x48fdf26200000385
- QUAD $0xf262000003832d59
- QUAD $0x000003913518487d
- QUAD $0x03773d5948fdf262
+ QUAD $0x48fdf26200000525
+ QUAD $0xf262000005232d59
+ QUAD $0x000005313518487d
+ QUAD $0x05173d5948fdf262
QUAD $0x0000441f0f660000
QUAD $0x7262c34048fd7262
QUAD $0x48b57162cb4048f5
@@ -4078,12 +5293,12 @@ TEXT ·randomF32AVX512(SB), NOSPLIT|NOFRAME, $0-32
QUAD $0xd02148c889480cc1
QUAD $0xd262c67c28fdf262
QUAD $0x0debf5c5c87c28fd
- QUAD $0x597de2c4000002a1
- QUAD $0x7de2c40000026c15
- QUAD $0xe2c40000026b1d59
- QUAD $0xc40000026a25597d
- QUAD $0x000002792d1879e2
- QUAD $0x00026835597de2c4
+ QUAD $0x597de2c400000441
+ QUAD $0x7de2c40000040c15
+ QUAD $0xe2c40000040b1d59
+ QUAD $0xc40000040a25597d
+ QUAD $0x000004192d1879e2
+ QUAD $0x00040835597de2c4
QUAD $0x2e66666666666600
QUAD $0x0000000000841f0f
QUAD $0xc5c5fa4028f5f262
@@ -4101,7 +5316,7 @@ TEXT ·randomF32AVX512(SB), NOSPLIT|NOFRAME, $0-32
QUAD $0x476d1ce4e5b9b948
QUAD $0x133111ebb849bf58
QUAD $0x0510fac594d049bb
- QUAD $0x7c15b949000001e5
+ QUAD $0x7c15b94900000385
QUAD $0x2e669e3779b97f4a
QUAD $0x0000000000841f0f
QUAD $0x491eeac149f28949
@@ -4112,7 +5327,55 @@ TEXT ·randomF32AVX512(SB), NOSPLIT|NOFRAME, $0-32
QUAD $0xff48870c11fac5c8
QUAD $0x75c23948ce014cc0
QUAD $0x77f8c55dec8948c7
- QUAD $0x00000000000000c3
+ QUAD $0x00000000000080c3
+ QUAD $0x0000000000009000
+ QUAD $0x000000000000a000
+ QUAD $0x000000000000b000
+ QUAD $0x000000000000c000
+ QUAD $0x000000000000d000
+ QUAD $0x000000000000e000
+ QUAD $0x000000000000f000
+ QUAD $0x0000000000000000
+ QUAD $0x0000000000001000
+ QUAD $0x0000000000002000
+ QUAD $0x0000000000003000
+ QUAD $0x0000000000004000
+ QUAD $0x0000000000005000
+ QUAD $0x0000000000006000
+ QUAD $0x0000000000007000
+ QUAD $0x0000000000000000
+ QUAD $0x0000000000001000
+ QUAD $0x0000000000002000
+ QUAD $0x0000000000003000
+ QUAD $0x0000000000004000
+ QUAD $0x0000000000005000
+ QUAD $0x0000000000006000
+ QUAD $0x0000000000007000
+ QUAD $0x0000000000008000
+ QUAD $0x0000000000009000
+ QUAD $0x000000000000a000
+ QUAD $0x000000000000b000
+ QUAD $0x000000000000c000
+ QUAD $0x000000000000d000
+ QUAD $0x000000000000e000
+ QUAD $0x000000000000f000
+ QUAD $0x0000000000000000
+ QUAD $0x0000000000001000
+ QUAD $0x0000000000002000
+ QUAD $0x0000000000003000
+ QUAD $0x0000000000004000
+ QUAD $0x0000000000005000
+ QUAD $0x0000000000006000
+ QUAD $0x0000000000007000
+ QUAD $0x0000000000000000
+ QUAD $0x0000000000001000
+ QUAD $0x0000000000002000
+ QUAD $0x0000000000003000
+ QUAD $0x0000000000004000
+ QUAD $0x0000000000005000
+ QUAD $0x0000000000006000
+ QUAD $0x0000000000007000
+ QUAD $0x0000000000000000
QUAD $0x0000000000000100
QUAD $0x0000000000000200
QUAD $0x0000000000000300
@@ -4144,6 +5407,10 @@ TEXT ·randomF32AVX512(SB), NOSPLIT|NOFRAME, $0-32
QUAD $0x0000000000000500
QUAD $0x0000000000000600
QUAD $0x0000000000000700
+ QUAD $0x0000000000010000
+ QUAD $0x0000000000008000
+ QUAD $0x0000000000010000
+ QUAD $0x0000000000008000
QUAD $0xbbcdcbfa53e0a800
QUAD $0x779b97f4a7c150f1
QUAD $0x336963eefba1f8e3
diff --git a/internal/amd64/numeric_sse2_amd64.s b/internal/amd64/numeric_sse2_amd64.s
index a4fee43..5bac7c1 100644
--- a/internal/amd64/numeric_sse2_amd64.s
+++ b/internal/amd64/numeric_sse2_amd64.s
@@ -176,24 +176,90 @@ TEXT ·polyEvalFloat32SSE2(SB), NOSPLIT|NOFRAME, $0-72
MOVQ x_len+32(FP), R8
MOVQ coeffs_len+56(FP), R9
QUAD $0x4dc14c0f4cc1394c
- QUAD $0x000000cb840fc985
- QUAD $0x0001678e0fc0854d
- QUAD $0xfc8a44100f42f300
- QUAD $0x0f02e88348c8894c
- QUAD $0x894855000000c48c
- QUAD $0x8d41f8e4834853e5
- QUAD $0xe1834103e183ff49
- QUAD $0x2e661cebd2314503
+ QUAD $0xe58948551674c985
+ QUAD $0xf88349f8e4834853
+ QUAD $0x0206e9c031217d10
+ QUAD $0x298e0fc0854d0000
+ QUAD $0x0f08f88349000003
+ QUAD $0xe9c031000001a883
+ QUAD $0x100f42f300000309
+ QUAD $0x0f01f98349fc8a44
+ QUAD $0x498d49000001248e
+ QUAD $0x518d4d00c0c60ffe
+ QUAD $0x314500000010bbfd
+ QUAD $0x2e6666666634ebdb
QUAD $0x0000000000841f0f
- QUAD $0xff499714110f42f3
- QUAD $0x0099840fc2394dc2
- QUAD $0x960c100f42f30000
- QUAD $0x8341d0280fc38949
- QUAD $0x49cb89481e7401f9
- QUAD $0x001f0fd0280fc389
+ QUAD $0x110f429f24110f42
+ QUAD $0x9f7c110f42109f74
+ QUAD $0x48309f44110f4620
+ QUAD $0x394cc3894910588d
+ QUAD $0x48000001918f0fc3
+ QUAD $0x429e0c100f42d889
+ QUAD $0x100f42109e54100f
+ QUAD $0x9e6c100f42209e5c
+ QUAD $0x0fe0280fcb894830
+ QUAD $0x280f44f8280ff028
+ QUAD $0x0f397501c1f641c0
+ QUAD $0x0f46f3e1590fe028
+ QUAD $0xc9c60f45f88a4c10
+ QUAD $0xf0280fe1580f4100
+ QUAD $0x0ff1580f41f2590f
+ QUAD $0x580f41fb590ff828
+ QUAD $0x590f44c0280f44f9
+ QUAD $0xd3894cc1580f45c5
+ QUAD $0xffff67840fc98548
+ QUAD $0x001f0f02c38348ff
+ QUAD $0x4c100f44f3e1590f
+ QUAD $0x9a54100f44f3f49a
+ QUAD $0x0f4100d2c60f45f8
+ QUAD $0x580f41f2590fe258
+ QUAD $0xfa580f41fb590ff2
+ QUAD $0xc2580f45c5590f44
+ QUAD $0x00c9c60f45e1590f
+ QUAD $0x41f2590fe1580f41
+ QUAD $0x0f41fb590ff1580f
+ QUAD $0x0f45c5590f44f958
+ QUAD $0x8348fec38348c158
+ QUAD $0xfffeffe9a47f01fb
+ QUAD $0xeac149f0508d4dff
+ QUAD $0x4900000088840f04
+ QUAD $0xe08348d0894cc2ff
+ QUAD $0x00c8c60fc8280ffe
+ QUAD $0x2e666600000010b9
+ QUAD $0x0000000000841f0f
+ QUAD $0x4c110fc08f4c110f
+ QUAD $0x0fe08f4c110fd08f
+ QUAD $0x8f0c110ff08f4c11
+ QUAD $0x4c110f108f4c110f
+ QUAD $0x48308f4c110f208f
+ QUAD $0x75fec0834820c183
+ QUAD $0xc2f641f0418d48cf
+ QUAD $0xf8b8484beb337501
+ QUAD $0x4c7fffffffffffff
+ QUAD $0x90c0570fc931c021
+ QUAD $0x8f44110f8f04110f
+ QUAD $0xc8394808c1834810
+ QUAD $0xb900000148e9ee75
+ QUAD $0xc60fc03100000010
+ QUAD $0x110f8704110f00c0
+ QUAD $0x208744110f108744
+ QUAD $0xc88948308744110f
+ QUAD $0x8e0fc22949c2894d
+ QUAD $0x100f42f30000010a
+ QUAD $0x8348c9894cfc8a44
+ QUAD $0x000000a38c0f02e9
+ QUAD $0x03e28341ff518d45
+ QUAD $0x2e661beb03e18341
+ QUAD $0x0000000000841f0f
+ QUAD $0xc0ff488714110ff3
+ QUAD $0x0000cd840fc0394c
+ QUAD $0x8949860c100ff300
+ QUAD $0x01f98341d0280fcb
+ QUAD $0xcb8949d3894c2074
+ QUAD $0x0000441f0fd0280f
QUAD $0x580f42f3d1590ff3
QUAD $0xcbff48cbff499a14
- QUAD $0xb87203f88348ee75
+ QUAD $0xb87203f98348ee75
QUAD $0x00401f0f04c38349
QUAD $0x580f42f3d1590ff3
QUAD $0xf3d1590ff3f09a54
@@ -202,27 +268,20 @@ TEXT ·polyEvalFloat32SSE2(SB), NOSPLIT|NOFRAME, $0-72
QUAD $0x0f42f3d1590ff3e8
QUAD $0xfcc38349e49a5458
QUAD $0x75e9ca7f03fb8349
- QUAD $0x8e0fc0854dffffff
- QUAD $0x08f883490000009c
- QUAD $0x000080e9c0311873
- QUAD $0x313d7308f8834900
- QUAD $0x5bf8658d486cebc0
- QUAD $0xfffffff8b848c35d
- QUAD $0x31c0214c7fffffff
- QUAD $0x2e666666c0570fc9
+ QUAD $0x7308fa8349ffffff
+ QUAD $0x89443aebc2894905
+ QUAD $0x49ca294907e183c1
+ QUAD $0xc8c60fc8280fc201
+ QUAD $0x00000000801f0f00
+ QUAD $0x874c110f870c110f
+ QUAD $0xc2394908c0834810
+ QUAD $0x661774c98548ee75
QUAD $0x0000000000841f0f
- QUAD $0x8f44110f8f04110f
- QUAD $0xc8394808c1834810
- QUAD $0xfff8b84847ebee75
- QUAD $0x214c7fffffffffff
- QUAD $0x00c8c60fc8280fc0
- QUAD $0x0000441f0f66c931
- QUAD $0x8f4c110f8f0c110f
- QUAD $0xc8394808c1834810
- QUAD $0xf31974c03949ee75
- QUAD $0xebc0ff488704110f
- QUAD $0x000000008704c7f1
- QUAD $0xf175c03949c0ff48
+ QUAD $0xff499704110f42f3
+ QUAD $0x8d48f275d0394dc2
+ QUAD $0x8704c7c35d5bf865
+ QUAD $0x49c0ff4800000000
+ LONG $0xf175c039
BYTE $0xc3
RET
@@ -236,68 +295,117 @@ TEXT ·convolveFloat32SSE2(SB), NOSPLIT|NOFRAME, $0-72
MOVQ ker_len+56(FP), R9
QUAD $0x56415741e5894855
QUAD $0xe483485354415541
- QUAD $0xcd894d18ec8348f8
- QUAD $0xc9854d08247c8948
- QUAD $0x0f41c8294dc0940f
- QUAD $0xbc850fc10841c19c
- QUAD $0x4c01488d4d000001
- QUAD $0x854dc94e0f4cc139
- QUAD $0x4d000001a88e0fc9
- QUAD $0x000001408e0fed85
- QUAD $0x41e88945aa0c8d4a
- QUAD $0xfffff8ba4903e083
- QUAD $0xca83497fffffffff
- QUAD $0x0c5e8d4cea214d04
- QUAD $0xfcc08348aa048d4a
- QUAD $0x49db311024448948
- QUAD $0xfffffffffffffebe
- QUAD $0xfffffffffdbf493f
- QUAD $0xfffffcbc493fffff
- QUAD $0x9022eb3fffffffff
- QUAD $0x110ff30824448b48
- QUAD $0xc38349c3ff489804
- QUAD $0xcb394c04c6834804
- QUAD $0x00012e840ffd8949
- QUAD $0x49c0570fef894c00
- QUAD $0xed3145137304fd83
- QUAD $0x2e666600000081e9
+ QUAD $0xc9854d18ec8348f8
+ QUAD $0x08244c894cc0940f
+ QUAD $0x41c19c0f41c8294d
+ QUAD $0x00000340850fc108
+ QUAD $0x4cc1394c01688d4d
+ QUAD $0x7d10fd8349e94e0f
+ QUAD $0x0000018ee9c93107
+ QUAD $0xc9854d08244c8b4c
+ QUAD $0x8d4c000000aa8e0f
+ QUAD $0x4500000010b83046
+ QUAD $0x2e6666666666d231
QUAD $0x0000000000841f0f
- QUAD $0xed31451024548b48
+ QUAD $0x894dc0570fc18948
+ QUAD $0x0fc9570fcb894cc3
+ QUAD $0x2e6666db570fd257
QUAD $0x0000000000841f0f
- QUAD $0xf3f4ab4c100f43f3
- QUAD $0x0ff3f8ab54100f43
+ QUAD $0xc60ffc9a64100ff3
+ QUAD $0x41d06b100f4100e4
+ QUAD $0x7b100f41e073100f
+ QUAD $0xec590f03100f45f0
+ QUAD $0x580ff4590fc5580f
+ QUAD $0x44d7580ffc590fce
+ QUAD $0x49d8580f41c4590f
+ QUAD $0xc075cbff4804c383
+ QUAD $0x110f429704110f42
+ QUAD $0x9754110f4210974c
+ QUAD $0x4830975c110f4220
+ QUAD $0x4940c0834910418d
+ QUAD $0x758e0fe8394cca89
+ QUAD $0x000000d6e9ffffff
+ QUAD $0x49c28949f0458d49
+ QUAD $0x8945c2ff4904eac1
+ QUAD $0xf8834803e08341d0
+ QUAD $0x00000010b80c7330
+ QUAD $0x4900000083e9c931
+ QUAD $0xc0570fc931fce283
+ QUAD $0x8f44110f8f04110f
+ QUAD $0x110f208f44110f10
+ QUAD $0x408f44110f308f44
+ QUAD $0x44110f508f44110f
+ QUAD $0x0f708f44110f608f
+ QUAD $0x0f000000808f8411
+ QUAD $0x0f000000908f8411
+ QUAD $0x0f000000a08f8411
+ QUAD $0x0f000000b08f8411
+ QUAD $0x0f000000c08f8411
+ QUAD $0x0f000000d08f8411
+ QUAD $0x0f000000e08f8411
+ QUAD $0x48000000f08f8411
+ QUAD $0x75fcc2834940c183
+ QUAD $0x8d483074c0854d8f
+ QUAD $0x001f0fc0570f1041
+ QUAD $0x8f44110f8f04110f
+ QUAD $0x110f208f44110f10
+ QUAD $0x8348c18948308f44
+ QUAD $0x48e175c8ff4910c0
+ QUAD $0x894dc18948f0c083
+ QUAD $0x018e8e0fc82949e8
+ QUAD $0x480824448b480000
+ QUAD $0x000001308e0fc085
+ QUAD $0x41c2894182048d4c
+ QUAD $0xfffffcbb4903e283
+ QUAD $0xc321497fffffffff
+ QUAD $0x0cc383488e1c8d48
+ QUAD $0x448948fc82448d48
+ QUAD $0xbe498e348d481024
+ QUAD $0x3ffffffffffffffe
+ QUAD $0xfffffffffffdbf49
+ QUAD $0xfffffffcbc493fff
+ QUAD $0x90661eeb3fffffff
+ QUAD $0xc1ff488f04110ff3
+ QUAD $0x04c6834804c38348
+ QUAD $0x840fc9394ccd894d
+ QUAD $0x48e9894d00000114
+ QUAD $0x0f09730408247c83
+ QUAD $0x907aebed3145c057
+ QUAD $0x1024548b48c0570f
+ QUAD $0x0000441f0fed3145
+ QUAD $0xf3f4ab4c100f42f3
+ QUAD $0x0ff3f8ab54100f42
QUAD $0x894cc8580ff30a59
- QUAD $0x14590ff3f0314ce8
- QUAD $0x0f43f3d1580ff381
- QUAD $0x4ce8894cfcab4c10
- QUAD $0xf3810c590ff3f831
- QUAD $0x04100f43f3ca580f
- QUAD $0xf3e0314ce8894cab
- QUAD $0xc1580ff38104590f
- QUAD $0xf0c2834804c58349
- QUAD $0xc0854da375ea394d
- QUAD $0x8d4affffff4a840f
- QUAD $0x8b4800000000ad04
- QUAD $0x8d4ec22948102454
- QUAD $0x00401f0fc031ae2c
+ QUAD $0x590f41f3f0314ce8
+ QUAD $0x42f3d1580ff38014
+ QUAD $0xe8894cfcab4c100f
+ QUAD $0x0c590f41f3f8314c
+ QUAD $0x0f42f3ca580ff380
+ QUAD $0x314ce8894cab0410
+ QUAD $0xf38004590f41f3e0
+ QUAD $0x4804c58349c1580f
+ QUAD $0xa075eb394df0c283
+ QUAD $0xffff57840fd2854d
+ QUAD $0x000000ad048d4aff
+ QUAD $0x29481024548b4800
+ QUAD $0x90c031ae2c8d4ec2
QUAD $0xf300854c100f41f3
QUAD $0x48c1580ff30a590f
QUAD $0x3949fcc28348c0ff
- QUAD $0xffffff10e9e575c0
- QUAD $0xc031047308f98349
- QUAD $0xfffffff8b8483deb
- QUAD $0x31c8214c7fffffff
- QUAD $0x24548b48c0570fc9
- QUAD $0x2e66666666666608
+ QUAD $0xffffff20e9e575c2
+ QUAD $0x8949057308f88349
+ QUAD $0xe083e8894435ebc8
+ QUAD $0x0fc80149c0294907
+ QUAD $0x0000441f0f66c057
+ QUAD $0x8f44110f8f04110f
+ QUAD $0xc8394908c1834810
+ QUAD $0x661974c08548ee75
QUAD $0x0000000000841f0f
- QUAD $0x8a44110f8a04110f
- QUAD $0xc8394808c1834810
- QUAD $0x481874c13949ee75
- QUAD $0x00401f0f08244c8b
- QUAD $0x48000000008104c7
- QUAD $0x48f175c13949c0ff
- QUAD $0x5d415c415bd8658d
- LONG $0x5f415e41
+ QUAD $0x000000008704c742
+ QUAD $0xf075c5394dc0ff49
+ QUAD $0x415c415bd8658d48
+ LONG $0x415e415d
+ BYTE $0x5f
BYTE $0x5d
BYTE $0xc3
RET
@@ -312,49 +420,106 @@ TEXT ·correlateFloat32SSE2(SB), NOSPLIT|NOFRAME, $0-72
MOVQ ker_len+56(FP), R9
QUAD $0x294dc0940fc9854d
QUAD $0xc20841c29c0f41c8
- QUAD $0x8d490000014d850f
- QUAD $0x4e0f48c1394c0140
- QUAD $0x01398e0fc08548c1
- QUAD $0xe58e0fc9854d0000
- QUAD $0x41e5894855000000
- QUAD $0xf8e4834853564157
- QUAD $0xb84903e183c98944
- QUAD $0x7ffffffffffffff8
- QUAD $0x4cc8214d04c88349
- QUAD $0x1aebdb31450c568d
- QUAD $0xff499f04110f42f3
- QUAD $0xc6834804c28349c3
- QUAD $0x0093840fc3394904
- QUAD $0x49db31c0570f0000
- QUAD $0x2e66665b7204f983
+ QUAD $0x485500000304850f
+ QUAD $0x485356415741e589
+ QUAD $0x4c01408d49f8e483
+ QUAD $0x8348c14e0f48c139
+ QUAD $0x9be9c931077d10f8
+ QUAD $0x8e0fc9854d000001
+ QUAD $0x30468d4c000000af
+ QUAD $0x314500000010bb41
+ QUAD $0x2e666666666666d2
+ QUAD $0x0000000000841f0f
+ QUAD $0x3145c0570fd9894c
+ QUAD $0x0fd2570fc9570fdb
+ QUAD $0x2e6666666666db57
+ QUAD $0x0000000000841f0f
+ QUAD $0xc60f9a24100f42f3
+ QUAD $0xd0986c100f4300e4
+ QUAD $0x0f43e09874100f43
+ QUAD $0x04100f47f0987c10
+ QUAD $0x0fc5580fec590f98
+ QUAD $0xfc590fce580ff459
+ QUAD $0x41c4590f44d7580f
+ QUAD $0x394dc3ff49d8580f
+ QUAD $0x9704110f42bd75d9
+ QUAD $0x0f4210974c110f42
+ QUAD $0x5c110f4220975411
+ QUAD $0x834910598d4c3097
+ QUAD $0xc33949ca894940c0
+ QUAD $0xe3e9ffffff728e0f
+ QUAD $0x49f0488d48000000
+ QUAD $0xff4904eac149ca89
+ QUAD $0x03e08341d08945c2
+ QUAD $0xba410d7330f98348
+ QUAD $0x8fe9c93100000010
+ QUAD $0x31fce28349000000
+ QUAD $0x2e666666c0570fc9
+ QUAD $0x0000000000841f0f
+ QUAD $0x8f44110f8f04110f
+ QUAD $0x110f208f44110f10
+ QUAD $0x408f44110f308f44
+ QUAD $0x44110f508f44110f
+ QUAD $0x0f708f44110f608f
+ QUAD $0x0f000000808f8411
+ QUAD $0x0f000000908f8411
+ QUAD $0x0f000000a08f8411
+ QUAD $0x0f000000b08f8411
+ QUAD $0x0f000000c08f8411
+ QUAD $0x0f000000d08f8411
+ QUAD $0x0f000000e08f8411
+ QUAD $0x48000000f08f8411
+ QUAD $0x75fcc2834940c183
+ QUAD $0x8d4c3074c0854d8f
+ QUAD $0x001f0fc0570f1051
+ QUAD $0x8f44110f8f04110f
+ QUAD $0x110f208f44110f10
+ QUAD $0x8349d1894c308f44
+ QUAD $0x49e175c8ff4910c2
+ QUAD $0x8949d1894cf0c283
+ QUAD $0x012e8e0fc82949c0
+ QUAD $0xd28e0fc9854d0000
+ QUAD $0x8341c88945000000
+ QUAD $0xfffffffcba4903e0
+ QUAD $0x4cca214d7fffffff
+ QUAD $0x480cc383498e1c8d
+ QUAD $0x001f0f1ceb8e348d
+ QUAD $0xc1ff488f04110ff3
+ QUAD $0x04c6834804c38349
+ QUAD $0x0000e7840fc13948
+ QUAD $0x8349db31c0570f00
+ QUAD $0x2e6666665c7204f9
QUAD $0x0000000000841f0f
- QUAD $0xf3f49a4c100f41f3
- QUAD $0x0ff3f89a54100f41
+ QUAD $0xf3f49b4c100f41f3
+ QUAD $0x0ff3f89b54100f41
QUAD $0x9a54590ff39a0c59
QUAD $0x0f41f3c8580ff304
- QUAD $0x5c590ff3fc9a5c10
+ QUAD $0x5c590ff3fc9b5c10
QUAD $0x41f3d1580ff3089a
- QUAD $0x44590ff39a04100f
+ QUAD $0x44590ff39b04100f
QUAD $0x0ff3da580ff30c9a
QUAD $0x394904c38348c358
- QUAD $0x8074c98548b575d8
+ QUAD $0x8074c0854db575da
QUAD $0x9e1c8d489a348d4c
QUAD $0x0000441f0fff3145
QUAD $0x43f3bb0c100f42f3
QUAD $0xc1580ff3be0c590f
- QUAD $0xe875f9394cc7ff49
- QUAD $0x658d48ffffff53e9
- QUAD $0xc35d5f415e415be8
- QUAD $0xc931047308f88348
- QUAD $0xfffffff8b94832eb
- QUAD $0x31c121487fffffff
- QUAD $0x2e666666c0570fd2
+ QUAD $0xe875f8394dc7ff49
+ QUAD $0xf88349ffffff53e9
+ QUAD $0x38ebc88949057308
+ QUAD $0xd0294907e283c289
+ QUAD $0x2e66c0570fc80149
+ QUAD $0x0000000000841f0f
+ QUAD $0x8f44110f8f04110f
+ QUAD $0xc8394908c1834810
+ QUAD $0x661974d28548ee75
QUAD $0x0000000000841f0f
- QUAD $0x9744110f9704110f
- QUAD $0xd1394808c2834810
- QUAD $0x008f04c70aebee75
- QUAD $0x3948c1ff48000000
- LONG $0xc3f175c8
+ QUAD $0x000000008704c742
+ QUAD $0xf075c0394cc0ff49
+ QUAD $0x415e415be8658d48
+ BYTE $0x5f
+ BYTE $0x5d
+ BYTE $0xc3
RET
// func normFloat64SSE2(a []float64) float64
@@ -522,24 +687,106 @@ TEXT ·polyEvalFloat64SSE2(SB), NOSPLIT|NOFRAME, $0-72
MOVQ x_len+32(FP), R8
MOVQ coeffs_len+56(FP), R9
QUAD $0x4dc14c0f4cc1394c
- QUAD $0x000000cb840fc985
- QUAD $0x00016a8e0fc0854d
- QUAD $0xf8ca44100f42f200
- QUAD $0x0f02e88348c8894c
- QUAD $0x894855000000c48c
- QUAD $0x8d41f8e4834853e5
- QUAD $0xe1834103e183ff49
- QUAD $0x2e661cebd2314503
+ QUAD $0xe58948551a74c985
+ QUAD $0xec8348f0e4834853
+ QUAD $0x31217d10f8834930
+ QUAD $0x854d0000027ee9c0
+ QUAD $0x49000003a68e0fc0
+ QUAD $0x0001fc830f04f883
+ QUAD $0x00000385e9c03100
+ QUAD $0x49f8ca4c100f42f2
+ QUAD $0x0001488e0f01f983
+ QUAD $0x498d49c9140f6600
+ QUAD $0x4500000010bb41ff
+ QUAD $0x66240c290f66d231
QUAD $0x0000000000841f0f
- QUAD $0xff49d714110f42f2
- QUAD $0x0099840fc2394dc2
- QUAD $0xd60c100f42f20000
- QUAD $0x41d0280f66c38949
- QUAD $0xcb89481d7401f983
- QUAD $0x90d0280f66c38949
+ QUAD $0xd604100f42d8894c
+ QUAD $0x0f4266102444290f
+ QUAD $0x100f426610d65410
+ QUAD $0x64100f426620d65c
+ QUAD $0xd66c100f426630d6
+ QUAD $0x50d674100f426640
+ QUAD $0x6660d67c100f4266
+ QUAD $0x894970d644100f46
+ QUAD $0x4466c9280f4466cb
+ QUAD $0xd9280f4466d1280f
+ QUAD $0x0f4466e1280f4466
+ QUAD $0x66f1280f4466e928
+ QUAD $0xc1280f66f9280f44
+ QUAD $0x0000000000841f0f
+ QUAD $0x590f4166c8280f66
+ QUAD $0x4466ff590f4466c8
+ QUAD $0xed590f4466f6590f
+ QUAD $0x0f4466e4590f4466
+ QUAD $0x66d2590f4466db59
+ QUAD $0x42f210244c590f44
+ QUAD $0x140f66f8da44100f
+ QUAD $0x4466c8580f4466c0
+ QUAD $0xd8580f4466d0580f
+ QUAD $0x0f4466e0580f4466
+ QUAD $0x66f0580f4466e858
+ QUAD $0xc1580f66f8580f44
+ QUAD $0x987fdb854dcbff49
+ QUAD $0x4666d70c110f4666
+ QUAD $0x0f466610d754110f
+ QUAD $0x110f466620d75c11
+ QUAD $0x6c110f466630d764
+ QUAD $0xd774110f466640d7
+ QUAD $0x60d77c110f466650
+ QUAD $0x4c70d744110f4266
+ QUAD $0x394dc2894910588d
+ QUAD $0x8e0f240c280f66c3
+ QUAD $0x00010be9fffffedc
+ QUAD $0xeac149f0508d4d00
+ QUAD $0x49000000c0840f04
+ QUAD $0xe08348d0894cc2ff
+ QUAD $0x140f66c1280f66fe
+ QUAD $0x906600000010b9c1
+ QUAD $0x0f6680cf44110f66
+ QUAD $0x44110f6690cf4411
+ QUAD $0xb0cf44110f66a0cf
+ QUAD $0x0f66c0cf44110f66
+ QUAD $0x44110f66d0cf4411
+ QUAD $0xf0cf44110f66e0cf
+ QUAD $0x110f66cf04110f66
+ QUAD $0xcf44110f6610cf44
+ QUAD $0x6630cf44110f6620
+ QUAD $0x110f6640cf44110f
+ QUAD $0xcf44110f6650cf44
+ QUAD $0x4870cf44110f6660
+ QUAD $0x75fec0834820c183
+ QUAD $0xc2f641f0418d4897
+ QUAD $0xfcb8486feb3b7501
+ QUAD $0x4c7fffffffffffff
+ QUAD $0x66c0570fc931c021
+ QUAD $0x0000000000841f0f
+ QUAD $0xcf44110fcf04110f
+ QUAD $0xc8394804c1834810
+ QUAD $0xb900000169e9ee75
+ QUAD $0x0f66c03100000010
+ QUAD $0x10c74c110f66c914
+ QUAD $0x110f66c70c110f66
+ QUAD $0xc74c110f6630c74c
+ QUAD $0x6650c74c110f6620
+ QUAD $0x110f6640c74c110f
+ QUAD $0xc74c110f6670c74c
+ QUAD $0x49c2894dc8894860
+ QUAD $0x0000010e8e0fc229
+ QUAD $0x4cf8ca44100f42f2
+ QUAD $0x8c0f02e98348c989
+ QUAD $0xff518d45000000a7
+ QUAD $0x03e1834103e28341
+ QUAD $0x2e66666666661feb
+ QUAD $0x0000000000841f0f
+ QUAD $0xc0ff48c714110ff2
+ QUAD $0x0000cd840fc0394c
+ QUAD $0x8949c60c100ff200
+ QUAD $0xf98341d0280f66cb
+ QUAD $0x8949d3894c1f7401
+ QUAD $0x001f0fd0280f66cb
QUAD $0x580f42f2d1590ff2
QUAD $0xcbff48cbff49da14
- QUAD $0xb87203f88348ee75
+ QUAD $0xb87203f98348ee75
QUAD $0x00401f0f04c38349
QUAD $0x580f42f2d1590ff2
QUAD $0xf2d1590ff2e0da54
@@ -548,28 +795,22 @@ TEXT ·polyEvalFloat64SSE2(SB), NOSPLIT|NOFRAME, $0-72
QUAD $0x0f42f2d1590ff2d0
QUAD $0xfcc38349c8da5458
QUAD $0x75e9ca7f03fb8349
- QUAD $0x8e0fc0854dffffff
- QUAD $0x04f883490000009f
- QUAD $0x000082e9c0311873
- QUAD $0x313f7304f8834900
- QUAD $0x5bf8658d486eebc0
- QUAD $0xfffffffcb848c35d
- QUAD $0x31c0214c7fffffff
- QUAD $0x2e6666c0570f66c9
- QUAD $0x0000000000841f0f
- QUAD $0x110f66cf04110f66
- QUAD $0x4804c1834810cf44
- QUAD $0xb84848ebec75c839
- QUAD $0x7ffffffffffffffc
- QUAD $0x66c8280f66c0214c
- QUAD $0x001f0fc931c8140f
- QUAD $0x110f66cf0c110f66
- QUAD $0x4804c1834810cf4c
- QUAD $0x74c03949ec75c839
- QUAD $0xff48c704110ff21a
- QUAD $0x00c704c748f1ebc0
- QUAD $0x3949c0ff48000000
- LONG $0xc3f075c0
+ QUAD $0x7304fa8349ffffff
+ QUAD $0x89443aebc2894905
+ QUAD $0x49ca294903e183c1
+ QUAD $0x0f66c8280f66c201
+ QUAD $0x0000441f0f66c814
+ QUAD $0x110f66c70c110f66
+ QUAD $0x4904c0834810c74c
+ QUAD $0x74c98548ec75c239
+ QUAD $0x00000000801f0f15
+ QUAD $0xff49d704110f42f2
+ QUAD $0x8d48f275d0394dc2
+ QUAD $0x04c748c35d5bf865
+ QUAD $0xc0ff4800000000c7
+ LONG $0x75c03949
+ BYTE $0xf0
+ BYTE $0xc3
RET
// func convolveFloat64SSE2(dst []float64, sig []float64, ker []float64)
@@ -582,66 +823,134 @@ TEXT ·convolveFloat64SSE2(SB), NOSPLIT|NOFRAME, $0-72
MOVQ ker_len+56(FP), R9
QUAD $0x56415741e5894855
QUAD $0xe483485354415541
- QUAD $0xcd894d18ec8348f8
- QUAD $0xc9854d08247c8948
- QUAD $0x0f41c8294dc0940f
- QUAD $0xbd850fc10841c19c
- QUAD $0x4c01488d4d000001
- QUAD $0x854dc94e0f4cc139
- QUAD $0x4d000001a98e0fc9
- QUAD $0x000001408e0fed85
- QUAD $0x41e88945ea0c8d4a
- QUAD $0xfffffcba4903e083
- QUAD $0xea214d7fffffffff
- QUAD $0xea048d4a185e8d4c
- QUAD $0x24448948f8c08348
- QUAD $0xfffffebe49db3110
+ QUAD $0xc9854d18ec8348f8
+ QUAD $0x08244c894cc0940f
+ QUAD $0x41c19c0f41c8294d
+ QUAD $0x000003e0850fc108
+ QUAD $0x4cc1394c01688d4d
+ QUAD $0x7d10fd8349e94e0f
+ QUAD $0x0000022be9c93107
+ QUAD $0xc9854d08244c8b4c
+ QUAD $0x8d4c000001328e0f
+ QUAD $0x4500000010b87046
+ QUAD $0x2e6666666666d231
+ QUAD $0x0000000000841f0f
+ QUAD $0x4dc0570f66c18948
+ QUAD $0x570f66cb894cc389
+ QUAD $0x570f66d2570f66c9
+ QUAD $0x570f66e4570f66db
+ QUAD $0x570f66f6570f66ed
+ QUAD $0x00000000801f0fff
+ QUAD $0x66f8da44100f44f2
+ QUAD $0x100f4566c0140f45
+ QUAD $0xa053100f4566904b
+ QUAD $0x4566b05b100f4566
+ QUAD $0x590f4566c063100f
+ QUAD $0x4566c1580f4166c8
+ QUAD $0xca580f4166d0590f
+ QUAD $0x0f4166d8590f4566
+ QUAD $0x66e0590f4566d358
+ QUAD $0x100f4566dc580f41
+ QUAD $0x66c8590f4566d04b
+ QUAD $0x100f4566e1580f41
+ QUAD $0x66c8590f4566e04b
+ QUAD $0x100f4566e9580f41
+ QUAD $0x66c8590f4566f04b
+ QUAD $0x100f4566f1580f41
+ QUAD $0x4166c8590f45660b
+ QUAD $0x4808c38349f9580f
+ QUAD $0xffffff68850fcbff
+ QUAD $0x4266d704110f4266
+ QUAD $0x0f426610d74c110f
+ QUAD $0x110f426620d75411
+ QUAD $0x64110f426630d75c
+ QUAD $0xd76c110f426640d7
+ QUAD $0x60d774110f426650
+ QUAD $0x4870d77c110f4266
+ QUAD $0x4980e8834910418d
+ QUAD $0xed8e0fe8394cca89
+ QUAD $0x000000ebe9fffffe
+ QUAD $0x04eac149f0558d4d
+ QUAD $0xff490000009f840f
+ QUAD $0xfee08348d0894cc2
+ QUAD $0x0f6600000010b841
+ QUAD $0x0000441f0f66c057
+ QUAD $0x6680c744110f4266
+ QUAD $0x426690c744110f42
+ QUAD $0x0f4266a0c744110f
+ QUAD $0x110f4266b0c74411
+ QUAD $0x44110f4266c0c744
+ QUAD $0xc744110f4266d0c7
+ QUAD $0xf0c744110f4266e0
+ QUAD $0x4266c704110f4266
+ QUAD $0x0f426610c744110f
+ QUAD $0x110f426620c74411
+ QUAD $0x44110f426630c744
+ QUAD $0xc744110f426640c7
+ QUAD $0x60c744110f426650
+ QUAD $0x4970c744110f4266
+ QUAD $0x75fec0834820c083
+ QUAD $0xc2f641f0488d4987
+ QUAD $0x10b8413eeb0a7501
+ QUAD $0x570f66c931000000
+ QUAD $0x0f66cf04110f66c0
+ QUAD $0x44110f6610cf4411
+ QUAD $0x30cf44110f6620cf
+ QUAD $0x0f6640cf44110f66
+ QUAD $0x44110f6650cf4411
+ QUAD $0x70cf44110f6660cf
+ QUAD $0x2949e8894dc1894c
+ QUAD $0x48000001918e0fc8
+ QUAD $0x0fc085480824448b
+ QUAD $0x048d4c000001338e
+ QUAD $0x03e28341c28941c2
+ QUAD $0xfffffffffffcbb49
+ QUAD $0x1c8d48c321497fff
+ QUAD $0x448d4818c38348ce
+ QUAD $0x481024448948f8c2
+ QUAD $0xfffffebe49ce348d
QUAD $0xfdbf491fffffffff
QUAD $0x491fffffffffffff
QUAD $0xfffffffffffffcbc
- QUAD $0x0000441f0f26eb1f
- QUAD $0x110ff20824448b48
- QUAD $0xc38349c3ff48d804
- QUAD $0xcb394c08c6834808
- QUAD $0x00012f840ffd8949
- QUAD $0xc0570f66ef894c00
- QUAD $0x3145127304fd8349
- QUAD $0x2e6600000080e9ed
- QUAD $0x0000000000841f0f
- QUAD $0xed31451024548b48
- QUAD $0x0000000000841f0f
- QUAD $0xf2e8eb4c100f43f2
- QUAD $0x0ff2f0eb54100f43
+ QUAD $0x0000441f0f21eb1f
+ QUAD $0xc1ff48cf04110ff2
+ QUAD $0x08c6834808c38348
+ QUAD $0x840fc9394ccd894d
+ QUAD $0x48e9894d00000114
+ QUAD $0x6609730408247c83
+ QUAD $0x79ebed3145c0570f
+ QUAD $0x24548b48c0570f66
+ QUAD $0x00401f0fed314510
+ QUAD $0xf2e8eb4c100f42f2
+ QUAD $0x0ff2f0eb54100f42
QUAD $0x894cc8580ff20a59
- QUAD $0x14590ff2f0314ce8
- QUAD $0x0f43f2d1580ff2c1
- QUAD $0x4ce8894cf8eb4c10
- QUAD $0xf2c10c590ff2f831
- QUAD $0x04100f43f2ca580f
- QUAD $0xf2e0314ce8894ceb
- QUAD $0xc1580ff2c104590f
- QUAD $0xe0c2834804c58349
- QUAD $0xc0854da375ea394d
- QUAD $0x8d4affffff4a840f
- QUAD $0x8b4800000000ed04
- QUAD $0x8d4ec22948102454
- QUAD $0x00401f0fc031ee2c
+ QUAD $0x590f41f2f0314ce8
+ QUAD $0x42f2d1580ff2c014
+ QUAD $0xe8894cf8eb4c100f
+ QUAD $0x0c590f41f2f8314c
+ QUAD $0x0f42f2ca580ff2c0
+ QUAD $0x314ce8894ceb0410
+ QUAD $0xf2c004590f41f2e0
+ QUAD $0x4804c58349c1580f
+ QUAD $0xa075eb394de0c283
+ QUAD $0xffff57840fd2854d
+ QUAD $0x000000ed048d4aff
+ QUAD $0x29481024548b4800
+ QUAD $0x90c031ee2c8d4ec2
QUAD $0xf200c54c100f41f2
QUAD $0x48c1580ff20a590f
QUAD $0x3949f8c28348c0ff
- QUAD $0xffffff10e9e575c0
- QUAD $0xc031047304f98349
- QUAD $0xfffffffcb8483feb
- QUAD $0x31c8214c7fffffff
- QUAD $0x548b48c0570f66c9
- QUAD $0x2e66666666660824
- QUAD $0x0000000000841f0f
- QUAD $0x110f66ca04110f66
- QUAD $0x4804c1834810ca44
- QUAD $0x74c13949ec75c839
- QUAD $0x906608244c8b4817
- QUAD $0x00000000c104c748
- QUAD $0xf075c13949c0ff48
+ QUAD $0xffffff20e9e575c2
+ QUAD $0x8949057304f88349
+ QUAD $0xe083e8894435ebc8
+ QUAD $0x66c80149c0294903
+ QUAD $0x0000441f0fc0570f
+ QUAD $0x110f66cf04110f66
+ QUAD $0x4904c1834810cf44
+ QUAD $0x74c08548ec75c839
+ QUAD $0x00000000801f0f17
+ QUAD $0x00000000c704c74a
+ QUAD $0xf075c5394dc0ff49
QUAD $0x415c415bd8658d48
LONG $0x415e415d
BYTE $0x5f
@@ -659,49 +968,123 @@ TEXT ·correlateFloat64SSE2(SB), NOSPLIT|NOFRAME, $0-72
MOVQ ker_len+56(FP), R9
QUAD $0x294dc0940fc9854d
QUAD $0xc20841c29c0f41c8
- QUAD $0x8d490000014e850f
- QUAD $0x4e0f48c1394c0140
- QUAD $0x013a8e0fc08548c1
- QUAD $0xe58e0fc9854d0000
- QUAD $0x41e5894855000000
- QUAD $0xf8e4834853564157
- QUAD $0xb84903e183c98944
- QUAD $0x7ffffffffffffffc
- QUAD $0x4518568d4cc8214d
- QUAD $0x00401f0f1eebdb31
- QUAD $0xff49df04110f42f2
- QUAD $0xc6834808c28349c3
- QUAD $0x0093840fc3394908
- QUAD $0xdb31c0570f660000
- QUAD $0x2e665a7204f98349
+ QUAD $0x485500000394850f
+ QUAD $0x485356415741e589
+ QUAD $0x4c01408d49f8e483
+ QUAD $0x8348c14e0f48c139
+ QUAD $0x28e9c931077d10f8
+ QUAD $0x8e0fc9854d000002
+ QUAD $0x70468d4c0000013a
+ QUAD $0x314500000010bb41
+ QUAD $0x2e666666666666d2
+ QUAD $0x0000000000841f0f
+ QUAD $0x45c0570f66d9894c
+ QUAD $0x0f66c9570f66db31
+ QUAD $0x0f66db570f66d257
+ QUAD $0x0f66ed570f66e457
+ QUAD $0x2e66ff570f66f657
QUAD $0x0000000000841f0f
- QUAD $0xf2e8da4c100f41f2
- QUAD $0x0ff2f0da54100f41
+ QUAD $0x4566da04100f46f2
+ QUAD $0x4c100f4766c0140f
+ QUAD $0xd854100f476690d8
+ QUAD $0xb0d85c100f4766a0
+ QUAD $0x66c0d864100f4766
+ QUAD $0x580f4166c8590f45
+ QUAD $0x4166d0590f4566c1
+ QUAD $0xd8590f4566ca580f
+ QUAD $0x0f4566d3580f4166
+ QUAD $0x66dc580f4166e059
+ QUAD $0x4566d0d84c100f47
+ QUAD $0xe1580f4166c8590f
+ QUAD $0x66e0d84c100f4766
+ QUAD $0x580f4166c8590f45
+ QUAD $0xf0d84c100f4766e9
+ QUAD $0x0f4166c8590f4566
+ QUAD $0xd80c100f4766f158
+ QUAD $0x0f4166c8590f4566
+ QUAD $0xd9394dc3ff49f958
+ QUAD $0x4266ffffff62850f
+ QUAD $0x110f4266d704110f
+ QUAD $0x54110f426610d74c
+ QUAD $0xd75c110f426620d7
+ QUAD $0x40d764110f426630
+ QUAD $0x6650d76c110f4266
+ QUAD $0x426660d774110f42
+ QUAD $0x598d4c70d77c110f
+ QUAD $0xca894980e8834910
+ QUAD $0xfffee78e0fc33949
+ QUAD $0x8d4c000000e5e9ff
+ QUAD $0x840f04eac149f050
+ QUAD $0x4cc2ff4900000099
+ QUAD $0xb841fee18348d189
+ QUAD $0xc0570f6600000010
+ QUAD $0x6680c744110f4266
+ QUAD $0x426690c744110f42
+ QUAD $0x0f4266a0c744110f
+ QUAD $0x110f4266b0c74411
+ QUAD $0x44110f4266c0c744
+ QUAD $0xc744110f4266d0c7
+ QUAD $0xf0c744110f4266e0
+ QUAD $0x4266c704110f4266
+ QUAD $0x0f426610c744110f
+ QUAD $0x110f426620c74411
+ QUAD $0x44110f426630c744
+ QUAD $0xc744110f426640c7
+ QUAD $0x60c744110f426650
+ QUAD $0x4970c744110f4266
+ QUAD $0x75fec1834820c083
+ QUAD $0xc2f641f0488d4987
+ QUAD $0x10b8413eeb0a7501
+ QUAD $0x570f66c931000000
+ QUAD $0x0f66cf04110f66c0
+ QUAD $0x44110f6610cf4411
+ QUAD $0x30cf44110f6620cf
+ QUAD $0x0f6640cf44110f66
+ QUAD $0x44110f6650cf4411
+ QUAD $0x70cf44110f6660cf
+ QUAD $0x2949c08949c1894c
+ QUAD $0x4d000001318e0fc8
+ QUAD $0x000000d58e0fc985
+ QUAD $0x4903e08341c88945
+ QUAD $0xfffffffffffffcba
+ QUAD $0xce1c8d4cca214d7f
+ QUAD $0xce348d4818c38349
+ QUAD $0x0000441f0f661feb
+ QUAD $0xc1ff48cf04110ff2
+ QUAD $0x08c6834808c38349
+ QUAD $0x0000e7840fc13948
+ QUAD $0x49db31c0570f6600
+ QUAD $0x2e66665b7204f983
+ QUAD $0x0000000000841f0f
+ QUAD $0xf2e8db4c100f41f2
+ QUAD $0x0ff2f0db54100f41
QUAD $0xda54590ff2da0c59
QUAD $0x0f41f2c8580ff208
- QUAD $0x5c590ff2f8da5c10
+ QUAD $0x5c590ff2f8db5c10
QUAD $0x41f2d1580ff210da
- QUAD $0x44590ff2da04100f
+ QUAD $0x44590ff2db04100f
QUAD $0x0ff2da580ff218da
QUAD $0x394904c38348c358
- QUAD $0x8074c98548b575d8
+ QUAD $0x8074c0854db575da
QUAD $0xde1c8d48da348d4c
QUAD $0x0000441f0fff3145
QUAD $0x43f2fb0c100f42f2
QUAD $0xc1580ff2fe0c590f
- QUAD $0xe875f9394cc7ff49
- QUAD $0x658d48ffffff53e9
- QUAD $0xc35d5f415e415be8
- QUAD $0xc931047304f88348
- QUAD $0xfffffffcb94832eb
- QUAD $0x31c121487fffffff
- QUAD $0x2e666666c0570fd2
+ QUAD $0xe875f8394dc7ff49
+ QUAD $0xf88349ffffff53e9
+ QUAD $0x38ebc88949057304
+ QUAD $0xd0294903e283c289
+ QUAD $0x66c0570f66c80149
QUAD $0x0000000000841f0f
- QUAD $0xd744110fd704110f
- QUAD $0xd1394804c2834810
- QUAD $0xcf04c7480bebee75
- QUAD $0x48c1ff4800000000
- LONG $0xf075c839
+ QUAD $0x110f66cf04110f66
+ QUAD $0x4904c1834810cf44
+ QUAD $0x74d28548ec75c839
+ QUAD $0x00000000801f0f17
+ QUAD $0x00000000c704c74a
+ QUAD $0xf075c0394cc0ff49
+ QUAD $0x415e415be8658d48
+ BYTE $0x5f
+ BYTE $0x5d
BYTE $0xc3
RET
@@ -1406,21 +1789,23 @@ TEXT ·movingAverageFloat32SSE2(SB), NOSPLIT|NOFRAME, $0-56
MOVQ width+48(FP), R8
QUAD $0x294cc09e0fc0854d
QUAD $0xc10841c19c0f41c1
- QUAD $0x8d48000000e3850f
- QUAD $0x4e0f48ca39480141
- QUAD $0x00cf8e0fc08548c2
- QUAD $0x4853e58948550000
- QUAD $0xc02a0f49f3f8e483
- QUAD $0x83c28944ff488d49
- QUAD $0xfffffff8b94907e2
- QUAD $0x4cc1214d7fffffff
- QUAD $0x22ebd231451c468d
- QUAD $0x0000000000841f0f
- QUAD $0x110f42f3c85e0ff3
- QUAD $0xc08349c2ff49970c
- QUAD $0xc2394904c6834804
- QUAD $0xdb3145c9570f7974
- QUAD $0x2e664f7207f98348
+ QUAD $0x4855000000f5850f
+ QUAD $0xe48348535641e589
+ QUAD $0xfffffffff8b848f8
+ QUAD $0x4801498d4c7fffff
+ QUAD $0x8349ca4e0f4cca39
+ QUAD $0x000000cc8d0f10f9
+ QUAD $0xb88d0fc9394cc931
+ QUAD $0x49f3c0570f000000
+ QUAD $0x45ff508d49c02a0f
+ QUAD $0x214c07e28341c289
+ QUAD $0xc083498e048d4cc0
+ QUAD $0x901aeb8e348d481c
+ QUAD $0x0c110ff3c85e0ff3
+ QUAD $0x04c08349c1ff488f
+ QUAD $0x74c9394c04c68348
+ QUAD $0x48db3145c9570f7a
+ QUAD $0x2e6666507207fa83
QUAD $0x0000000000841f0f
QUAD $0xf3e4984c580f43f3
QUAD $0x43f3e8984c580f43
@@ -1429,14 +1814,50 @@ TEXT ·movingAverageFloat32SSE2(SB), NOSPLIT|NOFRAME, $0-56
QUAD $0x4c580f43f3f4984c
QUAD $0x984c580f43f3f898
QUAD $0x49980c580f43f3fc
- QUAD $0xc075d9394d08c383
- QUAD $0x1c8d4e8b74d28548
+ QUAD $0xc075d8394c08c383
+ QUAD $0x1c8d4e8b74d2854d
QUAD $0x0000441f0fdb319e
QUAD $0xff489b0c580f41f3
- QUAD $0x6de9f275da3948c3
- QUAD $0x5bf8658d48ffffff
- BYTE $0x5d
- BYTE $0xc3
+ QUAD $0x6de9f275da3949c3
+ QUAD $0x5bf0658d48ffffff
+ QUAD $0x2a0f49f3c35d5e41
+ QUAD $0x508d4800c0c60fc0
+ QUAD $0x34568d4cc2214c06
+ QUAD $0xdb314500000010bb
+ QUAD $0x0000441f0f6670eb
+ QUAD $0x280fe3280fec280f
+ QUAD $0x9e348d4ed1280fda
+ QUAD $0xca580f9e0c100f41
+ QUAD $0x580f109e54100f41
+ QUAD $0x0f209e5c100f41d3
+ QUAD $0x309e64100f41dc58
+ QUAD $0x0f42c85e0fe5580f
+ QUAD $0x0f42d05e0f9f0c11
+ QUAD $0x42d85e0f109f5411
+ QUAD $0xe05e0f209f5c110f
+ QUAD $0x8d48309f64110f42
+ QUAD $0x894940c283491059
+ QUAD $0xfea88f0fcb394ccb
+ QUAD $0xc9570fd98948ffff
+ QUAD $0xdb570fd2570fdb31
+ QUAD $0x0f01f88349e4570f
+ QUAD $0x2e6666ffffff7b84
+ QUAD $0x0000000000841f0f
+ QUAD $0x580fcc9a6c100f41
+ QUAD $0x0fd09a4c100f41e9
+ QUAD $0xdc9a6c100f41cd58
+ QUAD $0x9a54100f41ea580f
+ QUAD $0x6c100f41d5580fe0
+ QUAD $0x100f41eb580fec9a
+ QUAD $0x0f41f4580ffc9a74
+ QUAD $0x41dd580ff09a5c10
+ QUAD $0x48e6580f9a24100f
+ QUAD $0xb075da394802c383
+ QUAD $0xff16850f01c0f641
+ LONG $0x44e9ffff
+ BYTE $0xff
+ BYTE $0xff
+ BYTE $0xff
RET
// func shiftDivFloat32SSE2(dst []float32, a []float32, shift float32, denom float32)
@@ -1502,21 +1923,59 @@ TEXT ·movingAverageFloat64SSE2(SB), NOSPLIT|NOFRAME, $0-56
MOVQ width+48(FP), R8
QUAD $0x294cc09e0fc0854d
QUAD $0xc10841c19c0f41c1
- QUAD $0x8d48000000e3850f
- QUAD $0x4e0f48ca39480141
- QUAD $0x00cf8e0fc08548c2
- QUAD $0x4853e58948550000
- QUAD $0xc02a0f49f2f8e483
- QUAD $0x83c28944ff488d49
- QUAD $0xfffffff8b94907e2
- QUAD $0x4cc1214d7fffffff
- QUAD $0x22ebd2314538468d
+ QUAD $0x485500000213850f
+ QUAD $0x48f8e4834853e589
+ QUAD $0x0f48ca394801418d
+ QUAD $0x077d10f88348c24e
+ QUAD $0xf20000011ce9c931
+ QUAD $0xc0140f66c02a0f49
+ QUAD $0x0010ba4170568d48
+ QUAD $0x001f0fc931450000
+ QUAD $0x45ff570f66d1894c
+ QUAD $0x66c0570f4566d231
+ QUAD $0x66ed570f66f6570f
+ QUAD $0x66db570f66e4570f
+ QUAD $0x66c9570f66d2570f
QUAD $0x0000000000841f0f
- QUAD $0x110f42f2c85e0ff2
- QUAD $0xc08349c2ff49d70c
- QUAD $0xc2394908c6834808
- QUAD $0x3145c9570f667974
- QUAD $0x664e7207f98348db
+ QUAD $0x6690d24c100f4666
+ QUAD $0x100f4666f9580f41
+ QUAD $0xc1580f4566a0d24c
+ QUAD $0x66b0d24c100f4666
+ QUAD $0x100f4666f1580f41
+ QUAD $0xe9580f4166c0d24c
+ QUAD $0x66d0d24c100f4666
+ QUAD $0x100f4666e1580f41
+ QUAD $0xd9580f4166e0d24c
+ QUAD $0x66f0d24c100f4666
+ QUAD $0x100f4666d1580f41
+ QUAD $0x49c9580f4166d20c
+ QUAD $0x669975d0394dc2ff
+ QUAD $0x3c110f4266f85e0f
+ QUAD $0x4666c05e0f4466cf
+ QUAD $0x5e0f6610cf44110f
+ QUAD $0x20cf74110f4266f0
+ QUAD $0x110f4266e85e0f66
+ QUAD $0x66e05e0f6630cf6c
+ QUAD $0x0f6640cf64110f42
+ QUAD $0xcf5c110f4266d85e
+ QUAD $0x0f4266d05e0f6650
+ QUAD $0xc85e0f6660cf5411
+ QUAD $0x4c70cf4c110f4266
+ QUAD $0x4980ea834810518d
+ QUAD $0xfd8e0fc23949c989
+ QUAD $0x8d0fc13948fffffe
+ QUAD $0xf2c0570f000000c7
+ QUAD $0xff508d49c02a0f49
+ QUAD $0x4907e18341c18945
+ QUAD $0xfffffffffffff8ba
+ QUAD $0xce048d4cc2214d7f
+ QUAD $0xce348d4838c08349
+ QUAD $0x0000441f0f661feb
+ QUAD $0x0c110ff2c85e0ff2
+ QUAD $0x08c08349c1ff48cf
+ QUAD $0x74c1394808c68348
+ QUAD $0xdb3145c9570f667a
+ QUAD $0x2e664f7207fa8348
QUAD $0x0000000000841f0f
QUAD $0xf2c8d84c580f43f2
QUAD $0x43f2d0d84c580f43
@@ -1525,11 +1984,11 @@ TEXT ·movingAverageFloat64SSE2(SB), NOSPLIT|NOFRAME, $0-56
QUAD $0x4c580f43f2e8d84c
QUAD $0xd84c580f43f2f0d8
QUAD $0x49d80c580f43f2f8
- QUAD $0xc075d9394d08c383
- QUAD $0x1c8d4e8b74d28548
+ QUAD $0xc075da394d08c383
+ QUAD $0x1c8d4e8b74c9854d
QUAD $0x0000441f0fdb31de
QUAD $0xff48db0c580f41f2
- QUAD $0x6de9f275da3948c3
+ QUAD $0x6de9f275d93949c3
QUAD $0x5bf8658d48ffffff
BYTE $0x5d
BYTE $0xc3
diff --git a/internal/arm64/numeric_neon_arm64.s b/internal/arm64/numeric_neon_arm64.s
index 3df269c..3f7dc75 100644
--- a/internal/arm64/numeric_neon_arm64.s
+++ b/internal/arm64/numeric_neon_arm64.s
@@ -233,20 +233,74 @@ TEXT ·polyEvalFloat32NEON(SB), NOSPLIT|NOFRAME, $0-72
MOVD coeffs_len+56(FP), R5
WORD $0xeb04007f
WORD $0x9a84b068
+ WORD $0xb40000c5
+ WORD $0x8b05084a
+ WORD $0xf100411f
+ WORD $0x5400012a
+ WORD $0xaa1f03e9
+ WORD $0x14000034
WORD $0xf100051f
- WORD $0xb4000305
- WORD $0x540007eb
- WORD $0x8b05084b
- WORD $0xf10008bf
- WORD $0xbc5fc160
- WORD $0x5400030b
+ WORD $0x54000d8b
+ WORD $0xf100211f
+ WORD $0x54000b62
+ WORD $0xaa1f03e9
+ WORD $0x14000063
+ WORD $0xd1001149
+ WORD $0xf10004ab
+ WORD $0x4d40c920
+ WORD $0x5400044d
+ WORD $0xaa1f03ec
+ WORD $0x5280020d
+ WORD $0x8b0c082e
+ WORD $0x4ea01c10
+ WORD $0x4ea01c07
+ WORD $0x4ea01c06
+ WORD $0x4ea01c02
+ WORD $0xaa0d03e9
+ WORD $0xad400dc1
+ WORD $0xd100214d
+ WORD $0xad4115c4
+ WORD $0xaa0b03ee
+ WORD $0xbc5fc5b1
+ WORD $0x6e21de10
+ WORD $0x6e23dce7
+ WORD $0x6e24dcc6
+ WORD $0x6e25dc42
+ WORD $0x4e040631
+ WORD $0xd10005ce
+ WORD $0xf10001df
+ WORD $0x4e30d630
+ WORD $0x4e27d627
+ WORD $0x4e26d626
+ WORD $0x4e22d622
+ WORD $0x54fffe8c
+ WORD $0x8b0c080e
+ WORD $0x9100412d
+ WORD $0xaa0903ec
+ WORD $0xeb0801bf
+ WORD $0xad001dd0
+ WORD $0xad0109c6
+ WORD $0x54fffc6d
+ WORD $0x14000009
WORD $0xaa1f03e9
- WORD $0xd10004aa
- WORD $0xd100216b
+ WORD $0x9100800b
+ WORD $0x9100812c
+ WORD $0x91004129
+ WORD $0xad3f0160
+ WORD $0xeb08019f
+ WORD $0xac820160
+ WORD $0x54ffff6d
+ WORD $0xeb09010b
+ WORD $0x5400072d
+ WORD $0xbc5fc140
+ WORD $0xf10008bf
+ WORD $0x5400024b
+ WORD $0xd10004ab
+ WORD $0xd100214a
WORD $0x1e204002
WORD $0xbc697821
- WORD $0xaa0b03ec
- WORD $0xaa0a03ed
+ WORD $0xaa0a03ec
+ WORD $0xaa0b03ed
WORD $0x1e220822
WORD $0xbc5fc583
WORD $0xd10005ad
@@ -257,38 +311,34 @@ TEXT ·polyEvalFloat32NEON(SB), NOSPLIT|NOFRAME, $0-72
WORD $0x91000529
WORD $0xeb08013f
WORD $0x54fffe61
- WORD $0x14000029
- WORD $0x5400050b
- WORD $0xf100211f
- WORD $0x540000e2
- WORD $0xaa1f03e9
- WORD $0x1400000f
- WORD $0xf100211f
- WORD $0x54000262
- WORD $0xaa1f03e9
- WORD $0x1400001b
- WORD $0x6f00e400
- WORD $0x927ded09
- WORD $0x9100400a
- WORD $0x927ded0b
- WORD $0xf100216b
- WORD $0xad3f8140
- WORD $0x9100814a
+ WORD $0x14000025
+ WORD $0xf100217f
+ WORD $0x540001c3
+ WORD $0x9240090a
+ WORD $0x4e040401
+ WORD $0x8b09080c
+ WORD $0xcb0a016b
+ WORD $0x8b0a012d
+ WORD $0x8b0b0129
+ WORD $0x9100418b
+ WORD $0xcb0801ac
+ WORD $0xb100218c
+ WORD $0xad3f8561
+ WORD $0x9100816b
WORD $0x54ffffa1
- WORD $0xeb09011f
- WORD $0x540002c0
+ WORD $0xb40002ca
WORD $0x8b09080a
WORD $0xcb090108
WORD $0xf1000508
- WORD $0xb800455f
+ WORD $0xbc004540
WORD $0x54ffffc1
WORD $0x14000010
- WORD $0x4e040401
+ WORD $0x6f00e400
WORD $0x927ded09
WORD $0x9100400a
WORD $0x927ded0b
WORD $0xf100216b
- WORD $0xad3f8541
+ WORD $0xad3f8140
WORD $0x9100814a
WORD $0x54ffffa1
WORD $0xeb09011f
@@ -296,7 +346,7 @@ TEXT ·polyEvalFloat32NEON(SB), NOSPLIT|NOFRAME, $0-72
WORD $0x8b09080a
WORD $0xcb090108
WORD $0xf1000508
- WORD $0xbc004540
+ WORD $0xb800455f
WORD $0x54ffffc1
WORD $0xd65f03c0
RET
@@ -309,41 +359,346 @@ TEXT ·convolveFloat32NEON(SB), NOSPLIT|NOFRAME, $0-72
MOVD dst_len+8(FP), R3
MOVD sig_len+32(FP), R4
MOVD ker_len+56(FP), R5
- WORD $0xb4000ae5
+ WORD $0xb40031e5
WORD $0xeb050088
- WORD $0x54000aab
+ WORD $0x540031ab
+ WORD $0xd10443ff
WORD $0xeb08007f
+ WORD $0x6d0c3bef
WORD $0x9a88d468
- WORD $0xf100051f
- WORD $0x54000a2b
- WORD $0xf10004bf
- WORD $0x5400078b
- WORD $0x8b05084b
+ WORD $0x6d0d33ed
+ WORD $0xf100411f
+ WORD $0x6d0e2beb
+ WORD $0x6d0f23e9
+ WORD $0xf90083fd
+ WORD $0x5400006a
WORD $0xaa1f03e9
- WORD $0x927decaa
- WORD $0x9100402c
+ WORD $0x14000128
+ WORD $0xf10000bf
+ WORD $0x540023ad
+ WORD $0x8b050849
+ WORD $0xaa1f03ef
+ WORD $0x927ef0aa
+ WORD $0x9100802b
WORD $0xd100104d
- WORD $0xd100416b
+ WORD $0x9100f02e
+ WORD $0xd100412c
+ WORD $0x52800210
+ WORD $0x1400000e
+ WORD $0x8b0f0811
+ WORD $0x91004130
+ WORD $0x9101016b
+ WORD $0xeb08021f
+ WORD $0x910101ce
+ WORD $0xaa0903ef
+ WORD $0xbd000224
+ WORD $0x3c804223
+ WORD $0x3c814225
+ WORD $0x3c824226
+ WORD $0x2d06f637
+ WORD $0xbd003e34
+ WORD $0x5400220c
+ WORD $0xf10010bf
+ WORD $0xaa1003e9
+ WORD $0x54000142
+ WORD $0x6f00e403
+ WORD $0x2f00e414
+ WORD $0xaa1f03f1
+ WORD $0x2f00e41d
+ WORD $0x2f00e417
+ WORD $0x2f00e404
+ WORD $0x6f00e405
+ WORD $0x6f00e406
+ WORD $0x140000de
+ WORD $0x2f00e414
+ WORD $0x2f00e41d
+ WORD $0xaa0b03e4
+ WORD $0x2f00e417
+ WORD $0x2f00e407
+ WORD $0xaa0c03f0
+ WORD $0x2f00e402
+ WORD $0x2f00e416
+ WORD $0x927ef0b1
+ WORD $0x2f00e406
+ WORD $0x2f00e40f
+ WORD $0xaa0b03e3
+ WORD $0x2f00e413
+ WORD $0x2f00e415
+ WORD $0x2f00e405
+ WORD $0x2f00e410
+ WORD $0x2f00e411
+ WORD $0x2f00e400
+ WORD $0x2f00e403
+ WORD $0x2f00e404
+ WORD $0x3cdf0618
+ WORD $0x3cc1c099
+ WORD $0x3cc1809a
+ WORD $0x3d802be0
+ WORD $0x4ea21c52
+ WORD $0x4ea00b18
+ WORD $0xad0157f1
+ WORD $0x3cc0c089
+ WORD $0x4eaf1df5
+ WORD $0xf1001231
+ WORD $0x6e184318
+ WORD $0x6e39df0b
+ WORD $0x3cc14099
+ WORD $0x6e3adf0c
+ WORD $0x3cc10c7a
+ WORD $0x6e39df0d
+ WORD $0x3cde0099
+ WORD $0x6e3adf0e
+ WORD $0x3cde409a
+ WORD $0x5e0c057b
+ WORD $0x1e2b2a80
+ WORD $0x6e39df01
+ WORD $0x5e0c059c
+ WORD $0x1e2c2ba2
+ WORD $0x5e140568
+ WORD $0x5e0c05bd
+ WORD $0x5e14059f
+ WORD $0x5e1c056b
+ WORD $0x5e0c05de
+ WORD $0x1e2e28e7
+ WORD $0x5e1405aa
+ WORD $0x3d802fe1
+ WORD $0x6e3adf01
+ WORD $0x1e3b2800
+ WORD $0x1e3c2842
+ WORD $0x5e1c058c
+ WORD $0x3cde809b
+ WORD $0x5e1405cf
+ WORD $0x6e29df14
+ WORD $0x5e1c05d9
+ WORD $0x1e3e28e7
+ WORD $0x6e3bdf11
+ WORD $0x4eb01e1e
+ WORD $0x3d8027e1
+ WORD $0x1e2d2ae1
+ WORD $0x1e282800
+ WORD $0x1e3f285a
+ WORD $0x5e1c05ad
+ WORD $0x3cc08090
+ WORD $0x1e342a57
+ WORD $0x3cdf8088
+ WORD $0x3cdec09c
+ WORD $0x6e30df1b
+ WORD $0x1e2f28e7
+ WORD $0xad02c7f4
+ WORD $0x1e3d2821
+ WORD $0x1e2b2800
+ WORD $0x3cc04091
+ WORD $0x5e0c0690
+ WORD $0xad7fbc89
+ WORD $0x6e31df02
+ WORD $0x6e28df1f
+ WORD $0x6e3cdf1c
+ WORD $0x1e3b2ace
+ WORD $0x5e0c0774
+ WORD $0x3cdf4096
+ WORD $0x1e2a2821
+ WORD $0xbd010fe0
+ WORD $0x1e2c2b40
+ WORD $0x1e302af0
+ WORD $0x3cdfc08a
+ WORD $0x6e29df1d
+ WORD $0x1e2228d2
+ WORD $0x1e204086
+ WORD $0x4eb31e64
+ WORD $0x5e0c0453
+ WORD $0x6e2adf08
+ WORD $0xad000bfb
+ WORD $0xbd010be0
+ WORD $0x1e2d2820
+ WORD $0x4ebb1f6a
+ WORD $0x6e36df1b
+ WORD $0x6e2fdf16
+ WORD $0x5e0c07fa
+ WORD $0x5e0c0798
+ WORD $0x5e0c07af
+ WORD $0xaa0303e4
+ WORD $0x1e28288c
+ WORD $0x4ea81d0b
+ WORD $0x5e0c0509
+ WORD $0xbd008fe0
+ WORD $0x1e3928e0
+ WORD $0x5e0c0767
+ WORD $0x1e292989
+ WORD $0x5e1406cc
+ WORD $0x3d801fe0
+ WORD $0x1e3429c0
+ WORD $0x4ebf1ff4
+ WORD $0x2d0943e0
+ WORD $0x1e332a40
+ WORD $0x1e362ab3
+ WORD $0x5e0c06d2
+ WORD $0xbd0047e0
+ WORD $0xad4103e1
+ WORD $0x1e3f2815
+ WORD $0xad4483f1
+ WORD $0x1e3c282e
+ WORD $0x3dc01be1
+ WORD $0x1e3b28bf
+ WORD $0x4ebd1fa5
+ WORD $0x1e322a7d
+ WORD $0x1e212800
+ WORD $0x5e0c0437
+ WORD $0x1e3a2aa8
+ WORD $0x5e140455
+ WORD $0x4eb41e82
+ WORD $0x1e252bcd
+ WORD $0x1e31287e
+ WORD $0x4ea11c23
+ WORD $0x3dc02fe1
+ WORD $0x1e272bf9
+ WORD $0x1e3829df
+ WORD $0x1e372813
+ WORD $0x3dc017e0
+ WORD $0x5e140557
+ WORD $0x1e2128c4
+ WORD $0x5e0c0430
+ WORD $0x5e140578
+ WORD $0x5e14040a
+ WORD $0x4ea11c26
+ WORD $0x5e0c0621
+ WORD $0x1e2f29ba
+ WORD $0x5e14078d
+ WORD $0x5e1404ae
+ WORD $0x5e14076f
+ WORD $0x5e1c079c
+ WORD $0x1e302887
+ WORD $0x5e140690
+ WORD $0x4eab1d74
+ WORD $0xbd404feb
+ WORD $0x1e212bd2
+ WORD $0x4ea51cbe
+ WORD $0x5e1404c5
+ WORD $0x5e140624
+ WORD $0x5e140461
+ WORD $0x1e2a296a
+ WORD $0x1e2c2bab
+ WORD $0x1e2f2b2c
+ WORD $0x1e302910
+ WORD $0x5e1c0459
+ WORD $0x5e1c06dd
+ WORD $0x5e1c07de
+ WORD $0x5e1c04c8
+ WORD $0xbd00a3ea
+ WORD $0x2d488bea
+ WORD $0x1e3d296f
+ WORD $0xbd410bfd
+ WORD $0x1e372857
+ WORD $0x1e352955
+ WORD $0x1e38292a
+ WORD $0x1e2e2b58
+ WORD $0x1e2d2bee
+ WORD $0x1e212a6d
+ WORD $0x1e242a41
+ WORD $0x1e2528f2
+ WORD $0xad4013e5
+ WORD $0x5e1c0407
+ WORD $0x5e1c069a
+ WORD $0x5e1c077f
+ WORD $0x5e1c0629
+ WORD $0x5e1c047b
+ WORD $0xbd40a3e0
+ WORD $0x5e1c0484
+ WORD $0x5e1c04a5
+ WORD $0x1e3c29d1
+ WORD $0xbd410ff4
+ WORD $0x1e272802
+ WORD $0x1e3a2953
+ WORD $0x3dc01fe7
+ WORD $0x1e3b29a0
+ WORD $0x1e292823
+ WORD $0x1e252af6
+ WORD $0x1e242aa6
+ WORD $0x1e392a15
+ WORD $0x1e3f2985
+ WORD $0x1e3e2b10
+ WORD $0x1e282a44
+ WORD $0xbd408ff7
+ WORD $0x54ffe881
+ WORD $0x6e0c0403
+ WORD $0x6e0c06a5
+ WORD $0xeb0a00bf
+ WORD $0x6e0c06c6
+ WORD $0x927ef0b1
+ WORD $0x6e140623
+ WORD $0x6e140665
+ WORD $0x6e140446
+ WORD $0x6e1c0603
+ WORD $0x6e1c05e5
+ WORD $0x6e1c04e6
+ WORD $0x54ffe160
+ WORD $0xd37ef630
+ WORD $0xcb1100b1
+ WORD $0xaa0e03e3
+ WORD $0x8b100064
+ WORD $0xbc7179a0
+ WORD $0xf1000631
+ WORD $0x2d7f4891
+ WORD $0xbc5c4081
+ WORD $0x3cdc8082
+ WORD $0x3cdd8087
+ WORD $0x91001063
+ WORD $0x3cde8090
+ WORD $0xbd400093
+ WORD $0x1e210801
+ WORD $0x4f809042
+ WORD $0x4f8090e7
+ WORD $0x1e310811
+ WORD $0x4f809210
+ WORD $0x1e320812
+ WORD $0x1e330800
+ WORD $0x1e212884
+ WORD $0x4e22d463
+ WORD $0x4e27d4a5
+ WORD $0x1e312af7
+ WORD $0x4e30d4c6
+ WORD $0x1e322bbd
+ WORD $0x1e202a94
+ WORD $0x54fffd01
+ WORD $0x17fffeee
+ WORD $0x6f00e400
+ WORD $0xaa1f03e9
+ WORD $0x9100800a
+ WORD $0x9100812b
+ WORD $0x91004129
+ WORD $0xad3f0140
+ WORD $0xeb08017f
+ WORD $0xac820140
+ WORD $0x54ffff6d
+ WORD $0xeb09010b
+ WORD $0x54000a4d
+ WORD $0xf10004bf
+ WORD $0x5400078b
+ WORD $0x8b09082a
+ WORD $0x8b05084d
+ WORD $0x927decab
+ WORD $0xd100104e
+ WORD $0x9100414c
+ WORD $0xd10041ad
WORD $0x14000007
WORD $0xbc297800
WORD $0x91000529
WORD $0x9100118c
WORD $0xeb08013f
- WORD $0x91001021
- WORD $0x54000840
+ WORD $0x9100114a
+ WORD $0x54000860
WORD $0x2f00e400
WORD $0xf10020bf
WORD $0x54000062
- WORD $0xaa1f03ef
+ WORD $0xaa1f03f0
WORD $0x14000021
- WORD $0xaa0c03ee
- WORD $0xaa0b03ef
- WORD $0x927decb0
+ WORD $0xaa0d03ef
+ WORD $0xaa0c03f0
+ WORD $0x927decb1
WORD $0xad7f85e2
- WORD $0xd1002210
- WORD $0xad7f91c3
+ WORD $0xd1002231
+ WORD $0xad7f9203
+ WORD $0x91008210
WORD $0xd10081ef
- WORD $0x910081ce
WORD $0x4ea00821
WORD $0x4ea00842
WORD $0x6e014021
@@ -364,38 +719,45 @@ TEXT ·convolveFloat32NEON(SB), NOSPLIT|NOFRAME, $0-72
WORD $0x5e1c0441
WORD $0x1e232800
WORD $0x1e212800
- WORD $0xb5fffcf0
- WORD $0xeb0a00bf
- WORD $0x927decaf
+ WORD $0xb5fffcf1
+ WORD $0xeb0b00bf
+ WORD $0x927decb0
WORD $0x54fffac0
- WORD $0x8b0f082e
- WORD $0xcb0f00af
- WORD $0xbc4045c1
- WORD $0xbc6f79a2
- WORD $0xf10005ef
+ WORD $0x8b10094f
+ WORD $0xcb1000b0
+ WORD $0xbc4045e1
+ WORD $0xbc7079c2
+ WORD $0xf1000610
WORD $0x1e220821
WORD $0x1e212800
WORD $0x54ffff61
WORD $0x17ffffcd
- WORD $0xf100211f
- WORD $0x54000062
- WORD $0xaa1f03e9
- WORD $0x1400000b
+ WORD $0xf100217f
+ WORD $0x540001c3
WORD $0x6f00e400
- WORD $0x927ded09
- WORD $0x9100400a
- WORD $0x927ded0b
- WORD $0xf100216b
- WORD $0xad3f8140
- WORD $0x9100814a
+ WORD $0x9240090a
+ WORD $0x8b09080c
+ WORD $0xcb0a016b
+ WORD $0x8b0a012d
+ WORD $0x8b0b0129
+ WORD $0x9100418b
+ WORD $0xcb0801ac
+ WORD $0xb100218c
+ WORD $0xad3f8160
+ WORD $0x9100816b
WORD $0x54ffffa1
- WORD $0xeb09011f
- WORD $0x540000c0
+ WORD $0xb40000ca
WORD $0x8b09080a
WORD $0xcb090108
WORD $0xf1000508
WORD $0xb800455f
WORD $0x54ffffc1
+ WORD $0x6d4f23e9
+ WORD $0xf94083fd
+ WORD $0x6d4e2beb
+ WORD $0x6d4d33ed
+ WORD $0x6d4c3bef
+ WORD $0x910443ff
WORD $0xd65f03c0
RET
@@ -407,38 +769,345 @@ TEXT ·correlateFloat32NEON(SB), NOSPLIT|NOFRAME, $0-72
MOVD dst_len+8(FP), R3
MOVD sig_len+32(FP), R4
MOVD ker_len+56(FP), R5
- WORD $0xb4000a45
+ WORD $0xb4003185
WORD $0xeb050088
- WORD $0x54000a0b
+ WORD $0x5400314b
+ WORD $0xd10503ff
WORD $0xeb08007f
+ WORD $0x6d0f3bef
WORD $0x9a88d468
- WORD $0xf100051f
- WORD $0x5400098b
+ WORD $0x6d1033ed
+ WORD $0xf100411f
+ WORD $0x6d112beb
+ WORD $0x6d1223e9
+ WORD $0xf9009bfd
+ WORD $0x5400006a
+ WORD $0xaa1f03e9
+ WORD $0x1400012a
+ WORD $0xf10000bf
+ WORD $0x540023ed
+ WORD $0xaa1f03ed
+ WORD $0x927ef0aa
+ WORD $0x9100802b
+ WORD $0x5280020e
+ WORD $0xaa0103ec
+ WORD $0x1400000e
+ WORD $0x8b0d080f
+ WORD $0x9100412e
+ WORD $0x9101016b
+ WORD $0xeb0801df
+ WORD $0x9101018c
+ WORD $0xaa0903ed
+ WORD $0xbd0001e4
+ WORD $0x3c8041e3
+ WORD $0x3c8141e5
+ WORD $0x3c8241e6
+ WORD $0x2d06f5f5
+ WORD $0xbd003df4
+ WORD $0x540022ac
+ WORD $0xf10010bf
+ WORD $0xaa0e03e9
+ WORD $0x54000142
+ WORD $0x6f00e403
+ WORD $0x2f00e414
+ WORD $0xaa1f03ef
+ WORD $0x2f00e41d
+ WORD $0x2f00e415
+ WORD $0x2f00e404
+ WORD $0x6f00e405
+ WORD $0x6f00e406
+ WORD $0x140000e1
+ WORD $0x2f00e400
+ WORD $0x2f00e414
+ WORD $0xaa0b03f1
+ WORD $0x2f00e41d
+ WORD $0x2f00e415
+ WORD $0xaa0203ee
+ WORD $0x2f00e407
+ WORD $0x2f00e416
+ WORD $0x927ef0af
+ WORD $0x2f00e412
+ WORD $0x2f00e406
+ WORD $0xaa0b03f0
+ WORD $0x3d8037e0
+ WORD $0x2f00e418
+ WORD $0x2f00e413
+ WORD $0x2f00e405
+ WORD $0x2f00e410
+ WORD $0x2f00e411
+ WORD $0x2f00e400
+ WORD $0x2f00e403
+ WORD $0x2f00e404
+ WORD $0x3d8033f8
+ WORD $0x3cc105d8
+ WORD $0x3cc1c239
+ WORD $0x3cc1823a
+ WORD $0xf10011ef
+ WORD $0x3d802be0
+ WORD $0x3cc0c229
+ WORD $0x6e39df0b
+ WORD $0x3cc14239
+ WORD $0x6e3adf0c
+ WORD $0x3cc10e1a
+ WORD $0x3d8023f3
+ WORD $0x6e39df0d
+ WORD $0x3cde0239
+ WORD $0x3d800bf0
+ WORD $0x6e3adf0e
+ WORD $0x3cde423a
+ WORD $0x3cc04230
+ WORD $0x5e0c057b
+ WORD $0x1e2b2a80
+ WORD $0x6e39df01
+ WORD $0x5e0c059c
+ WORD $0x1e2c2ba2
+ WORD $0x5e140568
+ WORD $0x5e0c05bd
+ WORD $0x5e14059f
+ WORD $0x5e1c056b
+ WORD $0x5e0c05de
+ WORD $0x1e2e28e7
+ WORD $0x5e1405aa
+ WORD $0x3d802fe1
+ WORD $0x6e3adf01
+ WORD $0x1e3b2800
+ WORD $0x1e3c2842
+ WORD $0x5e1c058c
+ WORD $0x3cde823b
+ WORD $0x5e1405cf
+ WORD $0x6e30df10
+ WORD $0x4eb11e3a
+ WORD $0x6e3bdf19
+ WORD $0x1e3e28e7
+ WORD $0x3cdec23c
+ WORD $0x3d8027e1
+ WORD $0x1e2d2aa1
+ WORD $0x1e282800
+ WORD $0x1e3f2842
+ WORD $0x5e1c05ad
+ WORD $0x3cc08228
+ WORD $0x5e1c05d5
+ WORD $0x3cdf023e
+ WORD $0x3cdf823f
+ WORD $0x6e29df13
+ WORD $0x6e28df14
+ WORD $0x1e3d2821
+ WORD $0x1e2b2800
+ WORD $0x1e2f28e7
+ WORD $0x3cdf422e
+ WORD $0x6e3cdf1b
+ WORD $0x6e3edf1c
+ WORD $0x6e3fdf1e
+ WORD $0x3d8003f0
+ WORD $0x3d803bf4
+ WORD $0x1e332acf
+ WORD $0x1e342a51
+ WORD $0x1e2a2821
+ WORD $0xbd013fe0
+ WORD $0x1e2c2840
+ WORD $0x5e0c0677
+ WORD $0x5e0c0694
+ WORD $0xad0267f3
+ WORD $0x1e3028d2
+ WORD $0x5e0c0613
+ WORD $0x3cdfc22a
+ WORD $0x3dc00236
+ WORD $0x6e2edf19
+ WORD $0x5e0c077f
+ WORD $0xbd013be0
+ WORD $0x1e2d2820
+ WORD $0x6e2adf1d
+ WORD $0x1e3729e1
+ WORD $0x6e36df16
+ WORD $0x4ebb1f6a
+ WORD $0x3dc017e2
+ WORD $0x5e0c07c9
+ WORD $0x5e0c078f
+ WORD $0x1e3928a6
+ WORD $0xaa1003f1
+ WORD $0xbd007fe0
+ WORD $0x1e3528e0
+ WORD $0x4ebd1fb8
+ WORD $0x5e0c06d7
+ WORD $0x5e0c0727
+ WORD $0x1e2a2b4e
+ WORD $0x4ebe1fda
+ WORD $0x3d801be0
+ WORD $0x1e342a20
+ WORD $0x5e0c07b1
+ WORD $0x1e2728c8
+ WORD $0x5e1407c7
+ WORD $0x1e3f29df
+ WORD $0x5e14078e
+ WORD $0x5e1c075a
+ WORD $0x2d0707e0
+ WORD $0x1e332a40
+ WORD $0xbd001fe0
+ WORD $0xad4607e0
+ WORD $0x1e382814
+ WORD $0xad444fe0
+ WORD $0x1e36283b
+ WORD $0xad454be1
+ WORD $0x1e3e2815
+ WORD $0x3dc00be0
+ WORD $0x1e332863
+ WORD $0x1e222825
+ WORD $0x5e0c0441
+ WORD $0x1e312a91
+ WORD $0x1e3c280d
+ WORD $0x5e0c0660
+ WORD $0x1e372b6b
+ WORD $0x4eaa1d5e
+ WORD $0x1e322884
+ WORD $0x5e0c064c
+ WORD $0x1e292aa9
+ WORD $0x5e140615
+ WORD $0x5e140646
+ WORD $0x1e2128b7
+ WORD $0x3dc03be1
+ WORD $0xbd00c3f1
+ WORD $0x1e202874
+ WORD $0x3dc013e0
+ WORD $0x1e2f29bd
+ WORD $0x5e140430
+ WORD $0x5e14054d
+ WORD $0xbd403fea
+ WORD $0x5e140403
+ WORD $0x1e2c2891
+ WORD $0x5e140704
+ WORD $0x5e1406cc
+ WORD $0x5e14072f
+ WORD $0x5e140665
+ WORD $0x5e140441
+ WORD $0x4ebc1f9b
+ WORD $0xbd401ffc
+ WORD $0x1e2e2bae
+ WORD $0x5e1c071d
+ WORD $0x1e232943
+ WORD $0xbd403bea
+ WORD $0x1e352b95
+ WORD $0x1e2c296b
+ WORD $0x1e2f290c
+ WORD $0x1e2d2bef
+ WORD $0x1e302950
+ WORD $0x5e1c06df
+ WORD $0x1e212aed
+ WORD $0x1e252a81
+ WORD $0x1e262a37
+ WORD $0x3dc003e5
+ WORD $0x5e1c0411
+ WORD $0x5e1c07dc
+ WORD $0x5e1c077e
+ WORD $0x5e1c04a6
+ WORD $0x5e1c0648
+ WORD $0x5e1c045b
+ WORD $0xbd00d3f0
+ WORD $0xbd40c3f0
+ WORD $0xbd413ff4
+ WORD $0xbd40d3e0
+ WORD $0x1e242a0a
+ WORD $0x1e272924
+ WORD $0x3dc03be7
+ WORD $0x5e1c0730
+ WORD $0x5e1c0669
+ WORD $0x1e312876
+ WORD $0x5e1c04e7
+ WORD $0x1e262aa6
+ WORD $0x1e3c29f1
+ WORD $0xbd407ff5
+ WORD $0x1e3d2958
+ WORD $0x1e3a2893
+ WORD $0x1e282ae4
+ WORD $0x1e302985
+ WORD $0x1e3e29d0
+ WORD $0x1e292823
+ WORD $0x1e272812
+ WORD $0x1e3f2960
+ WORD $0x3dc01be7
+ WORD $0xbd413bfd
+ WORD $0x3d8037e0
+ WORD $0x1e3b29a0
+ WORD $0x54ffe861
+ WORD $0x6e0c0403
+ WORD $0x6e0c0665
+ WORD $0x3dc037e0
+ WORD $0x6e0c0646
+ WORD $0xeb0a00bf
+ WORD $0x927ef0af
+ WORD $0x6e140623
+ WORD $0x6e140705
+ WORD $0x6e1406c6
+ WORD $0x6e1c0603
+ WORD $0x6e1c0405
+ WORD $0x6e1c04e6
+ WORD $0x54ffe100
+ WORD $0xd37ef5ee
+ WORD $0xcb0f00af
+ WORD $0xaa0c03f0
+ WORD $0xaa0203f1
+ WORD $0x8b0e0203
+ WORD $0xbc6e6a20
+ WORD $0xf10005ef
+ WORD $0x2d46c871
+ WORD $0xbd400061
+ WORD $0x3cc04062
+ WORD $0x3cc14067
+ WORD $0x91001231
+ WORD $0x3cc24070
+ WORD $0xbd403c73
+ WORD $0x1e210801
+ WORD $0x4f809042
+ WORD $0x4f8090e7
+ WORD $0x1e310811
+ WORD $0x4f809210
+ WORD $0x1e320812
+ WORD $0x1e330800
+ WORD $0x91001210
+ WORD $0x1e212884
+ WORD $0x4e22d463
+ WORD $0x4e27d4a5
+ WORD $0x1e312ab5
+ WORD $0x4e30d4c6
+ WORD $0x1e322bbd
+ WORD $0x1e202a94
+ WORD $0x54fffce1
+ WORD $0x17fffee9
+ WORD $0x6f00e400
+ WORD $0xaa1f03e9
+ WORD $0x9100800a
+ WORD $0x9100812b
+ WORD $0x91004129
+ WORD $0xad3f0140
+ WORD $0xeb08017f
+ WORD $0xac820140
+ WORD $0x54ffff6d
+ WORD $0xeb09010b
+ WORD $0x540009ad
WORD $0xf10004bf
WORD $0x540006eb
- WORD $0xaa1f03e9
- WORD $0x927decaa
- WORD $0x9100402b
+ WORD $0x8b09082a
+ WORD $0x927decab
+ WORD $0x9100414c
WORD $0x14000007
WORD $0xbc297800
WORD $0x91000529
- WORD $0x9100116b
+ WORD $0x9100118c
WORD $0xeb08013f
- WORD $0x91001021
- WORD $0x54000800
+ WORD $0x9100114a
+ WORD $0x54000820
WORD $0x2f00e400
WORD $0xf10020bf
WORD $0x54000062
- WORD $0xaa1f03ec
+ WORD $0xaa1f03ed
WORD $0x1400001d
- WORD $0xaa0b03ec
- WORD $0x9100404d
- WORD $0x927decae
- WORD $0xad7f91a1
- WORD $0xf10021ce
- WORD $0xad7f8d82
+ WORD $0xaa0c03ed
+ WORD $0x9100404e
+ WORD $0x927decaf
+ WORD $0xad7f91c1
+ WORD $0xf10021ef
+ WORD $0xad7f8da2
+ WORD $0x910081ce
WORD $0x910081ad
- WORD $0x9100818c
WORD $0x6e21dc41
WORD $0x5e0c0422
WORD $0x1e212800
@@ -456,39 +1125,46 @@ TEXT ·correlateFloat32NEON(SB), NOSPLIT|NOFRAME, $0-72
WORD $0x1e232800
WORD $0x1e212800
WORD $0x54fffd61
- WORD $0xeb0a00bf
- WORD $0x927decac
+ WORD $0xeb0b00bf
+ WORD $0x927decad
WORD $0x54fffb40
- WORD $0xd37ef58e
- WORD $0xcb0c00ac
- WORD $0x8b0e004d
- WORD $0x8b0e002e
- WORD $0xbc4045c1
- WORD $0xf100058c
- WORD $0xbc4045a2
+ WORD $0xd37ef5af
+ WORD $0xcb0d00ad
+ WORD $0x8b0f004e
+ WORD $0x8b0f014f
+ WORD $0xbc4045e1
+ WORD $0xf10005ad
+ WORD $0xbc4045c2
WORD $0x1e220821
WORD $0x1e212800
WORD $0x54ffff61
WORD $0x17ffffcf
- WORD $0xf100211f
- WORD $0x54000062
- WORD $0xaa1f03e9
- WORD $0x1400000b
+ WORD $0xf100217f
+ WORD $0x540001c3
WORD $0x6f00e400
- WORD $0x927ded09
- WORD $0x9100400a
- WORD $0x927ded0b
- WORD $0xf100216b
- WORD $0xad3f8140
- WORD $0x9100814a
+ WORD $0x9240090a
+ WORD $0x8b09080c
+ WORD $0xcb0a016b
+ WORD $0x8b0a012d
+ WORD $0x8b0b0129
+ WORD $0x9100418b
+ WORD $0xcb0801ac
+ WORD $0xb100218c
+ WORD $0xad3f8160
+ WORD $0x9100816b
WORD $0x54ffffa1
- WORD $0xeb09011f
- WORD $0x540000c0
+ WORD $0xb40000ca
WORD $0x8b09080a
WORD $0xcb090108
WORD $0xf1000508
WORD $0xb800455f
WORD $0x54ffffc1
+ WORD $0x6d5223e9
+ WORD $0xf9409bfd
+ WORD $0x6d512beb
+ WORD $0x6d5033ed
+ WORD $0x6d4f3bef
+ WORD $0x910503ff
WORD $0xd65f03c0
RET
@@ -733,20 +1409,97 @@ TEXT ·polyEvalFloat64NEON(SB), NOSPLIT|NOFRAME, $0-72
MOVD coeffs_len+56(FP), R5
WORD $0xeb04007f
WORD $0x9a84b068
+ WORD $0xb40000c5
+ WORD $0x8b050c4a
+ WORD $0xf100411f
+ WORD $0x5400012a
+ WORD $0xaa1f03e9
+ WORD $0x1400004b
WORD $0xf100051f
- WORD $0xb4000305
- WORD $0x540007eb
- WORD $0x8b050c4b
- WORD $0xf10008bf
- WORD $0xfc5f8160
- WORD $0x5400030b
+ WORD $0x5400106b
+ WORD $0xf100111f
+ WORD $0x54000e42
+ WORD $0xaa1f03e9
+ WORD $0x1400007a
+ WORD $0xfc5f8140
+ WORD $0xf10004ab
+ WORD $0x5400066d
+ WORD $0x4e080400
+ WORD $0xaa1f03ec
+ WORD $0x5280020d
+ WORD $0x8b0c0c2e
+ WORD $0x4ea01c18
+ WORD $0x4ea01c17
+ WORD $0x4ea01c16
+ WORD $0x4ea01c15
+ WORD $0xaa0d03e9
+ WORD $0xad4009c1
+ WORD $0x4ea01c14
+ WORD $0xad4111c3
+ WORD $0x4ea01c13
+ WORD $0xad4219c5
+ WORD $0x4ea01c12
+ WORD $0xad4341c7
+ WORD $0x4ea01c11
+ WORD $0xd100414d
+ WORD $0xaa0b03ee
+ WORD $0xfc5f85b9
+ WORD $0x6e61df18
+ WORD $0x6e62def7
+ WORD $0x6e63ded6
+ WORD $0x6e64deb5
+ WORD $0x6e65de94
+ WORD $0x6e66de73
+ WORD $0xd10005ce
+ WORD $0x6e67de52
+ WORD $0x6e70de31
+ WORD $0xf10001df
+ WORD $0x4e080739
+ WORD $0x4e78d738
+ WORD $0x4e77d737
+ WORD $0x4e76d736
+ WORD $0x4e75d735
+ WORD $0x4e74d734
+ WORD $0x4e73d733
+ WORD $0x4e72d732
+ WORD $0x4e71d731
+ WORD $0x54fffd8c
+ WORD $0x8b0c0c0e
+ WORD $0x9100412d
+ WORD $0xaa0903ec
+ WORD $0xeb0801bf
+ WORD $0xad005dd8
+ WORD $0xad0155d6
+ WORD $0xad024dd4
+ WORD $0xad0345d2
+ WORD $0x54fffa6d
+ WORD $0x14000010
WORD $0xaa1f03e9
- WORD $0xd10004aa
- WORD $0xd100416b
+ WORD $0x9101000b
+ WORD $0x9100812c
+ WORD $0x6d3c0160
+ WORD $0x91004129
+ WORD $0x6d3d0160
+ WORD $0xeb08019f
+ WORD $0x6d3e0160
+ WORD $0x6d3f0160
+ WORD $0x6d000160
+ WORD $0x6d010160
+ WORD $0x6d020160
+ WORD $0x6d030160
+ WORD $0x9102016b
+ WORD $0x54fffe8d
+ WORD $0xeb09010b
+ WORD $0x5400072d
+ WORD $0xfc5f8140
+ WORD $0xf10008bf
+ WORD $0x5400024b
+ WORD $0xd10004ab
+ WORD $0xd100414a
WORD $0x1e604002
WORD $0xfc697821
- WORD $0xaa0b03ec
- WORD $0xaa0a03ed
+ WORD $0xaa0a03ec
+ WORD $0xaa0b03ed
WORD $0x1e620822
WORD $0xfc5f8583
WORD $0xd10005ad
@@ -757,38 +1510,34 @@ TEXT ·polyEvalFloat64NEON(SB), NOSPLIT|NOFRAME, $0-72
WORD $0x91000529
WORD $0xeb08013f
WORD $0x54fffe61
- WORD $0x14000029
- WORD $0x5400050b
- WORD $0xf100111f
- WORD $0x540000e2
- WORD $0xaa1f03e9
- WORD $0x1400000f
- WORD $0xf100111f
- WORD $0x54000262
- WORD $0xaa1f03e9
- WORD $0x1400001b
- WORD $0x6f00e400
- WORD $0x927ef109
- WORD $0x9100400a
- WORD $0x927ef10b
- WORD $0xf100116b
- WORD $0xad3f8140
- WORD $0x9100814a
+ WORD $0x14000025
+ WORD $0xf100117f
+ WORD $0x540001c3
+ WORD $0x9240050a
+ WORD $0x4e080401
+ WORD $0x8b090c0c
+ WORD $0xcb0a016b
+ WORD $0x8b0a012d
+ WORD $0x8b0b0129
+ WORD $0x9100418b
+ WORD $0xcb0801ac
+ WORD $0xb100118c
+ WORD $0xad3f8561
+ WORD $0x9100816b
WORD $0x54ffffa1
- WORD $0xeb09011f
- WORD $0x540002c0
+ WORD $0xb40002ca
WORD $0x8b090c0a
WORD $0xcb090108
WORD $0xf1000508
- WORD $0xf800855f
+ WORD $0xfc008540
WORD $0x54ffffc1
WORD $0x14000010
- WORD $0x4e080401
+ WORD $0x6f00e400
WORD $0x927ef109
WORD $0x9100400a
WORD $0x927ef10b
WORD $0xf100116b
- WORD $0xad3f8541
+ WORD $0xad3f8140
WORD $0x9100814a
WORD $0x54ffffa1
WORD $0xeb09011f
@@ -796,7 +1545,7 @@ TEXT ·polyEvalFloat64NEON(SB), NOSPLIT|NOFRAME, $0-72
WORD $0x8b090c0a
WORD $0xcb090108
WORD $0xf1000508
- WORD $0xfc008540
+ WORD $0xf800855f
WORD $0x54ffffc1
WORD $0xd65f03c0
RET
@@ -809,41 +1558,252 @@ TEXT ·convolveFloat64NEON(SB), NOSPLIT|NOFRAME, $0-72
MOVD dst_len+8(FP), R3
MOVD sig_len+32(FP), R4
MOVD ker_len+56(FP), R5
- WORD $0xb40009a5
+ WORD $0xb40024c5
WORD $0xeb050088
- WORD $0x5400096b
+ WORD $0x5400248b
+ WORD $0xd10143ff
WORD $0xeb08007f
+ WORD $0x6d013bef
WORD $0x9a88d468
- WORD $0xf100051f
- WORD $0x540008eb
- WORD $0xf10004bf
- WORD $0x5400064b
- WORD $0x8b050c4b
+ WORD $0x6d0233ed
+ WORD $0xf100411f
+ WORD $0x6d032beb
+ WORD $0x6d0423e9
+ WORD $0x5400006a
WORD $0xaa1f03e9
- WORD $0x927ef0aa
- WORD $0x9100402c
+ WORD $0x140000cb
+ WORD $0xf10000bf
+ WORD $0x5400176d
+ WORD $0x8b050c49
+ WORD $0xaa1f03ef
+ WORD $0x927ff4aa
+ WORD $0x9101002b
WORD $0xd100204d
- WORD $0xd100416b
+ WORD $0x9101e02e
+ WORD $0xd100412c
+ WORD $0x52800210
+ WORD $0x14000010
+ WORD $0x8b0f0c0f
+ WORD $0x91004130
+ WORD $0x9102016b
+ WORD $0xeb08021f
+ WORD $0x910201ce
+ WORD $0x6d0051f7
+ WORD $0x6d0159f2
+ WORD $0x6d0245f5
+ WORD $0x6d0341f3
+ WORD $0x6d041de6
+ WORD $0x6d0539e5
+ WORD $0x6d0609e3
+ WORD $0x6d0701e1
+ WORD $0xaa0903ef
+ WORD $0x5400162c
+ WORD $0x2f00e400
+ WORD $0x2f00e401
+ WORD $0xf10004bf
+ WORD $0x2f00e402
+ WORD $0x2f00e403
+ WORD $0xaa1003e9
+ WORD $0x2f00e40e
+ WORD $0x2f00e405
+ WORD $0x2f00e407
+ WORD $0x2f00e406
+ WORD $0x2f00e410
+ WORD $0x2f00e413
+ WORD $0x2f00e411
+ WORD $0x2f00e415
+ WORD $0x2f00e416
+ WORD $0x2f00e412
+ WORD $0x2f00e414
+ WORD $0x2f00e417
+ WORD $0x54000061
+ WORD $0xaa1f03f1
+ WORD $0x1400005d
+ WORD $0xaa0b03e4
+ WORD $0xaa0c03f0
+ WORD $0x927ff4b1
+ WORD $0xaa0b03e3
+ WORD $0x3cdf0618
+ WORD $0x3cdc0099
+ WORD $0xad41749a
+ WORD $0xf1000a31
+ WORD $0x6e184318
+ WORD $0x3cc2809c
+ WORD $0x3cdd8089
+ WORD $0x3cde808b
+ WORD $0x3cdf008c
+ WORD $0xfd0007ee
+ WORD $0x3cc18084
+ WORD $0x6e79df1b
+ WORD $0x3cc38099
+ WORD $0x6e7adf1e
+ WORD $0x6e7cdf1f
+ WORD $0x6e7ddf1d
+ WORD $0x6e69df09
+ WORD $0x6e79df08
+ WORD $0xad7ea89c
+ WORD $0x3cdc8099
+ WORD $0x6e64df04
+ WORD $0x1e7e2863
+ WORD $0x1e7b2af7
+ WORD $0x6e79df1a
+ WORD $0x6e7cdf19
+ WORD $0x6e6adf0a
+ WORD $0x6e6bdf1c
+ WORD $0x6e6cdf0b
+ WORD $0x5e1807cc
+ WORD $0x5e1807ed
+ WORD $0x5e1807ae
+ WORD $0x5e18050f
+ WORD $0x1e682800
+ WORD $0x1e7d2821
+ WORD $0x1e7f2842
+ WORD $0x3cdf809d
+ WORD $0x3dc0009e
+ WORD $0x3cc0809f
+ WORD $0x3cc10c68
+ WORD $0x1e692ad6
+ WORD $0x6e7ddf1d
+ WORD $0x6e7edf1e
+ WORD $0x6e7fdf1f
+ WORD $0x6e68df08
+ WORD $0x5e180778
+ WORD $0x5e18053b
+ WORD $0x1e6a2ab5
+ WORD $0x5e180549
+ WORD $0x1e6b2a73
+ WORD $0x5e18056a
+ WORD $0xfd4007eb
+ WORD $0x1e7c2a31
+ WORD $0x1e7f28e7
+ WORD $0x5e1807ff
+ WORD $0x1e7d2a10
+ WORD $0x1e6828a5
+ WORD $0x5e180508
+ WORD $0x1e64296b
+ WORD $0x5e180484
+ WORD $0x5e1807bd
+ WORD $0x1e7e28c6
+ WORD $0x5e1807de
+ WORD $0x5e18079c
+ WORD $0x1e7a2a94
+ WORD $0x5e18075a
+ WORD $0x1e792a52
+ WORD $0x5e180739
+ WORD $0x1e6f2800
+ WORD $0x1e6e2821
+ WORD $0x1e6d2842
+ WORD $0x1e6c2863
+ WORD $0x1e64296e
+ WORD $0x1e6828a5
+ WORD $0x1e7f28e7
+ WORD $0x1e7e28c6
+ WORD $0x1e7d2a10
+ WORD $0x1e6a2a73
+ WORD $0x1e7c2a31
+ WORD $0xaa0303e4
+ WORD $0x1e692ab5
+ WORD $0x1e7b2ad6
+ WORD $0x1e792a52
+ WORD $0x1e7a2a94
+ WORD $0x1e782af7
+ WORD $0x54fff581
+ WORD $0xeb0a00bf
+ WORD $0x927ff4b1
+ WORD $0x54fff020
+ WORD $0xd37df230
+ WORD $0xcb1100b1
+ WORD $0xaa0e03e3
+ WORD $0x8b100064
+ WORD $0xfc7179b8
+ WORD $0xf1000631
+ WORD $0x6d78e484
+ WORD $0x91002063
+ WORD $0x6d79ec9a
+ WORD $0x6d7ca488
+ WORD $0x1e790b19
+ WORD $0x1e640b04
+ WORD $0x1e7a0b1a
+ WORD $0x1e7b0b1b
+ WORD $0x6d7af49c
+ WORD $0x1e680b08
+ WORD $0x6d7bfc9e
+ WORD $0x1e792a94
+ WORD $0x6d7de48a
+ WORD $0x1e642af7
+ WORD $0x1e690b04
+ WORD $0x1e7c0b1c
+ WORD $0x6d7fb48c
+ WORD $0x1e7d0b1d
+ WORD $0x1e6a0b09
+ WORD $0x6d7eac8a
+ WORD $0x1e7e0b1e
+ WORD $0x1e7f0b1f
+ WORD $0x1e7a2a52
+ WORD $0x1e790b19
+ WORD $0x1e7b2ad6
+ WORD $0x1e6c0b1b
+ WORD $0x1e6a0b1a
+ WORD $0x1e6b0b0a
+ WORD $0x1e7c2ab5
+ WORD $0x1e6d0b18
+ WORD $0x1e7d2a31
+ WORD $0x1e6828c6
+ WORD $0x1e7e2a73
+ WORD $0x1e7f2a10
+ WORD $0x1e6428e7
+ WORD $0x1e6928a5
+ WORD $0x1e7929ce
+ WORD $0x1e7b2821
+ WORD $0x1e7a2863
+ WORD $0x1e6a2842
+ WORD $0x1e782800
+ WORD $0x54fffa81
+ WORD $0x17ffff50
+ WORD $0x6f00e400
+ WORD $0xaa1f03e9
+ WORD $0x9101200a
+ WORD $0x9100812b
+ WORD $0xf81b815f
+ WORD $0x91004129
+ WORD $0xad3e0140
+ WORD $0xeb08017f
+ WORD $0xad3f0140
+ WORD $0xad000140
+ WORD $0x3d800940
+ WORD $0xf900195f
+ WORD $0x9102014a
+ WORD $0x54fffecd
+ WORD $0xeb09010b
+ WORD $0x5400090d
+ WORD $0xf10004bf
+ WORD $0x5400064b
+ WORD $0x8b090c2a
+ WORD $0x8b050c4d
+ WORD $0x927ef0ab
+ WORD $0xd100204e
+ WORD $0x9100414c
+ WORD $0xd10041ad
WORD $0x14000007
WORD $0xfc297800
WORD $0x91000529
WORD $0x9100218c
WORD $0xeb08013f
- WORD $0x91002021
- WORD $0x54000700
+ WORD $0x9100214a
+ WORD $0x54000720
WORD $0x2f00e400
WORD $0xf10010bf
WORD $0x54000062
- WORD $0xaa1f03ef
+ WORD $0xaa1f03f0
WORD $0x14000017
- WORD $0xaa0c03ee
- WORD $0xaa0b03ef
- WORD $0x927ef0b0
+ WORD $0xaa0d03ef
+ WORD $0xaa0c03f0
+ WORD $0x927ef0b1
WORD $0xad7f85e2
- WORD $0xd1001210
- WORD $0xad7f91c3
+ WORD $0xd1001231
+ WORD $0xad7f9203
+ WORD $0x91008210
WORD $0xd10081ef
- WORD $0x910081ce
WORD $0x6e014021
WORD $0x6e024042
WORD $0x6e61dc61
@@ -854,38 +1814,44 @@ TEXT ·convolveFloat64NEON(SB), NOSPLIT|NOFRAME, $0-72
WORD $0x5e180422
WORD $0x1e612800
WORD $0x1e622800
- WORD $0xb5fffe30
- WORD $0xeb0a00bf
- WORD $0x927ef0af
+ WORD $0xb5fffe31
+ WORD $0xeb0b00bf
+ WORD $0x927ef0b0
WORD $0x54fffc00
- WORD $0x8b0f0c2e
- WORD $0xcb0f00af
- WORD $0xfc4085c1
- WORD $0xfc6f79a2
- WORD $0xf10005ef
+ WORD $0x8b100d4f
+ WORD $0xcb1000b0
+ WORD $0xfc4085e1
+ WORD $0xfc7079c2
+ WORD $0xf1000610
WORD $0x1e620821
WORD $0x1e612800
WORD $0x54ffff61
WORD $0x17ffffd7
- WORD $0xf100111f
- WORD $0x54000062
- WORD $0xaa1f03e9
- WORD $0x1400000b
+ WORD $0xf100117f
+ WORD $0x540001c3
WORD $0x6f00e400
- WORD $0x927ef109
- WORD $0x9100400a
- WORD $0x927ef10b
- WORD $0xf100116b
- WORD $0xad3f8140
- WORD $0x9100814a
+ WORD $0x9240050a
+ WORD $0x8b090c0c
+ WORD $0xcb0a016b
+ WORD $0x8b0a012d
+ WORD $0x8b0b0129
+ WORD $0x9100418b
+ WORD $0xcb0801ac
+ WORD $0xb100118c
+ WORD $0xad3f8160
+ WORD $0x9100816b
WORD $0x54ffffa1
- WORD $0xeb09011f
- WORD $0x540000c0
+ WORD $0xb40000ca
WORD $0x8b090c0a
WORD $0xcb090108
WORD $0xf1000508
WORD $0xf800855f
WORD $0x54ffffc1
+ WORD $0x6d4423e9
+ WORD $0x6d432beb
+ WORD $0x6d4233ed
+ WORD $0x6d413bef
+ WORD $0x910143ff
WORD $0xd65f03c0
RET
@@ -897,38 +1863,247 @@ TEXT ·correlateFloat64NEON(SB), NOSPLIT|NOFRAME, $0-72
MOVD dst_len+8(FP), R3
MOVD sig_len+32(FP), R4
MOVD ker_len+56(FP), R5
- WORD $0xb4000945
+ WORD $0xb4002425
WORD $0xeb050088
- WORD $0x5400090b
+ WORD $0x540023eb
+ WORD $0xd10143ff
WORD $0xeb08007f
+ WORD $0x6d013bef
WORD $0x9a88d468
- WORD $0xf100051f
- WORD $0x5400088b
+ WORD $0x6d0233ed
+ WORD $0xf100411f
+ WORD $0x6d032beb
+ WORD $0x6d0423e9
+ WORD $0x5400006a
+ WORD $0xaa1f03e9
+ WORD $0x140000c9
+ WORD $0xf10000bf
+ WORD $0x5400172d
+ WORD $0xaa1f03ed
+ WORD $0x927ff4aa
+ WORD $0x9101002b
+ WORD $0x9101e02c
+ WORD $0x5280020e
+ WORD $0x14000010
+ WORD $0x8b0d0c0d
+ WORD $0x9100412e
+ WORD $0x9102016b
+ WORD $0xeb0801df
+ WORD $0x9102018c
+ WORD $0x6d0051b7
+ WORD $0x6d0159b2
+ WORD $0x6d0245b5
+ WORD $0x6d0341b3
+ WORD $0x6d041da6
+ WORD $0x6d0539a5
+ WORD $0x6d0609a3
+ WORD $0x6d0701a1
+ WORD $0xaa0903ed
+ WORD $0x5400164c
+ WORD $0x2f00e400
+ WORD $0x2f00e401
WORD $0xf10004bf
- WORD $0x540005eb
+ WORD $0x2f00e402
+ WORD $0x2f00e403
+ WORD $0xaa0e03e9
+ WORD $0x2f00e40e
+ WORD $0x2f00e405
+ WORD $0x2f00e407
+ WORD $0x2f00e406
+ WORD $0x2f00e410
+ WORD $0x2f00e413
+ WORD $0x2f00e411
+ WORD $0x2f00e415
+ WORD $0x2f00e416
+ WORD $0x2f00e412
+ WORD $0x2f00e414
+ WORD $0x2f00e417
+ WORD $0x54000061
+ WORD $0xaa1f03ef
+ WORD $0x1400005c
+ WORD $0xaa0b03f1
+ WORD $0xaa0203ee
+ WORD $0x927ff4af
+ WORD $0xaa0b03f0
+ WORD $0x3cc105d8
+ WORD $0x3cdc0239
+ WORD $0xad41763a
+ WORD $0xf10009ef
+ WORD $0x3cc2823c
+ WORD $0x6e79df1b
+ WORD $0x3cc38239
+ WORD $0x3cdd8229
+ WORD $0x3cde822b
+ WORD $0x3cdf022c
+ WORD $0x6e7adf1e
+ WORD $0x6e7cdf1f
+ WORD $0x6e7ddf1d
+ WORD $0x6e79df08
+ WORD $0xad7eaa3c
+ WORD $0x3cdc8239
+ WORD $0xfd0007ee
+ WORD $0x6e69df09
+ WORD $0x3cc18224
+ WORD $0x1e7b2af7
+ WORD $0x6e79df1a
+ WORD $0x6e7cdf19
+ WORD $0x6e6adf0a
+ WORD $0x6e6bdf1c
+ WORD $0x6e6cdf0b
+ WORD $0x5e1807cc
+ WORD $0x5e1807ed
+ WORD $0x5e1807ae
+ WORD $0x5e18050f
+ WORD $0x1e682800
+ WORD $0x1e7d2821
+ WORD $0x1e7f2842
+ WORD $0x1e7e2863
+ WORD $0x3cdf823d
+ WORD $0x3dc0023e
+ WORD $0x3cc0823f
+ WORD $0x3cc10e08
+ WORD $0x6e7ddf1d
+ WORD $0x6e7edf1e
+ WORD $0x6e64df04
+ WORD $0x6e7fdf1f
+ WORD $0x6e68df08
+ WORD $0x5e180778
+ WORD $0x1e692ad6
+ WORD $0x5e18053b
+ WORD $0x1e6a2ab5
+ WORD $0x5e180549
+ WORD $0x1e6b2a73
+ WORD $0x5e18056a
+ WORD $0xfd4007eb
+ WORD $0x1e7d2a10
+ WORD $0x5e1807bd
+ WORD $0x1e7f28e7
+ WORD $0x5e1807ff
+ WORD $0x1e6828a5
+ WORD $0x5e180508
+ WORD $0x1e64296b
+ WORD $0x5e180484
+ WORD $0x1e7e28c6
+ WORD $0x5e1807de
+ WORD $0x1e7c2a31
+ WORD $0x5e18079c
+ WORD $0x1e7a2a94
+ WORD $0x5e18075a
+ WORD $0x1e792a52
+ WORD $0x5e180739
+ WORD $0xaa1003f1
+ WORD $0x1e6f2800
+ WORD $0x1e6e2821
+ WORD $0x1e6d2842
+ WORD $0x1e6c2863
+ WORD $0x1e64296e
+ WORD $0x1e6828a5
+ WORD $0x1e7f28e7
+ WORD $0x1e7e28c6
+ WORD $0x1e7d2a10
+ WORD $0x1e6a2a73
+ WORD $0x1e7c2a31
+ WORD $0x1e692ab5
+ WORD $0x1e7b2ad6
+ WORD $0x1e792a52
+ WORD $0x1e7a2a94
+ WORD $0x1e782af7
+ WORD $0x54fff5a1
+ WORD $0xeb0a00bf
+ WORD $0x927ff4af
+ WORD $0x54fff040
+ WORD $0xd37df1ee
+ WORD $0xcb0f00af
+ WORD $0xaa0c03f0
+ WORD $0xaa0203f1
+ WORD $0x8b0e0203
+ WORD $0xfc6e6a38
+ WORD $0xf10005ef
+ WORD $0x6d78e464
+ WORD $0x91002231
+ WORD $0x6d79ec7a
+ WORD $0x91002210
+ WORD $0x6d7ca468
+ WORD $0x1e790b19
+ WORD $0x1e640b04
+ WORD $0x1e7a0b1a
+ WORD $0x1e7b0b1b
+ WORD $0x6d7af47c
+ WORD $0x1e680b08
+ WORD $0x6d7bfc7e
+ WORD $0x1e792a94
+ WORD $0x6d7de46a
+ WORD $0x1e642af7
+ WORD $0x1e690b04
+ WORD $0x1e7c0b1c
+ WORD $0x6d7fb46c
+ WORD $0x1e7d0b1d
+ WORD $0x1e6a0b09
+ WORD $0x6d7eac6a
+ WORD $0x1e7e0b1e
+ WORD $0x1e7f0b1f
+ WORD $0x1e7a2a52
+ WORD $0x1e790b19
+ WORD $0x1e7b2ad6
+ WORD $0x1e6c0b1b
+ WORD $0x1e6a0b1a
+ WORD $0x1e6b0b0a
+ WORD $0x1e7c2ab5
+ WORD $0x1e6d0b18
+ WORD $0x1e7d2a31
+ WORD $0x1e6828c6
+ WORD $0x1e7e2a73
+ WORD $0x1e7f2a10
+ WORD $0x1e6428e7
+ WORD $0x1e6928a5
+ WORD $0x1e7929ce
+ WORD $0x1e7b2821
+ WORD $0x1e7a2863
+ WORD $0x1e6a2842
+ WORD $0x1e782800
+ WORD $0x54fffa61
+ WORD $0x17ffff4f
+ WORD $0x6f00e400
WORD $0xaa1f03e9
- WORD $0x927ef0aa
- WORD $0x9100402b
+ WORD $0x9101200a
+ WORD $0x9100812b
+ WORD $0xf81b815f
+ WORD $0x91004129
+ WORD $0xad3e0140
+ WORD $0xeb08017f
+ WORD $0xad3f0140
+ WORD $0xad000140
+ WORD $0x3d800940
+ WORD $0xf900195f
+ WORD $0x9102014a
+ WORD $0x54fffecd
+ WORD $0xeb09010b
+ WORD $0x540008ad
+ WORD $0xf10004bf
+ WORD $0x540005eb
+ WORD $0x8b090c2a
+ WORD $0x927ef0ab
+ WORD $0x9100414c
WORD $0x14000007
WORD $0xfc297800
WORD $0x91000529
- WORD $0x9100216b
+ WORD $0x9100218c
WORD $0xeb08013f
- WORD $0x91002021
- WORD $0x54000700
+ WORD $0x9100214a
+ WORD $0x54000720
WORD $0x2f00e400
WORD $0xf10010bf
WORD $0x54000062
- WORD $0xaa1f03ec
+ WORD $0xaa1f03ed
WORD $0x14000015
- WORD $0xaa0b03ec
- WORD $0x9100404d
- WORD $0x927ef0ae
- WORD $0xad7f91a1
- WORD $0xf10011ce
- WORD $0xad7f8d82
+ WORD $0xaa0c03ed
+ WORD $0x9100404e
+ WORD $0x927ef0af
+ WORD $0xad7f91c1
+ WORD $0xf10011ef
+ WORD $0xad7f8da2
+ WORD $0x910081ce
WORD $0x910081ad
- WORD $0x9100818c
WORD $0x6e61dc41
WORD $0x5e180422
WORD $0x1e612800
@@ -938,39 +2113,45 @@ TEXT ·correlateFloat64NEON(SB), NOSPLIT|NOFRAME, $0-72
WORD $0x1e612800
WORD $0x1e622800
WORD $0x54fffe61
- WORD $0xeb0a00bf
- WORD $0x927ef0ac
+ WORD $0xeb0b00bf
+ WORD $0x927ef0ad
WORD $0x54fffc40
- WORD $0xd37df18e
- WORD $0xcb0c00ac
- WORD $0x8b0e004d
- WORD $0x8b0e002e
- WORD $0xfc4085c1
- WORD $0xf100058c
- WORD $0xfc4085a2
+ WORD $0xd37df1af
+ WORD $0xcb0d00ad
+ WORD $0x8b0f004e
+ WORD $0x8b0f014f
+ WORD $0xfc4085e1
+ WORD $0xf10005ad
+ WORD $0xfc4085c2
WORD $0x1e620821
WORD $0x1e612800
WORD $0x54ffff61
WORD $0x17ffffd7
- WORD $0xf100111f
- WORD $0x54000062
- WORD $0xaa1f03e9
- WORD $0x1400000b
+ WORD $0xf100117f
+ WORD $0x540001c3
WORD $0x6f00e400
- WORD $0x927ef109
- WORD $0x9100400a
- WORD $0x927ef10b
- WORD $0xf100116b
- WORD $0xad3f8140
- WORD $0x9100814a
+ WORD $0x9240050a
+ WORD $0x8b090c0c
+ WORD $0xcb0a016b
+ WORD $0x8b0a012d
+ WORD $0x8b0b0129
+ WORD $0x9100418b
+ WORD $0xcb0801ac
+ WORD $0xb100118c
+ WORD $0xad3f8160
+ WORD $0x9100816b
WORD $0x54ffffa1
- WORD $0xeb09011f
- WORD $0x540000c0
+ WORD $0xb40000ca
WORD $0x8b090c0a
WORD $0xcb090108
WORD $0xf1000508
WORD $0xf800855f
WORD $0x54ffffc1
+ WORD $0x6d4423e9
+ WORD $0x6d432beb
+ WORD $0x6d4233ed
+ WORD $0x6d413bef
+ WORD $0x910143ff
WORD $0xd65f03c0
RET
@@ -982,21 +2163,58 @@ TEXT ·movingAverageFloat32NEON(SB), NOSPLIT|NOFRAME, $0-56
MOVD a_len+32(FP), R3
MOVD width+48(FP), R4
WORD $0xf100049f
- WORD $0x540006cb
+ WORD $0x54000b6b
WORD $0xeb040068
- WORD $0x5400068b
+ WORD $0x54000b2b
WORD $0xeb08005f
WORD $0x9a88d448
- WORD $0xf100051f
- WORD $0x5400060b
- WORD $0x9e230080
+ WORD $0xf100411f
+ WORD $0x5400006a
WORD $0xaa1f03e9
- WORD $0x927dec8a
- WORD $0x9100402b
+ WORD $0x14000022
+ WORD $0x9e230080
+ WORD $0xaa1f03eb
+ WORD $0x9100802a
+ WORD $0x5280020c
+ WORD $0x4e040400
+ WORD $0x6f00e401
+ WORD $0x6f00e404
+ WORD $0xaa0c03e9
+ WORD $0x6f00e403
+ WORD $0x6f00e402
+ WORD $0xaa0403ec
+ WORD $0xaa0a03ed
+ WORD $0xad7f19a5
+ WORD $0xf100058c
+ WORD $0xad4041a7
+ WORD $0x910011ad
+ WORD $0x4e21d4a1
+ WORD $0x4e24d4c4
+ WORD $0x4e23d4e3
+ WORD $0x4e22d602
+ WORD $0x54ffff01
+ WORD $0x6e20fc21
+ WORD $0x8b0b080b
+ WORD $0x9100412c
+ WORD $0xeb08019f
+ WORD $0x9101014a
+ WORD $0x6e20fc84
+ WORD $0x6e20fc63
+ WORD $0xad001161
+ WORD $0x6e20fc42
+ WORD $0xad010963
+ WORD $0xaa0903eb
+ WORD $0x54fffcad
+ WORD $0xeb08013f
+ WORD $0x5400060a
+ WORD $0x9e230080
+ WORD $0x8b09082a
+ WORD $0x927dec8b
+ WORD $0x9100414c
WORD $0x14000008
WORD $0x1e201821
- WORD $0x9100116b
- WORD $0x91001021
+ WORD $0x9100118c
+ WORD $0x9100114a
WORD $0xbc297801
WORD $0x91000529
WORD $0xeb08013f
@@ -1004,13 +2222,13 @@ TEXT ·movingAverageFloat32NEON(SB), NOSPLIT|NOFRAME, $0-56
WORD $0x2f00e401
WORD $0xf100209f
WORD $0x54000062
- WORD $0xaa1f03ed
+ WORD $0xaa1f03ee
WORD $0x14000018
- WORD $0xaa0b03ec
- WORD $0x927dec8d
- WORD $0xad7f8d82
- WORD $0xf10021ad
- WORD $0x9100818c
+ WORD $0xaa0c03ed
+ WORD $0x927dec8e
+ WORD $0xad7f8da2
+ WORD $0xf10021ce
+ WORD $0x910081ad
WORD $0x5e0c0444
WORD $0x1e222821
WORD $0x5e140445
@@ -1026,13 +2244,13 @@ TEXT ·movingAverageFloat32NEON(SB), NOSPLIT|NOFRAME, $0-56
WORD $0x1e242821
WORD $0x1e222821
WORD $0x54fffde1
- WORD $0xeb0a009f
- WORD $0x927dec8d
+ WORD $0xeb0b009f
+ WORD $0x927dec8e
WORD $0x54fffbc0
- WORD $0x8b0d082c
- WORD $0xcb0d008d
- WORD $0xbc404582
- WORD $0xf10005ad
+ WORD $0x8b0e094d
+ WORD $0xcb0e008e
+ WORD $0xbc4045a2
+ WORD $0xf10005ce
WORD $0x1e222821
WORD $0x54ffffa1
WORD $0x17ffffd7
@@ -1153,21 +2371,74 @@ TEXT ·movingAverageFloat64NEON(SB), NOSPLIT|NOFRAME, $0-56
MOVD a_len+32(FP), R3
MOVD width+48(FP), R4
WORD $0xf100049f
- WORD $0x540005cb
+ WORD $0x54000c6b
WORD $0xeb040068
- WORD $0x5400058b
+ WORD $0x54000c2b
WORD $0xeb08005f
WORD $0x9a88d448
- WORD $0xf100051f
- WORD $0x5400050b
- WORD $0x9e630080
+ WORD $0xf100411f
+ WORD $0x5400006a
WORD $0xaa1f03e9
- WORD $0x927ef08a
- WORD $0x9100402b
+ WORD $0x14000032
+ WORD $0x9e630080
+ WORD $0xaa1f03eb
+ WORD $0x9101002a
+ WORD $0x5280020c
+ WORD $0x4e080400
+ WORD $0x6f00e405
+ WORD $0x6f00e410
+ WORD $0xaa0c03e9
+ WORD $0x6f00e407
+ WORD $0x6f00e406
+ WORD $0xaa0a03ec
+ WORD $0x6f00e404
+ WORD $0x6f00e403
+ WORD $0xaa0403ed
+ WORD $0x6f00e402
+ WORD $0x6f00e401
+ WORD $0xad7e4991
+ WORD $0xf10005ad
+ WORD $0xad7f5193
+ WORD $0xad405995
+ WORD $0x4e65d625
+ WORD $0x4e70d650
+ WORD $0xad414991
+ WORD $0x4e67d667
+ WORD $0x4e66d686
+ WORD $0x4e64d6a4
+ WORD $0x4e63d6c3
+ WORD $0x9100218c
+ WORD $0x4e62d622
+ WORD $0x4e61d641
+ WORD $0x54fffe41
+ WORD $0x6e60fca5
+ WORD $0x8b0b0c0b
+ WORD $0x9100412c
+ WORD $0xeb08019f
+ WORD $0x9102014a
+ WORD $0x6e60fe10
+ WORD $0x6e60fce7
+ WORD $0xad004165
+ WORD $0x6e60fcc6
+ WORD $0x6e60fc84
+ WORD $0xad011967
+ WORD $0x6e60fc63
+ WORD $0x6e60fc42
+ WORD $0xad020d64
+ WORD $0x6e60fc21
+ WORD $0xad030562
+ WORD $0xaa0903eb
+ WORD $0x54fffaad
+ WORD $0xeb08013f
+ WORD $0x5400050a
+ WORD $0x9e630080
+ WORD $0x8b090c2a
+ WORD $0x927ef08b
+ WORD $0x9100414c
WORD $0x14000008
WORD $0x1e601821
- WORD $0x9100216b
- WORD $0x91002021
+ WORD $0x9100218c
+ WORD $0x9100214a
WORD $0xfc297801
WORD $0x91000529
WORD $0xeb08013f
@@ -1175,13 +2446,13 @@ TEXT ·movingAverageFloat64NEON(SB), NOSPLIT|NOFRAME, $0-56
WORD $0x2f00e401
WORD $0xf100109f
WORD $0x54000062
- WORD $0xaa1f03ed
+ WORD $0xaa1f03ee
WORD $0x14000010
- WORD $0xaa0b03ec
- WORD $0x927ef08d
- WORD $0xad7f8d82
- WORD $0xf10011ad
- WORD $0x9100818c
+ WORD $0xaa0c03ed
+ WORD $0x927ef08e
+ WORD $0xad7f8da2
+ WORD $0xf10011ce
+ WORD $0x910081ad
WORD $0x5e180444
WORD $0x1e622821
WORD $0x5e180462
@@ -1189,13 +2460,13 @@ TEXT ·movingAverageFloat64NEON(SB), NOSPLIT|NOFRAME, $0-56
WORD $0x1e632821
WORD $0x1e622821
WORD $0x54fffee1
- WORD $0xeb0a009f
- WORD $0x927ef08d
+ WORD $0xeb0b009f
+ WORD $0x927ef08e
WORD $0x54fffcc0
- WORD $0x8b0d0c2c
- WORD $0xcb0d008d
- WORD $0xfc408582
- WORD $0xf10005ad
+ WORD $0x8b0e0d4d
+ WORD $0xcb0e008e
+ WORD $0xfc4085a2
+ WORD $0xf10005ce
WORD $0x1e622821
WORD $0x54ffffa1
WORD $0x17ffffdf
diff --git a/internal/arm64/numeric_sve2_arm64.s b/internal/arm64/numeric_sve2_arm64.s
index 79940bd..ed71ac3 100644
--- a/internal/arm64/numeric_sve2_arm64.s
+++ b/internal/arm64/numeric_sve2_arm64.s
@@ -233,20 +233,75 @@ TEXT ·polyEvalFloat32SVE2(SB), NOSPLIT|NOFRAME, $0-72
MOVD coeffs_len+56(FP), R5
WORD $0xeb04007f
WORD $0x9a84b068
+ WORD $0xb40000c5
+ WORD $0x8b05084a
+ WORD $0xf100411f
+ WORD $0x5400014a
+ WORD $0xaa1f03e9
+ WORD $0x14000035
WORD $0xf100051f
- WORD $0xb4000305
- WORD $0x540008ab
- WORD $0x8b05084b
- WORD $0xf10008bf
- WORD $0xbc5fc160
- WORD $0x5400032b
+ WORD $0x54000e8b
+ WORD $0x0460e3e9
+ WORD $0xeb09011f
+ WORD $0x54000c02
WORD $0xaa1f03e9
- WORD $0xd10004aa
- WORD $0xd100216b
+ WORD $0x1400006a
+ WORD $0xd1001149
+ WORD $0xf10004ab
+ WORD $0x4d40c920
+ WORD $0x5400044d
+ WORD $0xaa1f03ec
+ WORD $0x5280020d
+ WORD $0x8b0c082e
+ WORD $0x4ea01c10
+ WORD $0x4ea01c07
+ WORD $0x4ea01c06
+ WORD $0x4ea01c02
+ WORD $0xaa0d03e9
+ WORD $0xad400dc1
+ WORD $0xd100214d
+ WORD $0xad4115c4
+ WORD $0xaa0b03ee
+ WORD $0xbc5fc5b1
+ WORD $0x6e21de10
+ WORD $0x6e23dce7
+ WORD $0x6e24dcc6
+ WORD $0x6e25dc42
+ WORD $0x4e040631
+ WORD $0xd10005ce
+ WORD $0xf10001df
+ WORD $0x4e30d630
+ WORD $0x4e27d627
+ WORD $0x4e26d626
+ WORD $0x4e22d622
+ WORD $0x54fffe8c
+ WORD $0x8b0c080e
+ WORD $0x9100412d
+ WORD $0xaa0903ec
+ WORD $0xeb0801bf
+ WORD $0xad001dd0
+ WORD $0xad0109c6
+ WORD $0x54fffc6d
+ WORD $0x14000009
+ WORD $0xaa1f03e9
+ WORD $0x9100800b
+ WORD $0x9100812c
+ WORD $0x91004129
+ WORD $0xad3f0160
+ WORD $0xeb08019f
+ WORD $0xac820160
+ WORD $0x54ffff6d
+ WORD $0xeb09010b
+ WORD $0x5400080d
+ WORD $0xbc5fc140
+ WORD $0xf10008bf
+ WORD $0x5400024b
+ WORD $0xd10004ab
+ WORD $0xd100214a
WORD $0x1e204002
WORD $0xbc697821
- WORD $0xaa0b03ec
- WORD $0xaa0a03ed
+ WORD $0xaa0a03ec
+ WORD $0xaa0b03ed
WORD $0x1e220822
WORD $0xbc5fc583
WORD $0xd10005ad
@@ -257,44 +312,41 @@ TEXT ·polyEvalFloat32SVE2(SB), NOSPLIT|NOFRAME, $0-72
WORD $0x91000529
WORD $0xeb08013f
WORD $0x54fffe61
- WORD $0x1400002f
- WORD $0x540005cb
- WORD $0x0460e3e9
- WORD $0xeb09011f
- WORD $0x54000102
- WORD $0xaa1f03e9
- WORD $0x14000012
- WORD $0x0460e3e9
- WORD $0xeb09011f
- WORD $0x540002a2
- WORD $0xaa1f03e9
- WORD $0x1400001f
- WORD $0x6f00e400
- WORD $0xcb0903e9
- WORD $0xaa0003ea
- WORD $0x8a090109
- WORD $0xaa0903eb
- WORD $0xe5804140
- WORD $0x0470e7eb
- WORD $0xe5804540
- WORD $0x0431e3ea
- WORD $0xb5ffff8b
- WORD $0xeb09011f
+ WORD $0x1400002c
+ WORD $0x0460e3ed
+ WORD $0xeb0d017f
+ WORD $0x54000243
+ WORD $0xd37ef52e
+ WORD $0xcb0d03ed
+ WORD $0x05242001
+ WORD $0x8a0d016d
+ WORD $0x2598e3e0
+ WORD $0xaa1f03ea
+ WORD $0x8b0e000c
+ WORD $0x0430e3ee
+ WORD $0x8b0d0129
+ WORD $0x8b0e000e
+ WORD $0xe54a4181
+ WORD $0xe54a41c1
+ WORD $0x0470e3ea
+ WORD $0xeb0a01bf
+ WORD $0x54ffff81
+ WORD $0xeb0d017f
WORD $0x54000300
WORD $0x8b09080a
WORD $0xcb090108
WORD $0xf1000508
- WORD $0xb800455f
+ WORD $0xbc004540
WORD $0x54ffffc1
WORD $0x14000012
+ WORD $0x6f00e400
WORD $0xcb0903e9
- WORD $0x05242001
WORD $0xaa0003ea
WORD $0x8a090109
WORD $0xaa0903eb
- WORD $0xe5804141
+ WORD $0xe5804140
WORD $0x0470e7eb
- WORD $0xe5804541
+ WORD $0xe5804540
WORD $0x0431e3ea
WORD $0xb5ffff8b
WORD $0xeb09011f
@@ -302,7 +354,7 @@ TEXT ·polyEvalFloat32SVE2(SB), NOSPLIT|NOFRAME, $0-72
WORD $0x8b09080a
WORD $0xcb090108
WORD $0xf1000508
- WORD $0xbc004540
+ WORD $0xb800455f
WORD $0x54ffffc1
WORD $0xd65f03c0
RET
@@ -315,89 +367,305 @@ TEXT ·convolveFloat32SVE2(SB), NOSPLIT|NOFRAME, $0-72
MOVD dst_len+8(FP), R3
MOVD sig_len+32(FP), R4
MOVD ker_len+56(FP), R5
- WORD $0xb4000a65
+ WORD $0xb4002565
WORD $0xeb050088
- WORD $0x54000a2b
+ WORD $0x5400252b
+ WORD $0x6db73bef
+ WORD $0x6d0133ed
+ WORD $0x6d022beb
+ WORD $0x6d0323e9
+ WORD $0xa9047bfd
+ WORD $0xa90567fa
+ WORD $0xa9065ff8
+ WORD $0xa90757f6
+ WORD $0xa9084ff4
+ WORD $0xd10083ff
+ WORD $0x043f57ff
WORD $0xeb08007f
+ WORD $0xa9008be1
WORD $0x9a88d468
- WORD $0xf100051f
- WORD $0x540009ab
- WORD $0xf10004bf
- WORD $0x0460e3e9
- WORD $0x540006ab
- WORD $0xd37ef4ad
- WORD $0xcb0903eb
+ WORD $0xf100411f
+ WORD $0x5400006a
+ WORD $0xaa1f03e9
+ WORD $0x140000b9
+ WORD $0xf10000bf
+ WORD $0x540015cd
+ WORD $0xaa0203e9
+ WORD $0x04a0e3ec
WORD $0x2598e3e0
+ WORD $0x0430e7e9
+ WORD $0xcb0c03ec
+ WORD $0xaa1f03f0
+ WORD $0xd100104b
+ WORD $0x9100f022
+ WORD $0x5280020a
+ WORD $0xd280002d
+ WORD $0xd280004e
+ WORD $0xd280006f
+ WORD $0x8b050929
+ WORD $0xd2800091
+ WORD $0x8a0c00a3
+ WORD $0xd28000a6
+ WORD $0xd28000c7
+ WORD $0xd28000f3
+ WORD $0xd2800114
+ WORD $0xd2800135
+ WORD $0xd2800156
+ WORD $0xd2800177
+ WORD $0xd2800198
+ WORD $0xd28001b9
+ WORD $0xd28001da
+ WORD $0xd28001fd
+ WORD $0xaa0103fe
+ WORD $0xf9000fe9
+ WORD $0x1400000e
+ WORD $0x8b10080c
+ WORD $0x9100412a
+ WORD $0x910103de
+ WORD $0xeb08015f
+ WORD $0x91010042
+ WORD $0xaa0903f0
+ WORD $0xbd000186
+ WORD $0x3c804183
+ WORD $0x3c814185
+ WORD $0x3c824184
+ WORD $0x2d068582
+ WORD $0xbd003d80
+ WORD $0x540011ac
+ WORD $0x04a0e3e9
+ WORD $0xeb0900bf
+ WORD $0xaa0a03e9
+ WORD $0x54000142
+ WORD $0x6f00e403
+ WORD $0x2f00e400
+ WORD $0xaa1f03ec
+ WORD $0x2f00e401
+ WORD $0x2f00e402
+ WORD $0x2f00e406
+ WORD $0x6f00e405
+ WORD $0x6f00e404
+ WORD $0x1400005a
+ WORD $0x2f00e400
+ WORD $0x2f00e401
+ WORD $0xf9400fe4
+ WORD $0x2f00e402
+ WORD $0x2f00e41d
WORD $0xaa1f03ea
+ WORD $0x2f00e410
+ WORD $0x2f00e412
+ WORD $0x2f00e404
+ WORD $0x2f00e411
+ WORD $0x2f00e414
+ WORD $0x2f00e416
+ WORD $0x2f00e405
+ WORD $0x2f00e413
+ WORD $0x2f00e415
+ WORD $0x2f00e417
+ WORD $0x2f00e403
+ WORD $0x2f00e406
+ WORD $0x910083ec
+ WORD $0x85804098
+ WORD $0x0430e7e4
+ WORD $0xe580419d
+ WORD $0x8b0a0bcc
+ WORD $0x05b83b1d
+ WORD $0xa55d4189
+ WORD $0xa54a43d8
+ WORD $0xa54d4199
+ WORD $0xa54e419a
+ WORD $0xa54f419b
+ WORD $0xa551419c
+ WORD $0xa546419e
+ WORD $0xa547419f
+ WORD $0x65890ba9
+ WORD $0xa5534188
+ WORD $0xa55a418a
+ WORD $0xa554418b
+ WORD $0xa559418c
+ WORD $0xa555418d
+ WORD $0xa558418e
+ WORD $0xa556418f
+ WORD $0xa5574187
+ WORD $0x65980bb8
+ WORD $0x65990bb9
+ WORD $0x910083ec
+ WORD $0x659a0bba
+ WORD $0x659b0bbb
+ WORD $0x04b0e3ea
+ WORD $0x659c0bbc
+ WORD $0x659e0bbe
+ WORD $0x659f0bbf
+ WORD $0x65880ba8
+ WORD $0x658a0baa
+ WORD $0x658b0bab
+ WORD $0xeb0a007f
+ WORD $0x658c0bac
+ WORD $0x658d0bad
+ WORD $0x658e0bae
+ WORD $0x65870ba7
+ WORD $0x65982120
+ WORD $0x658f0ba9
+ WORD $0x8580419d
+ WORD $0x65982141
+ WORD $0x65982182
+ WORD $0x659821dd
+ WORD $0x659820f0
+ WORD $0x65982132
+ WORD $0x659821a4
+ WORD $0x65982171
+ WORD $0x65982114
+ WORD $0x659823f6
+ WORD $0x659823c5
+ WORD $0x65982393
+ WORD $0x65982375
+ WORD $0x65982357
+ WORD $0x65982323
+ WORD $0x65982306
+ WORD $0x54fff8c1
+ WORD $0x6e0c06e3
+ WORD $0x6e0c06c5
+ WORD $0xeb0300bf
+ WORD $0x6e0c0644
+ WORD $0xaa0303ec
+ WORD $0x6e1406a3
+ WORD $0x6e140685
+ WORD $0x6e140604
+ WORD $0x6e1c0663
+ WORD $0x6e1c0625
+ WORD $0x6e1c07a4
+ WORD $0x54fff1c0
+ WORD $0xd37ef58a
+ WORD $0xcb0c00a4
+ WORD $0xaa0203ec
+ WORD $0x8b0a0181
+ WORD $0xbc647967
+ WORD $0xf1000484
+ WORD $0x2d7f5434
+ WORD $0xbc5c4030
+ WORD $0x3cdc8031
+ WORD $0x3cdd8032
+ WORD $0x9100118c
+ WORD $0x3cde8033
+ WORD $0xbd400036
+ WORD $0x1e3008f0
+ WORD $0x4f879231
+ WORD $0x4f879252
+ WORD $0x1e3408f4
+ WORD $0x4f879273
+ WORD $0x1e3508f5
+ WORD $0x1e3608e7
+ WORD $0x1e3028c6
+ WORD $0x4e31d463
+ WORD $0x4e32d4a5
+ WORD $0x1e342842
+ WORD $0x4e33d484
+ WORD $0x1e352821
+ WORD $0x1e272800
+ WORD $0x54fffd01
+ WORD $0x17ffff71
+ WORD $0x6f00e400
+ WORD $0xaa1f03e9
+ WORD $0x9100800a
+ WORD $0x9100812b
+ WORD $0x91004129
+ WORD $0xad3f0140
+ WORD $0xeb08017f
+ WORD $0xac820140
+ WORD $0x54ffff6d
+ WORD $0xa940bfec
+ WORD $0xeb09010a
+ WORD $0x54000a2d
+ WORD $0xf10000bf
+ WORD $0x540006cd
+ WORD $0xd37ef4ae
+ WORD $0x0460e3ea
+ WORD $0x8b09098c
+ WORD $0xcb0a03eb
+ WORD $0x2598e3e0
+ WORD $0xaa0e03ed
+ WORD $0x0431e7ee
WORD $0x8a0b00ab
- WORD $0xd100104e
- WORD $0xaa0d03ec
- WORD $0x0431e7ed
- WORD $0x0430e7ec
- WORD $0x8b0d004d
- WORD $0x8b0c004c
+ WORD $0x0430e7ed
+ WORD $0x8b0e01ee
+ WORD $0x8b0d01ed
+ WORD $0xd10011ef
WORD $0x14000006
- WORD $0xbc2a7800
- WORD $0x9100054a
- WORD $0x91001021
- WORD $0xeb08015f
- WORD $0x54000720
- WORD $0xeb0900bf
+ WORD $0xbc297800
+ WORD $0x91000529
+ WORD $0x9100118c
+ WORD $0xeb08013f
+ WORD $0x540007a0
+ WORD $0xeb0a00bf
WORD $0x54000082
WORD $0x2f00e400
- WORD $0xaa1f03f0
+ WORD $0xaa1f03f1
WORD $0x14000015
WORD $0x2f00e400
- WORD $0xaa1f03ef
- WORD $0xaa0103f0
- WORD $0xa54f4181
- WORD $0x85804202
+ WORD $0xaa1f03f0
+ WORD $0xaa0c03f1
+ WORD $0xa55041a1
+ WORD $0x85804222
WORD $0x05b83821
WORD $0x65810841
- WORD $0x85804602
- WORD $0x0431e3f0
+ WORD $0x85804622
+ WORD $0x0431e3f1
WORD $0x65982020
- WORD $0xa54f41a1
- WORD $0x0470e7ef
+ WORD $0xa55041c1
+ WORD $0x0470e7f0
WORD $0x05b83821
- WORD $0xab0f017f
+ WORD $0xab10017f
WORD $0x65810841
WORD $0x65982020
WORD $0x54fffe61
WORD $0xeb0b00bf
- WORD $0xaa0b03f0
+ WORD $0xaa0b03f1
WORD $0x54fffc60
- WORD $0xd37ef60f
- WORD $0xcb1000b0
- WORD $0xbc6f6821
- WORD $0xbc7079c2
- WORD $0xf1000610
- WORD $0x910011ef
+ WORD $0xd37ef630
+ WORD $0xcb1100b1
+ WORD $0xbc706981
+ WORD $0xbc7179e2
+ WORD $0xf1000631
+ WORD $0x91001210
WORD $0x1e220821
WORD $0x1e212800
WORD $0x54ffff41
WORD $0x17ffffd9
- WORD $0xeb09011f
- WORD $0x54000062
- WORD $0xaa1f03e9
- WORD $0x1400000d
+ WORD $0x0460e3ed
+ WORD $0xeb0d015f
+ WORD $0x54000243
+ WORD $0xd37ef52e
WORD $0x6f00e400
- WORD $0xcb0903e9
- WORD $0xaa0003ea
- WORD $0x8a090109
- WORD $0xaa0903eb
- WORD $0xe5804140
- WORD $0x0470e7eb
- WORD $0xe5804540
- WORD $0x0431e3ea
- WORD $0xb5ffff8b
- WORD $0xeb09011f
+ WORD $0xcb0d03ed
+ WORD $0x8a0d014d
+ WORD $0x2598e3e0
+ WORD $0xaa1f03eb
+ WORD $0x8b0e000c
+ WORD $0x0430e3ee
+ WORD $0x8b0d0129
+ WORD $0x8b0e000e
+ WORD $0xe54b4180
+ WORD $0xe54b41c0
+ WORD $0x0470e3eb
+ WORD $0xeb0b01bf
+ WORD $0x54ffff81
+ WORD $0xeb0d015f
WORD $0x540000c0
WORD $0x8b09080a
WORD $0xcb090108
WORD $0xf1000508
WORD $0xb800455f
WORD $0x54ffffc1
+ WORD $0x043f503f
+ WORD $0x910083ff
+ WORD $0xa9484ff4
+ WORD $0xa94757f6
+ WORD $0xa9465ff8
+ WORD $0xa94567fa
+ WORD $0xa9447bfd
+ WORD $0x6d4323e9
+ WORD $0x6d422beb
+ WORD $0x6d4133ed
+ WORD $0x6cc93bef
WORD $0xd65f03c0
RET
@@ -409,78 +677,287 @@ TEXT ·correlateFloat32SVE2(SB), NOSPLIT|NOFRAME, $0-72
MOVD dst_len+8(FP), R3
MOVD sig_len+32(FP), R4
MOVD ker_len+56(FP), R5
- WORD $0xb4000905
+ WORD $0xb4002325
WORD $0xeb050088
- WORD $0x540008cb
+ WORD $0x540022eb
+ WORD $0x6db73bef
+ WORD $0x6d0133ed
+ WORD $0x6d022beb
+ WORD $0x6d0323e9
+ WORD $0xa9047bfd
+ WORD $0xa90567fa
+ WORD $0xa9065ff8
+ WORD $0xa90757f6
+ WORD $0xa9084ff4
+ WORD $0xd10043ff
+ WORD $0x043f57ff
WORD $0xeb08007f
+ WORD $0xaa0203ea
+ WORD $0xf90007e1
WORD $0x9a88d468
- WORD $0xf100051f
- WORD $0x5400084b
- WORD $0xf10004bf
- WORD $0x0460e3e9
- WORD $0x5400054b
- WORD $0xcb0903eb
+ WORD $0xf100411f
+ WORD $0x5400006a
+ WORD $0xaa1f03e9
+ WORD $0x140000b0
+ WORD $0xf10000bf
+ WORD $0x540014ad
+ WORD $0x04a0e3e9
WORD $0x2598e3e0
- WORD $0xaa1f03ea
- WORD $0x8a0b00ab
+ WORD $0xaa1f03e4
+ WORD $0xcb0903e9
+ WORD $0x52800202
+ WORD $0xd280002b
+ WORD $0xd280004c
+ WORD $0xd280006d
+ WORD $0x8a0900ae
+ WORD $0xd280008f
+ WORD $0xd28000b0
+ WORD $0xd28000d1
+ WORD $0xd28000e3
+ WORD $0xd2800106
+ WORD $0xd2800127
+ WORD $0xd2800153
+ WORD $0xd2800174
+ WORD $0xd2800195
+ WORD $0xd28001b6
+ WORD $0xd28001d7
+ WORD $0xd28001f8
+ WORD $0xaa0103f9
+ WORD $0x1400000d
+ WORD $0x8b040801
+ WORD $0x91004122
+ WORD $0x91010339
+ WORD $0xeb08005f
+ WORD $0xaa0903e4
+ WORD $0xbd000026
+ WORD $0x3c804023
+ WORD $0x3c814025
+ WORD $0x3c824024
+ WORD $0x2d068422
+ WORD $0xbd003c20
+ WORD $0x5400116c
+ WORD $0x04a0e3e9
+ WORD $0xeb0900bf
+ WORD $0xaa0203e9
+ WORD $0x54000142
+ WORD $0x6f00e403
+ WORD $0x2f00e400
+ WORD $0xaa1f03e2
+ WORD $0x2f00e401
+ WORD $0x2f00e402
+ WORD $0x2f00e406
+ WORD $0x6f00e405
+ WORD $0x6f00e404
+ WORD $0x14000056
+ WORD $0x2f00e400
+ WORD $0x2f00e401
+ WORD $0xaa1f03fa
+ WORD $0x2f00e402
+ WORD $0x2f00e41d
+ WORD $0x2f00e410
+ WORD $0x2f00e412
+ WORD $0x2f00e404
+ WORD $0x2f00e411
+ WORD $0x2f00e414
+ WORD $0x2f00e416
+ WORD $0x2f00e405
+ WORD $0x2f00e413
+ WORD $0x2f00e415
+ WORD $0x2f00e417
+ WORD $0x2f00e403
+ WORD $0x2f00e406
+ WORD $0x8b1a0b22
+ WORD $0x910043fd
+ WORD $0xa55a4338
+ WORD $0xe58043bd
+ WORD $0xa55a415d
+ WORD $0x04b0e3fa
+ WORD $0xa5584049
+ WORD $0xa54b4059
+ WORD $0xa54c405a
+ WORD $0xa54d405b
+ WORD $0xa54f405c
+ WORD $0xa550405e
+ WORD $0xa551405f
+ WORD $0x65980bb8
+ WORD $0xeb1a01df
+ WORD $0x65890ba9
+ WORD $0xa5434048
+ WORD $0xa557404a
+ WORD $0xa546404b
+ WORD $0xa556404c
+ WORD $0x65990bb9
+ WORD $0xa547404d
+ WORD $0xa555404e
+ WORD $0x659a0bba
+ WORD $0xa553404f
+ WORD $0xa5544047
+ WORD $0x659b0bbb
+ WORD $0x659c0bbc
+ WORD $0x659e0bbe
+ WORD $0x659f0bbf
+ WORD $0x65880ba8
+ WORD $0x658a0baa
+ WORD $0x658b0bab
+ WORD $0x658c0bac
+ WORD $0x658d0bad
+ WORD $0x658e0bae
+ WORD $0x65870ba7
+ WORD $0x65982120
+ WORD $0x658f0ba9
+ WORD $0x858043bd
+ WORD $0x65982141
+ WORD $0x65982182
+ WORD $0x659821dd
+ WORD $0x659820f0
+ WORD $0x65982132
+ WORD $0x659821a4
+ WORD $0x65982171
+ WORD $0x65982114
+ WORD $0x659823f6
+ WORD $0x659823c5
+ WORD $0x65982393
+ WORD $0x65982375
+ WORD $0x65982357
+ WORD $0x65982323
+ WORD $0x65982306
+ WORD $0x54fff921
+ WORD $0x6e0c06e3
+ WORD $0x6e0c06c5
+ WORD $0xeb0e00bf
+ WORD $0x6e0c0644
+ WORD $0xaa0e03e2
+ WORD $0x6e1406a3
+ WORD $0x6e140685
+ WORD $0x6e140604
+ WORD $0x6e1c0663
+ WORD $0x6e1c0625
+ WORD $0x6e1c07a4
+ WORD $0x54fff260
+ WORD $0xd37ef45a
+ WORD $0xcb0200bd
+ WORD $0xaa1903fe
+ WORD $0xaa0a03e2
+ WORD $0x8b1a03c1
+ WORD $0xbc7a6847
+ WORD $0xf10007bd
+ WORD $0x2d46d434
+ WORD $0xbd400030
+ WORD $0x3cc04031
+ WORD $0x3cc14032
+ WORD $0x91001042
+ WORD $0x3cc24033
+ WORD $0xbd403c36
+ WORD $0x1e3008f0
+ WORD $0x4f879231
+ WORD $0x4f879252
+ WORD $0x1e3408f4
+ WORD $0x4f879273
+ WORD $0x1e3508f5
+ WORD $0x1e3608e7
+ WORD $0x910013de
+ WORD $0x1e3028c6
+ WORD $0x4e31d463
+ WORD $0x4e32d4a5
+ WORD $0x1e342842
+ WORD $0x4e33d484
+ WORD $0x1e352821
+ WORD $0x1e272800
+ WORD $0x54fffce1
+ WORD $0x17ffff74
+ WORD $0x6f00e400
+ WORD $0xaa1f03e9
+ WORD $0x9100800b
+ WORD $0x9100812c
+ WORD $0x91004129
+ WORD $0xad3f0160
+ WORD $0xeb08019f
+ WORD $0xac820160
+ WORD $0x54ffff6d
+ WORD $0xf94007ee
+ WORD $0xeb09010b
+ WORD $0x540008ed
+ WORD $0x0460e3ed
+ WORD $0xf10000bf
+ WORD $0x5400058d
+ WORD $0xcb0d03ec
+ WORD $0x8b0909cb
+ WORD $0x2598e3e0
+ WORD $0x8a0c00ac
WORD $0x14000006
- WORD $0xbc2a7800
- WORD $0x9100054a
- WORD $0x91001021
- WORD $0xeb08015f
- WORD $0x540006a0
+ WORD $0xbc297800
+ WORD $0x91000529
+ WORD $0x9100116b
+ WORD $0xeb08013f
+ WORD $0x54000740
+ WORD $0xeb0d00bf
+ WORD $0x54000082
WORD $0x2f00e400
- WORD $0xeb0900bf
- WORD $0x54000062
- WORD $0xaa1f03ec
- WORD $0x14000013
- WORD $0xaa0103ec
- WORD $0xaa0203ed
- WORD $0xaa0b03ee
- WORD $0x85804181
- WORD $0x858041a2
- WORD $0x0470e7ee
+ WORD $0xaa1f03ee
+ WORD $0x14000015
+ WORD $0xaa0b03ef
+ WORD $0x2f00e400
+ WORD $0xaa1f03ee
+ WORD $0x0430e3ef
+ WORD $0xaa0a03f0
+ WORD $0xa54e4161
+ WORD $0x85804202
WORD $0x65820821
- WORD $0x858045a2
- WORD $0x0431e3ed
+ WORD $0x85804602
+ WORD $0x0431e3f0
WORD $0x65982020
- WORD $0x85804581
- WORD $0x0431e3ec
+ WORD $0xa54e41e1
+ WORD $0x0470e3ee
WORD $0x65820821
+ WORD $0xeb0e019f
WORD $0x65982020
- WORD $0xb5fffeae
- WORD $0xeb0b00bf
- WORD $0xaa0b03ec
- WORD $0x54fffca0
- WORD $0xbc6c7821
- WORD $0xbc6c7842
- WORD $0x9100058c
+ WORD $0x54fffea1
WORD $0xeb0c00bf
+ WORD $0xaa0c03ee
+ WORD $0x54fffc60
+ WORD $0xbc6e7961
+ WORD $0xbc6e7942
+ WORD $0x910005ce
+ WORD $0xeb0e00bf
WORD $0x1e220821
WORD $0x1e212800
WORD $0x54ffff41
- WORD $0x17ffffdd
- WORD $0xeb09011f
- WORD $0x54000062
- WORD $0xaa1f03e9
- WORD $0x1400000d
+ WORD $0x17ffffdb
+ WORD $0xeb0d017f
+ WORD $0x54000243
+ WORD $0xd37ef52e
WORD $0x6f00e400
- WORD $0xcb0903e9
- WORD $0xaa0003ea
- WORD $0x8a090109
- WORD $0xaa0903eb
- WORD $0xe5804140
- WORD $0x0470e7eb
- WORD $0xe5804540
- WORD $0x0431e3ea
- WORD $0xb5ffff8b
- WORD $0xeb09011f
+ WORD $0xcb0d03ed
+ WORD $0x8a0d016d
+ WORD $0x2598e3e0
+ WORD $0xaa1f03ea
+ WORD $0x8b0e000c
+ WORD $0x0430e3ee
+ WORD $0x8b0d0129
+ WORD $0x8b0e000e
+ WORD $0xe54a4180
+ WORD $0xe54a41c0
+ WORD $0x0470e3ea
+ WORD $0xeb0a01bf
+ WORD $0x54ffff81
+ WORD $0xeb0d017f
WORD $0x540000c0
WORD $0x8b09080a
WORD $0xcb090108
WORD $0xf1000508
WORD $0xb800455f
WORD $0x54ffffc1
+ WORD $0x043f503f
+ WORD $0x910043ff
+ WORD $0xa9484ff4
+ WORD $0xa94757f6
+ WORD $0xa9465ff8
+ WORD $0xa94567fa
+ WORD $0xa9447bfd
+ WORD $0x6d4323e9
+ WORD $0x6d422beb
+ WORD $0x6d4133ed
+ WORD $0x6cc93bef
WORD $0xd65f03c0
RET
@@ -725,20 +1202,98 @@ TEXT ·polyEvalFloat64SVE2(SB), NOSPLIT|NOFRAME, $0-72
MOVD coeffs_len+56(FP), R5
WORD $0xeb04007f
WORD $0x9a84b068
+ WORD $0xb40000c5
+ WORD $0x8b050c4a
+ WORD $0xf100411f
+ WORD $0x5400014a
+ WORD $0xaa1f03e9
+ WORD $0x1400004c
WORD $0xf100051f
- WORD $0xb4000305
- WORD $0x540008ab
- WORD $0x8b050c4b
- WORD $0xf10008bf
- WORD $0xfc5f8160
- WORD $0x5400032b
+ WORD $0x5400116b
+ WORD $0x04a0e3e9
+ WORD $0xeb09011f
+ WORD $0x54000ee2
WORD $0xaa1f03e9
- WORD $0xd10004aa
- WORD $0xd100416b
+ WORD $0x14000081
+ WORD $0xfc5f8140
+ WORD $0xf10004ab
+ WORD $0x5400066d
+ WORD $0x4e080400
+ WORD $0xaa1f03ec
+ WORD $0x5280020d
+ WORD $0x8b0c0c2e
+ WORD $0x4ea01c18
+ WORD $0x4ea01c17
+ WORD $0x4ea01c16
+ WORD $0x4ea01c15
+ WORD $0xaa0d03e9
+ WORD $0xad4009c1
+ WORD $0x4ea01c14
+ WORD $0xad4111c3
+ WORD $0x4ea01c13
+ WORD $0xad4219c5
+ WORD $0x4ea01c12
+ WORD $0xad4341c7
+ WORD $0x4ea01c11
+ WORD $0xd100414d
+ WORD $0xaa0b03ee
+ WORD $0xfc5f85b9
+ WORD $0x6e61df18
+ WORD $0x6e62def7
+ WORD $0x6e63ded6
+ WORD $0x6e64deb5
+ WORD $0x6e65de94
+ WORD $0x6e66de73
+ WORD $0xd10005ce
+ WORD $0x6e67de52
+ WORD $0x6e70de31
+ WORD $0xf10001df
+ WORD $0x4e080739
+ WORD $0x4e78d738
+ WORD $0x4e77d737
+ WORD $0x4e76d736
+ WORD $0x4e75d735
+ WORD $0x4e74d734
+ WORD $0x4e73d733
+ WORD $0x4e72d732
+ WORD $0x4e71d731
+ WORD $0x54fffd8c
+ WORD $0x8b0c0c0e
+ WORD $0x9100412d
+ WORD $0xaa0903ec
+ WORD $0xeb0801bf
+ WORD $0xad005dd8
+ WORD $0xad0155d6
+ WORD $0xad024dd4
+ WORD $0xad0345d2
+ WORD $0x54fffa6d
+ WORD $0x14000010
+ WORD $0xaa1f03e9
+ WORD $0x9101000b
+ WORD $0x9100812c
+ WORD $0x6d3c0160
+ WORD $0x91004129
+ WORD $0x6d3d0160
+ WORD $0xeb08019f
+ WORD $0x6d3e0160
+ WORD $0x6d3f0160
+ WORD $0x6d000160
+ WORD $0x6d010160
+ WORD $0x6d020160
+ WORD $0x6d030160
+ WORD $0x9102016b
+ WORD $0x54fffe8d
+ WORD $0xeb09010b
+ WORD $0x5400080d
+ WORD $0xfc5f8140
+ WORD $0xf10008bf
+ WORD $0x5400024b
+ WORD $0xd10004ab
+ WORD $0xd100414a
WORD $0x1e604002
WORD $0xfc697821
- WORD $0xaa0b03ec
- WORD $0xaa0a03ed
+ WORD $0xaa0a03ec
+ WORD $0xaa0b03ed
WORD $0x1e620822
WORD $0xfc5f8583
WORD $0xd10005ad
@@ -749,44 +1304,41 @@ TEXT ·polyEvalFloat64SVE2(SB), NOSPLIT|NOFRAME, $0-72
WORD $0x91000529
WORD $0xeb08013f
WORD $0x54fffe61
- WORD $0x1400002f
- WORD $0x540005cb
- WORD $0x04a0e3e9
- WORD $0xeb09011f
- WORD $0x54000102
- WORD $0xaa1f03e9
- WORD $0x14000012
- WORD $0x04a0e3e9
- WORD $0xeb09011f
- WORD $0x540002a2
- WORD $0xaa1f03e9
- WORD $0x1400001f
- WORD $0x6f00e400
- WORD $0xcb0903e9
- WORD $0xaa0003ea
- WORD $0x8a090109
- WORD $0xaa0903eb
- WORD $0xe5804140
- WORD $0x04b0e7eb
- WORD $0xe5804540
- WORD $0x0431e3ea
- WORD $0xb5ffff8b
- WORD $0xeb09011f
+ WORD $0x1400002c
+ WORD $0x04a0e3ed
+ WORD $0xeb0d017f
+ WORD $0x54000243
+ WORD $0xd37df12e
+ WORD $0xcb0d03ed
+ WORD $0x05282001
+ WORD $0x8a0d016d
+ WORD $0x25d8e3e0
+ WORD $0xaa1f03ea
+ WORD $0x8b0e000c
+ WORD $0x0430e3ee
+ WORD $0x8b0d0129
+ WORD $0x8b0e000e
+ WORD $0xe5ea4181
+ WORD $0xe5ea41c1
+ WORD $0x04b0e3ea
+ WORD $0xeb0a01bf
+ WORD $0x54ffff81
+ WORD $0xeb0d017f
WORD $0x54000300
WORD $0x8b090c0a
WORD $0xcb090108
WORD $0xf1000508
- WORD $0xf800855f
+ WORD $0xfc008540
WORD $0x54ffffc1
WORD $0x14000012
+ WORD $0x6f00e400
WORD $0xcb0903e9
- WORD $0x05282001
WORD $0xaa0003ea
WORD $0x8a090109
WORD $0xaa0903eb
- WORD $0xe5804141
+ WORD $0xe5804140
WORD $0x04b0e7eb
- WORD $0xe5804541
+ WORD $0xe5804540
WORD $0x0431e3ea
WORD $0xb5ffff8b
WORD $0xeb09011f
@@ -794,7 +1346,7 @@ TEXT ·polyEvalFloat64SVE2(SB), NOSPLIT|NOFRAME, $0-72
WORD $0x8b090c0a
WORD $0xcb090108
WORD $0xf1000508
- WORD $0xfc008540
+ WORD $0xf800855f
WORD $0x54ffffc1
WORD $0xd65f03c0
RET
@@ -807,89 +1359,326 @@ TEXT ·convolveFloat64SVE2(SB), NOSPLIT|NOFRAME, $0-72
MOVD dst_len+8(FP), R3
MOVD sig_len+32(FP), R4
MOVD ker_len+56(FP), R5
- WORD $0xb4000a65
+ WORD $0xb4002805
WORD $0xeb050088
- WORD $0x54000a2b
+ WORD $0x540027cb
+ WORD $0xd10283ff
WORD $0xeb08007f
+ WORD $0xfd000bee
WORD $0x9a88d468
- WORD $0xf100051f
- WORD $0x540009ab
- WORD $0xf10004bf
- WORD $0x04a0e3e9
- WORD $0x540006ab
- WORD $0xd37df0ad
- WORD $0xcb0903eb
+ WORD $0x6d0233ed
+ WORD $0xf100411f
+ WORD $0x6d032beb
+ WORD $0x6d0423e9
+ WORD $0xa9057bfd
+ WORD $0xa90667fa
+ WORD $0xa9075ff8
+ WORD $0xa90857f6
+ WORD $0xa9094ff4
+ WORD $0xa9000be1
+ WORD $0x5400006a
+ WORD $0xaa1f03e9
+ WORD $0x140000d0
+ WORD $0xf10000bf
+ WORD $0x5400180d
+ WORD $0xaa0203e9
+ WORD $0x04e0e3ec
WORD $0x25d8e3e0
+ WORD $0x0430e7e9
+ WORD $0xcb0c03ec
+ WORD $0xaa1f03f5
+ WORD $0xd100204b
+ WORD $0x9101e022
+ WORD $0x5280020a
+ WORD $0xd280002d
+ WORD $0xd280004e
+ WORD $0xd280006f
+ WORD $0x8b050d29
+ WORD $0xd2800090
+ WORD $0x8a0c00b1
+ WORD $0xd28000a4
+ WORD $0xd28000c6
+ WORD $0xd28000e7
+ WORD $0xd2800113
+ WORD $0xd2800134
+ WORD $0xd2800156
+ WORD $0xd2800177
+ WORD $0xd2800198
+ WORD $0xd28001b9
+ WORD $0xd28001da
+ WORD $0xd28001fd
+ WORD $0xaa0103fe
+ WORD $0xf9000fe9
+ WORD $0x14000010
+ WORD $0x8b150c0c
+ WORD $0x9100412a
+ WORD $0x910203de
+ WORD $0xeb08015f
+ WORD $0x91020042
+ WORD $0xaa0903f5
+ WORD $0x6d005997
+ WORD $0x6d015195
+ WORD $0x6d024993
+ WORD $0x6d034191
+ WORD $0x6d041987
+ WORD $0x6d051185
+ WORD $0x6d060983
+ WORD $0x6d070181
+ WORD $0x5400144c
+ WORD $0x04e0e3e9
+ WORD $0xeb0900bf
+ WORD $0xaa0a03e9
+ WORD $0x54000262
+ WORD $0x2f00e400
+ WORD $0x2f00e401
+ WORD $0xaa1f03ec
+ WORD $0x2f00e402
+ WORD $0x2f00e403
+ WORD $0x2f00e404
+ WORD $0x2f00e405
+ WORD $0x2f00e406
+ WORD $0x2f00e407
+ WORD $0x2f00e410
+ WORD $0x2f00e411
+ WORD $0x2f00e412
+ WORD $0x2f00e413
+ WORD $0x2f00e414
+ WORD $0x2f00e415
+ WORD $0x2f00e416
+ WORD $0x2f00e417
+ WORD $0x1400004d
+ WORD $0x2f00e400
+ WORD $0x2f00e401
+ WORD $0xf9400fe3
+ WORD $0x2f00e402
+ WORD $0x2f00e403
WORD $0xaa1f03ea
+ WORD $0x2f00e404
+ WORD $0x2f00e405
+ WORD $0x2f00e406
+ WORD $0x2f00e407
+ WORD $0x2f00e410
+ WORD $0x2f00e411
+ WORD $0x2f00e412
+ WORD $0x2f00e413
+ WORD $0x2f00e414
+ WORD $0x2f00e415
+ WORD $0x2f00e416
+ WORD $0x2f00e417
+ WORD $0x85804078
+ WORD $0x8b0a0fcc
+ WORD $0x0430e7e3
+ WORD $0xa5fa419a
+ WORD $0xa5fd4199
+ WORD $0xa5ed419b
+ WORD $0x05f83b18
+ WORD $0xa5f8419f
+ WORD $0xa5f94188
+ WORD $0xa5ee419c
+ WORD $0xa5ef419d
+ WORD $0xa5f0419e
+ WORD $0xa5e44189
+ WORD $0xa5e6418a
+ WORD $0xa5f7418b
+ WORD $0x65da0b1a
+ WORD $0x65d90b19
+ WORD $0xa5f6418c
+ WORD $0x65c80b08
+ WORD $0xa5f4418d
+ WORD $0xa5e7418e
+ WORD $0x65dc0b1c
+ WORD $0x65cb0b0b
+ WORD $0x65dd0b1d
+ WORD $0x65cc0b0c
+ WORD $0x65d82341
+ WORD $0x65db0b1a
+ WORD $0x65df0b1b
+ WORD $0x65d82320
+ WORD $0xa5ea43d9
+ WORD $0xa5f3419f
+ WORD $0x65d82102
+ WORD $0x65de0b1e
+ WORD $0x65cd0b08
+ WORD $0x65d82164
+ WORD $0x04f0e3ea
+ WORD $0x65d90b19
+ WORD $0x65df0b1f
+ WORD $0x65d82363
+ WORD $0x65c90b1b
+ WORD $0x65ca0b09
+ WORD $0x65ce0b18
+ WORD $0xeb0a023f
+ WORD $0x65d82185
+ WORD $0x65d82106
+ WORD $0x65d823e7
+ WORD $0x65d82310
+ WORD $0x65d82131
+ WORD $0x65d82372
+ WORD $0x65d823d3
+ WORD $0x65d823b4
+ WORD $0x65d82395
+ WORD $0x65d82356
+ WORD $0x65d82337
+ WORD $0x54fff941
+ WORD $0xeb1100bf
+ WORD $0xaa1103ec
+ WORD $0x54fff200
+ WORD $0xd37df18a
+ WORD $0xcb0c00a3
+ WORD $0xaa0203ec
+ WORD $0x8b0a0181
+ WORD $0xfc637978
+ WORD $0xf1000463
+ WORD $0x6d78e839
+ WORD $0x9100218c
+ WORD $0x6d79f03b
+ WORD $0x6d7ca829
+ WORD $0x1e7a0b1a
+ WORD $0x1e790b19
+ WORD $0x1e7b0b1b
+ WORD $0x1e7c0b1c
+ WORD $0x6d7af83d
+ WORD $0x1e690b09
+ WORD $0x6d7ba03f
+ WORD $0x1e7a2ad6
+ WORD $0x6d7de82b
+ WORD $0x1e792af7
+ WORD $0x1e6a0b19
+ WORD $0x1e7d0b1d
+ WORD $0x6d7fb82d
+ WORD $0x1e7e0b1e
+ WORD $0x1e6b0b0a
+ WORD $0x6d7eb02b
+ WORD $0x1e7f0b1f
+ WORD $0x1e680b08
+ WORD $0x1e7b2ab5
+ WORD $0x1e7a0b1a
+ WORD $0x1e7c2a94
+ WORD $0x1e6d0b1c
+ WORD $0x1e6b0b1b
+ WORD $0x1e6c0b0b
+ WORD $0x1e7d2a73
+ WORD $0x1e6e0b18
+ WORD $0x1e7e2a52
+ WORD $0x1e6928e7
+ WORD $0x1e7f2a31
+ WORD $0x1e682a10
+ WORD $0x1e7928c6
+ WORD $0x1e6a28a5
+ WORD $0x1e7a2884
+ WORD $0x1e7c2821
+ WORD $0x1e7b2863
+ WORD $0x1e6b2842
+ WORD $0x1e782800
+ WORD $0x54fffa81
+ WORD $0x17ffff5f
+ WORD $0x6f00e400
+ WORD $0xaa1f03e9
+ WORD $0x9101200a
+ WORD $0x9100812b
+ WORD $0xf81b815f
+ WORD $0x91004129
+ WORD $0xad3e0140
+ WORD $0xeb08017f
+ WORD $0xad3f0140
+ WORD $0xad000140
+ WORD $0x3d800940
+ WORD $0xf900195f
+ WORD $0x9102014a
+ WORD $0x54fffecd
+ WORD $0xa9403fec
+ WORD $0xeb09010a
+ WORD $0x54000a2d
+ WORD $0xf10000bf
+ WORD $0x540006cd
+ WORD $0xd37df0ae
+ WORD $0x04a0e3ea
+ WORD $0x8b090d8c
+ WORD $0xcb0a03eb
+ WORD $0x25d8e3e0
+ WORD $0xaa0e03ed
+ WORD $0x0431e7ee
WORD $0x8a0b00ab
- WORD $0xd100204e
- WORD $0xaa0d03ec
- WORD $0x0431e7ed
- WORD $0x0430e7ec
- WORD $0x8b0d004d
- WORD $0x8b0c004c
+ WORD $0x0430e7ed
+ WORD $0x8b0e01ee
+ WORD $0x8b0d01ed
+ WORD $0xd10021ef
WORD $0x14000006
- WORD $0xfc2a7800
- WORD $0x9100054a
- WORD $0x91002021
- WORD $0xeb08015f
- WORD $0x54000720
- WORD $0xeb0900bf
+ WORD $0xfc297800
+ WORD $0x91000529
+ WORD $0x9100218c
+ WORD $0xeb08013f
+ WORD $0x540007a0
+ WORD $0xeb0a00bf
WORD $0x54000082
WORD $0x2f00e400
- WORD $0xaa1f03f0
+ WORD $0xaa1f03f1
WORD $0x14000015
WORD $0x2f00e400
- WORD $0xaa1f03ef
- WORD $0xaa0103f0
- WORD $0xa5ef4181
- WORD $0x85804202
+ WORD $0xaa1f03f0
+ WORD $0xaa0c03f1
+ WORD $0xa5f041a1
+ WORD $0x85804222
WORD $0x05f83821
WORD $0x65c10841
- WORD $0xa5ef41a2
- WORD $0x04b0e7ef
+ WORD $0xa5f041c2
+ WORD $0x04b0e7f0
WORD $0x05f83842
- WORD $0xab0f017f
+ WORD $0xab10017f
WORD $0x65d82020
- WORD $0x85804601
- WORD $0x0431e3f0
+ WORD $0x85804621
+ WORD $0x0431e3f1
WORD $0x65c20821
WORD $0x65d82020
WORD $0x54fffe61
WORD $0xeb0b00bf
- WORD $0xaa0b03f0
+ WORD $0xaa0b03f1
WORD $0x54fffc60
- WORD $0xd37df20f
- WORD $0xcb1000b0
- WORD $0xfc6f6821
- WORD $0xfc7079c2
- WORD $0xf1000610
- WORD $0x910021ef
+ WORD $0xd37df230
+ WORD $0xcb1100b1
+ WORD $0xfc706981
+ WORD $0xfc7179e2
+ WORD $0xf1000631
+ WORD $0x91002210
WORD $0x1e620821
WORD $0x1e612800
WORD $0x54ffff41
WORD $0x17ffffd9
- WORD $0xeb09011f
- WORD $0x54000062
- WORD $0xaa1f03e9
- WORD $0x1400000d
+ WORD $0x04a0e3ed
+ WORD $0xeb0d015f
+ WORD $0x54000243
+ WORD $0xd37df12e
WORD $0x6f00e400
- WORD $0xcb0903e9
- WORD $0xaa0003ea
- WORD $0x8a090109
- WORD $0xaa0903eb
- WORD $0xe5804140
- WORD $0x04b0e7eb
- WORD $0xe5804540
- WORD $0x0431e3ea
- WORD $0xb5ffff8b
- WORD $0xeb09011f
+ WORD $0xcb0d03ed
+ WORD $0x8a0d014d
+ WORD $0x25d8e3e0
+ WORD $0xaa1f03eb
+ WORD $0x8b0e000c
+ WORD $0x0430e3ee
+ WORD $0x8b0d0129
+ WORD $0x8b0e000e
+ WORD $0xe5eb4180
+ WORD $0xe5eb41c0
+ WORD $0x04b0e3eb
+ WORD $0xeb0b01bf
+ WORD $0x54ffff81
+ WORD $0xeb0d015f
WORD $0x540000c0
WORD $0x8b090c0a
WORD $0xcb090108
WORD $0xf1000508
WORD $0xf800855f
WORD $0x54ffffc1
+ WORD $0xa9494ff4
+ WORD $0xfd400bee
+ WORD $0xa94857f6
+ WORD $0xa9475ff8
+ WORD $0xa94667fa
+ WORD $0xa9457bfd
+ WORD $0x6d4423e9
+ WORD $0x6d432beb
+ WORD $0x6d4233ed
+ WORD $0x910283ff
WORD $0xd65f03c0
RET
@@ -901,78 +1690,299 @@ TEXT ·correlateFloat64SVE2(SB), NOSPLIT|NOFRAME, $0-72
MOVD dst_len+8(FP), R3
MOVD sig_len+32(FP), R4
MOVD ker_len+56(FP), R5
- WORD $0xb4000905
- WORD $0xeb050088
- WORD $0x540008cb
- WORD $0xeb08007f
- WORD $0x9a88d468
- WORD $0xf100051f
- WORD $0x5400084b
- WORD $0xf10004bf
- WORD $0x04a0e3e9
- WORD $0x5400054b
- WORD $0xcb0903eb
+ WORD $0xb40024a5
+ WORD $0xeb050089
+ WORD $0x5400246b
+ WORD $0xd10283ff
+ WORD $0xeb09007f
+ WORD $0xaa0203e8
+ WORD $0xfd000bee
+ WORD $0x9a89d462
+ WORD $0x6d0233ed
+ WORD $0xf100405f
+ WORD $0x6d032beb
+ WORD $0x6d0423e9
+ WORD $0xa9057bfd
+ WORD $0xa90667fa
+ WORD $0xa9075ff8
+ WORD $0xa90857f6
+ WORD $0xa9094ff4
+ WORD $0xf90003e1
+ WORD $0x5400006a
+ WORD $0xaa1f03e9
+ WORD $0x140000bd
+ WORD $0xf10000bf
+ WORD $0x540015ad
+ WORD $0x04e0e3e9
WORD $0x25d8e3e0
- WORD $0xaa1f03ea
- WORD $0x8a0b00ab
- WORD $0x14000006
- WORD $0xfc2a7800
- WORD $0x9100054a
- WORD $0x91002021
- WORD $0xeb08015f
- WORD $0x540006a0
+ WORD $0xaa1f03fa
+ WORD $0xcb0903e9
+ WORD $0x9101e02a
+ WORD $0x5280020b
+ WORD $0xd280002c
+ WORD $0xd280004d
+ WORD $0x8a0900ae
+ WORD $0xd280006f
+ WORD $0xd2800090
+ WORD $0xd28000b1
+ WORD $0xd28000c3
+ WORD $0xd28000e4
+ WORD $0xd2800106
+ WORD $0xd2800127
+ WORD $0xd2800153
+ WORD $0xd2800174
+ WORD $0xd2800195
+ WORD $0xd28001b6
+ WORD $0xd28001d7
+ WORD $0xd28001f8
+ WORD $0xaa0103f9
+ WORD $0xf9000fe0
+ WORD $0xf90007e2
+ WORD $0x14000011
+ WORD $0xf94007e2
+ WORD $0x8b1a0c01
+ WORD $0x9100412b
+ WORD $0x91020339
+ WORD $0x9102014a
+ WORD $0xaa0903fa
+ WORD $0xeb02017f
+ WORD $0x6d005837
+ WORD $0x6d015035
+ WORD $0x6d024833
+ WORD $0x6d034031
+ WORD $0x6d041827
+ WORD $0x6d051025
+ WORD $0x6d060823
+ WORD $0x6d070021
+ WORD $0x5400122c
WORD $0x2f00e400
+ WORD $0x2f00e401
+ WORD $0x04e0e3e9
+ WORD $0x2f00e402
+ WORD $0x2f00e403
WORD $0xeb0900bf
+ WORD $0x2f00e404
+ WORD $0x2f00e405
+ WORD $0xaa0b03e9
+ WORD $0x2f00e406
+ WORD $0x2f00e407
+ WORD $0x2f00e410
+ WORD $0x2f00e411
+ WORD $0x2f00e412
+ WORD $0x2f00e413
+ WORD $0x2f00e414
+ WORD $0x2f00e415
+ WORD $0x2f00e416
+ WORD $0x2f00e417
WORD $0x54000062
- WORD $0xaa1f03ec
- WORD $0x14000013
- WORD $0xaa0103ec
- WORD $0xaa0203ed
+ WORD $0xaa1f03eb
+ WORD $0x1400003a
+ WORD $0xaa1f03fd
+ WORD $0x8b1d0f2b
+ WORD $0xa5fd4118
+ WORD $0xa5f7417a
+ WORD $0xa5f84179
+ WORD $0xa5ec417b
+ WORD $0xa5f6417f
+ WORD $0xa5f54168
+ WORD $0xa5ed417c
+ WORD $0xa5ef417d
+ WORD $0xa5f0417e
+ WORD $0xa5f14169
+ WORD $0x65da0b1a
+ WORD $0x65d90b19
+ WORD $0xa5e3416a
+ WORD $0x65df0b1f
+ WORD $0xa5f4416b
+ WORD $0xa5f3416c
+ WORD $0xa5e7416d
+ WORD $0xa5e4416e
+ WORD $0x65dc0b1c
+ WORD $0x65dd0b1d
+ WORD $0x65de0b1e
+ WORD $0x65d82341
+ WORD $0x65db0b1a
+ WORD $0x65c80b1b
+ WORD $0x65d82320
+ WORD $0xa5fd4339
+ WORD $0xa5e64168
+ WORD $0x65cb0b0b
+ WORD $0x65cc0b0c
+ WORD $0x04f0e3fd
+ WORD $0x65d823e2
+ WORD $0x65cd0b1f
+ WORD $0x65d90b19
+ WORD $0x65c80b08
+ WORD $0x65d82363
+ WORD $0x65c90b1b
+ WORD $0x65ca0b09
+ WORD $0x65ce0b18
+ WORD $0xeb1d01df
+ WORD $0x65d82164
+ WORD $0x65d82185
+ WORD $0x65d823e6
+ WORD $0x65d82107
+ WORD $0x65d82310
+ WORD $0x65d82131
+ WORD $0x65d82372
+ WORD $0x65d823d3
+ WORD $0x65d823b4
+ WORD $0x65d82395
+ WORD $0x65d82356
+ WORD $0x65d82337
+ WORD $0x54fff981
+ WORD $0xeb0e00bf
+ WORD $0xaa0e03eb
+ WORD $0x54fff440
+ WORD $0xd37df17d
+ WORD $0xcb0b00be
+ WORD $0xaa0a03eb
+ WORD $0xaa0803e2
+ WORD $0x8b1d0161
+ WORD $0xfc7d6858
+ WORD $0xf10007de
+ WORD $0x6d78e839
+ WORD $0x91002042
+ WORD $0x6d79f03b
+ WORD $0x9100216b
+ WORD $0x6d7ca829
+ WORD $0x1e7a0b1a
+ WORD $0x1e790b19
+ WORD $0x1e7b0b1b
+ WORD $0x1e7c0b1c
+ WORD $0x6d7af83d
+ WORD $0x1e690b09
+ WORD $0x6d7ba03f
+ WORD $0x1e7a2ad6
+ WORD $0x6d7de82b
+ WORD $0x1e792af7
+ WORD $0x1e6a0b19
+ WORD $0x1e7d0b1d
+ WORD $0x6d7fb82d
+ WORD $0x1e7e0b1e
+ WORD $0x1e6b0b0a
+ WORD $0x6d7eb02b
+ WORD $0x1e7f0b1f
+ WORD $0x1e680b08
+ WORD $0x1e7b2ab5
+ WORD $0x1e7a0b1a
+ WORD $0x1e7c2a94
+ WORD $0x1e6d0b1c
+ WORD $0x1e6b0b1b
+ WORD $0x1e6c0b0b
+ WORD $0x1e7d2a73
+ WORD $0x1e6e0b18
+ WORD $0x1e7e2a52
+ WORD $0x1e6928e7
+ WORD $0x1e7f2a31
+ WORD $0x1e682a10
+ WORD $0x1e7928c6
+ WORD $0x1e6a28a5
+ WORD $0x1e7a2884
+ WORD $0x1e7c2821
+ WORD $0x1e7b2863
+ WORD $0x1e6b2842
+ WORD $0x1e782800
+ WORD $0x54fffa61
+ WORD $0x17ffff6f
+ WORD $0x6f00e400
+ WORD $0xaa1f03e9
+ WORD $0x9101200a
+ WORD $0x9100812b
+ WORD $0xf81b815f
+ WORD $0x91004129
+ WORD $0xad3e0140
+ WORD $0xeb02017f
+ WORD $0xad3f0140
+ WORD $0xad000140
+ WORD $0x3d800940
+ WORD $0xf900195f
+ WORD $0x9102014a
+ WORD $0x54fffecd
+ WORD $0xf94003eb
+ WORD $0xeb09004a
+ WORD $0x5400090d
+ WORD $0xf10000bf
+ WORD $0x540005ad
+ WORD $0x04a0e3ea
+ WORD $0x8b090d6b
+ WORD $0x25d8e3e0
+ WORD $0xcb0a03ec
+ WORD $0x8a0c00ac
+ WORD $0x14000006
+ WORD $0xfc297800
+ WORD $0x91000529
+ WORD $0x9100216b
+ WORD $0xeb02013f
+ WORD $0x54000760
+ WORD $0xeb0a00bf
+ WORD $0x54000082
+ WORD $0x2f00e400
+ WORD $0xaa1f03ed
+ WORD $0x14000015
WORD $0xaa0b03ee
- WORD $0x85804181
- WORD $0x858041a2
- WORD $0x04b0e7ee
+ WORD $0x2f00e400
+ WORD $0xaa1f03ed
+ WORD $0x0430e3ee
+ WORD $0xaa0803ef
+ WORD $0xa5ed4161
+ WORD $0x858041e2
WORD $0x65c20821
- WORD $0x858045a2
- WORD $0x0431e3ed
+ WORD $0x858045e2
+ WORD $0x0431e3ef
WORD $0x65d82020
- WORD $0x85804581
- WORD $0x0431e3ec
+ WORD $0xa5ed41c1
+ WORD $0x04b0e3ed
WORD $0x65c20821
+ WORD $0xeb0d019f
WORD $0x65d82020
- WORD $0xb5fffeae
- WORD $0xeb0b00bf
- WORD $0xaa0b03ec
- WORD $0x54fffca0
- WORD $0xfc6c7821
- WORD $0xfc6c7842
- WORD $0x9100058c
+ WORD $0x54fffea1
WORD $0xeb0c00bf
+ WORD $0xaa0c03ed
+ WORD $0x54fffc60
+ WORD $0xfc6d7961
+ WORD $0xfc6d7902
+ WORD $0x910005ad
+ WORD $0xeb0d00bf
WORD $0x1e620821
WORD $0x1e612800
WORD $0x54ffff41
- WORD $0x17ffffdd
- WORD $0xeb09011f
- WORD $0x54000062
- WORD $0xaa1f03e9
- WORD $0x1400000d
+ WORD $0x17ffffdb
+ WORD $0x04a0e3ec
+ WORD $0xeb0c015f
+ WORD $0x54000243
+ WORD $0xd37df12d
WORD $0x6f00e400
- WORD $0xcb0903e9
- WORD $0xaa0003ea
- WORD $0x8a090109
- WORD $0xaa0903eb
- WORD $0xe5804140
- WORD $0x04b0e7eb
- WORD $0xe5804540
- WORD $0x0431e3ea
- WORD $0xb5ffff8b
- WORD $0xeb09011f
+ WORD $0xcb0c03ec
+ WORD $0x8a0c014c
+ WORD $0x25d8e3e0
+ WORD $0xaa1f03e8
+ WORD $0x8b0d000b
+ WORD $0x0430e3ed
+ WORD $0x8b0c0129
+ WORD $0x8b0d000d
+ WORD $0xe5e84160
+ WORD $0xe5e841a0
+ WORD $0x04b0e3e8
+ WORD $0xeb08019f
+ WORD $0x54ffff81
+ WORD $0xeb0c015f
WORD $0x540000c0
- WORD $0x8b090c0a
- WORD $0xcb090108
- WORD $0xf1000508
- WORD $0xf800855f
+ WORD $0x8b090c08
+ WORD $0xcb090049
+ WORD $0xf1000529
+ WORD $0xf800851f
WORD $0x54ffffc1
+ WORD $0xa9494ff4
+ WORD $0xfd400bee
+ WORD $0xa94857f6
+ WORD $0xa9475ff8
+ WORD $0xa94667fa
+ WORD $0xa9457bfd
+ WORD $0x6d4423e9
+ WORD $0x6d432beb
+ WORD $0x6d4233ed
+ WORD $0x910283ff
WORD $0xd65f03c0
RET
@@ -2461,50 +3471,216 @@ TEXT ·movingAverageFloat32SVE2(SB), NOSPLIT|NOFRAME, $0-56
MOVD a_len+32(FP), R3
MOVD width+48(FP), R4
WORD $0xf100049f
- WORD $0x5400056b
+ WORD $0x5400090b
WORD $0xeb040068
- WORD $0x5400052b
+ WORD $0x540008cb
+ WORD $0xd10283ff
WORD $0xeb08005f
+ WORD $0x6d013bef
WORD $0x9a88d448
- WORD $0xf100051f
- WORD $0x540004ab
+ WORD $0x6d0233ed
+ WORD $0xf100411f
+ WORD $0x6d032beb
+ WORD $0x6d0423e9
+ WORD $0xa9057bfd
+ WORD $0xa90667fa
+ WORD $0xa9075ff8
+ WORD $0xa90857f6
+ WORD $0xa9094ff4
+ WORD $0xf90007e1
+ WORD $0x5400070a
+ WORD $0xaa1f03e9
+ WORD $0xf94007ee
+ WORD $0xeb08013f
+ WORD $0x5400052a
+ WORD $0xd37ef52b
WORD $0x9e230080
WORD $0x0460e3ea
+ WORD $0xcb0a03ec
WORD $0x2598e3e0
- WORD $0xcb0a03eb
- WORD $0xaa1f03e9
- WORD $0x8a0b008b
- WORD $0x14000007
+ WORD $0xaa0b03ed
+ WORD $0x8b0b01cb
+ WORD $0x8a0c008c
+ WORD $0x0430e3ed
+ WORD $0x8b0d01cd
+ WORD $0x14000008
WORD $0x1e201821
- WORD $0x91001021
+ WORD $0x910011ad
+ WORD $0x9100116b
WORD $0xbc297801
WORD $0x91000529
WORD $0xeb08013f
- WORD $0x54000300
+ WORD $0x540002e0
WORD $0x2f00e401
WORD $0xeb0a009f
WORD $0x54000062
- WORD $0xaa1f03ec
- WORD $0x1400000d
- WORD $0xaa0103ec
- WORD $0xaa0b03ed
- WORD $0x85804182
- WORD $0x0470e7ed
+ WORD $0xaa1f03ee
+ WORD $0x1400000c
+ WORD $0xaa1f03ee
+ WORD $0xa54e4162
WORD $0x65982041
- WORD $0x85804582
- WORD $0x0431e3ec
+ WORD $0xa54e41a2
+ WORD $0x0470e3ee
+ WORD $0xeb0e019f
WORD $0x65982041
- WORD $0xb5ffff4d
- WORD $0xeb0b009f
- WORD $0xaa0b03ec
- WORD $0x54fffd40
- WORD $0xbc6c7822
- WORD $0x9100058c
+ WORD $0x54ffff41
WORD $0xeb0c009f
+ WORD $0xaa0c03ee
+ WORD $0x54fffd40
+ WORD $0xbc6e7962
+ WORD $0x910005ce
+ WORD $0xeb0e009f
WORD $0x1e222821
WORD $0x54ffff81
WORD $0x17ffffe4
+ WORD $0xa9494ff4
+ WORD $0xa94857f6
+ WORD $0xa9475ff8
+ WORD $0xa94667fa
+ WORD $0xa9457bfd
+ WORD $0x6d4423e9
+ WORD $0x6d432beb
+ WORD $0x6d4233ed
+ WORD $0x6d413bef
+ WORD $0x910283ff
WORD $0xd65f03c0
+ WORD $0x9e230080
+ WORD $0x04a0e3ea
+ WORD $0x2598e3e0
+ WORD $0xcb0a03e9
+ WORD $0xaa1f03f0
+ WORD $0x9100c02b
+ WORD $0x5280021a
+ WORD $0xd280002c
+ WORD $0xd280004d
+ WORD $0xd280006e
+ WORD $0xd280008f
+ WORD $0xd28000b1
+ WORD $0x4e040400
+ WORD $0xd28000c2
+ WORD $0x8a090083
+ WORD $0xd28000e5
+ WORD $0xd2800106
+ WORD $0xd2800127
+ WORD $0xd2800153
+ WORD $0xd2800174
+ WORD $0xd2800195
+ WORD $0xd28001b6
+ WORD $0xd28001d7
+ WORD $0xd28001f8
+ WORD $0xaa0103f9
+ WORD $0xaa0a03e1
+ WORD $0x14000010
+ WORD $0x6e0c04c5
+ WORD $0x6e20fc42
+ WORD $0x8b10080a
+ WORD $0x9100413a
+ WORD $0x91010339
+ WORD $0x9101016b
+ WORD $0xeb08035f
+ WORD $0xaa0903f0
+ WORD $0x6e180485
+ WORD $0x6e20fca4
+ WORD $0x6e20fc63
+ WORD $0xad000944
+ WORD $0x6e20fc21
+ WORD $0xad010543
+ WORD $0x54fff42c
+ WORD $0xeb01009f
+ WORD $0xaa1a03e9
+ WORD $0x54000122
+ WORD $0x2f00e404
+ WORD $0x6f00e402
+ WORD $0xaa1f03fd
+ WORD $0x2f00e406
+ WORD $0x2f00e405
+ WORD $0x6f00e403
+ WORD $0x6f00e401
+ WORD $0x14000043
+ WORD $0x2f00e407
+ WORD $0x2f00e410
+ WORD $0xaa1f03fa
+ WORD $0x2f00e413
+ WORD $0x2f00e401
+ WORD $0x2f00e411
+ WORD $0x2f00e415
+ WORD $0x2f00e417
+ WORD $0x2f00e403
+ WORD $0x2f00e414
+ WORD $0x2f00e416
+ WORD $0x2f00e418
+ WORD $0x2f00e402
+ WORD $0x2f00e412
+ WORD $0x2f00e404
+ WORD $0x2f00e406
+ WORD $0x2f00e405
+ WORD $0x8b1a0b3d
+ WORD $0xa55a4339
+ WORD $0x04b0e3fa
+ WORD $0xa55843be
+ WORD $0xa54c43ba
+ WORD $0xa54d43bb
+ WORD $0xa54e43bc
+ WORD $0xa54f43bd
+ WORD $0xa55143bf
+ WORD $0xa54243a8
+ WORD $0xa54543a9
+ WORD $0xa54643aa
+ WORD $0xa54743ab
+ WORD $0xa55743ac
+ WORD $0xeb1a007f
+ WORD $0x659823c7
+ WORD $0xa55643be
+ WORD $0xa55343ad
+ WORD $0xa55443ae
+ WORD $0xa55543af
+ WORD $0x65982190
+ WORD $0x659823d3
+ WORD $0x659821e1
+ WORD $0x659821d1
+ WORD $0x659821b5
+ WORD $0x65982177
+ WORD $0x65982143
+ WORD $0x65982134
+ WORD $0x65982116
+ WORD $0x659823f8
+ WORD $0x659823a2
+ WORD $0x65982392
+ WORD $0x65982364
+ WORD $0x65982346
+ WORD $0x65982325
+ WORD $0x54fffba1
+ WORD $0x6e0c0702
+ WORD $0x6e0c06e3
+ WORD $0xeb03009f
+ WORD $0x6e0c0661
+ WORD $0x6e0c0644
+ WORD $0xaa0303fd
+ WORD $0x6e1406c2
+ WORD $0x6e1406a3
+ WORD $0x6e140601
+ WORD $0x6e1c0682
+ WORD $0x6e1c0623
+ WORD $0x6e1c04e1
+ WORD $0x54fff4a0
+ WORD $0xd37ef7ba
+ WORD $0xcb1d009d
+ WORD $0xaa0b03fe
+ WORD $0x8b1a03ca
+ WORD $0xf10007bd
+ WORD $0x910013de
+ WORD $0x2d7a4151
+ WORD $0xfc5d8147
+ WORD $0xad7f4d52
+ WORD $0x3dc00154
+ WORD $0x0e24d4e4
+ WORD $0x1e252a25
+ WORD $0x1e262a06
+ WORD $0x4e21d681
+ WORD $0x4e22d642
+ WORD $0x4e23d663
+ WORD $0x54fffe61
+ WORD $0x17ffff93
RET
// func shiftDivFloat32SVE2(dst []float32, a []float32, shift float32, denom float32)
@@ -2632,50 +3808,227 @@ TEXT ·movingAverageFloat64SVE2(SB), NOSPLIT|NOFRAME, $0-56
MOVD a_len+32(FP), R3
MOVD width+48(FP), R4
WORD $0xf100049f
- WORD $0x5400056b
+ WORD $0x540008cb
WORD $0xeb040068
- WORD $0x5400052b
+ WORD $0x5400088b
+ WORD $0xd10243ff
WORD $0xeb08005f
+ WORD $0x6d0133ed
WORD $0x9a88d448
- WORD $0xf100051f
- WORD $0x540004ab
+ WORD $0x6d022beb
+ WORD $0xf100411f
+ WORD $0x6d0323e9
+ WORD $0xa9047bfd
+ WORD $0xa90567fa
+ WORD $0xa9065ff8
+ WORD $0xa90757f6
+ WORD $0xa9084ff4
+ WORD $0xf90007e1
+ WORD $0x540006ea
+ WORD $0xaa1f03e9
+ WORD $0xf94007ee
+ WORD $0xeb08013f
+ WORD $0x5400052a
+ WORD $0xd37df12b
WORD $0x9e630080
WORD $0x04a0e3ea
+ WORD $0xcb0a03ec
WORD $0x25d8e3e0
- WORD $0xcb0a03eb
- WORD $0xaa1f03e9
- WORD $0x8a0b008b
- WORD $0x14000007
+ WORD $0xaa0b03ed
+ WORD $0x8b0b01cb
+ WORD $0x8a0c008c
+ WORD $0x0430e3ed
+ WORD $0x8b0d01cd
+ WORD $0x14000008
WORD $0x1e601821
- WORD $0x91002021
+ WORD $0x910021ad
+ WORD $0x9100216b
WORD $0xfc297801
WORD $0x91000529
WORD $0xeb08013f
- WORD $0x54000300
+ WORD $0x540002e0
WORD $0x2f00e401
WORD $0xeb0a009f
WORD $0x54000062
- WORD $0xaa1f03ec
- WORD $0x1400000d
- WORD $0xaa0103ec
- WORD $0xaa0b03ed
- WORD $0x85804182
- WORD $0x04b0e7ed
+ WORD $0xaa1f03ee
+ WORD $0x1400000c
+ WORD $0xaa1f03ee
+ WORD $0xa5ee4162
WORD $0x65d82041
- WORD $0x85804582
- WORD $0x0431e3ec
+ WORD $0xa5ee41a2
+ WORD $0x04b0e3ee
+ WORD $0xeb0e019f
WORD $0x65d82041
- WORD $0xb5ffff4d
- WORD $0xeb0b009f
- WORD $0xaa0b03ec
- WORD $0x54fffd40
- WORD $0xfc6c7822
- WORD $0x9100058c
+ WORD $0x54ffff41
WORD $0xeb0c009f
+ WORD $0xaa0c03ee
+ WORD $0x54fffd40
+ WORD $0xfc6e7962
+ WORD $0x910005ce
+ WORD $0xeb0e009f
WORD $0x1e622821
WORD $0x54ffff81
WORD $0x17ffffe4
+ WORD $0xa9484ff4
+ WORD $0xa94757f6
+ WORD $0xa9465ff8
+ WORD $0xa94567fa
+ WORD $0xa9447bfd
+ WORD $0x6d4323e9
+ WORD $0x6d422beb
+ WORD $0x6d4133ed
+ WORD $0x910243ff
WORD $0xd65f03c0
+ WORD $0x9e630080
+ WORD $0x04e0e3ea
+ WORD $0x25d8e3e0
+ WORD $0xcb0a03e9
+ WORD $0xaa1f03f8
+ WORD $0x9101e02b
+ WORD $0x5280021a
+ WORD $0xd280002c
+ WORD $0x8a09008d
+ WORD $0xd280004e
+ WORD $0xd280006f
+ WORD $0xd2800090
+ WORD $0xd28000b1
+ WORD $0xd28000c2
+ WORD $0xd28000e3
+ WORD $0xd2800105
+ WORD $0xd2800126
+ WORD $0xd2800147
+ WORD $0xd2800173
+ WORD $0xd2800194
+ WORD $0xd28001b5
+ WORD $0xd28001d6
+ WORD $0xd28001f7
+ WORD $0xaa0103f9
+ WORD $0xaa0a03e1
+ WORD $0x14000020
+ WORD $0x1e601b18
+ WORD $0x8b180c0a
+ WORD $0x9100413a
+ WORD $0xeb08035f
+ WORD $0x91020339
+ WORD $0x9102016b
+ WORD $0xaa0903f8
+ WORD $0x1e601af7
+ WORD $0x1e601ad6
+ WORD $0x6d005d58
+ WORD $0x1e601ab5
+ WORD $0x1e601a94
+ WORD $0x6d015556
+ WORD $0x1e601a73
+ WORD $0x1e601a52
+ WORD $0x6d024d54
+ WORD $0x1e601a31
+ WORD $0x1e601a10
+ WORD $0x6d034552
+ WORD $0x1e6018e7
+ WORD $0x1e6018c6
+ WORD $0x6d041d50
+ WORD $0x1e6018a5
+ WORD $0x1e601884
+ WORD $0x6d051546
+ WORD $0x1e601863
+ WORD $0x1e601842
+ WORD $0x6d060d44
+ WORD $0x1e601821
+ WORD $0x6d070542
+ WORD $0x54fff26c
+ WORD $0x2f00e401
+ WORD $0x2f00e402
+ WORD $0xeb01009f
+ WORD $0x2f00e403
+ WORD $0x2f00e404
+ WORD $0xaa1a03e9
+ WORD $0x2f00e405
+ WORD $0x2f00e406
+ WORD $0x2f00e407
+ WORD $0x2f00e410
+ WORD $0x2f00e411
+ WORD $0x2f00e412
+ WORD $0x2f00e413
+ WORD $0x2f00e414
+ WORD $0x2f00e415
+ WORD $0x2f00e416
+ WORD $0x2f00e417
+ WORD $0x2f00e418
+ WORD $0x54000062
+ WORD $0xaa1f03fd
+ WORD $0x14000029
+ WORD $0xaa1f03fa
+ WORD $0x8b1a0f3d
+ WORD $0xa5fa4339
+ WORD $0x04f0e3fa
+ WORD $0xa5f743bb
+ WORD $0xa5f643bc
+ WORD $0xa5f543a8
+ WORD $0xa5ec43ba
+ WORD $0xa5ef43bd
+ WORD $0xa5f043be
+ WORD $0xa5f143bf
+ WORD $0xa5f443a9
+ WORD $0xa5e343aa
+ WORD $0x65d82361
+ WORD $0xa5ee43bb
+ WORD $0xa5e543ab
+ WORD $0x65d82382
+ WORD $0xa5e243bc
+ WORD $0xa5f343ac
+ WORD $0x65d82103
+ WORD $0xa5e643a8
+ WORD $0xa5e743ad
+ WORD $0x65d82124
+ WORD $0xeb1a01bf
+ WORD $0x65d82185
+ WORD $0x65d821a6
+ WORD $0x65d82107
+ WORD $0x65d82170
+ WORD $0x65d82151
+ WORD $0x65d82392
+ WORD $0x65d823f3
+ WORD $0x65d823d4
+ WORD $0x65d823b5
+ WORD $0x65d82376
+ WORD $0x65d82357
+ WORD $0x65d82338
+ WORD $0x54fffba1
+ WORD $0xeb0d009f
+ WORD $0xaa0d03fd
+ WORD $0x54fff4a0
+ WORD $0xd37df3ba
+ WORD $0xcb1d009d
+ WORD $0xaa0b03fe
+ WORD $0x8b1a03ca
+ WORD $0xf10007bd
+ WORD $0x910023de
+ WORD $0x6d78e959
+ WORD $0x6d79f15b
+ WORD $0x6d7af95d
+ WORD $0x1e782b38
+ WORD $0x1e772b57
+ WORD $0x1e762b76
+ WORD $0x1e752b95
+ WORD $0x6d7be959
+ WORD $0x1e742bb4
+ WORD $0x6d7cf15b
+ WORD $0x1e732bd3
+ WORD $0x6d7ff95d
+ WORD $0x1e722b32
+ WORD $0x1e712b51
+ WORD $0x1e702b70
+ WORD $0x1e672b87
+ WORD $0x6d7de959
+ WORD $0x1e622ba2
+ WORD $0x6d7ef15b
+ WORD $0x1e612bc1
+ WORD $0x1e662b26
+ WORD $0x1e652b45
+ WORD $0x1e642b64
+ WORD $0x1e632b83
+ WORD $0x54fffca1
+ WORD $0x17ffff85
RET
// func shiftDivFloat64SVE2(dst []float64, a []float64, shift float64, denom float64)
diff --git a/internal/loong64/numeric_lasx_loong64.go b/internal/loong64/numeric_lasx_loong64.go
index 5a5cff4..432b6ca 100644
--- a/internal/loong64/numeric_lasx_loong64.go
+++ b/internal/loong64/numeric_lasx_loong64.go
@@ -99,12 +99,18 @@ func tileUint64LASX(dst []uint64, pattern []uint64)
//go:noescape
func gatherUint64LASX(dst []uint64, src []uint64, idx []int32)
+//go:noescape
+func movingAverageFloat32LASX(dst []float32, a []float32, width int)
+
//go:noescape
func shiftDivFloat32LASX(dst []float32, a []float32, shift float32, denom float32)
//go:noescape
func layerNormFloat32LASX(dst []float32, a []float32, gamma []float32, beta []float32, shift float32, denom float32)
+//go:noescape
+func movingAverageFloat64LASX(dst []float64, a []float64, width int)
+
//go:noescape
func shiftDivFloat64LASX(dst []float64, a []float64, shift float64, denom float64)
diff --git a/internal/loong64/numeric_lasx_loong64.s b/internal/loong64/numeric_lasx_loong64.s
index 121db1d..4926710 100644
--- a/internal/loong64/numeric_lasx_loong64.s
+++ b/internal/loong64/numeric_lasx_loong64.s
@@ -213,22 +213,90 @@ TEXT ·polyEvalFloat32LASX(SB), NOSPLIT|NOFRAME, $0-72
WORD $0x0013a908
WORD $0x001328e7
WORD $0x001520e7
- WORD $0x40007920
- WORD $0x6401dc07
- WORD $0x002c992a
- WORD $0x2b3ff140
- WORD $0x03800806
- WORD $0x60007926
+ WORD $0x40001920
+ WORD $0x0380400a
+ WORD $0x002c9928
+ WORD $0x640020ea
WORD $0x00150006
- WORD $0x02fffd28
- WORD $0x02ffe149
+ WORD $0x50010c00
+ WORD $0x6402d807
+ WORD $0x03801005
+ WORD $0x6c017ce5
+ WORD $0x00150005
+ WORD $0x50020c00
+ WORD $0x2b3ff100
+ WORD $0x03800406
+ WORD $0x640088c9
+ WORD $0x0015000c
+ WORD $0x7707c000
+ WORD $0x02fffd2a
+ WORD $0x02ffe10b
+ WORD $0x0380400e
+ WORD $0x03400000
+ WORD $0x002c9586
+ WORD $0x0041098d
+ WORD $0x384834a1
+ WORD $0x2c8080c3
+ WORD $0x001501c6
+ WORD $0x0015016e
+ WORD $0x0015014f
+ WORD $0x77d40002
+ WORD $0x77d40004
+ WORD $0x03400000
+ WORD $0x03400000
WORD $0x03400000
+ WORD $0x322001c5
+ WORD $0x75388484
+ WORD $0x753090a4
+ WORD $0x75388c42
+ WORD $0x753088a2
+ WORD $0x02fffdef
+ WORD $0x02fff1ce
+ WORD $0x63ffe40f
+ WORD $0x002c918c
+ WORD $0x384c3484
+ WORD $0x02c040ce
+ WORD $0x2cc08182
+ WORD $0x001500cc
+ WORD $0x67ff9cee
+ WORD $0x50006800
+ WORD $0x00150006
+ WORD $0x02c0808a
+ WORD $0x03400000
+ WORD $0x03400000
+ WORD $0x03400000
+ WORD $0x2b7f8140
+ WORD $0x2b7f9140
+ WORD $0x2b7fa140
+ WORD $0x2b7fb140
+ WORD $0x2b7fc140
+ WORD $0x2b7fd140
+ WORD $0x2b7fe140
+ WORD $0x2b7ff140
+ WORD $0x2b400140
+ WORD $0x2b401140
+ WORD $0x2b402140
+ WORD $0x2b403140
+ WORD $0x2b404140
+ WORD $0x2b405140
+ WORD $0x2b406140
+ WORD $0x2b407140
+ WORD $0x02c080cb
+ WORD $0x02c040c6
+ WORD $0x02c1014a
+ WORD $0x67ffb4eb
+ WORD $0x6401d0c7
+ WORD $0x2b3ff100
+ WORD $0x0380080a
+ WORD $0x60005d2a
+ WORD $0x02fffd29
+ WORD $0x02ffe108
WORD $0x03400000
WORD $0x03400000
WORD $0x004108ca
WORD $0x383028a1
- WORD $0x0015012b
- WORD $0x0015010c
+ WORD $0x0015010b
+ WORD $0x0015012c
WORD $0x01149402
WORD $0x03400000
WORD $0x03400000
@@ -243,23 +311,19 @@ TEXT ·polyEvalFloat32LASX(SB), NOSPLIT|NOFRAME, $0-72
WORD $0x38382882
WORD $0x5fffc0c7
WORD $0x50016c00
- WORD $0x64016807
- WORD $0x03801005
- WORD $0x6c001ce5
- WORD $0x00150005
- WORD $0x5000ac00
- WORD $0x03801005
- WORD $0x6c001ce5
- WORD $0x00150005
- WORD $0x50012c00
+ WORD $0x001198e5
+ WORD $0x03801008
+ WORD $0x6c001ca8
+ WORD $0x001500c5
+ WORD $0x50013c00
WORD $0x03804005
WORD $0x6c001ce5
WORD $0x00150005
WORD $0x50005400
- WORD $0x03804005
- WORD $0x6c00a4e5
- WORD $0x00150005
- WORD $0x5000d400
+ WORD $0x03804008
+ WORD $0x6c00a4a8
+ WORD $0x00150008
+ WORD $0x5000d800
WORD $0x034030e6
WORD $0x00fe10e5
WORD $0x004110a5
@@ -274,7 +338,7 @@ TEXT ·polyEvalFloat32LASX(SB), NOSPLIT|NOFRAME, $0-72
WORD $0x02ffc129
WORD $0x02c10108
WORD $0x47fff13f
- WORD $0x5800ece5
+ WORD $0x5800fce5
WORD $0x40003cc0
WORD $0x001500a8
WORD $0x00fe08e5
@@ -289,7 +353,7 @@ TEXT ·polyEvalFloat32LASX(SB), NOSPLIT|NOFRAME, $0-72
WORD $0x02c010c6
WORD $0x02c04108
WORD $0x47fff4df
- WORD $0x5800b0e5
+ WORD $0x5800c0e5
WORD $0x001194e6
WORD $0x002c90a4
WORD $0x03400000
@@ -297,35 +361,39 @@ TEXT ·polyEvalFloat32LASX(SB), NOSPLIT|NOFRAME, $0-72
WORD $0x02fffcc6
WORD $0x02c01084
WORD $0x47fff4df
- WORD $0x50009000
- WORD $0x034030e6
- WORD $0x00fe10e5
- WORD $0x004110a5
+ WORD $0x5000a000
+ WORD $0x03403ce9
+ WORD $0x0011a4a8
WORD $0x7707c001
- WORD $0x02c08088
- WORD $0x001500a9
- WORD $0x03400000
- WORD $0x2cff8101
- WORD $0x2cc00101
- WORD $0x02ffc129
- WORD $0x02c10108
- WORD $0x47fff13f
- WORD $0x58005ce5
- WORD $0x40003cc0
- WORD $0x001500a8
- WORD $0x00fe08e5
- WORD $0x004108a5
+ WORD $0x002c90ca
+ WORD $0x02c0814a
+ WORD $0x0010a4cb
+ WORD $0x00119d6b
+ WORD $0x2cff8141
+ WORD $0x2cc00141
+ WORD $0x02c0416b
+ WORD $0x02c1014a
+ WORD $0x47fff17f
+ WORD $0x40006d20
+ WORD $0x0380100a
+ WORD $0x6800692a
+ WORD $0x03400ce9
+ WORD $0x0011a4a5
+ WORD $0x001094c5
WORD $0x72f7e001
- WORD $0x00119506
- WORD $0x002c9108
- WORD $0x03400000
+ WORD $0x004108ca
+ WORD $0x002ca90a
+ WORD $0x0010a88a
+ WORD $0x0010a0c6
+ WORD $0x0010a4c6
+ WORD $0x00119cc6
WORD $0x03400000
WORD $0x03400000
- WORD $0x2c400101
+ WORD $0x2c400141
WORD $0x02c010c6
- WORD $0x02c04108
+ WORD $0x02c0414a
WORD $0x47fff4df
- WORD $0x580020e5
+ WORD $0x40002120
WORD $0x001194e6
WORD $0x002c90a4
WORD $0x03400000
@@ -334,6 +402,8 @@ TEXT ·polyEvalFloat32LASX(SB), NOSPLIT|NOFRAME, $0-72
WORD $0x02c01084
WORD $0x47fff4df
WORD $0x4c000020
+ WORD $0x0010a0c5
+ WORD $0x53ffdfff
RET
// func convolveFloat32LASX(dst []float32, sig []float32, ker []float32)
@@ -344,22 +414,70 @@ TEXT ·convolveFloat32LASX(SB), NOSPLIT|NOFRAME, $0-72
MOVV dst_len+8(FP), R7
MOVV sig_len+32(FP), R8
MOVV ker_len+56(FP), R9
- WORD $0x40013120
- WORD $0x60012d09
+ WORD $0x40021120
+ WORD $0x60020d09
WORD $0x0011a508
WORD $0x02c0050a
WORD $0x00121d08
WORD $0x0013214a
WORD $0x0013a0e7
WORD $0x00151d47
- WORD $0x64011007
- WORD $0x64006409
+ WORD $0x03804008
+ WORD $0x64000ce8
+ WORD $0x00150008
+ WORD $0x5000bc00
+ WORD $0x64009009
+ WORD $0x0015000c
+ WORD $0x02c080aa
+ WORD $0x002c9928
+ WORD $0x02fff10b
+ WORD $0x0380400d
+ WORD $0x77e00000
+ WORD $0x03400000
+ WORD $0x001501a8
+ WORD $0x0015016d
+ WORD $0x0015014e
+ WORD $0x0015012f
+ WORD $0x77d40001
+ WORD $0x77d40002
+ WORD $0x03400000
+ WORD $0x03400000
+ WORD $0x2cbf81c3
+ WORD $0x322001a4
+ WORD $0x2c8001c5
+ WORD $0x75388c83
+ WORD $0x75308c21
+ WORD $0x75389483
+ WORD $0x75308c42
+ WORD $0x02fffdef
+ WORD $0x02c011ce
+ WORD $0x02fff1ad
+ WORD $0x47ffd9ff
+ WORD $0x002c918d
+ WORD $0x0041098c
+ WORD $0x384c3081
+ WORD $0x2cc081a2
+ WORD $0x02c0410d
+ WORD $0x02c1014a
+ WORD $0x0015010c
+ WORD $0x67ff98ed
+ WORD $0x50002c00
WORD $0x00150008
+ WORD $0x02c0808a
+ WORD $0x77e00000
+ WORD $0x03400000
+ WORD $0x2cff8140
+ WORD $0x2cc00140
+ WORD $0x02c0810b
+ WORD $0x02c04108
+ WORD $0x02c1014a
+ WORD $0x67ffeceb
+ WORD $0x64012907
+ WORD $0x64005c09
WORD $0x002c9926
WORD $0x02fff0c6
+ WORD $0x002c9505
WORD $0x0114a400
- WORD $0x03400000
- WORD $0x03400000
WORD $0x001500aa
WORD $0x001500cb
WORD $0x0015012c
@@ -377,42 +495,50 @@ TEXT ·convolveFloat32LASX(SB), NOSPLIT|NOFRAME, $0-72
WORD $0x02c00508
WORD $0x02c010a5
WORD $0x5fffc107
- WORD $0x5000ac00
- WORD $0x03801005
- WORD $0x6c000ce5
- WORD $0x00150005
- WORD $0x50008000
- WORD $0x03804005
- WORD $0x6c000ce5
- WORD $0x00150005
- WORD $0x50003800
- WORD $0x034030e6
- WORD $0x00fe10e5
- WORD $0x004110a5
- WORD $0x02c08088
+ WORD $0x5000cc00
+ WORD $0x0011a0e5
+ WORD $0x03801006
+ WORD $0x6c000ca6
+ WORD $0x00150105
+ WORD $0x50009c00
+ WORD $0x03804006
+ WORD $0x6c000ca6
+ WORD $0x00150006
+ WORD $0x50004800
+ WORD $0x03403ce9
+ WORD $0x0011a4a6
+ WORD $0x002c910a
+ WORD $0x02c0814a
+ WORD $0x0010a50b
+ WORD $0x00119d6b
WORD $0x77e00000
- WORD $0x001500a9
- WORD $0x2cff8100
- WORD $0x2cc00100
- WORD $0x02ffc129
- WORD $0x02c10108
- WORD $0x47fff13f
- WORD $0x58005ce5
- WORD $0x40003cc0
- WORD $0x001500a8
- WORD $0x00fe08e5
- WORD $0x004108a5
- WORD $0x00119506
- WORD $0x002c9108
- WORD $0x73e00000
WORD $0x03400000
WORD $0x03400000
+ WORD $0x2cff8140
+ WORD $0x2cc00140
+ WORD $0x02c0416b
+ WORD $0x02c1014a
+ WORD $0x47fff17f
+ WORD $0x40006d20
+ WORD $0x0380100a
+ WORD $0x6800692a
+ WORD $0x03400ce9
+ WORD $0x0011a4a5
+ WORD $0x00109505
+ WORD $0x0041090a
+ WORD $0x002ca8ca
+ WORD $0x0010a88a
+ WORD $0x00109906
+ WORD $0x0010a4c6
+ WORD $0x00119cc6
+ WORD $0x73e00000
+ WORD $0x03400000
WORD $0x03400000
- WORD $0x2c400100
+ WORD $0x2c400140
WORD $0x02c010c6
- WORD $0x02c04108
+ WORD $0x02c0414a
WORD $0x47fff4df
- WORD $0x580020e5
+ WORD $0x40002120
WORD $0x001194e6
WORD $0x002c90a4
WORD $0x03400000
@@ -421,6 +547,8 @@ TEXT ·convolveFloat32LASX(SB), NOSPLIT|NOFRAME, $0-72
WORD $0x02c01084
WORD $0x47fff4df
WORD $0x4c000020
+ WORD $0x00109905
+ WORD $0x53ffdfff
RET
// func correlateFloat32LASX(dst []float32, sig []float32, ker []float32)
@@ -431,18 +559,70 @@ TEXT ·correlateFloat32LASX(SB), NOSPLIT|NOFRAME, $0-72
MOVV dst_len+8(FP), R7
MOVV sig_len+32(FP), R8
MOVV ker_len+56(FP), R9
- WORD $0x40012120
- WORD $0x60011d09
+ WORD $0x40021120
+ WORD $0x60020d09
WORD $0x0011a508
WORD $0x02c0050a
WORD $0x00121d08
WORD $0x0013214a
WORD $0x0013a0e7
WORD $0x00151d47
- WORD $0x64010007
- WORD $0x64005409
+ WORD $0x03804008
+ WORD $0x64000ce8
+ WORD $0x00150008
+ WORD $0x5000bc00
+ WORD $0x64009009
+ WORD $0x0015000b
+ WORD $0x02c080aa
+ WORD $0x0380400c
+ WORD $0x77e00000
+ WORD $0x03400000
+ WORD $0x03400000
+ WORD $0x03400000
+ WORD $0x00150188
+ WORD $0x001500cc
+ WORD $0x0015014d
+ WORD $0x0015012e
+ WORD $0x77d40001
+ WORD $0x77d40002
+ WORD $0x03400000
+ WORD $0x03400000
+ WORD $0x2cbf81a3
+ WORD $0x32200184
+ WORD $0x2c8001a5
+ WORD $0x75388c83
+ WORD $0x75308c21
+ WORD $0x75389483
+ WORD $0x75308c42
+ WORD $0x02fffdce
+ WORD $0x02c011ad
+ WORD $0x02c0118c
+ WORD $0x47ffd9df
+ WORD $0x002c916c
+ WORD $0x0041096b
+ WORD $0x384c2c81
+ WORD $0x2cc08182
+ WORD $0x02c0410c
+ WORD $0x02c1014a
+ WORD $0x0015010b
+ WORD $0x67ff98ec
+ WORD $0x50002c00
WORD $0x00150008
+ WORD $0x02c0808a
+ WORD $0x77e00000
+ WORD $0x03400000
+ WORD $0x2cff8140
+ WORD $0x2cc00140
+ WORD $0x02c0810b
+ WORD $0x02c04108
+ WORD $0x02c1014a
+ WORD $0x67ffeceb
+ WORD $0x64012907
+ WORD $0x64005c09
+ WORD $0x002c9505
WORD $0x0114a400
+ WORD $0x03400000
+ WORD $0x03400000
WORD $0x001500aa
WORD $0x001500cb
WORD $0x0015012c
@@ -460,42 +640,50 @@ TEXT ·correlateFloat32LASX(SB), NOSPLIT|NOFRAME, $0-72
WORD $0x02c00508
WORD $0x02c010a5
WORD $0x5fffc107
- WORD $0x5000ac00
- WORD $0x03801005
- WORD $0x6c000ce5
- WORD $0x00150005
- WORD $0x50008000
- WORD $0x03804005
- WORD $0x6c000ce5
- WORD $0x00150005
- WORD $0x50003800
- WORD $0x034030e6
- WORD $0x00fe10e5
- WORD $0x004110a5
- WORD $0x02c08088
+ WORD $0x5000cc00
+ WORD $0x0011a0e5
+ WORD $0x03801006
+ WORD $0x6c000ca6
+ WORD $0x00150105
+ WORD $0x50009c00
+ WORD $0x03804006
+ WORD $0x6c000ca6
+ WORD $0x00150006
+ WORD $0x50004800
+ WORD $0x03403ce9
+ WORD $0x0011a4a6
+ WORD $0x002c910a
+ WORD $0x02c0814a
+ WORD $0x0010a50b
+ WORD $0x00119d6b
WORD $0x77e00000
- WORD $0x001500a9
- WORD $0x2cff8100
- WORD $0x2cc00100
- WORD $0x02ffc129
- WORD $0x02c10108
- WORD $0x47fff13f
- WORD $0x58005ce5
- WORD $0x40003cc0
- WORD $0x001500a8
- WORD $0x00fe08e5
- WORD $0x004108a5
- WORD $0x00119506
- WORD $0x002c9108
- WORD $0x73e00000
WORD $0x03400000
WORD $0x03400000
+ WORD $0x2cff8140
+ WORD $0x2cc00140
+ WORD $0x02c0416b
+ WORD $0x02c1014a
+ WORD $0x47fff17f
+ WORD $0x40006d20
+ WORD $0x0380100a
+ WORD $0x6800692a
+ WORD $0x03400ce9
+ WORD $0x0011a4a5
+ WORD $0x00109505
+ WORD $0x0041090a
+ WORD $0x002ca8ca
+ WORD $0x0010a88a
+ WORD $0x00109906
+ WORD $0x0010a4c6
+ WORD $0x00119cc6
+ WORD $0x73e00000
+ WORD $0x03400000
WORD $0x03400000
- WORD $0x2c400100
+ WORD $0x2c400140
WORD $0x02c010c6
- WORD $0x02c04108
+ WORD $0x02c0414a
WORD $0x47fff4df
- WORD $0x580020e5
+ WORD $0x40002120
WORD $0x001194e6
WORD $0x002c90a4
WORD $0x03400000
@@ -504,6 +692,8 @@ TEXT ·correlateFloat32LASX(SB), NOSPLIT|NOFRAME, $0-72
WORD $0x02c01084
WORD $0x47fff4df
WORD $0x4c000020
+ WORD $0x00109905
+ WORD $0x53ffdfff
RET
// func normFloat64LASX(a []float64) float64
@@ -709,22 +899,98 @@ TEXT ·polyEvalFloat64LASX(SB), NOSPLIT|NOFRAME, $0-72
WORD $0x0013a908
WORD $0x001328e7
WORD $0x001520e7
- WORD $0x40007920
- WORD $0x64012c07
- WORD $0x002d192a
- WORD $0x2bbfe140
- WORD $0x03800806
- WORD $0x60007926
- WORD $0x00150006
- WORD $0x02fffd28
- WORD $0x02ffc149
+ WORD $0x40001920
+ WORD $0x03804008
+ WORD $0x002d1926
+ WORD $0x640020e8
+ WORD $0x00150008
+ WORD $0x50012c00
+ WORD $0x64024807
+ WORD $0x03802005
+ WORD $0x6c019ce5
+ WORD $0x00150005
+ WORD $0x5001c000
+ WORD $0x2bbfe0c0
+ WORD $0x03800408
+ WORD $0x6400b109
+ WORD $0x0015000c
+ WORD $0x7707e000
+ WORD $0x02fffd2a
+ WORD $0x02ffc0cb
+ WORD $0x0380400e
+ WORD $0x03400000
+ WORD $0x002d1588
+ WORD $0x00410d8d
+ WORD $0x384834a1
+ WORD $0x2c808103
+ WORD $0x2c810104
+ WORD $0x2c818105
+ WORD $0x001501c8
+ WORD $0x0015016e
+ WORD $0x0015014f
+ WORD $0x77d40002
+ WORD $0x77d40006
+ WORD $0x77d40007
+ WORD $0x77d40008
WORD $0x03400000
WORD $0x03400000
WORD $0x03400000
- WORD $0x00410cca
- WORD $0x383428a1
- WORD $0x0015012b
+ WORD $0x321001c9
+ WORD $0x75390508
+ WORD $0x75312128
+ WORD $0x75390ce7
+ WORD $0x75311d27
+ WORD $0x753910c6
+ WORD $0x75311926
+ WORD $0x75391442
+ WORD $0x75310922
+ WORD $0x02fffdef
+ WORD $0x02ffe1ce
+ WORD $0x63ffd40f
+ WORD $0x002d118c
+ WORD $0x384c3488
+ WORD $0x2cc08187
+ WORD $0x2cc10186
+ WORD $0x02c0410e
+ WORD $0x2cc18182
WORD $0x0015010c
+ WORD $0x67ff74ee
+ WORD $0x50006000
+ WORD $0x00150008
+ WORD $0x02c1008a
+ WORD $0x03400000
+ WORD $0x2bff0140
+ WORD $0x2bff2140
+ WORD $0x2bff4140
+ WORD $0x2bff6140
+ WORD $0x2bff8140
+ WORD $0x2bffa140
+ WORD $0x2bffc140
+ WORD $0x2bffe140
+ WORD $0x2bc00140
+ WORD $0x2bc02140
+ WORD $0x2bc04140
+ WORD $0x2bc06140
+ WORD $0x2bc08140
+ WORD $0x2bc0a140
+ WORD $0x2bc0c140
+ WORD $0x2bc0e140
+ WORD $0x02c0810b
+ WORD $0x02c04108
+ WORD $0x02c2014a
+ WORD $0x67ffb4eb
+ WORD $0x64012107
+ WORD $0x2bbfe0c0
+ WORD $0x0380080a
+ WORD $0x60005d2a
+ WORD $0x02fffd29
+ WORD $0x02ffc0c6
+ WORD $0x03400000
+ WORD $0x03400000
+ WORD $0x00410d0a
+ WORD $0x383428a1
+ WORD $0x001500cb
+ WORD $0x0015012c
WORD $0x01149802
WORD $0x03400000
WORD $0x03400000
@@ -735,19 +1001,15 @@ TEXT ·polyEvalFloat64LASX(SB), NOSPLIT|NOFRAME, $0-72
WORD $0x02fffd8c
WORD $0x02ffe16b
WORD $0x63ffec0c
- WORD $0x02c004c6
+ WORD $0x02c00508
WORD $0x383c2882
- WORD $0x5fffc0c7
+ WORD $0x5fffc107
WORD $0x5000bc00
- WORD $0x6400b807
- WORD $0x03802005
- WORD $0x6c001ce5
- WORD $0x00150005
- WORD $0x50004000
- WORD $0x03802005
- WORD $0x6c0054e5
- WORD $0x00150005
- WORD $0x50008000
+ WORD $0x0011a0e5
+ WORD $0x03802006
+ WORD $0x6c0054a6
+ WORD $0x00150105
+ WORD $0x50009000
WORD $0x00fe0ce5
WORD $0x00410ca5
WORD $0x02c08086
@@ -758,27 +1020,31 @@ TEXT ·polyEvalFloat64LASX(SB), NOSPLIT|NOFRAME, $0-72
WORD $0x02ffe108
WORD $0x02c100c6
WORD $0x47fff11f
- WORD $0x58006ce5
+ WORD $0x58007ce5
WORD $0x001194e6
WORD $0x002d10a4
WORD $0x29c00080
WORD $0x02fffcc6
WORD $0x02c02084
WORD $0x47fff4df
- WORD $0x50005000
- WORD $0x00fe0ce5
- WORD $0x00410ca5
+ WORD $0x50006000
+ WORD $0x03401ce6
+ WORD $0x001198a5
+ WORD $0x00109505
WORD $0x7707e001
- WORD $0x02c08086
- WORD $0x001500a8
+ WORD $0x002d1109
+ WORD $0x02c08129
+ WORD $0x00109908
+ WORD $0x00119d08
WORD $0x03400000
WORD $0x03400000
- WORD $0x2cff80c1
- WORD $0x2cc000c1
- WORD $0x02ffe108
- WORD $0x02c100c6
+ WORD $0x03400000
+ WORD $0x2cff8121
+ WORD $0x2cc00121
+ WORD $0x02c02108
+ WORD $0x02c10129
WORD $0x47fff11f
- WORD $0x58001ce5
+ WORD $0x40001cc0
WORD $0x001194e6
WORD $0x002d10a4
WORD $0x2bc00080
@@ -796,19 +1062,79 @@ TEXT ·convolveFloat64LASX(SB), NOSPLIT|NOFRAME, $0-72
MOVV dst_len+8(FP), R7
MOVV sig_len+32(FP), R8
MOVV ker_len+56(FP), R9
- WORD $0x4000e120
- WORD $0x6000dd09
+ WORD $0x4001e120
+ WORD $0x6001dd09
WORD $0x0011a508
WORD $0x02c0050a
WORD $0x00121d08
WORD $0x0013214a
WORD $0x0013a0e7
WORD $0x00151d47
- WORD $0x6400c007
- WORD $0x64006409
+ WORD $0x03804008
+ WORD $0x64000ce8
+ WORD $0x00150008
+ WORD $0x5000e400
+ WORD $0x6400b009
+ WORD $0x0015000c
+ WORD $0x02c100aa
+ WORD $0x002d1928
+ WORD $0x02ffe10b
+ WORD $0x0380400d
+ WORD $0x77e00000
+ WORD $0x03400000
+ WORD $0x001501a8
+ WORD $0x0015016d
+ WORD $0x0015014e
+ WORD $0x0015012f
+ WORD $0x77d40001
+ WORD $0x77d40002
+ WORD $0x77d40003
+ WORD $0x77d40004
+ WORD $0x2cbf01c5
+ WORD $0x321001a6
+ WORD $0x2cbf81c7
+ WORD $0x753914c5
+ WORD $0x75311421
+ WORD $0x2c8001c5
+ WORD $0x75391cc7
+ WORD $0x2c8081c8
+ WORD $0x75311c42
+ WORD $0x753914c5
+ WORD $0x75311463
+ WORD $0x753920c5
+ WORD $0x75311484
+ WORD $0x02fffdef
+ WORD $0x02c021ce
+ WORD $0x02ffe1ad
+ WORD $0x47ffc1ff
+ WORD $0x002d118d
+ WORD $0x00410d8c
+ WORD $0x384c3081
+ WORD $0x2cc081a2
+ WORD $0x2cc101a3
+ WORD $0x2cc181a4
+ WORD $0x02c0410d
+ WORD $0x02c2014a
+ WORD $0x0015010c
+ WORD $0x67ff78ed
+ WORD $0x50003400
WORD $0x00150008
+ WORD $0x02c1008a
+ WORD $0x77e00000
+ WORD $0x03400000
+ WORD $0x2cff0140
+ WORD $0x2cff8140
+ WORD $0x2cc00140
+ WORD $0x2cc08140
+ WORD $0x02c0810b
+ WORD $0x02c04108
+ WORD $0x02c2014a
+ WORD $0x67ffe4eb
+ WORD $0x6400d107
+ WORD $0x64006409
WORD $0x002d1926
WORD $0x02ffe0c6
+ WORD $0x002d1505
WORD $0x0114a800
WORD $0x03400000
WORD $0x03400000
@@ -829,23 +1155,27 @@ TEXT ·convolveFloat64LASX(SB), NOSPLIT|NOFRAME, $0-72
WORD $0x02c00508
WORD $0x02c020a5
WORD $0x5fffc107
- WORD $0x50005c00
- WORD $0x03802005
- WORD $0x6c000ce5
- WORD $0x00150005
- WORD $0x50003400
- WORD $0x00fe0ce5
- WORD $0x00410ca5
- WORD $0x02c08086
+ WORD $0x50006c00
+ WORD $0x0011a0e5
+ WORD $0x03802006
+ WORD $0x6c000ca6
+ WORD $0x00150105
+ WORD $0x50004000
+ WORD $0x03401ce6
+ WORD $0x001198a5
+ WORD $0x00109505
+ WORD $0x002d1109
+ WORD $0x02c08129
+ WORD $0x00109908
+ WORD $0x00119d08
WORD $0x77e00000
- WORD $0x001500a8
WORD $0x03400000
- WORD $0x2cff80c0
- WORD $0x2cc000c0
- WORD $0x02ffe108
- WORD $0x02c100c6
+ WORD $0x2cff8120
+ WORD $0x2cc00120
+ WORD $0x02c02108
+ WORD $0x02c10129
WORD $0x47fff11f
- WORD $0x58001ce5
+ WORD $0x40001cc0
WORD $0x001194e6
WORD $0x002d10a4
WORD $0x29c00080
@@ -863,17 +1193,77 @@ TEXT ·correlateFloat64LASX(SB), NOSPLIT|NOFRAME, $0-72
MOVV dst_len+8(FP), R7
MOVV sig_len+32(FP), R8
MOVV ker_len+56(FP), R9
- WORD $0x4000d120
- WORD $0x6000cd09
+ WORD $0x4001d120
+ WORD $0x6001cd09
WORD $0x0011a508
WORD $0x02c0050a
WORD $0x00121d08
WORD $0x0013214a
WORD $0x0013a0e7
WORD $0x00151d47
- WORD $0x6400b007
- WORD $0x64005409
+ WORD $0x03804008
+ WORD $0x64000ce8
WORD $0x00150008
+ WORD $0x5000e400
+ WORD $0x6400b009
+ WORD $0x0015000b
+ WORD $0x02c100aa
+ WORD $0x0380400c
+ WORD $0x77e00000
+ WORD $0x03400000
+ WORD $0x03400000
+ WORD $0x03400000
+ WORD $0x00150188
+ WORD $0x0015014c
+ WORD $0x001500cd
+ WORD $0x0015012e
+ WORD $0x77d40001
+ WORD $0x77d40002
+ WORD $0x77d40003
+ WORD $0x77d40004
+ WORD $0x2cbf0185
+ WORD $0x321001a6
+ WORD $0x2cbf8187
+ WORD $0x753914c5
+ WORD $0x75311421
+ WORD $0x2c800185
+ WORD $0x75391cc7
+ WORD $0x2c808188
+ WORD $0x75311c42
+ WORD $0x753914c5
+ WORD $0x75311463
+ WORD $0x753920c5
+ WORD $0x75311484
+ WORD $0x02fffdce
+ WORD $0x02c021ad
+ WORD $0x02c0218c
+ WORD $0x47ffc1df
+ WORD $0x002d116c
+ WORD $0x00410d6b
+ WORD $0x384c2c81
+ WORD $0x2cc08182
+ WORD $0x2cc10183
+ WORD $0x2cc18184
+ WORD $0x02c0410c
+ WORD $0x02c2014a
+ WORD $0x0015010b
+ WORD $0x67ff78ec
+ WORD $0x50003400
+ WORD $0x00150008
+ WORD $0x02c1008a
+ WORD $0x77e00000
+ WORD $0x03400000
+ WORD $0x2cff0140
+ WORD $0x2cff8140
+ WORD $0x2cc00140
+ WORD $0x2cc08140
+ WORD $0x02c0810b
+ WORD $0x02c04108
+ WORD $0x02c2014a
+ WORD $0x67ffe4eb
+ WORD $0x6400c107
+ WORD $0x64005409
+ WORD $0x002d1505
WORD $0x0114a800
WORD $0x001500aa
WORD $0x001500cb
@@ -892,23 +1282,27 @@ TEXT ·correlateFloat64LASX(SB), NOSPLIT|NOFRAME, $0-72
WORD $0x02c00508
WORD $0x02c020a5
WORD $0x5fffc107
- WORD $0x50005c00
- WORD $0x03802005
- WORD $0x6c000ce5
- WORD $0x00150005
- WORD $0x50003400
- WORD $0x00fe0ce5
- WORD $0x00410ca5
- WORD $0x02c08086
+ WORD $0x50006c00
+ WORD $0x0011a0e5
+ WORD $0x03802006
+ WORD $0x6c000ca6
+ WORD $0x00150105
+ WORD $0x50004000
+ WORD $0x03401ce6
+ WORD $0x001198a5
+ WORD $0x00109505
+ WORD $0x002d1109
+ WORD $0x02c08129
+ WORD $0x00109908
+ WORD $0x00119d08
WORD $0x77e00000
- WORD $0x001500a8
WORD $0x03400000
- WORD $0x2cff80c0
- WORD $0x2cc000c0
- WORD $0x02ffe108
- WORD $0x02c100c6
+ WORD $0x2cff8120
+ WORD $0x2cc00120
+ WORD $0x02c02108
+ WORD $0x02c10129
WORD $0x47fff11f
- WORD $0x58001ce5
+ WORD $0x40001cc0
WORD $0x001194e6
WORD $0x002d10a4
WORD $0x29c00080
@@ -3098,6 +3492,83 @@ TEXT ·gatherUint64LASX(SB), NOSPLIT|NOFRAME, $0-72
WORD $0x4c000020
RET
+// func movingAverageFloat32LASX(dst []float32, a []float32, width int)
+TEXT ·movingAverageFloat32LASX(SB), NOSPLIT|NOFRAME, $0-56
+ MOVV dst_base+0(FP), R4
+ MOVV a_base+24(FP), R5
+ MOVV dst_len+8(FP), R6
+ MOVV a_len+32(FP), R7
+ MOVV width+48(FP), R8
+ WORD $0x64010c08
+ WORD $0x600108e8
+ WORD $0x0011a0e7
+ WORD $0x02c004e9
+ WORD $0x001218e7
+ WORD $0x00131d29
+ WORD $0x00139cc6
+ WORD $0x00151926
+ WORD $0x0380400b
+ WORD $0x0114a900
+ WORD $0x64000ccb
+ WORD $0x00150007
+ WORD $0x50008400
+ WORD $0x0015000a
+ WORD $0x011d1801
+ WORD $0x7707c021
+ WORD $0x02c080a9
+ WORD $0x77e00002
+ WORD $0x03400000
+ WORD $0x03400000
+ WORD $0x00150167
+ WORD $0x0015010b
+ WORD $0x0015012c
+ WORD $0x77d40044
+ WORD $0x77d40043
+ WORD $0x03400000
+ WORD $0x03400000
+ WORD $0x03400000
+ WORD $0x2cbf8185
+ WORD $0x2c800186
+ WORD $0x753090a4
+ WORD $0x75308cc3
+ WORD $0x02fffd6b
+ WORD $0x02c0118c
+ WORD $0x47ffe97f
+ WORD $0x002c914b
+ WORD $0x0041094a
+ WORD $0x753a8484
+ WORD $0x384c2884
+ WORD $0x753a8463
+ WORD $0x2cc08163
+ WORD $0x02c040eb
+ WORD $0x02c10129
+ WORD $0x001500ea
+ WORD $0x67ffa0cb
+ WORD $0x640058e6
+ WORD $0x011d1800
+ WORD $0x002c94e5
+ WORD $0x0114a401
+ WORD $0x03400000
+ WORD $0x03400000
+ WORD $0x03400000
+ WORD $0x001500a9
+ WORD $0x0015010a
+ WORD $0x01149422
+ WORD $0x03400000
+ WORD $0x2b000123
+ WORD $0x01008c42
+ WORD $0x02fffd4a
+ WORD $0x02c01129
+ WORD $0x47fff15f
+ WORD $0x01068042
+ WORD $0x004108e9
+ WORD $0x38382482
+ WORD $0x02c004e7
+ WORD $0x02c010a5
+ WORD $0x5fffc8e6
+ WORD $0x4c000020
+ RET
+
// func shiftDivFloat32LASX(dst []float32, a []float32, shift float32, denom float32)
TEXT ·shiftDivFloat32LASX(SB), NOSPLIT|NOFRAME, $0-56
MOVV dst_base+0(FP), R4
@@ -3274,6 +3745,91 @@ TEXT ·layerNormFloat32LASX(SB), NOSPLIT|NOFRAME, $0-104
WORD $0x4c000020
RET
+// func movingAverageFloat64LASX(dst []float64, a []float64, width int)
+TEXT ·movingAverageFloat64LASX(SB), NOSPLIT|NOFRAME, $0-56
+ MOVV dst_base+0(FP), R4
+ MOVV a_base+24(FP), R5
+ MOVV dst_len+8(FP), R6
+ MOVV a_len+32(FP), R7
+ MOVV width+48(FP), R8
+ WORD $0x64012c08
+ WORD $0x600128e8
+ WORD $0x0011a0e7
+ WORD $0x02c004e9
+ WORD $0x001218e7
+ WORD $0x00131d29
+ WORD $0x00139cc6
+ WORD $0x00151926
+ WORD $0x0380400b
+ WORD $0x0114a900
+ WORD $0x64000ccb
+ WORD $0x00150007
+ WORD $0x5000a400
+ WORD $0x0015000a
+ WORD $0x011d2801
+ WORD $0x7707e021
+ WORD $0x02c100a9
+ WORD $0x77e00002
+ WORD $0x03400000
+ WORD $0x03400000
+ WORD $0x00150167
+ WORD $0x0015010b
+ WORD $0x0015012c
+ WORD $0x77d40046
+ WORD $0x77d40045
+ WORD $0x77d40044
+ WORD $0x77d40043
+ WORD $0x03400000
+ WORD $0x2cbf0187
+ WORD $0x2cbf8188
+ WORD $0x2c800189
+ WORD $0x2c80818a
+ WORD $0x753118e6
+ WORD $0x75311505
+ WORD $0x75311124
+ WORD $0x75310d43
+ WORD $0x02fffd6b
+ WORD $0x02c0218c
+ WORD $0x47ffd97f
+ WORD $0x002d114b
+ WORD $0x00410d4a
+ WORD $0x753b04c6
+ WORD $0x384c2886
+ WORD $0x753b04a5
+ WORD $0x2cc08165
+ WORD $0x753b0484
+ WORD $0x2cc10164
+ WORD $0x753b0463
+ WORD $0x2cc18163
+ WORD $0x02c040eb
+ WORD $0x02c20129
+ WORD $0x001500ea
+ WORD $0x67ff80cb
+ WORD $0x640058e6
+ WORD $0x011d2800
+ WORD $0x002d14e5
+ WORD $0x0114a801
+ WORD $0x03400000
+ WORD $0x03400000
+ WORD $0x03400000
+ WORD $0x001500a9
+ WORD $0x0015010a
+ WORD $0x01149822
+ WORD $0x03400000
+ WORD $0x2b800123
+ WORD $0x01010c42
+ WORD $0x02fffd4a
+ WORD $0x02c02129
+ WORD $0x47fff15f
+ WORD $0x01070042
+ WORD $0x00410ce9
+ WORD $0x383c2482
+ WORD $0x02c004e7
+ WORD $0x02c020a5
+ WORD $0x5fffc8e6
+ WORD $0x4c000020
+ RET
+
// func shiftDivFloat64LASX(dst []float64, a []float64, shift float64, denom float64)
TEXT ·shiftDivFloat64LASX(SB), NOSPLIT|NOFRAME, $0-64
MOVV dst_base+0(FP), R4
diff --git a/internal/loong64/numeric_register_lasx_loong64.go b/internal/loong64/numeric_register_lasx_loong64.go
index 59bac6e..a5b9a22 100644
--- a/internal/loong64/numeric_register_lasx_loong64.go
+++ b/internal/loong64/numeric_register_lasx_loong64.go
@@ -242,6 +242,14 @@ func GatherUint64LASX(dst []uint64, src []uint64, idx []int32) {
gatherUint64LASX(dst, src, idx)
}
+func MovingAverageFloat32LASX(dst []float32, a []float32, width int) {
+ if len(dst) < 16 {
+ ref.MovingAverage(dst, a, width)
+ return
+ }
+ movingAverageFloat32LASX(dst, a, width)
+}
+
func ShiftDivFloat32LASX(dst []float32, a []float32, shift float32, denom float32) {
n := min(len(dst), len(a))
if n < 16 {
@@ -260,6 +268,14 @@ func LayerNormFloat32LASX(dst []float32, a []float32, gamma []float32, beta []fl
layerNormFloat32LASX(dst[:n:n], a, gamma, beta, shift, denom)
}
+func MovingAverageFloat64LASX(dst []float64, a []float64, width int) {
+ if len(dst) < 16 {
+ ref.MovingAverage(dst, a, width)
+ return
+ }
+ movingAverageFloat64LASX(dst, a, width)
+}
+
func ShiftDivFloat64LASX(dst []float64, a []float64, shift float64, denom float64) {
n := min(len(dst), len(a))
if n < 16 {
@@ -331,8 +347,10 @@ func registerNumericLASX(s *kernel.Set) {
s.U32.Gather = GatherUint32LASX
s.U64.Tile = TileUint64LASX
s.U64.Gather = GatherUint64LASX
+ s.F32.MovingAverage = MovingAverageFloat32LASX
s.F32.ShiftDiv = ShiftDivFloat32LASX
s.F32.LayerNorm = LayerNormFloat32LASX
+ s.F64.MovingAverage = MovingAverageFloat64LASX
s.F64.ShiftDiv = ShiftDivFloat64LASX
s.F64.LayerNorm = LayerNormFloat64LASX
s.U64.Random = RandomU64LASX
diff --git a/internal/riscv64/numeric_rvv_riscv64.s b/internal/riscv64/numeric_rvv_riscv64.s
index 552bfe0..541ab7a 100644
--- a/internal/riscv64/numeric_rvv_riscv64.s
+++ b/internal/riscv64/numeric_rvv_riscv64.s
@@ -263,20 +263,80 @@ TEXT ·polyEvalFloat32RVV(SB), NOSPLIT|NOFRAME, $0-72
MOV coeffs_len+56(FP), X15
WORD $0x00e6c463
WORD $0x00070693
- WORD $0x06078663
- WORD $0x0ad05e63
- WORD $0x00279713
- WORD $0x00e60733
- WORD $0xffc72787
- WORD $0x00200613
- WORD $0x08c7c063
- WORD $0x00000613
+ WORD $0x00078c63
+ WORD $0x01000713
+ WORD $0x00279813
+ WORD $0x02e6dc63
+ WORD $0x00000713
+ WORD $0x0e80006f
+ WORD $0x16d05c63
+ WORD $0x00000593
+ WORD $0x0d16f657
+ WORD $0x00259713
+ WORD $0x00e50733
+ WORD $0x5e003457
+ WORD $0x40c686b3
+ WORD $0x02076427
+ WORD $0x00c585b3
+ WORD $0xfe0692e3
+ WORD $0x1500006f
+ WORD $0x010602b3
+ WORD $0xffc2a787
+ WORD $0x00100713
+ WORD $0x08f75063
+ WORD $0x00000713
+ WORD $0xcd147057
+ WORD $0x5e07d457
+ WORD $0xfff78893
+ WORD $0xff828293
+ WORD $0x01000393
+ WORD $0x00271313
+ WORD $0x00658733
+ WORD $0x02076507
+ WORD $0x02070713
+ WORD $0x02076607
+ WORD $0x00038713
+ WORD $0x00028393
+ WORD $0x00088e13
+ WORD $0x9e80b857
+ WORD $0x9e80b757
+ WORD $0x0003a787
+ WORD $0x93051857
+ WORD $0x92e61757
+ WORD $0xfffe0e13
+ WORD $0x0307d857
+ WORD $0x02e7d757
+ WORD $0xffc38393
+ WORD $0xffc042e3
+ WORD $0x00650333
+ WORD $0x02036827
+ WORD $0x02030313
+ WORD $0x01070393
+ WORD $0x02036727
+ WORD $0xfa76d2e3
+ WORD $0x0300006f
+ WORD $0x00000713
+ WORD $0xcd147057
+ WORD $0x5e07d457
+ WORD $0x00050893
+ WORD $0x02088293
+ WORD $0x0208e427
+ WORD $0x0202e427
+ WORD $0x02070293
+ WORD $0x01070713
+ WORD $0x04088893
+ WORD $0xfe56d4e3
+ WORD $0x08d75a63
+ WORD $0x01060633
+ WORD $0xffc62787
+ WORD $0x00200813
+ WORD $0x0507c863
WORD $0xfff78793
- WORD $0xff870713
- WORD $0x00261813
+ WORD $0xff860613
+ WORD $0x00271813
WORD $0x010588b3
WORD $0x0008a707
- WORD $0x00070893
+ WORD $0x00060893
WORD $0x00078293
WORD $0x20f786d3
WORD $0x0008a607
@@ -286,22 +346,14 @@ TEXT ·polyEvalFloat32RVV(SB), NOSPLIT|NOFRAME, $0-72
WORD $0xffc88893
WORD $0xfe5046e3
WORD $0x01050833
- WORD $0x00160613
+ WORD $0x00170713
WORD $0x00d82027
- WORD $0xfcd612e3
- WORD $0x0580006f
- WORD $0x04d05a63
- WORD $0x00000593
- WORD $0x0d16f657
- WORD $0x00259713
- WORD $0x00e50733
- WORD $0x5e003457
- WORD $0x40c686b3
- WORD $0x02076427
- WORD $0x00c585b3
- WORD $0xfe0692e3
- WORD $0x02c0006f
+ WORD $0xfcd712e3
+ WORD $0x0380006f
WORD $0x00000593
+ WORD $0x40e686b3
+ WORD $0x00271713
+ WORD $0x00e50533
WORD $0x0d107657
WORD $0x5e07d457
WORD $0x0c76f657
@@ -322,14 +374,189 @@ TEXT ·convolveFloat32RVV(SB), NOSPLIT|NOFRAME, $0-72
MOV dst_len+8(FP), X13
MOV sig_len+32(FP), X14
MOV ker_len+56(FP), X15
- WORD $0x0c078c63
- WORD $0x0cf74a63
+ WORD $0x3a078c63
+ WORD $0x3af74a63
WORD $0x40f70733
WORD $0x00d75463
WORD $0x00170693
- WORD $0x0cd05263
- WORD $0x08f05e63
+ WORD $0xfe010113
+ WORD $0x00813c23
+ WORD $0x00913823
+ WORD $0xc2202773
+ WORD $0x00171713
+ WORD $0x40e10133
+ WORD $0x01000713
+ WORD $0x00e6d663
+ WORD $0x00000713
+ WORD $0x29c0006f
WORD $0x00000713
+ WORD $0x26f05663
+ WORD $0x00279893
+ WORD $0x01000e13
+ WORD $0x00100813
+ WORD $0x011608b3
+ WORD $0xffc00293
+ WORD $0x00000393
+ WORD $0x00271313
+ WORD $0x000e0713
+ WORD $0x00658e33
+ WORD $0x0d007ed7
+ WORD $0x42006c57
+ WORD $0x5e0c0cd7
+ WORD $0x5e0c0dd7
+ WORD $0x5e0c0d57
+ WORD $0x5e0c0657
+ WORD $0x5e0c06d7
+ WORD $0x5e0c0757
+ WORD $0x5e0c07d7
+ WORD $0x5e0c0857
+ WORD $0x5e0c08d7
+ WORD $0x5e0c0957
+ WORD $0x5e0c09d7
+ WORD $0x5e0c0a57
+ WORD $0x5e0c0ad7
+ WORD $0x5e0c0b57
+ WORD $0x5e0c0bd7
+ WORD $0x00078e93
+ WORD $0x0d0eff57
+ WORD $0xfff3cf93
+ WORD $0x00239413
+ WORD $0x002f9f93
+ WORD $0x41e804b3
+ WORD $0x008e0433
+ WORD $0x01f88fb3
+ WORD $0x00249493
+ WORD $0x009f8fb3
+ WORD $0x00440493
+ WORD $0x0204e507
+ WORD $0x00840493
+ WORD $0x0204e587
+ WORD $0x00c40493
+ WORD $0x0204e407
+ WORD $0x01010493
+ WORD $0x02848427
+ WORD $0x01040493
+ WORD $0x0204ee07
+ WORD $0x01440493
+ WORD $0x0204ee87
+ WORD $0x01840493
+ WORD $0x0204ef07
+ WORD $0x01c40493
+ WORD $0x0204ef87
+ WORD $0x02040493
+ WORD $0x0204e387
+ WORD $0x02440493
+ WORD $0x0204e307
+ WORD $0x02840493
+ WORD $0x0204e287
+ WORD $0x02c40493
+ WORD $0x0204e207
+ WORD $0x03040493
+ WORD $0x0204e187
+ WORD $0x03440493
+ WORD $0x0204e107
+ WORD $0x002f1493
+ WORD $0x01f48fb3
+ WORD $0x02046087
+ WORD $0xffcf8f93
+ WORD $0x0a5fe007
+ WORD $0x03840f93
+ WORD $0x03c40413
+ WORD $0x020fe407
+ WORD $0x02046487
+ WORD $0x920090d7
+ WORD $0xc2202ff3
+ WORD $0x01f10fb3
+ WORD $0x010f8f93
+ WORD $0x028f80a7
+ WORD $0x920510d7
+ WORD $0x92059557
+ WORD $0x01010f93
+ WORD $0x028f8587
+ WORD $0x920595d7
+ WORD $0x920e1e57
+ WORD $0x920e9ed7
+ WORD $0x920f1f57
+ WORD $0x920f9fd7
+ WORD $0x920393d7
+ WORD $0x92031357
+ WORD $0x920292d7
+ WORD $0x92021257
+ WORD $0x920191d7
+ WORD $0x92011157
+ WORD $0x92041457
+ WORD $0x920494d7
+ WORD $0x0e9c9cd7
+ WORD $0x0e8d9dd7
+ WORD $0x0e2d1d57
+ WORD $0x0e361657
+ WORD $0x0e4696d7
+ WORD $0x0e571757
+ WORD $0x0e6797d7
+ WORD $0x0e781857
+ WORD $0x0ff898d7
+ WORD $0x0fe91957
+ WORD $0x0fd999d7
+ WORD $0x0fca1a57
+ WORD $0x0eba9ad7
+ WORD $0x0eab1b57
+ WORD $0x0e1b9bd7
+ WORD $0xc2202ff3
+ WORD $0x01f10fb3
+ WORD $0x010f8f93
+ WORD $0x028f8407
+ WORD $0x0e8c1c57
+ WORD $0x41ee8eb3
+ WORD $0x01e383b3
+ WORD $0xe80e9ce3
+ WORD $0x439017d7
+ WORD $0x43b01757
+ WORD $0x43a016d7
+ WORD $0x42c01657
+ WORD $0x42d015d7
+ WORD $0x42e01557
+ WORD $0x42f01057
+ WORD $0x430010d7
+ WORD $0x43101157
+ WORD $0x432011d7
+ WORD $0x43301257
+ WORD $0x434012d7
+ WORD $0x43501357
+ WORD $0x436013d7
+ WORD $0x43701857
+ WORD $0x438018d7
+ WORD $0x00650333
+ WORD $0x01070e13
+ WORD $0x01132027
+ WORD $0x01032227
+ WORD $0x00732427
+ WORD $0x00632627
+ WORD $0x00532827
+ WORD $0x00432a27
+ WORD $0x00332c27
+ WORD $0x00232e27
+ WORD $0x02132027
+ WORD $0x02032227
+ WORD $0x02a32427
+ WORD $0x02b32627
+ WORD $0x02c32827
+ WORD $0x02d32a27
+ WORD $0x02e32c27
+ WORD $0x02f32e27
+ WORD $0xdbc6dae3
+ WORD $0x02c0006f
+ WORD $0xcd147057
+ WORD $0x5e003457
+ WORD $0x00050813
+ WORD $0x02080893
+ WORD $0x02086427
+ WORD $0x0208e427
+ WORD $0x02070893
+ WORD $0x01070713
+ WORD $0x04080813
+ WORD $0xff16d4e3
+ WORD $0x0cd75663
+ WORD $0x08f05c63
WORD $0x00279893
WORD $0x0d007857
WORD $0x42006457
@@ -366,8 +593,11 @@ TEXT ·convolveFloat32RVV(SB), NOSPLIT|NOFRAME, $0-72
WORD $0x00170713
WORD $0x00f2a027
WORD $0xf8d716e3
- WORD $0x0280006f
+ WORD $0x0340006f
WORD $0x00000593
+ WORD $0x40e686b3
+ WORD $0x00271713
+ WORD $0x00e50533
WORD $0x0d16f657
WORD $0x00259713
WORD $0x00e50733
@@ -376,6 +606,12 @@ TEXT ·convolveFloat32RVV(SB), NOSPLIT|NOFRAME, $0-72
WORD $0x02076427
WORD $0x00c585b3
WORD $0xfe0692e3
+ WORD $0xc2202573
+ WORD $0x00151513
+ WORD $0x00a10133
+ WORD $0x01813403
+ WORD $0x01013483
+ WORD $0x02010113
WORD $0x00008067
RET
@@ -387,14 +623,188 @@ TEXT ·correlateFloat32RVV(SB), NOSPLIT|NOFRAME, $0-72
MOV dst_len+8(FP), X13
MOV sig_len+32(FP), X14
MOV ker_len+56(FP), X15
- WORD $0x0a078463
- WORD $0x0af74263
+ WORD $0x36078663
+ WORD $0x36f74463
WORD $0x40f70733
WORD $0x00d75463
WORD $0x00170693
- WORD $0x08d05a63
- WORD $0x06f05663
+ WORD $0x01000713
+ WORD $0x00e6d863
+ WORD $0x00000713
+ WORD $0x2ad04a63
+ WORD $0x3480006f
+ WORD $0xff010113
+ WORD $0xc2202773
+ WORD $0x00171713
+ WORD $0x40e10133
WORD $0x00000713
+ WORD $0x24f05e63
+ WORD $0x01000293
+ WORD $0x00000893
+ WORD $0x00271813
+ WORD $0x00028713
+ WORD $0x010582b3
+ WORD $0x0d007357
+ WORD $0x42006c57
+ WORD $0x5e0c0d57
+ WORD $0x5e0c0557
+ WORD $0x5e0c05d7
+ WORD $0x5e0c0657
+ WORD $0x5e0c06d7
+ WORD $0x5e0c0757
+ WORD $0x5e0c07d7
+ WORD $0x5e0c0857
+ WORD $0x5e0c08d7
+ WORD $0x5e0c0957
+ WORD $0x5e0c09d7
+ WORD $0x5e0c0a57
+ WORD $0x5e0c0ad7
+ WORD $0x5e0c0b57
+ WORD $0x5e0c0bd7
+ WORD $0x00078313
+ WORD $0x0d0373d7
+ WORD $0x00289e13
+ WORD $0x01c60eb3
+ WORD $0x01c28e33
+ WORD $0x020eec87
+ WORD $0x020e6407
+ WORD $0xc2202ef3
+ WORD $0x01d10eb3
+ WORD $0x010e8e93
+ WORD $0x028e8427
+ WORD $0x004e0e93
+ WORD $0x020ee407
+ WORD $0x01010e93
+ WORD $0x028e8427
+ WORD $0x008e0e93
+ WORD $0x020eee07
+ WORD $0x00ce0e93
+ WORD $0x020eee87
+ WORD $0x010e0e93
+ WORD $0x020eef07
+ WORD $0x014e0e93
+ WORD $0x020eef87
+ WORD $0x018e0e93
+ WORD $0x020ee387
+ WORD $0x01ce0e93
+ WORD $0x020ee307
+ WORD $0x020e0e93
+ WORD $0x020ee287
+ WORD $0x024e0e93
+ WORD $0x020ee207
+ WORD $0x028e0e93
+ WORD $0x020ee187
+ WORD $0x02ce0e93
+ WORD $0x020ee107
+ WORD $0x030e0e93
+ WORD $0x020ee087
+ WORD $0x034e0e93
+ WORD $0x020ee007
+ WORD $0x038e0e93
+ WORD $0x03ce0e13
+ WORD $0x020ee407
+ WORD $0x020e6487
+ WORD $0xc2202e73
+ WORD $0x01c10e33
+ WORD $0x010e0e13
+ WORD $0x028e0d87
+ WORD $0x939d9dd7
+ WORD $0xc2202e73
+ WORD $0x01c10e33
+ WORD $0x010e0e13
+ WORD $0x028e0da7
+ WORD $0x01010e13
+ WORD $0x028e0d87
+ WORD $0x939d9dd7
+ WORD $0x939e1e57
+ WORD $0x939e9ed7
+ WORD $0x939f1f57
+ WORD $0x939f9fd7
+ WORD $0x939393d7
+ WORD $0x93931357
+ WORD $0x939292d7
+ WORD $0x93921257
+ WORD $0x939191d7
+ WORD $0x93911157
+ WORD $0x939090d7
+ WORD $0x93901057
+ WORD $0x93941457
+ WORD $0x939494d7
+ WORD $0xc2202e73
+ WORD $0x01c10e33
+ WORD $0x010e0e13
+ WORD $0x028e0c87
+ WORD $0x0f9c1c57
+ WORD $0x0e9d1d57
+ WORD $0x0e851557
+ WORD $0x0e0595d7
+ WORD $0x0e161657
+ WORD $0x0e2696d7
+ WORD $0x0e371757
+ WORD $0x0e4797d7
+ WORD $0x0e581857
+ WORD $0x0e6898d7
+ WORD $0x0e791957
+ WORD $0x0ff999d7
+ WORD $0x0fea1a57
+ WORD $0x0fda9ad7
+ WORD $0x0fcb1b57
+ WORD $0x0fbb9bd7
+ WORD $0x40730333
+ WORD $0x007888b3
+ WORD $0xe8031ce3
+ WORD $0x43a017d7
+ WORD $0x42a01757
+ WORD $0x42b016d7
+ WORD $0x42c01657
+ WORD $0x42d015d7
+ WORD $0x42e01557
+ WORD $0x42f01057
+ WORD $0x430010d7
+ WORD $0x43101157
+ WORD $0x432011d7
+ WORD $0x43301257
+ WORD $0x434012d7
+ WORD $0x43501357
+ WORD $0x436013d7
+ WORD $0x43701857
+ WORD $0x438018d7
+ WORD $0x01050833
+ WORD $0x01070293
+ WORD $0x01182027
+ WORD $0x01082227
+ WORD $0x00782427
+ WORD $0x00682627
+ WORD $0x00582827
+ WORD $0x00482a27
+ WORD $0x00382c27
+ WORD $0x00282e27
+ WORD $0x02182027
+ WORD $0x02082227
+ WORD $0x02a82427
+ WORD $0x02b82627
+ WORD $0x02c82827
+ WORD $0x02d82a27
+ WORD $0x02e82c27
+ WORD $0x02f82e27
+ WORD $0xda56dae3
+ WORD $0x02c0006f
+ WORD $0xcd147057
+ WORD $0x5e003457
+ WORD $0x00050813
+ WORD $0x02080893
+ WORD $0x02086427
+ WORD $0x0208e427
+ WORD $0x02070893
+ WORD $0x01070713
+ WORD $0x04080813
+ WORD $0xff16d4e3
+ WORD $0xc2202873
+ WORD $0x00181813
+ WORD $0x01010133
+ WORD $0x01010113
+ WORD $0x08d75e63
+ WORD $0x06f05463
WORD $0x0d007857
WORD $0x42006457
WORD $0x00000893
@@ -419,8 +829,11 @@ TEXT ·correlateFloat32RVV(SB), NOSPLIT|NOFRAME, $0-72
WORD $0x00170713
WORD $0x00f82027
WORD $0xfad716e3
- WORD $0x0280006f
+ WORD $0x0340006f
WORD $0x00000593
+ WORD $0x40e686b3
+ WORD $0x00271713
+ WORD $0x00e50533
WORD $0x0d16f657
WORD $0x00259713
WORD $0x00e50733
@@ -684,20 +1097,105 @@ TEXT ·polyEvalFloat64RVV(SB), NOSPLIT|NOFRAME, $0-72
MOV coeffs_len+56(FP), X15
WORD $0x00e6c463
WORD $0x00070693
- WORD $0x06078663
- WORD $0x0ad05e63
+ WORD $0x00078c63
+ WORD $0x01000813
WORD $0x00379713
- WORD $0x00e60733
- WORD $0xff873787
- WORD $0x00200613
- WORD $0x08c7c063
- WORD $0x00000613
+ WORD $0x0306dc63
+ WORD $0x00000813
+ WORD $0x14c0006f
+ WORD $0x1cd05e63
+ WORD $0x00000593
+ WORD $0x0d96f657
+ WORD $0x00359713
+ WORD $0x00e50733
+ WORD $0x5e003457
+ WORD $0x40c686b3
+ WORD $0x02077427
+ WORD $0x00c585b3
+ WORD $0xfe0692e3
+ WORD $0x1b40006f
+ WORD $0x00e602b3
+ WORD $0xff82b787
+ WORD $0x00100813
+ WORD $0x0af85c63
+ WORD $0x00000813
+ WORD $0xcd927057
+ WORD $0x5e07d457
+ WORD $0xfff78893
+ WORD $0xff028293
+ WORD $0x01000393
+ WORD $0x00381313
+ WORD $0x00658833
+ WORD $0x02087507
+ WORD $0x02080e13
+ WORD $0x020e7607
+ WORD $0x04080e13
+ WORD $0x06080813
+ WORD $0x020e7807
+ WORD $0x02087a07
+ WORD $0x00038813
+ WORD $0x00028393
+ WORD $0x00088e13
+ WORD $0x9e80b957
+ WORD $0x9e80bc57
+ WORD $0x9e80bb57
+ WORD $0x9e80b757
+ WORD $0x0003b787
+ WORD $0x93251957
+ WORD $0x93861c57
+ WORD $0x93681b57
+ WORD $0x92ea1757
+ WORD $0xfffe0e13
+ WORD $0x0327d957
+ WORD $0x0387dc57
+ WORD $0x0367db57
+ WORD $0x02e7d757
+ WORD $0xff838393
+ WORD $0xfdc04ae3
+ WORD $0x00650333
+ WORD $0x02030393
+ WORD $0x0203fc27
+ WORD $0x04030393
+ WORD $0x0203fb27
+ WORD $0x01080393
+ WORD $0x02037927
+ WORD $0x06030313
+ WORD $0x02037727
+ WORD $0xf676d6e3
+ WORD $0x05c0006f
+ WORD $0x00000813
+ WORD $0x04050893
+ WORD $0xfcf8b027
+ WORD $0xfcf8b427
+ WORD $0xfcf8b827
+ WORD $0xfcf8bc27
+ WORD $0xfef8b027
+ WORD $0xfef8b427
+ WORD $0xfef8b827
+ WORD $0xfef8bc27
+ WORD $0x00f8b027
+ WORD $0x00f8b427
+ WORD $0x00f8b827
+ WORD $0x00f8bc27
+ WORD $0x02f8b027
+ WORD $0x02f8b427
+ WORD $0x02f8b827
+ WORD $0x02f8bc27
+ WORD $0x02080293
+ WORD $0x01080813
+ WORD $0x08088893
+ WORD $0xfa56dae3
+ WORD $0x08d85a63
+ WORD $0x00e60633
+ WORD $0xff863787
+ WORD $0x00200713
+ WORD $0x04e7c863
WORD $0xfff78793
- WORD $0xff070713
- WORD $0x00361813
- WORD $0x010588b3
+ WORD $0xff060613
+ WORD $0x00381713
+ WORD $0x00e588b3
WORD $0x0008b707
- WORD $0x00070893
+ WORD $0x00060893
WORD $0x00078293
WORD $0x22f786d3
WORD $0x0008b607
@@ -706,32 +1204,24 @@ TEXT ·polyEvalFloat64RVV(SB), NOSPLIT|NOFRAME, $0-72
WORD $0x02c6f6d3
WORD $0xff888893
WORD $0xfe5046e3
- WORD $0x01050833
- WORD $0x00160613
- WORD $0x00d83027
- WORD $0xfcd612e3
- WORD $0x0580006f
- WORD $0x04d05a63
- WORD $0x00000593
- WORD $0x0d96f657
- WORD $0x00359713
WORD $0x00e50733
- WORD $0x5e003457
- WORD $0x40c686b3
- WORD $0x02077427
- WORD $0x00c585b3
- WORD $0xfe0692e3
- WORD $0x02c0006f
+ WORD $0x00180813
+ WORD $0x00d73027
+ WORD $0xfcd812e3
+ WORD $0x0380006f
WORD $0x00000593
- WORD $0x0d907657
+ WORD $0x41068633
+ WORD $0x00381813
+ WORD $0x01050533
+ WORD $0x0d9076d7
WORD $0x5e07d457
- WORD $0x0c66f657
+ WORD $0x0c6676d7
WORD $0x00359713
WORD $0x00e50733
- WORD $0x40c686b3
+ WORD $0x40d60633
WORD $0x02077427
- WORD $0x00c585b3
- WORD $0xfe0694e3
+ WORD $0x00d585b3
+ WORD $0xfe0614e3
WORD $0x00008067
RET
@@ -743,14 +1233,196 @@ TEXT ·convolveFloat64RVV(SB), NOSPLIT|NOFRAME, $0-72
MOV dst_len+8(FP), X13
MOV sig_len+32(FP), X14
MOV ker_len+56(FP), X15
- WORD $0x0c078c63
- WORD $0x0cf74a63
+ WORD $0x3c078a63
+ WORD $0x3cf74863
WORD $0x40f70733
WORD $0x00d75463
WORD $0x00170693
- WORD $0x0cd05263
- WORD $0x08f05e63
+ WORD $0xfe010113
+ WORD $0x00813c23
+ WORD $0x00913823
+ WORD $0xc2202773
+ WORD $0x00171713
+ WORD $0x40e10133
+ WORD $0x01000713
+ WORD $0x00e6d663
+ WORD $0x00000713
+ WORD $0x2b80006f
WORD $0x00000713
+ WORD $0x26f05663
+ WORD $0x00379893
+ WORD $0x01000e13
+ WORD $0x00100813
+ WORD $0x011608b3
+ WORD $0xff800293
+ WORD $0x00000393
+ WORD $0x00371313
+ WORD $0x000e0713
+ WORD $0x00658e33
+ WORD $0x0d807ed7
+ WORD $0x42006c57
+ WORD $0x5e0c0cd7
+ WORD $0x5e0c0dd7
+ WORD $0x5e0c0d57
+ WORD $0x5e0c0657
+ WORD $0x5e0c06d7
+ WORD $0x5e0c0757
+ WORD $0x5e0c07d7
+ WORD $0x5e0c0857
+ WORD $0x5e0c08d7
+ WORD $0x5e0c0957
+ WORD $0x5e0c09d7
+ WORD $0x5e0c0a57
+ WORD $0x5e0c0ad7
+ WORD $0x5e0c0b57
+ WORD $0x5e0c0bd7
+ WORD $0x00078e93
+ WORD $0x0d8eff57
+ WORD $0xfff3cf93
+ WORD $0x00339413
+ WORD $0x003f9f93
+ WORD $0x41e804b3
+ WORD $0x008e0433
+ WORD $0x01f88fb3
+ WORD $0x00349493
+ WORD $0x009f8fb3
+ WORD $0x00840493
+ WORD $0x0204f507
+ WORD $0x01040493
+ WORD $0x0204f587
+ WORD $0x01840493
+ WORD $0x0204f407
+ WORD $0x01010493
+ WORD $0x02848427
+ WORD $0x02040493
+ WORD $0x0204fe07
+ WORD $0x02840493
+ WORD $0x0204fe87
+ WORD $0x03040493
+ WORD $0x0204ff07
+ WORD $0x03840493
+ WORD $0x0204ff87
+ WORD $0x04040493
+ WORD $0x0204f387
+ WORD $0x04840493
+ WORD $0x0204f307
+ WORD $0x05040493
+ WORD $0x0204f287
+ WORD $0x05840493
+ WORD $0x0204f207
+ WORD $0x06040493
+ WORD $0x0204f187
+ WORD $0x06840493
+ WORD $0x0204f107
+ WORD $0x003f1493
+ WORD $0x01f48fb3
+ WORD $0x02047087
+ WORD $0xff8f8f93
+ WORD $0x0a5ff007
+ WORD $0x07040f93
+ WORD $0x07840413
+ WORD $0x020ff407
+ WORD $0x02047487
+ WORD $0x920090d7
+ WORD $0xc2202ff3
+ WORD $0x01f10fb3
+ WORD $0x010f8f93
+ WORD $0x028f80a7
+ WORD $0x920510d7
+ WORD $0x92059557
+ WORD $0x01010f93
+ WORD $0x028f8587
+ WORD $0x920595d7
+ WORD $0x920e1e57
+ WORD $0x920e9ed7
+ WORD $0x920f1f57
+ WORD $0x920f9fd7
+ WORD $0x920393d7
+ WORD $0x92031357
+ WORD $0x920292d7
+ WORD $0x92021257
+ WORD $0x920191d7
+ WORD $0x92011157
+ WORD $0x92041457
+ WORD $0x920494d7
+ WORD $0x0e9c9cd7
+ WORD $0x0e8d9dd7
+ WORD $0x0e2d1d57
+ WORD $0x0e361657
+ WORD $0x0e4696d7
+ WORD $0x0e571757
+ WORD $0x0e6797d7
+ WORD $0x0e781857
+ WORD $0x0ff898d7
+ WORD $0x0fe91957
+ WORD $0x0fd999d7
+ WORD $0x0fca1a57
+ WORD $0x0eba9ad7
+ WORD $0x0eab1b57
+ WORD $0x0e1b9bd7
+ WORD $0xc2202ff3
+ WORD $0x01f10fb3
+ WORD $0x010f8f93
+ WORD $0x028f8407
+ WORD $0x0e8c1c57
+ WORD $0x41ee8eb3
+ WORD $0x01e383b3
+ WORD $0xe80e9ce3
+ WORD $0x439017d7
+ WORD $0x43b01757
+ WORD $0x43a016d7
+ WORD $0x42c01657
+ WORD $0x42d015d7
+ WORD $0x42e01557
+ WORD $0x42f01057
+ WORD $0x430010d7
+ WORD $0x43101157
+ WORD $0x432011d7
+ WORD $0x43301257
+ WORD $0x434012d7
+ WORD $0x43501357
+ WORD $0x436013d7
+ WORD $0x43701857
+ WORD $0x438018d7
+ WORD $0x00650333
+ WORD $0x01070e13
+ WORD $0x01133027
+ WORD $0x01033427
+ WORD $0x00733827
+ WORD $0x00633c27
+ WORD $0x02533027
+ WORD $0x02433427
+ WORD $0x02333827
+ WORD $0x02233c27
+ WORD $0x04133027
+ WORD $0x04033427
+ WORD $0x04a33827
+ WORD $0x04b33c27
+ WORD $0x06c33027
+ WORD $0x06d33427
+ WORD $0x06e33827
+ WORD $0x06f33c27
+ WORD $0xdbc6dae3
+ WORD $0x0480006f
+ WORD $0xcd927057
+ WORD $0x5e003657
+ WORD $0xcda47057
+ WORD $0x5e003457
+ WORD $0x00050813
+ WORD $0x02080893
+ WORD $0xcd927057
+ WORD $0x02087627
+ WORD $0xcda47057
+ WORD $0x0208f427
+ WORD $0x06080893
+ WORD $0xcd927057
+ WORD $0x0208f627
+ WORD $0x02070893
+ WORD $0x01070713
+ WORD $0x08080813
+ WORD $0xfd16dae3
+ WORD $0x0cd75663
+ WORD $0x08f05c63
WORD $0x00379893
WORD $0x0d807857
WORD $0x42006457
@@ -787,8 +1459,11 @@ TEXT ·convolveFloat64RVV(SB), NOSPLIT|NOFRAME, $0-72
WORD $0x00170713
WORD $0x00f2b027
WORD $0xf8d716e3
- WORD $0x0280006f
+ WORD $0x0340006f
WORD $0x00000593
+ WORD $0x40e686b3
+ WORD $0x00371713
+ WORD $0x00e50533
WORD $0x0d96f657
WORD $0x00359713
WORD $0x00e50733
@@ -797,6 +1472,12 @@ TEXT ·convolveFloat64RVV(SB), NOSPLIT|NOFRAME, $0-72
WORD $0x02077427
WORD $0x00c585b3
WORD $0xfe0692e3
+ WORD $0xc2202573
+ WORD $0x00151513
+ WORD $0x00a10133
+ WORD $0x01813403
+ WORD $0x01013483
+ WORD $0x02010113
WORD $0x00008067
RET
@@ -808,14 +1489,195 @@ TEXT ·correlateFloat64RVV(SB), NOSPLIT|NOFRAME, $0-72
MOV dst_len+8(FP), X13
MOV sig_len+32(FP), X14
MOV ker_len+56(FP), X15
- WORD $0x0a078463
- WORD $0x0af74263
+ WORD $0x38078463
+ WORD $0x38f74263
WORD $0x40f70733
WORD $0x00d75463
WORD $0x00170693
- WORD $0x08d05a63
- WORD $0x06f05663
+ WORD $0x01000713
+ WORD $0x00e6d863
WORD $0x00000713
+ WORD $0x2cd04863
+ WORD $0x3640006f
+ WORD $0xff010113
+ WORD $0xc2202773
+ WORD $0x00171713
+ WORD $0x40e10133
+ WORD $0x00000713
+ WORD $0x24f05e63
+ WORD $0x01000293
+ WORD $0x00000893
+ WORD $0x00371813
+ WORD $0x00028713
+ WORD $0x010582b3
+ WORD $0x0d807357
+ WORD $0x42006c57
+ WORD $0x5e0c0d57
+ WORD $0x5e0c0557
+ WORD $0x5e0c05d7
+ WORD $0x5e0c0657
+ WORD $0x5e0c06d7
+ WORD $0x5e0c0757
+ WORD $0x5e0c07d7
+ WORD $0x5e0c0857
+ WORD $0x5e0c08d7
+ WORD $0x5e0c0957
+ WORD $0x5e0c09d7
+ WORD $0x5e0c0a57
+ WORD $0x5e0c0ad7
+ WORD $0x5e0c0b57
+ WORD $0x5e0c0bd7
+ WORD $0x00078313
+ WORD $0x0d8373d7
+ WORD $0x00389e13
+ WORD $0x01c60eb3
+ WORD $0x01c28e33
+ WORD $0x020efc87
+ WORD $0x020e7407
+ WORD $0xc2202ef3
+ WORD $0x01d10eb3
+ WORD $0x010e8e93
+ WORD $0x028e8427
+ WORD $0x008e0e93
+ WORD $0x020ef407
+ WORD $0x01010e93
+ WORD $0x028e8427
+ WORD $0x010e0e93
+ WORD $0x020efe07
+ WORD $0x018e0e93
+ WORD $0x020efe87
+ WORD $0x020e0e93
+ WORD $0x020eff07
+ WORD $0x028e0e93
+ WORD $0x020eff87
+ WORD $0x030e0e93
+ WORD $0x020ef387
+ WORD $0x038e0e93
+ WORD $0x020ef307
+ WORD $0x040e0e93
+ WORD $0x020ef287
+ WORD $0x048e0e93
+ WORD $0x020ef207
+ WORD $0x050e0e93
+ WORD $0x020ef187
+ WORD $0x058e0e93
+ WORD $0x020ef107
+ WORD $0x060e0e93
+ WORD $0x020ef087
+ WORD $0x068e0e93
+ WORD $0x020ef007
+ WORD $0x070e0e93
+ WORD $0x078e0e13
+ WORD $0x020ef407
+ WORD $0x020e7487
+ WORD $0xc2202e73
+ WORD $0x01c10e33
+ WORD $0x010e0e13
+ WORD $0x028e0d87
+ WORD $0x939d9dd7
+ WORD $0xc2202e73
+ WORD $0x01c10e33
+ WORD $0x010e0e13
+ WORD $0x028e0da7
+ WORD $0x01010e13
+ WORD $0x028e0d87
+ WORD $0x939d9dd7
+ WORD $0x939e1e57
+ WORD $0x939e9ed7
+ WORD $0x939f1f57
+ WORD $0x939f9fd7
+ WORD $0x939393d7
+ WORD $0x93931357
+ WORD $0x939292d7
+ WORD $0x93921257
+ WORD $0x939191d7
+ WORD $0x93911157
+ WORD $0x939090d7
+ WORD $0x93901057
+ WORD $0x93941457
+ WORD $0x939494d7
+ WORD $0xc2202e73
+ WORD $0x01c10e33
+ WORD $0x010e0e13
+ WORD $0x028e0c87
+ WORD $0x0f9c1c57
+ WORD $0x0e9d1d57
+ WORD $0x0e851557
+ WORD $0x0e0595d7
+ WORD $0x0e161657
+ WORD $0x0e2696d7
+ WORD $0x0e371757
+ WORD $0x0e4797d7
+ WORD $0x0e581857
+ WORD $0x0e6898d7
+ WORD $0x0e791957
+ WORD $0x0ff999d7
+ WORD $0x0fea1a57
+ WORD $0x0fda9ad7
+ WORD $0x0fcb1b57
+ WORD $0x0fbb9bd7
+ WORD $0x40730333
+ WORD $0x007888b3
+ WORD $0xe8031ce3
+ WORD $0x43a017d7
+ WORD $0x42a01757
+ WORD $0x42b016d7
+ WORD $0x42c01657
+ WORD $0x42d015d7
+ WORD $0x42e01557
+ WORD $0x42f01057
+ WORD $0x430010d7
+ WORD $0x43101157
+ WORD $0x432011d7
+ WORD $0x43301257
+ WORD $0x434012d7
+ WORD $0x43501357
+ WORD $0x436013d7
+ WORD $0x43701857
+ WORD $0x438018d7
+ WORD $0x01050833
+ WORD $0x01070293
+ WORD $0x01183027
+ WORD $0x01083427
+ WORD $0x00783827
+ WORD $0x00683c27
+ WORD $0x02583027
+ WORD $0x02483427
+ WORD $0x02383827
+ WORD $0x02283c27
+ WORD $0x04183027
+ WORD $0x04083427
+ WORD $0x04a83827
+ WORD $0x04b83c27
+ WORD $0x06c83027
+ WORD $0x06d83427
+ WORD $0x06e83827
+ WORD $0x06f83c27
+ WORD $0xda56dae3
+ WORD $0x0480006f
+ WORD $0xcd927057
+ WORD $0x5e003657
+ WORD $0xcda47057
+ WORD $0x5e003457
+ WORD $0x00050813
+ WORD $0x02080893
+ WORD $0xcd927057
+ WORD $0x02087627
+ WORD $0xcda47057
+ WORD $0x0208f427
+ WORD $0x06080893
+ WORD $0xcd927057
+ WORD $0x0208f627
+ WORD $0x02070893
+ WORD $0x01070713
+ WORD $0x08080813
+ WORD $0xfd16dae3
+ WORD $0xc2202873
+ WORD $0x00181813
+ WORD $0x01010133
+ WORD $0x01010113
+ WORD $0x08d75e63
+ WORD $0x06f05463
WORD $0x0d807857
WORD $0x42006457
WORD $0x00000893
@@ -840,8 +1702,11 @@ TEXT ·correlateFloat64RVV(SB), NOSPLIT|NOFRAME, $0-72
WORD $0x00170713
WORD $0x00f83027
WORD $0xfad716e3
- WORD $0x0280006f
+ WORD $0x0340006f
WORD $0x00000593
+ WORD $0x40e686b3
+ WORD $0x00371713
+ WORD $0x00e50533
WORD $0x0d96f657
WORD $0x00359713
WORD $0x00e50733
@@ -2067,14 +2932,261 @@ TEXT ·movingAverageFloat32RVV(SB), NOSPLIT|NOFRAME, $0-56
MOV dst_len+8(FP), X12
MOV a_len+32(FP), X13
MOV width+48(FP), X14
- WORD $0x06e05c63
- WORD $0x06e6ca63
+ WORD $0x44e05a63
+ WORD $0x44e6c863
WORD $0x40e686b3
WORD $0x00c6d463
WORD $0x00168613
- WORD $0x06c05263
+ WORD $0x01000893
+ WORD $0x01165863
+ WORD $0x00000693
+ WORD $0x3cc04c63
+ WORD $0x4300006f
+ WORD $0xff010113
+ WORD $0xc22026f3
+ WORD $0x00c00793
+ WORD $0x02f686b3
+ WORD $0x40d10133
WORD $0x00000693
WORD $0xd03777d3
+ WORD $0x00000813
+ WORD $0x00269793
+ WORD $0x00088693
+ WORD $0x00f588b3
+ WORD $0x0d0072d7
+ WORD $0x42006457
+ WORD $0x9e80b557
+ WORD $0xc22022f3
+ WORD $0x00a00313
+ WORD $0x026282b3
+ WORD $0x005102b3
+ WORD $0x01028293
+ WORD $0x22828427
+ WORD $0xc22022f3
+ WORD $0x00329293
+ WORD $0x005102b3
+ WORD $0x01028293
+ WORD $0x22828427
+ WORD $0x9e80b857
+ WORD $0x9e80b957
+ WORD $0x9e80ba57
+ WORD $0x9e80bb57
+ WORD $0x9e80bc57
+ WORD $0x9e80bd57
+ WORD $0x9e80be57
+ WORD $0x9e80bf57
+ WORD $0x9e80b357
+ WORD $0x9e80b257
+ WORD $0x9e80b157
+ WORD $0x9e80b057
+ WORD $0x9e80b657
+ WORD $0x00070293
+ WORD $0x0d12f357
+ WORD $0x00281393
+ WORD $0x007883b3
+ WORD $0x0203e707
+ WORD $0x00838e13
+ WORD $0x020e6407
+ WORD $0xc2202e73
+ WORD $0x001e1e13
+ WORD $0x01c10e33
+ WORD $0x010e0e13
+ WORD $0x228e0427
+ WORD $0x01038e13
+ WORD $0x020e6407
+ WORD $0x01010e13
+ WORD $0x228e0427
+ WORD $0x03c38e13
+ WORD $0x020e6407
+ WORD $0xc2202e73
+ WORD $0x00600e93
+ WORD $0x03de0e33
+ WORD $0x01c10e33
+ WORD $0x010e0e13
+ WORD $0x228e0427
+ WORD $0x03838e13
+ WORD $0x0ee61657
+ WORD $0xc2202ef3
+ WORD $0x002e9e93
+ WORD $0x01d10eb3
+ WORD $0x010e8e93
+ WORD $0x228e8627
+ WORD $0x020e6407
+ WORD $0x03438e13
+ WORD $0xc2202ef3
+ WORD $0x00600f13
+ WORD $0x03ee8eb3
+ WORD $0x01d10eb3
+ WORD $0x010e8e93
+ WORD $0x228e8607
+ WORD $0x0ec51557
+ WORD $0xc2202ef3
+ WORD $0x00600f13
+ WORD $0x03ee8eb3
+ WORD $0x01d10eb3
+ WORD $0x010e8e93
+ WORD $0x228e8527
+ WORD $0x020e6507
+ WORD $0x03038e13
+ WORD $0x9e003657
+ WORD $0x9e203057
+ WORD $0x9e403157
+ WORD $0x9e603257
+ WORD $0x9fe03357
+ WORD $0x9fc03f57
+ WORD $0x9fa03e57
+ WORD $0x9f803d57
+ WORD $0x9f603c57
+ WORD $0xc2202ef3
+ WORD $0x00a00f13
+ WORD $0x03ee8eb3
+ WORD $0x01d10eb3
+ WORD $0x010e8e93
+ WORD $0x228e8707
+ WORD $0x0e871757
+ WORD $0xc2202ef3
+ WORD $0x00a00f13
+ WORD $0x03ee8eb3
+ WORD $0x01d10eb3
+ WORD $0x010e8e93
+ WORD $0x228e8727
+ WORD $0xc2202ef3
+ WORD $0x003e9e93
+ WORD $0x01d10eb3
+ WORD $0x010e8e93
+ WORD $0x228e8707
+ WORD $0x9f803b57
+ WORD $0x9fa03c57
+ WORD $0x9fc03d57
+ WORD $0x9fe03e57
+ WORD $0x9e603f57
+ WORD $0x9e403357
+ WORD $0x9e203257
+ WORD $0x9e003157
+ WORD $0x9ec03057
+ WORD $0xc2202ef3
+ WORD $0x002e9e93
+ WORD $0x01d10eb3
+ WORD $0x010e8e93
+ WORD $0x228e8607
+ WORD $0x020e6407
+ WORD $0x02c38e13
+ WORD $0x0ea71757
+ WORD $0xc2202ef3
+ WORD $0x003e9e93
+ WORD $0x01d10eb3
+ WORD $0x010e8e93
+ WORD $0x228e8727
+ WORD $0x020e6507
+ WORD $0x02838e13
+ WORD $0x0e881857
+ WORD $0x020e6407
+ WORD $0x02438e13
+ WORD $0x0ea91957
+ WORD $0x020e6507
+ WORD $0x02038e13
+ WORD $0x0e8a1a57
+ WORD $0x020e6407
+ WORD $0x01c38e13
+ WORD $0x0eab1b57
+ WORD $0x020e6507
+ WORD $0x01838e13
+ WORD $0x0e8c1c57
+ WORD $0x020e6407
+ WORD $0x01438e13
+ WORD $0x0ead1d57
+ WORD $0x020e6507
+ WORD $0x00438e13
+ WORD $0x00c38393
+ WORD $0x0e8e1e57
+ WORD $0x0203e407
+ WORD $0x0eaf1f57
+ WORD $0x020e6507
+ WORD $0x01010393
+ WORD $0x22838707
+ WORD $0x0ee31357
+ WORD $0x0e821257
+ WORD $0xc22023f3
+ WORD $0x00139393
+ WORD $0x007103b3
+ WORD $0x01038393
+ WORD $0x22838407
+ WORD $0x0e811157
+ WORD $0x0ea01057
+ WORD $0xc22023f3
+ WORD $0x00600e13
+ WORD $0x03c383b3
+ WORD $0x007103b3
+ WORD $0x01038393
+ WORD $0x22838507
+ WORD $0x406282b3
+ WORD $0x00680833
+ WORD $0xdc0298e3
+ WORD $0x42a01757
+ WORD $0xc2202873
+ WORD $0x00a00893
+ WORD $0x03180833
+ WORD $0x01010833
+ WORD $0x01080813
+ WORD $0x22880407
+ WORD $0x428016d7
+ WORD $0xc2202873
+ WORD $0x00381813
+ WORD $0x01010833
+ WORD $0x01080813
+ WORD $0x22880407
+ WORD $0x42801657
+ WORD $0x430015d7
+ WORD $0x43201557
+ WORD $0x43401057
+ WORD $0x436010d7
+ WORD $0x43801157
+ WORD $0x43a011d7
+ WORD $0x43c01257
+ WORD $0x43e012d7
+ WORD $0x42601357
+ WORD $0x424013d7
+ WORD $0x42201857
+ WORD $0x420018d7
+ WORD $0x42c01e57
+ WORD $0xcd027057
+ WORD $0x5e0e5457
+ WORD $0x5e0354d7
+ WORD $0x5e015557
+ WORD $0x5e05d5d7
+ WORD $0x3e88d457
+ WORD $0x3e92d4d7
+ WORD $0x3ea0d557
+ WORD $0x3eb655d7
+ WORD $0x3e885457
+ WORD $0x3e9254d7
+ WORD $0x3ea05557
+ WORD $0x3eb6d5d7
+ WORD $0x3e91d4d7
+ WORD $0x8297d4d7
+ WORD $0x3ea55557
+ WORD $0x82a7d557
+ WORD $0x3e83d457
+ WORD $0x8287d457
+ WORD $0x3eb755d7
+ WORD $0x82b7d5d7
+ WORD $0x00f507b3
+ WORD $0x01078813
+ WORD $0x020864a7
+ WORD $0x02078813
+ WORD $0x02086527
+ WORD $0x01068893
+ WORD $0x0207e427
+ WORD $0x03078793
+ WORD $0x0207e5a7
+ WORD $0xc71654e3
+ WORD $0xc22027f3
+ WORD $0x00c00813
+ WORD $0x030787b3
+ WORD $0x00f10133
+ WORD $0x01010113
+ WORD $0x06c6d063
+ WORD $0xd03777d3
WORD $0x0d0077d7
WORD $0x42006457
WORD $0x00000813
@@ -2161,14 +3273,265 @@ TEXT ·movingAverageFloat64RVV(SB), NOSPLIT|NOFRAME, $0-56
MOV dst_len+8(FP), X12
MOV a_len+32(FP), X13
MOV width+48(FP), X14
- WORD $0x06e05c63
- WORD $0x06e6ca63
+ WORD $0x46e05263
+ WORD $0x46e6c063
WORD $0x40e686b3
WORD $0x00c6d463
WORD $0x00168613
- WORD $0x06c05263
+ WORD $0x01000893
+ WORD $0x01165863
+ WORD $0x00000693
+ WORD $0x3ec04463
+ WORD $0x4400006f
+ WORD $0xff010113
+ WORD $0xc22026f3
+ WORD $0x00c00793
+ WORD $0x02f686b3
+ WORD $0x40d10133
WORD $0x00000693
WORD $0xd23777d3
+ WORD $0x00000813
+ WORD $0x00369793
+ WORD $0x00088693
+ WORD $0x00f588b3
+ WORD $0x0d8072d7
+ WORD $0x42006457
+ WORD $0x9e80b557
+ WORD $0xc22022f3
+ WORD $0x00a00313
+ WORD $0x026282b3
+ WORD $0x005102b3
+ WORD $0x01028293
+ WORD $0x22828427
+ WORD $0xc22022f3
+ WORD $0x00329293
+ WORD $0x005102b3
+ WORD $0x01028293
+ WORD $0x22828427
+ WORD $0x9e80b857
+ WORD $0x9e80b957
+ WORD $0x9e80ba57
+ WORD $0x9e80bb57
+ WORD $0x9e80bc57
+ WORD $0x9e80bd57
+ WORD $0x9e80be57
+ WORD $0x9e80bf57
+ WORD $0x9e80b357
+ WORD $0x9e80b257
+ WORD $0x9e80b157
+ WORD $0x9e80b057
+ WORD $0x9e80b657
+ WORD $0x00070293
+ WORD $0x0d92f357
+ WORD $0x00381393
+ WORD $0x007883b3
+ WORD $0x0203f707
+ WORD $0x01038e13
+ WORD $0x020e7407
+ WORD $0xc2202e73
+ WORD $0x001e1e13
+ WORD $0x01c10e33
+ WORD $0x010e0e13
+ WORD $0x228e0427
+ WORD $0x02038e13
+ WORD $0x020e7407
+ WORD $0x01010e13
+ WORD $0x228e0427
+ WORD $0x07838e13
+ WORD $0x020e7407
+ WORD $0xc2202e73
+ WORD $0x00600e93
+ WORD $0x03de0e33
+ WORD $0x01c10e33
+ WORD $0x010e0e13
+ WORD $0x228e0427
+ WORD $0x07038e13
+ WORD $0x0ee61657
+ WORD $0xc2202ef3
+ WORD $0x002e9e93
+ WORD $0x01d10eb3
+ WORD $0x010e8e93
+ WORD $0x228e8627
+ WORD $0x020e7407
+ WORD $0x06838e13
+ WORD $0xc2202ef3
+ WORD $0x00600f13
+ WORD $0x03ee8eb3
+ WORD $0x01d10eb3
+ WORD $0x010e8e93
+ WORD $0x228e8607
+ WORD $0x0ec51557
+ WORD $0xc2202ef3
+ WORD $0x00600f13
+ WORD $0x03ee8eb3
+ WORD $0x01d10eb3
+ WORD $0x010e8e93
+ WORD $0x228e8527
+ WORD $0x020e7507
+ WORD $0x06038e13
+ WORD $0x9e003657
+ WORD $0x9e203057
+ WORD $0x9e403157
+ WORD $0x9e603257
+ WORD $0x9fe03357
+ WORD $0x9fc03f57
+ WORD $0x9fa03e57
+ WORD $0x9f803d57
+ WORD $0x9f603c57
+ WORD $0xc2202ef3
+ WORD $0x00a00f13
+ WORD $0x03ee8eb3
+ WORD $0x01d10eb3
+ WORD $0x010e8e93
+ WORD $0x228e8707
+ WORD $0x0e871757
+ WORD $0xc2202ef3
+ WORD $0x00a00f13
+ WORD $0x03ee8eb3
+ WORD $0x01d10eb3
+ WORD $0x010e8e93
+ WORD $0x228e8727
+ WORD $0xc2202ef3
+ WORD $0x003e9e93
+ WORD $0x01d10eb3
+ WORD $0x010e8e93
+ WORD $0x228e8707
+ WORD $0x9f803b57
+ WORD $0x9fa03c57
+ WORD $0x9fc03d57
+ WORD $0x9fe03e57
+ WORD $0x9e603f57
+ WORD $0x9e403357
+ WORD $0x9e203257
+ WORD $0x9e003157
+ WORD $0x9ec03057
+ WORD $0xc2202ef3
+ WORD $0x002e9e93
+ WORD $0x01d10eb3
+ WORD $0x010e8e93
+ WORD $0x228e8607
+ WORD $0x020e7407
+ WORD $0x05838e13
+ WORD $0x0ea71757
+ WORD $0xc2202ef3
+ WORD $0x003e9e93
+ WORD $0x01d10eb3
+ WORD $0x010e8e93
+ WORD $0x228e8727
+ WORD $0x020e7507
+ WORD $0x05038e13
+ WORD $0x0e881857
+ WORD $0x020e7407
+ WORD $0x04838e13
+ WORD $0x0ea91957
+ WORD $0x020e7507
+ WORD $0x04038e13
+ WORD $0x0e8a1a57
+ WORD $0x020e7407
+ WORD $0x03838e13
+ WORD $0x0eab1b57
+ WORD $0x020e7507
+ WORD $0x03038e13
+ WORD $0x0e8c1c57
+ WORD $0x020e7407
+ WORD $0x02838e13
+ WORD $0x0ead1d57
+ WORD $0x020e7507
+ WORD $0x00838e13
+ WORD $0x01838393
+ WORD $0x0e8e1e57
+ WORD $0x0203f407
+ WORD $0x0eaf1f57
+ WORD $0x020e7507
+ WORD $0x01010393
+ WORD $0x22838707
+ WORD $0x0ee31357
+ WORD $0x0e821257
+ WORD $0xc22023f3
+ WORD $0x00139393
+ WORD $0x007103b3
+ WORD $0x01038393
+ WORD $0x22838407
+ WORD $0x0e811157
+ WORD $0x0ea01057
+ WORD $0xc22023f3
+ WORD $0x00600e13
+ WORD $0x03c383b3
+ WORD $0x007103b3
+ WORD $0x01038393
+ WORD $0x22838507
+ WORD $0x406282b3
+ WORD $0x00680833
+ WORD $0xdc0298e3
+ WORD $0x42a01757
+ WORD $0xc2202873
+ WORD $0x00a00893
+ WORD $0x03180833
+ WORD $0x01010833
+ WORD $0x01080813
+ WORD $0x22880407
+ WORD $0x428016d7
+ WORD $0xc2202873
+ WORD $0x00381813
+ WORD $0x01010833
+ WORD $0x01080813
+ WORD $0x22880407
+ WORD $0x42801657
+ WORD $0x430015d7
+ WORD $0x43201557
+ WORD $0x43401057
+ WORD $0x436010d7
+ WORD $0x43801157
+ WORD $0x43a011d7
+ WORD $0x43c01257
+ WORD $0x43e012d7
+ WORD $0x42601357
+ WORD $0x424013d7
+ WORD $0x42201857
+ WORD $0x420018d7
+ WORD $0x42c01e57
+ WORD $0x00f507b3
+ WORD $0x01068893
+ WORD $0x1afe7e53
+ WORD $0x1af8f8d3
+ WORD $0x1af87853
+ WORD $0x1af3f3d3
+ WORD $0x1af37353
+ WORD $0x1af2f2d3
+ WORD $0x1af27253
+ WORD $0x1af1f1d3
+ WORD $0x1af17153
+ WORD $0x1af0f0d3
+ WORD $0x1af07053
+ WORD $0x1af57553
+ WORD $0x1af5f5d3
+ WORD $0x1af67653
+ WORD $0x1af6f6d3
+ WORD $0x1af77753
+ WORD $0x01c7b027
+ WORD $0x0117b427
+ WORD $0x0107b827
+ WORD $0x0077bc27
+ WORD $0x0267b027
+ WORD $0x0257b427
+ WORD $0x0247b827
+ WORD $0x0237bc27
+ WORD $0x0427b027
+ WORD $0x0417b427
+ WORD $0x0407b827
+ WORD $0x04a7bc27
+ WORD $0x06b7b027
+ WORD $0x06c7b427
+ WORD $0x06d7b827
+ WORD $0x06e7bc27
+ WORD $0xc5165ce3
+ WORD $0xc22027f3
+ WORD $0x00c00813
+ WORD $0x030787b3
+ WORD $0x00f10133
+ WORD $0x01010113
+ WORD $0x06c6d063
+ WORD $0xd23777d3
WORD $0x0d8077d7
WORD $0x42006457
WORD $0x00000813
diff --git a/internal/tests/docs/kernels_guide_test.go b/internal/tests/docs/kernels_guide_test.go
new file mode 100644
index 0000000..33457be
--- /dev/null
+++ b/internal/tests/docs/kernels_guide_test.go
@@ -0,0 +1,171 @@
+package docs
+
+import (
+ "go/ast"
+ "go/parser"
+ "go/token"
+ "os"
+ "path/filepath"
+ "regexp"
+ "strings"
+ "testing"
+)
+
+// docs/kernels.md's worked example names things that have to exist.
+//
+// The guide is the answer to "the operation I need isn't in the 403", and its
+// centre is one `spec.Kernel` literal with eight fields, each annotated with
+// what it is for. A reader copies that literal. If a field is renamed the
+// guide goes on naming the old one, and the person following it gets a
+// compile error in generated code they did not write and cannot easily read
+// -- which is the moment the guide exists to prevent.
+//
+// The link gate and the test-name gate already cover this document. Neither
+// looks inside a code block.
+//
+// THE FIELD LIST IS READ FROM THE DOCUMENT, not written here. A list in this
+// file would have to be kept in step with the guide by hand, which is the
+// thing being gated, one level up.
+func TestTheKernelGuidesManifestFieldsExist(t *testing.T) {
+ doc := readDoc(t, filepath.Join("docs", "kernels.md"))
+
+ // The one block that shows a manifest entry, from `spec.Kernel{` to the
+ // fence that ends it.
+ block := regexp.MustCompile("(?s)```go\\s*\nspec\\.Kernel\\{(.*?)\n```").FindStringSubmatch(doc)
+ if block == nil {
+ t.Fatal("docs/kernels.md no longer contains a ```go block starting " +
+ "`spec.Kernel{`. That literal is the guide's worked example and " +
+ "this gate reads it; if the example moved, repoint the gate rather " +
+ "than leaving it green over nothing")
+ }
+
+ key := regexp.MustCompile(`(?m)^\s*([A-Z][A-Za-z0-9]*):`)
+ var named []string
+ for _, m := range key.FindAllStringSubmatch(block[1], -1) {
+ named = append(named, m[1])
+ }
+ if len(named) < 5 {
+ t.Fatalf("the worked example names %d fields (%v); it documented eight, "+
+ "so the extraction has lost the block rather than the block having "+
+ "shrunk", len(named), named)
+ }
+
+ fields := structFields(t, filepath.Join("tools", "simdgen", "spec", "spec.go"), "Kernel")
+ if len(fields) < 5 {
+ t.Fatalf("spec.Kernel parsed to %d fields; the parse has lost the "+
+ "struct", len(fields))
+ }
+ for _, n := range named {
+ if !fields[n] {
+ t.Errorf("docs/kernels.md's worked example sets spec.Kernel.%s, "+
+ "which the struct does not declare. Someone following the guide "+
+ "writes a manifest entry that does not compile", n)
+ }
+ }
+
+ // AND THE TWO HAND-WRITTEN WIRINGS THE EXAMPLE POINTS AT.
+ //
+ // `RefFunc` and `Field` are the two files `docs/plans` wants generated
+ // precisely because they are the two easiest things to forget when adding a
+ // kernel -- forgetting the ref wiring shipped past the whole normal lane
+ // once. The guide names a REAL kernel for its example, so both ends of it
+ // can be checked rather than described.
+ lit := regexp.MustCompile(`(?m)^\s*(RefFunc|Field):\s*"([A-Za-z0-9]+)"`)
+ seen := 0
+ for _, m := range lit.FindAllStringSubmatch(block[1], -1) {
+ seen++
+ switch m[1] {
+ case "RefFunc":
+ if !refExports(t)[m[2]] {
+ t.Errorf("the guide's example declares RefFunc %q, which "+
+ "internal/ref does not export. The guard would fall back to "+
+ "a function that does not exist", m[2])
+ }
+ case "Field":
+ if !kernelOpsFields(t)[m[2]] {
+ t.Errorf("the guide's example declares Field %q, which "+
+ "internal/kernel does not declare on any kernel group. That "+
+ "is the wiring a new kernel is most often missing", m[2])
+ }
+ }
+ }
+ if seen != 2 {
+ t.Fatalf("found %d of the example's RefFunc/Field literals, want 2; "+
+ "the two assertions above then covered less than they claim", seen)
+ }
+}
+
+// structFields is the field names of one struct in one file, multi-name
+// declarations (`Group, Field string`) included -- which is how two of the
+// eight are declared, so a parser that took one name per line would report
+// the guide as wrong about fields that exist.
+//
+// Parsed rather than imported: `tools/` is a separate module and must never
+// become a consumer dependency, which is a rule in CLAUDE.md.
+func structFields(t *testing.T, rel, name string) map[string]bool {
+ t.Helper()
+ f, err := parser.ParseFile(token.NewFileSet(), path(rel), nil, 0)
+ if err != nil {
+ t.Fatalf("parsing %s: %v", rel, err)
+ }
+ out := map[string]bool{}
+ ast.Inspect(f, func(n ast.Node) bool {
+ ts, ok := n.(*ast.TypeSpec)
+ if !ok || ts.Name.Name != name {
+ return true
+ }
+ st, ok := ts.Type.(*ast.StructType)
+ if !ok {
+ return true
+ }
+ for _, fld := range st.Fields.List {
+ for _, id := range fld.Names {
+ out[id.Name] = true
+ }
+ }
+ return false
+ })
+ return out
+}
+
+// refExports is every exported function internal/ref declares.
+func refExports(t *testing.T) map[string]bool {
+ t.Helper()
+ return declaredIn(t, filepath.Join("internal", "ref"),
+ regexp.MustCompile(`(?m)^func ([A-Z][A-Za-z0-9]*)`))
+}
+
+// kernelOpsFields is every function-typed field on any group in
+// internal/kernel -- the `Field` half of a manifest entry.
+func kernelOpsFields(t *testing.T) map[string]bool {
+ t.Helper()
+ return declaredIn(t, filepath.Join("internal", "kernel"),
+ regexp.MustCompile(`(?m)^\t([A-Z][A-Za-z0-9]*)\s+func\(`))
+}
+
+func declaredIn(t *testing.T, dir string, re *regexp.Regexp) map[string]bool {
+ t.Helper()
+ out := map[string]bool{}
+ matches, err := filepath.Glob(filepath.Join(path(dir), "*.go"))
+ if err != nil {
+ t.Fatal(err)
+ }
+ for _, p := range matches {
+ if strings.HasSuffix(p, "_test.go") {
+ continue
+ }
+ b, err := os.ReadFile(p)
+ if err != nil {
+ t.Fatal(err)
+ }
+ for _, m := range re.FindAllStringSubmatch(string(b), -1) {
+ out[m[1]] = true
+ }
+ }
+ if len(out) == 0 {
+ t.Fatalf("no declarations matched in %s; the pattern has stopped "+
+ "finding them and every lookup against this set would pass or fail "+
+ "for the wrong reason", dir)
+ }
+ return out
+}
diff --git a/internal/tests/docs/test_names_test.go b/internal/tests/docs/test_names_test.go
new file mode 100644
index 0000000..b751aed
--- /dev/null
+++ b/internal/tests/docs/test_names_test.go
@@ -0,0 +1,105 @@
+package docs
+
+import (
+ "os"
+ "path/filepath"
+ "regexp"
+ "strings"
+ "testing"
+)
+
+// A document naming a test is claiming a guarantee is enforced.
+//
+// It is the same promise as a link and it rots the same way: a test is
+// renamed, the document goes on naming the old one, and a reader who greps
+// for it finds nothing and cannot tell whether the guarantee was dropped or
+// the name moved. Both are worse than no sentence.
+//
+// The subject is activeDocs — the set links_test.go checks, for the reason
+// stated there. CHANGELOG.md, docs/wrong.md, docs/plans and docs/research are
+// historical or external: a changelog entry describing what a release did is
+// accurate about that release even after a rename, and docs/research quotes
+// benchmark names from other people's blog posts (`BenchmarkNoopAsm` is
+// mmcloughlin's, not this repository's). Gating those would force history to
+// be rewritten to satisfy a checker, which is the thing docs/wrong.md exists
+// to prevent.
+func TestActiveDocsNameRealTests(t *testing.T) {
+ declared := declaredTestFuncs(t)
+ if len(declared) == 0 {
+ t.Fatal("no test functions were found in the tree, so this gate measures nothing")
+ }
+
+ cited := regexp.MustCompile("`((?:Test|Fuzz|Benchmark|Example)[A-Za-z0-9_]+)`")
+ checked := 0
+ for _, doc := range testNameDocs() {
+ b, err := os.ReadFile(filepath.Join(repoRoot, doc))
+ if err != nil {
+ t.Fatalf("%s: %v", doc, err)
+ }
+ for i, line := range strings.Split(string(b), "\n") {
+ for _, m := range cited.FindAllStringSubmatch(line, -1) {
+ checked++
+ if !declared[m[1]] {
+ t.Errorf("%s:%d names %s, which no test in this repository "+
+ "declares: the guarantee on that line is prose",
+ doc, i+1, m[1])
+ }
+ }
+ }
+ }
+ if checked == 0 {
+ t.Fatal("no test names were cited in any active document, so this gate " +
+ "measures nothing -- if the citations were deliberately removed, " +
+ "delete this test rather than leaving it green over nothing")
+ }
+ t.Logf("%d citations across %d documents, all declared", checked, len(testNameDocs()))
+}
+
+// testNameDocs is activeDocs plus the two documents that describe the SUITE
+// itself, which is where most citations naturally are.
+//
+// Not by widening activeDocs: CLAUDE.md describes that set as what the LINK
+// gate covers -- README, CONTRIBUTING, ROADMAP and the main references and
+// guides -- and says the LLDs, plans and records are checked by hand. This
+// gate has its own subject rather than quietly changing the meaning of that
+// sentence. The LLD citations were checked by hand on 2026-08-16 and all
+// resolve.
+func testNameDocs() []string {
+ return append(append([]string(nil), activeDocs...),
+ "docs/verification.md",
+ "internal/tests/README.md",
+ )
+}
+
+// declaredTestFuncs is every Test/Fuzz/Benchmark/Example function in the tree.
+func declaredTestFuncs(t *testing.T) map[string]bool {
+ t.Helper()
+ decl := regexp.MustCompile(`(?m)^func ((?:Test|Fuzz|Benchmark|Example)[A-Za-z0-9_]*)\(`)
+ out := map[string]bool{}
+ err := filepath.WalkDir(repoRoot, func(path string, d os.DirEntry, err error) error {
+ if err != nil {
+ return err
+ }
+ if d.IsDir() {
+ if n := d.Name(); n == ".git" || n == "node_modules" {
+ return filepath.SkipDir
+ }
+ return nil
+ }
+ if !strings.HasSuffix(path, "_test.go") {
+ return nil
+ }
+ b, err := os.ReadFile(path)
+ if err != nil {
+ return err
+ }
+ for _, m := range decl.FindAllStringSubmatch(string(b), -1) {
+ out[m[1]] = true
+ }
+ return nil
+ })
+ if err != nil {
+ t.Fatal(err)
+ }
+ return out
+}
diff --git a/tools/simdgen/emit/dispatch_gen.go b/tools/simdgen/emit/dispatch_gen.go
index 0782a1d..8148f5e 100644
--- a/tools/simdgen/emit/dispatch_gen.go
+++ b/tools/simdgen/emit/dispatch_gen.go
@@ -62,6 +62,21 @@ func sigTypes(k spec.Kernel) string {
return sig
}
+// complexElem maps a complex group name to its Go element type, and
+// partsElem the split groups that pair a complex type with its real one.
+//
+// These were absent, and their absence was the whole defect: numericGroupsIn
+// skipped every complex group, so no per-tier table was emitted for them and
+// complexOps had nothing to overlay from. Nine tiers of generated complex
+// assembly across six architectures were linked into the test-only Sets()
+// aggregator and reachable from nothing a consumer calls.
+var complexElem = map[string]string{"C64": "complex64", "C128": "complex128"}
+
+var partsElem = map[string][2]string{
+ "C64Parts": {"complex64", "float32"},
+ "C128Parts": {"complex128", "float64"},
+}
+
// numericElem maps a numeric group name to its Go element type.
var numericElem = map[string]string{
"F32": "float32", "F64": "float64",
@@ -142,6 +157,34 @@ func RootDispatch(arch string, tiers []string, present map[string]map[string]boo
fmt.Fprintf(&b, "}\n\n")
}
+ // Complex groups, the same shape over kernel.Complex and
+ // kernel.ComplexParts. Kept separate from the numeric loop because the
+ // struct type differs and ComplexParts carries two type parameters.
+ for _, g := range groupsIn(all, func(name string) bool { _, ok := complexElem[name]; return ok }) {
+ emitTierTable(&b, arch, tiers, present, all, g,
+ "kernel.Complex["+complexElem[g]+"]", "cplx"+g)
+ }
+ for _, g := range groupsIn(all, func(name string) bool { _, ok := partsElem[name]; return ok }) {
+ e := partsElem[g]
+ emitTierTable(&b, arch, tiers, present, all, g,
+ "kernel.ComplexParts["+e[0]+", "+e[1]+"]", "parts"+g)
+ }
+
+ // Every group in the manifest must have produced a table. A group this
+ // file does not know about silently falls back to the reference on every
+ // machine, with every test green -- which is exactly what happened to
+ // C64/C128/C64Parts/C128Parts for the whole life of the complex kernels.
+ // The allowlists above are the same shape that let it happen, so the
+ // check is here rather than in a test: a generator that cannot dispatch
+ // a group must fail loudly instead of emitting a working-looking file.
+ if missing := ungrouped(all); len(missing) > 0 {
+ panic(fmt.Sprintf("simdgen: no dispatch table for kernel group(s) %v.\n"+
+ "Add the group to numericElem, complexElem, partsElem or flatGroups in "+
+ "dispatch_gen.go. Without an entry the group compiles, passes every "+
+ "test, and runs the portable reference on every architecture.",
+ missing))
+ }
+
out := b.String()
if formatted, err := format.Source([]byte(out)); err == nil {
return string(formatted)
@@ -149,11 +192,78 @@ func RootDispatch(arch string, tiers []string, present map[string]map[string]boo
return out
}
+// ungrouped returns the manifest groups no table is emitted for, sorted.
+func ungrouped(all []spec.Kernel) []string {
+ bad := map[string]bool{}
+ for _, k := range all {
+ if flatGroups[k.Group] {
+ continue
+ }
+ if _, ok := numericElem[k.Group]; ok {
+ continue
+ }
+ if _, ok := complexElem[k.Group]; ok {
+ continue
+ }
+ if _, ok := partsElem[k.Group]; ok {
+ continue
+ }
+ bad[k.Group] = true
+ }
+ out := make([]string, 0, len(bad))
+ for g := range bad {
+ out = append(out, g)
+ }
+ sort.Strings(out)
+ return out
+}
+
// exportedGuard mirrors guardName: the exported threshold wrapper.
func exportedGuard(k spec.Kernel) string {
return strings.ToUpper(k.GoName[:1]) + k.GoName[1:]
}
+// emitTierTable writes one partial struct per tier plus the tier-indexed
+// pointer array the dispatcher overlays from, for any group whose kernel.Set
+// field is a struct of function fields. Slot 0 is nil: the reference base
+// needs no overlay.
+func emitTierTable(b *strings.Builder, arch string, tiers []string,
+ present map[string]map[string]bool, all []spec.Kernel, group, typ, prefix string) {
+
+ for _, t := range tiers {
+ suffix := strings.ToUpper(t)
+ fmt.Fprintf(b, "var %s%s = %s{\n", prefix, suffix, typ)
+ for _, k := range all {
+ if k.Group != group || !present[t][k.CName] {
+ continue
+ }
+ fmt.Fprintf(b, "\t%s: %s.%s%s,\n", k.Field, arch, exportedGuard(k), suffix)
+ }
+ fmt.Fprintf(b, "}\n\n")
+ }
+ fmt.Fprintf(b, "var %sByTier = [...]*%s{\n\tnil,\n", prefix, typ)
+ for _, t := range tiers {
+ fmt.Fprintf(b, "\t&%s%s,\n", prefix, strings.ToUpper(t))
+ }
+ fmt.Fprintf(b, "}\n\n")
+}
+
+// groupsIn returns the sorted group names in all that keep matches true.
+func groupsIn(all []spec.Kernel, keep func(string) bool) []string {
+ seen := map[string]bool{}
+ for _, k := range all {
+ if keep(k.Group) {
+ seen[k.Group] = true
+ }
+ }
+ out := make([]string, 0, len(seen))
+ for g := range seen {
+ out = append(out, g)
+ }
+ sort.Strings(out)
+ return out
+}
+
func numericGroupsIn(all []spec.Kernel) []string {
seen := map[string]bool{}
for _, k := range all {
@@ -197,6 +307,13 @@ func RootDispatchFallback(arches []string, all []spec.Kernel, prov Provenance) s
for _, g := range numericGroupsIn(all) {
fmt.Fprintf(&b, "var ops%sByTier = [...]*kernel.Ops[%s]{nil}\n", g, numericElem[g])
}
+ for _, g := range groupsIn(all, func(n string) bool { _, ok := complexElem[n]; return ok }) {
+ fmt.Fprintf(&b, "var cplx%sByTier = [...]*kernel.Complex[%s]{nil}\n", g, complexElem[g])
+ }
+ for _, g := range groupsIn(all, func(n string) bool { _, ok := partsElem[n]; return ok }) {
+ e := partsElem[g]
+ fmt.Fprintf(&b, "var parts%sByTier = [...]*kernel.ComplexParts[%s, %s]{nil}\n", g, e[0], e[1])
+ }
out := b.String()
if formatted, err := format.Source([]byte(out)); err == nil {
@@ -284,6 +401,35 @@ func RootDispatchTest(arch string, tiers []string, all []spec.Kernel, prov Prove
fmt.Fprintf(&b, "},\n")
}
fmt.Fprintf(&b, "}\n\n")
+
+ // The struct groups -- numeric and complex -- reach the runtime through
+ // per-tier partials, not per-operation tables, so allFlatTables cannot
+ // name them and the inventory walk fell back to archSets(), the test-only
+ // aggregator. That is why four whole complex groups could be absent from
+ // the runtime tables with the suite green. These expose the partials the
+ // runtime actually indexes, keyed by group, so a test can walk them.
+ fmt.Fprintf(&b, "// allGroupTables enumerates the per-tier partial structs the runtime\n")
+ fmt.Fprintf(&b, "// overlays from, keyed by kernel.Set group. Slot 0 is the reference\n")
+ fmt.Fprintf(&b, "// and is nil; the rest are one per accelerated tier, in tier order.\n")
+ fmt.Fprintf(&b, "var allGroupTables = map[string][]any{\n")
+ emitGroupRow := func(group, prefix string) {
+ fmt.Fprintf(&b, "\t%q: {nil", group)
+ for _, t := range tiers {
+ fmt.Fprintf(&b, ", %s%s", prefix, strings.ToUpper(t))
+ }
+ fmt.Fprintf(&b, "},\n")
+ }
+ for _, g := range numericGroupsIn(all) {
+ emitGroupRow(g, "ops"+g)
+ }
+ for _, g := range groupsIn(all, func(n string) bool { _, ok := complexElem[n]; return ok }) {
+ emitGroupRow(g, "cplx"+g)
+ }
+ for _, g := range groupsIn(all, func(n string) bool { _, ok := partsElem[n]; return ok }) {
+ emitGroupRow(g, "parts"+g)
+ }
+ fmt.Fprintf(&b, "}\n\n")
+
fmt.Fprintf(&b, "// archSets hands the tests every tier's complete kernel set.\n")
fmt.Fprintf(&b, "var archSets func() map[string]kernel.Set = %s.Sets\n", arch)
@@ -315,6 +461,9 @@ func RootDispatchFallbackTest(arches []string, all []spec.Kernel, prov Provenanc
fmt.Fprintf(&b, "\t%q: {%s[0]},\n", k.Group+"."+k.Field, TableName(k))
}
fmt.Fprintf(&b, "}\n\n")
+ fmt.Fprintf(&b, "// No generated backend here: every group table is the reference slot\n")
+ fmt.Fprintf(&b, "// alone, so the walk has nothing to check.\n")
+ fmt.Fprintf(&b, "var allGroupTables = map[string][]any{}\n\n")
fmt.Fprintf(&b, "// No generated backend here: the numeric half of the inventory walk\n")
fmt.Fprintf(&b, "// has nothing to check and skips on nil.\n")
fmt.Fprintf(&b, "var archSets func() map[string]kernel.Set\n")
diff --git a/tools/simdgen/emit/ungrouped_test.go b/tools/simdgen/emit/ungrouped_test.go
new file mode 100644
index 0000000..9028572
--- /dev/null
+++ b/tools/simdgen/emit/ungrouped_test.go
@@ -0,0 +1,26 @@
+package emit
+
+import (
+ "testing"
+
+ "github.com/sebishogun/simd/tools/simdgen/spec"
+)
+
+// A group with no dispatch table falls back to the reference on every
+// machine with every test green. The generator must refuse to emit rather
+// than produce a working-looking file -- this is the check that would have
+// caught the complex groups.
+func TestUngroupedGroupIsRefused(t *testing.T) {
+ if got := ungrouped(nil); len(got) != 0 {
+ t.Fatalf("an empty manifest reported %v", got)
+ }
+ for _, g := range []string{"F32", "C128", "C64Parts", "Bytes"} {
+ if got := ungrouped([]spec.Kernel{{Group: g}}); len(got) != 0 {
+ t.Errorf("known group %s reported as ungrouped: %v", g, got)
+ }
+ }
+ got := ungrouped([]spec.Kernel{{Group: "C32"}, {Group: "F32"}})
+ if len(got) != 1 || got[0] != "C32" {
+ t.Fatalf("ungrouped = %v, want [C32]", got)
+ }
+}
diff --git a/tools/simdgen/main.go b/tools/simdgen/main.go
index c1a36a2..96575c3 100644
--- a/tools/simdgen/main.go
+++ b/tools/simdgen/main.go
@@ -285,6 +285,19 @@ func build(cc *compile.Clang, src kernels.Source, tgt target.Target, root, outDi
if verbose {
fmt.Printf(" %s\n", r.Summary())
}
+ // A kernel that does arithmetic and none of it in lanes is printed
+ // whether or not the run is verbose, because it is exactly what
+ // RequireVector exists to catch and exactly what it cannot see: on
+ // amd64 every scalar float instruction lives in %xmm, so the "does any
+ // instruction touch a vector register" test passes for scalar code.
+ // Eight kernels ship this way today -- convolve, correlate, movavg and
+ // polyeval in both precisions -- and docs/wrong.md entry 76 has the
+ // disassembly. Not fatal and not dropped: falling back to internal/ref
+ // instead is a behaviour change that wants a benchmark first.
+ if r.ScalarOnly() {
+ fmt.Printf(" SCALAR-ONLY %s: %d scalar arithmetic instructions, "+
+ "no packed ones\n", r.Func, r.ScalarArith)
+ }
for _, p := range r.Problems {
problems = append(problems, fmt.Sprintf("%s: %s", r.Func, p))
}
diff --git a/tools/simdgen/verify/arith_test.go b/tools/simdgen/verify/arith_test.go
new file mode 100644
index 0000000..5e5f99f
--- /dev/null
+++ b/tools/simdgen/verify/arith_test.go
@@ -0,0 +1,108 @@
+package verify
+
+import (
+ "testing"
+
+ "github.com/sebishogun/simd/tools/simdgen/target"
+)
+
+// arithKind separates work from moves, and packed work from scalar work.
+//
+// The mnemonics below are copied from llvm-objdump output for kernels in this
+// repository, not invented: the amd64 scalar cases are what simd_convolve_f32
+// actually contains at sse2 and avx2, and the "neither" cases are the two
+// instructions that made it pass as vectorized.
+func TestArithKindSeparatesPackedFromScalarAndFromMoves(t *testing.T) {
+ for _, tc := range []struct {
+ arch target.Arch
+ mnemonic string
+ operands string
+ want arithForm
+ }{
+ // amd64, scalar: one lane, in an %xmm register, which is why
+ // vectorWidth counts them as vector.
+ {target.AMD64, "mulss", "(%rsi), %xmm0", arithScalar},
+ {target.AMD64, "addss", "%xmm1, %xmm0", arithScalar},
+ {target.AMD64, "vmulss", "%xmm2, %xmm1, %xmm0", arithScalar},
+ {target.AMD64, "vaddss", "%xmm2, %xmm1, %xmm0", arithScalar},
+ {target.AMD64, "mulsd", "%xmm1, %xmm0", arithScalar},
+ {target.AMD64, "vfmadd213sd", "%xmm2, %xmm1, %xmm0", arithScalar},
+ {target.AMD64, "divss", "%xmm1, %xmm0", arithScalar},
+ {target.AMD64, "maxsd", "%xmm1, %xmm0", arithScalar},
+
+ // amd64, packed.
+ {target.AMD64, "mulps", "%xmm1, %xmm0", arithPacked},
+ {target.AMD64, "vaddps", "%ymm2, %ymm1, %ymm0", arithPacked},
+ {target.AMD64, "vfmadd213ps", "%zmm2, %zmm1, %zmm0", arithPacked},
+ {target.AMD64, "vmulpd", "%ymm2, %ymm1, %ymm0", arithPacked},
+ {target.AMD64, "paddd", "%xmm1, %xmm0", arithPacked},
+ {target.AMD64, "vpmulld", "%ymm2, %ymm1, %ymm0", arithPacked},
+ {target.AMD64, "vpmaxsd", "%ymm2, %ymm1, %ymm0", arithPacked},
+ {target.AMD64, "vpsubb", "%xmm2, %xmm1, %xmm0", arithPacked},
+
+ // amd64, NEITHER. These two are the whole finding: they are the only
+ // vector-register instructions in simd_convolve_f32 at sse2, and
+ // counting them is what let a scalar kernel ship as accelerated.
+ {target.AMD64, "movups", "(%rdi), %xmm0", arithNone},
+ {target.AMD64, "xorps", "%xmm0, %xmm0", arithNone},
+ {target.AMD64, "vmovups", "(%rdi), %ymm0", arithNone},
+ {target.AMD64, "vmovss", "(%rsi), %xmm0", arithNone},
+ {target.AMD64, "movq", "%rax, %rcx", arithNone},
+ {target.AMD64, "vshufps", "$0, %xmm0, %xmm0, %xmm0", arithNone},
+
+ // arm64: the OPERAND decides, because `fmul` spells both forms.
+ {target.ARM64, "fmul", "v0.4s, v1.4s, v2.4s", arithPacked},
+ {target.ARM64, "fadd", "v0.2d, v1.2d, v2.2d", arithPacked},
+ {target.ARM64, "fmul", "s0, s1, s2", arithScalar},
+ {target.ARM64, "fadd", "d0, d1, d2", arithScalar},
+ {target.ARM64, "fmul", "z0.s, p0/m, z0.s, z1.s", arithPacked},
+ {target.ARM64, "ldr", "q0, [x0]", arithNone},
+ {target.ARM64, "mov", "v0.16b, v1.16b", arithNone},
+
+ // An architecture the classifier does not claim: everything is
+ // unclassified, and ArithKnown is what says so.
+ {target.PPC64LE, "xvmuldp", "vs0, vs1, vs2", arithNone},
+ {target.S390X, "vfmdb", "%v0, %v1, %v2", arithNone},
+ } {
+ got := arithKind(Instr{Mnemonic: tc.mnemonic, Operands: tc.operands}, tc.arch)
+ if got != tc.want {
+ t.Errorf("%s %s %q = %v, want %v", tc.arch, tc.mnemonic, tc.operands, got, tc.want)
+ }
+ }
+}
+
+// ScalarOnly is false where the classifier does not run, so a zero count is
+// never read as a finding.
+//
+// Without this, adding an architecture to the report and forgetting to add it
+// to arithKind would mark every one of its kernels scalar-only -- a check that
+// fires everywhere is as useless as one that fires nowhere, and this one would
+// have been noise on four of the six architectures.
+func TestScalarOnlyIsSilentWhereTheClassifierDoesNotRun(t *testing.T) {
+ for _, arch := range []target.Arch{target.AMD64, target.ARM64, target.PPC64LE,
+ target.S390X, target.RISCV64, target.LOONG64} {
+ known := arithClassified(arch)
+ r := Report{ArithKnown: known, ScalarArith: 10, PackedArith: 0}
+ if got := r.ScalarOnly(); got != known {
+ t.Errorf("%s: ScalarOnly()=%v with ArithKnown=%v; an unclassified "+
+ "architecture must not report a finding it did not measure", arch, got, known)
+ }
+ }
+ // And where it does run, the three states are distinct.
+ for _, tc := range []struct {
+ name string
+ packed, scalar int
+ want bool
+ }{
+ {"vectorized", 12, 0, false},
+ {"mixed", 12, 50, false},
+ {"scalar only", 0, 10, true},
+ {"a permute kernel: no arithmetic at all", 0, 0, false},
+ } {
+ r := Report{ArithKnown: true, PackedArith: tc.packed, ScalarArith: tc.scalar}
+ if got := r.ScalarOnly(); got != tc.want {
+ t.Errorf("%s (packed=%d scalar=%d) = %v, want %v",
+ tc.name, tc.packed, tc.scalar, got, tc.want)
+ }
+ }
+}
diff --git a/tools/simdgen/verify/verify.go b/tools/simdgen/verify/verify.go
index 0adf6ed..01c8e0e 100644
--- a/tools/simdgen/verify/verify.go
+++ b/tools/simdgen/verify/verify.go
@@ -66,6 +66,26 @@ type Report struct {
TotalInstrs int
// WidestVector is the widest vector register class seen, in bits.
WidestVector int
+ // PackedArith and ScalarArith count ARITHMETIC instructions by form.
+ //
+ // VectorInstrs above answers "does any instruction touch a vector
+ // register", which on amd64 every float instruction does: scalar SSE
+ // arithmetic lives in %xmm, so `vmulss %xmm0, %xmm1, %xmm2` counts.
+ // RequireVector therefore cannot see a scalar float kernel on the one
+ // architecture that runs natively here, and eight do ship that way --
+ // convolve, correlate, movavg and polyeval, in both precisions, with zero
+ // packed arithmetic at sse2, avx2 and avx512 alike. See docs/wrong.md
+ // entry 76.
+ //
+ // These two count the arithmetic itself, so "does work, none of it in
+ // lanes" is a question that can be asked. A permute kernel does no
+ // arithmetic at all and has both at zero, which is why the question is
+ // not "is PackedArith > 0".
+ //
+ // ArithKnown is false on architectures whose mnemonics this does not
+ // classify, so a zero there reads as "not measured" rather than "none".
+ PackedArith, ScalarArith int
+ ArithKnown bool
// BodyEnd is the byte offset of the function's return instruction, which
// is where the copied body must stop.
//
@@ -163,7 +183,7 @@ func ObjectWithDisasm(objPath string, tgt target.Target, funcs []string, opt Opt
}
func checkFunc(name string, instrs []Instr, tgt target.Target, opt Options) Report {
- r := Report{Func: name, TotalInstrs: len(instrs)}
+ r := Report{Func: name, TotalInstrs: len(instrs), ArithKnown: arithClassified(tgt.Arch)}
for _, in := range instrs {
if f, w := featureOf(in, tgt.Arch); f > r.MaxFeature {
@@ -176,6 +196,12 @@ func checkFunc(name string, instrs []Instr, tgt target.Target, opt Options) Repo
r.WidestVector = bits
}
}
+ switch kind := arithKind(in, tgt.Arch); kind {
+ case arithPacked:
+ r.PackedArith++
+ case arithScalar:
+ r.ScalarArith++
+ }
if n := stackAdjust(in, tgt.Arch); n > r.StackBytes {
r.StackBytes = n
}
@@ -715,6 +741,107 @@ func vectorWidth(in Instr, arch target.Arch) int {
return 0
}
+// ---------- arithmetic form ----------
+
+type arithForm int
+
+const (
+ arithNone arithForm = iota
+ arithPacked
+ arithScalar
+)
+
+// arithClassified reports whether arithKind understands this architecture.
+//
+// amd64 and arm64 only, deliberately. Both spell the difference in the
+// mnemonic or the operand and both are checkable here: amd64 is the only
+// architecture this repository runs natively, and arm64 is the one with real
+// hardware in the cross lane. On the rest, a count of zero would mean "this
+// function did not look", and reporting that as "no scalar arithmetic" is the
+// shape of a check that cannot fail.
+func arithClassified(arch target.Arch) bool {
+ return arch == target.AMD64 || arch == target.ARM64
+}
+
+var (
+ // amd64: the operation, then the form suffix. `ss`/`sd` are scalar single
+ // and scalar double -- one lane, in an %xmm register, which is why
+ // vectorWidth sees them as vector. `ps`/`pd` are packed. The integer
+ // family (`p*`) is packed by construction.
+ reX86FloatOp = regexp.MustCompile(`^v?(add|sub|mul|div|max|min|sqrt|rsqrt|rcp|cmp|fmadd|fmsub|fnmadd|fnmsub|and|andn|or|xor|round|scalef|getexp|getmant|range|reduce)[0-9a-z]*?(ss|sd|ps|pd)$`)
+ reX86PackedInt = regexp.MustCompile(`^v?p(add|sub|mul|mad|max|min|avg|abs|sad|and|or|xor|sll|srl|sra|cmp)`)
+
+ // The zeroing idiom. `xorps %xmm0, %xmm0` is how every x86 compiler
+ // produces a zero register, and it is not work -- simd_convolve_f32
+ // contains two of them at sse2 and three at avx2, and those, with the
+ // moves, are the entirety of what made it look vectorized. The bitwise
+ // family stays in the arithmetic set otherwise, because a mask-and-blend
+ // kernel does its work with `andps`/`orps` and counting that is right.
+ reX86Xor = regexp.MustCompile(`^v?p?xor(ps|pd)?$`)
+ reX86Reg = regexp.MustCompile(`%[a-z0-9]+`)
+
+ // arm64: the operand tells the form. `v0.4s` is a vector arrangement,
+ // `s0`/`d0` a single lane. The mnemonic alone cannot decide -- `fmul`
+ // spells both.
+ reA64Arith = regexp.MustCompile(`^(f?(add|sub|mul|div|max|min|abs|neg|sqrt|cmp|mla|mls|madd|msub|nmul)|s(add|mul|max|min)|u(add|mul|max|min)|and|orr|eor|bic|shl|ushr|sshr)`)
+ reA64Vector = regexp.MustCompile(`\bv[0-9]+\.[0-9]*[bhsdq]|\bz[0-9]+\.[bhsdq]`)
+ reA64Scalar = regexp.MustCompile(`\b[sdhq][0-9]+\b`)
+)
+
+// arithKind classifies one instruction as packed arithmetic, scalar
+// arithmetic, or neither.
+//
+// MOVES AND ZEROING ARE NEITHER, which is the whole point: simd_convolve_f32
+// at sse2 contains two `movups` and two `xorps` and no packed arithmetic at
+// all, and counting the moves is what let it pass as vectorized. A register
+// zero is not work.
+func arithKind(in Instr, arch target.Arch) arithForm {
+ m := in.Mnemonic
+ switch arch {
+ case target.AMD64:
+ if reX86Xor.MatchString(m) && sameRegisters(in.Operands) {
+ return arithNone
+ }
+ if reX86PackedInt.MatchString(m) {
+ return arithPacked
+ }
+ if g := reX86FloatOp.FindStringSubmatch(m); g != nil {
+ switch g[2] {
+ case "ss", "sd":
+ return arithScalar
+ default:
+ return arithPacked
+ }
+ }
+ case target.ARM64:
+ if !reA64Arith.MatchString(m) {
+ return arithNone
+ }
+ if reA64Vector.MatchString(in.Operands) {
+ return arithPacked
+ }
+ if reA64Scalar.MatchString(in.Operands) {
+ return arithScalar
+ }
+ }
+ return arithNone
+}
+
+// sameRegisters reports whether every register operand names the same
+// register, which for a xor means the instruction produces a zero.
+func sameRegisters(operands string) bool {
+ regs := reX86Reg.FindAllString(operands, -1)
+ if len(regs) < 2 {
+ return false
+ }
+ for _, r := range regs[1:] {
+ if r != regs[0] {
+ return false
+ }
+ }
+ return true
+}
+
// ---------- frame pointer ----------
var (
@@ -998,6 +1125,21 @@ func disassemble(objPath string, tgt target.Target, objdump string) (map[string]
// Summary renders a one-line description of a report, for the generator's log.
func (r Report) Summary() string {
- return fmt.Sprintf("%-28s %3d instrs, %3d vector (%d-bit), feature=%s, stack=%dB",
- r.Func, r.TotalInstrs, r.VectorInstrs, r.WidestVector, r.MaxFeature, r.StackBytes)
+ arith := ""
+ if r.ArithKnown {
+ arith = fmt.Sprintf(", arith=%d packed/%d scalar", r.PackedArith, r.ScalarArith)
+ }
+ return fmt.Sprintf("%-28s %3d instrs, %3d vector (%d-bit)%s, feature=%s, stack=%dB",
+ r.Func, r.TotalInstrs, r.VectorInstrs, r.WidestVector, arith, r.MaxFeature, r.StackBytes)
+}
+
+// ScalarOnly reports that this kernel does arithmetic and none of it is in
+// lanes -- a kernel that would be dispatched to as an acceleration and run one
+// element at a time.
+//
+// False when the architecture's mnemonics are not classified, because "no
+// packed arithmetic counted" and "no packed arithmetic present" are not the
+// same statement and only one of them is a finding.
+func (r Report) ScalarOnly() bool {
+ return r.ArithKnown && r.ScalarArith > 0 && r.PackedArith == 0
}