IR fuzzer produces postcard-encoded programs with out-of-bounds ShutdownScriptVariant values, panicking the executor

[erickcestari]

Running the IR fuzzer (against both the ldk and lnd targets) from a completely fresh state
(docker build --no-cache, fresh cargo build --release -p smite-ir-mutator,
empty /tmp/smite-out, single-byte seed) consistently produces crash inputs
within ~30 seconds whose postcard bytes decode into a Program containing a
LoadShutdownScript instruction with values outside the BOLT 2 / BIP 141
ranges enforced by ShutdownScriptVariant::encode. The executor then panics in
smite-ir/src/operation.rs when it tries to encode the variant.

Observed across two independent machines with fresh builds, and reproduced
on current master.

Observed panics

All in smite-ir/src/operation.rs:

• AnySegwit version 41 out of range
• AnySegwit version 45 out of range
• AnySegwit version 34 out of range
• OpReturn data length 111 out of range

Valid ranges (declared on ShutdownScriptVariant):

• AnySegwit.version: 1..=16
• AnySegwit.program.len(): 2..=40
• OpReturn.data.len(): 6..=80

Where the bytes come from

The AFL crash filename includes op:libsmite_ir_mutator.so,pos:0, i.e. AFL
attributes the crashing bytes to our custom mutator's output. With
AFL_CUSTOM_MUTATOR_ONLY=1 and afl_custom_splice_optout exported, AFL's
built-in byte mutations and splicing are disabled, so the bytes can only have
been produced by:

1. OpenChannelGenerator -> ShutdownScriptVariant::random(rng) -- bounded by
   the constants on the type. Verified by shutdown_script_random_respects_bounds.
2. OperationParamMutator::mutate_shutdown_script ->
   mutate_shutdown_script_bytes (length-preserving mutate_fixed_bytes on
   &mut [u8]; AnySegwit.version is never touched because the match arm
   binds AnySegwit { program, .. }), or a variant swap via random().
3. InputSwapMutator -- only swaps input indices, doesn't touch payloads.

By inspection none of these paths can produce the observed values. I also
hammered afl_custom_fuzz end-to-end through the FFI shim (the exact entry
point AFL loads) for 2,000,000 iterations, decoding every output and asserting
all LoadShutdownScript variants are in range. Zero violations. Yet the live
fuzzer reliably finds one in tens of thousands of execs.

So either:

• there is a code path I'm missing that produces these bytes, or
• something in the AFL + Nyx integration is corrupting the postcard stream
  between when our shim returns it and when the executor reads it.

Reproduction

export SCENARIO=ir
export TARGET=ldk

docker build -t smite-$TARGET-$SCENARIO -f workloads/$TARGET/Dockerfile \
    --build-arg SCENARIO=$SCENARIO . --no-cache

./scripts/setup-nyx.sh /tmp/smite-nyx smite-$TARGET-$SCENARIO ~/AFLplusplus

cargo build --release -p smite-ir-mutator

mkdir -p /tmp/smite-seeds
printf '\x00' > /tmp/smite-seeds/empty

AFL_CUSTOM_MUTATOR_LIBRARY=target/release/libsmite_ir_mutator.so \
AFL_CUSTOM_MUTATOR_ONLY=1 \
AFL_DISABLE_TRIM=1 \
    ~/AFLplusplus/afl-fuzz -X -i /tmp/smite-seeds -o /tmp/smite-out \
        -- /tmp/smite-nyx

# A crash typically appears within ~30 seconds.
# Pick one from /tmp/smite-out/default/crashes/ and replay it:

cp /tmp/smite-out/default/crashes/id:000000,sig:00,src:000008,\
time:12476,execs:27175,op:libsmite_ir_mutator.so,pos:0 ./crash

docker run --rm -v $PWD/crash:/input.bin -e SMITE_INPUT=/input.bin \
    smite-$TARGET-$SCENARIO /$TARGET-scenario

Expected output (truncated):

INFO  [smite::scenarios] Scenario initialized! Executing input...
INFO  [smite::runners] Reading input from "/input.bin"
DEBUG [smite_scenarios::scenarios::ir] [7.13us] Executing IR program (23 instructions, 321 input bytes)

thread 'main' (1) panicked at smite-ir/src/operation.rs:224:17:
AnySegwit version 34 out of range

The exact panic message varies between runs (AnySegwit version N with
N > 16, or OpReturn data length N with N > 80), but every fresh-state
fuzzing session reproduces one of them quickly.

Environment

• master at commit dc2ce06a3c3a7340e9615f988446b9bfbb1fc884
• AFL++ 4.41a, Nyx mode
• Targets reproduced on: ldk and lnd. Scenario: ir
• AFL_CUSTOM_MUTATOR_ONLY=1,
  AFL_CUSTOM_MUTATOR_LIBRARY=target/release/libsmite_ir_mutator.so
• Two machines, fresh cargo build --release and docker build --no-cache

Logs:

smite on  master [$?] is 📦 v0.0.0 via 🦀 v1.95.0 
❯ export SCENARIO=ir

smite on  master [$?] is 📦 v0.0.0 via 🦀 v1.95.0 
❯ export TARGET=ldk

smite on  master [$?] is 📦 v0.0.0 via 🦀 v1.95.0 
❯ docker build -t smite-$TARGET-$SCENARIO -f workloads/$TARGET/Dockerfile --build-arg SCENARIO=$SCENARIO . --no-cache
[+] Building 455.5s (37/37) FINISHED                                                                                                             docker:default
 => [internal] load build definition from Dockerfile                                                                                                       0.0s
 => => transferring dockerfile: 4.61kB                                                                                                                     0.0s
 => [internal] load metadata for docker.io/library/debian:bookworm-slim                                                                                    0.9s
 => [internal] load metadata for docker.io/library/debian:bookworm                                                                                         0.9s
 => [internal] load .dockerignore                                                                                                                          0.0s
 => => transferring context: 2B                                                                                                                            0.0s
 => [internal] load build context                                                                                                                          0.0s
 => => transferring context: 5.88kB                                                                                                                        0.0s
 => CACHED [stage-1  1/10] FROM docker.io/library/debian:bookworm-slim@sha256:67b30a61dc87758f0caf819646104f29ecbda97d920aaf5edc834128ac8493d3             0.0s
 => CACHED [builder  1/22] FROM docker.io/library/debian:bookworm@sha256:85019db29298555fd1a5f4bb57673ae989414a9884117c75d7a3e1a6cce21688                  0.0s
 => [stage-1  2/10] RUN apt-get update && apt-get install -y --no-install-recommends     libstdc++6     libgcc-s1     && rm -rf /var/lib/apt/lists/*       3.3s
 => [builder  2/22] RUN apt-get update && apt-get install -y --no-install-recommends     software-properties-common     wget     ca-certificates          30.5s
 => [builder  3/22] RUN mkdir -p /etc/apt/keyrings &&     wget -qO- https://apt.llvm.org/llvm-snapshot.gpg.key | tee /etc/apt/keyrings/llvm.asc > /dev/nu  1.0s 
 => [builder  4/22] RUN apt-get update && apt-get install -y --no-install-recommends     build-essential     clang-19     curl     git     && rm -rf /va  39.8s 
 => [builder  5/22] RUN curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y                                                           37.6s 
 => [builder  6/22] RUN rustup install nightly && rustup default nightly                                                                                  29.9s 
 => [builder  7/22] RUN cargo install cargo-afl                                                                                                           19.4s 
 => [builder  8/22] WORKDIR /tmp                                                                                                                           0.1s 
 => [builder  9/22] RUN wget https://bitcoincore.org/bin/bitcoin-core-29.2/bitcoin-29.2-x86_64-linux-gnu.tar.gz &&     tar -xzf bitcoin-29.2-x86_64-linu  38.6s 
 => [builder 10/22] WORKDIR /ldk-wrapper                                                                                                                   0.1s 
 => [builder 11/22] COPY workloads/ldk/Cargo.toml workloads/ldk/Cargo.lock ./                                                                              0.1s 
 => [builder 12/22] COPY workloads/ldk/src/ src/                                                                                                           0.1s 
 => [builder 13/22] RUN cargo afl build --release                                                                                                        241.2s 
 => [builder 14/22] WORKDIR /smite                                                                                                                         0.1s 
 => [builder 15/22] COPY Cargo.toml Cargo.lock ./                                                                                                          0.1s 
 => [builder 16/22] COPY smite/ smite/                                                                                                                     0.1s 
 => [builder 17/22] COPY smite-ir/ smite-ir/                                                                                                               0.1s 
 => [builder 18/22] COPY smite-ir-mutator/ smite-ir-mutator/                                                                                               0.1s 
 => [builder 19/22] COPY smite-nyx-sys/ smite-nyx-sys/                                                                                                     0.1s 
 => [builder 20/22] COPY smite-scenarios/ smite-scenarios/                                                                                                 0.1s
 => [builder 21/22] RUN TARGET_PATH=/ldk-wrapper/target/release/ldk-node-wrapper     cargo build -p smite-scenarios --bin ldk_ir --release --features ny  12.2s
 => [builder 22/22] RUN clang-19 -fPIC -DENABLE_NYX -DNO_PT_NYX -D_GNU_SOURCE     smite-nyx-sys/src/nyx-crash-handler.c -ldl -shared -o /nyx-crash-handle  0.4s
 => [stage-1  3/10] COPY --from=builder /ldk-wrapper/target/release/ldk-node-wrapper /usr/local/bin/ldk-node-wrapper                                       0.1s 
 => [stage-1  4/10] COPY --from=builder /nyx-crash-handler.so /crash-handler.so /                                                                          0.1s 
 => [stage-1  5/10] COPY --from=builder /usr/local/bin/bitcoind /usr/local/bin/bitcoind                                                                    0.1s 
 => [stage-1  6/10] COPY --from=builder /usr/local/bin/bitcoin-cli /usr/local/bin/bitcoin-cli                                                              0.1s 
 => [stage-1  7/10] COPY --from=builder /smite/target/release/ldk_ir /ldk-scenario                                                                         0.1s 
 => [stage-1  8/10] COPY workloads/ldk/init.sh /init.sh                                                                                                    0.1s
 => [stage-1  9/10] RUN chmod +x /init.sh /ldk-scenario                                                                                                    0.2s
 => exporting to image                                                                                                                                     1.1s
 => => exporting layers                                                                                                                                    1.0s
 => => writing image sha256:d27c3bb60e7541fe46a7923d29c4dce296d664139c3d77635fcf2a5118278fc0                                                               0.0s
 => => naming to docker.io/library/smite-ldk-ir                                                                                                            0.0s

smite on  master [$?] is 📦 v0.0.0 via 🦀 v1.95.0 
❯ source /home/erick/open-source/smite/.venv/bin/activate.fish

smite on  master [$?] is 📦 v0.0.0 via 🐍 v3.14.5 (.venv) via 🦀 v1.95.0 
❯ ./scripts/setup-nyx.sh /tmp/smite-nyx smite-$TARGET-$SCENARIO ~/AFLplusplus
Creating sharedir at: /tmp/smite-nyx
Exporting Docker container to container.tar...
Copying packer binaries...
Generating Nyx config...
Creating fuzz_no_pt.sh...

Sharedir created successfully at: /tmp/smite-nyx

Contents:
total 146M
-rw-r--r-- 1 erick erick  531 May 16 17:29 config.ron
-rw------- 1 erick erick 137M May 16 17:29 container.tar
-rwxr-xr-x 1 erick erick  738 May 16 17:29 fuzz_no_pt.sh
-rwxr-xr-x 1 erick erick 809K May 16 17:29 habort
-rwxr-xr-x 1 erick erick 809K May 16 17:29 habort_no_pt
-rwxr-xr-x 1 erick erick 809K May 16 17:29 hcat
-rwxr-xr-x 1 erick erick 809K May 16 17:29 hcat_no_pt
-rwxr-xr-x 1 erick erick 809K May 16 17:29 hget
-rwxr-xr-x 1 erick erick 817K May 16 17:29 hget_bulk
-rwxr-xr-x 1 erick erick 817K May 16 17:29 hget_bulk_no_pt
-rwxr-xr-x 1 erick erick 809K May 16 17:29 hget_no_pt
-rwxr-xr-x 1 erick erick 813K May 16 17:29 hpush
-rwxr-xr-x 1 erick erick 813K May 16 17:29 hpush_no_pt
-rwxr-xr-x 1 erick erick  15K May 16 17:29 libnyx.so
-rwxr-xr-x 1 erick erick 828K May 16 17:29 loader

To start fuzzing, run:
  mkdir -p /tmp/smite-seeds && echo 'AAAA' > /tmp/smite-seeds/seed1
  /home/erick/AFLplusplus/afl-fuzz -X -i /tmp/smite-seeds -o /tmp/smite-out -- /tmp/smite-nyx

smite on  master [$?] is 📦 v0.0.0 via 🐍 v3.14.5 (.venv) via 🦀 v1.95.0 
❯ cargo build --release -p smite-ir-mutator
   Compiling typenum v1.19.0
   Compiling version_check v0.9.5
   Compiling proc-macro2 v1.0.106
   Compiling quote v1.0.43
   Compiling unicode-ident v1.0.22
   Compiling shlex v1.3.0
   Compiling find-msvc-tools v0.1.6
   Compiling bitcoin-io v0.1.4
   Compiling arrayvec v0.7.6
   Compiling bitcoin-internals v0.3.0
   Compiling thiserror v2.0.18
   Compiling cfg-if v1.0.4
   Compiling subtle v2.6.1
   Compiling cfg_aliases v0.2.1
   Compiling cpufeatures v0.2.17
   Compiling serde_core v1.0.228
   Compiling libc v0.2.180
   Compiling zeroize v1.8.2
   Compiling hex_lit v0.1.1
   Compiling nix v0.30.1
   Compiling opaque-debug v0.3.1
   Compiling cc v1.2.51
   Compiling serde v1.0.228
   Compiling bitcoin v0.32.8
   Compiling log v0.4.29
   Compiling bech32 v0.11.1
   Compiling hex-conservative v0.2.2
   Compiling bitflags v2.10.0
   Compiling generic-array v0.14.7
   Compiling bitcoin-units v0.1.2
   Compiling rand_core v0.10.0
   Compiling hex v0.4.3
   Compiling simple_logger v5.1.0
   Compiling rand v0.10.0
   Compiling bitcoin_hashes v0.14.1
   Compiling syn v2.0.114
   Compiling secp256k1-sys v0.10.1
   Compiling crypto-common v0.1.7
   Compiling inout v0.1.4
   Compiling universal-hash v0.5.1
   Compiling aead v0.5.2
   Compiling cipher v0.4.4
   Compiling poly1305 v0.8.0
   Compiling chacha20 v0.9.1
   Compiling chacha20poly1305 v0.10.1
   Compiling base58ck v0.1.0
   Compiling thiserror-impl v2.0.18
   Compiling serde_derive v1.0.228
   Compiling cobs v0.3.0
   Compiling postcard v1.1.3
   Compiling secp256k1 v0.29.1
   Compiling smite v0.0.0 (/home/erick/open-source/smite/smite)
   Compiling smite-ir v0.0.0 (/home/erick/open-source/smite/smite-ir)
   Compiling smite-ir-mutator v0.0.0 (/home/erick/open-source/smite/smite-ir-mutator)
    Finished `release` profile [optimized] target(s) in 9.12s

smite on  master [$?] is 📦 v0.0.0 via 🐍 v3.14.5 (.venv) via 🦀 v1.95.0 
❯ mkdir -p /tmp/smite-seeds

smite on  master [$?] is 📦 v0.0.0 via 🐍 v3.14.5 (.venv) via 🦀 v1.95.0 
❯ printf '\x00' > /tmp/smite-seeds/empty

smite on  master [$?] is 📦 v0.0.0 via 🐍 v3.14.5 (.venv) via 🦀 v1.95.0 
❯ AFL_CUSTOM_MUTATOR_LIBRARY=target/release/libsmite_ir_mutator.so AFL_CUSTOM_MUTATOR_ONLY=1 AFL_DISABLE_TRIM=1 ~/AFLplusplus/afl-fuzz -X -i /tmp/smite-seeds -o
 /tmp/smite-out -- /tmp/smite-nyx
[+] Enabled environment variable AFL_CUSTOM_MUTATOR_ONLY with value 1
[+] Enabled environment variable AFL_CUSTOM_MUTATOR_LIBRARY with value target/release/libsmite_ir_mutator.so
afl-fuzz++4.41a based on afl by Michal Zalewski and a large online community
[+] AFL++ is maintained by Marc "van Hauser" Heuse, Dominik Maier, Andrea Fioraldi and Heiko "hexcoder" Eißfeldt
[+] AFL++ is open source, get it at https://github.com/AFLplusplus/AFLplusplus
[+] NOTE: AFL++ >= v3 has changed defaults and behaviours - see README.md
[+] AFL++ Nyx mode is enabled (developed and maintained by Sergej Schumilo)
[+] Nyx is open source, get it at https://github.com/Nyx-Fuzz
[+] No -M/-S set, autoconfiguring for "-S default"
[+] Enabled environment variable AFL_DISABLE_TRIM with value 1
[*] Getting to work...
[+] Using exploration-based constant power schedule (EXPLORE)
[+] Enabled testcache with 50 MB
[+] Generating fuzz data with a length of min=1 max=1048576
[+] FrameShift status: enabled (10% overhead configured)
[*] Trying to load libnyx.so plugin...
[+] libnyx plugin is ready!
[+] You have 16 CPU cores and 4 runnable tasks (utilization: 25%).
[+] Try parallel jobs - see docs/fuzzing_in_depth.md#c-using-multiple-cores
[*] Setting up output directories...
[+] Output directory exists but deemed OK to reuse.
[*] Deleting old session data...
[+] Output dir cleanup successful.
[*] Checking CPU core loadout...
[+] Found a free CPU core, try binding to #0.
[*] Loading custom mutator library from 'target/release/libsmite_ir_mutator.so'...
[+] Found 'afl_custom_mutator'.
[*] optional symbol 'afl_custom_fuzz_count' not found.
[*] optional symbol 'afl_custom_post_process' not found.
[*] optional symbol 'afl_custom_init_trim' not found.
[*] optional symbol 'afl_custom_trim' not found.
[*] optional symbol 'afl_custom_post_trim' not found.
[*] optional symbol 'afl_custom_havoc_mutation' not found.
[*] optional symbol 'afl_custom_havoc_mutation_probability' not found.
[*] optional symbol 'afl_custom_queue_get' not found.
[+] Found 'afl_custom_splice_optout'.
[*] optional symbol 'afl_custom_fuzz_send' not found.

        american fuzzy lop ++4.41a {default} (/tmp/smite-nyx) [explore] - Nyx        
┌─ process timing ────────────────────────────────────┬─ overall results ────┐
│        run time : 0 days, 0 hrs, 0 min, 47 sec      │  cycles done : 0     │
│   last new find : 0 days, 0 hrs, 0 min, 7 sec       │ corpus count : 48    │
│last saved crash : 0 days, 0 hrs, 0 min, 35 sec      │saved crashes : 1     │
│ last saved hang : none seen yet                     │  saved hangs : 0     │
├─ cycle progress ─────────────────────┬─ map coverage┴──────────────────────┤
│  now processing : 42.1 (87.5%)       │    map density : 0.68% / 0.95%      │
│  runs timed out : 0 (0.00%)          │ count coverage : 2.38 bits/tuple    │
├─ stage progress ─────────────────────┼─ findings in depth ─────────────────┤
│  now trying : libsmite_ir_mutator.so │ favored items : 20 (41.67%)         │
│ stage execs : 8888/12.8k (69.44%)    │  new edges on : 35 (72.92%)         │
│ total execs : 67.4k                  │ total crashes : 6 (1 saved)         │
│  exec speed : 1046/sec               │  total tmouts : 0 (0 saved)         │
├─ fuzzing strategy yields ────────────┴─────────────┬─ item geometry ───────┤
│   bit flips : disabled (custom-mutator-only mode)  │    levels : 8         │
│  byte flips : disabled (custom-mutator-only mode)  │   pending : 23        │
│ arithmetics : disabled (custom-mutator-only mode)  │  pend fav : 1         │
│  known ints : disabled (custom-mutator-only mode)  │ own finds : 47        │
│  dictionary : n/a                                  │  imported : 0         │
│havoc/splice : 0/0, 0/0                             │ stability : 85.32%    │
│py/custom/rq : unused, 41/51.6k, unused, unused     ├───────────────────────┘
│    trim/eff : disabled, disabled                   │          [cpu000: 12%]
└─ strategy: explore ────────── state: started :-) ──┘^C[QEMU-NYX] Error: Bye! (pid: 136892 / signal: 2)

+++ Testing aborted by user +++
[*] Writing /tmp/smite-out/default/fastresume.bin ...
[+] fastresume.bin successfully written with 5310224 bytes.
[!] libnyx: sending SIGKILL to QEMU-Nyx process...
[+] We're done here. Have a nice day!


smite on  master [$?] is 📦 v0.0.0 via 🐍 v3.14.5 (.venv) via 🦀 v1.95.0 took 53s 
❯ cp /tmp/smite-out/default/crashes/id:000000,sig:00,src:000008,time:12476,execs:27175,op:libsmite_ir_mutator.so,pos:0 ./crash

smite on  master [$?] is 📦 v0.0.0 via 🐍 v3.14.5 (.venv) via 🦀 v1.95.0 
❯ docker run --rm -v $PWD/crash:/input.bin -e SMITE_INPUT=/input.bin smite-$TARGET-$SCENARIO /$TARGET-scenario
INFO  [smite_scenarios::targets::bitcoind] Starting bitcoind...
INFO  [smite_scenarios::targets::bitcoind] Waiting for bitcoind to be ready...
INFO  [smite_scenarios::targets::bitcoind] bitcoind is ready
INFO  [smite_scenarios::targets::ldk] Starting ldk-node-wrapper...
INFO  [smite_scenarios::targets::ldk] LDK identity pubkey: 023a5e3520bd85ab6f19c0a8b42b119a159c3bceb3e2890153d8d9abfbc87a745d
INFO  [smite_scenarios::targets::ldk] ldk-node-wrapper is ready and synced
INFO  [smite_scenarios::targets::ldk] Both daemons are running, ready to fuzz
DEBUG [smite_scenarios::scenarios] Handshake complete, received target init
INFO  [smite::scenarios] Scenario initialized! Executing input...
INFO  [smite::runners] Reading input from "/input.bin"
DEBUG [smite_scenarios::scenarios::ir] [7.13µs] Executing IR program (23 instructions, 321 input bytes)

thread 'main' (1) panicked at smite-ir/src/operation.rs:224:17:
AnySegwit version 34 out of range
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
DEBUG [smite::process] ldk-node-wrapper: dropping running process, attempting shutdown
DEBUG [smite::process] ldk-node-wrapper: sending SIGTERM to process group 53
DEBUG [smite::process] ldk-node-wrapper: exited with exit status: 0
DEBUG [smite::process] bitcoind: dropping running process, attempting shutdown
DEBUG [smite::process] bitcoind: sending SIGTERM to process group 7
DEBUG [smite::process] bitcoind: exited with exit status: 0

[morehouse]

Your startup output does not match mine and makes me suspicious that the custom mutator isn't getting installed correctly. Notably, your output is missing the [+] Custom mutator 'target/release/libsmite_ir_mutator.so' installed successfully line that mine has. Here's my startup output for reference (afl++ 4.35a):

[*] Loading custom mutator library from 'target/release/libsmite_ir_mutator.so'...                                                                                                                                  
[+] Found 'afl_custom_mutator'.                                                                                                                                                                                     
[*] optional symbol 'afl_custom_fuzz_count' not found.                                                                                                                                                              
[*] optional symbol 'afl_custom_post_process' not found.                                                                                                                                                            
[*] optional symbol 'afl_custom_init_trim' not found.                                                                                                                                                               
[*] optional symbol 'afl_custom_trim' not found.                                                                                                                                                                    
[*] optional symbol 'afl_custom_post_trim' not found.                                                                                                                                                               
[*] optional symbol 'afl_custom_havoc_mutation' not found.                                                                                                                                                          
[*] optional symbol 'afl_custom_havoc_mutation_probability' not found.                                                                                                                                              
[*] optional symbol 'afl_custom_queue_get' not found.                                                                                                                                                               
[+] Found 'afl_custom_splice_optout'.                                                                                                                                                                               
[*] optional symbol 'afl_custom_fuzz_send' not found.                                                                                                                                                               
[*] optional symbol 'afl_custom_post_run' not found.                                                                                                                                                                
[*] optional symbol 'afl_custom_queue_new_entry' not found                                                                                                                                                          
[+] Found 'afl_custom_describe'.                                                                                                                                                                                    
[+] Custom mutator 'target/release/libsmite_ir_mutator.so' installed successfully.                                                                                                                                  
[*] Validating target binary...                                                                                                                                                                                     
[*] Scanning 'ldk-ir-minimized/'...                                                                                                                                                                                 
[*] Creating hard links for all input files...                                                                                                                                                                      
[+] Loaded a total of 186 seeds.                                                                                                                                                                                    
[*] Spinning up the NYX backend...

Your output is also missing the [+] Found 'afl_custom_describe' line, and your non-descriptive queue filenames op:libsmite_ir_mutator.so,pos:0 also indicate that afl_custom_describe isn't working properly.

For comparison, here's what my filenames look like when using the custom mutators: id:000221,src:000194,time:253604,execs:215623,smite-ir:op-param,op-param,input-swap,input-swap,input-swap,op-param,input-swap,input-swap,op-param,op-param,op-param,op-param,input-swap,input-swap,op-param,input-swap,+cov.

[morehouse]

I was able to reproduce this after upgrading AFL++ to 4.40c.

The problem is the new FrameShift stage added in 4.40c. It currently bypasses custom mutators even if AFL_CUSTOM_MUTATORS_ONLY=1 is set.

We can workaround this by disabling FrameShift when using custom mutators: #85.

I also filed a bug upstream for AFL++: AFLplusplus/AFLplusplus#2791.

[vanhauser-thc]

fixed in the AFL++ dev branch