repro for LTO/ORC segfault by tersec · Pull Request #832 · status-im/nim-eth

tersec · 2025-11-30T03:27:14Z

Run all_tests instead of seperate parts tests #831 (comment)

Run

rm ~/.cache/nim/ -rf && ~/nim224/bin/nim c -f -d:release --verbosity:0 -r eth/enr/enr; ~/nim226/bin/nim c -f -d:release --verbosity:0 -r eth/enr/enr; ~/nim227/bin/nim c -f -d:release --verbosity:0 -r eth/enr/enr; ~/nim23/bin/nim c -f -d:release --verbosity:0 -r eth/enr/enr

with nim224, nim226, nim227, and nim23 all containing the respective Nim versions to get Segmentation fault in all 4 cases.

tersec · 2025-11-30T18:09:21Z

Reduced somewhat. Now

reset && for i in $(seq 1 7); do rm ~/.cache/nim/ -rf && ~/nim226/bin/nim c -f -d:release --verbosity:0 -o:$(mktemp) -r eth/common/keys.nim; ~/nim227/bin/nim c -f -d:release --verbosity:0 -o:$(mktemp) -r eth/common/keys.nim; ~/nim23/bin/nim c -f -d:release --verbosity:0 -o:$(mktemp) -r eth/common/keys.nim; done

produces, for example:

lto-wrapper: warning: using serial compilation of 4 LTRANS jobs
lto-wrapper: note: see the ‘-flto’ option documentation for more information
MPT trie proof verification SIGSEGV: Illegal storage access. (Attempt to read from nil?)
Segmentation fault
Error: execution of an external program failed: '/tmp/tmp.jIRlZXQFSe'
lto-wrapper: warning: using serial compilation of 4 LTRANS jobs
lto-wrapper: note: see the ‘-flto’ option documentation for more information
MPT trie proof verification SIGSEGV: Illegal storage access. (Attempt to read from nil?)
Segmentation fault
Error: execution of an external program failed: '/tmp/tmp.zn8IVIo0f5'
lto-wrapper: warning: using serial compilation of 4 LTRANS jobs
lto-wrapper: note: see the ‘-flto’ option documentation for more information
MPT trie proof verification SIGSEGV: Illegal storage access. (Attempt to read from nil?)
Segmentation fault
Error: execution of an external program failed: '/tmp/tmp.a5cmsNskTx'
lto-wrapper: warning: using serial compilation of 4 LTRANS jobs
lto-wrapper: note: see the ‘-flto’ option documentation for more information
MPT trie proof verification SIGSEGV: Illegal storage access. (Attempt to read from nil?)
Segmentation fault
Error: execution of an external program failed: '/tmp/tmp.XVyzmCyvzp'
lto-wrapper: warning: using serial compilation of 4 LTRANS jobs
lto-wrapper: note: see the ‘-flto’ option documentation for more information
MPT trie proof verification SIGSEGV: Illegal storage access. (Attempt to read from nil?)
Segmentation fault
Error: execution of an external program failed: '/tmp/tmp.8HX9pgFbPG'
lto-wrapper: warning: using serial compilation of 4 LTRANS jobs
lto-wrapper: note: see the ‘-flto’ option documentation for more information
MPT trie proof verification SIGSEGV: Illegal storage access. (Attempt to read from nil?)
Segmentation fault
Error: execution of an external program failed: '/tmp/tmp.KpiqQC4nQ7'
lto-wrapper: warning: using serial compilation of 4 LTRANS jobs
lto-wrapper: note: see the ‘-flto’ option documentation for more information
MPT trie proof verification SIGSEGV: Illegal storage access. (Attempt to read from nil?)
Segmentation fault
Error: execution of an external program failed: '/tmp/tmp.JW2OuwbHUh'
lto-wrapper: warning: using serial compilation of 4 LTRANS jobs
lto-wrapper: note: see the ‘-flto’ option documentation for more information
MPT trie proof verification SIGSEGV: Illegal storage access. (Attempt to read from nil?)
Segmentation fault
Error: execution of an external program failed: '/tmp/tmp.fSb4wWSQud'
lto-wrapper: warning: using serial compilation of 4 LTRANS jobs
lto-wrapper: note: see the ‘-flto’ option documentation for more information
MPT trie proof verification SIGSEGV: Illegal storage access. (Attempt to read from nil?)
Segmentation fault
Error: execution of an external program failed: '/tmp/tmp.X11e6kd1bZ'
lto-wrapper: warning: using serial compilation of 4 LTRANS jobs
lto-wrapper: note: see the ‘-flto’ option documentation for more information
MPT trie proof verification SIGSEGV: Illegal storage access. (Attempt to read from nil?)
Segmentation fault
Error: execution of an external program failed: '/tmp/tmp.bVd7OMZyks'
lto-wrapper: warning: using serial compilation of 4 LTRANS jobs
lto-wrapper: note: see the ‘-flto’ option documentation for more information
MPT trie proof verification SIGSEGV: Illegal storage access. (Attempt to read from nil?)
Segmentation fault
Error: execution of an external program failed: '/tmp/tmp.uW49CL2Dpd'
lto-wrapper: warning: using serial compilation of 4 LTRANS jobs
lto-wrapper: note: see the ‘-flto’ option documentation for more information
MPT trie proof verification SIGSEGV: Illegal storage access. (Attempt to read from nil?)
Segmentation fault
Error: execution of an external program failed: '/tmp/tmp.XHHPqI77OL'
lto-wrapper: warning: using serial compilation of 4 LTRANS jobs
lto-wrapper: note: see the ‘-flto’ option documentation for more information
MPT trie proof verification SIGSEGV: Illegal storage access. (Attempt to read from nil?)
Segmentation fault
Error: execution of an external program failed: '/tmp/tmp.PSzdwLM2oK'
lto-wrapper: warning: using serial compilation of 4 LTRANS jobs
lto-wrapper: note: see the ‘-flto’ option documentation for more information
MPT trie proof verification SIGSEGV: Illegal storage access. (Attempt to read from nil?)
Segmentation fault
Error: execution of an external program failed: '/tmp/tmp.uS9s0hxYmd'
lto-wrapper: warning: using serial compilation of 4 LTRANS jobs
lto-wrapper: note: see the ‘-flto’ option documentation for more information
MPT trie proof verification SIGSEGV: Illegal storage access. (Attempt to read from nil?)
Segmentation fault
Error: execution of an external program failed: '/tmp/tmp.bkkjftXOwR'
lto-wrapper: warning: using serial compilation of 4 LTRANS jobs
lto-wrapper: note: see the ‘-flto’ option documentation for more information
MPT trie proof verification SIGSEGV: Illegal storage access. (Attempt to read from nil?)
Segmentation fault
Error: execution of an external program failed: '/tmp/tmp.SLZpoVQmvn'
lto-wrapper: warning: using serial compilation of 4 LTRANS jobs
lto-wrapper: note: see the ‘-flto’ option documentation for more information
MPT trie proof verification SIGSEGV: Illegal storage access. (Attempt to read from nil?)
Segmentation fault
Error: execution of an external program failed: '/tmp/tmp.8iqyIIq7np'
lto-wrapper: warning: using serial compilation of 4 LTRANS jobs
lto-wrapper: note: see the ‘-flto’ option documentation for more information
MPT trie proof verification SIGSEGV: Illegal storage access. (Attempt to read from nil?)
Segmentation fault
Error: execution of an external program failed: '/tmp/tmp.bh1DkSa1TF'
lto-wrapper: warning: using serial compilation of 4 LTRANS jobs
lto-wrapper: note: see the ‘-flto’ option documentation for more information
MPT trie proof verification SIGSEGV: Illegal storage access. (Attempt to read from nil?)
Segmentation fault
Error: execution of an external program failed: '/tmp/tmp.0rI0rQumXc'
lto-wrapper: warning: using serial compilation of 4 LTRANS jobs
lto-wrapper: note: see the ‘-flto’ option documentation for more information
MPT trie proof verification SIGSEGV: Illegal storage access. (Attempt to read from nil?)
Segmentation fault
Error: execution of an external program failed: '/tmp/tmp.xcelBsX8Rx'
lto-wrapper: warning: using serial compilation of 4 LTRANS jobs
lto-wrapper: note: see the ‘-flto’ option documentation for more information
MPT trie proof verification SIGSEGV: Illegal storage access. (Attempt to read from nil?)
Segmentation fault
Error: execution of an external program failed: '/tmp/tmp.E9M4464FJn'

gcc (Debian 15.2.0-9) 15.2.0
Copyright (C) 2025 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

Nim Compiler Version 2.2.6 [Linux: amd64]
Compiled at 2025-11-29
Copyright (c) 2006-2025 by Andreas Rumpf

git hash: ab00c56904e3126ad826bb520d243513a139436a
active boot switches: -d:release

Nim Compiler Version 2.2.7 [Linux: amd64]
Compiled at 2025-11-29
Copyright (c) 2006-2025 by Andreas Rumpf

git hash: 800384176e8b2cf51c77f14b70f2916d764c8c0d
active boot switches: -d:release

Nim Compiler Version 2.3.1 [Linux: amd64]
Compiled at 2025-11-29
Copyright (c) 2006-2025 by Andreas Rumpf

git hash: 66560840043d2ea8a96b4ce46ab55f0faed37349
active boot switches: -d:release

tersec · 2025-11-30T18:15:44Z

$ valgrind --track-origins=yes /tmp/tmp.xcelBsX8Rx
==90346== Memcheck, a memory error detector
==90346== Copyright (C) 2002-2024, and GNU GPL'd, by Julian Seward et al.
==90346== Using Valgrind-3.25.1 and LibVEX; rerun with -h for copyright info
==90346== Command: /tmp/tmp.xcelBsX8Rx
==90346== 
MPT trie proof verification ==90346== Conditional jump or move depends on uninitialised value(s)
==90346==    at 0x401C6E3: eqdestroy___system_u3692 (in /tmp/tmp.xcelBsX8Rx)
==90346==    by 0x403DA14: verifyProof__OOZtrieZhexary95proof95verification_u807 (in /tmp/tmp.xcelBsX8Rx)
==90346==    by 0x403FDF8: runTestX60gensym60___keys_u1894 (in /tmp/tmp.xcelBsX8Rx)
==90346==    by 0x4044197: NimMainModule (in /tmp/tmp.xcelBsX8Rx)
==90346==    by 0x40022EE: main (in /tmp/tmp.xcelBsX8Rx)
==90346==  Uninitialised value was created by a stack allocation
==90346==    at 0x403B821: verifyProof__OOZtrieZhexary95proof95verification_u807 (in /tmp/tmp.xcelBsX8Rx)
==90346== 
==90346== Use of uninitialised value of size 8
==90346==    at 0x401C6E5: eqdestroy___system_u3692 (in /tmp/tmp.xcelBsX8Rx)
==90346==    by 0x403DA14: verifyProof__OOZtrieZhexary95proof95verification_u807 (in /tmp/tmp.xcelBsX8Rx)
==90346==    by 0x403FDF8: runTestX60gensym60___keys_u1894 (in /tmp/tmp.xcelBsX8Rx)
==90346==    by 0x4044197: NimMainModule (in /tmp/tmp.xcelBsX8Rx)
==90346==    by 0x40022EE: main (in /tmp/tmp.xcelBsX8Rx)
==90346==  Uninitialised value was created by a stack allocation
==90346==    at 0x403B821: verifyProof__OOZtrieZhexary95proof95verification_u807 (in /tmp/tmp.xcelBsX8Rx)
==90346== 
==90346== Invalid read of size 1
==90346==    at 0x401C6E5: eqdestroy___system_u3692 (in /tmp/tmp.xcelBsX8Rx)
==90346==    by 0x403DA14: verifyProof__OOZtrieZhexary95proof95verification_u807 (in /tmp/tmp.xcelBsX8Rx)
==90346==    by 0x403FDF8: runTestX60gensym60___keys_u1894 (in /tmp/tmp.xcelBsX8Rx)
==90346==    by 0x4044197: NimMainModule (in /tmp/tmp.xcelBsX8Rx)
==90346==    by 0x40022EE: main (in /tmp/tmp.xcelBsX8Rx)
==90346==  Address 0x6d is not stack'd, malloc'd or (recently) free'd
==90346== 
SIGSEGV: Illegal storage access. (Attempt to read from nil?)
==90346== 
==90346== Process terminating with default action of signal 11 (SIGSEGV)
==90346==    at 0x4A5A95C: __pthread_kill_implementation (pthread_kill.c:44)
==90346==    by 0x4A05CC1: raise (raise.c:26)
==90346==    by 0x4A05DEF: ??? (in /usr/lib/x86_64-linux-gnu/libc.so.6)
==90346==    by 0x401C6E4: eqdestroy___system_u3692 (in /tmp/tmp.xcelBsX8Rx)
==90346==    by 0x403DA14: verifyProof__OOZtrieZhexary95proof95verification_u807 (in /tmp/tmp.xcelBsX8Rx)
==90346==    by 0x403FDF8: runTestX60gensym60___keys_u1894 (in /tmp/tmp.xcelBsX8Rx)
==90346==    by 0x4044197: NimMainModule (in /tmp/tmp.xcelBsX8Rx)
==90346==    by 0x40022EE: main (in /tmp/tmp.xcelBsX8Rx)
==90346== 
==90346== HEAP SUMMARY:
==90346==     in use at exit: 1,024 bytes in 1 blocks
==90346==   total heap usage: 1 allocs, 0 frees, 1,024 bytes allocated
==90346== 
==90346== LEAK SUMMARY:
==90346==    definitely lost: 0 bytes in 0 blocks
==90346==    indirectly lost: 0 bytes in 0 blocks
==90346==      possibly lost: 0 bytes in 0 blocks
==90346==    still reachable: 1,024 bytes in 1 blocks
==90346==         suppressed: 0 bytes in 0 blocks
==90346== Rerun with --leak-check=full to see details of leaked memory
==90346== 
==90346== For lists of detected and suppressed errors, rerun with: -s
==90346== ERROR SUMMARY: 3 errors from 3 contexts (suppressed: 0 from 0)
Segmentation fault         valgrind --track-origins=yes /tmp/tmp.xcelBsX8Rx
```

tersec · 2025-11-30T18:18:36Z

Also happens with -d:useMalloc (here with Nim devel of stated commit in previous comments):

lto-wrapper: warning: using serial compilation of 3 LTRANS jobs
lto-wrapper: note: see the ‘-flto’ option documentation for more information
MPT trie proof verification free(): invalid pointer
SIGABRT: Abnormal termination.
Aborted
Error: execution of an external program failed: '/tmp/tmp.r6WqtMtRJc'

$ valgrind --track-origins=yes /tmp/tmp.r6WqtMtRJc
==91674== Memcheck, a memory error detector
==91674== Copyright (C) 2002-2024, and GNU GPL'd, by Julian Seward et al.
==91674== Using Valgrind-3.25.1 and LibVEX; rerun with -h for copyright info
==91674== Command: /tmp/tmp.r6WqtMtRJc
==91674== 
MPT trie proof verification ==91674== Conditional jump or move depends on uninitialised value(s)
==91674==    at 0x401B863: eqdestroy___system_u3676 (in /tmp/tmp.r6WqtMtRJc)
==91674==    by 0x403BBCD: verifyProof__OOZtrieZhexary95proof95verification_u807 (in /tmp/tmp.r6WqtMtRJc)
==91674==    by 0x403CF3F: runTestX60gensym60___keys_u1894 (in /tmp/tmp.r6WqtMtRJc)
==91674==    by 0x40413B8: NimMainModule (in /tmp/tmp.r6WqtMtRJc)
==91674==    by 0x40022DE: main (in /tmp/tmp.r6WqtMtRJc)
==91674==  Uninitialised value was created by a stack allocation
==91674==    at 0x4039D11: verifyProof__OOZtrieZhexary95proof95verification_u807 (in /tmp/tmp.r6WqtMtRJc)
==91674== 
==91674== Use of uninitialised value of size 8
==91674==    at 0x401B865: eqdestroy___system_u3676 (in /tmp/tmp.r6WqtMtRJc)
==91674==    by 0x403BBCD: verifyProof__OOZtrieZhexary95proof95verification_u807 (in /tmp/tmp.r6WqtMtRJc)
==91674==    by 0x403CF3F: runTestX60gensym60___keys_u1894 (in /tmp/tmp.r6WqtMtRJc)
==91674==    by 0x40413B8: NimMainModule (in /tmp/tmp.r6WqtMtRJc)
==91674==    by 0x40022DE: main (in /tmp/tmp.r6WqtMtRJc)
==91674==  Uninitialised value was created by a stack allocation
==91674==    at 0x4039D11: verifyProof__OOZtrieZhexary95proof95verification_u807 (in /tmp/tmp.r6WqtMtRJc)
==91674== 
==91674== Conditional jump or move depends on uninitialised value(s)
==91674==    at 0x499A835: free (vg_replace_malloc.c:989)
==91674==    by 0x403BBCD: verifyProof__OOZtrieZhexary95proof95verification_u807 (in /tmp/tmp.r6WqtMtRJc)
==91674==    by 0x403CF3F: runTestX60gensym60___keys_u1894 (in /tmp/tmp.r6WqtMtRJc)
==91674==    by 0x40413B8: NimMainModule (in /tmp/tmp.r6WqtMtRJc)
==91674==    by 0x40022DE: main (in /tmp/tmp.r6WqtMtRJc)
==91674==  Uninitialised value was created by a stack allocation
==91674==    at 0x4039D11: verifyProof__OOZtrieZhexary95proof95verification_u807 (in /tmp/tmp.r6WqtMtRJc)
==91674== 
==91674== Invalid free() / delete / delete[] / realloc()
==91674==    at 0x499A87F: free (vg_replace_malloc.c:989)
==91674==    by 0x403BBCD: verifyProof__OOZtrieZhexary95proof95verification_u807 (in /tmp/tmp.r6WqtMtRJc)
==91674==    by 0x403CF3F: runTestX60gensym60___keys_u1894 (in /tmp/tmp.r6WqtMtRJc)
==91674==    by 0x40413B8: NimMainModule (in /tmp/tmp.r6WqtMtRJc)
==91674==    by 0x40022DE: main (in /tmp/tmp.r6WqtMtRJc)
==91674==  Address 0x414bf88 is 8 bytes inside data symbol "TM__M68nGWgBGfdRpIiz8SqAwg_87"
==91674== 
==91674== 
==91674== HEAP SUMMARY:
==91674==     in use at exit: 42,593 bytes in 38 blocks
==91674==   total heap usage: 93 allocs, 56 frees, 46,183 bytes allocated
==91674== 
==91674== LEAK SUMMARY:
==91674==    definitely lost: 41 bytes in 1 blocks
==91674==    indirectly lost: 0 bytes in 0 blocks
==91674==      possibly lost: 25,592 bytes in 28 blocks
==91674==    still reachable: 16,960 bytes in 9 blocks
==91674==         suppressed: 0 bytes in 0 blocks
==91674== Rerun with --leak-check=full to see details of leaked memory
==91674== 
==91674== For lists of detected and suppressed errors, rerun with: -s
==91674== ERROR SUMMARY: 4 errors from 4 contexts (suppressed: 0 from 0)

tersec · 2026-03-11T01:34:32Z

Did some minimization, ended up with https://github.com/status-im/nim-eth/blob/lto-orc-segfault/test_release_segfault.sh to run the tests.

gcc (Debian 15.2.0-14) 15.2.0
Copyright (C) 2025 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

Nim Compiler Version 2.2.8 [Linux: amd64]
Compiled at 2026-03-04
Copyright (c) 2006-2026 by Andreas Rumpf

git hash: 4f500679b196fad944caa50a753f5bbfaefda001
active boot switches: -d:release

Nim Compiler Version 2.3.1 [Linux: amd64]
Compiled at 2026-03-09
Copyright (c) 2006-2026 by Andreas Rumpf

git hash: 0395af2b3459837fcdf6bf8c38d470ee682dd9cd
active boot switches: -d:release

kdeme and others added 2 commits November 30, 2025 03:21

Run all_tests instead of seperate parts tests

1f2e6d2

foo

d9db015

tersec mentioned this pull request Nov 30, 2025

Run all_tests instead of seperate parts tests #831

Open

tersec added 2 commits November 30, 2025 14:50

reduce via eth/common/keys.nim

d286ac2

incorporate unittest2 and partly reduce it

ef18678

foo

8d0b6a4

tersec force-pushed the lto-orc-segfault branch from ad3fd85 to 8d0b6a4 Compare March 10, 2026 19:30

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

repro for LTO/ORC segfault#832

repro for LTO/ORC segfault#832
tersec wants to merge 5 commits into
masterfrom
lto-orc-segfault

tersec commented Nov 30, 2025

Uh oh!

tersec commented Nov 30, 2025

Uh oh!

tersec commented Nov 30, 2025

Uh oh!

tersec commented Nov 30, 2025

Uh oh!

tersec commented Mar 11, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

tersec commented Nov 30, 2025

Uh oh!

tersec commented Nov 30, 2025

Uh oh!

tersec commented Nov 30, 2025

Uh oh!

tersec commented Nov 30, 2025

Uh oh!

tersec commented Mar 11, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants