Test failures with DYNAMIC=1

[hasufell]

• PR: Build dynamic GHC in github CI #137
• pipeline: https://github.com/stable-haskell/ghc/actions/runs/21209299917/job/61022037886?pr=137#step:10:4879

snip:

Unexpected failures:
   ghci.debugger/scripts/break001.run                                 break001 [bad stderr] (ghci-ext)
   ghci.debugger/scripts/break002.run                                 break002 [bad stderr] (ghci-ext)
   ghci.debugger/scripts/break003.run                                 break003 [bad stderr] (ghci-ext)
   ghci.debugger/scripts/break005.run                                 break005 [bad stdout] (ghci-ext)
   ghci.debugger/scripts/break006.run                                 break006 [bad stderr] (ghci-ext)
   ghci.debugger/scripts/break007.run                                 break007 [bad stderr] (ghci-ext)
   ghci.debugger/scripts/break008.run                                 break008 [bad stdout] (ghci-ext)
   ghci.debugger/scripts/break009.run                                 break009 [bad stdout] (ghci-ext)
   ghci.debugger/scripts/break010.run                                 break010 [bad stderr] (ghci-ext)
   ghci.debugger/scripts/break011.run                                 break011 [bad stdout] (ghci-ext)
   ghci.debugger/scripts/break012.run                                 break012 [bad stderr] (ghci-ext)
   ghci.debugger/scripts/break013.run                                 break013 [bad stderr] (ghci-ext)
   ghci.debugger/scripts/break014.run                                 break014 [bad stderr] (ghci-ext)
   ghci.debugger/scripts/break016.run                                 break016 [bad stdout] (ghci-ext)
   ghci.debugger/scripts/break017.run                                 break017 [bad stdout] (ghci-ext)
   ghci.debugger/scripts/break018.run                                 break018 [bad stderr] (ghci-ext)
   ghci.debugger/scripts/break019.run                                 break019 [bad stderr] (ghci-ext)
   ghci.debugger/scripts/break020.run                                 break020 [bad stderr] (ghci-ext)
   ghci.debugger/scripts/break021.run                                 break021 [bad stderr] (ghci-ext)
   ghci.debugger/scripts/break024.run                                 break024 [bad stdout] (ghci-ext)
   ghci.debugger/scripts/break025.run                                 break025 [bad stderr] (ghci-ext)
   ghci.debugger/scripts/break026.run                                 break026 [bad stderr] (ghci-ext)
   ghci.debugger/scripts/break027.run                                 break027 [bad stdout] (ghci-ext)
   ghci.debugger/scripts/break028.run                                 break028 [bad stderr] (ghci-ext)
   ghci.debugger/scripts/dynbrk001.run                                dynbrk001 [bad stderr] (ghci-ext)
   ghci.debugger/scripts/dynbrk003.run                                dynbrk003 [bad stderr] (ghci-ext)
   ghci.debugger/scripts/dynbrk004.run                                dynbrk004 [bad stderr] (ghci-ext)
   ghci.debugger/scripts/dynbrk007.run                                dynbrk007 [bad stderr] (ghci-ext)
   ghci.debugger/scripts/dynbrk008.run                                dynbrk008 [bad stderr] (ghci-ext)
   ghci.debugger/scripts/dynbrk009.run                                dynbrk009 [bad stderr] (ghci-ext)
   ghc-api/dynCompileExpr/dynCompileExpr.run                          dynCompileExpr [bad exit code (1)] (normal)
   ghci.debugger/scripts/getargs.run                                  getargs [bad stderr] (ghci-ext)
   ghci/scripts/ghci004.run                                           ghci004 [bad stdout] (ghci-ext)
   ghci/scripts/ghci006.run                                           ghci006 [bad stdout] (ghci-ext)
   ghci/scripts/ghci062.run                                           ghci062 [bad stderr] (ghci-ext)
   ghci.debugger/scripts/hist001.run                                  hist001 [bad stderr] (ghci-ext)
   ghci.debugger/scripts/hist002.run                                  hist002 [bad stderr] (ghci-ext)
   rts/linker/linker_unload.run                                       linker_unload [bad exit code (2)] (normal)
   rts/linker/unload_multiple_objs/linker_unload_multiple_objs.run    linker_unload_multiple_objs [bad exit code (2)] (normal)
   ghci.debugger/scripts/listCommand001.run                           listCommand001 [bad stdout] (ghci-ext)
   ghci.debugger/scripts/listCommand002.run                           listCommand002 [bad stderr] (ghci-ext)
   ghci/should_run/PackedDataCon/PackedDataCon.run                    PackedDataCon [bad stderr] (ghci-ext)
   ghci.debugger/scripts/print001.run                                 print001 [bad stderr] (ghci-ext)
   ghci.debugger/scripts/print002.run                                 print002 [bad stdout] (ghci-ext)
   ghci.debugger/scripts/print003.run                                 print003 [bad stderr] (ghci-ext)
   ghci.debugger/scripts/print004.run                                 print004 [bad stderr] (ghci-ext)
   ghci.debugger/scripts/print005.run                                 print005 [bad stdout] (ghci-ext)
   ghci.debugger/scripts/print006.run                                 print006 [bad stderr] (ghci-ext)
   ghci.debugger/scripts/print007.run                                 print007 [bad stderr] (ghci-ext)
   ghci.debugger/scripts/print008.run                                 print008 [bad stderr] (ghci-ext)
   ghci.debugger/scripts/print009.run                                 print009 [bad stderr] (ghci-ext)
   ghci.debugger/scripts/print010.run                                 print010 [bad stderr] (ghci-ext)
   ghci.debugger/scripts/print011.run                                 print011 [bad stderr] (ghci-ext)
   ghci.debugger/scripts/print012.run                                 print012 [bad stderr] (ghci-ext)
   ghci.debugger/scripts/print013.run                                 print013 [bad stderr] (ghci-ext)
   ghci.debugger/scripts/print014.run                                 print014 [bad stderr] (ghci-ext)
   ghci.debugger/scripts/print016.run                                 print016 [bad stderr] (ghci-ext)
   ghci.debugger/scripts/print017.run                                 print017 [bad stderr] (ghci-ext)
   ghci.debugger/scripts/print018.run                                 print018 [bad stderr] (ghci-ext)
   ghci.debugger/scripts/print019.run                                 print019 [bad stderr] (ghci-ext)
   ghci.debugger/scripts/print021.run                                 print021 [bad stderr] (ghci-ext)
   ghci.debugger/scripts/print022.run                                 print022 [bad stderr] (ghci-ext)
   ghci.debugger/scripts/print023.run                                 print023 [bad stderr] (ghci-ext)
   ghci.debugger/scripts/print024.run                                 print024 [bad stderr] (ghci-ext)
   ghci.debugger/scripts/print025.run                                 print025 [bad stderr] (ghci-ext)
   ghci.debugger/scripts/print026.run                                 print026 [bad stderr] (ghci-ext)
   ghci.debugger/scripts/print027.run                                 print027 [bad stderr] (ghci-ext)
   ghci.debugger/scripts/print028.run                                 print028 [bad stderr] (ghci-ext)
   ghci.debugger/scripts/print029.run                                 print029 [bad stderr] (ghci-ext)
   ghci.debugger/scripts/print030.run                                 print030 [bad stderr] (ghci-ext)
   ghci.debugger/scripts/print031.run                                 print031 [bad stderr] (ghci-ext)
   ghci.debugger/scripts/print032.run                                 print032 [bad stderr] (ghci-ext)
   ghci.debugger/scripts/print033.run                                 print033 [bad stderr] (ghci-ext)
   ghci.debugger/scripts/print034.run                                 print034 [bad stderr] (ghci-ext)
   ghci.debugger/scripts/print035.run                                 print035 [bad stdout] (ghci-ext)
   ghci/prog001/prog001.run                                           prog001 [bad stderr] (ghci-ext)
   ghci/recompTHghci/recompTHghci.run                                 recompTHghci [bad stderr] (ghci-ext)
   ghci.debugger/scripts/result001.run                                result001 [bad stderr] (ghci-ext)
   plugins/static-plugins.run                                         static-plugins [bad exit code (1)] (normal)
   ghc-api/T10052/T10052.run                                          T10052 [bad exit code (2)] (normal)
   ghci/linking/dyn/T10458.run                                        T10458 [bad stderr] (ghci)
   ghc-api/T10508_api.run                                             T10508_api [bad exit code (1)] (normal)
   ghci.debugger/scripts/T10616.run                                   T10616 [bad stderr] (ghci-ext)
   ghci.debugger/scripts/T10617.run                                   T10617 [bad stderr] (ghci-ext)
   ghci.debugger/scripts/T12449.run                                   T12449 [bad stdout] (ghci-ext)
   ghci.debugger/scripts/T12458.run                                   T12458 [bad stderr] (ghci-ext)
   ghci.debugger/scripts/T13825-debugger.run                          T13825-debugger [bad stderr] (ghci-ext)
   quasiquotation/T14028.run                                          T14028 [bad exit code (2)] (dyn)
   ghci.debugger/scripts/T14628.run                                   T14628 [bad stderr] (ghci-ext)
   ghci.debugger/scripts/T14690.run                                   T14690 [bad stderr] (ghci-ext)
   ghci.debugger/scripts/T1620.run                                    T1620 [bad stderr] (ghci-ext)
   ghci/T16392/T16392.run                                             T16392 [bad stderr] (ghci-ext)
   ghci.debugger/scripts/T16700.run                                   T16700 [bad stderr] (ghci-ext)
   ghci/scripts/T17669.run                                            T17669 [bad stderr] (ghci)
   ghci.debugger/scripts/T17989.run                                   T17989 [bad stderr] (ghci-ext)
   ghci.debugger/scripts/T18045.run                                   T18045 [bad stderr] (ghci-ext)
   dynlibs/T18072debug.run                                            T18072debug [bad exit code (2)] (normal)
   ghc-api/T18181.run                                                 T18181 [bad exit code (1)] (normal)
   ghci.debugger/scripts/T19157.run                                   T19157 [bad stderr] (ghci-ext)
   ghci.debugger/scripts/T19355.run                                   T19355 [bad stderr] (ghci-ext)
   ghci.debugger/scripts/T19394.run                                   T19394 [bad stderr] (ghci-ext)
   ghci.debugger/scripts/T2215.run                                    T2215 [bad stderr] (ghci-ext)
   ghci.debugger/scripts/T24306.run                                   T24306 [bad stderr] (ghci-ext)
   ghci.debugger/scripts/T24712.run                                   T24712 [bad stderr] (ghci-ext)
   ghci.debugger/scripts/T25109.run                                   T25109 [bad stderr] (ghci-ext)
   plugins/T25217.run                                                 T25217 [bad exit code (1)] (normal)
   ghc-api/T25577.run                                                 T25577 [bad exit code (1)] (normal)
   ghci.debugger/scripts/T25932.run                                   T25932 [bad stderr] (ghci-ext)
   bytecode/T25975.run                                                T25975 [bad stderr] (ghci-ext)
   ghci.debugger/scripts/T26042b.run                                  T26042b [bad stderr] (ghci-ext)
   ghci.debugger/scripts/T26042c.run                                  T26042c [bad stderr] (ghci-ext)
   ghci.debugger/scripts/T26042d.run                                  T26042d [bad stderr] (ghci-ext)
   ghci.debugger/scripts/T26042d2.run                                 T26042d2 [bad stderr] (ghci-ext)
   ghci.debugger/scripts/T26042e.run                                  T26042e [bad stderr] (ghci-ext)
   ghci.debugger/scripts/T26042f1.run                                 T26042f1 [bad stderr] (ghci-ext)
   ghci.debugger/scripts/T26042f2.run                                 T26042f2 [bad stderr] (ghci-ext)
   ghci.debugger/scripts/T26042g.run                                  T26042g [bad stderr] (ghci-ext)
   ghci.debugger/scripts/T2740.run                                    T2740 [bad stderr] (ghci-ext)
   ghci.debugger/scripts/T2950.run                                    T2950 [bad stdout] (ghci-ext)
   ghci.debugger/scripts/T3000.run                                    T3000 [bad stderr] (ghci-ext)
   ghci/linking/dyn/T3372.run                                         T3372 [bad exit code (1)] (normal)
   driver/T5313.run                                                   T5313 [bad exit code (1)] (normal)
   ghci.debugger/scripts/T7386.run                                    T7386 [bad stderr] (ghci-ext)
   ghci.debugger/scripts/T8487.run                                    T8487 [bad stderr] (ghci-ext)
   ghci.debugger/scripts/T8557.run                                    T8557 [bad stderr] (ghci-ext)
   ghc-api/T8628.run                                                  T8628 [bad exit code (1)] (normal)
   ghc-api/T8639_api.run                                              T8639_api [bad exit code (1)] (normal)
   ghci/should_run/UnboxedTuples/UnboxedTuples.run                    UnboxedTuples [bad stderr] (ghci-ext)
   ghci/should_run/UnliftedDataTypeInterp/UnliftedDataTypeInterp.run  UnliftedDataTypeInterp [bad stderr] (ghci-ext)

[hasufell]

Reproduce:

• git clone --recursive https://github.com/stable-haskell/ghc.git --branch stable-ghc-9.14-dynamic
• cd ghc
• make CABAL=_build/stage0/bin/cabal DYNAMIC=1 _build/bindist
• run one of the tests, e.g.: PATH="$(pwd)/_build/bindist/bin:$PATH" make -C testsuite/ CLEANUP=0 TEST=dynbrk009

[hasufell]

I can make T18181 succeed, by preloading the RTS:

LD_LIBRARY_PATH=$(pwd)/_build/bindist/lib/x86_64-unknown-linux/ \
  LD_PRELOAD="libHSrts-1.0.3-ghc9.14.so libHSrts-1.0.3-threaded-debug-ghc9.14.so" \
  PATH=$(pwd)/_build/bindist/bin:$PATH make -C testsuite/ CLEANUP=0 SKIP_PERF_TESTS=YES TEST=T18181

[angerman]

Looking at dynbrk009, we see:

=====> dynbrk009(ghci) 1 of 1 [0, 0, 0]
cd "ghci.debugger/scripts/dynbrk009.run" && HC="/home/angerman/ghc-debug-dynamic/_build/bindist/bin/ghc" HC_OPTS="-dcore-lint -dstg-lint -dcmm-lint -fno-dump-with-ways -no-user-package-db -rtsopts -fno-warn-missed-specialisations -fshow-warning-groups -fdiagnostics-color=never -fno-diagnostics-show-caret -fprint-error-index-links=never -Werror=compat -dno-debug-output " "/home/angerman/ghc-debug-dynamic/_build/bindist/bin/ghc" --interactive -fno-unoptimized-core-for-interpreter -O0 -v0 -ignore-dot-ghci -fno-ghci-history +RTS -I0.1 -RTS -fghci-leak-check -dcore-lint -dstg-lint -dcmm-lint -fno-dump-with-ways -no-user-package-db -rtsopts -fno-warn-missed-specialisations -fshow-warning-groups -fdiagnostics-color=never -fno-diagnostics-show-caret -fprint-error-index-links=never -Werror=compat -dno-debug-output     -ignore-dot-ghci< dynbrk009.script
=====> dynbrk009(ghci-ext) 1 of 1 [0, 0, 0]
cd "ghci.debugger/scripts/dynbrk009.run" && HC="/home/angerman/ghc-debug-dynamic/_build/bindist/bin/ghc" HC_OPTS="-dcore-lint -dstg-lint -dcmm-lint -fno-dump-with-ways -no-user-package-db -rtsopts -fno-warn-missed-specialisations -fshow-warning-groups -fdiagnostics-color=never -fno-diagnostics-show-caret -fprint-error-index-links=never -Werror=compat -dno-debug-output " "/home/angerman/ghc-debug-dynamic/_build/bindist/bin/ghc" --interactive -v0 -ignore-dot-ghci -fno-ghci-history -fexternal-interpreter +RTS -I0.1 -RTS -dcore-lint -dstg-lint -dcmm-lint -fno-dump-with-ways -no-user-package-db -rtsopts -fno-warn-missed-specialisations -fshow-warning-groups -fdiagnostics-color=never -fno-diagnostics-show-caret -fprint-error-index-links=never -Werror=compat -dno-debug-output     -ignore-dot-ghci< dynbrk009.script

Thus without external iserv, this seems to work. If we look at ghc we see that ghc somehow depends on:

- libHSrts-1.0.3-threaded-debug-ghc9.14.so
- libHSrts-1.0.3-threaded-nodebug-ghc9.14.so
- libHSrts-1.0.3-ghc9.14.so

while the ghc-iserv executable only depends on

- libHSrts-1.0.3-nonthreaded-nodebug-ghc9.14.so
- libHSrts-1.0.3-ghc9.14.so

This raises three questions:

1. why does ghc depend on two different (debug and non-debug) rts?
2. why does ghc-iserv crash. I don't really think this can be just because of a different rts.
3. Why is the error output so bad?

ghc-iserv: internal error: evacuate: strange closure type 300235704
    (GHC version 9.14 for x86_64_unknown_linux)
    Please report this as a GHC bug:  https://github.com/stable-haskell/ghc/issues

is rather uninformative. It does tell us that something during GC failed, but not why, or what. Running this will also print random closure types, so this is clearly some form of memory corruption.

We also seem to have a bunch of changes in #45 that should make the need for patchelf obsolete.