Testsuite fails on several archs

[sthibaul]

Hello,
The sphinxbase testsuite fails on some architectures in Debian :

ppc64el: https://buildd.debian.org/status/fetch.php?pkg=sphinxbase&arch=ppc64el&ver=0.8%2B5prealpha%2B1-8&stamp=1582414870&raw=0

FAIL: test_log_shifted
======================

INFO: logmath.c(283): Writing log table file 'tmp.logadd'
INFO: logmath.c(174): Reading log table file 'tmp.logadd'
FAIL: abs((logmath_log(lmath, 1e-150)) - (-13493)) < LOG_EPSILON
log(1e-150) = -13493
exp(log(1e-150)) = 9.843575e-151
log(1e-48) = -4318
exp(log(1e-48)) = 9.888735e-49
log(42) = 146
exp(log(42)) = 41.989229
log(1e-3 + 5e-3) = -270 l+ -207 = -200
log(1e-3 + 5e-3) = 9.961084e-04 + 4.996912e-03 = 5.977553e-03
log(1e-150) = -3454050
FAIL test_log_shifted (exit status: 1)

alpha: https://buildd.debian.org/status/fetch.php?pkg=sphinxbase&arch=alpha&ver=0.8%2B5prealpha%2B1-8&stamp=1583125429&raw=0

../../../../test-driver: line 107: 17908 Bus error               "$@" > $log_file 2>&1
FAIL: test_log_int16

INFO: logmath.c(283): Writing log table file 'tmp.logadd'
INFO: logmath.c(174): Reading log table file 'tmp.logadd'
FAIL test_log_int16 (exit status: 138)

Samuel

[dhdaines]

Hello, there seem to be some long-standing problems with binary data. I am rewriting the PocketSphinx build system, would it be possible for you to test my fork at https://github.com/dhdaines/pocketsphinx ?

Sadly I have not been a Debian developer for several decades so do not have access to all these wonderful platforms anymore :)

[sthibaul]

This is still failing to a large extent, e.g. on s390x:

% make test
Running tests...
Test project /home/sthibault/pocketsphinx/build
      Start  1: test-cards.sh
 1/93 Test  #1: test-cards.sh ....................   Passed    1.33 sec
      Start  2: test-lm.sh
 2/93 Test  #2: test-lm.sh .......................***Failed    0.23 sec
      Start  3: test-align.sh
 3/93 Test  #3: test-align.sh ....................   Passed    4.33 sec
      Start  4: test-tidigits-fsg.sh
 4/93 Test  #4: test-tidigits-fsg.sh .............   Passed    0.98 sec
      Start  5: test-tidigits-simple.sh


 5/93 Test  #5: test-tidigits-simple.sh ..........***Failed  512.69 sec
      Start  6: test_acmod
 6/93 Test  #6: test_acmod .......................   Passed    3.31 sec
      Start  7: test_acmod_grow
 7/93 Test  #7: test_acmod_grow ..................   Passed    1.43 sec
      Start  8: test_alignment
 8/93 Test  #8: test_alignment ...................   Passed    0.21 sec
      Start  9: test_allphone
 9/93 Test  #9: test_allphone ....................***Failed    0.05 sec
      Start 10: test_bitvec
10/93 Test #10: test_bitvec ......................   Passed    0.48 sec
      Start 11: test_dict2pid
11/93 Test #11: test_dict2pid ....................   Passed    0.18 sec
      Start 12: test_dict
12/93 Test #12: test_dict ........................   Passed    0.27 sec
      Start 13: test_fe
13/93 Test #13: test_fe ..........................   Passed    0.01 sec
      Start 14: test_fwdflat
14/93 Test #14: test_fwdflat .....................***Failed    0.20 sec
      Start 15: test_fwdtree_bestpath
15/93 Test #15: test_fwdtree_bestpath ............***Failed    0.20 sec
      Start 16: test_fwdtree
16/93 Test #16: test_fwdtree .....................***Failed    0.20 sec
      Start 17: test_init
17/93 Test #17: test_init ........................***Failed    0.21 sec
      Start 18: test_jsgf
18/93 Test #18: test_jsgf ........................   Passed    0.88 sec
      Start 19: test_keyphrase
19/93 Test #19: test_keyphrase ...................   Passed    1.65 sec
      Start 20: test_lattice
20/93 Test #20: test_lattice .....................***Failed    0.20 sec
      Start 21: test_ngram_model_read
21/93 Test #21: test_ngram_model_read ............***Failed    0.04 sec
      Start 22: test_log_shifted
22/93 Test #22: test_log_shifted .................   Passed    0.00 sec
      Start 23: test_log_int8
23/93 Test #23: test_log_int8 ....................   Passed    0.00 sec
      Start 24: test_log_int16
24/93 Test #24: test_log_int16 ...................   Passed    0.01 sec
      Start 25: test_mllr
25/93 Test #25: test_mllr ........................***Failed    0.02 sec
      Start 26: test_nbest
26/93 Test #26: test_nbest .......................***Failed    0.23 sec
      Start 27: test_pitch
27/93 Test #27: test_pitch .......................   Passed    0.35 sec
      Start 28: test_posterior
28/93 Test #28: test_posterior ...................***Failed    0.19 sec
      Start 29: test_ptm_mgau
29/93 Test #29: test_ptm_mgau ....................   Passed    0.71 sec
      Start 30: test_reinit
30/93 Test #30: test_reinit ......................   Passed    0.40 sec
      Start 31: test_senfh
31/93 Test #31: test_senfh .......................***Failed    0.18 sec
      Start 32: test_set_search
32/93 Test #32: test_set_search ..................***Failed    0.60 sec
      Start 33: test_simple
33/93 Test #33: test_simple ......................***Failed    0.19 sec
      Start 34: test_state_align
34/93 Test #34: test_state_align .................   Passed    1.82 sec
      Start 35: test_ckd_alloc
35/93 Test #35: test_ckd_alloc ...................   Passed    0.00 sec
      Start 36: test_listelem_alloc
36/93 Test #36: test_listelem_alloc ..............   Passed    0.00 sec
      Start 37: test_ckd_alloc_catch
37/93 Test #37: test_ckd_alloc_catch .............   Passed    0.00 sec
      Start 38: test_ckd_alloc_fail
38/93 Test #38: test_ckd_alloc_fail ..............   Passed    0.00 sec
      Start 39: test_ckd_alloc_abort
39/93 Test #39: test_ckd_alloc_abort .............   Passed    0.00 sec
      Start 40: _lcase1.test
40/93 Test #40: _lcase1.test .....................   Passed    0.00 sec
      Start 41: _lcase2.test
41/93 Test #41: _lcase2.test .....................   Passed    0.00 sec
      Start 42: _lcase3.test
42/93 Test #42: _lcase3.test .....................   Passed    0.00 sec
      Start 43: _strcmp1.test
43/93 Test #43: _strcmp1.test ....................   Passed    0.00 sec
      Start 44: _strcmp2.test
44/93 Test #44: _strcmp2.test ....................   Passed    0.00 sec
      Start 45: _strcmp3.test
45/93 Test #45: _strcmp3.test ....................   Passed    0.02 sec
      Start 46: _ucase1.test
46/93 Test #46: _ucase1.test .....................   Passed    0.01 sec
      Start 47: _ucase2.test
47/93 Test #47: _ucase2.test .....................   Passed    0.00 sec
      Start 48: _ucase3.test
48/93 Test #48: _ucase3.test .....................   Passed    0.00 sec
      Start 49: _test_parse_badargs.test
49/93 Test #49: _test_parse_badargs.test .........   Passed    0.02 sec
      Start 50: _test_parse_defaults_r.test
50/93 Test #50: _test_parse_defaults_r.test ......   Passed    0.01 sec
      Start 51: _test_parse_defaults.test
51/93 Test #51: _test_parse_defaults.test ........   Passed    0.01 sec
      Start 52: _test_parse_goodargs.test
52/93 Test #52: _test_parse_goodargs.test ........   Passed    0.00 sec
      Start 53: _test_parse_multiple.test
53/93 Test #53: _test_parse_multiple.test ........   Passed    0.00 sec
      Start 54: _test_feat.test
54/93 Test #54: _test_feat.test ..................   Passed    0.00 sec
      Start 55: test_feat_live
55/93 Test #55: test_feat_live ...................   Passed    0.00 sec
      Start 56: test_feat_fe
56/93 Test #56: test_feat_fe .....................   Passed    0.06 sec
      Start 57: test_subvq
57/93 Test #57: test_subvq .......................   Passed    0.00 sec
      Start 58: test_fsg_read
58/93 Test #58: test_fsg_read ....................   Passed    0.00 sec
      Start 59: test_fsg_jsgf
59/93 Test #59: test_fsg_jsgf ....................   Passed    0.00 sec
      Start 60: test_fsg_write_fsm
60/93 Test #60: test_fsg_write_fsm ...............   Passed    0.00 sec
      Start 61: test_hash_iter
61/93 Test #61: test_hash_iter ...................   Passed    0.00 sec
      Start 62: _hash_delete1.test
62/93 Test #62: _hash_delete1.test ...............   Passed    0.01 sec
      Start 63: _hash_delete2.test
63/93 Test #63: _hash_delete2.test ...............   Passed    0.01 sec
      Start 64: _hash_delete3.test
64/93 Test #64: _hash_delete3.test ...............   Passed    0.01 sec
      Start 65: _hash_delete4.test
65/93 Test #65: _hash_delete4.test ...............   Passed    0.00 sec
      Start 66: _hash_delete5.test
66/93 Test #66: _hash_delete5.test ...............   Passed    0.00 sec
      Start 67: test_lineiter
67/93 Test #67: test_lineiter ....................   Passed    0.01 sec
      Start 68: _test_determinant.test
68/93 Test #68: _test_determinant.test ...........   Passed    0.00 sec
      Start 69: _test_invert.test
69/93 Test #69: _test_invert.test ................   Passed    0.00 sec
      Start 70: _test_solve.test
70/93 Test #70: _test_solve.test .................   Passed    0.00 sec
      Start 71: test_lm_read
71/93 Test #71: test_lm_read .....................***Failed    2.39 sec
      Start 72: test_lm_score
72/93 Test #72: test_lm_score ....................***Failed    0.00 sec
      Start 73: test_lm_add
73/93 Test #73: test_lm_add ......................   Passed    0.09 sec
      Start 74: test_lm_casefold
74/93 Test #74: test_lm_casefold .................   Passed    0.08 sec
      Start 75: test_lm_class
75/93 Test #75: test_lm_class ....................***Failed    0.08 sec
      Start 76: test_lm_set
76/93 Test #76: test_lm_set ......................***Failed    0.09 sec
      Start 77: test_lm_write
77/93 Test #77: test_lm_write ....................***Failed    0.01 sec
      Start 78: _fread_line.test
78/93 Test #78: _fread_line.test .................   Passed    0.02 sec
      Start 79: _nextword.test
79/93 Test #79: _nextword.test ...................   Passed    0.00 sec
      Start 80: _str2words.test
80/93 Test #80: _str2words.test ..................   Passed    0.00 sec
      Start 81: _string_join.test
81/93 Test #81: _string_join.test ................   Passed    0.00 sec
      Start 82: _string_trim.test
82/93 Test #82: _string_trim.test ................   Passed    0.00 sec
      Start 83: test_atof
83/93 Test #83: test_atof ........................   Passed    0.06 sec
      Start 84: test_thread
84/93 Test #84: test_thread ......................   Passed    0.29 sec
      Start 85: test_event
85/93 Test #85: test_event .......................   Passed    0.50 sec
      Start 86: test_msgq
86/93 Test #86: test_msgq ........................   Passed    0.00 sec
      Start 87: test_fopen
87/93 Test #87: test_fopen .......................   Passed    0.01 sec
      Start 88: test_bitarr
88/93 Test #88: test_bitarr ......................   Passed    0.00 sec
      Start 89: test_bit_encode
89/93 Test #89: test_bit_encode ..................   Passed    0.00 sec
      Start 90: test_build_directory
90/93 Test #90: test_build_directory .............   Passed    0.00 sec
      Start 91: test_heap
91/93 Test #91: test_heap ........................   Passed    0.00 sec
      Start 92: test_filename
92/93 Test #92: test_filename ....................   Passed    0.00 sec
      Start 93: test_readfile
93/93 Test #93: test_readfile ....................   Passed    0.00 sec

78% tests passed, 20 tests failed out of 93

Total Test time (real) = 537.77 sec

The following tests FAILED:
	  2 - test-lm.sh (Failed)
	  5 - test-tidigits-simple.sh (Failed)
	  9 - test_allphone (Failed)
	 14 - test_fwdflat (Failed)
	 15 - test_fwdtree_bestpath (Failed)
	 16 - test_fwdtree (Failed)
	 17 - test_init (Failed)
	 20 - test_lattice (Failed)
	 21 - test_ngram_model_read (Failed)
	 25 - test_mllr (Failed)
	 26 - test_nbest (Failed)
	 28 - test_posterior (Failed)
	 31 - test_senfh (Failed)
	 32 - test_set_search (Failed)
	 33 - test_simple (Failed)
	 71 - test_lm_read (Failed)
	 72 - test_lm_score (Failed)
	 75 - test_lm_class (Failed)
	 76 - test_lm_set (Failed)
	 77 - test_lm_write (Failed)
Errors while running CTest
Output from these tests are in: /home/sthibault/pocketsphinx/build/Testing/Temporary/LastTest.log
Use "--rerun-failed --output-on-failure" to re-run the failed cases verbosely.
make: *** [Makefile:71: test] Error 8

[dhdaines]

Hmm, good to know - big-endian platform I assume? Can you rerun with make test CTEST_OUTPUT_ON_FAILURE=1 and post the output?

[dhdaines]

also the contents of build/config.h would be helpful!

[sthibaul]

yes, it's big-endian, here they are:

test.log

config.h.txt

[dhdaines]

Thanks! It seems that a new binary language model format was implemented sometime after PocketSphinx 0.8 with zero consideration for byte ordering. Currently considering if this is another 5prealpha "feature" that should just get reverted...

[dhdaines]

I'm guessing the Alpha problem has something to do with unaligned access to memory. In which case it is probably my fault :)

[dhdaines]

Most of the tests should be fixed on big-endian machines by https://github.com/dhdaines/pocketsphinx/commit/192ac3f1141558d91381bc1700bddbabed4d878b - please try and let me know!