Crash when trying to run `ml/main.py`

[unprosaiclabyrinth]

A few traces were collected using the logger and saved to the traces directory by default. The instructions don't mention whether any preprocessing of the traces is necessary before running the ML module, so the main.py script was run as instructed in the README immediately after collecting the traces. It was expected to demo the training and inference, however it crashed with the following backtrace:

Traceback (most recent call last):
  File "BasicBlockMLToolkit/ml/main.py", line 121, in <module>
    main()
    ~~~~^^
  File "BasicBlockMLToolkit/ml/main.py", line 92, in main
    print(analyze_sequence_stats(sequences))
          ~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^
  File "BasicBlockMLToolkit/ml/dataset.py", line 115, in analyze_sequence_stats
    'length_25th': np.percentile(lengths, 25),
                   ~~~~~~~~~~~~~^^^^^^^^^^^^^
  File "BasicBlockMLToolkit/.venv/lib/python3.13/site-packages/numpy/lib/_function_base_impl.py", line 4292, in percentile
    return _quantile_unchecked(
        a, q, axis, out, overwrite_input, method, keepdims, weights)
  File "BasicBlockMLToolkit/.venv/lib/python3.13/site-packages/numpy/lib/_function_base_impl.py", line 4569, in _quantile_unchecked
    return _ureduce(a,
                    func=_quantile_ureduce_func,
    ...<5 lines>...
                    overwrite_input=overwrite_input,
                    method=method)
  File "BasicBlockMLToolkit/.venv/lib/python3.13/site-packages/numpy/lib/_function_base_impl.py", line 3914, in _ureduce
    r = func(a, **kwargs)
  File "BasicBlockMLToolkit/.venv/lib/python3.13/site-packages/numpy/lib/_function_base_impl.py", line 4744, in _quantile_ureduce_func
    result = _quantile(arr,
                       quantiles=q,
    ...<2 lines>...
                       out=out,
                       weights=wgt)
  File "BasicBlockMLToolkit/.venv/lib/python3.13/site-packages/numpy/lib/_function_base_impl.py", line 4866, in _quantile
    slices_having_nans = np.isnan(arr[-1, ...])
                                  ~~~^^^^^^^^^
IndexError: index -1 is out of bounds for axis 0 with size 0

[abscosmos]

You're right; there isn't any need to preprocess the traces before running the script.

Since the script failed while separating the trace sequences into training, validation, and test sets, I'd assume the issue is either that there weren't any traces, or the traces had zero basic blocks.

What operating system did you run this on, and what programs did you trace? Debugging this issue will be easier once I implement a way to make the trace files human readable (#1)

[unprosaiclabyrinth]

I ran this on an Ubuntu Linux VM. The executables I traced were ls, grep, cat, awk, sed. For what it's worth, the traces were certainly not empty since I checked their sizes and I was able to dump their binary contents to the console. I tried changing the trace extensions to .txt, and the module names became human-readable but other stuff remained binary-encoded.

[abscosmos]

Got it. These shouldn't have been any issues with the contents of the trace files then, or it would've crashed during the load_all_traces call during create_load_tokenizer.

Based on the error, I'm assuming the sequences list was empty, and none of the traces were loaded.

Can you check len(sequences) before the call to analyze_sequence_stats in main.py?

And trace_path correctly points to the traces, right?

[unprosaiclabyrinth]

I believe these steps/checks are just the initial two of a full debugging process. It would be easier if you carried them out and followed through with the debugging since the results of the checks would make more sense to you as the author of the code 🙂. I ran the code without any edits to a fresh clone if that helps.

[abscosmos]

Yeah, agreed. Can you zip up the traces & send it to me (either attach a link here or email me).

I'm pretty sure it might just be an issue of not having the trace path set in the code. Did you do step 2 in the README instructions: Update the paths in ml/main.py to point to your traces (line 85)?