`bigwigaverageoverbed` average outputs fixed 3 decimal precision unlike Kent `bigWigAverageOverBed`

[rabyj]

bigwigaverageoverbed appears to format the average/mean output values (last two columns) with a fixed precision of three decimal places, whereas the UCSC bigWigAverageOverBed utility outputs variable floating-point precision.

This leads to a loss of precision compared to the UCSC/Kent implementation.

Example using the same bigWig and BED input (head -n10):

bigtools

rank1	8800	8800	2006.485	0.228	0.228
rank2	24000	24000	6108.989	0.255	0.255
rank3	13400	13400	5949.426	0.444	0.444
rank4	23800	23800	8905.968	0.374	0.374
rank5	5200	5200	1188.002	0.228	0.228
rank6	4400	4400	786.351	0.179	0.179
rank7	7600	7600	1841.650	0.242	0.242
rank8	11600	11600	2512.321	0.217	0.217
rank9	2200	2200	643.230	0.292	0.292
rank10	2400	2400	439.444	0.183	0.183

UCSC bigWigAverageOverBed (once sorted)

rank1	8800	8800	2006.48	0.22801	0.22801
rank2	24000	24000	6108.99	0.254541	0.254541
rank3	13400	13400	5949.43	0.443987	0.443987
rank4	23800	23800	8905.97	0.3742	0.3742
rank5	5200	5200	1188	0.228462	0.228462
rank6	4400	4400	786.351	0.178716	0.178716
rank7	7600	7600	1841.65	0.242322	0.242322
rank8	11600	11600	2512.32	0.216579	0.216579
rank9	2200	2200	643.23	0.292377	0.292377
rank10	2400	2400	439.444	0.183102	0.183102

Across 303114 regions, the number of digits after the decimal point differs substantially:

bigtools:
303114 3

Kent:
     30 0
      2 1
     41 2
    365 3
   3248 4
  46830 5
 229797 6
  22791 7
     10 8

This suggests bigwigaverageoverbed truncates or rounds values to 3 decimal places.

If the goal is to match the Kent utility behavior, it might be preferable to output full floating-point precision (or match the formatting used in bigWigAverageOverBed), since the current output may lose information.

[rabyj]

I was hoping to use this tool for large-scale processing (averaging ~303k regions across ~20k bigWig files), but the current precision difference makes it difficult to use as a drop-in replacement for the Kent utility.

[rabyj]

I found this interesting manifesto, it's mainly about AI rewrite of bioinformatics tools, but absolutely applies to non-AI rewrites too:
https://rewrites.bio

Reading it made me think of this issue right here, about decimal precision changes

2.2

Emulate exactly

The goal is a faster tool that produces the same results. A rewrite that changes outputs, even as "improvements", is a different tool. It cannot inherit the original's validation or be substituted without re-validation.

Emulate exactly means:

Deterministic tools: byte-for-byte identical output files
Floating-point tools: results within acceptable numerical precision (defined by scientists, not convenience)
Everything counts: header formats, column ordering, file naming, summary statistics

The section on stewardship also seems relevant to this repo, considering the last commit was made 7 months ago (and the two most recent issues have not had any main contributor comment). Creating a tool that could have significant usage, and then leaving, is not exactly responsible. The fact users can fork if the project is abandoned isn't enough.

[jackh726]

Hi @rabyj

Sorry, I did see this issue and I did briefly look at it (including long before this issue was filed). I didn't find the source of the float truncation, but I also have not had the time to dig in fully.

I'll also say, as it pertains to stewardship, etc.: I wrote this during my PhD (albeit not directly related to my PhD project), and could more easily dedicate time to it. I've since graduated and am doing entirely different things, not even in bioinformatics. I am more than happy to work with authors submitting pull requests, and I definitely wish I could spend more time digging in myself on these issues, but my time is limited. I'm sorry for that - I am trying my best to balance this versus the other things that I must do. It's particularly hard when the time I can spend doesn't lead to any direct changes (like, the time I spend investigating without discovering the root cause), which can lead it to drop off my radar.

I will also say, on emulation: I do not entirely agree with the definition provided. The bigWig and bigBed files have a spec as defined in their original paper. Technically, changes within that spec is "allowed" - the best example for bigtools around this is the position of the different data blocks within the file. In this case, "floating-point tools" are covered in the quote: there is an argument (which I have thought about before) that three decimal places is sufficient numerical precision for this type of data. From the many, many analyses I have done on bigWigs, three decimal places is more than enough precision.

---

All that being said, I hear you. I would like to fix this issue. If you can find it and submit a PR, I'll merge probably near-immediately. Otherwise, I need to find the time (and keep it on my radar) to get to this.

[rabyj]

Thank you, I understand none of this is simple to handle. We haven't invested as societies into infrastructures to steward important tools into the future. Open-source is complicated.

For additional context on my end, fact that the output wasn't a one-to-one match made me back off, because I was having weirdness with the data coming out of this tool when it was used for training a neural network. I didn't have time to investigate more deeply, it could have been unrelated to this format difference.

It's difficult to know without massive test coverage using real data if this difference could be causing weird edge cases. The fact that we gave up on reprocessing the data through our usual pipeline because of some paper related deadlines is not helping either. You might be entirely right that this isn't an actual problem, but the UCSC tools are battle-hardened, so matching their output is a "safe" bet.

[maximilianh]

@rabyi you can just ask any AI coding tool to add an option to change precision and then ask the tool to make a PR with that ?

…

On Wed, May 6, 2026 at 15:42 Kat ***@***.***> wrote: *rabyj* left a comment (jackh726/bigtools#97) <#97 (comment)> thank you — Reply to this email directly, view it on GitHub <#97 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AACL4TKBUNILT65HXHJZZBL4ZOITZAVCNFSM6AAAAACWVLBEDSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHM2DGOJRGQYTIMJXHE> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>. You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[jackh726]

I fixed this in fe72cd6 - will cut a release fairly soon.