Backported chainwork fix by HenrikJannsen · Pull Request #45 · bisq-network/bitcoinj

HenrikJannsen · 2025-12-23T04:48:45Z

This cherry-picked the commits suggested in:

but excluding the checkpoint related commits as suggested by @schildbach in #45 (comment) to reduce risks and as those changes are not strictly required for the hotfix.

Additional commits for missing methods and updates in build files and CI config have been applied as well.

This is a fix for Issue bitcoinj#3410 that will allow main net blocks above 849,137 to be processed. As of block 849,138 we can no longer fit total chainwork in a 12-byte *signed* field. This fix "kicks the can down the road" by making the field 12-bytes *unsigned*. We should open a new issue to address the long term need for bigger values. Note that converting the field to 12-byte unsigned precludes us from using the most-significant bit as a flag for a new format, but we should be able to pick some arbitrary value, say 0xA0 as a version flag and declare that values less than 0xA0 are "unversioned". This is BACKPORT to the 0.16 branch. In the master/0.17 branch the bigIntegerToBytes method has been moved to base.internal.BytUtils, so this commit differs in that one respect.

The old format will soon run out of bytes for the chain work value.

HenrikJannsen · 2025-12-23T10:25:09Z

Windows CI build fails at

bitcoinj/core/src/test/java/org/bitcoinj/core/PeerGroupTest.java

Line 543 in 3cc5f29

    
           assertTrue(watch.elapsed(TimeUnit.MILLISECONDS) >= timeout); // should not disconnect before timeout

. I guess this is not related to the code changes.

… backport of the chainwork bugfix. See bisq-network/bitcoinj#45 Signed-off-by: HenrikJannsen <boilingfrog@gmx.com>

schildbach · 2025-12-23T11:47:59Z

Not sure if it's worth the effort, but in order to lower the risk of regressions you could leave out the BuildCheckpoint-related changes (including the picocli dependency) for now. Reasons:

It will take a bit until a new checkpoint is due and then there is the "safety margin" (default: 7 days) that comes on top of it.
Even if the checkpoints would not be updated any more, the effect on user experience would be gradual (minimal at first, only getting more severe after a couple of months have passed without up-to-date checkpoints).
If you want to, you can always generate an up-to-date checkpoints file using bitcoinj 0.17 – no need to use your fork for that.

If you decide you don't need more recent checkpoints for the next couple of weeks at all (again, the impact will be minimal at first), then you could also exclude the changes related to CheckPointmanager from this PR.

In other words: For a hotfix, only the changes related to SPVBlockStore are truly relevant. Checkpoints could come next year when the responsible developer is available again.

In any case, I'd strongly recommend developing a strategy to update bitcoinj at least to version 0.16.x which we will probably support with critical updates for some time.

HenrikJannsen · 2025-12-23T12:06:16Z

Not sure if it's worth the effort, but in order to lower the risk of regressions you could leave out the BuildCheckpoint-related changes (including the picocli dependency) for now. Reasons:
* It will take a bit until a new checkpoint is due and then there is the "safety margin" (default: 7 days) that comes on top of it.

* Even if the checkpoints would not be updated any more, the effect on user experience would be gradual (minimal at first, only getting more severe after a couple of months have passed without up-to-date checkpoints).

* If you want to, you can always generate an up-to-date checkpoints file using bitcoinj 0.17 – no need to use your fork for that.
If you decide you don't need more recent checkpoints for the next couple of weeks at all (again, the impact will be minimal at first), then you could also exclude the changes related to CheckPointmanager from this PR.

In other words: For a hotfix, only the changes related to SPVBlockStore are truly relevant. Checkpoints could come next year when the responsible developer is available again.

In any case, I'd strongly recommend developing a strategy to update bitcoinj at least to version 0.16.x which we will probably support with critical updates for some time.

Thanks! OK, I will remove those commits. I did not plan to provide checkpoint updates in that release, only the minimal fixes, but for sure lowers the risk in such a quick hotfix.

Yes we are aware that its very bad that we are such far behind the upstream version, but as Musig on Bisq 2 will use BDK and we put all dev efforts into that, we tried to limit efforts on Bisq 1 - which went obviously too far by not following up better with this issue...

schildbach · 2025-12-23T12:06:22Z

On another note, I'm confused about the presence of a backport of our "kicking the can down the road" hotfix on this PR. I thought Bisq has already hotfixed this when the issue came up in June 2024?

Are you sure this PR is against the correct branch that is actually used in Bisq?

HenrikJannsen · 2025-12-23T12:07:48Z

@schildbach Do you think the failed test (#45 (comment)) is critical? It seems to me a CI environment issue with timing. I tried a bit with newer versions but that caused then other issues.

This gets rid of several bytes to/from string conversions.

This is meant to make sure the buffer position is always right.

…hreading`

It's available as a field already.

Existing V1 files are automatically migrated to V2 format. Includes a test for migration from V1 to V2 format. This requires `getRingCursor()` to be changed from private to package-private.

- Use protobuf-java:3.18.0 instead of protobuf-java:3.6.1 - Use mavenCentral instead of jcenter

HenrikJannsen · 2025-12-23T12:12:16Z

On another note, I'm confused about the presence of a backport of our "kicking the can down the road" hotfix on this PR. I thought Bisq has already hotfixed this when the issue came up in June 2024?

Are you sure this PR is against the correct branch that is actually used in Bisq?

Yes, the latest Bisq release used commit b005953e5eec339a82daf4866f85518091f6b9f6 of our BitcoinJ branch.

schildbach · 2025-12-23T12:18:17Z

@schildbach Do you think the failed test (#45 (comment)) is critical? It seems to me a CI environment issue with timing. I tried a bit with newer versions but that caused then other issues.

I don't think it's caused by a regression introduced by this PR. More likely it's a fluke of these PeerGroupTest tests. If you re-run the failed test using the GitHub UI, is the failure persistent?

HenrikJannsen · 2025-12-23T12:20:28Z

@schildbach Do you think the failed test (#45 (comment)) is critical? It seems to me a CI environment issue with timing. I tried a bit with newer versions but that caused then other issues.

I don't think it's caused by a regression introduced by this PR. More likely it's a fluke of these PeerGroupTest tests. If you re-run the failed test using the GitHub UI, is the failure persistent?

Before it was failing on Windows with Java 15 now with Java 11. So its not persistent, but seems Windows is mostly affected.

… backport of the chainwork bugfix. See bisq-network/bitcoinj#45

bisqadmin · 2025-12-23T13:24:50Z

utACK

hiciefte

utACK

… backport of the chainwork bugfix. See bisq-network/bitcoinj#45

msgilligan and others added 2 commits June 24, 2024 00:28

StoredBlock: add serializeCompactV2(), deserializeCompactV2()

2f4cebd

The old format will soon run out of bytes for the chain work value.

HenrikJannsen force-pushed the backported_chainwork_fix branch 12 times, most recently from 898691f to 3cc5f29 Compare December 23, 2025 10:00

This was referenced Dec 23, 2025

Use bitcoinj with chainwork fix bisq-network/bisq#7567

Merged

stuck in an error loop during SPV resync bisq-network/bisq#7566

Closed

schildbach and others added 8 commits December 23, 2025 19:09

SPVBlockStore: lock the store file as early as possible

af33656

SPVBlockStore: compare magic header as bytes, not string

52dc892

This gets rid of several bytes to/from string conversions.

SPVBlockStore: rewind buffer directly before accesses to magic header

2bed866

This is meant to make sure the buffer position is always right.

Add ReentrantLock lock(Class clazz) method in `org.bitcoinj.utils.T…

0248c4d

…hreading`

SPVBlockStore: don't pass buffer into setRingCursor(), getRingCursor()

238d57c

It's available as a field already.

SPVBlockStore: support 32 byte chain work

d10de9c

Existing V1 files are automatically migrated to V2 format. Includes a test for migration from V1 to V2 format. This requires `getRingCursor()` to be changed from private to package-private.

Add Utils.resetMocking() method

83054b8

Update github workflow and build files:

7dc0851

- Use protobuf-java:3.18.0 instead of protobuf-java:3.6.1 - Use mavenCentral instead of jcenter

HenrikJannsen force-pushed the backported_chainwork_fix branch from 3cc5f29 to 7dc0851 Compare December 23, 2025 12:13

HenrikJannsen requested review from alvasw and hiciefte December 23, 2025 12:23

HenrikJannsen added a commit to HenrikJannsen/bisq that referenced this pull request Dec 23, 2025

Use bitcoinj commit 7dc0851a807857ba19f76824e48d75ab0390eca7 with the…

132e15d

… backport of the chainwork bugfix. See bisq-network/bitcoinj#45

hiciefte approved these changes Dec 23, 2025

View reviewed changes

bisqadmin merged commit 52bfceb into bisq-network:main Dec 23, 2025
11 of 12 checks passed

HenrikJannsen mentioned this pull request Dec 23, 2025

Use backported bitcoinj fix for release branch bisq-network/bisq#7569

Closed

HenrikJannsen added a commit to HenrikJannsen/bisq that referenced this pull request Dec 23, 2025

Use bitcoinj commit 7dc0851a807857ba19f76824e48d75ab0390eca7 with the…

04c3ef4

… backport of the chainwork bugfix. See bisq-network/bitcoinj#45

HenrikJannsen mentioned this pull request Jan 1, 2026

For Cycle 77 bisq-network/compensation#1938

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Backported chainwork fix#45

Backported chainwork fix#45
bisqadmin merged 10 commits intobisq-network:mainfrom
HenrikJannsen:backported_chainwork_fix

HenrikJannsen commented Dec 23, 2025 •

edited

Loading

Uh oh!

HenrikJannsen commented Dec 23, 2025

Uh oh!

schildbach commented Dec 23, 2025 •

edited

Loading

Uh oh!

HenrikJannsen commented Dec 23, 2025

Uh oh!

schildbach commented Dec 23, 2025

Uh oh!

HenrikJannsen commented Dec 23, 2025

Uh oh!

HenrikJannsen commented Dec 23, 2025

Uh oh!

schildbach commented Dec 23, 2025

Uh oh!

HenrikJannsen commented Dec 23, 2025

Uh oh!

bisqadmin commented Dec 23, 2025

Uh oh!

hiciefte left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Conversation

HenrikJannsen commented Dec 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

HenrikJannsen commented Dec 23, 2025

Uh oh!

schildbach commented Dec 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

HenrikJannsen commented Dec 23, 2025

Uh oh!

schildbach commented Dec 23, 2025

Uh oh!

HenrikJannsen commented Dec 23, 2025

Uh oh!

HenrikJannsen commented Dec 23, 2025

Uh oh!

schildbach commented Dec 23, 2025

Uh oh!

HenrikJannsen commented Dec 23, 2025

Uh oh!

bisqadmin commented Dec 23, 2025

Uh oh!

hiciefte left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

HenrikJannsen commented Dec 23, 2025 •

edited

Loading

schildbach commented Dec 23, 2025 •

edited

Loading