New stratum-core structure

[lucasbalieiro]

address #1799 (comment);

• remove roles, since they moved to sv2-apps repo
• move protocols/v2 --> sv2/
• move protocols/v2 --> sv1/
• move protocols/stratum-translation --> stratum-core/stratum-translation
• move utils/buffer --> sv2/buffer-sv2
• adapted scripts and workflows where needed

[lucasbalieiro]

@GitGab19 I'm currently making some updates to run-integration-tests.sh, and it now points to the sv2-apps repo.

Could you rebase the sv2-apps repo so I can properly test the script?

Some integration tests are failing because certain changes (like those in #1856) aren’t being picked up.

[GitGab19]

@lucasbalieiro just force-pushed sv2-apps.

Let me know if you need anything else. Also, ping me when you want a review on this PR.

[lucasbalieiro]

@GitGab19, thanks for updating the repo.

I got the script working and made a few changes, including updates to the GitHub Workflows.

Right now, the integration test workflow is failing. It seems the reusable workflow from sv2-apps is overriding paths that don’t exist in this PR. You can check the logs for details. I think we’ll need to make some changes there as well.

Also, I’ve noticed that the test jds_should_not_panic_if_jdc_shutsdown sometimes hangs on my local setup (I haven’t investigated this yet).

If you have a chance, it would be really helpful if you could review the overall structure to see if I’ve made any mistakes.

[lucasbalieiro]

force-pushed a modification in the script to run the integration tests with cargo nextest to avoid the flaky tests that I was facing earlier.

Also, rebased with the latest changes on main

[GitGab19]

@GitGab19, thanks for updating the repo.

I got the script working and made a few changes, including updates to the GitHub Workflows.

Right now, the integration test workflow is failing. It seems the reusable workflow from sv2-apps is overriding paths that don’t exist in this PR. You can check the logs for details. I think we’ll need to make some changes there as well.

Also, I’ve noticed that the test jds_should_not_panic_if_jdc_shutsdown sometimes hangs on my local setup (I haven’t investigated this yet).

If you have a chance, it would be really helpful if you could review the overall structure to see if I’ve made any mistakes.

Sorry for being late, I'll take a look tomorrow morning and report my findings here!

[lucasbalieiro]

@GitGab19 I adapted the CI and tweaked the script to make it cross-compatible.

Could you update sv2-apps? CI need this PR there: #1864

[GitGab19]

Force-pushed sv2-apps repo, triggered again the CI and now IT pass!

[plebhash]

key-utils is used also by codec_sv2 for its examples

codec_sv2 can point to the published version on crates.io

cc @lucasbalieiro can you change that here?

[Shourya742]

Pinning it here, for later reflection: #1947 (comment)

[lucasbalieiro]

Updated this PR to catch-up with main.

Adapted to recent changes on sv2-apps repo.

CI Green

[GitGab19]

ACK.

I would wait for #1902 to get merged before merging this one though, since that PR is touching both protocol crates and apps.

[average-gary]

#1966 is preferred to #1902 as it moves persistence concerns to the application layer.

[GitGab19]

Now that #1966 has been merged, and the work about persistence will be done on the application layer, I think we can proceed with this PR.

@lucasbalieiro can you please solve conflicts and rebase it?

I'm pretty sure it will be the last time 😉

[lucasbalieiro]

rebased with the changes from #1966

[plebhash]

sorry guys, I just realized this #1967

should be fixed via #1968

[GitGab19]

@lucasbalieiro I just merged #1968 and updated sv2-apps repo.

You can rebase this PR and we should be really ready to get it merged :)

[Shourya742]

@lucasbalieiro Can you remove sv2.h from scripts, we don't use it.