feat: Consider additionally supporting downloading Northstar from Thunderstore

[GeckoEidechse]

What feature would you like added?
Viper currently downloads Northstar from GitHub but we also upload Northstar to Thunderstore as well. In fact Thunderstore version gets auto-upload by CI on release.

Why should this feature be added?
The reason for supporting Thunderstore Northstar download is a bit of selfish one. We're planning to also auto-upload release candidates to their own package on Thunderstore (NorthstarReleaseCandidate vs Northstar). By supporting Thunderstore download, it would be really easy to then add support for selecting a release channel in Viper. So player could then pick whether they want to be in "stable" or "release candidate" channel. "stable" would obviously be the default while "release candidate" would have to be changed to manually.
That way when we enable auto-upload release candidates we can then point playtesters to just select "release candidate" channel in Viper to playtest the newest release candidates which is way easier for them to do compared to having to manually download a release-candidate from GitHub actions everytime we do a new one. Especially as Viper already supports auto-download on update.

Additional Info
I'd still keep support for GitHub and then have an option to select download source. Whether GitHub or Thunderstore should be the default is a decision I'll leave up to you. Currently it makes no difference :P

[0neGal]

I completely agree with everything and I'll have a look at implementing this, it'll just be a switch for selecting where the Northstar release will be fetched and downloaded from, which would then have GitHub, Thunderstore Stable and Unstable.

I think "release candidate" can be confusing than something like "unstable", while they aren't exactly the same thing, explaining to a user how releases are drafted and tested is a lot just to explain that it's a less stable but more upstream version.

Feel free to lmk if you think something else is better...

As for implementing this, just to confirm, the Zip archive on Thunderstore is identical in layout to the GitHub release, correct? If so we only actually need to update some stuff in requests.js and it would be able to download from there (not counting implementing a switch for picking the option, just purely getting it to install...)

Whether GitHub or Thunderstore should be the default is a decision I'll leave up to you. Currently it makes no difference :P

Personally I think Thunderstore should be default...

By supporting Thunderstore download, it would be really easy to then add support for selecting a release channel in Viper.

I do wonder, how does our code for checking if Northstar is up to date, work with rc suffixed releases? I assume it just assumes it's always outdated, so might have to revisit that...

[Alystrasz]

Why bothering creating GitHub, Thunderstore Stable or Unstable channels, when we can just fetch all available versions from Thunderstore and let users choose themselves (à-la-Minecraft Launcher, if it still works this way)?

@GeckoEidechse release notes are not published on Thunderstore, only on GitHub releases, right?

[0neGal]

Why bothering creating GitHub, Thunderstore Stable or Unstable channels, when we can just fetch all available versions from Thunderstore and let users choose themselves (à-la-Minecraft Launcher, if it still works this way)?

It really wouldn't take a lot to implement, and if Thunderstore is down it's always good to have an alternative, and likewise with GitHub.

[GeckoEidechse]

which would then have GitHub, Thunderstore Stable and Unstable.

Sounds like a good idea. Probably best to hint somewhere in UI that there's not a difference between "Thunderstore stable" and "GitHub", maybe display the most current version next to download source when selecting download source?

I think "release candidate" can be confusing than something like "unstable", while they aren't exactly the same thing, explaining to a user how releases are drafted and tested is a lot just to explain that it's a less stable but more upstream version.
Feel free to lmk if you think something else is better...

Don't really have an idea so I'll leave it up to you :P

We're planning to also do a rolling release "git main" package in the future but we still need to figure out how to do infrastructure for that. I guess that one could be called "dev" then ^^

As for implementing this, just to confirm, the Zip archive on Thunderstore is identical in layout to the GitHub release, correct?

Sadly no but almost. When downloading from Thunderstore the resulting zip contains a folder called Northstar in its root. Going into that folder the files are the same as those from GitHub.
(On Thunderstore we do not ship ns_startup_args{_dedi}.txt as it would cause r2mm to overwrite each Northstar update. We still have to drop them from GitHub release. NorthstarLauncher will self generate those files if they do not exist.)

By supporting Thunderstore download, it would be really easy to then add support for selecting a release channel in Viper.

I do wonder, how does our code for checking if Northstar is up to date, work with rc suffixed releases? I assume it just assumes it's always outdated, so might have to revisit that...

I'd go for picking newest in that channel. So if newest unstable is older then newest stable, still stick with newest unstable. For version checking on NorthstarMasterServer we used the semver package, see https://github.com/R2Northstar/NorthstarMasterServer/pull/55/files#diff-7ae45ad102eab3b6d7e7896acd08c427a9b25b346470d7bc6507b6481575d519R25

 

@GeckoEidechse release notes are not published on Thunderstore, only on GitHub releases, right?

Yup, so release note fetching would stay the same regardless of selected channel. We currently don't really do release notes for release candidates, though release note drafts should be publicly visible.

[0neGal]

Don't really have an idea so I'll leave it up to you :P

We're planning to also do a rolling release "git main" package in the future but we still need to figure out how to do infrastructure for that. I guess that one could be called "dev" then ^^

How about "GitHub (Stable)", "Thunderstore (Stable)", "Thunderstore (Canary)" (rc builds), and "Thunderstore (Unstable)"

I'd go for picking newest in that channel. So if newest unstable is older then newest stable, still stick with newest unstable. For version checking on NorthstarMasterServer we used the semver package, see https://github.com/R2Northstar/NorthstarMasterServer/pull/55/files#diff-7ae45ad102eab3b6d7e7896acd08c427a9b25b346470d7bc6507b6481575d519R25

Hmm, we'll just modify what requests.getLatestNsVersion() spits out (depending on the release source), and compare versions slightly differently then...

[GeckoEidechse]

How about "GitHub (Stable)", "Thunderstore (Stable)", "Thunderstore (Canary)" (rc builds), and "Thunderstore (Unstable)"

I'd use development over unstable, as unstable just feels too vague to me. Otherwise seems fine to me ^^

[0neGal]

I'd use development over unstable, as unstable just feels too vague to me. Otherwise seems fine to me ^^

The idea is rather to really empathize that it's not really stable... Developer may not exactly imply the stability of it to some people... Idk. We could also rename Canary to Testing, anything is really possible of course.

I'm not sure what better words there really are to describe it really. We could also go all the way with the Discord route and have Stable, PTB (Public Test Build), and Canary. And then note that Canary is not stable, PTB is meant for testing, and Stable is the recommended build.

[GeckoEidechse]

Maybe combine the two words and use unstable, development? idk ^^

[0neGal]

Maybe combine the two words and use unstable, development? idk ^^

Considering everything else is a single short word and nothing more, that'd be a very long line...

GitHub (Stable)
Thunderstore (Stable)
Thunderstore (PTB)
Thunderstore (Unstable Development)

Keep in mind the UI this would be in, though, now that I think of it, In-Development might work, aka:

GitHub (Stable)
Thunderstore (Stable)
Thunderstore (PTB)
Thunderstore (In-Development)

Ofc whether it's PTB or Canary, or whatever doesn't matter much and if nobody has an opinion I'll figure out what fits best, from my opinion that is...

[GeckoEidechse]

So turns out Thunderstore only allows MAJOR.MINOR.PATCH versioning (c.f.: thunderstore-io/Thunderstore#634). This means stuff like v1.9.1-rc1 wouldn't work.

To circumvent this, we are considering doing MAJOR.MINOR.100*PATCH+RCNUMBER, i.e. v1.9.1-rc1 -> v1.9.101 for the time until Thunderstore changes their requirements (if ever).

At least for Viper this means a change in versioning detection library won't be needed for now :P

[GeckoEidechse]

The release candidate Thunderstore package (manually uploaded for now):
https://northstar.thunderstore.io/package/northstar/NorthstarReleaseCandidate/

[GeckoEidechse]

At least for Viper this means a change in versioning detection library won't be needed for now :P

One pain point I just realised is that the Northstar mods version will still say v1.9.1-rc1 (the way they should). Meaning checking if version installed is newest on Viper will be a bit of pain. Basically you'd have to implement the same calculation as above to go from "proper rc semver release number" to "thunderstore release number" :c

[0neGal]

So turns out Thunderstore only allows MAJOR.MINOR.PATCH versioning (c.f.: thunderstore-io/Thunderstore#634). This means stuff like v1.9.1-rc1 wouldn't work.

I suggest waiting until that is fully sorted, as it would not only be a waste of work getting it solved and then having to scrap it later, but it'll also likely require a bit of work for it to function properly.

One pain point I just realised is that the Northstar mods version will still say v1.9.1-rc1 (the way they should). Meaning checking if version installed is newest on Viper will be a bit of pain. Basically you'd have to implement the same calculation as above to go from "proper rc semver release number" to "thunderstore release number" :c

Again, it seems easier to wait till everything is solved with the whole suffix...

[GeckoEidechse]

So based on discussion in discussion in thunderstore-io/Thunderstore#634 as well as in direct messages with Mythic I'm under the impression that this will take quite some time to implement. In particular it's not yet clear whether there will ever be suffix support as it might just be changed to add support different release channels, which of course takes time.

From the Northstar side of things, we're planning to add auto-uploading rc builds to separate package some time this weekend. Given the limitations by Thunderstore we'll stick to encoded version numbers e.g. v1.9.101 as mentioned above.

For Viper I guess the best course of action is to just add support for downloading the stable package from Thunderstore for now, preferably in a more generic way to allow for easy extending to other download sources in the future.
After that I could look into downloading release candidate package in Viper. Maybe to keep things simpler from Viper side of things we just take over the version number of Thunderstore i.e. also display v1.9.101 and force Northstar reinstall on switching download source to avoid having an rc like v1.9.101 block a technically newer version like v1.9.2 (stable).