Experimental: Update XDR check between core and the host to handle ifdefs

[sisuresh]

Core currently hashes the .x files and compares them to the xdr in rs-soroban-env for curr builds. As part of https://github.com/stellar/pod-fsr/issues/12, we're looking into adding ifdefs to the xdr files to gate individual features. The ifdefs will cause xdr mismatches, so we'll have to figure out a solution here.

[leighmcculloch]

It might be enough to use the embedded git revision for the xdr in use. I don't remember why we went as far as hashing individual files, maybe we still have .x files in multiple locations back then. Today rs-stellar-xdr embeds the git sha of the stellar-xdr repo in use, and env already exposes it. Core should be able to check it equals the stellar-xdr git sha it is using. That should be enough.

• https://github.com/stellar/rs-stellar-xdr/blob/19ecc43e17797ad9233861af93b100a2a941ecfe/src/lib.rs#L128
• https://github.com/stellar/rs-soroban-env/blob/60398e4cf317eccc8ed32802104834bd1d994928/soroban-env-common/src/lib.rs#L49

[sisuresh]

One issue with using git revisions is that the current hash check compares per file, and certain xdr files are ignored. This allows core to take certain non-protocol related xdr updates without requiring a host update.

[sisuresh]

Some ideas -

1. Strip ifdefs on both sides (stellar-core and rs-stellar-xdr) before hashing.
2. Skip the check in core when XDR feature flags are active. We already skip it for vnext builds.

[leighmcculloch]

3. Hash the original .x files before preprocessing.
4. Preprocess the files first and hash those. (Not sure if that's what you mean by strip idefs)

[leighmcculloch]

I think the issue with why we saw failures on that prototype build is because I actually had a discrepancy, I was using different XDR in Env and Core. I'm not convinced there's actually an issue here now. Assuming both the rs-stellar-xdr and Core build pipelines hash the same files, either pre-processed, or original, the values should match.

Hashing the pre-processed file would be preferred, because that's what the code is being code generated on. But that may be more difficult to implement as it essentially requires two pipelines.

[sisuresh]

I think the issue with why we saw failures on that prototype build is because I actually had a discrepancy, I was using different XDR in Env and Core. I'm not convinced there's actually an issue here now.

Yeah in this case, only the "vnext" run should've had the updated xdr. In that case, we could set the requirement that a host bump is required as well.

We already have some discrepancies today because we allow core to build a vnext build using the current host, which is why we skip the xdr hash checks for vnext builds. We could change how this is done though - all builds for the next protocol version would use the next protocols host, not the current one, which much use matching xdr.

[sisuresh]

4. Preprocess the files first and hash those. (Not sure if that's what you mean by strip idefs)

Yeah but we would need to preprocess with no features enabled.

[sisuresh]

The xdr hashes in rs-stellar-xdr are generated in xdrgen, so any changes to that should be made after stellar/rs-stellar-xdr#498 is done.