Suboptimal TXID calculation

[tevador]

Some issues with the TXID calculation:

1. Doesn't explain why the input key images are hashes twice. IMO it's sufficient to have them in the prefix, which would save 1 hash.
2. The final hash hashes more than 128 bytes of data, which requires two calls to the Blake2b compression function. This has the same performance impact as one additional hash. Since 3 hashes take 96 bytes, the domain separation stuff could still fit into the remaining 32 bytes if it was less verbose.

[UkoeHB]

1. Doesn't explain why the input key images are hashes twice.

The input images are hashed separately to decouple them from the tx proposal prefix so future enote image structure changes don't affect the prefix. It also avoids needing to commit to input masks when you define a proposal, which may have subtle long-term impacts on innovations around how txs are constructed.

2. The final hash hashes more than 128 bytes of data, which requires two calls to the Blake2b compression function.

This kind of expense is unavoidable when using the 'Fiat-Shamir' transcript mode, which includes labels, tags, and lengths (in addition to domain separators). I did optimize all uses of the 'KDF' transcript mode to fit within 128 bytes.

In order to get a tx hash you need a ton of blake2b compression rounds for the internal data anyway, which makes me uncertain that the final hash is worth optimizing.

[tevador]

The input images are hashed separately to decouple them from the tx proposal prefix so future enote image structure changes don't affect the prefix. It also avoids needing to commit to input masks when you define a proposal, which may have subtle long-term impacts on innovations around how txs are constructed.

Some clients don't care about the proofs (wallets, light nodes and some pruned nodes), so the masked key and the masked commitment are useless for them. If they need to be hashed separately, it would be better to hide them together with the "proofs" hash:

hash1 (prefix) ------------------------\
                                        \
                                        |--- hash5 (TXID)
hash2 (e-note images) ------\           / 
                            |-- hash4--/
hash3 (proofs) -------------/

Light clients (which are bottlenecked by hashing rather than proof validation) would download prefix and hash4 and would only need to calculate hash1 and hash5. Full nodes would need 1 additional hash, but that's negligible compared to the full TX validation.

In order to get a tx hash you need a ton of blake2b compression rounds for the internal data anyway, which makes me uncertain that the final hash is worth optimizing.

If my math is correct, a 1-in/2-out TX has a prefix size of just under 256 bytes, which needs 2 Blake2b compressions. Light clients/wallets will get the proof data pre-hashed, so the additional compression call when calculating the TXID can be as high as 33% of the whole calculation for them.

[UkoeHB]

If they need to be hashed separately, it would be better to hide them together with the "proofs" hash:

Ok I agree with your approach and arguments, I will update this.

If my math is correct, a 1-in/2-out TX has a prefix size of just under 256 bytes

I think it's at least 257-262 bytes: tx version (3) + key image (32) + 2x( enote (93 = 32+32+8+20+1)) + fee (4-9 [this is a varint which can be up to 9 bytes]) + ephemeral pubkey (32). Since the prefix is using the FSTranscript mode, there's a bunch more bytes for labels, tags, lengths, and the domain separator + transcript prefix.

I just did a test and got 281 bytes using KDF mode and 662 bytes using FS mode for the tx proposal prefix transcript (if I remove labels then the FS mode is 328 bytes).

[UkoeHB]

Ok I agree with your approach and arguments, I will update this.

Done, the input images and tx proofs are now hidden behind a 'tx artifacts merkle root'. After slimming down labels and the domain separator, the tx id's transcript is 127 bytes in FS mode.

Note that the coinbase tx id's transcript is currently 345 bytes with one output.