Skip to content

Support different Parentchain configs for Integritee, TargetA, TargetB and default to AssetTip for TargetB (aka support Asset Hub)#1652

Merged
brenzi merged 41 commits intomasterfrom
cl/support-assethub-try
Dec 20, 2024
Merged

Support different Parentchain configs for Integritee, TargetA, TargetB and default to AssetTip for TargetB (aka support Asset Hub)#1652
brenzi merged 41 commits intomasterfrom
cl/support-assethub-try

Conversation

@clangenb
Copy link
Contributor

@clangenb clangenb commented Nov 22, 2024

This was mostly accomplished introducing additional type parameters, or generics to support config discrepancies.

Other changes:

  • Remove the concrete ParentchainApi to prevent ambiguities. We explicitly use now IntegriteeRpcApi, TargetARpcApi, TargetBRpcApi`, or just have generic type parameters.

Testing Against Asset Hub:

  • Currently failing at funding vault account on assethub.
[2024-11-26T18:03:54.396132Z TRACE itp_stf_state_handler::file_io::sgx] writing state to: "/home/ubuntu/worker/integritee-worker/bin/shards/4ntWXUmxH3QCZJdjj2rwXv3TmagV6EX6rKFBpTu3F841/1732644234396106070_state.bin"
[2024-11-26T18:03:54.396323Z TRACE itp_stf_state_handler::file_io::sgx] state encrypted and stored in 0.0002s
[2024-11-26T18:03:54.396438Z INFO  enclave_runtime::shard_vault] [TargetB] send existential funds from enclave account to vault account: 9312854203
[2024-11-26T18:03:54.396481Z INFO  enclave_runtime::shard_vault] [TargetB] vault funding call: 0x0a0300d8141be0567296162299933097eed1e0a46f4a3d77f9f716472c289450ae20c707bbe0162b02
[2024-11-26T18:03:54.396Z DEBUG integritee_service::ocall_bridge::bridge_api] Requesting WorkerOnChain OCall API instance
[2024-11-26T18:03:54.396Z DEBUG integritee_service::ocall_bridge::worker_on_chain_ocall]     Entering ocall_worker_request
[2024-11-26T18:03:54.430Z INFO  integritee_service::main_impl] [TargetB] Received finalized header update (16), syncing parent chain...
[2024-11-26T18:03:54.430Z TRACE integritee_service::parentchain_handler] [TargetB] Getting current head
[2024-11-26T18:03:54.448Z INFO  integritee_service::parentchain_handler] [TargetB] Syncing blocks from 16 to 16
[2024-11-26T18:03:54.448720Z TRACE enclave_runtime::shard_creation_info] fetched shard creation header from state: ShardCreationInfo { integritee: Some(BlockMetadata { number: 7, hash: 0x17f0c7387b53ea3545fc91de9ae6039cf7ce62eebd94e3ccad6e8042969a132a, timestamp: None }), target_a: None, target_b: Some(BlockMetadata { number: 20, hash: 0x7c357405d27c757c4c10e48b50c4b7be2d529cecb18353512665026ef1cadc0a, timestamp: None }) }
[2024-11-26T18:03:54.448Z TRACE integritee_service::parentchain_handler] [TargetB] shard creation block: BlockMetadata { number: 20, hash: 0x7c357405d27c757c4c10e48b50c4b7be2d529cecb18353512665026ef1cadc0a, timestamp: None }
[2024-11-26T18:03:54.448Z DEBUG integritee_service::parentchain_handler] [TargetB] Found 0 block(s) to sync in this chunk. immediate import=false
[2024-11-26T18:03:54.449599Z TRACE enclave_runtime::utils] extrinsic mortality checkpoint: 8 0xa7e8…6878
[2024-11-26T18:03:54.449Z DEBUG integritee_service::ocall_bridge::bridge_api] Requesting WorkerOnChain OCall API instance
[2024-11-26T18:03:54.490Z DEBUG integritee_service::ocall_bridge::worker_on_chain_ocall] Enclave wants to send 1 extrinsics to parentchain: TargetB. await each inclusion: true
[2024-11-26T18:03:54.497Z WARN  substrate_api_client::rpc::tungstenite_client::client] Expected subscription, but received an id response instead: Object {"error": Object {"code": Number(1010), "data": String("Transaction has a bad signature"), "message": String("Invalid Transaction")}, "id": String("1"), "jsonrpc": String("2.0")}
[2024-11-26T18:03:59.399Z INFO  integritee_service::main_impl] [Integritee] Received finalized header update (9), syncing parent chain...
[2024-11-26T18:03:59.399Z TRACE integritee_service::parentchain_handler] [Integritee] Getting current head
[2024-11-26T18:03:59.399Z INFO  integritee_service::main_impl] [TargetB] Received finalized header update (17), syncing parent chain...
[2024-11-26T18:03:59.399Z TRACE integritee_service::parentchain_handler] [TargetB] Getting current head
[2024-11-26T18:03:59.418Z INFO  integritee_service::parentchain_handler] [TargetB] Syncing blocks from 16 to 17

Critical line from above log:

[2024-11-26T18:03:54.497Z WARN  substrate_api_client::rpc::tungstenite_client::client] Expected subscription, but received an id response instead: Object {"error": Object {"code": Number(1010), "data": String("Transaction has a bad signature"), "message": String("Invalid Transaction")}, "id": String("1"), "jsonrpc": String("2.0")}

Bad signature error, we can fund the enclave account from Alice. So our extrinsic handling concept is ok, but I have to double check the extrinsic factory code in the enclave side, which is producing the signature.

@clangenb clangenb changed the base branch from ab/asset-hub-second-attempt to master November 22, 2024 13:21
@clangenb clangenb added A0-core Affects a core part B1-releasenotes C1-low 📌 Does not elevate a release containing this beyond "low priority" E3-hardmerge PR introduces a lot changes in a lot of files. Requires some effort to merge or rebase to. labels Nov 22, 2024
} else {
AccountData::default()
};
let account_data = api.get_account_data(&accountid).unwrap().unwrap_or_default();
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have changed the implementation. It returns now, our AccountData instead of the one from the api-client. This simplifies some things.

Comment on lines +31 to +33
pub struct ExtrinsicsFactoryMock<C> {
_phantom: PhantomData<C>,
}
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can't derive default if the concrete type doesn't implement default. This is a well-known rust bug, and we have to implement default manually.

rpc::{tungstenite_client::TungsteniteRpcClient, Error as RpcClientError},
};

pub type ParentchainApi = Api<ParentchainRuntimeConfig, TungsteniteRpcClient>;
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I removed this type to prevent ambiguities. We use IntegriteeApi, TargetAApi, TargetBApi in all cases now, or we are generic over traits.

@clangenb clangenb changed the title WIP asset hub support Support different Parentchain configs for Integritee, TargetA, TargetB and default to AssetTip for TargetB (aka support Asset Hub) Nov 22, 2024
@clangenb clangenb requested a review from brenzi November 22, 2024 15:38
@brenzi
Copy link
Collaborator

brenzi commented Dec 16, 2024

testing with

  • polkadot 1.16.1-835e0767fe8
  • polkadot-parachain 1.16.1-835e0767fe8
  • zombienet-linux-x64 spawn --provider native zombienet/rococo-local-with-integritee-and-asset-hub.toml

observations:

If I don't specify target_b as shielding target:
./integritee-service -c -u ws://172.17.0.1 --target-b-parentchain-rpc-url ws://172.17.0.1 --target-b-parentchain-rpc-port 9954 --ws-external --enable-metrics run --skip-ra --dev &> worker-ah.log

  • everything starts fine and sidechain syncs both parentchain and then produces blocks
  • enclave account got endowed with 0.026 ROC on target_b

If I do specify shielding-target = target-b:
./integritee-service -c -u ws://172.17.0.1 --target-b-parentchain-rpc-url ws://172.17.0.1 --target-b-parentchain-rpc-port 9954 --ws-external --enable-metrics run --shielding-target target_b --skip-ra --dev &> worker-ah.log

Therefore, the enclave IS capable of sending extrinsics to target_b Asset Hub!

Next, let's see if shielding works:

  1. Charlie sends 2.34 ROC to Shielding target on AH.
  2. ./integritee-cli trusted --mrenclave $MRENCLAVE balance //Charlie
  3. zero. didn't work. shielding target balance did increase on AH, but no issuance on L2

edit: reproducing the above leads to varying outcome. there seems to be some race condition:
creating proxy can end in "Inability to pay some fees (e.g. account balance too low)")

�[0m�[38;5;8m[�[0m2024-12-16T14:34:06.120276Z �[0m�[32mINFO �[0m enclave_runtime::shard_vault�[0m�[38;5;8m]�[0m [TargetB] send existential funds from enclave account to vault account: 9313639703
�[0m�[38;5;8m[�[0m2024-12-16T14:34:06.120300Z �[0m�[32mINFO �[0m enclave_runtime::shard_vault�[0m�[38;5;8m]�[0m [TargetB] vault funding call: 0x0a0300cdd6e63c32474d475bbe81f4a2c3861d8b9e306ca7672f3d62bf258a2c401e700717dd222b02
[2024-12-16T14:34:06.149Z WARN  substrate_api_client::rpc::tungstenite_client::client] Expected subscription, but received an id response instead: Object {"error": Object {"code": Number(1010), "data": String("Inability to pay some fees (e.g. account balance too low)"), "message": String("Invalid Transaction")}, "id": String("1"), "jsonrpc": String("2.0")}

tried to send 9mROC. balance of enclave account was 26mROC. proxy deposit base is 0.66 mROC

next try gets one step further:

[2024-12-16T14:44:30.171Z INFO  integritee_service::account_funding] Proxy Deposit is 667999980 base + 1099989 per proxy
�[0m�[38;5;8m[�[0m2024-12-16T14:44:30.171554Z �[0m�[33mWARN �[0m enclave_runtime::shard_vault�[0m�[38;5;8m]�[0m Failed to fetch shard vault account: Other("failed to fetch shard vault account. has it been initialized?")
[TargetB] initializing proxied shard vault account now
�[0m�[38;5;8m[�[0m2024-12-16T14:44:30.171716Z �[0m�[32mINFO �[0m enclave_runtime::shard_vault�[0m�[38;5;8m]�[0m shard vault account derived pubkey: 0xcdd6e63c32474d475bbe81f4a2c3861d8b9e306ca7672f3d62bf258a2c401e70
�[0m�[38;5;8m[�[0m2024-12-16T14:44:30.171795Z �[0m�[33mWARN �[0m sp_io::storage�[0m�[38;5;8m]�[0m storage::start_transaction unimplemented
�[0m�[38;5;8m[�[0m2024-12-16T14:44:30.171827Z �[0m�[33mWARN �[0m sp_io::storage�[0m�[38;5;8m]�[0m storage::commit_transaction unimplemented
�[0m�[38;5;8m[�[0m2024-12-16T14:44:30.172458Z �[0m�[32mINFO �[0m enclave_runtime::shard_vault�[0m�[38;5;8m]�[0m [TargetB] send existential funds from enclave account to vault account: 9313639703
�[0m�[38;5;8m[�[0m2024-12-16T14:44:30.172506Z �[0m�[32mINFO �[0m enclave_runtime::shard_vault�[0m�[38;5;8m]�[0m [TargetB] vault funding call: 0x0a0300cdd6e63c32474d475bbe81f4a2c3861d8b9e306ca7672f3d62bf258a2c401e700717dd222b02
[2024-12-16T14:44:30.205Z INFO  integritee_service::main_impl] [TargetB] Received finalized header update (8), syncing parent chain...
[2024-12-16T14:44:30.207Z INFO  integritee_service::parentchain_handler] [TargetB] Syncing blocks from 8 to 8
[L1Event:TargetB] Balances::Endowed
[L1Event:TargetB] BalanceTransfer { from: 6d6f646c506f745374616b650000000000000000000000000000000000000000 (5EYCAe5c...), to: b684ec0c49ded00365338c6620358e54bbcd58ab9b70f7ff546e7cbdef63a873 (5GC24LJB...), amount: 6964864 }
[2024-12-16T14:44:30.229Z WARN  substrate_api_client::rpc::tungstenite_client::client] Expected subscription, but received an id response instead: Object {"id": String("1"), "jsonrpc": String("2.0"), "result": String("gEXU6RVTvCkJ40pq")}
[2024-12-16T14:44:33.217Z INFO  integritee_service::main_impl] [TargetB] Received finalized header update (9), syncing parent chain...
[2024-12-16T14:44:33.217Z INFO  integritee_service::main_impl] [Integritee] Received finalized header update (5), syncing parent chain...
[2024-12-16T14:44:33.218Z INFO  integritee_service::parentchain_handler] [Integritee] Syncing blocks from 4 to 5
[2024-12-16T14:44:33.218Z INFO  integritee_service::parentchain_handler] [TargetB] Syncing blocks from 8 to 9
[L1Event:TargetB] Balances::Endowed
[L1Event:TargetB] BalanceTransfer { from: 6d6f646c506f745374616b650000000000000000000000000000000000000000 (5EYCAe5c...), to: 8444498b6cce6acd2fff8628bc06ddcd8d6ffcbad920e9c3952f7a426a4de938 (5F48TqUR...), amount: 3482432 }
[L1Event:TargetB] Balances::Endowed
[L1Event:TargetB] BalanceTransfer { from: 1051397465c70b7c658fb986a0b16665bc153c264831dedebb1c3d9cfb8ce0d9 (5CS6ka2D...), to: cdd6e63c32474d475bbe81f4a2c3861d8b9e306ca7672f3d62bf258a2c401e70 (5GibWeWk...), amount: 9313639703 }
�[0m�[38;5;8m[�[0m2024-12-16T14:44:36.049785Z �[0m�[32mINFO �[0m enclave_runtime::shard_vault�[0m�[38;5;8m]�[0m [TargetB] register enclave signer as proxy for shard vault
�[0m�[38;5;8m[�[0m2024-12-16T14:44:36.049841Z �[0m�[32mINFO �[0m enclave_runtime::shard_vault�[0m�[38;5;8m]�[0m [TargetB] add proxy call: 0x2a01001051397465c70b7c658fb986a0b16665bc153c264831dedebb1c3d9cfb8ce0d90000000000
[L1Event:TargetB] Balances::Endowed
[L1Event:TargetB] BalanceTransfer { from: 6d6f646c506f745374616b650000000000000000000000000000000000000000 (5EYCAe5c...), to: 8444498b6cce6acd2fff8628bc06ddcd8d6ffcbad920e9c3952f7a426a4de938 (5F48TqUR...), amount: 3482432 }
[TargetB] initialized shard vault account: : 5GibWeWkmzsafynBpP9JgtbB4Rfp7aakS5UsrHeAi6K8AXjA

yet, proxy.ProxyAdded doesn't happen. xt got swallowed

It can even happen that the first funding extrinsic fails:
thread 'main' panicked at '[TargetB] Could not fund parentchain enclave account: ApiClient(UnexpectedTxStatus(Invalid))', service/src/main_impl.rs:885:9

@brenzi
Copy link
Collaborator

brenzi commented Dec 16, 2024

seems we see this very often. most likely the cause why scanning blocks for shielding trasnfers fails as the block doesn't even get processed: HeaderAncestryMismatch

Guess we need to change the pruning mode of the collators and validators in zombienet:
done here: integritee-network/parachain@c213c77

But the problem is another one: the initial sync imports until block 8 and the next light client import is block 14. see https://gist.github.com/brenzi/6a2cb568c41d1e437834dc745ae15904

hypothesis:

  • we miss block imports while we wait for the shard vault to be created

let xts = vault_extrinsics_factory.create_extrinsics(&[(call, mortality)], None)?;

ocall_api.send_to_parentchain(xts, &parentchain_id, false)?;
ocall_api.send_to_parentchain(xts, &parentchain_id, true)?;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this may help to not accidentally use the proxy before it got created fine

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

or maybe waiting for confirmations in the enclave blocks parentchain sync?

use itp_types::parentchain::ParentchainCall;
pub use peer_block_sync::*;

pub trait Verifier<ParentchainBlock, SignedSidechainBlock>: Send + Sync
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

all the sidechain stuff effectively only needs the header which is of equal type for all parentchains. Tried to untangle the mess....

@clangenb
Copy link
Contributor Author

So, interestingly, this commit 5698d95, which reverts another commit from this branch makes creating the proxy account work 5 out of 7 times, the initial funding has always worked. This unfortunately implies to me that we have a timing related issue. When it failed, I also got the ancestry error.

Below are the logs of a success and a failed run. I will analyze them later.

@clangenb
Copy link
Contributor Author

clangenb commented Dec 18, 2024

We still get header ancestry errors, this grep of the log seems very suspicious:

[2024-12-18T15:43:07.390968Z TRACE aura] import TargetB blocks until 10: 0x1bcb1bfd9accf6ec54cff1e3912957fb747375c34c4181115a3a80b0ee14e167
[2024-12-18T15:43:08.425345Z TRACE aura] import TargetB blocks until 8: 0x412362565dcd6e5314e41f80bdef84dfa262a3c4c07500bed2e61cc63a13684c
[2024-12-18T15:43:09.362615Z TRACE aura] import TargetB blocks until 8: 0x412362565dcd6e5314e41f80bdef84dfa262a3c4c07500bed2e61cc63a13684c
[2024-12-18T15:43:10.365110Z TRACE aura] import TargetB blocks until 11: 0x16e8e2b168373698bacc009ace3c74253122473382e3eece0629706777bacb5c
[2024-12-18T15:43:11.667661Z TRACE aura] import TargetB blocks until 8: 0x412362565dcd6e5314e41f80bdef84dfa262a3c4c07500bed2e61cc63a13684c
[2024-12-18T15:43:12.513881Z TRACE aura] import TargetB blocks until 8: 0x412362565dcd6e5314e41f80bdef84dfa262a3c4c07500bed2e61cc63a13684c
[2024-12-18T15:43:13.344340Z TRACE aura] import TargetB blocks until 8: 0x412362565dcd6e5314e41f80bdef84dfa262a3c4c07500bed2e61cc63a13684c
[2024-12-18T15:43:14.320314Z TRACE aura] import TargetB blocks until 8: 0x412362565dcd6e5314e41f80bdef84dfa262a3c4c07500bed2e61cc63a13684c
[2024-12-18T15:43:15.395286Z TRACE aura] import TargetB blocks until 12: 0x8034fbe73b1ea0b475a2c5913aa4727b5a539be959e6e7e71e16830ef8730cf0
[2024-12-18T15:43:16.356253Z TRACE aura] import TargetB blocks until 8: 0x412362565dcd6e5314e41f80bdef84dfa262a3c4c07500bed2e61cc63a13684c
[2024-12-18T15:43:17.549382Z TRACE aura] import TargetB blocks until 8: 0x412362565dcd6e5314e41f80bdef84dfa262a3c4c07500bed2e61cc63a13684c
[2024-12-18T15:43:18.742140Z TRACE aura] import TargetB blocks until 8: 0x412362565dcd6e5314e41f80bdef84dfa262a3c4c07500bed2e61cc63a13684c
[2024-12-18T15:43:19.469053Z TRACE aura] import TargetB blocks until 8: 0x412362565dcd6e5314e41f80bdef84dfa262a3c4c07500bed2e61cc63a13684c
[2024-12-18T15:43:20.426689Z TRACE aura] import TargetB blocks until 8: 0x412362565dcd6e5314e41f80bdef84dfa262a3c4c07500bed2e61cc63a13684c
[2024-12-18T15:43:21.299318Z TRACE aura] import TargetB blocks until 8: 0x412362565dcd6e5314e41f80bdef84dfa262a3c4c07500bed2e61cc63a13684c
[2024-12-18T15:43:22.282043Z TRACE aura] import TargetB blocks until 8: 0x412362565dcd6e5314e41f80bdef84dfa262a3c4c07500bed2e61cc63a13684c
[2024-12-18T15:43:23.368965Z TRACE aura] import TargetB blocks until 13: 0x8452d29b1b611ba7f06e803bbfb13f9004c36fc2f37acc3ad92b24a74c54acc5
[2024-12-18T15:43:24.320334Z TRACE aura] import TargetB blocks until 8: 0x412362565dcd6e5314e41f80bdef84dfa262a3c4c07500bed2e61cc63a13684c
[2024-12-18T15:43:25.298061Z TRACE aura] import TargetB blocks until 8: 0x412362565dcd6e5314e41f80bdef84dfa262a3c4c07500bed2e61cc63a13684c
[2024-12-18T15:43:26.313467Z TRACE aura] import TargetB blocks until 8: 0x412362565dcd6e5314e41f80bdef84dfa262a3c4c07500bed2e61cc63a13684c
[2024-12-18T15:43:27.275881Z TRACE aura] import TargetB blocks until 14: 0x165c556b676aedfbb88f9f1e523ec4956e47f232fc3e0ec4c326d54adaf1c25f

The only way I can see that this makes sense is here:

match self.peek_latest_target_b_parentchain_header() {

Edit: Ok this makes sense actually. Whenever there a parentchain block in the queue we can peek it (and get the number > 8), and then we fail to import it due to the ancestry rematch. Whenever there is nothing in the queue, we return the last finalized from our light client db.

@clangenb
Copy link
Contributor Author

the following command yields:

clear && less worker.log | grep -C 2 "filter"   
[2024-12-18T16:09:13.286758Z TRACE itc_parentchain_indirect_calls_executor::executor] xt_statuses:: [Success, Success, Success]
[2024-12-18T16:09:13.286864Z TRACE itc_parentchain_indirect_calls_executor::executor] extrinsic count :: 3
[2024-12-18T16:09:13.297792Z TRACE ita_parentchain_interface::integritee::event_handler] filtering transfer events to shard vault account: 0x9974260cb3a4d525a6f192b709796569799dabd983f1a5448d94fdd014ce4aab
[2024-12-18T16:09:13.298754Z TRACE ita_parentchain_interface::integritee] ExtrinsicFilter: attempting to execute indirect call with index [1, 0]
[2024-12-18T16:09:13.298839Z TRACE ita_parentchain_interface::integritee] ExtrinsicFilter: attempting to execute indirect call with index [3, 0]
--
[2024-12-18T16:09:15.511606Z TRACE itc_parentchain_indirect_calls_executor::executor] xt_statuses:: [Success, Success]
[2024-12-18T16:09:15.511709Z TRACE itc_parentchain_indirect_calls_executor::executor] extrinsic count :: 2
[2024-12-18T16:09:15.516665Z TRACE ita_parentchain_interface::integritee::event_handler] filtering transfer events to shard vault account: 0x9974260cb3a4d525a6f192b709796569799dabd983f1a5448d94fdd014ce4aab
[2024-12-18T16:09:15.517353Z TRACE ita_parentchain_interface::integritee] ExtrinsicFilter: attempting to execute indirect call with index [1, 0]
[2024-12-18T16:09:15.517405Z TRACE ita_parentchain_interface::integritee] ExtrinsicFilter: attempting to execute indirect call with index [3, 0]
--
[2024-12-18T16:09:27.358944Z TRACE itc_parentchain_indirect_calls_executor::executor] xt_statuses:: [Success, Success, Success, Success, Success]
[2024-12-18T16:09:27.359011Z TRACE itc_parentchain_indirect_calls_executor::executor] extrinsic count :: 5
[2024-12-18T16:09:27.378990Z TRACE ita_parentchain_interface::integritee::event_handler] filtering transfer events to shard vault account: 0x9974260cb3a4d525a6f192b709796569799dabd983f1a5448d94fdd014ce4aab
[2024-12-18T16:09:27.379532Z TRACE ita_parentchain_interface::integritee] ExtrinsicFilter: attempting to execute indirect call with index [1, 0]
[2024-12-18T16:09:27.379572Z TRACE ita_parentchain_interface::integritee] ExtrinsicFilter: attempting to execute indirect call with index [3, 0]
--
[2024-12-18T16:09:39.311636Z TRACE itc_parentchain_indirect_calls_executor::executor] xt_statuses:: [Success, Success]
[2024-12-18T16:09:39.311657Z TRACE itc_parentchain_indirect_calls_executor::executor] extrinsic count :: 2
[2024-12-18T16:09:39.315628Z TRACE ita_parentchain_interface::integritee::event_handler] filtering transfer events to shard vault account: 0x9974260cb3a4d525a6f192b709796569799dabd983f1a5448d94fdd014ce4aab
[2024-12-18T16:09:39.316180Z TRACE ita_parentchain_interface::integritee] ExtrinsicFilter: attempting to execute indirect call with index [1, 0]
[2024-12-18T16:09:39.316199Z TRACE ita_parentchain_interface::integritee] ExtrinsicFilter: attempting to execute indirect call with index [3, 0]
--
[2024-12-18T16:09:51.409562Z TRACE itc_parentchain_indirect_calls_executor::executor] xt_statuses:: [Success, Success, Success, Success, Success]
[2024-12-18T16:09:51.409676Z TRACE itc_parentchain_indirect_calls_executor::executor] extrinsic count :: 5
[2024-12-18T16:09:51.440125Z TRACE ita_parentchain_interface::integritee::event_handler] filtering transfer events to shard vault account: 0x9974260cb3a4d525a6f192b709796569799dabd983f1a5448d94fdd014ce4aab
[2024-12-18T16:09:51.441618Z TRACE ita_parentchain_interface::integritee] ExtrinsicFilter: attempting to execute indirect call with index [1, 0]
[2024-12-18T16:09:51.441677Z TRACE ita_parentchain_interface::integritee] ExtrinsicFilter: attempting to execute indirect call with index [3, 0]
--
[2024-12-18T16:10:03.333112Z TRACE itc_parentchain_indirect_calls_executor::executor] xt_statuses:: [Success, Success, Success]
[2024-12-18T16:10:03.333135Z TRACE itc_parentchain_indirect_calls_executor::executor] extrinsic count :: 3
[2024-12-18T16:10:03.342183Z TRACE ita_parentchain_interface::integritee::event_handler] filtering transfer events to shard vault account: 0x9974260cb3a4d525a6f192b709796569799dabd983f1a5448d94fdd014ce4aab
[2024-12-18T16:10:03.342647Z TRACE ita_parentchain_interface::integritee] ExtrinsicFilter: attempting to execute indirect call with index [1, 0]
[2024-12-18T16:10:03.342660Z TRACE ita_parentchain_interface::integritee] ExtrinsicFilter: attempting to execute indirect call with index [3, 0]

Which is missing logs from target b even though there should be logs too. So I guess we are missing something in the initialization of target b

@brenzi
Copy link
Collaborator

brenzi commented Dec 18, 2024

I reproduced the current state. It seems that the service and the enclave regularly - but not deterministically fail to send extrinsics. Be it service or enclave, they fail with txStatus "Invalid"

Also, I tested that the initial sync isn't disturbed by a transfer extrinsic which happened before registering the enclave (not such a surprise as we don't scan these blocks)

@brenzi
Copy link
Collaborator

brenzi commented Dec 19, 2024

@clangenb your plan made sense:

I removed AssetTip customizations and tried running target_b = relaychain as shielding target.

./integritee-service -c -u ws://172.17.0.1 --target-b-parentchain-rpc-url ws://172.17.0.1 --target-b-parentchain-rpc-port 9999 --ws-external --enable-metrics run --shielding-target target_b --skip-ra --dev &> worker-rococorelay.log

Same ancestry mismatch issues.

Next step: try the same with master and see if the issue was already there before this PR

@brenzi
Copy link
Collaborator

brenzi commented Dec 19, 2024

there seems to be a problem on master too:
https://gist.github.com/brenzi/e994fcf4223d8e27583e12fbf5adf929

We get BadSignature and it seems there is a race condition between the initial funding of the enclave account and the creation of the shard vault account

@clangenb
Copy link
Contributor Author

So, on master the bug doesn't happen for target a, does it? So maybe we want to test that target a still works on this branch?

@clangenb
Copy link
Contributor Author

clangenb commented Dec 19, 2024

The race condition might imply that something from target a writes to target b's storage, and depending on whether this already happened, funding the enclave works or not. What do you think?

Edit: or the integritee parentchain writes to target b.

Copy link
Collaborator

@brenzi brenzi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tested this with

zombienet-linux-x64 spawn --provider native zombienet/rococo-local-with-integritee-and-asset-hub.toml 
./integritee-service -c -u ws://172.17.0.1 --target-b-parentchain-rpc-url ws://172.17.0.1 --target-b-parentchain-rpc-port 9954 --ws-external --enable-metrics run --skip-ra --dev --shielding-target target_b &> worker-ah.log

shielding and unshielding to/from Asset hub works!

@brenzi brenzi merged commit 3e9ebb0 into master Dec 20, 2024
36 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

A0-core Affects a core part B1-releasenotes C1-low 📌 Does not elevate a release containing this beyond "low priority" E3-hardmerge PR introduces a lot changes in a lot of files. Requires some effort to merge or rebase to.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants