From our discussion, we decided it would be beneficial to refactor the way the polling operation works, eliminating:
- Unnecessary re-subscriptions when not checking for "filter not found" in the error message (since the error message may not be consistent across all client implementations).
- Checking for missing events between time of unsubscription and re-subscription
- Subscription management (mapping)
Inspired by the way ethers.js handles subscriptions, the polling loop can be reduced down to checking for a new block at each polling interval. Once a new block is encountered, block subscription handlers can be called, and we can check the log bloom to see if it makes any of our log filter subscriptions. If there is a match, the logs for a block range, address, and topics are retrieved. The retrieved logs are intersected with the filters and eventually the callbacks are called 1,3.
- As new event subscriptions (block subscriptions not included) are subscribed, generate an 32-byte id, adding the
EventFilter to the logFilters table using the id as the key. Each EventFilter contains the topics (event signatures) and the address of the MP contract. Unsubscribing removes the EventFilter from the table.
- Keep last polled block number in memory.
- Call
eth_getBlockNumber, updating the last polled block number if it has increased. Note that the block number could have increased by more than 1, so we would need to consider handling a block range.
- Using the current block number (or for each block in the range), call
eth_getBlockByNumber, calling any block subscription callbacks with the result.
- Check if any
topics from the filters property are included in the log bloom of the block (using BloomFilter from nim-eth2).
- If the topic(s) are included, it could be a false positive, so we then have to get logs for that block (or block range) using
eth_getLogs, providing the block range, contract address and all collated topics (array of event signatures) from the logFilters property. Providing all topics from all filters in the array will reduce the needed RPC calls.
- Intersect the returned logs and the
topics from the filters property. For each intersected log, call the callback (for the filter id) with the deserialised log value.
References:
1 When an application wants to find all the log entries from a given contract, or with specific indexed fields (or both), the node can quickly scan over the header of each block, checking the bloom filter to see if it may contain relevant logs. If it does, the node re-executes the transactions from that block, regenerating the logs, and returning the relevant ones to the application.
2 nim-eth bloom filter implementation
3 Pattern for reading bloom filters in Ethereum
From our discussion, we decided it would be beneficial to refactor the way the polling operation works, eliminating:
Inspired by the way ethers.js handles subscriptions, the polling loop can be reduced down to checking for a new block at each polling interval. Once a new block is encountered, block subscription handlers can be called, and we can check the log bloom to see if it makes any of our log filter subscriptions. If there is a match, the logs for a block range, address, and topics are retrieved. The retrieved logs are intersected with the filters and eventually the callbacks are called 1,3.
EventFilterto thelogFilterstable using theidas the key. EachEventFiltercontains the topics (event signatures) and the address of the MP contract. Unsubscribing removes theEventFilterfrom the table.eth_getBlockNumber, updating the last polled block number if it has increased. Note that the block number could have increased by more than 1, so we would need to consider handling a block range.eth_getBlockByNumber, calling any block subscription callbacks with the result.topicsfrom thefiltersproperty are included in the log bloom of the block (usingBloomFilterfromnim-eth2).eth_getLogs, providing the block range, contract address and all collatedtopics(array of event signatures) from thelogFiltersproperty. Providing all topics from allfiltersin the array will reduce the needed RPC calls.topicsfrom thefiltersproperty. For each intersected log, call the callback (for the filter id) with the deserialised log value.References:
1 When an application wants to find all the log entries from a given contract, or with specific indexed fields (or both), the node can quickly scan over the header of each block, checking the bloom filter to see if it may contain relevant logs. If it does, the node re-executes the transactions from that block, regenerating the logs, and returning the relevant ones to the application.
2
nim-ethbloom filter implementation3 Pattern for reading bloom filters in Ethereum