Hi, thanks for open-sourcing this dataset.
I found what looks like a timestamp issue in the orderfilled.parquet files.
I compared the exported timestamp against Polygon block timestamps using both:
- Polygon RPC
eth_getBlockByNumber
- Etherscan v2
eth_getBlockByNumber
For sampled fills, the two chain sources agree with each other, but the exported parquet timestamp differs.
From a larger 100-sample check on my side:
- RPC timestamp == Etherscan timestamp:
100/100
- Exported parquet timestamp == RPC/Etherscan:
0/100
So the exported timestamp does not appear to be the chain block.timestamp.
Here is a minimal reproduction script using a few reference fills:
import os
import time
import requests
RPC_URL = "https://polygon-bor-rpc.publicnode.com"
API_KEY = os.environ["POLYGONSCAN_API_KEY"]
SAMPLES = [
{
"fill_id": "0x876afb3e013514ba3091eafeff4c64c8bdaf9bc66d4e528d59c9a84b3a26b6a3_348",
"tx_hash": "0x876afb3e013514ba3091eafeff4c64c8bdaf9bc66d4e528d59c9a84b3a26b6a3",
"exported_timestamp": 1674519220,
},
{
"fill_id": "0xf59d3f04414c2d291b4cb632f33645988fef636381ca56af802a2ad170b458b3_186",
"tx_hash": "0xf59d3f04414c2d291b4cb632f33645988fef636381ca56af802a2ad170b458b3",
"exported_timestamp": 1670544960,
},
{
"fill_id": "0xe9a45356419702dc6a3298d4d4aa6338fe9ba7d50e8d77078f5f0c38aee98635_156",
"tx_hash": "0xe9a45356419702dc6a3298d4d4aa6338fe9ba7d50e8d77078f5f0c38aee98635",
"exported_timestamp": 1697846476,
},
]
def rpc_call(method, params):
payload = {"jsonrpc": "2.0", "id": 1, "method": method, "params": params}
r = requests.post(RPC_URL, json=payload, timeout=30)
r.raise_for_status()
data = r.json()
if "error" in data:
raise RuntimeError(data["error"])
return data["result"]
def get_block_timestamp_from_rpc(tx_hash):
receipt = rpc_call("eth_getTransactionReceipt", [tx_hash])
block_number = int(receipt["blockNumber"], 16)
block = rpc_call("eth_getBlockByNumber", [hex(block_number), False])
return block_number, int(block["timestamp"], 16)
def get_block_timestamp_from_etherscan(block_number):
url = "https://api.etherscan.io/v2/api"
params = {
"chainid": "137",
"module": "proxy",
"action": "eth_getBlockByNumber",
"tag": hex(block_number),
"boolean": "false",
"apikey": API_KEY,
}
r = requests.get(url, params=params, timeout=30)
r.raise_for_status()
data = r.json()
if "result" not in data or data["result"] is None:
raise RuntimeError(data)
return int(data["result"]["timestamp"], 16)
for sample in SAMPLES:
block_number, rpc_ts = get_block_timestamp_from_rpc(sample["tx_hash"])
etherscan_ts = get_block_timestamp_from_etherscan(block_number)
time.sleep(0.4) # avoid free-tier rate limit
print(
sample["fill_id"],
{
"exported_timestamp": sample["exported_timestamp"],
"rpc_block_ts": rpc_ts,
"etherscan_block_ts": etherscan_ts,
"exported_eq_rpc": sample["exported_timestamp"] == rpc_ts,
"rpc_eq_etherscan": rpc_ts == etherscan_ts,
},
)
Example output from this script:
0x876afb3e013514ba3091eafeff4c64c8bdaf9bc66d4e528d59c9a84b3a26b6a3_348 {
'exported_timestamp': 1674519220,
'rpc_block_ts': 1674519246,
'etherscan_block_ts': 1674519246,
'exported_eq_rpc': False,
'rpc_eq_etherscan': True
}
0xf59d3f04414c2d291b4cb632f33645988fef636381ca56af802a2ad170b458b3_186 {
'exported_timestamp': 1670544960,
'rpc_block_ts': 1670544922,
'etherscan_block_ts': 1670544922,
'exported_eq_rpc': False,
'rpc_eq_etherscan': True
}
0xe9a45356419702dc6a3298d4d4aa6338fe9ba7d50e8d77078f5f0c38aee98635_156 {
'exported_timestamp': 1697846476,
'rpc_block_ts': 1697846488,
'etherscan_block_ts': 1697846488,
'exported_eq_rpc': False,
'rpc_eq_etherscan': True
}
Could you confirm what the exported timestamp field is intended to represent?
- If it is supposed to be Polygon
block.timestamp, it seems incorrect.
- If it represents another timestamp semantic, it would be helpful to document that explicitly.
Hi, thanks for open-sourcing this dataset.
I found what looks like a timestamp issue in the orderfilled.parquet files.
I compared the exported
timestampagainst Polygon block timestamps using both:eth_getBlockByNumbereth_getBlockByNumberFor sampled fills, the two chain sources agree with each other, but the exported parquet
timestampdiffers.From a larger 100-sample check on my side:
100/1000/100So the exported
timestampdoes not appear to be the chainblock.timestamp.Here is a minimal reproduction script using a few reference fills:
Example output from this script:
Could you confirm what the exported
timestampfield is intended to represent?block.timestamp, it seems incorrect.