Skip to content

timestamp in orderfilled.parquet does not match Polygon block timestamp #3

Description

@Yjejuy

Hi, thanks for open-sourcing this dataset.

I found what looks like a timestamp issue in the orderfilled.parquet files.

I compared the exported timestamp against Polygon block timestamps using both:

  • Polygon RPC eth_getBlockByNumber
  • Etherscan v2 eth_getBlockByNumber

For sampled fills, the two chain sources agree with each other, but the exported parquet timestamp differs.

From a larger 100-sample check on my side:

  • RPC timestamp == Etherscan timestamp: 100/100
  • Exported parquet timestamp == RPC/Etherscan: 0/100

So the exported timestamp does not appear to be the chain block.timestamp.

Here is a minimal reproduction script using a few reference fills:

import os
import time
import requests

RPC_URL = "https://polygon-bor-rpc.publicnode.com"
API_KEY = os.environ["POLYGONSCAN_API_KEY"]

SAMPLES = [
    {
        "fill_id": "0x876afb3e013514ba3091eafeff4c64c8bdaf9bc66d4e528d59c9a84b3a26b6a3_348",
        "tx_hash": "0x876afb3e013514ba3091eafeff4c64c8bdaf9bc66d4e528d59c9a84b3a26b6a3",
        "exported_timestamp": 1674519220,
    },
    {
        "fill_id": "0xf59d3f04414c2d291b4cb632f33645988fef636381ca56af802a2ad170b458b3_186",
        "tx_hash": "0xf59d3f04414c2d291b4cb632f33645988fef636381ca56af802a2ad170b458b3",
        "exported_timestamp": 1670544960,
    },
    {
        "fill_id": "0xe9a45356419702dc6a3298d4d4aa6338fe9ba7d50e8d77078f5f0c38aee98635_156",
        "tx_hash": "0xe9a45356419702dc6a3298d4d4aa6338fe9ba7d50e8d77078f5f0c38aee98635",
        "exported_timestamp": 1697846476,
    },
]

def rpc_call(method, params):
    payload = {"jsonrpc": "2.0", "id": 1, "method": method, "params": params}
    r = requests.post(RPC_URL, json=payload, timeout=30)
    r.raise_for_status()
    data = r.json()
    if "error" in data:
        raise RuntimeError(data["error"])
    return data["result"]

def get_block_timestamp_from_rpc(tx_hash):
    receipt = rpc_call("eth_getTransactionReceipt", [tx_hash])
    block_number = int(receipt["blockNumber"], 16)
    block = rpc_call("eth_getBlockByNumber", [hex(block_number), False])
    return block_number, int(block["timestamp"], 16)

def get_block_timestamp_from_etherscan(block_number):
    url = "https://api.etherscan.io/v2/api"
    params = {
        "chainid": "137",
        "module": "proxy",
        "action": "eth_getBlockByNumber",
        "tag": hex(block_number),
        "boolean": "false",
        "apikey": API_KEY,
    }
    r = requests.get(url, params=params, timeout=30)
    r.raise_for_status()
    data = r.json()
    if "result" not in data or data["result"] is None:
        raise RuntimeError(data)
    return int(data["result"]["timestamp"], 16)

for sample in SAMPLES:
    block_number, rpc_ts = get_block_timestamp_from_rpc(sample["tx_hash"])
    etherscan_ts = get_block_timestamp_from_etherscan(block_number)
    time.sleep(0.4)  # avoid free-tier rate limit

    print(
        sample["fill_id"],
        {
            "exported_timestamp": sample["exported_timestamp"],
            "rpc_block_ts": rpc_ts,
            "etherscan_block_ts": etherscan_ts,
            "exported_eq_rpc": sample["exported_timestamp"] == rpc_ts,
            "rpc_eq_etherscan": rpc_ts == etherscan_ts,
        },
    )

Example output from this script:

0x876afb3e013514ba3091eafeff4c64c8bdaf9bc66d4e528d59c9a84b3a26b6a3_348 {
  'exported_timestamp': 1674519220,
  'rpc_block_ts': 1674519246,
  'etherscan_block_ts': 1674519246,
  'exported_eq_rpc': False,
  'rpc_eq_etherscan': True
}

0xf59d3f04414c2d291b4cb632f33645988fef636381ca56af802a2ad170b458b3_186 {
  'exported_timestamp': 1670544960,
  'rpc_block_ts': 1670544922,
  'etherscan_block_ts': 1670544922,
  'exported_eq_rpc': False,
  'rpc_eq_etherscan': True
}

0xe9a45356419702dc6a3298d4d4aa6338fe9ba7d50e8d77078f5f0c38aee98635_156 {
  'exported_timestamp': 1697846476,
  'rpc_block_ts': 1697846488,
  'etherscan_block_ts': 1697846488,
  'exported_eq_rpc': False,
  'rpc_eq_etherscan': True
}

Could you confirm what the exported timestamp field is intended to represent?

  • If it is supposed to be Polygon block.timestamp, it seems incorrect.
  • If it represents another timestamp semantic, it would be helpful to document that explicitly.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions