Add support to `EPHEMERAL` as row

[DenisCarriere]

Add support to EPHEMERAL field

This allows us to drop block_hash without having to re-run the Wasm cache, this would be helpful to drop SQL fields that are no longer needed, but not change the underlying db_out inserts

Table

SHOW CREATE TABLE erc20_balances

Query id: a56e20fd-aafb-4766-8c07-accce1560fc6

   ┌─statement───────────────────────────────────────────────────────────────────────────────┐
1. │ CREATE TABLE default.erc20_balances                                                    ↴│
   │↳(                                                                                      ↴│
   │↳    `block_num` UInt32,                                                                ↴│
   │↳    `block_hash` String EPHEMERAL,                                                     ↴│
   │↳    `timestamp` DateTime('UTC'),                                                       ↴│
   │↳    `minute` UInt32 MATERIALIZED toRelativeMinuteNum(timestamp),                       ↴│
   │↳    `contract` String COMMENT 'token contract address',                                ↴│
   │↳    `address` String COMMENT 'token holder address',                                   ↴│
   │↳    `balance` UInt256 COMMENT 'token balance',                                         ↴│
   │↳    `is_deleted` UInt8 MATERIALIZED balance = 0,                                       ↴│
   │↳    INDEX idx_balance balance TYPE minmax GRANULARITY 1,                               ↴│
   │↳    PROJECTION prj_contract_count                                                      ↴│
   │↳    (                                                                                  ↴│
   │↳        SELECT                                                                         ↴│
   │↳            contract,                                                                  ↴│
   │↳            min(balance),                                                              ↴│
   │↳            max(balance),                                                              ↴│
   │↳            count(),                                                                   ↴│
   │↳            max(block_num),                                                            ↴│
   │↳            min(block_num),                                                            ↴│
   │↳            max(timestamp),                                                            ↴│
   │↳            min(timestamp),                                                            ↴│
   │↳            max(minute),                                                               ↴│
   │↳            min(minute)                                                                ↴│
   │↳        GROUP BY contract                                                              ↴│
   │↳    ),                                                                                 ↴│
   │↳    PROJECTION prj_address_contract                                                    ↴│
   │↳    (                                                                                  ↴│
   │↳        SELECT *                                                                       ↴│
   │↳        ORDER BY                                                                       ↴│
   │↳            address,                                                                   ↴│
   │↳            contract                                                                   ↴│
   │↳    )                                                                                  ↴│
   │↳)                                                                                      ↴│
   │↳ENGINE = ReplacingMergeTree(block_num, is_deleted)                                     ↴│
   │↳ORDER BY (contract, address)                                                           ↴│
   │↳SETTINGS deduplicate_merge_projection_mode = 'rebuild', index_granularity = 8192       ↴│
   │↳COMMENT 'ERC-20 & Native balance changes per block for a given address / contract pair' │
   └─────────────────────────────────────────────────────────────────────────────────────────┘

1 row in set. Elapsed: 0.001 sec. 

Error

2026-04-13T10:35:05.804-0400 ERRO (sink-sql) handle BlockScopedData message at block #24276716 (e0c03c0146cd1bfa003d437a1bcb1745f1de45e18934103fb1484fe6ec2615cd): failed to flush at block #24276716 (e0c03c0146cd1bfa003d437a1bcb1745f1de45e18934103fb1484fe6ec2615cd): failed to flush after 3 attempts: dialect flush: failed to get values: cannot find column "block_hash" for table ""default"."erc20_balances"" (valid columns are "contract, address, balance, block_num, timestamp")

[copilot-swe-agent]

The agent encountered an error and was unable to start working on this issue: The service is temporarily unavailable. Please try again later. (Request id: E09E:2CCA62:B8FB9A2:ED3ECDF:69DCFFC2)

[copilot-swe-agent]

The agent encountered an error and was unable to start working on this issue: The service is temporarily unavailable. Please try again later. (Request id: E15C:1F949F:14697F69:1A1089A1:69DD02B6)

[DenisCarriere]

Proposed fix is up in #26.

What was happening:

• the ClickHouse loader was using the generic schema.Tables(...) path, which depends on SELECT * FROM table LIMIT 0 metadata
• for tables with EPHEMERAL columns, that metadata path was dropping those columns from columnsByName
• when the sink later flushed a row that still contained block_hash, the ClickHouse dialect rejected it because the discovered schema no longer knew that column existed

How this PR fixes it:

• ClickHouse LoadTables() now reads column metadata from system.columns
• EPHEMERAL columns are preserved as insertable columns
• generated columns that still should not be written by the sink (MATERIALIZED and ALIAS) are filtered out
• the ClickHouse type string is converted through the ClickHouse driver’s column parser so existing value conversion keeps working

Validation:

• added a regression test that proves EPHEMERAL columns are kept while generated columns are skipped
• ran go test ./db -run 'TestLoadClickhouseInsertableColumns|TestShouldSkipClickhouseColumn|TestUpsertUsesInsertForInsertOnlyDialects'

If you want, I can also add a follow-up integration test against a live ClickHouse table shape matching the issue example.