Skip to content

Scrape Solana program names from Solscan for program_id values #296

Description

@DenisCarriere

Context

We previously maintained a manual ClickHouse mapping of Solana program_id values to readable program names in substreams-svm:

https://github.com/pinax-network/substreams-svm/blob/4543009f99f01540c56850df43f07fcd8587b9aa/svm-dex/clickhouse/schema.0.functions.programs.sql

That list was painful to maintain by hand. We may want to use this scraper to extract the program name from Solscan for each program_id instead.

Current need

For swaps, the list of program IDs is relatively small and can be obtained with:

SELECT
    program_id,
    count()
FROM swaps
GROUP BY program_id
ORDER BY count() DESC

Example high-volume IDs from the current output include:

675kPX9MHTjS2zt1qfr1NYHuzeLXfQM9H24wFSUt1Mp8
JUP6LkbZbjS1jKKwapdHNy74zcZ3tLUZoi5QNyVTaV4
pAMMBay6oceH9fJKBRHGP5D4bD4sWpmSwMn52FMfXEA
CAMMCzo5YL8w4VFF8KVHrK22GGUsp5VTaW7grrKgrWqK
6EF8rrecthR5Dkzon8Nwu78hRvfCKubJ14M5uBEwF6P
LBUZKhRxPF3XUpBCjp4YzTKgLccjZhTSDM9YuVaPwxo
CPMMoo8L3F4NbTegBCKVNunggL7H1ZpdTHKxQB5qKP1C
cpamdpZCGKUy5JxQXB4dcpGPiikHawvSWAd6mEn1sGG
whirLbMiicVdio4qvUfM5KAg6Ct8VwpYZGff3uctyCc
JUP4Fb2cqiRUcaTHdrPC8h2gNsA2ETXiPDD33WcGuJB

Proposal

Add scraper support for looking up a Solana Program ID on Solscan and extracting the program/project name, then output a small mapping we can use downstream.

Potential target output formats:

  • JSON/CSV mapping of program_id -> name
  • ClickHouse-compatible CASE/function snippet similar to the old program_names function

Acceptance criteria

  • Given a list of Solana program_id values, the scraper fetches the corresponding Solscan page/API data.
  • It extracts a human-readable program name when available.
  • Unknown or missing names are handled explicitly without failing the whole run.
  • Results are deterministic and easy to diff/re-run.
  • Include a small test fixture or sample run covering known IDs such as Raydium, Jupiter, Pump.fun, Meteora, and Whirlpool.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions