Avoid poisoning metadata_errors with transient RPC failures

[DenisCarriere]

Context

Avalanche and Arbitrum One show a much lower metadata scrape success rate than the other EVM chains. The code path suggests a likely cause: retryable RPC failures are recognized during request execution, but after retries are exhausted many of those same transient failures are still persisted into metadata_errors, which suppresses retries for up to a week.

Why this looks plausible

• lib/rpc.ts correctly treats many conditions as retryable: 429, 503, 504, timeouts, overloaded/busy responses, and several transient JSON-RPC error codes/messages.
• services/metadata/index.ts only treats a very small subset as infrastructure errors when deciding whether to skip metadata_errors insertion (unable to connect, URL typo, 502, 404).
• sql.schemas/schema.metadata_evm.sql gives metadata_errors a 1-week TTL.
• The metadata candidate SQL excludes contracts already present in metadata_errors.
• On RPC-noisy chains, a burst of rate limiting or backend instability can therefore mark many valid contracts as failed for a full week, crushing the observed success rate.

Proposed fix

• Unify retryable/infrastructure error classification so the same error families are handled consistently in both lib/rpc.ts and services/metadata/index.ts.
• Do not insert retryable/transient RPC failures into metadata_errors.
• Optionally store transient failures in a separate table/metric for observability without poisoning the permanent candidate set.
• Add tests covering 429/503/timeout/overloaded JSON-RPC failures.

Acceptance criteria

• Exhausted-but-retryable RPC failures do not suppress reprocessing for one week.
• Only deterministic application failures (for example truly missing decimals() or self-destructed contracts) go into metadata_errors.
• Tests verify consistent behavior between retry logic and metadata error filtering.

Relevant code

• lib/rpc.ts
• services/metadata/index.ts
• sql.schemas/schema.metadata_evm.sql
• services/metadata/get_contracts_by_transfers.sql
• services/metadata/get_contracts_by_swaps.sql
• services/metadata/get_contracts_by_balances.sql