Add watchdog timeouts so metadata workers cannot hang forever

[DenisCarriere]

Context

Base and BSC metadata scrapers can apparently stop producing successful scrapes for days until the process is restarted. The most likely failure mode in the current code is an iteration that never finishes, so the auto-restart loop never gets a chance to start the next pass.

Why this looks plausible

• services/metadata/run.ts queues the entire batch, then waits on queue.onIdle() before the next iteration.
• Each contract can perform multiple async RPC calls in processMetadata().
• If any queued job never settles, the queue never becomes idle and the service can look "alive" while making no forward progress.
• This would be much easier to hit on high-volume chains because they sustain more concurrent in-flight work for longer.

Proposed fix

• Add a per-contract watchdog timeout (for example METADATA_TASK_TIMEOUT_MS) around each queued metadata job.
• Use abortable I/O where possible so timed-out tasks do not keep dangling network work around.
• Treat task timeouts as infrastructure failures, not permanent token failures.
• Optionally exit the process after N stuck-task timeouts in a row so the outer supervisor can recover automatically.

Acceptance criteria

• A single hung contract cannot block an entire metadata iteration forever.
• Timed-out jobs are logged at warn/error level with contract + network context.
• The service either continues processing remaining work or exits cleanly for auto-restart.

Relevant code

• services/metadata/run.ts
• services/metadata/index.ts
• lib/rpc.ts