feat(traceroute): switch views and APIs to materialized route and hop readers

[joroses]

TLDR

Uses the new traceroute tables from the previous PR: significant improvement in /traceroute-hops loading times and actually shows all the traceroutes in the selected range/filter.

Summary (PR 2: Materialized Traceroute Readers)

Switches all traceroute pages, API endpoints, and visualization tools from on-the-fly protobuf decoding of raw packet_history to direct SQL queries against the materialized traceroute_routes and traceroute_hops tables introduced in PR 1.

PR Stack Context:
This is PR 2 of 4 in the traceroute performance & network analysis series:

1. PR 1 (feat(traceroute): pre-decode and store traceroute routes and hops on capture #122): Storage Layer & Backfill CLI (traceroute_routes & traceroute_hops) — Prerequisite / Merged first
2. PR 2 (This PR): Materialized Traceroute Readers — Switches all views & endpoints to SQL queries
3. PR 3: Discard Bogus Near-Zero GPS Coordinates (Null Island)
4. PR 4: UI Pagination Visibility & Map Date Filter Retention

---

Key Changes

1. Materialized Read Repository (traceroute_read_repository.py):

   • Implements high-performance queries for:
     • Main traceroute table (get_traceroute_packets): supports two-stage SQL grouping, pagination, and multi-field filtering.
     • Link analysis (get_traceroute_link): aggregates SNR stats, link counts, and matching traceroute packets between two nodes.
     • Network topology graph & maps (get_traceroute_hops_for_graph): extracts directional hops and signal levels directly in SQL.
     • Longest links & paths (get_traceroute_hops_for_longest_links): retrieves filtered hop collections for distance aggregation.
     • Route patterns (get_route_patterns_data): identifies recurring node sequences between endpoints.
     • Node traceroute statistics (get_node_traceroute_statistics): calculates source, destination, and intermediate routing participation.
     • Batch node location history (get_nodes_location_history): fetches recent position history for multiple nodes in single windowed queries.

2. Removed Packet Scan Limits:

   • Eliminates artificial caps (e.g., LIMIT 20000) across all traceroute readers. Full multi-day (24h, 7d) and multi-week windows now return complete, unskewed link topologies without silent truncation.

3. Strict Hop Continuity & Path Integrity:

   • In TracerouteService, indirect connections and longest paths now require contiguous hop sequences. Routes with missing or corrupted middle hops no longer collapse into false shortcut links.

4. Legacy Code Cleanup:

   • Deprecated and removed ~600 lines of obsolete raw-packet parsing loops and helper methods in TracerouteRepository (repositories.py).

5. Table UI Exact Count Display (modern-table.js):

   • Grouped packet views now display exact total counts from the SQL grouping queries instead of displaying estimated N+ count indicators.

---

Test Status

• Test Suite Results: 231 passed, 0 failed.
• Unit Test Fix Included: In tests/unit/test_traceroute_read_repository.py:220 (test_get_route_patterns_data), the assertion checks pattern_key[1] == (905,) matching the composite dictionary key pattern_key = (endpoints, route_nodes) returned by get_route_patterns_data().
• E2E Test Fix Included: In tests/e2e/test_traceroute_filters_e2e.py, test_traceroute_route_node_filter_e2e checks that a filtered node matches anywhere along the entire route path (endpoints or intermediate hops).

---

Commits

• 9f5f24c (Core traceroute pages and analyses reader migration):
  • Core traceroute pages and analyses now read from the saved route and hop tables instead of parsing thousands of raw packets on every request. Link analysis between nodes, related node lists, the hop picker, and the main traceroute table now query the database directly with fast filtering and pagination. Updates table UI to display exact total counts.
• a6b5542 (Full reader migration & legacy cleanup):
  • All traceroute features now read directly from the saved route and hop tables instead of re-decoding raw packets on every page request. The network graph, maps, longest links, route patterns, node statistics, and packet details now query pre-calculated hops and routes, eliminating slow packet loops and repeated database lookups. Also removes obsolete legacy reader code.
• 56de02a (Strict path continuity):
  • Traceroutes with a missing or invalid middle hop are no longer shown as shorter connected paths. Hop queries now keep the full route structure, and indirect connections and longest-path results are only built from hops that actually link up end to end, so distances, hop counts, and route previews stay correct.
• e5631de (Fix route_node e2e assertion):
  • Fixes route_node e2e assertion in test_traceroute_filters_e2e.py to check the entire route path (endpoints or intermediate hops).

---

Verification

• Automated Tests: All unit tests in test_traceroute_read_repository.py, test_traceroute_service.py, and test_traceroute_materialization.py pass.
• Code Quality: ruff check src/ tests/ passes cleanly with no warnings.