feat: Implement backfilling of historical data

[gracious01-tech]

Description

Implemented historical ledger backfilling support for the indexer, enabling ingestion and processing of past ledger data starting from a specified sequence number.

This feature allows the system to recover missed data, bootstrap new environments, and rebuild analytics datasets from historical Stellar ledger activity.

---

Related Issue

Closes #135

---

Type of Change

• New feature
• Backend enhancement
• Indexer infrastructure improvement
• Bug fix
• Refactor
• Documentation

---

Changes Included

Historical Backfill Support

• Added ability to start indexing from a specific ledger sequence
• Implemented sequential historical ledger ingestion
• Added configurable backfill start parameters
• Added progress tracking for backfill operations
• Improved handling of large historical ranges

Indexer Improvements

• Enhanced indexer service orchestration
• Added safeguards for duplicate processing
• Improved retry/error handling during backfill
• Added logging for historical sync progress

Updated File

• packages/indexer/src/services/indexer-service.ts

---

Problem Solved

Previously:

• Indexer only supported forward/live ingestion
• Historical ledger recovery required manual intervention
• Missed ledger ranges could not be rebuilt automatically
• New deployments lacked historical synchronization support

This PR enables reliable historical replay and synchronization.

---

Security Impact

No security impact.

Improves operational reliability and data completeness.

---

Testing

• Verified backfill starts from specified sequence number
• Verified sequential ledger processing correctness
• Tested large-range historical ingestion
• Verified duplicate processing protection
• Tested recovery from interrupted backfill runs
• Existing live indexing behavior remains functional

---

Manual Testing Steps

• Started indexer with custom historical sequence
• Verified historical ledgers processed correctly
• Compared indexed data against expected ledger range
• Interrupted backfill process and resumed successfully
• Verified live indexing continues after backfill completion

---

Acceptance Criteria Coverage

• Supports historical ledger ingestion
• Accepts configurable starting sequence number
• Handles sequential backfill processing correctly
• Preserves existing live indexing behavior

---

Breaking Changes

No breaking changes.

Adds optional historical synchronization capability.

---

Checklist

• Code builds successfully
• Tests pass
• Follows project conventions
• No sensitive data exposed
• Existing indexer functionality preserved

---

Additional Context

Historical backfilling is essential for analytics systems to maintain complete datasets, recover from outages, and support environment bootstrapping without data gaps.

This implementation is designed to scale for large ledger ranges while remaining compatible with current indexing workflows.

---

Reviewer Notes

Please focus on:

• Backfill sequencing correctness
• Duplicate ingestion protection
• Recovery behavior after interruptions
• Performance impact during large historical syncs
• Transition from backfill mode to live indexing

[drips-wave]

@gracious01-tech Great news! 🎉 Based on an automated assessment of this PR, the linked Wave issue(s) no longer count against your application limits.

You can now already apply to more issues while waiting for a review of this PR. Keep up the great work! 🚀

Learn more about application limits