Skip to content

Implement first database/storage layer #38

Description

@BytecodeBrewer

Goal

Implement the first DuckDB-based storage layer for ARGUS based on the storage research.

Why

ARGUS needs persistent local storage to move beyond temporary API requests and in-memory analytics. A first storage layer makes it possible to save normalized historical market data, read it back for analytics and prepare the project for later storage-first workflows.

This ticket focuses only on the database/storage foundation. Workflow orchestration, API fallback and service refactoring are handled in separate sub-issues.

Scope

  • Add DuckDB as the first storage dependency

  • Create the first minimal DuckDB schema based on the storage research:

    • data_sources
    • instruments
    • price_bars
  • Store normalized internal model data, not raw API response structures

  • Add a small storage/repository module for:

    • initializing the database
    • inserting/upserting data sources
    • inserting/upserting instruments
    • saving price bars
    • reading price bars by instrument, source and date range
  • Keep storage logic separate from API clients, UI code and metric functions

  • Add tests for schema creation, saving and reading historical data

  • Document basic setup and usage

Out of scope

  • Storage-first/read-through workflow
  • API fallback when data is missing
  • Refactoring TimeSeriesService into MarketDataService
  • Refactoring existing services
  • GUI integration
  • Analytics/reporting integration
  • Cache strategy

Acceptance criteria

  • DuckDB is added as the selected first storage technology
  • A minimal DuckDB schema exists for data_sources, instruments and price_bars
  • The schema follows the storage research in docs/research-databases-and-storage.md
  • Historical FX or market data can be saved using the internal data model
  • Stored price bars can be read back by instrument, source and date range
  • FX rates are stored as close values in price_bars
  • Storage code does not depend directly on raw API response structures
  • Storage logic is separated from API clients, UI code and metric functions
  • Basic tests cover database initialization, saving data and reading data
  • README or docs explain how to use the storage layer

Note

Priority: Should

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

codingNew feature or request

Projects

No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions