Skip to content

DEVELOPER.md

Latest commit

 

History

History
283 lines (224 loc) · 6.14 KB

DEVELOPER.md

File metadata and controls

283 lines (224 loc) · 6.14 KB
Raw

SignalHub APIs - Developer Guide

Quick Start

Prerequisites

  • Python 3.12+
  • Node.js 20+
  • npm or yarn

Start Everything

# Windows
start.bat

# Manual start
# Terminal 1 - Backend
cd apps/api
venv\Scripts\activate
set PYTHONPATH=C:\dev\signalhub-apis\apps\api;C:\dev\signalhub-apis
uvicorn app.main:app --host 0.0.0.0 --port 8000 --reload

# Terminal 2 - Frontend
cd apps/web
npm run dev

Access

Project Structure

signalhub-apis/
├── apps/
│   ├── api/                    # FastAPI backend
│   │   ├── app/
│   │   │   ├── db/            # Models, engine, base
│   │   │   ├── routers/       # API endpoints
│   │   │   ├── schemas/       # Pydantic schemas
│   │   │   ├── services/      # Business logic
│   │   │   ├── config.py      # Configuration
│   │   │   └── main.py        # FastAPI app
│   │   ├── alembic/           # Database migrations
│   │   ├── venv/              # Python virtual environment
│   │   ├── signalhub.db       # SQLite database
│   │   └── requirements.txt   # Python dependencies
│   │
│   └── web/                   # Next.js frontend
│       ├── src/
│       │   ├── app/           # App router pages
│       │   ├── components/    # React components
│       │   └── lib/           # Utilities
│       └── package.json       # Node dependencies
│
├── packages/
│   └── ingestion/             # Data pipeline
│       ├── connectors/        # API connectors
│       ├── jobs/              # Job runner
│       ├── quality/           # Quality checks
│       └── transforms/        # Data transforms
│
├── docs/                      # Documentation
├── scripts/                   # Utility scripts
├── PLAN.md                    # Development plan
├── SUMMARY.md                 # Project summary
└── VALIDATION.md              # Validation report

Common Tasks

Trigger Connectors Manually

cd apps/api
venv\Scripts\activate
python trigger_connectors.py

Validate API Endpoints

cd apps/api
venv\Scripts\activate
python validate_api.py

Inspect Database

cd apps/api
venv\Scripts\activate
python inspect_db.py
python inspect_data.py

Run Migrations

cd apps/api
venv\Scripts\activate
set PYTHONPATH=C:\dev\signalhub-apis\apps\api;C:\dev\signalhub-apis
alembic upgrade head

Create New Migration

cd apps/api
venv\Scripts\activate
set PYTHONPATH=C:\dev\signalhub-apis\apps\api;C:\dev\signalhub-apis
alembic revision --autogenerate -m "description"

Database Schema

Tables

  1. sources - Data source definitions
  2. runs - Execution history
  3. raw_payloads - Raw API responses
  4. normalized_signals - Normalized data points
  5. freshness_status - Data freshness tracking
  6. quality_checks - Quality validation results
  7. event_logs - System events

Relationships

  • Source → Runs (1:N)
  • Source → FreshnessStatus (1:1)
  • Run → RawPayloads (1:N)
  • Run → NormalizedSignals (1:N)
  • Run → QualityChecks (1:N)

API Endpoints

Health

  • GET /health - System health check

Sources

  • GET /api/v1/sources - List all sources
  • GET /api/v1/sources/{slug} - Get source detail

Runs

  • GET /api/v1/runs - List runs (paginated)
  • GET /api/v1/runs/{id} - Get run detail

Data

  • GET /api/v1/signals - List signals (paginated)
  • GET /api/v1/freshness - Get freshness status
  • GET /api/v1/quality - List quality checks

Metrics

  • GET /api/v1/metrics/summary - System metrics

Actions

  • POST /api/v1/runs/trigger/{slug} - Trigger manual run

Configuration

Environment Variables (.env)

# Database
DATABASE_URL=sqlite+aiosqlite:///C:/dev/signalhub-apis/apps/api/signalhub.db
DATABASE_URL_SYNC=sqlite:///C:/dev/signalhub-apis/apps/api/signalhub.db

# API
API_HOST=0.0.0.0
API_PORT=8000
API_DEBUG=false

# Frontend
NEXT_PUBLIC_API_URL=http://localhost:8000

# Connectors
COINGECKO_API_KEY=

# Open-Meteo
OPEN_METEO_LATITUDE=52.52
OPEN_METEO_LONGITUDE=13.41

Connectors

Open-Meteo (Weather)

Frankfurter (Currency)

CoinGecko (Crypto)

Troubleshooting

Backend won't start

  1. Check if venv is activated
  2. Verify PYTHONPATH is set correctly
  3. Check if port 8000 is available
  4. Review logs for errors

Frontend won't start

  1. Run npm install in apps/web
  2. Check if port 3000/3001 is available
  3. Verify NEXT_PUBLIC_API_URL in .env

Connectors failing

  1. Check internet connection
  2. Verify API endpoints are accessible
  3. Check rate limits
  4. Review error logs in database

Database issues

  1. Check if signalhub.db exists
  2. Verify file permissions
  3. Run migrations if needed
  4. Check SQLite version

Testing

Backend Tests

cd apps/api
venv\Scripts\activate
pytest

Frontend Tests

cd apps/web
npm test

Deployment

Production Checklist

  • Switch to PostgreSQL
  • Set API_DEBUG=false
  • Configure CORS for production domain
  • Add rate limiting
  • Set up monitoring (Sentry, etc.)
  • Configure SSL/TLS
  • Set up CI/CD pipeline
  • Add authentication
  • Configure backup strategy
  • Set up logging aggregation

Support

For issues or questions:

  1. Check VALIDATION.md for known issues
  2. Review PLAN.md for development status
  3. Check SUMMARY.md for project overview
  4. Create issue on GitHub

License

MIT