Newswire — Real-Time Financial Intelligence & Market Platform

Autonomous News Aggregator, Live Equity Ticker & Market Sentiment Tracking Engine
Next.js 16 • React 19 • LibSQL / Turso Edge Database • Yahoo Finance API • Automated RSS Workers

---

⚡ Overview

Newswire is an automated financial news platform and market intelligence dashboard. It continuously aggregates, deduplicates, and classifies high-impact market news from global publications, correlates articles with real-time equity tickers via Yahoo Finance, and stores indexed feeds inside a high-speed LibSQL / Turso edge SQLite database.

Designed for investors, analysts, and tech-forward readers, Newswire pairs clean, distraction-free typography with fast edge data retrieval.

---

🚀 Key Features

• 📰 Autonomous RSS Ingestion Engine: Background workers (scripts/run-ingest.ts, src/lib/ingest-rss.ts) that poll and parse syndication feeds from top financial outlets, normalizing schemas and extracting article metadata.
• 📊 Real-Time Market Tracking: Live indices, commodities, and sector performance dashboards powered by the yahoo-finance2 API (src/lib/stocks.ts, /markets).
• 🔮 Market Predictions & Sentiment Insights: Predictive sentiment trackers and macroeconomic indicator cards (src/lib/predictions.ts, /predictions).
• 🖼️ Image Scraping & Safe Media Caching: Automated OpenGraph and fallback image scraper (scripts/scrape-images.ts, SafeImage.tsx) to guarantee clean thumbnails without broken links.
• ☁️ Serverless Edge Storage: Hybrid database architecture supporting local better-sqlite3 during development and Turso / LibSQL for distributed global deployment.
• 🔍 Full-Text Story Search & Categorization: Category-based routing (/category/[slug]), keyword search (/search), and full story views (/story/[id]).

---

🏗️ Architecture & Technical Stack

                          ┌────────────────────────────────┐
                          │    Next.js 16 (App Router)     │
                          │   React 19 Server Components   │
                          └───────────────┬────────────────┘
                                          │
                  ┌───────────────────────┼───────────────────────┐
                  ▼                       ▼                       ▼
        ┌──────────────────┐    ┌──────────────────┐    ┌──────────────────┐
        │  RSS Workers &   │    │  Yahoo Finance   │    │  LibSQL / Turso  │
        │ Ingestion Pipeline│   │   Market API     │    │  Edge SQLite DB  │
        └──────────────────┘    └──────────────────┘    └──────────────────┘

• Framework: Next.js 16 with React 19 Server & Client Components
• Persistence: LibSQL Client (@libsql/client) & better-sqlite3
• Data Ingestion: rss-parser, Custom TSX ingestion crons
• Design System: Tailwind CSS v4, Lucide icons, Geist typography
• Type Safety: TypeScript 5 strict mode

---

📂 Project Structure

newswire/
├── src/
│   ├── app/                    # Next.js App Router endpoints & routes
│   │   ├── category/           # Dynamic category news pages
│   │   ├── markets/            # Financial market tickers & sector stats
│   │   ├── predictions/        # Market sentiment & predictive analysis
│   │   ├── search/             # Full-text news search
│   │   ├── story/              # Individual article view
│   │   ├── weather/            # Contextual weather module
│   │   ├── layout.tsx          # Root layout with responsive navbar
│   │   └── page.tsx            # Live headlines homepage
│   ├── components/             # Reusable UI components (ArticleCard, SafeImage, Tabs)
│   ├── config/                 # Feed configuration & sector definitions
│   └── lib/                    # Core library services
│       ├── db.ts               # LibSQL/SQLite database connection pool
│       ├── ingest-rss.ts       # RSS feed parser & database writer
│       ├── predictions.ts      # Sentiment heuristics & forecasting models
│       ├── stocks.ts           # Yahoo Finance live ticker integration
│       └── types.ts            # Domain TypeScript interfaces
├── scripts/
│   ├── run-ingest.ts           # Standalone CLI / cron job to trigger RSS sync
│   ├── check-sectors.ts        # Sector taxonomy validator
│   ├── scrape-images.ts        # OG image scraping & CDN caching worker
│   └── migrate-turso.ts        # Database migration script for Turso
└── data/                       # Local SQLite development database

---

🛠️ Quickstart & Setup

Prerequisites

• Node.js v20+ or v22 LTS
• npm or pnpm

1. Installation

git clone https://github.com/LifeOfDevD/newswire.git
cd newswire
npm install

2. Environment Variables

Create a .env.local file in the root directory:

# Optional: Turso Cloud DB (defaults to local SQLite if omitted)
TURSO_DATABASE_URL=libsql://your-db.turso.io
TURSO_AUTH_TOKEN=your-turso-token

3. Run Ingestion & Start Server

# Ingest latest news articles
npm run dev

# Or run the ingestion worker directly
npx tsx scripts/run-ingest.ts

Open http://localhost:3000 to view the live dashboard.

---

📄 License

MIT License. Open source and free for community use.