Skip to content

Latest commit

Β 

History

86 Commits

Folders and files

NameName
Last commit message
Last commit date
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 

Repository files navigation

Mutual Aid NYC β€” Service Directory

A zero-cost community resource directory built on HSDS 3.0, powered by Cloudflare Workers and Vercel. Features natural-language search via LLM chatbot, an MCP server for AI agent access, and Google Geocoding for accurate map pins.

Built for Mutual Aid NYC and Open Referral.

Architecture

β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”   cron every 15min   β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚   Airtable  β”‚ ───────────────────▢ β”‚  Cloudflare Worker (hsds-api)   β”‚
β”‚   (source)  β”‚   incremental sync   β”‚  β”œβ”€ Hono router (HSDS 3.0 API)   β”‚
β”‚   (Airtable)β”‚                      β”‚  β”œβ”€ D1 database (SQLite edge)  β”‚
β”‚             β”‚                      β”‚  β”œβ”€ Workers AI (Llama 3.3 70B) β”‚
β”‚             β”‚                      β”‚  β”œβ”€ MCP server (Durable Object)β”‚
β”‚             β”‚                      β”‚  └─ Search token index         β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜                      β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
                                                 β”‚
                                     β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
                                     β”‚  Vercel (services-wegov-nyc)   β”‚
                                     β”‚  β”œβ”€ Next.js (App Router)       β”‚
                                     β”‚  β”œβ”€ MapLibre GL interactive   β”‚
                                     β”‚  └─ LLM chat widget           β”‚
                                     β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜

Total hosting cost: $0/month (Cloudflare Free + Vercel Hobby)

Live URLs

Component URL
Frontend (Production) https://services.wegov.nyc
API (Production) https://services-api.wegov.nyc
API Health https://services-api.wegov.nyc/health
MCP Server https://services-api.wegov.nyc/mcp

Key Features

Modern Serverless Stack (v2)

Feature Legacy (Hetzner) v2 (Cloudflare + Vercel)
Backend FastAPI (Python) Hono / Cloudflare Worker (TypeScript)
Database SQLite file + FTS5 Cloudflare D1 (edge SQLite)
Search FTS5 full-text Token index with stemming + ranking
Geocoding Nominatim (spotty) Google Geocoding API (98% coverage)
AI Chat None Workers AI Llama 3.3 70B with RAG
MCP None 5-tool MCP server for AI agents
Hosting ~$5-10/mo VPS $0/month
Deploy SSH + Docker wrangler deploy + vercel build && deploy

Search

Token-based search with stemming replaces raw LIKE queries:

  • Stemming: "food pantries" matches "food pantry" (37 results vs 1)
  • Relevance ranking: Name (3x) > Description (2x) > Org name (1x)
  • Indexed: Prefix matching on the search_tokens table
  • Fallback: Degrades to LIKE when token index is empty

Sync

Incremental sync compares Airtable's modifiedTime against D1's updated_at:

  • Steady-state: 0-10 writes/cycle (down from ~2,134)
  • Token index + icon cache rebuilt after each sync
  • Category icons cached as base64 in D1 (avoids expiring Airtable signed URLs)

Quick Start

Prerequisites

  • Node.js 18+
  • Wrangler CLI: npm install -g wrangler
  • Airtable Personal Access Token (read-only scope)

1. Clone and setup

git clone https://github.com/sarapis/hsd.git
cd hsd

2. Worker (API backend)

cd worker
npm install

# Authenticate with Cloudflare
wrangler login

# Create D1 database
wrangler d1 create hsds-directory

# Update wrangler.toml with your database_id, then execute local schema:
wrangler d1 execute hsds-directory --local --file=src/db/schema.sql

# Set secrets
wrangler secret put AIRTABLE_API_KEY
wrangler secret put SYNC_SECRET
wrangler secret put GOOGLE_GEOCODING_API_KEY  # optional, for geocoding

# Local dev
npm run dev

# Deploy
wrangler deploy

3. Frontend

cd hsdirectory-v2
npm install

# Configure API URL for local development
echo "NEXT_PUBLIC_API_URL=http://localhost:8787" > .env.local

# Local dev
npm run dev

# Deploy to Vercel (Production)
# 1. Pull settings and variables
npx vercel pull --yes --environment production --scope YOUR_SCOPE
# 2. Build Next.js locally with target env
NEXT_PUBLIC_API_URL=https://services-api.wegov.nyc npx vercel build --prod --yes --scope YOUR_SCOPE
# 3. Deploy prebuilt bundle
npx vercel deploy --prebuilt --prod --scope YOUR_SCOPE

4. Initial data seed

# Trigger Airtable β†’ D1 sync
curl -X POST https://services-api.wegov.nyc/sync/trigger \
  -H "Authorization: Bearer YOUR_SYNC_SECRET"

# Cache category icons (run until remaining=0)
curl -X POST https://services-api.wegov.nyc/sync/icons \
  -H "Authorization: Bearer YOUR_SYNC_SECRET"

# Geocode addresses (batches of 10, run until remaining=0)
curl -X POST https://services-api.wegov.nyc/sync/geocode \
  -H "Authorization: Bearer YOUR_SYNC_SECRET"

API Endpoints

Public

Method Endpoint Description
GET / API metadata (HSDS 3.0)
GET /health Health check with service count
GET /services Paginated services (?search=, ?page=, ?per_page=)
GET /services/:id Full service detail with nested relations (UUID format support)
GET /organizations Paginated organizations
GET /organizations/:id Organization detail
GET /map/services Services with coordinates + categories for map
GET /icons/:name Cached category icons (stable URLs)
POST /api/chat LLM chat with RAG (SSE streaming)
ALL /mcp MCP server (Streamable HTTP)

Admin (requires Authorization: Bearer SYNC_SECRET)

Method Endpoint Description
POST /sync/trigger Full Airtable sync
POST /sync/table/:name Sync single table
POST /sync/icons Cache category icons (batch of 5)
POST /sync/geocode Geocode addresses (batch of 10)
GET /sync/status Sync metadata + stats

Project Structure

hsd/
β”œβ”€β”€ worker/                        # Cloudflare Worker (API backend)
β”‚   β”œβ”€β”€ src/
β”‚   β”‚   β”œβ”€β”€ index.ts               # Hono app + routes + cron handler
β”‚   β”‚   β”œβ”€β”€ env.ts                 # Environment bindings type
β”‚   β”‚   β”œβ”€β”€ mapper.ts              # Airtable β†’ HSDS field mapping
β”‚   β”‚   β”œβ”€β”€ db/
β”‚   β”‚   β”‚   β”œβ”€β”€ schema.sql         # D1 schema (17 tables + indexes)
β”‚   β”‚   β”‚   └── queries.ts         # Typed D1 queries + token search
β”‚   β”‚   β”œβ”€β”€ sync/
β”‚   β”‚   β”‚   β”œβ”€β”€ sync.ts            # Incremental sync + token/icon cache
β”‚   β”‚   β”‚   └── airtable-client.ts # Paginated Airtable fetcher
β”‚   β”‚   β”œβ”€β”€ routes/
β”‚   β”‚   β”‚   β”œβ”€β”€ services.ts        # /services endpoints
β”‚   β”‚   β”‚   β”œβ”€β”€ organizations.ts   # /organizations endpoints
β”‚   β”‚   β”‚   β”œβ”€β”€ map.ts             # /map/services (geocoded)
β”‚   β”‚   β”‚   └── ...
β”‚   β”‚   β”œβ”€β”€ chat/
β”‚   β”‚   β”‚   └── handler.ts         # Workers AI chat with RAG
β”‚   β”‚   └── mcp/
β”‚   β”‚       └── server.ts          # MCP Durable Object (5 tools)
β”‚   β”œβ”€β”€ wrangler.toml              # Worker config + D1 binding
β”‚   └── package.json
β”‚
β”œβ”€β”€ hsdirectory-v2/                # Next.js frontend (Vercel)
β”‚   β”œβ”€β”€ src/
β”‚   β”‚   β”œβ”€β”€ app/                   # App Router pages
β”‚   β”‚   β”‚   β”œβ”€β”€ page.tsx           # Home (hero + categories)
β”‚   β”‚   β”‚   β”œβ”€β”€ services/          # Service list + map + detail
β”‚   β”‚   β”‚   └── organizations/     # Org list + detail
β”‚   β”‚   β”œβ”€β”€ components/
β”‚   β”‚   β”‚   β”œβ”€β”€ ui/                # Header, SearchBar, Footer, Chat
β”‚   β”‚   β”‚   └── map/               # MapLibre GL map component
β”‚   β”‚   └── lib/
β”‚   β”‚       └── api.ts             # Typed API client
β”‚   └── package.json
β”‚
└── README.md                      # ← you are here

Configuration

Worker Secrets (via wrangler secret put)

Secret Description
AIRTABLE_API_KEY Airtable PAT with data.records:read scope
SYNC_SECRET Bearer token for admin endpoints
GOOGLE_GEOCODING_API_KEY Google Maps Geocoding API key (server-side)

Worker Environment (in wrangler.toml)

Variable Default Description
AIRTABLE_BASE_ID β€” Airtable base ID (appXXX)
PUBLISHED_STATUS_VALUE Published Filter services by status
SYNC_INTERVAL_MINUTES 15 Cron sync interval

Frontend Environment (.env.production.local / Vercel Env)

Variable Description
NEXT_PUBLIC_API_URL Worker API URL (https://services-api.wegov.nyc)

MCP Server

The Worker exposes an MCP server at /mcp with 5 tools for AI agents:

Tool Description
search_services Keyword search with pagination
get_service Full service detail by ID
list_organizations Browse/search organizations
get_organization Organization detail by ID
get_directory_stats Summary stats + category list

Connect from any MCP client (like Cursor, Claude Desktop, or Devin):

{
  "mcpServers": {
    "mutualaid-nyc": {
      "url": "https://services-api.wegov.nyc/mcp"
    }
  }
}

Free Tier Limits

Resource Limit Current Usage
Worker requests 100K/day Low
D1 reads 5M/day ~3K/day
D1 writes 100K/day ~5K/day (incremental)
D1 storage 5 GB ~2 MB
Workers AI 10K neurons/day ~100/chat
Vercel bandwidth 100 GB/mo Low

Standards Compliance

License

MIT

About

A free-to-host (Cloudflare/Vercel) app that takes an HSDS3.0 Airtable Template and generates a front end resource directory and ORUK compliant API feed.

Resources

Contributing

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages