Македонски народни песни со ноти · Macedonian Folk Songs with Sheet Music

Live at: https://pesnarka.mk

A fast, offline-capable, searchable archive of Macedonian folk songs with their sheet music (ноти). Type in Cyrillic or Latin and results filter instantly.

PDFs are hosted on Cloudflare R2; metadata and user submissions are managed by a small Cloudflare Worker + D1 API with moderated review.

Live site

GitHub Pages serves index.html at the repo root. PDF links point at the public R2 URL configured in site.config.json.

What's here

Path	Description
index.html	Search UI with all song titles embedded inline. Generated — don't edit by hand.
add.html	User submission form (PDF + titles → pending review). Generated from add.html.template.
admin.html	Admin moderation queue (approve / reject pending songs).
index.json	Song list: { f: file, t: title (Cyrillic), l: title (Latin) }.
site.config.json	Public (non-secret) URLs: R2 PDF base, Worker API base, Turnstile site key.
build_html.py	Regenerates index.html and add.html from index.json + site.config.json.
worker/	Cloudflare Worker API (/api/submit, /api/songs.json, admin routes) + D1 schema.
scripts/	Provisioning, R2 upload, D1 import, verification helpers. See below.
.env.local.example	Template for local secrets/URLs. Copy to .env.local (gitignored).
pdf_list.txt	List of PDF filenames used by the legacy downloader.
download.sh	Resumable parallel downloader from pelister.org (legacy / bootstrap).
pages/	Raw source index pages the song list was scraped from.

scripts/

Script	Purpose
provision_cloudflare.sh	One-time: create R2 bucket, D1 database, apply schema.
apply_env_config.py	Syncs .env.local → site.config.json + worker/wrangler.toml.
upload_pdfs_s3.py	Bulk-upload pdf/*.pdf to R2 via S3 API (fastest, parallel; recommended).
upload_pdfs.sh	Alternative bulk upload via wrangler r2 object put (slower, no extra deps).
upload_pdfs_rclone.sh	Alternative bulk upload via rclone (S3-compatible).
generate_import_sql.py	Builds worker/import_songs.sql from index.json for D1 import.
verify_pdfs.sh	Spot-checks a random sample of PDFs against the live R2 public URL.
wait_for_worker.sh	Polls a URL (e.g. the Worker's /api/health) until it responds or times out — useful right after enabling a new workers.dev route while DNS catches up.

One-time Cloudflare setup

Prerequisites: a Cloudflare account, R2 enabled on it, and wrangler available (installed as a dev dependency in worker/, or globally).

# 1. Auth
cd worker && npx wrangler login

# 2. Provision R2 bucket + D1 database, apply schema
bash ../scripts/provision_cloudflare.sh
# Copy the printed database_id into worker/wrangler.toml if not done automatically.

# 3. Enable public access on the R2 bucket
#    R2 dashboard → pesnimk-pdfs → Settings → Public access → Enable
#    Copy the pub-*.r2.dev URL.

# 4. Create a Turnstile widget (dashboard → Turnstile) for the domain
#    the "Add a song" form will run on, e.g. YOUR_USERNAME.github.io.

# 5. Copy the env template and fill in real values
cd ..
cp .env.local.example .env.local
#    - R2_ENDPOINT, R2_BUCKET, ACCESS_KEY_ID_WRITE/SECRET_ACCESS_KEY_WRITE
#      (R2 → Manage API Tokens → Create User API Token → Read & Write)
#    - R2_PUBLIC_BASE_URL (from step 3, plus /public/)
#    - GITHUB_PAGES_ORIGIN / ALLOWED_ORIGINS
#    - TURNSTILE_SECRET_KEY (secret key from step 4) + ADMIN_SECRET (any long random string you choose)
#    WORKER_URL is filled in after step 7 below — leave the placeholder for now.

# 6. Sync .env.local into site.config.json + worker/wrangler.toml
python3 scripts/apply_env_config.py

# 7. Set Worker secrets and deploy
cd worker
npx wrangler secret put TURNSTILE_SECRET_KEY
npx wrangler secret put ADMIN_SECRET
npm install
npm run deploy

Important — confirm the real Worker URL. Wrangler may report a workers.dev subdomain that doesn't match your worker's name; the account's workers.dev subdomain is chosen once and shared across all your workers (e.g. pesnimk-api.pesnarka.workers.dev, not pesnimk-api.<your-name>.workers.dev). Always confirm the exact URL in Workers & Pages → pesnimk-api → Domains tab before wiring it up. Then:

# Sanity-check it responds
curl https://pesnimk-api.<your-subdomain>.workers.dev/api/health   # {"ok":true}
# Or, if you just changed DNS/routes and want to wait for it to come up:
bash scripts/wait_for_worker.sh https://pesnimk-api.<your-subdomain>.workers.dev/api/health

# Put the confirmed URL in .env.local as WORKER_URL, then re-sync:
python3 scripts/apply_env_config.py

# 8. Upload existing PDFs to R2 (from repo root)
python3 scripts/upload_pdfs_s3.py

# 9. Import song metadata into D1
python3 scripts/generate_import_sql.py
cd worker && npm run db:import   # wrangler d1 execute ... --remote --file=import_songs.sql

# 10. Rebuild the static site + verify
cd ..
python3 build_html.py
bash scripts/verify_pdfs.sh

Note on D1: wrangler d1 execute --remote rejects explicit BEGIN TRANSACTION/COMMIT statements (each file execution is already atomic) — generate_import_sql.py accounts for this.

Rebuilding the static site

# From approved songs in index.json (or freshly fetched from the API)
python3 build_html.py

build_html.py sorts songs by Cyrillic title, marks any song with m: 1 or a missing local PDF as unavailable (нема in the UI), and also regenerates add.html from add.html.template using site.config.json.

GitHub Action sync

Workflow .github/workflows/sync-songs.yml fetches /api/songs.json from the Worker nightly and rebuilds index.html.

Set repository secret SONGS_API_URL to your confirmed Worker URL (e.g. https://pesnimk-api.pesnarka.workers.dev).

User submissions

1. User opens add.html, uploads a PDF + titles, passes Turnstile.
2. Worker stores the file at pending/<uuid>.pdf in R2 and inserts a pending row in D1.
3. Admin opens admin.html, previews the PDF, approves (moves it to public/NNNN.pdf) or rejects.
4. The nightly sync (or a manual build_html.py run) picks up newly approved songs.

Security notes

• worker/wrangler.toml's [vars] block is not a place for secrets — it ships with the deployed Worker config. Only PDF_PUBLIC_BASE and ALLOWED_ORIGINS (non-secret) belong there.
• Real secrets (TURNSTILE_SECRET_KEY, ADMIN_SECRET) are set via wrangler secret put (production) and worker/.dev.vars (local dev only, gitignored).
• .env.local holds your local R2/Worker credentials and is gitignored — only .env.local.example (a template with no real values) is committed.

Source & attribution

Lyrics and sheet music © George Goce Mitrevski, pelister.org. This repository is a personal, non-commercial archive and search front-end over that material.