Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,9 @@ jobs:

secret-scan:
runs-on: ubuntu-latest
permissions:
contents: read
pull-requests: read
steps:
- name: Checkout
uses: actions/checkout@v4
Expand All @@ -38,5 +41,3 @@ jobs:
uses: gitleaks/gitleaks-action@v2
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
config-path: .gitleaks.toml
3 changes: 0 additions & 3 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -58,9 +58,6 @@ next-env.d.ts
# Scrollback config (written by onboarding; may contain credentials)
scrollback.config.json

# Legacy config filename from the pre-release project name
feedsilo.config.json

# SQLite generated Prisma client
/lib/generated/prisma-sqlite

Expand Down
89 changes: 87 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,30 @@ It gives you a browser-extension capture flow for X/Twitter, RSS syncing, a poli

Built with Next.js 16, React 18, Prisma 7, PostgreSQL/pgvector, SQLite, Tailwind CSS, and optional OpenAI or Google Gemini AI providers.

## Launch Trailer

The production launch trailer is committed with the repo and published by the app as a static asset:

- Video: [`/launch/scrollback-launch-trailer.mp4`](public/launch/scrollback-launch-trailer.mp4)
- Poster: [`/launch/scrollback-launch-trailer-poster.jpg`](public/launch/scrollback-launch-trailer-poster.jpg)
- Source kit: [`scrollback-launch-trailer/`](scrollback-launch-trailer/)

After deployment, the public video URL will be:

```text
https://scrollback.atomtan.studio/launch/scrollback-launch-trailer.mp4
```

Launch asset map:

| Path | Purpose |
| --- | --- |
| `public/launch/scrollback-launch-trailer.mp4` | Public launch trailer served by the deployed app |
| `public/launch/scrollback-launch-trailer-poster.jpg` | Poster/share image for the launch trailer |
| `public/trailer-capture/*.svg` | Lightweight feature cards used for trailer and launch collateral |
| `scrollback-launch-trailer/` | Full editable HyperFrames production kit |
| `scrollback-launch-trailer/renders/scrollback-launch-trailer-command-center-glasscore-v4.mp4` | Selected final render kept with the source kit |

## Features

- **One-click X/Twitter capture** - Save posts, threads, linked article metadata, images, GIFs, and videos from the browser extension.
Expand All @@ -28,6 +52,21 @@ Scrollback is focused on two capture paths:

It can enrich linked article cards and some long-form X article content when that data is available in the page or X API responses, but it is not meant to be a universal web clipper yet.

## Browser Extension and X API Safety

The Scrollback Capture browser extension can save individual X/Twitter posts, threads, media, and linked article cards without an X API token. That keeps setup simple and is useful for normal one-click capture.

For bulk capture, especially syncing Likes and Bookmarks, Scrollback strongly recommends using an official X API bearer token or OAuth connection. API mode is more reliable, avoids scraping private X web endpoints, and is the lowest-risk way to import large parts of your account activity.

Without the X API, the extension can still attempt page-based capture from visible X pages as you scroll. That mode is best-effort and depends on X's web app behavior. Because it may rely on internal X web requests or page scraping, use it thoughtfully and understand that it can carry more account risk than official API access.

Practical guidance:

- Use the extension without X API for occasional one-click saves.
- Add an X API bearer token before bulk importing Likes or Bookmarks.
- Use OAuth from Settings when you want direct bookmark and like syncing.
- If account safety matters more than convenience, prefer API mode.

## Quick Start

### Requirements
Expand Down Expand Up @@ -64,6 +103,40 @@ The onboarding wizard does the same setup work you can do manually:

Admin account creation is required. AI provider, X API, RSS feeds, local/R2 media storage, and extension pairing can all be configured later from Settings.

## SQLite Onboarding Smoke Test

Use this before launch, before publishing a release, or whenever setup code changes. Run it from a clean checkout or temporary copy so existing `.env.local`, `scrollback.config.json`, and database files do not mask first-run behavior.

```bash
nvm use 22
npm install
SCROLLBACK_SETUP_TOKEN="$(openssl rand -base64 32)" npm run dev
```

Then open [http://localhost:3000/onboarding](http://localhost:3000/onboarding) and follow the wizard:

1. Paste the setup token from `SCROLLBACK_SETUP_TOKEN`.
2. Choose **Local SQLite**.
3. Keep the default database path, or enter another local path such as `./scrollback-smoke.db`.
4. Continue through migration.
5. Create the first admin account.
6. Skip AI provider and X API setup unless you are testing those integrations.
7. Skip or save the extension pairing token.
8. Confirm the app opens at `/` and you are signed in.

Expected local files after a successful SQLite onboarding smoke test:

- `scrollback.config.json`
- `.env.local`
- the SQLite database file you selected, for example `scrollback.db`

Expected app state:

- `/api/setup/status` returns `configured: true` and `databaseType: "sqlite"`.
- The SQLite database contains the core tables, including `users`, `content_items`, `categories`, and `rss_feeds`.
- The `users` table has one admin user.
- Reloading `/` shows the authenticated Scrollback app, not onboarding.

## Configuration

For local development, either use onboarding or copy `.env.example` to `.env.local`.
Expand All @@ -88,14 +161,16 @@ Environment variables override values from `scrollback.config.json`.
| `R2_SECRET_ACCESS_KEY` | No | R2 API token secret key |
| `R2_BUCKET_NAME` | No | R2 bucket name |
| `LOCAL_MEDIA_PATH` | No | Local directory for media storage when not using R2 |
| `XAPI_BEARER_TOKEN` | No | App-only X API bearer token for tweet lookup fallback |
| `XAPI_CLIENT_ID` | No | X OAuth 2.0 client ID for bookmark/like sync |
| `XAPI_BEARER_TOKEN` | Recommended for bulk capture | App-only X API bearer token for safer extension bulk capture and tweet lookup fallback |
| `XAPI_CLIENT_ID` | No | X OAuth 2.0 client ID for direct bookmark/like sync |
| `XAPI_CLIENT_SECRET` | No | X OAuth 2.0 client secret |
| `XAPI_ENCRYPTION_KEY` | No | 32-byte hex key for encrypted X OAuth token storage |
| `DEMO_EMAIL`, `DEMO_PASSWORD` | No | Creates a read-only demo user at startup |

## Production Deploy

Production target: `https://scrollback.atomtan.studio` on a Hetzner VPS behind HTTPS.

### 1. Use Node 22

Scrollback is pinned to Node 22.x. Set your host, CI, Docker image, or runtime to Node 22 before installing dependencies.
Expand Down Expand Up @@ -156,6 +231,12 @@ http://localhost:3000

For production, use your deployed HTTPS origin. After code changes in `extension/`, reload the unpacked extension in your browser.

For the safest bulk capture experience, also configure an X API bearer token in the extension popup or Settings before importing Likes or Bookmarks. One-click saves can work without it, but API mode is recommended for account safety and reliability.

### 6. Static launch assets

The launch trailer is served by Next.js from `public/launch/`. No separate object storage or CDN step is required for the launch video unless traffic exceeds the VPS bandwidth budget.

## Scripts

| Script | Description |
Expand Down Expand Up @@ -194,6 +275,10 @@ Expected result for a clean launch branch:

See [RELEASE_CHECKLIST.md](RELEASE_CHECKLIST.md) for the full release checklist.

## Launch Posting Plan

Use [docs/LAUNCH_PLAN.md](docs/LAUNCH_PLAN.md) for the launch-day posting checklist, asset URLs, and channel plan.

## Known Limitations

- X/Twitter capture is best-effort. X changes its markup and article surfaces
Expand Down
8 changes: 5 additions & 3 deletions RELEASE_CHECKLIST.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ Use this before tagging or publishing the open-source release.
- [ ] Merge the release branch into `dev`.
- [ ] Run `npm run verify`.
- [ ] Run `npm audit --audit-level=moderate`.
- [ ] Smoke-test a fresh first-run setup with `SCROLLBACK_SETUP_TOKEN`.
- [ ] Smoke-test a fresh first-run SQLite setup with `SCROLLBACK_SETUP_TOKEN`; follow the README's "SQLite Onboarding Smoke Test" and confirm login works.
- [ ] Capture one X/Twitter post with media through the extension.
- [ ] Add and sync one RSS feed.
- [ ] Export or query a saved item through the remote query API.
Expand All @@ -33,6 +33,8 @@ Use this before tagging or publishing the open-source release.

## Launch Assets

- [ ] Capture 2-4 clean product screenshots.
- [ ] Create a launch image or short demo clip.
- [ ] Confirm `public/launch/scrollback-launch-trailer.mp4` opens locally after `npm run dev`.
- [ ] Confirm `public/launch/scrollback-launch-trailer-poster.jpg` opens locally after `npm run dev`.
- [ ] Confirm the deployed trailer URL opens at `https://scrollback.atomtan.studio/launch/scrollback-launch-trailer.mp4`.
- [ ] Confirm the source trailer kit in `scrollback-launch-trailer/` is committed.
- [ ] Write the launch post with the project URL, scope, and known limitations.
17 changes: 10 additions & 7 deletions app/api/setup/migrate/route.ts
Original file line number Diff line number Diff line change
Expand Up @@ -54,10 +54,12 @@ async function pushSchema(config: ScrollbackConfig, schemaFlag: string): Promise
});
}

async function ensureSqliteParentDirectory(databaseUrl: string): Promise<void> {
async function ensureSqliteDatabaseFile(databaseUrl: string): Promise<void> {
const sqlitePath = resolveSqliteFilePath(databaseUrl);
if (!sqlitePath) return;
await fs.mkdir(path.dirname(sqlitePath), { recursive: true });
await fs.mkdir(/* turbopackIgnore: true */ path.dirname(sqlitePath), { recursive: true });
const handle = await fs.open(/* turbopackIgnore: true */ sqlitePath, "a");
await handle.close();
}

async function verifySqliteSchema(databaseUrl: string): Promise<void> {
Expand Down Expand Up @@ -136,10 +138,6 @@ export async function POST(request: NextRequest) {
localMedia: {},
};

// Write config files
writeConfig(config);
invalidateConfigCache();

// Run Prisma db push
const schemaFlag =
config.database.type === "sqlite"
Expand All @@ -148,7 +146,7 @@ export async function POST(request: NextRequest) {

try {
if (config.database.type === "sqlite") {
await ensureSqliteParentDirectory(config.database.url);
await ensureSqliteDatabaseFile(config.database.url);
}
await pushSchema(config, schemaFlag);
// eslint-disable-next-line @typescript-eslint/no-explicit-any
Expand Down Expand Up @@ -187,6 +185,11 @@ export async function POST(request: NextRequest) {
}
}

// Write config files only after schema setup succeeds. This avoids locking
// onboarding into a partially configured instance after a failed migration.
writeConfig(config);
invalidateConfigCache();

// For PostgreSQL, try to create pgvector extension
if (config.database.type !== "sqlite") {
try {
Expand Down
76 changes: 76 additions & 0 deletions docs/LAUNCH_PLAN.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
# Scrollback Launch Plan

## Launch Goal

Launch Scrollback at **9:00 AM Pacific on Friday, May 8, 2026**.

Definition of launched:

- The PR is merged and deployed.
- `https://scrollback.atomtan.studio` loads over HTTPS.
- The trailer URL opens publicly.
- The GitHub repository/release is public and accurate.
- Announcement posts go out within the same launch window.
- The first hour is monitored for broken links, setup issues, replies, and questions.

## Canonical Links

- App: `https://scrollback.atomtan.studio`
- Trailer: `https://scrollback.atomtan.studio/launch/scrollback-launch-trailer.mp4`
- Trailer poster: `https://scrollback.atomtan.studio/launch/scrollback-launch-trailer-poster.jpg`
- GitHub repository: `https://github.com/atomtanstudio/scrollback`

## Asset Checklist

- `public/launch/scrollback-launch-trailer.mp4` is the deployed trailer file.
- `public/launch/scrollback-launch-trailer-poster.jpg` is the sharing/poster image.
- `scrollback-launch-trailer/` contains the full editable HyperFrames production kit.
- `scrollback-launch-trailer/renders/scrollback-launch-trailer-command-center-glasscore-v4.mp4` is the selected final render.

## Pre-Post Checklist

- Run `npm run verify`.
- Run `npm audit --audit-level=moderate`.
- Deploy the latest `main` branch to the Hetzner VPS.
- Confirm DNS points `scrollback.atomtan.studio` at the VPS.
- Confirm HTTPS is active for `scrollback.atomtan.studio`.
- Open the app, trailer, poster, README, privacy policy, and release checklist from the public domain.
- Smoke-test first-run SQLite onboarding from the README.
- Capture one X/Twitter post with the browser extension.
- Confirm README language recommends X API mode for bulk Likes and Bookmarks.

## Simultaneous Posting Window

Target all public posts for **9:00-9:15 AM Pacific**. Prepare all drafts before 8:45 AM so launch is copy/paste and link-checking, not writing.

Launch channels:

- GitHub release: trailer URL, self-hosting scope, setup notes, known limitations.
- X/Twitter: trailer-first post with app URL and GitHub URL.
- LinkedIn: privacy/self-hosting angle with trailer and repository link.
- Hacker News: concise "Show HN" post after the public URL and repository are verified.
- Reddit: targeted communities only, with a useful self-hosting/open-source framing rather than drive-by promotion.
- Relevant Discord/Slack communities: short post, trailer, GitHub, and what feedback would be useful.

Recommended posting order inside the 15-minute window:

1. Publish GitHub release.
2. Post X/Twitter with trailer.
3. Post LinkedIn.
4. Submit Hacker News.
5. Post Reddit/community links.
6. Reply to early questions and pin/favorite the best launch post as the canonical thread.

## Launch Drafts Needed

- X/Twitter: short trailer post plus a 4-6 post follow-up thread.
- LinkedIn: slightly longer post emphasizing self-hosting, privacy, and personal archives.
- Hacker News: title and first comment with technical details and limitations.
- Reddit: one version for self-hosted/open-source audiences and one for AI/builder audiences.
- GitHub release notes: what it is, how to run it, known limitations, and safety note for X API bulk capture.

## Positioning

Scrollback is a self-hosted archive for X/Twitter saves, RSS, media, prompts, search, AI enrichment, and agent-native retrieval. The launch message should stay concrete: own the archive, search it locally, enrich it when desired, and expose it to tools without giving a hosted service the library.

Account-safety wording for extension posts: one-click capture works without the X API, but the official X API bearer token or OAuth path is recommended for bulk importing Likes and Bookmarks.
6 changes: 1 addition & 5 deletions lib/config/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -6,14 +6,10 @@ import { configSchema, type ScrollbackConfig, type DatabaseType } from "./schema

const PROJECT_ROOT = /* turbopackIgnore: true */ process.cwd();
const DEFAULT_CONFIG_PATH = path.join(PROJECT_ROOT, "scrollback.config.json");
const LEGACY_CONFIG_PATH = path.join(PROJECT_ROOT, "feedsilo.config.json");
const DEFAULT_ENV_PATH = path.join(PROJECT_ROOT, ".env.local");

function resolveConfigPath(configPath: string): string {
if (configPath !== DEFAULT_CONFIG_PATH) return configPath;
if (fs.existsSync(DEFAULT_CONFIG_PATH)) return DEFAULT_CONFIG_PATH;
if (fs.existsSync(LEGACY_CONFIG_PATH)) return LEGACY_CONFIG_PATH;
return DEFAULT_CONFIG_PATH;
return configPath;
}

export function readConfig(
Expand Down
6 changes: 3 additions & 3 deletions package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

3 changes: 2 additions & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
"bugs": {
"url": "https://github.com/atomtanstudio/scrollback/issues"
},
"homepage": "https://github.com/atomtanstudio/scrollback#readme",
"homepage": "https://scrollback.atomtan.studio",
"engines": {
"node": "22.x"
},
Expand Down Expand Up @@ -89,6 +89,7 @@
},
"overrides": {
"@hono/node-server": "1.19.13",
"hono": "4.12.18",
"anymatch": {
"picomatch": "2.3.2"
},
Expand Down
4 changes: 1 addition & 3 deletions proxy.ts
Original file line number Diff line number Diff line change
Expand Up @@ -97,9 +97,7 @@ export async function proxy(request: NextRequest) {
!pathname.startsWith("/api/")
) {
const hasEnv = Boolean(process.env.DATABASE_URL && process.env.DATABASE_TYPE);
const configured =
request.cookies.get("scrollback-configured") ??
request.cookies.get("feedsilo-configured");
const configured = request.cookies.get("scrollback-configured");
if (!hasEnv && configured?.value !== "true") {
return NextResponse.redirect(new URL("/onboarding", request.url));
}
Expand Down
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added public/launch/scrollback-launch-trailer.mp4
Binary file not shown.
10 changes: 10 additions & 0 deletions public/trailer-capture/launch-thread.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
10 changes: 10 additions & 0 deletions public/trailer-capture/media-storage.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
10 changes: 10 additions & 0 deletions public/trailer-capture/prompt-library.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Loading