Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 39 additions & 0 deletions .github/workflows/status.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
# Status page CI: typecheck, unit tests and a production build. No secrets needed. The build reads the live
# status API once (ISR); if it cannot, the page still builds and says so.
name: status

on:
pull_request:
paths: ['apps/status/**', '.github/workflows/status.yml']
push:
branches: [main]
paths: ['apps/status/**', '.github/workflows/status.yml']

concurrency:
group: status-${{ github.ref }}
cancel-in-progress: true

permissions:
contents: read

jobs:
build:
runs-on: ubuntu-latest
timeout-minutes: 15
defaults:
run:
working-directory: apps/status
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: 24.22.1
- uses: oven-sh/setup-bun@v2
with:
bun-version: 1.3.9
- run: bun install --frozen-lockfile
- run: bun run typecheck
- run: bun run test
- run: bun run build
env:
NEXT_TELEMETRY_DISABLED: '1'
2 changes: 2 additions & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,7 @@ HelloNear is an AI agent persona platform on NEAR, built for NEAR HACKATHON (1
|---|---|---|
| `apps/fe` | Web app (Next.js App Router, NEAR Connect) | Ancung |
| `apps/landing` | Landing page | Nabil |
| `apps/status` | Status page, status.hellonear.xyz (Vercel): whether HelloNear and the services it runs on answer, read from the backend's `GET /v1/status` | Axel |
| `apps/be` | Backend API, Postgres + pgvector, NEP-413 login, OutLayer + Composio | James |
| `engine` | AI engine (Mastra + NEAR AI Cloud) | Axel |
| `sc` | Mind registry contract (Rust `near-sdk`) | Axel |
Expand Down Expand Up @@ -71,6 +72,7 @@ Each folder has its own workflow in `.github/workflows/`. It runs on pull reques
| `fe` | `apps/fe` | lint, typecheck, `next build` |
| `sc` | `sc` | unit tests, sandbox tests (incl. upgrades from the fixtures), clippy |
| `landing` | `apps/landing` | typecheck, `next build` |
| `status` | `apps/status` | typecheck, unit tests, `next build` |

CI does not replace the local integration and load tests required by the rules below. Nothing deploys from CI itself. A merged `main` is deployed to the hosted demo after its CI passes, by a poller on James's always-on LAN machine that comments the result on the merged PR (rule 3, `deploy/README.md`).

Expand Down
4 changes: 3 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@ HelloNear gives a person a **Mind**: an AI agent with a persona, memory, skills
| **Web app** (no account needed: *Continue as guest*) | **[app.hellonear.xyz](https://app.hellonear.xyz)** |
| **Landing page** | **[hellonear.xyz](https://hellonear.xyz)** |
| **API** (NEAR testnet) | **[api.hellonear.xyz/v1/health](https://api.hellonear.xyz/v1/health)** |
| **Status** (HelloNear and every provider it uses, checked every 5 minutes) | **[status.hellonear.xyz](https://status.hellonear.xyz)** · raw: [api.hellonear.xyz/v1/status](https://api.hellonear.xyz/v1/status) |
| **Mind registry contract** | **[`hellonear-registry.testnet`](https://testnet.nearblocks.io/address/hellonear-registry.testnet)** (v0.3.0, NEP-330, [source verified](https://testnet.nearblocks.io/address/hellonear-registry.testnet?tab=contract)) |
| **Swarm auction contract** | **[`auction.hellonear-swarm.testnet`](https://testnet.nearblocks.io/address/auction.hellonear-swarm.testnet?tab=contract)** (NEP-330, [source verified](https://testnet.nearblocks.io/address/auction.hellonear-swarm.testnet?tab=contract), [a real round](#the-swarm-auction-one-real-round-on-testnet)) |
| **API reference** | **[OpenAPI spec](docs/openapi.json)** · [README section](#api-reference) |
Expand Down Expand Up @@ -652,7 +653,7 @@ Every route and event is in [`docs/api-contract.md`](docs/api-contract.md).

## API reference

The web app is a client of a public HTTP API, and everything it does can be done with `curl`. The full spec is **[`docs/openapi.json`](docs/openapi.json)** (OpenAPI 3.1, 62 operations): open it in [Swagger Editor](https://editor.swagger.io/?url=https://raw.githubusercontent.com/Hellonear-AI/hellonear/main/docs/openapi.json) or any OpenAPI viewer to read every route and try it against the hosted API. Every server-sent event type is described in [`docs/api-contract.md`](docs/api-contract.md) §3.
The web app is a client of a public HTTP API, and everything it does can be done with `curl`. The full spec is **[`docs/openapi.json`](docs/openapi.json)** (OpenAPI 3.1, 63 operations): open it in [Swagger Editor](https://editor.swagger.io/?url=https://raw.githubusercontent.com/Hellonear-AI/hellonear/main/docs/openapi.json) or any OpenAPI viewer to read every route and try it against the hosted API. Every server-sent event type is described in [`docs/api-contract.md`](docs/api-contract.md) §3.

- **Base URL:** `https://api.hellonear.xyz` (NEAR testnet), or `http://localhost:4000`.
- **Auth:** a session cookie, `hn_session`. Get one with `POST /v1/auth/demo` (guest, no wallet) or with a NEAR wallet: `POST /v1/auth/challenge`, sign the message (NEP-413), then `POST /v1/auth/login`. `curl` works as is (no `Origin` header); a browser page on another site gets `403` for any call that changes something (CSRF protection).
Expand Down Expand Up @@ -736,6 +737,7 @@ The web app is a client of a public HTTP API, and everything it does can be done
| `GET` | `/v1/minds/{mindId}/wallet` | Balance, spent today, frozen, and the policy | 200 | yes |
| `GET` | `/v1/minds/{mindId}/wallet/payments` | Payment receipts, refused ones included | 200 | yes |
| `GET` | `/v1/minds/{mindId}/payments.csv` | Every payment as a CSV an accountant can open (date UTC, status in words, amount, counterparty, purpose, who approved, tx hash, NearBlocks link) | 200 `text/csv` | yes |
| `GET` | `/v1/status` | Public status of HelloNear and its providers (NEAR AI chat and embeddings, 1Click quotes, OutLayer, NEAR RPC, Agent Market, TEE Explorer): status, latency, uptime, 48 half-hour buckets and incidents of the last 7 days. CORS `*`, cached 30 s | 200 | |
| `POST` | `/v1/relay` | Relay a NEP-366 signed delegate action to the registry (only `unequip_skill` / `set_wallet`, exactly 1 yoctoNEAR); the relayer pays the gas, never a deposit. Not for try-out users | 200, `503 relay_not_configured` until the relayer is set | |
| `POST` | `/v1/minds/{mindId}/wallet/policy` | Build a store_wallet_policy transaction for the owner to sign | 200 | |
| `POST` | `/v1/minds/{mindId}/wallet/freeze` | Build a freeze_wallet transaction for the owner to sign | 200 | |
Expand Down
4 changes: 4 additions & 0 deletions apps/be/.env.example
Original file line number Diff line number Diff line change
Expand Up @@ -70,3 +70,7 @@ OUTLAYER_HANDOFF_URL=
OUTLAYER_SUB_SKILL_PAYMENTS=
# Optional, default https://proof.t16z.com: Phala's TEE Attestation Explorer, where POST /v1/runs/{id}/proof/tee-explorer uploads a run's Intel quote (#277; tests point it at a fake)
TEE_EXPLORER_URL=
# Optional, status page checks (src/status.ts, GET /v1/status); defaults are the real services, tests point them at fakes
STATUS_ONECLICK_URL=
STATUS_MARKET_URL=
STATUS_NEAR_RPC_URL=
3 changes: 2 additions & 1 deletion apps/be/CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ Backend API: Postgres + pgvector, NEP-413 login, run/event storage + SSE, encryp
- **Thread ids are issued by the backend** (`thr_…`). A `threadId` from the browser is only accepted if a run of that same Mind already used it: engine memory is keyed by thread, so a foreign id would read another Mind's history.
- The chat relay (`src/runs.ts`) reads the engine stream to the end even when the browser leaves, so `run_events` is complete. Store every event before forwarding it; deltas are batched. A resumed run continues the paused run's `seq`; if the engine restarted (its counter is in memory) the backend shifts the new events after the stored ones.
- **Verification proof (#254):** the engine puts the proof behind the badge on `run.verified.verification.proof` (~12 KB, mostly the Intel quote; hashes and keys' addresses only, no conversation text). It is stored whole in `run_events`, but `withoutProof` drops it from every streamed event and `NO_PROOF` from `GET /v1/runs/{runId}` and the Mind Graph's `…/events`; only `GET /v1/runs/{runId}/proof` (owner only, `runProofs`) returns it, one entry per `run.verified` that has one, `404 no_proof` when none. Any new route that returns stored events must strip it too.
- **Status page data (`src/status.ts`, `src/statusMath.ts`, `GET /v1/status`, status.hellonear.xyz):** every 5 minutes (`startStatusProbes`, first round 20 s after start, not in tests) the backend checks 9 services in parallel, each with its own timeout and degraded threshold (`SERVICES`), and stores one row each in `service_checks` (migration 016, kept 8 days). `hellonear-api` (Postgres), `hellonear-engine` (`/v1/health`), `nearai-chat` / `nearai-embeddings` (the engine's `GET /internal/v1/probes`, since only the engine has the NEAR AI key; nothing stored for them when the engine does not answer), `near-intents-quote` (a dry 1Click quote, 10 USDC NEAR → Base), `outlayer` (`/health`), `near-rpc` (FastNear `status`), `agent-market`, `tee-explorer`. `GET /v1/status` is handled before the app's CORS: public, `access-control-allow-origin: *`, no cookies, `cache-control: public, max-age=30`, answer cached 30 s and dropped after each round. The numbers (latest status, `unknown` after 15 minutes, uptime 24 h / 7 d with degraded as up, 48 half-hour buckets with `checks` / `bad` / `detail`, incidents as runs of non-operational checks ended by one good check) are pure functions in `statusMath.ts`. Upstreams can be pointed at fakes with `STATUS_ONECLICK_URL`, `STATUS_MARKET_URL`, `STATUS_NEAR_RPC_URL`, `TEE_EXPLORER_URL`, `OUTLAYER_URL`. Details are short and never carry keys or URLs. Contract and thresholds: `docs/api-contract.md` §7. Tests: `test/status-math.test.ts` (unit), `test/status.test.ts` (local Postgres, fake upstreams).
- **TEE Explorer link (#277, `src/teeExplorer.ts`):** `POST /v1/runs/{runId}/proof/tee-explorer` (owner only, `getRun`) uploads each Intel quote in the run's stored proofs (`quotesOf`, one per attestation) to Phala's TEE Attestation Explorer through its own upload, `POST {TEE_EXPLORER_URL}/api/upload` with multipart `file` = the quote's bytes (default `https://proof.t16z.com`; read live 12 Nov from its page script: there is no URL that prefills a quote, and the API sends no CORS headers, so the FE cannot call it). It answers `{ runId, reports: [{ attestation, kind, url, verified }] }`, `url` = `/reports/<checksum>`, a page that already reads VERIFIED (a local run's real gateway quote came back `verified: true`). The checksum must be 64 lowercase hex before it goes into a link. One upload per quote and process (in-memory `reports`, keyed by the quote's SHA-256). Only the quote leaves (NEAR AI's hardware report and nonce, no conversation text); it then shows in the Explorer's public "Recent attestations". `404 no_proof` without one, `503 tee_explorer_unreachable` / `tee_explorer_error` when the Explorer fails (never 502), `422 not_a_quote` for anything but hex. Tests: `test/fake-explorer.ts` and the two `TEE Explorer link (#277)` tests in `test/api.test.ts`.
- **Cancel (#121):** each relayed run gets an `AbortController` (passed to `engineFetch` as `signal`, kept in the in-memory `live` map in `src/runs.ts` once its `runId` is known). `POST /v1/runs/{id}/cancel` aborts it; the relay then stores `run.finished` `{ status: "canceled" }` (builder: `workflow.finished`) as the last event, sets `runs.status = 'canceled'` and ends the stream, and the route answers once that is stored. A browser leaving still does **not** cancel. Not live (finished, paused, or after a restart): `409 not_running`; a paused run is declined, not canceled. Single process only: with more than one backend process the map would have to move to Postgres.
- **Stuck runs (#218):** `sweepStuckRuns` (`src/runs.ts`, at start and every 5 minutes) ends a run left `running` by a backend that died mid-stream: not in `live`, and no event for `STUCK_AFTER_MS` (5 minutes; the engine gives up after 2). It must look stuck on two sweeps in a row at the same last `seq`, because an approval claims the run (`running`) just before its resumed stream starts. It then gets status `error` and a `run.error` event `run_interrupted`, so it can't block an approval or the chat's delete.
Expand Down Expand Up @@ -64,4 +65,4 @@ Backend API: Postgres + pgvector, NEP-413 login, run/event storage + SSE, encryp

## Env

See `.env.example`: `DATABASE_URL`, `ENGINE_URL`, `ENGINE_API_KEY`, `SECRETS_MASTER_KEY` (32 bytes base64; losing it loses every stored secret), optional `PORT`, `FE_ORIGIN`, `AUTH_RECIPIENT`, `NEAR_RPC_URL`, `COOKIE_SECURE`, `OUTLAYER_URL`, `OUTLAYER_CONTRACT_ID`, `DEMO_MODE`, `DEMO_OUTLAYER_API_KEY`, `COMPOSIO_API_KEY`, `COMPOSIO_URL`, `AGENTMAIL_API_KEY`, `AGENTMAIL_API_KEY_2`, `AGENTMAIL_URL`, `DEMO_AGENTMAIL_INBOX`, `TEE_EXPLORER_URL`, `SHOP_PAY_TO`, `SHOP_SELLERS`, `RELAYER_ACCOUNT_ID`, `RELAYER_PRIVATE_KEY`, `REGISTRY_CONTRACT_ID`. `.env` is gitignored; never commit it.
See `.env.example`: `DATABASE_URL`, `ENGINE_URL`, `ENGINE_API_KEY`, `SECRETS_MASTER_KEY` (32 bytes base64; losing it loses every stored secret), optional `PORT`, `FE_ORIGIN`, `AUTH_RECIPIENT`, `NEAR_RPC_URL`, `COOKIE_SECURE`, `OUTLAYER_URL`, `OUTLAYER_CONTRACT_ID`, `DEMO_MODE`, `DEMO_OUTLAYER_API_KEY`, `COMPOSIO_API_KEY`, `COMPOSIO_URL`, `AGENTMAIL_API_KEY`, `AGENTMAIL_API_KEY_2`, `AGENTMAIL_URL`, `DEMO_AGENTMAIL_INBOX`, `TEE_EXPLORER_URL`, `SHOP_PAY_TO`, `SHOP_SELLERS`, `RELAYER_ACCOUNT_ID`, `RELAYER_PRIVATE_KEY`, `REGISTRY_CONTRACT_ID`, `STATUS_ONECLICK_URL`, `STATUS_MARKET_URL`, `STATUS_NEAR_RPC_URL`. `.env` is gitignored; never commit it.
12 changes: 12 additions & 0 deletions apps/be/migrations/016_service_checks.sql
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
-- Status page (status.hellonear.xyz): one row per check of HelloNear or a service it depends on, every 5 minutes,
-- kept 8 days (src/status.ts). `GET /v1/status` reads the last 7 days from here.
create table service_checks (
id bigserial primary key,
service text not null,
checked_at timestamptz not null default now(),
status text not null check (status in ('operational', 'degraded', 'down')),
latency_ms integer,
detail text
);
create index service_checks_service_at on service_checks (service, checked_at desc);
create index service_checks_at on service_checks (checked_at);
2 changes: 1 addition & 1 deletion apps/be/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
"dev": "node --env-file=.env --import tsx src/server.ts",
"migrate": "node --env-file=.env --import tsx src/migrate.ts",
"test": "node --env-file-if-exists=.env --import tsx --test --test-concurrency=1 test/*.test.ts",
"test:unit": "node --import tsx --test test/crypto.test.ts test/sse.test.ts test/telegram-format.test.ts test/approvals.test.ts test/relay.test.ts",
"test:unit": "node --import tsx --test test/crypto.test.ts test/sse.test.ts test/telegram-format.test.ts test/approvals.test.ts test/relay.test.ts test/status-math.test.ts",
"load": "node --env-file-if-exists=.env --import tsx load/api.ts",
"load:shop": "node --env-file-if-exists=.env --import tsx load/shop.ts",
"typecheck": "tsc --noEmit"
Expand Down
5 changes: 5 additions & 0 deletions apps/be/src/server.ts
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ import { HttpError, assertSameOrigin, cors, logServerError, readBody, sendJson,
import { TOOLKITS, connect, disconnect, listConnections, toolkitSchema } from './connections.ts'
import { createMind, createMindSchema, deleteMind, listMinds, mindConfig, playgroundAttackOf, playgroundMind, requireMind, updateMind, updateMindSchema, type Mind } from './minds.ts'
import { cancelRun, getRun, isAwaiting, listRuns, mindEvents, pendingApprovals, relayRun, runEvents, runProofs, sweepStuckRuns } from './runs.ts'
import { serveStatus, startStatusProbes } from './status.ts'
import { explorerReport, quotesOf } from './teeExplorer.ts'
import { relay } from './relay.ts'
import { deleteSecret, listSecrets, putSecret, putSecretSchema, secretNameSchema, secretsForRun } from './secrets.ts'
Expand Down Expand Up @@ -211,6 +212,8 @@ export const telegramRunner: MindRunner = { chat: runTelegramChat, answer: answe

export const server = createServer(async (req, res) => {
try {
// The public status page's data (status.hellonear.xyz): open to any origin, without cookies, so before the app's CORS.
if ((req.url ?? '').split('?')[0] === '/v1/status') return await serveStatus(req, res)
cors(req, res)
if (req.method === 'OPTIONS') return res.writeHead(204).end()
assertSameOrigin(req)
Expand Down Expand Up @@ -605,6 +608,8 @@ if (import.meta.url === `file://${process.argv[1]}`) {
const applied = await migrate()
if (applied.length) console.log('migrations applied', applied.join(', '))
server.listen(config.port, () => console.log(`backend listening on :${config.port}`))
// Status page checks (src/status.ts): every 5 minutes, the first one shortly after start.
startStatusProbes()
// Judge try-out users are deleted after the retention: at start and once a day.
const sweep = () => cleanupDemoUsers(async (mindId) => { await deleteAllWorkflows(mindId); await disconnectTelegram(mindId).catch(() => {}) }).then((n) => { if (n) console.log('demo users removed', n) }, (e) => console.error('demo cleanup failed', e instanceof Error ? e.message : e))
void sweep()
Expand Down
Loading
Loading