diff --git a/apps/be/CLAUDE.md b/apps/be/CLAUDE.md index 9bcaef6b..7d76e1cc 100644 --- a/apps/be/CLAUDE.md +++ b/apps/be/CLAUDE.md @@ -45,7 +45,7 @@ Backend API: Postgres + pgvector, NEP-413 login, run/event storage + SSE, encryp - **Paid data (#150, `src/shop.ts`):** `GET /internal/v1/shop/{item}` answers 402 with a price and `SHOP_PAY_TO`; with `?tx=` it serves a live NEAR RPC report only for this Mind's own `sent` payment row (to `SHOP_PAY_TO`, at least the price, last hour), confirmed by RPC `tx` (success, signer = the Mind's wallet), and records the hash in `shop_purchases` (009) so it is used once. Plain `rpc.testnet.near.org` forgets transactions after a few days, hence the one-hour window. `SHOP_PAY_TO` must be on the paying wallet's address list. Tests use the fake OutLayer's `/rpc` (`tx`, `status`, `gas_price`). - **Several sellers (#176, Best Price Buyer):** `SHOP_SELLERS` (`account:multiplier,…`) lists sellers `s1`, `s2`, … by their place in the list; empty means `SHOP_PAY_TO` alone as `s1`. A bad list stops the backend at start. `?quotes=1` lists them cheapest first with `onYourList`, read from the paying wallet's own OutLayer policy (`addressRuleOf` in `wallets.ts`), so the Mind only picks sellers OutLayer would let it pay. A 402 without `seller` offers the cheapest. A payment counts for the seller its `wallet_payments.to_account` names, at that seller's price; with `?seller=` it must be that seller. Every seller must also be on the demo wallet's list for try-outs, which is a policy the demo wallet's owner signs in Meteor. Intended (11 Nov, not set yet): `SHOP_SELLERS="lin1er.testnet:1.0,testing7.testnet:0.8"`. - **Negotiation (agents track, `src/shop.ts`):** `?seller=sN&offer=` asks one seller to sell for less. Each seller has a floor, a share of its price (third `SHOP_SELLERS` field `account:multiplier:floor`, default 0.8): at or above it the offer is `accepted` (never above the list price), below it the seller `countered`s once, halfway to its price but not under the floor. Stateless; the paid step carries the same `offer` and checks the payment against the agreed price (an offer under the floor never lowers it). The engine's `make_offer` and `buy_data.offerNear` call exactly this; keep the shape (`docs/api-contract.md`). -- **Payments CSV (realworld track):** `GET /v1/minds/{id}/payments.csv` (`paymentsCsv` in `src/payments.ts`), owner only. "Approved by" is read from the run events of the call that made the payment (`tool.result.result.paymentId` → same `toolCallId`'s `tool.called.approvedBy` / `tool.approval_required`), in one query with CTEs: per-row subqueries made it 3× slower under load. Cells that start like a formula get a leading `'`. +- **Payments CSV (realworld track):** `GET /v1/minds/{id}/payments.csv` (`paymentsCsv` in `src/payments.ts`), owner only. "Approved by" is read from the run events of the call that made the payment (`tool.result.result.paymentId` → same `toolCallId`'s `tool.called.approvedBy` / `tool.approval_required`), in one query with CTEs: per-row subqueries made it 3× slower under load. Cells that start like a formula get a leading `'`. **Payments Excel (#298):** `GET /v1/minds/{id}/payments.xlsx` (`paymentsXlsx`), the same rows (`paymentRows`) as a real workbook from `src/xlsx.ts`, a small writer with no dependency (stored zip, `zlib.crc32`). Cells are typed (date serial in UTC with a date format, amounts as numbers, the explorer link as a hyperlink, text as inline strings, which are never formulas), with a bold frozen header and a filter. It exists because a comma CSV opens in one column in Excel on a comma-decimal locale (Indonesia), and no single CSV separator fits every locale. An amount a JS number can't hold exactly stays text. - **Meta-transaction relayer (realworld track, `src/relay.ts`):** `POST /v1/relay { signedDelegate }` (NEP-366, base64 Borsh). The relayer account (`RELAYER_ACCOUNT_ID`, `RELAYER_PRIVATE_KEY`) pays gas **and any inner deposit** (docs.near.org "Meta Transactions", read 13 Nov), and the registry refunds storage to the caller, so only `unequip_skill` / `set_wallet` with exactly 1 yoctoNEAR are relayed; `register_mind` and `equip_skill` need storage (and a price) and are not. Everything that can be refused is refused before sending (sender = signed-in account, receiver = `REGISTRY_CONTRACT_ID`, gas, ed25519 signature over sha256(NEP-461 prefix + delegate), expiry ≤ 1000 blocks, full-access key, nonce) so a bad request costs no gas. 10 per owner per hour, in memory. RPC: `NEAR_RPC_URL`, default `https://test.rpc.fastnear.com`. `borsh` 2.0.0 is a direct dependency for decoding (same version near-api-js uses). Not run on testnet yet: no relayer account is funded; a wallet that signs delegate actions without sending them (Meteor?) is not proven either. - **Mind as Seller (#181, `src/offers.ts`, migration 011):** a Mind publishes an offer with its own wallet token (`PUT /internal/v1/offers/{item}`, stored in `mind_offers`); another Mind buys it with `GET /internal/v1/minds/{sellerMindId}/offers/{item}[?tx=]`, the same 402 and receipt check as the shop but `payTo` is the seller Mind's own wallet address and the RPC `tx` must show receiver = that address; hashes are used once (`offer_purchases`). Try-out and playground Minds can neither sell nor be bought from (the shared demo wallet is nobody's own): `403 demo_not_allowed` / `404 no_offer`. The buyer's wallet must list the seller's 64-hex address, or OutLayer refuses the payment (`policy_denied`). - **Telegram (#168, `src/telegram.ts`, migration 010):** one bot per Mind, made by the owner with @BotFather. `PUT /v1/minds/{id}/telegram { token }` checks it with `getMe`, refuses a bot another Mind uses (`409 telegram_bot_in_use`), stores it AES-GCM with AAD `telegram:{mindId}`, and either sets a webhook (`TELEGRAM_WEBHOOK_BASE` + `/v1/telegram/{mindId}/webhook`, with a random `secret_token`) or, locally, deletes it so `TELEGRAM_POLLING=true` can poll `getUpdates`. The answer has a one-time `t.me/?start=`: the private chat that sends that code becomes the owner; every other chat is ignored without a reply. `POST /v1/telegram/{mindId}/webhook` has no session: the `X-Telegram-Bot-Api-Secret-Token` header must match (`401` otherwise), it answers `200` at once and handles the update after; `last_update_id` drops redelivered updates. An owner message runs as a chat through `runTelegramChat` (`src/server.ts`: `relayRun` into an in-memory sink, so events, threads, the live view and try-out limits are the same as the app), in one thread per bot (`thread_id`), in UTC (no time zone from Telegram). **Streaming (#222, `LiveReply` in `src/telegram.ts`):** the sink passes each relayed event to Telegram as it is written (`memorySink(onEvent)`; `relayRun` and the app path are unchanged). "typing…" is sent every 4 s until the first text shows and again while a tool runs (Telegram clears it after 5 s). The first text is sent at once, then the same message is edited as it grows, at most once per 1.5 s (`telegramTiming`), each time the whole text so far through the converter (a half-written mark stays literal until its pair arrives, so it is always valid HTML); past 4096 characters it continues in a new message. `run.finished` triggers the last edit with the complete formatted reply, so the owner does not wait for `run.verified`. An edit answered "message is not modified" is ignored. Approvals still come as their own messages after the reply. The reply after an approval button press is not streamed yet (sent whole when the resumed run ends). **Rich text (#221, `src/telegramFormat.ts`):** every bot message is sent with `parse_mode: "HTML"`. The Mind's Markdown is converted by our own small converter (no library): bold, italic, strikethrough, inline and fenced code (`
`), links `[text](url)` and bare URLs (http/https only; any other scheme stays visible as text), headings as bold lines, lists as "•"/"1." lines, blockquotes, tables as one line per row; all other text is escaped (`< > & "` only). `telegramChunks` splits at 4096 between blocks or lines, never inside a tag, so each message is valid on its own. A `400` "can't parse entities" resends that message as plain text (`plainOf`). Approval cards (#208) use the same path, but only the title is bold and every other word is escaped, never read as Markdown: their words come from tool arguments (an email subject, an event title someone else wrote), so a stranger cannot inject a link into a message that asks the owner to approve. **Approvals happen in Telegram too (#208, Axel's call 11 Nov; replaces "never in Telegram"):** each approval a Telegram run asks for is its own message (`askOwner`) with the app card's checks or details (`src/approvals.ts`, ported from apps/fe `wallet/data/checks.ts` and `runs/data/approvalDetails.ts`; keep the words in step, and give every new approval tool a case in both, or Telegram shows only its name; long values are cut by `clip` to stay under Telegram's 4096 characters) and buttons Approve / Decline / Open in app (`/minds/{id}/chat/{threadId}`); no Approve when the policy would refuse or the wallet holds less than the payment (`next: 'fund'`, #228; an unfunded wallet's account isn't on chain yet and OutLayer reads its balance as `"0"`, spike 01, so it counts as 0), a button to `/minds/{id}/wallet` instead. Order of reasons: frozen, limit, today's limit, list, balance. Today's limit (#149) is OutLayer's rule `spentTodayNear + amount > dailyNear` (a payment exactly on the cap goes through; the day starts 00:00 UTC), and a `policy_denied` message names it too, since a refused payment doesn't count toward the day. A wallet read that fails still shows no checks and keeps Approve (OutLayer decides). `callback_data` is an opaque one-time code in `telegram_approvals` (013); only `callback_query.from.id` = the owner chat id counts, everyone else gets "Only the owner can approve"; `answerCallbackQuery` is always called; the first press uses up both codes. A press resumes through `resumeRun` (`src/server.ts`), the same function as the app's approve/decline route, then edits the message to the result (never "rejected on-chain", no tx link for a refusal) and sends the Mind's reply and any next approval. `resumeRun` accepts only a call of the run's current pause (`isAwaiting` in `src/runs.ts`: at most one `run.finished` after its `tool.approval_required`) and claims the run (`awaiting_approval` → `running` in one update, undone if the engine refuses), so the app and Telegram cannot resume one run twice. `allowed_updates` is `["message", "callback_query"]`, and every start re-runs `setWebhook` for all bots (`registerTelegramWebhooks`) so bots linked before #208 get presses; a URL button Telegram refuses (`400`) is resent as a link in the text. `DELETE` deletes the webhook (best effort) and the row. Polling skips a bot that errors, so one bad row cannot stop the others. Tests use `test/fake-telegram.ts` (also `answerCallbackQuery`, `editMessageText`, `editMessageReplyMarkup`, `refuseLocalUrls`, `refuseHtmlOnce`, `telegramHtmlError`, which refuses HTML Telegram would refuse, `deleteMessage`, and "message is not modified"; every call has its time `at`) and the fake engine's `fake.scripts` (scripted replies with slow tools, text pacing and verification delay; `fake.timings`) with fresh bot ids per run; the fake engine's `pay  NEAR to [ and …]` asks for `send_payment` one pause at a time and pays through the backend's internal route (`fake.backendUrl`). A labelled link shows its host after it (`[Your bank](http://evil.example/login)` reads "Your bank (evil.example)"), unless the label names it; the plain-text fallback drops that note since it shows the whole URL (Lin1er, #223).
@@ -57,7 +57,7 @@ Backend API: Postgres + pgvector, NEP-413 login, run/event storage + SSE, encryp
 
 ## Commands
 
-- `bun run dev` (port 4000, applies migrations at start), `bun run migrate`, `bun run test:unit` (offline), `bun run test` (unit + integration: local Postgres + fake engine), `bun run load:shop` (the shop's quotes and offers and the payments CSV under load, fake engine and fake OutLayer; fails on any wrong answer), `bun run load` (fake engine and fake Telegram; fails if a run loses events, if a paused run answered twice at once is resumed other than once, or if a Telegram reply streamed alongside the app traffic does not end with the full text), `bun run typecheck`.
+- `bun run dev` (port 4000, applies migrations at start), `bun run migrate`, `bun run test:unit` (offline), `bun run test` (unit + integration: local Postgres + fake engine), `bun run load:shop` (the shop's quotes and offers and the payments CSV and Excel file under load, fake engine and fake OutLayer; fails on any wrong answer), `bun run load` (fake engine and fake Telegram; fails if a run loses events, if a paused run answered twice at once is resumed other than once, or if a Telegram reply streamed alongside the app traffic does not end with the full text), `bun run typecheck`.
 - Postgres: `docker compose up -d postgres` from the repo root; `docker compose stop` when done. If port 55432 is taken by another project, run it on another host port with a compose override (`ports: !override ["55433:5432"]`) and pass `DATABASE_URL=…:55433/hellonear` to `bun run test` / `bun run load`.
 - Schema changes: add a new `migrations/NNN_name.sql` (never edit an applied one) and update `docs/api-contract.md` §5.
 - Tests never call NEAR AI or the NEAR RPC: `test/fake-engine.ts` stands in for the engine and `setProvider()` fakes the access-key lookup; signatures are real. Run `bun run test` and `bun run load` before every push/merge that touches `apps/be`, and report results in the PR. CI: `.github/workflows/be.yml`.
diff --git a/apps/be/load/shop.ts b/apps/be/load/shop.ts
index fb90dae6..ed4b5cd9 100644
--- a/apps/be/load/shop.ts
+++ b/apps/be/load/shop.ts
@@ -1,6 +1,6 @@
 // Short local load test of the paths the track work changed: the paid-data shop's price list and negotiation
 // (`/internal/v1/shop/{item}?quotes=1` and `?seller=&offer=`, with the run's wallet token, as the engine calls them) and
-// the payments CSV (`GET /v1/minds/{id}/payments.csv`, each Mind holding 200 payments with their runs' events). Every
+// the payments CSV and Excel file (`GET /v1/minds/{id}/payments.csv` and `.xlsx`, each Mind holding 200 payments with their runs' events). Every
 // answer must be 200 with the right shape: an offer is accepted or countered at the expected price, and the CSV has one
 // line per payment. Fake engine and fake OutLayer, local Postgres. Never point this at a public endpoint.
 // Run: bun run load:shop   (env: VUS=10, DURATION_S=10)
@@ -58,7 +58,7 @@ async function user() {
 }
 
 const users = await Promise.all(Array.from({ length: VUS }, user))
-const latencies: Record = { quotes: [], offer: [], csv: [] }
+const latencies: Record = { quotes: [], offer: [], csv: [], xlsx: [] }
 let errors = 0
 const fail = (what: string) => { if (errors++ < 5) console.error(what) }
 async function timed(name: string, f: () => Promise) {
@@ -93,6 +93,12 @@ await Promise.all(users.map(async (u, n) => {
       const lines = text.trimEnd().split('\r\n')
       return r.status === 200 && lines.length === PAYMENTS_PER_MIND + 1 && lines.filter((l) => l.includes(',your rule,')).length === PAYMENTS_PER_MIND / 2
     })
+    if (i % 3 === 1) await timed('xlsx', async () => {
+      const r = await fetch(`${base}/v1/minds/${u.mindId}/payments.xlsx`, { headers: { cookie: u.cookie } })
+      const buf = Buffer.from(await r.arrayBuffer())
+      // A zip (PK) holding one row per payment plus the header.
+      return r.status === 200 && buf.subarray(0, 2).toString() === 'PK' && (buf.toString('latin1').match(/ {
+export interface PaymentRow { at: Date; status: string; direction: 'out'; amountNear: string; counterparty: string; purpose: string; approvedBy: string; txHash: string; explorer: string; note: string }
+// The rows behind both downloads (CSV and Excel), newest first.
+export async function paymentRows(mindId: string, limit = 10_000): Promise {
   // One pass: the payment calls in this Mind's runs, then who approved each, then the rows (no per-row subqueries).
   const rows = await query<{ id: string; created_at: Date; status: PaymentStatus; amount_yocto: string; to_account: string; tx_hash: string | null; message: string | null; shop_item: string | null; offer_item: string | null; approved_by: string | null }>(
     `with calls as (
@@ -146,18 +149,40 @@ export async function paymentsCsv(mindId: string, limit = 10_000): Promise [
-    p.created_at.toISOString().replace('T', ' ').slice(0, 19),
-    CSV_STATUS[p.status] ?? p.status,
-    'out',
-    toNear(BigInt(p.amount_yocto)),
-    p.to_account,
-    p.shop_item ? `data report: ${p.shop_item}` : p.offer_item ? `bought from another Mind: ${p.offer_item}` : '',
-    p.approved_by ?? '',
-    p.tx_hash ?? '',
-    p.tx_hash ? `https://testnet.nearblocks.io/txns/${encodeURIComponent(p.tx_hash)}` : '',
-    p.message ?? '',
+  return rows.map((p) => ({
+    at: p.created_at,
+    status: CSV_STATUS[p.status] ?? p.status,
+    direction: 'out' as const,
+    amountNear: toNear(BigInt(p.amount_yocto)),
+    counterparty: p.to_account,
+    purpose: p.shop_item ? `data report: ${p.shop_item}` : p.offer_item ? `bought from another Mind: ${p.offer_item}` : '',
+    approvedBy: p.approved_by ?? '',
+    txHash: p.tx_hash ?? '',
+    explorer: p.tx_hash ? `https://testnet.nearblocks.io/txns/${encodeURIComponent(p.tx_hash)}` : '',
+    note: p.message ?? '',
+  }))
+}
+
+export async function paymentsCsv(mindId: string, limit = 10_000): Promise {
+  const lines = [CSV_HEADER, ...(await paymentRows(mindId, limit)).map((p) => [
+    p.at.toISOString().replace('T', ' ').slice(0, 19), p.status, p.direction, p.amountNear, p.counterparty, p.purpose, p.approvedBy, p.txHash, p.explorer, p.note,
   ])]
   // A BOM so Excel opens it as UTF-8; CRLF line ends per RFC 4180.
   return '' + lines.map((l) => l.map(csvCell).join(',')).join('\r\n') + '\r\n'
 }
+
+// The same rows as an Excel workbook (#298): typed cells, so the columns, the amounts and the dates come out right in
+// any locale, which a CSV can't do (Excel on a comma-decimal locale splits on `;`). An amount that a JS number can't
+// hold exactly (more than ~15 significant digits) stays text rather than being rounded.
+export async function paymentsXlsx(mindId: string, limit = 10_000): Promise {
+  const amount = (s: string) => (String(Number(s)) === s ? Number(s) : s)
+  return xlsx({
+    name: 'Payments',
+    header: CSV_HEADER,
+    widths: [20, 30, 10, 14, 28, 34, 12, 48, 60, 40],
+    rows: (await paymentRows(mindId, limit)).map((p) => [
+      p.at, p.status, p.direction, amount(p.amountNear), p.counterparty, p.purpose, p.approvedBy, p.txHash,
+      p.explorer ? { text: p.explorer, link: p.explorer } : '', p.note,
+    ]),
+  })
+}
diff --git a/apps/be/src/server.ts b/apps/be/src/server.ts
index d7be21a1..7702c64b 100644
--- a/apps/be/src/server.ts
+++ b/apps/be/src/server.ts
@@ -15,7 +15,7 @@ import { explorerReport, quotesOf } from './teeExplorer.ts'
 import { relay } from './relay.ts'
 import { deleteSecret, listSecrets, putSecret, putSecretSchema, secretNameSchema, secretsForRun } from './secrets.ts'
 import { addressRuleOf, assertWalletEmpty, createWallet, freezeTransaction, getWallet, hasWallet, isDemoMind, policySchema, policyTransaction } from './wallets.ts'
-import { listPayments, paymentsCsv, sendPayment, sendPaymentSchema, walletBalance } from './payments.ts'
+import { listPayments, paymentsCsv, paymentsXlsx, sendPayment, sendPaymentSchema, walletBalance } from './payments.ts'
 import { itemSchema, offerNearSchema, sellerSchema, shop, txHashSchema } from './shop.ts'
 import { buyOffer, offerItemSchema, putOffer, putOfferSchema } from './offers.ts'
 import { issueMailToken, issueWalletToken, verifyMailToken, verifyWalletToken } from './walletToken.ts'
@@ -462,6 +462,12 @@ export const server = createServer(async (req, res) => {
         res.writeHead(200, { 'content-type': 'text/csv; charset=utf-8', 'content-disposition': `attachment; filename="hellonear-${mind.id}-payments.csv"`, 'cache-control': 'no-store', 'access-control-expose-headers': 'content-disposition' }).end(csv)
         return
       }
+      // The same as an Excel workbook, for a spreadsheet on any locale (#298).
+      if (sub === '/payments.xlsx' && req.method === 'GET') {
+        const file = await paymentsXlsx(mind.id)
+        res.writeHead(200, { 'content-type': 'application/vnd.openxmlformats-officedocument.spreadsheetml.sheet', 'content-disposition': `attachment; filename="hellonear-${mind.id}-payments.xlsx"`, 'cache-control': 'no-store', 'access-control-expose-headers': 'content-disposition' }).end(file)
+        return
+      }
       if (sub === '/wallet/policy' && req.method === 'POST') denyDemo(user, 'set_policy')
       if (sub === '/wallet/policy' && req.method === 'POST') return sendJson(res, 200, { transaction: await policyTransaction(mind.id, user.nearAccountId, await readBody(req, policySchema)) })
       if ((sub === '/wallet/freeze' || sub === '/wallet/unfreeze') && req.method === 'POST') denyDemo(user, 'freeze')
diff --git a/apps/be/src/xlsx.ts b/apps/be/src/xlsx.ts
new file mode 100644
index 00000000..b7ddcbf6
--- /dev/null
+++ b/apps/be/src/xlsx.ts
@@ -0,0 +1,113 @@
+// A minimal .xlsx writer (one sheet), no dependencies. An .xlsx is a zip of a few XML files; the zip here is "stored"
+// (no compression), which every spreadsheet app opens. Cells are typed, so a number stays a number and a date a date in
+// any locale: the reason it exists is that a comma CSV opens in one column in Excel on a comma-decimal locale (Indonesia,
+// most of Europe), and a `;` CSV breaks every other locale (#298).
+import { crc32 } from 'node:zlib'
+
+export type Cell = string | number | Date | { text: string; link: string } | null
+export interface Sheet { name: string; header: string[]; rows: Cell[][]; widths?: number[] }
+
+// XML 1.0 can't hold most control characters, even escaped; drop them. Tab, LF and CR stay.
+const xml = (s: string) => s
+  .replace(/[\u0000-\u0008\u000b\u000c\u000e-\u001f￾￿]/g, '')
+  .replace(/[&<>"]/g, (c) => ({ '&': '&', '<': '<', '>': '>', '"': '"' })[c]!)
+  .slice(0, 32767)
+
+export function colName(i: number): string {
+  let s = ''
+  for (let n = i + 1; n > 0; n = Math.floor((n - 1) / 26)) s = String.fromCharCode(65 + ((n - 1) % 26)) + s
+  return s
+}
+
+// Excel's date serial: days since 1899-12-30, in UTC (the sheet says "(UTC)" in its header).
+const serial = (d: Date) => d.getTime() / 86_400_000 + 25_569
+
+// Style ids in styles.xml below: 0 default, 1 bold header, 2 date-time, 3 hyperlink.
+const STYLES = `
+
+
+
+
+
+
+
+
+`
+
+function sheetXml(sheet: Sheet): { xml: string; links: string[] } {
+  const links: string[] = []
+  const refs: string[] = []
+  const cell = (v: Cell, ref: string, header = false): string => {
+    if (v === null || v === '') return ''
+    if (typeof v === 'number') return Number.isFinite(v) ? `${v}` : ''
+    if (v instanceof Date) return `${serial(v)}`
+    if (typeof v === 'object') {
+      links.push(v.link)
+      refs.push(``)
+      return `${xml(v.text)}`
+    }
+    // Inline strings are never formulas, so text like "=1+1" stays text without a leading quote.
+    return `${xml(v)}`
+  }
+  const rows = [sheet.header, ...sheet.rows]
+  const data = rows.map((r, i) => `${r.map((v, j) => cell(v, `${colName(j)}${i + 1}`, i === 0)).join('')}`).join('')
+  const last = `${colName(sheet.header.length - 1)}${rows.length}`
+  const cols = sheet.widths?.length ? `${sheet.widths.map((w, i) => ``).join('')}` : ''
+  return {
+    links,
+    xml: `
+
+
+${cols}${data}${refs.length ? `${refs.join('')}` : ''}
+`,
+  }
+}
+
+export function xlsx(sheet: Sheet): Buffer {
+  const { xml: sheetBody, links } = sheetXml(sheet)
+  const files: [string, string][] = [
+    ['[Content_Types].xml', `
+`],
+    ['_rels/.rels', `
+`],
+    ['xl/workbook.xml', `
+`],
+    ['xl/_rels/workbook.xml.rels', `
+`],
+    ['xl/styles.xml', STYLES],
+    ['xl/worksheets/sheet1.xml', sheetBody],
+  ]
+  if (links.length) files.push(['xl/worksheets/_rels/sheet1.xml.rels', `
+${links.map((l, i) => ``).join('')}`])
+  return zip(files.map(([name, body]) => [name, Buffer.from(body, 'utf8')]))
+}
+
+// A stored (uncompressed) zip: local headers + data, then the central directory and its end record.
+export function zip(entries: [string, Buffer][]): Buffer {
+  const parts: Buffer[] = []
+  const central: Buffer[] = []
+  let offset = 0
+  for (const [name, data] of entries) {
+    const n = Buffer.from(name, 'utf8')
+    const crc = crc32(data)
+    const local = Buffer.alloc(30)
+    local.writeUInt32LE(0x04034b50, 0); local.writeUInt16LE(20, 4); local.writeUInt16LE(0x0800, 6); local.writeUInt16LE(0, 8)
+    local.writeUInt16LE(0, 10); local.writeUInt16LE(0x21, 12) // 1980-01-01 00:00, a fixed time keeps the file reproducible
+    local.writeUInt32LE(crc, 14); local.writeUInt32LE(data.length, 18); local.writeUInt32LE(data.length, 22)
+    local.writeUInt16LE(n.length, 26); local.writeUInt16LE(0, 28)
+    const dir = Buffer.alloc(46)
+    dir.writeUInt32LE(0x02014b50, 0); dir.writeUInt16LE(20, 4); dir.writeUInt16LE(20, 6); dir.writeUInt16LE(0x0800, 8); dir.writeUInt16LE(0, 10)
+    dir.writeUInt16LE(0, 12); dir.writeUInt16LE(0x21, 14)
+    dir.writeUInt32LE(crc, 16); dir.writeUInt32LE(data.length, 20); dir.writeUInt32LE(data.length, 24)
+    dir.writeUInt16LE(n.length, 28); dir.writeUInt16LE(0, 30); dir.writeUInt16LE(0, 32); dir.writeUInt16LE(0, 34); dir.writeUInt16LE(0, 36)
+    dir.writeUInt32LE(0, 38); dir.writeUInt32LE(offset, 42)
+    parts.push(local, n, data)
+    central.push(dir, n)
+    offset += local.length + n.length + data.length
+  }
+  const cd = Buffer.concat(central)
+  const end = Buffer.alloc(22)
+  end.writeUInt32LE(0x06054b50, 0); end.writeUInt16LE(entries.length, 8); end.writeUInt16LE(entries.length, 10)
+  end.writeUInt32LE(cd.length, 12); end.writeUInt32LE(offset, 16)
+  return Buffer.concat([...parts, cd, end])
+}
diff --git a/apps/be/test/api.test.ts b/apps/be/test/api.test.ts
index 97b7cecc..b065c1a2 100644
--- a/apps/be/test/api.test.ts
+++ b/apps/be/test/api.test.ts
@@ -1221,6 +1221,69 @@ describe('payment tools through the backend (#42, option B)', () => {
     }
   })
 
+  test('payments Excel (#298): the same rows as typed cells (numbers, dates, links) in a valid .xlsx; owner only', async () => {
+    const { crc32 } = await import('node:zlib')
+    // Read a stored zip: every entry's CRC must match, and names map to their text.
+    const unzip = (buf: Buffer) => {
+      const files: Record = {}
+      const end = buf.lastIndexOf(Buffer.from([0x50, 0x4b, 0x05, 0x06]))
+      let p = buf.readUInt32LE(end + 16)
+      for (let i = 0; i < buf.readUInt16LE(end + 10); i++) {
+        assert.equal(buf.readUInt32LE(p), 0x02014b50)
+        const crc = buf.readUInt32LE(p + 16), size = buf.readUInt32LE(p + 20), nameLen = buf.readUInt16LE(p + 28), at = buf.readUInt32LE(p + 42)
+        const name = buf.subarray(p + 46, p + 46 + nameLen).toString('utf8')
+        assert.equal(buf.readUInt32LE(at), 0x04034b50)
+        const data = buf.subarray(at + 30 + buf.readUInt16LE(at + 26), at + 30 + buf.readUInt16LE(at + 26) + size)
+        assert.equal(crc32(data), crc, `crc of ${name}`)
+        files[name] = data.toString('utf8')
+        p += 46 + nameLen + buf.readUInt16LE(p + 30) + buf.readUInt16LE(p + 32)
+      }
+      return files
+    }
+    const saved = [config.shopPayTo, config.shopSellers] as const
+    try {
+      config.shopPayTo = 'shop.testnet'
+      config.shopSellers = ''
+      const a = await signIn()
+      const { mindId } = await walletMind(a)
+      const token = await tokenFromChat(a, mindId)
+      const pay = async (to: string, amountNear: string) => (await (await engineCall('POST', '/internal/v1/wallet/send_payment', token, { to, amountNear })).json())
+      const sent = await pay('shop.testnet', '0.001')
+      // A note that looks like a formula and needs escaping.
+      await query('update wallet_payments set message = $1 where id = $2', ['=1+1  & "q"', sent.paymentId])
+      const refused = await pay('stranger.testnet', '0.003')
+      assert.equal(refused.status, 'policy_denied')
+
+      const res = await a.call('GET', `/v1/minds/${mindId}/payments.xlsx`)
+      assert.equal(res.status, 200)
+      assert.equal(res.headers.get('content-type'), 'application/vnd.openxmlformats-officedocument.spreadsheetml.sheet')
+      assert.equal(res.headers.get('content-disposition'), `attachment; filename="hellonear-${mindId}-payments.xlsx"`)
+      assert.equal(res.headers.get('access-control-expose-headers'), 'content-disposition')
+      const files = unzip(Buffer.from(await res.arrayBuffer()))
+      assert.deepEqual(Object.keys(files).sort(), ['[Content_Types].xml', '_rels/.rels', 'xl/_rels/workbook.xml.rels', 'xl/styles.xml', 'xl/workbook.xml', 'xl/worksheets/_rels/sheet1.xml.rels', 'xl/worksheets/sheet1.xml'])
+      const sheet = files['xl/worksheets/sheet1.xml']
+      // Header row, bold, the same columns as the CSV.
+      for (const h of ['Date (UTC)', 'Status', 'Amount (NEAR)', 'Approved by', 'Explorer link', 'Note']) assert.match(sheet, new RegExp(`${h.replace(/[()]/g, '\\$&')}`))
+      const rows = [...sheet.matchAll(/(.*?)<\/row>/g)]
+      assert.equal(rows.length, 3)
+      // Amounts are numbers, dates are date serials with the date style, and the explorer link is a hyperlink.
+      assert.match(sheet, /0\.001<\/v><\/c>/)
+      assert.match(sheet, /0\.003<\/v><\/c>/)
+      assert.match(sheet, /\d+\.\d+<\/v><\/c>/)
+      assert.match(files['xl/worksheets/_rels/sheet1.xml.rels'], new RegExp(`Target="https://testnet.nearblocks.io/txns/${sent.txHash}" TargetMode="External"`))
+      assert.match(sheet, /<\/hyperlinks>/)
+      // The note stays text (an inline string is never a formula) and is escaped.
+      assert.match(sheet, /=1\+1 <tag> & "q"<\/t>/)
+      assert.match(sheet, /refused by the wallet policy \(no money moved\)/)
+      assert.match(files['xl/workbook.xml'], //)
+
+      assert.equal((await (await signIn()).call('GET', `/v1/minds/${mindId}/payments.xlsx`)).status, 404)
+      assert.equal((await client()('GET', `/v1/minds/${mindId}/payments.xlsx`)).status, 401)
+    } finally {
+      ;[config.shopPayTo, config.shopSellers] = saved
+    }
+  })
+
   test('Mind as Seller (#181): a Mind publishes an offer; another Mind pays the seller\'s own wallet and gets it, once', async () => {
     const a = await signIn()
     const seller = await walletMind(a)
diff --git a/apps/docs/api.mdx b/apps/docs/api.mdx
index 75fce77a..400caddc 100644
--- a/apps/docs/api.mdx
+++ b/apps/docs/api.mdx
@@ -87,6 +87,7 @@ The web app is a client of a public HTTP API, and everything it does can be done
 | `GET` | `/v1/minds/{mindId}/wallet` | Balance, spent today, frozen, and the policy | 200 | yes |
 | `GET` | `/v1/minds/{mindId}/wallet/payments` | Payment receipts, refused ones included | 200 | yes |
 | `GET` | `/v1/minds/{mindId}/payments.csv` | Every payment as a CSV an accountant can open (date UTC, status in words, amount, counterparty, purpose, who approved, tx hash, NearBlocks link) | 200 `text/csv` | yes |
+| `GET` | `/v1/minds/{mindId}/payments.xlsx` | The same rows as an Excel workbook with typed cells (date, number, hyperlink), so it opens in columns on any locale | 200 `.xlsx` | yes |
 | `GET` | `/v1/status` | Public status of HelloNear and its providers (NEAR AI chat and embeddings, 1Click quotes, OutLayer, NEAR RPC, Agent Market, TEE Explorer): status, latency, uptime, 48 half-hour buckets and incidents of the last 7 days. CORS `*`, cached 30 s | 200 | |
 | `POST` | `/v1/relay` | Relay a NEP-366 signed delegate action to the registry (only `unequip_skill` / `set_wallet`, exactly 1 yoctoNEAR); the relayer pays the gas, never a deposit. Not for try-out users | 200, `503 relay_not_configured` until the relayer is set | |
 | `POST` | `/v1/minds/{mindId}/wallet/policy` | Build a store_wallet_policy transaction for the owner to sign | 200 |  |
diff --git a/apps/docs/payments-csv.mdx b/apps/docs/payments-csv.mdx
index 7f03d75b..89ab18bb 100644
--- a/apps/docs/payments-csv.mdx
+++ b/apps/docs/payments-csv.mdx
@@ -5,7 +5,7 @@ description: "A file of every payment a Mind made or tried, with who approved it
 
 **Track:** `realworld`.
 
-The Wallet page has **Download CSV** beside Recent activity. The file has one row per payment attempt: date, status in words, amount, counterparty, purpose, who approved, tx hash and NearBlocks link.
+The Wallet page has **Download CSV** and **Excel** beside Recent activity. The file has one row per payment attempt: date, status in words, amount, counterparty, purpose, who approved, tx hash and NearBlocks link.
 
 ## Try it
 
@@ -15,6 +15,7 @@ Through the API (owner only, with the session cookie):
 
 ```sh
 curl -s -b jar $B/v1/minds/$MIND/payments.csv
+curl -s -b jar -o payments.xlsx $B/v1/minds/$MIND/payments.xlsx
 ```
 
 ## Details
@@ -23,4 +24,6 @@ curl -s -b jar $B/v1/minds/$MIND/payments.csv
 - Refused payments are listed too, with their reason and no tx hash, because a payment the policy refuses has no transaction.
 - Cells that start like a spreadsheet formula get a leading `'`, so opening the file cannot run a formula.
 
-Source: `paymentsCsv` in `apps/be/src/payments.ts`.
+- **Excel** has the same rows as an `.xlsx` with typed cells: the date is a date, the amount a number, the NearBlocks link a hyperlink. Excel on a locale whose list separator is `;` (Indonesian, most of Europe) opens a comma CSV in one column; the Excel file opens in columns everywhere.
+
+Source: `paymentsCsv` and `paymentsXlsx` in `apps/be/src/payments.ts`.
diff --git a/apps/fe/CLAUDE.md b/apps/fe/CLAUDE.md
index 7b593ab0..3653c457 100644
--- a/apps/fe/CLAUDE.md
+++ b/apps/fe/CLAUDE.md
@@ -128,7 +128,7 @@ Approvals (#14): **no page since 11 Nov** (Axel and Ancung: it repeated the chat
 - The live view's "Waiting for you" rows open the chat that asked. Emergency stop and recent payments live in Wallet. Telegram approvals happen in Telegram (#208); its "Open in app" should go to the chat, not the old page.
 
 Wallet (`/minds/[mindId]/wallet`, `src/modules/wallet`, design "5 · Money" W1, W2, W4):
-- `WalletScreen`: balance with the address (copy on click), "Spent today" (OutLayer's own counter `spentTodayNear` when the backend sends it, else the sent payments since 00:00 UTC; the daily cap resets at 00:00 UTC, measured by James on #149, so local midnight read wrong; with a daily cap it reads "of X NEAR"), Recent activity from `GET …/wallet/payments` with All / Sent / Blocked, a **Download CSV** button beside its title once there are payments (a plain link to `GET …/payments.csv`, so the browser downloads it with the session cookie; `FileDownIcon` from lucide), and "Your limits" beside it. States: normal, empty (Add funds becomes the main button), frozen (banner, Unfreeze), loading, no wallet yet (W1.6, "Set up a wallet"), and a wallet with no limits ("Set limits"). Try-out users and the shared demo wallet only read.
+- `WalletScreen`: balance with the address (copy on click), "Spent today" (OutLayer's own counter `spentTodayNear` when the backend sends it, else the sent payments since 00:00 UTC; the daily cap resets at 00:00 UTC, measured by James on #149, so local midnight read wrong; with a daily cap it reads "of X NEAR"), Recent activity from `GET …/wallet/payments` with All / Sent / Blocked, a **Download CSV** and **Excel** buttons beside its title once there are payments (`GET …/payments.csv` and `…/payments.xlsx`, fetched with the session cookie and saved as a blob named from `Content-Disposition`, since on the hosted site the API is another origin where `download` is ignored, #297; Excel is the one that opens right on an Indonesian-locale Excel, #298; `FileDownIcon` and `FileSpreadsheetIcon` from lucide), and "Your limits" beside it. States: normal, empty (Add funds becomes the main button), frozen (banner, Unfreeze), loading, no wallet yet (W1.6, "Set up a wallet"), and a wallet with no limits ("Set limits"). Try-out users and the shared demo wallet only read.
 - Money coming in isn't in the payments list (it records what the Mind tried to send), so the design's "Added by you" rows don't show; W5 (all activity, payment detail) isn't built yet.
 - `LimitsDialog` (W2): per-payment limit, an optional "Per day" cap (#149, `dailyError` in `minds/data/limits.ts`: above 0, at most 100, not below the per-payment limit; empty = none; it starts from the stored cap, because a save without it would remove it) and addresses with the rules from `minds/data/limits.ts`, "Was 2" on a changed limit, "New" on added addresses, a summary of what the Mind may do after saving, then `useSaveLimits`: create the wallet if there is none (409 is fine), `POST …/wallet/policy`, the owner signs `store_wallet_policy` in Meteor (it costs a small deposit), and the wallet is read back until the new limits show.
 - `AddFundsDialog` (W4): the owner's Meteor wallet sends a plain `Transfer` to the Mind's address (no backend call), then the balance is read back until it grew. The owner's sendable balance comes from the testnet RPC (`view_account`, `NEAR_RPC_URL` in `src/lib/config.ts`): the balance minus storage at 10^19 yoctoNEAR per byte over the free 770 bytes (docs.near.org, checked 9 Nov). Quick picks are 1, 2, 5 NEAR, not the design's 5, 10, 20, because the faucet gives 5.
diff --git a/apps/fe/src/modules/wallet/components/WalletScreen.tsx b/apps/fe/src/modules/wallet/components/WalletScreen.tsx
index 5686d308..c19cefc8 100644
--- a/apps/fe/src/modules/wallet/components/WalletScreen.tsx
+++ b/apps/fe/src/modules/wallet/components/WalletScreen.tsx
@@ -1,6 +1,6 @@
 "use client"
 
-import { CheckIcon, CircleSlashIcon, CopyIcon, FileDownIcon, HourglassIcon, LockIcon, PlusIcon, SnowflakeIcon, WalletIcon, XIcon } from "lucide-react"
+import { CheckIcon, CircleSlashIcon, CopyIcon, FileDownIcon, FileSpreadsheetIcon, HourglassIcon, LockIcon, PlusIcon, SnowflakeIcon, WalletIcon, XIcon } from "lucide-react"
 import { useState, type ReactNode } from "react"
 import { toast } from "sonner"
 import { cn } from "cn"
@@ -200,18 +200,20 @@ const TONE = {
   waiting: { icon: HourglassIcon, dot: "bg-warning-soft text-warning", badge: "warning" as const, amount: "text-foreground" },
 }
 
-async function downloadCsv(mindId: string) {
+// "csv" for any tool; "xlsx" for Excel on any locale (a comma CSV opens in one column on a comma-decimal locale, #298).
+async function downloadPayments(mindId: string, kind: "csv" | "xlsx") {
+  const failed = `Could not download the ${kind === "csv" ? "CSV" : "Excel file"}. Try again.`
   try {
-    const res = await fetch(apiUrl(`/minds/${encodeURIComponent(mindId)}/payments.csv`), { credentials: "include" })
-    if (!res.ok) throw new Error(res.status === 401 ? "Your session ended. Sign in again to download." : "Could not download the CSV. Try again.")
-    const name = /filename="([^"]+)"/.exec(res.headers.get("content-disposition") ?? "")?.[1] ?? "hellonear-payments.csv"
+    const res = await fetch(apiUrl(`/minds/${encodeURIComponent(mindId)}/payments.${kind}`), { credentials: "include" })
+    if (!res.ok) throw new Error(res.status === 401 ? "Your session ended. Sign in again to download." : failed)
+    const name = /filename="([^"]+)"/.exec(res.headers.get("content-disposition") ?? "")?.[1] ?? `hellonear-payments.${kind}`
     const url = URL.createObjectURL(await res.blob())
     const a = Object.assign(document.createElement("a"), { href: url, download: name })
     a.click()
     URL.revokeObjectURL(url)
   } catch (e) {
     // A network failure is a TypeError ("Failed to fetch"); only our own messages are shown as they are.
-    toast.error(e instanceof Error && e.name !== "TypeError" ? e.message : "Could not download the CSV. Try again.")
+    toast.error(e instanceof Error && e.name !== "TypeError" ? e.message : failed)
   }
 }
 
@@ -224,14 +226,20 @@ function Activity({ mindId, name }: { mindId: string; name: string }) {
     
       

Recent activity

- {/* Every payment as a CSV an accountant can open (GET …/payments.csv). Fetched and saved as a blob, + {/* Every payment as a CSV an accountant can open (GET …/payments.csv, and …/payments.xlsx for Excel). Fetched and saved as a blob, not a plain link: on the hosted site the API is another origin, where browsers ignore `download`, and an expired session would open the JSON error in the tab (Ancung, #297 review). */} {payments?.length ? ( - +
+ + +
) : null}
{isPending ? ( diff --git a/docs/api-contract.md b/docs/api-contract.md index 1cdc0cec..aa7fcc53 100644 --- a/docs/api-contract.md +++ b/docs/api-contract.md @@ -453,6 +453,7 @@ One OutLayer wallet per Mind, on testnet. The backend keeps the agent key and ne | `POST /v1/minds/{mindId}/wallet/freeze`, `…/unfreeze` | `{}` | `{ transaction }` for `freeze_wallet` / `unfreeze_wallet` (30 Tgas, no deposit). A frozen wallet's transfers get `403 wallet_frozen` before signing **[tested]** | | `GET /v1/minds/{mindId}/wallet/payments?limit=20` | | `{ payments }`, the same receipts the engine's `list_payments` reads (Receipts screen) **[tested]** | | `GET /v1/minds/{mindId}/payments.csv` | | Every payment of the Mind as a CSV for an accountant (realworld track): `text/csv; charset=utf-8`, UTF-8 with a BOM, CRLF, `Content-Disposition: attachment; filename="hellonear-{mindId}-payments.csv"`. Columns `Date (UTC)` (`YYYY-MM-DD HH:MM:SS`), `Status` in words (`sent`, `pending`, `refused by the wallet policy (no money moved)`, `refused, wallet frozen (no money moved)`, `failed (no money moved)`), `Direction` (`out`), `Amount (NEAR)`, `Counterparty`, `Purpose` (`data report: ` / `bought from another Mind: `, else empty), `Approved by` (`owner` when the run paused on `tool.approval_required` for that call, `your rule` when `tool.called` carries `approvedBy: "rule"`, else empty), `Transaction hash`, `Explorer link` (NearBlocks testnet), `Note` (the stored message). Newest first, up to 10,000 rows. RFC 4180 quoting; a cell starting with `=`, `+`, `-`, `@` gets a leading `'` so spreadsheets don't run it **[tested; load-tested locally with 200 payments per Mind]** | +| `GET /v1/minds/{mindId}/payments.xlsx` | | The same rows and columns as `payments.csv`, as an Excel workbook (#298): `application/vnd.openxmlformats-officedocument.spreadsheetml.sheet`, `Content-Disposition: attachment; filename="hellonear-{mindId}-payments.xlsx"`, one sheet `Payments`. Typed cells, so it opens right in Excel on any locale (a comma CSV opens in one column where `;` is the list separator, e.g. Indonesian): `Date (UTC)` is a date (`yyyy-mm-dd hh:mm:ss`), `Amount (NEAR)` a number, `Explorer link` a hyperlink, everything else text (never a formula). Bold frozen header with a filter. Owner only, like the CSV **[tested; load-tested locally with 200 payments per Mind]** | | `POST /v1/relay` | `{ signedDelegate }`: base64 of a Borsh `SignedDelegate` (NEP-366, near-api-js `encodeSignedDelegate`) the owner signed | Meta-transaction relayer (realworld track, `src/relay.ts`): the backend's relayer account (`RELAYER_ACCOUNT_ID` / `RELAYER_PRIVATE_KEY`) wraps the delegate in a transaction to the owner and pays the gas. `200` `{ txHash, status: "success" \| "failure", senderId, receiverId, methods, failure? }`. Refused before anything is sent: `403 delegate_sender_mismatch` (sender ≠ the signed-in account), `400` `receiver_not_allowed` (only `REGISTRY_CONTRACT_ID`), `method_not_allowed` (only `unequip_skill`, `set_wallet`), `deposit_not_allowed` (exactly 1 yoctoNEAR: the relayer pays inner deposits, docs.near.org "Meta Transactions"), `action_not_allowed`, `too_many_actions` (> 4), `gas_too_high` (> 30 Tgas a call), `args_too_large`, `args_not_json`, `unsupported_key_type` (ed25519 only), `bad_signature`, `invalid_signed_delegate`, `delegate_expired`, `delegate_ttl_too_long` (> 1000 blocks), `unknown_access_key`, `key_not_full_access`, `stale_nonce`; `429 relay_rate_limited` (10 per owner per hour), `403 demo_not_allowed`, `503 relay_not_configured \| rpc_unreachable \| relay_failed` **[tested: unit + fake RPC; never run on testnet: no relayer account is funded or configured]** | `transaction` = `{ receiverId: "outlayer.testnet", actions: [{ type: "FunctionCall", params: { methodName, args, gas, deposit } }] }`. OutLayer down → `503 outlayer_unreachable` (`reason`); refusing → `503 outlayer_error` (`outlayerStatus`). Deleting a Mind deletes its wallet row: move the balance out first (a later route will do it). diff --git a/docs/openapi.json b/docs/openapi.json index 2aaac9b6..f3661292 100644 --- a/docs/openapi.json +++ b/docs/openapi.json @@ -1289,6 +1289,43 @@ ] } }, + "/v1/minds/{mindId}/payments.xlsx": { + "get": { + "tags": [ + "Wallet" + ], + "summary": "Every payment as an Excel workbook (opens in columns on any locale)", + "responses": { + "200": { + "description": "`application/vnd.openxmlformats-officedocument.spreadsheetml.sheet`, downloaded as `hellonear-{mindId}-payments.xlsx`. The same rows and columns as `payments.csv`, in one sheet `Payments` with typed cells: Date (UTC) is a date, Amount (NEAR) a number, Explorer link a hyperlink, everything else text (never a formula); bold frozen header with a filter. Use it where Excel's list separator is `;` (e.g. an Indonesian locale), which opens a comma CSV in one column. Newest first, up to 10,000 rows", + "content": { + "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet": { + "schema": { + "type": "string", + "format": "binary" + } + } + } + }, + "401": { + "description": "Not signed in" + }, + "404": { + "description": "`mind_not_found` (not yours)" + } + }, + "parameters": [ + { + "name": "mindId", + "in": "path", + "required": true, + "schema": { + "type": "string" + } + } + ] + } + }, "/v1/minds/{mindId}/wallet/policy": { "post": { "tags": [