Skip to content

data-market.md

Latest commit

 

History

History
200 lines (161 loc) · 12.7 KB

data-market.md

File metadata and controls

200 lines (161 loc) · 12.7 KB

Data Market

This is the canonical status doc for the Data Market.

Purpose

The Data Market prices access to useful context under explicit permissions.

Kernel-facing objects:

  • DataAsset
  • AccessGrant
  • PermissionPolicy
  • DeliveryBundle
  • RevocationReceipt

The data market exists so access to artifacts, datasets, local context, and private knowledge is explicit, permissioned, and receipted rather than being smuggled through opaque prompt state.

Current repo verdict

Dimension Status Notes
Product surface seller authoring + read surface + narrow buyer request surface there is now a dedicated read-only data-market pane in the desktop app plus a Data Seller conversational authoring lane with a structured local draft, seller-specific Codex session wiring, auto-provisioned first-party seller skills, typed openagents.data_market.* tools, seller-side targeted-request intake/evaluation, seller payment-required issuance, seller delivery-bundle/result publication, and explicit seller revoke/expire controls with RevocationReceipt read-back; the panes now also surface package metadata, exact preview state, request relay/kind, and fulfillment posture so shell-first/headless publication remains visually legible; there is now also a narrow Data Buyer pane that selects a visible asset, shows bundle/posture context, and publishes a targeted DS-DVM fulfillment request, but broader buyer transaction UX is still incomplete
Kernel authority retired compatibility slice compatibility data types still exist in openagents-kernel-core and desktop app state, but the DS-first launch path no longer uses or depends on Nexus authority routes
Wire/proto retired compatibility slice the checked-in openagents.data.v1 package remains in-repo for compatibility code, but relay-native DS publication/fulfillment is the canonical operator path
Local prototype implemented richer provenance, packaging, and private-data economics live mostly in docs and adjacent desktop concepts
Planned yes broader discovery, pricing, payouts, provider economics, and product-facing UX remain planned

Implemented now

  • register a DataAsset
  • create an AccessGrant
  • accept an access grant
  • issue a DeliveryBundle
  • revoke access and emit a RevocationReceipt
  • open a dedicated Data Seller desktop shell with transcript, draft-card, exact preview card, and gated confirm/publish scaffolding
  • auto-provision and pin the first-party autopilot-data-seller and autopilot-data-market-control skills for the dedicated seller lane
  • expose typed openagents.data_market.* dynamic tools for seller status, draft, exact preview, blocked publish, and snapshot flows
  • send seller prompts from the dedicated pane into the dedicated Codex seller thread and render the resulting transcript back into the pane
  • require an explicit preview-confirm step before publication can be armed
  • publish a DataAsset-compatible seller row from the pane while also publishing the canonical DS listing to the relay
  • reflect newly published assets into the read-only Data Market pane from the same local seller state and relay catalog path
  • surface redacted Codex-export/package metadata in the seller draft card and in the read-only market rows when those fields are present in published asset metadata
  • carry reusable grant-policy templates plus a first seller-side grant draft posture in the conversational seller flow
  • preview and publish AccessGrant-compatible seller rows from the same flow while also publishing the canonical DS offer to the relay
  • show a seller-side inventory/status card with published asset/default-offer summaries, exact preview state, latest request relay/kind context, and draft-vs-published warnings
  • derive and publish an explicit OpenAgents-local NIP-90 data-vending profile from the seller pane into the shared relay/runtime lane
  • parse incoming OpenAgents data-vending requests on the desktop relay lane as openagents.data.access demand instead of treating them as malformed compute jobs
  • advertise the current data-vending request kind and coarse asset-family/delivery metadata on the provider NIP-89 handler when a seller profile is present
  • surface incoming targeted data-access requests in the seller pane and seller status tools with explicit evaluation outcomes such as no_published_asset, grant_required, scope_mismatch, and ready_for_payment_quote
  • generate seller-side Lightning invoices for matched targeted DS-DVM data requests, publish NIP-90 payment-required feedback, and track the request through invoice_requested, publishing_feedback, awaiting_payment, and paid
  • prepare seller-side delivery drafts for paid targeted DS-DVM requests, issue delivery metadata, and publish linked DS access-contract/result events from the same seller flow
  • let the seller revoke or expire access from the same flow and immediately reflect the terminal grant/delivery state into both panes
  • record recent asset/grant/payment/delivery/revocation lifecycle entries in the read-only Data Market pane so operator-facing activity includes policy, counterparty, and receipt context
  • open a dedicated Data Buyer desktop pane that derives a request draft from the visible market snapshot, selects an active asset/default offer, and publishes a targeted DS-DVM data-access request without widening into public discovery
  • show buyer-side bundle and market-posture context for the currently selected asset before request publication
  • package local files or directories into deterministic listing-template.json, grant-template.json, packaging-manifest.json, and packaging-summary.json artifacts through scripts/autopilot/data_market_package.py
  • package recent or explicit Codex rollout conversations into a redacted seller-ready bundle through scripts/autopilot/package_codex_conversations.py
  • publish canonical DS dataset listings (30404) and DS dataset offers (30406) for seller assets and grants
  • publish DS access contracts (30407) as the relay-native fulfillment and settlement surface
  • surface relay-backed DS catalog rows in the read-only Data Market pane and narrow Data Buyer pane
  • attach optional NIP-99 classified wrappers, optional NIP-15 storefront wrappers, and optional NIP-28 dataset discussion channels back onto the canonical DS relay rows when present
  • drive the same seller path through autopilotctl data-market ... without opening the visible UI window
  • start a no-window Data Market runtime through autopilot_headless_data_market
  • use the repo-owned skills/autopilot-data-seller-cli skill for shell-first packaging and publication discipline
  • resolve a delivered local DeliveryBundle back into copied buyer-side files through autopilotctl data-market consume-delivery
  • run scripts/autopilot/headless-data-market-e2e.sh to mechanically verify the local headless publish -> request -> delivery -> consume path
  • run scripts/autopilot/headless-data-market-public-e2e.sh as an operator probe against live public relays
  • run scripts/autopilot/verify-data-market-cli-headless.sh to mechanically verify the publish/consume path plus the critical lifecycle checks
  • allow both seller request intake and buyer result tracking to run in a relay-only online posture without requiring a compute-ready local inference runtime
  • normalize targeted buyer/seller identity matching across npub and raw hex Nostr pubkeys for the current DS-DVM targeted request/result flow
  • import a targeted request or buyer response back from configured relays through autopilotctl data-market seller-import-request and autopilotctl data-market buyer-import-response

The legacy compatibility code still present in-repo is limited to:

  • crates/openagents-kernel-core/src/data.rs
  • crates/openagents-kernel-core/src/authority.rs
  • apps/nexus-control/src/kernel.rs

The former /v1/kernel/data/* HTTP routes in apps/nexus-control are retired and no longer mounted.

Local prototype or partial only

  • richer provenance modeling beyond the starter permission and delivery objects
  • private-data packaging and policy detail beyond the current starter shapes, though there is now a deterministic local packaging helper for turning a selected file or folder boundary into truthful draft inputs
  • local or adjacent desktop concepts for context packaging, but not a generalized canonical data-market product surface
  • broader public discovery and richer indexing ideas still live in docs rather than in a full market-facing catalog surface

Local packaging helper

The current repo now includes a deterministic local packaging helper:

  • scripts/autopilot/data_market_package.py

Its job is narrow:

  • select a package boundary from one or more local files or directories
  • compute a canonical bundle digest from a sorted manifest of file digests
  • emit a stable provenance_ref
  • write listing-template.json
  • optionally write grant-template.json
  • write packaging-manifest.json and packaging-summary.json

The helper is intentionally local and preparatory. It does not publish by itself.

The generated outputs map directly into the current seller flow:

  • listing-template.json -> autopilotctl data-market draft-asset --file ...
  • grant-template.json -> autopilotctl data-market draft-grant --file ...
  • packaging-summary.json -> local operator or agent audit surface for the chosen package boundary and resulting digests

CLI and headless control

The repo now also includes a real shell-first control path:

  • apps/autopilot-deprecated/src/bin/autopilotctl.rs
  • apps/autopilot-deprecated/src/bin/autopilot_headless_data_market.rs
  • skills/autopilot-data-seller-cli/
  • docs/headless-data-market.md

This path is intentionally not a second seller implementation. It targets the same app-owned seller state and kernel mutation logic through the typed desktop-control contract.

The current DS-first verification posture is:

  • seller DS listing kind 30404 publishes before buyer request publication
  • seller DS offer kind 30406 is then visible in buyer selection state
  • buyer request kind 5960 is now the DS-DVM fulfillment request, not the market listing itself
  • seller result kind 6960 is now the DS-DVM fulfillment result, not the market listing itself
  • the portable repo-owned verifier proves the local DS-first path with a local relay, DS selection state, seller request matching, relay-only consume resolution, and local relay event-kind evidence
  • the repo also retains a public-relay harness, but live relay health is external and not a deterministic launch gate
  • the explicit relay import commands still exist as operator recovery tools if a relay regression needs to be worked around
  • normal operator headless runs keep Codex enabled so autopilotctl data-market seller-prompt ... still drives the conversational seller lane
  • repo-owned smoke and E2E harnesses set OPENAGENTS_DISABLE_CODEX=true because they use the typed DS-first CLI path rather than the conversational seller lane

The freshest launch-path audit is:

  • docs/audits/2026-03-22-relay-only-headless-data-market-paid-e2e-audit.md

The current buyer-side consume step is local by design:

  • autopilotctl data-market consume-delivery resolves the matching delivery from DS relay result/access-contract state, falling back to local DeliveryBundle rows when present
  • current headless verification brings the buyer online in a relay-only posture so targeted DS-DVM result events are actually observed before local consume
  • it currently supports local file:// and plain local-path delivery_ref values
  • it copies the delivered payload and local manifest refs into a chosen output directory
  • it is not yet a general remote blob retrieval client

Not implemented yet

  • a full transactional buyer-facing data market in Autopilot beyond the current narrow targeted-request pane
  • fully integrated seller publication UX beyond the current asset/grant/payment/delivery/revocation control slice
  • public discovery, listing, and richer pricing/comparison surfaces for data buyers
  • payout and provider-economics flows specific to data access

Current repo truth lives in

  • crates/nostr/core/src/nip_ds.rs
  • crates/openagents-kernel-core/src/data.rs
  • apps/autopilot-deprecated/src/data_market_control.rs
  • apps/autopilot-deprecated/src/data_seller_control.rs
  • apps/autopilot-deprecated/src/data_buyer_control.rs
  • apps/autopilot-deprecated/src/input/tool_bridge.rs
  • ../economy-kernel.md
  • ../economy-kernel-proto.md

Boundary notes

  • data sells permissioned access to context
  • compute sells bounded machine capacity
  • labor sells machine work and outcome delivery
  • liquidity moves value between rails and participants
  • risk prices uncertainty, verification difficulty, and liability