Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
93 changes: 93 additions & 0 deletions skills/verifiable-web-research/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,93 @@
---
name: verifiable-web-research
description: Build an auditable web research packet where each claim is tied to source metadata, exact extracts, and replay guidance.
runx:
category: research
---

# Verifiable Web Research

Create a bounded research packet from web-source evidence with enough
provenance for an independent reviewer to replay the work. The skill turns a
research objective plus fetched source snapshots into claims, direct extracts,
content digests, and verification steps.

Use this when a downstream decision depends on web claims that must be checked
later. Use `web-fetch` when the caller already knows a single URL. Use
`research` when provenance is useful but digest-level replay is not required.

## Inputs

- `objective` (required): the question the evidence packet must answer.
- `source_fixture_path` (required): package-relative JSON fixture containing
source snapshots. The deterministic publish harness may use
`builtin:ai-agent-frameworks` when external fixture files are unavailable.
- `verification_level` (optional): `basic`, `detailed`, or `audit_ready`.
Defaults to `detailed`.
- `max_claims` (optional): maximum number of evidence claims to include.
- `output_dir` (optional): package-relative directory for `evidence.json` and
`report.md`.

## Source fixture contract

The fixture is a JSON object:

```json
{
"sources": [
{
"url": "https://example.com",
"final_url": "https://example.com",
"fetched_at": "2026-06-22T00:00:00Z",
"status": 200,
"content": "Source text",
"extracts": [
{ "claim": "Claim text", "quote": "Exact supporting quote" }
]
}
]
}
```

The runner never reaches the network. It operates on already-fetched public
snapshots so verification can be deterministic and safe in harness runs.

## Procedure

1. Validate the objective, fixture path, and verification level.
2. Load the fixture from inside the skill directory.
3. For each source, compute `sha256` over the captured content.
4. Convert each extract into a claim record with source URL, final URL,
accessed timestamp, digest, exact quote, and confidence.
5. Produce a verification guide explaining how to re-fetch the URLs and compare
fresh content against the stored extracts and digests.
6. Write optional artifacts and print the packet as JSON.

## Output

The runner emits `verifiable_research_packet`:

```yaml
schema: runx.verifiable_web_research.result.v1
data:
objective: string
verification_level: basic | detailed | audit_ready
summary: string
claims:
- claim: string
source_url: string
final_url: string
accessed_at: string
content_digest: string
extract: string
confidence: verified | likely | uncertain
confidence_reasoning: string
verification_guide:
overview: string
steps: array
evidence_archive:
sources: array
```

`basic` omits the evidence archive. `detailed` includes source digests and
claims. `audit_ready` includes replay commands and every source record.
100 changes: 100 additions & 0 deletions skills/verifiable-web-research/X.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,100 @@
skill: verifiable-web-research
version: "0.1.0"

catalog:
kind: skill
audience: public
visibility: public
role: canonical

harness:
cases:
- name: verifiable-web-research-audit-ready-packet
runner: default
inputs:
objective: Compare captured AI agent framework evidence.
source_fixture_path: builtin:ai-agent-frameworks
verification_level: audit_ready
max_claims: 2
expect:
status: sealed
receipt:
schema: runx.receipt.v1

- name: verifiable-web-research-missing-fixture-fails
runner: default
inputs:
objective: Reject runs without a captured source fixture.
verification_level: detailed
expect:
status: failure

runx:
mutating: false
idempotency:
key: source_fixture_sha256
scopes:
- research.evidence.read
policy:
data_classification: public_web_snapshot_metadata
network:
allowed: []
forbidden:
- live network fetches during deterministic harness runs
- private or authenticated sources
verifier_notes:
- The runner uses already-fetched public source snapshots.
- Each claim includes an exact extract and a sha256 digest of the captured source content.
artifacts:
emits:
- verifiable_research_packet
- evidence_json
- report_md
wrap_as: verifiable_research_packet

runners:
default:
default: true
type: cli-tool
command: node
args:
- run.mjs
scopes:
- research.evidence.read
policy:
reads:
- package-local public source fixtures
writes:
- evidence.json
- report.md
disallows:
- network calls
- credential material
- private source snapshots
inputs:
objective:
type: string
required: true
description: Research question to answer from captured web evidence.
source_fixture_path:
type: string
required: true
description: Package-relative JSON fixture with captured source snapshots.
verification_level:
type: string
required: false
default: detailed
description: basic, detailed, or audit_ready.
max_claims:
type: number
required: false
default: 10
description: Maximum number of evidence claims to include.
output_dir:
type: string
required: false
description: Directory inside the skill directory where artifacts should be written.
outputs:
verifiable_research_packet: object
evidence_json: object
report_md: string
30 changes: 30 additions & 0 deletions skills/verifiable-web-research/fixtures/ai-agent-frameworks.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
{
"sources": [
{
"url": "https://example.org/langchain-pricing",
"final_url": "https://example.org/langchain-pricing",
"fetched_at": "2026-06-22T00:00:00Z",
"status": 200,
"content": "LangChain is open source. LangSmith is an optional hosted observability product with a free developer tier.",
"extracts": [
{
"claim": "LangChain has an open-source framework and an optional hosted observability product.",
"quote": "LangChain is open source. LangSmith is an optional hosted observability product with a free developer tier."
}
]
},
{
"url": "https://example.org/crewai-docs",
"final_url": "https://example.org/crewai-docs",
"fetched_at": "2026-06-22T00:00:05Z",
"status": 200,
"content": "CrewAI provides a multi-agent framework with hosted enterprise options for teams.",
"extracts": [
{
"claim": "CrewAI offers a multi-agent framework with hosted enterprise options.",
"quote": "CrewAI provides a multi-agent framework with hosted enterprise options for teams."
}
]
}
]
}
103 changes: 103 additions & 0 deletions skills/verifiable-web-research/references/evidence.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,103 @@
{
"schema": "runx.verifiable_web_research.delivery_evidence.v1",
"summary": "Published and verified a reusable Runx skill that builds auditable web research packets from captured source snapshots. The skill emits exact extracts, content digests, confidence reasoning, replay guidance, and optional evidence/report artifacts without reaching the network during execution.",
"skill": {
"name": "verifiable-web-research",
"published_ref": "jienigoto/verifiable-web-research@sha-f0d326500848",
"owner": "jienigoto",
"publisher_principal": "user_476edbcb37a0fa1c989987ad",
"registry": "https://api.runx.ai",
"public_url": "https://runx.ai/x/jienigoto/verifiable-web-research",
"versioned_public_url": "https://runx.ai/x/jienigoto/verifiable-web-research@sha-f0d326500848",
"source_url": "https://github.com/jienigoto/runx/tree/fd8ab4418d8bb9420b76d5d02178bcf53868755e/skills/verifiable-web-research",
"pr_url": "https://github.com/runxhq/runx/pull/112",
"digest": "7547a9783368d97b03be077d5ba822b3a817a6394634756091e9507f47775882",
"profile_digest": "6230275c1ddf69851b58e1028f002ff9bf83ffc61c9311a948889f857ddc83e6",
"trust_tier": "community"
},
"checks": {
"runx_cli": {
"status": "passed",
"command": "runx --version",
"output": "runx-cli 0.6.13"
},
"local_tests": {
"status": "passed",
"command": "node --test skills/verifiable-web-research/test.mjs",
"tests": 4
},
"local_harness": {
"status": "passed",
"command": "runx harness ./skills/verifiable-web-research --receipt-dir .runx-delivery-receipts --json",
"case_count": 2,
"case_names": [
"verifiable-web-research-audit-ready-packet",
"verifiable-web-research-missing-fixture-fails"
],
"receipt_ids": [
"sha256:e3d0d862edde1959bf7e14540554f9d8359f8b1b49eb500c02eae5601979333e",
"sha256:c31b620b06736a5bb7507c8353619612618a440d5c41737d2780ccc1ff20f626"
]
},
"receipt_verification": {
"status": "passed",
"verify_command": "runx verify --receipt-dir .runx-delivery-receipts sha256:e3d0d862edde1959bf7e14540554f9d8359f8b1b49eb500c02eae5601979333e --json",
"verify_kid": "local-harness-key",
"verify_public_key_base64": "IVL40Zt5HSRFMkLhXy6rbLfP+ntqXtMAl5YOBpiB2xI=",
"valid_receipts": [
"sha256:e3d0d862edde1959bf7e14540554f9d8359f8b1b49eb500c02eae5601979333e",
"sha256:c31b620b06736a5bb7507c8353619612618a440d5c41737d2780ccc1ff20f626"
]
},
"registry_publish": {
"status": "published",
"command": "runx registry publish ./skills/verifiable-web-research/SKILL.md --registry https://api.runx.ai --json",
"install_command": "runx add jienigoto/verifiable-web-research@sha-f0d326500848 --registry https://api.runx.ai",
"run_command": "runx skill jienigoto/verifiable-web-research@sha-f0d326500848 --registry https://api.runx.ai"
},
"registry_readback": {
"status": "passed",
"command": "runx registry read jienigoto/verifiable-web-research@sha-f0d326500848 --registry https://api.runx.ai --json",
"runner_names": [
"default"
],
"publisher_handle": "jienigoto",
"markdown_contract_checked": true
},
"registry_install": {
"status": "installed",
"command": "runx add jienigoto/verifiable-web-research@sha-f0d326500848 --registry https://api.runx.ai --json"
},
"registry_dogfood_run": {
"status": "sealed",
"command": "runx skill jienigoto/verifiable-web-research@sha-f0d326500848 --registry https://api.runx.ai -i objective=<text> -i source_fixture_path=builtin:ai-agent-frameworks -i verification_level=audit_ready -i max_claims=2 --json",
"receipt_id_observed": "sha256:939eafcfcb9215633445658edc5e2b97b2a82211fcb4cc3e4955b7e942c5c83e",
"note": "The registry skill was admitted and sealed. The durable receipt proof for this delivery is the local harness receipt set above, because the current Windows/WSL receipt directory boundary did not retain the dogfood receipt file."
},
"public_urls": {
"status": "passed",
"checked_urls": [
"https://runx.ai/x/jienigoto/verifiable-web-research",
"https://runx.ai/x/jienigoto/verifiable-web-research@sha-f0d326500848"
],
"http_status": 200
}
},
"observations": [
"runx --version returned runx-cli 0.6.13, satisfying the stated CLI floor for the bounty.",
"The published registry ref is jienigoto/verifiable-web-research@sha-f0d326500848 under the authenticated jienigoto namespace.",
"runx registry read jienigoto/verifiable-web-research@sha-f0d326500848 --registry https://api.runx.ai --json resolves the package metadata, digest, profile digest, publisher, trust tier, and default runner.",
"runx add jienigoto/verifiable-web-research@sha-f0d326500848 --registry https://api.runx.ai --json installed the package from the remote registry.",
"The local harness passed both declared cases: an audit-ready packet from builtin:ai-agent-frameworks and a missing-fixture failure case.",
"runx verify returned valid=true for the harness receipt sha256:e3d0d862edde1959bf7e14540554f9d8359f8b1b49eb500c02eae5601979333e with kid local-harness-key and public key IVL40Zt5HSRFMkLhXy6rbLfP+ntqXtMAl5YOBpiB2xI=.",
"runx verify returned valid=true for the harness receipt sha256:c31b620b06736a5bb7507c8353619612618a440d5c41737d2780ccc1ff20f626 with the same public key.",
"The skill runner never reaches the network; it converts captured public source snapshots into auditable claims, extracts, digests, and replay instructions.",
"The versioned public adoption page https://runx.ai/x/jienigoto/verifiable-web-research@sha-f0d326500848 returned HTTP 200."
],
"artifact_refs": {
"public_url": "https://runx.ai/x/jienigoto/verifiable-web-research@sha-f0d326500848",
"source_url": "https://github.com/jienigoto/runx/tree/fd8ab4418d8bb9420b76d5d02178bcf53868755e/skills/verifiable-web-research",
"pr_url": "https://github.com/runxhq/runx/pull/112",
"receipt_ref": "runx:receipt:sha256:e3d0d862edde1959bf7e14540554f9d8359f8b1b49eb500c02eae5601979333e"
}
}
56 changes: 56 additions & 0 deletions skills/verifiable-web-research/references/report.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
# Verifiable Web Research Delivery Report

## Package

- Registry ref: `jienigoto/verifiable-web-research@sha-f0d326500848`
- Public URL: https://runx.ai/x/jienigoto/verifiable-web-research@sha-f0d326500848
- Source PR: https://github.com/runxhq/runx/pull/112
- Package digest: `7547a9783368d97b03be077d5ba822b3a817a6394634756091e9507f47775882`
- Profile digest: `6230275c1ddf69851b58e1028f002ff9bf83ffc61c9311a948889f857ddc83e6`

## What the Skill Does

`verifiable-web-research` builds an auditable research packet from captured
source snapshots. It emits claim text, source URLs, final URLs, access times,
content digests, exact extracts, confidence reasoning, and replay steps. The
runner does not reach the network, so harness runs are deterministic and safe.

## Verification Summary

- `runx --version` returned `runx-cli 0.6.13`.
- `node --test skills/verifiable-web-research/test.mjs` passed 4 tests.
- `runx harness ./skills/verifiable-web-research --receipt-dir .runx-delivery-receipts --json` passed 2 declared cases.
- `runx registry publish ./skills/verifiable-web-research/SKILL.md --registry https://api.runx.ai --json` published the package.
- `runx registry read jienigoto/verifiable-web-research@sha-f0d326500848 --registry https://api.runx.ai --json` resolved the registry package.
- `runx add jienigoto/verifiable-web-research@sha-f0d326500848 --registry https://api.runx.ai --json` installed the package from the remote registry.
- The versioned public page returned HTTP 200.

## Receipt Proof

Primary receipt:

`runx:receipt:sha256:e3d0d862edde1959bf7e14540554f9d8359f8b1b49eb500c02eae5601979333e`

Verification command:

```bash
RUNX_RECEIPT_VERIFY_KID=local-harness-key \
RUNX_RECEIPT_VERIFY_ED25519_PUBLIC_KEY_BASE64=IVL40Zt5HSRFMkLhXy6rbLfP+ntqXtMAl5YOBpiB2xI= \
runx verify --receipt-dir .runx-delivery-receipts \
sha256:e3d0d862edde1959bf7e14540554f9d8359f8b1b49eb500c02eae5601979333e \
--json
```

Result: `valid: true`.

The negative-path harness receipt
`sha256:c31b620b06736a5bb7507c8353619612618a440d5c41737d2780ccc1ff20f626`
also verifies with `valid: true`.

## Notes

The registry skill dogfood run was admitted and sealed under
`jienigoto/verifiable-web-research@sha-f0d326500848`. On this Windows/WSL
machine, the registry dogfood receipt file did not remain readable across the
receipt directory boundary, so the durable receipt proof submitted here is the
harness receipt set.
Loading