Add platform runtime and Node middleware

[gharibyan]

Summary

• Add @agentrail/runtime as the platform-neutral request runtime for AI-bot classification, Markdown serving, stale handling, and background warmup.
• Refactor @agentrail/worker into a Cloudflare adapter that delegates request handling to the shared runtime while keeping scheduled and queue crawling support.
• Add @agentrail/node with dependency-free Express/Nest-style middleware helpers and a file-backed resource store for local/self-hosted trials.
• Recognize versioned Claude user agents such as Mozilla/5.0 (compatible; Claude/1.0) so real Claude fetch paths can receive AgentRail responses.
• Improve Markdown extraction for richer agent-facing output by preserving useful JSON-LD summaries and formatting block-level feature cards as readable headings, descriptions, and links.
• Add generated Cloudflare KV maintenance commands for dropping one cached resource or clearing AgentRail page resources from remote AGENTRAIL_RESOURCES.
• Update docs for Node middleware, non-Cloudflare deployment options, Cloudflare KV cleanup, bot behavior, and operational testing.

Validation

• npm test
• npm run typecheck
• git diff --check
• Generated scripts/agentrail-kv.mjs syntax check with node --check
• Generated kv:clear helper refusal path verified without --yes
• Staged secret/local-path scan returned no matches

Notes

The Node adapter does not depend on Express or Nest packages. It uses structural request/response types, so users can mount it with app.use(...) in Express or Nest without framework-specific heavy lifting.

For deployed Cloudflare projects, stale Markdown can now be refreshed by deleting the cached key, for example:

npx wrangler kv key delete "page:https://www.finalbitai.com/features" --binding AGENTRAIL_RESOURCES --remote

After the key is deleted, the next AI-agent request falls back to origin and schedules background warmup; a later request receives regenerated Markdown.