Repository navigation
Generic hosted sources and the Linear source - #679
alex-clickhouse wants to merge 10 commits into
Conversation
A generic hosted source sends each record's data as a JSON object that the provider defines, and its subscription carries allow/deny rules on JSON Pointer paths into that data. The channel gateway applies the rules before it stores a record, and the inbox guardrail must apply the same rules again, so both sides drop the same records. nerve/sources/source_json.py holds what the channel contract and the guardrail share: the checks of a source JSON object, its canonical form (RFC 8785), and JSON Pointers. DataRule in filters.py evaluates a rule on a record's metadata with the contract's value semantics, and the contract's rule vectors pin them. The runner's drop log names the path and the value of the rule that dropped a record. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Every provider outside the closed vocabularies (github, github_repos, email) is a generic source. Its records carry a schema and data, a JSON object that the provider defines, in place of metadata, and a subscription to it carries provider settings, the record schemas that Nerve reads, and rules on the data. Nerve must decode such records and send such subscriptions exactly as the gateway checks them, or the gateway closes the stream. The mirror keeps the data and the settings as parsed JSON with every member, refuses what the gateway refuses (repeated member names, NaN, lone surrogates, numbers outside binary64, more than 8 levels, data or settings above their canonical byte bounds), checks record types and schemas, and treats a subscription as generic only when it names a schema. A member with its zero value counts as absent, except settings and data. Conversation kind item names a tracked item. Byte-exact copies of the contract's synthetic vectors pin the canonical form, the bounds, the record and subscription verdicts, the generic subscriptions frame, and the canonical data in the digest projection. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The channel gateway polls Linear for each agent with the agent's Linear connector in the MCP catalog (its credential, project scope and GitHub filter) and streams the records as a generic source. Nerve needs a setting that subscribes to them and a source that drains them into the inbox. sync.linear has enabled, mode (hosted only; poll refuses to load with a message that it is not available), events, inbox, include_bots, drop_own_actions, rules, schedule, condense and batch_size. While it is on, Nerve subscribes to provider linear with the settings that differ from the provider's defaults, the schema linear.issue_event/1, and the rules. HostedSource drains the buffer of any generic provider and keeps the record data as the metadata, with every member; HostedGitHubSource builds on it and keeps its own records. The registry runs the Linear drain with the rules as the guardrail, and nerve doctor reports the source. Rules that the gateway would refuse refuse to load. Docs: docs/sources.md "Linear (hosted)" and "Rules of generic hosted sources", docs/config.md (keys and the restart row), the example config, and a line in the TOOLS.md templates. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A Linear record stores its data (nested JSON) as the record's metadata. The header chips of the detail pane show such values as raw text, and nothing showed the issue at a glance. LinearRenderer shows an issue card (identifier, title, event, state, actor, and a link to the comment or the issue, http(s) only) above the content, and the record data behind a toggle. Any other source whose metadata holds nested JSON, the data of a generic hosted source, gets the content and the record data as indented JSON. The header leaves out its chips for both. The card's structure is based on the Linear renderer by Frank Rosner, adapted to the record data of the hosted gateway. Co-authored-by: Frank Rosner <3427394+FRosner@users.noreply.github.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The contract's rule vectors do not tell an index with a leading zero from the element it would name, so a direct test pins that "01" and "-" name nothing. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The config tests require every list setting to hold scalars or dataclasses, and a typed rule (SourceRuleConfig: path, allow, deny) also reads better at the two places that use it: the subscription and the guardrail. The guardrail builds its DataRule objects from it directly. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The poll tools wrote each metadata value with str(), so the data of a Linear record, which names the MCP catalog server that acts on it, reached the agent as Python dict text. Nested values (an object, or a list that holds an object or a list) are now compact JSON; flat values stay as they were. docs/cron.md names Linear among the hosted sources that run on delivery. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review round 1 of the generic hosted sources:
- A subscription with rules is never sent without them. When the
subscriptions frame does not fit, it goes first without the GitHub
filter patterns, which the guardrail still applies, and then without
the subscriptions that carry rules: the gateway stores nothing for
them, the stream logs an error that names the rules to shorten, and
nerve doctor reports it. Before, the fallback dropped the rules, and
the gateway stored records that the user had denied.
- The runner's drop log names only the record ID, the rule's field and
the list that fired, never a value or a summary of a dropped record.
- A stored generic record keeps its record schema next to its data
(metadata {"schema", "data"}); the rules read the data, and the poll
tools show the schema and every member of the data as JSON.
- The Linear fixtures follow the gateway's Linear provider: a byte-exact
copy of the schema file of linear.issue_event/1 validates every one,
the server ID is the catalog server ID, the connection ID is derived
from the tenant and the server, and the record ID stays the same for
every version of an event, so a changed event replaces the stored one.
- A provider without schemas may name repositories within the bounds,
as the contract permits; github, email and generic sources may not.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A record of a generic hosted source stores its schema and its data. The view came from the source name and from the shape of the data, so flat data of a generic source showed as chips and two major versions could not be told apart. The Linear card shows records of linear.issue_event/1, the record data view every other schema, flat data included, and the toggle names the schema. A change without an actor shows its kind. The tests render record data from a fixture file that the backend tests validate against the schema of linear.issue_event/1. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The gateway's Linear provider reads only the mentions of the credential's Linear user from its Linear inbox (via inbox only with mentioned); assignments, comments and state changes come from the issues of the scope, and an item too large to read gives no record. The docs, the config comments and the fixtures say the same, and a fixture test pins that via inbox goes only with mentioned. The schema copy is unchanged. The lines that this branch adds use no em-dashes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Nerve PR review · ✅ Ready to merge This PR adds the generic side of the hosted channel contract: records carry provider-defined JSON Completeness: tests cover the contract vectors, Linear end to end over the fake gateway, the frame-limit boundary and doctor. Also covered: docs, example config, the restart-table row and the TOOLS.md templates. No migration is needed. Worth considering:
Checked: full suite in an offline sandbox at |
What
github,github_reposandemailis generic: its records carry aschemaanddata, aJSON object that the provider defines, and a subscription to it carries
settings, therecord
schemasthat Nerve reads, andruleson the data. Nerve keeps every member of thedata, also the members that it does not know. Conversation kind
itemnames a tracked item.32 KiB of data and 8 KiB of settings in canonical form), the canonical form (RFC 8785) and
JSON Pointers.
DataRulein the inbox guardrail applies the rules of a generic source to the record datawith the same semantics as the channel gateway: a deny rule drops a record when a value at its
path matches, an allow rule drops a record without a matching value.
HostedSourcedrains the hosted buffer of any generic provider; the record's metadata holdsits schema and its data. The hosted GitHub source keeps its own records.
sync.linear:enabled,mode(hostedonly;pollrefuses to load),events,inbox,include_bots,drop_own_actions,rules,schedule,condense,batch_size. While it ison, Nerve subscribes to provider
linearwith the settings that differ from the defaults, theschema
linear.issue_event/1, and the rules, and thesource:linearrunner drains therecords with the rules as guardrail.
nerve doctorreports the source.so the agent reads the MCP catalog server that a Linear record names.
or a summary of the dropped record, for every source.
state, actor, link) for
linear.issue_event/1, and a JSON view of the record data for anyother schema.
the example config, and the TOOLS.md templates.
Why
The channel gateway can poll Linear for an agent with the agent's Linear connector (its
credential, project scope and GitHub filter) and deliver the records while Nerve is stopped. The
contract for such a source is generic: a strict envelope and provider-owned JSON data with a
versioned schema, so that a new provider or record type needs no contract change in Nerve.
How
dataandsettingsas parsed JSON, tells an absent member fromnull, andrefuses what the gateway refuses: repeated member names,
NaN, lone surrogates, numbersoutside binary64, too deep or too large objects. A member with its zero value counts as absent,
except
settingsanddata.subscription that it builds. The frames of GitHub, the repository watch and email do not
change.
rules: if the subscriptions frame is too large, it goes first without the GitHub filter
patterns, then without the subscriptions that carry rules. The gateway then stores nothing
for them, the stream logs an error, and
nerve doctorreports it.startup); switching it off follows a reload.
Rollout: the gateway must know generic sources before an agent switches Linear on; an older
gateway closes the stream on
settings,schemasorrules. With the default config Nervesends the same frames as before.
The Linear card is based on a Linear renderer by Frank Rosner.
Tests
and subscription verdicts, rule evaluation, the generic subscriptions frame, and the canonical
data in the digest projection.
linear.issue_event/1validates every Linearfixture, backend and frontend; the fixtures follow the gateway's ID rules, and a changed event
replaces the stored copy.
to end over real streams with the fake gateway, the agent's view, and
nerve doctor.🤖 Generated with Claude Code