Repository navigation
feat: Group 3 — AI-native CDC (MCP, vector sink, enrichment, SDKs) - #48
Merged
Merged
Conversation
Sign webhook FlushBatch requests with AWS SigV4 after the final body bytes are assembled. Mutually exclusive with bearer/basic and HMAC.
…S ripple Add json.RawMessage AIContext to ChangeEvent so enrichment can attach opaque AI metadata without locking a Go interior schema; ripple fixtures, OpenAPI descriptions, and @kaptanto/events types/validators accordingly.
…C-01)
Foundation for the vector sink (1/4): Sinks.Vector config with strict
${VAR} secrets, columns/template extraction, and best-effort hash cache.
Ship packages/kaptanto-python (PyPI name kaptanto) with ChangeEvent/AIContext models, reconnecting SSE client, shared NDJSON fixtures via make fixtures, and CI drift + pytest gates. Closes #86.
Introduce internal/mcp with streamable-HTTP listener (official go-sdk v1.6.1), API-key auth with constant-time compare, per-key ACL/redaction, and size-rotated NDJSON audit. MCP disabled remains zero-cost (MCP-04).
Add serverless ACT-01 presets with SigV4 sync/async Lambda Function URL invocation, Cloudflare Worker and Vercel HTTPS presets, golden httptest tests, and cold-start guidance docs.
Plain net/http clients for OpenAI-compatible /v1/embeddings and Cohere v2/embed, with golden httptest coverage, VEC-02 count-mismatch guards, and empty api-key auth omission for local endpoints.
Wire Kaptanto SSE into Mastra workflows via kaptantoTrigger (createRun →
start({ inputData })), add toAgentContext for agent prompts, and publish
on mastra-v* tags. Includes compose example for public.orders.
Implement VEC-03 stable IDs and the three store backends behind one interface so the vector sink can upsert/delete without provider-specific callers; freeze Pinecone/Qdrant HTTP shapes in golden httptest tests.
…/g3-13-vector-consumer # Conflicts: # internal/output/vector/config.go
…trix Optional kaptanto[langchain] with lazy as_tool StructuredTool, OIDC PyPI publish on python-v* tags (tests-gated), and Python 3.10–3.13 CI. Closes #87
Optional enrichment: between parse and EventLog.Append wraps the log so matching events POST to an HTTP sidecar; timeouts/errors fail open with metrics and never block checkpoint durability.
Register schema introspection tools with per-key ACL filtering and audit lines so agents only see authorized tables and redacted column markers.
…d output wiring Wire Deliver/FlushBatch for the vector sink: hash-skip (VEC-01), consecutive upsert/delete runs (VEC-02), post-ack hashcache, 429-never-DLQ, and 4xx one-by-one isolation that reuses already-embedded vectors. Add --output vector factory wiring and package metrics. Closes #85
Provide a compose-backed reference enricher that implements the fail-open HTTP contract so matching ticket inserts get ai_context entities and intent, with 204 when no text is extractable.
…tools Implement MCP-02/04 subscribe_to_changes, unsubscribe, list_subscriptions, and get_recent_events with non-blocking ring consumers, 100ms coalesced resources/updated nudges, session-close cleanup, and router Unregister.
…pipeline wiring Bounded ULID→event index fed by an always-on internal consumer (only when MCP is enabled with API keys), get_event_by_id with ACL/redaction, throughput regression + benches, and startup/shutdown wiring that clears all MCP consumers.
…o fix/g3-17-ai-examples
…fix/g3-19-edge-cases # Conflicts: # internal/observability/metrics.go
… fix/g3-19-edge-cases
…to fix/g3-19-edge-cases
…fix/g3-17-ai-examples # Conflicts: # internal/observability/metrics.go
… fix/g3-17-ai-examples
…-edge-cases # Conflicts: # internal/config/config.go # internal/observability/metrics.go
…amples Ship AI-native demos for MCP agents, zero-cloud pgvector RAG, LangChain SSE reactivity, and async Lambda Function URL invocation; update examples index.
Resolve config.go and metrics.go conflicts by keeping MCP/vector structs alongside EnrichmentConfig and EnrichmentFailuresTotal.
Close gaps for TOAST extract, ai_context ripple across webhook/actions/DLQ/MCP, SigV4 provider-chain naming, Python mid-event reconnect, and MCP-04 CI floor.
Document MCP, vector/RAG, ai_context enrichment, serverless, and SDKs across README, CLAUDE.md invariants/packages, and Qwik landing docs.
…at/group3-ai-native
Remove hardcoded mcp-demo-key defaults from the MCP agent example and docs; require MCP_API_KEY at compose startup. Cloudflare Worker actions now reject unauthenticated POSTs unless allow-unauthenticated: true is set explicitly. Introduced-by: #48
…on, e2e Add AIC-02 exact 16KiB boundary, MCP-04 concurrent race coverage and 0.72 floor, example YAML smoke for MCP/enrich/vector, enrich/MCP integration tests, Group 3 e2e config validation, TS fixture drift CI, SDK ai_context SSE tests, vector in integration -race, and report-only gremlins targets for new packages. Introduced-by: #48
Trigger landing workflow on pull_request paths, add vitest needles for AIC/MCP/VEC/ACT codes, normalize Prettier formatting, and harden install URL smoke tests for multiline JSX. Introduced-by: #48
Introduced-by: #48
CI runners report MCP-04 median ratios ~0.65–0.70 under parallel load; 0.72 was too tight. Keep 0.60 as a coarse guard; precise regression coverage stays in BenchmarkMCP04_Deliver_*. Also run Prettier on test/install-command.test.ts for fmt.check. Introduced-by: #48
olucasandrade
added a commit
that referenced
this pull request
Aug 10, 2026
Allow a probe request after the open circuit cools down and exclude parent context cancellation from timeout streak counting. Introduced-by: #48
olucasandrade
added a commit
that referenced
this pull request
Aug 10, 2026
Resolve hostnames once and dial the checked IP directly so DNS rebinding cannot bypass private/metadata blocklists. Introduced-by: #48
olucasandrade
added a commit
that referenced
this pull request
Aug 10, 2026
Replace linear pending slice scans with a per-partition ID set when VEC-01 hash-skip checks in-flight buffers. Introduced-by: #48
olucasandrade
added a commit
that referenced
this pull request
Aug 19, 2026
Actions-only none still served /metrics, /healthz, and /openapi.json without a bearer token. Treat none like other network outputs unless --insecure is set. Introduced-by: #48
olucasandrade
added a commit
that referenced
this pull request
Aug 19, 2026
Add webhook/vector to Compatibility, drop implied ES/Redis sinks, fix cursor/CHK-01 copy, and mark --ha as Postgres-only. Introduced-by: #48
olucasandrade
added a commit
that referenced
this pull request
Aug 19, 2026
Mongo capture is Change Streams, Docker needs --tables, --ha is Postgres-only, Connect Postgres now documents REPLICATION/SELECT, and the rust 30-40% CPU claim is replaced with the published bench result. Introduced-by: #48
olucasandrade
added a commit
that referenced
this pull request
Aug 19, 2026
Crawlers were still sent to kaptanto.dev with a stale slug list, and the primary meta description omitted webhook, vector, MCP, and actions. Introduced-by: #48
olucasandrade
added a commit
that referenced
this pull request
Aug 21, 2026
Hash-skip now treats cache as miss when the same vector ID is already buffered (VEC-01). Flush embed cache keys by id+hash so consecutive same-ID upserts with different text each embed (VEC-02). Delete poison errors isolate per-seq like upserts instead of DLQing the first seq. Introduced-by: #48
olucasandrade
added a commit
that referenced
this pull request
Aug 21, 2026
Harden MCP tool paths against subscription/event existence oracles, strip AIContext under column redaction, require explicit tables ACL, and gate output:none observability when MCP is enabled. Introduced-by: #48
olucasandrade
added a commit
that referenced
this pull request
Aug 21, 2026
…aker
Enforce strict ${VAR} auth-token refs, block private/metadata destinations
with optional allowlist, enable HTTP keep-alives, fast-fail after consecutive
timeouts, and propagate shutdown context through enrich.Wrap.
Introduced-by: #48
olucasandrade
added a commit
that referenced
this pull request
Aug 21, 2026
Remove synthetic SHA EventLog pad, assert paired-trial median ratios with consistent logging, and store event pointers in the recent index and ring buffers to cut Deliver copies. Introduced-by: #48
olucasandrade
added a commit
that referenced
this pull request
Aug 21, 2026
Add in-process LRU in front of SQLite for Unchanged checks on Deliver, batch Put/Del in a single transaction per FlushBatch ack, and reduce hot-path allocations via pooled After maps and stack-backed HashText. Introduced-by: #48
olucasandrade
added a commit
that referenced
this pull request
Aug 21, 2026
Align config/API pages with current output modes, add MCP and vector auth callouts, label unpublished SDK installs, add a TypeScript SDK page, and cross-link the vector sink vs vector-upsert action. Introduced-by: #48
olucasandrade
added a commit
that referenced
this pull request
Aug 21, 2026
Remove hardcoded mcp-demo-key defaults from the MCP agent example and docs; require MCP_API_KEY at compose startup. Cloudflare Worker actions now reject unauthenticated POSTs unless allow-unauthenticated: true is set explicitly. Introduced-by: #48
olucasandrade
added a commit
that referenced
this pull request
Aug 21, 2026
…on, e2e Add AIC-02 exact 16KiB boundary, MCP-04 concurrent race coverage and 0.72 floor, example YAML smoke for MCP/enrich/vector, enrich/MCP integration tests, Group 3 e2e config validation, TS fixture drift CI, SDK ai_context SSE tests, vector in integration -race, and report-only gremlins targets for new packages. Introduced-by: #48
olucasandrade
added a commit
that referenced
this pull request
Aug 21, 2026
Trigger landing workflow on pull_request paths, add vitest needles for AIC/MCP/VEC/ACT codes, normalize Prettier formatting, and harden install URL smoke tests for multiline JSX. Introduced-by: #48
olucasandrade
added a commit
that referenced
this pull request
Aug 21, 2026
CI runners report MCP-04 median ratios ~0.65–0.70 under parallel load; 0.72 was too tight. Keep 0.60 as a coarse guard; precise regression coverage stays in BenchmarkMCP04_Deliver_*. Also run Prettier on test/install-command.test.ts for fmt.check. Introduced-by: #48
olucasandrade
added a commit
that referenced
this pull request
Aug 21, 2026
feat: Group 3 — AI-native CDC (MCP, vector sink, enrichment, SDKs)
olucasandrade
added a commit
that referenced
this pull request
Aug 21, 2026
Allow a probe request after the open circuit cools down and exclude parent context cancellation from timeout streak counting. Introduced-by: #48
olucasandrade
added a commit
that referenced
this pull request
Aug 21, 2026
Resolve hostnames once and dial the checked IP directly so DNS rebinding cannot bypass private/metadata blocklists. Introduced-by: #48
olucasandrade
added a commit
that referenced
this pull request
Aug 21, 2026
Replace linear pending slice scans with a per-partition ID set when VEC-01 hash-skip checks in-flight buffers. Introduced-by: #48
olucasandrade
added a commit
that referenced
this pull request
Sep 24, 2026
Docs claimed inbound TLS was SSE/gRPC-only while sinks, webhook, vector, and output:none also gate on requireServerTLS; examples now match the binary and clarify none+MCP auth. Introduced-by: #48
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Consolidates all 19 Group 3 (AI-native) issues into a single PR on public
kaptanto. This is the full AI-native milestone: MCP server, vector sink, fail-open enrichment, Python/TypeScript SDKs, Mastra adapter, serverless actions, SigV4 webhooks, examples, docs, and edge-case verification.Core Go features
ai_contextfield onChangeEventwith OpenAPI/TS fixture ripple (G3-01)SDKs & integrations
@kaptanto/mastraadapter +examples/mastra(G3-16)kaptanto-pythoncore SDK + langchain extra + PyPI CI (G3-14, G3-15)Docs & polish
Merge strategy
All 19 mirror branches (
fix/g3-*) merged in dependency order ontomain. Merge conflicts in shared files (config.go,metrics.go,examples/README.md) resolved to retain the union of all features.Validation
Local validation is the gate — mirror CI (#147–#165) is blocked by GitHub Actions billing quota, not test failures.
Mirror PRs superseded
This PR replaces mirror PRs #147–#165 on
kaptanto-private-mirror. After merge, sync mirror and close those PRs + issues #73–#91.Test plan
make lint && make test && make verify-no-cgocd packages/kaptanto-events && npm testcd packages/kaptanto-mastra && npm testcd packages/kaptanto-python && uv run pytestcd landing && npm run lint && npm run build.types && npm test--mcp.enabled, vector sink config validates, enricher fail-open on bad HTTP response