Repository navigation
feat(meetings): server-side project detection + detection review UX - #354
Merged
Merged
Conversation
added 3 commits
September 24, 2026 23:00
…vector search)
Mirrors the person-embedding pipeline so the Sonic `suggest_project` tool runs
server-side and tenant-safe instead of matching projects in the browser.
Backend:
- Projects schema: summaryEmbedding (json) + summaryEmbeddingSource/UpdatedAt +
sparse summaryEmbeddingPending GSI (listSummaryEmbeddingPending).
- generate-project-embedding Lambda + shared run-project-embedding core: embeds
"<name> — <first summary section>" (Titan v2, 1024d), native DDB write,
source-diffing. Armed via a one-time EventBridge schedule (~2 min) off the
Projects stream when name/summary changes (schedule-handler + scheduler).
- ProjectSummaryEmbeddingIndex via CDK custom resource (setupProjectVectorIndex,
self-healing). Added a redeployNonce to both project AND person index custom
resources so CFN always sends an Update event and the idempotent ensure-index
handler actually runs (it previously could be skipped, leaving the index
uncreated).
- project-vector-search Lambda + suggestProjectByVoice query (owner post-filter,
returns {projectId, name, summarySnippet, score}).
- Project-embedding backfill: enqueue + worker Lambdas + SQS, and
scripts/mark-projects-embedding-pending.js.
Client:
- suggest_project tool now takes {query, name?} (no projectId); handleToolUse
calls searchProjects -> suggestProjectByVoice and the server resolves the
projectId. suggest_project is explicitly anchored in the system prompt.
- useSonicTranscription: searchProjects wired in; SuggestedProject carries
name/score/confirmed.
Note: not yet deployed — sandbox asset publish is currently blocked by transient
S3 socket timeouts from this environment (synth + type-checks pass).
…newal, detection filtering UI (detection review): - Persistent detection pills on the meeting page for BOTH people and projects (stay visible for the whole recording); the header keeps the discreet marker. - Confirmation gate after stop, before the summary is written: the user confirms/rejects each detected person and suggested project; only confirmed ones feed the summary (stop() opens the gate, confirmDetectionsAndSummarize() runs it). - Person pill: choosing a match auto-confirms the mention (no extra click); "Neue Person anlegen" dialog (name + company + role) creates the Person, resolves/creates the Account, links a PersonAccount, and assigns+confirms. - Engine: pendingConfirmation gate, rejectDetectedPerson, assignCreatedPerson, toggleConfirmProject; SuggestedProject gains confirmed. Sonic session renewal (fixes recording cutting out ~8 min): - Nova 2 Sonic has a hard 8-minute connection limit. SonicClient now renews the stream at ~7 min: opens a new session, switches audio over, gracefully ends the old one — seamless. De-dupe sets + cumulative usage live on the client so nothing is re-emitted and token totals carry across the swap. Tool results go back on the originating session. Over-detection fixes: - onToolUse now ignores suggest_project tool-uses (they were wrongly creating phantom detected people) and suppresses names that match a current meeting participant (already in the meeting = noise). - System prompt tightened: report a person only when a concrete, relevant named person is actually discussed; never the participants, casual mentions, or greetings.
The Provider + CustomResource(onEvent) wiring never actually invoked the ensure-index handlers in this Amplify setup — the provider framework ran but didn't call through, so the DynamoDB vector index depended on a manual invoke (the person index in prod). Replace it with an AwsCustomResource that performs a direct lambda:Invoke of the ensure handler on every deploy (per-deploy physical id → onUpdate always fires). Verified: both PersonNameEmbeddingIndex and ProjectSummaryEmbeddingIndex are now created automatically and report ACTIVE; the handler logs "already exists; no-op" on subsequent deploys. Handlers accept a direct invoke (RequestType optional) in addition to the CFN custom-resource event shape. Note: `aws dynamodb describe-table --query Table.VectorIndexes` returns null from an older local CLI service model even when the index exists — verify with a newer SDK or the handler logs, not the CLI.
cabcookie
pushed a commit
that referenced
this pull request
Sep 24, 2026
…ild) The prod deploy of #354 crashed in the frontend build's prebuild step (tsc --noEmit) with 'JavaScript heap out of memory' (exit 134): the full-project typecheck plus next build exceed Node's default ~2 GB heap. Set NODE_OPTIONS=--max-old-space-size=8192 before npm run build. The backend deploy itself succeeded; only the frontend build failed.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Overview
Server-side, vector-based project detection for live meetings (Block B/C), plus the meeting-page detection UX, Sonic session renewal, and over-detection filtering.
Mirrors the existing person-embedding pipeline. Projects get a summary embedding; during a live meeting Sonic can call a tool that semantically matches spoken context to a project, and detections surface as reviewable pills behind a confirmation gate.
Backend — project embeddings & vector search
project-schema.ts):ProjectsgainssummaryEmbedding(json),summaryEmbeddingSource,summaryEmbeddingUpdatedAt,summaryEmbeddingPending+ a sparse GSIlistSummaryEmbeddingPendingfor backfill.project-summary/):run-project-embedding.tsbuilds the source as"<name> — <first summary section>";project-embedding-handler.tswrites the embedding;schedule-handler.tsenqueues on project / projectSummary change (debounced via scheduler,PROJECT_EMBEDDING_DELAY_MINUTES=2).backfill-project-embedding-{enqueue,worker}-handler.ts,scripts/mark-projects-embedding-pending.js): mark-pending + enqueue/worker to embed existing projects.project-vector-index/): ensures a DynamoDB vector index (ProjectSummaryEmbeddingIndex). Triggered on deploy viaAwsCustomResourcedoing a directlambda:Invoke— CDK-native, reliable (the previousProvider/CustomResourcenever invoked the handler).project-vector-search/): owner-scopedSearchVectorsquery with post-filter, returns{ projectId, name, summarySnippet, score }.ai-schema.ts):ProjectVoiceMatchtype +suggestProjectByVoicequery.Client — detection UX
helpers/sonic/client.ts):suggest_projecttool now takes{ query, name? }and calls the server search;KNOWN_PROJECT_THRESHOLD=0.5.report_detected_person, filter out current meeting participants, tightened system prompt.meeting-detection-pills.tsx,meeting-confirmation-gate.tsx,create-detected-person-dialog.tsx(creates Person + find-or-create Account + PersonAccount). Removes the oldmeeting-suggested-projects-bar.tsx.Testing
npm run lint— clean (0 errors).npm test— 26/26 pass.Notes
mark-projects-embedding-pending.js -env prod --commit, then invoke the enqueue lambda).