Repository navigation
feat(people): on-demand people data layer with recency + semantic search - #351
Merged
Merged
Conversation
Replaces "load all people into memory" (Person.list limit 4000) — which made a referenced person invisible if it wasn't in the loaded set (e.g. a meeting's participants showing no names) — with an incremental, cache-and-keep store. Data layer: - Person.lastSeen (datetime) + sparse GSI listPeopleByLastSeen (owner PK, lastSeen SK). Initial set is now the 50 most recently seen people. - touch-person-last-seen Lambda: off the MeetingParticipant / NoteBlockPerson DynamoDB streams (INSERT), stamps Person.lastSeen via a forward-only, owner-preserving UpdateItem. No debounce. CDK-wired in custom/backend. - searchPeople query: semantic (vector) search for typed lookups, sharing the existing person-vector-search Lambda (owner enforced server-side). Store (api/usePeople.ts): - Module-level cache-and-keep map wrapped by SWR; loads recent-50 initially. - getPersonById / getPeopleByIds fetch missing people on demand by id and merge them in (a referenced person is never permanently invisible). - searchPeopleRemote + queryPerson use the vector search and merge matches. - resolvePeopleByIds for non-render callers (SWR fetchers). Consumers migrated off people.find(): - people-selector + Cmd-K palette: debounced semantic search (ComboBox gains an optional onSearch that disables client-side filtering). - meeting participants, PeopleList, activity-meeting-list, payer person/header, resolveParticipant, useInvolvedPeople, mention editors. Dev scripts: - copy-recent-meetings-to-sandbox.js: copy last N days of PROD meetings (+ all relations incl. NoteBlockPerson) into the sandbox. - backfill-person-last-seen.js: seed Person.lastSeen from existing relations. Note: the lastSeen GSI replaces the Person table on deploy (Amplify recreates the table on a secondary-index change).
cabcookie
added a commit
that referenced
this pull request
Sep 25, 2026
…etch) (#355) Applies the people data-layer principles (PR #351) to projects and fixes the prod list silently showing < 10 projects. Root cause: client.models.Projects.list() returned only the first DynamoDB page (~1 MB) and never followed nextToken; once PR #348 added the heavy projectSummary markdown to the list selectionSet, items grew so large that a page held only a handful of the (1146) projects. Changes: - Split selectionSet into leanSelectionSet (list/active set, drops the projectSummary full text, keeps the tiny projectSummaryUpdatedAt) and fullSelectionSet (on-demand by-id: detail page, Sonic context). The summaryEmbedding vector was never client-selected (native DDB write). - fetchProjects now paginates via nextToken (limit 1000) so the active set is complete again, with much smaller pages. - Module-level project cache (cache-and-keep): loadProjectsByIds / resolveProjectsByIds / searchProjectsRemote / ensureProjectSummaries, with a cache-version SWR key to re-render consumers. getProjectById merges the full summary onto the fresh active object; getProjectNamesByIds loads missing ids on demand so a referenced done/archived project is never permanently invisible. - searchProjects AI query (reuses the existing project-vector-search Lambda / ProjectVoiceMatch) for typed semantic lookups. - useMeetingSonicData: pre-warm full summaries (ensureOpenProjectSummaries) at recording start; getOpenProjects reads them from the cache. - project-details: ensureProjectSummaries on mount to render the summary. Verified: tsc --noEmit clean, eslint clean, next build succeeds. Co-authored-by: Carsten Koch <carstenbkoch@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Builds on #350 (base branch: feat/sonic-meeting-stage2).
Problem
The app loaded ALL people into memory (Person.list, limit 4000) and every
consumer did people.find(id). Anyone not in that set rendered no name — e.g. a
meeting's participants ("Matt", "David") showed blank. It also doesn't scale.
Solution — incremental, cache-and-keep store + semantic search
Backend
lastSeen SK). Initial in-memory set = the 50 most recently seen people.
DynamoDB streams (INSERT): forward-only, owner-preserving UpdateItem stamps
Person.lastSeen. No debounce. CDK-wired.
existing person-vector-search Lambda (owner enforced server-side).
Store (api/usePeople.ts)
them in — a referenced person is never permanently invisible.
SWR fetchers.
Consumers migrated off people.find()
people-selector + Cmd-K palette (debounced semantic search; ComboBox gains an
optional onSearch that disables client-side filtering), meeting participants,
PeopleList, activity-meeting-list, payer person/header, resolveParticipant,
useInvolvedPeople, and all mention editors.
Dev scripts
relations (incl. NoteBlockPerson) into the sandbox.
Deploy note
The lastSeen GSI replaces the Person table on deploy (Amplify recreates the
table on a secondary-index change). Sandbox was redeployed, last-28-days PROD
meetings copied in, and lastSeen backfilled.
Testing
search returns semantic matches.