Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 32 additions & 1 deletion .procoder/ask/decisions.md
Original file line number Diff line number Diff line change
Expand Up @@ -6547,6 +6547,8 @@ tx.Commit()` and the errors returned (ErrSuppressed, ErrNotOpen,
`@before::integer`: the same plan, and the same out-of-range error for
a cursor above 2^31-1 as before. LookupCalls and LookupRecordings already
cast theirs to bigint and still do.
Superseded (2026-11-17): both now cast `@before::bigint`, as answered
under "CDR and recording list cursor type".
- **Run-time statements.** lookupWhere takes the caller's
`momo.Named("before")` and `momo.Named("limit")` and appends one Named per
condition it adds (`@allowed`, `@extension`, `@digits`, `@suffix`,
Expand All @@ -6567,6 +6569,8 @@ The old `($1 = 0 OR id < $1)` typed the paging cursor as integer, so a cursor id
- Keep `::integer` (current behaviour, same plan).
- Switch to `::bigint`, as LookupCalls and LookupRecordings already use: no cursor limit, a behaviour change.

Answer (2026-11-17): switch to `::bigint`.

## momo Phase 3, domain 13: internal/store AI on @name (2026-11-17, momo/store-ai)

- **Scope.** `messaging_ai.go`, `ai_core.go`, `ai_sessions.go` and
Expand Down Expand Up @@ -6639,7 +6643,10 @@ The old `($1 = 0 OR id < $1)` typed the paging cursor as integer, so a cursor id
equal until those files move and the `$n` form goes. `queueRedactions` is
called from `messaging.go` (on `$n`) as well: it binds the batch as
`@batch` when its fragment's arguments are `momo.Named`, else as the next
`$n`, as before.
`$n`, as before. Superseded (2026-11-17, momo/store-followup): with
`messaging_calls.go` and ErasePeer on `@name`, `visible`,
`poolMemberParam`, `TestVisibleToMatchesVisible` and `queueRedactions`'
`$n` branch are gone; `queueRedactions` always binds `@batch`.
- **Run-time statements.** ListConversations' WHERE fragments each carry
their `@name` and their Named; the report's grouping keys bind `@tz`
(the `strings.ReplaceAll` of `$5` is gone); exportMedia binds
Expand Down Expand Up @@ -6706,6 +6713,10 @@ The old `($1 = 0 OR id < $1)` typed the paging cursor as integer, so a cursor id
(`messaging_privacy.go`) wraps it and numbers its batch `$n+1`.
`poolMemberParam` (in `messaging_route.go`) stays `$1::bigint`: only
`visible` uses it.
Superseded (2026-11-17, momo/store-followup): the visibility read binds
`@id` with `visibleTo`, ErasePeer's redaction select binds `@peer_key`,
`messaging_calls.go` is in `namedFiles`, and `visible` and
`poolMemberParam` are deleted.
- **Carrier credentials.** sealChannelCredentials binds the sealed bytes
as `@credentials_enc`; nothing logs argument values (momo's logger never
does, and the tests print only recorded listing and CDR statements).
Expand Down Expand Up @@ -6762,3 +6773,23 @@ passes.
`SELECT pg_advisory_xact_lock(@key)` with `int64(chatEventLock)`, one
bigint bind as before.
- **maintainChat's three deletes** take no arguments and are unchanged.

## momo Phase 3 follow-ups: bigint cursor, messaging cleanup, final $n sweep (2026-11-17, momo/store-followup)

- **Cursor.** ListCDRs and ListRecordings cast `@before::bigint` on both
uses, as answered under "CDR and recording list cursor type": a cursor id
above 2^31-1 now pages instead of failing out of range. LookupCalls and
LookupRecordings were already bigint.
- **Messaging.** ConversationCalls' visibility read binds `@id` and joins
`visibleTo` with `inboxAccessArgs`; `messaging_calls.go` is in
`namedFiles`. `visible` (`messaging_inbox.go`) and `poolMemberParam`
(`messaging_route.go`) had no other user and are deleted, with
`TestVisibleToMatchesVisible`; `poolMemberNamed` and `poolMemberUser` are
the two forms left. ErasePeer's redaction select binds `@peer_key`, so
`queueRedactions` no longer numbers its batch after a `$n` fragment and
always binds `@batch`. `TestConversationCallsSends` now also pins the
visibility read's binds.
- **Sweep.** No non-test file of `internal/store` binds `$n` but
`trylock.go` (`TryLock`'s pinned connection, raw `database/sql`); `dbq.go`
and `backup.go`'s goose read stay raw `database/sql` and bind nothing
positional. Every other file is in `namedFiles`.
58 changes: 20 additions & 38 deletions docs/database.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,45 +25,27 @@ Moved so far:
transaction is momo's and whose hooks still get the `*sql.Tx` behind it.
- The directory sync store (`internal/directory`), the SCIM store
(`internal/scim`) and the SSO store (`internal/sso`).
- `internal/store`, at its seam. Its files move to `@name` binds one
domain at a time (below); the files still on `$n` are those not listed
here. Converted so far:
- `internal/store`, at its seam. Every domain is converted to `@name`
binds, and every file but the raw exceptions below is in `namedFiles`:
- core and configuration: `store.go`, `batch.go`, `config.go`, `dialspace.go`, `dirprovision.go`, `flow_canvas.go`
- faxes, mail and backup: `faxes.go`, `mail.go`, `backup.go`
- users and authentication: `users.go`, `totp.go`, `oauth.go`, `ldap.go`, `onboarding.go`, `sidekey.go`
- phonebook: `phonebook.go`, `phonebook_lookup.go`, `personal_contacts.go`, `locations.go`
- call features: `ivr.go`, `queues.go`, `queue_callbacks.go`, `followme.go`, `pickup.go`, `park.go`, `paging.go`, `ringmap.go`, `ringtag.go`, `conference.go`, `multicast.go`
- voicemail and media: `pbx.go`, `voicemail_groups.go`, `media.go`, `moh.go`, `ringtones.go`, `tts.go`, `recording_links.go`, `video_recording.go`, `cdr.go`
- provisioning: `prov.go`, `prov_admin.go`, `prov_handler.go`, `prov_lines.go`, `prov_push.go`, `prov_overrides.go`, `prov_books.go`, `prov_multicast.go`, `syslog.go`, `rtcpxr.go`, `clientquality.go`, `clients.go`
- routing and time: `routing.go`, `timecond.go`, `blocklist.go`, `originate.go`, `boss.go`, `calendar_dnd.go`
- security and door: `scep.go`, `wifi.go`, `door.go`, `door_open.go`, `door_camera.go`, `door_webhook.go`
- AI: `messaging_ai.go`, `ai_core.go`, `ai_sessions.go`, `ai_findings.go` (`TryLock` moved to `trylock.go`, which stays raw and keeps its `$1`)
- AI: `messaging_ai.go`, `ai_core.go`, `ai_sessions.go`, `ai_findings.go`
- messaging inbox: `messaging_inbox.go`, `messaging_media.go`, `messaging_privacy.go`, `messaging_deskphone.go`, `messaging_reports.go`
- messaging core: `messages.go`, `messaging.go`, `messaging_route.go`, `messaging_route_views.go`, `messaging_carrier.go`, `messaging_events.go`, and `messaging_calls.go` but for its visibility read, which joins the inbox's `visible` condition and binds `$n` until the inbox moves; ErasePeer's select for `queueRedactions` (privacy) binds `$1` as that helper numbers after it
- messaging core: `messages.go`, `messaging.go`, `messaging_route.go`, `messaging_route_views.go`, `messaging_carrier.go`, `messaging_events.go`, `messaging_calls.go`
- webchat: `webchat.go`, `webchat_sites.go`
- `internal/store`, at its seam, with the files of the core and
configuration domain converted to `@name` binds: `store.go`, `batch.go`,
`config.go`, `dialspace.go`, `dirprovision.go` and `flow_canvas.go`, and
of the faxes, mail and backup domain: `faxes.go`, `mail.go` and
`backup.go`. Its other files still bind `$n` and move one domain at a time (below).
`config.go`, `dialspace.go`, `dirprovision.go` and `flow_canvas.go`; and
those of the users and authentication domain: `users.go`, `totp.go`,
`oauth.go`, `ldap.go`, `onboarding.go` and `sidekey.go`. Its other files
still bind `$n` and move one domain at a time (below).
those of the phonebook domain: `phonebook.go`, `phonebook_lookup.go`,
`personal_contacts.go` and `locations.go`; and those of the call
features domain: `ivr.go`, `queues.go`, `queue_callbacks.go`,
`followme.go`, `pickup.go`, `park.go`, `paging.go`, `ringmap.go`,
`ringtag.go`, `conference.go` and `multicast.go`. Its other files still bind
`personal_contacts.go` and `locations.go`; and those of the voicemail and
media domain: `pbx.go`, `voicemail_groups.go`, `media.go`, `moh.go`,
`ringtones.go`, `tts.go`, `recording_links.go`, `video_recording.go` and
`cdr.go`. Its other files still bind
`$n` and move one domain at a time (below).
`personal_contacts.go` and `locations.go`; and those of the
provisioning domain: `prov.go`, `prov_admin.go`, `prov_handler.go`,
`prov_lines.go`, `prov_push.go`, `prov_overrides.go`, `prov_books.go`,
`prov_multicast.go`, `syslog.go`, `rtcpxr.go`, `clientquality.go` and
`clients.go`. Its other files still bind `$n` and move one domain at a
time (below).
`personal_contacts.go` and `locations.go`; and those of the routing and
time domain: `routing.go`, `timecond.go`, `blocklist.go`, `originate.go`,
`boss.go` and `calendar_dnd.go`. Its other files still bind `$n` and move
one domain at a time (below).

Raw on purpose: the seam itself (`dbq.go`, which passes every statement
through and begins transactions), `TryLock`'s pinned session-lock
connection (`trylock.go`, which keeps its native `$1` and is not in
`namedFiles`), and `SchemaVersion`'s read of goose's `goose_db_version`
(`backup.go`, which binds nothing and goes straight to `database/sql`).

Every package above builds its handle with `db.New` (or `db.NewPool`) where
the binary constructs it, except `internal/store`, which builds its own in
Expand All @@ -73,10 +55,10 @@ the binary constructs it, except `internal/store`, which builds its own in
pool and its transactions are types with `database/sql`'s method set
(`ExecContext`, `QueryContext`, `QueryRowContext`, `BeginTx`) whose every
statement is a momo raw statement, on the store's `*momo.DB` or on momo's
view of the open `*sql.Tx` (`db.AdoptTx`). A converted file binds `@name`;
an unconverted one still binds `$n`, which momo accepts on PostgreSQL as
positional arguments (one placeholder style per statement: a `$n` statement
cannot also use `?` or `@name`). The handle and the transaction types are
view of the open `*sql.Tx` (`db.AdoptTx`). Every converted file binds
`@name`; momo also accepts `$n` on PostgreSQL as positional arguments, which
only the raw exceptions above still use (one placeholder style per
statement: a `$n` statement cannot also use `?` or `@name`). The handle and the transaction types are
`momo.Querier`s, so code can also pass them straight to `momo.Exec`,
`momo.Query[T]`, `momo.Scalars[V]` and the builders. Transactions are still
begun by `database/sql` and adopted, so methods that take a caller's
Expand Down
2 changes: 1 addition & 1 deletion internal/store/cdr.go
Original file line number Diff line number Diff line change
Expand Up @@ -157,7 +157,7 @@ const blockedReasonPrefix = "blocked: "
// ("" at the end).
func (s *Store) ListCDRs(ctx context.Context, f CDRFilter, before int64, limit int) ([]CDR, string, error) {
rows, err := s.db.QueryContext(ctx, `SELECT `+cdrCols+`
FROM cdrs WHERE (@before::integer = 0 OR id < @before::integer) AND (@direction::text = '' OR direction = @direction::text)
FROM cdrs WHERE (@before::bigint = 0 OR id < @before::bigint) AND (@direction::text = '' OR direction = @direction::text)
AND (NOT @failed::boolean OR final_status NOT BETWEEN 200 AND 299)
AND (@disposition::text = '' OR (@disposition::text = 'blocked' AND left(failure_reason, length(@blocked_prefix::text)) = @blocked_prefix::text))
ORDER BY id DESC LIMIT @limit`, momo.Named("before", before), momo.Named("limit", limit+1), momo.Named("direction", f.Direction),
Expand Down
1 change: 1 addition & 0 deletions internal/store/dbq_offline_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -108,6 +108,7 @@ var namedFiles = map[string]bool{
"messaging_events.go": true,
"messaging_route.go": true,
"messaging_route_views.go": true,
"messaging_calls.go": true,
"webchat.go": true,
"webchat_sites.go": true,
}
Expand Down
2 changes: 1 addition & 1 deletion internal/store/media.go
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,7 @@ func (s *Store) ListRecordings(ctx context.Context, extension string, before int
rows, err := s.q(ctx).QueryContext(ctx, `SELECT `+recordingCols+recordingFrom+`
WHERE (@extension::text = '' OR EXISTS (SELECT 1 FROM cdrs c
WHERE c.correlation_id = r.correlation_id AND (c.source = @extension::text OR c.destination = @extension::text)))
AND (@before::integer = 0 OR r.id < @before::integer)
AND (@before::bigint = 0 OR r.id < @before::bigint)
ORDER BY r.id DESC LIMIT @limit`, momo.Named("extension", extension), momo.Named("before", before), momo.Named("limit", limit+1))
if err != nil {
return nil, "", err
Expand Down
2 changes: 1 addition & 1 deletion internal/store/messaging.go
Original file line number Diff line number Diff line change
Expand Up @@ -736,7 +736,7 @@ func (s *Store) ErasePeer(ctx context.Context, actor, peerKey string) (EraseRepo
if err := queueRedactions(ctx, tx, rep.Batch, `
SELECT m.channel_id, ch.provider, m.provider_message_id FROM conversation_messages m
JOIN conversations c ON c.id = m.conversation_id JOIN msg_channels ch ON ch.id = m.channel_id
WHERE c.peer_key = $1 AND m.provider_message_id IS NOT NULL AND ch.provider <> 'sip_trunk'`, peerKey); err != nil {
WHERE c.peer_key = @peer_key AND m.provider_message_id IS NOT NULL AND ch.provider <> 'sip_trunk'`, momo.Named("peer_key", peerKey)); err != nil {
return err
}
// The SIP text queue holds a copy of what went to desk phones and
Expand Down
9 changes: 4 additions & 5 deletions internal/store/messaging_calls.go
Original file line number Diff line number Diff line change
Expand Up @@ -25,11 +25,10 @@ func cdrDigits(x string) string {
// and so no calls.
func (s *Store) ConversationCalls(ctx context.Context, acc InboxAccess, convID int64, limit int) ([]CDR, error) {
var peerKey, peerKind string
// The visibility condition is the inbox's (visible, messaging_inbox.go),
// which still binds $1 and $2, so this read binds $n until the inbox
// moves to @name.
err := s.db.QueryRowContext(ctx, `SELECT c.peer_key, c.peer_kind FROM conversations c WHERE c.id = $3 AND `+visible,
acc.UserID, acc.All, convID).Scan(&peerKey, &peerKind)
// The visibility condition is the inbox's (visibleTo, messaging_inbox.go),
// which binds @user_id and @all.
err := s.db.QueryRowContext(ctx, `SELECT c.peer_key, c.peer_kind FROM conversations c WHERE c.id = @id AND `+visibleTo,
append(inboxAccessArgs(acc.UserID, acc.All), momo.Named("id", convID))...).Scan(&peerKey, &peerKind)
if err != nil {
return nil, mapErr(err)
}
Expand Down
12 changes: 3 additions & 9 deletions internal/store/messaging_inbox.go
Original file line number Diff line number Diff line change
Expand Up @@ -47,21 +47,15 @@ const visibleTo = `(@all::boolean OR c.assignee_user_id = @user_id OR EXISTS (
SELECT 1 FROM conversation_messages vx WHERE vx.conversation_id = c.id AND vx.sender_user_id = @user_id AND vx.kind = 'message')
OR (c.assignee_user_id IS NULL AND ` + poolMemberNamed + `))`

// poolMemberNamed is poolMemberParam (messaging_route.go) binding the caller
// as @user_id. TestVisibleToMatchesVisible keeps the two the same.
// poolMemberNamed is the condition "the caller is one of the people who may
// take the unassigned conversation c" (poolMemberUser in messaging_route.go
// is the same for a row u of users), binding the caller as @user_id.
const poolMemberNamed = `(EXISTS (SELECT 1 FROM queues pq JOIN queue_members pm ON pm.queue_id = pq.id
JOIN user_extensions pe ON pe.extension_id = pm.extension_id
WHERE pq.id = c.queue_id AND pq.enabled AND pq.text_enabled AND pe.user_id = @user_id::bigint)
OR EXISTS (SELECT 1 FROM ring_group_members pg JOIN user_extensions pge ON pge.extension_id = pg.extension_id
WHERE pg.group_id = c.ring_group_id AND pge.user_id = @user_id::bigint))`

// visible is visibleTo for the files whose statements still bind with $n
// (messaging_calls.go): $1 is the user and $2 All. It goes when they move
// to @name.
const visible = `($2::boolean OR c.assignee_user_id = $1 OR EXISTS (
SELECT 1 FROM conversation_messages vx WHERE vx.conversation_id = c.id AND vx.sender_user_id = $1 AND vx.kind = 'message')
OR (c.assignee_user_id IS NULL AND ` + poolMemberParam + `))`

// inboxAccessArgs are the arguments of visibleTo (and of convSelect, which
// binds @user_id too) for a reader: @user_id and @all.
func inboxAccessArgs(userID int64, all bool) []any {
Expand Down
20 changes: 4 additions & 16 deletions internal/store/messaging_inbox_offline_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -13,19 +13,6 @@ import (
"github.com/azrtydxb/momo/dialect"
)

// TestVisibleToMatchesVisible keeps the inbox's access condition the same in
// both its forms: visibleTo (and poolMemberNamed) binding by name, and visible
// (with poolMemberParam), kept for the files still on $n.
func TestVisibleToMatchesVisible(t *testing.T) {
r := strings.NewReplacer("$1", "@user_id", "$2", "@all")
if got := r.Replace(visible); got != visibleTo {
t.Errorf("visible with @name binds =\n%s\nwant visibleTo =\n%s", got, visibleTo)
}
if got := r.Replace(poolMemberParam); got != poolMemberNamed {
t.Errorf("poolMemberParam with @name binds =\n%s\nwant poolMemberNamed =\n%s", got, poolMemberNamed)
}
}

// oneRowDriver answers every query with one row that scans as zero values and
// every exec with one row affected, recording what PostgreSQL would receive,
// so a store method runs its statements without a database. A statement
Expand Down Expand Up @@ -159,10 +146,11 @@ func TestInboxRuntimeStatementsBuild(t *testing.T) {
err := queueRedactions(ctx, tx, "b", `SELECT channel_id, @provider::text, provider_message_id FROM conversation_messages WHERE id = @id`,
momo.Named("id", 1), momo.Named("provider", "telnyx"))
notValidation(t, "queueRedactions with @name", err)
// A fragment still on $n (messaging.go).
// ErasePeer's fragment (messaging.go), binding @peer_key.
err = queueRedactions(ctx, tx, "b", `SELECT m.channel_id, ch.provider, m.provider_message_id FROM conversation_messages m
JOIN conversations c ON c.id = m.conversation_id JOIN msg_channels ch ON ch.id = m.channel_id WHERE c.peer_key = $1`, "+32")
notValidation(t, "queueRedactions with $n", err)
JOIN conversations c ON c.id = m.conversation_id JOIN msg_channels ch ON ch.id = m.channel_id WHERE c.peer_key = @peer_key`,
momo.Named("peer_key", "+32"))
notValidation(t, "queueRedactions with @peer_key", err)
if len(d.sent) != 2 {
t.Errorf("queueRedactions sent %d statements, want 2", len(d.sent))
}
Expand Down
9 changes: 9 additions & 0 deletions internal/store/messaging_offline_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -207,6 +207,15 @@ func TestConversationCallsSends(t *testing.T) {
if err != nil || out == nil || len(out) != 0 {
t.Fatalf("ConversationCalls = %v, %v; want an empty list", out, err)
}
// The visibility read binds @id and visibleTo's @all and
// @user_id, one bind per use in order.
if len(d.sent) > 0 {
wantVis := []any{int64(3), tc.all, int64(7), int64(7), int64(7), int64(7)}
if v := d.sent[0]; strings.Contains(v.sql, "@") || !strings.Contains(v.sql, "c.id = $1 AND ($2::boolean") ||
fmt.Sprintf("%#v", v.args) != fmt.Sprintf("%#v", wantVis) {
t.Errorf("visibility read sent %#v\nwant args %#v", v, wantVis)
}
}
var (
conds []string
args []any
Expand Down
15 changes: 4 additions & 11 deletions internal/store/messaging_privacy.go
Original file line number Diff line number Diff line change
Expand Up @@ -34,23 +34,16 @@ func newRedactionBatch(what string) string {
// under batch. A message queued before and not yet done joins this batch
// and starts its tries again; one already done stays done.
//
// q binds by name (args are momo.Named) and the batch is then bound as
// @batch, a name q must not use; a q of a file not yet converted to @name
// (messaging.go) binds with $n, and the batch is then the next $n.
// q binds by name (args are momo.Named) and the batch is bound as @batch, a
// name q must not use.
func queueRedactions(ctx context.Context, tx *dbTx, batch, q string, args ...any) error {
bind, arg := "@batch", any(momo.Named("batch", batch))
if len(args) > 0 {
if _, named := args[0].(momo.NamedArg); !named {
bind, arg = "$"+strconv.Itoa(len(args)+1), batch
}
}
//nolint:gosec // G202: q is a fixed query of this package with bound parameters.
_, err := tx.ExecContext(ctx, `
INSERT INTO msg_provider_redactions (channel_id, provider, provider_message_id, batch)
SELECT x.channel_id, x.provider, x.provider_message_id, `+bind+` FROM (`+q+`) AS x(channel_id, provider, provider_message_id)
SELECT x.channel_id, x.provider, x.provider_message_id, @batch FROM (`+q+`) AS x(channel_id, provider, provider_message_id)
ON CONFLICT (channel_id, provider_message_id) DO UPDATE SET batch = EXCLUDED.batch, state = 'pending', attempts = 0,
next_attempt_at = now(), claimed_until = NULL, last_error = '', finished_at = NULL
WHERE msg_provider_redactions.state <> 'done'`, append(args, arg)...)
WHERE msg_provider_redactions.state <> 'done'`, append(args, momo.Named("batch", batch))...)
return err
}

Expand Down
Loading