Skip to content

feat: bidirectional issue sync (GitHub, GitLab, Jira Cloud)#2

Merged
0xHexE merged 2 commits into
mainfrom
feat/bidirectional-issue-sync
Jul 6, 2026
Merged

feat: bidirectional issue sync (GitHub, GitLab, Jira Cloud)#2
0xHexE merged 2 commits into
mainfrom
feat/bidirectional-issue-sync

Conversation

@0xHexE

@0xHexE 0xHexE commented Jul 6, 2026

Copy link
Copy Markdown
Member

Summary

Adds a new bidirectional issue-sync engine that mirrors Multica issues with external trackers (GitHub Issues, GitLab Issues, Jira Cloud). Projects can attach multiple remote containers (GitHub repos, GitLab projects, Jira projects) whose issues sync both ways: title, description, status (with mapping), comments (with attribution), and assignees + labels.

This was a greenfield outbound direction — the existing GitHub/GitLab integrations only mirrored PRs/MRs inbound. No outbound path, no issue sync, no background worker existed before.

Architecture

Inbound:  provider webhook → handler normalizes → Engine.ApplyRemote (upsert local issue/comment, ActorType="issue_sync")
Outbound: event-bus listener → issue_sync_outbox row → worker pushes to provider, records content hash

Three-layer echo suppression prevents sync loops:

  1. Actor filter — outbound listeners skip events with ActorType == "issue_sync"
  2. Connection identity + content hash + bot login — inbound webhooks from the connection's own identity are dropped; inbound content hashing to last_pushed_hash is dropped
  3. Stale-replay clock — inbound events older than remote_updated_at are dropped (last-write-wins)

What's included

Backend (server/)

  • internal/integrations/issuesync/ — provider-agnostic engine (provider.go interface, engine.go inbound apply + outbound enqueue, outbox.go polling worker with FOR UPDATE SKIP LOCKED + exponential backoff, mapping.go status maps) + three provider impls:
    • github.go — GitHub App installation tokens, reuses existing App credentials
    • gitlab.go — GitLab OAuth token via existing connection + secretbox
    • jira.go + jira_adf.go — Jira Cloud REST + Atlassian Document Format ↔ markdown converter
  • Migration 136jira_connection, issue_sync_source, external_issue_link, external_comment_link, external_identity, issue_sync_outbox (+ sqlc queries)
  • handler/issue_sync.go — sync-source CRUD API, remote-container picker, inbound webhook routing
  • handler/jira.go — Jira OAuth 3LO (rotating refresh tokens), connection lifecycle, webhook handler
  • cmd/server/issuesync_listeners.go — outbound event-bus listeners (issue/comment/labels → outbox)
  • Extended HandleGitHubWebhook + HandleGitLabWebhook with issues/issue_comment and Issue/Note hooks
  • Outbox worker + listener registration wired in main.go/router.go
  • New gitlab_repo project resource type (alongside existing github_repo)

Frontend (packages/)

  • packages/coretypes/issue-sync.ts (zod schemas), issue-sync/ hooks (React Query, wsId-scoped keys), WS event invalidation, malformed-response tests
  • packages/viewsjira-tab.tsx settings tab, project-sync-sources-section.tsx (attach/manage sources), issue-sync-badges.tsx
  • i18n — en + zh-Hans + ja + ko keys (locale parity tested)

REST API

Method Path Purpose
GET/POST /api/projects/{id}/sync-sources List/create sync sources
PUT/DELETE /api/projects/{id}/sync-sources/{sourceId} Update/delete a source
GET /api/workspaces/{id}/sync/{provider}/remote-projects Remote repo/project picker
GET /api/workspaces/{id}/jira/connect Jira OAuth authorize URL
GET /api/jira/oauth/callback Jira OAuth callback
POST /api/webhooks/jira/{connectionId} Jira webhook ingress
GET/DELETE /api/workspaces/{id}/jira/connections[/{connectionId}] Jira connection lifecycle

Testing

  • Go: go build ./... ✅ · go vet clean ✅ · 26 issuesync tests pass (engine hash/normalization, GitLab provider httptest, Jira ADF round-trip + refresh-token rotation) · go test ./internal/handler/ passes
  • Frontend: pnpm typecheck all 6 packages ✅ · pnpm test (views) 1627 pass ✅ · core 769 pass ✅ · locale parity ✅
  • sqlc: regenerates with zero diff

Notes

  • Jira Cloud OAuth 3LO uses rotating refresh tokens (the refresh response includes a new refresh token; always persisted — tested)
  • Jira dynamic webhooks expire after 30 days (refresh scheduler is stubbed for a follow-up; the webhook handler works today via manual registration)
  • The outbox worker is multi-replica safe (FOR UPDATE SKIP LOCKED)
  • Per-project push_default source controls where new Multica issues are created remotely (partial unique index enforces one per project)

Migration notes for operators

New optional env vars (all gated — absence disables that provider cleanly):

  • MULTICA_JIRA_SECRET_KEY — at-rest encryption for Jira OAuth tokens
  • JIRA_OAUTH_CLIENT_ID / JIRA_OAUTH_CLIENT_SECRET — Jira Cloud OAuth app credentials
  • Existing MULTICA_GITLAB_SECRET_KEY + GITHUB_APP_ID/GITHUB_APP_PRIVATE_KEY are reused

🤖 Generated with Claude Code

Add a new issue-sync engine that mirrors Multica issues bidirectionally
with external trackers. Projects can attach multiple remote containers
(GitHub repos, GitLab projects, Jira Cloud projects) whose issues sync
both ways: title, description, status (with provider-specific mapping),
comments (with attribution), and assignees + labels.

Architecture:
- server/internal/integrations/issuesync/ — provider-agnostic engine
  (provider.go, engine.go, outbox.go, mapping.go) + GitHub/GitLab/Jira
  provider impls. Inbound: webhook -> normalize -> ApplyRemote (upsert
  local issue/comment with ActorType 'issue_sync'). Outbound: event-bus
  listeners -> issue_sync_outbox -> worker pushes to provider.
- Three-layer echo suppression: actor-type filter, connection-identity +
  content-hash + bot-login, and stale-replay clock.
- DB: migration 136 (jira_connection, issue_sync_source,
  external_issue_link, external_comment_link, external_identity,
  issue_sync_outbox) + sqlc queries.
- REST: sync-source CRUD under /api/projects/{id}/sync-sources, remote
  container picker, Jira OAuth 3LO + connection lifecycle.
- Frontend: core types/zod hooks + WS invalidation, settings jira-tab,
  project sync-sources section, issue sync badges, i18n (en/zh/ja/ko).

Jira Cloud uses OAuth 3LO with rotating refresh tokens; GitLab reuses the
existing OAuth connection; GitHub reuses App installation tokens. A new
gitlab_repo project resource type is also added so GitLab repos can be
attached alongside GitHub repos.
@0xHexE
0xHexE marked this pull request as ready for review July 6, 2026 09:46
The gitlab_repo section header used a literal "GitLab" string which
violated the i18next/no-literal-string lint rule. Add a gitlab_label
key to all 4 locales (en/zh-Hans/ja/ko) and reference it via t().
@0xHexE
0xHexE merged commit e29819a into main Jul 6, 2026
6 checks passed
0xHexE pushed a commit that referenced this pull request Jul 9, 2026
…UL-4195) (multica-ai#5068)

* fix(comments): guarantee at-least-once processing of user comments (MUL-4195)

Consecutive comments on an issue were silently dropped: a new comment that
arrived while the agent already had a queued/dispatched task was discarded by
the HasPendingTaskForIssueAndAgent dedup, losing the user's follow-up
instruction with no visible trace. Comments — unlike chat — are deliberate,
addressed, persisted input and must never vanish.

This makes comment handling at-least-once while keeping concurrency bounded to
one run per (issue, agent):

- Merge, don't drop (PR1): a comment landing while a not-yet-started task
  exists is folded into that task — the prior trigger becomes a coalesced
  comment and the new one becomes the trigger, so a single run still covers
  every deliberate comment. Falls back to a fresh enqueue if the pending task
  was claimed mid-flight, so nothing is lost in the race.
- Completion reconciliation (PR2): on task completion, a member comment newer
  than the run's started_at schedules exactly one follow-up via the normal
  trigger pipeline. Loop-safe: member-authored only, capped by the existing
  per-(issue,agent) dedup, and terminating.
- Visibility (PR3): coalesced_comment_ids is surfaced on the task API and in
  the run prompt so the covered comments are explicit.

Migration 145 adds agent_task_queue.coalesced_comment_ids UUID[].

Tests: merge-not-drop preserves all three of a rapid burst and repoints the
trigger to the newest; reconciliation query gates on member/since; e2e
CompleteTask enqueues a follow-up for a mid-run member comment and does not for
none.

Co-authored-by: multica-agent <github@multica.ai>

* fix(comments): address review — originator gate, agent-scoped reconcile, cross-thread coalesced prompt (MUL-4195)

Resolves GPT-Boy's Request-changes review on PR multica-ai#5068.

Must-fix #1 — merge no longer inherits a stale originator/runtime context.
MergeCommentIntoPendingTask now only folds a comment into a pending task
whose originator_user_id IS NOT DISTINCT FROM the new comment's originator.
runtime_mcp_overlay / runtime_connected_apps are a pure function of
(originator, agent) and the agent is fixed, so a matching originator keeps
the stored overlay/attribution valid; a differing originator (e.g. user B
commenting on a task originated by user A) matches no row and the caller
enqueues a fresh follow-up with B's own context instead of reusing A's.
trigger_summary is refreshed to the new trigger comment.

Must-fix #2 — completion reconcile no longer re-wakes unrelated agents.
reconcileCommentsOnCompletion computes the latest member comment's triggers
and keeps ONLY the agent that just completed, instead of fanning the comment
out through the full pipeline. An @-mention of agent B during agent A's run
is triggered once at creation time and is no longer replayed (double-run)
when A completes.

Should-fix #3 — coalesced-comment prompt no longer assumes a single thread.
The claim response now carries each folded comment's thread id / author /
created_at / content (CoalescedCommentData); the prompt embeds them directly
so the agent addresses cross-thread folded comments without the wrong
"they are in the triggering thread" hint. Old servers that ship only ids
fall back to an issue-wide fetch, still without the same-thread assumption.

Tests: TestMergeCommentIntoPendingTask_OriginatorGate (query gate),
TestCompleteTask_DoesNotReTriggerOtherAgentMentionedDuringRun (reconcile
scoping), TestBuildCommentPromptCoalescedCrossThread / IDsOnlyFallback
(prompt). Existing MUL-4195 suites still pass.

Co-authored-by: multica-agent <github@multica.ai>

* fix(comments): close unique-index drop + dispatched-window race in comment coalescing (MUL-4195)

Second-round review follow-up on PR multica-ai#5068.

Must-fix #1 — originator-mismatch no longer drops the comment.
The previous originator gate returned ErrNoRows on a different originator and
the caller fell through to a fresh enqueue, which collided with the
idx_one_pending_task_per_issue_agent unique index (one queued/dispatched task
per (issue, agent)) — silently dropping the second user's comment. Replaced
the gate with recompute-on-merge: MergeCommentIntoPendingTask now re-stamps
originator_user_id, runtime_mcp_overlay, runtime_connected_apps and
trigger_summary to the new comment's originator. A different member's comment
folds into the single coalescing run carrying the latest instruction's own
identity/overlay (no cross-user capability bleed, no drop, no collision).

Must-fix #2 — comment arriving in the claim→StartTask window is no longer lost.
Merge now targets only PRE-CLAIM states ('queued','deferred'); a
dispatched/running task is never a merge target, so a post-claim comment is
never falsely stamped into coalesced_comment_ids as "delivered". Completion
reconcile is re-anchored on dispatched_at (the moment the claim response is
built) instead of started_at, and sweeps ALL undelivered member comments since
that anchor — replaying each through the normal enqueue path so they coalesce
into one bounded, agent-scoped follow-up run. This covers the dispatch→start
window a started_at anchor missed.

Enqueue path: on a merge miss the caller no longer blindly fresh-enqueues
(which could collide with a dispatched sibling); it defers to the active
task's completion reconcile via HasActiveTaskForIssueAndAgent, and only
fresh-enqueues when no active task exists.

Tests: rewrote the query test to
TestMergeCommentIntoPendingTask_RecomputesOriginatorAndSkipsDispatched;
added TestConsecutiveCommentsDifferentOriginatorsFullEnqueuePath (full handler
enqueue path, two distinct originators) and
TestCompleteTask_ReconcilesDispatchedWindowComment (claim→start window). All
existing MUL-4195 handler/cmd-server/daemon/service suites still pass.

Co-authored-by: multica-agent <github@multica.ai>

* fix(comments): catch pre-dispatch merge-race comment in completion reconcile (MUL-4195)

Third-round review follow-up on PR multica-ai#5068.

Race: a member comment is created while the task is still queued, but its
merge loses the race to the daemon claiming the task (queued→dispatched). The
merge then finds no pre-claim row (ErrNoRows), the enqueue path defers to
reconcile — but the comment's created_at is BEFORE dispatched_at, so the
dispatched_at-anchored reconcile skipped it and the comment vanished with no
task coverage.

Fix: anchor completion reconcile on the task's created_at (which always
precedes dispatch) instead of a dispatch/start timestamp, and exclude the
run's DELIVERED SET — trigger_comment_id ∪ coalesced_comment_ids. Because
merges only ever touch pre-claim rows, that set is exactly what the claim
response carried, so any member comment created since the task was made that
is NOT in it was genuinely undelivered and earns a bounded follow-up. This
catches the pre-dispatch merge-race comment and the dispatch→start comment,
while never re-firing a comment that was delivered as a pre-claim coalesced
entry.

Test: TestCompleteTask_ReconcilesPreDispatchMergeRaceComment reproduces the
race (comment created pre-dispatch, task dispatched before merge, plus a
delivered coalesced comment) and asserts exactly one follow-up, triggered by
the race comment, with the delivered coalesced comment excluded. Existing
reconcile fixtures updated to set a realistic created_at (the production
invariant that created_at is the earliest task timestamp).

Co-authored-by: multica-agent <github@multica.ai>

* fix(comments): merge only into the queued task, never a deferred fallback (MUL-4195)

Fourth-round review follow-up on PR multica-ai#5068.

MergeCommentIntoPendingTask targeted status IN ('queued','deferred') ordered
by created_at DESC. When a (issue, agent) pair had both an older queued task
(the run about to be claimed) and a newer deferred assignee-fallback task, a
new comment merged into the deferred row instead of the queued one — so the
comment missed the imminent run and the deferred fallback could later promote
into a duplicate/conflicting run.

This merge is only ever reached when HasPendingTaskForIssueAndAgent matched a
queued/dispatched task (it never inspects deferred), so the coalescing target
must be the queued row. Restricted the merge target to status = 'queued'
(the unique index guarantees at most one). Deferred fallbacks keep their own
fire_at/promotion escalation lifecycle and are never a merge target.

Test: TestMergeCommentIntoPendingTask_TargetsQueuedNotDeferred seeds an older
queued task + a newer deferred fallback for the same (issue, agent), merges a
new comment, and asserts it lands on the queued task (trigger repointed, old
trigger coalesced) while the deferred fallback is left untouched.

Co-authored-by: multica-agent <github@multica.ai>

---------

Co-authored-by: Eve <eve@multica-ai.local>
Co-authored-by: multica-agent <github@multica.ai>
0xHexE pushed a commit that referenced this pull request Jul 10, 2026
Syncs the fork with multica-ai/multica upstream (35 commits ahead of the
previous sync). Brings in: thread quick-jump minimap on issue detail,
Mod-Enter IME-composition guard in the editor, Runtimes tab CLI-update red
dot removal, GitHub PR/check_suite webhook fan-out to all bound workspaces
(MUL-4343), daemon task transcript ordering + log rotation, Codex gpt-5.6
model series + dynamic catalog, coalesced per-thread direct-chat replies
(MUL-4348/MUL-4351), Lark binding token clock-skew tolerance, merged-comment
delivery preservation, claim-time comment scope + attachment path guardrails
(MUL-4252), unified avatar size tiers + rounded avatars (MUL-4277/MUL-4184),
overflowing desktop tab additions, chat FAB unread-badge removal, and the
v0.3.43 release.

Conflicts resolved:
- server/internal/handler/quick_create_parent_test.go: take fork (ours). The
  fork removed the daemon CLI version gate for quick-create
  (d588f8e "remove daemon CLI version gate"), so MinQuickCreateCLIVersion no
  longer exists in non-test code. Upstream's improved version-gate test setup
  (bump cli_version on the agent's bound runtime) references that now-undefined
  symbol; taking ours keeps the test compiling against the gateless handler.

Additional fixes (merge-introduced regressions in fork-only code):
- server/internal/handler/gitlab.go: restore repoIdentityFromURL as a local
  helper. Upstream's webhook fan-out refactor (53f05cc, MUL-4343) removed
  repoIdentityFromURL + resolveWorkspaceForRepo from github.go because GitHub
  routing no longer gates on workspace.repos. The fork's GitLab webhook
  auto-registration still uses it to match repos-registry entries against a
  connection's instance — a different concern (registering hooks, not routing)
  — so the helper is re-added local to its only remaining caller.
- packages/views/settings/components/mattermost-tab.tsx: pass size="lg" instead
  of size={32} to ActorAvatar. Upstream's avatar refactor (f4de094,
  MUL-4277/MUL-4184) retyped size from a raw pixel number to the AvatarSize
  union ("xs".."2xl"); 32px == "lg" tier (AVATAR_SIZE_PX.lg === 32).
- server/migrations: renumber 157_issue_origin_type_reconcile -> 161. Upstream
  claimed prefix 157 (157_agent_task_delivered_comments). The fork's reconcile
  migration restores the full issue.origin_type CHECK union after the two
  same-prefix-149 migrations (agent_create + mattermost_chat) each drop the
  other's value; it is idempotent and forward-only, so running it last at 161
  is correct on both fresh and already-migrated databases.

Verified: go build ./... + go vet ./... clean; migration uniqueness/direction
lint passes; turbo typecheck passes (6/6 packages); @multica/views tests pass
(174 files / 1810 tests). The remaining local test failures
(@multica/core workspace/mutations + @multica/desktop runtime-config-loader)
are pre-existing sandbox environment issues (Node experimental localStorage
without --localstorage-file; Electron not installed) on files the merge did
not touch.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant