Assess org-wide ad hoc IX Reviewer invocation modes (workflow vs webhook vs Windows Service poller)

[PrzemyslawKlys]

Summary

Assess how IntelligenceX Reviewer could support ad hoc, org-wide PR reviews invoked by comment/mention (for example @intelligencex review) without requiring the reviewer workflow to be deployed into every target repository.

This issue is for architecture assessment only: document the viable operating models, their trust/privacy implications, and what would need to change in the current implementation.

Problem Statement

Today IX Reviewer is primarily a repo-installed workflow experience:

• A target repo gets .github/workflows/review-intelligencex.yml
• The workflow calls the reusable reviewer workflow
• Optional BYO GitHub App identity/secrets are used for posting as a branded app
• Review behavior is driven by .intelligencex/reviewer.json

The desired experience is an org-wide ad hoc invocation model where a maintainer could write something like:

• @intelligencex review
• @intelligencex please review
• @intelligencex rerun

and have IX Reviewer act on that PR even when the target repository has not been onboarded with the reviewer workflow.

Current State Observed

What already exists

• The reviewer engine already supports direct repo + pr_number execution, so it is possible to point the reviewer at a specific PR centrally.
• The reusable workflow already supports manual/workflow-call inputs for repo and pr_number.
• The setup flows already support BYO GitHub App creation and installation-token usage for org onboarding.
• BYO/self-hosted trust positioning already exists for onboarding and reviewer auth.

What does not exist yet

• No current org-wide event/command receiver for PR comments/mentions.
• No current GitHub App webhook event subscription path for reviewer commands.
• No current poller/daemon/Windows Service that watches org PR comments for commands.
• No current target-repo config fetch path for loading .intelligencex/reviewer.json centrally before review.
• No current central execution contract for ad hoc review mode vs repo-installed workflow mode.

Options To Assess

Option 1: Status quo / repo-installed workflow only

How it works:

• Each repo installs the reviewer workflow and optional config/secrets.
• PR events trigger review in that repo via Actions.

Pros:

• No always-on service required.
• Cleanest existing implementation path.
• Uses current onboarding flow with minimal new architecture.

Cons:

• Does not solve org-wide ad hoc invocation.
• Requires setup in every target repository.
• Harder to position as a lightweight "just mention the bot" experience.

Assessment question:

• Is this sufficient for teams that want org-wide coverage, or is ad hoc central execution important enough to justify a new runtime mode?

Option 2: Central webhook receiver + worker

How it works:

• Customer installs their own GitHub App across selected org repos.
• The app sends webhook events (for example issue_comment) to a central receiver.
• The receiver validates the event, parses commands such as @intelligencex review, and enqueues work.
• A worker obtains an installation token, optionally fetches target repo config, runs IX Reviewer centrally, and posts results back as the app.

Pros:

• Real-time / near-real-time behavior.
• Clean event-driven model.
• Good UX for comment-driven commands.
• Strong BYO story when customer hosts it themselves.

Cons:

• Requires hosting and exposing a public HTTPS endpoint.
• Introduces ingress, TLS, firewall, availability, and webhook hardening concerns.
• More operationally complex for privacy-sensitive or cautious customers.

Trust/privacy positioning:

• Strong if framed as customer-owned runtime: BYO App, BYO hosting, BYO secrets, optional BYO model/provider.
• We should be explicit that this is not "no service needed"; it is "self-hosted service in your environment."

Assessment question:

• Is the operational cost of public ingress acceptable for target customers, or does it weaken the privacy-first positioning too much?

Option 3: Windows Service / background poller (outbound-only)

How it works:

• Customer installs their own GitHub App across selected org repos.
• A Windows Service runs inside the customer environment.
• Every 1-5 minutes it polls GitHub for new PR comments/commands.
• It looks for commands like @intelligencex review, de-dupes by comment id, and runs IX Reviewer centrally.
• It posts the result back as the app.

Pros:

• No public webhook endpoint.
• Outbound-only traffic to GitHub API.
• Easier privacy/trust story for cautious orgs.
• Comfortable fit for .NET + Windows-heavy customers.
• Likely the best first self-hosted org-wide mode.

Cons:

• Command handling is delayed by poll interval.
• Higher API usage than webhooks.
• Requires local durable state (cursor/idempotency store, likely SQLite).
• Polling strategy must be careful to avoid scanning the whole org inefficiently.

Trust/privacy positioning:

• Very strong: BYO App, BYO machine, no inbound endpoint, local secret custody.
• Probably the easiest mode to explain for privacy-first/self-hosted customers.

Assessment question:

• Should this become the first implementation for org-wide ad hoc review because it avoids internet-exposed hosting?

Option 4: Hybrid model

How it works:

• Support both webhook mode and poller mode.
• Same command semantics and reviewer engine.
• Different event-ingestion/runtime adapters.

Pros:

• Best flexibility.
• Teams choose real-time webhooks or privacy-first poller mode.
• Strong long-term product story.

Cons:

• More implementation and documentation surface.
• Requires a clear shared execution contract to avoid drift between modes.

Assessment question:

• Should we design a shared command/runtime abstraction now so webhook and poller modes can coexist later without duplication?

Key Design Questions

1. Command scope

What commands should be supported initially?

Suggested MVP:

• @intelligencex review
• @intelligencex rerun
• @intelligencex review summary-only

2. Where does config come from?

For central ad hoc execution, should we:

• always use central/default config,
• fetch .intelligencex/reviewer.json from the target repo if present,
• or support a precedence model combining central defaults with repo overrides?

Suggested direction:

• Fetch target repo .intelligencex/reviewer.json when present.
• Fall back to a central/default config when absent.
• Keep a clearly documented precedence model for ad hoc mode.

3. Should ad hoc mode support static analysis?

Choices:

• API-only review of PR diff/comments/threads
• Optional temp checkout for analysis/gate parity
• Full clone/checkout always

Suggested direction:

• MVP should be API-only review.
• Add optional temp checkout later if analysis/gate parity is needed.

4. How do we prevent duplicate runs?

Need idempotency around:

• comment id
• installation id
• repo
• PR number
• command type
• head SHA

5. What should the runtime abstraction look like?

Potential shared pieces:

• command parser
• installation token provider
• target repo config fetcher
• review job store / idempotency store
• reviewer invocation adapter
• result poster / sticky comment updater

Suggested MVP Direction

Recommended phased path:

Phase 1: Windows Service poller

Rationale:

• avoids public webhook hosting
• strongest privacy-first story
• fits the current user/customer preference
• easiest self-hosted deployment for Windows-centric environments

Expected MVP behavior:

• poll every 1-5 minutes
• inspect open PR comments updated since last cursor
• detect @intelligencex review
• de-dupe by comment id
• fetch target repo .intelligencex/reviewer.json if present
• run IX Reviewer centrally using target repo + pr_number
• post one sticky summary comment as the app

Phase 2: Optional webhook mode

Rationale:

• add low-latency event-driven mode later
• reuse the same command/review execution pipeline
• position it as an optional advanced deployment for teams comfortable with public ingress

Proposed Implementation Shape

Potential new components:

• IntelligenceX.Reviewer.Commands or similar shared command/orchestration library
• IntelligenceX.Reviewer.Poller as a .NET Worker Service / Windows Service
• GitHubAppInstallationTokenProvider
• TargetRepoReviewConfigFetcher
• ReviewJobStore (SQLite)
• CommandMentionParser
• ReviewerInvocationAdapter

Potential later component:

• IntelligenceX.Reviewer.Webhook for webhook receiver mode

Acceptance Criteria For Assessment

This issue should result in:

• a decision on whether org-wide ad hoc review is worth pursuing
• a decision on which mode should be implemented first
• a trust/privacy positioning statement for the chosen approach
• a clear config strategy for central execution
• an MVP scope definition
• a follow-up implementation breakdown issue/epic if approved

Recommendation For Discussion

Current recommendation:

• Treat org-wide ad hoc review as viable and worth pursuing.
• Prefer Windows Service poller as the first implementation.
• Keep webhook mode as a later optional runtime.
• Reuse the existing reviewer engine and add a small command/orchestration layer around it.

Labels

Suggested labels:

• enhancement
• ix/category:feature
• ix/vision:needs-review