Fix stale-issue bug, add integrations (SNS+Slack), and DSPM data discovery

[therandomsecurityguy]

Summary

Closes the gap analysis by addressing three of the highest-priority findings from the Wiz-parity review: the dead autoTicketConfig, the stale-issue lifecycle bug, and the absence of automated data discovery (DSPM).

Phase 1.1 — Stale-issue fix + posture trends

• Bug fix: runner.ts existing-issue branch only added to activeIssueIds without refreshing fields — issues were frozen at first creation. Now refreshExistingIssue re-derives riskScore, severity, pathSummary, and metadata.scoringFactors on every re-match, guarded by #status <> :resolved so resolved issues aren't clobbered.
• New PostureTrendStore (packages/risk-engine/src/posture-trend-store.ts) writes daily snapshots to DynamoDB PostureTrends (metrics: openIssuesBySeverity, publicBuckets, crossAccountTrusts, usersWithoutMfa).
• New API: GET /trends/:metric?days=90 returns the historical series.

Phase 1.2 — Integrations core (SNS + Slack)

• New @khalifa/integrations package with pluggable IssueActionSink interface — the previously dead autoTicketConfig is now wired to live sinks.
• SNS sink publishes issue JSON envelopes to ISSUES_TOPIC_ARN (drives email/Lambda/SQS fan-out) with severity/rule/issueId message attributes.
• Slack sink posts Block Kit cards to an incoming webhook, filtered by SLACK_MIN_SEVERITY (default high).
• IssueIntegrationsOrchestrator loads sinks from comma-separated ISSUE_SINKS env var; dedupes names; ignores unknown sinks. Best-effort emit — sink errors never fail the rule run.
• Issue.externalRefs field added to types (risk-engine + api-service), persisted via updateExternalRefs (dedup by system+id).
• New API routes: GET /integrations/status (Viewer+), POST /issues/:id/suppress (Analyst+), POST /issues/:id/reopen (Analyst+).
• CDK: SNS topic khalifa-issues, risk-engine role granted sns:Publish + PostureTrendsTable write.
• ConfigMap: ISSUES_TOPIC_ARN, ISSUE_SINKS, SLACK_WEBHOOK_URL, SLACK_MIN_SEVERITY, UI_BASE_URL, POSTURE_TRENDS_TABLE.

Phase 2.2 — DSPM data discovery

• New dspm-scanner Lambda — agentless S3 sampler with pluggable classifiers.
• Regex classifier (regex-classifier.ts) detects: AWS access keys (AKIA…), AWS secret keys, GCP service-account JSON, private keys (-----BEGIN … PRIVATE KEY-----), JWTs, credit cards (Luhn-validated), emails, phone, SSN, high-entropy secrets (Shannon entropy ≥ 4.5).
• Macie classifier (opt-in via MACIE_ENABLED=true) enriches with managed PII findings via ListFindings/GetFindings.
• promoteClassification() maps findings to secret / restricted / internal tiers with confidence scores.
• Graph schema: new DataClassificationFinding vertex + CLASSIFIES edge to S3Bucket/RdsInstance; new prop data_class_source: 'tag'|'scanner'|'macie'.
• New rules: RULE-011 (public datastore with PII), RULE-012 (secret committed in S3 object).
• CDK: DspmScannerFn (15m timeout, 2GB mem), EventBridge daily at 03:00 UTC, S3 read IAM grants; DSPM_SCAN_MODE=off|tagged-only|all (default tagged-only for cost control).
• Privacy/OPSEC: sample-size cap (50 objects/bucket, 256KB/object), never stores sampled values — only metadata (piiTypes, secretCount, confidence).

Test plan

• npm run build --workspaces — all 9 workspaces compile clean
• CDK stack compiles (cdk workspace tsc)
• npx jest — 121 tests passing across 12 suites (risk-engine, integrations, api-service, dspm-scanner, collector)
• Lint clean on all modified workspaces (0 errors; pre-existing policy-evaluator warnings untouched)
• runner.test.ts verifies re-match updates existing issue fields (not frozen)
• runner.test.ts verifies emitToIntegrations skips suppressed issues and disabled autoTicketConfig
• orchestrator.test.ts verifies sink loading, dedup, and unknown-sink handling
• classifier.test.ts verifies regex patterns (AWS keys, private keys, JWT, Luhn credit cards, SSN, GCP SA, high-entropy) and promoteClassification tiers
• rules.test.ts updated to expect 12 rules (11 enabled — RULE-003 still disabled)

Files changed

37 files, +2,180 / -302 lines.

Not in this PR (Phase 2.1 — designed, not built)

Jira sink, Security Hub ASFF export, and SSM Automation auto-remediation actions. The IssueActionSink interface and RemediationAction type in packages/integrations/src/types.ts are ready to accept them as a follow-up.

[Shasheen8]

@therandomsecurityguy I wanted to ask you about the direction for this.

Would be good to discuss what a final state of this tool would look like.

[Shasheen8]

@Anusha-RG can we get this data into the dashboard too? But we need to first plug it into prod