Skip to content

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

20 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

because-go

Production causal inference on known causal DAGs — Pearl-style structural causal models for Go. You bring the graph (service topology, metric dependencies); because-go fits the mechanisms and answers three families of questions at alert-time latency: what happens if we change this?, what would have happened if we hadn't?, and who is to blame?

The library never discovers causal structure. The DAG is an input, assumed correct — which is exactly the situation in infrastructure telemetry, where the call graph is known and the interesting questions start after it.

The model in four ideas

SCM = graph + mechanisms + noise. A structural causal model assigns every node X a mechanism X := f(parents(X)) + ε with an independent noise term ε. The graph (because.Structure) is pure topology: node IDs, kinds (continuous metrics, binary events), directed edges. A because.Model adds one fitted mechanism per node (package mech: linear ANMs, empirical or Gaussian roots, Bernoulli event roots). The noise terms are where the world's randomness lives — everything else is deterministic propagation.

do() — intervene, then predict. Model.Do, Model.Effect, and Model.Exceedance implement Pearl's do-operator by Monte Carlo ancestral sampling under an edited model. Four intervention kinds: Set pins a node to a constant and severs its incoming edges (hard do); Shift adds a constant after the mechanism runs (the "canary measured +30 ms locally" primitive); Scale multiplies (traffic multipliers); Replace swaps in a whole new marginal. Effect and Exceedance always compute a no-intervention baseline from the same random numbers, so the reported delta is variance-reduced — for linear mechanisms a Shift's DeltaMean is exact to machine precision.

Counterfactual — abduction, action, prediction. Model.Counterfactual answers questions about one realized episode, not a population: recover every node's noise from the observed evidence (abduction), apply the interventions (action), replay the mechanisms with the noise frozen (prediction). No sampling; exact for invertible additive-noise mechanisms.

Attribution — Shapley over mechanisms. Package attrib assigns blame. AttributeAnomaly explains one anomalous observation by Shapley attribution over the recovered noise terms (Budhathoki et al., ICML 2022): blame lands on the node whose own mechanism misbehaved, not on nodes that faithfully propagated the fault. AttributeChange explains a distribution shift between two fitted models by Shapley attribution over mechanism replacement (Budhathoki et al., AISTATS 2021): fit one model before the deploy and one after, and it names the mechanisms that actually changed. RankRootCauses wraps anomaly attribution into a ranked top-k with normalized shares — the shape a paging pipeline consumes.

Assumptions. because-go's answers are conditional on: the DAG being correct; causal sufficiency — every common cause of two modeled nodes is itself in the graph (no latent confounders); additive-noise mechanisms for continuous nodes (binary events are Bernoulli roots, meant to be intervened on, e.g. do(deploy = 1)); and caller-unrolled time — no feedback loops; model temporal dependence with time-indexed node IDs (svc.latency@t-1).

Quickstart A — will this change breach the SLO?

Fit a model from observational telemetry, then predict the effect of a +30 ms database regression on checkout latency (full worked example: go run ./examples/slo-impact):

package main

import (
	"context"
	"fmt"
	"log"

	because "github.com/loewenthal-corp/because-go"
	"github.com/loewenthal-corp/because-go/fit"
)

func main() {
	ctx := context.Background()

	// The DAG is an input, not something the library discovers. Nodes
	// referenced only by edges are inferred as Continuous; event nodes are
	// declared Binary.
	s, err := because.NewStructure(
		because.Nodes{{ID: "deploy.checkout", Kind: because.Binary}},
		because.Edges{
			{From: "traffic.rps", To: "gw.latency"},
			{From: "traffic.rps", To: "db.latency"},
			{From: "db.latency", To: "payments.latency"},
			{From: "gw.latency", To: "checkout.latency"},
			{From: "db.latency", To: "checkout.latency"},
			{From: "payments.latency", To: "checkout.latency"},
			{From: "deploy.checkout", To: "checkout.latency"},
		})
	if err != nil {
		log.Fatal(err)
	}

	// Columnar telemetry: node ID -> aligned sample column (thousands of
	// rows in practice; see examples/slo-impact for a full generator).
	telemetry := fit.Dataset{
		"traffic.rps":      {1071, 915, 707, 1273, 1102, 1239, 738, 883, 1159, 864, 966, 965},
		"gw.latency":       {72.3, 66.7, 52.2, 82.9, 79.2, 83.6, 54.4, 63.9, 81.2, 63.7, 64.6, 71.9},
		"db.latency":       {97.3, 83.9, 78.0, 101.1, 93.2, 98.6, 72.9, 81.6, 97.5, 83.3, 84.6, 92.8},
		"payments.latency": {141.2, 131.9, 125.4, 134.6, 128.1, 136.3, 111.5, 124.6, 134.3, 124.4, 117.4, 137.0},
		"deploy.checkout":  {0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0},
		"checkout.latency": {445.3, 401.3, 364.6, 508.9, 425.4, 447.6, 346.8, 389.9, 440.4, 445.3, 381.4, 422.0},
	}
	m, report, err := fit.Fit(s, telemetry)
	if err != nil {
		log.Fatal(err)
	}
	r, _ := report.Node("checkout.latency")
	fmt.Printf("checkout.latency fit R^2 = %.3f\n", r.R2)

	// "The canary measured the database +30 ms slower": Shift is the soft
	// intervention X := f(parents) + noise + 30 — db still tracks traffic.
	shift := []because.Intervention{{Node: "db.latency", Kind: because.Shift, Value: 30}}
	eff, err := m.Effect(ctx, "checkout.latency", shift)
	if err != nil {
		log.Fatal(err)
	}
	fmt.Printf("delta E[checkout.latency] = %+.2f ms +/- %.2g\n",
		eff.DeltaMean, eff.DeltaMCStdErr)

	// The SLO question: breach probability before vs under the shift.
	exc, err := m.Exceedance(ctx, "checkout.latency", 500, shift)
	if err != nil {
		log.Fatal(err)
	}
	fmt.Printf("P(checkout.latency > 500ms): %.1f%% -> %.1f%%\n",
		100*exc.BaselineProbability, 100*exc.Probability)
}

Output (deterministic — default N=10000, seed 1):

checkout.latency fit R^2 = 0.996
delta E[checkout.latency] = +76.52 ms +/- 4.7e-16
P(checkout.latency > 500ms): 2.8% -> 43.3%

Note the 4.7e-16: with linear mechanisms and common random numbers the paired delta is exact, not merely low-variance. The full example (examples/slo-impact, 8 nodes, 5000 fitted rows) ends:

intervention: do(db.latency += 30ms)   [Shift, N=10000 samples]
  E[checkout.latency]: 440.9 -> 513.8 ms   (delta +72.95 +/- 5.3e-16 ms)
  p99[checkout.latency]: 556.4 -> 629.3 ms

SLO: checkout.latency <= 500 ms
  P(breach) baseline:     11.2%  (+/- 0.32)
  P(breach) under shift:  60.8%  (+/- 0.49)
  verdict: the +30 ms db regression multiplies breach risk 5.4x

Quickstart B — root-cause analysis

Given a fitted *because.Model and one anomalous joint observation (a value for every node in the target's ancestral closure), rank the candidate root causes:

ranked, err := attrib.RankRootCauses(ctx, m, "checkout.latency", observation)
if err != nil {
	log.Fatal(err)
}
for i, rc := range ranked {
	fmt.Printf("#%d %-18s score=%6.3f share=%5.1f%%\n",
		i+1, rc.Node, rc.Score, 100*rc.Share)
}

examples/rca builds the same 8-node topology, manufactures an incident by propagating hand-picked noises (every node typical except the database at +6σ), and attributes checkout's anomaly. The trap RCA must avoid: payments.latency (174 ms vs typical 131) and checkout itself look anomalous too — but only because they faithfully propagated db's fault. Noise-based attribution exonerates them (go run ./examples/rca):

ranked root causes of the checkout.latency anomaly:
  #  node               score     +/-       share
  1  db.latency          3.781    0.007   100.0%
  2  cache.latency       0.001    0.001     0.0%
  3  gw.latency         -0.000    0.001     0.0%
  4  payments.latency   -0.007    0.001     0.0%
  5  checkout.latency   -0.036    0.003     0.0%

OK: the injected culprit db.latency is ranked #1

Counterfactual — would we have breached absent the deploy?

A deploy went out at 14:02; at 14:07 checkout p99 read 531 ms — an SLO breach. Abduction–action–prediction answers the after-incident question "would p99 have exceeded 500 ms absent the deploy?":

s, _ := because.NewStructure(
	because.Nodes{{ID: "deploy.checkout", Kind: because.Binary}},
	because.Edges{
		{From: "db.latency", To: "checkout.latency"},
		{From: "deploy.checkout", To: "checkout.latency"},
	})
// checkout's parents in canonical (lexicographic) order: [db, deploy].
m, err := because.NewModel(s, map[string]mech.Mechanism{
	"db.latency":       &mech.GaussianRoot{Mean: 90, Std: 12},
	"deploy.checkout":  &mech.BernoulliRoot{P: 0.05},
	"checkout.latency": &mech.LinearANM{Intercept: 60, Coefs: []float64{3.0, 90}, Noise: mech.GaussianNoise{Std: 25}},
})
if err != nil {
	log.Fatal(err)
}

// Everything we measured during the incident window:
evidence := map[string]float64{
	"db.latency":       110,
	"deploy.checkout":  1,
	"checkout.latency": 531,
}
cf, err := m.Counterfactual(ctx, evidence,
	[]because.Intervention{{Node: "deploy.checkout", Kind: because.Set, Value: 0}})
if err != nil {
	log.Fatal(err)
}
fmt.Printf("factual: 531 ms, counterfactual without deploy: %v ms\n",
	cf["checkout.latency"])

Output (deterministic — no sampling is involved):

factual: 531 ms, counterfactual without deploy: 441 ms

Reading: abduction recovers checkout's episode noise ε = 531 − (60 + 3.0·110 + 90·1) = +51 ms; replaying that same episode with do(deploy = 0) gives 60 + 330 + 0 + 51 = 441 ms — below the 500 ms SLO, so the breach would not have happened absent the deploy. Exact, not a Monte Carlo estimate: for linear ANMs the counterfactual is arithmetic.

The CLI

cmd/because exposes the whole pipeline to non-Go callers: every subcommand reads files and writes one deterministic JSON document to stdout. A round trip:

go build -o dist/because ./cmd/because

Topology can start as plain arrow text:

printf '%s\n' 'db.latency -> payments.latency' \
  'db.latency -> checkout.latency' \
  'payments.latency -> checkout.latency' > topology.arrows
dist/because validate --arrows topology.arrows
{
  "ok": true,
  "nodes": 3,
  "edges": 3,
  "model": false
}

Fit a model bundle from CSV telemetry (header row of node IDs, one column per node):

cat > telemetry.csv <<'EOF'
db.latency,payments.latency,checkout.latency
97.3,141.2,445.3
83.9,131.9,401.3
78.0,125.4,364.6
101.1,134.6,448.9
93.2,128.1,425.4
98.6,136.3,447.6
72.9,111.5,346.8
81.6,124.6,389.9
97.5,134.3,440.4
83.3,124.4,385.3
84.6,117.4,381.4
92.8,137.0,422.0
EOF
dist/because fit --arrows topology.arrows --data telemetry.csv --out model.json

Predict a Shift intervention's downstream effect (--iv grammar: node=v Set, node+=v Shift, node*=v Scale):

dist/because effect --model model.json --target checkout.latency --iv 'db.latency+=30'

The output is the full EffectResult as JSON — means, quantiles, the baseline twin, and the paired delta; note the machine-precision delta_mc_std_err (linear mechanisms + common random numbers):

{
  "target": "checkout.latency",
  "n": 10000,
  "mean": 522.0221228030274,
  ...
  "baseline_mean": 409.5147075105653,
  ...
  "delta_mean": 112.50741529246206,
  "delta_mc_std_err": 5.029850224591088e-16
}

Attribute an anomalous observation:

printf '{"db.latency": 138, "payments.latency": 174.2, "checkout.latency": 549.2}' > observation.json
dist/because attribute anomaly --model model.json --target checkout.latency --observation observation.json
{
  "target": "checkout.latency",
  "attributions": [
    {
      "node": "db.latency",
      "score": 7.7546801549097655,
      "std_err": 0.04593920041152513
    },
    ...
  ],
  "observed_score": 8.517393171418915,
  "baseline": 1.0105378521337574
}

Determinism is byte-for-byte: every sampling command takes --seed (default 1), and the same seed produces identical output bytes at any parallelism — running the same effect command twice with --seed 7 and diffing the two outputs yields no difference (verified). See because --help and the package documentation of cmd/because for the full output-schema and exit-code contract.

Package map

Package What lives there
because (root) Structure, Model, interventions, Do / Effect / Exceedance, Counterfactual, sentinel errors
spec JSON bundle + arrow-text serialization; normative format doc: spec/SPEC.md
mech Mechanism families: LinearANM, EmpiricalRoot, GaussianRoot, BernoulliRoot; noises
fit Fit / Refit from columnar data: OLS + empirical noise, Bernoulli MLE, custom fitters
infer Free-function facade over model queries (infer.Effect, infer.WithSeed, ...)
graphq Pure graph queries: d-separation, backdoor sets, Markov blankets, closures
attrib AttributeAnomaly, AttributeChange, RankRootCauses
cmd/because The CLI: validate, fit, sample, effect, exceedance, counterfactual, attribute, graph

Determinism contract. Every stochastic entry point takes a seed (default 1). Noise draws are keyed by (seed, node, sample index) — never by worker — so a fixed seed gives bit-identical results at any parallelism, with or without pruning. Parents are everywhere ordered lexicographically by node ID; all output orderings (edges, reports, rankings, JSON keys) are deterministic.

Performance. Queries prune to the target's ancestral closure (~10² nodes even in 10⁴-node graphs) and sample on dense structure-of-arrays buffers with a zero-allocation inner loop; see BENCHMARKS.md.

Development

. bin/activate-hermit   # pinned toolchain (or invoke ./bin/go directly)
task do                 # format + lint + test + build
task test               # go test ./...
task bench              # performance-contract benchmarks

About

Causal Inference primitives and functions

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages