docs/etsi-watchdog.html at main · etsi-ai/docs · GitHub

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="UTF-8">
  <title>etsi-watchdog Documentation</title>
  <link rel="stylesheet" href="style.css">
</head>
<body>
  <nav>
    <h1>etsi</h1>
    <ul>
      <li><a href="etsi-watchdog.html" class="active">etsi-watchdog</a></li>
      <li><a href="etsi-failprint.html">etsi-failprint</a></li>
      <li><a href="etsi-etna.html">etsi-etna</a></li>
    </ul>
  </nav>
  <main>
    <h2>etsi-watchdog</h2>
    <p><strong>etsi-watchdog</strong> is a lightweight Python library for detecting tabular data drift using statistical tests like PSI. It's modular, production-ready, and ideal for ML validation pipelines and scheduled monitoring tasks.</p>
    <br>
    <h3>Installation</h3>
    <pre><code>pip install etsi-watchdog</code></pre>

    <h3>Core Components</h3>
    <table>
      <thead>
        <tr>
          <th>Class</th>
          <th>Purpose</th>
        </tr>
      </thead>
      <tbody>
        <tr>
          <td><code>DriftCheck</code></td>
          <td>Compare reference vs. current dataset using PSI or other drift algorithms.</td>
        </tr>
        <tr>
          <td><code>Monitor</code></td>
          <td>Rolling monitoring system to detect drift over time with logging support.</td>
        </tr>
        <tr>
          <td><code>DriftComparator</code></td>
          <td>Compares drift results between two different data/model versions.</td>
        </tr>
      </tbody>
    </table>

    <h3>DriftCheck API</h3>
    <table>
      <thead>
        <tr>
          <th>Method</th>
          <th>Description</th>
        </tr>
      </thead>
      <tbody>
        <tr>
          <td><code>check.run(current_df, features)</code></td>
          <td>Runs drift detection on a list of features from <code>current_df</code> vs the reference.</td>
        </tr>
      </tbody>
    </table>

    <h3>Monitor API</h3>
    <table>
      <thead>
        <tr>
          <th>Method</th>
          <th>Description</th>
        </tr>
      </thead>
      <tbody>
        <tr>
          <td><code>monitor.enable_logging(path)</code></td>
          <td>Enable CSV-based logging of drift results to the specified path.</td>
        </tr>
        <tr>
          <td><code>monitor.watch_rolling(df, window, freq, features)</code></td>
          <td>Monitors drift over a time-indexed DataFrame. Aggregates and checks drift for each period.</td>
        </tr>
      </tbody>
    </table>

    <h3>DriftResult API</h3>
    <table>
      <thead>
        <tr>
          <th>Attribute / Method</th>
          <th>Description</th>
        </tr>
      </thead>
      <tbody>
        <tr>
          <td><code>.is_drifted</code></td>
          <td>Returns <code>True</code> if the score exceeds threshold.</td>
        </tr>
        <tr>
          <td><code>.summary()</code></td>
          <td>Prints a short summary like “Drift Detected” with score and method name.</td>
        </tr>
        <tr>
          <td><code>.plot()</code></td>
          <td>Renders a histogram plot of PSI or other scores (requires matplotlib).</td>
        </tr>
        <tr>
          <td><code>.to_json(path)</code></td>
          <td>Saves result as a structured JSON file. If no path is passed, returns a dict.</td>
        </tr>
      </tbody>
    </table>

    <h3>Example: One-time Check</h3>
    <pre><code>from etsi.watchdog import DriftCheck

check = DriftCheck(reference_df)
results = check.run(current_df, features=["age", "salary"])

for feature, result in results.items():
    print(result.summary())
    result.plot()
    result.to_json(f"logs/drift_{feature}.json")
</code></pre>

    <h3>Example: Scheduled Monitoring</h3>
    <pre><code>from etsi.watchdog import Monitor

monitor = Monitor(reference_df)
monitor.enable_logging("logs/rolling_log.csv")

monitor.watch_rolling(
    df=live_df,
    window=50,
    freq="D",
    features=["age", "salary"]
)
</code></pre>

    <h3>Example: Version Comparison</h3>
    <pre><code>from etsi.watchdog import DriftComparator

comp = DriftComparator(run1_results, run2_results)
diff = comp.diff()

for feature, delta in diff.items():
    print(f"{feature}: Δ PSI = {delta:+.4f}")
</code></pre>

    <h3>Resources</h3>
    <div class="resource-links">
    <a class="resource-link" href="https://pypi.org/project/etsi-watchdog/" target="_blank">PyPI Release</a>
    <a class="resource-link" href="" target="_blank">Google Colab Demo</a>
    <a class="resource-link" href="https://github.com/etsi-ai/etsi-watchdog" target="_blank">GitHub Repository</a>
    </div>

  </main>
</body>
</html>