PyETWkit

A modern, high-performance ETW (Event Tracing for Windows) toolkit for Python, powered by a Rust backend.

---

Features

Core

• Real-time ETW streaming with sync API
• Kernel providers: process, thread, registry, file, disk, network
• User providers: DNS, Audio, and more via profiles
• ETL file reading: Parse existing trace logs
• Full property decoding via TDH: arrays, nested structures, SIDs, and WPP events given a PDB or .tmf — see Event Properties
• Rust backend (pyo3): High throughput, zero-copy event delivery
• Windows 10 / 11 / Server supported

v2.0 - Enterprise Features

• Multi-session support: Run multiple ETW sessions simultaneously
• Manifest-based typed events: Parse ETW manifests for structured event data
• Rust-side filtering: High-performance event filtering in Rust
• Provider discovery: Search and list available providers
• Pre-configured profiles: Audio, network, security scenarios

v3.0 - Advanced Analysis

• Live Dashboard: Browser-based real-time visualization with Gradio
• Event Correlation Engine: Auto-correlate events by PID/TID/Handle
• Recording & Replay: Capture and replay ETW sessions (.etwpack format)
• OpenTelemetry Exporter: Send events to an OTLP collector over HTTP (Jaeger, Grafana, Datadog), or write spans to a file. No extra dependency

Export Formats

• CSV, JSON, JSONL, Parquet, Arrow

---

Installation

pip install pyetwkit

# Optional: Dashboard support
pip install pyetwkit[dashboard]

# Optional: Export to Parquet/Arrow
pip install pyetwkit[export]

---

Quick Start

CLI Usage

# List available providers
pyetwkit providers
pyetwkit providers --search Kernel

# List profiles
pyetwkit profiles

# Listen to events (requires admin)
pyetwkit listen Microsoft-Windows-DNS-Client
pyetwkit listen --profile network

# Launch live dashboard (requires admin)
pyetwkit dashboard Microsoft-Windows-Kernel-Process
pyetwkit dashboard --profile network --port 8080

# Export ETL file
pyetwkit export trace.etl -o events.csv
pyetwkit export trace.etl -o events.parquet -f parquet

Python API

from pyetwkit._core import EtwProvider, EtwSession

# Create session
session = EtwSession("MySession")

# Add provider
provider = EtwProvider(
    "Microsoft-Windows-DNS-Client",
    "DNS-Client"
)
provider = provider.level(4)  # Info level
session.add_provider(provider)

# Start and process events
session.start()

try:
    while True:
        event = session.next_event_timeout(1000)
        if event:
            print(f"Event {event.event_id}: {event.provider_name}")
except KeyboardInterrupt:
    pass
finally:
    session.stop()

Live Dashboard

from pyetwkit import Dashboard

# Create and launch dashboard
dashboard = Dashboard(port=7860)
dashboard.add_provider("Microsoft-Windows-Kernel-Process")
dashboard.add_provider("Microsoft-Windows-DNS-Client")

# Opens browser at http://localhost:7860
dashboard.launch()

Event Correlation

from pyetwkit import CorrelationEngine

# Create correlation engine
engine = CorrelationEngine()
engine.add_provider("Microsoft-Windows-Kernel-Process")
engine.add_provider("Microsoft-Windows-Kernel-Network")

# Add events from your ETW session
for event in events:
    engine.add_event(event)

# Correlate events by process ID
correlated = engine.correlate_by_pid(1234)
for event in correlated:
    print(f"Event {event.event_id} from {event.provider_name}")

# Export to timeline JSON
timeline = engine.to_timeline_json(pid=1234)

Recording & Replay

from pyetwkit import Recorder, Player, CompressionType, RecorderConfig

# Record events
config = RecorderConfig(compression=CompressionType.ZSTD)
recorder = Recorder("session.etwpack", config=config)
recorder.add_provider("Microsoft-Windows-DNS-Client")
recorder.start()

# ... capture events ...
recorder.stop()

# Replay events
player = Player("session.etwpack")
print(f"Duration: {player.duration:.2f}s, Events: {player.event_count}")

for event in player.events():
    print(f"Event {event['event_id']}")

OpenTelemetry Export

Spans are sent as OTLP/HTTP with JSON encoding, so no extra dependency is needed. Note 4318 — 4317 is the gRPC port and will not answer HTTP.

from pyetwkit import OtlpExporter, SpanMapper

# Map ETW events to spans
mapper = SpanMapper()
mapper.add_rule(
    provider="Microsoft-Windows-Kernel-Process",
    event_id=1,
    span_name="process.start",
    attributes=["ProcessID", "ImageName"],
)

exporter = OtlpExporter(
    endpoint="http://collector:4318",
    service_name="my-service",
    resource_attributes={"deployment.environment": "production"},
    span_mapper=mapper,
)

for event in events:
    exporter.export(event)

# False means nothing was delivered; the batch is kept so it can be retried.
if not exporter.flush():
    log.warning("OTLP export failed; see the log for the reason")

To write spans to a file instead, for a collector agent to pick up:

from pyetwkit import OtlpFileExporter

exporter = OtlpFileExporter("traces.json", service_name="my-service")
for event in events:
    exporter.export(event)
exporter.flush()

Kernel Tracing

from pyetwkit._core import PyKernelFlags, PyKernelSession

flags = PyKernelFlags()
flags = flags.with_process()  # Enable process events

session = PyKernelSession(flags)
session.start()

for _ in range(10):
    event = session.next_event_timeout(1000)
    if event and event.event_id == 1:  # Process start
        props = event.to_dict().get("properties", {})
        print(f"Process: {props.get('ImageFileName')}")

session.stop()

Provider Discovery

from pyetwkit._core import list_providers, search_providers

# List all providers
for p in list_providers()[:10]:
    print(f"{p.name}: {p.guid}")

# Search by name
for p in search_providers("Kernel"):
    print(p.name)

Export Events

from pyetwkit._core import EtlReader
from pyetwkit.export import to_csv, to_parquet

# Read ETL file
reader = EtlReader("trace.etl")
events = list(reader.events())

# Export to various formats
to_csv(events, "events.csv")
to_parquet(events, "events.parquet")

---

Architecture

Python API / CLI
  ↓
pyetwkit (Python package)
  ↓
pyetwkit._core (Rust/pyo3)
  ↓
ferrisetw (Rust ETW library)
  ↓
Windows ETW subsystem

---

Documentation

• Tutorial - Comprehensive usage guide
• API Reference - Detailed API documentation
• Examples - Sample scripts
• Architecture - Design documents
• Event Properties - How values are decoded, display strings, raw payloads, and WPP

---

Changelog

v3.2.0 (2026-09)

• OpenTelemetry export actually sends (#90). Spans are POSTed to /v1/traces as OTLP/HTTP with JSON encoding, using only the standard library — no new dependency. flush() returns False and logs the reason on failure, keeping the batch so events can be retried rather than lost
• Fixed: the OTLP JSON encoding sent enum names, which a collector rejects. The spec allows integers only, so kind and status.code are now numbers
• Fixed: event_to_span() crashed on every real event. EtwEvent.timestamp is an RFC 3339 string and the code called float() on it; only mocks and plain numbers had ever been passed to it

Note the OTLP/HTTP port is 4318. The 4317 in earlier examples is the gRPC port and will not answer an HTTP request.

v3.1.0 (2026-09)

Event properties are now decoded from the schema via TDH, rather than guessed from a list of twelve names. See Event Properties.

• All properties decoded: whatever the provider declares, not a guess list (#72, #76)
• Arrays and nested structures: lists and dicts, instead of only the first element (#84)
• WPP events: decoded given a .pdb or .tmf — set_wpp_pdb_path() needs no SDK tooling (#72)
• Undecodable events keep their payload in raw_data instead of losing it
• formatted_properties: TDH's own display strings, opt-in, with value maps resolved
• Correct SIDs, win:Boolean, pointer widths from 32-bit processes, and counted strings
• pip install . works — it never had (#79)
• OtlpExporter no longer reports success while discarding events; its transport was never implemented (#88). OtlpFileExporter is unaffected

Note: v3.0.2 was tagged and released on GitHub but never reached PyPI, so this is the first release since v3.0.1 that PyPI users will see.

v3.0.0 (2024-12)

• Live Dashboard: Gradio-based real-time UI (pyetwkit dashboard CLI)
• Event Correlation Engine: Link events by PID/TID/Handle with timeline export
• Recording & Replay: Capture sessions to .etwpack format with compression
• OpenTelemetry Exporter: Export to OTLP endpoints (Jaeger, Grafana, etc.) — announced here, but the transport was not actually implemented until v3.2.0 (#88, #90)

v2.0.0 (2024-12)

• Multi-session support: Run multiple ETW sessions simultaneously
• Manifest-based typed events: Parse ETW provider manifests
• Rust-side filtering: High-performance filtering with RustEventFilter
• Enhanced CLI: Provider profiles, export options

v1.0.0 (2024-12)

• Initial release
• Real-time ETW streaming
• Kernel and user-mode providers
• ETL file reading
• Export to CSV, JSON, JSONL, Parquet, Arrow
• CLI tool with provider discovery

---

Examples

See the examples/ directory for complete sample scripts:

• basic_session.py - Simple ETW session
• kernel_trace.py - Kernel-level process monitoring
• export_events.py - Capture and export events
• provider_discovery.py - Find ETW providers
• profiles.py - Use pre-configured profiles
• read_etl.py - Read ETL files
• demo_v2_features.py - v2.0 features demo
• demo_v3_features.py - v3.0 features demo

---

Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

1. Fork the repository
2. Create your feature branch (git checkout -b feature/amazing-feature)
3. Commit your changes (git commit -m 'Add some amazing feature')
4. Push to the branch (git push origin feature/amazing-feature)
5. Open a Pull Request

---

License

MIT

---

Author

m96-chan