Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 34 additions & 4 deletions .github/workflows/python-quality.yml
Original file line number Diff line number Diff line change
Expand Up @@ -470,11 +470,39 @@ jobs:
- uses: astral-sh/setup-uv@v6
- run: uv run --quiet --frozen --no-dev --project plugins/foundation/darrow-skill-authoring/skills/author-agent-skill/backend python plugins/foundation/darrow-skill-authoring/skills/author-agent-skill/backend/tests/fresh_install.py

observability-windows:
name: Observability Langfuse Python ${{ matrix.python-version }} on windows-latest
needs: changes
if: needs.changes.outputs.observability_langfuse == 'true'
strategy:
fail-fast: false
matrix:
python-version: ["3.10", "3.11", "3.12", "3.13"]
runs-on: windows-latest
defaults:
run:
shell: bash
env:
UV_PYTHON: ${{ matrix.python-version }}
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}
- uses: astral-sh/setup-uv@v6
with:
enable-cache: true
- run: scripts/check-python --package plugins/capability/darrow-observability-langfuse/backend

observability-fresh-install:
name: Fresh install observability-langfuse on ubuntu-latest
name: Fresh install observability-langfuse on ${{ matrix.os }}
needs: changes
if: needs.changes.outputs.observability_langfuse == 'true'
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
os: [ubuntu-latest, macos-latest, windows-latest]
runs-on: ${{ matrix.os }}
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
Expand Down Expand Up @@ -555,6 +583,7 @@ jobs:
- ticket-pipeline-fresh-install
- inventory
- skill-authoring-fresh-install
- observability-windows
- observability-fresh-install
- discovery-fresh-install
- verification-fresh-install
Expand All @@ -576,6 +605,7 @@ jobs:
VERIFICATION_WINDOWS_RESULT: ${{ needs.verification-windows.result }}
INVENTORY_RESULT: ${{ needs.inventory.result }}
SKILL_INSTALL_RESULT: ${{ needs.skill-authoring-fresh-install.result }}
OBSERVABILITY_WINDOWS_RESULT: ${{ needs.observability-windows.result }}
OBSERVABILITY_INSTALL_RESULT: ${{ needs.observability-fresh-install.result }}
DISCOVERY_INSTALL_RESULT: ${{ needs.discovery-fresh-install.result }}
VERIFICATION_INSTALL_RESULT: ${{ needs.verification-fresh-install.result }}
Expand Down Expand Up @@ -606,8 +636,8 @@ jobs:
"$CHANGES_RESULT" "$INVENTORY_RESULT" \
"$PACKAGE_SELECTED" "$PACKAGE_RESULT" \
"$SKILL_AUTHORING_CHANGED" "$WINDOWS_RESULT" "$SKILL_INSTALL_RESULT" \
"$OBSERVABILITY_CHANGED" "$OBSERVABILITY_INSTALL_RESULT" \
"$PERFORMANCE_RESULT" \
"$OBSERVABILITY_CHANGED" "$OBSERVABILITY_WINDOWS_RESULT" \
"$OBSERVABILITY_INSTALL_RESULT" "$PERFORMANCE_RESULT" \
"$DISCOVERY_CHANGED" "$DISCOVERY_WINDOWS_RESULT" \
"$DISCOVERY_INSTALL_RESULT" \
"$VERIFICATION_CHANGED" "$VERIFICATION_WINDOWS_RESULT" \
Expand Down
35 changes: 21 additions & 14 deletions docs/specs/observability-langfuse.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,10 +19,11 @@ configuration guidance, tests, evals, and documentation all live inside its
plugin directory. It does not reference another Darrow plugin or skill.

This capability is an explicit, issue-authorized exception to Darrow's default
portable-Bash plugin-runtime boundary. The deterministic launcher remains
portable Bash, but trace reconstruction and export use Python managed by UV.
The plugin declares and locks its Python dependencies and refuses safely when
UV or the managed environment is unavailable. This exception is local to this
portable-Bash plugin-runtime boundary. The deterministic host launcher uses
Bash on Linux and macOS and native Windows PowerShell on Windows; trace
reconstruction and export use Python managed by UV on every platform. The
plugin declares and locks its Python dependencies and refuses safely when UV
or the managed environment is unavailable. This exception is local to this
plugin and does not change the default boundary for other plugins.

## Runtime contract
Expand Down Expand Up @@ -123,9 +124,10 @@ already-final snapshots. Local unresolved turns retain redacted parser data
until an authoritative receipt allows privacy-filtered envelope materialization.

Local capture transactions and each session's single background drainer use
separate process locks. State updates are atomic and durable. Concurrent and
reordered hooks cannot overwrite another hook's capture or acknowledge work
they did not deliver. Network waits never hold the capture lock.
separate cross-process locks on Linux, macOS, and native Windows. State updates
are atomic and durable. Concurrent and reordered hooks cannot overwrite another
hook's capture or acknowledge work they did not deliver. Network waits never
hold the capture lock.

Every envelope has a stable identity, an expected observation count, a frozen
privacy-filtered trace document, and exactly one state: `pending`,
Expand Down Expand Up @@ -250,9 +252,11 @@ Installation documentation names the required Codex hook support, UV,
supported Python version, Langfuse server/SDK compatibility, configuration
files and variables, first-run dependency behavior, and verification command.
The hook registration resolves the packaged launcher through Codex's
`PLUGIN_ROOT` environment variable. The launcher then resolves its own plugin
root and uses the committed UV lock. It must not assume the source checkout
location or another plugin installation.
`PLUGIN_ROOT` environment variable and selects a native Windows command through
`commandWindows`. Each launcher then resolves its own plugin root and uses the
committed UV lock. Neither launcher may assume the source checkout location or
another plugin installation. Native Windows operation requires PowerShell, not
Bash, WSL, or a POSIX compatibility layer.

The hook exits successfully without export when tracing is disabled. Missing
UV, missing credentials, malformed hook input, unreadable transcript, invalid
Expand All @@ -269,10 +273,13 @@ export retry snapshots, prompt-time provisional attribution, interrupted-turn
continuity, missing-snapshot quarantine, epoch session segmentation, detached
and non-ticket Git state, rollout reconstruction, deduplication, content
privacy, malformed input, missing runtime or configuration, and exporter
failure. Backend checks run through UV. Portable hook-launcher tests run with
both supported Bash executables. The backend conforms to the repository-wide
[Python quality standard](python-quality.md), including separate 95% statement
and branch coverage gates on every supported Python and CI platform.
failure. Backend checks run through UV. Hook-launcher tests run with both
supported Bash executables on Unix and the registered PowerShell command on
native Windows. The backend conforms to the repository-wide [Python quality
standard](python-quality.md), including separate 95% statement and branch
coverage gates on every supported Python and CI platform. Fresh copied-artifact
verification exercises the registered hook command on Linux, macOS, and native
Windows.

Performance evidence distinguishes startup from steady-state foreground
capture, confirms network-independent foreground completion, bounded
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"name": "darrow-observability-langfuse",
"description": "Export Codex rollout turns to Langfuse with attribution epochs",
"version": "0.5.3",
"version": "0.6.0",
"license": "BUSL-1.1",
"author": {
"name": "Björn Rochel",
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "darrow-observability-langfuse",
"version": "0.5.3",
"version": "0.6.0",
"description": "Export Codex rollout turns to Langfuse with attribution epochs",
"author": {
"name": "Björn Rochel",
Expand Down
31 changes: 18 additions & 13 deletions plugins/capability/darrow-observability-langfuse/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,10 +15,11 @@ plugin. The hook never contacts or mutates a tracker.

## Runtime

`hooks/stop.sh` is a Unix Bash launcher, while
transcript reconstruction and Langfuse export run in Python managed by UV. The
plugin commits `backend/pyproject.toml` and `backend/uv.lock`; no sibling plugin
or repository runtime is required.
Codex selects `hooks/stop.sh` through Bash on Linux and macOS and
`hooks/stop.ps1` through Windows PowerShell on native Windows. Transcript
reconstruction and Langfuse export run in Python managed by UV on every
platform. The plugin commits `backend/pyproject.toml` and `backend/uv.lock`; no
sibling plugin or repository runtime is required.

To prepare the hook environment before its first execution, run from the plugin
root:
Expand All @@ -41,7 +42,7 @@ Codex session after installation. You can inspect the registered hooks with
`/hooks`. Check [hosts and prerequisites](#hosts-and-prerequisites) before
enabling export. Delivery uses the supported
OTLP traces endpoint and the v4 ingestion header. Codex must support native
asynchronous command hooks (verified with CLI 0.153.4).
asynchronous command hooks and `commandWindows` (verified with CLI 0.154.0).

## Configure

Expand Down Expand Up @@ -261,22 +262,26 @@ uv run --quiet --frozen --no-dev --project backend python backend/tests/fresh_in
bun run check:python
```

The hook tests invoke both `bash` and `/bin/bash` from pytest.
The repository command verifies the UV lock, formatting, lint, strict typing,
tests, property tests, and separate statement and branch coverage gates.
The hook tests invoke both `bash` and `/bin/bash` on Unix and Windows PowerShell
on native Windows. The copied-artifact check executes the registered host
command on its current platform. The repository command verifies the UV lock,
formatting, lint, strict typing, tests, property tests, and separate statement
and branch coverage gates.

## When to use

Configure or explain Codex turn telemetry, privacy, and attribution. Do not use it to monitor arbitrary applications, deploy Langfuse, or operate tickets.

## Hosts and prerequisites

Codex is the observed runtime. Export requires Codex async hooks,
Codex is the observed runtime. Linux, macOS, and native Windows are supported.
Export requires Codex async hooks and `commandWindows` support,
[UV and Python](https://github.com/BjRo/darrow/blob/main/docs/installing-plugins.md#uv-and-python-for-plugin-helpers),
and a compatible Langfuse v4 server or Langfuse Cloud. Claude Code turns are not exported.
The hook requires a Unix environment with Bash; the backend uses Unix file
locking. Native Windows support is tracked separately in
[#205](https://github.com/BjRo/darrow/issues/205).
and a compatible Langfuse v4 server or Langfuse Cloud. Linux and macOS require
Bash. Native Windows requires Windows PowerShell 5.1 or later and does not
require Bash, WSL, or a POSIX compatibility layer. Backend and sidecar locks
use the platform's native cross-process file locking. Claude Code turns are not
exported.
Claude installation and guidance invocation are unverified: Claude Code 2.1.223
rejects this package's Codex-specific `Interrupt` hook during native validation.
The presence of a Claude manifest is not a compatibility guarantee.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ version = "0.1.0"
description = "Codex rollout reconstruction and Langfuse export backend"
requires-python = ">=3.10,<3.14"
dependencies = [
"filelock>=3.20,<4",
"langfuse>=4.0,<5",
"opentelemetry-exporter-otlp-proto-http>=1.36,<2",
]
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,14 +2,12 @@

from __future__ import annotations

import fcntl
import hashlib
import json
import os
import sqlite3
import time
from collections.abc import Callable, Iterator
from contextlib import closing, contextmanager
from collections.abc import Callable
from contextlib import closing
from dataclasses import dataclass
from pathlib import Path
from typing import Any
Expand All @@ -18,6 +16,7 @@
from .config import Config
from .context import delivery_context, require_context
from .export import DeliveryError, export_document
from .locking import exclusive_lock


def await_capture(rollout: Path, turn_id: str, timeout: float = 45) -> bool:
Expand Down Expand Up @@ -58,7 +57,7 @@ def drain(
lock_path = (
lock_directory / f"{hashlib.sha256(session_id.encode()).hexdigest()}.lock"
)
with _exclusive_lock(lock_path) as acquired:
with exclusive_lock(lock_path, blocking=False) as acquired:
return _Drainer(rollout, config, exporter).run() if acquired else 0


Expand All @@ -76,20 +75,6 @@ def _session_id(rollout: Path, config: Config, cwd: str) -> str | None:
return str(json.loads(row[0])) if row is not None else None


@contextmanager
def _exclusive_lock(path: Path) -> Iterator[bool]:
descriptor = os.open(path, os.O_CREAT | os.O_RDWR, 0o600)
try:
try:
fcntl.flock(descriptor, fcntl.LOCK_EX | fcntl.LOCK_NB)
except BlockingIOError:
yield False
return
yield True
finally:
os.close(descriptor)


@dataclass(frozen=True)
class _Batch:
rows: list[sqlite3.Row]
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
"""Portable durability helpers for plugin-owned state."""

from __future__ import annotations

import os
from pathlib import Path


def sync_directory(path: Path) -> None:
if os.name == "nt": # pragma: no cover - Windows cannot open directories this way.
return
descriptor = os.open(path, os.O_RDONLY)
try:
os.fsync(descriptor)
finally:
os.close(descriptor)
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@

from .config import Config
from .context import delivery_context, require_context
from .filesystem import sync_directory
from .sidecar import _load_state_path, load_provisional_attribution_snapshots


Expand All @@ -31,11 +32,7 @@ def _atomic_record(path: Path, value: Any) -> None:
os.fsync(handle.fileno())
with suppress(FileExistsError):
os.link(temporary, path) # First receipt wins; never replace evidence.
directory = os.open(path.parent, os.O_RDONLY)
try:
os.fsync(directory)
finally:
os.close(directory)
sync_directory(path.parent)
finally:
os.unlink(temporary)

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
"""Cross-process file locks for capture sidecars and delivery drainers."""

from __future__ import annotations

from collections.abc import Iterator
from contextlib import contextmanager
from pathlib import Path

from filelock import FileLock, Timeout


@contextmanager
def exclusive_lock(path: Path, *, blocking: bool = True) -> Iterator[bool]:
lock = FileLock(path, timeout=-1 if blocking else 0)
try:
lock.acquire()
except Timeout:
yield False
return
try:
yield True
finally:
lock.release()
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
from __future__ import annotations

import fcntl
import hashlib
import json
import os
Expand All @@ -13,6 +12,8 @@
from typing import Any, ParamSpec, TypeVar, cast

from .config import validate_work_item_id
from .filesystem import sync_directory
from .locking import exclusive_lock

_CONTROL_CHARACTER = re.compile(r"[\x00-\x1f\x7f]")
_GIT_HEAD = re.compile(r"(?:[0-9a-f]{40}|[0-9a-f]{64})", re.I)
Expand All @@ -28,12 +29,9 @@ def decorate(function: Callable[P, R]) -> Callable[P, R]:
def invoke(*args: P.args, **kwargs: P.kwargs) -> R:
path = path_for(*args, **kwargs)
path.parent.mkdir(parents=True, exist_ok=True)
descriptor = os.open(f"{path}.lock", os.O_CREAT | os.O_RDWR, 0o600)
try:
fcntl.flock(descriptor, fcntl.LOCK_EX)
with exclusive_lock(Path(f"{path}.lock")) as acquired:
assert acquired
return function(*args, **kwargs)
finally:
os.close(descriptor)

return invoke

Expand Down Expand Up @@ -171,11 +169,7 @@ def _write_state_path(path: Path, state: dict[str, Any]) -> None:
os.fsync(handle.fileno())
os.chmod(temporary, 0o600)
os.replace(temporary, path)
directory = os.open(path.parent, os.O_RDONLY)
try:
os.fsync(directory)
finally:
os.close(directory)
sync_directory(path.parent)
temporary = None
finally:
if temporary is not None:
Expand Down
Loading
Loading