fix: harden ATOF HTTP endpoint export#295
Conversation
Adds an endpoint-local field_name_policy option for ATOF streaming exporters. The default preserves canonical ATOF JSON, while replace_dots recursively rewrites dotted JSON object keys to underscores for HTTP receivers that reject dotted field names. This also fixes ATOF HTTP streaming robustness by installing the rustls crypto provider before endpoint workers create HTTP clients, preserving the endpoint config in the NDJSON worker after spawning the upload task, and logging bounded non-2xx response bodies so rejected events are diagnosable. Exposes the new option through plugin config and Rust, Python, Node, and Go bindings, with tests for validation, serialization, recursive key rewriting, and collision handling. Signed-off-by: Bryan Bednarski <bbednarski@nvidia.com>
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: ASSERTIVE Plan: Enterprise Run ID: 📒 Files selected for processing (6)
📜 Recent review details⏰ Context from checks skipped due to timeout. (2)
🧰 Additional context used📓 Path-based instructions (29)go/nemo_relay/**/*.go📄 CodeRabbit inference engine (.agents/skills/add-binding-feature/SKILL.md)
Files:
go/**/*.go📄 CodeRabbit inference engine (.agents/skills/add-binding-feature/SKILL.md)
Files:
{crates/adaptive/**,python/nemo_relay/plugin.py,go/nemo_relay/**,**/node/**,**/wasm/**}📄 CodeRabbit inference engine (.agents/skills/maintain-optimizer/SKILL.md)
Files:
{crates/adaptive/**/*.rs,**/*test*.{rs,py,go,ts,js},**/*adaptive*test*.{rs,py,go,ts,js},docs/plugins/adaptive/**}📄 CodeRabbit inference engine (.agents/skills/maintain-optimizer/SKILL.md)
Files:
{go/nemo_relay/go.mod,go/**/*.go}📄 CodeRabbit inference engine (.agents/skills/maintain-packaging/SKILL.md)
Files:
**/*.go📄 CodeRabbit inference engine (.agents/skills/rename-surfaces/SKILL.md)
Files:
**/*.{rs,py,js,ts,tsx,jsx,go,sh,toml,yaml,yml,md}📄 CodeRabbit inference engine (AGENTS.md)
Files:
**/*.{rs,py,go,js,ts,tsx}📄 CodeRabbit inference engine (AGENTS.md)
Files:
**⚙️ CodeRabbit configuration file
Files:
go/nemo_relay/**/*⚙️ CodeRabbit configuration file
Files:
{crates/**/tests/**,python/tests/**,go/nemo_relay/**/*_test.go}⚙️ CodeRabbit configuration file
Files:
{crates/adaptive/**,python/nemo_relay/adaptive.py,python/nemo_relay/plugin.py,go/nemo_relay/adaptive/**,go/nemo_relay/!(adaptive)/**,**/node/**,**/wasm/**}📄 CodeRabbit inference engine (.agents/skills/maintain-optimizer/SKILL.md)
Files:
crates/{python,ffi,node,wasm}/**/*⚙️ CodeRabbit configuration file
Files:
**/test_*.{py,py}📄 CodeRabbit inference engine (.agents/skills/add-integration/SKILL.md)
Files:
{pyproject.toml,**/*.py}📄 CodeRabbit inference engine (.agents/skills/maintain-packaging/SKILL.md)
Files:
**/*.{py,txt,toml,cfg,yaml,yml}📄 CodeRabbit inference engine (.agents/skills/rename-surfaces/SKILL.md)
Files:
python/**/*test*.py📄 CodeRabbit inference engine (.agents/skills/test-python-binding/SKILL.md)
Files:
python/**/{conftest.py,*test*.py}📄 CodeRabbit inference engine (.agents/skills/test-python-binding/SKILL.md)
Files:
**/*.py📄 CodeRabbit inference engine (.agents/skills/validate-change/SKILL.md)
Files:
**/*.{md,mdx,py,sh,yaml,yml,toml,json}📄 CodeRabbit inference engine (.agents/skills/contribute-docs/SKILL.md)
Files:
**/*.rs📄 CodeRabbit inference engine (.agents/skills/add-binding-feature/SKILL.md)
Files:
**/{Cargo.toml,**/*.rs}📄 CodeRabbit inference engine (.agents/skills/maintain-packaging/SKILL.md)
Files:
**/*.{h,hpp,c,cpp,rs}📄 CodeRabbit inference engine (.agents/skills/maintain-packaging/SKILL.md)
Files:
{crates/core,crates/adaptive}/**/*📄 CodeRabbit inference engine (.agents/skills/prepare-pr/SKILL.md)
Files:
**/*.{rs,toml}📄 CodeRabbit inference engine (.agents/skills/rename-surfaces/SKILL.md)
Files:
crates/core/**/*.rs📄 CodeRabbit inference engine (.agents/skills/test-go-binding/SKILL.md)
Files:
crates/{core,adaptive}/**📄 CodeRabbit inference engine (.agents/skills/validate-change/SKILL.md)
Files:
crates/**/*.rs📄 CodeRabbit inference engine (AGENTS.md)
Files:
crates/{core,adaptive}/**/*.rs⚙️ CodeRabbit configuration file
Files:
🔇 Additional comments (6)
WalkthroughAdds ChangesAtofEndpointFieldNamePolicy end-to-end
Sequence Diagram(s)sequenceDiagram
participant Caller as Caller (any binding)
participant PluginComponent as plugin_component
participant AtofWorker as AtofEndpointWorker
participant EEJ as endpoint_event_json
participant Transport as HTTP/WS/NDJSON
Caller->>PluginComponent: field_name_policy = "replace_dots"
PluginComponent->>PluginComponent: AtofEndpointFieldNamePolicy::parse("replace_dots")
PluginComponent->>AtofWorker: AtofEndpointConfig { field_name_policy: ReplaceDots }
AtofWorker->>AtofWorker: install_rustls_crypto_provider()
loop per event
AtofWorker->>EEJ: raw_json + ReplaceDots policy
EEJ->>EEJ: recursively rewrite dot-keys → underscore-keys
EEJ-->>AtofWorker: transformed JSON
AtofWorker->>Transport: send transformed payload
Transport-->>AtofWorker: HTTP response
alt non-2xx
AtofWorker->>AtofWorker: log_http_error(status, truncated_body)
end
end
Estimated code review effort🎯 4 (Complex) | ⏱️ ~60 minutes 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Warning There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure. 🔧 golangci-lint (2.12.2)level=error msg="[linters_context] typechecking error: pattern ./...: directory prefix . does not contain main module or its selected dependencies" Comment |
Signed-off-by: Bryan Bednarski <bbednarski@nvidia.com>
|
/merge |
Overview
Adds an endpoint-local field-name policy for ATOF streaming exporters and hardens HTTP endpoint delivery failure handling.
Details
field_name_policy = "preserve" | "replace_dots"to ATOF endpoint config, defaulting topreserveso canonical ATOF JSON remains unchanged unless configured.replace_dotsrecursively to endpoint payload JSON object keys for HTTP POST, WebSocket, and NDJSON endpoint transports while keeping local JSONL file output canonical.Where should the reviewer start?
Start with
crates/core/src/observability/atof.rsfor the endpoint payload transformation, HTTP error logging, and runtime robustness fixes. Then reviewcrates/core/src/observability/plugin_component.rsfor config parsing and validation.Validation run:
cargo test -p nemo-relay --features atof-streaming observability::atof::testscargo test -p nemo-relay --features atof-streaming observability::plugin_component::testsuv run pytest python/tests/test_observability_plugin.py::TestObservabilityConfigHelpers::test_atof_endpoint_config_serializes_streaming_fields python/tests/test_types.py::TestAtofExporterType::test_config_defaults_mutation_and_reprnode --test --test-name-pattern="serializes ATOF streaming endpoints" tests/observability_plugin_tests.mjsGOCACHE=/private/tmp/nemo-relay-atof-http-field-policy/.tmp/go-build-cache go test -run 'Test(NewAtofExporterConfigDefaults|ObservabilityConfigHelpers)$' ./...uv run ruff check python/nemo_relay/observability.py python/tests/test_observability_plugin.py python/tests/test_types.pygit diff --checkRelated Issues: (use one of the action keywords Closes / Fixes / Resolves / Relates to)
Summary by CodeRabbit
field_name_policyfor ATOF streaming endpoints (preservedefault, orreplace_dotsto rewrite JSON object keys by replacing.with_, including nested structures).