WIP on Rust based processors

[rjenkins]

No description provided.

[rjenkins]

  Async Rust Processors                                                                                                                                                                                                                                                                                                                        
                                                                                                                                                                                                                                                                                                                                               
  Writing a processor                                                                      

  Create a new Rust library crate:

  # Cargo.toml
  [package]
  name = "my_processor"
  version = "0.1.0"
  edition = "2021"

  [lib]
  crate-type = ["cdylib"]

  [dependencies]
  rotel_rust_processor_sdk = { path = "path/to/rotel/rotel_rust_processor_sdk" }

  Implement the AsyncProcessor trait with async fn methods:

  use rotel_rust_processor_sdk::prelude::*;
  use rotel_rust_processor_sdk::{export_async_processor, ROption};

  #[derive(Default)]
  pub struct MyProcessor;

  impl AsyncProcessor for MyProcessor {
      async fn process_spans(
          &self,
          mut spans: RResourceSpans,
          _context: ROption<RRequestContext>,
      ) -> RResourceSpans {
          // You can use .await here - e.g. HTTP calls, database lookups, etc.
          let enrichment = fetch_from_api().await;

          for scope_spans in spans.scope_spans.iter_mut() {
              for span in scope_spans.spans.iter_mut() {
                  span.attributes.push(RKeyValue::string("my.attribute", enrichment.as_str()));
              }
          }
          spans
      }
  }

  export_async_processor!(MyProcessor);

  The export_async_processor! macro handles tokio runtime creation and FFI bridging automatically — no need to manage a runtime yourself.

  Building

  cargo build --release
  # produces: target/release/libmy_processor.dylib (macOS) or libmy_processor.so (Linux)

  Running with Rotel

  rotel start \
    --async-rust-trace-processor ./target/release/libmy_processor.dylib \
    --debug-log traces \
    --debug-log-verbosity detailed \
    --exporter blackhole

  Send test traces:

  telemetrygen traces --otlp-insecure

  The debug log output will show your enrichment attributes on the spans.

  Async processors are always executed via spawn_blocking so they never block the pipeline's tokio worker threads. For processors that don't need async, use the sync RotelProcessor trait with export_processor! and --rust-trace-processor instead.

[rjenkins]

@brightsparc example output

./target/release/rotel start \                                                                                                                                                                                 ✔  2m 13s   15:23:29                                                                                                                                                                           
  --async-rust-trace-processor ./examples/rust_processors/async_enrichment_processor/target/debug/libasync_enrichment_processor.dylib \                                                                                                                                                                                                                                                                                                                         
 --debug-log traces \                                                                                                                                                                                                                                                                                                                                                                                                                                           
 --debug-log-verbosity detailed \                                                                                                                                                                                                                                                                                                                                                                                                                               
 --exporter blackhole                                                                                                                                                                                                                                                                                                                                                                                                                                           
2026-02-12T23:23:43.575481Z  INFO Starting Rotel.                                                                                                                                                                                                                                                                
2026-02-12T23:23:43.709851Z  INFO Loaded async Rust processor: libasync_enrichment_processor.dylib from ./examples/rust_processors/async_enrichment_processor/target/debug/libasync_enrichment_processor.dylib                                                                                                                                                                                             
2026-02-12T23:23:43.710037Z  INFO OTLP receiver listening grpc_endpoint="127.0.0.1:4317" http_endpoint="127.0.0.1:4318"                                                                                                                                                                                          
2026-02-12T23:23:47.930896Z  INFO OTLP payload before processing                                                                                                                                                                                                                                                 
ResourceSpans #0                                                                                                                                                                                                                                                                                                 
Resource SchemaURL: https://opentelemetry.io/schemas/1.4.0                                                                                                                                                                                                                                                       
Resource attributes:                                                                                                                                                                                                                                                                                             
     -> service.name: Str(telemetrygen)                                                                                                                                                                                                                                                                          
ScopeSpans #0                                                                                                                                                                                                                                                                                                    
ScopeSpans SchemaURL:                                                                                                                                                                                                                                                                                            
InstrumentationScope telemetrygen                                                                                                                                                                                                                                                                                
Span #0                                                                                                                                                                                                                                                                                                          
    Trace ID       : df0852ca7cf18ceb5d50486e3de8f732                                                                                                                                                                                                                                                            
    Parent ID      : 16b505310fe6eda4                                                                                                                                                                                                                                                                            
    ID             : d51f268246697ee6                                                                                                                                                                                                                                                                            
    Name           : okey-dokey-0                                                                                                                                                                                                                                                                                
    Kind           : SPAN_KIND_SERVER                                                                                                                                                                                                                                                                            
Start time: 1770938627927536000                                                                                                                                                                                                                                                                                  
End time: 1770938627927659000                                                                                                                                                                                                                                                                                    
    Status code    : STATUS_CODE_UNSET                                                                                                                                                                                                                                                                           
    Status message :                                                                                                                                                                                                                                                                                             
Attributes:                                                                                                                                                                                                                                                                                                      
     -> net.peer.ip: Str(1.2.3.4)                                                                                                                                                                                                                                                                                
     -> peer.service: Str(telemetrygen-client)                                                                                                                                                                                                                                                                   
Span #1                                                                                                                                                                                                                                                                                                          
    Trace ID       : df0852ca7cf18ceb5d50486e3de8f732                                                                                                                                                                                                                                                            
    Parent ID      :                                                                                                                                                                                                                                                                                             
    ID             : 16b505310fe6eda4                                                                                                                                                                                                                                                                            
    Name           : lets-go                                                                                                                                                                                                                                                                                     
    Kind           : SPAN_KIND_CLIENT                                                                                                                                                                                                                                                                            
Start time: 1770938627927536000                                                                                                                                                                                                                                                                                  
End time: 1770938627927659000                                                                                                                                                                                                                                                                                    
    Status code    : STATUS_CODE_UNSET                                                                                                                                                                                                                                                                           
    Status message :                                                                                                                                                                                                                                                                                             
Attributes:                                                                                                                                                                                                                                                                                                      
     -> net.peer.ip: Str(1.2.3.4)                                                                                                                                                                                                                                                                                
     -> peer.service: Str(telemetrygen-server)                                                                                                                                                                                                                                                                   
2026-02-12T23:23:47.932677Z  INFO OTLP payload after processing                                                                                                                                                                                                                                                  
ResourceSpans #0                                                                                                                                                                                                                                                                                                 
Resource SchemaURL: https://opentelemetry.io/schemas/1.4.0                                                                                                                                                                                                                                                       
Resource attributes:                                                                                                                                                                                                                                                                                             
     -> service.name: Str(telemetrygen)                                                                                                                                                                                                                                                                          
ScopeSpans #0                                                                                                                                                                                                                                                                                                    
ScopeSpans SchemaURL:                                                                                                                                                                                                                                                                                            
InstrumentationScope telemetrygen                                                                                                                                                                                                                                                                                
Span #0                                                                                                                                                                                                                                                                                                          
    Trace ID       : df0852ca7cf18ceb5d50486e3de8f732                                                                                                                                                                                                                                                            
    Parent ID      : 16b505310fe6eda4                                                                                                                                                                                                                                                                            
    ID             : d51f268246697ee6                                                                                                                                                                                                                                                                            
    Name           : okey-dokey-0                                                                                                                                                                                                                                                                                
    Kind           : SPAN_KIND_SERVER                                                                                                                                                                                                                                                                            
Start time: 1770938627927536000                                                                                                                                                                                                                                                                                  
End time: 1770938627927659000                                                                                                                                                                                                                                                                                    
    Status code    : STATUS_CODE_UNSET                                                                                                                                                                                                                                                                           
    Status message :                                                                                                                                                                                                                                                                                             
Attributes:                                                                                                                                                                                                                                                                                                      
     -> net.peer.ip: Str(1.2.3.4)                                                                                                                                                                                                                                                                                
     -> peer.service: Str(telemetrygen-client)                                                                                                                                                                                                                                                                   
     -> enrichment.source: Str(async-enriched)                                                                                                                                                                                                                                                                   
     -> rotel.async_processor: Str(enrichment)                                                                                                                                                                                                                                                                   
Span #1                                                                                                                                                                                                                                                                                 
    Trace ID       : df0852ca7cf18ceb5d50486e3de8f732                                                                                                                                                                                                                                   
    Parent ID      :                                                                                                                                                                                                                                                                    
    ID             : 16b505310fe6eda4                                                                                                                                                                                                                                                   
    Name           : lets-go                                                                                                                                                                                                                                                            
    Kind           : SPAN_KIND_CLIENT                                                                                                                                                                                                                                                   
Start time: 1770938627927536000                                                                                                                                                                                                                                                         
End time: 1770938627927659000                                                                                                                                                                                                                                                           
    Status code    : STATUS_CODE_UNSET                                                                                                                                                                                                                                                  
    Status message :                                                                                                                                                                                                                                                                    
Attributes:                                                                                                                                                                                                                                                                             
     -> net.peer.ip: Str(1.2.3.4)                                                                                                                                                                                                                                                       
     -> peer.service: Str(telemetrygen-server)                                      
     -> enrichment.source: Str(async-enriched)                                      
     -> rotel.async_processor: Str(enrichment)                                      
^C2026-02-12T23:23:54.382642Z  INFO Shutdown signal received.