Skip to content

StreamSubscriber merges initial PROTO encoding SAMPLE updates until sync_response, losing origin/prefix boundaries #177

Description

@aryaarun-work

When using pygnmi StreamSubscriber with STREAM + SAMPLE subscriptions and PROTO encoding for OpenConfig paths, the initial SubscribeResponse messages received before sync_response are coalesced into a single update by the StreamSubscriber implementation.

This behavior occurs in StreamSubscriber._next_update(), where the first call merges all updates until a sync_response is observed via _get_updates_till_sync(), even though they were sent by the gNMI server as distinct SubscribeResponse.update messages with different prefixes.

This merging causes loss of prefix and origin boundaries in the parsed output and makes it difficult to correctly associate updates with their original OpenConfig paths, especially when automation logic depends on precise path semantics.


Relevant code path

class StreamSubscriber(_Subscriber):
    """
    The first time next() is called, updates from the target are coalesced
    until a message with the sync_response field is seen, and the update
    is returned.
    """
    def _next_update(self, timeout):
        if not self._first_update_seen:
            self._first_update_seen = True
            return self._get_updates_till_sync(timeout=timeout)
        else:
            return self._get_one_update(timeout=timeout)

Example subscription

{
  "subscription": [
    {
      "path": "openconfig-system:system/ntp/state/enabled",
      "mode": "SAMPLE",
      "sample_interval": 10000000000
    },
    {
      "path": "openconfig-system:system/state/hostname",
      "mode": "SAMPLE",
      "sample_interval": 10000000000
    },
    {
      "path": "openconfig-system:system/ntp/state/enable-ntp-auth",
      "mode": "SAMPLE",
      "sample_interval": 10000000000
    }
  ],
  "mode": "STREAM",
  "encoding": "PROTO"
}

Raw gNMI server responses (PROTO)

The gNMI server sends multiple SubscribeResponse.update messages with distinct prefixes before sync_response:

update {
  timestamp: 1769756715350817637
  prefix {
    origin: "openconfig-system"
    elem {
      name: "system"
    }
    elem {
      name: "ntp"
    }
    elem {
      name: "state"
    }
  }
  update {
    path {
      elem {
        name: "enabled"
      }
    }
    val {
      bool_val: false
    }
  }
  update {
    path {
      elem {
        name: "enable-ntp-auth"
      }
    }
    val {
      bool_val: false
    }
  }
}

update {
  timestamp: 1769756715351283414
  prefix {
    origin: "openconfig-system"
    elem {
      name: "system"
    }
    elem {
      name: "ntp"
    }
    elem {
      name: "state"
    }
  }
  update {
    path {
      elem {
        name: "enabled"
      }
    }
    val {
      bool_val: false
    }
  }
  update {
    path {
      elem {
        name: "enable-ntp-auth"
      }
    }
    val {
      bool_val: false
    }
  }
}

update {
  timestamp: 1769756715377899950
  prefix {
    origin: "openconfig-system"
    elem {
      name: "system"
    }
    elem {
      name: "state"
    }
  }
  update {
    path {
      elem {
        name: "hostname"
      }
    }
    val {
      string_val: "ios"
    }
  }
}

sync_response: true

Observed behavior

StreamSubscriber merges these responses into a single parsed update prior to sync_response, flattening updates across different prefixes and OpenConfig containers.

As a result:

  • Updates from different OpenConfig paths lose their original prefix separation

Expected behavior

  • Each SubscribeResponse.update should preserve its prefix and origin until explicitly combined by the client
  • Initial sync responses should not merge updates across different prefixes without maintaining per-prefix boundaries
  • Alternatively, provide an option to disable coalescing for PROTO encoding so consumers can process raw updates as received

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions