Skip to content

intermittent gRPC "channel closed!" error when no data is returned from device #170

Description

@lucasalvatore

pygnmi version = 0.8.15
Python version = 3.10.12
device = nokia 7750 (sros 23.10)

When subscribing once (the only way SROS will work afaik) if a particular path does not return any data, i'm seeing an intermittent gRPC error

in this example ports 1/1/c27/1 and 1/1/c28/1 do not exist on the device

subscribe_request = {'subscription': [{'path': 'state/port[port-id=1/1/c27/1]/oper-state/'},
  {'path': 'state/port[port-id=1/1/c28/1]/oper-state/'}],
 'mode': 'ONCE',
 'encoding': 'json'}

for _ in range(10):
     try:
         with gNMIclient(target=host, username="foo", password=grpc_password, insecure=True, timeout=20) as client:
             responses = list(client.subscribe2(subscribe_request))
     except Exception as e:
        print(e)
     print("SUCCESS", responses)
     print()

when running this loop, sometimes it works, sometimes it throws the error below:

SUCCESS [{'sync_response': True}]

Exception in thread Thread-194 (enqueue_updates):
Traceback (most recent call last):
  File "/usr/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.10/threading.py", line 953, in run
    self._target(*self._args, **self._kwargs)
  File "/home/lsalvatore/venvs/python3/lib/python3.10/site-packages/pygnmi/client.py", line 1080, in enqueue_updates
    raise error
  File "/home/lsalvatore/venvs/python3/lib/python3.10/site-packages/pygnmi/client.py", line 1076, in enqueue_updates
    for update in subscription:
  File "/home/lsalvatore/venvs/python3/lib/python3.10/site-packages/grpc/_channel.py", line 543, in __next__
    return self._next()
  File "/home/lsalvatore/venvs/python3/lib/python3.10/site-packages/grpc/_channel.py", line 969, in _next
    raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
	status = StatusCode.CANCELLED
	details = "Channel closed!"
	debug_error_string = "UNKNOWN:Error received from peer  {created_time:"2025-04-16T11:32:30.797396803-04:00", grpc_status:1, grpc_message:"Channel closed!"}"
>
SUCCESS [{'sync_response': True}]

SUCCESS [{'sync_response': True}]

Exception in thread Thread-208 (enqueue_updates):
Traceback (most recent call last):
  File "/usr/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.10/threading.py", line 953, in run
    self._target(*self._args, **self._kwargs)
  File "/home/lsalvatore/venvs/python3/lib/python3.10/site-packages/pygnmi/client.py", line 1080, in enqueue_updates
    raise error
  File "/home/lsalvatore/venvs/python3/lib/python3.10/site-packages/pygnmi/client.py", line 1076, in enqueue_updates
    for update in subscription:
  File "/home/lsalvatore/venvs/python3/lib/python3.10/site-packages/grpc/_channel.py", line 543, in __next__
    return self._next()
  File "/home/lsalvatore/venvs/python3/lib/python3.10/site-packages/grpc/_channel.py", line 969, in _next
    raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
	status = StatusCode.CANCELLED
	details = "Channel closed!"
	debug_error_string = "UNKNOWN:Error received from peer  {created_time:"2025-04-16T11:32:31.968167337-04:00", grpc_status:1, grpc_message:"Channel closed!"}"
>
SUCCESS [{'sync_response': True}]

I never see this when asking for ports that do exist.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions