Bug:[Smartswitch] Data applied through gnmi configuration to the DPU is inacessible from DPU database

[gpunathilell]

Is it platform specific

generic

Importance or Severity

Critical

Description of the bug

This issue is seen because the orchagent on DPU is subscribed to the APPL_DB dash entries which is transferred to the DPU over ZMQ, and the gnmi server on the switch applies the data to the APPL_DB in the databasedpun docker which is inaccessible on the DPU. All the dash related configuration applied on the DPU is not visible on the DPU, and is stored in the APPL_DB present in the databasedpun docker on the switch, which is not erased on config reload on DPU as well.

Steps to Reproduce

Apply DASH configuration using a gnmi configuration tool
Observe DASH configuration from switch console:

root@sonic:/home/admin# docker exec -it databasedpu0 bash
root@sonic:/# sonic-db-cli APPL_DB keys "*DASH*"
DASH_ROUTE_GROUP_TABLE:RouteGroup2
DASH_ROUTE_TABLE:RouteGroup2:10.0.0.0/16
DASH_TUNNEL_TABLE:exgw_tunnel_1

Confirm configuration is not seen on DPU:
From DPU console:

root@sonic:/home/admin# sonic-db-cli APPL_DB keys "*DASH*"
root@sonic:/home/admin#

DB data is available on switch in the databasedpun dockers, but not in the DPU itself, it is also not available in DPU_APPL_DB

This doesnt affect orchagent code in DPU, since this obtains the configuration over ZMQ

Actual Behavior and Expected Behavior

Actual behaviour:
DASH configuration available only on switch
Expected Behaviour:
DASH configuration available only on DPU on any of the DBs

Relevant log output

Output of show version, show techsupport

Attach files (if any)

No response

[KrisNey-MSFT]

Hi @prsunny and @gpunathilell - is this a P0?

I don't think so at this point. It can be a P1.

[prabhataravind]

@gpunathilell Data not being on dpu APPL_DB is by design. You can access DPU_APPL_DB on NPU using the following command from the DPU assuming we use DPU_APPL_DB to program objects via gNMI.

sudo sonic-db-cli DPU_APPL_DB keys *

This should connect to the correct remote database.

We do need to fix the issue where DPU databases on NPU are not reset when DPU crashes or reloads. @liuh-80 @ganglyu for viz on this ask.

[gpunathilell]

@prabhataravind it is not visible in DPU_APPL_DB either, After application of config to DPU0, this is the view from the switch console:

root@sonic:/home/admin# docker exec -it databasedpu0 sonic-db-cli APPL_DB keys *
DASH_APPLIANCE_TABLE:100
DASH_METER_RULE_TABLE:MeterPolicyV4:1
DASH_ROUTE_GROUP_TABLE:RouteGroup1
DASH_ENI_TABLE:497f23d7-f0ac-4c99-a98f-59b470e8c7bd
DASH_ROUTE_RULE_TABLE:497f23d7-f0ac-4c99-a98f-59b470e8c7bd:100:101.1.2.3/32
DASH_ROUTE_TABLE:RouteGroup1:10.0.0.0/16
DASH_ROUTE_TABLE:RouteGroup1:10.2.0.0/16
DASH_ENI_ROUTE_TABLE:497f23d7-f0ac-4c99-a98f-59b470e8c7bd
DASH_METER_POLICY_TABLE:MeterPolicyV4
DASH_VNET_TABLE:Vnet1
DASH_METER_RULE_TABLE:MeterPolicyV4:2
DASH_ROUTE_RULE_TABLE:497f23d7-f0ac-4c99-a98f-59b470e8c7bd:4321:25.1.1.1/32
DASH_TUNNEL_TABLE:Tunnel2
DASH_VNET_MAPPING_TABLE:Vnet1:10.2.0.100
DASH_VNET_TABLE:Vnet2
DASH_ROUTING_TYPE_TABLE:privatelink
root@sonic:/home/admin# docker exec -it databasedpu0 sonic-db-cli DPU_APPL_DB keys *

root@sonic:/home/admin#

And this is from the DPU console:

root@sonic:/home/admin# sonic-db-cli APPL_DB keys "*DASH*"

root@sonic:/home/admin# sonic-db-cli DPU_APPL_DB keys "*DASH*"

root@sonic:/home/admin# sonic-db-cli DPU_APPL_DB keys "*"

root@sonic:/home/admin#

As you can see, the data is written to the APPL_DB in redis instance in databasedpu0 docker on the switch, which is not reachable from DPU, and no data is written to the DPU_APPL_DB at all

[prabhataravind]

@gpunathilell can you share the gNMI API you use? What is the path you give?

[prabhataravind]

Working on a potential fix to use DPU_APPL_DB instead of APPL_DB for all DASH object programming.

[KrisNey-MSFT]

Fix in progress (Prabhat). Good to have for PoC