Skip to content

sysfs: add SR-IOV VF PCI address and NUMA node helpers for net interfaces#795

Open
aharivel wants to merge 1 commit intoprometheus:masterfrom
aharivel:sriov
Open

sysfs: add SR-IOV VF PCI address and NUMA node helpers for net interfaces#795
aharivel wants to merge 1 commit intoprometheus:masterfrom
aharivel:sriov

Conversation

@aharivel
Copy link

Add two helpers to the sysfs package for SR-IOV Physical Function (PF) network interfaces:

  • NetClassVFPCIAddress(iface string, vfIndex uint32) (string, error)
    Resolves the PCI BDF address of a Virtual Function by reading the sysfs
    virtfn symlink at /sys/class/net/<iface>/device/virtfn<vfIndex>.
    Returns the BDF address e.g. 0000:65:01.0.

  • NetClassNumaNode(iface string) (int, error)
    Returns the NUMA node of the PCI device backing the interface, read from
    /sys/class/net/<iface>/device/numa_node.
    Returns -1 if the device is not NUMA-aware (kernel convention).

These are intended to support SR-IOV VF collectors (e.g. in node_exporter) that need to correlate VF traffic metrics with workloads referencing VFs by PCI address or NUMA topology (e.g. OpenStack Nova, libvirt, DPDK).

Test fixtures added under testdata/fixtures/sys/class/net/enp3s0f0/.

@SuperQ
Copy link
Member

SuperQ commented Mar 10, 2026

Why is this not part of net_class.go?

@aharivel
Copy link
Author

@SuperQ I followed the precedent of net_class_aer.go and net_class_ecn.go which group functionality by feature area in separate files. But re-reading those, they're justified by their size (100+ lines of complex parsing each) — our two functions are tiny by comparison and don't really warrant their own file.

You're right that net_class.go is the natural home. Both functions read attributes from the underlying PCI device of a net interface (device/numa_node, device/virtfn), which is exactly the kind of thing net_class.go already handles. I can move them there and drop net_class_sriov.go.

That said — would you prefer they stay grouped together (e.g. with a comment marking them as SR-IOV related), or just inline them with the rest of the net class helpers?

@SuperQ
Copy link
Member

SuperQ commented Mar 10, 2026

Hmm, looking into this a bit more. The /sys/class/net/<iface>/device path is just a symlink to the PCI device. We already support parsing numa_node in pci_device.go. Maybe what we should is have a function that returns the sysfs.PciDevice by calling parser.

@aharivel
Copy link
Author

Hmm ok cleaner approach — agreed. A NetClassPCIDevice(iface string) (*PCIDevice, error) that resolves the device symlink and reuses the existing PCI parser avoids duplicating numa_node logic. I'll rework it that way.

For virtfnX resolution, I'd read those symlinks from the resolved PCI device path.
Does that sound right to you?

Add two new functions to support SR-IOV network VF metrics collection:

- NetClassPCIDevice(iface string) (*PciDevice, error) in net_class.go:
  Resolves /sys/class/net/<iface>/device symlink and returns the backing
  PciDevice using the existing PCI device parser. This gives callers
  access to all PCI device properties (NumaNode, link speed, SR-IOV
  fields, etc.).

- PciDeviceVFAddress(device *PciDevice, vfIndex uint32) (string, error)
  in pci_device.go: Returns the PCI BDF address of a Virtual Function by
  resolving the virtfn symlink at /sys/bus/pci/devices/<bdf>/virtfn<n>.

Test fixtures added under testdata/fixtures/sys/class/net/enp3s0f0/
using the existing 0000:a2:00.0 PCI device fixture.

Signed-off-by: Anthony Harivel <aharivel@redhat.com>
@aharivel aharivel marked this pull request as draft March 10, 2026 15:17
@aharivel aharivel marked this pull request as ready for review March 11, 2026 11:50
@aharivel
Copy link
Author

aharivel commented Mar 11, 2026

@SuperQ Ok so I've reworked the patch in a way that takes your feedback into consideration.

I've also tested this morning the end goal of this patch (e.g adding SR-IOV network Virtual Function statistics in node_exporter prometheus/node_exporter#3544 ) on my server with E810 nic and the result looks as expected (i.e PCI address are good, numa node are also good):

~/node_exporter sriov-testing# curl -s http://localhost:9100/metrics | grep node_net_vf
# HELP node_net_vf_broadcast_packets_total Number of broadcast packets received by the VF.
# TYPE node_net_vf_broadcast_packets_total counter
node_net_vf_broadcast_packets_total{device="eno12399",numa_node="0",pci_address="0000:8a:01.0",vf="0"} 131390
node_net_vf_broadcast_packets_total{device="eno12399",numa_node="0",pci_address="0000:8a:01.1",vf="1"} 131774
# HELP node_net_vf_info Virtual Function configuration information.
# TYPE node_net_vf_info gauge
node_net_vf_info{device="eno12399",link_state="auto",mac="12:6d:66:5b:35:8a",numa_node="0",pci_address="0000:8a:01.0",spoof_check="true",trust="false",vf="0",vlan="0"} 1
node_net_vf_info{device="eno12399",link_state="auto",mac="ba:7c:63:25:b0:f4",numa_node="0",pci_address="0000:8a:01.1",spoof_check="true",trust="false",vf="1",vlan="0"} 1
# HELP node_net_vf_multicast_packets_total Number of multicast packets received by the VF.
# TYPE node_net_vf_multicast_packets_total counter
node_net_vf_multicast_packets_total{device="eno12399",numa_node="0",pci_address="0000:8a:01.0",vf="0"} 0
node_net_vf_multicast_packets_total{device="eno12399",numa_node="0",pci_address="0000:8a:01.1",vf="1"} 2547
# HELP node_net_vf_receive_bytes_total Number of received bytes by the VF.
# TYPE node_net_vf_receive_bytes_total counter
node_net_vf_receive_bytes_total{device="eno12399",numa_node="0",pci_address="0000:8a:01.0",vf="0"} 4.6839828e+07
node_net_vf_receive_bytes_total{device="eno12399",numa_node="0",pci_address="0000:8a:01.1",vf="1"} 4.718765e+07
# HELP node_net_vf_receive_dropped_total Number of dropped received packets by the VF.
# TYPE node_net_vf_receive_dropped_total counter
node_net_vf_receive_dropped_total{device="eno12399",numa_node="0",pci_address="0000:8a:01.0",vf="0"} 131390
node_net_vf_receive_dropped_total{device="eno12399",numa_node="0",pci_address="0000:8a:01.1",vf="1"} 0
# HELP node_net_vf_receive_packets_total Number of received packets by the VF.
# TYPE node_net_vf_receive_packets_total counter
node_net_vf_receive_packets_total{device="eno12399",numa_node="0",pci_address="0000:8a:01.0",vf="0"} 131390
node_net_vf_receive_packets_total{device="eno12399",numa_node="0",pci_address="0000:8a:01.1",vf="1"} 134321
# HELP node_net_vf_transmit_bytes_total Number of transmitted bytes by the VF.
# TYPE node_net_vf_transmit_bytes_total counter
node_net_vf_transmit_bytes_total{device="eno12399",numa_node="0",pci_address="0000:8a:01.0",vf="0"} 0
node_net_vf_transmit_bytes_total{device="eno12399",numa_node="0",pci_address="0000:8a:01.1",vf="1"} 0
# HELP node_net_vf_transmit_dropped_total Number of dropped transmitted packets by the VF.
# TYPE node_net_vf_transmit_dropped_total counter
node_net_vf_transmit_dropped_total{device="eno12399",numa_node="0",pci_address="0000:8a:01.0",vf="0"} 0
node_net_vf_transmit_dropped_total{device="eno12399",numa_node="0",pci_address="0000:8a:01.1",vf="1"} 0
# HELP node_net_vf_transmit_packets_total Number of transmitted packets by the VF.
# TYPE node_net_vf_transmit_packets_total counter
node_net_vf_transmit_packets_total{device="eno12399",numa_node="0",pci_address="0000:8a:01.0",vf="0"} 0
node_net_vf_transmit_packets_total{device="eno12399",numa_node="0",pci_address="0000:8a:01.1",vf="1"} 0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants