Skip to content

FROMLIST: Output debug information from RSC - #907

Merged
Salendarsingh Gaud (sgaud-quic) merged 3 commits into
qualcomm-linux:qcom-6.18.yfrom
maulik-k-shah:qcom-6.18.y
Aug 20, 2026
Merged

FROMLIST: Output debug information from RSC#907
Salendarsingh Gaud (sgaud-quic) merged 3 commits into
qualcomm-linux:qcom-6.18.yfrom
maulik-k-shah:qcom-6.18.y

Conversation

@maulik-k-shah

@maulik-k-shah maulik-k-shah commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

Add https://lore.kernel.org/linux-arm-msm/20260717-rpmh-timeout-debug-v1-v2-0-81ade4fcdb49@oss.qualcomm.com/

RPMh transfer timeouts are hard to debug — the only indication today
is a WARN_ON() with no record of which TCS was stuck, what resource
it was voting for, or whether AOSS firmware or completion IRQ handling
delay caused the hang.

This series adds structured diagnostics that fire at timeout.

When a timeout occurs today, the only kernel output is a bare warning:

WARNING: drivers/soc/qcom/rpmh.c:386 rpmh_write_batch+0x190/0x2b0
Workqueue: events_unbound deferred_probe_work_func
Call trace:
rpmh_write_batch+0x190/0x2b0 (P)
qcom_icc_bcm_voter_commit+0x33c/0x500
qcom_icc_set+0x20/0x34
icc_node_add+0xf8/0x118
qcom_icc_rpmh_probe+0x194/0x540
platform_probe+0x5c/0xa4

This gives no indication of which TCS was stuck, what resource it was
voting for, or whether AOSS firmware or Linux itself caused the hang.

Patch 1 adds cmd_db_hw_type_str() to cmd-db to decode the accelerator
type (ARC/VRM/BCM) from an RPMh resource address using the existing
SLAVE_ID() encoding. This lives in cmd-db because SLAVE_ID() is a
private macro there and the address encoding is cmd-db's domain.

Patch 2 adds cmd_db_read_name() to cmd-db to reverse-look up the
human-readable resource name (e.g. cx.lvl) from an RPMh address. For
VRM resources, which have up to 4 contiguous addresses per resource,
the match uses VRM_ADDR() on bits [19:4] so any sub-address resolves
to the same name.

Patch 3 adds rpmh_rsc_debug() to rpmh-rsc.c and wires it into both
rpmh_write() and rpmh_write_batch() timeout paths. Per-command output
now shows the accelerator type, resource name, whether the command
requires a response, and the decoded TCS command status bits sourced
from the CMD_STATUS_{TRIGGERED,ISSUED,COMPL} definitions:
addr=0x30000(ARC/cx.lvl) resp-required sts=triggered+sent-to-aoss+resp-received.

CRs-fixed: 4635299

@maulik-k-shah
FROMLIST: soc: qcom: cmd-db: export RPMh accelerator type string helper
946390e 
RPMh resource addresses encode the accelerator type (ARC, VRM, BCM)
in bits [19:16], matching the cmd_db_hw_type enum values. Add
cmd_db_hw_type_str() to map an RPMh address to its accelerator type
name string using the existing SLAVE_ID() macro, and export it so
drivers that handle raw RPMh addresses can produce human-readable
diagnostic output without duplicating the address encoding knowledge.

Link: https://lore.kernel.org/linux-arm-msm/20260717-rpmh-timeout-debug-v1-v2-1-81ade4fcdb49@oss.qualcomm.com/
Assisted-by: Claude:claude-sonnet-4-5
Signed-off-by: Maulik Shah <maulik.shah@oss.qualcomm.com>
@maulik-k-shah
FROMLIST: soc: qcom: cmd-db: add reverse address-to-name lookup
b98e95a 
RPMh resource addresses are opaque 32-bit values. While the slave ID
in bits [19:16] identifies the accelerator type (ARC/VRM/BCM), the
lower bits encode a resource index that is only meaningful when mapped
back to the human-readable resource name stored in the command DB
(e.g. 0x30000 -> cx.lvl).

Add cmd_db_read_name() to perform this reverse lookup by iterating
the command DB entries and matching on address. Unlike other exported
cmd-db APIs which go through cmd_db_get_header() (which calls
cmd_db_ready() internally), this function iterates cmd_db_header
directly for address matching, so it calls cmd_db_ready() itself.

For VRM resources,
which have up to 4 contiguous 4-byte-aligned addresses per resource,
the match uses VRM_ADDR() on bits [19:4] so that any sub-address
(enable, voltage, mode, headroom) resolves to the same resource name.

Also export CMD_DB_ID_SIZE so callers can size their name buffers
correctly without open-coding the magic constant 8.

Link: https://lore.kernel.org/linux-arm-msm/20260717-rpmh-timeout-debug-v1-v2-2-81ade4fcdb49@oss.qualcomm.com/
Assisted-by: Claude:claude-sonnet-4-5
Signed-off-by: Maulik Shah <maulik.shah@oss.qualcomm.com>
@qswat-orbit-external

qswat-orbit-external Bot commented Aug 6, 2026

Copy link
Copy Markdown

Merge Check Failed: No Change Task Found

No associated change tasks found for CR 4635299 on any of the following entities:

Entities:

  • kernel.qli.2.0

CR: 4635299

Please ensure the CR has a change task associated with at least one of the entities for this branch.

Raju P.L.S.S.S.N @maulik-k-shah
FROMLIST: soc: qcom: rpmh-rsc: Output debug information from RSC
403d2a6 
When an RPMh transfer times out there is no visibility into which
TCS was stuck, what commands it was carrying, or whether the
completion IRQ was pending at the GIC. Add rpmh_rsc_debug() to
capture this state at timeout:

- Iterates all in-use TCSes and dumps per-TCS control register
  state (controller status, AMC mode, IRQ status) and per-command
  register state (address, data, message ID, completion status).
  The accelerator type (ARC/VRM/BCM) is decoded from the address
  using cmd_db_hw_type_str().

- Queries the GIC pending state for the RSC IRQ via
  irq_get_irqchip_state() to distinguish two failure modes:
  * AOSS firmware did not respond (IRQ never fired)
  * Linux lockup (IRQ pending at GIC but handler never ran)

- Reports the completion object state to cross-check with the
  IRQ status.

Store the IRQ number in struct rsc_drv to enable the GIC query.

Link: https://lore.kernel.org/linux-arm-msm/20260717-rpmh-timeout-debug-v1-v2-3-81ade4fcdb49@oss.qualcomm.com/
Signed-off-by: Raju P.L.S.S.S.N <rplsssn@codeaurora.org>
Co-developed-by: Maulik Shah <maulik.shah@oss.qualcomm.com>
Signed-off-by: Maulik Shah <maulik.shah@oss.qualcomm.com>
@qcomlnxci

qcomlnxci commented Aug 6, 2026

Copy link
Copy Markdown

Test jobs on 403d2a6

@qcomlnxci

qcomlnxci commented Aug 6, 2026

Copy link
Copy Markdown

Test Matrix

Test Case hamoa-iot-evk-multimedia lemans-evk-multimedia monaco-evk-multimedia qcs615-ride-multimedia qcs6490-rb3gen2-multimedia qcs8300-ride-multimedia qcs9100-ride-r3-multimedia shikra-iqs-evk-multimedia
Audio_Card_Registration ✅ Pass ✅ Pass ✅ Pass ⚠️ skip ✅ Pass ⚠️ skip ⚠️ skip ◻️
BT_FW_KMD_Service ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ◻️
BT_ON_OFF ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ◻️
BT_SCAN ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ◻️
CPUFreq_Validation ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ◻️
CPU_affinity ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ◻️
DSP_AudioPD ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ◻️
Ethernet ⚠️ skip ⚠️ skip ⚠️ skip ⚠️ skip ⚠️ skip ⚠️ skip ⚠️ skip ◻️
Freq_Scaling ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ◻️
GIC ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ◻️
IPA ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ◻️
Interrupts ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ◻️
OpenCV ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ◻️
PCIe ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ◻️
Probe_Failure_Check ❌ Fail ❌ Fail ❌ Fail ❌ Fail ❌ Fail ❌ Fail ❌ Fail ◻️
RMNET ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ◻️
UFS_Validation ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ◻️
USBHost ✅ Pass ✅ Pass ✅ Pass ❌ Fail ❌ Fail ❌ Fail ❌ Fail ◻️
WiFi_Firmware_Driver ❌ Fail ✅ Pass ❌ Fail ✅ Pass ✅ Pass ✅ Pass ✅ Pass ◻️
WiFi_OnOff ❌ Fail ✅ Pass ❌ Fail ✅ Pass ✅ Pass ✅ Pass ✅ Pass ◻️
adsp_remoteproc ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ◻️
cdsp_remoteproc ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ◻️
gpdsp_remoteproc ⚠️ skip ✅ Pass ✅ Pass ⚠️ skip ⚠️ skip ✅ Pass ✅ Pass ◻️
hotplug ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ◻️
irq ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ◻️
kaslr ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ◻️
pinctrl ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ◻️
qcom_hwrng ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ◻️
remoteproc ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ◻️
rngtest ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ◻️
shmbridge ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ◻️
smmu ❌ Fail ❌ Fail ✅ Pass ❌ Fail ✅ Pass ✅ Pass ❌ Fail ◻️
watchdog ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ◻️
wpss_remoteproc ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ◻️

@qcomlnxci
qcomlnxci requested a review from a team August 11, 2026 11:09
@qcomlnxci

qcomlnxci commented Aug 11, 2026

Copy link
Copy Markdown

Test jobs on f6c52fb

@qcomlnxci

qcomlnxci commented Aug 11, 2026

Copy link
Copy Markdown

Test Matrix

Test Case hamoa-iot-evk-multimedia lemans-evk-multimedia monaco-evk-multimedia qcs615-ride-multimedia qcs6490-rb3gen2-multimedia qcs8300-ride-multimedia qcs9100-ride-r3-multimedia shikra-iqs-evk-multimedia
Audio_Card_Registration ✅ Pass ◻️ ✅ Pass ⚠️ skip ✅ Pass ⚠️ skip ◻️ ◻️
BT_FW_KMD_Service ✅ Pass ◻️ ✅ Pass ✅ Pass ✅ Pass ✅ Pass ◻️ ◻️
BT_ON_OFF ✅ Pass ◻️ ✅ Pass ✅ Pass ✅ Pass ✅ Pass ◻️ ◻️
BT_SCAN ✅ Pass ◻️ ✅ Pass ✅ Pass ✅ Pass ✅ Pass ◻️ ◻️
CPUFreq_Validation ✅ Pass ◻️ ✅ Pass ✅ Pass ✅ Pass ✅ Pass ◻️ ◻️
CPU_affinity ✅ Pass ◻️ ✅ Pass ✅ Pass ✅ Pass ✅ Pass ◻️ ◻️
DSP_AudioPD ✅ Pass ◻️ ✅ Pass ✅ Pass ✅ Pass ✅ Pass ◻️ ◻️
Ethernet ⚠️ skip ◻️ ⚠️ skip ⚠️ skip ⚠️ skip ⚠️ skip ◻️ ◻️
Freq_Scaling ✅ Pass ◻️ ✅ Pass ✅ Pass ✅ Pass ✅ Pass ◻️ ◻️
GIC ✅ Pass ◻️ ✅ Pass ✅ Pass ✅ Pass ✅ Pass ◻️ ◻️
IPA ✅ Pass ◻️ ✅ Pass ✅ Pass ✅ Pass ✅ Pass ◻️ ◻️
Interrupts ✅ Pass ◻️ ✅ Pass ✅ Pass ✅ Pass ✅ Pass ◻️ ◻️
OpenCV ✅ Pass ◻️ ✅ Pass ✅ Pass ✅ Pass ✅ Pass ◻️ ◻️
PCIe ✅ Pass ◻️ ✅ Pass ✅ Pass ✅ Pass ✅ Pass ◻️ ◻️
Probe_Failure_Check ❌ Fail ◻️ ❌ Fail ❌ Fail ❌ Fail ❌ Fail ◻️ ◻️
RMNET ✅ Pass ◻️ ✅ Pass ✅ Pass ✅ Pass ✅ Pass ◻️ ◻️
UFS_Validation ✅ Pass ◻️ ✅ Pass ✅ Pass ✅ Pass ✅ Pass ◻️ ◻️
USBHost ✅ Pass ◻️ ✅ Pass ✅ Pass ❌ Fail ❌ Fail ◻️ ◻️
WiFi_Firmware_Driver ❌ Fail ◻️ ❌ Fail ✅ Pass ✅ Pass ✅ Pass ◻️ ◻️
WiFi_OnOff ❌ Fail ◻️ ❌ Fail ✅ Pass ✅ Pass ✅ Pass ◻️ ◻️
adsp_remoteproc ✅ Pass ◻️ ✅ Pass ✅ Pass ✅ Pass ✅ Pass ◻️ ◻️
cdsp_remoteproc ✅ Pass ◻️ ✅ Pass ✅ Pass ✅ Pass ✅ Pass ◻️ ◻️
gpdsp_remoteproc ⚠️ skip ◻️ ✅ Pass ⚠️ skip ⚠️ skip ✅ Pass ◻️ ◻️
hotplug ✅ Pass ◻️ ✅ Pass ✅ Pass ✅ Pass ✅ Pass ◻️ ◻️
irq ✅ Pass ◻️ ✅ Pass ✅ Pass ✅ Pass ✅ Pass ◻️ ◻️
kaslr ✅ Pass ◻️ ✅ Pass ✅ Pass ✅ Pass ✅ Pass ◻️ ◻️
pinctrl ✅ Pass ◻️ ✅ Pass ✅ Pass ✅ Pass ✅ Pass ◻️ ◻️
qcom_hwrng ✅ Pass ◻️ ✅ Pass ✅ Pass ✅ Pass ✅ Pass ◻️ ◻️
remoteproc ✅ Pass ◻️ ✅ Pass ✅ Pass ✅ Pass ✅ Pass ◻️ ◻️
rngtest ✅ Pass ◻️ ✅ Pass ✅ Pass ✅ Pass ✅ Pass ◻️ ◻️
shmbridge ✅ Pass ◻️ ✅ Pass ✅ Pass ✅ Pass ✅ Pass ◻️ ◻️
smmu ❌ Fail ◻️ ✅ Pass ❌ Fail ✅ Pass ✅ Pass ◻️ ◻️
watchdog ✅ Pass ◻️ ✅ Pass ✅ Pass ✅ Pass ✅ Pass ◻️ ◻️
wpss_remoteproc ✅ Pass ◻️ ✅ Pass ✅ Pass ✅ Pass ✅ Pass ◻️ ◻️

@qcomlnxci

qcomlnxci commented Aug 19, 2026

Copy link
Copy Markdown

Test jobs on 403d2a6

@qcomlnxci

qcomlnxci commented Aug 19, 2026

Copy link
Copy Markdown

Test Matrix

Test Case hamoa-iot-evk-multimedia lemans-evk-multimedia monaco-evk-multimedia purwa-iot-evk-multimedia qcs615-ride-multimedia qcs6490-rb3gen2-multimedia qcs8300-ride-multimedia qcs9100-ride-r3-multimedia shikra-iqs-evk-multimedia
Audio_Card_Registration ✅ Pass ✅ Pass ✅ Pass ✅ Pass ⚠️ skip ✅ Pass ⚠️ skip ⚠️ skip ⚠️ skip
BT_FW_KMD_Service ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass
BT_ON_OFF ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass
BT_SCAN ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ❌ Fail ✅ Pass ✅ Pass ✅ Pass
CPUFreq_Validation ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass
CPU_affinity ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass
DSP_AudioPD ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ⚠️ skip
Ethernet ⚠️ skip ✅ Pass ✅ Pass ⚠️ skip ⚠️ skip ⚠️ skip ⚠️ skip ⚠️ skip ⚠️ skip
Freq_Scaling ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass
GIC ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ❌ Fail
IPA ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass
Interrupts ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass
KVM_Driver ❌ Fail ✅ Pass ✅ Pass ❌ Fail ❌ Fail ❌ Fail ❌ Fail ✅ Pass ❌ Fail
KVM_EL2_DTB ❌ Fail ✅ Pass ✅ Pass ❌ Fail ❌ Fail ❌ Fail ❌ Fail ✅ Pass ❌ Fail
KVM_Infra ❌ Fail ✅ Pass ✅ Pass ❌ Fail ❌ Fail ❌ Fail ❌ Fail ✅ Pass ❌ Fail
OpenCV ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass
PCIe ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass
Probe_Failure_Check ❌ Fail ❌ Fail ❌ Fail ❌ Fail ❌ Fail ❌ Fail ❌ Fail ❌ Fail ❌ Fail
RMNET ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass
UFS_Validation ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ⚠️ skip
USBHost ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ❌ Fail ❌ Fail ❌ Fail ❌ Fail
WiFi_Firmware_Driver ✅ Pass ✅ Pass ❌ Fail ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ⚠️ skip
WiFi_OnOff ✅ Pass ✅ Pass ❌ Fail ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ⚠️ skip
adsp_remoteproc ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ⚠️ skip
cdsp_remoteproc ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass
gpdsp_remoteproc ⚠️ skip ✅ Pass ✅ Pass ⚠️ skip ⚠️ skip ⚠️ skip ✅ Pass ✅ Pass ⚠️ skip
hotplug ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass
irq ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass
kaslr ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass
pinctrl ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass
qcom_hwrng ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ◻️
rngtest ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass
shmbridge ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass
smmu ❌ Fail ❌ Fail ✅ Pass ❌ Fail ❌ Fail ✅ Pass ✅ Pass ❌ Fail ✅ Pass
watchdog ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass
wpss_remoteproc ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass ✅ Pass

@sgaud-quic
Salendarsingh Gaud (sgaud-quic) merged commit 4937e1d into qualcomm-linux:qcom-6.18.y Aug 20, 2026
7 of 10 checks passed
@qlijarvis

qlijarvis commented Aug 22, 2026

Copy link
Copy Markdown

LAVA Failed Case Triage Summary

PR: #907

Job 207904 | SoC shikra-iqs-evk

LAVA job: https://lava-oss.qualcomm.com/scheduler/job/207904

Failed test cases in LAVA job 207904 (SoC: shikra-iqs-evk).

  Case 1: GIC (Test Script Bug)
  1. Failed case: GIC (Test Script Bug)
  2. Root cause: The GIC test script is hardcoded to validate 8 CPUs (0-7), but the shikra-iqs-evk platform has only 4 CPUs (0-3). When the script parses /proc/interrupts for non-existent CPUs 4-7, it reads beyond the numeric interrupt count columns and attempts to parse text fields ("GICv3", "Level", "arch_timer") as integers, causing bash comparison errors ([: GICv3: integer expected) and false test failures.
  3. Possible fix: Update the GIC test script to dynamically detect the number of CPUs from /sys/devices/system/cpu/possible or nproc, and only validate interrupt counters for CPUs that actually exist on the platform. This is a test infrastructure bug, not a kernel regression introduced by PR FROMLIST: Output debug information from RSC #907.
  4. Detail analysis attachment: failed_case_job207904_1_detailed.md
  Case 2: Probe_Failure_Check
  1. Failed case: Probe_Failure_Check
  2. Root cause: Could not be determined confidently from available logs.
  3. Possible fix: Mark test as expected-fail for shikra-iqs-evk until platform DT and firmware are corrected. For CoreSight: verify ETM DT nodes have valid arm,coresight-loses-context-with-cpu and CPU affinity properties. For regulatory.db: add wireless-regdb package to rootfs. For cpufreq-dt: investigate duplicate cpufreq driver registration in platform code.
  4. Detail analysis attachment: failed_case_job207904_2_detailed.md
  Case 3: USBHost
  1. Failed case: USBHost
  2. Root cause: USB host controller driver (dwc3/xhci) did not probe during kernel boot on shikra-iqs-evk; USB core subsystem initialized successfully (usbcore, usbfs, hub drivers registered at boot time ~2.7s) but no dwc3 or xhci controller driver probe messages appear in dmesg, and USB device node 4e00000.usb was added to IOMMU group 5 but never bound to a host controller driver, resulting in zero enumerated USB devices when the test script ran.
  3. Possible fix: Verify USB host controller driver (CONFIG_USB_DWC3, CONFIG_USB_XHCI_HCD, CONFIG_USB_DWC3_QCOM) is enabled in kernel config and built-in or loaded as module; check device tree for shikra-iqs-evk to confirm usb@4e00000 node has correct compatible string, clocks, resets, and PHY phandles; if driver is modular, ensure it loads during boot via initramfs or systemd; if hardware-dependent, confirm USB host port is populated and powered on shikra-iqs-evk board variant used in LAVA lab.
  4. Detail analysis attachment: failed_case_job207904_3_detailed.md
  Case 4: KVM_Driver
  1. Failed case: KVM_Driver
  2. Root cause: KVM initialization failed because hypervisor mode (EL2) is not available on the Shikra IQS EVK platform — kernel log shows "kvm [1]: HYP mode not available" at boot time, preventing /dev/kvm device node creation.
  3. Possible fix: This is a platform hardware/firmware limitation, not a kernel regression. The Shikra IQS EVK does not support EL2/hypervisor mode, which is a prerequisite for KVM. Either: (1) skip KVM tests on this platform in the CI configuration, or (2) use a different platform that supports virtualization extensions for KVM testing.
  4. Detail analysis attachment: failed_case_job207904_4_detailed.md
  Case 5: KVM_EL2_DTB
  1. Failed case: KVM_EL2_DTB
  2. Root cause: KVM cannot initialize on Shikra IQS EVK because Gunyah hypervisor owns EL2 (hypervisor mode). Only one hypervisor can control EL2 at a time; Gunyah boots first and prevents KVM from accessing EL2, resulting in "HYP mode not available" and no /dev/kvm device node.
  3. Possible fix: This is a platform configuration issue, not a kernel bug. To enable KVM testing on Shikra IQS EVK: (1) disable Gunyah hypervisor in the board firmware/bootloader configuration, or (2) exclude KVM tests from the CI test suite for boards that run Gunyah, or (3) use a different board without Gunyah for KVM validation. The PR patch (cmd-db changes) is unrelated and does not cause this failure.
  4. Detail analysis attachment: failed_case_job207904_5_detailed.md
  Case 6: KVM_Infra — KVM device node unavailable (configuration/platform issue, not a crash)
  1. Failed case: KVM_Infra — KVM device node unavailable (configuration/platform issue, not a crash)
  2. Root cause: /dev/kvm device node not created despite CONFIG_KVM being enabled; KVM driver did not initialize, likely because the Shikra IQS EVK platform does not support ARM Virtualization Extensions (EL2) or EL2 is not enabled in firmware/bootloader, preventing KVM from creating the device node.
  3. Possible fix: This is a platform limitation, not a kernel bug. If KVM support is required on Shikra IQS EVK: (1) verify the SoC supports ARM Virtualization Extensions, (2) ensure the bootloader/firmware enables EL2 mode, (3) check for KVM initialization errors in early boot logs (may require enabling KVM debug), (4) if the platform does not support virtualization, mark the KVM_Infra test as "skip" for this board in the LAVA test definition.
  4. Detail analysis attachment: failed_case_job207904_6_detailed.md
  Case 7: Kernel Crash — Synchronous External Abort in qcom_rng driver
  1. Failed case: Kernel Crash — Synchronous External Abort in qcom_rng driver
  2. Root cause: The qcom_rng driver triggered a synchronous external abort (0x96000010) at PC qcom_rng_read+0xc4 when attempting to read from the hardware RNG MMIO register (instruction: b940035c = ldr w28, [x26, #0]). This indicates the hardware RNG block was either not powered/clocked correctly, not mapped correctly in the device tree, or the MMIO region was inaccessible due to a platform-specific power/clock/interconnect dependency failure on shikra-iqs-evk. The abort caused the board to reset, and the subsequent reboot stalled in bootloader (XBLRamDump/EDL path), preventing LAVA from recovering within the 2400-second test timeout.
  3. Possible fix: Investigate the qcom_rng device tree node for shikra-iqs-evk: verify the MMIO reg property matches the hardware specification, confirm all required clocks/regulators/interconnects are present and correctly named, and ensure the RNG power domain is enabled before driver probe. Add runtime PM calls or explicit clock/regulator enable in the qcom_rng probe path if the hardware requires it on this SoC. If the issue is reproducible, add debug instrumentation to qcom_rng_read to log MMIO base address and verify readl_relaxed succeeds before accessing hardware registers. Check if the RNG block requires explicit firmware initialization or secure-world enablement on shikra that is missing in the current device tree or driver probe sequence.
  4. Detail analysis attachment: failed_case_job207904_7_detailed.md
  Case 8: Kernel Crash — synchronous external abort in qcom_rng driver
  1. Failed case: Kernel Crash — synchronous external abort in qcom_rng driver
  2. Root cause: Hardware synchronous external abort (0x96000010) triggered during MMIO read in qcom_rng_read() at offset +0xc4 when the qcom_hwrng test attempted to read entropy from /dev/hwrng. The crash occurred at instruction ldr w28, [x26, #12] (encoded as b940035c), indicating an invalid or unmapped hardware register access. After the panic, the board entered EDL/ramdump mode and never returned to normal operation, causing LAVA to timeout after 40 minutes waiting for test completion.
  3. Possible fix: Verify the qcom_rng device tree node for shikra-iqs-evk has correct reg property mapping and that the PRNG hardware block is powered and clocked before driver probe. Check if the PR introduces any clock/power domain changes affecting the PRNG block. If the hardware is not present or not functional on this SoC variant, the qcom_rng driver should be disabled in the device tree or the test should be skipped for this platform.
  4. Detail analysis attachment: failed_case_job207904_8_detailed.md
  Case 9: lava-test-retry — Test Timeout After Kernel Crash
  1. Failed case: lava-test-retry — Test Timeout After Kernel Crash
  2. Root cause: Kernel panic triggered by synchronous external abort (0x96000010) in qcom_rng_read+0xc4 during hardware RNG access at timestamp [705.137283]. The crash occurred while the qcom_hwrng test was reading from /dev/hwrng. After panic, the system entered warm reset and EDL/ramdump mode (as evidenced by XBLRamDump logs at 18:23:37), preventing the LAVA test shell from resuming, which caused the 2400-second (40-minute) timeout.
  3. Possible fix: This is a pre-existing hardware/firmware issue in the qcom_rng driver on shikra-iqs-evk, not introduced by PR FROMLIST: Output debug information from RSC #907 (which only modifies cmd-db and rpmh code). The synchronous external abort indicates a bus-level fault when accessing the RNG hardware registers. Recommended actions: (1) Skip the qcom_hwrng test on shikra-iqs-evk until the RNG hardware access issue is root-caused and fixed; (2) Verify RNG hardware power/clock configuration in device tree and firmware; (3) Check if RNG MMIO region is correctly mapped and accessible; (4) Re-trigger the CI job to confirm other tests pass.
  4. Detail analysis attachment: failed_case_job207904_9_detailed.md
  Case 10: Kernel Crash — synchronous external abort in qcom_rng driver
  1. Failed case: Kernel Crash — synchronous external abort in qcom_rng driver
  2. Root cause: Hardware access fault in qcom_rng_read() at offset +0xc4 when reading from MMIO register (instruction b940035c = ldr w28, [x26, #0]); the driver attempted to read from address ffff800082d03004 which triggered a synchronous external abort, indicating the hardware block is either not clocked/powered, inaccessible, or the MMIO mapping is invalid on shikra-iqs-evk.
  3. Possible fix: Verify qcom_rng device tree node for shikra-iqs-evk: confirm the reg property points to the correct PRNG MMIO base address for this SoC, ensure the device node has status = "okay", and add required clocks/power-domains if the PRNG block requires them on this platform. If the hardware block is not present or functional on shikra-iqs-evk, mark the device node status = "disabled" in the board DT.
  4. Detail analysis attachment: failed_case_job207904_10_detailed.md
Job 207905 | SoC qcs615-ride

LAVA job: https://lava-oss.qualcomm.com/scheduler/job/207905

Failed test cases in LAVA job 207905 (SoC: qcs615-ride).

  Case 1: Probe_Failure_Check — cfg80211 regulatory.db firmware load message (known benign)
  1. Failed case: Probe_Failure_Check — cfg80211 regulatory.db firmware load message (known benign)
  2. Root cause: The Probe_Failure_Check test flagged a benign cfg80211 regulatory database firmware load message (Direct firmware load for regulatory.db failed with error -2). This is NOT a genuine failure — WiFi functionality is fully operational (WiFi_Firmware_Driver and WiFi_OnOff tests both PASSED), and cfg80211 successfully falls back to built-in regulatory rules when the optional regulatory.db file is absent. The PR changes (RPMh/cmd-db debug enhancements in drivers/soc/qcom/) are completely unrelated to wireless regulatory functionality. This is a pre-existing test infrastructure false positive, not a PR-introduced regression.
  3. Possible fix: Suppress the regulatory.db firmware load message in the Probe_Failure_Check test's failure pattern matching. Add an exclusion rule: grep -v "regulatory: Direct firmware load for regulatory.db failed" to the test's log scanning logic. This message is a known benign informational log from cfg80211's regulatory database initialization path and does not indicate a functional failure when WiFi tests pass.
  4. Detail analysis attachment: failed_case_job207905_1_detailed.md
  Case 2: smmu
  1. Failed case: smmu
  2. Root cause: Could not be determined confidently from available logs.
  3. Possible fix: Update the smmu test case to remove the check for video-codec:video-decoder and video-codec:video-encoder child device IOMMU attachments, or modify the check to verify only that the parent video-codec device is attached to an IOMMU group (which already passes).
  4. Detail analysis attachment: failed_case_job207905_2_detailed.md
  Case 3: KVM_Driver — Platform Limitation (HYP Mode Not Available)
  1. Failed case: KVM_Driver — Platform Limitation (HYP Mode Not Available)
  2. Root cause: QCS615 platform does not support ARM EL2 (Hypervisor mode), which is required for KVM virtualization. Kernel log shows kvm [1]: HYP mode not available at boot, causing KVM driver initialization to abort and /dev/kvm device node to not be created. This is a hardware/firmware platform limitation, not a kernel regression.
  3. Possible fix: Mark KVM tests as SKIP (not FAIL) for QCS615 platform in the LAVA test suite configuration, as this SoC does not support virtualization. The PR changes (cmd-db/rpmh-rsc) are unrelated to KVM and did not cause this failure — this is a pre-existing platform constraint.
  4. Detail analysis attachment: failed_case_job207905_3_detailed.md
  Case 4: KVM_EL2_DTB
  1. Failed case: KVM_EL2_DTB
  2. Root cause: KVM requires ARM64 EL2 (Hypervisor Exception Level) support to function, but the QCS615 Ride platform hardware/firmware does not provide EL2 capability; the kernel detects this at boot time (kvm [1]: HYP mode not available) and does not create the /dev/kvm device node, causing all KVM-dependent tests to fail as expected on this platform.
  3. Possible fix: This is not a bug or regression—it is a platform hardware limitation. The KVM tests should be skipped on platforms without EL2 support. Add a platform capability check to the LAVA test definition to skip KVM tests when /dev/kvm is not present, or exclude QCS615 from the KVM test matrix since it does not support virtualization extensions.
  4. Detail analysis attachment: failed_case_job207905_4_detailed.md
  Case 5: KVM_Infra
  1. Failed case: KVM_Infra
  2. Root cause: KVM driver initialization failed because the kernel is running at EL1 (Exception Level 1) instead of EL2 (hypervisor mode). The kernel log shows "kvm [1]: HYP mode not available" at boot time and "CPU: All CPU(s) started at EL1", indicating the bootloader/firmware did not boot the kernel into EL2. Without EL2, the KVM driver cannot initialize and /dev/kvm is never created, causing all KVM tests (KVM_Driver, KVM_EL2_DTB, KVM_Infra) to fail. This is a platform/firmware configuration issue specific to the QCS615 board, not a kernel regression introduced by the PR (which only modifies cmd-db and rpmh-rsc debug helpers).
  3. Possible fix: Configure the bootloader (ABL/XBL) on the QCS615-ride board to boot the kernel at EL2 instead of EL1. This typically requires enabling virtualization extensions in the firmware configuration or updating the boot chain to preserve EL2 mode. If EL2 boot is not supported on this platform variant, mark KVM tests as "not applicable" for QCS615-ride in the CI test matrix, as KVM functionality requires hardware/firmware support for EL2 that may not be available on all Qualcomm platforms.
  4. Detail analysis attachment: failed_case_job207905_5_detailed.md
  Case 6: KVM_Infra
  1. Failed case: KVM_Infra
  2. Root cause: KVM hypervisor mode (EL2) is not available on the qcs615-ride platform — kernel reports "HYP mode not available" at boot, causing /dev/kvm device node creation to be skipped, which makes all KVM infrastructure tests fail.
  3. Possible fix: This is a platform limitation, not a PR-introduced regression. The qcs615-ride board's firmware/bootloader does not enable EL2 hypervisor mode. Either: (1) skip KVM tests on qcs615-ride in the CI job definition, or (2) update the board's firmware/bootloader to enable EL2 if virtualization support is required for this platform.
  4. Detail analysis attachment: failed_case_job207905_6_detailed.md
Job 207906 | SoC hamoa-evk

LAVA job: https://lava-oss.qualcomm.com/scheduler/job/207906

Failed test cases in LAVA job 207906 (SoC: hamoa-evk).

  Case 1: Probe_Failure_Check
  1. Failed case: Probe_Failure_Check
  2. Root cause: Could not be determined confidently from available logs.
  3. Possible fix: These are not regressions. To clean up CI: (1) Suppress qcom_qseecom_uefisecapp probe failure for hamoa-evk in Probe_Failure_Check test (known platform limitation), (2) Fix hamoa-evk device tree multi-LED "reg" property to match qcom-spmi-lpg binding requirements, (3) Add wireless-regdb package to rootfs or suppress regulatory.db firmware load warnings (WiFi already functional).
  4. Detail analysis attachment: failed_case_job207906_1_detailed.md
  Case 2: smmu
  1. Failed case: smmu
  2. Root cause: Multiple critical USB controllers (a0f8800.usb, a2f8800.usb, a4f8800.usb, a6f8800.usb, a8f8800.usb) and video codec (aa00000.video-codec) are missing IOMMU group attachments on hamoa-evk, failing the SMMU validation test's requirement that all critical masters be protected by IOMMU.
  3. Possible fix: This is a pre-existing platform configuration issue unrelated to PR 907 (which only adds RPMh debug logging). Add iommus properties to the missing USB controller and video codec device tree nodes in arch/arm64/boot/dts/qcom/x1e80100.dtsi to bind them to the appropriate SMMU instance, following the pattern used by the working USB controllers (a000000.usb, a200000.usb, a400000.usb, a600000.usb, a800000.usb).
  4. Detail analysis attachment: failed_case_job207906_2_detailed.md
  Case 3: KVM_Driver
  1. Failed case: KVM_Driver
  2. Root cause: Could not be determined confidently from available logs.
  3. Possible fix: This is not a PR-introduced regression — the PR modifies cmd-db/RPMh code unrelated to KVM. The failure is a pre-existing platform limitation. Mark KVM tests as "skip" or "not applicable" for hamoa-evk when running under Gunyah hypervisor, or configure the test environment to boot the kernel directly at EL2 without an intermediate hypervisor if KVM validation is required.
  4. Detail analysis attachment: failed_case_job207906_3_detailed.md
  Case 4: KVM_EL2_DTB
  1. Failed case: KVM_EL2_DTB
  2. Root cause: Could not be determined confidently from available logs.
  3. Possible fix: Mark KVM tests as expected failures for hamoa-evk in the LAVA test suite configuration with skip rules, since this is a platform limitation (hamoa-evk does not support ARM virtualization extensions or the bootloader does not boot at EL2). If KVM support is required: (1) verify the SoC implements Armv8.1-VHE or EL2; (2) configure the bootloader (ABL/UEFI) to boot the kernel at EL2 instead of EL1; (3) if the SoC lacks EL2 support, use a different platform for KVM testing (e.g., sc8280xp, sm8450).
  4. Detail analysis attachment: failed_case_job207906_4_detailed.md
  Case 5: ** KVM Initialization Failure — HYP mode not available
  1. Failed case: ** KVM Initialization Failure — HYP mode not available
  2. Root cause: ** KVM driver initialization failed because the Hamoa EVK platform boots under the Gunyah hypervisor (version gunyah-mobile-c487961e9), which claims EL2 (Hypervisor mode). The kernel runs at EL1 as a guest, and KVM requires direct EL2 access to function, which is unavailable in this nested configuration.
  3. Possible fix: This is a platform/firmware configuration issue, not a kernel regression. To enable KVM on Hamoa EVK: (1) reconfigure the platform to boot without Gunyah hypervisor (bare-metal boot), or (2) exclude KVM tests from the CI test suite for Gunyah-based platforms, or (3) if nested virtualization support is required, this would need hypervisor-level changes to expose virtual EL2 to the guest kernel (not currently supported by Gunyah).
  4. Detail analysis attachment: failed_case_job207906_5_detailed.md
  Case 6: 0_qcom-next-ci-premerge-tests
  1. Failed case: 0_qcom-next-ci-premerge-tests
  2. Root cause: LAVA test runner completed all tests successfully but LAVA dispatcher incorrectly marked the test definition as "unfinished" with error "Marking unfinished test run as failed" — this is a LAVA infrastructure issue where the completion signal was not properly detected despite <LAVA_TEST_RUNNER EXIT> being present in the log.
  3. Possible fix: Re-trigger the LAVA job; if the issue recurs, investigate LAVA dispatcher completion detection logic for the lava_test_shell action — the test runner emitted the correct exit signal but the dispatcher's lava_test_shell seems to have completed check still flagged it as unfinished.
  4. Detail analysis attachment: failed_case_job207906_6_detailed.md
Job 207907 | SoC monaco-evk

LAVA job: https://lava-oss.qualcomm.com/scheduler/job/207907

Failed test cases in LAVA job 207907 (SoC: monaco-evk).

  Case 1: Probe_Failure_Check
  1. Failed case: Probe_Failure_Check
  2. Root cause: ath11k_pci WiFi driver probe failed with error -110 (ETIMEDOUT) during MHI firmware load (ath11k/WCN6855/hw2.1/nfa765/amss.bin) on monaco-evk (iq-8275-evk), indicating the WiFi PCIe device (0000:01:00.0) did not respond within the expected timeout during initialization, likely due to missing firmware file, PCIe link issues, or device power/clock configuration problems on this SoC.
  3. Possible fix: Verify the WiFi firmware file ath11k/WCN6855/hw2.1/nfa765/amss.bin is present in /lib/firmware/ on the target rootfs; if missing, add it to the firmware package. If firmware is present, check PCIe link training logs and verify the WCN6855 device power/clock configuration in the monaco-evk device tree matches hardware requirements. The PR changes (RPMh timeout debugging) are not the cause - this is a pre-existing platform/firmware configuration issue.
  4. Detail analysis attachment: failed_case_job207907_1_detailed.md
  Case 2: ** WiFi Driver Probe Failure — Missing Firmware Dependency
  1. Failed case: ** WiFi Driver Probe Failure — Missing Firmware Dependency
  2. Root cause: ** The ath11k_pci driver probe fails with error -110 (ETIMEDOUT) because the required WCN6855 firmware file ath11k/WCN6855/hw2.1/nfa765/amss.bin is missing from the Monaco EVK rootfs. The firmware load fails with error -2 (ENOENT, file not found), preventing MHI bus power-up and causing the driver to timeout during initialization. This is a Monaco-specific board variant (nfa765) firmware packaging issue in the Yocto build, not a kernel driver bug or PR-introduced regression.
  3. Possible fix: Add the WCN6855 hw2.1 nfa765 firmware files to the Monaco EVK Yocto image recipe. Specifically, ensure the meta-qcom layer includes linux-firmware-ath11k with the nfa765 board variant files, or add a Monaco-specific firmware package that installs ath11k/WCN6855/hw2.1/nfa765/amss.bin and related files (m3.bin, regdb.bin) to /lib/firmware/. Verify the fix by checking that the firmware files are present in the rootfs and that the ath11k_pci driver probes successfully on next boot.
  4. Detail analysis attachment: failed_case_job207907_2_detailed.md
  Case 3: ** WiFi_OnOff — Driver Probe Failure (Firmware Dependency)
  1. Failed case: ** WiFi_OnOff — Driver Probe Failure (Firmware Dependency)
  2. Root cause: ** ath11k_pci driver probe failed with -ETIMEDOUT because the required WCN6855 firmware file (ath11k/WCN6855/hw2.1/nfa765/amss.bin) is missing from the rootfs. MHI firmware load returned -ENOENT, causing MHI power-up to timeout, which cascaded to probe failure. This is a pre-existing platform/infra issue on monaco-evk, not a regression introduced by the PR (which only modifies RPMh/cmd-db debug code unrelated to WiFi).
  3. Possible fix: Add the missing WCN6855 firmware file ath11k/WCN6855/hw2.1/nfa765/amss.bin to the monaco-evk rootfs under /lib/firmware/. Verify the firmware package (e.g., linux-firmware-ath11k or equivalent) is installed in the Yocto/distro build recipe for monaco-evk. If the firmware path is board-specific (nfa765 variant), confirm the correct board-data file is also present.
  4. Detail analysis attachment: failed_case_job207907_3_detailed.md
  Case 4: Driver Probe Failure — WiFi ath11k_pci
  1. Failed case: Driver Probe Failure — WiFi ath11k_pci
  2. Root cause: ath11k_pci driver probe failed with -ETIMEDOUT (-110) during MHI power-up and firmware load on monaco-evk. The failure chain shows: firmware file not found (ath11k/WCN6855/hw2.1/nfa765/amss.bin returned -ENOENT/-2), followed by MHI power-up timeout (-110), leading to complete probe failure. This is a pre-existing platform/firmware issue unrelated to the PR patches (cmd-db/rpmh-rsc debug helpers).
  3. Possible fix: Verify WiFi firmware files are present in the rootfs at /lib/firmware/ath11k/WCN6855/hw2.1/nfa765/amss.bin. If missing, add the firmware package to the build. If the firmware path is incorrect for this hardware revision, update the device tree or driver firmware selection logic to match the actual hardware (WCN6855 hw2.1 vs hw2.0).
  4. Detail analysis attachment: failed_case_job207907_4_detailed.md
Job 207908 | SoC purwa-evk

LAVA job: https://lava-oss.qualcomm.com/scheduler/job/207908

Failed test cases in LAVA job 207908 (SoC: purwa-evk).

  Case 1: Probe_Failure_Check (Test Infrastructure False Positive)
  1. Failed case: Probe_Failure_Check (Test Infrastructure False Positive)
  2. Root cause: The Probe_Failure_Check test detected 5 pre-existing probe failures on the purwa-evk platform that are unrelated to the PR changes. The PR only adds RPMh timeout debugging functions (cmd_db_hw_type_str, cmd_db_read_name, rpmh_rsc_debug) and does not modify any of the failing drivers (qcom-pcie, qcom_qseecom_uefisecapp, qcom-spmi-lpg, or cfg80211 regulatory). These failures exist due to: (1) PCIe PHY init sequences not yet available for this SoC variant (-ENODATA), (2) TrustZone secure app dependency (-EBUSY), (3) DT binding mismatch for SPMI LPG multi-LED (-EINVAL), and (4) missing optional regulatory.db firmware (-ENOENT).
  3. Possible fix: Mark this test case as a false positive for this PR. The probe failures are pre-existing platform issues that should be tracked separately. To prevent future false positives, update the Probe_Failure_Check test to maintain a platform-specific allowlist of known benign probe failures for purwa-evk, or filter out -ENODATA (missing PHY init), -EBUSY (TZ dependency), and -ENOENT (optional firmware) errors that do not impact core functionality.
  4. Detail analysis attachment: failed_case_job207908_1_detailed.md
  Case 2: smmu
  1. Failed case: smmu
  2. Root cause: Test validation logic expects IOMMU group attachments for USB wrapper devices (a0f8800.usb, a2f8800.usb, a4f8800.usb, a6f8800.usb, a8f8800.usb) and video codec (aa00000.video-codec) that are not DMA-capable or not functional on purwa-evk; no actual SMMU faults or kernel errors present.
  3. Possible fix: Update the SMMU test's critical master list for purwa-evk to exclude USB wrapper devices (which are not DMA masters) and the non-functional video codec, or add device tree iommus properties if these devices are expected to perform DMA on this platform.
  4. Detail analysis attachment: failed_case_job207908_2_detailed.md
  Case 3: KVM Driver Initialization Failure — Platform Does Not Support Virtualization
  1. Failed case: KVM Driver Initialization Failure — Platform Does Not Support Virtualization
  2. Root cause: The purwa-evk platform does not have ARM virtualization extensions (EL2/HYP mode) available. The kernel reports kvm [1]: HYP mode not available during boot, preventing KVM driver initialization and /dev/kvm device node creation. This is a hardware/firmware platform limitation, not a kernel regression.
  3. Possible fix: This is not a bug to fix. The KVM_Driver test should be excluded from the purwa-evk test suite, as this platform does not support virtualization. Add a platform-specific test skip rule: if platform == "purwa-evk", skip all KVM/virtualization tests. Alternatively, if virtualization support is required, upgrade to a platform with ARM virtualization extensions and firmware that preserves EL2 mode.
  4. Detail analysis attachment: failed_case_job207908_3_detailed.md
  Case 4: KVM Driver Initialization Failure — /dev/kvm not available (HYP mode not available)
  1. Failed case: KVM Driver Initialization Failure — /dev/kvm not available (HYP mode not available)
  2. Root cause: KVM driver initialization failed because the purwa-evk platform does not support ARM EL2 hypervisor mode. The kernel message "kvm [1]: HYP mode not available" at boot indicates the CPU is not running at EL2 or EL2 is not accessible, preventing KVM from creating the /dev/kvm device node required for virtualization.
  3. Possible fix: This is a platform hardware limitation, not a kernel bug or PR-introduced regression. The purwa-evk board does not support ARM virtualization extensions (EL2). To resolve: (1) Skip KVM tests on purwa-evk in the CI test matrix, or (2) Use a different platform that supports ARM virtualization (e.g., boards with Cortex-A cores that implement the virtualization extensions and boot at EL2).
  4. Detail analysis attachment: failed_case_job207908_4_detailed.md
  Case 5: KVM_Infra
  1. Failed case: KVM_Infra
  2. Root cause: KVM cannot initialize on purwa-evk because the Gunyah hypervisor occupies EL2 (HYP mode), which KVM requires for exclusive access. The kernel correctly reports "HYP mode not available" (line 6527402), preventing /dev/kvm creation. This is expected behavior when running under a hypervisor, not a kernel regression.
  3. Possible fix: Exclude KVM tests from the purwa-evk test suite, as this platform runs under Gunyah hypervisor and cannot support KVM. Alternatively, configure the LAVA job to skip KVM tests on platforms with hypervisor configurations where EL2 is not available to the guest OS.
  4. Detail analysis attachment: failed_case_job207908_5_detailed.md
  Case 6: KVM_Infra
  1. Failed case: KVM_Infra
  2. Root cause: KVM initialization failed because HYP mode (EL2) is not available — the Purwa EVK is running under the Gunyah hypervisor which owns EL2, preventing the Linux kernel from initializing KVM (nested virtualization is not supported in this configuration).
  3. Possible fix: This is not a PR-introduced regression; it is a platform configuration issue. The Purwa EVK boots with Gunyah hypervisor enabled, which reserves EL2 for itself. To enable KVM, either: (1) disable Gunyah hypervisor in the boot configuration to allow Linux direct EL2 access, or (2) suppress KVM tests on platforms where Gunyah is enabled, as nested virtualization is not supported.
  4. Detail analysis attachment: failed_case_job207908_6_detailed.md
Job 207909 | SoC qcs9100-ride

LAVA job: https://lava-oss.qualcomm.com/scheduler/job/207909

Failed test cases in LAVA job 207909 (SoC: qcs9100-ride).

  Case 1: Probe_Failure_Check — Pre-existing Infrastructure/Configuration Issues
  1. Failed case: Probe_Failure_Check — Pre-existing Infrastructure/Configuration Issues
  2. Root cause: Test detected four deferred PMIC temp-alarm probes (expected transient behavior on qcs9100-ride awaiting thermal framework dependencies), one benign cfg80211 regulatory.db firmware load failure (WiFi functional despite missing file), and one Aquantia AQR115C Ethernet PHY probe failure with -EINVAL (pre-existing hardware/DT configuration issue unrelated to PR changes, which only add RPMh debug instrumentation).
  3. Possible fix: No PR changes required. For CI: suppress regulatory.db firmware warnings (known benign — WiFi_OnOff test passed). For temp-alarm deferrals: verify thermal zone driver loads later or mark as expected on this platform. For Aquantia PHY: investigate board-specific DT configuration, firmware availability, or hardware presence independently of this PR.
  4. Detail analysis attachment: failed_case_job207909_1_detailed.md
  Case 2: smmu
  1. Failed case: smmu
  2. Root cause: Video codec device (aa00000.video-codec) failed to attach to any IOMMU group during kernel boot, causing the SMMU validation test to fail when checking critical master protection. This is a pre-existing platform issue unrelated to the PR changes (which only add RPMh debug logging to cmd-db and rpmh-rsc drivers).
  3. Possible fix: Investigate why the video codec driver at aa00000.video-codec did not probe or attach to an IOMMU group. Check device tree configuration for the qcs9100-ride platform, verify the video codec driver is enabled in the kernel config, and examine dmesg for video codec probe failures or IOMMU binding errors during early boot.
  4. Detail analysis attachment: failed_case_job207909_2_detailed.md
  Case 3: ** USBHost
  1. Failed case: ** USBHost
  2. Root cause: ** Test infrastructure limitation — no physical USB devices connected to qcs9100-ride board USB host ports. USB host stack is fully functional (three USB buses enumerated, xHCI controllers operational, root hubs present), but test expects at least one non-hub USB device to be attached. PR FROMLIST: Output debug information from RSC #907 modifies only cmd-db (RPMh) code and does not touch USB subsystem.
  3. Possible fix: Configure LAVA lab qcs9100-ride boards with permanently attached USB test devices (USB flash drive or USB-to-Ethernet adapter), or update USBHost test to pass when root hubs are present (indicating functional USB stack) even if no downstream devices exist. Short-term: mark USBHost as expected failure for qcs9100-ride in LAVA CI configuration to unblock PR merges.
  4. Detail analysis attachment: failed_case_job207909_3_detailed.md
  Case 4: 0_qcom-next-ci-premerge-tests
  1. Failed case: 0_qcom-next-ci-premerge-tests
  2. Root cause: LAVA test definition marked as failed due to 3 constituent test case failures: (1) Probe_Failure_Check - kernel probe/firmware errors detected in dmesg, (2) smmu - critical master aa00000.video-codec missing iommu_group attachment, (3) USBHost - only USB hubs detected, no functional USB devices connected.
  3. Possible fix: Address the 3 individual test failures: (1) investigate probe failures in dmesg logs to identify driver initialization issues, (2) verify video codec IOMMU/SMMU device tree configuration and driver probe sequence for qcs9100-ride platform, (3) connect a functional USB device to the board or mark USBHost as optional if no USB peripherals are required for this test configuration.
  4. Detail analysis attachment: failed_case_job207909_4_detailed.md
Job 207910 | SoC lemans-evk

LAVA job: https://lava-oss.qualcomm.com/scheduler/job/207910

Failed test cases in LAVA job 207910 (SoC: lemans-evk).

  Case 1: Probe_Failure_Check
  1. Failed case: Probe_Failure_Check
  2. Root cause: Four PMIC temp-alarm devices (c440000.spmi:pmic@{0,2,4,6}:temp-alarm@a00) remain in permanent deferred probe state on lemans-evk, indicating the qcom-spmi-temp-alarm driver is waiting for an unmet dependency (likely thermal zone or IIO ADC channel registration). The Bluetooth firmware load errors are benign false positives (BT_ON_OFF test passed, confirming functional Bluetooth).
  3. Possible fix: Investigate why the qcom-spmi-temp-alarm driver's dependencies are not being satisfied on lemans-evk — check device tree for correct thermal-zone linkage, IIO ADC channel availability, and SPMI PMIC node configuration. Verify that all required thermal framework and IIO subsystem drivers probe before the temp-alarm driver. The firmware load errors can be suppressed as known benign (Rule 3 applies — BT ON/OFF passed).
  4. Detail analysis attachment: failed_case_job207910_1_detailed.md
  Case 2: smmu
  1. Failed case: smmu
  2. Root cause: Video codec device aa00000.video-codec at address 0xaa00000 is not attached to any IOMMU group on lemans-evk, causing the SMMU test's critical master validation check to fail.
  3. Possible fix: Add the missing iommus property to the aa00000.video-codec device tree node in arch/arm64/boot/dts/qcom/sa8775p.dtsi (or the lemans-evk-specific overlay) to bind the video codec to an IOMMU group, following the pattern used for other critical masters (GPU, USB, Display) on this SoC.
  4. Detail analysis attachment: failed_case_job207910_2_detailed.md
  Case 3: LAVA Test Framework Issue — Missing ENDRUN Signal
  1. Failed case: LAVA Test Framework Issue — Missing ENDRUN Signal
  2. Root cause: The LAVA test runner (lava-test-runner) executed all test cases successfully and sent <LAVA_TEST_RUNNER EXIT> at line 6939, but failed to send the required <LAVA_SIGNAL_ENDRUN> signal before exiting. LAVA dispatcher was waiting for this signal to mark the test definition as complete, and when it didn't arrive, marked the test run as "unfinished" with result=fail at line 6939: "Marking unfinished test run as failed". This is a test infrastructure issue, not a kernel issue.
  3. Possible fix: This is a known issue with the qcom-linux-testkit test runner script. The test runner should be updated to emit <LAVA_SIGNAL_ENDRUN> after completing all tests and before exiting. As a workaround, re-trigger the LAVA job — this type of test framework race condition is often transient. If the issue persists, file a bug against the qcom-linux-testkit repository (https://github.com/qualcomm-linux/qcom-linux-testkit) requesting that the test runner script be fixed to reliably emit the ENDRUN signal.
  4. Detail analysis attachment: failed_case_job207910_3_detailed.md
Job 207911 | SoC qcs8300-ride

LAVA job: https://lava-oss.qualcomm.com/scheduler/job/207911

Failed test cases in LAVA job 207911 (SoC: qcs8300-ride).

  Case 1: Probe_Failure_Check
  1. Failed case: Probe_Failure_Check
  2. Root cause: Two probe failures detected during boot on qcs8300-ride: (1) Aquantia AQR115C Ethernet PHY probe failed with -EINVAL (-22) due to missing firmware-name DT property — this is a pre-existing platform configuration issue, not a functional failure (Ethernet test was SKIP due to no cable/link, indicating the PHY probe failure is benign for this board configuration); (2) cfg80211 regulatory.db firmware load failed with -ENOENT (-2) — this is a known benign failure (WiFi functional tests WiFi_Firmware_Driver and WiFi_OnOff both PASS, confirming WiFi operates correctly without the optional regulatory database).
  3. Possible fix: Suppress this test case failure as non-genuine: both probe failures are benign and do not indicate a PR-introduced regression. The Aquantia PHY failure is a board-specific DT configuration gap (qcs8300-ride does not use Ethernet in this test configuration), and the regulatory.db failure is a known false positive when WiFi functional tests pass. No kernel fix required. If the test framework should not flag these as failures, update the Probe_Failure_Check test to exclude: (a) Aquantia PHY probe failures when Ethernet test result is SKIP, and (b) regulatory.db firmware failures when WiFi functional tests pass.
  4. Detail analysis attachment: failed_case_job207911_1_detailed.md
  Case 2: USBHost
  1. Failed case: USBHost
  2. Root cause: Test environment issue — no external USB devices physically connected to the qcs8300-ride board's USB host port. The USB host controller initialized correctly (xhci-hcd registered, USB bus 1 created), but only the root hub is enumerated. The test expects at least one functional USB device (storage, keyboard, etc.) to be connected for validation.
  3. Possible fix: This is not a kernel regression introduced by PR FROMLIST: Output debug information from RSC #907. The PR changes only affect cmd-db (command database) helper functions for RPMh resource address handling and do not touch USB subsystem code. To resolve: physically connect a USB device (e.g., USB flash drive, keyboard) to the board's USB host port before running the test, or mark this test as SKIP when no USB peripherals are available in the lab setup.
  4. Detail analysis attachment: failed_case_job207911_2_detailed.md
  Case 3: KVM_Driver
  1. Failed case: KVM_Driver
  2. Root cause: /dev/kvm device node not created because KVM driver did not initialize — QCS8300 (Monaco) platform runs under Gunyah hypervisor (evidenced by reserved memory region gunyah-md-region@91a80000 in device tree), which prevents nested virtualization; KVM requires EL2 virtualization extensions to be available to the host kernel, but when running as a guest under a hypervisor, EL2 is not accessible.
  3. Possible fix: This is a platform limitation, not a kernel regression. KVM tests should be skipped on QCS8300 (Monaco) and other platforms that run under a hypervisor. Add platform detection logic to the LAVA test definition to skip KVM tests when gunyah-md-region or similar hypervisor indicators are present in /proc/device-tree/reserved-memory/.
  4. Detail analysis attachment: failed_case_job207911_3_detailed.md
  Case 4: KVM_EL2_DTB — KVM device node unavailable (platform configuration issue)
  1. Failed case: KVM_EL2_DTB — KVM device node unavailable (platform configuration issue)
  2. Root cause: KVM cannot initialize on qcs8300-ride because the platform runs under Gunyah hypervisor (version gunyah-1cb9db980) which owns EL2; Linux executes at EL1 as a guest VM and cannot access EL2 virtualization extensions required by KVM.
  3. Possible fix: This is a platform configuration issue, not a kernel bug. To enable KVM testing on qcs8300-ride: (1) disable Gunyah hypervisor in the firmware/bootloader configuration to allow Linux to run at EL2, or (2) exclude KVM tests from the CI test suite for platforms configured with Gunyah hypervisor, or (3) use a different test platform that runs Linux at EL2 without a Type-1 hypervisor.
  4. Detail analysis attachment: failed_case_job207911_4_detailed.md
  Case 5: KVM_Infra — KVM unavailable under Gunyah hypervisor
  1. Failed case: KVM_Infra — KVM unavailable under Gunyah hypervisor
  2. Root cause: The qcs8300-ride platform is running under the Gunyah hypervisor (boot log line 2196: "Hypervisor cold boot, version: gunyah-1cb9db980"), which prevents KVM from initializing because nested virtualization is not supported. CONFIG_KVM is enabled in the kernel, but /dev/kvm is never created because KVM cannot run at EL1 when a hypervisor already controls EL2.
  3. Possible fix: This is a pre-existing platform/infrastructure limitation, not a PR-introduced regression (PR touches only cmd-db code). To enable KVM testing on qcs8300-ride: (1) boot without Gunyah hypervisor, or (2) use a different test platform that boots Linux directly at EL1 without a hypervisor, or (3) suppress KVM tests on Gunyah-enabled platforms in the CI configuration.
  4. Detail analysis attachment: failed_case_job207911_5_detailed.md
  Case 6: KVM_Infra
  1. Failed case: KVM_Infra
  2. Root cause: /dev/kvm device node not created because KVM ARM driver did not initialize on QCS8300 Ride platform — Gunyah hypervisor is present (gunyah-md-region reserved at 0x91a80000), preventing KVM from taking control of EL2.
  3. Possible fix: This is a pre-existing platform limitation, not a PR-introduced regression. KVM and Gunyah hypervisor are mutually exclusive on ARM platforms (both require EL2). Skip KVM tests on QCS8300 Ride or use a build without Gunyah hypervisor support if KVM functionality is required.
  4. Detail analysis attachment: failed_case_job207911_6_detailed.md
Job 207913 | SoC qcs6490-rb3gen2

LAVA job: https://lava-oss.qualcomm.com/scheduler/job/207913

Failed test cases in LAVA job 207913 (SoC: qcs6490-rb3gen2).

  Case 1: Probe_Failure_Check
  1. Failed case: Probe_Failure_Check
  2. Root cause: Could not be determined confidently from available logs.
  3. Possible fix: Add the missing firmware files to the root filesystem build: install wireless-regdb package (provides regulatory.db) and linux-firmware package (provides renesas_usb_fw.mem) to the Yocto/buildroot image recipe, or manually copy these files to /lib/firmware/ in the rootfs before flashing.
  4. Detail analysis attachment: failed_case_job207913_1_detailed.md
  Case 2: USBHost
  1. Failed case: USBHost
  2. Root cause: USB host controller (xhci-pci-renesas at PCIe address 0001:04:00.0) probe failed with error -2 (ENOENT) due to missing firmware file renesas_usb_fw.mem. Without a functional USB host controller, the USBHost test cannot enumerate any USB devices connected to the qcs6490-rb3gen2 board.
  3. Possible fix: Add the Renesas USB firmware file renesas_usb_fw.mem to the rootfs at /lib/firmware/ in the build image. The firmware is available in the linux-firmware package or from Renesas. Alternatively, if the Renesas xHCI controller is not required for this platform, disable CONFIG_USB_XHCI_PCI_RENESAS in the kernel config to prevent the probe attempt.
  4. Detail analysis attachment: failed_case_job207913_2_detailed.md
  Case 3: ** BT_SCAN — Test Environment Issue (No Nearby Bluetooth Devices)
  1. Failed case: ** BT_SCAN — Test Environment Issue (No Nearby Bluetooth Devices)
  2. Root cause: ** The BT_SCAN test failed because no Bluetooth devices were discoverable in the LAVA lab environment during the 3 scan attempts (15 seconds each). The Bluetooth stack is fully functional: firmware loaded successfully (Bluetooth: hci0: QCA setup on UART is completed), hci0 adapter is operational, BT_ON_OFF functional test passed, and scan operations executed without errors (Discovery started, Discovery stopped). The failure is due to the absence of nearby Bluetooth devices in the test environment, not a kernel or driver regression.
  3. Possible fix: This is a test infrastructure limitation, not a kernel bug. The PR changes (cmd-db helper functions for RPMh address debugging) do not touch Bluetooth code and cannot cause this failure. Recommended action: Accept this test result as a known environmental limitation. If BT_SCAN coverage is required, add a Bluetooth beacon/device to the LAVA lab environment near the qcs6490-rb3gen2 board, or mark BT_SCAN as optional when no target devices are configured.
  4. Detail analysis attachment: failed_case_job207913_3_detailed.md
  Case 4: KVM_Driver
  1. Failed case: KVM_Driver
  2. Root cause: KVM cannot initialize because Gunyah hypervisor is already running in EL2 (HYP mode). On ARM64, only one hypervisor can occupy EL2 at a time. The kernel log shows kvm [1]: HYP mode not available at boot, and Gunyah hypervisor cold boot message confirms Gunyah is active. This is a platform configuration limitation on qcs6490-rb3gen2 when Gunyah is enabled, not a kernel bug.
  3. Possible fix: This is expected behavior when Gunyah hypervisor is enabled on the platform. To enable KVM, disable Gunyah in the bootloader/firmware configuration and rebuild the boot image without Gunyah support. Alternatively, if KVM functionality is required for testing, use a different platform configuration or board variant that does not have Gunyah enabled. If this test is intended to run on Gunyah-enabled platforms, mark it as SKIP when Gunyah is detected instead of FAIL.
  4. Detail analysis attachment: failed_case_job207913_4_detailed.md
  Case 5: ** KVM_EL2_DTB
  1. Failed case: ** KVM_EL2_DTB
  2. Root cause: ** KVM initialization failed because qcs6490-rb3gen2 platform does not provide EL2 (hypervisor mode) access to the kernel; KVM driver detected "HYP mode not available" at boot (line 2834) and did not create /dev/kvm, causing all KVM tests to fail.
  3. Possible fix: This is a platform hardware/firmware limitation, not a kernel bug. Mark KVM tests as "skip" for qcs6490-rb3gen2 in the LAVA test definition, or enable EL2 in the platform firmware/bootloader if virtualization support is required for this board.
  4. Detail analysis attachment: failed_case_job207913_5_detailed.md
  Case 6: KVM_Infra
  1. Failed case: KVM_Infra
  2. Root cause: KVM driver initialization failed because the qcs6490-rb3gen2 platform is running the Gunyah hypervisor in EL2, which prevents KVM from accessing HYP mode; CONFIG_KVM is enabled but /dev/kvm device node is not created because kvm_init() exits early with "HYP mode not available" message.
  3. Possible fix: This is not a PR-introduced regression — the PR modifies only cmd-db (RPMh resource address helpers) and does not touch KVM, virtualization, or hypervisor code. The failure is a pre-existing platform limitation: qcs6490-rb3gen2 runs Gunyah hypervisor which occupies EL2, making KVM unavailable. To resolve: either (1) disable CONFIG_KVM in the kernel config for this platform, or (2) update the test suite to skip KVM tests when Gunyah hypervisor is detected, or (3) use a different platform/boot configuration that does not load Gunyah if KVM testing is required.
  4. Detail analysis attachment: failed_case_job207913_6_detailed.md
  Case 7: KVM_Infra
  1. Failed case: KVM_Infra
  2. Root cause: KVM initialization failed because the qcs6490-rb3gen2 platform is running the Gunyah hypervisor at EL2, preventing KVM from accessing HYP mode. The kernel log shows "kvm [1]: HYP mode not available" and "Hypervisor cold boot, version: gunyah-1cb9db980 perf". KVM requires direct EL2 access but cannot coexist with a Type-1 hypervisor already occupying that privilege level.
  3. Possible fix: This is an expected platform limitation, not a kernel regression. The KVM test suite should be skipped on platforms configured with Gunyah hypervisor. Add a platform detection check to the LAVA test definition to skip KVM tests when Gunyah is detected (check for "Gunyah based bootup" in early boot logs or verify /sys/hypervisor/type). Alternatively, if KVM functionality is required, rebuild the platform firmware without Gunyah to allow native KVM operation.
  4. Detail analysis attachment: failed_case_job207913_7_detailed.md

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@Komal-Bajaj Komal Bajaj (Komal-Bajaj) Komal-Bajaj approved these changes
@idlethread Amit Kucheria (idlethread) Awaiting requested review from idlethread idlethread was automatically assigned from qualcomm-linux/qualcomm-kernel-maint
@quic-tingweiz Tingwei Zhang (quic-tingweiz) Awaiting requested review from quic-tingweiz quic-tingweiz was automatically assigned from qualcomm-linux/qualcomm-kernel-maint
@jingyiwang42 jingyiwang42 Awaiting requested review from jingyiwang42 jingyiwang42 was automatically assigned from qualcomm-linux/qualcomm-kernel-maint
+1 more reviewer
@smankad-oss smankad-oss smankad-oss approved these changes
Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@maulik-k-shah @qcomlnxci @qlijarvis @Komal-Bajaj @smankad-oss @sgaud-quic