Supplemental Material "Uncovering Failures in Cyber-Physical System State Transitions: A Fuzzing-Based Approach Applied to sUAS"

Theodore Chambers, Arturo Miguel Russell Bernal, Michael Vierhauser, Jane Cleland-Huang

accepted at the 48th International Conference on Software Engineering (ICSE) 2026, Technical Track

Citation:

@inproceedings{statefuzz_icse26,
title = {Uncovering Failures in Cyber-Physical System State Transitions: A Fuzzing-Based Approach Applied to sUAS},
 author = {Chambers, Theodore and Bernal, Arturo Miguel Russell and Vierhauser, Michael and Cleland-Huang, Jane},
 booktitle = {Proceedings of the 48th International Conference on Software Engineering},
 doi = {N/A},
 publisher = {Association for Computing Machinery},
 series = {ICSE 2026},
 year = {2026}
}

Overview

SaFUZZ is an automated fuzz testing pipeline designed to validate the autonomous behavior of small Uncrewed Aerial Systems (sUAS). By targeting mode transitions and failsafe mechanisms in layered sUAS state machines, SaFUZZ exposes subtle failures with fuzzing that may arise due to environmental disturbances, timing variability, and RC Input.

This framework enhances software system safety assurance by systematically generating fuzz specifications, executing tests in simulation, and producing fault trees to aid root cause analysis.

---

Features

• Automated generation of test scenarios from hazard analysis.
• Layered state machine analysis covering application-level logic and varied autopilot firmware (PX4, ArduPilot).
• Fuzzing pipeline injecting realistic environmental and timing disturbances along with RC Inputs.
• Decision-tree based labeling to classify test outcomes (success, failure, invalid).
• Fault tree visualization for root cause analysis.
• Support for simulation-based and real-world testing of sUAS.

---

Motivation

sUAS operate in unpredictable environments and rely on complex state machines for safe and reliable behavior. Existing testing approaches often focus on low-level input mutations or specific functionalities but lack systematic validation of state transitions and failsafe activations under realistic conditions that can be extended to the real world.

SaFUZZ addresses this challenge by enabling behaviorally meaningful fuzz testing to aid detection of potential faults early in development.

---

Architecture

The SaFUZZ pipeline consists of 8 main phases:

Figure: High-level SaFUZZ pipeline architecture.

Validation

To evaluate the effectiveness of SaFUZZ we used a multi-sUAS application (real name withheld for DB-Review Process - We will provide a link to the SuT and further details upon acceptance of the paper).

We evaluate the effectiveness, scalability, and practical utility of SaFUZZ through three research questions, each accompanied by corresponding supplemental artifacts:

Research Question	Description	Artifact
RQ1: Failure Detection	To what extent can SaFUZZ identify previously unknown behavioral failures in a real-world sUAS system? This question examines whether our framework can effectively detect and categorize failures in the SuT. For each type of failure, we identify potential actions such as code modifications, requirements analysis, or updates to the decision tree.	Test Scenarios, Decision Tree
RQ2: Detection of Transition Errors	How well do the transition-related errors detected by SaFUZZ align with those identified by the development team over time? We conduct a detailed analysis of mode and state-related transition errors that were identified by the Drone Response development team within a time frame of 18 months, between January 2024 and July 2025, and whether SaFUZZ also identified them.	Detected Failures, Fault Trees
RQ3: Real-World Reproducibility	To what extent are the failures identified by SaFUZZ in simulation reproducible in real-world flight tests? This assesses the correspondence between simulation-detected failures and their manifestation in physical flight, examining consistency for both unresolved faults and those marked as mitigated.	Flight Logs

These artifacts are provided as supplemental material to support the validation of SaFUZZ.

Code Samples Notice:
We have included some code samples to demonstrate parts of our automated pipeline. You can browse them in the CodeSamples directory. We have left other major files tied to our onboard pilot (to preserve anonymity), and the code base is not directly executable since we use a custom Docker interface and other functions to interface with our SuT (more details to come if accepted).