Research artifacts accompanying:
Urrea, C. Hybrid Deep Learning-Reinforcement Learning for Adaptive Human-Robot Task Allocation in Industry 5.0. Systems 2025, 13, 631. https://doi.org/10.3390/systems13080631
Published 26 July 2025 (received 21 June 2025, accepted 24 July 2025).
Every result comes from simulation. A virtual Doosan H2017 cobot and a synthetic human operator work inside a RoboDK 5.9 cell orchestrated from MATLAB R2025a. The operator's fatigue, skill level and task-completion times are drawn from declared distributions (Appendix A, Tables A5–A6); no human participant was recorded at any stage, and the physiological signals that drive the workload metric are synthetic.
That design permits controlled, repeatable trials over 1,000 episodes. It does not establish factory-floor performance. The 21% throughput gain, the 7% workload reduction and the 99.90% collision-free rate are properties of this simulation. Validation on a physical H2017 cell with volunteer operators is the declared next step and has not been performed.
A Convolutional Neural Network classifies nine fatigue–skill combinations from simulated sensor inputs; its output feeds a Double Deep Q-Network whose state vector also carries task-queue and robot-status features. The DDQN optimises a multi-objective reward balancing throughput, workload and safety, and executes at 10 Hz inside a closed-loop pipeline with a 90 ms hard deadline.
Benchmarked over 1,000 episodes (2,500 allocations per episode) against five baselines: rule-based, SARSA, Dueling DQN, PPO and A3C.
├── README.md this file
├── Metadata.md file descriptions, column definitions, dependencies
├── Dataset/
│ ├── Results/
│ │ ├── HRC_Simulation_Results.csv per-episode performance metrics
│ │ ├── HRC_Simulation_Log_Episodes.txt simulation log
│ │ └── HRC_Synthetic_Dataset.parquet synthetic dataset
│ ├── Tables/
│ │ ├── Table_2.csv aggregate performance (Table 2)
│ │ └── Table_3.csv performance by human state (Table 3)
│ └── Figures/
│ ├── Debug_Figure_4.txt debug log for Figure 4
│ └── Debug_Figure_6.txt debug log for Figure 6
├── Scripts/
│ ├── hrc_simulation.m main simulation (1,000 episodes)
│ ├── generate_synthetic_dataset.m generates the Parquet dataset
│ ├── Table_2.m generates Table 2
│ ├── Table_3.m generates Table 3
│ ├── Figure_2.m RoboDK snapshot placeholder
│ ├── Figure_3.m Q-value heat-map
│ ├── Figure_4.m conditional heat-maps
│ ├── Figure_5.m dynamic task allocation
│ └── Figure_6.m proof-of-concept visualisation
├── Figures/
│ └── Figure_1.tiff … Figure_6.tiff figures as published
└── Simulation_Assets/
└── Placeholder_Metadata.txt notes on the excluded RoboDK assets
MATLAB R2025a with the Reinforcement Learning, Deep Learning, Robotics System, and Statistics and Machine Learning toolboxes. CNN training was executed on an NVIDIA RTX 4090; a GPU is recommended but the analysis scripts run without one.
RoboDK 5.9 is optional. It is needed only for the visualisation of Figure 2.
Set the API path to C:\RoboDK\API\MATLAB and adjust as required. The scripts fall
back gracefully when the connection fails — see the robodkAvailable flag in
hrc_simulation.m.
git clone https://github.com/ClaudioUrrea/doosan.git
cd doosanThe dataset is also archived on Figshare at
10.6084/m9.figshare.29323520, and the
compute_metrics() implementation of Appendix A at
10.6084/m9.figshare.29604932.v1.
Place the files in a MATLAB-accessible working directory and ensure
HRC_Simulation_Results.csv is reachable from the scripts.
| Paper artefact | Command |
|---|---|
| 1,000-episode campaign | hrc_simulation.m |
| Table 2 (aggregate performance) | Table_2.m |
| Table 3 (performance by human state) | Table_3.m |
| Figure 3 (Q-value heat-map) | Figure_3.m |
| Figure 4 (conditional heat-maps) | Figure_4.m |
| Figure 5 (dynamic allocation) | Figure_5.m |
| Figure 6 (proof of concept) | Figure_6.m |
| Synthetic dataset | generate_synthetic_dataset.m |
Figures 4–6 require HRC_Simulation_Results.csv and, in some cases, the qTable
produced by hrc_simulation.m. Figure 1 is a system diagram and Figure 2 a RoboDK
render; both are supplied as TIFF rather than regenerated by script.
All simulations run under rng(42, 'twister'). Hyperparameters are listed in
Table 1 of the paper: CNN with three convolutional layers (32/64/128 filters, 3×3),
Adam at lr = 0.001, batch 32, 50 epochs with early stopping at patience 5; DDQN with
hidden layers 128/64/32, γ = 0.99, α = 0.001, ε-greedy from 0.1 decaying at 0.995,
replay buffer of 10,000 transitions, mini-batch 64, target update every 100 steps.
Over 1,000 evaluation episodes, the hybrid CNN+DDQN controller reached 60.48 ± 0.08 tasks·min⁻¹ (+21% over rule-based, +12% SARSA, +9% A3C, +8% Dueling DQN, +5% PPO), 4.25 ± 0.10 cumulative fatigue units (−7% versus rule-based) and 99.90 ± 0.10% collision-free execution. One-way ANOVA gave p < 0.001 with Tukey HSD contrasts at α = 0.05; partial η² between 0.29 and 0.46. Post-hoc power was 1 − β = 0.87.
Ablation over 200 episodes: removing the CNN cut throughput by 15% and raised workload to 4.90 ± 0.12; replacing the DDQN with a single-estimation DQN lowered safety to 97.00 ± 0.05%.
Simulation assets — RoboDK cell templates, STEP and URDF models — are excluded under proprietary restrictions. They can be requested from the author. RoboDK itself is not redistributed; obtain it from RoboDK Inc. under whichever licence applies to you.
Metadata.md documents file descriptions, column definitions (for example
StateIndex = 1–9 for the fatigue–skill combinations) and dependency information.
CC BY 4.0, consistent with the article's open-access terms.
Urrea, C. Hybrid Deep Learning-Reinforcement Learning for Adaptive Human-Robot Task Allocation in Industry 5.0. Systems 2025, 13, 631. https://doi.org/10.3390/systems13080631
Claudio Urrea — claudio.urrea@usach.cl Electrical Engineering Department, University of Santiago of Chile