Edge-Assisted Reinforcement-Learning Face Tracking

An edge-side control prototype for a face-tracking teleconference terminal built with OpenHarmony and HiSilicon development boards. A camera endpoint detects a participant's face, this Python service converts the reported bounding box into either rule-based or DDPG-assisted motion decisions, and a second embedded endpoint drives a multi-servo arm.

The complete system received the National First Prize (Graduate Division), Application Track National Finals, 5th National College Student Embedded Chip and System Design Competition, 2022. See the official University of Electronic Science and Technology of China announcement.

Important

This repository contains only the historical Python edge controller. It is not a complete release of the award-winning system and cannot reproduce the end-to-end prototype by itself.

System overview

flowchart LR
    A["Hi3516DV300 camera endpoint<br/>OpenHarmony · NNIE · YOLOv2"]
    B["Python edge controller<br/>rule baseline or DDPG"]
    C["Hi3861V100 actuator endpoint<br/>OpenHarmony · serial/PWM control"]
    D["Multi-servo tracking arm"]

    A -- "face box over TCP" --> B
    B -- "servo command over TCP" --> C
    C --> D

Loading

The competition report describes the camera endpoint as a Hi3516DV300-based Taurus board and the actuator endpoint as a Hi3861V100-based Pegasus board. The camera-side detector sends face coordinates to the edge controller; the controller returns a discrete vertical/horizontal target, encoded as a servo command.

What is in this repository

Path	Role
main.py	Historical entry point and mode selection.
Environment.py	TCP endpoints, state acquisition, reward calculation, and actuator dispatch.
Manual.py	Deterministic face-centering baseline.
Algorithm.py	DDPG interaction, replay-memory, training, and checkpoint orchestration.
Servo.py	Mapping from logical arm positions to the serial servo protocol.
RL/	Actor-critic networks, exploration noise, and replay memory.
test/tcp_server.py	Early TCP test utility retained as project history.

The following components are not included:

• Hi3516DV300 or Hi3861V100 firmware and OpenHarmony build projects;
• YOLOv2/NNIE model conversion files, detector weights, and camera pipeline;
• trained DDPG checkpoints, replay buffers, experimental logs, or datasets;
• PCB, mechanical, wiring, and enclosure design assets; and
• a hardware-independent simulator or automated end-to-end test harness.

Control model

The controller expects a comma-separated face box in this order:

right,left,bottom,top

Environment.py augments those four coordinates with the current vertical and horizontal arm positions, yielding the six-value state

[right, left, bottom, top, vertical, horizontal]

The two logical actions are discrete vertical and horizontal target positions in the inclusive range 0..10. Servo.py translates them into PWM commands for four servo channels. The reward used by the DDPG path is

reward = 300 - distance(face_center, frame_center)

for a nominal 1920 × 1080 frame. The rule baseline uses the same face-center offsets with hand-tuned movement thresholds.

Recreating the software environment

The dependency set below was smoke-tested for importing the controller modules and instantiating the neural networks. It does not constitute hardware validation.

conda env create -f environment.yml
conda activate rl-face-tracking-edge
python -c "import gym, numpy, torch, tensorboardX; from Servo import Servo; from RL.ddpg import Actor, Critic"

The pinned environment uses Python 3.8.20, NumPy 1.23.5, PyTorch 1.13.1 (CPU), Gym 0.21.0, and TensorBoardX 2.6.2.2. Its MKL 2021.4 pins are intentional for compatibility with the historical PyTorch Windows binary. CPU-only PyTorch is sufficient for code inspection and smoke tests; the historical controller can use CUDA under a separately validated CUDA-enabled PyTorch installation.

Running against the original hardware protocol

main.py is a hardware-coupled historical entry point: importing or running it opens sockets and enters a long control loop. Before using it, review the module-level flags and network constants rather than treating it as a general command-line application.

Setting	Current default	Effect
rl_brain	False	Uses the deterministic controller in Manual.py; set to True for DDPG.
test	True	Sends actions without collecting transitions or training.
self_ip	0.0.0.0	Listens on every network interface.
t_port	3516	Receives camera-side face boxes.
p_port	3861	Sends commands to the actuator endpoint.

With both trusted hardware peers available and the constants reviewed:

python main.py

The repository does not include trained checkpoints. The DDPG path attempts to load local model and replay-memory files and falls back when they are absent, so its behavior should not be interpreted as a reproduction of the competition result.

Security and deployment limits

This prototype predates a production threat model. The TCP protocol has no authentication, encryption, integrity protection, explicit message framing, or peer authorization; the service binds to all interfaces by default. Model and replay-memory loading also uses PyTorch/pickle serialization, which must never be used with untrusted files.

Run the controller only on an isolated, trusted laboratory network. Do not expose ports 3516 or 3861 to the public Internet. A production redesign should add authenticated transport, bounded parsing, timeouts, connection lifecycle handling, command validation, and safe model-artifact verification.

Reproducibility status

This repository is best treated as a source snapshot of the edge-control component. Static compilation, dependency imports, neural-network construction, and deterministic servo-message generation can be checked without the boards. End-to-end face tracking, latency, accuracy, stability, and award-demo behavior require the missing firmware, models, mechanical platform, calibration, and experimental records and are therefore not reproducible from this repository alone.

Project record and citation

The system was developed by Zhuobin Huang, Yuhang Shen, and Ruxin Qi for the 2022 competition. If you reuse or discuss this source-code snapshot, cite the repository metadata in CITATION.cff. The original Chinese technical report documents the complete prototype; it is not redistributed in this repository.

Licensing and third-party code

No project-wide open-source license is granted at present. Some files preserve or appear to adapt upstream reinforcement-learning implementations under their own licenses. See THIRD_PARTY_NOTICES.md before reuse or redistribution. The absence of a root license means all other rights remain reserved unless stated otherwise.