bench-automation

An end-of-line test station for a 12 V embedded PC, written in Python — complete with simulated instruments, so you can run the whole thing without a single piece of hardware.

It does what a production test does: power the board up through a lab supply, check inrush and idle current, measure every supply rail through a relay multiplexer and a DMM, capture the power-on with an oscilloscope, watch the serial console during boot and compare it with a golden log, log in, and run on-device checks (identity, RAM, SSD health, USB ports, Ethernet link, CPU temperature, RTC). Every unit gets a pass/fail verdict and a report.

Quick start

Needs Python 3.10+. The simulated instruments, scpi-sim, come from PyPI.

python -m venv .venv && . .venv/bin/activate
pip install -e ".[dev]"

bench demo                 # healthy device -> PASS
bench demo --fault smart   # SSD with bad sectors -> FAIL in "Storage"

  [PASS ] Power on           Inrush peak 2.248 A, Supply voltage 12.0 V
  [PASS ] Supply rails       5V0 5.0148 V, 3V3 3.2851 V, 1V8 1.7997 V
  [PASS ] Boot               Boot time 1.86 s
  [PASS ] Boot log           Known faults 0, Facts changed 0, Missing lines 0
  ...
  [FAIL ] Storage            Reallocated sectors, Pending sectors out of limits
  ...
  Result: FAIL  (EPC-24-00137, 3.4 s)
  Report: reports/EPC-24-00137_20261025-173106/report.html

Each run writes result.json, measurements.csv and a self-contained report.html (with the full console log) to reports/<serial>_<timestamp>/.

Injectable faults

The simulated device can be broken on purpose, to show that the test actually catches real failure modes:

--fault	Defect	Caught by
short	short circuit on the 12 V input	Power on — supply hits current limit, power is removed, rest skipped
inrush_spike	2 ms inrush spike of 4 A	Power on — only the scope: polling the supply misses it
rail_slow	3.3 V rail ramps up too slowly	Power on — only the scope
rail_1v05	1.05 V core rail missing	Supply rails
boot_hang	kernel hangs in a SATA reset loop	Boot — timeout, last console line in the report; Boot log — diagnosis sata-link, boot-incomplete
sata_slow	SATA link falls back to 3.0 Gbps, board otherwise fine	Boot log only — diagnosis sata-link
can_missing	CAN controller not detected, no error printed	Boot log only — diagnosis can-missing
usb_port	one dead USB port	USB ports
smart	SSD with reallocated/pending sectors	Storage
no_link	no Ethernet link	Ethernet
overtemp	CPU running hot	Temperature

Power-on capture

Polling a supply's current every few milliseconds misses what happens in between. So the scope is armed before the supply switches on: channel 1 sees the input current through a current probe, channel 2 a rail. The step checks the peak current, the rail's level and its ramp time, and the report shows both curves.

scope:                       # leave the section out to run without a scope
  time_per_div: 0.05
  trigger_amps: 0.5
  max_rail_ramp_s: 0.05      # 10 % to 90 %
  current: {channel: 1, volts_per_amp: 0.1, volts_per_div: 0.1, center_volts: 0.3}
  rail: {name: "3V3", channel: 2, volts_per_div: 1, center_volts: 2, min_volts: 3.0}

The commands are those of Keysight InfiniiVision scopes. Other models need their own driver in drivers.py. None of this has run against a real scope yet.

Boot log check

A board can boot, log in and pass every on-device check while something is already wrong: a SATA link at half speed, a controller nothing in the sequence talks to. So the console output of each boot is compared with the log of a known-good board, using boot-log-diff: timestamps, addresses and other values that change between boots are masked, and each deviation fails the "Boot log" step. Known faults get a diagnosis in the report, also when the boot hangs.

boot:
  golden_log: golden_boot.log   # relative to station.yaml; leave out to skip the step
  mask: ['epc-[0-9a-f]+']       # values that differ between good boards
  ignore: ['optional-module']   # lines to leave out
  rules: [our-boards.toml]      # own known faults

To create the golden log for your board, save the console output of one boot of a known-good unit, from power-on up to the login prompt.

Architecture

 bench run / bench demo
        │
  EndOfLineTest  ── limits & wiring from station.yaml
        │
  drivers.py ── PowerSupply, Multimeter, Multiplexer  (PyVISA, SCPI)
            └── SerialConsole                          (pyserial)
        │
  ┌─────┴──────────── simulator (bench simulate) ──────────────┐
  │ PSU :5025  DMM :5026  MUX :5027  console :5028  scope :5029 │
  │   scpi-sim instruments, wired to one EmbeddedPC device      │
  └─────────────────────────────────────────────────────────────┘

• The simulated instruments are scpi-sim: SCPI over raw TCP sockets, exactly like LAN instruments on port 5025 — short/long command forms, the SCPI path rule, error queue (SYST:ERR?) and status registers.
• The device under test, EmbeddedPC in src/bench/sim/dut.py, is a scpi-sim device: its 12 V input is wired to the supply, its four rails to multiplexer channels 101–104, its serial console is a TCP port like a terminal server (socket://host:port).
• The device model behaves physically: inrush current, constant-current limit pulling the supply voltage down on a short, boot timing, rails that only come up with sufficient input voltage. A running board loads the supply like a resistor, so a current limit below its demand pulls the input down (and below 10.8 V the board browns out). Closing two rail channels at once is refused (-221), as it would short the rails on a real fixture.

Running on real hardware

Only the configuration changes. Point the resources in station.yaml at your instruments and pass it with --config:

visa_backend: ""                                   # NI-VISA / Keysight IO Libraries
resources:
  psu: "USB0::0x2A8D::0x1002::MY12345678::INSTR"
  dmm: "TCPIP0::192.168.1.20::INSTR"
  mux: "TCPIP0::192.168.1.21::5025::SOCKET"
  console: "/dev/ttyUSB0"

bench run --config my-station.yaml

The drivers use common SCPI commands (VOLT, CURR, OUTP, MEAS:VOLT:DC?, ROUT:CLOS (@101)); instruments with different dialects need a small driver subclass.

Tests

pytest

The suite runs the full sequence against a healthy device and against every injected fault, and checks that exactly the expected step fails.

License

MIT