Getting a Sony IMX462 (Arducam Pivariety) near-IR / ultra-low-light camera working on a Raspberry Pi, with a simple Python interface, a live web feed, and a breathing / sleep-apnea detection prototype from night video.
TL;DR of the hard part: this camera "detected but would never capture" on every Pi we tried. It turned out to be an Arducam Pivariety module (MCU at I²C
0x0c), not a raw IMX462 — so the standardimx462/imx290driver was wrong. The fix + full debugging story is in DIAGNOSIS.md.
| File | What it is |
|---|---|
nir_camera.py |
Picamera2 wrapper: photos, video, night mode, numpy frames. CLI + importable class. |
breathing_detect.py |
Estimate respiratory rate and flag apnea events from a night video (ROI → band-pass → spectrum + envelope). |
mjpeg_server.py |
Live MJPEG video feed over HTTP — open http://<pi-ip>:8000/ in a browser. |
DIAGNOSIS.md |
The full "detected but won't capture" debugging story and the fix. |
USAGE.md |
Detailed student/usage guide. |
1. Get the camera working (see DIAGNOSIS.md for why):
# /boot/firmware/config.txt
dtoverlay=arducam-pivariety # add ,cam0 on a Pi 5
# install Arducam's libcamera (stock libcamera lacks Pivariety support)
wget -O install_pivariety_pkgs.sh https://github.com/ArduCAM/Arducam-Pivariety-V4L2-Driver/releases/download/install_script/install_pivariety_pkgs.sh
chmod +x install_pivariety_pkgs.sh
./install_pivariety_pkgs.sh -p libcamera_dev
./install_pivariety_pkgs.sh -p libcamera_apps
# tuning file (IMX462 ≈ IMX290 family); vc4/ on Pi 3-4, pisp/ on Pi 5
sudo cp /usr/share/libcamera/ipa/rpi/vc4/imx290.json \
/usr/share/libcamera/ipa/rpi/vc4/arducam-pivariety.json
sudo reboot2. Capture:
python3 nir_camera.py info
python3 nir_camera.py photo shot.jpg
python3 nir_camera.py video clip.mp4 --duration 15
python3 nir_camera.py night sleep.mp4 --duration 60 --exposure-ms 50 --gain 123. Live feed:
python3 mjpeg_server.py # then open http://<pi-ip>:8000/4. Analyse breathing (on a laptop with opencv-python scipy matplotlib):
python3 breathing_detect.py sleep.mp4 --show-roi roi.png # position the ROI
python3 breathing_detect.py sleep.mp4 --roi 700,300,500,400 # rate + apnea + plot
from nir_camera import NIRCamera
with NIRCamera() as cam:
frame = cam.capture_array() # (H, W, 3) RGB uint8 -> OpenCV / PyTorch
cam.capture_photo("shot.jpg")
cam.record_video("clip.mp4", 10)
# night / apnea: FIXED exposure & gain so the breathing signal is stable
with NIRCamera(resolution=(1280, 720), framerate=15) as cam:
cam.night(exposure_ms=50, gain=8.0)
cam.record_video("sleep.mp4", 60)The IMX462 is a Sony STARVIS sensor: 2 MP, ultra-low-light, sensitive into the near-infrared. It is a color sensor (Bayer RGGB) that also sees NIR — so in visible light it produces a normal RGB image. You get true NIR / night vision by:
- removing visible light and adding an IR illuminator (850 nm faint-red, or 940 nm invisible),
- optionally an IR-pass filter on the lens (then the image is monochrome NIR).
Quick sensor test: point a TV remote at the lens on the live feed — if the remote's IR LED glows bright on screen, the sensor is seeing near-IR.
For breathing detection, color is irrelevant (the pipeline uses grayscale).
ROI mean-intensity (or motion) → detrend → band-pass 0.1–0.6 Hz (6–36 bpm) →
respiratory rate from the spectral peak (Welch) → Hilbert-envelope amplitude →
flag spans where breathing is suppressed for ≥ 10 s as apnea/hypopnea events.
Supports a chest ROI (robust primary) plus an optional nostril ROI to
corroborate. See USAGE.md for details and next steps
(motion magnification, ground-truth validation, on-device real-time).
- ✅ Working & verified on Raspberry Pi 3 Model B+ (Bookworm, kernel 6.12).
- Same fix applies to Pi 5 (use
dtoverlay=arducam-pivariety,cam0and thepisp/tuning path).
MIT — see LICENSE.
Developed in the Sustainability Lab, IIT Gandhinagar, for an NIR-based sleep-apnea / breathing-detection project.