Skip to content

fix(libcamera): publish mid-exposure timestamp#2489

Open
JosephTLockwood wants to merge 1 commit into
PhotonVision:mainfrom
JosephTLockwood:feature/libcamera-mid-exposure-timestamp
Open

fix(libcamera): publish mid-exposure timestamp#2489
JosephTLockwood wants to merge 1 commit into
PhotonVision:mainfrom
JosephTLockwood:feature/libcamera-mid-exposure-timestamp

Conversation

@JosephTLockwood

Copy link
Copy Markdown
Contributor

Why

PhotonVision's libcamera path currently publishes the start-of-exposure timestamp from libcamera::controls::SensorTimestamp directly. The scene a detector integrates over spans [SOE, SOE+exposure], so the closest single-instant approximation is the midpoint of that window. The existing e2e-latency design doc already acknowledges this residual ("sensor will remain collecting light for up to the total integration time").

Magnitude at typical FRC exposures (~5–15 ms): a 2.5–7.5 ms earlier-bias is removed, which is ~12–37 mm of pose error at 5 m/s drive speed.

What

  • LibcameraGpuFrameProvider: read getFrameExposureTimeUs and add exposure/2 (ns) to the published timestamp.
  • When the driver returns 0 (libcamera did not populate ExposureTime for the frame), the correction is skipped and behaviour matches today.
  • e2e-latency.md: documents the new behaviour and the rolling-shutter caveat.

Dependency

Requires PhotonVision/photon-libcamera-gl-driver#33 (the paired JNI surface). That PR must merge and a driver release that bumps libcameraDriverVersion in build.gradle must land before this PR compiles against the upstream Maven artifact. Local smoke build was verified with ./gradlew publishToMavenLocal on the driver branch.

Scope notes

  • libcamera path only. The V4L/USB path has separate caveats around proprietary exposure scales (some Arducam V4L drivers) and unreliable per-frame metadata under auto-exposure, and is explicitly out of scope.
  • No feature flag — the driver-returns-0 fallback already covers the unsupported-sensor case, so a config toggle would be redundant.
  • Residual row-dependent bias on rolling-shutter sensors is not corrected here.

SensorTimestamp marks start-of-exposure. The scene a detector
integrates over spans [SOE, SOE+exposure], so mid-exposure is the
closest single-instant approximation. Add exposure/2 to the
published timestamp using the new getFrameExposureTimeUs JNI call;
falls back to SOE when the driver returns 0.

Depends on PhotonVision/photon-libcamera-gl-driver#33.
@JosephTLockwood JosephTLockwood requested a review from a team as a code owner May 12, 2026 23:04
@github-actions github-actions Bot added documentation Anything relating to https://docs.photonvision.org backend Things relating to photon-core and photon-server labels May 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backend Things relating to photon-core and photon-server documentation Anything relating to https://docs.photonvision.org

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant