[LukeTheProtogen]

Eye tracking: correctness fixes, gaze filtering, and pupillometry

Ok, so this will be long while to fit in but I'll try to answer most questions I can and cover up most of the modifications done here.

I decided to make a research on the eye-tracking path, one of the feats added is the 1 euro filter (which I used on my tracking projects), improve the fidelity of what reaches SteamVR, then build on top.

OBS: Each layer is independent and can be reviewed or reverted separately.

30 files changed, and approximately ~1160 insertions.

1. Correctness fixes

'customHandleData' - three memory-safety defects

usb_thread_hook.cpp parses VD general-data blocks straight off the USB pipe, so it is a trust boundary for data the driver DOES NOT control. It has three problems:

• Item bounds checked against the wrong base. item>offset is relative to the current block, but the check compared offset + size against the size of the whole payload. From the second block onwards it was too permissive by exactly the number of bytes already consumed, allowing reading past the end of the buffer.

• 'numItems' was trusted without bonds. A 16-bit count of 12-byte entries (up to 760 KB) could be read past the block header before any other check ran in.

• A zero header>size hung the USB thread. The walk advanced by 'header>size' each iteration, so zero meant no forward progress.

So the block walking is now in common/hmd2_gen_data.h, free of I/O and SteamVR dependencies so it can be tested without hardware.

claimDriverMutex — infinite spin, inverted return

auto now = std::chrono::steady_clock::now();
while (!held && now < now + std::chrono::milliseconds(5000)) {   // always true
  held = IpcMutex_TryLock(m_driverActiveMutex);
}
return !held;   // false on success

now was never re-read, so the documented 5 second timeout did not exist as far as I know and the loop was an unthrottled TryLock spin. 100% of a core for as long as another process held the mutex, the return value was also inverted, so the called discard it which is why it probably wasn't noticed

Now it uses a fixed deadline with 1 ms backoff, and returns the true value.

The caller in command_thread.cpp logs and keeps running instead of exiting, the mutex only backs the "driver active" flag, where areas TriggerEffectManager::Update drives controller trigger effects every 10ms and must NOT stop.
releaseDriverMutex is now only called if the mutex is actually claimed.

Short USB transfers published stale gaze frames

TransferPipe returns a byte count, annnnd a timed-out but partial transfer returns a positive value smaller than hmd2_gaze_tatus_t. PollAndProcess only branched on ==0 and < 0, then forwarded a full-size struct. So the problem was that the buffer was `static, and the untouched tail had the previous frame so a partial read published a half-stale frame carrying a valid magic and a timestamp, which nothing downstream could be detecting.

Now the transfers are now discarded and logged to be throttled. This means the buffer is no longer static

HookLib::InstallHook discarded failures

detour-->hook() result was ignored, so a hook that failed to install was unindentifiable from one whose target nerver ran, now it returns a bool and logs the target address.

After this, it surfaced five hooks that I noticed a silent failure behind it
caesar_usb_thread.cpp:558-568 - GetInterface, GetEndpoint, OnConnected and OnDisconnected
PollAndProcess. Those are pre-existing, not introduced here, all five pass a non-null ppOriginal, so nothing in this change affects whether they succeed. Probably short virtual stubs with no room for a 5-byte jump. Probably deserves its own issue.

BTW: fixed a bug in the same function (the trampoline pointer used when ppOriginal is null was a stack local handed to a heap-allocated detour that outlives it.

WARNING: A NOTE REGARDING THE DUPLICATED RVA AT 0X123130

CaesarManager::SetupManager and CaesarManager::initialize were both hooked at 0x123130. this is maybe a copy-pasted RVA, so one of the fixes I looked into it was to disable the duplicate.

Well, the reading is wrong, and acting on it is a downsize. Polyhook chains detours at the same address: setupManagerHook wrapped initializeHook, so its body ran on every initialize, clearing firmwareLoaded and firmwareVersion after the original returned. Those two lines are the only place in the toolkit that writes those fields.

This opened PR folds the reset into CaesarManager_initializeHook in the same order the chain produced it, so the behaviour is not modified, and leaves the seteupManager hook disabled with a TODO because its real RVA is unkown yet. PS: a hook on an unexpected address is not necessarily inert.

2. Data fidelity

vGazeTarget was receiving a direction, not a point...

openvr_driver.h documents vGazeTarget as a fixation pointer in the same space as vGazeOrigin, This means the driver was writing the normalised gaze direction, Consumers computing normalize(target - origin) therefore got a ray skewed by the 30mm eye offset against a unit-length vector.

Now computed as origin + direction * convergence_distance, using foveated.convergence_distance_mm
(clamped 0.1–20 m, with a configurable fallback). This is the single largest accuracy change here.

Validation composition

bActive / bTracked / bValid all derived from is_gaze_dir_combined_valid alone.
is_gaze_origin_combined_valid was never consulted, so an invalid origin was published as good and after the changes made above, a bad origin corrupts the target too. bActive now means the "tracker is working" and survives a momentary dropout. bTracked/ bValid describes whether if the sample is usable.

Eye Gaze was advertised even when disabled

Prop_SupportsXrEyeGazeInteraction_Bool and CreateEyeTrackingComponent ran undonditionally whole the hooks that feed the mare gated on disableGaze. with the gaze disabled, applications enabled gaze-driven UI that never moved which is worse for the mthan being told there is no eye tracking. Both are now gated on the same DriverSettings::IsGazeEnabled() helper so both cannot drift.

3. Robustness + changes

++ psvr2_toolkit_gaze_image_copy: this existing call returns a pointer into shared memory after releasing the lock; the producer can recycle that slot mid-read. the old entry point is kept and deprecated.

++ per-slot image sizes: the slot is a fixed 0x200100 bytes but the headset does not fill it. the measure comes from the hardware, the total_size is 80256 bytes (400x200) plus a 256 byte header) , so this cuts 26x off a memcpy that runs on the USB thread under a mutex. Falls back to the full slot if total_size looks wrong.

++ Calibration write: the blob was truncated in place on the USB thread on every matching packet. A crash mid-write left the user with no usable calibration. Now temp-file + rename, skipping entirely when unchanged, and the load path validates magic, size and transfer result instead of breaking an unchecked file at the headset.

++ volatile bool -> std::atomic_ref: isFulfilled is the cross-process completion handshake and was read without the mutex. meaning volatile is not a synchronisation primiteve. atomic_ref rather than thread_local frame counters. The CAPI's "last frame seen" counters were global, so two threads polling gaze stole each other's new frame edges and each silently missed roughly half the stream. Loader reports unresolved symbols. Every entry point is an inline shim though a func pointer, so one missing symbol became a null call on the first use, crashing with no module name and no message. The loader now counts failures and callers can refuse to start with a readable error.

4. New features

Gaze filtering (common/gaze.filter.h)

Nothing filteredthe gaze stream, raw per-frame direction went directly to SteamVR, microsaccades and all. A fixed low-pass strong enough to settle a fixation and ends adding visible lag to every saccade, so I decided to implement a one-euro filter (I STRONGLY RECOMMEND READING ABOUT IT!) with velocity-adaptive cutoff and an I-VT classifier that detects saccades and passes them through untouched, plus a short hold across blinks!

Two flags to mention:

• The I-VT threshgold must sit above the tracker's noise floor, not just above eye drift.
• One-euro's beta is dimensioned by the input's units. Gaze directions are unit vectors whose derivate is 0.44 rads/s at 25°/s, so the reading 0.007 moves the cutoff by 0.003 Hz - no adaptation at all. It is now expressed in Hz per deg/s.

This applies and was tested to the SteamVR path only! The C API and legacy IPC server keep delivering raw samples.

5. Pupillometry (common/pupillometry.h)

pupil_dia_mm has always been there but since nothing was using it, I decided to work with it. The pipeline: blink gating with guard bands on both sides, foreshortening correction, binocular fusion, low-pass, rolling median baseline, and a luminance-robust activity index.

Every stage was validated against a real 120 Hz capture.

Fitting the eye camera axis rather than assuming it removes *all of the gaze-angle dependence. PupilCameraAxisCalibrator does that fit from live data, the axes persist to disk so a calibration done in one application is picked up by every other CAPI client.

LIMITATION THAT I RAN ACROSS, STATED IN THE HEADER AND IN THE UI: with no display-luminance signal, I didn't find a way to calculate a way to detect if a "scene got brigther or not" to work with the pupils, BUT, that's why I thought of a activityIndex to work it out, it count abrupt dilations rather than measuring level, so it rides out light changes. deltaMm and relative are labelled descriptive and explicitly not presented as cognitive-load measurements.

Test tooling (psvr2_toolkit_capi_test)

• Session recorder every field of hmd2_gaze_status_t to CSV plus a host timestamp, polled on its own thread so nothing is dropped when gaze outruns vsync, with frame-gap detection. IR eye dumper with it too!

• Diagnostics panel: measured IPD, pos_guide fit values, blink rate/duration/PERCLOS, convergence distance, and the pupillometry output with a calibration flow.

This is implemented because several remaining tasks depends on questions which only the hardware can answer, what pos_guide reads on a badly fitted headset, whether measured IPD agrees with what SteamVR sends back, it measures and displays, it deliberately does not change driver behavior on the strength of a guess.

Compatibility

• BufferData gained a field, so the driver, psvr2_toolkit_capi.dlland libcustomshare must ship together now. Every IPC object name is now suffixed with a layout generationm, so a mismatched build sees NOTHING rather than a silently reading the same memory in the wrong offsets, the driver simply reads as inactive, which is a symptom someone can diagnose.

• New C API exports are additive. Existing clients resolve names they know, verified against a third-party module built before these changes.

• psvr2_toolkit_gaze_image is DEPRECATED, but not removed,

• common.h stays C++17 safe, it ships in the public capi_dist headers, do the one C++20 construct is behind a feature check.

SOME FINAL OBSERVATIONS (What this DOESN'T INCLUDE)

• setupManager's real RVA — hook disabled with a TODO; its effect is preserved (see above).
• Eyelid openness — the open field on the IPC wire is still filled from binary blink. Now
  unblocked: the IR buffer is 400x200.
• IPD → Prop_UserIpdMeters_Float — measured and displayed only, until it is settled whether
  something upstream already sets it.
• Legacy IPC protocol bump — those clients still receive raw values.
• Luminance compensation for pupillometry — not a scoping decision; the telemetry does not exist.

That's it, for any further questions or reviews, I'll gladly be opened for them, my goal here is to assist in any way I can, I've been working with jailbreaks and VR for a while now, so I would be very happy to help!

[meowybnuuy]

Holy vibecode, this PR is pure slop.

[Aerialflame7125]

"My agent is better than you"

[LukeTheProtogen]

Holy vibecode, this PR is pure slop.

Not 100% vibecoded, there was researches, reading, reviews and written code behind it, if is there any way I could help with implementations, I can write them by hand, explain it or work with deeper researches.

Again, the goal here is to make something so everyone can use

[meowybnuuy]

Dude, I've used LLMs for coding before, I can tell when code is vibecoded. Give up the act.

[LukeTheProtogen]

Dude, I've used LLMs for coding before, I can tell when code is vibecoded. Give up the act.

Not acting, I have no reasons for it. You can ask about it, I did use LLMs for coding, I don't hide it, BUT, again, I did researches, reviewed it, read it, and there is hand written code.

I can write it by hand, it WILL take a while, and I don't see a problem about it, again, goal here to help and assist

[WolfWings]

You're also chucking a single enormous squashed pull request in that does a LOT more things than it advertises all bundled into a single 'take it or leave it' thing.

Break this thing up into orderly chunks if you need / want to overhaul certain aspects as prep-work or cleanup, and don't hide your revision history by squashing things to a single massive patch like this.

Basically: Show your work, and don't do multiple different things in a single PR, or else it screams "no matter what I say this was vibe coded with me just pressing 👍 occasionally" to many coders.