Head scan scene doesn't look 3D in binocular view

[goatchurchprime]

The head scan example looked flat to my eyes, so I separated it out into a greybox scene called simple_vr_binocular with the right controller joystick control to rotate, and which I could export to the Quest3. (I've also changed the shading to bring out more colours and structure.)

It was still very flat.

I put in a couple of yellow markers in 3D to use as parallax reference points and confirmed it was a flat image.

Getting these shader things to work in VR with its binocular view isn't straightforward (see all the work that goes into making mirrors work), and it's not something I have ever done (I've wisely stayed away from it).

I made a quick hack to volume_shader.gdshader of the volume_layers_addon to add in the EYE_OFFSET value to the world camera.

void vertex() {
	world_position = VERTEX;
	world_camera = (inverse(MODELVIEW_MATRIX) * vec4(0, 0, 0, 1)).xyz; //object space
	//world_camera = ( CAMERA_MATRIX  * vec4(0, 0, 0, 1)).xyz; //uncomment this to raymarch in world space
	world_camera += EYE_OFFSET;
}

This produced a much more 3D looking visualization, but it's not right according to the parallax of my marker points. I don't know what I am doing here, because I thought that the two EYE offsets were already part of the CAMERA_MATRIX etc, but there might be some issue created by the scaling of the model by (-0.25,-0.5,-0.25) that this counteracts.

This sort of thing can only be properly debugged with a clean reference data set (of a dense cube or wire outline of crossed cylinders) where the alignment and positions of the geometry is obvious and easy to reproduce and specify, rather than some huge malformed fuzzy head model. (If we can prove there's an issue, we can submit a ticket to the godot_volume_layers project.

In addition, we could also talk about what kind of interface we want for these models. Arguably instancing this as something you pick up and throw round the room in VR isn't as good as putting it on a pedestal with some controls to rotate and set the colour thresholds of it. If it works, I also think being able to draw marker points into it that are in the head coordinate space could be useful in terms of facilitating communication between players.

[ronpandolfi]

You're totally right here. I'm surprised I didn't notice the eye offset was missing.

I rewrote the raymarching shader based on this information. I think it looks good, but I could use your opinion. The grayboxed scene should have the raymarching shader implemented now.

https://github.com/lbl-camera/Ascribe-XR/blob/master/shaders/raymarching.gdshader

I also have an optimized version of the shader from volume_layers_addon here: https://github.com/lbl-camera/Ascribe-XR/blob/master/shaders/volume_shader_improved.gdshader

I'll look at applying this info there as well. It'd be good to be able to swap between shaders. The raymarching shader doesn't interact well with other objects in its space because it is projected onto the cube faces. The layered shader is better in those cases.

I don't know how to benchmark these, but the raymarching shader should be the best quality in most cases, and doesn't feel any different performance-wise.

I agree we should plan out a UI for interacting with volume specimens. I'm also not attached to the scifi-themed room. Something more museum-like would be good. My 3d modeling skills are very limited though.

[ronpandolfi]

I looked at this more closely. The reason the layered volume shader needs EYE_OFFSET is that world_camera = (inverse(MODELVIEW_MATRIX) * vec4(0, 0, 0, 1)).xyz; //object space gives you the center camera, not the eye-specific camera. Inverting MODELVIEW_MATRIX removes the EYE_OFFSET because VIEW_MATRIX contains it, and you’re undoing that transform.

To preserve correct stereoscopic vision, you must manually re-add EYE_OFFSET after the inversion.

My understanding is that the correctness of the stereoscopic view shouldn't be affected by the model transformations. The MODEL_MATRIX encodes those transformations already. To test this, I added an AnimationPlayer to the greybox (d1b2830) that changes the model scaling. As the scaling changes, the perception of depth does not seem to be affected.