Question for Parallel Light Sources

[HarryPeverell]

Hi ,

First of all, I would like to express my gratitude for the excellent work you've done on this project. It’s truly impressive, and I’ve found your approach to be very insightful.

I have a question regarding the dataset used in the project. It seems that the dataset is designed with point light sources in mind. If the light rays are parallel (as in the case of distant light sources or sunlight), could you please share the necessary modifications to the workflow for handling such changes in the light source? Any guidance on this would be greatly appreciated.

I look forward to your response, and thank you again for your great work.

Best regards,
Harry

[iamNCJ]

Hi, thanks for your interest. Parallel / Directional lighting can be implemented by setting the shadow_ray_d to the lighting direction and shadow_ray_o to a far distant point. Related code is at:

NRHints/models/neus_hint_model.py

Lines 373 to 381 in b10f965

	def get_visibility(self, pls, target_points, up_sample_steps=4,
	cos_anneal_ratio=1.0,
	offset=1e-2, perturb=False):
	device = pls.device
	n_samples = self.config.renderer.n_shadow_samples
	n_importance_samples = self.config.renderer.n_shadow_importance_samples
	with nullcontext() if self.config.renderer.shadow_hint_gradient else torch.no_grad():
	shadow_ray_o = pls
	shadow_ray_d = target_points - shadow_ray_o

which can be changed into:

def get_visibility(self, olat_ray_dir, target_points, up_sample_steps=4,
                   cos_anneal_ratio=1.0,
                   offset=1e-2, perturb=False):
    device = olat_ray_dir.device
    n_samples = self.config.renderer.n_shadow_samples
    n_importance_samples = self.config.renderer.n_shadow_importance_samples
    with nullcontext() if self.config.renderer.shadow_hint_gradient else torch.no_grad():
        shadow_ray_o = target_points + olat_ray_dir * 10.
        shadow_ray_d = target_points - shadow_ray_o
        light_norms = torch.linalg.norm(shadow_ray_d, ord=2, dim=-1, keepdim=True)
        sample_dist = light_norms / n_samples

Besides, you should also change the lighting direction in the specular hint calculation part. From:

NRHints/models/neus_hint_model.py

Line 591 in b10f965

lit_dirs = F.normalize(rays_pl - hit_points, dim=-1, p=2)

to:

lit_dirs = rays_pl

To pass the lighting direction into the system with minimal changes, you can set the pl_pos in your custom dataset as your lighting direction (as they are all 3-channel vectors), and use pl_pos in the codebase as lighting direction.

It is also highly recommended to check the shadow and specular hint maps after your modification to make sure everything is working as expected.