When the first (ambient) layer is anisotropic and θ ≠ 0, the calculation of the in-plane wavevector ξ = nx·sin(θ) uses only the x-component of the refractive index. For a rotated anisotropic ambient, this is not well-defined and produces NaN.
Fixing this properly requires computing ξ from the full dielectric tensor of the ambient medium, accounting for the direction-dependent refractive index at the given angle of incidence.
When the first (ambient) layer is anisotropic and θ ≠ 0, the calculation of the in-plane wavevector ξ = nx·sin(θ) uses only the x-component of the refractive index. For a rotated anisotropic ambient, this is not well-defined and produces NaN.
Fixing this properly requires computing ξ from the full dielectric tensor of the ambient medium, accounting for the direction-dependent refractive index at the given angle of incidence.