Introduction

The photonic spin Hall effect (PSHE) is an optical analogy of electronic spin Hall effect and its inherent physical mechanism is the spin-orbit interaction of light, which delineates the mutual influence of the spin (circular polarization) and the trajectory of light beam. [1][2][3][4][5] When a linearly polarized light propagates in an inhomogeneous medium, the components with the opposite spins drift along the opposite directions perpendicular to the refractiveindex gradient, causing the light beam to split into two circularly polarized beams separated on either side of the incident plane. The spin-dependent splitting in the PSHE is sensitive to the state of the incident photons and the physical parameters of the interface, thus it holds great promise for various applications, such as the biosensor, optical differential manipulation, image processing, and precision metrology, and so forth. [6][7][8][9][10] Unfortunately, the PSHE is typically very weak and the spindependent displacements are always at the subwavelength scale, severely inhibiting its practical applications. 11,12 Until now, even if many strategies including polarization control, 13 anisotropic impedance mismatching, 14 constructing surface plasmonic platform 15 and various metastructures [16][17][18] have been proposed to enhance the PSHE, the largest photonic spin Hall shifts achieved by these methods are only hundreds of times of the incident wavelength. Besides, the element materials in most of previously reported modulation methods preserve the time-reversal symmetry such that the Hall conductivity (equivalently the off-diagonal permittivity element) is generally zero. [13][14][15][16][17][18] In our recent studies, we have shown that the PSHE can be greatly enhanced via breaking the time-reversal symmetry of system. [19][20][21] Particularly, the in-plane spin-dependent shift can be theoretically increased to more than 1000 times of the incident wavelength in a monolayer of black phosphorus with a nonzero Hall conductivity. 21 Nevertheless, rigorous physical conditions (e.g., intense magnetic field, ultralow temperature, and appropriate straining) are necessary for our previously reported model systems, which are difficult to be realized in practical observations. [19][20][21] In consequence, it is significant to exploit naturally low-symmetry crystalline materials to manipulate the PSHE.

Recently, the low-symmetry Bravais crystal β-Ga2O3, which is endowed with nontrivial relative orientation (neither parallel nor orthogonal) of several optical transitions, has spurred tremendous research interest due to its non-orthogonal principal crystal axes. 22,23 Especially, the discovering of hyperbolic shear polaritons (HShPs) in monoclinic β-Ga2O3 further inspires new Please do not adjust margins Please do not adjust margins directions for nanophotonics in low-symmetry materials. [23][24][25][26] The HShPs are a new class of polariton modes, which were first proposed and experimentally confirmed by N. C. Passler et al in 2022. 23 They emerge in crystalline materials in which the permittivity tensor cannot be diagonalized. They have been directly observed by near-field real-space nanoimaging and preliminarily shown remarkable potential for enhancing the near-field thermal radiation. 27,28 Even so, the study on HShPs is in its infancy, and lots of new directions including the modulation of PSHE by the HShPs have not yet been explored thus far.

Propelled by the advances in PSHE and natural hyperbolic polaritons, in the present work, we theoretically propose a new method to colossally amend the PSHE. Giant photonic spin Hall shifts are demonstrated in an Otto-type multilayer structure of KRS5 prism/sensing medium/β-Ga2O3, which are ascribed to the broken crystal lattice symmetry and the related excitation of HShPs in monoclinic β-Ga2O3 crystal. Furthermore, evolutions of in-plane and transverse spin-dependent displacements with various factors (e.g., incident angle, excitation wavelength, azimuth angle and doping concentration of β-Ga2O3, thickness and refractive index of sensing medium) are clearly unveiled. Finally, the biosensing performance of our proposed model system is also inspected in detail.

Model system and the theoretical method

There are five different polymorphisms of bulk Ga2O3, i.e., α-, β-, γ-, σ-, and ε-Ga2O3. 29 Among them, β-Ga2O3 is the most thermodynamically stable phase. β-Ga2O3 belongs to the C2/m (or C2h 3 ) space group (number 12) and has a base centered monoclinic crystal structure, as shown in Fig. 1(a). The unit cell contains four Ga2O3 molecules and has two crystallographically different Ga atoms. The Ga(I) ions are at tetrahedral sites (with four oxygen neighbors) and the Ga(II) ions are at octahedral sites (with six oxygen neighbors). These low-symmetry Bravais lattices make β-Ga2O3 exhibit non-orthogonal principal crystal axes. As a result, the dielectric permittivity tensor of β-Ga2O3 has major polarizability directions which are strongly dependent on the wavelength λ, with off-diagonal terms that cannot be completely removed through coordinate rotation. The wavelength-dependent permittivity of β-Ga2O3 can be described as a third-rank tensor with identical off-diagonal elements 0 0 0 0

The monoclinic permittivity tensor consists of the highfrequency contributions, the dipole charge resonances, and the free charge-carrier contributions. Its main calculation equations are given in eqn (S1)-(S6) in the ESI †, and its detailed parametrization and calculation method are available in ref. 22. Fig. 2 presents the optical spectra of diagonal and offdiagonal permittivity elements ɛxx, ɛyy, ɛzz and ɛxy at a free chargecarrier density N = 1 × 10 18 cm -3 . The parameters of anisotropic charge carrier mobility are μx = 296 cm 2 V -1 s -1 and μy = μz = 37 cm 2 V -1 s -1 . In general, the hyperbolic polaritons can be produced when either the real part of one diagonal permittivity element is negative and the other two are positive or two are negative and one is positive. Nonetheless, the strong dielectric response of offdiagonal elements (ɛxy = ɛyx ≠ 0) in monoclinic β-Ga2O3 crystal typically prevents the electromagnetic propagation angle from being aligned with the principal axes, giving rise to a remarkable shearing effect of polariton modes. This shear phenomenon in the dielectric response results in extreme anisotropic propagation of surface waves, supporting a polariton class of HShPs. 23 The dispersion isofrequency contours (IFCs) of HShPs at a frequency ω with a momentum (kx, ky, kz) can be obtained via solving the Maxwell equations as follows 27

(

)

where E = [Ex, Ey, Ez] T , k and k0 are the wave vectors in β-Ga2O3 and vacuum, respectively. The dispersion relation including the IFCs can thus be calculated by

In the experimental model, an Otto-type prism-coupling geometry is designed to excite the HShPs, as sketched in Fig. 1(b). The incident medium and substrate are KRS5 glass prism and bulk β-Ga2O3, respectively. The KRS5 glass has a good transparency in the spectral range from 5 to 40 μm and its refractive index is derived from 30

There is a biosensing medium with a micron-sized thickness d to be sandwiched between the prism and β-Ga2O3. The refractive index of the initial calibration biosensing medium is ns = 1.419, corresponding to the organic solvent of ethyl acetoacetate.

Hereafter, unless explicitly specified, the thickness of biosensing medium is set as 8.3 μm.

It is assumed that one Gaussian wave packet with a monochromatic wavelength impinges from the prism upon the surface of biosensing medium with an incident angle θi. The coordinate system used to define the optical response of β-Ga2O3 is illustrated Fig. 1. To evaluate the influence of azimuth angle  on the PSHE, we also tune the rotation angle of β-Ga2O3 around the z axis in plane. In the spin basis set, the linearly polarized state of electromagnetic wave can be decomposed into two orthogonal spin components, i.e., the left-and right-circular polarization (LCP and RCP) components. 21,31 Under the paraxial approximation, spin-dependent Hall shifts of these two orthogonal components in reflected beam can be mathematically derived. 31 The PSHE is sensitive to the polarization states of incident photons. We inspect the condition that incident photons are only p-polarized in this work. The corresponding in-plane and transverse spin Hall shifts for LCP and RCP components, as indicated in Fig. 1(b), are given by 21,31 1 Re

where ki is the wave vector in the incident medium, the The Fresnel reflection coefficients rpp, rsp, rss, and rps can be calculated via using a generalized 4 × 4 transfer matrix formalism for light propagation in anisotropic stratified media. 32,33

Results and discussions

We first consider the influence of the incident angle θi and the wavelength λ on the PSHE. Fig. 3 21,34 The incident angles at which the reflection of p-polarized light is nearly extinguished correspond to the Brewster's angles. Fig. 3(a) demonstrates that the Brewster's angles are diverse for different λ values.

Previous studies have shown that photonic spin Hall shifts are sensitive to the minimum values of |rpp| such that they generally give extreme values around the Brewster's angles. [19][20][21] This change rule is reproduced through comparing Fig. 3(a) with Fig. 3(c-d). Especially, the HShPs stem from the nontrivial relative orientation (neither parallel nor orthogonal) of several optical transitions which contribute to a net polarization (cannot be aligned with the crystal axes) at a given frequency. As a result, even in the rotated coordinate system, the permittivity tensor of β-Ga2O3 has off-diagonal terms associated with shear phenomena. The PSHE originates from the presence of permittivity gradient. The off-diagonal terms of monoclinic β-Ga2O3 crystal contribute more degrees of freedom to permittivity variation gradient, which can effectively enhance the PSHE. Besides, previous study has shown that the largest displacement of one spin component can be up to only half of Gaussian beam waist. 35 In practical experiment, thus, the beam waist should be large enough in order to observe the displacement with three orders of magnitude of the incident wavelength. and Re(ɛzz) gradually decrease as the doping concentration increases. Therefore, not only does the wave-vector magnitude of HShPs change but the direction of hyperbolic IFC also rotates within the monoclinic plane as the doping concentration varies [Fig. 4(b)].

Influenced by the changed permittivity tensor (accompanied by the tuned hyperbolic IFC), the largest values of S1 in the ESI †, we may draw a conclusion that the enhancement of PSHE is associated with the HShPs supported by monoclinic β-Ga2O3 crystal.

We continue to focus on the case of log10N = 18.0 and λ = 19.6 μm. Fig. 6 exhibits the optical spectra of Fresnel reflection coefficients and photonic spin Hall shifts versus the incident angle θi and the azimuth angle . To demonstrate the effects of symmetry breaking of β-Ga2O3 crystal structure in the PSHE, we also calculate the Fresnel reflection coefficients and photonic spin Hall shifts after artificially setting the off-diagonal permittivity element ɛxy of β-Ga2O3 to be zero, as shown in Figs.

Conclusions

To summarize, the enhancement of spin-dependent splitting of reflected light via the HShPs is investigated through designing an Otto-type configuration of KRS5 prism/sensing medium/β-Ga2O3. We show that the in-plane (or transverse) photonic spin Hall shifts can be amplified to be more than 1000 (or 200) times of the incident wavelength at appropriate structural and physical parameters. Besides, the azimuthal dispersions of photonic spin Hall shifts present non-mirror-symmetric patterns at tuning the rotation angle of β-Ga2O3 around the z axis in plane. All of these extraordinary merits are attributed to the natural symmetry breaking and the related excitation of HShPs in monoclinic β-Ga2O3 crystal. By virtue of the giant PSHE, the maximum biosensing sensitivity can reach to ~10 7 μm/RIU which is at least two orders of magnitude larger than previously reported PSHE