Search for: All records

This paper proposes a real-size, single-shot, high-speed, and energy-efficient tensorized optical multimodal fusion network (TOMFuN) on an electro-photonic large-scale III–V-on-Si in-memory compute engine. The TOMFuN architecture leverages a memory-efficient and low-complexity self-attention for the embedding network for the text information and tensor-train and CANDECOMP/PARAFAC decompositions for compressing the model parameters in the large-scale fully connected layers. Compared to full-size counterparts, our proposed network maintains a compatible inference accuracy in multimodal sentiment analysis tasks while requiring 92.8× fewer model parameters and 51.3× fewer hardware resources. Furthermore, the impact of photonic device imperfections on the TOMFuN architecture is investigated. The simulation results show that noise-aware on-chip training exhibits superior robustness. Finally, chip performance analysis shows that our TOMFuN inference accelerator has 230.73 PetaOps computational speed, 6.51 TOPS/W power efficiency, and 2.7 µs latency with the input dimensions of 1024. 

Abstract Silicon photonics has evolved from lab research to commercial products in the past decade as it plays an increasingly crucial role in data communication for next‐generation data centers and high‐performance computing. Recently, programmable silicon photonics has also found new applications in quantum and classical information processing. A key component of programmable silicon photonic integrated circuits (PICs) is the phase shifter, traditionally realized via thermo‐optic or free‐carrier effects that are weak, volatile, and power hungry. A non‐volatile phase shifter can circumvent these limitations by requiring zero power to maintain the switched phases. Previously non‐volatile phase modulation is achieved via phase‐change or ferroelectric materials, but the switching energy remains high (pico to nano joules) and the speed is slow (micro to milliseconds). Here, a non‐volatile III‐V‐on‐silicon photonic phase shifter based on a HfO₂memristor with sub‐pJ switching energy (≈400 fJ), representing over an order of magnitude improvement in energy efficiency compared to the state of the art, is reported. The non‐volatile phase shifter can be switched reversibly using a single 100 ns pulse and exhibits excellent endurance over 800 cycles. This technology can enable future energy‐efficient programmable PICs for data centers, optical neural networks, and quantum information processing.