Search for: All records

Satellite image time series (SITS) segmentation is crucial for many applications, like environmental monitoring, land cover mapping, and agricultural crop type classification. However, training models for SITS segmentation remains a challenging task due to the lack of abundant training data, which requires fine-grained annotation. We propose S4, a new self-supervised pretraining approach that significantly reduces the requirement for labeled training data by utilizing two key insights of satellite imagery: (a) Satellites capture images in different parts of the spectrum, such as radio frequencies and visible frequencies. (b) Satellite imagery is geo-registered, allowing for fine-grained spatial alignment. We use these insights to formulate pretraining tasks in S4. To the best of our knowledge, S4 is the first multimodal and temporal approach for SITS segmentation. S4’s novelty stems from leveraging multiple properties required for SITS self-supervision: (1) multiple modalities, (2) temporal information, and (3) pixel-level feature extraction. We also curate m2s2-SITS, a large-scale dataset of unlabeled, spatially aligned, multimodal, and geographic-specific SITS that serves as representative pretraining data for S4. Finally, we evaluate S4 on multiple SITS segmentation datasets and demonstrate its efficacy against competing baselines while using limited labeled data. Through a series of extensive comparisons and ablation studies, we demonstrate S4’s ability as an effective feature extractor for downstream semantic segmentation. 

Distribution matching can be used to learn invariant representations with applications in fairness and robustness. Most prior works resort to adversarial matching methods but the resulting minimax problems are unstable and challenging to optimize. Non-adversarial likelihood-based approaches either require model invertibility, impose constraints on the latent prior, or lack a generic framework for distribution matching. To overcome these limitations, we propose a non-adversarial VAE-based matching method that can be applied to any model pipeline. We develop a set of alignment upper bounds for distribution matching (including a noisy bound) that have VAE-like objectives but with a different perspective. We carefully compare our method to prior VAE-based matching approaches both theoretically and empirically. Finally, we demonstrate that our novel matching losses can replace adversarial losses in standard invariant representation learning pipelines without modifying the original architectures—thereby significantly broadening the applicability of non-adversarial matching methods. 

Mobile devices continuously beacon Bluetooth Low Energy (BLE) advertisement packets. This has created the threat of attackers identifying and tracking a device by sniffing its BLE signals. To mitigate this threat, MAC address randomization has been deployed at the link-layer in most BLE transmitters. However, attackers can bypass MAC address randomization using lower-level physical-layer fingerprints resulting from manufacturing imperfections of radios. In this work, we demonstrate a practical and effective method of obfuscating physical-layer hardware imperfection fingerprints. Through theoretical analysis, simulations, and field evaluations, we design and evaluate our approach to hardware imperfection obfuscation. By analyzing data from thousands of BLE devices, we demonstrate obfuscation significantly reduces the accuracy of identifying a target device. This makes an attack impractical, even if a target is continuously observed for 24 hours. Furthermore, we demonstrate the practicality of this defense by implementing it by making firmware changes to commodity BLE chipsets. 

Interfacing electronics with optical fiber networks is key to the long-distance transfer of classical and quantum information. Piezo-optomechanical transducers enable such interfaces by using gigahertz-frequency acoustic vibrations as mediators for converting microwave photons to optical photons via the combination of optomechanical and piezoelectric interactions. However, despite successful demonstrations, efficient quantum transduction remains out of reach due to the challenges associated with hybrid material integration and increased loss from piezoelectric materials when operating in the quantum regime. Here, we demonstrate an alternative approach in which we actuate 5-GHz phonons in a conventional silicon-on-insulator platform. In our experiment, microwave photons resonantly drive a phononic crystal oscillator via the electrostatic force realized in a charge-biased narrow-gap capacitor. The mechanical vibrations are subsequently transferred via a phonon waveguide to an optomechanical cavity, where they transform into optical photons in the sideband of a pump laser field. Operating at room temperature and atmospheric pressure, we measure a microwave-to-optical photon conversion efficiency of 1.72±0.14×10⁻⁷in a 3.3 MHz bandwidth. Our results mark a stepping stone towards quantum transduction with integrated devices made from crystalline silicon, which promise efficient high-bandwidth operation and integration with superconducting qubits. Additionally, the lack of need for piezoelectricity or other intrinsic nonlinearities makes our approach applicable to a wide range of materials for potential applications beyond quantum technologies.