An official website of the United States government
Abstract The exponential growth of information stored in data centers and computational power required for various data-intensive applications, such as deep learning and AI, call for new strategies to improve or move beyond the traditional von Neumann architecture. Recent achievements in information storage and computation in the optical domain, enabling energy-efficient, fast, and high-bandwidth data processing, show great potential for photonics to overcome the von Neumann bottleneck and reduce the energy wasted to Joule heating. Optically readable memories are fundamental in this process, and while light-based storage has traditionally (and commercially) employed free-space optics, recent developments in photonic integrated circuits (PICs) and optical nano-materials have opened the doors to new opportunities on-chip. Photonic memories have yet to rival their electronic digital counterparts in storage density; however, their inherent analog nature and ultrahigh bandwidth make them ideal for unconventional computing strategies. Here, we review emerging nanophotonic devices that possess memory capabilities by elaborating on their tunable mechanisms and evaluating them in terms of scalability and device performance. Moreover, we discuss the progress on large-scale architectures for photonic memory arrays and optical computing primarily based on memory performance.
more »
« less
Neural Network Compression for Noisy Storage Devices
Compression and efficient storage ofneural network (NN)parameters is critical for applications that run on resource-constrained devices. Despite the significant progress in NN model compression, there has been considerably less investigation in the actualphysicalstorage of NN parameters. Conventionally, model compression and physical storage are decoupled, as digital storage media witherror-correcting codes (ECCs)provide robust error-free storage. However, this decoupled approach is inefficient as it ignores the overparameterization present in most NNs and forces the memory device to allocate the same amount of resources to every bit of information regardless of its importance. In this work, we investigate analog memory devices as an alternative to digital media – one that naturally provides a way to add more protection for significant bits unlike its counterpart, but is noisy and may compromise the stored model’s performance if used naively. We develop a variety of robust coding strategies for NN weight storage on analog devices, and propose an approach to jointly optimize model compression and memory resource allocation. We then demonstrate the efficacy of our approach on models trained on MNIST, CIFAR-10, and ImageNet datasets for existing compression techniques. Compared to conventional error-free digital storage, our method reduces the memory footprint by up to one order of magnitude, without significantly compromising the stored model’s accuracy.
more »
« less
- Award ID(s):
- 2106467
- PAR ID:
- 10478470
- Publisher / Repository:
- ACM
- Date Published:
- Journal Name:
- ACM Transactions on Embedded Computing Systems
- Volume:
- 22
- Issue:
- 3
- ISSN:
- 1539-9087
- Page Range / eLocation ID:
- 1 to 29
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
The hallmark of the information age is the ease with which information is stored, accessed, and shared throughout the globe. This is enabled, in large part, by the simplicity of duplicating digital information without error. Unfortunately, an ever-growing consequence is the global threat to security and privacy enabled by our digital reliance. Specifically, modern secure communications and authentication suffer from formidable threats arising from the potential for copying of secret keys stored in digital media. With relatively little transfer of information, an attacker can impersonate a legitimate user, publish malicious software that is automatically accepted as safe by millions of computers, or eavesdrop on countless digital exchanges. To address this vulnerability, a new class of cryptographic devices known as physical unclonable functions (PUFs) are being developed. PUFs are modern realizations of an ancient concept, the physical key, and offer an attractive alternative for digital key storage. A user derives a digital key from the PUF’s physical behavior, which is sensitive to physical idiosyncrasies that are beyond fabrication tolerances. Thus, unlike conventional physical keys, a PUF cannot be duplicated and only the holder can extract the digital key. However, emerging machine learning (ML) methods are remarkably adept at learning behavior via training, and if such algorithms can learn to emulate a PUF, then the security is compromised. Unfortunately, such attacks are highly successful against conventional electronic PUFs. Here, we investigate ML attacks against a nonlinear silicon photonic PUF, a novel design that leverages nonlinear optical interactions in chaotic silicon microcavities. First, we investigate these devices’ resistance to cloning during fabrication and demonstrate their use as a source of large volumes of cryptographic key material. Next, we demonstrate that silicon photonic PUFs exhibit resistance to state-of-the-art ML attacks due to their nonlinearity and finally validate this resistance in an encryption scenario.more » « less
-
Abstract Rapidly growing flexible and wearable electronics highly demand the development of flexible energy storage devices. Yet, these devices are susceptible to extreme, repeated mechanical deformations under working circumstances. Herein, the design and fabrication of a smart, flexible Li‐ion battery with shape memory function, which has the ability to restore its shape against severe mechanical deformations, bending, twisting, rolling or elongation, is reported. The shape memory function is induced by the integration of a shape‐adjustable solid polymer electrolyte. This Li‐ion battery delivers a specific discharge capacity of≈140 mAh g‐1at 0.2 C charge/discharge rate with≈92% capacity retention after 100 cycles and≈99.85% Coulombic efficiency, at 20°C. Besides recovery from mechanical deformations, it is visually demonstrated that the shape of this smart battery can be programmed to adjust itself in response to an internal/external heat stimulus for task‐specific and advanced applications. Considering the vast range of available shape memory polymers with tunable chemistry, physical, and mechanical characteristics, this study offers a promising approach for engineering smart batteries responsive to unfavorable internal or external stimulus, with potential to have a broad impact on other energy storage technologies in different sizes and shapes.more » « less
-
Demand for fast data sharing among smart devices is rapidly increasing. This trend creates challenges towards ensuring essential security for online shared data while maintaining the resource usage at a reasonable level. Existing research studies attempt to leverage compression based encryption for enabling such secure and fast data transmission replacing the traditional resource-heavy encryption schemes. Current compression-based encryption methods mainly focus on error insensitive digital data formats and prone to be vulnerable to different attacks. Therefore, in this paper, we propose and implement a new Huffman compression based Encryption scheme using lightweight dynamic Order Statistic tree (HEliOS) for digital data transmission. The core idea of HEliOS involves around finding a secure encoding method based on a novel notion of Huffman coding, which compresses the given digital data using a small sized "secret" (called as secret_intelligence in our study). HEliOS does this in such a way that, without the possession of the secret intelligence, an attacker will not be able to decode the encoded compressed data. Hence, by encrypting only the small-sized intelligence, we can secure the whole compressed data. Moreover, our rigorous real experimental evaluation for downloading and uploading digital data to and from a personal cloud storage Dropbox server validates efficacy and lightweight nature of HEliOS.more » « less
-
Abstract Compact symbolic expressions have been shown to be more efficient than neural network (NN) models in terms of resource consumption and inference speed when implemented on custom hardware such as field-programmable gate arrays (FPGAs), while maintaining comparable accuracy (Tsoiet al2024EPJ Web Conf.29509036). These capabilities are highly valuable in environments with stringent computational resource constraints, such as high-energy physics experiments at the CERN Large Hadron Collider. However, finding compact expressions for high-dimensional datasets remains challenging due to the inherent limitations of genetic programming (GP), the search algorithm of most symbolic regression (SR) methods. Contrary to GP, the NN approach to SR offers scalability to high-dimensional inputs and leverages gradient methods for faster equation searching. Common ways of constraining expression complexity often involve multistage pruning with fine-tuning, which can result in significant performance loss. In this work, we propose , a NN approach to SR specifically designed as a model compression technique, aimed at enabling low-latency inference for high-dimensional inputs on custom hardware such as FPGAs. This framework allows dynamic pruning of model weights, input features, and mathematical operators in a single training process, where both training loss and expression complexity are optimized simultaneously. We introduce a sparsity regularization term for each pruning type, which can adaptively adjust its strength, leading to convergence at a target sparsity ratio. Unlike most existing SR methods that struggle with datasets containing more than inputs, we demonstrate the effectiveness of our model on the LHC jet tagging task (16 inputs), MNIST (784 inputs), and SVHN (3072 inputs).more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Journal Article:
- https://doi.org/10.1145/3588436
