More Like this

1. Does Unsupervised Architecture Representation Learning Help Neural Architecture Search?

   Yan, Shen; Zheng, Yu; Ao, Wei; Zeng, Xiao; Zhang, Mi (January 2020, Conference on Neural Information Processing Systems)

   Existing Neural Architecture Search (NAS) methods either encode neural architectures using discrete encodings that do not scale well, or adopt supervised learning-based methods to jointly learn architecture representations and optimize architecture search on such representations which incurs search bias. Despite the widespread use, architecture representations learned in NAS are still poorly understood. We observe that the structural properties of neural architectures are hard to preserve in the latent space if architecture representation learning and search are coupled, resulting in less effective search performance. In this work, we find empirically that pre-training architecture representations using only neural architectures without their accuracies as labels improves the downstream architecture search efficiency. To explain this finding, we visualize how unsupervised architecture representation learning better encourages neural architectures with similar connections and operators to cluster together. This helps map neural architectures with similar performance to the same regions in the latent space and makes the transition of architectures in the latent space relatively smooth, which considerably benefits diverse downstream search strategies. 

   more » « less
2. Persistent Processor Architecture

   https://doi.org/10.1145/3613424.3623772  

   Zeng, Jianping; Jeong, Jungi; Jung, Changhee (October 2023, IEEE/ACM International Symposium on Microarchitecture)
3. AHomogeneous Transformer Architecture

   Gan, Yulu; Poggio, Tomaso (September 2023, Center for Brains, Minds and Machines (CBMM))

   While the Transformer architecture has made a substantial impact in the field of machine learning, it is unclear what purpose each component serves in the overall architecture. Heterogeneous nonlinear circuits such as multi-layer RELU networks are interleaved with layers of soft-max units. We introduce here a homogeneous architecture based on Hyper Radial Basis Function (HyperBF) units. Evalua- tions on CIFAR10, CIFAR100, and Tiny ImageNet demonstrate a performance comparable to standard vision transformers. 

   more » « less
4. The SuperCodelet architecture

   Jose M Monsalve Diaz, Kevin Harms (January 2022, The 1st International Workshop on Extreme Heterogeneity Solutions (ExHet’ 22, in conjunction with PPoPP’22)
5. SPLICEcube Architecture: An Extensible Wi-Fi Monitoring Architecture for Smart-Home Networks

   Malik, Namya (May 2022, Dartmouth Digital Commons)

   The vision of smart homes is rapidly becoming a reality, as the Internet of Things and other smart devices are deployed widely. Although smart devices offer convenience, they also create a significant management problem for home residents. With a large number and variety of devices in the home, residents may find it difficult to monitor, or even locate, devices. A central controller that brings all the home’s smart devices under secure management and a unified interface would help homeowners and residents track and manage their devices. We envision a solution called the SPLICEcube whose goal is to detect smart devices, locate them in three dimensions within the home, securely monitor their network traffic, and keep an inventory of devices and important device information throughout the device’s lifecycle. The SPLICEcube system consists of the following components: 1) a main cube, which is a centralized hub that incorporates and expands on the functionality of the home router, 2) a database that holds network data, and 3) a set of support cubelets that can be used to extend the range of the network and assist in gathering network data. To deliver this vision of identifying, securing, and managing smart devices, we introduce an architecture that facilitates intelligent research applications (such as network anomaly detection, intrusion detection, device localization, and device firmware updates) to be integrated into the SPLICEcube. In this thesis, we design a general-purpose Wi-Fi architecture that underpins the SPLICEcube. The architecture specifically showcases the functionality of the cubelets (Wi-Fi frame detection, Wi-Fi frame parsing, and transmission to cube), the functionality of the cube (routing, reception from cubelets, information storage, data disposal, and research application integration), and the functionality of the database (network data storage). We build and evaluate a prototype implementation to demonstrate our approach is scalable to accommodate new devices and extensible to support different applications. Specifically, we demonstrate a successful proof-of-concept use of the SPLICEcube architecture by integrating a security research application: an "Inside-Outside detection" system that classifies an observed Wi-Fi device as being inside or outside the home. 

   more » « less