An official website of the United States government
Benchmarks are the standards by which technologies can be evaluated and fairly compared. In the field of digital circuits, benchmarks were critical for the development of CAD and FPGA tools decades ago. Hardware security is an emerging field of research where new techniques of security and vulnerability of hardware designs are being proposed in higher volume each year. Using decade-old VLSI/CAD oriented benchmarks for analyzing the techniques has many issues as these benchmarks were not developed for security research. Additionally, the rise of statistical analysis or machine learning to model vulnerabilities and solve security issues demands a very large set of samples for training purposes. Since the number of available VLSI/CAD benchmarks is limited, such volume can only be obtained through synthetic benchmark generation tools. To accommodate both of these needs, the first hardware security oriented synthetic circuit benchmark generation framework is developed in this paper. With the use of principal component analysis (PCA) and linear optimization tool, the benchmarks generated by the proposed framework are “divergent”, that is having maximum variation in structures from each other. By accommodating user inputs for desired features, the framework offers customization for generating richer and more challenging benchmarks for data-driven hardware security. With thorough experimentation, we demonstrate our framework’s scalability, the structural and functional variations in the generated benchmarks, and the advantage of structurally variant synthetic benchmarks in hardware security applications.
more »
« less
Adaptable and Divergent Synthetic Benchmark Generation for Hardware Security
Benchmarking can drive the development of technologies by facilitating standardization of features for comparison of different methods. While hardware security has seen an exponential growth in innovation throughout the last decade, the lack of sufficient benchmarks for data-driven analysis is prominent. Researchers must currently rely on decades-old VLSI benchmarks, which in most cases were not designed with security evaluation in mind. Considering the present day computational power, these benchmarks lack in both quality and quantity for usage in hardware security topics such as obfuscation and hardware Trojans. Many advanced techniques, like statistical analysis and machine learning, require a large number of samples in order to sufficiently examine the feature space. In an attempt to resolve this issue, we have developed the first synthetic benchmark generation process flow. This paper describes our novel technique that utilizes linear optimization to generate an endless number of synthetic combinational benchmarks that are adaptable to user input constraints and divergent in quantifiable structural features from input reference benchmarks. Thus, our framework offers customization for generating richer and more challenging benchmarks for data-driven hardware security. Through experimentation, we verify that our benchmarks offers more structural variation than the current benchmark suites.
more »
« less
- PAR ID:
- 10187137
- Date Published:
- Journal Name:
- IEEE/ACM International Conference on Computer-Aided Design (ICCAD)
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
The increasing complexity of System-on-Chip (SoC) designs and the rise of third-party vendors in the semiconductor industry have led to unprecedented security concerns. Traditional formal methods struggle to address software-exploited hardware bugs, and existing solutions for hardware-software co-verification often fall short. This paper presents Microscope, a novel framework for inferring software instruction patterns that can trigger hardware vulnerabilities in SoC designs. Microscope enhances the Structural Causal Model (SCM) with hardware features, creating a scalable Hardware Structural Causal Model (HW-SCM). A domain-specific language (DSL) in SMT-LIB represents the HW-SCM and predefined security properties, with incremental SMT solving deducing possible instructions. Microscope identifies causality to determine whether a hardware threat could result from any software events, providing a valuable resource for patching hardware bugs and generating test input. Extensive experimentation demonstrates Microscope's capability to infer the causality of a wide range of vulnerabilities and bugs located in SoC-level benchmarks.more » « less
-
We consider the table union search problem which has emerged as an important data discovery problem in data lakes. Semantic problems like table union search cannot be benchmarked using only synthetic data. Our current methods for creating benchmarks for this problem involve the manual curation and human label- ing of real data. These methods are not robust or scalable and perhaps more importantly, it is not clear how comprehensive the created benchmarks are. We propose to use generative AI models to create structured data benchmarks for table union search. We present a novel method for using generative models to create ta- bles with specied properties. Using this method, we create a new benchmark containing pairs of tables that are both unionable and non-unionable, but related. We use this benchmark to provide new insights into the strengths and weaknesses of existing methods. We evaluate state-of-the-art table union search methods over both existing benchmarks and our new benchmarks. We also present and evaluate a new table search method based on large language models over all benchmarks. We show that the new benchmarks are more challenging for all methods than hand-curated benchmarks. We examine why this is the case and show that our new methodology for creating benchmarks permits more detailed analysis and com- parison of methods. We discuss how our generation method (and benchmarks created using it) sheds more light into the successes and failures of table union search methods sparking new insights that can help advance the eld. We also discuss how our benchmark generation methodology can be applied to other semantic problems including entity matching and related table search.more » « less
-
Academic tabular benchmarks often contain small sets of curated features. In contrast, data scientists typically collect as many features as possible into their datasets, and even engineer new features from existing ones. To prevent over-fitting in subsequent downstream modeling, practitioners commonly use automated feature selection methods that identify a reduced subset of informative features. Existing benchmarks for tabular feature selection consider classical downstream models, toy synthetic datasets, or do not evaluate feature selectors on the basis of downstream performance. We construct a challenging feature selection benchmark evaluated on downstream neural networks including transformers, using real datasets and multiple methods for generating extraneous features. We also propose an input-gradient-based analogue of LASSO for neural networks that outperforms classical feature selection methods on challenging problems such as selecting from corrupted or second-order features.more » « less
-
Logic locking is a promising solution against emerging hardware security threats, which entails protecting a Boolean circuit using a “keying” mechanism. The latest and hitherto unbroken logic-locking techniques are based on the “corrupt-and-correct (CAC)” principle, offering provable security against input-output query attacks. However, it remains unclear whether these techniques are susceptible to structural attacks. This paper exploits the properties of integrated circuit (IC) design tools, also termed electronic design automation (EDA) tools, to undermine the security of the CAC techniques. Our proposed attack can break all the CAC techniques, including the unbroken CACrem technique that 40+ hackers taking part in a competition for more than three months could not break. Our attack can break circuits processed with any EDA tools, which is alarming because, until now, none of the EDA tools can render a secure locking solution: logic locking cannot make use of the existing EDA tools. We also provide a security property to ensure resilience against structural attacks. The commonly-used circuits can satisfy this property but only in a few cases where they cannot even defeat brute-force; thus, questions arise on the use of these circuits as benchmarks to evaluate logic locking and other security techniques.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- The DOI is not currently available.
