An official website of the United States government
Deep reinforcement learning (deep RL) has emerged as an effective tool for developing controllers for legged robots. However, vanilla deep RL often requires a tremendous amount of training samples and is not feasible for achieving robust behaviors. Instead, researchers have investigated a novel policy architecture by incorporating human experts' knowledge, such as Policies Modulating Trajectory Generators (PMTG). This architecture builds a recurrent control loop by combining a parametric trajectory generator (TG) and a feedback policy network to achieve more robust behaviors. In this work, we propose Policies Modulating Finite State Machine (PM-FSM) by replacing TGs with contact-aware finite state machines (FSM), which offers more flexible control of each leg. This invention offers an explicit notion of contact events to the policy to negotiate unexpected perturbations. We demonstrated that the proposed architecture could achieve more robust behaviors in various scenarios, such as challenging terrains or external perturbations, on both simulated and real robots.
more »
« less
Reverse Engineering of RTL Controllers from Look-Up Table Netlists
Verification of FPGA-based designs and comprehension of legacy designs can be aided by the process of reverse engineering the flattened Look-up Table (LUT) level netlists to high-level RTL representations. We propose a tool flow to extract Finite State Controllers by identifying control registers and progressively improving the accuracy of register classification. A control unit consists of one or more Finite State Machines (FSMs) which manage the execution of datapath units. The proposed tool flow has two phases. Phase 1 extracts the potential state/control registers. Phase 2 identifies the exact list of state/control registers and groups FSMs. The main goal of the proposed work is to improve the accuracy of control register identification. Three types of controllers used for experimental evaluation are standalone FSM designs with no datapath units, datapaths with a single FSM, and datapaths with multiple FSMs. Accuracy is observed to be 73% to 100% in controllers with multiple FSMs, 100% in controllers with a single FSM and standalone FSM controller designs. The average accuracy of control register detection over all the real-world designs considered is 98%.
more »
« less
- Award ID(s):
- 1916722
- PAR ID:
- 10536731
- Publisher / Repository:
- IEEE
- Date Published:
- ISBN:
- 979-8-3503-2769-4
- Page Range / eLocation ID:
- 1 to 6
- Format(s):
- Medium: X
- Location:
- Foz do Iguacu, Brazil
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Due to the globalization of Integrated Circuit supply chain, hardware Trojans and the attacks that can trigger them have become an important security issue. One type of hardware Trojans leverages the “don’t care transitions” in Finite-state Machines (FSMs) of hardware designs. In this article, we present a symbolic approach to detecting don’t care transitions and the hidden Trojans. Our detection approach works at both register-transfer level (RTL) and gate level, does not require a golden design, and works in three stages. In the first stage, it explores the reachable states. In the second stage, it performs an approximate analysis to find the don’t care transitions and any discrepancies in the register values or output lines due to don’t care transitions. The second stage can be used for both predicting don’t care triggered Trojans and for guiding don’t care aware reachability analysis. In the third stage, it performs a state-space exploration from reachable states that have incoming don’t care transitions to explore the Trojan payload and to find behavioral discrepancies with respect to what has been observed in the first stage. We also present a pruning technique based on the reachability of FSM states. We present a methodology that leverages both RTL and gate-level for soundness and efficiency. Specifically, we show that don’t care transitions and Trojans that leverage them must be detected at the gate-level, i.e., after synthesis has been performed, for soundness. However, under specific conditions, Trojan payload exploration can be performed more efficiently at RTL. Additionally, the modular design of our approach also provides a fast Trojan prediction method even at the gate level when the reachable states of the FSM is known a priori . Evaluation of our approach on a set of benchmarks from OpenCores and TrustHub and using gate-level representation generated by two synthesis tools, YOSYS and Synopsis Design Compiler (SDC), shows that our approach is both efficient (up to 10× speedup w.r.t. no pruning) and precise (0% false positives both at RTL and gate-level netlist) in detecting don’t care transitions and the Trojans that leverage them. Additionally, the total analysis time can achieve up to 1.62× (using YOSYS) and 1.92× (using SDC) speedup when synthesis preserves the FSM structure, the foundry is trusted, and the Trojan detection is performed at RTL.more » « less
-
null (Ed.)Finite-state machine (FSM) is a fundamental computation model used by many applications. However, FSM execution is known to be “embarrassingly sequential” due to the state dependences among transitions. Existing solutions leverage enumerative or speculative parallelization to break the dependences. However, the efficiency of both parallelization schemes highly depends on the properties of the FSM and its inputs. For those exhibiting unfavorable properties, the former suffers from the overhead of maintaining multiple execution paths, while the latter is bottlenecked by the serial reprocessing among the misspeculation cases. Either way, the FSM parallelization scalability is seriously compromised. This work addresses the above scalability challenges with two novel techniques. First, for enumerative parallelization, it proposes path fusion. Inspired by the classic NFA to DFA conversion, it maps a vector of states in the original FSM to a new (fused) state. In this way, path fusion can reduce multiple FSM execution paths into a single path, minimizing the overhead of path maintenance. Second, for speculative parallelization, this work introduces higher-order speculation to avoid the serial reprocessing during validations. This is a generalized speculation model that allows speculated states to be validated speculatively. Finally, this work integrates different schemes of FSM parallelization into a framework—BoostFSM, which automatically selects the best based on the relevant properties of the FSM. Evaluation using real-world FSMs with diverse characteristics shows that BoostFSM can raise the average speedup from 3.1× and 15.4× of the existing speculative and enumerative parallelization schemes, respectively, to 25.8× on a 64-core machine.more » « less
-
null (Ed.)Various hardware security solutions have been developed recently to help counter hardware level attacks such as hardware Trojan, integrated circuit (IC) counterfeiting and intellectual property (IP) clone/piracy. However, existing solutions often provide specific types of protections. While these solutions achieve great success in preventing even advanced hardware attacks, the compatibility of among these hardware security methods are rarely discussed. The inconsistency hampers with the development of a comprehensive solution for hardware IC and IP from various attacks. In this paper, we develop a security primitive generator to help solve the compatibility issue among different protection techniques. Specifically, we focus on two modern IC/IP protection methods, logic locking and watermarking. A combined locking and watermarking technique is developed based on enhanced finite state machines (FSMs). The security primitive generator will take user-specified constraints and automatically generate an FSM module to perform both logic locking and watermarking. The generated FSM can be integrated into any designs for protection. Our experimental results show that the generator can facilitate circuit protection and provide the flexibility for users to achieve a better tradeoff between security levels and design overheads.more » « less
-
We develop 5GBaseChecker— an efficient, scalable, and dynamic security analysis framework based on differential testing for analyzing 5G basebands' control plane protocol interactions. 5GBaseChecker first captures basebands' protocol behaviors as a finite state machine (FSM) through black-box automata learning. To facilitate efficient learning and improve scalability, 5GBaseChecker introduces novel hybrid and collaborative learning techniques. 5GBaseChecker then identifies input sequences for which the extracted FSMs provide deviating outputs. Finally, 5GBaseChecker leverages these deviations to efficiently identify the security properties from specifications and use those to triage if the deviations found in 5G basebands violate any properties. We evaluated 5GBaseChecker with 17 commercial 5G basebands and 2 open-source UE implementations and uncovered 22 implementation-level issues, including 13 exploitable vulnerabilities and 2 interoperability issues.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- https://doi.org/10.1109/ISVLSI59464.2023.10238540
