We assess the Value of Information (VoI) for inspecting components in systems managed
by multiple agents, using game theory and Nash equilibrium analysis. We focus on binary systems made up by binary components which can be either intact or damaged. Agents taking maintenance actions are responsible for the repair costs of their own components, and the penalty for system failure is shared among all agents. The precision of inspection is also considered, and we identify the prior and posterior Nash equilibrium with perfect or imperfect inspections. The VoI is assessed for the individual agents as well as for the whole set of agents, and the analysis consider series, parallel and general systems. A negative VoI can trigger the phenomenon of Information Avoidance (IA), where rational agents prefer not to collect free information. We discuss whether it is possible that the VoI is negative for one or for all agents, for the agents with inspected or uninspected components, and for the total sum of VoIs.
more »
« less
Optimal adaptive inspection and maintenance for redundant systems
Optimal exploration of engineering systems can be guided by the principle of Value of Information (VoI), which accounts for the topological important of components, their reliability and the management costs. For series systems, in most cases higher inspection priority should be given to unreliable components. For redundant systems such as parallel systems, analysis of one-shot decision problems shows that higher inspection priority should be given to more reliable components. This paper investigates the optimal exploration of redundant systems in long-term decision making with sequential inspection and repairing. When the expected, cumulated, discounted cost is considered, it may become more efficient to give higher inspection priority to less reliable components, in order to preserve system redundancy. To investigate this problem, we develop a Partially Observable Markov Decision Process (POMDP) framework for sequential inspection and maintenance of redundant systems, where the VoI analysis is embedded in the optimal selection of exploratory actions. We investigate the use of alternative approximate POMDP solvers for parallel and more general systems, compare their computation complexities and performance, and show how the inspection priorities depend on the economic discount factor, the degradation rate, the inspection precision, and the repair cost.
more » « less- Award ID(s):
- 1653716
- PAR ID:
- 10547150
- Publisher / Repository:
- SAGE Publications
- Date Published:
- Journal Name:
- Proceedings of the Institution of Mechanical Engineers, Part O: Journal of Risk and Reliability
- Volume:
- 235
- Issue:
- 4
- ISSN:
- 1748-006X
- Format(s):
- Medium: X Size: p. 568-579
- Size(s):
- p. 568-579
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Many infrastructure systems can be modeled as networks of components with binary states (intact, damaged). Information about components’ conditions is crucial for the maintenance process of the system. However, it is often impossible to collect information of all components due to budget constraints. Several metrics have been developed to assess the importance of the components in relation to maintenance actions: an important component is one that should receive high maintenance priority. Instead, in this paper we focus on the priority to be assigned for component inspections and information collection. We investigate metrics based on system level (global) and component level (local) decision making after inspection for networks with different topology, and compare these results with traditional ones. We then discuss the computational challenges of these metrics and provide possible approximation approaches.more » « less
-
null (Ed.)We develop computable metrics to assign priorities for information collection on binary systems composed of binary components. Components are worth inspecting because their condition states are uncertain, and system functioning depends on them. The Value of Information (VoI) enables assessment of the impact of information in decision making under uncertainty, including the component’s reliability and role in the system, the precision of the observation, the available maintenance actions and the expected economic loss. We introduce the VoI-based metrics for system-level (“global”) and component-level (“local”) maintenance actions, analyze the properties of these metrics, and apply them to series and parallel systems. We discuss their computational complexity in applications to general network systems and, to tame the complexity for the local metric assessment, we present a heuristic and assess its performance on some case studies.more » « less
-
This paper introduces a simple efficient learning algorithms for general sequential decision making. The algorithm combines Optimism for exploration with Maximum Likelihood Estimation for model estimation, which is thus named OMLE. We prove that OMLE learns the near-optimal policies of an enormously rich class of sequential decision making problems in a polynomial number of samples. This rich class includes not only a majority of known tractable model-based Reinforcement Learning (RL) problems (such as tabular MDPs, factored MDPs, low witness rank problems, tabular weakly-revealing/observable POMDPs and multi-step decodable POMDPs ), but also many new challenging RL problems especially in the partially observable setting that were not previously known to be tractable. Notably, the new problems addressed by this paper include (1) observable POMDPs with continuous observation and function approximation, where we achieve the first sample complexity that is completely independent of the size of observation space; (2) well-conditioned low-rank sequential decision making problems (also known as Predictive State Representations (PSRs)), which include and generalize all known tractable POMDP examples under a more intrinsic representation; (3) general sequential decision making problems under SAIL condition, which unifies our existing understandings of model-based RL in both fully observable and partially observable settings. SAIL condition is identified by this paper, which can be viewed as a natural generalization of Bellman/witness rank to address partial observability. This paper also presents a reward-free variant of OMLE algorithm, which learns approximate dynamic models that enable the computation of near-optimal policies for all reward functions simultaneously.more » « less
-
Abstract To be responsive to dynamically changing real-world environments, an intelligent agent needs to perform complex sequential decision-making tasks that are often guided by commonsense knowledge. The previous work on this line of research led to the framework called interleaved commonsense reasoning and probabilistic planning (i corpp ), which used P-log for representing commmonsense knowledge and Markov Decision Processes (MDPs) or Partially Observable MDPs (POMDPs) for planning under uncertainty. A main limitation of i corpp is that its implementation requires non-trivial engineering efforts to bridge the commonsense reasoning and probabilistic planning formalisms. In this paper, we present a unified framework to integrate i corpp ’s reasoning and planning components. In particular, we extend probabilistic action language pBC + to express utility, belief states, and observation as in POMDP models. Inheriting the advantages of action languages, the new action language provides an elaboration tolerant representation of POMDP that reflects commonsense knowledge. The idea led to the design of the system pbcplus2pomdp , which compiles a pBC + action description into a POMDP model that can be directly processed by off-the-shelf POMDP solvers to compute an optimal policy of the pBC + action description. Our experiments show that it retains the advantages of i corpp while avoiding the manual efforts in bridging the commonsense reasoner and the probabilistic planner.more » « less
