Cybersecurity is an ever growing concern for businesses and individuals alike. While it is not always possible to prevent cybersecurity incidents, one should always strive to mitigate them promptly and to learn from them as much as possible. Thus, it is imperative in the aftermath of an incident to promptly discover the techniques that the threat actors used to breach security. By discovering how threat actors operate, we can learn to develop more effective cyber defences and detect-sufficiently early-cyber-attacks that bypass these defences. To this end, victims of cyber incidents can launch cyber-forensic investigations.
Since we do not know in advance which adversarial techniques the threat actors used, we have to prioritize the investigation of these techniques under uncertainty, not knowing if the investigation of a technique will waste precious time or reveal crucial information. In prior work, Nisioti et al. (2021a) demonstrated that one can effectively prioritize the investigation of techniques based on datasets of prior incidents, exploiting the fact that most threat actors follow common tactics. However, this existing data-driven approach is based on a heuristic that does not approximate optimal decisions, regardless of the size of the dataset or the available computational power.
To address this limitation, we propose a principled datadriven approach for prioritization. We introduce a novel model of cyber-forensic investigations based on Markov decision processes, which enables us to formally define the prioritization problem. Note that this problem is inherently challenging for multiple reasons. First, datasets of prior incidents are limited in size as many businesses are reluctant to share detailed information about their security. Second, although threat actors follow common tactics, it is inherently difficult to predict which particular techniques were used in a particular incident.
In light of these challenges, we propose a computational approach for decision support that combines a nonparametric machine-learning model, based on k-nearest neighbor, with a Monte Carlo tree search. By working directly with the data instead of training a parametric model, we get as much "mileage" out of the limited data as possible.
The remainder of this paper is organized as follows. In Section 2, we model cyber-forensic investigations as Markov decision processes and formulate the problem of cyber-forensic decision support. In Section 3, we describe our computational approach, which is based on k-NN regression and Monte Carlo tree search. In Section 4, we evaluate our approach on datasets of real-world incidents, demonstrating that it outperforms the state-of-the-art approach. In Section 5, we give a brief overview of related work. Finally, in Section 6, we provide concluding remarks.
To formally define the problem of providing optimal decision support for cyber forensics, we first introduce fundamental assumptions and notations for the key elements of
Adversarial Techniques We consider a forensic investigation whose objective is to discover-as soon as possiblewhat techniques the threat actors used in the incident that is under investigation. The motivation for discovering how the threat actors breached security is to learn from the incident and to prevent or mitigate future breaches. We let A denote the set of all techniques that threat actors may have used in an incident (e.g., DLL search order hijacking, spearphishing attachment, drive-by compromise). In practice, the set A can be taken from a well-known knowledge base, such as MITRE ATT&CK (Barnum 2012), which provides a comprehensive list of common adversarial techniques.
Incident Next, we let I Y ⊆ A denote the set of techniques that were used by threat actors in incident I; and for ease of presentation, we let I N = A \ I Y denote the set of techniques that were not used in incident I. A key aspect of cyber-forensic investigations is that I Y is not known by the forensic experts at the beginning of the investigation. Since an investigation is typically launched when a breach is detected, forensic experts might know some elements of I Y (e.g., techniques that triggered an intrusion detection system, leading to the detection of the breach); however, discovering set I Y is the very objective of the investigation. To capture this uncertainty that forensic experts face, we model the set of techniques I Y as a random variable, whose realization is unknown at the beginning of the investigation. The distribution of this random variable represents the threat actors' tactics, that is, how likely they are to use certain subsets of techniques in a cyber attack.
Prior Incidents In practice, we can estimate the distribution of I Y from prior incidents, which have already been investigated. Note that for this estimation, we can use a public repository of prior incidents, which were perpetrated by other threat actors against other targets (e.g., MITRE Cyber Threat Intelligence Repository from MITRE Corporation). While there are differences between how different threat actors operate, most threat actors do follow common tactics. Therefore, subsets of techniques that were frequently used in prior incidents are likely to have been used in the incident that we are currently investigating. We let I denote the set of prior incidents; and we assume that for each prior incident Î ∈ I, we know the set ÎY ⊆ A and that ÎY was drawn from the same distribution as I Y .
Cyber-forensic Investigation During the investigation of incident I, forensic experts discover the realization of I Y step-by-step. In each step, the experts choose and investigate a technique a ∈ A that they have not investigated yet, and they discover whether or not the threat actors used technique a. Note that we assume this discovery to be perfect (i.e., if a technique is investigated, experts correctly learn whether it was used or not); our model and computational approach could be generalized in a straightforward manner to account for false negatives and positives in discovery, but this is not the focus of our work.
To investigate whether a particular technique was used by the threat actors, forensic experts have to spend time, effort, resources, etc. We let constant C a denote the cost of investigating technique a ∈ A, which represents the time, effort, resources, etc. spent. In practice, we can estimate these costs based on domain experts' knowledge (Nisioti et al. 2021a). On the other hand, discovering that a technique was used by the threat actors yields benefit since it provides information that we can use to prevent or mitigate future breaches. We let constant B a denote the benefit obtained from discovering that technique a ∈ A was used in the incident. In practice, we can estimate these benefit values based on the impacts of using various techniques, which we can take from well-known knowledge bases (e.g., MITRE ATT&CK (Barnum 2012)). Some experts may inherently prefer investigating certain actions over others, e.g., because certain actions provide crucial information on how the adversary breached the system. All such preferences can be captured by the benefit values.
Based on the assumptions and notations introduced in the preceding subsection, we now model the cyber-forensic investigation of an incident as a Markov decision process (MDP) (Puterman 2014), whose steps correspond to the step-by-step investigation of the adversarial techniques.
Modeling the cyber-forensic investigation as an MDP provides the foundation for formulating our decision-support problem in the next subsection. To define an MDP, we have to specify the state space of the process, the set of actions that can be taken in each state, the transition probabilities between subsequent states, and the immediate reward for taking a particular action in a particular state.
and
For ease of notation, we will let Pr[a | Y t , N t ] denote the first probability (i.e., probability that the threat actors used technique a given that the state is ⟨Y t , N t ⟩). In practice, we have to estimate these conditional probabilities based on the set of prior incidents I.
Rewards We formulate rewards to capture the benefits of the cyber-forensic investigation. The experts obtain a benefit B a from investigating technique a only if technique a was used in the incident. Hence, if the process transitions from state ⟨Y t , N t ⟩ to state ⟨Y t+1 , N t+1 ⟩ = ⟨Y t ∪ {a}, N t ⟩, then we let the reward for step t be B a ; if the process transitions to state ⟨Y t+1 , N t+1 ⟩ = ⟨Y t , N t ∪ {a}⟩, then we let the reward for step t be 0.
We represent the decision-support system as a policy π, which maps a state ⟨Y t , N t ⟩ to a recommended action a t ∈ A \ (Y t ∪ N t ) in each time step t. Our goal is to provide a policy that maximizes the expected rewards obtained during the forensic investigation. Following Nisioti et al. (2021a), we formulate this objective with a budget limit G on the total cost C at :
where T limit = max T T t=0 C at ≤ G (i.e., T limit is the last step before the investigation budget G is exhausted), and C at is the cost of investigating the action that is chosen in time step t. Note that we focus on this objective because it is practical and enables a fair comparison with DISCLOSE (Nisioti et al. 2021a); in Equation ( 8), we will provide a more conventional formulation with temporal discounting.
In practice, the budget may be flexible, in which case our goal is to provide a policy that attains a good cost-benefit tradeoff. We will quantify this tradeoff in our experiments using the area under the cost-benefit curve (i.e., AUC for the expected benefit as a function of the budget limit).
To solve the decision-support problem based on the objective in Equation ( 3), we propose a computational approach based on Monte Carlo tree search (MCTS) and k-nearest neighbour (k-NN) algorithms. Specifically, we implement the policy π as an MCTS algorithm (Section 3.2), relying on k-NN for estimating transition probabilities (Section 3.1).
Note that in recent years, deep reinforcement learning (DRL) algorithms have garnered significant attention from researchers for their applications in cybersecurity decision support (e.g., (Ganesan et al. 2016;Kurt et al. 2018)). While we could apply DRL algorithms to our problem in principle, they are ill-suited to our problem for multiple reasons. First, DRL algorithms tend to be sample inefficient (i.e., require a large number of training experiences to learn a performant policy), which poses a significant challenge since it is difficult to collect large datasets of prior incidents with sufficient details. Second, the limited size of the dataset enables us to make decisions based directly on the dataset, instead of having to train a parametric machine-learning model. By working directly with the dataset, decisions can take advantage of all the information in the dataset.
In a nutshell, Monte Carlo tree search finds which action to take in a given state by randomly sampling sequences of actions, simulating how they play out starting from the given state, and choosing the action that leads to the highest rewards on average. We can simulate the cyber-forensic investigation process from any given state using the statetransition probabilities (Equations ( 1) and ( 2)).
Empirical Probabilities However, since we do not know the underlying probability distribution in practice, we have to estimate the probabilities based on the set of prior incidents I. We can estimate the state-transition probability Pr[a | Y t , N t ] as the conditional empirical probability:
where
In other words, I ⟨Yt,Nt⟩ is the set of prior incidents which "match" the current state of the investigation, that is, the set of prior incidents in which threat actors did use all of the techniques from Y t but did not use any of the techniques from N t . Equation ( 5) estimates the probability Pr[a | Y t , N t ] as the ratio of incidents from I ⟨Yt,Nt⟩ in which threat actors used technique a.
k-nearest Neighbors The weakness of this estimation is that its accuracy depends on the cardinality of the set I ⟨Yt,Nt⟩ , and as the sets Y t and N t grow with each step of the cyber-forensic investigation, the set I ⟨Yt,Nt⟩ shrinks. In fact, due to the limited number of prior incidents, the number of prior incidents that "match" the current state of the investigation may reach zero, which precludes us from calculating the estimates at all. To address this issue, we extend the set I ⟨Yt,Nt⟩ to include prior incidents that do not exactly "match" the current state of the investigation but are sufficiently similar. We measure similarity between the current state ⟨Y t , N t ⟩ and a prior incident Î ∈ I using a Hamming distance over the techniques Y t ∪ N t that have already been investigated:
The first term of the right-hand side is the number of techniques that were used in incident I but not in prior incident Î, while the second term is the number of techniques that were used in prior incident Î but not in incident I. Equivalently, we could formulate the following metric: s(⟨Y t , N t ⟩, Î) = Y t ∩ ÎY + N t ∩ ÎN , which measures similarity by counting techniques where the current state and the prior incident are the same; in contrast, our formulation measures dissimilarity (specifically, Hamming distance) by counting techniques where the current state and the prior incident differ. These measures are practically equivalent since
Finally, we replace the set of "matching" prior incidents I ⟨Yt,Nt⟩ in Equation ( 5) with the set of k prior incidents that are closest with respect to metric d (breaking ties arbitrarily). Notice that this estimation of the probability Pr[a | Y t , N t ] is actually a k-nearest neighbor regression with prior incidents I as the dataset, d as the distance metric, and a binary value representing if technique a was used in an incident as the output feature. While the value of k could be a constant hyper-parameter, we found that our approach performs better if k varies throughout the investigation. Hence, we let k = β 1 + β 2 • t, where β 1 and β 2 are hyper-parameters, which can find experimentally.
Since Monte Carlo tree search randomly samples sequences of actions, its coverage of the state space becomes sparser and sparser as it looks further into the future. Therefore, when choosing between actions, it should assign more importance to the near future than the uncertain far future. We can express this consideration by reformulating the objective as maximizing the expected discounted sum of rewards:
where γ ∈ (0, 1) is a temporal discount factor: rewards obtained at step t are discounted by a factor γ t .
Note that we also replace the reward B at with the benefit to cost ratio B at /C at . The rationale behind this is to incentivize the tree search to consider cost C a regardless of how far the investigation is from reaching a budget limit G; otherwise, the tree search would focus only on immediate benefit B a . Note that we found the ratio B at /C at formulation to work best in practice (e.g., better than the more intuitive semi-MDP formulation with γ t τ =0 Ca τ temporal discount and reward B at ).
In each step of the investigation, we run a Monte Carlo tree search (Algorithm 2), starting from the current state ⟨Y t , N t ⟩, which outputs an action a t that is estimated to result in the maximum expected discounted sum of rewards. To estimate the expected rewards for each action, the algorithm performs a number of iterations. In each iteration, it first generates a sequence of actions and states, starting from the current state, which is a random sample of how the investigation might play out if certain actions are selected. Then, in the back-propagation phase of the iteration, it updates its estimates of the expected rewards based on the experience of the sampled sequence. While our algorithm follows the common principles of MCTS, it is tailored to our problem and takes advantage of the specific rules of our MDP.
Variables and Initialization Here, we describe the variables that are maintained by the algorithm throughout the iterations and how they are initialized; we will describe later how they are updated.
Variable n[Y, N, a] is the number of times that the algorithm has tried action a in state ⟨Y, N ⟩ (initialized to 0 at the beginning of the search). Note that this and all other initializations can be lazy, i.e., we can store these variables in a dictionary that is initially empty, and we can assign an initial value to a variable when we use it for the first time. For Algorithm 2 MCTS for Forensic Decision Support
ease of presentation, the pseudocode initializes all variables explicitly at the beginning.
Variable R[Y, N, a] is an estimate of the expected discounted sum of rewards if we started from state ⟨Y, N ⟩, took action a first, and then followed an optimal policy.
Finally, variable R[Y, N ] is an estimate of the expected discounted sum of rewards if we started from state ⟨Y, N ⟩ and then followed an optimal policy. While these variables could be initialized to 0, we can improve the performance of the search by initializing them with an estimate:
Starting from state ⟨Y, N ⟩, we can investigate techniques A \ (Y ∪ N ); however, we do not know the optimal order in which to investigate them. Our estimate calculates expected rewards assuming a random order: each technique a ∈ A \ (Y ∪ N ) is equally likely to be investigated first (discount factor 1), second (factor γ), third (factor γ 2 ), ..., and last (factor γ |A\(Y ∪N )|-1 ). Our estimate can be calculated very quickly, especially since the probability estimates Pr[a|Y t , N t ] need to be calculated anyway for the back-propagation phase. Note that for terminal states (i.e., when A \ (Y ∪ N ) = ∅), this formula correctly calculates the expected rewards to be 0, following the notational convention that summation over an empty set yields 0. , a] gives preference to actions that should be selected becauseaccording to our current estimates-they lead to high expected rewards; while the second term gives preference to actions that have been explored less (i.e., tried fewer times) than other actions in the given state. The exploration factor M is a constant hyper-parameter that balances exploration and exploitation, which we can find experimentally. Note that we add 1 to the divisor to avoid division by zero since n[Y, N, a] is initialized to 0; however, this is just for ease of exposition, the addition of 1 is negligible as we run a large number of iterations. After selecting an action a i , we update the number of times n[Y i , N i , a i ] that action a i has been tried, and we simulate the random transition to state ⟨Y i+1 , N i+1 ⟩. However, instead of basing the transition on the actual probability Pr[a i |Y i , N i ], we select one of the two possible transitions with the same probability (i.e., 0.5 probability that a i was used, and 0.5 probability that it was not). We use uniform probabilities for two reasons. First, we factor in the actual probability Pr[a i |Y i , N i ] during back-propagation; hence, we obtain unbiased estimates regardless of the selection probabilities. Second, uniform probabilities lead to a more balanced and thorough exploration of the transitions; we found that with the actual probabilities, transitions that have high impact but low probability are not explored enough.
We stop generating the sequence of states and actions when we reach either a terminal state (Y i ∪ N i = A) or the depth limit (i ≥ t + D, where D is a hyper-parameter).
Myopic Pruning To improve search performance, we prune the search tree during selection by exploring only those actions that are most attractive in terms of the expected immediate reward. Specifically, in state ⟨Y, N ⟩, Algorithm 1 explores only those F actions that have the highest
where F is a hyper-parameter. While myopically focusing on actions with the highest expected immediate reward may occasionally disregard optimal actions, it typically provides better results by letting the search thoroughly explore the most promising branches of the tree. Backpropagation During the back-propagation phase of each iteration, we loop over the sequence of states and actions backwards, and we update our estimates based on the experience of this sequence. For each action a i , we update the estimate R[Y i , N i , a i ] using the following formula:
This formula calculates a best estimate since in each state ⟨Y i , N i ⟩, we should take the optimal action a that maximizes the expected discounted sum of rewards (based on our best estimates R[Y i , N i , a]).
We evaluate our proposed approach numerically on public datasets of real-world cyber incidents. For each cyber incident, we simulate how our approach would have prioritized the investigation, and plot the benefit obtained through discovery as a function of the effort spent. We compare our approach to two baselines, DISCLOSE and a naïve policy.
Our implementation and datasets are publicly available. 1
Dataset We use the MITRE ATT&CK Enterprise repository (Barnum 2012), which is a public repository of adversarial tactics, techniques, & procedures, referencing realworld cyber incidents in which some of these techniques were used. Since its original publication, both the ATT&CK framework and its CTI dataset have been regularly updated. We use three versions of the repository in our evaluation: v6.3 ( 2019), which is the version that Nisioti et al. (2021a) used to evaluate DISCLOSE;v10.1 (2021);and v11.3 (2022), which is the latest version at the time of writing. Evaluating our approach on multiple versions demonstrates that it can be applied to newer versions without any changes (other than standard hyper-parameter optimization).
For a fair comparison, we use the same 31 techniques A and the same benefit B and cost C values as Nisioti et al. (2021a). Since the categorization of techniques changed slightly between versions, we had to map some techniques to equivalent ones for later versions. This leaves us with 29 techniques for versions v10.1 and v11.3. The benefit and cost values are based on the Common Vulnerability Scoring System and interviews with cyber-forensic experts.
For each technique, we collected cyber incidents in which the technique was used via the external references field (replicating Nisioti et al. (2021a)). Our v6.3, v10.1, and v11.3 datasets contain 331, 670, and 716 cyber incidents, respectively. Version v11.3 includes 425 incidents that are new compared to v6.3, and 73 incidents that are new compared to v10.1. In these datasets, every incident used at least 2 techniques. In dataset v11.3, incidents used 4.1 techniques on average (min. 2; max. 17). One example of an incident that used many techniques is the Frankenstein campaign (Biasini 2019), which employed Phishing, OS Credential Dumping, Exfiltration Over C2 Channel, and other techniques.
Baselines We compare our approach to two baselines, DISCLOSE (implemented as specified in Nisioti et al. (2021a)) and a naïve baseline, which we call the static policy. At every step, the static policy selects the technique (from the set of techniques that have not been investigated yet) that is most frequent across all prior incidents, instead of considering conditional probabilities. This naïve baseline represents investigation without decision support (i.e., decisions that ignore the state).
Simulation Setup and Metrics Since the datasets are relatively small, we use a leave-one-out cross-validation: when evaluating a policy on an incident, we treat all other incidents in our dataset as prior incidents I. We always simulate the investigation starting with a single randomly chosen technique from I Y as the singleton set Y 0 (and letting N 0 = ∅). Same as Nisioti et al. (2021a), we terminate an investigation when the cumulative effort cost (i.e., a∈Yt∪Nt C a ) reaches 45 or 65 to assess which techniques could have been promptly discovered by the approach. We also conducted experiments with two more budget limits (70 and 100) and without a budget limit to provide a more comprehensive comparison (we include results in Appendix A.1). Further, to quantify promptness, we measure the area under the benefit-effort curve (AUCBE): we plot the discovery of benefits obtained as a function of the cumulative effort cost for three different approaches (e.g., see Figure 1). Higher AUCBE values are better. Solid lines are averages over all incidents, dotted lines are 25% quantiles, and dashed lines are 75% quantiles.
Hyperparameter Optimization While the baseline does not have any hyper-parameters, our proposed approach has a number of them. First, we optimized the hyper-parameters for the k-NN probability estimation (β 1 , β 2 ) using a grid search, maximizing the average AUCBE over all incidents. During this search, we restricted the MCTS to one-action depth (D = 1, in which case the other parameters of the MCTS are irrelevant), providing us with good hyperparameters for probability estimation. Then, we optimized the hyper-parameters for MCTS using Hyperopt Python library, again maximizing AUCBE. Note that we optimized the hyper-parameters for datasets separately. We provide results from the hyper-parameter search in Appendix A.2.
Running Time For a single decision, the MCTS algorithm takes less than 7 seconds on average using a single core of a 2.4GHz Intel Core i9 CPU, and less than a second using multiple cores. Compared to the amount of time that forensic analysts need to investigate the selected technique (at the very least minutes), these running times are negligible. 1 and2 show cumulative benefits obtained during the cyber-forensic investigations (i.e., a∈Yt B a ) as functions of the cumulative effort costs (i.e., a∈Yt∪Nt C a ) using MCTS, DISCLOSE, and static approach on the v6.3 dataset. We conducted the experiments with budgets 45 and 65. We provide results for other versions of the dataset and different budgets in Appendix A.1. Each curve is based on all incidents in the dataset. For the v6.3 dataset, our approach outperforms the baselines in both of the scenarios (45, and 65). The average AUCBE of MCTS, DISCLOSE, and static policy for budget limit 45 are 3,503, 3,411, and 3,175 respectively. The average AUCBE of our approach, DISCLOSE, and static policy for budget limit 65 are 6,288, 6,108, and 5,826, respectively. For the v11.3 dataset, our approach outperforms the baselines in both of the scenarios as well. The average AUCBE of MCTS, DISCLOSE, and static policy for budget limit 45 are 4, 061, 3,982, and 3,865, and for budget limit 65 are 7,072, 6,975, and 6,816 respectively. This demonstrates that while our principled approach provides superior prioritization, the decision-support problem is very challenging due to the inherent uncertainty of cyber incidents.
Several prior research efforts focused on enhancing the efficacy of forensic investigations by decreasing the time or resources required for an investigation without impacting its quality and objectivity (Hossain et al. 2018(Hossain et al. , 2017;;Hossain, Sheikhi, and Sekar 2020;Hassan et al. 2020;Satvat, Gjomemo, and Venkatakrishnan 2021). Our approach is primarily motivated by DISCLOSE (Nisioti et al. 2021a), a data-driven decision-support framework, which creates TTP space using the MITRE ATT&CK dataset, computes conditional probabilistic relations and proximity values between TTPs, and proposes actions based on these relations. DIS-CLOSE can assist an expert in each step of the investigation by considering benefits, costs, and available budget. We extend this work by introducing a formal model of the cyberforensic investigation process, modeling it as a Markov decision process. Further, we propose a computational approach which, at each step of the investigation, considers all the techniques that have been investigated, instead of considering only the very last technique-as DISCLOSE does. Horsman et al. (2014) present CBR-FT, a case driven reasoning based technique for device triage. Similar to our approach, CBR-FT uses a knowledge base of past cases to calculate probable subsequent actions based on system file paths, which is then used to offer prediction of triage process of the current case under investigation. Unlike the system file paths, we use TTPs which allows for more flexible analysis and reasoning of adversarial behavior. Attack graphs were included in forensic investigation by Lui et al. (2012) to guide the decision process. The notion of anti-forensic steps, parallel to other adversarial actions, were included in the attack graphs to represent that the adversary may hide relevant forensic evidence to dissuade the defender. Likewise, Nassif and Hruschka (2012) propose the use of clustering techniques to support forensic investigation. Using a similar approach, Barrere et al. ( 2017) proposed an algorithm using condensed attack graphs that transformed the structure of the original attack graph allowing for more effective exploration. Algorithms are also proposed to support forensic investigation of online social networks (Arshad et al. 2020) and for unsupervised prediction for proactive forensic investigation of insider threats (Wei, Chow, and Yiu 2021). Similar to our work, these algorithms aim to increase the efficacy of forensic investigation by decreasing the investigation time; however, they approach the problem differently by decreasing the time required for crucial tasks. Quick and Choo (2014) highlight the effects of large amounts of digital-forensic data on modern forensic investigations. Saeed et al. (2020) and Nisioti et al. (2021b) present game-theoretic approaches used to model interactions between an investigator and an adversary using antiforensic method. By finding the Nash equilibrium of the game, the authors find the optimal trade-off strategy for each player. Even though our work does not explicitly model antiforensic techniques and their analysis, our approach is developed considering the potential existence of anti-forensic techniques for an incident under investigation.
Defenders face a similar prioritization problem with sensitive intrusion-detection systems, which may raise a large number of false alarms (Tong et al. 2020;Yan et al. 2019Yan et al. , 2018;;Laszka et al. 2017).
To address the limitations of DISCLOSE, we introduced a principled approach by modeling cyber-forensic investigation as Markov decision processes and proposing an MCTS and k-NN regression based computational approach. A key advantage of our proposed approach is that it works directly with the data (i.e., at every step and every iteration, probabilities are estimated based on the dataset of all prior incidents), instead of training a parametric machine-learning model. By relying on non-parametric machine-learning models, we aim to get as much "mileage" out of the data as possible, which is both enabled and necessitated by the limited amount of public data about cyber incidents. Another key advantage of our proposed approach is that the tree search approximates best estimates-and hence optimal decisions-based on the dataset (as the number of iterations increases).
While these advantages enabled our approach to outperform baselines, including DISCLOSE, we found the prioritization problem to be very challenging. The primary reason for this is the inherent difficulty of predicting how threat actors work. In fact, DISCLOSE itself is not too far from the naïve baseline policy.
A.1 Evaluation with Different Budgets
For each budget limit and each dataset we performed hyperparameter optimization (β 1 , and β 2 ) using Myopic approach. For each dataset and different budget limit, there is a heatmap (see Figures 16 to 30). Each cell of heatmap shows AUCBE of average percentage of benefit attained up to the budget limit. β 1 variable ranges from 1 to 130, and β 2 ranges from 0 to 6 with 0.1 intervals. For the sake of better visualization, heatmaps are cropped at y-axis (β 1 ) ranges 1 to 61, 1 to 101, and 1 to 121 for datasets v6.3, v10.1, and v11.3 respectively. For all heatmaps the y-axis shows β 1 and x-axis shows β 2 .