POMDP + Information-decay: Incorporating defender's behaviour in autonomous penetration testing

Penetration testing (pen-testing) aims to assess vulnerabilities in a computer network by emulating possible attacks. Autonomous pen-testing allows frequent and regular pen-testing to be performed, which is increasingly necessary as networks become larger and more complex. Autonomous pen-testing is a planning under uncertainty problem, where the uncertainty is caused by partial observability of the network, lack of reliability of attack tools, and possible changes in the network that are triggered by the network administrator (the defender). Approaches that account for the first two causes of uncertainty have been developed based on the mathematically principled framework, Partially Observable Markov Decision Process (POMDP). However, they do not account for the third type of uncertainty. On the other hand, work that accounts for the defender's actions do not account for both partial observability and unreliability of the attack tools. This paper proposes a POMDP-based autonomous pen-testing framework that accounts for the defender's behaviour, thereby accounting for all of the above three causes of uncertainty. Key to our model is the observation that the defender's actions can be abstracted into two types: Network analysis, which does not alter the network, and active defence operations, which alter the network. This observation enables us to represent the defender's behaviour as a single variable: An information decay factor. This variable is based on the expected time the defender takes to move from analysing to actively defending the network, and therefore represents the decay of a pen-tester's knowledge about the network. We propose D-PenTesting, which assumes the decay factor is known prior to execution, and LD-PenTesting, which learns the decay factor as it attempts to break into the network. Simulation tests on two benchmark scenarios indicate that D-PenTesting and LD-PenTesting outperform existing POMDP-based pen-tester and is more robust than one that incorporates a POMDP-based defender.