BEHAVIOR-BASED CYBER SECURITY: USING TRADITIONAL AND BEHAVIORAL GAME THEORY TO IMPROVE ATTACK DETECTION

It is undeniable that computer network security is of paramount importance. As unauthorized users increase in sophistication, it is necessary to develop tools to not only respond to their new methods but to anticipate the attack methods before they are deployed. This dissertation will investigate the possibility of employing traditional and behavioral game theory to improve intrusion detection within a computer network as opposed to at a network’s firewall. It will first show how to incorporate knowledge of an attacker’s behavior to improve detection within a network’s perimeter. The second contribution will be to establish a new strategic foundation for attackers and defenders that includes an explicit model of the signal classification problem faced by the defender. This inclusion purifies the mixed-strategy equilibrium results that are ubiquitous in strategic analyses of computer network attack. The final contribution will be to synthesize the statistical method of improving attack detection and the strategic intuition to analyze how boundedly rational (in the form of level-k thinkers) defenders and attackers interact in a noisy network environment. The results show that specifying exactly when and at which hosts an attacker will attack leaves the defender vulnerable to attacks at other “unmonitored” locations. However, if the defender assumes an attacker will exploit all vulnerabilities, then he can optimally define a likelihood ratio attack detector that anticipates the attacker’s strategy.