On the detection of Kernel-level rootkits using hardware performance counters

Abstract

Recent work has investigated the use of hardware perfor- mance counters (HPCs) for the detection of malware run- ning on a system. These works gather traces of HPCs for a variety of applications (both malicious and non-malicious) and then apply machine learning to train a detector to dis- tinguish between benign applications and malware. In this work, we provide a more comprehensive analysis of the ap- plicability of using machine learning and HPCs for a specific subset of malware: kernel rootkits. We design five synthetic rootkits, each providing a single piece of rootkit functionality, and execute each while collect- ing HPC traces of its impact on a specific benchmark ap- plication. We then apply machine learning feature selection techniques in order to determine the most relevant HPCs for the detection of these rootkits. We identify 16 HPCs that are useful for the detection of hooking based roots, and also find that rootkits employing direct kernel object manipula- tion (DKOM) do not significantly impact HPCs. We then use these synthetic rootkit traces to train a detection system capable of detecting new rootkits it has not seen previously with an accuracy of over 99%. Our results indicate that HPCs have the potential to be an effective tool for rootkit detection, even against new rootkits not previously seen by the detector.