A Layered Architecture for Detecting Malicious Behaviors

We address the semantic gap problem in behavioral monitoring by using hierarchical behavior graphs to infer high-level behaviors from myriad low-level events that could be parts of many different kinds of behavior. Our experimental system traces the execution of a process, performing data-flow analysis to identify meaningful actions such as “proxying”, “keystroke logging”, “data leaking”, and “downloading and executing a program” from complex combinations of rudimentary system calls. To preemptively address evasive malware behavior, our specifications are carefully crafted to detect alternate sequences of events that achieve the same high-level goal. We tested seven malicious bots and eleven benign programs and found that we were able to thoroughly identify high-level behaviors across this diverse code base. Moreover, we were able to distinguish malicious execution of high-level behaviors from benign by distinguishing remotely-initiated from locally-initiated actions.