Understanding the behaviors of system intrusions is important for malware defense systems to discover their vulnerabilities and prevent them to be exploited for the future. However, existing approaches fail to explain the complete details of intrusion paths trying to balance between overheads from logging/analysis and performance.
We introduce a new security analysis system that enables tracking and understanding system intrusions fully and precisely, using deterministic virtual machine replay and virtual machine introspection. We adopt deterministic VM replay to:
(1) record the whole execution of a target system,
(2) take the replay log off-line, and
(3) perform analysis on a replayed session of the execution.
Further, our analysis engine effectively overcomes the semantic gap between an analysis algorithm and the low-level state of a guest VM implementing a powerful debug symbol library and core VM introspection component. Along with the replay and analysis engines, our new analysis model naturally breaks the complex behavior of an intrusion into a number of small problems and tries to answer to them one by one in a repeated, retrospective, and precise manner.
This research is a part of the Advanced Adaptive Applications (A3) project in the Flux Research Group.
Research
Iterative Backtracking via Deterministic Virtual Machine Replay and Virtual Machine Introspection -- MS project report
GPUs have extremely powerful computation ability under certain limitations, such as large overheads from transferring data to and from the GPU memory. We believe that DBT is a type of computation that GPUs can speed up. To improve the performance, we plan to aggressively translate the instruction blocks that will likely be executed in the near future and cache them for later use. The main challenges of this work are (1) how well one can predict the control flow of the process being translated and (2) balancing the tradeoffs between the amount of computation and the overheads from data transmission.
When a manufacturer designs a processor, it must publish the instruction set because software engineers need the specifications in order to write programs. However, the instruction set also serves as a "double-edged sword" since it enables hackers to modify benign code for malicious reasons. Our approach to solve this problem was complete encryption of executable binary code. With this concept we designed a new method that allows a system to run encrypted code safely and rapidly by using virtual-memory remapping.