Overcoming the Antivirus Emulation Defense

by Dan Lowe, on 17 October 2013

Cybercriminals are bright and have learned how to overcome some virus defenses.

For awhile now, antivirus vendors have been using different techniques to help identify unknown or variations of unknown malware. By allowing malware to fully execute in a safe environment, antivirus vendors can review which files are malicious and develop signatures as well as tweak their antivirus scanning engines to reduce infections. However, criminals are very bright and have learned how to overcome some of these antivirus defenses.

Virtualization and emulation are similar antimalware detection techniques designed to look at the behavior of a file. These are two different concepts and sometimes are confusing as they are used interchangeably. It is interesting to understand how these techniques are used, where they are generally being used, and how malware authors are creating ways to circumvent these defenses. At a high-level, I will provide one way of getting around the antivirus behavior approach.

In a traditional sense, a virtual environment creates a layer between the native hardware and controlling access to that hardware. If you are using an x86 machine, then you are using the chipset of the native hardware to run. It generally runs faster as there is no translation layer needed like emulation[1]. Many automated network based or signature based antivirus companies build many virtual instances to test different files to determine whether it is infected or not.  

Emulation takes the properties of a system and reproduces it within a different type of system. For example: on a PowerPC, with emulation software, you can emulate the hardware and architecture of an x86 based system. This allows the antivirus vendor to see how an executable file behaves in a safe environment. It is optimal to use this technology within a single user environment as it allows a file to execute while analyzing the behavior of the file. If a file has the characteristics of being malicious, it will be flagged.

Many malware authors use multiple tools to encode and obfuscate their files to bypass these detection methods. The XOR cipher is one of the tools malware authors use to bypass antivirus detection. Once they bypass the antivirus behavior technique, the file needs to be decoded and decrypted for the malicious code to operate. There are many variations to this example as malware authors continue to innovate and improve their approach to circumventing antimalware defenses.

It is truly a challenge to identify unknown malware as criminals have many tools at their disposal to create multiple variations. Unknown to some malware writers, sometimes they leave clues to help antivirus companies become better at identifying threats. Sometimes it is an aggregate number of minute details that form patterns to help companies understand potential malware variations. Though much of malware identification is scientific, sometimes you just need some luck!    

Dan Lowe

Dan Lowe, an OEM Senior Marketing Manager, has been working with Bitdefender for the last 3 ½ years. His familiarity with multiple security products from Firewalls to Antivirus has provided him a unique perspective on the security industry.