Distributed Service Control Technique for Detecting ... - IEEE Xplore

6 downloads 0 Views 176KB Size Report
Faculty of Science, Macquarie University, Sydney, Australia. Abstract — We propose Distributed Service Control (DSC) technique for securing critical services.
Distributed Service Control Technique for Detecting Security Attacks Udaya Tupakula

Vijay Varadharajan

Information and Networked Systems Security Research Faculty of Science, Macquarie University, Sydney, Australia

Abstract — We propose Distributed Service Control (DSC) technique for securing critical services. One of the main aims of DSC is to deal with the attacks by minimising the attack surface between two hosts. In our model, light weight security policies are enforced at the client machines to ensure that the client can access the services using legitimate traffic only. This will minimise the number of attacks that can be generated by the malicious client machine on the server. We will show that our model can increase the availability of the critical services. Our model can also be used as an early detection technique for the outbreak of worms. Keywords – Virtual machine monitor, Distributed Service Control, Local Area Networks.

I.

INTRODUCTION

Today, Local Area Networks (LAN) are part of every large, medium, and small organisations. One of the fundamental requirements of the LAN is to enable sharing of information between different users or computers and hence the traditional LANs were designed to enable free communication between the hosts. Ethernet is one of the widely used LAN technologies. The current Internet environment is vulnerable to a range of different types of attacks [1-3] such as malware, phishing, spam, and denial of service. The size and complexity of the current operating systems and applications is continuously increasing and it is not an easy task to ensure the secure operation of such systems. Although there are several tools such as intrusion detection systems [4], honey pots, antivirus and anti malware, the dynamic nature of the attacks makes it difficult to detect and prevent attacks. On the other hand, it is simple task for the attacker to compromise such systems and generate different types of attacks. As a result we are witnessing an increasing number of zero day attacks on a daily basis. Zero day attacks are the attacks which are previously not known. In particular LAN environments are highly vulnerable to such attacks since there is free communication between the hosts in LAN environment. Today Virtualisation technology is widely being deployed. A Virtual Machine Monitor (VMM) [5] is an additional software layer which has complete control on the physical resources and enables to run multiple operating systems on a scalable computer. The current virtual monitors can be The authors would like to thank Departments of the Prime Minister and Cabinet (PM&C) and Defence Signals Directorate (DSD), Australia, for their financial support of the research project on Secure Virtualization Systems. The PM&C and DSD funding should not be taken to imply endorsement of the content or conclusions of the research project.

classified into two types. In Type 1 VMM, the hypervisor runs as an application on the host operating system. In type 2 VMM, the hypervisor directly runs on top of the hardware. Our model supports both types of VMM. Currently there is considerable research interest to develop security tools that are based on virtualisation technology. Although some of the VMM based security techniques [6-8] have been proposed, none of the techniques consider the specific features of LAN to detect and prevent the attacks. Hence this is the main focus of our work. In this paper we first consider the challenges to deal with the attacks in the traditional LAN environments using traditional security tools. Then we propose techniques to deal with the attacks by taking into account the specific characteristics of the LAN. Since there is free communication of hosts in the LAN environment, the attack surface between two hosts is considerably high. For example, a compromised host can send any type of attack traffic to other hosts. One of the main aims of our work is to deal with the attacks by minimising the attack surface between the hosts. Our model makes use of the virtualisation technology to deal with the attacks. Our model can prevent the attacks at the virtual machine monitor that is hosting the compromised virtual machine. The paper is organized as follows. In Section II we first present the challenges for dealing with the attacks in traditional LANs and then propose Distributed Service Control (DSC) model to deal with the attacks. Section III presents some of the related works that are relevant to this paper. Section IV concludes. II.

DISTRIBUTED SERVICE CONTROL

In this section we will first consider an attack scenario in the traditional LAN and discuss the challenges to deal with the attacks. Then we propose Distributed Service Control model to minimise the attack surface between the hosts and deal with the attacks. Our model makes use of virtualisation technology to enhance the security of the existing LAN’s. A. Attack Scenario As shown in Figure 1, let us consider a simple scenario with several client and server machines installed on a separate physical machine and are connected in Local Area Network. The hosts share the resources of the available LAN medium and often there is free communication between the hosts in the LAN. Also as shown in the Figure, the client and server machines can have some host based security tools (HBST) such as antivirus or host based intrusion detection that are

978-1-4673-0269-2/12/$31.00 ©2012 IEEE

574

Client 1

Service 1

HBST

HBST

Client N

Service N

HBST

Security Gateway

HBST

Figure 1. Traditional LAN

installed on every machine to detect attacks on the hosts. In additional, network based security tools can be installed on the security gateway to monitor the network traffic for suspicious behaviour and detect attacks. Although there are several tools such as intrusion detection systems, honey pots, antivirus and anti malware, the dynamic nature of the attacks makes it difficult to detect and prevent attacks. The host based tools have good visibility of internal state of the monitored system and can detect the attacks more efficiently. However since the tools are implemented on the monitored system itself, they are vulnerable to attacks by the attacker. The network based tools detect the attacks by monitoring the incoming and outgoing traffic from the monitored machines. They have less visibility into the state of monitored machines but offer high attack resistance. For efficient detection of attacks, it is desirable for the tools to have good visibility of the monitored system while at the same time offering high resistance to attacks. In the current LAN environment, there is no technique to determine if any of the sources is dominating the usage of resources. As shown in Figure 1, a single compromised client can flood the LAN medium with different types of malicious messages and severely degrade the services in the LAN environment. Often, since there is free communication between the hosts in the LAN the attack surface between the hosts is considerably large. The compromised hosts can generate any type of attack traffic on other hosts in the LAN. Even if the destination hosts have some host based security tools they can only secure the destination node from the attack traffic but cannot guarantee the availability of the destination host services. This is because all the available resources of the destination machine and/or the LAN medium can be consumed by the compromised host. One of the common behaviour with any of the attacks such as slammer, conficker, torpig, storm is that they result in congestion of the LAN medium. For example, Slammer worm floods the network with malicious UDP packet, conficker and torpig perform brute force attack on the neighbouring hosts to obtain administrative access to other machines and storm worm will result in bulk email messages. Also, it has to be noted that the hosts in the LAN are the primary target for the attacker to spread the malware since the hit rate is considerably higher with a rare chance of detection. Hit rate is used as a measure of finding vulnerable hosts for the spread of malware. In case of LAN, there is more possibility for the hosts to have similar applications and operating systems since they belong to single administrative domain. The detection chances are rare since free communication is permitted between the hosts in the LAN.

In case of zero day attack, often it is manual task for the administrator to determine the malicious host that is generating the attack traffic. The detection process is complicated since the compromised host can be flooding the LAN medium with spoofed source address. Although there are some security tools on each client and server machines there are several challenges for the traditional security tools to deal with the emerging attacks. The emerging attacks such as conficker and torpig have the capabilities to disable the security tools and features such as auto updates that are running in the client and server machines. Although the network based security tools are isolated from the compromised host, they cannot efficiently detect the attacks. For example, the capability of such tools is limited to detect the attacks by monitoring the traffic in the LAN medium. Since the LAN medium is shared among all the hosts, it becomes extremely difficult to determine the attacking source. Hence it depends on the ability of the Network or Security administrator to determine the compromised host and isolate it from degrading the services in the LAN. So, there is need for techniques that can be used to detect and prevent such attacks in the LAN environment. If the attacks are prevented only at the destination end, this can impact the availability of the services in the LAN environment. Hence there is need for techniques to deal with the attacks at the source end. B. Architecture Overview Figure 2, shows the proposed architecture for enhancing the security in the LAN. As shown in the Figure 2, the client and server machines which are implemented as standalone machines in traditional LAN are running as virtual machine on top of the virtual machine monitor. Our architecture also supports hosting of multiple virtual machines on each VMM and the operation of our model is not dependent on the number of virtual machines being hosted on the virtual machine monitor. However for simplicity, we will only consider that a single virtual machine is hosted on each physical server. Server VM21

Client VM11

HBST

HBST

VMM1

VMM2

DSC

DSC

CAC

SAC

Hardware

Hardware

Security Gateway Figure 2. DSC Virtualised LAN

The DSC will be used to monitor all the interactions of virtual machine and deal with the zero day attacks in case of compromise of the system. Figure 3 shows the DSC architecture which can be integrated into the VMM or the host machine. Note that the DSC architecture is similar at the client

2012 IEEE Network Operations and Management Symposium (NOMS): Short Papers

575

and server virtual machines. However lightweight security policies are enforced at the client end and comprehensive security policies are enforced at the server end. Since the VMM can have complete control of the resources, they have good visibility of the internal state of the virtual machines. Also the security tools placed in VMM are isolated from the virtual machines. Hence the DSC module placed in VMM has the capabilities of performing both host based and network based intrusion detection. However one of the challenges with the VMM based security tools is that it is not always possible to determine internal state of the virtual machines. This is known as the semantic gap problem. As shown in the Figure 2, the client and server virtual machines are also equipped with the host based security tools (HBST) such as host based intrusion detection system or antivirus or firewall. Although, the DSC is capable of performing host based and network based security monitoring one of the design choices is to use DSC as an additional layer of defence instead of considering it as a replacement to the host based security tools. Furthermore, it can also deal with the semantic gap problem which is relevant to the VMM based security tools. Also note that the operation of our model does not depend on the operation of the HBST in the virtual machine. The security policies stored in HBST and the security policies in the DSC module can be different.

the security policies that need be enforced by the DSC/CAC. At a minimum it includes the services running on each IP address and the legitimate protocols at different layers of the TCP/IP protocol stack. C. Operation DSC monitors all the traffic from the client or server virtual machines. Figure 3 shows the DSC architecture for securing the services in LAN environment and also for detecting the attacks at an early stage. The traffic generated by the client virtual machines is received by the Service Extraction Unit (SEU). The SEU enforces the security policies for the secure operation of the LAN. For example, this module can prevent if the virtual machine is trying to dominate the usage of resources within the LAN. This component also validates the traffic that is originating from the virtual machine. Only the traffic that is considered as legitimate is placed on the LAN medium. If the traffic is found to be suspicious, then the traffic is dropped and in some cases the virtual machine can be isolated from other virtual machines in the LAN. The virtual machines can be connected only after validation by the security administrator.

SCE

Table 1: Sample DSC policies S.No 1

2

IP Address

32-bit IP Address

32-bit IP Address

Services Mail

DNS

Legitimate Traffic Pattern SMTP, TCP, max/min packet length, packet arrival rate, TCP/IP header values, port numbers, attack signatures, etc. DNS, UDP, max/min packet length, packet arrival rate, UDP/IP header values, port numbers, attack signatures, etc.

The Distributed Service Control model has two components. The Server Access Control (SAC) component has global security policies for securing each VM service hosted on the VMM. For example, it can contain the statistical data for each service and the attack patterns for each service. The sample format of the statistical data for Mail and DNS services is shown in Table 1. The IP address is the 32-bit IP address of each server and the services specify different services running on the virtual machine. If a virtual machine is running more than one critical application it becomes easy for the attacker to exploit the vulnerability in any of the critical application and generate attacks on all the applications. Hence we consider that each virtual machine is running only a single critical application. The legitimate traffic pattern specifies the different parameters of the traffic at different layers that are considered as legitimate for each server. For example, this can have the information about the maximum or minimum packet length, packet arrival rate, TCP/IP header values, and application layer protocols. The Client Access Control (CAC) is a lightweight access control policy for the client virtual machines to access the server virtual machine. The DSC/SAC at the server specifies

576

Legitimate? Yes

No Drop Traffic

OUT

SEU In

Figure 3. DSC Architecture

Let us consider some of the sample policies that can be used to validate the traffic generated by the virtual machine. The SEU ensures that any traffic originating from VM has correct MAC address and IP address. If the source IP address or MAC address of the VM traffic is spoofed then the SEU isolates the virtual machine from the LAN. Note that since the source address of all the packets is validated by the SEU component, it is not possible for the virtual machines to generate attack traffic with spoofed source address. However at this stage, it is still possible to generate attack traffic with correct source address. Now let us consider how SEU prevents attacks from the client virtual machine to the server virtual machines. The SEU also determines the services accessed by the client traffic and forwards the traffic to the Service Control Enforcement component. The server identity can be determined from the destination address of the client traffic and the services accessed by the client can be determined from the payload information available at the different layers of the TCP/IP protocol stack.

2012 IEEE Network Operations and Management Symposium (NOMS): Short Papers

Service Control Enforcement (SCE) validates the services accessed by the client traffic. If the services accessed by the client machine are actually running at the destination server then the traffic is considered as legitimate. In this case the client traffic is placed on the LAN medium for forwarding to the server. If the services accessed by the client traffic are not running at the server end then the traffic is considered as malicious. In this case the traffic is dropped and further analysis can be performed to determine the application that is generating the malicious traffic. There are several advantages with the proposed technique. First the attacks surface is minimised between the client and server machine since the clients can only access the server machines using legitimate traffic. Even if the client machine is compromised, the attacks from the client machine to the server machine will be successful only if the traffic is accessing legitimate services on the server virtual machines. Also, this can be used as an early sign of detection of malware. Hence we can see that the operation of our model is simple. Also, since the current VMM’s are capable of running the virtual machines at native speeds, our model has minimal overhead since it only requires validation of the traffic before placing the traffic on the LAN medium. Furthermore, our model can simplify the tasks of network and security administrators when some attacks are detected in the LAN. D. Attack Scenario Let us consider an example scenario to deal with the attacks from a compromised client virtual machine on the DNS server virtual machine.

as malicious since DNS protocol is the legitimate application layer protocol for DNS server. III.

Although, several VMM based security techniques have been proposed, none of the techniques take into consideration the specific features of the LAN to deal with the attacks. However, one of the common behaviour with any of the emerging attacks [1-3] is that abnormalities are detected with the LAN. Also, our model mainly deals with the attacks by minimising the attack surface between the client and server virtual machines. IV.

CONCLUSION

Today we are witnessing an increasing amount of zero day attacks. The traditional LANs are highly vulnerable to such attacks since there is free communication between the hosts in the LAN. Hence a single compromised host can severely degrade the services in the LAN. In this paper we have proposed Distributed Service Control model to deal with the attacks in LAN. The DSC model deals with the attack my minimising the attack surface between the hosts.

[1] Figure 4. Attack traffic on DNS Server [2]

In Figure 2, consider the case of attack traffic generated from client virtual machine (VM11) to a DNS server machine (VM21). As shown in Figure 4, malicious TCP SYN traffic is generated from a compromised client virtual machine to the DNS server. From Table 1, since only UDP traffic is considered as legitimate traffic for the DNS server, the traffic generated by the attacking VM is considered as malicious by the DAC/CAC enforcement module and the traffic is dropped. Hence the attack traffic is dropped at the VMM that is hosting the attacking VM. This is very efficient compared to the prevention of the attack at the destination. Also, if such traffic is allowed to be forwarded in LAN, this can cause congestion within the LAN. It has to be noted that in some cases, DNS servers can also configured to operate on TCP protocol. In such cases TCP traffic to the DNS server has to be considered as legitimate. Even if we assume that the DNS support TCP protocol, the traffic in this case is destined to port 80 which is not standard port for DNS services. Also the application layer protocol in this attack case is HTTP which can be considered

RELATED WORK

Recently there is considerable research interest for developing security tools that are based on Virtualization. Dunlap et al [6] proposed ReVirt for securing the logs by placing the logging tool inside the VMM. The attacker does not have access to the logs even in case of compromise of the virtual machines. Hence, the administrator can use this information for analysing the attacks on virtual machines. Garfinkel [7] proposed a Livewire intrusion detection system which makes use of the components placed in the VMM to determine the state of the virtual machine and detect attacks. Antfarm [8] can be used for the detection of hidden processes in the virtual machine.

[3]

[4] [5] [6]

[7] [8]

REFERENCES Moore, D., Paxson, V., Savage, S., Shannon, C., Staniford, S., and Weaver, N. “Inside the Slammer worm”, IEEE Security and Privacy 1, 4, Jul. 2003. Brett Stone-Gross, Marco Cova, Bob Gilbert, Richard Kemmerer, Christopher Kruegel, Giovanni Vigna, “ Analysis of a Botnet Takeover”, IEEE Security & Privacy, Sept. 2010. Seungwon Shin, Guofei Gu, "Conficker and Beyond: A Large-Scale Empirical Study", Proceedings of ACSAC 2010, Texas, USA, Dec 2010. The Open Source Network Intrusion Detection System: Snort. http://www.snort.org J.E. Smith, Ravi Nair, “The Architecture of Virtual Machines”, IEEE Internet Computing, May 2005. George W. Dunlap, Samuel T. King, Sukru Cinar, Murtaza A. Basrai, Peter M. Chen, “ReVirt: Enabling Intrusion Analysis through Virtual-Machine Logging and Replay”, Proceedings of OSDI, 2002. T. Garfinkel and M. Rosenblum, “A virtual machine introspection based architecture for intrusion detection”, Proceedings of NDSS, February 2003. Stephen T. Jones, Andrea C. Arpaci-Dusseau, Remzi H. Arpaci-Dusseau,“VMM-based hidden process detection and identification using Lycosid”, Proc. of ACM VEE, March 2008.

2012 IEEE Network Operations and Management Symposium (NOMS): Short Papers

577