Towards Intelligent Cross Protocol Intrusion Detection in the Next Generation Networks based on Protocol Anomaly Detection Bazara I. A. Barry and H. Anthony Chan Department of Electrical Engineering, University of Cape Town Emails:
[email protected];
[email protected]
Abstract—The open nature of the Next Generation Networks (NGNs) and the involvement of multiple protocols in a single session, along with the attacks that spread multiple protocols, pose new challenges to Intrusion Detection Systems (IDSs). Detecting attacks based on information taken from a single protocol or a group of protocols at a certain layer results in a high rate of false positives or false negatives. In this paper, we introduce a new Cross Protocol design for IDSs in the NGNs based on Protocol Anomaly Detection. Our design aims at correlating various detection results from the protocols involved in a session, both, horizontally and vertically. By horizontal correlation we aim at monitoring sessions taking place within a single layer of the protocol stack, whereas, vertical correlation addresses sessions taking place across multiple protocol layers. In addition, our design is supported by intelligent mechanism based on fuzzy logic to help the system reduce the rate of false alarms which is relatively high in many anomaly based intrusion detection systems. This paper presents the basic features of our design, emphasizing the components and the interactions between them. Index Terms—Cross protocol, intrusion detection, protocol anomaly detection, fuzzy logic.
I
I. INTRODUCTION
n recent years, Intrusion Detection Systems (IDSs) have become an essential component of computer security. So many organizations have been implementing them as part of the defense in depth policy. The technologies and methodologies of Intrusion Detection Systems (IDSs) have been changing at the speed of light to follow the rapidly changing nature of attacks and threats. On the other hand, there is a remarkable growth in the demand for Next Generation Networks services and infrastructure. The Next Generation Networks (NGNs) infrastructure is open towards, or shared by, various types of service, such as voice, multimedia, and data, including the global Internet, and the services provisioned by multiple stakeholders. Signaling and control flows are carried in-band jointly with end-user data traffic [1]. All these facts put a Manuscript received 23 December, 2006. Work supported in part by Broadband Network Center of Excellence at UCT with funding support by Telkom, Siemens, and NRF.
heavy burden on the wide variety of security solutions in general and specially on IDSs not to be outpaced by malicious attacks. Applications in the NGNs employ multiple protocols for a single session or connection. A good example can be Voice over IP (VoIP) systems which are gaining popularity as the technology for transferring voice traffic over IP networks. They pose new challenges to IDS designers, because they employ multiple protocols for call management and data delivery. For instance, a typical VoIP session involves the Session Initiation Protocol (SIP) for call management, and the Real Time Transport Protocol (RTP) for data delivery, along with the rest of protocols at lower layers. Considering the vulnerabilities and the attacks targeting each protocol, and the poor efficiency of IDSs which use data from a single layer or protocol, a functionality that performs detection which spans multiple protocols is urgently needed. Useful detection information should be exchanged across protocols to enable the IDS to make a more accurate decision about the intrusion in the system. We introduce a novel Intrusion Detection architecture based on a cross-protocol design in the Next Generation Networks environment. Our architecture aims at monitoring protocols traffic, and reporting any violations of the protocol standards and semantics. We believe that protocol standards form a good basis for detection because they are well defined in the Request for Comment (RFC) documents and other vendor documents. In addition, our proposed architecture correlates different detection results from different protocols, both, horizontally and vertically. By horizontal correlation we aim at monitoring sessions taking place within a single layer of the protocol stack, whereas, vertical correlation addresses sessions taking place across multiple protocol layers. We contend that cross-protocol approach will enable us to analyze the attack scenario in its entirety and as a result, we can achieve better efficiency in terms of both higher true positives and lower false positives rates. Since benign traffic inevitably exhibits some anomalies and peculiarities, and since the difference between the normal and abnormal activities is not distinct in practice, but rather fuzzy, our design takes advantage of fuzzy logic techniques as an intelligent mechanism to reduce the false positives rate in
determining intrusive activities. The rest of the paper is organized as follows. Section II reviews intrusion detection technologies including Protocol Anomaly detection, and discusses how to model protocols behavior. Section III gives an overview regarding fuzzy logic and its use in intrusion detection. Section IV analyzes the problem space and gives the necessary motivation. Section V discusses the proposed design, describing the components and their functionality. Section VI sheds some light on the related work. Section VII concludes the paper. II. INTRUSION DETECTION SYSTEMS OVERVIEW A. Intrusion detection technologies Intrusion detection technologies have evolved from signature recognition based technologies which attempt to identify events that misuse the system. In other words, such systems operate by modeling attacks and intrusions as attack signatures, and comparing incoming traffic against these signatures. The simple form of detection used in such systems is the simple pattern matching which compares incoming packets against binary signatures of attacks. Obviously, signature recognition IDSs are good at detecting known attacks, but new and unknown attacks are far beyond their reach. On the other hand, anomaly detection IDSs are all about what is allowed rather than what is not allowed. They model the normal use of the system and consider any deviation from it as being malicious. Clearly, anomaly detection based IDSs have the capacity to detect unknown attacks, but it is always hard to define normality. A study on internet traffic in [2] concluded that statistics such as packet arrival times, connection arrival times, and website hits have much variation. The very fact about the difficulty of defining a reliable base line upon which we perform our analysis, resulted in a new variation of anomaly detection which is protocol anomaly detection. Protocol anomaly detection models protocols using their specification and real-world implementations of the protocols. As well known, the implementation and the proper use of a protocol is dictated by specifications, known as RFC (Request For Comment) documents and some vendor’s documents. Although RFCs are not always complete, they serve as a good starting point to build an acceptable model. Many of the new discovered attacks violate protocol standards, and it is always simpler to model the normal use of the protocol rather than to model its misuse. In a narrow sense, Protocol Anomaly Detection means looking for behavior in network traffic that is peculiar in terms of the specification for the protocol the traffic is using. In a broader sense, the term could mean applying anomaly detection on the semantics of traffic as expressed using the protocol. In this approach, traffic isn't peculiar due to a particular protocol element it's using, but rather what in aggregate it's trying to achieve with the protocol. A strongly related concept to protocol analysis is stateful protocol analysis, which involves performing protocol analysis
for an entire connection or session, capturing and storing certain pieces of relevant data seen in the session, and using that data to identify attacks that involve multiple requests and responses. Protocol anomaly by itself is a very powerful technique, but it is limited to examining a single request or response. Of course, many attacks cannot be detected by looking at one request - the attack may involve a series of requests. When we add stateful characteristics to protocol anomaly, we monitor and analyze all the events within a connection or session. Without the ability to keep state, we can only examine each packet, request or response on its own, completely independent of the rest of the session. B. Modeling protocols behavior Finite state machines (FSM) are commonly used for describing protocols. The graphical representation of an FSM is a visual aid to the designer of the protocol. This representation helps in testing and verifying the correctness of the design. An FSM consists of a set of states (represented by circles) and a set of transitions between these states (represented by directed arrows connecting the states). At each state, a set of events may occur. Each event triggers a specific transition out of a state. Before a transition occurs, a set of actions (possibly an empty set) is taken. One of the states in an FSM is designated the initial state [11]. Figure 1 illustrates a typical FSM.
Initial state
Event 1
State 2
Event 1
State 3
Fig. 1. Typical FSM.
Network protocols are also modeled using extended finite state machines (EFSM). An EFSM is similar to a finite-state machine, with the following differences: (a) an EFSM makes transitions on events that may have arguments, and (b) it can use a finite set of state variables in which values can be stored [12]. III. FUZZY LOGIC FOR INTRUSION DETECTION A good reason fuzzy logic is introduced for intrusion detection is that security itself includes fuzziness [3]. Classical approaches in intrusion detection define a range value or an interval to denote a normal value, then any values falling outside the range are considered anomalies regardless of their
distance to the interval. Unfortunately, this causes an abrupt separation between normality and anomaly. Fuzzy logic helps smooth this abrupt separation, and produce more general rules which will increase the flexibility of the IDSs. Fuzzy sets are an extension of the classical set theory used in fuzzy logic. A fuzzy set is characterized by a membership function, which maps the set of values that the object can take into the unit interval [0, 1]. Table I from [4] shows the difference between classic sets and fuzzy sets. The simplest membership function is the triangular membership function which is depicted in figure 2. TABLE I COMPARISON BETWEEN FUZZY SETS AND CLASSIC SETS Fuzzy Sets
Classic Sets
In fuzzy sets an object can partially be in a set.
In classic sets an object is entirely in a set or is not.
The membership degree takes values between 0 and 1.
The membership degree takes only two values 0 or 1.
1 means entirely in the set, 0 means entirely not in the set, other values. mean partially in the set.
1 means entirely in the set, 0 means entirely not in the set. Other values are not allowed.
The degree of membership of the object to a fuzzy set is called the truth-value (TV) of the fuzzy expression. TVs are calculated using a special formula and measured by numbers between 0 and 1. Wherefore, the fuzzy expression evaluation process is reduced to arithmetic operations. A fuzzy rule follows the form: IF condition THEN consequent [weight] Where, • condition is a complex fuzzy expression that uses fuzzy logic operators and atomic expressions. • consequent is a fuzzy atomic expression. • weight is a real number that defines the confidence of the rule. The following is an example of a fuzzy rule: IF the number of packets with invalid fields is more than n in the last s seconds THEN activity is abnormal [0.7]. Simple fuzzy rules allow us to construct if-then rules to reflect common ways of describing security attacks. In the intrusion detection problem we usually deal with two classes (fuzzy sets). These classes are called normal and abnormal. Two rules are defined accordingly (RN, RA) with RN goes for the normal class and RA for the abnormal class. One of the most used techniques to determine the class that an object belongs to, is to calculate the truth-value (TV) of each rule and to take the rule that produces the maximum TV. IV. PROBLEM SPACE ANALYSIS
abnormal 1.0
0.4 x
Set of possible object’s values
Fig. 2. Triangular membership function for a fuzzy set.
In figure 2, the object x has 0.4 degree of membership to the fuzzy set abnormal, i.e., x does not entirely belong to the set abnormal, but x belongs to the fuzzy set and does not belong to the set at the same time. Fuzzy logic allows an object to belong to different classes at the same time by considering more than one fuzzy set which form so called fuzzy space. This feature is useful to deal with benign traffic that exhibits anomalies. A fuzzy atomic expression has the form: object is [not] fuzzyset For instance: Activity1 is [not] normal
Protocol Anomaly based IDSs are supposed to understand how various protocols such as (HTTP, SMTP, TCP,…) work, and detect whether the traffic does not follow the expected behavior for the specific protocol. Obviously, different protocols run on different layers of the protocol stack, and each protocol (layer) has its own vulnerabilities, which necessitate an intrusion detection module at each layer. Traditionally, IDSs have been using data only from single layers to decide whether an attack is taking place or not. This results usually in a high rate of false positives, or false negatives produced by the IDS engine. For example, an attack that tries to create an unauthorized access to a service at the application layer may seem perfectly legitimate to the lower layers. What is really needed is a mechanism to coordinate intrusion detection among different protocols, and to aid detection decisions on one layer by using information from different layers. What necessitates such a mechanism, is the way applications work on the protocol stack. For example, a typical file transfer protocol on the application layer may use TCP protocol to establish the connection and to exchange control information with a server, and UDP protocol to transfer files [9]. Another good example can be Voice over IP (VoIP) systems which are gaining popularity as the technology for transferring voice traffic over IP networks. They pose new challenges to IDS designers, because they employ multiple
protocols for call management and data delivery. For instance, a typical VoIP session involves the Session Initiation Protocol (SIP) for call management, and the Real Time Transport Protocol (RTP) for data delivery, along with the rest of protocols at lower layers. Until an authentication and encryption solution is widely adopted, VoIP protocols will continue to pose a major source of vulnerabilities. Both, signaling protocols such as SIP and media delivery protocols such as RTP, transmit packet headers in clear text which enables attackers to manipulate packet fields to launch an array of different attacks. In his paper about Bro system [10], Vern Paxon mentioned what he called “The Problem of Crud”. Based on monitoring a large volume of network traffic, he realized that legitimate traffic exhibits abnormal behavior. He stated that the diversity of legitimate network traffic, including the implementation errors sometimes reflected within it, leads to a very real problem for intrusion detection, namely discerning in some circumstances between a true attack versus an innocuous implementation error. He concluded by mentioning the difficulty of relying on “clearly” broken protocol behavior as definitely indicating an attack because it very well may simply reflect the operation of an incorrect implementation of that protocol. Clearly, an intelligent functionality that performs detection which spans multiple protocols is urgently needed. V. SYSTEM DESIGN We propose a novel cross-protocol architecture for our protocol anomaly IDS. The proposed architecture exploits information across the different protocols involved to efficiently identify an intrusion. Two types of correlation are required based on the nature of the session taking place. That is, horizontal cross-protocol correlation and vertical cross-protocol correlation. Horizontal cross-protocol correlation deals with the protocols working on the same protocol stack layer, e.g., SIP and SDP, whereas, vertical correlation deals with protocols across stack layers, e.g., TFTP and TCP. Our proposed Design is composed of the following: 1. A protocol anomaly based detection module for each and every protocol in the protocol stack to monitor the status of the protocol traffic. 2. A horizontal cross-protocol module at each layer of the protocol stack layers to monitor the session taking place between multiple protocols on a single layer. 3. A vertical cross-protocol module to monitor the session spanning across multiple protocols on multiple layers. 4. A decision making module which gets information regarding potential attacks from the different modules above, and decides whether an attack is taking place or not. Figure 3 presents an overview of the proposed design.
HTTP Module Decision Making System
SIP Module
SDP Module
TCP Module
Decision
IP Module
Incoming traffic Signature Database
Fig. 3. Overview of the architecture components.
The incoming packets are fragmented into protocol headers, and each header is examined by the corresponding protocolanomaly based module (represented by ovals). If a session is detected, the appropriate cross-protocol module is triggered either vertically (as represented by the session involving SIP and SDP) or horizontally (as represented by the session involving (HTTP, TCP, and IP)). All modules, protocol based and session based, pass their analysis results to the decision making module. For each protocol, the analysis is based not only on protocol standards, particularly the RFCs, but also on how things are implemented in the “real world”. Many implementations violate protocol standards, so it is very important that analysis reflects how things are really done, not how they are ideally done, or many false positives and negatives will occur. Obviously, this is achieved through well designed fuzzy logic rules in the modules and the decision making system. As can be seen from the figure, protocol anomaly based modules can exchange information with regard to detection. Due to the richer semantic information available in the upper layers some attacks may be detected much earlier there than in lower layers, so upper layers can aid lower ones to make the right decision. Our proposed architecture has the capacity to perform both types of protocol analysis, namely semantic and syntax analysis. Semantic analysis is carried on by building a state machine for each protocol. A protocol state machine is responsible for keeping the current state of the connection and guarding against any unexpected behavior during the session. On the other hand, syntax analysis is performed by a packet checker associated with every protocol. A packet checker’s main responsibility is to cover other aspects apart from protocol semantics such as verifying packet size and header fields. Both protocol state machines and packet checkers work in a co-operative manner between the different protocols, embodying the cross protocol nature of the system. So, the most important parts of the research are: • Finding out the existence of a session and correlating the
packets belong to the session either vertically or horizontally, by means of protocol anomaly. • Providing a mechanism that enables detection modules to exchange useful information with regard to detection, so as to aid decision making on the protocol detection module level. For example, before a server establishes a connection with a client, and upon receiving the ACK packet from the client, the server’s TCP layer should look back at the IP layer to validate the connection. Specifically, it should check the IP source address of the incoming packet to make sure it has not changed from what it was in the first SYN packet. If a change is realized, it is an indication of a TCP hijacking attack. • Creating the appropriate fuzzy rules in the protocol based modules and the decision making system . In order to overcome the difficulties introduced by encryption, the proposed system should be host-based. This will enable the IDS to get full access to the whole packet, and to gain better understanding of the situation. VI. RELATED WORK The concept of integrating multiple layers of the protocol stack for efficient intrusion detection was introduced by Zhang and Lee [5]. They developed a cross layer based IDS architecture to study the abnormalities in the network using anomaly detection. Another remarkable attempt was made by G. Thamilarasu et al [6] in the CIDS intrusion detection architecture. However, both architectures are based on anomaly detection techniques in wireless networks, rather than Protocol Anomaly techniques which are more reliable as mentioned earlier. In addition, CIDS system is confined to a certain type of attack which is Denial of Service (DoS) attacks. The WebSTAT system [7] provides intrusion detection for web servers. It works by correlating both network-level and operating system-level events with entries contained in server logs. However, no correlation is performed within concurrently executing protocols at the same layer. This shortcoming is addressed in SCIDIVE system [8], which is on the other hand dedicated to VoIP protocols. SCIDIVE can be considered a signature based detection system rather than an anomaly based. It works by accessing packets from multiple protocols in a system and compare them against well created cross-protocol rules. As mentioned previously, signature based systems lack the ability to detect new and novel attacks, and the rules database needs to be updated on a regular basis following new attacks. In addition, none of the previous attempts considered using artificial intelligence techniques in the detection process to reduce the number of false alarms. We contend that, our proposed design is one of the very few ambitious attempts to address the problem of intrusion detection at different layers using both types of correlation, vertical and horizontal, based on protocol anomaly detection with fuzzy logic refinements, without being confined to a certain type of attack or application.
VII. CONCLUSION AND FUTURE WORK In this paper, we introduced novel cross-protocol design for Intrusion Detection Systems in the Next Generation Networks. Our detection is based on a cross protocol design that correlates anomalies from different protocols involved in a session both vertically and horizontally across stack layers, and aided by fuzzy logic. We believe that the proposed architecture will help improve the efficiency of IDSs by covering the whole range of threatened protocols based on the solid platform provided by Protocol Anomaly detection techniques. False positives and false negatives are expected to be reduced dramatically, due to the richness of information provided to the Decision Making mechanism, the use of fuzzy logic, and information exchange between different protocolanomaly based modules. The architecture shown in figure 3 is now under construction. Currently, we are building a vertical detection module that involves both TCP and IP protocols. Our next step will be building a horizontal module which addresses SIP and RTP protocols. The last stage is to add fuzzy logic in the packet checkers and the decision making system to improve the system efficiency. REFERENCES [1]
M. Mampaey, O. Pariadaens, “Alcatel Vision for Secured Next Generation Networks,” Technology white paper, Alcatel, 1st March 2005, Available at: http://whitepapers.zdnet.co.uk/0,39025945,60151912p39000411q,00.htm. [2006 April]. [2] V. Paxson, “Why Understanding Anything about the Internet Is Painfully Hard,” UCB Berkeley MIG Seminar, April 1999, http://www.icir.org/vern/talks/vp-painfully-hard.UCBmig.99.ps.gz. [2006 May]. [3] J. Luo, “Integrating Fuzzy Logic With Data Mining Methods for Intrusion Detection,” Msc Thesis, Mississippi State University, 1999. [4] J. Gomez, D. Dasgupta, “Evolving Fuzzy Classifiers for Intrusion Detectio,” In Proceddings of the 2002 IEEE Workshop on Information Assurance, United States Military Academy, West Point, NY June 2001. [5] Y. Zhang and W. Lee, “Intrusion detection in wireless ad-hoc networks,” in Mobile Computing and Networking, pp. 275–283, 2000. [6] G. Thamilarasu, A. Balasubramanian, S. Mishra, and R. Sridhar, “A Cross-layer based Intrusion Detection Approach for Wireless Ad hoc Networks,” Submitted in International Workshop on Wireless Sensor Networks and Security, in conjunction with IEEE MASS 2005. [7] G. Vigna, W. Robertson, V. Kher, R.A. Kemmerer, “A Stateful Intrusion Detection System for World-Wide Web Servers,” In Proceedings of the 19th Annual Computer Security Applications Conference (ACSAC ’03), December ‘03. [8] Y. Wu, S. Bagchi, S. Garg, N. Singh, T. Tsai, ” SCIDIVE: A Stateful and Cross Protocol Intrusion Detection Architecture for Voice-over-IP Environments,” In Proceedings of the 2004 International Conference on Dependable Systems and Networks (DSN’04). [9] G. Howard Cooper, “An Argument for Soft Layering of Protocols,” Msc Thesis, Massachusetts Institute of Technology, 1983. [10] V. Paxon, “Bro: A System for Detecting Network Intruders in RealTime,” In Proceedings of the 7th USENIX Security Symposium, San Antonio, TX, 1998. [11] Ibrahim S. Abdullah, and Daniel A. Menasce “PROTOCOL SPECIFICATION AND AUTOMATIC IMPLEMENTATION USING XML AND CBSE,” In the Proc. of Int. Conf. on Communications, Internet and Information Tech. (CIIT2003), Scottsdale, AZ, 17-19 Nov. 03. [12] R. Sekar, A. Gupta, J. Frullo, T. Shanbhag, A. Tiwari, H. Yang and S. Zhou ” Specification-based Anomaly Detection: A New Approach for
Detecting Network Intrusions,” CCS’02, November 18–22, 2002, Washington, DC, USA.
Bazara I A Barry was born in Madani, Sudan. He received his B.Sc and M.Sc degrees in computer science at University of Khartoum – Faculty of Mathematical Sciences in 2001 and 2004 respectively. He joined University of Khartoum in 2002 as a teaching assistant then became a lecturer in 2004. He is currently a PhD student in Electrical Engineering at the University of Cape Town. His research interest is on security in the next generation networks. H Anthony Chan received his PhD in physics at University of Maryland, College Park in 1982 and then continued post-doctorate research there in basic science. After joining the former AT&T Bell Labs in 1986, his work moved to industry-oriented research in areas of interconnection, electronic packaging, reliability, and assembly in manufacturing, and then moved again to network management, network architecture and standards for both wireless and wireline networks. He had designed the Wireless section of the year 2000 state-of-the-art Network Operation Center in AT&T. He was the AT&T delegate in several standards work groups under 3rd generation partnership program (3GPP). During 2001-2003, he was visiting Endowed Pinson Chair Professor in Networking at San Jose State University. In 2004, he joined University of Cape Town as professor in the Department of Electrical Engineering. Prof. Chan is Administrative Vice President of IEEE CPMT Society and had chaired or served numerous technical committees and conferences. He is distinguished speaker of IEEE CPMT Society and is in the speaker list of IEEE Reliability Society since 1997.
