Multi Layer Architecture For Collaborative Intrusion Detection

Published Date: 02 Nov 2017

Abstract

Current reactive and individual network security products are not capable of withstanding the onslaught of diversified network threats. As a result, a new security paradigm, where integrated security devices or systems collaborate closely to achieve enhanced protection and provide multi layer defenses is emerging. In this paper, we present the design of a collaborative architecture for multiple intrusion detection systems to employ mutually to detect network intrusions at real time. The detection is made more efficient and effective by using collaborative smart agents, relevant knowledge base and combination of multiple detection sensors. The architecture is composed of three parts: Collaborative Alert Indexing, Signature based Alert Estimation and Alert Correlation. The architecture is meant at sinking the alert excess by correlating outcomes from numerous sensors to produce condensed views, dropping false positives by combining host system and network information into the estimation technique and correlating measures based on reasonable relations to produce global and manufactured alert report. The architecture is designed as a layer above intrusion detection for post detection alert analysis and security actions. The architecture has been executed and the executed results are presented in this paper.

Keywords: Network security; Intrusion detection; Alert; Smart agents; Collaborative work.

1. INTRODUCTION

Network computing has become an integral part of our daily life with the emergence of the digital society. With the advances in Collaborative work techniques, network based collaboration among individuals and groups can dramatically boost our productivity. But on the other hand, with the advent of the Internet Era, the widespread reliance on network as well as Collaborative work technology makes computer attacks launched over the Network more devastating than ever before. Due to its distributed nature, a distributed Collaborative work application can be more vulnerable than traditional individual applications. For instance, intruders can easily exploit a Collaborative work application by disguising themselves as remote trusted entities and sending an executable file to the victim and executing it via a shared application. Consequently, the success of a Collaborative work application will be largely determined by the success of its security systems. Therefore, security systems inevitably become an essential part of Collaborative work applications. On the other hand, in response to the daunting threats of network attacks especially the increasing amount of distributed, coordinated malicious attacks such as DDoS attacks, the collaboration among different security systems, security information as well as the collaboration among experts from all over the world are becoming crucial in winning this war against spiteful network hackers. Consequently, now-a-days Collaborative work techniques are playing an important role in designing, developing and deploying security systems, devices and policies.

In response to the challenges from malicious network attacks, a promising approach to deterring intruders is to detect, record and analyze attacks and finally prosecute spiteful attackers using the evidence collected through the above process. Computer and network forensics is such an approach. This newly emerging discipline is becoming more important as the society is recognizing the seriousness of network attacks. It involves capturing, recording and analyzing network incidents in order to find out the resource of security attacks or additional setback events. It attempts to deter hackers from attacking a system as the chance of their discovery increases. It also searches for evidence after an attack has occurred. This evidence will be useful in prosecuting the hackers or in planning counter measures.

Intrusion Detection System (IDS) is an essential part of computer and network examining. Intrusion detection is a kind of security device or system that is deployed to observe network and host behavior including data flows and information accesses etc. and discover suspicious behavior and capture relevant evidence for later use. Its main purpose is to detect real time, ongoing intrusions and warn system administrators through alarms so that actions could be taken to stop the intrusions or discover the damage if the attacks had already succeeded.

Currently, there are two basic approaches to detection of an intrusion [1]. The first approach, called the anomaly detection (also called the behavioral detection), is to define and characterize the correct static form and the adequate vibrant activities of the system and then to detect illegal changes or unfair behavior. The second approach, called the misuse detection (also called the signature detection), involves distinguishing known ways to break into a system.

Every known penetration technique is typically depicted as a prototype. The misuse detection system looks for explicit prototypes. The prototype may be a fixed bit string such as a precise virus bit string inclusion. Instead, the prototype may describe a suspect set or sequence of actions.

Intrusion detection systems (IDS) have been built in the past 15–20 years using both the approaches: the anomaly detection and the misuse detection. Some of the IDS systems including EMERALD [2] and DIDS [3], are exploring a hybrid approach. In recent years, intrusion detection products have been widely deployed and are beginning to gain acceptance as a worthwhile investment. But neither of these two detection systems is quite satisfactory. For example, the anomaly detection systems often generate too many false positives. This happens because the deviation from the normal behavior does not always correspond to the occurrence of an attack. Besides, the critical threshold is hard to define precisely. The misuse detection systems can only detect those intrusions depicted in their signature repository. Therefore, new or slightly modified intrusions cannot be captured by misuse based intrusion detection systems.

In addition to the above weaknesses, IDS products are subjected to many other problems including alert flooding, too many false positive and false negative alerts, isolated alerts against a series of attacks, blindness to network and hosts they are observing etc.

Many of the above weaknesses in traditional IDS products are due to the lack of various collaborations including: (a) collaboration among different detection systems, (b) collaboration between intrusion detection and other network management operations, (c) collaboration between detection and other security systems. In order to overcome this deficiency, in this paper, we propose architecture to enable collaboration among multiple intrusion detection systems using distributed smart agents and Collaborative work techniques. The ultimate goal of the proposed collaborative architecture is to make intrusion detection more accurate, more efficient and easier to use by system administrators. Where appropriate, some of these processes may even be automated. The key to the realization of this goal is the use of a collaborative architecture that uses distributed smart agents and relevant information knowledge bases. In the proposed architecture, different IDS products and the protected hosts and network asset information are integrated as a holistic security system through the deployment of a distributed system of autonomous smart agents interacting via message exchange and thus making decisions in a cooperative and coordinated manner. The architecture sits as a layer above intrusion detection system for post detection alert analysis and security actions.

2. RELATED WORK

Realizing the limitations of single detection systems, researchers began to explore the benefits of collaboration among different IDS products. The main objective of the IDS cooperation is to reduce the number of alerts generated by correlating different IDS outputs and discard false alerts. By threading multiple alerts generated by related attacks, cooperating IDS modules will be able to provide a global view of intrusion behavior.

The first IDS collaboration research was initiated in the IDES [4] project and then refined in the EMERALD [2] project. Currently, there are a number of ongoing projects in the area of alert indexing and the false positive reduction.

Honeywell is developing Argus, a qualitative Bayesian estimation technology to combine results from multiple intrusion detection systems [5]. Cuppens [6,7] is developing an intrusion detection indexing and correlation module (MIRADOR) using snort and e-trust. An expert-system based approach for similarity formulation is used in their work. SRI international [8] is using a probability-based approach to attribute similarity recognition. Another approach is the Tivoli Enterprise suggested by Debar and Wespi [9]. Their correlation technique uses a significant system to state what types of alerts may pursue a given alert category.

Concurrent Engineering Research Center (CERC) of West Virginia University [10] have developed a general collaborative alert management. We are developed a generic Collaborative IDS and analysis architecture for multiple IDS products with smart agents and Signature based alert estimation. Our approach differs from the above ones in the following aspects. First, we categorize the correlation into two different types: (a) information asset correlation and (b) the alert correlation. Information asset correlation uses the network and host hardware and software information to estimate the alert priority and the likelihood of the success of the attack. The alert correlation is to correlate different alerts based on the logical relations among them to provide a global vision of the effects of intrusion. Second, we designed a generic collaborative architecture for multiple diversified IDS and alert estimation. This architecture sits as a layer above IDS products. Third, based on the specific alerts, appropriate security solutions collected from multiple vulnerability monitoring corporations and organizations will be provided with the alert in order to assist security administration in taking the appropriate actions. Although this architecture design that uses smart agents and relevant knowledge base to provide a layer above intrusion detection is novel, using agents in detecting malicious intrusions itself.

Distributed Intrusion Detection System (DIDS) [3] uses agents to aggregate audit reports from hosts. The architecture consists of a host manager, an observing process or collection of procedures; Autonomous Agents For Intrusion Detection (AAFID) [11] is a distributed anomaly detection system that employs autonomous agents at the lowest level for data collection and analysis. At the higher levels, other agent entities are then used to obtain a global view of behavior; Helmer et al. [12] use mobile lightweight agents in intrusion detection. Their agents travel between monitored systems to obtain, classify and correlate information. Finally, this information is used to detect suspicious behavior.

All the above researches use agents to some extent to collect information for the purpose of detection. Therefore, the information is often retrieved from audit logs etc.

Besides, in many agent based detection approaches such as DIDS, agent technology is used only minimally. Our design is a layer above intrusion detection. The information is mainly used for the alert estimation and security decision making. We also exploit agent technologies such as agent communication languages and services for the collaboration among distributed smart agents. The architecture design we have been developing is dubbed as Multi layer architecture.

The implication is that the third eye is the system we are developing to watch over a computer network.

3. THE MULTI LAYER ARCHITECTURE DESIGN

As shown in Figure.1, the collaborative architecture consists of three main components: (a) collaborative alert indexing, (b) Signature based alert estimation and (c) alert correlation. Each of these three parts is depicted in the following sections.

3.1. Collaborative alert indexing

Collaborative alert indexing is mainly aimed at mitigating alert flooding. Meanwhile, a loosely coupled collaboration among multiple IDS products is also achieved by using smart agents and advanced algorithms. This component provides three functions: (i) alert preprocessing, (ii) alert clustering and (iii) collaborative alert merging.

These functions are provided by the following distributed agents.

3.1.1. IDMEF agents

To aggregate alerts from multiple IDS products with different output formats, first the component needs to convert the diversified formats into a unified standard representation. The format we chose is the Intrusion Detection Message Exchange Format (IDMEF) [13], which is emerging as an industry standard.

The conversion of heterogeneous alert output into the standard IDMEF format is performed by IDMEF agent. An IDMEF agent is an independently running entity that collects alerts emanated from an IDS product and converts them into the standard IDMEF format. In our design, for each IDS product, a single IDMEF agent is developed and deployed close to that IDS product. The IDMEF agent is then dedicated to that IDS product and converts its output into IDMEF format. Therefore, an IDMEF agent can be implemented in any language and deployed in any system atmosphere provided the output is in the standard IDMEF format. Whenever a new IDS is integrated into the system, we only need to provide a connection to IDMEF agent to serve it. Therefore, the architecture is highly scalable and can evolve with advances in IDS systems. IDMEF agent do not communicate directly with each other. Instead, they store all their outputs in an IDMEF database for later analysis.

3.1.2. Clustering agent

As IDMEF agent store alerts in the IDMEF database, another kind of agent called clustering agent is fulfilling its responsibilities. The Clustering agent groups the alerts into different clusters according to their source, target, time and classification. Each cluster is then used by a ‘representative alert’ or Meta alert to represent that cluster. Clustering is used to eliminate duplicate alerts and group alerts from the occurrence of the same attack. After clustering, the amount of alerts can be significantly reduced. Clusters are then used in alert merging process by a Merging Agent.

3.1.3. Merging agent

Currently, the collaboration among multiple IDS products in Multi Layer architecture may be characterized as loose and it is transparent. We have designed the Multi Layer architecture as a layer above IDS products. In the architecture, IDS products are integrated into it as black boxes. The collaboration is then achieved by this upper layer. In practice, when IDS vendors develop their own IDS products, they seldom have the collaboration with others in mind. Consequently, their alert formats are heterogeneous and detection systems are proprietary and not released to public. Therefore, direct collaboration among multiple IDS products from different vendors is very difficult and may not be feasible at all.

The collaboration among multiple IDS products is done indirectly by the alert merging agent in the Multi Layer architecture. The analysis of alerts from multiple IDS products results in useful information that could be gathered collectively and resolution of conflicts is all carried out by this kind of alert merging agent. This agent is also used to merge the alerts from different IDS products into blended alerts.

Alert merging agent is highly smart because it uses a voting algorithm to solve conflicts when IDS products have different ‘voices’. The algorithm works as follows: if the number of IDS reporting alerts is greater than the number of non-active IDS systems when an event occurs, the voting algorithm will notify that an attack has occurred and generate a synthetic alert. But when only two IDS are deployed, we chose a priority based voting algorithm. When conflict arises, the algorithm will generate a synthesized alert only when the priority levels of the generated alert pass the associated priority screening levels. The priority of the blended alert will still be reduced one level lower compared to its original one. After the alert merging process, clean and synthesized alerts containing the detailed information from all of the active IDS systems are sent to the Signature based estimation component for further analysis.

3.2. Signature and Anomaly based alert estimation

One of the main weaknesses of current IDS products is that they often generate too many false positive alerts or system immune alerts. False positive alerts correspond to those false alerts generated against normal or slightly skewed normal behavior. System immune alerts are real attacks but not harmful to the targets. In practice, they should be treated with different priority compared to real harmful attacks. But current IDS products often are unable to filter out these two kinds of alerts and even assign critical priorities to these alerts. For instance, in our laboratory, we once observed an IDS system send out an alert with highest severity in response to an attack that was exploiting a well known vulnerability of a Windows 2000 operating system even though the system that the IDS was observing was a Linux operating system. System security operators will be distracted by these impossible-to-succeed attacks or low priority probes. High volume of false positive alerts will make the system security operator miss the real spiteful attacks that are more likely to succeed. There are several reasons why current IDS products are prone to generate excessive false positive alerts or system immune alerts. The poor quality of signature sets is one. The difficulty to define precisely the boundary between abnormal and normal behavior is another one. But another important reason is attributable to the fact that current IDS systems are often unaware of the system context they are protecting. They often are not fully integrated into the system atmosphere they are running on. They only serve as an add-on security product instead of being an integral part of the whole security system. Fundamentally, this situation is the result of a lack of collaboration among IDS products with other network or system management entities. That is to say, there is a lack of message and information sharing and exchanging between them.

IDS products are isolated from the rest of the system and the network. This isolation between the IDS products and the system security atmosphere results in the high rate of false positives and system immune alerts. Common techniques suggested by IDS vendors and used in practice to solve the above problem are alert filtering or to manually tune up the signature sets in misuse based detection systems and to reconfigure the parameters in abnormal detection systems. But these techniques only push the burden on to system administrators’ shoulders, but not solve the problem. Furthermore, alert filtering and signature tuning will require the system security operators to understand the IDS signatures thoroughly. Filters are also difficult to maintain. For instance, different filters will be needed for different IDS products, since each IDS product has its unique signature sets or detection algorithms. Our proposed solution to the above problem is the use of Signature based alert estimation. We decided to use autonomous smart agents to fully integrate IDS systems into the network atmosphere.

The message exchanging and information sharing among these smart agents and extensive knowledge about existing vulnerabilities in protected system and potential attack techniques enable us to efficiently and effectively estimate generated alerts. In practice, most of the existing IDS products especially commercial products employ signature based detection approaches. This is because signature based approach is much easier to implement and is much more successful in practice. Furthermore, many IDS products, such as Snort [14], refer their signatures to standard CVE [15], Bugtraq [16] or CERT [17] etc. exploits. Meanwhile, these security observing organizations such as CVE, Bugtraq and CERT generally list all the vulnerable systems and application information infected by this attack. Consequently, this attack and the infected system information match can be used to estimate the attack severity and its likelihood of success.

The function of this Signature based estimation is to eliminate these kinds of system immune or false positives alerts and enable the system security operators to focus on real threatening alerts. In this context, the ‘knowledge’ implies the integration of the network topology structure, the hardware devices, the operating systems and the application information combined with well known vulnerability information into the threat management and analysis system. In addition, ‘knowledge’ also refers to the knowledge about the protected system and ongoing intrusion techniques. An information asset containing all the above hosts and network information is used in the estimation process to filter out false positives and rank the severity of attacks. The vulnerability knowledge base (Figure. 1) stores all well known exploits and system vulnerability information together with the corresponding security solutions. The functions provided by this Signature based estimation are similar to alert filtering and signature tuning. But it significantly relieves the heavy burden on system administrators and is more efficient and easier to configure and maintain. Figure. 2 show the architecture of the estimation component and illustrate the alert estimation process in a typical scenario. In the estimation process, the following key components are involved:

3.2.1. Host agents

A smart host agent is installed on each host in the observed network. They collect low level details of the host’s system and configuration information. Host agents are autonomous. They are also smart and are aware of the system atmosphere they are running. Host agents do not communicate with each other directly. Instead, they all communicate with a director agent.

3.2.2. Director agent

Director agent coordinates with host agents. The communication and collaboration between host agents and director agent is vital to the whole estimation component. Host information collected from host agents are first sent to the director agent. The director agent then stores the information in appropriate places in the network and host asset knowledge base. Meanwhile, if any information needed by an estimation process is missing in the knowledge base, director agent then requests the corresponding host agent to collect that information and send it back.

3.2.3. The communication language and coordination systems between host agents and director agent

To enable coordination among distributed agents, we have adopted Collaborative work and groupware techniques. For instance, we need to employ a common communication language to facilitate message exchange and information sharing. An Agent Communication Language (ACL) provides agents with a means to exchange information and knowledge. In the design of communication between director agent and host agents, we do not use the standard ACL, such as KQML [18] or FIPA ACL [19]. Instead, we decided to design and implement a simple ACL language or protocol to be used in Multi Layer Architecture. We did not choose a fully specified ACL such as KQML or FIPA ACL because the communication between director agent and host agents is simple—mainly of the query–answer type. The domain ontology is the same: system atmosphere information. But the syntax of the message in our protocol is adapted from KQML. For instance, a simple message sent from a host agent to the director agent can be encoded as:

(Inform Director

:sender hostagent A

:content ( Linux OS 3)

:content (message)

:receiver director)

In this simple message, the host agent A is telling the director that the host it is observing is running a Linux operating system with version 3. A query from the director agent to the host agent can be encoded as:

(Enquiry Agent

:sent Director

:content (PATCH LEVEL)

:receiver hostagent A)

To facilitate communication between director agent and host agents, we also need an infrastructure of services with naming, registration and other services such as encryption. Naming services are used to provide matching between host agent name and IP address. Registration service is used by host agent to specify the system information it can collect and provide to director. This information is then used by director agent to determine which information it can query to which agent. Encryption service is very important in the communication among agents because the message being exchanged contains sensitive information, which should be prevented from unauthorized access. In addition, the message should also be signed by using the host agent’s secret key to verify its integrity.

The communication or coordination model between the director agent and the host agent is a ‘push–pull’ model. The host agent pushes system atmosphere information to the director under three scenarios: (1) After each reboot of the host, host agents automatically start to collect system information and send it to the director agent. (2) After a user defined update time interval, the host agents look for updated system information and send it to the director agent. (3) Upon arrival of the director agent’s request, the requested information is sent. The director agent actively pulls information from host agents. This model corresponds to the dominant ‘request reply’ communication paradigm over the Network. The director pulls information from host agents under two situations: (1) Upon the arrival of a request from an estimation process. (2) After a certain user defined updating time interval. Besides host agents and director agents, there is a special agent, which is a network agent deployed for each network segment.

3.2.4. Network agent

Network agent manages an internal topology map of the protected network. Network agent identifies the available assets on the network, their current state (up or down), IP address to hostname match, open ports and the application behind those ports, active TCP and UDP network services and hardware information etc. Network agent runs at intervals to maintain an updated topology map of the protected network. The network agent does not communicate with director agent directly. The host information and network information collected by host agents and network agents are all stored in a network and hosts asset knowledge base. This knowledge refers to the knowledge about the protected network.

3.2.5. Knowledge base

This knowledge base contains vibrant network and host asset information. The information is used to compare with vulnerability requirement information to estimate alerts and provide appropriate security solutions for real harmful attacks. In addition to knowledge about attacks there is a need to know can about Vulnerability. The knowledge base contains known vulnerabilities as well as the known penetration approaches exploiting these vulnerabilities identified by the security observing organizations and their corresponding security solutions. Vulnerabilities are categorized and sorted according to CVE, bugtraq and CERT vulnerability references. For each vulnerability reference, there is a corresponding security solution to facilitate security decision making.

3.2.6. Expert system engine

In our alert estimation framework, the estimation processes are executed by an expert system engine. After indexing process, IDS alerts are asserted as ‘facts’ into the knowledge base. Appropriate estimation process is then triggered by the assertion of new alerts. We bring in an expert system into the framework due to the following reasons: (1) IDS products should never be individual. They should be used closely coupled with security policy, contingency plan and other defense systems. In reality, security policy and contingency plan are often implemented with predefined security rules. Therefore, after the estimation process, certain defense actions triggered by these rules can be executed by the expert system to deter intrusions. (2) By representing alerts as facts in knowledge base, we can query the knowledge base to find relationships between facts (alerts). This is the primary purpose of our third component Alert Correlation in the Multi layer Architecture. In addition, predefined defense rules can take actions based on the contents of one or more facts (alerts), thus leading to automation of some defense processes.

3.2.7. Alert estimation in a typical Circumstance

Figure. 2 depict the alert estimation process in a typical scenario. After the collaborative alert indexing process, IDS alerts are asserted as ‘facts’ into the knowledge base. After that, an estimation process is loaded and executed. The estimation process first queries the vulnerability knowledge base to see whether the NIV-2013 vulnerability information is available or not. If not, the alert is marked as a higher priority alert. Otherwise, the vulnerability information including security solution suggestions concerning NIV-2013 is retrieved from the knowledge base. The estimation process then identifies the target and uses network topology information to roughly estimate the alert. If the target’s Operating System is in the vulnerability list, the estimation process then goes to the next step. Otherwise, the process marks the alert as a lower priority alert. In this example, the estimation process goes to next step to retrieve target’s host information. The estimation process then compares the target information against the vulnerability requirement information. A relevance score is calculated based on the matches of the information. The relevance score ranges from 0 to 1 with 1 meaning a perfect match and 0 meaning no match at all. Finally the estimation process presents the estimated result as well as suggested security solution corresponding to the attack to system administrator to facilitate decision making.

3.3. Alert Correlation

The main purpose of this component is to find out the logical relationships among the alerts. Attackers are likely to launch a series of attacks against their targets. Current IDS systems can only generate isolated alerts based on each step it detects. Smart hackers are more likely to disguise their real purpose by launching many other minor attacks. Alert correlation component is used to correlate alerts based on logical relationships among the alerts. This function will provide the system security operator with a great insight into where the initial attacks came from and where they actually end up. This function can also be used to find prototypes among series of attacks. After the alert correlation, a high-level report providing an overall view of the attacks will be presented to the system security operators. The Alert Correlation component has other functionalities. By appropriate correlation of the alerts from different IDS systems, we will be able to verify the occurrence of a certain attack. For instance, a network based IDS detects a suspicious remote buffer overflow attack to get shell access to a server machine. But due to its limitation, it does not know what is really going on inside that host after that. Meanwhile, a host-based IDS system deployed inside the same server detects a suspicious shell process and generates an alert. Therefore by correlating these alerts from the two different IDS products, the system security operator can further confirm that some remote shell access attack is in progress. Furthermore, since each IDS product has its own blind spots, a correlation can help to remove some of the false negatives.

4. IMPLEMENTATION AND EXPERIMENTS

As of the writing of this paper, we have implemented the collaborative alert indexing component and the Signature based estimation component of the architecture in our prototype system. We used two different network based IDS products: Snort [4] and Prelude [20]. These two IDSs were installed in a Redhat Linux system to observe a subnet with heterogeneous operating systems and different configurations. The alert preprocessing, alert clustering and alert merging were all implemented in Perl. For each IDS, a Perl script was written. This Perl script served as an IDMEF agent to that IDS sensor, converting its alerts into the standard IDMEF format and storing alerts into a relational IDMEF database. Another Perl script was written as a clustering agent to cluster the alerts into different groups based on the combination of the source, target, time and the classification information. The merging agent was also written using Perl script, which includes the voting algorithm. Host, director and network agents were also implemented in Perl. To enable communication among agents, we also needed to provide processes that can parse incoming messages, compose messages for transport and channel them through the network using some lower level network protocols. In our implementation, these were implemented as shared Perl libraries. A Jess Expert Engine [21] was deployed for the estimation process. We designed and implemented two different estimation processes, one for Unix (Linux) and the other for Windows. All estimation processes were implemented as Java classes and loaded on demand to estimate the corresponding alerts. Finally, a PHP front end was provided as a web portal to view the alerts, system and network information. We validated our approach using an attack base of 50 attacks.

During that experiment, Snort generated 447 alerts and Prelude generated 724 alerts. Of these 50 attacks, Snort detected 24 attacks and Prelude detected 21 attacks. Five attacks were missed by both of them. Thus, two metrics are to be used to evaluate the proposed CIDS performance, namely, the intrusion DR and FAR [22,23]. Our indexing component generated 45 indexing. The voting algorithm in the merging process successfully removed 10 false positives of Snort and eight false positives of Prelude. We deliberately designed our attacks in order to generate system immune alerts and false positives. For instance, we attacked one of our Linux hosts with several IIS buffer overflow attacks, and attacked one of our Window 2000 machines with Service Pack 3 installed containing the necessary hotfix with an attack exploiting certain vulnerability in Service Pack 2. But during the experiment, all of the attacks were detected by Snort and Prelude and some of the generated alerts were marked with the highest severity. But the Signature based estimation process successfully estimated them as system immune alerts. Furthermore, we also launched some exploits, which will certainly make the targets vulnerable. This time the alert estimation process not only marked them as high priority alerts, but also suggested us the corresponding appropriate security solutions. So far, we have conducted the attacks within our isolated test bed. Our next plan is to test the system in real atmospheres. Performance of the alert indexing component and the alert estimation component will be estimated in an open and real atmosphere.

5. CONCLUSION AND FUTURE WORK

In this paper, we have presented the need for collaborative intrusion detection and collaboration among intrusion detection and other network management systems. We proposed a collaborative architecture design, named as Multi layer Architecture to coordinate multiple intrusion detection systems using various distributed smart agents, relevant information knowledge base and Collaborative work techniques used to cluster and merge alerts from multiple IDS products to achieve an indirect collaboration among them. By combining the strengths of the heterogeneous IDS products, alert correlation can also be used to verify the occurrence of certain attacks and eliminate some false negatives.

The collaborative intrusion detection and knowledge based alert estimation architecture presented in Multi Layer Architecture can be widely applied to the design of integrated intrusion detection products and related network security engineering applications, which will be more efficient and effective in terms of detecting real spiteful attacks. As of this writing, we have already implemented the first two components in our prototype system. We then tested our system with a set of attacks and the results we obtained were quite satisfactory.