Development of Intrusion Detection System Software

Published Date: 02 Mar 2018

INTRODUCTION

Heavy reliance on the Internet and worldwide connectivity has greatly increased that can be imposed by attacks plunged over the Internet against systems. It is very difficult to prevent such attacks by the only use of security policies, firewall or other mechanism because system and application software always contains unknown weaknesses or many bugs. In addition, complex, often unforeseen, interactions between software components and or network protocols are continually exploited by attackers. Successful attacks inevitably occur despite the best security precautions. There for intrusion detection system has become an essential part of the system because they can detect the attacks before they inflict widespread damage. Some approaches detect attacks in real time and can stop an attack in progress. Others provide after-the-fact information about attacks and can help repair damage, understand the attack mechanism, and reduce the possibility of future attacks of the same type. More advanced intrusion detection systems detect never-before-seen, new, attacks, while the more typical systems detect previously seen, known attacks [1].

MOTIVATION

The speed of growth of Internet is very fast without any end. With this growth the threat of attacks is also increasing. Because as we all know that theft can be occurred over the Internet from all over the world. So we require a system which can detect the attack or theft before there is some loss of information and reputation of organization or any individual. There are many solutions has been provided by the researchers and from many companies like firewall, intrusion detection system and IPS to stop the attacks. But still it is very hard to detect the attacks like DoS and worm propagation before they widespread, because regularly thousands of attacks are being developed and for a signature based intrusion detection system it is very hard to detect these kinds of new attacks with perfect accuracy. Mostly intrusion detection system generates many false alarms. These false alarms can affect the other processing of the network.

If somehow any attacker gets to know that there is an intrusion detection system in the network then, the attacker will want to disable the intrusion detection system. His/her first target will be the intrusion detection system before attacking the network. So there should be proper security policies for deploying the IDS to take proper advantages of it.

PROJECT OBJECTIVES

Security is the main concern for any network. Every day thousands of attacks are created so that alarms and logs should be generated properly for reducing their effect. intrusion detection system and IPS are mostly used devices for providing these kinds of solutions. But there are many issues like performance and accuracy. So the main objective of the project is to develop a signature based intrusion detection system for DoS attacks with better scalability and performance i.e. intrusion detection system with minimum false alarms and with better throughput. In this study the example of TCP SYN flood attack will be taken for implementing and evaluating the performance and scalability of the developed intrusion detection system.

Second Objective of this study is to discuss the policies for implementing the intrusion detection system securely. And these policies shall also be evaluated.

Intrusion detection system

intrusion detection systems (IDS) are software or hardware systems that automate the process of monitoring the events occurring in a computer system or network, analyzing them for signs of security problems. As network attacks have increased in number and severity over the past few years, intrusion detection systems have become a necessary addition to the security infrastructure of most organizations [2, 48]. There are many different types of intrusion detection system and they can be characterized by different monitoring and analysis approaches. Each approach has different advantages and disadvantages. All approaches can be described in terms of generic process model for intrusion detection systems. Many intrusion detection systems can be described in terms of three fundamental functional components information source, analysis, and response [2].

OVERVIEW

Chapter 1 In this chapter we will give a brief introduction of whole project, what is the motivation for selecting this project. What are the main objectives of this project? And what is the main problem which will be considered in this project.

Chapter 2 is all about the literature review. In this chapter many different aspects of the intrusion detection system will be discussed like why we require intrusion detection system, different type of intrusion detection system, need for intrusion detection system, about attacks different types of attacks and many other different facts about intrusion detection system which can help to improve the knowledge about intrusion detection system.

Chapter 3 will focus on the analysis and designing part of the intrusion detection system. How a computer system can be designed. What s the system engineering and different type’s models will be discussed.

CHAPTER 2

NEED FOR INTRUSION DETECTION SYSTEM

Internet is carrying more traffic than ever before and still growing in the size without any end. Along with the explosive growth comes an increased threat from Internet related attacks. The Internet allows theft to occur from anywhere of the world [14].

Many threats impact on the operation of your computer network. Natural threats such as flood fire and tornadoes, causes unexpected disruptions. Most companies have well-defined procedure to handle these natural attacks. Security procedures designed to combat hacker attacks, an unsecured network will definitely be attacked. The only question is when the attack will occur [14].

COMPUTER ATTACKS AND VULRANABILITIES

intrusion detection systems have been adopted by many organizations because the organizations know that intrusion detection systems are necessary component of the security architectures. But still intrusion detection system is not too much popular, most organizations lack experienced intrusion detection system operators. intrusion detection system can be most effective if the human operates it. But before developing a signature based intrusion detection system the knowledge of the attacks is must. Signatures is a set of rules that sensor uses to detect typical intrusive activities. These rules are based on various criteria i.e. IP protocol parameters, transport protocol parameter and packet data [12].

THE PHASES OF THE ATTACKS

Attack can be divided into three different phases. The first phase is defining the goal for attack. The second phase is the reconnaissance attack, also known as the information gathering. After collecting the information the attacker proceed to the third phase, the attacking phase [12].

FIRST PHASE: GOALS OF ATTACK

Before attacking a network or system, an attacker sets her goals or objectives. When attacking network the attacker can have various goals:

• Data manipulation
• System access
• Elevated privileges
• Denying availability of the network resources

MOTIVATION

• Revenge
• Political activism
• Financial gain

Attackers attempt to disrupt network to discredit the particular organization’s image [12].

RECONNAISSANCE BEFORE THE ATTACK

Collecting the information is the attacker’s second step in launching an attack against the network. Successful reconnaissance is also important for successful attack. Attackers use two main mechanisms to collect the information about the network.

• Public data source
• Scanning and probing

An attacker sometime starts his knowledge search by examining public information available about company. By using these kind of information the attacker can determine that where the business is located, the business partners, the value of the company assets and much more.

And through scanning, the attackers use remote reconnaissance to find specific resource on the network.

The goal of the information gathering is to pinpoint weak points on the network where an attack is likely to succeed. By pinpointing specific weakness on the network, the attacker can launch an attack in the future that generates minimal traffic or noise on the network. This greatly reduces the likelihood of detection during the actual attack [12]. For example: ping sweep, vertical scan, horizontal attack, DNS query, block scan and many more.

THE ACTUAL ATTACK

After an attacker maps the network, he researches known vulnerabilities for the system that he detected. The attacker’s goal at this stage is to gain access to resources of the network i.e.

Unauthorized data manipulation, system access, or privilege escalation.

ATTACK METHODOLOGY

Regardless of the motivation or personal preferences, an attacker has several attack methodologies from which to choose [12]:

• Ad hoc (random)
• Methodological
• Surgical strike (lightning quickly)
• Patient (slow)

AD HOC (Random)

An ad hoc attack methodology is unstructured. An attacker using this methodology is usually disorganized and those types of attacks frequently fail. It is difficult to comprehensively locate targets on the network.

METHODOLOGICAL

It provides a well-defined sequence of steps to attack a network. First, the attackers use the reconnaissance to locate the targets. Next the attacker locates the exploits for known vulnerability on the target. Finally when he satisfies with his toolkit he starts attacking system on the target network.

SURGICAL STRIKE (Lightning Quick)

Many times the attacker uses an automated script against a network. The entire attack is completed in a few seconds. Before the system administrator or security analysts have time to react and make any decision.

PATIENT (Slow)

It refers to how quickly the attacker executes his attacks. Usually the one uses a patient (slow) methodology to avoid detection. Many intrusion detection systems have difficulty detecting attacks that occurs over long period of time.

BACK DOORS

Viruses and worms provide a vehicle for an attacker to wreak havoc on your network and potentially the Internet. However, the spread of viruses and worms is much harder to determine in advance. Viruses and worms are much harder to determine in advance.

Trojan horse program enables an attacker to establish back door on systems. However Trojan horse requires some type of transport vehicle [12].

DENIAL OF SERVICE TECHNIQUES

The purpose of DoS attacks is to deny legitimate access to the network resources. These attacks include everything from simple one-line commands to sophisticated programs written by knowledgeable hackers. There are different types of DoS attacks some of them are-

• Network resource overload
• Host resource starvation
• Out-of-band attacks
• Distributed attacks

NETWORK RESOURCE OVERLOAD

One common way to deny the network access is by overloading a common resource necessary for network components to operate. The main common resource that can be attacked in the network bandwidth in several ways generating lots of traffic, distributing the attack across numerous hosts, and using a protocol flaws that amplifies the attack by soliciting help from many different hosts on the target [12].

Example- Smurf and Fraggle attack.

HOST RESOURCE STARVATION

The resources available at the hosts are also known as the attack point as well. One such resource is the buffer that a host uses to track TCP connections.

OUT-OF-BOUNDS ATTACKS

The first out-of-bounds attack category uses over-sized packet, it overflows the allocated buffer and causes the system crash. An over-sized packet attack is ping of death.

DISTRIBUTED ATTACKS

The latest trend in DoS attacks is for an attacker to compromise numerous hosts and then use all these compromised hosts to provide a massive against a specific target. These types of attacks are known as the distributed denial of service attack (DDoS).

DISTRIBUTION EFFECT

To disrupt the victims communication very badly, the attacker must compromise an agent machine that has more network resources than the victim. Locating and breaking into such a machine may prove difficult, if the target of the attack is well-provisioned site [16].

Distribution brings number of benefits to the attackers:

• By using distribution techniques, the attacker can multiply the resources on the attacking end, allowing him to deny service to more powerful machines at the target end [16].
• To stop a simple DoS attack from a single agent, a defender needs to identify that agent and take some action that prevents it from sending such a large volume of traffic. In many cases, the attack from a machine can be stopped only if the machine’s human administrator, or network operator, takes action. If there are thousands agents participating in the attack, however, stopping any single one of them may provide little benefit to the victim. Only by stopping most or all of them can the DoS effect be palliated [16].
• If the attacker choose agents that are spread widely throughout the Internet, attempts to stop the attack are more difficult, since the only point at which all of the attack traffic merges is close to the victim. This point is called aggregation point. Other nodes in the network might experience no telltale signs of the attack and might have difficulty distinguishing the attack traffic from legitimate traffic [16].
• In DoS attack executed from a single agent, the victim might be able to recover by obtaining more resources. For example, an overwhelmed Web server might be able to recruit other local servers to help handle the extra load. Regardless of how powerful a single agent might be, the defender can add more capacity until he outstrips the attacker’s ability to generate load. This approach is less effective in defending against DDoS attacks. If the defender doubles his resources to handle twice as many requests, the attacker merely needs to double the number of agents- often an easy task [16].

TCP-SYN ATTACK

The SYN-flooding attack is a Distributed denial-of-service method disturbing hosts that run TCP server processes. The attack take benefit of the state retention TCP performs for some time after receiving a SYN segment to a port that has been put into the listen state. The basic idea is to utilize this behavior by causing a host to retain enough state for bogus half-connections that there are no resources to establish new genuine connections [51, 52].

A TCP implementation may allocate to LISTEN state to be entered with either all, some, or none of the pair of IP addresses and port numbers specified by the application. In many common applications like web servers, none of the remote host’s information is pre known or preconfigured, so that a connection can be established with any client whose details are unidentified to the server ahead of time. This type of “unbound” LISTEN is the goal of SYN flooding attacks due to the way it is typically implemented by operating systems [51, 52].

For success, [51, 52] the SYN flooding attack relies on the victim host TCP implementation’s behavior. In particular, it assumes that the victim allocates state for every TCP SYN segment when it is received and that there is perimeter on the amount of such state than can be kept at any time.

The [51, 52] SYN flooding attack does not attempt to overload the networks recourses or the end host memory, but merely attempts to exhaust the backlog of half-open connections associated with the port number. The goal is to send a quick barrage of SYN segments from IP addresses (often spoofed) that will not generate replies to the SYN-ACKs that are produced. By keeping the backlog full of bogus half-opened connections, legitimate requests will be rejected. Three important attack parameters for success are the size of the barrage, the frequency with which barrages2 are generated, and the means of the selecting IP addresses to spoof.

Usually, [51, 52] systems implements a parameter to the typical listen () system calls that allows the application to suggest a value for this limit, called the backlog.

1 To be effective, the size of the barrage must be made large enough to reach the backlog. Ideally, the barrage size is no larger than the backlog, minimizing the volume of the traffic the attacker must source. Typical default backlog values vary from half-dozen to several dozen, so the attack might be tailored to the particular value determined by the victim host and application. On machines intended to be servers, especially for a high volume of the traffic, the backlogs are often administratively configured to higher.

Another aspect makes both DoS and DDoS attacks hard to handle: Defenses that work well against many other kinds of attacks are not necessarily effective against denial of service. For years, system administrators have been advised to install a firewall and keep its configuration up to date, to close unnecessary ports on all machines, to stay current with patches of operating systems and other important software, and to run intrusion detection system to discover any attacks that have managed to penetrate the outer bastions of defense [16].

Unfortunately, these security measures often will not help against denial of service. The attack can consist of traffic that the firewall finds acceptable. intrusion detection systems are of limited value in dealing with DoS, since, unlike break-ins and thefts, DoS attacks rarely hide themselves [16].

WHAT IS INTRUSION DETECTION SYSTEM?

intrusion detection systems gather information from a computer or network of computers and attempt to detect intruders or system abuse. Generally, an intrusion detection system will notify a human analyst of a possible intrusion and take no further action, but some newer systems take active steps to stop an intruder at the time of detection [4].

The goal of intrusion detection is seemingly simple: to detect intrusions. However, the task is difficult, and in fact intrusion detection systems do not detect intrusions at all—they only identify evidence of intrusions, either while they’re in progress or after the fact. Such evidence is sometimes referred to as an attacks “manifestation.” If there is no manifestation, if the manifestation lacks sufficient information, or if the information it contains is untrustworthy, then the system cannot detect the intrusion [5].

intrusion detection systems are classified into two general types known as signature based and heuristic based. Pfleeger and Pfleeger describe signature-based systems as “pattern-matching” systems that detect threats based on the signature of the attack matching a known pattern. Heuristic based systems, which are synonymous with anomaly-based systems, detect attacks through deviations from a model of normal behavior [6].

intrusion detection systems that operate on a single workstation are known as host intrusion detection system (HIDS), while those that operate as stand-alone devices on a network are known as NIDS. HIDS monitor traffic on its host machine by utilizing the resources of its host to detect attacks. NIDS operate as a stand-alone device that monitors traffic on the network to detect attacks. NIDS come in two general forms; signature based NIDS and heuristic based NIDS [7].

PROCESS MODEL FOR INTRUSION DETECTION SYSTEM

intrusion detection systems can be described in terms of three fundamental functional components [2, 48]:

• Information Sources the different sources of event information used to determine whether an intrusion has taken place. These sources can be drawn from different levels of the system, with network, host, and application monitoring most common.
• Analysis the part of intrusion detection systems that actually organizes and makes sense of the events derived from the information sources, deciding when those events indicate that intrusions are occurring or have already taken place. The most common analysis approaches are misuse detection based (signature based) and anomaly detection.
• Response the set of actions that system takes once it detects intrusions. These are typically grouped into active and passive measures, with active measures involving some automated intervention on the part of the system, and passive measures involving reporting intrusion detection system findings to humans, who are then expected to take action based on those reports.

INFORMATION SOURCE

The most common way to classify intrusion detection system is to group them by information source. Some intrusion detection systems analyze network packets, captured from network backbones or LAN segments, to find attackers [2]. It can be describe by dividing three different parts.

NETWORK BASED INTRUSION DETECTION SYSTEM

NIDS are intrusion detection systems that capture data packets traveling on the network media (cables, wireless) and match them to a database of signatures. Depending upon whether a packet is matched with an intruder signature, an alert is generated or the packet is logged to a file or database [8, 48].

Network-based intrusion detection systems often consist of a set of single-purpose sensors or hosts placed at various points in a network. These units monitor network traffic, performing local analysis of that traffic and reporting attacks to a central management console. As the sensors are limited to running the intrusion detection system, they can be more easily secured against attack. Many of these sensors are designed to run in “stealth” mode, in order to make it more difficult for an attacker to determine their presence and location [2, 48].

HOST INTRUSION DETECTION SYSTEM or HIDS

Host-based intrusion detection systems or HIDS are installed as agents on a host. These intrusion detection systems can look into system and application log files to detect any intruder activity. Some of these systems are reactive, meaning that they inform you only when something has happened. Some HIDS are proactive; they can sniff the network traffic coming to a particular host on which the HIDS is installed and alert you in real time [8, 48].

These types of intrusion detection systems run on host to reveal inappropriate activities on these hosts. The HIDSs are used for detecting the attacks from the inside and outside network. They provide snap shot about the existing system files and connect them to the previous. If the important system files were modified or deleted, the warning is sent to the administrator for inspection. The HIDS example is notice able on the machines with significant task; these machines do not expect the change of their configuration [9, 48].

APPLICATION-BASED INTRUSION DETECTION SYSTEM

Application-based intrusion detection systems are a special subset of host-based intrusion detection systems that analyze the events transpiring within a software application. The most common information sources used by application-based intrusion detection systems are the application’s transaction log files. The ability to interface with the application directly, with significant domain or application-specific knowledge included in the analysis engine, allows application-based intrusion detection systems to detect suspicious behavior due to authorized users exceeding their authorization. This is because such problems are more likely to appear in the interaction between the user, the data, and the application [2, 48].

INTRUSION DETECTION SYSTEM ANALYSIS

There are two primary approaches to analyzing events to detect attacks: misuse detection and anomaly detection. Misuse detection in which the analysis targets something known to be “bad”, is the technique used by most commercial systems. Anomaly detection, in which the analysis looks for abnormal patterns of activity, has been, and continues to be, the subject of a great deal of research. Anomaly detection is used in limited form by a number of intrusion detection systems. There are strengths and weaknesses associated with each approach, and it appears that the most effective intrusion detection systems use mostly misuse detection methods with a smattering of anomaly detection components [2, 48].

ANOMALY BASED DETECTION

Anomaly detection uses models of the intended behavior of users and applications, interpreting deviations from this “normal” behavior as a problem.

A basic assumption of anomaly detection is that attacks differ from normal behavior. For example, we can model certain users’ daily activity (type and amount) quite precisely. Suppose a particular user typically logs in around 10 Am., reads mail, performs database transactions, takes a break between noon and 1 Pm., has very few file access errors, and so on. If the system notices that this same user logs in at 3 Am., starts using compilers and debugging tools, and has numerous file access errors, it will flag this activity as suspicious.

The main advantage of anomaly detection systems is that they can detect previously unknown attacks. By defining what’s normal, they can identify any violation, whether it is part of the threat model or not. In actual systems, however, the advantage of detecting previously unknown attacks is paid for in terms of high false-positive rates. Anomaly detection systems are also difficult to train in highly dynamic environments [5].

MISUSE DETECTION

Misuse detection systems essentially define what’s wrong. They contain attack descriptions (or “signatures”) and match them against the audit data stream, looking for evidence of known attacks. One such attack, for example, would occur if someone created a symbolic link to a UNIX system’s password file and executed a privileged application that accesses the symbolic link. In this example, the attack exploits the lack of file access checks [5, 10].

The main advantage of misuse-based systems is that they usually produce very few false positives: attack description languages usually allow for modeling of attacks at such fine level of detail that only a few legitimate activities match an entry in the knowledge base.

However, this approach has drawbacks as well. First of all, populating the knowledge base is a difficult, resource intensive task. Furthermore, misuse based systems cannot detect previously unknown attacks, or, at most, they can detect only new variations of previously modeled attacks. Therefore, it is essential to keep the knowledge base up-to-date when new vulnerabilities and attack techniques are discovered. Figure 2 shows how the misuse detection based intrusion detection system works is [11].

RESPONSE OPTION FOR INTRUSION DETECTION SYSTEM

Once intrusion detection systems have obtained event information and analyzed it to find symptoms of attacks, they generate responses. Some of these responses involve reporting results and findings to a pre-specified location. Others involve more active automated responses. Though researchers are tempted to underrate the importance of good response functions in intrusion detection systems, they are actually very important. Commercial intrusion detection systems support a wide range of response options, often categorized as active responses, passive responses, or some mixture of the two [2].

IMPORTANCE OF THE INTRUTION DETECTION SYSTEM

Usually we place a burglar alarm on the doors and windows of our home. We are installing an intrusion detection system (intrusion detection system) for our house. The intrusion detection systems used to protect our computer network operate in similar fashion. An intrusion detection system is a software and possibly hardware that detects attacks against our network. They detect intrusive activities that enter into our network. We can locate intrusive activity by examining network traffic, host logs, system calls, and other areas that signal an attack against our network [14].

There are different benefits that an intrusion detection system provides. Besides detecting attacks, most intrusion detection systems also provide some type of response to the attacks, such as resetting TCP connections [14].

DESIRABLE CHARACTERSTICS OF INTRUSION DETECTION SYSTEM

There are different characteristics for an ideal intrusion detection system, which are listed below [many references]:

1. An ideal intrusion detection system must run with minimum human supervision.
2. An ideal intrusion detection system must be easy to deploy.
3. An ideal intrusion detection system must be able to detect attacks
• intrusion detection system must not produce false negative alarms.
• intrusion detection system must not produce false positive alarms.
• intrusion detection system must report intrusion as soon as possible after the attacks occur.
• intrusion detection system must be general enough to detect different types of attacks.
4. An ideal intrusion detection system must be fault tolerant; it must be able to recover from crashes and must restore previous state, either accidental or caused by malicious activities.
5. An ideal intrusion detection system must impose minimal overhead on the system.
6. An ideal intrusion detection system must be configurable to implement the securities policies of the system.

THE PERIMETER MODEL AND DoS

The perimeter model is an architecture commonly used by today’s organizations to protect critical infrastructures. This security model divides network architectures into two distinct groups; trusted and entrusted. The trusted group is often the finite internal infrastructure, whilst the entrusted group consists of infinite external networks. In this model two types of devices are used; firewall to control the traffic entering and leaving the trusted domain, and intrusion detection system to detect misbehavior of trust with in the trusted area boundary [18].

WHERE IDS SHOULD BE PLACED IN NETWORK TOPOLOGY

Depending upon network topology, the intrusion detection system can be positioned one or more places. It’s also depends upon what type of intrusion activities should be detected: internet external or both. For example if the external intrusion activities should be detected, and only one router is connected to the internet, the best place for an intrusion detection system may be just inside the router or firewall. If there are many different paths to the internet, then the intrusion detection system should be placed at every entry point. However, if the internal attacks should be detected then the intrusion detection system should be placed in every network segment 2. Placement of the intrusion detection system really depends upon security policies 3 [8].

1. Note that more intrusion detection systems mean more work and more maintenance costs.
2. Which defines that what should be protected from the hackers [8]?

IDS AGAINST DENIAL-OF-SERVICE ATTACKS (DoS)

The goal of a DoS attack is to disrupt some legitimate activity, such as browsing, web pages, an on line radio and many more. The denial of service is achieved by sending message to the target that interferes with its operation and makes it hang, crash, reboot or do useless work [16].

A denial-of-service attack is different in goal, form, and effect than most of the attacks that are launched at networks and computers. Most attackers involves in cyber crime seek to break into a system, extract its secrets, or fool it into providing a service that they should not be allowed to use. Attackers commonly try to steal credit card numbers or proprietary information, gain control of machines to install their software or save their data, deface Web pages, or alter important content on victim’s machines. Frequently, compromised machines are valued by attackers as resources that can be turned to whatever purpose they currently deem important [16].

NEED OF EARLY DETECTION OF DoS ATTACKS

The effectiveness of the DoS [18] attacks has been much reported in recent years, even though organizations continue to employ perimeter model security devices. Case such as cloud nine incident [53].

DoS attacks prevent a legal network user from performing his/her functions [54]. They overwhelm the victim host to the point of unresponsiveness to the legitimate user of that host [55]. As demonstrated by the CBI/FBI survey [56], these attacks are prevalent ‘in the wild’. With today’s reliance on networks and computing technologies, these attacks can have serious effect on the victim.

CONCLUSION

To conclude the literature there have been three main research areas identified with in the literature that appertain to the undertaking of this study. By the nature of the research the field of study of the whole project will be between performance and scalability issues with those much more concentrated on that what techniques and algorithms can be used for developing an intrusion detection system with these characteristics.

In addition the literature review has allowed what security policies should be implemented for securing an intrusion detection system itself from the attackers. And where there are the actual problems, especially in the area of alarm generation i.e. scalabilities issues.

As a result the literature review has greater focus on the three different research areas:

1. Scalability issues of intrusion detection system means how the false alarms can be reduce against the Denial-of-service attacks.
2. Throughput of the intrusion detection system means how the better performance can be obtained for better response against the attacks.
3. intrusion detection system itself is a target for attackers so that what security policies should be adopted for reducing the risk of attackers for intrusion detection system.

And the great focus of this study to develop an intrusion detection system by considering these three aspects.

CHAPTER 3

Although intelligent intrusion and detection strategies are used to detect any false alarms within the network critical segments of the network infrastructures, reducing false positives is still a major challenge. Up to this moment, these strategies focus on either detection or response features, but often lack of having both features together. Without considering these features together, intrusion detection systems will not be able to highly detect on low false rates [30].

This chapter describes the analysis part of the project which includes mechanism, algorithms and software development life cycle for developing an intrusion detection system with better performance and scalability. In this research an intrusion detection system will be developed with better performance and scalability by using the system engineering.

DATA TYPES FOR INTRUSION DETECTION SYSTEM

In 1998, under DARPA intrusion evolution program, an environment was set up to acquire raw TCP/IP dump data for a network for simulating a typical US air force LAN. The LAN was operated like a real environment, but was blasted with multiple attacks [31, 30]. For each TCP/IP connection, 41 various quantitative and qualitative features (See appendix A) were extracted [32, 30]. Of this database, a subset of 494021 data were used which compromised normal patterns. Attacks types were divided into the following 4 main categories [30]:

• PROBING It is class of attacks where an attacker scans a network to gather information in order to find known vulnerability. An attacker with a map of machines and services that are available on a network can manipulate the information to look for exploits. There are different types of probes: some of them abuse computer’s legitimate features; and some of them use social engineering techniques. This class of attacks is most common because it requires very little technical expertise.
• DENIAL OF SERVICE Denial Of Service is a class of attacks where an attacker make some computing or memory resource too busy or too full to handle legitimate requests, denying legitimate users access to a machine.
• USER TO ROOT In this attack, an attackers starts with access to a normal user account on the system by gaining root access.
• REMOTE TO USER This attack happens when an attacker sends packets to a machine over a network that exploits the machine’s vulnerability to gain local access as a user illegally.

SYSTEM

System engineering is concerned with all aspect of the development and evolution of complex systems where software plays a major role. System engineering is therefore concerned with hardware development, policy and process design and system deployment and as well as software engineering. System engineers also involved in specifying the system, defining it overall architecture and then integrating the different parts to create the finished system. They are less concern with the engineering of the system components (hardware, software etc.) [41].

A system is a purposeful collection of interrelated components that work together to achieve some objective.

Systems that include software fall into two categories [41]:

• TECHNICAL COMPUTER BASED SYSTEMS are systems that include hardware and software components but not procedures and processes. Examples of the technical system include televisions, mobile phone and most personal computer software.
• SOCIO-TECHNICAL SYSTEM includes one or more technical systems but, crucially, also include knowledge of how the system should be used to achieve some broader objective.

Essential characteristics of socio-technical systems are as follow [41].

1. They have the emergent properties that are the properties of the system as a whole rather than associated with individual parts of the system. Emergent properties depend on both the system components and the relationship between them. As this is so complex, the emergent properties can only be evaluated once the system has been assembled.
2. They are often nondeterministic. This means that, when presented with a specific input, they may not always produce the same output. The system’s behaviour depends upon the human operators, and people do not always react in the same way. Furthermore, use of the system may create new relationships between the system components and hence change its emergent behavior.
3. The extent to which the system supports organizational objectives does not just depend on the system itself. It also depends on the stability of these objectives, the relationships and conflicts between organizational objectives and how people in the organization interpret these objectives. New management may reinterpret the organizational objective that a system is designed to support and a successful system may become failure.

A characteristic of all systems is that the properties and the behavior of the system components are inextricably intermingled. The successful functioning of each system component depends on the functioning of some other components. Thus, the software can only operate if the processor is operational. The processor can only carry out computations if the software system defining these computations has been successfully installed [41].

Systems are usually hierarchal and so include other systems. These other systems are called sub-systems. A characteristic of sub-system is that they can operate as independent system in their own right. Therefore, the same geographical information system may be used in different systems [41].

EMERGENT SYSTEM PROPERTY

The complex relationships between the components in a system mean that the system is more than simply the sum of its parts. It has properties that are properties of the system as a whole. These emergent properties cannot be attributed to any specific part of the system. Rather, emerge only once the system components have been integrated. Some of these properties can be derived directly from the comparable properties of the sub systems. However, more often, they result from complex sub-system interrelationship that cannot, in practice, be derived from the properties of the individual system components [41].

There are two different types of emergent properties [41]:

FUNCTIONAL EMERGENT PROPERTIES appear when all the parts of the system work together to achieve some objective.

NON FUNCTIONAL EMERGENT PROPERTOES relate to the behavior of the system in its operational environment. Examples of non-functional properties are reliability, performance, safety and security. These are often critical for computer-based systems, as failure to achieve some minimal defined level in these properties may make the system unusable. Some users may not need some system functions so the system may be acceptable without them. However, a system that is unreliable or too slow is likely to be rejected by all its users.

SYSTEM RELIABILITY Reliability is a complex concept that must always be considered at the system level rather than the individual component level. The components in a system are independent, so failure in one component can be propagated through the system and affect the operation of the other components [41].

Like reliability, other emergent properties such as performance or usability are hard to assess but can be measured after the system is operational. Properties such as safety and security, however pose different problems. A secure system is one that does not allow unauthorized access to its data but it is clearly impossible to predict all possible modes of access and explicitly forbid them. Therefore, it may only be possible to assess these properties by default. That is, you only know that a system is insecure when someone breaks into it [41].

SYSTEM ENGINEERING

System engineering is the activity of specifying, designing, implementing, validating, deploying, and maintaining socio-technical systems. The phases of the system engineering process are shown in Figure 1. This process was an important influence on the waterfall model of the software process.

SYSTEM REQUIRENMENT DEFINITION

System requirements definitions specify what the system should do (its function) and its essential and desirable properties. As with software requirement analysis, creating system requirement definitions involves consultation with system customers and end-users. This requirement phase usually concentrates on deriving three types of requirement [41]:

An important part of the requirements definition phase is to establish a set of overall objectives that the system should meet. These should not necessarily be expressed in terms of system’s functionality but should define why the system being procured for a particular environment [41].

To illustrate that what this means, we are specifying a system for a company’s network to provide the protection against the attacks, worms and viruses. A statement of objective based on system functionality might be:

To provide an intrusion detection system for the network that will provide internal and external warning of unauthorized intrusion.

This objective states explicitly that there needs to be a detection system that provides warnings of undesired events. By contrast, a broader statement of objectives might be:

To ensure that the normal functioning of the work carried out over the network is not seriously disrupted by events such as virus, worm or unauthorized intrusion.

If we set out the objective like this, we can broaden and limit the design choices, this objective allows for intrusion detection using sophisticated pattern’s signatures. It may also exclude the signatures which can affect the working of overall network [41].

WICKED PROBLEM It is a problem that is so complex and where there are so many related entities that there is no definitive problem specification. The nature of the problem emerges only as solution is developed. For example no one can create the signature which can detect all the expected attacks [41]. We can just create the signatures base on previous attacks.

REQUIRENMENT ANALYSIS

Current countermeasures to DoS rely on the perimeter model of network security. However, this model, which relies on firewalls and intrusion detection systems (intrusion detection system), does not provide the defence required against DoS attacks as long as these devices are an internal part of the victim system. This is because they only respond to an attack, rather than prevent them from being successful. Consequently, when the attacks are detected the services are shut down [18].

HOW COMMON ARE DoS and DDoS ATTACKS

In February 2000, a series of massive denial-of-service (DoS) attacks disabled several high-visibility Internet e-commerce sites, including Yahoo, Ebay and many more. Then, In January 2001 Microsoft’s name server infrastructure was incapacitated by similar attacks. The root DNS server were beleaguered in 2002, over the last six years, denial-of- service attacks against highly visible sites or services have become commonplace. However, the vast majority of attacks is not publicized and includes wide range of global victims, from small commercial sites, to educational institutions, public chat servers and government organizations [17].

Using backscatter analysis, we have established the presence of roughly 2,000{3,000 active denial-of-service attacks per week. Over a three-year period we have collected 22 distinct traces, revealing 68,700 attacks on over 34,700 distinct Internet hosts belonging to more than 5,300 distinct organizations. We are also able to estimate a lower-bound on the intensity of such attacks | some of which are in excess of 100,000 packets-per-second (pps) | and characterize the nature of the sites victimized [17].

SURVEY OF RESEARCH DEFENCE APPROACHES

Denial of service targets the heart of today’s information economy, connectivity, by preventing the access to service to the legitimate users. This may be achieved in number of ways. However, 94 percent attacks utilize TCP to achieve their aim [20, 22].

A number of approaches have been proposed to counter the denial of service problem. These mechanism include payment for network resources [20, 23], strong authentication [20, 24], Pushback [20, 25], traffic identification [20, 26], D-WARD [16, 26, 28], and NETBOUNCER [16, 27]. However, issues such as their inability to scale differentiate malicious from benign traffic with little overhead, requirement for state full information, or deal with little overhead, requirement for state full information, or deal with high- volume flows has ensured that these approaches have not achieved widespread development. There for new approach is required [20].

WEAKNESSES OF SIGNATURE BASED IDS

The role of the intrusion detection system starts when any organization deploy intrusion detection system, it must monitor the system and respond to the alerts that it reports. Deployment issues to address include placement of sensors to maximize protection for the most critical assets, configuring the intrusion detection system to reflect security policy, installing appropriate signatures and other initial conditions, establishing forensic procedures to preserve evidence for possible prosecutions, and determining when (if ever) and what automatic responses are allowed. Users must develop procedures for handling intrusion detection system alerts and consider how to correlate alerts with other information such as system or application logs [3].

intrusion detection systems themselves are logical targets for attack. Smart intruders who realize that an intrusion detection system has been deployed on a network they are attacking will likely attack the intrusion detection system first, disabling it or forcing it to provide false information [3].

Although signature based intrusion detection system provides various benefits but there are some drawbacks as well. Some of them are listed below:

• Updating signature database
• False negative
• False positive
• Inability to detect unknown attacks
• Maintaining the state information (Event horizon 4)

4 To detect an attack, a signature based intrusion detection system examines the data presented to it, sometime many pieces of data are necessary to match an attack signature. The maximum amount of time over which an attack signature can be successfully detected is known as the event horizon.

The biggest drawback of a signature based intrusion detection system is its inability to detect previously unpublished attacks. Signature based intrusion detection system detects the attack on the basis of previous attack signatures. So it is very hard to detect a new attack by using the old signatures .

FALSE POSITIVE

intrusion detection system generate alarm to signal when attacks are occurring on the network. When an attack is generating because of the normal behavior, the alarm is known as false positive.

FALSE NEGATIVE

When an intrusion detection system fails to generate an alarm for known intrusive activity, it is called a false negative. False negative represent an actual attacks that the intrusion detection system missed even though it is programmed to detect the attack.

WHY BETTERSCALABILITY AND PERFORMANCE REQUIRED

Packet classification is important function in network security appliances such as firewalls and intrusion detection system. Signature based intrusion detection system use the deep packet inspection in that different multi pattern algorithms are used. These packet matching algorithms check whether the packet payload or flow content contains a specified signature in the signature set [36].

Rapid expansion of the network traffic has increased the significance of the NIDS performance. Most of the intrusion detection system relies on exact string matching from network packet payloads against thousand of intrusion signatures. The performance of signature based intrusion detection system has been shown to be conquered by the speed of string matching algorithms used to judge packet against signatures. A NIDS must utilize an efficient string matching algorithm because an underperforming passive system drops many packets and may miss many attacks, while an underperforming inline system creates a bottleneck for network performance [57].

The quality of intrusion detection system is described by the percentage of true attacks detected combined with the number of false alerts. However, even a high quality pattern matching algorithm is not effective if its processing cost is too high, since the resulting loss of packets increases the probability that an attack is not detected [58].

Usually, the performance of an intrusion detection system is characterized by the probability that an attack is detected in amalgamation with the number of false alerts. Though, uniformly important is the system’s capacity to process traffic at the maximum rate offered by the network with minimal packet loss. Significant packet loss can leave a number of attacks undetected and reduce the overall efficiency of the system. A higher performance sensor is not only able to process packets at a higher rate, but can also apply more complicated detection techniques to reduce the number of false alert [58].

Multi-pattern matching is known to require exhaustive memory access and is often a performance bottle neck [59].

SECURITY CONSIDRATION WITH INTRUSION DETECTION SYSTEM

Even though, a network in that a intrusion detection system has been used to improve the security, making sure that intrusion detection system is as secure as possible will make the data more trustworthy. If someone breaks into the intrusion detection system, there is no reason to trust the alerts that it sends, thereby making the system completely useless [41].

Because that intrusion detection system requires a operating system. With that said, an intrusion detection system installation is subject to attacks, both in intrusion detection system itself and in the underlying operating system. Why? Even if we want to get in remotely (SSH), and we will probably want to store the alerts in a database like MySQL. And we will probably want to view the alerts with a dapper interface that might require a web server. Any listening service is possible surfaces for attacks, and some driver attacks can even target a listening interface that is not advertising any services in particular at all. This makes our intrusion detection system just like any other application [41].

Frequently an operating system creates a single process that has at least one thread with which an application runs. A number of operating systems permit and support the capability for a single process to be composed of multiple threads. This is significant because sometimes a single process needs to do numerous things at the same time (parallel) [41].

Threads can be consideration of as individual processes with special attribute that make them more resourceful for today’s more complex applications. The unique attributes threads contain are shared process address space, global variables, registers, stack, state and other process type information. In addition to sharing all of these resources, threads also preserve their own separate data as well. For instance, individual threads deal with their own registers, stack and state [41].

The main point is that threads are becoming ordinary and the majority of software applications today are being keenly written to a threaded model. In addition, multiprocessor systems are becoming somewhat commonplace throughout the home, corporate world and the data centers by way of inexpensive and dominant new technologies and architectures, such as dual-core processors that offer a substantial increase in performance and a good return on the investment [41].

So, the method in which the operating system interacts with the CPUs may be an area where you can realize performance gains. Even though some application cannot take clear advantage of multiple processors, there are ways in which we can “help” these applications to exploit their use, provided artistic gene is up for it [41]!

WHY BETTER SECURITY POLICIES REQUIRED

A with any other system that is planned to attach to a network, it is crucial to consider the security of the network’s intrusion detection system. Can it be hacked? Will it need to be patched? What known attacks are available and are being used against a intrusion detection system? What type of threat could it pose to the network if an attacker managed to compromise the system? These are the important questions for any system, but they are doubly important if one of the most important security devices is considered [60] .

When designing the security policies for any network, it is always wise to take a defense in depth approach. Of course, all the network systems should be protected to the best of network administrator ability. However, it’s also wise to plan so that not even a single point of failure exists. All the systems of the network should be susceptible to attack. A robust plan of defense will consider the security of each individual system, including the intrusion detection system of a network, and it should be definite that no one machine would be a single point of failure [60].

There are two classes of attacks against intrusion detection system. The first is designed to make an intrusion detection system ineffective. Programs like stick and snot. They can be used to attempt to overwhelm intrusion detection system with noisy garbage alerts, perhaps distracting an analyzer from the real attack hidden somewhere in all that junk. Denial-of-service attacks against intrusion detection system, such as the ICMP header size DoS [61] attack took place against snort 5. The second category of attacks is designed to use intrusion detection system an exploitable network service, aiming to execute code or gain privilege on the intrusion detection system itself [60].

5Snort is an open source network intrusion prevention and detection system (IDS/IPS) developed by Sourcefire. Combining the benefits of signature, protocol and anomaly-based inspection, Snort is the most widely deployed IDS/IPS technology worldwide. With millions of downloads and over 250,000 registered users, Snort has become the de facto standard for IPS

The main purpose of the project is development of an intrusion detection system with better performance and scalability against the Denial-Of-Service attacks. For achieving that purpose different mechanism and algorithms can be used. And we can also divide the signature into different parts for reducing the false positive and false negative.

SYSTEM DESIGN

System design (Figure 2) is concerned with how the system functionality is to be provided by the components of the system. The activities involved in the process are [41]:

• PARTIONING REQUIREMENT We analyze the requirements and organize them into related groups. There are usually several possible partitioning options and we may suggest a number of alternatives at this stage of process.

  intrusion detection system for DoS attack requires partitioning because there are several functions that has to be done with in itself.

  First of all the network packets from the wire should be captured, for that a sniffer is required. That can sniff the network packets and can forward it for further processing.

  At the present time there are several standard present over the network for data link layer means like FDDI, Token ring, Ethernet and many more. So it is mandatory that intrusion detection system should detect the attack on any kind of interface so a system is required which can convert every type any kind of data in to a particular standard.

  After getting the data in particular standard it is required that it should be controlled by using some congestion control algorithms. So that another sub-system is required that can control the traffic and forward it for further processing.

  Later than a system is required which can detect the attacks within the controlled traffic and can generate the alarms.

  After that these generated alarms should be saved somewhere where they can be utilized for further processing.

• IDENTIFY SUB-SYSTEM Sub-systems that can individually or collectively meet the requirements. Group of requirements are usually related to Sub-systems, so this activity and requirements partitioning may be mingled.

  As conclude the partitioning requirement there are five sub systems required for developing an intrusion detection system.

  1. Libpcap
  2. Decoder
  3. Traffic control system
  4. Detection engine
  5. Output generating system
  6. ASSIGN REQUIREMENTS TO THE SUB-SYSTEMS In practice; there is never a clean match between requirements partitions and identified sub-systems.
  7. SPECIFY THE SUB-SYSTEM FUNCTINALITY Specification of the specific function provided by each sub system should be cleared. This may be seen as part of the system design phase or, if the subsystem is a software system, part of requirements specification activity for that system.
  8. DEFINE SUB-SYSTEM INTERFACE We should define the interface that are provided and required by each sub-system. Once these interfaces have been agreed upon it becomes possible to develop these sub-systems in parallel.

  Libpcap will capture the traffic from the wire and forward it to the decoder then decoder will convert that data into a particular standard and forward it to the traffic control system. Traffic control system will use the congestion control algorithm for controlling the traffic. And it will divide the traffic into two different parts first will be the normal traffic means if there is no congestion then the traffic control system will forward the traffic normally otherwise it will drop the traffic. In both cases it will forward the traffic to the detection engine. After getting the data from the traffic control system detection engine will match this data against the pre-defined rules. By using the multi pattern matching algorithms. If there is any match then it will generate the alarms and forward it to the output plug-ins. Then the output plug-ins will save the data for data analyst or network administrator.

  SYSTEM MODELLING

  During the system requirements and design activities, system may be modeled as a set of components and relationship between these components. These are normally illustrated graphically in a system architecture model that gives the reader an overview of the system organization [41].

  For example figure 3 shows the decomposition of a reliable intrusion detection system for DoS attacks into its principle components.

  Figure 3 IDS for DoS attacks data flow

  • LIBPCAP The Libpcap can be used to read record, inject and in general deal with network packets at a higher level than raw sockets. Essentially Libpcap can be used to easily collect up or manipulate the packets. Libpcap function also abstract a lot of the difference between operating systems network API making programs that leverage Libpcap generally more portable or perhaps saving the programmer headache of writing their own network API layer [42].
  • DECODER The packet decoder takes packets from different types of network interfaces and prepares the packets to pass through congestion control part or to be sent to the detection engine. The interface may be Ethernet, SLIP, and PPP [38].
  • TRAFFIC CONTROL SYSTEM This system will be used for controlling the congestion of the network by using the different congestion control algorithms like RED and DROPTAIL etc.
  • DETECTION ENGINE Its responsibility is to detect if any intrusion activity exists in a packet. The detection engine matches the packet with already saved signatures and employs the rules for this purpose [38].
  • OUTPUT PLUG-IN The purpose of output plug-in is to dump altering data to another resource of file. Multiple outputting plug-in is to dump altering data to another resource or file [43].

  At this level of detail, the system is decomposed into a set of interacting sub-systems. Each sub-system should be represented in a similar way until the system is decomposed into functional components. Functional components are the components that, when viewed from the perspective of the subsystem, provide a single function. By contrast, a sub system usually is multi functional. Of course when viewed from another perspective, a functional component may itself be a system in its own right [41].

  SUB-SYSTEM DEVELOPMENT During the sub-system development, the subsystem identified during system design is implemented. This may involve starting another system engineering process for individual sub-system or, if the sub-system is software, software process involving requirements, design implementation and testing.

  FEEDING DATA WITH LIBPCAP

  Some prior labor is required to send the packets into the congestion control system and the detection engine. So we can use the LIBPCAP as packet capture and is a platform independent facility. It can be run on every popular combination of hardware and OS. We can utilize the Libpcap library to grab the packets off