Analysis of Intrusion Detection Systems (IDS)

Published Date: 28 Feb 2018

Introduction

Intrusion detection systems (IDS) were developed in 1990’s, when the network hackers and worms appeared, initially for the identification and reporting of such attacks. The intrusion detection systems didn’t have the ability to stop such attacks rather than detecting and reporting to the network personnel.

The Intrusion Prevention Systems got both characteristics i.e. threat detection and prevention. The detection process analyzes the events for any possible threats while the intrusion prevention stops the detected possible threats and reports the network administrator.

Purpose & Scope

The main purpose of the project is to evaluate the security capabilities of different types of IDPS technologies in maintaining the network security. It provides detail information about the different classes & components of IDPS technologies, for example, detection methods, security capabilities, prevention capabilities & internals of IDPS. It is mainly focused on different detection techniques & responses by these technologies.

1.2 Audience

The information can be useful for computer network administrators, network security personnel, who have little knowledge about these IDPS technologies.

1.3 Project Structure

The project is organized into the following major structure:

• Section 2 provides a general introduction of IDPS.

• Section 3 provides detail information about of IDPS technologies, components & architecture, detection methodologies, security capabilities & prevention capabilities.

• Section 4 provides the internals of IDPS & incident response.

Section 2: Introduction of IDPS

This Chapter Explains the Intrusion Detection & Prevention Process, Uses, Functions and Different Types of IDPS

The modern computer networks provide fast, reliable and critical information not only to small group of people but also to ever expanding group of users. This need led the development of redundant links, note book computers, wireless networks and many others. On one side, the development of these new technologies increased the importance and value of these access services and on other side they provide more paths to attacks.

During the past, In the presence of firewalls and anti-virus software, organizations suffered huge losses in minutes to their businesses in terms of their confidentiality and availability to the legitimate clients. These modern threats highlighted the need for more advance protection systems. Intrusion detection & prevention systems are designed to protect the systems and networks from any unauthorized access and damage.

An intrusion is an active sequence of related events that deliberately try to cause harm, such as rendering system unusable, accessing unauthorized information or manipulating such information. In computer terminology, Intrusion detection is the process of monitoring the events in a computer network or a host resource and analyzing them for signs of possible incidents, deliberately or incidentally. The primary functions of IDPS are the identification of incident, logging information about them, stopping them & preventing them from causing any damage. The security capabilities of IDPS can be divided into three main categories:

• Detection : Identification of malicious attacks on network & host systems

• Prevention: stopping of attack from executing

• Reaction: Immunization of the system from future attacks.

On the basis of location and type of events they monitor, there are two types IDPS technologies, host-based & network based. The network-based IDPS monitors traffic for particular network segment and analyze the network & application protocol activity for suspicious events. It is commonly deployed at the borders between networks. While on the other hand, host-based IDPS monitors the activity of a single host and events occurring within that host for suspicious activity.

There are two complementary approaches in detecting intrusions, knowledge-based approach and behavior based approach. In knowledge-based approach an IDPS looks for specific traffic patterns called Signatures, which indicates the malicious or suspicious content while in the behavior-based approach an intrusion can be detected by observing a deviation from normal or unexpected behavior of the user or the system.

What is an IDS?

The Intrusion Detection Systems (IDS) can be defined as: tools, methods & resources to identify, assess & report unauthorized or unapproved network activity.

It is the ability to detect attacks against a network or host and sending logs to management console providing the information about malicious attacks on the network and host resources. IDSs fall into two main categories:

• Host-Based Intrusion Detection System (HIDS): A HIDS system require some software that resides on the system and can scan all host resources for activity. It will log any activities it discovers to a secure database and check to see whether the events match any malicious event record listed in the knowledge base.

• Network-Based Intrusion Detection Systems (NIDS): A NIDS system is usually inline on the network and it analyzes network packets looking for attacks. A NIDS receives all packets on a particular network segment via one of several methods, such as taps or port mirroring. It carefully reconstructs the streams of traffic to analyze them for patterns of malicious behavior.

The basic process for IDS is that it passively collects data and preprocesses and classifies them. Statistical analysis can be done to determine whether the information falls outside normal activity, and if so, it is then matched against a knowledge base. If a match is found, an alert is sent. Figure 1-1 outlines this activity.

Response

Manager

• GUI
• Host System
• Pre-processing
• Statistical Analysis
• Alert Manager
• Knowledge
• Base
• Long-Term Storage
• Signature
• Matching

Fig 1.1 Standard IDS System

What is an IPS?

IPS technology has all capabilities of an intrusion detection system and can also attempt to stop possible incidents. IPS technologies can be differentiated from the IDS by one characteristic, the prevention capability. Once a threat is detected, it prevents the threat from succeeding. IPS can be a host-based (HIPS), which work best at protecting applications, or a network-based IPS (NIPS) which sits inline, stops and prevents the attack.

A typical IPS performs the following actions upon the detection of an attack:

• IPS terminates the network connection or user session.

• It blocks access to target .i.e. IP address, user account or sever.

• It reconfigures the devices i.e. firewall, switch or router.

• It replace the malicious portion of an attack to make it benign

An IPS typically consists of four main components:

• Traffic Normalizer: Interpret the network traffic and do packet analysis and packet reassembly & traffic is fed into the detection engine & service scanner.

• Service Scanner: Builds a reference table that classifies the information & helps the traffic shaper manage the flow of the information.

• Detection Engine: Detection engine does pattern matching against the reference table.

Figure 1.2 outlines this process:

• Response
• Manager
• GUI
• Traffic Normalizer
• System Scanner
• Detection Engine
• Alert Manager
• Reference Table
• Long-Term Storage
• Signature
• Matching

FIG 1-2 Standard IPS

Uses of IDPS Technologies

The identification of possible incidents is the main focus of an IDPS, for example, if an intruder has successfully compromised a system by exploiting the vulnerability in the system, the IDPS could report this to the security personnel. Logging of information is another important function of IDPS. This information is vital for security people for further investigation of attack. IDPS has also the ability to identify the violation of security policy of an organization which could be intentionally or unintentionally, for example, an unauthorized access to a host or application.

Identification of reconnaissance activity is one of the major capabilities of IDPS, which is the indication of an imminent attack, for example, scanning of hosts and ports for launching further attacks. In this case, an IDPS can either block the reconnaissance activity or it can alter the configurations of other network devices

Functions of IDPS Technologies

The main difference between different types of IDPS technologies is the type of events they can recognize. Following are some main functions;

• Recording of information regarding observed events, this information could be stored locally or could be sent to the logging server.

• Sending of alerts is one of the vital functions of IDPS. Alerts are sent through different methods i.e. email, SNMP traps, syslog messages etc.

• In case of detection of a new threat, some IDPS do have the ability to change their security profile, for example, when a new threat is detected, it might be able to collect more detail information about the threat.

IDPS not only performs detection but it also performs prevention by stopping the threat to succeed. Following are some prevention capabilities:

• It can stop the attack by terminating either network connection or user session, by blocking access to a target host.

• It could change the configuration of other network devices (firewalls, routers & switches) to block the attack or disrupt it.

• Some IDPS could change the contents of a malicious IP packet, for example, it can replace the header of an IP packet with a new one.

Types of IDPS Technologies

IDPS technologies can be divided into following two major categories:

• Network-Based IDPS

• Host-Based IDPS

Network-Based IDPS

Network-based IDPS monitors network traffic for a particular network segment. They analyze the network and application protocol activity to identify any suspicious activity.

A network based IDPS is usually sits inline on the network and it analyzes network packets looking for attacks. It receives all packets on a particular network segment, including switched networks. It carefully reconstructs the streams of traffic to analyze them for patterns of malicious behavior. They are equipped with facilities to log their activities and report or alarm on questionable events. Main strengths of network-based IDPS are:

• Packet Analysis: Network-based IDPSs perform packet analysis. They examine headers of all IP packets for malicious contents. This helps in detection of the common denial of service (DOS) attack. For example, LAND attack, in which both the source & destination addresses and source & destination ports are same as of the target machine. This cause the target machine to open connection with itself, causing the target machine either performs slowly or crash. It can also investigate the payload of an IP packet for specific commands.

• Real Time Detection & Response: Network-based IDPS detects attacks in real time as they are occurring in the real time and provides faster response. For example, if a hacker initiated a TCP based DoS attack, IDPS can drop the connection by sending a TCP reset.

• Malicious Content Detection: Network-based IDPS remove & replaces suspicious portion of the attack. For example, if an email has infected attachment, an IDPS removes the infected file and permits the clean email.

• Evidence for Prosecution: Network-based IDPS monitors real time traffic and if an attack is detected and captured the hacker cannot remove the evidence. Because the captured attack has data in it but also the information about his or her identification which helps in the prosecution.

Host-Based IDPS

A Host-Based system monitors the characteristics of a single host and the events occurring within that host for suspicious activity. It require some software that resides on the system and monitors the network traffic, syslog, processes, file access & modification and configuration or system changes. It logs any activities it discovers to a secure database and check to see whether the events match any malicious event record listed in the knowledge base. Some of the major strengths of Host-Based IDPS are as under:

• Verification of Attack: Host-based IDPS uses logs which contains events that have actually occurred. It has the advantage of knowing if the attack is successful or not. This type of detection is more accurate and generates fewer false alarms.

• Monitoring of Important Components: Host-Based IDPS monitors key components for example, executables files, specific DDLs and NT registry. All of these can cause damage to the host or network.

• System Specific Activity: Host-based IDPS monitors user and file access activity. It monitors the logoff or login procedure and monitors it on the basis of current policy. It also monitors the file access for example, opening of a non shared file.

• Switched & Encrypted Environments: Host-Based IDPSs provide greater visibility into purely switched environment by residing on as many critical hosts as needed. Encryption is a challenging problem for network-based IDPS but not a major problem for host-based IDPS. If the host in question has log-based analysis the encryption will have no impact on what goes in to the log files.

• Near Real Time Detection: A host-based IDPS relies on the log analysis which is not a true real time analysis. But it can detect & respond as soon as the log is written to and compared to the active attack signatures.

• Real Time Detection & Response: Stack-based IDPS monitors the packets as they transverse the TCP/IP stack. It examines inbound & outbound packets and examines in real time if an attack is being executed. If it detects an attack in real the time then it can responds to that attack in the real time.

Section 2: IDPS Analysis Schemes

IDPSs Perform Analysis: This Chapter is about the Analysis Process- What Analysis does and Different Phases of Analysis.

2.2 Analysis

In the context of intrusion detection & prevention, analysis is the organization of the constituent parts of data and their relationships to identify any anomalous activity of interest. Real time analysis is analysis done on the fly as the data travels the path to the network or host. The fundamental goal of intrusion-detection & prevention analysis is to improve an information system’s security.

This goal can be further broken down:

• Create records of relevant activity for follow-up.

• Determine flaws in the network by detecting specific activities.

• Record unauthorized activity for use in forensics or criminal prosecution of intrusion attacks.

• Act as a deterrent to malicious activity.

• Increase accountability by linking activities of one individual across system.

2.3 Anatomy of Intrusion Analysis

There are many possible analysis schemes but in order to understand them, the intrusion process can be broken down into following four phases:

• Preprocessing

• Analysis

• Response

• Refinement

1. Pre-Processing

Preprocessing is the key function once the data is collected from IDPS sensor. The data is organized in some fashion for classification. The preprocessing helps in determining the format the data are put into, which is usually some canonical format or could be a structured database. Once the data are formatted, they are broken down further into classifications.

These classifications can depend on the analysis schemes being used. For example, if rule-based detection is being used, the classification will involve rules and patterns descriptors. If anomaly detection is used, then statistical profile based on different algorithms in which the user behavior is baseline over the time and any behavior that falls outside of that classification is flagged as an anomaly.

Upon completion of the classification process, the data is concatenated and put into a defined version or detection template of some object by replacing variables with values. These detection templates populate the knowledgebase which are stored in the core analysis engine.

2. Analysis

Once the processing is completed, the analysis stage begins. The data record is compared to the knowledge base, and the data record will either be logged as an intrusion event or it will be dropped. Then the next data record is analyzed. The next phase is response.

3. Response

Once information is logged as an intrusion, a response is initiated. The inline sensor can provide real time prevention through an automated response. Response is specific to the nature of the intrusion or the different analysis schemes used. The response can be set to be automatically performed or it can be done manually after someone has manually analyzed the situation.

4. Refinement

The final phase is the refinement stage. This is where the fine tuning of the system is done, based on the previous usage and detected intrusions. This gives the opportunity to reduce false-positive levels and to have a more accurate security tool.

Analysis Process By Different Detection Methods

The intrusion analysis process is solely depends on the detection method being used. Following is the information regarding the four phases of intrusion analysis by different detection methods:

Analysis Process By Rule-Based Detection

Rule-based detection, also known as signature detection, pattern matching and misuse detection. Rule-based detection uses pattern matching to detect known attack patterns. The four phases of intrusion analysis process applied in rule-based detection system are as under:

• Preprocessing: The data is collected about the intrusions, vulnerabilities and attacks and then it is putted down into classification scheme or pattern descriptors. From the classification scheme a behavior model is built and then into a common format;

• Signature Name: The given name of the signature

• Signature ID: The unique ID for the signature

• Signature Description: The description of the signature & what it does

• Possible False Positive Description: An explanation of any “false positives” that may appear to be an exploit but are actually normal network activity.

• Related Vulnerability Information: This field has any related vulnerability information

The pattern descriptors are typically either content-based signatures, which examine the payload and header of packet, or context-based signatures that evaluate only the packet headers to identify an alert. The pattern descriptors can be atomic (single) or composite (multiple) descriptors. Atomic descriptor requires only one packet to be inspected to identify an alert, while composite descriptor requires multiple packets to be inspected to identify an alert. The pattern descriptors are then put into a knowledge base that contains the criteria for analysis.

• Analysis: The event data are formatted and compared against the knowledge base by using pattern-matching analysis engine. The analysis engine looks for defined patterns that are known as attacks.

• Response: If the event matches the pattern of an attack, the analysis engine sends an alert. If the event is partial match, the next event is examined. Partial matches can only be analyzed with a stateful detector, which has the ability to maintain state, as many IDS systems do. Different responses can be returned depending on the specific event records.

• Refinement: Refinement of pattern-matching analysis comes down to updating signatures, because an IDS is only as good as its signature update.

Analysis Process By Profile-Based Detection (Anomaly Detection)

An anomaly is something that is different from the norm or that cannot be easily classified. Anomaly detection, also referred to as Profile-based detection, creates a profile system that flags any events that strays from a normal pattern and passes this information on to output routines. The analysis process by profile-based detection is as following:

• Preprocessing: The first step in the analysis process is collecting the data in which behavior considered normal on the network is baselined over a period of time. The data are put into a numeric form and then formatted. Then the information is classified into a statistical profile that is based on different algorithms is the knowledge base.

• Analysis: The event data are typically reduced to a profile vector, which is then compared to the knowledge base. The contents of the profile vector are compared to a historical record for that particular user, and any data that fall outside of the baseline of normal activity is labeled as deviation.

• Response: At this point, a response can be triggered either automatically or manually.

• Refinement: The profile vector history is typically deleted after a specific time. In addition, different weighting systems can be used to add more weight to recent behavior than past behaviors.

Section 3: IDPS Technologies

This section provides an overview of different technologies. It covers the major components, architecture, detection methodologies & security capabilities of IDPS.

Components

Following are the major components and architecture of IDPS;

Sensor & Agents: Sensors & Agents monitors and analyze the network traffic for malicious traffic.

Sensor:The technologies that use sensors are network based intrusion detection & prevention systems, wireless based intrusion detection & prevention systems and network behavior analysis systems.

Agents: The term “Agent” is used for Host-Based Intrusion detection & prevention technologies.

• Database Server: The information recorded by the sensors and agents are kept safely in a database server.

• Console: A console is software that provides an interface for the IDPS users. Console software is installed on the administrator’s PC. Consoles are used for configuring, monitoring, updating and analyzing the sensors or agents.

• Management Server: It is a centralized device, receives information from sensors & agents and manages that information. Some management server can also perform analysis on the information provided by sensor & agents, for example correlation of events. Management server can be both appliance based or software based.

3.1 Network architecture

IDPS components are usually connected with each other through organization’s network or through Management network. If they are connected through management network, each agent or sensor has additional interface known as management Interface that connects it to the management network. IDPS cannot pass any traffic between management interface and its network interface for security reasons. The components of an IDPS i.e. consoles and database servers are attached only with the Management network. The main advantage of this type of architecture is to hide its existence from hackers & intruders and ensure it has enough bandwidth to function under DoS attacks

Another way to conceal the information & communication is to create a separate VLAN for its communication with the management. This type of architecture doesn’t provide a much protection as the management network does.

3.2 Security capabilities

IDPS provide different security capabilities. Common security capabilities are information gathering, logging, detection and prevention.

3.2.1 Information gathering

Some IDPS gather general characteristics of a network, for example, information of hosts and network. They identify the hosts, operating system and application they use, from observed activity.

3.2.2 Logging capabilities

When a malicious activity is detected by the IDPS, it performs logging. Logs contain date & time, event type, rating and prevention action if performed. This data is helpful in investigating the incident. Some network-based IDPS captures packet while host-based IDPS records user ID. IDPS technologies allow log to be store locally and send copies of centralized logging server i.e. syslog.

3.2.3 Detection capabilities

The main responsibility of an IDPS is to detect malicious activity. Most IDPS uses combination of detection techniques. The accuracy and types of events they detect greatly depends on the type of IDPS. IDPS gives great results once they are properly tuned. Tuning gives more accuracy, detection and prevention. Following are some the tuning capabilities:

• Thresholds: It is a value that sets the limit for normal and abnormal behavior. For example, the number of maximum login attempts. If the attempts exceed the limit then it is considered to be anomalous.

• Blacklists & Whitelists: A blacklist is list which contains TCP or UDP port numbers, users, applications, files extensions etc that is associated with malicious activity. A whitelist is a list of discrete entities that are known to be benign. Mainly used to reduce false positive.

• Alert Setting: It enables IDPS to suppress alerts if an attacker generates too much alerts in a short time and blocking all future traffic from that host. Suppressing of alerts provide IDPS from being overwhelmed.

3.2.4 Prevention Capabilities

IDPS offers multiple prevention capabilities. The prevention capability can be configured for each type of alert. Depending on the type of IDPS, some IDPS sensors are more intelligent. They have learning & simulation mode which enables them to know when an action should be performed-reducing the risk of blocking benign activity.

3.2.5 Types of Alarms

When IDPS detects an intrusion it generates some types of alarms but no IDPS generates 100% true alarm. An IDPS can generate alarm for legitimate activity and can be failed to alarm when an actual attack occurs. These alarms can be categorized as:

• False Alarms: When an IDPS fails to accurately indicate what is actually happening in the network, it generates false alarms. False alarm fall into two main categories:

• False Positives: These are the most common type of alarms. False positive occurs when an IDPS generates alarm based on normal network activity.

• False Negatives: When an IDPS fails to generate an alarm for intrusion, it is called false negative. It happens when IDPS is programmed to detect ck but the attack went undetected.

2. True Alarms: When an IDPS accurately indicates what is actually happening in the network, it generates true alarms. True alarms fall into two main categories:

• True Positives: When an IDPS detects an intrusion and sends alarm correctly in response to actually detecting the attack in the traffic. True positive is opposite of false negative.

• True Negative: It represents a situation in which an IDPS signature does not send alarm when it is examining normal user traffic. This is the correct behavior.

• ARCHITECTURE DESIGHN

• Architecture design is of vital importance for the proper implementation of an IDPS. The considerations include the following:

• The location of sensors or agents.

• The reliability of the solutions & the measurements to achieve that reliability. For example using of multiple sensors, for monitoring the same activity, as a backup.

• The number & location of other components of IDPS for usability, redundancy and load balancing.

• The systems with which IDPS needs interfacing, including:

• System to which it provides the data i.e. log servers, management softwares.

• System to which it initiates the prevention responses i.e. routers, firewalls or switches.

• The systems used to manage the IDPS components i.e. network management software.

• The protection of IDPS communications on the standard network.

3.3 Maintenance & Operation

Mostly IDPS are operated & maintained by user graphic interface called Console. It allows administrator to configure and update the sensors and servers as well as monitor their status. Console also allows users to monitor and analyze IDPS data and generate reports. Separate accounts could be setup for administrators and users.

Command Line Interface (CLI) is also used by some IDPS products. CLI is used for local administration but it can be used for remote access through encrypted tunnel.

3.3.1 Common Use of Consoles

Many consoles offer drill down facilities for example, if an IDPS generates an alert, it gives more detail information in layers. It also give extensive information to the user i.e. packet captures and related alerts.

Reporting is an important function of console. User can configured the console to send reports at set time. Reports can be transferred or emailed to appropriate user or host. Users can obtain and customized reports according to their needs.

3.3.2 Acquiring & applying updates

There are two types of updates –software updates and signature updates. Software updates for enhancing the performance or functionality and fixing the bugs in IDPS while the signature updates for adding detection capabilities or refining existing capabilities.

Software updates are not limited for any special component but it could include all or one of them i.e. sensor, console, server and agents. Mostly updates are available from the vendor’s web site.

New Chapter

Detection Methodologies

Most IDPS uses multiple detection methodologies for broad & accurate detection of threats but following are primary detection methodologies:

• Signature Based Detection

• Anomaly Based Detection

• Stateful Protocol Analysis

3.3.1 Signature Based Detection

The term Signature refers to the pattern that corresponds to a known threat. In signature based detection, the predefined signatures, stored in a database, are compared with the network traffic for series of bytes or packet sequence known to be malicious, for example, an email with the subject of free screen savers and an attachment of screensavers.exe, which are characteristics of known form of malware Or a telnet log attempt with a false username is the violation of an organization’s security policy.

A signature is a string that is part of what an attacking host sends to an intended victim host that uniquely identifies a particular attack. Input strings are passed through on to detection routines match a pattern in the IDPS’s signature files. This method is the simplest detection method because the current unit of an activity, which could be either a packet or a log entry, is compared with the predefined list of signatures using a string comparison.

This is very effective for detecting known threats but is ineffective in detecting unknown threats. Signature based technologies have very less understanding of network and application protocols and because of this they are ineffective for handling complex data communication. For example, in the above case if the attacker changes the name of the file to scrrensaver2.exe instead of screensaver.exe, the signature based technology , using simple string comparison, could not detect it be a malware. So for this reason, whenever a new threat is detected (by other means) a new signature has to be created to stop such attacks in future.

3.3.2 Anomaly-Based Detection

Anomaly-Based detection is based on the behavior of an event in a system or in a network. It is the comparison of observed events, over a period of time which is considered to be normal, against the events of significant deviations. When some unusual behavior occurs, which could be in events, state or content, it triggers an alarm. This methodology uses Profiles. The profiles are built over a period of time by monitoring the characteristics of a typical activity, for example, a profile for a network might show that Web activity comprises an average of 20% of network bandwidth any more bandwidth usage will be considered as anomaly.

Initial profile is generated over a period of time called a training period. Profiles set the baseline for the normal accepted behavior of the network. Any activity which deviates from this baseline would be considered as anomalous, for example, the normal port used by HTTP is 80 but if it uses other non standard ports then it would be considered as abnormal or during the day. Profiles can either be dynamic or static.

• Dynamic Profiles: Dynamic profiles are developed by monitoring a typical activity over a period of time called training period.

• Static Profiles: Static profiles are configured manually.

It is difficult for the network administrator to manually configure the profiles because of the complexity of network traffic while the dynamic profile can adjust itself as new events are observed. With the passage of time, the networks & systems do change, in such a situation static profiles needed to be updated while the dynamic learn themselves and update the profiles accordingly but they are susceptible to invasion techniques for example, an attacker can perform small amount of malicious activity occasionally then slowly increasing the quantity and frequency of activity.

One key distinction between anomaly detection and other analysis schemes is that anomaly-based schemes not only define activities that are not allowed, but also activities that are allowed. In addition, anomaly detection is typically used for its ability to collect statistical behavior and characteristic behavior. Statistics are quantitative and characteristics are more qualitative. For example, ‘’a server’s UDP traffic never exceeds 25 percent of capacity’’, describes a statistical behavior, and a user ‘’X does not normally FTP files outside of the company’’ describes a characteristic behavior.

Anomaly-Based detection methods are quite effective for detecting previously unknown threats because it detects network traffic that is new or unusual, which is opposite to the anomaly-based detection.

3.3.3 Stateful Protocol Analysis

Stateful Protocol Analysis is another comparison technique, which depends on the pre-defined universal standards that specify how a particular protocol should behave. It compares predetermined profiles of generally accepted definitions of benign protocol activity for each protocol state against observed events to identify deviations. It looks for network protocol violations or misuse based on RFC-based behavior. Stateful protocol analysis relies on vendor-developed universal profiles that specify how particular protocols should and should not be used.

The “protocol analysis” performed by stateful protocol analysis methods includes reasonableness checks for individual commands, such as minimum and maximum lengths for arguments. If a command typically has a username argument, and usernames have a maximum length of 20 characters, then an argument with a length of 1000 characters is suspicious. If the large argument contains binary data, then it is even more suspicious.

Stateful protocol analysis performs the inspection of headers as well as the contents of an IP packet up to the application layer. The information obtained from various layers i.e. transport, session and network layer, the stateful protocol analysis decides whether the traffic is legitimate or not. It also examines the state of a connection & store information in a state table. The following looks at the interaction of stateful inspection in the context of an IDPS:

• Internet Protocols: Some applications create complex patterns of network traffic, for example, FTP and web traffic. In order to analyze the ‘’state’’ of these network connections, IDPS keeps a track in a central cache. When IDPS receives a packet, it is analyzed against the state table to assess whether or not to allow it to its destination. For example, FTP utilizes multiple simultaneous network connections. When a user opens a connection to a server on the internet and requests a file, the stateful protocol analysis searches for outgoing PORT commands and subsequently adds a cache entry for the anticipated data connection. Since the port command contains the address & port information, the connection would be identified.

• TCP Connections: A normal TCP connection follows a three-way handshake process to setup a connection. In the TCP initiation packet, the SYN flag is set and its ACK flag is cleared but the following packets do not have the same structure because they contain data. These subsequent packets are bi-directional & stateful inspection monitors these packets in order to make it sure that they are legitimate.

• UDP Connections: UDP is considered to be the unreliable protocol as the packets do not contain any connection information. The header contains only source, destination IP addresses, port numbers and message type. In case of UDP, an entry is created in the cache to buildup a virtual connection. The entry will contain the IP addresses and port numbers. So for a short period of time the IP packets coming from the same IP addresses and port numbers would be allowed.

When IDPS monitors the behavior of the application protocol, it performs decoding. While decoding it can detect the following anomalies:

• Running a protocol or service on non standard ports.

• Changes in the field values

• Illegal commands usage.

• COMPONENTS OF AN IDPS

The typical components of an IDPS solution are Sensors, Agents and manager (management server, database server).

• Sensors:

• Agents

• Manager ( Management Server)

• SENSORS

Sensor is the main repository in Network-Based IDPS. They are critical in intrusion detection and prevention architectures. Sensors are the beginning point of intrusion detection and prevention systems. They supply the initial data about the potentially malicious activity. Within a particular network architecture, sensors are usually (but not always) considered the lowest end components because sensors typically do not have very sophisticated functionality. They are usually designed to obtain only certain data and pass them on. The Network Interface Cards (NIC), monitoring the network data are placed into the promiscuous mode, which accept all incoming traffic regardless of their destinations.

There are two basic types of sensors:

• Hardware Based/Appliance Based Sensors

• Software Based Sensors

Hardware Based or Appliance Based Sensors

Appliance based or hardware based sensors are dedicated machines that monitor the network traffic. It comprises of specialized processors for optimize performance. They are efficient in capturing & analyzing the raw data for possible malicious activity.

Software Based Sensors

These are the software that can be installed on the hosts. They capture data from packets and prints packet header that match a particular filter expression. The packets parameter that are particularly useful in the intrusion detection & prevention are time, source & destination address, source & destination ports, TCP flags, initial sequence number from the source IP for the initial connection , ending sequence number, number of bytes and window size.

Previously the programs that most frequently use as sensors were TCP dump and Libcap. TCP dump is an application but Libcap is a library called by an application. The main function of Libcap is to gather packet data from the kernel of the operating system and then move it to one or more applications, for example Ethernet card may obtain packet data from a network. The operating system over which libcap runs will process each packet.

Starting with determining what kind of packet it is by removing the Ethernet header to get to the next layer up the stack. The next layer will be IP layer; if so, the IP header must be removed to determine the protocol at the next layer of the stack i.e. ICMP, TCP or UDP. If the packet is TCP, the TCP header is also removed and the contents of the packet are then passed on to the next layer up, the application layer. Libcap provides intrusion detection and prevention applications with this data so that these applications can analyze the content to look for attack signatures, names of hacking tools and so forth libcap provides a standard interface to these applications.

Sensors Deployment Considerations

Many sensors require that a host be running one or more network interfaces in promiscuous mode. Sensors can be placed outside of exterior firewalls, inside them or both.

Sensors that reside outside of exterior firewalls record information about internet attacks. Web server, FTP servers, extended DNS servers and mail servers often placed outside of the firewall, making them much more likely to be attacked than other hosts. Placing these systems within an organizations internal network potentially makes them lesser targets, because being within the internal network at least afford some protection such as filtering barriers provided by firewalls and screening routers.

On the other hand, at the same time, having these servers within the internal network will increase the traffic load for internal network and will also expose the internal network more if any of these servers become compromised. Given that servers placed outside of the internal network are more vulnerable to attack.

Sensors can be deployed in two modes:

• Inline Mode

• Passive Mode

• Inline Mode: In this mode, they need to pass the network traffic through them in order to analyze the traffic for any malicious activity.

The basic reason, to put the sensor inline, is to stop attacks by blocking network traffic just like firewall.

Inline sensors are usually placed between two networks, for example, connections with external networks or at the borders between different internal subnets that needed to segregated.

FIG (>>>>)

• Passive Mode: In this mode, the traffic does not pass through the sensor itself rather it analyze the copy of network traffic for malicious activity. The sensors in passive mode are typically deployed at important key locations in the network. For example, a passive sensor can be deployed at Demilitarized Zone in order to watch the traffic for web server. It can watch the network traffic through different methods:

• Spanning Port: A ‘’spanning port’’ is a switch port that can see all the traffic passing through the switch, so attaching a sensor to the spanning port can enable it to monitor all traffic which is coming to or passing out of this port.

• Network Tap: network refers to the direct communication between the sensor & the physical medium. The tap provides the sensor the copy of all network traffic being carried by the media.

• IDS Load Balancer: An IDS load balancer directs the network traffic to the monitoring system (sensors). It receives copies of network traffic from different spanning port or ports and sends the copies of this network traffic to listening devices .e.g. sensors, based on the configured rules. Common configurations include the following:

If multiple sensors are deployed in the network to analyze the same activity, then it sends all traffic to multiple IDPS sensors.

If there is high volume of traffic then it dynamically split the traffic among multiple sensors.

Traffic is sent to the individual dedicated sensors On the basis of protocols or IP addresses. For example, one sensor might be monitoring the web activity while the other might be monitoring the traffic of a specific subnet.

Security Capabilities of Sensors

The security capabilities of each IDPS depend on the type of technology being used. The main security capabilities of Sensors include the following:

3.5.1 Information Gathering Capabilities

Sensors (Network based IDPS) can collect information on hosts & the network activity of those hosts for example;

• An IDPS sensor may have the list of the hosts in a network on basis of IP or MAC addresses. This list can be used to identify new hosts on the network.

• An IDPS sensor could use different techniques to identify the operating system used by the hosts. For example, it could have the ability of tracking the open ports on the host, which could indicate the family of the operating system. It could analyze the packet header for certain unusual characteristics, the technique is known as fingerprinting.

• Sensor can identify the version of an application by keeping track of the ports.

• To detect any change in configuration of the network, sensors gather general information about the network data.

3.5.2 Logging Capabilities

In the case of detected events, IDPS performs logging of data. This data is of great importance for further investigation of the incident. The commonly logged data are:

• IP addresses of source & destination

• Source & destination ports ( incase of TCP & UDP)

• Message type (in case of ICMP)

• Event or alert type

• Time stamp

3.5.3 Detection Capabilities

Network based sensors has broad range of detection capabilities. IDPS can use either one or all of the following detection mechanisms:

• Signature based detection

• Anomaly based detection

• Stateful protocol analysis

Depending on the type of mechanism they used, sensors can detect the following types of events:

• Application Layer Reconnaissance Attacks

• Transport Layer Reconnaissance Attacks: packet fragmentation, port scanning, SYN flood

• Network Layer Attacks: Spoofed IP addresses, analysis of IP, ICMP and IGMP

• Unexpected Application Services: Unexpected application services can be detected through stateful protocol analysis, for example, unauthorized applications running on hosts or any changes in the network through anomaly based detection.

• Policy Violations: Security violations of an organization can be detected through network based IDPS. For example, unauthorized access to IP addresses, unauthorized access to ports, inappropriate web sites and use of forbidden application protocols.

3.5.4 Prevention Capabilities of Sensors

Network based IDPS sensors not only perform the detection but prevention is also an important function of such types of technologies. Following are the prevention capabilities:

• By Ending the TCP Session (Passive): By ‘’Session Snipping’’ a passive sensor can terminate a TCP session. It sends a TCP reset packets to both endpoints. Both the end point thinks that the other end wants to end the session.

• By Performing Firewalling (Inline): When sensors are placed inline, they behaved like a firewall. If they detect any activity as suspicious, they have the capability to drop or reject the suspicious activity

• Bandwidth Allocation (inline): If a protocol is used for any malicious activity then IDPS sensor has the ability to limit the percentage of bandwidth that a protocol can use.

• Repackaging or Replacing the Content (inline): If an attacker changed the content of the packet for any malicious activity, the inline sensor can replace the payload in new packets, thus normalizing the traffic.

• Reconfiguring Other Devices: If an internal host has been compromised, IDPS sensor can instruct other network devices to block certain types of activity.

• Using Other Programs: If an IDPS sensor doesn’t support the prevention action that is desirable by the administration, it can triggered programs specified by the administration in case of any detected attack.

3.6 Agents

Agent/s is the repository of a Host-based IDPS. Host-Based IDPS have detection software known as Agents installed on the hosts to monitor the activities of that single host or dedicated appliances running agent software installed on them. Each appliance monitor the network activity coming to and going from a particular host.

The primary function of the agents is the analysis of input provided by sensors. An agent can be defined as a group of processes that run independently and that are programmed to analyze system behavior or network events or both to detect anomalous and violations of an organization’s security policy.

Each agent performs a specialized function independently, for example some agents may examine network traffic and host-based events generally i.e. such as checking whether normal TCP connections have occurred, their start and stop times, the amount of data transmitted or weather certain services have crashed.

Other Agents might look at specific aspects of application layer protocols such as FTP, TFTP, HTTP and SMTP as well as authentication sessions to determine whether data in packets or system behavior is consistent with known attack patterns.

The independent running of Agents means that if one Agent crashes or it is impaired in some manner, the other will continue to run normally.

It also means that Agents can be added or deleted from an IDPS, although each Agent runs independently on the particular host on which it resides, Agents often co-operate with each other. Each Agent may receive and analyze only one part of the data regarding a particular system, network or device.

Agents normally share information they have obtained with each other by using a particular communication protocol over the network. When an Agent detects an anomaly or policy violation (such an attempt to root or a massive flood of packets over the network), the Agent will immediately notify the other Agents of what it has found. This new information, combined with the information the another Agent has, may cause that Agent to report that an attack on another host also occurred.

At a minimum, an Agent needs to incorporate three functions or components:

• A communication interface to communicate with other components of an IDPS.

• A listener that waits in the background for data from sensors and messages from other Agents and receives them.

• A sender that transmits data and messages to other components, such as other Agents and the manage components using established means of communication such as network protocols.

Besides the above, the Agents can also provide a variety of additional functions for example; Agents can perform correlation analysis on input received from a wide range of sensors. In some Agents implementation, the Agents themselves generate alerts and alarms. In some other implementations, Agents access large database to launch queries to obtain more information about specific source and destination IP addresses associated with certain types of attacks, times at which known attacks have occurred, frequencies of scans and other types of malicious activity and so forth. From this kind of additional information, Agents can perform functions such as tracking the specific phases of attacks and estimating the threat that each attack constitutes.

Host-based IDPS often have extensive knowledge about characteristics and configuration and due this ability they can determine whether or not an attack against the host would succeed if not stopped.

Agents Deployment Considerations

Agent can and should be configured to the operating environment in which its run. In Host-Based intrusion detection each Agent generally monitors one host, sometimes sensors on multiple hosts send data to one or more central Agents.

Generally Host-based Agents are deployed to publically accessable servers. In Network-Based intrusion detection, Agents are generally placed in two locations:

• Where They Are Most Efficient: Efficiency is related to the particular part of a network where connections to sensors and other components are placed. The more locally co resident the sensors and agents are, the better the efficiency.

• Where They Will Be Sufficiently Secure: The threat of subversion of Agents is a major issue. Agents are typically much smarter than sensors. If an Agent is successfully attacked, not only will the attacker be able to stop or subvert the type of analysis that the Agent performs, but the attack can also be able to glean information that is likely to prove useful in attacking the other components of an IDPS. Compromised Agents thus can rapidly become a security liability.

The way Agents are typically deployed provides some level of defense against attacks that are directed at them. Agents (especially in network-based IDPS) are generally distributed through out a network or networks. Each Agent must therefore be individually discovered and attacked. This substantially increases the work involved in attacking Agents, something that is very desirable from a security perspective, each to some degree unique challenge to the attacker.

Agents need to be secure by doing many things that must be done to protect sensors hardening the platform on which they run, ensuring that they can be accessed by authorized persons and so on.

Security Capabilities of Agents

The security capabilities of Host-Based IDPS are as follows

3.7.1 Logging Capabilities

One of the important functions of Agents or host-based IDPS is logging of data to detected events. The logged data is useful for the further investigation of the incident. The commonly logged information include the following:

• Date & time

• Alert type

• IP addresses & port information

• Applications, paths & filenames

3.7.2 Detection Capabilities

Host-based IDPS detect different types of events, depending on type of detection technology. Some of the techniques are:

• Code Analysis: Agents can analyze attempts to execute code, which could be the signs of possible malware activity, by the following techniques:

• Analysis Of Code Behavior: Before running the code on the host, it is executed in a virtual environment & the output is analyzed with the profiles of accepted or unaccepted behavior.

• Buffer Overflow: Attempts to perform buffer overflow is detected by looking certain sequence of instructions & accessing to the memory that is not allocated to the process.

• System Call Monitoring: Some malicious activities involve triggering of other applications or processes. Agent watch such attempts.

• Lists of Applications & Library: If a user attempts to load an application or library, an agent monitors it by comparing with the list of authorized & unauthorized applications & libraries.

• Analysis Of Network Traffic: Agents analyze the network, transport & application layer protocols for any malicious activity. In addition, they also do processing for some applications, for example, email clients.

• Filtering Of Network Traffic: Agents can filter incoming & outgoing traffic from each application. Their traffic filtering capability prevent unauthorized access.

• Monitoring Of File System: Different techniques are used to monitor the file system.

• Checking Of File Integrity: This can be done by generating message digests or cryptographic checksums for files & comparing it with the reference values.

• Checking Of File Attributes: It involves checking of file attributes, for example, ownership & permissions to specific file.

• Attempts to File Access: An agent has policies regarding file access. It compares the current attempt or type of access with the policies. The attempt could be from a user or an application.

• Monitoring Of Network Configuration: Agent can monitor a host’s network configuration & detect if it has been changed, For example, additional ports of a host being used or additional network protocols.

3.7.3 Prevention Capabilities of Agents

The prevention capabilities of the Host-Based IDPS Agents mainly depend on the detection techniques used by them.

• Analysis of Code: To prevent the code from being executed, host-based IDPS uses code analysis technique, which could prevent malware and unauthorized applications. Some also prevent invoking of shells, which may cause certain type of attack, by stopping network applications.

• Analysis of Network Traffic: By analyzing the network traffic, it stops the incoming traffic that might be malicious or outgoing traffic that might be unauthorized. The analysis can also prevent unauthorized downloading or transferring of files to a host. Network analysis technique might reject or drop the network traffic.

• Filtering of Network Traffic: If the activity is identified by the IP address, ports, ICMP message type or unauthorized policy violation, it can stop unauthorized access.

• Monitoring of File System: This capability prevents system files from being accessed, modified or deleted, which could stop any malicious activity.

3.8 Manager

The final component in multitier architecture is the “Manager” (also known as Server). The fundamental purpose of this component is to provide an executive or master control capability for an IDPS.

Functions

Sensors are usually low–level components and that Agents are usually more sophisticated components, at a minimum, analyze the data they receive from sensors and possibly from each other.

Although Sensors and Agents are capable of functioning without a master control component, but master component is extremely advantageous in helping all components work in a coordinated manner along with some other valuable functions:

• Date Management: IDPS can gather massive amount of data. One way to deal with this amount of data is to compress (to conserve disk space), archive it, and then purge it.

Having sufficient disk space for management purposes is a major consideration. One good solution is RAID (redundant array of inexpensive disks), which writes data to multiple disks and provides redundancy in case of any disk failing. Another option is optical media, such as worm drives.

The manager component of an IDPS will also organize the stored data i.e. a rational database. Once a database is designed and implemented, new data can be added on the fly and queries against database entries can be made.

• Alerting: Another important function that the manager component can perform is generating alerts whenever events that constitute high levels of threat occur. Agents are designed to provide detection capability, but Agents are normally not involved in alerting because it is more efficient to do so from a central host. Agents usually send information to a central server that sends alert whenever predefined criteria are met. This requires that the server not only contain the addresses of operators who need to be notified, but also have an alerting mechanism.

Alerts are either sent via email or Syslog facility, the message content is usually encrypted. The main advantage of Syslog facility is its flexibility, Syslog can send messages about nearly anything to just about everybody if desired.

• Event Correlation: It is extremely important function of the manager component is correlating events that have occurred to determine whether they have a common source, whether they were part of a series of related attacks and so fourth.

• High-Level Analysis: The Manager component can perform high-level analysis of the events that the intrusion-detection or intrusion-prevention tool discovers. The Manager component can track the progression of each attack from stage to stage, starting with the preparatory stage. Additionally, this component can analyze the threat each event constitutes, sending notification to the alert generation whether a threat reaches a certain specified value.

• Monitoring other Components: Being the centralized, the Manager is ideal to perform this function. The Manger can send packets to each Sensor and Agent to determine whether each is up and running. If the Manager component determines that any other component has failed, it can notify its alerting facility to generate an alert.

Manager can also monitor each host to ensure that logging or auditing is functioning correctly.

• Policy Generation & Distribution: Another important function of the Manager component is Policy Generation & Distribution. The term ‘’Policy’’ refers to the setting that affect how the various components of an intrusion-detection or intrusion-prevention system function.

Based on the data that the Manager component receives, it creates and then distributes a policy or a change in policy to individual hosts. The policy might tell each host to not accept input for a particular source IP address or not to execute a particular system call. The Manger component is usually in charge of creating, updating and enforcing policy.

• Management Console: Manager Component also provides an interface to the users through management console. It displays critical information-Alerts, the status of each component, data in individual packets, audit log data etc and also allow the ope