Driving Factors Over Ids

Published Date: 02 Nov 2017

Abstract

Computer systems of today are subject to many attacks and it can be anticipated that these problems will increase in the future. One way of protecting the systems is to use better authentication and other types of preventive security mechanisms. These mechanisms do not offer good enough protection in most cases and they should therefore be complemented with monitoring and detection mechanisms. Intrusion detection over the network is indeed an important field of the information security. Although many intrusion detection systems are widely available today, the technology is still young and the combat against threats from both internal and external sources seems to be an endless. Intrusion detection systems (IDS) have become an important component in the security officer's toolbox. However, many security experts are still in the dark about IDS, unsure about what IDS tools do, how to use them, or why they are a must. In the following we will offer a brief overview of intrusion detection systems, including: a description of what IDSs are, the functions they serve, and the different methods of intrusion detection that they may employ.

Keywords

Authenticity; Availability; Attacks; Detection; Expansion; Intrusion; Security; Sensitive; Converging; Techniques; Threats; Legitimate

Introduction

Before we start to answer this important question asked from everyone in every network age that "Why we need network security?" we have to take a deep look about our current running systems. For the first few decades of their existence, computer networks were primarily used by university researchers for sending e-mail and by corporate employees for sharing printers. Under these conditions, security did not get a lot of attention. As Internet expanded, so did the opportunities for its misuse, the result of a host of security flaws. For instance, e-mail was easy to spoof, passwords were transmitted in clear and connections could be hijacked. Nevertheless, most users had no real interest in security failings until the 1980`s Internet worm case, which provided a glimpse of how damaging these defects Error: Reference source not found. But now, as millions of ordinary citizens are using networks for banking, shopping, and filing their tax returns, network security is looming on the horizon as a potentially massive problem. Computer crimes are common today and there are many indications of that the problem is going to increase. The concept of security is defined as a continuous process of protecting an object from attack. That object may be a person, an organization such as a business, or property such as a computer system or a file Error: Reference source not found. Network security can be defined as a process of securing network assets from internal and external threats. It also can be defined as protecting data that are stored on or that travel over a network against either accidental and intentional unauthorized disclosure or modification Error: Reference source not found. Many people view network security as having 3 main goals (Availability, Integrity and confidentiality) are described in figure 1, as a core trinity of a network security Error: Reference source not found, Error: Reference source not found. Availability is a characteristic that ensures that our information, service or asset is accessible and can provide the service it is designed for, when it is needed. There are several processes in networking such as: redundancy, backups; that can offer a higher level of system availability. The denial of service attacks are aimed to harm availability of the system. Integrity is a characteristic about the insurance of software or data completeness and accuracy as well as its authenticity. When we talk about the network integrity it has the purpose to provide and ensure that data must be protected from unauthorized modification and destruction. It can be achieved by cryptography. Confidentiality is a characteristic of protecting sensitive information from manipulation in a form of disclosure and interception. Cryptography is also used here to provide confidentiality.

Figure ‎1. Network security trinity [2]

The importance of those characteristics varies depending on the company’s business. For example banks will be most concerned about the integrity of data in contrast of an Internet Service Provider company that values the availability as the most important characteristic for their business. Furthermore, all characteristics are connected among themselves and cooperate to provide an adequate level of security. Based on everything we know, this truly seems to be the golden age of hacking Error: Reference source not foundthis statement is true for many reasons we will explain this as follows;

Networks are easy to crack. The Internet grew so quickly that few gave any thought to security. Attackers have the upper hand and it will take a while before companies secure their systems. The best thing for companies to do is disconnect from the Internet until their systems are secure, but no one will do that. The other thing that makes matters worse is how companies have built their networks. In the past, every company’s network and systems were different. In late 80`s, companies hired programmers to customize their applications and systems, so if an attacker wanted to break into your network, he had to learn a lot about your environment. The information did not help the attacker when he tried to break into another company’s network, because its systems were totally different. Now, every company uses the same equipment with the same software. If an attacker learns Cisco and he can break into practically any system on the Internet; because networks are so similar, and software and hardware are so standardized, the attacker’s job is much easier [7].

Attacks Are Easy to Obtain and Easy to Use. Not only are systems easy to break into, but the tools for automating attacks are very easy to obtain on the Internet. Even though an attacker might have a minimal amount of sophistication, he can download tools that allow him to run very sophisticated attacks. The ease at which these tools and techniques can be obtained transforms anyone with access to the Internet into a possible attacker [7].

Boundless Nature of Internet. Another issue is the ease in which a user connected to the Internet can travel across local, state, and international boundaries. Accidentally typing one wrong number in an IP address can be the difference of connecting to a machine across the room and connecting to a machine across the world. When connecting to a machine outside this country, international cooperation is required to trace the connection. Based on the ease of connecting to a machine anywhere in the world, attackers can hide their path by hopping through several computers in several countries before attacking a target machine. In many cases, picking countries that are not allies can almost eliminate the possibility of a successful trace. So to trace this attacker couple of things is needed: First, it takes a lot of time, and second, it requires timely cooperation among all the regions, which would be difficult at best [7].

No one is Policing the Internet. Currently, because there is no one policing the Internet, when problems occur, there are not clear lines over who should investigate and what crime has been committed. Most countries are trying to take conventional laws and apply them to the Internet. In some cases, they apply, but in other cases they do not adapt well. Even if there were an entity policing the Internet, it would still be difficult because people are committing the crimes virtually [7].

So the need to computer network is not an option during the present days, it became a must and the question should not be whether to secure your network or not; but it must be how to secure it. An intrusion detection system (IDS) can be defined as the tools, methods, and resources to help identify, assess, and report unauthorized or unapproved network activity . Intrusion detection systems do exactly as the name suggests: they detect possible intrusions. More specifically, IDS tools aim to detect computer attacks and/or computer misuse, and to alert the proper individuals upon detection. An IDS installed on a network provides much the same purpose as a burglar alarm system installed in a house. Through various methods, both detect when a burglar is present, and both subsequently issue some type of warning or alert. Although IDSs may be used in conjunction with firewalls, which aim to regulate and control the flow of information into and out of a network, the two security tools should not be considered the same thing. Using the previous example, firewalls can be thought of as a fence or a security guard placed in front of a house. They protect a network and attempt to prevent intrusions, while IDS tools detect whether or not the network is under attack or has, in fact, been breached. IDS tools thus form an integral part of a thorough and complete security system. They don’t fully guarantee security, but when used with security policy, vulnerability assessments, data encryption, user authentication, access control, and firewalls, they can greatly enhance network safety.

Driving Factors over IDS

The concept of intrusion detection is not a new one, but it has been discussed more than thirty years ago and from that time many researchers addressed it. In this chapter we will take a look on the previous and current works in the field of IDS. Originally, the first step in intrusion detection was done by system administrators they performed intrusion detection by sitting in front of a console and monitoring user activities. They might detect intrusions by noticing, for example, that a vacationing user is logged in locally or that a seldom used printer is unusually active. Although effective enough at the time, this early form of intrusion detection was ad hoc and not scalable. The next step in intrusion detection involved audit logs, which system administrators reviewed for evidence of unusual or malicious behavior. In the late 1980`s, administrators typically printed audit logs on fan-folded paper, which were often stacked four to five feet high by the end of an average week. Searching through such a stack was obviously very resource consuming. With this overabundance of information and only manual analysis, administrators mainly used audit logs as a forensic tool to determine the cause of a particular security incident after the fact. There was little hope of catching an attack in progress. As storage became cheaper, audit logs moved online and researchers developed programs to analyze the data.

However, analysis was slow and often computationally intensive and, therefore, intrusion detection programs were usually run at night when the system’s user load was low. Therefore, most intrusions were still detected after they occurred. In early ’90s, researchers developed real time intrusion detection systems that reviewed audit data as it was produced. This enabled the detection of attacks and attempted attacks as they occurred, which in turn allowed for real time response, and, in some cases, attack preemption [1]. More recent intrusion detection efforts have centered on developing products that users can effectively deploy in large networks. This is no easy task, given increasing security concerns, countless new attack techniques, and continuous changes in the surrounding computing environment. The first attempt to define the process of intrusion was done in 1980`s and he defined some terms in this field as follows;

Threat: the potential possibility of a deliberate unauthorized attempt to; access information, manipulate information, render a system unreliable or unusable [19].

Risk: accidental and unpredictable exposure of information, or violation of operations integrity due to malfunction of hardware or incomplete or incorrect software design [19].

Vulnerability: a known or suspected flaw in the hardware or software design or operation of a system that exposes the system to penetration of its information to accidental disclosure [19].

Attack: a specific formulation or execution of a plan to carry out a threat [19].

Penetration: a successful attack; the ability to obtain unauthorized (undetected) access to files and programs or control state of a computer [19].

In the article [15] it is proposed that audit trials should be used to monitor threats and all security procedures were focused on denying access to sensitive data from an unauthorized source. Later in [2] it was proposed the concept of intrusion detection as a solution to the problem of providing a sense of security in computer systems. The basic idea is that intrusion behavior involves abnormal usage of the system. The model is a rule based pattern matching system. Some models of normal usage of the system could be constructed and verified against usage of the system and any significant deviation from the normal usage flagged as abnormal usage. Statistical approaches compare the recent behavior of a user of a computer system with observed behavior and any significant deviation is considered as intrusion. This approach requires construction of a model for normal user behavior. Predictive pattern generation uses a rule base of user profiles defined as statistically weighted event sequencesError: Reference source not found. This method of intrusion detection attempts to predict future events based on events that have already occurred. State transition analysis approach construct the graphical representation of intrusion behavior as a series of state changes that lead from an initial secure state to a target compromised state. Using the audit trail as input, an analysis tool can be developed to compare the state changes produced by the user to state transition diagrams of known penetrations [4].

Keystroke monitoring technique utilizes a user’s keystrokes to determine the intrusion attempt. The main approach is to pattern match the sequence of keystrokes to some predefined sequences to detect the intrusion. Model Based approach attempts to model intrusions at a higher level of abstraction than audit trail records. This allows administrators to generate their representation of the penetration abstractly, which shifts the burden of determining what audit records are part of a suspect sequence to the expert system. This technique differs from the rule based expert system technique, which simply attempt to pattern match audit records to expert rulesError: Reference source not found. The pattern matchingError: Reference source not found approach encodes known intrusion signatures as patterns that are then matched against the audit data. Intrusion signatures are classified using structural interrelationships among the elements of the signatures. The patterned signatures are matched against the audit trails and any matched pattern can be detected as an intrusion [5]. During recent years, several data mining approaches have been also used to construct IDS. Also in Error: Reference source not found it was proposed a novel IDS architecture utilizing both anomaly and misuse detection. The hybrid IDS architecture consists of an anomaly detection module, a misuse detection module and a decision support system combining the results of these two detection modules. The proposed anomaly detection module uses a Self Organizing Map (SOM) structure to model normal behavior. The proposed misuse detection module uses decision tree algorithm to classify various types of attacks. A rule based Decision Support System (DSS) is also developed for interpreting the results of both anomaly and misuse detection modules.

The author in suggested two data mining methodologies involving artificial neural networks (ANN) and support vector machine (SVM) and two encoding methods namely simple frequency based scheme and TFIDF scheme to detect potential system intrusions. Their experiments show that SVM with TFIDF scheme achieved the best performance, while ANN with simple frequency based scheme achieved the worst. Furthermore in the article Error: Reference source not found presented security agent architecture, which is useful as an administrative tool for intrusion detection. The agent based monitoring and detection system, could detect malfunctions, faults, abnormalities, misuse, deviations, intrusions, and provide recommendations (in the form of common intrusion detection language). We have seen in Error: Reference source not found that the author has formulated intrusion detection as a binary classification problem, using SVM and additionally, some text processing techniques are also employed for intrusion detection, based on the characterization of the frequencies of the system calls executed by the privileged programs.

Rohrmair and Lowar Error: Reference source not found demonstrate the modeling and analysis of IDS using the process algebra communicating sequential processes and its model checker FDR. Authors show that this analysis can be used to discover attack strategies that can be used to blind efficient IDS, even a hypothetically perfect one that knows all the weaknesses of its protected host. Current IDS examine all data features to detect intrusion or misuse patterns. Some of the features may be redundant or contribute little (if anything) to the detection process. Chebrolu et al. Error: Reference source not found investigated the performance of two feature selection algorithms involving Bayesian networks (BN) and Classification and Regression Trees (CART). Empirical results indicate that significant input feature selection is important to design an IDS that is lightweight, efficient and effective for real world detection systems. Most IDS have a single level structure can only detect either misuse or anomaly attacks. Some IDSs with multi level structure or multi classifier are proposed to detect both attacks, but they are limited in adaptive learning. Zhang et al. proposed a serial hierarchical IDS (SHIDS) to identify misuse attacks accurately and anomaly attacks adaptively. Beghdad Error: Reference source not found introduces an anomaly intrusion detection method based on a Within Class Dissimilarity (WCD). This approach functions by using an appropriate metric WCD to measure the distance between an unknown user and a known user defined respectively by their profile vectors [16].

Classifications and trait of IDS

Our existing computer systems which are supposed to provide confidentiality and assurance against denial of service. However, due to increased connectivity and the vast spectrum of financial possibilities that are opening up, more and more systems are subject to attack by intruders. These subversion attempts try to exploit flaws in the operating system as well as in application programs. This may rise another important question "How we can handle subversion attempts?". There are two ways to handle subversion attempts. One way is to prevent subversion itself by building a completely secure system. For example, require all users to identify and authenticate themselves; we could protect data by cryptographic methods and very tight access control mechanisms. In practice, it is not possible to build a completely secure system because of; Bug free software is still a dream and no one seems to want to make the effort to try to develop such software. So, the designing and implementing is totally secure system and thus an extremely difficult task too.

The vast installed base of systems worldwide guarantee that any transition to a secure system, (if it is ever developed) will be long in coming [7].

Cryptographic methods have their own problems. Passwords can be cracked, users can lose their passwords, and entire crypto-systems can be broken [7].

Even a truly secure system is vulnerable to abuse by insiders who abuse their privileges [7].

It has been seen that the relationship between the level of access control and user efficiency is an inverse one, which means that the stricter the mechanism, the lower the efficiency becomes [7].

We thus see that we are stuck with systems that have vulnerabilities for a while to come. If there are attacks on a system, we would like to detect them as soon as possible and take appropriate action. This is essentially what an intrusion detection system does. An IDS does not usually take preventive measures when an attack is detected; it is a reactive rather than proactive agent. It plays the rule of an informant rather than a police officer. Error: Reference source not found. So the question should not be whether to use intrusion detection, but which intrusion detection features and capabilities to use. Intrusions can be divided into 6 main types;

Attempted break-ins, which are detected by a typical behavior profiles or violations of security constraints [16].

Masquerade attacks, which are detected by a typical behavior profiles or violations of security constraints [16].

Penetration of the security control system, which are detected by monitoring for specific patterns of activity [16].

Leakage, which is detected by a typical use of system resources [16].

Denial of service, which is detected by a typical use of system resources [16].

Malicious use, which is detected by a typical behavior profiles, violations of security constraints, or use of special privileges [16].

Intrusion detection systems can be classified according to different criteria such as information sources, detection techniques, and Response Options. The most common way to classify IDS`s is to group them by information source. Some IDS`s analyze network packets. Other IDS`s analyze information sources generated by the operating system or application software for signs of intrusion. The majority of commercial IDS`s are network based. These IDS`s detect attacks by capturing and analyzing network packets. Listening on a network segment or switch, one network-based IDS can monitor network traffic affecting multiple hosts that are connected to the network segment, thereby protecting these hosts. Network-based IDS`s often consists of a set of sensors or hosts placed at various points in a network. These units monitor network traffic, performing local analysis of the traffic and reporting attacks to a central management console Error: Reference source not found. The ultimate objective of network security is to ensure that protected applications and the information used as input and generated as output by these applications are not compromised by malicious or unintentional security breaches. As a result, it is possible to define the major basic network security functional elements that are needed to build a network security system, in terms of the following well known security services needed for secure message exchanges: privacy, authentication, authorization, message integrity, and non-repudiation Error: Reference source not found.

Security views associated with IDS

Commonly known, host based IDS (HIDS) and network based IDS (NIDS) are two types of intrusion detection systems differ significantly from each other, but complement one another well as defined in table1. The network architecture of host-based is agent-based, which means that a software agent resides on each of the hosts that will be governed by the system. In addition, more efficient host-based intrusion detection systems are capable of monitoring and collecting system audit trails in real time as well as on a scheduled basis, thus distributing both CPU utilization and network overhead and providing for a flexible means of security administration. In a proper IDS implementation, it would be advantageous to fully integrate the network intrusion detection system, such that it would filter alerts and notifications in an identical manner to the host-based portion of the system, controlled from the same central location. In doing so, this provides a convenient means of managing and reacting to misuse using both types of intrusion detection.

NIDS

HIDS

Broad in scope (watches all network activities)

Narrow in scope (watches only specific host activities)

Easier setup

More complex setup

Better for detecting attacks from the outside

Better for detecting attacks from the inside

Less expensive to implement

More expensive to implement

Detection is based on what can be recorded on the entire network

Detection is based on what any single host can record

Examines packet headers

Does not see packet headers

Near real-time response

Usually only responds after a suspicious log entry has been made

OS-independent

OS-specific

Detects network attacks as payload is analyzed

Detects local attacks before they hit the network

Detects unsuccessful attack attempts

Verifies success or failure of attacks

Table ‎2: The differences between NIDS and HIIDS

Most internal threats come from two sources: employees and accidents. Employee threats may be intentional or accidental as well. We will not stress on threats occurred by accidents. In most cases, employees know more about a network and the computers on it than any outsider. At the very least, they have legitimate access to user accounts. IT personnel, of course, have various levels of increased access. Intentional employee security threats include the following;

Person who employ hacking techniques to upgrade their legitimate access to root/administrator access, allowing them to reveal trade secrets, steal money, and so on for personal or political gain.

Person who take advantage of legitimate access to reveal trade secrets, steal money, and so on for personal or political gain.

Family members of employees who are visiting the office and have been given access to company computers to occupy them while waiting.

Person who breaks into secure machine rooms to gain physical access to mainframe and other large system consoles.

Hackers and crackers are the sources of external threats to any existing system. So we will take a deep tour to investigate who are them and why they do that. A hacker is a person intensely interested in the arcane and recondite workings of any computer operating system. Most often, hackers are programmers. As such, hackers obtain advanced knowledge of operating systems and programming languages. They may know of holes within systems and the reasons for such holes. Hackers constantly seek further knowledge, freely share what they have discovered, and never, ever intentionally damage data. A cracker is a person who breaks into or otherwise violates the system integrity of remote machines, with malicious intent. Crackers, having gained unauthorized access, destroy vital data, deny legitimate users service, or basically cause problems for their targets. Crackers can easily be identified because their actions are malicious.

Detection Techniques and further response options

We can classify IDSs according the detection techniques to misuse detection and anomaly detection. Misuse detection, in which the analysis targets something known to be "bad", is the technique used by most commercial systems. While in anomaly detection the analysis looks for abnormal patterns of activity. Although anomaly detection is used in limited IDSs today, it stills the subject of a great deal research.

Misuse Detection

Misuse detectors analyze system activity, looking for events that match predefined pattern of events describe a known attack. Since the pattern corresponding to known attack is called signature, misuse detection is commonly called signature-based detection. It is also known in literature as Rule-based detection. This method is similar to method of detection new viruses where an appropriate signature or pattern should be known in advance. A block diagram of a typical misuse detection system is shown in Figure 2.

Figure 2. typical misuse detection system [16]

Anomaly Detection

Anomaly detectors are designed to uncover abnormal patterns of behavior, the IDS establishes a baseline of normal usage patterns, and anything that widely deviates from it gets flagged as a possible intrusion. What is considered to be an anomaly can vary, but normally, any incident that occurs on frequency greater than or less than two standard deviations from the statistical norm raise an eyebrow. An example of this would be if a user logs on and off of a machine 20 times a day instead of the normal 1 or 2. At another level, anomaly detection can investigate user patterns, such as profiling the programs executed daily. If a user in the graphics department suddenly starts accessing accounting programs or compiling code, the system can properly alert its administrators.

Figure 3. typical misuse detection system [18]

Response Options for IDSs

Once IDSs have obtained event information and analyzed it to find symptoms of attacks, they generate responses. Some of these responses involve reporting results and findings to a pre-specified location. Others involve more active automated responses. Though researchers are tempted to undervalue the importance of good response functions in IDSs, they are actually very important. Commercial IDSs support a wide range of response options, often categorized as active responses, passive responses. Active IDS responses are automated actions taken when certain types of intrusions are detected. There are three categories of active responses.

Feats against Intruder by assembling information

The most innocuous, but at times most productive, active response is to collect additional information about a suspected attack. Each one of us has probably done the equivalent of this when awakened by a strange noise at night. The first thing one does in such a situation is to listen more closely, searching for additional information that allows you to decide whether you should take action. In the IDS case, this might involve increasing the level of sensitivity of information sources (for instance, turning up the number of events logged by an operating system audit trail, or increasing the sensitivity of a network monitor to capture all packets, not just those targeting a particular port or target system.) Collecting additional information is helpful for several reasons. The additional information collected can help resolve the detection of the attack.

Another active response is to halt an attack in progress and then block subsequent access by the attacker. Typically, IDSs do not have the ability to block a specific person’s access, but instead block Internet Protocol (IP) addresses from which the attacker appears to be coming. Passive IDS responses provide information to system users, relying on humans to take subsequent actions based on that information. Many commercial IDSs rely solely on passive responses. Some who follow intrusion detection discussions, especially in information warfare circles, believe that the first option in active response is to take action against the intruder. The most aggressive form of this response involves launching attacks against or attempting to actively gain information about the attacker’s host or site. However tempting it might be, this response is ill advised. Due to legal ambiguities about civil liability, this option can represent a greater risk that the attack it is intended to block. The first reason for approaching this option with a great deal of caution is that it may be illegal. Furthermore, as many attackers use false network addresses when attacking systems, it carries with it a high risk of causing damage to innocent Internet sites and users. Finally, strike back can escalate the attack, provoking an attacker who originally intended only to browse a site to take more aggressive action.

Alarms and Notifications

Alarms and notifications are generated by IDSs to inform users when attacks are detected. Most commercial IDSs allow users a great deal of latitude in determining how and when alarms are generated and to whom they are displayed.

Figure 4. typical misuse detection system [5]

The most common form of alarm is an onscreen alert or popup window. This is displayed on the IDS console or on other systems as specified by the user during the configuration of the IDS. The information provided in the alarm message varies widely, ranging from a notification that an intrusion has taken place to extremely detailed messages outlining the IP addresses of the source and target of the attack, the specific attack tool used to gain access, and the outcome of the attack. Another set of options that are of utility to large or distributed organizations are those involving remote notification of alarms or alerts. These allow organizations to configure the IDS so that it sends alerts to cellular phones and pagers carried by incident response teams or system security personnel. Some products also offer email as another notification channel. This is ill advised, as attackers often routinely monitor email and might even block the message.

Conclusion

As security incidents become more numerous, IDS tools are becoming increasingly necessary. They round out the security arsenal, working in conjunction with other information security tools, such as firewalls, and allow for the complete supervision of all network activity. This information can, in turn, help to determine network misuse, its nature, and its source. These intrusion detection tools use several techniques to help them determine what qualifies as an intrusion versus normal traffic. Whether a system uses anomaly detection, misuse detection, target monitoring, or stealth probes, they generally fall into one of two categories: network-based or host-based. Each category has strengths and weaknesses that should be measured against the requirements for each separate target environment. Ideally, the best IDS tools combine both approaches under one management console. That way, the user gets comprehensive coverage, making sure to guard against as many threats as possible. Whatever the choice, whether it is host-based, network-based, or a hybrid of the two, it is clear that using intrusion detection systems is an important and necessary tool in the security manager's arsenal. Intrusion detection based upon computational intelligence is currently attracting considerable interest from the research community. Its characteristics, such as adaptation, fault tolerance, high computational speed and error resilience in the face of noisy information, fit the requirement of building a good intrusion detection system.