An Important Aspect Of Intrusion Detection

Published Date: 02 Nov 2017

CHAPTER 6

The Internet has brought dramatic changes in the interaction between individuals, businesses, and governments. Moreover, global access to the Internet has become ubiquitous. With broadband networks, large amounts of data can be transferred rapidly between parties over the Internet. Users take these advances for granted until security attacks cripple the global Internet. Attacks spread rapidly through the same broadband networks that made the Internet revolution possible. The cost of these attacks to individuals, companies, and governments has increased rapidly as well. According to Trend Micro, virus attacks cost global businesses an estimated $55 billion in damages in 2003, up from about $20 billion in 2002.

In academia, researchers have proposed the Internet2 and Supernet networks to provide a networking test bed for all university campuses in the United States to use for real-time collaboration at speeds that are 1000’s of times faster than those available on currently deployed campus networks. Many scientists predict that Internet data network speed will increase from the 10 Gbps to the Tbps range. With this explosive growth in Internet ecommerce and the advent of the terabit network on the horizon, security experts must reevaluate current security solutions. Clearly, the critical challenge facing the Internet in the future is establishing security and trust. Many researchers argue that automatic detection and protection is the only solution for stopping fast-spreading worms such as the SQL Slammer and Code Red.

Realizing the fact that traditional intrusion detection systems have not adapted adequately to new networking paradigms like wireless and mobile networks and nor have they scaled to meet the requirements posed by high-speed (gigabit and terabit) networks, this research presents techniques to address the threats posed by network based denial-of-service attacks in such networks. Although there have been recent attempts [37, 39] to train anomaly detection models over noisy data, to the best of our knowledge, the research presented in this dissertation is the first attempt to investigate the effect of incomplete data sets on the anomaly detection process. This chapter describes the work performed during this dissertation and highlights significant achievements and possible extensions to the existing state of the art.

6.1 SUMMARY

This research provides a methodology to detect intrusions when only a subset of the audit data is available. To achieve this goal, the proposed anomaly detection scheme integrated a number of components in a unique way to overcome the limitations of previous intrusion detections systems. To tackle the problem of intrusion detection in high bandwidth networks the following modules were implemented and tested.

Sampling for data reduction: We believe that the key to efficient and cost-effective intrusion detection in high bandwidth networks is implementing a system that samples the incoming network traffic instead of attempting to parse and analyze all of it. From this perspective, in this dissertation, we have presented an adaptive sampling algorithm that uses the weighted least squares predictor to dynamically alter the sampling rate based on the accuracy of the predictions. Results have shown that compared to simple random sampling, the proposed adaptive sampling algorithm performs well on random, bursty data. Our simulation results have shown that the proposed sampling scheme is effective in reducing the volume of sampled data while retaining the intrinsic characteristics of the network traffic. In addition, we have also demonstrated that the proposed sampling scheme preserves the self-similar property of network traffic in the sampling process.

Bloom filters based fast flow aggregation: In high speed networks, flow aggregation and state maintenance becomes a problem as the number of flows increase. As a result, delays are introduced in the intrusion detection process which, in the worst case scenario, makes the intrusion detection system totally ineffective. To mitigate the delay problem and provide for a fast lookup for maintaining the state of an incoming flow, this dissertation proposes a Bloom filter based flow aggregation scheme.

Improvements in speed of clustering: The classical EM algorithm has many desirable features that includes a strong statistical basis, theoretical guarantees about optimality, easily explainable results, robustness to noise and to highly skewed data. Nevertheless, the classical EM algorithm also has several disadvantages. In general, the EM algorithm converges to a poor locally optimal solution and requires a number of passes over the dataset for every iteration. To address these problems, this dissertation proposed improvements in speed by employing sufficient statistics and learning steps to accelerate convergence and reduce the number of passes per iteration.

Anomalous flow detection Lastly based on the improvements described above, this dissertation also proposes a novel a intrusion detection scheme that is capable of detecting anomalous/intrusive flows in the incoming network traffic which would be indicative of a network based denial of service attack when only a subset of the audit data is available. Results described above, have shown that the proposed system has a high rate of detection even when only 75% of the audit data is present.

6.2 FUTUREWORK

In the last twenty years, intrusion detection systems have slowly evolved from host- and operating system-specific applications to distributed systems that involve a wide array of operating systems. The challenges that lie ahead for the next generation of intrusion detection systems and, more specifically, for anomaly detection systems are many.

Over the years, numerous techniques, models, and full-fledged intrusion detection systems have been proposed and built in the commercial and research sectors. However, there is no globally acceptable standard or metric for evaluating an intrusion detection system. Although the ROC curve has been widely used to evaluate the accuracy of intrusion detection systems and analyze the tradeoff between the false positives rate and the detection rate, evaluations based on the ROC curve are often misleading and/or incomplete [44,113]. Recently, several methods have been proposed to address this issue [7, 44, 113]. However, a majority of the proposed solutions rely on parameters values (such as the cost associated with each false alarm or missed attack instance) that are difficult to obtain and are subjective to a particular network or system. As a result, such metrics may lack the objectivity required to conduct a fair evaluation of a given system. Therefore, one of the open challenges is the development of a general systematic methodology and/or a set of metrics that can be used to fairly evaluate intrusion detection systems.

An important aspect of intrusion detection, which has also been proposed as an evaluation metric [102, 109, 116], is the ability of an intrusion detection system to defend itself from attacks. Attacks on intrusion detection systems can take several forms. As a specific example, consider an attacker sending a large volume of non-attack packets that are specially crafted to trigger many alarms within an intrusion detection system, thereby overwhelming the human operator with false positives or crashing the alert processing or display tools. Axelsson [6], in his 1998 survey of intrusion detection systems, found that a majority of the available intrusion detection systems at that time performed very poorly when it came to defending themselves from attacks. Since then, the ability of intrusion detection systems at defending themselves from attacks has improved only marginally.

Lastly, an increasing problem in today’s corporate networks is the threats posed by insiders, viz., disgruntled employees. In a survey [88] conducted by the United States Secret Service and CERT of Carnegie Mellon University, 71% of respondents out of 500 participants reported that 29% of the attacks that they experienced were caused by insiders. Respondents identified current or former employees and contractors as the second greatest cyber security threat, preceded only by hackers. Configuring an intrusion detection system to detect internal attacks is very difficult. The greatest challenge lies in creating a good rule set for detecting "internal" attacks or anomalies. Different network users require different degrees of access to different services, servers, and systems for their work, thus making it extremely difficult to define and create user- or system-specific usage profiles.

Although there is some existing work in this area (e.g., [77, 94]), more research is needed to find practical solutions.