Effects Of False Positives On Csirt

Published Date: 02 Nov 2017

The aim of this paper is to focus on the much neglected issue of false positives in an organization. False positives are the false threat incidents which are assumed to be real threats in incident detection. These incidents need to be controlled as they will not only stop the real threats to be detected but also take a lot of time to be identified as false and will ultimately delay all the other operations. This topic discusses about what exactly are false positives, how do they occur, how such incidents affect the incident response team CSIRT and finally how it affects the organization’s well being. The measures to reduce such incidents from occurring and the ways to identify them are also covered here. Most organizations do not realize that big security incidents occur because of ignoring small incidents like false positives. This paper provides a deep explanation about false positive incidents and how important it is for an organization to identify them.

1 Introduction

An incident can be defined as a security hindrance in an organization. Incidents can be of various kinds either technical or business related. False positives are the incidents which are wrongly identified as actual incidents, such incidents mainly occur in technically related incidents. CSIRT (Computer Security Incident Response Team) is responsible for detecting and responding to the incidents in an organization. It is the foremost area which is concerned about the false positives as its entire operation is dealt with incident response. CSIRT has many other functions to carry out apart from only detecting and responding to incidents in such case dedicated groups are formed to deal with different functions and incidents, these groups involve in the routine operations given to them but none of them are worried about maintaining and reviewing any flaws or other abnormalities in the operations, here comes the trouble of false positives which is not even thought about. This is the major reason why we have to know more about such incidents before they become a troublesome issue. Most of the CSIRTs work on these issues only when they occur and do not plan for them beforehand due to the lack of understanding about these interruptions. For this we are going to make an attempt to dig in more about these incidents to enlighten the organizations as to what might happen if we do not pay attention to them and the measures we have to take to reduce them.

The motive behind this paper is to highlight the problem of false positives which has not been given so much of attention. The paper proceeds covering the following sections Section 2. False Positives. Section 3. Effects of False positives on CSIRT. Section 4. Organizational issues. Section 5. Control strategies. Section 6. Conclusion.

2 False Positives

As we have defined before, false positives are incidents which are detected as real threats but actually they are not real. In a CSIRT the incidents are reported in many ways, it could be through end users, help desks, administrators etc. but all network related incidents are detected by Intrusion detected systems (IDS), here we come across our concerned issue about false positives. These incidents occur mainly in IDS. When false positives are from other sources like a false incident response request email or a misleading phone call, they can be very easily detected as not real, but when they are from IDS it is difficult to find out if it is a false positive or a real incident. We have to first see how such incidents take place.

2.1 Occurrence

False positives occur mainly from IDS when it triggers an alarm for an incident which was never a threat, due to some wrong configuration or malfunction. Before we can discuss about the occurrence of these incidents we have to first see how an IDS detects incidents and how it functions so that we can have a better understanding about false positives as they occur from an IDS.

IDS is an alarm generating system when there is an intrusion in the network. Organizations prefer using IDS for their network, as they are capable of identifying the activities which go against the normal use of the network and can monitor network traffic very effectively. Some of the activities may be DOS attacks, insider attacks, thefts etc. IDS has two mechanisms to identify network traffic, signature based and anomaly detection. Signature based detection uses the mechanism to match signatures which are programmed in the system with the incoming traffic to detect an intrusion and anomaly detection uses statistical measures by evaluating normal network traffic and produces a baseline or a limit to compare the incoming traffic and detect malicious activity. We shall now see how false positives occur in both of these mechanisms.

2.1.1 False positives in Signature based IDS

Signature based IDS uses the mechanism of matching signatures of well known attacks configured in it with the data packets that it comes across to notify about the attacks. This mechanism is widely used as it depends on the signature matching and pattern matching system to identify attacks and not on the normal activities of the users and generates very few false positives. One might wonder how false positives can arise in this case? the answer is very simple if the pattern of another event is matched partially with the configured attack patterns in the IDS then false alerts are generated but this happens very rarely.

2.1.2 False positives in Statistical Anomaly Based IDS

Statistical anomaly based IDS detects the incidents by first taking the frequency of normal network activities and once it is ready with the statistics of the normal network behavior it creates a baseline, it means that it creates a limit to decide if an activity is legitimate or an attack, this limit is the baseline. Every activity on the network when passes through the IDS is first checked with the statistics of normal network activities and then compared to the baseline, if it is more than the clipping level that is the triggering level then an incident is detected. This IDS generates lots of false positives because if there is a slight variation in the normal network activities then an alarm might trigger even though there is no harm. This can be very annoying as there are many activities which can vary time and again, that is why this kind of IDS is not encouraged.

3 Effects of false positives on CSIRT

After seeing how false positives occur we now proceed to the concerned goal of the paper that is, effects of false positives on the CSIRT. In this section we are going to brief about the structure and services provided by CSIRT and next we see how the CSIRT structure is effected by false positives, what are the services majorly affected by such false alarms and how other inter dependent services run into delays.

3.1 CSIRT structure

CSIRT has the following four elements which it is built on in an organization they are 1.Mission statement 2. Constituency 3. Place in the organization and 4. Outer CSIRT Relationships. In the mission statement CSIRT first forms its main goal towards providing services to the organization. It frames the statement in accordance with the provisions expected by the organization and what the company exactly wants from CSIRT. The second element of a CSIRT is the constituency it works for. CSIRT constituencies can be of different kinds, a CSIRT can be only for users, an institution, a company, an organization or other national and international CSIRTs, it all depends on the mission statement of the CSIRT and the services it offers. In other words it can be internal or an external CSIRT. Internal CSIRT is works for a dedicated organization and an external CSIRT works for whoever is in need of it. The next element if it is an internal CSIRT is the place it takes in an organization, here the position of a CSIRT and what kind of authority it possesses on the constituency is considered. A CSIRT can have full authority on an organization; in the sense it has to obey whatever the CSIRT recommends it to do, for its security or it can have a partial or no authority at all, in this case the CSIRT can only suggest the management to improve security but it cannot make them do it. The fourth element is its relationship with other CSIRTs, it is important because it has to develop its relationships and trust so that it can grow. Here the CSIRT has to inform about the services it provides to other CSIRTs to make them clear as to what it provides and for what it has to be approached. It has to even understand what other CSIRTs are providing and has to develop trust by exchanging quality services.

3.2 CSIRT services

CSIRT provides three kinds of services reactive, proactive and security quality management. Most of the CSIRTs focus on the reactive services.

3.2.1 Reactive services: The reactive services are mainly concerned with how the CSIRT reacts to an intrusion alert or an IDS alarm. Reactive services are majorly alerts, warnings, incident handling, vulnerability handling and artifact handling. Alerts and warnings are provided as a service and all the new intrusions, vulnerabilities, viruses hoaxes are notified to the organization to make it prepared for incidents and prevent them. Incident handling is a service which involves the analysis, response and recovery of the incidents detected. It has four elements incident analysis, incident response on site, incident response support and incident response coordination. Incident analysis has the incidents and their origin examined, it also suggests the recovery methods for them. Incident response on site involves the CSIRT providing response on site, where the incident is reported from, and incident response support has the CSIRT supporting the site remotely and recovering the attack. Incident response coordination is a service which has the CSIRT exchanging information about its strategies and helping other CSIRTS and also taking their help. The vulnerability handling service examines the vulnerabilities reported, recommends recovery strategies for them. It involves vulnerability analysis, vulnerability response and vulnerability response coordination. A vulnerability analysis has the vulnerabilities analyzed, tracks their location and estimates their exploitation and provides solutions. Vulnerability response involves recovering the vulnerabilities and developing fixing methods and in vulnerability response coordination the CSIRT interacts with different experts, constituencies, CSIRTs, vendors and exchanges information about all the vulnerability updates and recovery strategies. Artifact handling is another service which examines the corrupt files that are used for an attack. It has artifact analysis, artifact response and artifact coordination as its categories where artifacts are analyzed, mitigated and others are informed about it respectively.

3.2.2 Proactive services: Proactive services are mainly concerned with long term security of the constituency. They help the organization to implement measures before an incident occurs to prevent it so that when they occur and are successful the impact is lowered. They are announcements, technology watch, security audits, configuration maintenance and development of tools and services, intrusion detection services. Announcements are basically made when there are any new attacks, vulnerabilities or any new mitigation strategies, technologies etc. to the constituency or other CSIRTs. Technology watch updates the latest technology in use, new attacks and their recovery methods to the constituency. Security audits are conducted as a service to check the security standards of the organization so that the needed security is implemented. Configuration, maintenance and development of tools and services is also provided as a service as most of the tools and services are not maintained and many security issues arise due to this. Intrusion detection services include incident detection, analysis of IDS logs, recovery.

3.2.3 Security quality management services: These services mainly focus on how to manage the quality of security provided and maintain the quality in the long run. They are risk analysis, business continuity and disaster recovery planning, security consulting, awareness building, education, product evaluation. Risk analysis makes the organization aware of the risks and their impact and prepares it to prevent them before they occur. Business continuity and disaster recovery planning is a service which plans the strategies to rebuild the business after a disaster. Awareness building enables the CSIRT to expose its constituency to all the latest security issues and the measures it has to take to improve its security quality. Education about the ongoing threats, vulnerabilities, viruses, technologies to the organization and training the members in this aspect is another very useful service provided by CSIRT. Product evaluation is a service where the CSIRT evaluates all the tools and services of the organization to check if they reach the standards expected by the organization or not and to certify them.

3.3 False positives effecting CSIRT structure and services

CSIRTs major role is to provide incident response service to its constituency and the main area from where it receives its requests is the IDS alarm and notifications. When there is any disturbance here then the entire goal of the CSIRT will be lost. False positives are the main disturbances which affect the normal functioning of the CSIRT. The mission statement of the CSIRT is the first thing which is damaged. The mission statements of different CSIRTs are of different kind but for any CSIRT has incident response as its main mission. When false positives occur the incident response will be delayed, quality of service and time is lost, this way the mission statement is not reached. The next element which is affected in the structure of a CSIRT is the constituency once the mission statement is not being fulfilled the constituency takes notice that the CSIRT is not functioning properly and loses its trust; this way CSIRT can lose its constituency which can majorly affect its status. The constituency might not entirely lose its connections with the CSIRT but it will definitely reduce itself from taking services from it and the position it gives to the CSIRT will definitely go down. The place in the constituency will be no more left and the advices CSIRT gives might be ignored. So if the CSIRT is having an authority on the constituency it will be simply reduced to a supporting aspect. Finally the external CSIRT relationships will be compromised as well, because if the CSIRT is functioning well it will develop good relations with others but if it is faulty it will not be approached for services or help. In this way the entire structure of the CSIRT will be ruined if false positives are not reduced.

Reactive services are the ones majorly affected by false positives as the IDS alarms are the main part of these services. Alerts and warnings as a service are affected by false positives as the CSIRT receives information about security issues from IDS and generates warnings based on that information. When there is false information the CSIRT alerts and gives wrong warnings to the organization thinking that there is a real need to notify and when the organization comes to know that there was no such threat it will lose its trust on the CSIRT. The incident handling service has most of its reports from the IDS notifications and the rest through other CSIRTs, users, mails etc. This service is extremely affected by false positives. When IDS detects an intrusion or something unusual CSIRT analyzes and responds to it, here is where the delay occurs if it is a false alarm. The CSIRT officer looks into the intrusion and analyzes it which takes some time and if it shows something which is an emergency alert the response team directly starts the response. Finally when they detect no such intrusion and realize that it is just a false alarm, it is too late and they might have already lost their time, other legitimate reports, money and a delay in other services. This is a serious issue with the constituency CSIRT works for and might lose its position. When intrusion analysis of false alarms are taken into consideration the true incidents have a delayed analysis, in the same way intrusion response of true incidents is delayed and the worst case is if the remote site has a false positive and the CSIRT goes all the way to the site for response it will not only waste time and money but also cannot respond to other remote sites which have true incidents. This way it would have a bad reputation from its customers; this in turn would affect its intrusion support and intrusion coordination services as no one would want to approach a CSIRT with bad functionality. It is the same with the other two services vulnerability handling and artifact handling but they are not dependent on IDS, they are only affected by the false positives produced by the tools used to detect them. These two services are affected indirectly by the false positives produced by IDS because if they are real incidents there will be a delay in responding to them; ultimately there will be a low quality service which will turn down the customers. In this way slowly CSIRT will start losing its reputation and also its business if it does not control false positive incidents.

3.4 Impact of false positives on interdependent CSIRT services

The proactive and security quality management services are dependent on reactive services. If the reactive services are not of good quality then this will also affect the functioning of proactive and security quality management services. That means false positives can indirectly cause harm to other interdependent services. We have seen the proactive services, they are for the long term development of the organization, when the reactive services are not functioning properly then the focus on the proactive services will be lost and thereby the security is lowered. The announcements will be reduced, not updated and with flaws; technical watch will also be deteriorated; Security audits will be conducted but not frequently as the focus will be on the reactive services, this will lead to delayed updating of security requirements and there will be many misperceptions regarding security; Configuration maintenance, development of tools and services will be given low importance, this will create additional problems like errors, interruptions, new vulnerabilities, in this way the burden on the CSIRT will increase; the main service of proactive services which is compromised is intrusion detection service. This is because it is affected by false positives directly as well as indirectly, in the sense it is directly affected if there are false positives in its intrusion detection system and is indirectly affected by the false positives in the reactive services. The CSIRT cannot handle both the services due to overload, as it has to not only correct the reactive services but also has to look into proactive intrusion detection service. In this way indirectly proactive service is neglected and because CSIRT has to put in additional effort to repair the service it will simply close the service. If this happens, the announcements, new intrusion updates, long term preparations will be submerged.

The security quality management services are the ones which will be totally ignored, even worse than the proactive services due to the burden of the reactive services on the CSIRT. This happens because the security quality management services can only be maintained if the core services of the CSIRT are being implemented properly and with a good quality, if there are false positives it is obvious that the concentration will be on the reactive services and CSIRT will not be giving the long term services any importance. The risks will be difficult to analyze as the legitimate reports are being lost and there will not be a right estimation of the risks and they will definitely not be analyzed properly in this way the risk analysis service will be affected by the false positives. Business continuity and disaster recovery planning service falls apart when CSIRT is busy with treatment of reactive services. This service is entirely for the future worst cases so this is totally neglected, even if this service is taken up it cannot do justice and will not be finished on time, if during this time a disastrous attack occurs and is missed by the IDS alarm due to false positives the organization and CSIRT will be in great danger. When the CSIRT does not have resources for reactive services it cannot think about continuing the awareness building service, anyway there will be nobody who will approach for this service even if it is provided because CSIRT is already not performing well with its core services so this service will come to a halt. Education and training services will also come to an end as there will not be sufficient time, staff and money with the CSIRT. The product evaluation service is also slowed down by the CSIRT due to the lack of resources, for any product evaluation there is a need for experience staff, time and tools when the CSIRT is not being able to provide all this then it cannot certify the product so therefore it stops the service.

All the above reaction on the interdependent services is due to CSIRTs negligence towards reducing false positives and developing reactive services.

4 Organizational Issues with false positives affecting CSIRT

CSIRT is the most important part of any organization, though it can have a security system within its own staff this would not be enough for all the security required by the organization, therefore a CSIRT is formed to reach all the require security by the organization. Any organization would look forward for a smooth functioning of all its departments that is why there is a CSIRT and when this does not function properly slowly other organizational domains are affected. Any organization typically has four main departments finance, human resources, management and technical and every department requires its own security to function properly and CSIRT is responsible for all the security and sometimes these departments become separate businesses when it is a huge organization and each department will have a separate CSIRT then security will become easy to handle but there will always be a dedicated CSIRT for the parent organization. When talking about the CSIRT of the parent organization only and assuming that there is no other separate CSIRT for each department, we see how important CSIRT becomes for the organization. When there are false positives the entire CSIRT’s functioning gets ruined and it affects all the organizational departments. All the departments need security in their own required levels and there are threats, new vulnerabilities and attacks almost every day, as all the departments have their important information stored on systems their work becomes more dependent on the security of the stored information. This dependency increases the responsibility of the CSIRT and it has to keep itself away from the false positives and protect all its services for the smooth functioning of the departments. Finance being the first important department of the organization handles all the financial details like sales, costs, budget, banking details, financial statistical information etc. when the systems holding this information are exposed to some intrusions and if the CSIRT does not respond to the intrusion on time or at all then this intrusion can become a disaster because this being the financial department there are more chances of disastrous attacks. Therefore CSIRT has to take steps to avoid this situation by taking measures against false positives. Human resource department actually does not need many CSIRT services but yes, it does need the basic services like the incident handling, alerts etc. to protect the vital information about the employees, recruitments, finance etc. and if something goes wrong and CSIRT is not available then there will be a loss of time resources money and information. Next is the department of management which is totally disturbed because it is responsible for managing all the projects, tasks, organizing meetings, and managing the functions of all other departments. When there are delays incurred in the rest of the organization this department has to bear the interruptions and will find it difficult to manage its tasks. In this way the management department is also affected with the improper functioning of the CSIRT. The technical department is the one which is majorly affected after the finance department due to the lack of quality services from CSIRT because of false positives as every task of it is based on the security of the system and if there are interruptions the work will come to a halt which will lead to loss. This department needs more attention from the CSIRT as it uses most of its services and this can make it more dependent on CSIRT so the best services are expected out of it. The other departments like logistics, marketing, production are also affected due to these delays and irresponsive CSIRT. In this way even the top organization can lose its reputation and public image in no time. The company can lose its connections with other organizations, even its stakeholders and customers. This can only become right with by forming a stronger dedicated CSIRT with good resources and less false alarms. These are few organizational issues concerning a company with bad CSIRT services.

5 Control strategies for false positives

This section is the most important for CSIRT because this is where we concentrate upon the strategies to control false positives from IDS in a network. Here we see almost four different methods to control false alarms and also few additional proposed strategies. The three main strategies are 1. IDS behind firewall 2. Tuning IDS signatures 3. Network analysis 4. Alarm filtering. Now we will see how each strategy solves false positives in its way.

5.1 IDS behind firewall

This is the easiest method which can control false alarms to some extent. It has a simple logic, the firewall is used to stop illegitimate traffic from entering the network so if the IDS is placed behind the firewall the legitimate traffic will pass through the IDS and if it produces an alarm the CSIRT can be sure that the traffic is legitimate and the alarm is false, but this is not always possible because, if there is something wrong with the firewall then illegitimate traffic can enter the network, the alarm might trigger and the CSIRT would be unaware as it would think it is an authorized data because the IDS is behind the firewall, so this can be a trouble so the CSIRT has to be very alert and keep monitoring the devices time to time.

5.2 Tuning IDS signatures

Tuning is a process where the IDS signature alert configuration is adjusted to reduce the false positives. This tuning is done according to how the IDS configuration is responding to legitimate traffic and why is it triggering a false alarm for legitimate traffic. In this process first the IDS is tested, when it generates a false alarm the configuration is changed by making it more insensitive to such traffic because these are the patterns of the required traffic and such traffic should not be obstructed.

5.3 Network analysis

In this method the IDS is analyzed for false positives under no security conditions that is it is placed before the firewall, now the IDS is exposed to all traffic illegitimate and legitimate. This time the network expert will analyze how the IDS is responding to traffic and to what legitimate traffic is it generating alarm according to this he changes the configuration. This will need lot of time, skill and experience to analyze.

5.4 Alarm filtering

In this method alarm filters are used to separate the malicious traffic and legitimate traffic. The network administrator will introduce the alarm filter after checking how the IDS will respond to traffic and after adjusting the configuration accordingly, so that the legitimate traffic can be separated correctly. In this way false positives can be controlled because if the alarm triggers when there is legitimate traffic it can be easily known that it is false as both the malicious and legitimate traffic are separated by the alarm filter.

There are many other proposed reduction techniques. These techniques are proposed according to two levels sensor level and after detection level. At the sensor level Mahmoud used a soft computing modeling technique, the FuzzyCognitiveMaps(FCM) which is based on the compensation of fuzzy logic and neural network. Cheung makes use of the agents and data mining technique to reduce the false alarms. Pi-Cheng proposed a scenario based technique by optimizing rule selection and attack identification during attack analysis. At the after detection level Abdulrahman proposed a method by classification of alert patterns into continuous alert pattern which are real alerts and discontinuous alert pattern which are alerts with noise. Njawa measured five attributes of quality of alerts and developed an intrusion alert quality framework, where these attributes are measured and the five threshold criteria is used to classify alerts once the total of these attributes is calculated. There are also other data mining methods to reduce false positives. Wenke developed a data mining framework for intrusion detection model which takes system audit data to improve the classifiers for efficient intrusion detection. Another method is to use automatic IDS for audit data for easy detection of attack patterns and reduction of false positives. Correlation techniques are also widely used to differentiate between different alerts so that it will be easier to detect and reduce false positives.

6 Conclusion

This paper provides an in depth discussion about the issue of false positives. Here we are made aware about how such incidents occur and what the consequences are if they are neglected. It intends to make the CSIRT and the organization think and act on the issue of false positives and makes them alert as to what can happen if they are ignoring this issue. The structure and services of the CSIRT are discussed; we also see how the false positives affect the quality of services of the CSIRT and how the interdependent services deteriorate due to this problem. The organization is largely dependent on the functioning of the CSIRT; this fact is discussed deeply as to what the organization can face if the CSIRT is offering low quality services due to false positives. Finally the measures to reduce the false positives are listed so that the CSIRT can implement different techniques to improve the quality of its services, to save other services and to serve for the betterment of the organization. The intention of this paper is to show false positives as a bigger picture to the CSIRT, so that it does not take the issue of the false positives easily and keeps itself alert, as it is made aware about not only losing its reputation but also damaging the organization.