Classification Based On Positive And Negative

Published Date: 02 Nov 2017

Chapter 5

Association Rules indicates how strongly two sets of attributes are associated in the sense that how frequently they co-occur in a dataset. Its applications started with analysis of purchase patterns of customers, utility in decision making process such product placement, catalogue design, cross-marketing strategies. Frequent Itemset Mining is crucial in finding association rules. Association between a set of conditional attributes and any class label is useful for classification. Because association rules explore highly confident associations among multiple attributes, this approach may overcome some constraints introduced by decision-tree induction, which considers only one attribute at a time. In many studies, associative classification has been found to be more accurate than some traditional classification methods, such as C4.5. In section 5.1, we present associative classification. In section 5.2, we review traditional classifiers. In section 5.4, we propose an approach for association classification has been discussed.

5.1 Introduction

Association analysis, Classification and Clustering are three different Data Mining techniques. The aim of any classification algorithm is to build a classification model to the given input examples of the classes. The model we obtain can then be used to classify new examples or simply to achieve a better understanding of the available data. Classification is a two step process consists of training phase and testing phase. The set of rules will be generated during the training phase from the training data. The test phase is used to test the accuracy of the classifier. Different approaches have been proposed to build accurate classifiers, such as, naive Bayes classification, Decision Trees, and Support Vector Machines (SVM).

Set of Class

Association Rules

Classifier

Training Data

Classification

Figure 5.1: Associative Classifier

A new technique for classification called Associative classification has been proposed [6]. It performs classification by using association rules. These rules are straight forward and simple to understand. In associative classification, the model consists of class association rules where each rule consequent is restricted to a class attribute. Recent studies show that the approach has achieved higher accuracy than traditional approaches.

Associative classification technique consists of three phases namely rule generation, construction of classifier and classification and is shown in Figure 5.1. Rule generation phase will generate Class Association Rules (CARs), an association rule consequent is restricted to only one attribute called class attribute, by using association rule mining techniques. Classifier is constructed from the rules obtained in the previous step. Classification phase assigns a class label to the given object [83].

5.2 Traditional Classifiers

Classification is a classical problem extensively studied by statisticians and machine learning researchers. According to one definition classification is the separation or ordering of objects (or things) into classes. If the classes are created without looking at the data (non-empirically), the classification is called apriori classification. If, however, the classes are created empirically (by looking at the data), the classification is called posteriori classification. In most literature on classification it is assumed that the classes have been deemed apriori and classification then consists of training the system so that when a new object is presented to the trained system it is able to assign the object to one of the existing classes. This approach is also called supervised learning. Some techniques are available for posteriori or unsupervised classification in which the classes are determined based on the given data.

Data mining has generated renewed interest in classification. Since the datasets in data mining are often large, new classification techniques have been developed to deal with millions of objects having perhaps dozens or even hundreds of attributes.

A classification process in which classes have been pre-defined needs a method that will train the classification system to allocate objects to the classes. The training is based on a training sample, a set of sample data where for each sample the class is already known. We assume each object to have a number of attributes, one of which tells us which class the object belongs to. This attribute is known for the training data but for data other than the training data (we call this other data as the test data). We assume that the value of the attribute is unknown and is to be determined by the classification method. This attribute may be considered as the output of all other attributes and is often referred to as the output attribute or the dependent attribute. The attributes other than the output attribute are called the input attributes or the independent attributes.

In supervised learning schemes, it is assumed that we have sufficient training data to build an accurate model during the training phase. The model that we build from the training data is never 100% accurate and classification based on the model will always lead to errors in some cases. In spite of such errors, classification can be useful for prediction and better understanding of the data.

The attributes may be of different types. Attributes whose domain is numerical are called numerical attributes while attributes whose domain is not numerical are called categorical attributes. Categorical attributes may be ordered (e.g. a student’s grade) or may be unordered (e.g. gender). Usually, the dependent attribute is assumed to be categorical if it is a classification problem and then the value of the attribute is the class label. If the problem is not a classification problem, the dependent attribute may be numerical. Usually such problems are called regression problems. Obviously, the two problems are closely related and one type of problem may sometimes be converted to another type, if necessary, by simple transformation of variables from either categorical to continuous (by converting categories to numerical values which may not be always possible) or from continuous to categorical (by bracketing numerical values and assigning categories, e.g. salaries may be assigned high, medium and low categories). Binning of continuous data into categories is quite simple although the selection of ranges can have a significant impact on the results.

Classification has many applications, for example prediction of customer behavior (e.g. predicting direct mail responses or identifying telecom customers that might switch companies) and identifying fraud. In many applications, for example in assessing credit card applicants, a credit card company may already have sample data based on past applicants and knowledge about the applicants that were good credit risks and those that were not. A classification method may use the sample to derive a set of rules for allocating new applications to either of the two classes.

There are a number of classification methods. We are not able to discuss them all. We will only discuss the Decision tree, Naïve Bayes and Neural Network techniques.

5.2.1 Decision Tree based Algorithms

Mainly there are three popular methods for constructing decision trees. They are ID3 (Iterative Dichotomiser), C4.5(a successor of ID3) and Classification And Regression Trees (CART)[33]. ID3 and CART were invented independently of one another at around the same time, yet follow a similar approach for learning decision trees from training tuples.

The decision tree approach is most useful in classification problems. With this technique, a tree is constructed to model the classification process. Once the tree is built, it is applied to each tuple in the database and results in a classification for that tuple.

The ID3 technique, to build a decision tree, is based on information theory and attempts to minimize the expected number of comparisons. The basic strategy used in ID3 is to choose splitting attributes with the highest information gain first. The amount of information associated with an attribute value is related to the probability of occurrence.

The concept used to quantify information is called entropy. Entropy is used to measure the amount of uncertainty or surprise or randomness in a set of data. Certainly, when all data in a set belong to a single class, there is no uncertainty. In this case the entropy is zero.

The ID3 approach favors attributes with many divisions and thus may lead to over fitting. In the extreme, an attribute that has a unique value for each tuple in the training set would be the best because there would be only one tuple (and thus one class) for each division. A modified version of ID3 is known as C4.5. This approach uses the GainRatio as a measure to split attribute.

CART is a technique that generates a binary decision tree [33]. It uses a measure called Gini Index to perform splitting of attribute.

Attribute Selection Measure

An attribute selection measure is a heuristic for selecting the splitting criterion that best separates a given data partition, D, of class-labeled training tuples into individual classes. If we are to split D into smaller partitions according to the outcomes of the splitting criterion, ideally each partition would be pure (i.e., all of the tuples that fall into a given partition would belong to the same class). There are three popular measures: Information Gain, Gain Ratio and Gini Index.

ID3 uses Information Gain as its attribute selection measure. Let node N represents or hold the tuples of partition D. The attribute with the highest information gain is chosen as the splitting attribute for node N. This attribute minimizes the information needed to classify the tuples in the resulting partitions and reflects the least randomness or impurity in these partitions. The expected information needed to classify a tuple in D is given by

(5.1)

Where pi is the probability that an arbitrary tuple in D belongs to class Ci and is estimated by |Ci,D|/ |D|. Info(D) is also known as the entropy of D.

The amount of information needed after the partitioning is measured by

(5.2)

Information gain is defined as the difference between the original information requirement (i.e., based on just the proportion of classes) and the new requirement (i.e., obtained after partitioning on A). That is,

Gain(A)= Info(D)- InfoA (D) (5.3)

The information gain measure is biased toward tests with many outcomes. That is, it prefers to select attributes having a large number of values.

C4.5, a successor of ID3, uses an extension to information gain known as gain ratio, which attempts to overcome the bias. It applies a kind of normalization to information gain using a split information value defined analogously with Info(D) as

(5.4)

The Gain Ratio is defined as

(5.5)

The Gini index is used in CART. It measures the impurity of D, a data partition or set of training tuples, as

(5.6)

Where pi is the probability that a tuple in D belongs to class Ci and is estimated by |Ci,D|/|D|. The sum is computed over m classes.

5.2.2 Bayesian Classification

Bayesian classification is quite different from the decision tree approach. In Bayesian classification we have a hypothesis that the given data belongs to a particular class. We then calculate the probability for the hypothesis to be true. This is among the most practical approaches for certain types of problems. The approach requires only one scan of the whole data. Also, if at some stage there are additional training data then each training example can incrementally increase/decrease the probability that a hypothesis is correct.

Before we define the Bayes’ theorem, we will define some notations. The expression P(A) refers to the probability that event A will occur. P(A|B) stands for the probability that event A will happen, given that event B has already happened. In other words P(A|B) is the conditional probability of A based on the condition that B has already happened. For example, A and B may be probabilities of passing a course A and passing another course B respectively. P(A|B) then is the probability of passing A when we know that B has been passed.

The Bayes’ theorem:

P(A|B)=P(B|A)P(A)/P(B) (5.7)

Actually it is rather easy to derive since we know the following.

P(A|B)=P(A & B)/P(B) (5.8)

and

P(B|A) = P(A & B)/P(A) (5.9)

Dividing the equation 5.8 by the equation 5.9 gives us the Bayes’ theorem.

Continuing with A and B being courses, we can compute the conditional probabilities if we know what the probability of passing both courses was, that is P(A &B), and what the probabilities of passing A and B separately were. If an event has already happened then we divide the joint probability P(A & B) with the probability of what has just happened and obtain the conditional probability.

If we consider X to be an object to be classified then Bayes’s theorem (5.7) may be read as giving the probability of it belonging to one of the classes C1, C2, C3, etc by calculating P(Ci|X). Once these probabilities have been computed for all the classes, we simply assign X to the class that has the highest conditional probability.

Let us now consider how probabilities P(Ci/X) may be calculated. We have

P(Ci|X)=[P(X|Ci)P(Ci)]/P(X) (5.10)

P(Ci|X) is the probability of the object X belonging to class Ci.

P(X|Ci) is the probability of obtaining attribute values X if we know that it belongs to class Ci.

P(Ci) is the probability of any object belonging to class Ci without any other information.

P(X) is the probability of obtaining attribute values X whatever class the object belongs to.

Given the attribute values X, what probabilities in the formula can we compute? The probabilities we need to compute are P(X|Ci), P(Ci) and P(X). Actually the denominator P(X) is independent of Ci and is not required to be known since we are interested only in comparing probabilities P(Ci|X). Therefore we only need to compute P(X|Ci) and P(Ci) for each class. Computing P(Ci) is rather easy since we count the number of instances of each class in the training data and divide each by the total number of instances. This may not be the most accurate estimation of P(Ci) but we have very little information, the training sample, and we have no other information to obtain a better estimate. This estimate will be reasonable if the training sample was large and was randomly chosen.

To compute P(X|Ci) we use a naïve approach (that is why it is called the Naïve Bayes model) by assuming that all attributes of X are independent which is often not true.

Using the independence of attributes assumption and based on the training data, we compute an estimate of the probability of obtaining the data X that we have by estimating the probability of each of the attribute values by counting the frequency of those values for class Ci.

We then determine the class allocation of X by computing [P(X|Ci)P(Ci)] for each of the classes and allocating X to the class with the largest value.

The beauty of the Bayesian approach is that the probability of the dependent attribute can be estimated by computing estimates of the probabilities of the independent attributes.

We should also note that it is possible to use this approach even if values of all the independent attributes are not known since we can still estimate the probabilities of the attribute values that we know. This is a significant advantage of the Bayesian approach.

5.2.3 Neural Networks

Neural networks have emerged as an important tool for classification [33]. The research activities in neural classification have established that neural networks are a promising alternative to various conventional classification methods. The advantage of neural networks lies in the following theoretical aspects. First, neural networks are data driven self-adaptive methods in that they can adjust themselves to the data without any explicit specification of functional or distributional form for the underlying model. Second, they are universal functional approximators in that neural networks can approximate any function with arbitrary accuracy. Since any classification procedure seeks a functional relationship between the group membership and the attributes of the object, accurate identification of this underlying function is doubtlessly important. Third, neural networks are nonlinear models, which makes them flexible in modeling real world complex relationships. Finally, neural networks are able to estimate the posterior probabilities, which provide the basis for establishing classification rule and performing statistical analysis.

Although significant progress has been made in classification related areas of neural networks, a number of issues in applying neural networks still remain and have not been solved successfully or completely.

5.3 Problem Description

The philosophy behind the task of classification is to construct a classifier from the training data. The model is then used for classifying a new object. Classification is used in majority of the avenues of science and technology. Though there are many classification techniques exist in the literature and the history of classification is long, still there is a scope for further improvements of classification. The traditional classifiers are Decision Trees, Bayesian Classifier, Neural Networks, and Support Vector Machines. For the past two decades classification based on association rules has attracted researchers. The following are the steps involved in the process of constructing an associative classifier:

Rules Generation

Using an association rule mining algorithm, the set of class association rules of the form Xci are generated from the training data, where X is a set of conditional attributes and ci is a class label.

Rules Pruning

When the minimum support is set too low then the previous step produces large number of class association rules. Hence pruning is required. This phase is used to prune those rules that causes over fitting or causing errors in the classification stage.

Classification

The task of classification starts here. To have good prediction, ranking of rules obtained in the previous phase is done here. In associative classification, classification rules are generated by using association rule mining algorithms. All most all algorithms generate same set of association rules. There are different ways to perform classification. Some algorithms perform classification by considering average confidence for each category or by computing the weight for each category using chi-square or first matching rule on a set of matching rules.

5.4 Proposed Approach

We are proposing an approach to perform Classification based on Positive and Negative Association Rules which are known as Class Association Rules. Primarily Class Association Rules of the form X c are mined where X is a set of attributes and c is a category or a class of an object. Here X c need not be only positive association rule rather it may be a negative association rule. Then a classifier is constructed by considering strong rules, which is called an Associative Classifier. It takes an object as input then it attaches a class label for the given input.

5.5 Algorithm

In this section we introduce an Associative Classifier algorithm called CPNAR (Classification based on Positive and Negative Association Rules) to predict to which classes new objects are attached. Database (DB), minimum support (ms) and minimum confidence (mc) are the inputs to the algorithm CPNAR. It consists of 5 procedures namely PCR ( ), NCR1 ( ), NCR2 ( ), NCR3 ( ), and AC ( ).

PCR ( ) generates Positive Classification Rules of the form Xc,

NCR1 ( ) generates negative Association Rules of the form ┐Xc,

NCR2 ( ) generates negative Association Rules of the form ┐XYc,

NCR3 ( ) generates negative Association Rules of the form ┐X┐Yc, and

CNOPNAR ( ) is the actual Associative Classifier.

Here we are using Apriori-based implementation as they are simple and efficient to generate association rules. To determine the validity of ARs the support and confidence measures have been used.

Algorithm: CPNAR ( )

{

Call Procedure PCR ( )

Call Procedure NCR1 ( )

Call Procedure NCR2 ( )

Call Procedure NCR3 ( )

Call Procedure CNOPNAR ( )

}

Procedure PCR ( )

{

Pcr = Φ

Find L1- Frequent 1-itemsets

L = L ∪ L1

for ( K = 2; L K-1 ≠ Ø; K++)

{

/* Generating PCK */

for each l1,l2 ∈ LK-1

{

if(l1[1]=l2[1]^………………l1[k-2]=l2[k-2]^l1[k-1]<l2[k-1])

PCK = PCK ∪ {{l1 [1]…….l1 [k-2],l1[k-1],l2[k-1]}

}

/* Pruning using Apriori property*/

for each (k-1)- subsets s of I ∈ PCK

{

if s ∉ L K-1

PCK = PCK – {I}

}

/*Pruning using Support Count*/

Scan the database and find supp(I) for all I ∈ PCK

for each I ∈ PCK

{

if supp (I) ≥ ms

LK = LK ∪ {I}

}

L =L ∪ LK

}

}

/* Generating Positive Class Association Rules of the form I(=XY)  c*/

for each I(=XY) ∈ L

{

for each c ∈ C

{

if conf (I c) ≥ mc

Pcr = Pcr ∪ {I  c}

}

}

}

Initially the set Pcr (Positive classification rules) is empty. First it finds L1- frequent 1-itemsets. Line 4 - 25 generates all positive frequent itemsets. Line 6 -10 generates positive candidate itemsets (PCK). The generated candidate itemsets are pruned using Apriori Principle (Line 12 -16) and support count (Line 18-23). Line 29-34 generates positive class association rules by considering positive frequent itemset (I) and a class label (c). If the confidence of the aforementioned rule is more than minimum confidence (mc) then it is considered as a valid Positive Class Association Rule and will be included in Pcr otherwise it will be discarded.

Procedure NCR1 ( )

{

Ncr1 = Φ /* Negative Class Association Rule set 1*/

NL = Φ /* Negative Frequent Itemset*/

for each I ∈ L

{

if 1-supp(I) ≥ ms

NL=NL ∪ { I }

}

/* Generating Negative Class Association Rules of the from ┐I(=XY)  c*/

for each I ∈ NL

{

for each c ∈ C

{

if conf( ┐Ic ) ≥ mc

Ncr1 = Ncr1 ∪ {┐Ic}

}

}

}

Initially Ncr1 and NL set to Φ. It generates negative class association rules of the form ┐I (=XY)  c. First it generates negative frequent itemsets from positive frequent itemsets generated in the previous procedure PAR by finding 1-supp (I). If it is more than minimum support (ms) then it is included in NL. Otherwise, it is not included in NL. It is shown in line 3 - 7. For each I in NL and a class label c it generates a rule ┐I c. If the confidence is more than mc then it is included in Ncr1. Otherwise it is discarded. It is shown in line 9 -15.

Procedure NCR2 ( )

{

Ncr2 = Φ /* Negative Class Association Rule set 2*/

NNL = Φ /*Negative Negative Frequent itemset*/

NNC2 = {┐{i1 }┐{i2 }|i1 , i2 ∈ L1 , i1 ≠ i2 } /* Negative Negative Candidate Itemset*/

for (K = 2; NNCK ≠ Ø; K + +)

{

for all I = ┐X┐Y ∈ NNCK

{

if supp(I) ≥ ms

NNLK = NNLK ∪ { I }

else

{

for all i ∉ XY

{

/* Generating Candidates */

Cand ={┐ ( X ∪ { i }) ┐Y, ┐X ( ┐ Y ∪ { i } )}

/* Pruning Cand*/

for each item in Cand

{

if (X{i} ∉ L or ┐X1 ┐Y1 ∈ NNL where X1X{i} and Y1Y)

Cand= Cand – {XY {i}}

NNCK+1=NNCK+1 ∪ Cand

}

}

}

}

/* Generating Negative class association Rules of the from I(= ┐X┐Y)  c*/

for each I ∈ NNL

{

for each c ∈ C

{

if conf(Ic) ≥ mc

Ncr2 = Ncr2 ∪ { Ic }

}

}

}

Initially Ncr2 (Negative class association rules of second type) and NNL (Negative Negative Frequent Itemset) set to Φ. Line 3 generates Negative Negative candidate2 itemset. It produces candidate 2-itemset by taking two positive frequent 1- itemsets from L1 and then applied negation to each item. Line 4 - 25 generates all valid Negative Negative Frequent itemsets. Line 8 - 9 generates negative negative frequent itemsets. Line 15 generates negative negative candidate itemsets for the next level. It generates negative negative candidate itemsets by adding a positive frequent 1-item, i.e., by adding frequent itemset i to an infrequent itemset ┐ X ┐Y. We will obtain two negative candidate itemsets ┐ X{i} ┐Y, ┐ X ┐Y{i}. Line 17 - 21 performs pruning on the generated candidate itemsets. Line 27- 34 generates Negative Class Association Rules for Negative Negative Frequent itemsets obtained in the previous steps.

Procedure NCR3 ( )

{

Ncr3 = Φ /* Negative Class Association Rule Set3*/

NPL= Φ /* Negative and Positive Frequent Itemset*/

NPC1,1 = {┐{i1 }{i2 }|i1 , i2 ∈ L1 , i1 ≠ i2 } /* Negative and Positive Candidate itemset*/

for (K = 1; NCK,1 ≠ Ø; K + +)

{

for (P = 1; NCK,P ≠ Ø; P + +)

{

for all I ∈ NCK,P

{

if supp(I) ≥ ms

NPLK,P = NPLK,P ∪ { I }

}

/*Generating Candidates*/

for all joinable I1 , I2 ∈ NPLK,P

{

X = I1 .negative, Y = I1 .positive ∪ I2 .positive

I = ┐XY

if ((∄X1 X)(supp(┐X1 Y ) ≥ ms) and (∄Y1 Y )(supp(┐XY1 ) < ms))

NPCK,P+1 = NPCK,P+1 ∪ { I }

}

}

for all X ∈ LK+1 , i ∈ L1

{

if ( ∄X1 X)(┐X1 {i} ∈ NPL)

NPCK+1,1 = NPCK+1,1 ∪ ┐X{i}

}

}

/* Generating Negative Class Association Rules of the from ┐XY c */

for each I ∈ NPL

{

for each c ∈ C

{

if conf ( I  c ) ≥ mc

Ncr3 = Ncr3 ∪ {Ic}

}

}

}

/* I1 and I2 are joinable if I1 I2 , I1 .negative = I2 .negative, I1 .positive and I2.positive share the same k − 1 items, and I1 .positive ∪ I2 .positive ∈ L(P1 )p+1 */

The set of rules that were generated by procedures PCR(), NCR1(), NCR2( ), NCR3() represent the actual classifier. The discovered set of positive and negative class association rules are arranged in the decreasing order by confidence and support. This sorted set of rules represents the Associative Classifier (AC). This categorizer is used to predict to which class new objects are attached. To the given a new object, the classification process searches in this set of rules for those classes that are relevant to the object presented for classification. The following procedure called CNOPNAR (Classification of New Object based on Positive and Negative Association Rules) discusses the approach for labeling new objects based on the set of Class Association Rules that forms the classifier. The inputs to the CNOPNAR are Associative Classifier, Confidence Margin (CM) and a new object O to be classified. It produces a category attached to the new object.

Procedure CNOPNAR ( )

{

S = Φ /* set of rules that match o*/

for each rule R in the sorted set of rules

{

if ( R ⊂ O) /* O is an object which is to be Classified*/

{count++}

if(count = = 1)

fr.conf = R.conf /* keep the first rule confidence*/

S = S ∪ R

else if( R.conf > fr.conf - CM)

S= S ∪ R

}

Divide S in subsets by category: S1, S2,…. Sn

for each subset S1,S2, . . . . Sn

{

}

O = cj, where cj = max { ConfidenceScore1, ....... ConfidenceScoren }

}

In the above algorithm CNOPNAR (Classification of a new object based on Positive and Negative Association Rules), a set of applicable rules is selected in the lines 2-11 within a Confidence Margin (CM). The interval of selected rules is between the confidence of the first rule and this confidence minus the confidence margin i.e., [ R.confidence – CM , fr.confidence]. The prediction process starts at line 11. The applicable set of rules is divided according to the classes in line 12. In line 12-14 the groups are arranged according to the average confidence per class. In line 17 the classification is made by assigning a class that has the highest ConfidenceScore to the new object.

5.6 Experimental Results and Performance Evaluation

The execution of the algorithm is through the implementation in Java. The experiments have been tested using BREST, HEART, HEPATITIS, IRIS and ZOO datasets downloaded from UCI machine learning repository. To run the experiments, a ten-fold cross validation test has been used to compute the accuracy of the classifier. To discretize the continuous attributes, the technique used in CBA has been used. All the experiments are performed on a Pentium (R) Dual-Core CPU T400 @ 2.00 GHz with 4.00 GB main memory running Microsoft Vista. From the table 5.2, it is observed that, CNOPNAR algorithm has performed well for Heart, Iris and Zoo datasets when compared to C4.5, CBA, CMAR and CNOPNAR.

DATASET

#ATTS

#CLS

#REC

#Rules Generated

BREST

10

2

699

478

HEART

13

2

270

209

IRIS

4

3

150

123

ZOO

16

7

101

68

Table 5.1: Number of CARs Generated by CNOPNAR on Various UCI ML Datasets

DATASET

C4.5

CBA

CMAR

CPAR

CNOPNAR

BREST

95.0

96.3

96.4

96.0

96.6

HEART

80.8

81.9

82.2

82.6

83.0

IRIS

95.3

94.7

94

94.7

95.6

ZOO

92.2

96.8

97.1

95.1

97.5

Table 5.2: Accuracy of Various Classifiers on UCI ML Datasets

5.7 Summary

In this chapter we introduced four algorithms namely PCR, Ncr1, Ncr2 and Ncr3 to generate Positive and Negative Class Association Rules. The aforesaid algorithms used the existing support confidence frame work and did not use any additional measures. The proposed approach integrates Classification and Association Rule generation. It mines both positive and negative class association rules. It generates positive and negative class association rules with existing support-confidence framework. Experiments have been conducted on UCI datasets. All the approaches are implemented by their authors.

In our future work, (1) we will investigate the classification stage with reference to its scoring scheme in building an associative classifier. It is observed that though good rules exist, classification becomes inaccurate and it indicates some limitations of confidence averaging scheme. In this case, selection of right rules may improve the accuracy of classification. (2) Studying classification stage with reference to scoring scheme in building associative classifier. Though good rules exist, sometimes inaccuracy of classification is observed and it attributes to study the limitation of confidence averaging scheme. Selection of right rules may improve the situation. (3) We would like to avoid pre-discretization and introduce sophisticated alternative techniques to mine CARs. (4) Developing a new classifier using direct and indirect association rules may improve the classifier’s accuracy as the indirect association rules are providing additional information.