Use Constraints For Corresponding Mutant Detection

Published Date: 02 Nov 2017

To calculate the appropriate mutation score it is necessary to check whether a mutant is equal to its program before starting a mutation testing. It is sad to say that this situation is not all the time possible by just taking a glance at the program’s structure. The procedure of solving the corresponding mutant issue by using a limited representation of the agenda and the mutant is introduced in this paper.

The paper consists of algorithms and results of first empirical. The concept is based on differentiating test subjects that has forced the agenda and its mutant to act in diverse directions.

1 Introduction

Here we are discussing about mutation testing methods. It is a practice which is used to examine the excellence of experiment suites as well as it is more competent than any other steps such as treatment based metrics(1). It is an error based system that utilizes properly explained sets of errors. The measured errors can be put forward as little variations of the real agenda that direct to an alternative of the plan

That is called as a mutant. The common proposal behind transformation examination is to check whether the presently accessible examination suite is able to diagnose the mutant or not. The transformed versions of the real agenda are executed under the accessible test suite for the purpose of figuring the mutant. The mutant is determined as dead or killed in the case of failure of test and if the executed test is successful then the mutant is considered as alive. One examination suite is much more effective than other one in case it able to detect more number of mutants. The credibility and quality is estimated by using the mutant score. The ratio between the quantity of mutants detected and the total number of mutants exclude the equal ones is called as mutation score. Generally the perfect mutation score is 1 which means that all mutants are effectively discovered. For further information on mutation testing then reader should follow (9).

A bigger trouble of transformation testing is the equal problem with mutant also. If there is no test case then mutant is called as equivalent as well as capable to distinguish between the production of the mutant and the yield of the real agenda. The discovery of all mutants is very important when mutation score definition is taken into consideration. In this paper we are focusing towards various techniques for equal mutant discovery and trying to point out methods of a tool used for the test. The author is using various mutation detection methods for serving the function and trying to diagnose problems and find out its solutions by utilizing tools and techniques.

_The research herein is partially conducted within the competence network Softnet Austria II (www.soft-net.at, COMET K-Projekt) and funded by the Austrian Federal Ministry of Economy, Family and Youth (bmwfj), the province of Styria, the Steirische Wirtschaftsf¨orderungsgesellschaft mbH. (SFG), and the city of Vienna in terms of the center for innovation and technology (ZIT).

C. Andr´es and L. Llana (Eds.): 2nd Workshop on Formal

Methods in the Development of Software 2012 (WS-FMDS 2012).

EPTCS 86, 2012, pp. 1–8, doi:10.4204/EPTCS.86.1

c S. Nica & F. Wotawa

This work is licensed under the Creative Commons Attribution License.

The author made an instrument that implements a statistical limitation algorithm. For understanding the comparable mutants the author had used limitations to know unproductive limitations that will produce equal mutants. The constraint which came in use will explain the conditions in which a mutant shall be detected that means if an examination case can discover the mutant, then the limitation system will be correct. Or else, no examination case is capable of killing the mutants and hence the same mutant is discovered.

A mutation is discovered when it is able to satisfy three situations:- reachability, sufficiency and necessity. Dissimilar from our approach, the author recommend three strategies to find out same mutants and they are: negation limitation splitting and constants evaluation. The author uses constraint negation for negation and half negation of two limitations for e.g. C1 and C2, again write one out of two limitations and evaluate them by comparison. In case they are same then the limitation system is unproductive and a mutant will be regarded as equivalent with this unproductive limitation system. However, control negation does not assist in shaping in case the necessary limitations clashes with the course expression. Moreover, for detecting clashes it is important to introduce limitations splitting. Both the limitations should use the format (V RelOp K), in which V is a variable and RelOp is a comparative operator and where K is a constant. The variables should remain the same in both constraints.

(15) The effect on executions and on the return values is utilized. The effect of mutations is examined by the approach at the coverage point. The exposure from the real execution is compared with the exposure of the mutants; however, the approach utilizes interpretations about the ways of code executed. The author regarded two types of dissimilar mutations keeping in mind the effect over the return values which eventually do not affect the exposure point. Therefore, mutations affect only the figures but do not influence the result values of an agenda as well as mutations are seen in the return values.

The author focuses on using constraints and its mutants in this paper by introducing certain approaches. We also talk about challenges and results that are very less because of applying approaches on timid programs.

In section II we are discussing the common definitions on particular topics and in section III we will focus on the introduction of constraint representation. In section IV the necessary algorithms are explained in detail and in section V we gave synopsis of our techniques, tools and shown the first empirical outcomes and in section VI we have concluded the paper.

2 Basic definitions

Now we will start this paper by discussing the common definitions which will be used at different levels. For the purpose of example we have used small agenda which is shown in figure1 (a). the program utilizes two natural numbers for multiplication. The essential programming language is not defined by us formally. A chronological programming is assumed without object oriented constructs. In starting of algorithm implementation we are not considering system calls and not even considering recursion. The common concept of similar mutants can be put into action to other languages which needs expertise limitation solver.

First of all we will start with the definition of test case that is responsible for the expected result behavior provided on an input.

In chronological languages the input and the outcome are known as variable environments. It is a mapping from a variable to the value. In the definition it is not compulsory that the principles of all output variables are mentioned.

Definition 1 [Test Case] We define a test case for a program P as a set (I;O) where I is the input variable environment specifying the values of all input variables used in P, and O the output variable environment if no output variable environment is specified, we set O to /0.

A test case which fail is known as a test case for which the resulted environment i.e. output is extracted from the program P and implied over input I, is not reliable with the desired output.

However, the test case is known as a passing test case.

The test case (f(a;1); (b;2)g; f(res;2)g) is a passing test case for the multiplication program from Fig. 1 (a).

A test suite TS for a program P is a collection of test cases for P.

Therefore, we establish the model of distinctive test cases.

Definition 2 [Distinguishing Test Case] specified programs P1 and P2. We say that a test case (I;O) is a unique test case, if the output variable environments combined by P1 and P2 with the same Input I is dissimilar. We now momentarily launch mutants and affirm the equivalent mutant difficulty.

Definition 3 [Mutant] specified a program [] and a statement SP 2 []. Let S0

[] be a statement that results from SP when applying changes like redefining a worker or a variable. Then program M is the mutant of program P with admiration to statement SP, acquired when altering S[] with S0[].

A mutant of program mult (Fig. 1 (a)) probably a program where Line 5 is altered to i = i + 2;.

Additional mutants can also be generated by varying constants, variables and other operators in the program. Now from here on we imagine that there exists a purpose that reproduces mutants by applying small transformations to the real program. Thus, a function gives a set of mutants that are significantly diverse to the original program. Regrettably, syntactical disparity does not promise that the mutant behaves in a different way to the original program.

Definition 4 [Equivalent Mutant] Given a program P, and one of the respective mutants M. We say that M is an equivalent mutant if semantically M reacts just like P. If we consider the distinctive test case definition for the equivalent mutant we will not identify a test case which is able to spot out the variance between the original program P and its parallel mutant, M.

The equivalent mutant trouble is a judgment problem that allows in deciding whether a program is behavioral matching to its mutant. This crisis is obviously equal to the program sameness problem that is well known to for non- deciding in general.

3 Representation of programs as constraints

To become self-reliant we shortly remember the alteration of sequential programs into their equivalent constraint illustration under definite assumptions. For the conversion of programs to their constraint representation we utilize prior work held in the constraint based on testing research are [3, 4, 6, 7]. Refer Figure 1 (a) as an example to display the conversion.

The first step of the alteration course is to remove all round statements. The idea here is to substitute a statement with a modified provisional statement of a pre-defined nesting the depth of nd; for sure the value of nd determine whether the unique program works corresponding to its parallel loop-free deviation. Once we choose a value for nd which is very small for a specified test case then the programs start acting in a diverse way. In contrast when selecting a big value for nd the consequential loop-free program becomes needlessly hefty. Therefore in exercising a trade-off for nd which is essential, collectively with means 4 Equivalent Mutant Detection

1. int i = 0; 1. bool loop 4 = false

2. int res = 0; 2. int i = 0;

3. while (i < a) f 3. int res = 0;

4. res = res + b; 4. if (i < a) f

5. i = i + 1; g 5. res = res + b;

6. i = i + 1;

7. if (i < a) f

8. loop 4 = true; g g

(a) The original program mult (b) mult after eliminating loops using a

nesting depth of 1

1. bool loop 4 1 = false; loop 4 1 = f alse;

2. int i 1 = 0; i 1 = 0;

3. int res 1 = 0; res 1 = 0;

4. res 2 = res 1 + b 0; res 2 = res 1+b 0;

5. i 2 = i 1 + 1; i 2 = i 1+1;

6. res 3 = Phi((i 1 < a 0), res 2, res 1); res 3 = F((i 1 < a 0); res 2; res 1);

7. i 3 = Phi((i 1 < a 0), i 2, i 1); i 3 = F((i 1 < a 0); i 2; i 1);

8. loop 4 2 = true; loop 4 2 = true;

9. loop 4 3 = Phi(((i 1 < a 0) and

(i 2 < a 0)),loop 4 2, loop 4 1); loop 4 3 = F(((i 1 < a 0)^(i 2 < a 0)); loop 4 2; loop 4 1);

(c) The loop-free variant of mult after SSA (d) The constraint representation of mult

loop

conversion

Refer Figure 1: A program mult for computing a _ b and its alteration to a set of constraint. Note that A and B are known as positive integers including zero and they are used as input variables, and variable res signifies the outcome for analyzing situations where nd is very small. We make sure this by introducing a new Boolean variable loopi for each loop, which is initialized with counterfeit and fixed to true whenever nesting depth nd is not big.

The loop-free program is transformed to its static single assignment form (SSA) In the second step [2].

In the static single assignment each and every variable is explained and defined only one time. Every variable is defined only once. We can make sure this by mapping each variable x taking place in a program to a variable x i. However, i represent an index variable starting from 0. Each time x is defined, the index i is amplified by 1 and supplemented to the variable. In case the variable is referenced after the current index, then i can be used until a new modified-definition of x takes place. Moreover adding indices there is a single additional alteration rule for conditional statements. Let us assume the following program fragment: if (x > 4) ( y = 0;)else ( y = 1;) After supplementing indices to the variable let us suppose the following situation: if (x 0> 4)

{ y 1 = 0;} else { y 2 = 1; } . In this case whether to use y 1 or y 2 after the conditional, when referring to y, is not defined. Therefore, in order to conquer this difficulty, conditional statements are exchanged with a F function. Now, the aim of the F function is to plan the variables explained in each of the branches of a conditional statement to the identical variable but with a fresh index that can be suggested after the statement. For our course section this supplementary rule leads to the following program:

y 1 = 0;

y 2 = 1;

y 3 = Phi((x 0 > 4),y 1,y 2);

Note:- if we suppose a function Phi accessible in the programming language representing the

function F , which is defined as follows:

F(C,x1,x2) ={x1 if C evaluates to true,

{x2 otherwise

It is also important to know that after the second step, the program comprises assignment statements only. Therefore in the final step of alteration we need to plan the assignment statements to equations (or constraints). This following step is responsible for taking care for the underlying constraint solver. In our case we use MINION [5]. Because of space limitations we do not talk about the mapping to MINION programs. Instead of that we use arithmetical equations written in italics for signifying the arrays, and we suppose a constraint solver that is able to compute a solution for a set of equations. So from here on we also think that a function convert ([[,nd) implements the conversion taking a program [] and a greatest nesting depth

nd as inputs. Figure 1 (b)-(d) depicts the conversion results step-by-step when applying convert to the multiplication program from Figure 1 (a) and using nd = 1 as maximum nesting depth.

4 The equivalent mutant detection algorithm

The common scheme behind the similar mutant detection algorithm is encouraged by the theory of differentiating test cases that are introduced for program debugging. The author in this paper is explaining that how different test cases can help to upgrade fault localization by stretching the accessible test suite. By computing differentiating tests the equivalent mutant problem can be solved. In case there is not test case that differentiates the program from its mutant then the program as well as its mutant has to be similar. However, it will be difficult to mention that two programs should be different with admiration to their respective behavior.

In preveious section constraint representation of a program is introduced and it is only behavioral similarity to the equivalent program for test cases that are not crossing the number of measured iterations in while statements. In pure fault localization no further test cases are taken place due to the number of iterations that has to be considered for producing the loop-free program which is known in prior notice as it is has less issues. One failing test case is necessary in fault localization that determines the amount of iterations supposing the program halts.

When using differentiating test case It is not favourable to determine similar mutants after the above observation. There are some consequences while distinguishing test case. We need to check whether respective test case is a viable test case or not. When differentiating test cases are feasible then programs are not similar i.e. equivalent. Therefore, for such purpose we have to find another distinguishing test case. The past computed input is not given permission to be computed any further thus, we can do this the above search of test case by adding this information.

Furthermore, there is a next difficulty because of the number of iterations measured throughout alteration. Due to chosen nesting depth the constraint solver returns back. Hence, we need to increase the nesting depth and research is required again. This is not possible every time so the nesting depth is set to limitations to a pre-explained maximum value. Below we have mentioned algorithm that implements the primary idea of disgnosing whether a mutant is equivalent to its similar program or not.

Algorithm equal Mutant Detection(P,M,nd, ndmax)

Input: A program P, its mutant M, the initial nesting depth nd, and the maximum nesting depth ndmax.

Output: true if P is equivalent to M and false, otherwise.

1. Convert the program into its constraint representation: CONP = convert(P;nd)

2. Let M0 be a program obtained from M by adding the postfix M to all variables.

3. Convert the mutant into a set of constraints:CONM = convert(M0;nd)

6 Equivalent Mutant Detection

Class LOC NoMut DetEqMut

tcas01 125 231 137

tcas02 125 231 139

tcas03 125 231 149

tcas04 125 231 142

. . . . . . . . . . . .

tcas25 125 - -

. . . . . . . . . . . .

tcas41 125 229 119

AVG 125 226 137

Class LOC NoMut DetEqMut

C432Order 382 2038 1341

ArrayOperations 101 496 11

BubbleSort 18 62 9

CalculateRectArea 14 18 2

CalculateRectPerimeter 13 21 4

CoffeeMachine 43 91 0

FindEvenOrOddNumber 15 34 3

FindLargestSmallestNumber 19 41 13

GcdATC 35 95 29

NumberFactorial 19 38 12

Table 1: EqMut Detection

4. Let CON be CONP [CONM.

5. For all input variables x of P, add the constraint x = x M to CON.

6. Let y1, . . . , yk be the k output variables of P. Add the constraint y1 6= y1 M_ : : : _yk 6= yk M to

CON.

7. Call a constraint solver on CON and let SOL be the set of solutions, e.g., mappings of variables to values that satisfy all constraints in CON.

8. If there exists no solution SOL, i.e., SOL = / 0, then P and M are potentially equivalent. In this case do the following:

(a) If nd _ ndmax, terminate the algorithm and return true.

(b) Otherwise, increase nd by 1 and go to 1.

9. Otherwise, there exists a (non-empty) solution SOL. If there is no variable loopj with an assigned value of true in SOL, then return false. Otherwise, add the information that the inputs computed in SOL are not valid to CON and go to 7. In the equa lMutant Detection algorithm lines 1–6 are for constructing the constraint system. In Line

5 constraints are added in order to force the input variables of the program and its mutant to be the same.

Line 6 is for stating that a distinguishing test case triggers at least one output variable to hold a different value after execution. Note also that the mutant variables are changed before conversion.

Line 7 calls the constraint solver.

In Line 8 it is handled the case where no distinguishing test case can be found. This either leads to the result that the mutants are equivalent or to an increase of the considered nesting depth and a re-computation.

Line 9 is for managing the case where a distinctive test case is found. There possibility is checked by testing whether the figure of iterations for computing a resolution has been exceeded. If the computed test case is not feasible, then search starts once more using the information previously gathered.

Note that the algorithm for all time terminates because of the introduced ndmax variable. Though, it can be the case that the algorithm returns true but there is a unique test case requiring the programs to be executed for a vast period. on the other hand, if the algorithm returns false, then the program as well as its mutant are certainly diverse with respect to their semantics. This restraint comes from the undecidability of the core equivalent mutant problem.

5 Empirical results

In order to authenticate the practicability of our approach we have to some extent implemented the equal Mutant- Detection algorithm, by neglecting increasing the bound of the nesting depth. This expansion will be implemented in the next version of our tool EqMutDetect which makes use of the MuJava. We aim to offer a reliable mutation test case generation tool that handles the equivalent mutant problem as good as possible. We developed EqMut- Detect in Java. We also modified MuJava in order to be JDK 1.5 compliant (see [11]). EqMutDetect comprises three main components: (1) Mutant generation using MuJava, (2) Equivalent mutants detection and reduction, and (3) Test case generation. The currently used mutation tool produces method level and class level mutations. However, in our research experiments, we excluded object oriented programs and generate only method level mutations, i.e., we make use of the method-level operators defined for MuJava, and mutate statements by replacing, deleting and inserting primitive operators, e.g., arithmetic

operator, relational operator, etc.

Using the implemented tool EqMutDetect we conducted first experiments. For this purpose we

applied EqMutDetect on some smaller programs varying from 13 to more than 380 lines of code. The used programs include the TCAS files [14], and programs that implement operations on arrays, and mathematical operations. The TCAS files were converted to Java syntax, in order to produce mutations with MuJava. In our experiments the nesting depths for representing loops vary from 2 to 5. For constraint solving we have customized a 5 minutes time bound. We executed EqMutDetect using the highest and lowest nesting depth, and noticed no dissimilarity concerning the achieved solutions. Consequently, we finish off that for the present experiments a little number of iterations is sufficient. In Table 1 we sum up the obtained results.

LOC denotes the lines of code, NoMut represents the total number of computed mutants, and DetEqMut is the number of equivalent mutants detected using EqMutDetect. We scrutinize from Table 1 that in a number of cases no corresponding mutants were generated, i.e., the constraint solver was always able to find a distinctive test case. We also come through the circumstances where the constraint solver tried to find a resolution within the pre-set time limit, but it did not handle. As a result we have terminated the search.

Concerning the number of results of our experiments, there are some fascinating findings. That around 40% of the mutants produced for the tcas files were found equivalent when using our method. This happens when we take mutations into consideration, which are accessible and have an effect on the return values. Moreover, concerning the tcas files we discovered conditions where we could not implement our approach. It is also interesting to note that for the largest program C432Order the constraint solver was able to provide solutions within the 5 minutes bound.

6 Conclusion

In determining the effectiveness of test suites the number of equivalent mutants has been reduced which plays an important role. When all equivalent mutants are removed, then mutation score is improved. Therefore, we have discussed about an approach for detecting similar mutants in this paper. Constraint representations of programs are combined with mutation testing in the current approach. In order to differentiate the program from its mutant we are searching for a differentiating test case. Because of the fact that the underlying problem is undecidable the approach does not guarantee to find a solution. The effect of particular parameters like the given nesting depth is abandoned for future research. As a derivative the approach allows for multiplying new test cases to the test suite. Computed distinctive test cases can be used to increase the mutation score.

In our recent experiments we have taken smaller programs for granted. As a result, the competence and scalability of the approach can be disputed. Thus, by using number of larger programs we can do our future research that will also include improvement of experimental bases. In addition, we also desire to widen the tool in order to handle object-oriented programs.