Hybrid Regression Testing Technique

Published Date: 02 Nov 2017

Test case reduction and

Fault detection.

Regression testing technique is better if it selects the smaller set of test cases that had 100% fault detection rate. There can be some compromise with the number of test cases if the difference is being limited to single or double digit but not with fault detection capabilities.

Table 4.4. Detailed description of results obtained

S.No.

Segment of Code

Type of Dependency

No. of Test Cases

(a)

Fault Detected (Yes/No)

(b)

Reason for (a)

(P1)

Reason for

(b)

(P2)

P1

P2

P1

P2

1

Path between variables in function Book( ) & added variable "fine".

D.D

04

02

Yes

No

Different Prioritization schemes.

Two faults out of 5 were identified since one is related to control structure and another was in unaffected parts.

C.D

06

04

Control Dependency exists in form of variable usage in control struct.

2

Path between variables in function Issue( ) and added variable "fine".

D.D

02

02

Yes

No

Yielded same set of test cases.

Three out of five went unnoticed.

C.D

06

00

Control Dependency not considered

3

Path between variables in the function i.e. Member_details( ) and added variable "fine".

D.D

04

03

Yes

Yes

Different Prioritization schemes.

All two seeded fault were identified since it was in variable usage.

C.D

08

04

Control Dependency considered to same extend due to some variable usage in control struct.

4

Path between variables generating output and variables dependent on it.

D.D

24

00

Yes

No

Unaffected statements are not considered.

Unaffected statements are not considered. So all five faults were not identified.

C.D

22

00

Unaffected statements are not considered

Redundancy in Test Cases

44

00

Redundancy only considered for deleted variables that has no usage.

Total Number of Test Cases

32

17

Basically 32 are what obtained after selecting test cases on basis of dependencies. Actual are 21 obtained after considering number of times path is executed.

Number of Faults Detected

Technique P1 detected 20 faults while P2 detected 10 out off 22.

P2 do not handles the unaffected statements and control dependent statements. Only those control dependencies are handled that has variable usage.

Where D.D refers to Data Dependency while C.D refers to Control Dependency. P1 refers to the proposed technique as presented in this proposal while P2 refers to that proposed in (Singh et al., 2010). The results obtained for two techniques for the same project of Library Management System, executed against the similar test suite, but against different test cases obtained, can be represented graphically as shown in figure 4.4.

Figure 4.4. Comparison results for two techniques.

Result Analysis

Figure 4.4 and Table 4.4 gives the details of results obtained after applying two regression testing techniques on a prototype project of the Library Management system. The proposed technique, P1 is better than P2 technique (Singh et al., 2010), Reason for the difference in reduction in the test case is that P2 technique selects only those test cases that were executing the changed variable and variables that are computed from these variables. Only these selected test cases were prioritized while in P1 technique, all the test cases that were part of the highlighted path were selected. Highlighted path is one that exists between the changed variable and other statements that is data or control dependent on it. Moreover, P2 technique does not consider the fact that unaffected statements might not execute properly. Due to this fact, P1 technique has some large test cases but difference in being limited to a few tens (since only few prioritized test cases executing control dependent paths and few executing unaffected statements are to be added).

Only test case reduction had no role to play in regression testing if it is not making a faulty program to fault when executed against selected test cases. Thus, scope of P2 technique is limited as it is not able to make program to generate faults. Thus, no doubt if proposed technique P1 is utilized, software testing team had to execute some extra test cases (the number being in single to double digit) but ensures better fault detection rate. The P1 technique finds the chromatic number of test cases that had a good likelihood of finding all errors housed in a program. Here, 32 test cases have not been further prioritized based on values of dependency and a number of times path (and hence statements) are executed. Thus, if further steps are employed, the program will be executed with 21 test cases. Execution with 21 test cases, executes 20 faults, thus leaving 2 undetected. But, such undetected faults will have no effect on quality of software since they are being used very less. This leaves difference between the number of test cases between the two techniques P1 and P2 as 5 but with the huge difference in fault detection rate.

Hybrid Regression Testing Technique: Based on Requirement Priorities, Fault & Modification History.

Introduction

Regression Testing is the effortful activity that had been the focus of various researchers. This activity aims to ensure that the non modified parts of source code are not affected as a result of changes in other parts. The relationship between requirement prioritization and regression testing is well established in the thesis. This work is an extension of the previous multilayered hybrid regression testing approach with the following changes:

This proposed technique provides the detailed algorithmic implementation of the multilayered technique proposed earlier.

This proposal does not employ dependencies as the regression testing based requirement prioritization approaches will ensure maximum dependencies between the delayed and already implemented requirements. For dependencies between implemented and new requirements, this proposal could be employed.

This modified proposal is evaluated on live software system and results proved to be very promising.

The proposal groups the clusters depending on the similarities between the requirements from implementation point of view. It is quite possible that the one requirement is light modification of the other (example, AHP and hierarchical AHP). In such scenarios, the testing of the one reusing almost entire code of the latter is wastage of time and effort. The testing of incrementally developed software employs the following steps.

Test the first increment of the software. To do this tester must generate test data, which must be carried out using effective test generation techniques since this test data will be reused to retest the next increment that is simply the modification of earlier increment. Thus quality of this test suit will not only determine the quality of the first increment, but also that of next increments. Here pair wise testing technique of test case generation can be employed that is optimized and executed on program.

Perform the implementation of next prioritized set of requirements. This set will comprise requirements that were delayed for next increment together with the changed requirements and new ones. Thus the delayed requirement set needs to be re analysed and whole set needs to be re prioritized so as to find out the set comprising delayed ones with changed and new requirements. The reprioritization of these requirements complicated the decision making process.

Perform testing of the modified code. Since one simple strategy will be to re execute the available test cases but this is very costly to perform. Thus tester must select the test cases from the available ones that test the modified program by executing the modified part of source code and the other parts that are directly or indirectly influenced by the changes. Thus regression testing will be employed by selectively selecting the test cases from test suite. This is beneficial since the sub set of test cases represent the pair wise test cases (since tester is choosing from same cluster) thus it is expected that modified program is re tested while accounting for ripple effects. The previous history of modules like number of changes, the number of faults caused by them, is used as one of the selection criteria.

The step 2 and 3 founds the foundation for evolution of the software. Thus these two steps are executed until all the functionality has been delivered and software is lifted from the market. Thus regression testing needs to be done to test the modified code. One of the strategies for regression testing is the hybrid approach. This approach is the combination of selection and prioritization approaches. Hybrid approaches are proposed by various researchers as in (Aggarwal et al., 2004; Malhotra et al., 2010; Singh et al., 2010). The proposed approach is basically a hybrid approach that performs selection and prioritization at various levels by using three parameters like (a) priorities of requirements, (b) changes and (c) faults resulted. The aim is to take advantage of sequences of selections followed by prioritizations by keeping in view the dynamically changing history of the modules of the incrementally developed software and requirement priorities. The proposed technique is well suited if regression prioritization based on the effects on regression testing as proposed in the chapter 6 and the priorities are changed with respect to time by employing the techniques as proposed in chapter 7 of the thesis.

Proposed Approach

The proposed approach is based on selectively selecting the test cases from clusters belonging to particular "Implementation Slice of Requirement", provided the modified code had direct or indirect impact on the above slice. Once the set of test cases are selected, two main parameters are employed i.e. Fim which denotes number of times the module (m) belonging to requirement (i) has caused faults in software; Mim denotes number of times module (m) is changed since it was delivered. The test cases are prioritized two times in this approach, one during impact analysis of modified code on rest program part and another by selecting those with higher values of Fim and Mim. The regression test cases will ensure that the test cases associated with impacted statements of the highest priority requirements gets executed first such that the module to which it belongs has caused many failures and changed several times. Proposed approach provides a method to test evolving software, and explains in detail that how software can be tested starting with its first increment. Thus the algorithm can be divided into two parts: first part tests the first increment and second part tests the further increments of the software. Following are the steps taken during the testing process:

PART 1: To test first increment, software testing team should generate test data, which must be carried out using effective test generation techniques. Quality of test generation methods determines the quality of increments since the test data will be reused to retest the next modified increments. It’s better to generate test data using optimized pairwise testing technique. General pairwise testing technique is not feasible and thus needs to be augmented with optimized methods. This means that the pair wise testing technique itself is based on some prioritization and selection criteria’s that are employed for making decisions while generating test cases. In the cases where the generation of test data using pairwise testing is not feasible, other designing techniques could be employed. The test cases that belongs to same "Implementation Slice of Requirement", forms a cluster. Thus cluster houses the test cases that test the implementation code for the given requirement. In this testing phase of first increment tester must create and maintain various types of information’s that includes the following:

Execution Slice for each requirement.

Cluster information.

Coverage Information i.e. Test coverage arrays.

Fim: Number of times the module (m) belonging to requirement (i) has created faults or failures during execution of the software.

Mim: Number of times the module (m) belonging to requirement (i) has been modified i.e. changed due to change in the working environment of software or due to faults or changing user requirements.

Last two parameters are updated after very regression testing activity on evolving software.

PART 2: To test further increments of software, the tester should reuse subset of the test suite for considering ripple effects and design new test cases for executing new requirements. To reduce the size of the test suite, one approach that could be employed is to take into account the direct as well as an indirect impact of the modified statements and hence variables in such statements on rest of the program. Variable usage and definition had been employed for impact analysis and regression testing in (Gupta et.al, 2010a, 2010b; Aggarwal et al., 2004; Malhotra et al., 2010; Singh et al., 2010).

Let us denote the set of test cases as T, which is a subset of test cases belonging to same cluster i.e. T={ {T1i, T1j,......, T1k}, {T2i, T2j,......, T2k},...., {Tpi, Tpj,......, Tpk} }. Here T1i denotes the test case that belongs to cluster 1. Thus in general Txy denotes the test case belonging to cluster x and numbered y in the original test suite of cluster x. To analyze the direct or indirect Impact, Feature allocation technique powered by analyzing "variable usage" and "requirement implementation" maintained is proposed. The two types of impacts are discussed as under:

Direct Impact: A module (m) and modified portion of the program (L’) is said to be directly related or L’ having Direct Impact on "m" if the variables used in modified part (either used or defined) is being transmitted to the module (m) either by using call by value or call by reference. Variable that defines control dependency between module (m) and program portion (L’) could also be taken as a catalyst for the direct impact.

Indirect Impact: A module (m’) is said to be indirectly related to the modified portion of the program (L’) if there exists some module (m) such that there is Direct Impact of L’ on module (m) and module (m) has a direct impact on module (m’). Same thing hold if any or both direct impacts are due to control dependency induced by any variable.

Once such impact is analyzed, effected module statements collected, and test cases that execute these statements are organized into the set, the tester must further prioritize this set such that only most suspicious test cases are selected from the set. Each cluster is housing test cases belonging to the same requirement. For the test cases that executes those statement belonging to requirement j influenced by statements of requirement i, test cases of requirement j will form cluster j. Test cases will always be categorized in clusters depending on the factor that to which requirement they belong. A Test case is suspicious if it has high value of Fim and Mim. The module that had been changed large number of times and had caused large failures must needs to be retested first. The above parameters have values associated with modules and not with test cases. This means that the tester must allot the values to test cases on the basis of its association with the modules. But in all this process the priorities of requirement plays an important role. The test cases belonging to same cluster will be further prioritized on the basis of the priorities of the requirements to which the cluster belongs. The highest priority requirements not affected by modifications are simply ignored. The steps to be taken in sequence are as given below.

Perform the impact analysis so as to find out the statements that are effected as a result of change.

Choose the test cases that execute above statements. The identified test cases for every requirement are clustered into same set.

Choose the cluster that belongs to requirement of higher priority and further prioritize the elements i.e. the test cases of this cluster that has higher values of Fim and Mim.

Repeat Step 3 for all the requirements in accordance with the priorities and until resources is available. The impacted clusters that are not associated with highest priority requirements and don’t have any involvement in large failures and modifications can be ignored for testing. If the resources allow, test cases associated with the lowest priority requirement could be targeted.

The priority of requirements can be calculated according to the technique formulated in earlier work on requirement prioritization as in (Gupta et al., 2011a). During software evolution, the module changes due to fault fixing or changing user requirements. The requirement, to which this module belongs, gets increment in the priority. This increment in priority helps to give preference during testing to the cluster to which this cluster belongs. The alteration in the priority is computed using the techniques as proposed in chapter 7 of the thesis.

The pseudo code for the above testing approach is as under:

Incremental_Testing (RP, MI, VUI, RII, M, F): This algorithm selects the prioritized set of test cases that are categorized into clusters, from the original test suite in accordance with the effects of the changed program code on Implementation Slices of the requirements. Requirement priorities are the selection criteria of the clusters while values of Fim and Mim (elements of set F and M) are used as prioritization criteria inside the cluster. Thus algorithm generates output by performing Inter-Cluster and Intra-Cluster prioritizations. RP is the 1-D array that stores the priorities of "N" requirements such that first location of array i.e. RP[0] represents priority of requirement R1 while RP[N-1] represents the priority of last requirement i.e. Rn. MI is module information array that stores the total numbers of modules in the requirement. Thus first location i.e. MI[0] specifies modules in requirement R1. VUI is the variable usage information table that stores details of variable usage and definition for every occurrence along with the modules that implements the particular requirement. To access particular module information associated with a requirement, it is required to access MI array to know how many modules makes a requirement. Finally for M modules, there are M rows of VUI array given for each module. RII is requirement implementation information table that stores the implementation details of requirement i.e. information of information slice of the requirements like modules implementing requirement, statements implementing such modules. M denotes the set of values that represent that how many times a module associated with a requirement has been modified. F denotes the set of values that represent that how many times a module associated with a requirement had resulted in failures. Both M and F are 2-D arrays with first represents the values associated with modules of requirement R1. Thus M[1][5] represents the value associated with module number M5 that implements requirement R1. Finally the test cases that executes module M5 that implements requirement R1 is allotted this value.

Input: RP, RII, M, F.

Output: T’, set of prioritized test cases belonging to different selected clusters and clusters as the classifying criteria.

Method:

NRows_MI( )

/* Rows_MI( ) returns the total number of rows i.e. the total number of requirements that are implemented in the software. Total rows of MI array gives total requirements to be implemented. */

FOR every statement x in S

DO

S’ Extract_variable_info(S)

/* Here S is the set of modified statements i.e. statements that are added, deleted or modified. S’ is the set of changed variables in the modified statements since such variables will be analysed so as to find their direct or inDirect Impact on other program parts. Extract_variable_info( ) is the function that extracts the variables from the statement.

END

END

FOR every requirement Ri from i=1 to N

DO

PExtract_moduleinfo(i)

//This functions returns the value stored at location i

// Thus the output is the total number of modules for requirement i.

FOR every element x in S’

DO

FOR j=1 to P //Total number of modules

DO

EffectedExtract_Effected_variables(RII[j][3], RII[j][4])

T[j]Extract_testcases (Effected)

/* Extract_Effected_variables( ) function takes as input the information of variable usage and definitions for a module identified by value of j that implements requirement identified by i. Effected is the array that stores the variables that matches with the modified statements variables and thus directly or indirectly related to them. Extract_testcases takes the effected array as input and generates the test cases that executes such variables belonging to module j of requirement i. */

END

END

/* create cluster of the test cases that belongs to modules of same requirement i.e. same implementation slice of requirement. But there may be duplication of test cases in the cluster, thus first remove duplicates from array T[ ], then combine the test cases into clusters. */

FOR j=1 to P

DO

Cluster[i]Make_set(T[j])

// Make_set( ) adds the test case to the set cluster.

Cluster[i]Remove_duplicates(Cluster)

// Remove_duplicates removes the duplicate test cases from the cluster and thus cluster for requirement as specified by value of i gets its selected test cases.

END

END

END

END

/* Group the clusters depending on the similarities of the two requirements. Let’s consider that Ri is superset of Rj in terms of functionality. The new cluster will house all the test cases belonging to the cluster of Ri and only those that are unique in R2. R1 and R2 could be already implemented, changed, delayed or newly arrived ones */

FOR j=1 to P

DO

Cluster[i]Cluster[i] U { Cluster[j]-Cluster[i] }

// This yields the minimized set of test cases.

Priority_Cluster[i] = Priotiy(Cluster[i])+Priority(Cluster[j])

//This adds the priorities of two requirements

END

END

END

END

/* Create final cluster, this Final_cluster is the set of sets i.e. members of this set is the set of test cases that are selected and organised into same cluster if and only if they belong to same cluster i.e. implementation slice of requirement. */

FOR i=1 to N

DO

Final_Cluster[i]Make_set(cluster[i])

END

END

/* Prioritize the test cases within the cluster such that such Intra-Cluster prioritization performs the selection of test cases from cluster and prioritizes the clusters on basis of requirement priority. */

FOR i=1 to N

DO

LocExtract_max(RP, i)

/* Here function Extract_max( ) fetches out the location of requirement of ith highest priority since RP is 1-D array and Final_cluster first location contains test cases clusters of requirement R1. The second argument to this function i.e. i represent the number of runs of this function i.e. for i=2, it finds out 2nd larget value from the set. */

Tcluster  Extract_cluster (Loc) // Extracts the cluster at location Loc

Tcluster  Extract_clustertestcases (Tcluster, Fim, Mim)

// Selects the Suspicious Test cases from same cluster in each iteration.

/* Extract_clustertestcases( ) extracts the test cases from cluster that has the value greater than Fim, Mim) */

Tregression_cluster[i]  Tcluster

/* Tregression_cluster is the final set that contains the set of test cases that has values greater than Fim, Mim and forms same cluster if they belong to same implementation slice, and contains all the clusters but one must execute them starting with location=1 i.e. Tregression_cluster [i] has clusters arranged in descending order of their priorities. */

END

Tregression_cluster[i]  Tregression_cluster ∪ Tnewr_req_cases

/* Regression testing test suit set combined using Union operation with Tnewr_req_cases test cases. This final test suite now covers test cases designed to test new requirements. */

END

This is the algorithm execution of which results in creation of Tregression_cluster that contains the test cases categorized by clusters. Finally starting with the starting cluster (highest priority), test cases are executed and next cluster is chosen and so on. This process if carried out as decided by resources that testing team had. Clusters itself contains the selective test cases and final prioritization cluster as given by Tregression_cluster, even contains selective clusters. These clusters are even prioritized as they are arranged in descending order of their priority. Thus the testers have now to execute the modified code with minimum number of test cases and thus lots of efforts can be saved.

Layered Approach

The proposed approach is a layered approach where the top most layers provides services to lower layers to achieve the objective of regression testing. The whole process of testing is divided into 4 stages or 4 layers yielding the hierarchical model. The top most layer performs its intended work and supplies its output as input to the next layer which does the same with next layer till the 4 layers has accomplished its work of regression testing.

C:\Users\varun\Desktop\V.jpeg

Figure 4.5. Layered Approach

The layered hierarchical model of hybrid regression testing is shown in Figure 1. The activities performed by each level are explained with the help of pseudo code as given in earlier sections. The details given here is just for quick up session.

Level 1: This level corresponds to the activities related to the feature allocation, defined as the problem of mapping the features of the software to its source code. In this proposal, the feature allocation refers to identification of the modules and hence the statements that makes up or implements individual requirements. This gives complete understanding of the code and hence simplifies the process of impact analysis. The changes made to one part of code may impact other parts due to existence of dependencies. The dependencies are of two types namely Inter module and Intra module dependency and can be though as inter cluster and Intra cluster dependencies. Such dependencies arises due to data and control dependencies resulting in two types of impacts i.e. direct impact and indirect impact

The feature allocation enables the tester to maintain various data structures like MI array, VUI array etc, which contains the code artefacts details for performing this process. The tester executes only the test cases associated with the statements or modules impacted by the changes and are related to the requirements of highest priorities. The inter cluster dependencies exist when changes in module effects only those modules that are in same implementation slice of requirement. For another situation, the dependency is Intra cluster dependency. The proposed feature allocation and impact analysis takes into consideration both types of dependencies and effects.

Level 2: This level corresponds to the Inter cluster prioritization. This level selects the test cases executing the statements belonging to particular module associated with particular requirement, affected by the changes. In other words, the changes should be linked with the statements by above mentioned dependencies. New cluster is created so as to keep the old cluster available thus supporting both general and version specific test case prioritization. The new clusters will contain the selective test cases executing affected statements.

Another important activity deals with the merging of the similar clusters. For example, in the live case study, the requirement to implement AHP and another requirement to implement Hierarchical AHP are almost similar to each other from implementation point of view. Few parameters like different in case of latter requirement. In such scenarios, it is better to reduce the size of test suite to minimize the efforts.

The activity of prioritization at this level is confined to higher level of abstraction i.e. at cluster level not at thinner level of abstractions i.e. within cluster. Clusters belonging to the requirements of highest priorities are selected first.

Level 3: This level corresponds to the Intra cluster prioritization. This level considers each cluster one by one on the basis of the priorities. For each cluster, the most suspicious test cases are selected. A Test case is suspicious if it has high value of Fim and Mim. In this level, the tester selects the test cases that belong to high priority clusters on the basis of values of Fim and Mim. The test cases designed to test new requirements are added to this regression testing test suite so as to test new requirements thereby making sure that they are properly executed and implemented.

Level 4: The tester executes all the test cases belonging to the highest priority cluster. This process is continued until maximum highest priority requirements are tested and resources allow the further testing of remaining clusters.

These four clusters form the proposed testing approach. This approach is suitable for both general and version specific test case prioritization.

Efficiency Metrics

In order to prove the efficiency of the approach, proposed technique has selected the two parameters that determine the efficiency of the proposed approach. The two parameters are:

Percentage savings in effort: Let’s consider that T denotes the set that contains the test cases that tests the old version of the software and Tregression_cluster is the regression test suite. Suppose that the test suit (T) contains X clusters, with each cluster containing maximum Y test cases and minimum 1. Thus maximum test cases in T will be XY while minimum will be X. Assume that the final test suite has X’ clusters (X’ can be X or even lesser) with each clusters having maximum Y’ elements and minimum of 1 test case. Then maximum test cases will be XY’ and minimum will be X. Thus maximum savings in effort will be XY-XY’ while minimum savings will be X-X’. Thus total savings is the test cases that are not executed at all from the available ones. The percentage savings are denoted by Tsavings (%) and given by equation 4.1.

Tsavings (%) = ((XY-XY’) / (XY)) * 100. (4.1)

This percentage of efforts savings is an indication of how much time, cost and hence effort is minimized in performing the regression testing.

Number of fault detected: This is most important metric since how effective and useful is first one, is determined by this metric since percentage savings in efforts will be useless until and unless it is able to identify all the faults. The proposed technique is evaluated not only on the basis of saving in test cases but also on ability to identify the faults.

Efficiency of Approach

This section illustrates the effectiveness of the proposal on hypothetical example. It had been discussed earlier that the tester must identify the statements involving the changed variables so that both direct and inDirect Impacts could be analyzed. In case of new requirements, the test cases will be added to test suite and will be executed without being subject to any prioritizations. This technique relies on the maintenance of various data structures like RII table and MI table as given in Table 4.5 and 4.6.

MI table

Table 4.5. Module Information Table

S.No

Requirement Number

Modules

1

R2

a( ), b( )

2

R4

a( ), c( )

3

R6

a( )

Here total number of rows of this table represents the total requirements that are implemented in the software. Hence provides an upper bound on total clusters to be prioritized.

RII table

Table 4.6. Requirement Implementation Information Table

S.No

Requirement Number

Modules

Statements

Variable Defined

Variables Used

Test cases

1

R2

(cluster 1)

a( )

S1, S2, S3, S4, S5, S6

a, b

a, b, x, y

T1, T2, T3 (T3 for x and y)

b( )

S7, S8, S9, S10, S11, S12

x, y

x, y

T4, T5, T6

2

R4

(cluster 2)

a( )

S1 to S6, S13 to S19

a, b

a, b, p, q

T1, T2, T3

c( )

S20 to S33

p, q

p, q, x, y

T7, T8, T9

(T8 and T9 for x and y)

3

R6

(cluster 3)

a( )

S1 to S6

a, b

a, b

T1, T2, T3

Let’s suppose that the variables x and y encounters a change. Suppose that new requirement R10 is added in form of code and three test cases i.e. T10, T11, T12 are created to test this requirement. Now the following are the orderings generated by prioritizations and selections:

For Intra cluster prioritization, for cluster 1, the test cases T3, T4, T5, T6 (they execute x and y) are selected. For cluster 2, T8 and T9 are selected since changes of x and y don’t have any impact on module a( ) but have effects on module c( ). For cluster 3, nothing is selected since changes in x and y doesn’t have any impact on variables a, b.

Now there are two clusters associated with two requirements i.e. R2 and R4. Suppose that requirement R2 has highest priority. Thus tester will choose cluster 1 followed by cluster 2 as the tester is left with two clusters only that could be executed within the resource limitations.

The test cases to be executed are T3, T4, T5, T6, T8, T9, T10, T11, and T12. Thus only 9 test cases out of 15 (including the multiple occurrences of test cases in test suites) are to be executed. Thus percentage saving is given by:

Tsavings (%) = (6/15) * 100

Tsavings (%) = 40 %.

Since it is only a hypothetical assumption on test cases, modules and variable usage, real benefits will be proved when applied on a case study (Section 4.4.6).

Case Study of Live system

To evaluate the proposal, it had been applied on the live project of "Tool for automatic analysis and comparison of different release planning methods" developed in Computer Programming laboratory at Jawaharlal Nehru Govt. Engineering College. The reason for the selection of this software lies in the fact that the requirements of the software are prioritized as well as reprioritized. The results are given in Appendix I & II. The total 14 requirement are considered for evaluation of proposed testing technique (3 were added after the first review, leading to the requirement of reprioritization).

It should be able to automate the Analytical Hierarchical Process (AHP).

It should be able to automate the Hierarchical AHP.

It should be able to automate the Cumulative Voting (CV).

It should be able to automate the Hierarchical CV.

It should be able to automate the technique of the Minimal Spanning Tree.

It should be able to automate the Bubble sort technique.

It should be able to automate the Binary Search Tree.

It should be able to automate the Priority Groups.

It should be able to analyze these methods against predefined comparison criteria’s.

It should be password protected.

It should be having a GUI.

It should be able to automate the technique as given by (Gupta and Srivastav, 2011).

It should be able to automate the Wieger’s method.

It should be able to automate the Cost-Value approach.

Table 4.7. Reprioritized Priorities

Normalized

Relative Priority

R1

0.0111

1.11

R2

0.0210

2.10

R3

0.0453

4.53

R4

0.0474

4.74

R6

----

4.23

R9

0.2433

24.33

R10

0.2244

22.44

R11

0.1846

18.46

R12

0.0985

9.85

R13

0.1343

13.43

R14

0.1465

14.65

The prioritization results are not available for requirement R5, R7 and R8 because they were ignored during reprioritization. The results of reprioritization are already validated, so these requirements will not be considered in the whole discussions. The requirements R12 to R14 are newly added during second release. The new priorities allotted as a result of reprioritization is only considered for evaluation of this proposal. The new requirement R12 and R13 does not create any ripple effects. On the other hand, implementation of R13 creates ripple effects and hence will affect functioning of R1, R2 and R9. The cost-value requirement will be implemented by calling the AHP technique two times, one for the cost and another for value. The requirement R6 will only be affected as a result of implementation. The distribution of test cases for each cluster is given below.

Creation of Clusters

The various clusters created for test cases associated with each requirement is as given below:

This cluster houses the test cases executing the code implementing the requirements R1 and R2. The requirement R14 is not added to the cluster because it is the new requirement and will be tested using newly designed test cases.

This cluster houses the test cases executing the code implementing requirements R3 and R4.

The test cases associated with the code for requirement R5 is housed in this cluster.

The test cases executing requirement R6 is collected in this cluster.

R9 requirement test cases are contained in this cluster.

Test cases associated with the Requirement R10 is contained in this cluster.

The three new requirements could be considered as being housed in three clusters denoted by 7, 8 and 9. Cluster 1 and 4 are impacted by the addition of the three requirements especially R14. Although it will not result in malfunctioning of requirements in cluster 1.

Test Cases: The three requirements were tested with total 21 test cases. Cluster 1 will house total 14 test cases (7 are common to both R1 and R2 plus 7 belongs to R2). Requirement R14 will execute the similar test cases, and after comparison with cluster 1, 7 are dropped from the cluster 1. Cluster 6 will house total 4 test cases to test statements impacted by implementation of R14. Thus total 32 test cases are to be executed. The proposal yielded total 11 regression test cases total 56 available ones. This yielded total saving of 80%. Similar saving could be attained using other hybrid techniques. This happens because the factors Fim and Mim are not known for this young software. Larger the life of the software, more accurate is the values associated with these parameters and hence larger will be the reduction in testing effort. Few faults were seeded in new as well as in AHP and requirement R6. All the faults were detected by the proposal.

Support for Parallelization

It is widely accepted that parallelization of processing activities results in lowering of processing time since there is simultaneous activities going to complete the processes. Thus due to overlap in such activities the overall time to perform such activities decreases. Thus it is expected from this technique to provide another benefit along with the two main benefits of increased fault detection rate and increased percentage of effort saving. This benefit is percentage reduce in time taken to perform testing when done parallel than sequentially. But one of the main requirements for this to happen is that the activities must be independent of each other which are only possible if they operate on independent pieces of item of interest i.e. clusters. Since the main function that invokes another function is always contained in implementation slice of the requirement, thus each implementation slice can be thought to be independent of each other. Thus the various clusters that represent different implementation slices can be distributed across various testers’ part of local i.e. centralized or global testing team that are nothing but virtual teams. Thus in global software development, tester will be in position to perform such testing. To do this, software engineer must distribute the various data structures like RII, MI tables etc to distributed units so that Inter and Intra cluster prioritizations can be performed. So proposed algorithm gets modified to following steps that makes it truly a solution to distributed parallel processing. Following is the outline of algorithm:

ALGORITHM: Incremental_Testing_DP (This algorithm performs the testing in distributed environment by parallelizing the various sub activities. It is simply the implementation of centralized incremental testing algorithm proposed in distributed environment by analyzing the scope for parallelization).

Distribute the following to distributed virtual testing teams:

Implementation slices of requirement.

Various data structures like MI, RI etc.

Statements that had been changed or added.

Perform impact analysis by going through the data structures supplied and implementation slices available. For occurrence of the variables in modified statements in the variable usage of same variables in modules of requirements whose slices are been supplied, select the test cases that executes such variables. Set of all these test cases is called a cluster.

From the details of M and F associated with each module and hence with test cases, select the test cases from the cluster created. By analysing the details of the priority of requirements, place the above created cluster in its proper location in array such that highest priority requirement cluster is placed in location array[0].

Distribute the cluster placed in array to another distributed team, where the cluster associated with requirement R2 will be placed in proper location.

Repeat step 5 until all requirement clusters are there in proper locations.

Execute the clusters at a centralized location. Such execution is not parallelized since there may exist some Inter cluster dependencies whose impacts will be analyzed by checking various clusters results, which otherwise parallelized will not result in equal work for each site. Thus load balancing strategies has to be applied that increases the efforts and hence may overweight the savings from parallelization.

This is how the proposed approach can be used in distributed environments. The way of performing activities remains same, only difference is that now in place of centralized teams; distributed virtual teams will be employed.

Tool Support

The savings earned by employing this technique can further be increased if major of its activities can be automated. Such automation support allows tester to test software faster and easily. The tool will be a web based application with static pages implemented with hyper text markup language i.e. HTML and dynamic pages implemented with hyper text preprocessor i.e. PHP language and MYSQL database as the backend. Since it is a web based application, thus it can be a good solution to overcome global distances in global software development. Thus the testing can be done parallel by multiple testing teams that are like collocated virtual teams, performs the testing activities simultaneously in distributed environment. Automated support provided by web based framework provides a good solution to various problems in phases of global software development that arises due to global distances and now organisations can think of distributing the development process not only phase wise or task wise but same phase activities can be distributed, with each team performing the same activity on different clusters. The various modules that make up this framework and thus perform testing together are given below:

Insertion Module: This module performs following activities:

Add the values of M, F, RP, RII, and MI in the database. Since software evolves, thus the values stored in these data structures needs to be changed. But as new requirement emerges, its details will be stored in these data structures.

Inserting the implementation slices of every requirement along with the associated test cases.

Insert the details of the distributed sites where work needs to be distributed. Here one way of doing this is to create a PHP script that automatically transfers the slices and associated test cases to distributed sites by using the stored site information in form of IP address or another way is to give each site a username, password and site address as input security parameters that needs to be entered in order to access the database to view details. In this case the database entries will be indexed by the site address that is foreign key in this table and attribute in username, password table.

Insert the source program that is modified as a requirement for evolution.

The module inserts the test suite generated as a result of intra cluster prioritizations have to be executed to debug faults at centralized locations. Thus creation of array with cluster being placed at particular location in accordance with its priority is done by this module. This prioritized set is then send or stored in database that can be seen by debuggers at same location for intra cluster prioritization inside the prioritized clusters.

Deletion Module: This module performs the action of deleting or removing the data from database fields. This may be required as a result of removing the wrong requirement from the software or as a result of preventive maintenance activities. The main actions performed are thus removing requirement information from database, removing the source code lines, removing site details or its administrators or even removing the past test details and test cases.

Updation Module: This module performs the action of changing the information stored in database and thus involves the following:

Changing the source code as a result of evolution.

Changing the information stored in data structures like RII, MI, F, M etc. This is required since due to software evolution the information represented by these data structures needs evolution.

Changing the test cases as a result of code changes.

Changing the username, password or site address stored in the database.

Execution module: This module takes data from the databases and performs the action of impact analysis, clustering, Intra and Inter cluster prioritization. This module also calculates the percentage saving in effort and average percentage fault detection metrics. Thus it contains several arithmetic and logical statements together with database access statements like select, join etc.

Report generation module: This module generates the details of prioritization like the list of faults in the program, percentage saving in effort and average percentage fault detection metrics, analysis reports on above metrics analysis etc.

Thus each module interacts with databases in order to implement the proposed technique testing technique in both centralized and distributed environments in both sequential and parallel fashion. This is illustrated with the help of an algorithm that shows that how the web framework works:

ALGORITHM: (Working of web framework supporting testing)

Click on web browser and enter the IP address of the machine that stores the web framework web pages (server). (This server can be in same organization campus or can be remotely located in any location of the world.)

Enter your username, password and site address. The site address will be used to access the implementation slices available for the site.

Click on Impact analysis link, and select the modified statement. This generates the list of test cases that executes the modified variables that are directly or indirectly liked to changed variables.

Click on Intra cluster prioritization link to select the test cases from clusters on the basis of values of F and M.

Click on Inter cluster prioritization, to arrange the clusters on the basis of requirement priorities.

Click on generate analysis to calculate values for percentage saving in effort and average percentage fault detection metrics.

Click on generate the report, to generate the testing report that includes the faults identified, the percentage saving in effort and average percentage fault detection metrics, analysis reports on above metrics analysis etc.

Click on Logout to end your session with this web application.

Conclusion and Future work

The goal of this chapter is to give the solution to the problems as encountered during the testing of evolving software’s in terms of minimization of test suites and enhancement of fault detection rate. The three different proposals are suggested with each one an extended version of the former.

The multilayered hybrid approach is a hybrid regression testing technique. The question of effectiveness could be answered only when it is clearly visualized when the technique is applied on that software that had evolved over years and large values are associated with the statements, priorities have changed while thousands of test cases are there. Thus one has to wait for some more time so as to have the above mentioned software on which this technique can be employed. The technique will be proved more effective when it is utilized in the industry based complex applications. The proposed web based framework will be implemented as a tool in the future and will be interesting to see that to what extend it will interface with test management suites in the industry.

A hybrid regression testing technique based on path pruning aims to find out the chromatic number of test cases with high fault detection capability. The proposal proposes to automate the hybrid regression testing approach by performing activities in parallel using web framework. The proposed hybrid regression testing technique leads to a reduction in the number of paths to be tested thereby reducing the number of test cases to be executed. Proposed technique will be useful in software industry dealing with maintenance of "aged" software in resource constrained environments. The proposed hybrid regression testing technique overcomes various drawbacks of existing techniques like non-coverage of unaffected parts and incomplete handling of control dependencies. Regression testing technique proved to be effective for a program of size 486 lines of code and test cases in the range of a few hundreds.

The results are obtained by using the above program as the experimental unit. The results are dependent on the values of number of times path is used (N), dependencies (D) etc. This proposal had not been evaluated on other software systems and hence it will be intresting to analyze the performance by using different softwares with different values of above parameters. The variation in the components of the software source code with different values of parameters like N and D would affect the result and had been one of the threat to validity. The variation in the results by varations in above coding and parameters will provide a future work to further improve the proposal.

It is expected that in the near future this technique will be employed on those software whose sizes and test cases are in the few thousands. It is also expected that web based framework will be implemented in the near future and some software development firm will employ the tool and take advantage of it. Work on automation and parallelization of regression testing is currently in progress.

The third and final hybrid regression technique performs the task of prioritization and selection on the basis of requirement priorities, fault and modification history. This technique had been illustrated and evaluated on the live case system of "Tool for automatic analysis and comparison of different release planning methods" developed in the Computer Programming laboratory at Jawaharlal Nehru Govt. Engineering College. Since the software is younger and not evolved to larger extend. This makes it impossible to employ the values of Fim and Mim. This makes it impossible to compare the proposal with existing hybrid regression testing techniques. It will be quite interesting to see that how much improvement will this technique will bring after being employed on the matured evolving software system. The regression testing based requirement prioritization techniques will make the set of requirements dependent to the maximum extentd. In such cases, the savings in regression testing efforts will be the value of prime interest to various researchers. The task of evaluation on matured software as the future work.