Metrics For Software Maintenance

Published Date: 02 Nov 2017

1 Introduction

The enormous economic impact of quality of the software makes it a major concern for all software development efforts. Software quality is the focus of a software project, from the perspective of the developer as much as the customer. At the same time, the development of concepts, methods and tools for software engineering quality comes to demanding new challenges for researchers (Buchberger et al., 2009).

Traditionally, software requirements are classified as functional or non-functional with any notions of quality hidden in the past. As the industry shift from functionality to improve quality, is emerging a new category of requirements focused on quality. To define this new quality, quality must itself be determined. A quality model provides the framework for a definition of quality. Engineers have long recognized that in order for something to find its way into a product, it should be well defined and specified (Alexis, 2006).

2 Software quality metrics

Quality metrics of the software have three categories: product metrics, process metrics, and project metrics. Product metrics describe the characteristics of the product, such as size, complexity, design features, performance and quality. Process metrics can be used to improve software development and maintenance. Project metrics describe the project characteristics and performance.

2.1 Product Quality Metrics

The actual definition of quality of the software consists of two levels: the intrinsic product quality and customer satisfaction. The metrics on below covered both levels:

Mean time to failure

Defect density

Customer problems

Customer satisfaction

2.2 Process Quality Metrics

Quality metrics play an important role, because our goal is to provide programming to understand and learn the quality engineer in the process. We have four basic activity in process quality metrics:

Defect Density During Machine Testing

Defect Arrival Pattern During Machine Testing

Phase-Based Defect Removal Pattern

Defect Removal Effectiveness

2.3 Metrics for Software Maintenance

When developing software product is complete and it is released to the market, is the maintenance phase of its life cycle. In this phase defect arrivals by time interval and customer problem calls (whether or not defects) over a time interval are de facto metrics. The following metrics ​​are very important for the defect rate of the product to improve:

Fix backlog and backlog management index

Fix response time and fix responsiveness

Percent delinquent fixes

Fix quality

Software complexity

In the software life cycle, the development and maintenance have high cost. If the complexity of the software is reduced, the cost of software can also be reduced.

software complexity have three classes:

essential complexity

selecting complexity

incidental complexity

The essential complexity is determined by the problems that the software tries to solve.

Selecting complexity is determined by the programming languages ​​the problem modeling and software design methods.

The incidental complexity is determined by the quality of the performer.

We can see some information about them in the table 1.

Class

Origin

Influence

Improvement

Essential

Problem

Huge

Extreme hard

Selecting

Method

Medium.

Hard

Incidental

Implement

Small

Fair

Table 1.THE CLASSIFICATION OF THE COMPLEXITY

We have some classic and efficient software complexity metrics, such as Lines of code (LOC), Halstead Complexity Metric (HCM) and cyclomatic Complexity Metric (CCM). In this case we use the CCM to measure our code. It is because of in the CCM we can find the number of the path in our code and we can control them in this technique we draw flowchart and all of analysis are based on that (McCabe, 1976).

Based on this technique, we can access to the results and the analysis faster than other techniques and it is showed the simplicity of this technique because other techniques will take time to get results (Cole, 2007).

In this technique we can calculate the data with this formula:

(n) is number of nods.

(e) is number of edges.

(p) is number of connected components.

The (e) and (n) are related to flow chart and (p) always is one, except we have the components that they are connected to the code so if connected components is one, (p) will be two (McCabe, 1976).

The following chart shows the relations between complexity and risk:

Cyclomatic Complexity

Risk Summary

1 - 10

Simple , low risk

11 – 20

Moderate complexity, medium risk

21 – 50

Complex , high risk

51+

Very high risk

Relation between CCM and Risk

Measuring Quality

In this report we want to measure the quality of program that written with visual C++ 2010, this program get 3 data (digit) and after analyzing them gave the angels between the sides.

Process steps:At first running the software after that the program ask to enter 3 numbers, we can choose any numbers so we are insert first numbers and press enter and then second one and press enter and at final the third one and press enter.

Now the program analyzing the numbers that inserted for find that is it possible to build a triangle with those 3 numbers or not, If it is not possible the program get error and say enter another numbers but if it was possible the programs shows the angels between the each side of the triangle.

Now we are calculating the Cyclomatic Complexity of this program. Figure 1 has shown the flow chart of the program.

C:\Users\behrooz\Desktop\Untitled.jpg

Flowchart of angels between sides of triangle

Based on this flowchart we have 17 nodes therefore (n=17) and we have 20 edges therefore (e=20) and because we have just one exit node and we don’t have any connected component therefore (p = 1).

Base on the formula:

=> CCM = 5

Conclusion

Quality can be measured by some metrics which determined by some standards such as IEEE and ISO 9126 to cover all five different quality views. There are also some software quality tools can be used parallel to increase the level of quality and decrease the number of defect and the amount of risk (Tian, 2005). Cyclomatic complexity is used as a software quality metric in this assignment to assess the complexity and level of risk in given program. In this report cyclomatic Complexity Metric (CCM) was 5 and Because of that risk of the program is low and the code is a simple. In additional, CCM help us to understand that all the parts of the code is correct or not.

Part B: therac-25

Introduction

Computers and software systems are intertwined with modern society. Worldwide users need to comply with the support of the essential software that works well in software life cycle, easy to use, and etc. User involvement in the development process is a way of achieving a high quality system to satisfy people. In a word, do the right things and doing things right are the key elements of the quality assurance (Tian, 2005).

In software development process, various software quality control activities - such as checking for data consistency, integrity of the database files, uncertainties in the data or database files - be performed at various locations within the requirements to ensure that the errors can be corrected in time (LingDa et al., 2011).

Overall, the terms "Quality Assurance (QA)" and "Quality Control (QC)" are the ways to improve the quality of a service or product. In other words, quality assurance is the planned and systematic activities in a quality, so the quality needed for a product or service will be accomplished (asq.org).Quality has many interpretations therefore they have different quality views. Hundreds of standards such as IEEE or ISO 9126:2001 exist for satisfy these different views and expectations (Voas, 2003).

Case review

The Therac-25 is a computerized radiation therapy machine produced by Atomic Energy of Canada Limited for cancer treatment. The new features in Therac-25 including its software design which has been taken from Therac-6 and Therac-20 made the system analyst assumed that there were design problems since Therac-6 and Therac-20 has been running for many years without detectable errors. This is a dual mode machine that use electron beam and x-ray to kill off the tumor in different layer of skin. It means, for shallow tissue electron beam will be used but for depth tissues x-ray mode well be more helpful. Nevertheless some of the problems of the Therac-25 are being originated from dual-mode equipment.

Between 1985 and 1987, there were six accidents involved cause to the death of two patients with the use of the Therac-25 in which patient were given massive overdoses of radiation which was approximately 100 times of the dose intended.

Functions of therac-25

In therac-25 machine all the process will be done by software like accepting desired treatment, set machine up for treatment and turn on/off the beam. The operator can watch all the details of parameters on CRT monitor and print those. The operators have some duties such as, position patient on table, set treatment manually, use VT-100 console to enter patient information and range of dosage, start machine or re-enter data and informations (Leveson & Turner, 1993).

Therac-25 failures identification

At about the same amount as application techniques have been presented in our lifestyle, the number of bad information about issues due to application issues improved (Weiss et al., 2009). To feature only one cause to a car incident is a serious error. One of the issue as Leveson and Turner (1993) said, "Many organizations developing safety-critical application not using appropriate techniques from a software-engineering and safety-engineering perspective". Even though, the main of the most injuries is complicated connections between elements and actions.

According to Nancy Leveson, (1995) to better understanding the accidents, divers’ characteristics of the Therac-25 should be noticed for example:

The Therac-25 application has more focus on issues of protection than application in past editions. It means in the Therac-20, for example, there are individual safety tour or technical interlocks to function safe but in the Therac-25, application is accountable to sustain the protection of managing.

Design of Turn table in the Therac-25 is one of the causes of injuries.

The serious risk in a double method device is that high-energy gleam might straight attack the affected person if the aluminum foil and flattener were not placed its way.

Overconfidence in the capability of application to make sure the protection of the Therac-25 was an essential aspect which led to the injuries.

The Therac-25 is managed by PDP-11 pc as same as the Therac-6 and the Therac-20 while the Therac-25 is designed as a pc managed device not as a stand- alone device.

Several misguided beliefs in the company's mind-set led to the injuries.

Source code:

We have two different ideas in therac-25 case, Garland and Guttag, (1989) believe that the code had problem and it doesn’t behave as expected but Muffy Thomas (1993) has different opinion, he tried to prove the code is correct. Malfunction 54 found several bugs in software coding. The reason is whole the code written just by one person therefore lack of proper review, testing and monitoring cause to injected that bugs to the software (cheme wordsmithy, 2012).

Human error

In this days the machines have complicated software and this complexity is the cause of unsafe behavior of the machines. Engineering treatment data is one of them, For example, in the East Texas Cancer Center (ETCC) in March 1986, operator made ​​a mistake and tried to wrong input quickly correct that was caused by X-ray mode instead of the lower intensity electron beam without proper filter or the goal to produce required dose radiation. Then " Malfunction 54" appeared relating to underdose or overdose. The point is documentation about the error was insufficient as the consequences of such an error was not clear for operator (cheme wordsmithy, 2012). In this respect, it can be said that a dual-mode is one of the characteristics of the machine, leading to errors in the system. This is the main cause what happened in the fourth and fifth accident (Leveson & Turner, 1993). Moreover, the AECL staff and operators no attention to the complaints. They also assume the fault error was another way of machine and overrule it (cheme wordsmithy, 2012).

The software problems

the software of the Therac-25 is developed version of the Therac-20’s software that itself was the developed version of the Thearc-6’s. It means one programmer is used for different machine with different features over several years. The software responsibilities is totally different, in the Therac-20 software just adds convenience to the hardware but in the Therac-25 is responsible for critical system checks, set appropriate position for turntable based on required treatment, monitoring the machine status, accepting input, set up the machine for treatment, and turn on and turn off the treatment beam in both successful treatment or facing with malfunction and also controls and checks the turntable position and control possible hazards. All the bugs in the Therac-20 were found in the Therac-25, however, hardware in the Therac-20 can be considered as an error prevention activity. The main problem in the Therac-25 software is caused by using poor engineering process and lack of formal testing procedure (Porrello, 2000).

CMC performed Fault Tree Analysis as a safety analysis to calculate the probabilities of hazard happening. Since the Therac-6 and Therac-20 software systems had been running without any problems discovered, the professionals presumed that there is no problem with the Therac-25. While producers should be aware that safety-critical application substantially need precise and tight design examining and failing studies. Even more important such examining and analysis should be applied by professionals (Leveson & Turner, 1993).in addition, the software errors led to set the wrong position for turntable "when the turntable is in test mode Class 3 is set to non-zero value. As long as the operator is testing the position of the light beam, the variable increments. Once testing procedures are complete, the variable is set to zero and the radiation beam is allowed to pass. Class 3 was stored in one byte of memory thus every 256th increament results in the value of zero assigned to it" (Porrello, 1993).

Deficiency of sufficient documentation

Deficiency of certification is one of the primary causes of all problem happened in this situation. Problem signing or producing all defect information and quality guarantee actions that has done to eliminate them allow producers to use such information for upcoming item and avoid same failing to provide to the system (Tian, 2005). Further associates can connect with documented information, customers can use this information as a guide to how to use the system (Sommerville, 2001).

Therefore, in the Therac-25 is designed without any certification. If there were certification about "Malfunction 54" it can link servicing technicians, designer and owner then employees was aware about repercussions of such mistakes. Deficiency of certification led to uncertain mistakes encounter by providers. Documentation allows designer to recreate the problem and find a way to eliminate or manage them.

Deficiency of improvement Process

Application procedure enhancement are using by software organizations as a way of improving the level of quality and reducing expenses. Process enhancement means having sufficient knowledge about current procedures and making the necessary changes to achieve higher item quality with lower expenses. In fact procedure quality and item quality are closely related and the quality of the item is affected by procedure quality especially in large organizations. In other words, procedure is the prominent quality identifying factor (Sommerville, 2001).

Quality Assurance Strategy

Quality assurance is about the required quality of final product. There are some metrics to measure product or process quality. Plus some quality assurance activities reduce the number of defects or remove errors and prevent them to inject to software system (Tian, 2005).

The goals of quality assurance strategy are:

1. Deliver the highest quality software possible with the available time and resources

2. Achieve quality economically

3. Ensure a lower total cost of ownership

Quality assurance activities are:

Defect prevention

Defect reduction

Defect containment

Defect prevention is finding the reason of errors like imprecise design, human-misconception, etc. to remove them through error blocking or error source removal. For reducing the faults of Therac-25 we have to do some activities like education and training, using formal method and axiomatic approach.

Testing and inspection are the main activities of defect reduction; in terms of Therac-25 lack of testing was one of the root causes.

Three accepted premises of strategy in software development:

1. It is usually significantly less expensive to deal with defects when they are caught early than it is to fix them later

2. The greater level of test assurance (coverage, frequency and effectiveness), the simpler it is to catch defects early

3. It is usually significantly less expensive to automate a test than it is to personally perform it

Waterfall would be suitable development method for Therac-25 because of documentation, reviewing and inspection in each stage before going to the next phase (Tian, 2005). Figure 3 has shown the waterfall stages and related quality assurance activities.

C:\Users\behrooz\Desktop\SQE\a.jpg

In this methodology, a criterion is set as a quality metric for transferring the product from one phase to another. This quality measure is set in planning phase based on customer expectation, organization goal and environmental constraint.

Preventing defect in Therac-25

Most of discovered errors were happened in product specification, design and coning section. With preventing error we can reduced the causes of errors for example with training and education we can increase the knowledge of personnel that involved in this process and the personnel should completely be inform about the product and development methodology (Leveson, 1995).

Formal development method

For developing software system often used some technique supported by tools that called formal methods. Users with mathematical calculation can analyze and verifying the errors at any part of life-cycle of the program.{reference( fmsurvey 1 )}

Formal methods have two formal methodology:

Formal verification

Formal specification

If formal specification/verification had been done in Therac-25 project manufacturer knew that the source code is not match with the Therac-25 design or they can validate whether the required functions presence in system or not.

Planning

Planning is the first step of waterfall development process, in this phase defined and determined the number of staff, skill level of staff, responsibilities of all involved personnel and exit quality criteria (Tian, 2005).

All activities that are integrated parts of software that supposed to use in critical position like nuclear device or medicine machine (like Therac-25) the safety is the main concern. All these activities would be reviewed by quality assurance team before proceeding to the next stage in order to defect prevention. In Therac-25 it is completely clear that no one of those activities done because no standard procedures were enforcement and knowledge and experience of programmer did not consider (Owens and Khazanchi, 2009)

Requirement and specification

Requirement analysis is the next process after planning, in this step the programmer has meeting with customer to understanding the requirements. This step in most important and crucial step because most miscommunication and misinterpretation happened at this step and injected to the software. After noted the requirements must be sure that we listed all details and accurate and there is no place for any ambiguity.{refrence internet)

System Design

Because of making easier the software process implementation broken down the requirements that were gathered in last phase into logical units. Maximum fault are referred to early phase especially in design phase thus within this phase needs to be done error blocking (Tian, 2005).

In Therac-25 was designed as a computer controlled machine but manufacturer reused the Therac-6 design that wasn’t designed for computer controlled software system.

System Implementation (Coding)

Software development takes place in this phase. Software program is written based on algorithms that were written in previous phase. For be sure that software have correct output in every module software code is tested. In Therac-25 case manufacturer didn’t test the code based on in Therac-20 no defect was not detected.(internet)

System Integration and Testing

Testing is an executed base activity to achieve quality through detecting and removing the problems. In this phase all the modules are integrated and software was tested for finding the bugs then all bugs will removed due integration and retest software this process is followed to make software free of bugs.

C:\Users\behrooz\Desktop\SQE\software-testing-framework.png

Dynamic problems happened during interdependency and execution that they occurred in implementation part therefore for acceptance testing should do defect removal activities like testing in middle to late phases (Tian, 2005). Testing fits into various software processes as an important phase. There are different testing techniques based on what artifact to test. In the case of Therac-25 coverage based testing and usage based testing, and structural testing (WBT) would be proper.

White Box Testing (WBT)

Structural testing or WBT should be done to verify correct implementation of internal units (code, data structures, and blocks). Code would be the primer part to test either as individual pieces or as an integrated whole. The focus of the WBT is on internal parts and relation between them. As long as WBT is a proper testing technique for early phases then it can be used in THerac-25 to concentrate on detecting and removing source code problems and other problems that coding led to.

Usage-based Statistical Testing (UBST)

Usage-based testing is actually application for customer utilization. In situation if any faults are recognized by customers, some information about them can be revealed to application providers and incorporated repairs may be established and sent to all the customers to prevent such defects. A new product for the expected usage can be captured and used in product consistency and testing software could most downright guarantee. This phase can be helpful in Therac-25 because the operators had lots of errors and problems in using the machine (Leveson, 1995).

Coverage-based Testing (CBT)

This type of testing views the things from a developer’s perspective and specializes in efficient or execution units. CBT can perform by professional tester or developers. In early phase in Therac-25 CBT can be applied for component and unit. Based on Therac-25 coverage goals can be consist of safety, usability, functionality and reliability.

Inspection

Inspection is a meeting during coding and designing that reviewed by peoples and developers. It helps the programmer to improve the quality of the program and removes defect efficiently and finally it cause to improve product quality (Tian, 2005). Applying Fagan inspection could be effective for Therac-25 because it done by both individual and group of inspector and detect false alarms and real defect.

Inspection process is shown in figure.

C:\Users\behrooz\Desktop\SQE\inspectionprocess.png

Release and Support

Release is the last phase of waterfall development. With all prevention and reduction defects still after release the product and receiving it to customer the software may have some defects or errors or faults, that to handle those and reducing the impact of them defect containment activities applied.

Fault-tolerance

Fault-tolerance is a technique that breaks the relation between fault and failures and do not let local fault increase to global failures. Recovery block, N-Version Programming (NVP), and N self –checking programming are techniques used for tolerating or containing faults (Xia, 2006). In addition fault- tolerance is containment measures to avoid disastrous consequences thus it can be used for real-time control software like Therac-25 to prevent damage to users and patients (Tian, 2005).

Process Improvement

Making change in order to making things better in meaning of process improvement that it’s important to understanding the current state of software engineering and helps to improve areas where changes can afford to the long-term benefits (reference). All the factors affecting the process quality such as selected method, models, QA activity and goals examined by a team (sommerville, 2001).

In-term of this explanation for achieving high quality on Therac-25 we need to making some changes in two previous machine (Therac-6 and Therac-20) that cause to identify and remove all weakness of those.

Quality Metrics

Quality metrics are needed for measure the quality of product and can be used as exit criterion from one phase to another in development process. These metrics should be selected to satisfy different quality views. In this regard IEEE standard or ISO 9126 can be enforce to select proper metrics in order to measure product quality.

Quality expectation of a user is that system performs only pre- specified functions. While producers expect that product conforms to product specifications, and everything should be conform to contractual obligations and agreement (Tian, 2005).

The International Organization for Standard (ISO) and International Electrotechnical Commission (IEC) have published ISO/IEC 9126 to define software quality model, quality characteristics, and related metric (Zeiss et al.,2007).

ISO 9126 has six top level quality characteristics which each has its own sub-characteristics, as summarized in table 3 (ISO, 2001).