Software Development Process And General Solutions

Published Date: 02 Nov 2017

Software development process and general solutions for organizing the software process belong to the standard themes of software engineering and have challenged theoreticians and practitioners for a long time.

The causes of the software crisis have been linked to the overall intricacy of the software process and the relative immaturity of software engineering as a profession. The crisis reveals itself in numerous ways:

Projects running over-budget.

Projects running over-time.

The software was very inefficient.

Software was of low quality.

Software often did not meet requirements.

Projects were unmanageable and code difficult to maintain.

Software was never delivered.

The overall success for the development of software is still not achieved because each software development process or model considers only one or few concerns and specify a phase wise abstraction for the development, but no definite approach or model is specified for the phases of software process models. To make the software development effective and reliable, a new software process model is required which will address all these concerns. Which will also consider each phase of software development as a software development process and provide an effective model for software development phases.

The purpose of the proposed research work is to analyze the issues, challenges and needs in software development and develop new software process model. The subject which will be addressed are review of literature, development of Fusion Process Model and phased model, named 3C Model which is for different phases of Fusion Process Model, Mapping of Fusion Process Model onto ISO/IEC 12207 and comparative study.

1.3 OBJECTIVES

The objective of the proposed work is the development of Fusion Process Model for effective and reliable software development. The following objectives will be addressed by the thesis:

Development of Fusion Process Model that will manage the concerns in software development.

Development of a phased model for different phases of Fusion Process Model that will integrate the concept of Context (environment), Capture (Problem Solving concept for various development phases) and Control (based on environment and development constraints, quality criteria, mathematical and optimization techniques) in one model.

Integrating processes of identification, evaluation and composition of various concerns in software development.

Classification and evaluation of software design.

Fusion is component based software process model, where each component implements a problem solving model. It includes the explicit processes for technically analyzing the problem, solution space analysis, alternative management, dynamic design and development and scope of dynamic testing. In the problem analysis process, technical problems are identified and structured into loosely coupled sub-problems that are first independently solved and later integrated into the overall solution. In the solution space analysis process, requirements are specified using some representation and this should be refined along the software development process until the final software is delivered. In the alternative analysis process, different alternative solutions are searched and evaluated against explicit quality criteria. Dynamic design and development is a component driven approach, which provides scope for dynamic changes during the development life cycle. As Fusion Process Model follows the component driven approach, it provides scope for dynamic testing (component based testing). This approach is totally unlike CBSD (Component Based Software Development) which is based on pre-built, standardize software components.

No software process model till date explicitly follows component development approach from requirement analysis till operation and maintenance. The Fusion Process Model implements proven engineering techniques. It trims down all the software development risks to a component only, and provide the firmer control over the software development process. Unfortunately, in current software engineering practices, an explicit problem analysis process generally does not completely exist, solution space analysis is still not fully integrated into software design processes and alternative management is usually done in an implicit manner or practically missing. Fixed design and development approach is followed and there is not much scope of dynamic testing.

1.4 RESEARCH METHODOLOGY

The purpose of the proposed work is to analyze existing software development process models and techniques, and to develop contemporary software development process model. The research has the following phases:

Analysis of Current Software Development Approaches

During this phase, all the well known software development models and techniques has been analyzed and studied in detail to find out there short coming in software development models and to reveal their problems in terms of requirements and constraints of the modern world.

Development of Process Model

In this phase, Fusion Process Model is developed, that manage the concerns in software development. It is a component driven software process model, where each component implements a phase model. The model also includes the explicit processes for technically analyzing the problem, solution space analysis, alternative management, dynamic design specification development and scope of dynamic testing.

Analysis of Developed Process Model

After the development of the Fusion Process Model, it is applied to the design of a real world information system and analyzed through case studies.

Mapping of Developed Model onto ISO/IEC 12207

This segment performs Mapping of Fusion Process Model onto ISO/OEC 12007 for standardization.

Comparative Study

An extensive comparative study is done to compare the proposed process model with the existing ones to demonstrate the effectiveness and ability to manage the concerns of software failure.

1.5 CHAPTER SCHEME

The thesis will be next comprise of seven chapters which have been planned as follows:

Chapter 2 Literature Survey contains an extensive survey of literature showing the work done previously in the area of proposed research. The basis here is a discussion of what software development model is and various existing models. Numerous failure reports have been studied and few major problems in software development is underlined.

Chapter 3 3C Model initiates the concept and development of the phased model in terms of problem solving.

Chapter 4 Fusion Process Model (FPM) presents the component driven development approach that provides appropriate solutions to the identified problems of design approaches, by applying phase model in each phase.

Chapter 5 Case Studies take a closer look at Fusion Process Model by investigating the practical aspect of it. The evaluation is performed through the analysis and comparison of actual results vs prior estimations.

Chapter 6 Mapping of FPM onto ISO/IEC 12207 discusses the International Standard, ISO/IEC 12207 on software life cycle processes and substantiates the standardization of Fusion Process Model.

Chapter 7 Comparative Study inspects the comparison of existing software development models and Fusion Process Model.

Chapter 8 Conclusion and Future Work summarize the contributions and evaluates the work that has been presented. Future work on the Fusion Process Model focus on in order to continue the research on it is also pointed out.

2.1 INTRODUCTION

Developing and maintaining software systems involves a variety of highly interrelated activities [14]. The software process is a critical factor for delivering quality software systems, as it aims to manage and transform the user need into a software product that meets this need. In this context, software process means the set of activities required to produce a software system, executed by a group of people organized according to a given organizational structure and counting on the support of techno-conceptual tools. In the literature studied so far, we find out that in order to manage these activities, various models have been developed over the years with different success rate. A software process model is an abstract representation of the architecture, design or definition of the software process [25]. The development of process models was to provide a guide for the development of software applications. Applying a process model and discussing its sub-processes help the team to recognize the gap between what should be and what is.

Software process models are problem-solving approaches that apply requirements engineering to help solve the problems based on varying degrees of problem specification. The use of requirements engineering necessitates user or stakeholder involvement to ensure that the software product is a valid solution to the underlying problem. However, the level of stakeholder involvement varies considerably across the various models. The process models assist to achieve consistency in a project, which involves a large number of people with different skills. The software development process model that a group uses to design software is important for determining the success of a project. Many such models have been proposed; these include Waterfall model, Iterative development, Prototyping, Spiral model, RAD (Rapid application development), Agile process models and CBSD (Component base software development) processes. All the models take a project through several primary phases: a requirements-gathering phase, a design phase, a construction or implementation phase, and a testing phase. Figure1 provides a simplified illustration of the general software development life cycle model.

Each phase produces feedback that affects the next phase. For instance, the requirements gathered during the requirements phase influence the design, which is translated into working software code during the implementation phase. The software code is verified against the requirements during the testing phase.

2.2 CURRENT APPROACHES

Royce [21] proposed the first and most influential approach which is referred to as the waterfall model and has become the base for other models. This model initially attempts to identify phases within software development as a linear series of actions, each of which must be completed before the next is commenced, as shown in the Figure 2 [PRE04]. The Waterfall model is marked by the apparently neat, concise and logical ordering of the series of obvious phases, which must be followed in order to obtain the final software product. It can be very useful in helping developers lay out what they need to do and its simplicity makes it easy to explain to customers who are not familiar with software development. Refinements to this model consider that completion is seldom and that iteration back to a previous stage is likely to happen, but it takes no account of bottom-up development and prototyping. It follows the linear sequence to software development. McCracken and Jackson (1998) point out that the model imposes a project management structure on system development.

Curtis, Krasner, Shen, Iscoe (1987) observed that the waterfall model’s major shortcoming is its failure to treat software as a problem-solving process. Humphrey and Kellner [20] criticize the model by discussing the problems which are faced after implementing the model. It has a linear structure and rigid design, rather than a dynamic problem-solving process, which would help in implementing the learning that result from user feedback and involvement.

Later the German defense organization introduced a modified version of the Waterfall called the V-shaped Model. Instead of moving down in a linear way, the process steps are bent upwards after the coding phase, to form the typical V shape, as depicted in Figure 3 [SBalaji]. This model included validation and verification processes by associating testing activities with the analysis and design phases [5]. This model had a higher chance of success over the Waterfall Model due to the development of the test plan. However, some developers believe it is too rigid for the evolving nature of IT (information technology) business environments [SBalaji].

Graham identifies the changes in software development approach with incremental and iterative models, also called phased development models [12]. This approach was different from a conventional waterfall model where the development process was divided into phases according to various levels in software development process. In these models, the phases are divided according to the functionality it provides. Each phase of incremental and iterative models is a complete system which provides the limited set of functionality. Pfleeger emphasized that, to measure the productivity more effectively, each iteration or phase should be small [26]. Lott’s staged approach is similar to Prototype and spiral model, where project risk reduced after each increment or iteration is reviewed [10]. This approach overcomes many issues of conventional development model but new issues arise like the requirements are not properly defined because user initiates the development. At a later stage when more requirements came, it was found that the current approach was not applied to these new requirements, because it never considers such type of requirements before creating the base for the development. According to Graham, this process is suitable for small system of short duration [12]. Even the software engineering terminology describes the importance of development in phases, where we can validate the output after each phase to keep track of the development process and should be present in the development model [15].

Boehm [6] proposed an approach called Spiral Model, which creates software development more flexible and has been proposed mainly to speed up software development through prototyping. To get a clear view of prototype paradigm, it has been represented through Figure 4 [Roger]. Similar to the Prototyping Model, an initial version of the system is developed and then repetitively modified based on input received from customer evaluations. Prototyping provides constructive feedback to designers and potential users so that the system requirements can be clarified and refined early during software development. Risk assessment is included as a step in the development process as a means of evaluating each version of the system to determine whether or not the development should continue, shown in Figure 5 [7]. Several papers indicate that for the development of a software system, the identification of concerns, objective and alternatives are vital [5], [7], and [8].

Later Boehm and Prasanta Bose [9] extended the spiral model to a variant called the Win-Win Spiral Model as depicted in Figure 6 [BOE98]. It is also called the Win-Win Stakeholder approach to converge on a system’s next level objectives, constraints and alternatives. It determines three milestones: life-cycle objectives, life-cycle architecture, and initial operational capability, which served as the basis for software development process. This model has been formally specified and analyzed for consistency but only little is known about the correctness and usefulness of assumptions made during this process. The process and outcome of negotiation is not well defined [3].

According to the IEEE definition "An architectural prototype consists of a set of executables created to investigate architectural qualities related to concerns raised by stakeholders of a system under development" [27]. It gives the end-user an outlook about the future system or analyzes the feasibility of critical components which are not an individual part of the system. But it was system in itself which provide the limited set of functionality initially. The term prototype is used for an initial system that is the first version of the target system upon which developers can build the system [16]. At a later stage, when new requirements take place, it is probable that the hardware chosen for the initial system does not support them and leads to risk for system development. The response time of the initial simplified system may also be significantly faster than the fully developed system. This generates high initial expectations from users, which might be subsequently disappointed as the system extends and performance deteriorates under further development complexity.

Incremental software development process model that emphasizes an extremely short development cycle is a Rapid Application Development (RAD). The RAD model is a high-speed adaptation of the linear sequential model in which rapid development is achieved by using component-based construction. If requirements are well understood and the project scope is constrained, the RAD process enables a development team to create a fully functional system within very short time periods mostly 60 to 90 days, illustrated in Figure 7 [MAR91]. The time constraints imposed on a RAD project demand scalable scope [KER94]. Not all types of applications are appropriate for RAD. If a system cannot be properly modularized, building the components necessary for RAD will be problematic. If high performance is an issue and performance is to be achieved through tuning the interfaces to system components, then this approach may not work. It is also not appropriate when technical risks are high. This occurs when a new application makes heavy use of new technology or when the new software requires a high degree of interoperability with existing computer programs.

A lot of research on the evolution of software development Process Models has reflected the changing needs of software customers. The methods for developing systems were amended with the demand for faster results and measures to determine risk and effectiveness. These rapid and numerous modifications in the system development environment led the development of more practical new Process Models and the termination of older models that were no longer useful [28].

Harlan Mills [11] proposed Cleanroom approach, a quality control driven philosophy which is intended to improve the manageability and predictability of the development process. The Cleanroom process applies the statistical correctness verification to the formal method of incremental software design, as represented in Figure 8 [KK11].

Through usage testing each increment, the programmer is given well-defined statistics (through applying mathematics to the usage tests) as to how closely the software system is functioning compared to the desired outcome. It addresses two fundamental principles, first to certify the software with respect to the specifications and second to produce zero-fault or near-zero fault software. A simple system or a system that wasn’t life critical may not need such rigorous method of producing reliable code. Additionally, the Cleanroom process requires formal training on all levels of software development, including managers and requirement engineers. Without everyone buying into the process, there will be issues with team cohesion and productivity. In the long run, if a software company is making the shift over to Cleanroom, it might be very beneficial, but trying to get a software team to switch over to a different method like Cleanroom can be harmful in the end [5]. According to Beizer (1997) clean room relies on statistical results to verify reliability, which itself is misleading. This approach does not provide complete life cycle model, it provides specifications for the software development.

Alan Cline [2] paper work shows Joint Application Development (JAD) technique, which is an attempt to build collaboration process model. It is a technique for engaging a group or team of software developers, testers, customers, and prospective end-users in requirement specification and development of prototype. The JAD process is based on four ideas:

1. People who actually work at a job have the best understanding of that job.

2. People who are trained in software development have the best understanding of the

possibilities of that technology.

3. Software-based information systems and business processes rarely exist in isolation --

they transcend the confines of any single system or office and effect work in related

departments. People working in these related areas have valuable insight on the role of a

system within a larger community.

4. The best information systems are designed when all of these groups work together on a

project as equal partners.

The JAD process proceeds through a number of phases each succeeding phase considering different aspects of the system in ever increasing detail. Each phase has three main activities. Planning, conducting the session and wrapping up. In Figure 9 the JAD process is depicted, here P – Plan, S- Session and W- Wrap-Up.

This is suitable for open source software development projects that rely on group email discussions among globally distributed users and developers [19]. J. Neighbors [12] laid stress on reusable software components guided by an application domain analysis, which is an alternative approach to developing software.

Agile software development approaches have gained widespread interest in last some years. It is a family of methodologies that has been developed to tackle the problems associated with other processes. Many of the principles originating from the JAD process have been incorporated into Agile development process. Where JAD emphasizes producing a requirements specification and optionally a prototype, Agile focuses on iterative development, where both requirements and the end software product evolve through regular collaboration between self-organizing cross-functional teams. Agile process models give less stress on analysis and design. Implementation begins much earlier in the life cycle of the software development. This process model demands fixed time and quality is non-negotiable.

One of the Agile Methodology is Extreme Programming (XP) which is based on iterative enhancement model. It is a deliberate and disciplined approach to software development, which is aimed to solve customer requirement change problem. The methodology is designed to deliver the software as needed by the customer. It is based on four essential values: communication, simplicity, feedback and courage. Unlike other traditional software development models, such as Waterfall, which conducts analysis, design, implementation and testing once in its long development cycle, or Spiral model, which has shorter, iterative development cycles, extreme programming is blending all these activities, a little at a time, throughout the entire software development life process. Like the other agile software development, XP attempts to reduce the cost of change by having multiple short development cycles, rather than one long one as portrayed in Figure 10 [LOF05].

It only works on teams of twelve or fewer people [11]. Industrial Extreme Programming (IXP) was introduced as an evolution of XP. It is intended to bring the ability to work in large and distributed teams. It then supported flexible values [8]. There is not enough data to prove its usability and management of project can view paired programming as a waste of resources.

These days, the majority of the software development project involve some level of reuse of existing artifact like design or code modules. Rational Unified Process (RUP) considers reuse of existing classes in one of the initial phases. The concentration is on object oriented development. According to Boehm and Abts (1999), both the waterfall model and evolutionary development are unsuitable for component based development.

The component-based development model leads to software reuse, and reusability provides software engineers with a number of measurable benefits, portrayed in Figure 11. The unified software development process is representative of a number of component-based development models that have been proposed. Using a combination of iterative and incremental development, the unified process defines the function of the system by applying a scenario-based approach [7]. Some of the component based software developments are: Somerville’s Model in which he suggested to follow sequential approach and requirements are outlined briefly and later on refined and modified so that they can comply with component. Kotanya Model (2003) takes into account activities related to component use as well as component management. The Y-Model (Capretz, 2005) is another alternative to address software reusability during component-based software production. It is very difficult to follow either a strict top-down or bottom-up approach and that it is often necessary to switch over between them. Qureshi and Hussain (2008) model are largely influenced by object oriented software development life cycle. Existence of component repository is essential in this life cycle model. Component repository supports the application developer during the analysis and selection phase. But there is no explicit mention of how and when components are added to the repository.

Anton Jansen and Jan Bosch [4] research show new perspectives on software architecture that views software architecture as a composition of a set of explicit design decisions. This makes the design decision an explicit part of software architecture, which has become an accepted concept in research and industry. The reuse model follows the component based approach, but this approach is not guided by domain analysis. It does not provide complete life cycle for software development because it considers only those systems which can be built using existing components only.

Li (2009) did a practical study on the adoption of the component based development process by the software industry. The study analyzed 133 software development projects from 127 companies. In around 75% of the studied project, the development process was chosen in advance and mainly waterfall, waterfall with prototyping and incremental mixed with prototyping processes are used. Kaur (2010) did a small study, which was carried out to know the component based development processes following in the software industry and restricted the research to new product development only. The study examined over 30 medium and large size companies, both national and international levels. It is surprising that linear structure and rigid designed waterfall model is still followed in the software industry. Waterfall model with prototyping for requirement analysis, spiral and classical V- model are still used for software development by various companies.

2.3 FAILURE REPORTS ON SOFTWARE PRODUCTS

The software industry has evolved through many eras. Each era has its own distinctive characteristics, but over the years the software’s have increased in size and complexity. Several problems are common to almost all of the eras.

The Software Crisis dates back to the 1960’s when the primary reasons for this situation were less than acceptable software engineering practices. In the early stages of software there was a lot of interest in computers, a lot of code written but no established standards. Then in early 70’s a lot of computer programs started failing and people lost confidence and thus an industry crisis was declared. Various reasons leading to the crisis included:

Hardware advances outpacing the ability to build software for this hardware.

The ability to build in pace with the demands.

Increasing dependency on software’s

Struggle to build reliable and high quality software

Poor design and inadequate resources.

This failure though identified in the early years, still do exist. Software projects fail when they do not meet the criteria for success. Most of the IT projects run over budget or are terminated prematurely and those that reach completion often fall far short of meeting user expectations and business performance goals. Various reports on failure of software product are projected by Dan Galorath [5].There is several updates to the Standish "Chaos" reports. The 2004 report shows in the Figure 12:

The report proves that Successful Projects were just 29%, Challenged Projects were 53% and Failed Projects were 18%.

Standish Findings is revealed in Figure 13 by year updated for 2009:

TCS (Tata Consultancy Services) in 2007 released the report of software failure, as portrayed in Figure 14:

The results of the report shows, 62% of organizations experienced IT projects that failed to meet their schedules, 49% suffered from budget overruns, 47% had higher-than-expected maintenance costs, 41% failed to deliver the expected business value and 33% failed to perform against expectations.

In Avanade Research Report of 2007 following Figure 15 finding were confirmed:

The failure due to system specification was 66%, 51% due requirement understanding and 49% due to technology selection.

ESSU (European Service Strategy Unit) Research Report in 2007 gives an idea about percentage of failure in software industry:

Jama Software partnered with Ravenflow [], conducted industry-wide survey including over 808 participants. In 2011 the survey report shows that only 17% of the projects had a success rate of 80%, which is too high failure rate. The team who spends at least half that is 50 % of their time managing changes to requirements, a whopping 25% admitted that less than 20% of their project or product launches were delivered on-time and on-budget. Successful product launches were much more likely for a team who spend less than 10% of their time managing change, as more than 23% had a success rate of greater than 80%.

In the survey two key questions were asked by all participants:

What’s a company’s biggest challenge in innovation, communication or documenting requirements?

Do you use agile or waterfall?

In the response to the first question, over seventy percent agree that the biggest challenge is gaining a clear understanding of what the customer or client wants. The answer of the other but ultimate question is depicted in Figure 18, the majority of participants responded that their software development process is a hybrid.

To avoid these failures and thus improve the record, what is needed is a better understanding of the process, better estimation techniques for cost time and quality measures. But the question is what is a process? Process transforms inputs to outputs i.e. a product. A software process is a set of activities, methods and practices involving transformation that people use to develop and maintain software.

The process that has been defined and adopted needs to be managed well and thus process management comes into play. Process management is concerned with the knowledge and management of the software process, its technical aspects also ensures that the processes are being followed as expected and improvements are shown.

From this we conclude that a set of defined processes can possibly save us from software project failures. But it is nonetheless important to note that the process alone cannot help us avoid all the problems, because with varying circumstances the need varies and the process has to be adaptive to these varying needs.

Rich Hilliard gives an overview of the contributions of IEEE 1471 to the discipline of software architecture representation which fits in the theory of phased development model for different phases in software development model [24], [25]. The research done by Jonathan Lee [18] describes the software engineering as a problem solving process. Where the software process model approaches divide the development process into various phases/activities or according to functionality. But these models still don't follow the technique of technically analyzing the problem, where the technical problems are identified and divided into sub-problems that are first independently solved and later integrated into the overall solution. The client problems may be ill-defined and include many vague requirements, but the main focus is on the precise formulation of objectives, quality criteria and the constraints for given requirement or problem. In technical analysis part, we can easily put this specification on each small unit of the problem.

Providing a solution for a given problem is not simple, it involves the accumulation and use of the huge amount of knowledge. The solution space analysis approach is still not integrated into software process models. It aims to identify the right solution domains for the given problems and extract the relevant knowledge from these domains to come up with a feasible solution. To provide quality software, it is necessary to identify the important knowledge sources for a given problem. Not all the solutions identified for a given problem are desirable. In the alternative management process, different alternative solutions are searched and evaluated against explicit quality criteria [13], [22]. The high risk in software development led to the inclusion of managerial, financial and psychological factors in models [1], [14], and [16]. Shaw and Garlan [15] identify seven levels of design specification capability which supports the concept of components, composition, validation, alternatives and finally automation. In the component based development, cost, time and reliability risk for an organization developing software system will shrink to component level that can be managed effectively at any stage.

2.4 FOREMOST PROBLEMS WITHIN THE SOFTWARE DEVELOPMENT APPROACHES

The most primary problem within the software development approaches is the inability to manage the cost, resources, functionality and schedule necessary for a software development. Few projects achieve all these and many more are delivered which fail on one or more of these criteria, and a substantial number are cancelled having failed badly. They are number of software projects that succeeded or failed. So are these the foremost factors for success of a project which are based on only these three criteria? Well they are no one factor that causes the failure of the project, a number of factors are involved.

As mentioned earlier the Jama Software partnered with Ravenflow [], presented a survey report in 2011 which consist of over 808 participants of the software industry. In this survey, one interesting question was asked to every participant: when a project or product isn’t successful, what are typical causes? The answers as represented in Figure draw attention to three main points most typical for failure are unrealistic schedule or expectation, missing or poorly defined requirements and scope creep. The success for a project or product requires a tight control on the process of software development.

In Planit Testing Index 2012: Causes of Project Failure, mention some of the more reasons for failure and its percentage of contribution in it for 2012, which has been shown through the Figure 20 below [Planit]. According to these details, the one area of the Software Development Life Cycle that is always associated with project failure is poor management of changing business requirements. In 2012 there was no difference, unfortunately it rises to 68 percent and increase of 9 percent over the previous years. The rest factors in contribution of failure were reduction of project or IT budgets, changing government policy, technical, mergers or acquisitions of project and other.

Let’s take a look at a few of the foremost problems within the software development approaches which require considerable attention are:

2.4.1 Extracting Requirements

Extracting the requirements of a desired software product is the first task in creating it. Sometimes the goal of a project may be only partially clear due to a poor requirement gathering in the definition stage of a project. Many projects have high levels, vague and generally unhelpful requirements. This leads the developers having no input from the user and build what they believe is required, without having any real knowledge of the business for which the project is being developed. Inevitably when the system is delivered, the user declares, it does not do what they needed it to. Defining clear requirements for a project can take time and lots of communication, but sometimes goals and objectives might be unclear because project sponsors lack the experience to describe what they really require. The user should know what they require from the project and be able to specify it clearly. However as a user is non-IT specialist, developer must extracting requirements from the user through his/her skills and experience in software engineering.

2.4.2 Lack of User/Client Involvement

In development of software projects, lack of user involvement can lead the project to miss or poorly defined requirements. User involvement makes the development team of the project committed to the product, which otherwise may be the reason of project failure. The senior management of the developing product needs to establish a working environment in which the customer can actively participate in the project and communicate with the team. This helps the development team to get a clear view of the functionality required for a project.

2.4.3 Varying Requirement

If software project requirements are poorly defined (unclear, incomplete, not reflective of business goals and objectives) projects are more likely to fail. Varying requirements which are not controlled are bound to hinder the project outcome. Establishing control on the varying requirement gives stable guidelines that determine how and when requirements can be added, removed or implemented, and how the costs can be applied to these changes, prior to the project execution, can help in the successful completion of the project.

2.4.4 Unrealistic Schedule or Expectations

Projects with unrealistic schedule or expectations are also about equally likely to be poorly managed projects that fail to validate the feasibility of satisfying user expectations, or well-managed projects that try to validate feasibility and find unavoidable factors, such as immature technology, overhyped COTS products, or a saturated market that justify a project’s prompt termination. Software developers are locked into their favorite solution or approach and are unwilling to accept a different idea. Spend more time leading and less time controlling.

2.4.5 Over Budget

Late projects usually cause costs. One of the causes of budget over run is the employment of more staff to try and make sure that the project is delivered on time, and with all the required features. Sometimes, the client is not even aware that the budget has been over shot until the bill arrives. This is a symptom of bad project management, and can be avoided. Any late cycle changes in hardware or software specifications will cause a project to run over budget. These needs of the client are required to be controlled. Of course, if the client has been made aware that their request will cause an overrun, then this might be acceptable for the benefits that the new development brings.

2.4.6 Testing

The project failure is often caused by lack of proper testing. While software developers focus on creating code, they do not deal with testing. Testers are those who should do "testing" during the development process. Without documented business procedures, there is no clear basis for testing the system and no way of gauging the benefits of implementation. Even the most sophisticated software testing methodologies and tools require a detailed definition of what is expected of the software to be given before their sophistication can be fully exploited. Without a process definition this is not possible to provide. Regression testing is not performed at each release and functionality formerly present in the software is not verified and can be lost. The result of all this is a system that is not fulfilling business needs, and which is falling short of expectations.

2.4.7 No Explicit Control Steps in Development

The projects generally fail due to poor or no control on the development. Proper control needs to be performed on vague/changing/new requirements, to overcome the undesirable effects on the system. Uncontrolled development leads to over budget, poor quality and testing. Each phase of development must progress under controlled monitor. Proper controlling can prevent poor performance. This is one of the golden rules of any undertaking project, but is even more important for IT projects. A complex system needs more control over development than other, simpler ones.

2.5 CONCLUSION

Thus, a number of software process models have been studied. Most of the existing techniques manage one or more concerns of the development process. From the insight gained from this study, contemporary software process models is needed, which handles various issues like requirement changes, software reuse, flexible design, user involvement and tight control over quality, cost and schedule.

Software Engineering problems are complex and include many different kinds of concerns. A problem may include various types of requirements; require different kinds of solution domain knowledge, various goals and different abstractions. For large and complex problems, it is just practically impossible to cope with all these concerns at a time. This means that the problem cannot be solved in one step. A traditional technique for coping with complexity is the decomposition of the problem into sub-problems. The engineering disciplines apply this technique and decompose the overall engineering process into phases. A phase represents a set of related activities to solve a particular problem. Each phase can be implemented using a model called 3C-Model, which we have developed. This model will help in generalizing the process of solving the problems in each phase. It also helps in implementing the component based development approach, which provides a dynamic environment to complete software development. This will widen the scope for software development and provide firmer control over the software development process. Since it is a component driven approach, the risk associated with cost and time will be limited to component only and ensure the overall quality of software systems. This will also reduce the development cost and time by considering the changing requirements of the client, risk assessment, identification, evaluation and composition of relative concerns at each phase of the development process.

3.2 FUNDAMENTAL PARTS OF 3C-MODEL

The 3C-Model in development process utilizes Domain Engineering and divides the problem solving approach of each phase in three fundamental parts: Capture, Control and Context

3.2.1 Capture

The real problem is defined or captured in this part, which arise due to client requirements. It comprises of five concepts: Need, Problem Description, Solution Domain Knowledge, Alternative, Solution Description and Artifact.

Need represents an unsatisfied situation existing in the context (environment). The function Input represents the cause of a need.

Problem Description represents the description of the problem. The function Conceive is the process of understanding what the need is and expressing it in terms of the concept Problem Description.

Solution Domain Knowledge represents the background information that is used to solve the problem. The function Search represents the process of finding the relevant background information that corresponds to the problem.

Alternative represents the possible alternative solutions. The function Generate serves for the generation of different alternatives from the solution domain knowledge. After alternatives have been generated, the problem description can be refined using the function Refine.

Solution Description represents a feasible solution for the given problem. It uses the relevant background information to provide a solution description that conforms to the problem description. The function Detail is used to for the description of a selected alternative.

Artifact represents the solution for the given need. The function Implement maps the solution description to an artifact. The function Output represents the delivery and impact of the concept artifact in the context. The function Initiate represents the cause of a new need because of the produced artifact.

3.2.2 Control

The development process in software engineering starts with the need, while the goal is to arrive at an artifact by applying a sequence of actions. Since this may be a complex process, the concepts and functions that are applied are usually controlled. This is represented by the Control part in the model. The controller observes variables from the system, evaluates this against the criteria and constraints, produces the difference, and performs some control actions to meet the criteria.

(Mathematical) Model represents a description of the concept Alternative. The function Analyze represents the process of analyzing the alternative.

(Quality) Criteria represent the relevant criteria that need to be met for the final artifact. The function Evaluate assesses the alternative with respect to (Quality) Criteria and Constraints.

Constraints represent the possible constraints either from the context or as described in Problem Statement.

Heuristics/Optimization Techniques represents the information for finding the necessary actions to meet the criteria and constraints. The function Select/Optimize selects the right alternative or optimizes a given alternative to meet the criteria and the constraints.

The Control part of 3C-Model is based on five concepts: Interpret, Representation of Concern, Criteria, Adapter and Improve.

Interpret represents the process of retrieving information from the concept or function that needs to be controlled.

Representation of Concern stands for a description of the concept or function that is controlled.

Criteria signifies the relevant criteria that need to be met for the given concept or function.

Adapter represents the information for finding the necessary actions to meet the criteria.

The concept Improve performs the required actions to meet the criteria.

3.2.3 Context

Both the control and the problem-solving activities take place in a particular context. Context can be expressed as the environment in which software development takes place including a broad set of external constraints that influence the final solution and the approach to the solution. Constraints are the rules, requirements, relations, conventions, and principles that define the context of software engineering, to be precise, anything which limits the final solution. Since constraints rule out alternative design solutions directing engineers into taking action on what is doable and feasible.

The context also defines the need, which is illustrated by a directed arrow from the context to the need concept. Apparently, the context may be very wide and include different aspects like the engineer’s experience and profession, culture, history, and environment.

3.3 DOMAIN ENGINEERING IN 3C-MODEL

The development of large and complex software systems requires a clear understanding of the desired system features and the capabilities which software required to implement these features. Software reuse, which has long promised improvements in the development process, will become feasible only when the features and capabilities common to the systems within a domain can be properly defined in advance of software development. Currently Domain Engineering is well known concept in software engineering. Domain Engineering is an activity of collecting, organizing and storing past experience in the building systems or parts of systems in a particular domain in the form of reusable assets (i.e. reusable work-products), as well as providing an adequate means for reusing these assets (i.e. retrieval, qualification, dissemination, adaptation, assembly, etc.) when building new systems[PA91].

Domain Engineering includes three main process components:

Domain Analysis

Domain Design

Domain Implementation

The main purpose of each of these components is given in Table 1.

Domain Analysis, the systematic exploration of the software systems to define and exploit commonality, defines the features and capabilities of a class of related software systems. Thus, the availability of Domain Analysis technology is a factor that can improve the software development process and promote the software reuse by providing a means of communication and a common understanding of the domain. Domain analysis was first introduced by Neighbors [Nei81] as "the activity of identifying the objects and operations of a class of similar systems in a particular problem domain". In other words, the Domain Analysis can be seen as a process where information used in developing software system in identified, capture, structure and organized for further reuse. More specifically, Domain Analysis deals with the development and evolution of information infrastructure to support reuse [Ruben Prieto-Diaz].Domain Analysis selects and define the domain of focus and collect relevant domain information. The sources of domain information include existing systems in the domain, domain experts, system handbooks, textbooks, prototyping, experiments, already known requirements on future systems, etc [Arr89].

The purpose of Domain Design is to develop a structural design for the systems in the domain. Domain Design is followed by Domain Implementation. During Domain Implementation appropriate technologies are applied to implement components, generators for automatic component assembly, reuse infrastructure (i.e. component retrieval, qualification, dissemination, etc.), and application production process (Czarnecki & Eisenecker, 2000).

The phased model uses domain analysis to identify the domains, bounding them and discovering commonalities and variability’s among the systems. This information is captured in models that are used in the domain implementation process to create artifacts such as reusable components, a domain-specific programming language or application generators that can be used to build new systems in the domain. A key idea in systematic software reuse is the domain, a software area that contains systems sharing commonalities.

3.4 LOGICAL VIEW OF 3C MODEL

The 3C-Model generalizes the problem solving process in each phase of the software development. It has logical building blocks which are three processes: technical problem analysis, solution space analysis and alternative management. These three processes in the 3C Model are based on the concepts as mentioned by Bedir TekinerdoÄŸan and Mehmet Aksit [].

3.4.1 Technical Problem Analysis

The definition of the problems and the sub problems that need to be solved are in the technical problem analysis. Identification of problem analysis process and technical problems are made and structured into loosely coupled sub-problems that are first independently solved and later integrated into the overall solution. The concept of technical problem describes the fundamental aspects that may not have been present in the original requirements. A clear understanding of the problem is part of the solution and as such this decrease the distance to the final solution, as depicted in Figure 2.

Let us explain these processes in more detail now.

Generalize The Requirements

Finding out the problems from a requirement specification is not a simple task. The reason for this is that the clients may not be able to accurately describe the initial state and the desired goals of the system. The client requirements may be precise and present only precise wordings of a more general problem. Thus, to provide the wider view and make the right problems detect we abstract and generalize from the requirement specification and try to solve the problem at that level. Usually, this abstraction and generalization process allows defining the client’s requirements in entirely different terms and therefore may suggest and help to find out problems that were not considered in the initial requirements.

Identify Sub-Problems

When the requirement specification has been arranged into a more general and broader form, we derive the technical problem that usually consists of several sub-problems. Problem solving is defined as the action of a process by which the transformation from the initial state to the target is achieved [Newell & Simon 76]. Thus we need first of all to find out and explain the problem. For this, in the generalized requirement specification we look for the vital feature that need to be considered while developing. These features are identified by considering the terms in the generalized requirements specification, the general knowledge of the software product which needs to be developed and the interaction with the clients. This process is supported by the results of the requirements analysis stage and utilizes the provided use-case models, scenarios, prototypes and formal requirements models.

Specify Sub-Problems

The identification of a sub-problem goes in parallel with its specification. The major distinction between the identification and the specification of a problem is that the first activity focuses on the process for finding the relevant problems, whereas the second activity is concerned with its exact formalization. A problem is defined as the gap between the initial state and the goal. Thereby, the specification of the technical problems consists of describing its name, its initial state and goal.

Prioritize Sub-Problems

After the decomposition of the problem into several sub-problems the process for solving each of the sub-problems can be started. The selection and ordering, in which the sub-problems are solved, though may have an impact on the final solution. Therefore, it is necessary to prioritize and order the sub-problems and then process the sub-problems according to the priority. The prioritization of the sub-problems may be defined by the client or the solution domain itself. The latter may be the case if a sub-problem can only be solved after a solution for another sub-problem has been defined.

3.4.2 Solution Domain Analysis

Here corresponding solution domains are identified and search for complete background information required to solve the problems is achieved. The general idea is that requirements are specified using some representation and this should be refined along with the software development process until the final software is delivered.

In the Figure 4, the arrow from the concept Solution Domain Analysis to the concept Solution represents the previous requirement of deriving solution abstractions from the Solution Domain Analysis. The double side arrow from the concept Technical Problem Analysis to the concept Solution Domain Analysis represents the search, leveraging of the Solution Domain Analysis by the identified problems and refine the solution concept before the final solution.

The solution domain analysis process consists of the following activities:

1) Identify and prioritize the solution domains for each sub-problem

2) Identify and prioritize knowledge sources for each solution domain.

3) Extract solution concepts from solution domain knowledge.

4) Structure and refine the solution domain concepts.

An abstract model for illustrating the relationships between the concepts of Technical Problem, Sub-Problem, Solution Domain, and Solution Domain is represented Figure 5 []. Hereby, the round rectangles represent the concepts and the directed arrows represent the associations between these concepts. From the figure it follows that for each Technical Problem a solution is provided by one or more Solution Domains. The concept Problem includes zero or more Sub-Problems. Each Solution Domain includes one or more Knowledge Sources from which one or more Solution Domain Concepts may be derived that solves the concepts Problem and Sub-Problem. Identify and Prioritize the Solution Domains.

Let us explicate these activities in more detail now.

Identify And Prioritize The Solution Domains For Each Sub-Problem

For the overall problem and each sub-problem, the search is conducted for the Solution Domains that provide the solution abstractions to solve the Technical Problem. The Solution Domains for the overall problem are further more general than the Solution Domains for the sub-problems. As well as, each sub-problem may be recursively structured into sub-problems requiring more definite Solution Domains. Barrier in the search for Solution Domains may be the probable wide space of Solution Domains leading to a prolonged search process. Categorizations of the Solution Domain Knowledge into smaller sub-domains are performed to support this process. There are dissimilar categorization possibilities. In library science, for example, the categories are represented by facets that are groupings of related terms that have been derived from a sample of selected titles [Rubin 98]. The Solution Domain Knowledge can be categorized into the application, mathematical and computer science domain knowledge.

There is no mandatory for a particular categorization of the Solution Domain Knowledge and similarly other classifications besides of the above approaches may be equally used. If the Solution Domains have been sufficiently organized one may still encounter some problems and the Solution Domain Analysis may not always declare a feasible solution domain model. This is especially the case if the Solution Domain does not exist or the concepts in the Solution Domain are not fully searched yet and/or compiled in a reusable format. The flow diagram for the feasibility study on Solution Domain Analysis is shown in Figure 6 []. The diamonds represent decisions, the rectangles the processes and the rounded rectangle the termination of the flow process.

If the Solution Domain Knowledge does not subsist, one can either terminate the feasibility analysis process or initiate a scientific research to explore and formalize the concepts of the required Solution Domain. The first case directs to the conclusion that the problem is actually not entirely solvable due to lack of knowledge. The latter case is the more long-term and difficult option and falls outside the project scope. If a suitable Solution Domain exists and sufficiently specified, it can be reused to extract the necessary knowledge and apply this for the development. It may also happen that the Solution Domain concepts are well-known but not formalized [Shaw & Garlan 96]. In that case it is necessary to specify the Solution Domain.

Identify And Prioritize Knowledge Sources For Each Solution Domain

Every identified Solution Domain may cover a wide range of Solution Domain Knowledge sources. These knowledge sources may not all be appropriate and vary in quality. For differentiating and validating the Solution Domain Knowledge sources, basically two quality factors are consider the objectivity and relevancy. The objectivity quality factor refers to the Solution Domain Knowledge sources itself, and defines the general acceptance of the knowledge source.

The relevancy factor refers to the relevancy of the Solution Domain Knowledge for solving the known Technical Problem. The relevancy of the Solution Domain Knowledge is different from the target quality. A Solution Domain Knowledge entity may have a high degree of objectivity quality because it is very accurately defined and supported by a community of experts, though, it may not be relevant for solving the identified problem because it addresses dissimilar concerns. To be suitable for solving a problem it is required that the Solution Domain Knowledge is both objectivity and relevant. Therefore, the identified Solution Domain Knowledge is prioritized according to their objectivity and relevancy factors.

This can be expressed in the empirical formula:

Priority(s) = (objectivity(s)), (relevance(s))

Hereby priority, objectivity and relevance represent functions that define the corresponding quality factors of the arguments that stand for the Solution Domain Knowledge Source. For solving the problem, first the Solution Domain Knowledge with the higher priorities is used. The measure of the objectivity degree can be determined from general knowledge and experiences. The measure for the relevancy factor can be determined by considering whether the identified Solution Domain source matches the goal of the problem. Note down, though, that this formula should not be interpreted too firmly and rather be considered as an intuitive and practical aid for prioritizing the identified Solution Domain Knowledge sources rather.

Extract Solution Domain Concepts From Solution Domain Knowledge

When the Solution Domains have been identified and prioritized, the knowledge acquirement from the Solution Domain Sources can be initiated. The Solution Domain Knowledge might include a lot of knowledge that is covered by books, research papers, case studies, reference manuals etc. Due to the vast size of the Solution Domain Knowledge, the knowledge acquisition process can be a labor-intensive activity and as such a organized approach for knowledge acquisition is required [Partridge & Hussain 95], [Gonzales & Dankel 93], [Wielinga et al. 92].

Basically differentiation is done in between the knowledge extraction and concept formation process. Knowledge extraction focuses on extracting the knowledge and verifying the correctness and consistency of the extracted data. Hereby, the irrelevant data is ignored and the relevant data is provided as input for the concept formation process. Knowledge extraction techniques have been described in several publications and its role in the knowledge acquisition process is reasonably well-understood [Wielinga et al. 92], [Meyer & Booker 91], [Diaper 89a], [Firlej & Hellens 91].

The concept formation process utilizes and abstracts from the knowledge to form concepts. In the liteseveral concept formation techniques have been identified [Parsons & Wand 97][Reich & Fenves 91][Lakoff 87]. One of the basic abstraction techniques in forming concepts is by identifying the variations and commonalities of extracted information from the knowledge sources [Stillings et al. 95][Howard 87]. Usually a concept is defined as a representation that explains the common properties of a set of instances and is identified through its name.

Structure And Refined The Solution Domain Concepts

The identified solution domain concepts are structured using generalization specialization relations and part-whole relations, respectively. In addition, also other structural association relations are used. Like the concepts themselves the structural relations between the concepts are also derived from the solution domains. For the structuring and representation of concepts, so-called concept graphs are used. A concept graph is a graph which nodes represent concepts and the edges between the nodes represent conceptual relations.

After identifying the top-level conceptual, structural design the focus is on each sub-problem and follows the similar process. The refinement may be essential if the structural design concepts have a complex structure themselves and this structure is of significant for the eventual system. The ordering of the refinement process is determined by the ordering of the problems with respect to their earlier determined priorities. Structural design concepts that represent problems with higher priorities are handled first and in the similar style the refinement of the structural design concepts is done.

3.4.3 Alternative Management