Software Process Improvement

by Robin B. Hunter (Editor), Richard H. Thayer (Editor), Mark C. Paulk (Foreword by)

Overview

The creation of quality software on time and within budget has been a major problem facing the software industry for several decades. Consideration of software development and maintenance as an engineering discipline to control the developmental process can help alleviate these obstacles. This new book focuses on the best practices for software process improvement (SPI) and related international standards providing a valuable guide and reference.

The text is a collection of original and republished papers providing a significant survey on the use of SPI and software process assessment (SPA) as practiced by companies such as Lockheed Martin, Siemens, and Hewlett Packard. Among the important features of the book are chapters on software process evaluation, how to best perform SPI, ISO 9000 and TickIT-an alternative approach to SPA, as well as the latest information on the CMM integration project. The text also provides vivid descriptions on the most important international and national standards for SPI, in particular ISO 9001, ISO 9000-3, ISO/IEC 9126. ISO/IEC 15504, ISO/IEC 12207.

Software Process Improvement benefits software managers who want to learn about the requirements and effects of SPI, software process staff who need to understand the mechanisms of SPA and SPI, software developers who are affected by SPI and need to know how to apply it, and college students who want to understand the various methods of SPA and SPI.

Product Details

ISBN-13:	9780769509990
Publisher:	Wiley
Publication date:	12/11/2001
Series:	Practitioners , #25
Pages:	638
Product dimensions:	8.30(w) x 10.90(h) x 1.30(d)

About the Author

Robin B. Hunter is a senior lecturer in the Computer Science Department in the University of Strathclyde in Glasgow, Scotland. He has been in this department for over thirty years, having previously been on the teaching staffs of the University of Glasgow and the University of Newcastle upon Tyne. He has a B.Sc. and Ph.D. degrees, both from the University of Glasgow, and is a member of the IEEE Computer Society.

Read an Excerpt

Software Process Improvement

---

John Wiley & Sons

ISBN: 0-7695-0999-1

---

Chapter One

Software Process Assessment

1. Introduction to Chapter

The production of high quality software on time and within budget has largely eluded the software industry world-wide for several decades now. What became known as the "software crisis"-a term coined in the late sixties-has refused to go away. As software has become more pervasive, as well as becoming larger and more complex, so the opportunities for software 'disasters' caused by faulty or overdue software have become much greater, more than offsetting any improvements to software development and maintenance methods. Consideration of software development and maintenance as an engineering discipline, and control of the processes involved, however, have helped to alleviate the situation and offer hope for the future.

When the term "software engineering" was introduced in 1969, it was hailed as identifying an approach to software production that would lead to an engineering-type process for developing (and maintaining) software. Since then, significant contributions to the development of software engineering been made, including

the use of structured analysis and design,

the introduction of the object-oriented approach to software development,

the application of software measurement.

While the benefits of the engineering approach have been real and significant, software "disasters" continue to occur and , according to Curtis, 25 percent of software projects still do not reach fruition, many projects are up to 40 percent over budget, and project schedules are only met about half of the time.

The emphasis on the engineering aspects of software development and maintenance can lead to an overemphasis on the use of methods and tools to produce software products, whereas it is now being realized that people and process issues also have to be addressed if quality products are to be produced. People issues arise in software development and maintenance due to the fact that they are largely human-based activities. Support for the tedious, and some of the error prone, parts of the process can, and should be, supported by tools where appropriate. However the creative parts of the process, including the design, are largely performed by people and, inevitably, the quality of the software product depends on the abilities of, and the care taken by, the people involved in its production and maintenence.

The process approach to software development and maintenance attempts to model all significant aspects of software development and maintenance. The ultimate aim of this approach is to produce software in a controlled way that will be on time, within budget, and of appropriate quality. The process approach is not so much concerned with the use of particular methods or tools but more with using a well-defined and controlled process that may be supported by appropriate methods and tools.

According to Humphrey, the software process is defined as

The set of activities, methods, and practices that are used in the production and evolution of software.

At the heart of the process approach to software development and maintenance/evolution is the concept of Software Process Assessment (SPA), which is concerned with assessing a software process against a process standard or framework. Such a standard or framework would normally incorporate a number of achievement levels of increasing sophistication. For example, the Capability Maturity Model for Software (SW-CMM) developed by the Software Engineering Institute (SEI) in Pittsburgh has five levels of achievement.

Software Process Assessment can be used for a number of purposes, the two principal ones being

Capability Determination (CD), used by software procurers to determine the capability of potential contractors;

Software Process Improvement (SPI), used by software producers to improve their software processes in line with their business aims.

In addition, the results of process assessments are sometimes used to represent the state of the practice in software development, though this should only be done with care, as the sample used for this purpose is rarely representative of the industry as a whole.

While the original purpose of SPA was primarily capability determination, it is now used extensively for SPI and it is with SPI that this tutorial is principally concerned. However, process improvement depends on process assessment, and much of this tutorial will also be concerned with SPA.

The notions of process assessment and process improvement are not unique to software but have been developed from more general notions, including:

Shewart's Plan-Do-Check-Act cycle for quality improvement, developed in the 1930's;

Juran's four steps to quality improvement:

Collect defect data

Develop a theory

Test the theory

Implement corrective actions;

Deming's approach to continuous improvement.

2. Description of Articles

The first paper in this chapter introduces and summarizes the main approaches to software process improvement. It is by Mark Paulk and is entitled "Models and Standards for Software Process Assessment and Improvement". In it Paulk introduces and describes the "big three" methods for assessing software products:

ISO 9000 - Quality Management Systems (see Chapter 7);

SW-CMM (The Capability Maturity Model for Software) (see Chapter 2);

ISO/IEC 15504/SPICE (see Chapter 8).

For each of these methods, he provides an overview, describes the method and its context, identifies its strengths and weaknesses, and discusses its future. In addition, the paper covers more briefly some other methods of process assessment such as Bootstrap (see Chapter 3), SDCE (Software Development and Capability Evaluation), TRILLIUM (see Chapter 3), and Software Productivity Research (see Capers Jones and Chapter 3).

In the conclusions to the paper, Paulk suggests that an organization will normally choose to use one of the "big three" methods of process assessment, based on the environment in which the organization operates. For example, large US companies will tend to use the SWCMM, while multinational companies may tend to use ISO/IEC 15504/SPICE since it is an emerging international standard specific to software assessment. The international series of standards ISO 9000 was pioneered for use with software in the United Kingdom. The series tends to be used most (at least in its software context) in Europe, as well as in certain other countries such as Japan and India.

The Paulk paper also contains a large number of references to published work in the area as well as references to websites where more information concerning the various software assessment methods may be found. More details of many of them can also be found in later chapters of this volume, and in the references associated with these chapters.

The second paper in the chapter is frequently referenced. It is entitled "The Frameworks Quagmire" and is by Sarah Sheard. The author categorizes the various frameworks by the purpose served, which may be one or more of

Standards and Guidelines

Process Improvement Models and Internal Appraisal Methods

Contractor Selection Vehicles

Quality Awards

Software Engineering Life Cycle Models

System Engineering Models

Each of the purposes is described in detail and the characteristics of seven of the frameworks are outlined. While the author sees signs of consolidation of the current frameworks as an encouraging trend, she also draws attention the complexity of the current situation in one of the figures in the paper.

The author gives a reference to a clickable version of the quagmire available on the internet, itself a valuable source of information on the various schemes and frameworks for software quality. The actual website where the "quagmire" is shown has been updated since the illustration shown in the paper was created and is well worth consulting.

Models and Standards for Software Process Assessment and Improvement

Mark C. Paulk Software Engineering Institute Carnegie Mellon University Pittsburgh PA 15213-3890 Tel: [phone] Fax: [phone] Internet: [e-mail]

Abstract

The "software crisis" has inspired a number of efforts by software suppliers and customers to improve the state-of-the-practice in building software. This chapter provides an overview of models and standards underlying the three best-known approaches to software process improvement: the ISO 9001 standard for quality management systems, the Software Engineering Institute's Capability Maturity Model(r) for Software, and the proposed ISO/IEC 15504 standard for software process assessment. Other approaches are also briefly discussed, including the SEI's Personal Software Process[SM] and Team Software Process[SM].

Keywords: Capability Maturity Model for Software, CMM(r), IDEAL[SM], ISO 9000-3, ISO 9001, ISO/DEC 12207, ISO/IEC 15504, software process assessment, software process improvement, Personal Software Process[SM], PSP[SM], SPICE, Team Software Process[SM], TickIT, TSP[SM].

Outline

1. Introduction

2. ISO 9000 - Quality Management Systems 2.1 An Overview of ISO 9001 2.2 Placing ISO 9001 in Context 2.3 ISO 9000 Certification 2.4 Strengths and Weaknesses of ISO 9000 2.5 The Future of ISO 9000

3. The Capability Maturity Model for Software 3.1 An Overview of the Software CMM 3.2 Placing the Software CMM in Context 3.3 CMM-Based Appraisals 3.4 Strengths and Weaknesses of the Software CMM 3.5 The Future of the Software CMM

4. ISO 15504 - An International Standard for Software Process Assessment 4.1 An Overview of ISO 15504 4.2 An Overview of the ISO 15504-2 Reference Model 4.3 Strengths and Weaknesses of ISO 15504 4.4 The Future of ISO 15504

5. Other Models and Standards

6. Conclusions

References

1. Introduction

The size, complexity, and power of software-intensive systems have exploded since the beginning of the Information Age, and software has become a core competency that is critical to high technology companies. Computers are now an integral part of our day-to-day lives. We use them, even if we don't realize it, when we drive a car, withdraw money from an automated teller machine, or telephone a friend.

At the same time, anyone familiar with computers is also familiar, frequently painfully so, with the "software crisis" [Gibbs94]. Our ability to build software-intensive systems is orders of magnitude greater today than it was five decades ago, but our appetite for software has grown even faster, and the software industry is still evolving from a craft to an engineering discipline [Shaw90]. Historically, the result has been the chronic software crisis: software is (almost) always later than expected, more expensive than planned, and with less functionality than hoped. There is hope, however, that we have turned the corner on the software crisis.

Why do we have a chronic software crisis? Partially it is because the software discipline is still maturing, but some of the pain is caused by human nature. In response to the question "Why does software cost so much?", Weinberg replies "Compared to what?" and DeMarco points out that the question is based on the assertion that software is too pricey [DeMarco95]. DeMarco suggests that this assertion is a negotiating position; people complain because they know we work harder when they complain. In one survey, most of the responding professional software managers reported that their estimates were dismal, but they weren't on the whole dissatisfied with the estimating process [Lederer92]!

All too many software professionals would agree with DeMarco, but many software managers and customers are vitally interested in understanding how to manage software projects more effectively. Can we plan and manage software projects effectively? While some have argued not, the evidence is that we can - within the bounds of the business paradigm that is chosen.

The business paradigm is crucial because for significant software projects we rarely understand all the requirements at the beginning of the software project. The waterfall software life cycle model, which assumes that the requirements are frozen at the beginning of the software project, has largely been superseded by evolutionary or incremental life cycles. Software project management thus emphasizes managing risks and controlling change.

Customers and managers who use schedule pressure and overtime as motivational tools have to deal with the resulting quality tradeoff. Customers and managers who are interested in truly managing software projects - and facing up to a sometimes unpleasant reality - have available a number of approaches for systematically improving the process for developing and maintaining software. The results of successfully applying these approaches give us hope that the software crisis is finally coming to an end. A growing number of software organizations credit their increasing ability to achieve functionality, quality, schedule, and budget goals to systematic improvement of their software processes.

Perhaps the best-known approaches to software process improvement are the International Organization for Standardization's ISO 9001 standard for quality management systems, the Software Engineering Institute's Capability Maturity Model for Software (CMM or Software CMM), and the proposed ISO/IEC 15504 standard for software process assessment. These approaches, among others, apply Total Quality Management (TQM) principles to the software process.

ISO 9000 is a suite of standards dealing with quality management systems that can be used for external quality assurance purposes. ISO 9001, which addresses quality assurance in design, development, production, installation, and servicing, is the standard of specific interest to the software community, but it is much broader in scope than just software; software-specific guidance is provided in ISO 9000-3. Although the scope of ISO 9001 is broader than software, its application can be of value to software organizations [Stelzer96], and ISO 9001 certification is required to do business in many markets.

The Capability Maturity Model for Software [Paulk95a] describes the process capability of software organizations and provides a roadmap for software process improvement. Developed at the request of the U.S. Department of Defense to help identify the capability of software contractors [Humphrey87, Byrnes96, Besselman93], its use for improving the software process has spread far beyond the DoD community. The Software CMM is arguably the best known and most widely used model for software process appraisal and improvement today. The author of this chapter is the product manager for the Software CMM, so there may be some bias in this discussion.

Many models and standards for software process improvement have been developed. This proliferation led to the development of ISO/EEC 15504, a suite of standards for software process assessment [Dorling93]. Popularly known as SPICE (Software Process Improvement and Capability dEtermination), ISO/EEC 15504 is currently under development and may change significantly before its final release. ISO/IEC 15504 will provide a framework for harmonizing different approaches to assessing and improving the software process.

The importance of high-quality software products cannot be overemphasized. Recent UK court decisions and proposed changes to the U.S. Uniform Commercial Code foreshadow a potential for legal action by dissatisfied customers: "UK Court of Appeal judges have ruled for the first time that software should be sold free of major bugs and should work as intended, like other commercial goods ... attempts by software vendors to exclude or limit liability for product performance will be judged in terms of the Unfair Contract Terms Act (1977)." [UK-Court96] Fortunately the increasing maturity of the software field encourages us to think this goal may now be within our grasp.

(Continues...)

---

Excerpted from Software Process Improvement Excerpted by permission.
All rights reserved. No part of this excerpt may be reproduced or reprinted without permission in writing from the publisher.
Excerpts are provided by Dial-A-Book Inc. solely for the personal use of visitors to this web site.

---

Table of Contents

Contributors of Original Papers.

Foreword by Mark C. Paulk.

Preface.

Chapter 1: Software Process Assessment.

Introduction.

Models and Standards for Software Process Assessment and Improvement (Mark C. Paulk)
Advances in Computers, volume 46, pp. 1-33, Academic Press, 1998 (revised).

The Frameworks Quagmire (Sarah A. Sheard)
Crosstalk, volume 10, number 9, pp. 17-22, 1997.

Chapter 2: The Capability Maturity Model for Software.

Introduction.

A Software Maturity Framework (Watts S. Humphrey)
Managing the Software Process, Chapter 1, pp. 3-16, Addison Wesley, 1989.

Software Process Improvement at Hughes Aircraft (Watts S. Humphrey, Terry R. Snyder, and Ronald R. Willis)
IEEE Software, volume 8, no. 4, pp. 11-23, 1991.

Capability Maturity Model version 1.1 (Mark C. Paulk, Bill Curtis, Mary Beth Chrissis, and Charles V. Weber)
IEEE Software, volume 10, no. 4, pp. 18-27, 1993.

Chapter 3: Other Approaches to Software Process Assessment.

Introduction.

BOOTSTRAP 3.0 - A SPICE Conformant Software Process Assessment Methodology (P. Kuvaja)
Software Quality Journal, volume 8, number 1, pp. 7-19, September 1999.

TRILLIUM : A Model for the Assessment of Telecom Product Development and Support Capability (Francois Coallier)
IEEE Software Process Newsletter, volume 2, pp. 3-8, 1995.

Sorting out Six Sigma and the CMM (David N. Card)
IEEE Software, volume 17, number 3, pp. 11-13, May/June 2000.

SEL's Software Process Improvement Program (Victor Basili, Marvin Zelkowitz, Frank McGarry, Jerry Page, Sharon Waligora, and Rose Pajerski)
IEEE Software, volume 12, pp. 83-87, 1995.

Measuring Software Process Improvement (Capers Jones).

Chapter 4: Software Process Improvement: How To Do It.

Introduction.

Overcoming Resistance to Standard Processes, or, "Herding Cats" (Sandra G. McGill)
Software Failure Analysis for High-Return Process Improvement Decisions (Robert B. Grady)
Hewlett-Packard Journal, p. 15, August 1996.

Statistically Managing the Software Process (William Florae, Anita Carleton, Julie Barnard)
IEEE Software, July/August, 2000 (revised).

Chapter 5: Developments Following from the SW-CMM.

Introduction.

The Personal Software Process (Watts S. Humphrey)
Cross Talk, March 1998 (retitled).

Using the TSP on the TaskView Project (David Webb and Watts S. Humphrey)
Cross Talk, pp. 3-10, February, 1999.

The Software Acquisition Capability Maturity Model (Peter A. Kind and Jack Ferguson)
Cross Talk, pp. 13-17, March 1997.

CMM Integration - Enterprise-Wide Process Improvement Using the Next Generation of CMM's (Mike Konrad and Sandy Shrum)
The Rapid Assessment of Software Process Capability (T.P. Rout, A. Tuffley, B. Cahill, and B. Hodgen)
SPICE 2000, pp. 47-55, Limerick, 2000.

SPICE for Space : A Method of Process Assessment for Space Software Projects (Ann Cass, Christian Volcker, Paolo Panaroni, Alec Dorling, and Lothar Winzer)
SPICE 2000, pp. 21-31 Limerick, 2000.

Relating the SPICE Framework and SEI Approach to Software Process Assessment (D. H. Kitson)
Software Quality Journal, volume 5, pp. 145-156, 1996.

Chapter 9: Experiences of Software Process Assessment.

Introduction.

Software Quality and the Capability Maturity Model (James D. Herbsleb, David Zubrow, Dennis R. Goldenson, Will Hayes, and Mark Paulk)
Communications of the ACM, vol. 40, pp. 30-40, 1997.

SPICE Trials Phase 2 (Bob Smith)
IEEE Software Process Newsletter, number 8, pp. 11-14, 1997.

SPICE Trials Assessment Profile (Robin Hunter)
IEEE Software Process Newsletter, number 12, pp. 12-18, 1998.

The Reliability of ISO/IEC PDTR 15504 Assessments (Jean-Martin Simon, Khaled El Emam, Sonia Rousseau, Eric Jacquet and Frederic Babey)
Software Process : Improvement and Practice, volume 3, issue 3, pp. 177-188, 1997.

European Experiences with Software Process Improvement (Fran O'Hara)
ICSE2000, pp. 635-640, IEEE Press, 2000.

Chapter 10: Software Process Improvement for Small Organizations.

Introduction.

Applying and Adjusting a Software Process Model in Practice.The Use of the IDEAL Model in a Small Software Enterprise (Karlheinz Kautz, Henrik Westergaard Hansen, Kim Thaysen)
ICSE2000, pp. 626-633, IEEE Press, 2000.

Process Improvement Towards ISO 9001 Certification in a Small Software Organisation (Elif Demirors,Onur Demirors, Oguz Dikenelli, and Billur Keskin)
ICSE 1998, IEEE Press, 1998.

Software Process Improvement in Small Organizations (Marty Sanders).

Chapter 11 Benefits of Software Process Improvement.

Introduction.

Journey to a Mature Software Process (C. Billings, J. Clifton, B. Kolkhorst, E. Lee, and W. B. Wingert)
IBM Systems Journal volume 33, number 1, pp. 46-61, 1994.

Schlumberger's Software Process Improvement Program (Harvey Wohlwend and Susan Rosenbaum)
IEEE Transactions on Software Engineering, volume 20, number 11, pp. 833-839, 1994.

Accumulating the Body of Evidence for the Payoff of Software Process Improvement—1997 (Herb Krasner).

Chapter 12: Software Process.

Introduction.

Software Processes are Software Too : Revisited; An Invited Talk on the Most Influential Paper of ICSE 9 (Leon J. Osterweil)
ICSE 1997, pp. 540-548, ACM Press, 1997.

Process Modelling—Where Next (M. M. Lehman)
ICSE 1997, pp. 549-552, ACM Press, 1997.

Software Process: A Roadmap (Alfonso Fuggetta)
The Future of Software Engineering, pp. 27-34, ACM, 2000.

Appendix.

Glossary (R.H. and M.C. Thayer).

Editors' Biographies.

Editorial Reviews

"Software Process Improvement is a useful book and an office resource must have." (Software Quality Professional Journal, March 2009)

From the Publisher