Testing IT: An Off-the-Shelf Software Testing Process / Edition 2

by John Watkins, Simon Mills

Overview

Testing IT provides a complete, off-the-shelf software testing process framework for any testing practitioner who is looking to research, implement, roll out, adopt, and maintain a software testing process. It covers all aspects of testing for software developed or modified in-house, modified or extended legacy systems, and software developed by a third party. Software professionals can customize the framework to match the testing requirements of any organization, and six real-world testing case studies are provided to show how other organizations have done this. Packed with a series of real-world case studies, the book also provides a comprehensive set of downloadable testing document templates, proformas, and checklists to support the process of customizing. This new edition demonstrates the role and use of agile testing best practices and includes a specific agile case study.

Product Details

ISBN-13:	9780521148016
Publisher:	Cambridge University Press
Publication date:	12/06/2010
Edition description:	New Edition
Pages:	352
Product dimensions:	6.90(w) x 9.90(h) x 0.90(d)

About the Author

John Watkins has more than thirty years of experience in the field of software development, with some twenty-five years in the field of software testing. During his career, Watkins has been involved at all levels and phases of testing and has provided high-level test process consultancy, training, and mentoring to numerous blue chip companies. He is both a Chartered IT Professional and a Fellow of the British Computer Society, where he is an active member of the Specialist Group in Software Testing (SIGiST), previously serving on committees of the Intellect Testing Group (representing the UK technology industry) and the SmallTalk User Group. He is author of Agile Testing: How to Succeed in an Extreme Testing Environment and currently works for IBM's software group.

Simon Mills has more than thirty years of experience in the field of software quality, having transferred into the world of system testing from a business role. Simon has been involved in testing software in both business and technical or scientific environments from major investment and insurance systems to laser control, cryogenic control, and superconducting applications. He is the founder of Ingenuity System Testing Services, the preeminent testing authority in the field of electronically traded insurance in the United Kingdom. Simon is widely published internationally in conference proceedings, papers, and contributions to books and has presented as an invited speaker in the United States, at EuroStar, and at the World Congress for Software Quality.

Table of Contents

1. Introduction; Part I. The Traditional Testing Process: 2. An overview of testing; 3. Testing techniques; 4. The management and planning of testing; 5. Unit testing; 6. Integration testing; 7. System testing; 8. Systems integration testing; 9. User acceptance testing; 10. Operations acceptance testing; 11. Regression testing; 12. Improving the testing process; 13. Introduction, adoption and maintenance of the testing process; 14. Agile testing; Part II. The Testing Process in the Real World: Illustrative Case Studies: 15. Case Study 1. The British Library; 16. Case Study 2. Reuters product acceptance group; 17. Case Study 3. Crown quality assurance group; 18. Case Study 4. The Wine Society; 19. Case Study 5. Automatic Data Processing Limited; 20. Case Study 6. Confirmit; Part III. The Appendices: Appendix A. Terms of reference for testing staff; Appendix B. Testing guides; Appendix C. Test plan document template; Appendix D. Test specification document template; Appendix E. Test script template; Appendix F. Test result record form template; Appendix G. Test log template; Appendix H. Test certificate template; Appendix I. Re-use pack checklist; Appendix J. Test summary report template; Appendix K. Equivalence partition example; Appendix L. Boundary value analysis example; Appendix M. State transition example; Appendix N. Pairwise testing example; Appendix O. Automated testing tool selection criteria; Appendix P. Usability testing; Appendix Q. Testing process health check; Appendix R. The testing of object-oriented software; Appendix S. Pragmatic test process adoption - a real world example.

Introduction

1.1 Purpose of the Book

This book provides comprehensive and rigorous guidance to workers in the field of software testing for researching or setting up a software testing process within organizations.

The book provides advice and guidance on all aspects of the testing process, including:

• The need to test software and the approach to testing
• Specific details of testing techniques with worked examples
• The planning and management of testing projects
• Testing roles and responsibilities
• Comprehensive details of the testing phases
• Extensive testing document templates, proformas, and checklists
• Recommendations for testing process improvement and the role and use of metrics
• The testing issues facing developers of Object-Oriented and ComponentBased systems

The book covers the testing of software from a number of sources, including software developed or modified in-house, software that represents the modification or extension of exisiting legacy software systems, and software developed on behalf of an organization by a third party.

The book also covers the acceptance testing of commercial off-the-shelf (COTS) software procured by an organization, or COTS software that has undergone development either internally or by a third party on behalf of an organization.

This book should be used in a pragmatic manner, in effect providing a testing framework that can be used by all members of staff involved in software development and testing within an organization to improve the quality of the software they deliver and to reduce timescales, effort, and cost of testing.

Alternatively, the testing process described in this book can be customized to match the specific testing requirements of any particular organization, and a series of realworld case studies are provided to illustrate how this can be achieved.

1.2 Readership

The target audience for this book includes the following people:

• Technical Director/Managers who need to improve the software testing process within their organization (in terms of quality, productivity, cost, and/or repeatability of the process)
• Quality Assurance (QA) professionals (such as company QA Directors or Managers) who need to put in place a formal organization-wide approach to software testing
• Project Managers/Leaders who need to save time, effort, and money and improve quality by adopting a complete, standard, off-the-shelf solution to their testing requirements
• Independent Information Technology (IT), QA, or Management Consultants who provide advice and guidance to clients on their software testing process, for whom the book will represent a key item in their "Consultants Tool Kit"
• Testing/QA Professionals (such as Test Analysts, Testers, or QA Representatives) who wish to save time and effort by adopting predefined testing artifacts (such as standard templates for Test Script, Test Plan, and Test Specification documents)
• IT Professionals who need to understand the software testing process (such as developers involved in Unit or Integration testing)
• Any staff members who are keen to improve their career prospects by advocating a complete testing solution to their organizations' software testing needs, particularly where there is a need to improve quality or save time, effort, and cost
• Training Managers/Trainers who are in the process of writing or amending testing training materials and who need to obtain a pragmatic view of the testing process and its application
• Students who need to obtain a pragmatic/real-world view of the application of testing theory and principles to organizational software testing requirements, or who have an interest in testing-process improvement and the role and use of metrics

1.3 How to Read This Book

This book is divided into three parts, all closely linked, but each of which can be read and applied separately.

Part 1 (Chapters 2-13) documents the "traditional view" of the components comprising a software testing process. Part 1 provides detailed information that can be used as the basis for setting up a testing-process framework tailored to the individual requirements of any organization involved in software testing.

Part 2 (Chapters 14-18) provides a series of case studies that show how a number of organizations have implemented their own testing process based on the "classic view" described in Part 1. These case studies can be read to provide real world guidance on how an individual organization can implement a testing-process framework to meet its own testing requirements.

Part 3 (the appendices) contains a set of standard testing document templates, pro- formas, and checklists~plus a number of appendices that expand on topics described in passing in the main body of the book. The standard testing document templates, proformas, and checklists are also available from the following URL: so that they can be used immediately without modi- fication or customized to reflect the particular requirements of any organization (such as a corporate style, branding, or documentation standard).

Terms in italics are fully defined in the glossary.

1.4 Structure and Content of This Book

Specifically, the chapters and appendices comprising this book are:

• Chapter 2, which discusses just how challenging it is to thoroughly test even the most simple software system, reviews a number of definitions of testing, provides a brief overview of the approach to software testing, and lists definitive testing references for further reading
• Chapter 3, which describes the principal techniques used in designing effective and efficient tests for testing software systems and provides, where appropriate, references to illustrative worked examples in the appendices
• Chapter 4, which deals with the issues associated with the management and planning of the testing process, provides guidance on the organization of testing and testing projects and on the need for thorough planning, describing a number of techniques for supporting the planning process
• Chapters 5-11, which provide details on each of the testing phases (from Unit Testing to Acceptance Testing and on to Regression Testing') and their interrelationships. Each chapter is presented in a standard format and covers:
  - The overall testing approach for that phase
  - Test data requirements for that phase
  - The roles and responsibilities associated with that phase
  - Any particular planning and resourcing issues for that phase
  - The inputs to and the outputs from that phase
  - A review of the specific testing techniques that are appropriate to that phase.

• Chapter 12, considers the need for process improvement within the testing process and reviews the role of metrics (proposing a pragmatic metrics set that can be used effectively within and across testing projects). It also provides ref- erences to further sources of information on test process improvement
• Chapter 13, which for organizations adopting the testing process described in this book or using it as the basis for setting up their own testing-process framework, discusses the approach to introducing testing into an organization and managing its successful adoption, reviews the need to maintain that testing process, and proposes an approach to satisfy this requirement
• Chapters 14-18, provide a series of real-world case studies describing how a number of commercial organizations have implemented their own customized view of the testing process described in Chapters 2-13. Specifically, the organizations covered in the case studies are:
  - The British Library
  - Reuters Product Acceptance Group
  - Crown Quality Assurance Group
  - The Wine Society
  - Automatic Data Processing (ADP] Limited

• Appendices A -j provide a set of testing document templates, proformas, and checklists:
  - Terms of reference for testing staff
  - Summary testing guides for each testing phase
  - A Test Plan document template
  - A Test Specification document template
  - A Test Script template
  - A Test Result Record Form template
  - A Test Log template
  - A Test Certificate template
  - A Re-use Pack checklist
  - A Test Summary Report template

• Appendices K-M present a series of worked examples of testing techniques described in Chapter 3.
• Appendices N-Q expand on topics described in passing in the main body of the book and include:
  - A scheme and set of criteria for evaluating the relative merits of commercially available automated software testing tools
  - An overview of the process of Usability Testing and its application
  - A scheme and set of criteria for performing an audit of a testing process
  - A discussion of the issues involved in the testing of object-oriented and component-based applications

• A list of the references cited in the book
• A glossary of terms used in this book

Editorial Reviews

'With the publication of the second edition of Testing IT, John Watkins and Simon Mills built on the success of the first edition, revising and bringing it up to date to ensure it continues to be relevant for the next ten years and beyond … [This] book goes a long way to helping practitioners involved in test process, test management and testing …' Geoff Thompson, from the Foreword of Software Testing, Verification and Reliability

Praise for the first edition: 'When asked for my opinion about software testing I referred naturally to Testing IT. Imagine a light being switched on! Almost overnight everyone in the organisation was walking around with versions of the V-lifecycle … has made a difference to two organisations that were lost in a testing strategy wilderness.' Martin Allen, Software Team Leader

Praise for the first edition: 'An off-the-shelf set of procedures, complete with forms and checklists, for much less than the price of a day's work, must be a bargain … The forms are a joy to anyone in quality assurance …' Brian Peaker, Computer Bulletin

Praise for the first edition: 'The third part consists of an extensive set of test checklists, templates and proformas, ready to use or to tailor to your specific requirements. To have these all in one place and already standardised is alone worth the price of the book … covers all aspects of testing for both in-house and third-party developed software. I heartily recommend that any test professional looking for an easier life gets a copy of this book. I believe that even the most hardened of us would gain from reading it.' Professional tester

Praise for the first edition: 'This is the most useful book on testing since Myers' The Art of Software Testing in the late 1970s. Besides having all of the material of Myers, Watkins has enlarged the scope of testing to include object-oriented code.' James Edward Tomayko, Computing Reviews

From the Publisher