"Regrettably, many organizations find it difficult enough to develop a plan, let alone to find the resources and time to conduct meaningful testing. Recovery testing has, for these organizations, held many similarities to the classic dilemma of application software documentation: always budgeted at the end of the project, yet seldom enough time or money left to do a decent job. Why is it that some organizations never get around to recovery testing, and others test aggressively?
"Where recovery testing is planned, budgeted and conducted, the predicament is, as often as not, a dearth of practical experience or knowledge. The typical disaster recovery book has a chapter near the back entitled "Testing and Maintenance," or something like that. In a handful of pages, the author communicates the desirability of testing, the basic types of tests, and little more. This is not meant to be a criticism of these valuable books, so much as an observation that they are usually focused on plan development and implementation. This book does not attempt to convey the process of recovery plan development - it assumes that the reader at least understands the mechanics of disaster recovery / business continuity / contingency planning.
"Among the clients of Rothstein Associates Inc.'s management consultancy, recovery testing has been a hot topic for some time. As publisher of The Rothstein Catalog On Disaster Recovery, recovery testing has been the single most requested subject, yet our research has shown that little has been published on the subject.
"From this book, the contingency planner can understand more than just how to test: why to test, when to test (and not test) and the necessary participants and resources. Further, this book addresses some often-ignored, real-world considerations: the justification, politics and budgeting affecting recovery testing. By having multiple authors share their respective areas of expertise, it is hoped that this book will provide the reader with a comprehensive resource addressing the significant aspects of recovery testing."
Table of ContentsPreface - Alan Freedman
Introduction - Philip Jan Rothstein
Managing Recovery Testing:
1 Justifying Recovery Testing - Dan Muecke
2 Managing The Testing Process - Marvin Wainschel
3 Budgeting for Recovery Testing - Robbie Atabaigi
4 The First Test - John Nevola
5 The Test Plan - David Sobolow
6 Hot Site Recovery Testing Checklist - Judith Hinds
7 Choosing and Developing the Test Scenario - Melvyn Musson
8 The Politics of Recovery Testing - Philip Jan Rothstein
9 Closing The Loop - Patrick LaValla, Robert Stoffel & Charles Erwin
Test Participants and Resources
10 The Test Team - Michael G. Courton
11 Management's Role - Dan Muecke
12 The Psychology of Recovery Testing - David Doepel
13 The Consultant s Role In Disaster Recovery Testing - Marvin Wainschel
14 The Recovery Vendor s Perspective - John Sensenich
15 Client Participation In Testing - Michael G. Courton
16 Recovery Planning Software s Role In Testing - Kenneth J. Bauman
17 The Recovery Plan Document In Testing - William J. Krouslis
18 The Restoration Vendor's Perspective - Ronald N. Chamberlain
19 The Risk Management Perspective - Melvyn Musson
Testing Methods and Processes
20 Testing Methods - Geoffrey H. Wold & Robert F. Shriver
21 Testing Levels and Resources - William J. Krouslis
22 The Test Report - David Sobolow
23 Surprise and Unannounced Testing - James Certoma
24 Disaster Recovery Testing Cycles - Judith Hinds
What is Being Tested
25 Data Center Recovery Testing - Joan Blum
26 Data Communications Recovery Testing - Paul F. Kirvan
27 Voice Communications Recovery Testing - Paul F. Kirvan
28 Local Area Network Recovery Testing - Paul F. Kirvan & Philip Jan Rothstein
29 Trading Floor Recovery Testing - James N. Loizides
30 Testing Public and Internal Communications - James E. Lukaszewski
31 The Emergency Management Exercise Process - Patrick LaValla, Robert Stoffel & Charles Erwin
32 Disaster Plan Simulates Plane Crash Into High-Rise Building - William H. Johnson & Warren R. Matthews
33 Rehearsing Your Crisis Management Plan - Alvin Arnell
34 Testing Emergency Plans and Capabilities: Two Case Studies - Roger W. Mickelson
35 Vital Records Testing - Federal Emergency Management Agency
36 The Bottom Line - Philip Jan Rothstein
On February 26, 1993, a bomb exploded in the World Trade Center in New York City. The result of this disaster included six people dead; over one thousand injured; 50,000 people displaced from their work locale; and 950 businesses put on the street. In addition to spending eight weeks around-the-clock and significant dollars to make the World Trade Center habitable, an additional $70 million was spent to erect an interim air conditioning plant to provide temporary cooling to the World Trade Center during the summer. Business loss was estimated at $600 million and many believe that figure to be conservative.
Perhaps the World Trade Center disaster had an even greater impact than described above. It took away whatever innocence we had that a terrorist attack would ever befall us at home. It also reinforced in a most tragic way the need for preparedness in the event of catastrophic circumstances.
Most disasters are not the work of terrorists but of mother nature or human error. The impacts of disasters take many forms with human and business devastation all too often the result. In business, we refer to these phenomena and the ability to recover from them as disaster recovery or contingency planning or business continuity. Regardless of label, this translates to the ability to conduct business at an alternate location when the primary business facility is no longer functional.
Business continuity is a necessity. Testing is crucial to business continuity. This book will help you work through a variety of testing scenarios and methods to better understand the requirement for a workable contingency plan. In examining these alternative test scenarios, you will in turn be able to better plan for your own business.
Viable business continuity plans need not be a three-inch book. They should only contain what is absolutely necessary for business resumption. First and foremost they must always address life-safety of the individual. Interviews with those who lived through Hurricane Andrew and other catastrophes have articulated two common themes: the well-being of the employee and immediate family is paramount; and, the recovery plan book is not used when it represents a symbol of an audit or regulatory requirement. It is used when its contents define what is required to recover.
"No two businesses are identical. One business' needs may be very different from those of another. This book will walk you through various test scenarios and approaches. It is designed to be a guide, to help you gain optimal results in business preparedness, to identify and quantify base needs, to facilitate response to the difficult questions of who, where, what, when, and how.
A business continuity plan that is not tested is of little to no value. Testing and learning from the experience is the only way to perfect the plan; to ensure the business needs are addressed. Ideally, a perfect test addresses all business requirements and thoroughly involves the real users. Anything less than optimum presents levels of business risk.
This book has been written to allow the reader to focus on a particular or multiple concerns (at the discretion of the reader); to examine and study one or a series of test scenarios. As this preface is being written the Midwest United States is trying to recover from the worst flood in its experience. The human devastation is incalculable while business losses have been estimated as high as eight billion dollars. The ability to recover in an appropriate time after a catastrophe is imperative. One difficulty has been creating a document and test mechanism that will do just that. You will discern what it takes to be prepared, to experience a catastrophe and survive without sacrifice of life-safety or going out of business.
Alan Freedman, (formerly Vice President, Technology Strategic Planning, Bankers Trust Company)