| Acknowledgments | xiii |
| Foreword | xvii |
| Preface | xxi |
Part I1 |
Chapter 1 | Why Care About Data Quality? | 3 |
1.1 | Introduction | 3 |
1.2 | Poor Data Quality Is Pervasive | 4 |
1.3 | Poor Data Quality Impacts Business Success | 6 |
1.3.1 | Poor Data Quality Lowers Customer Satisfaction | 6 |
1.3.2 | Poor Data Quality Leads to High and Unnecessary Costs | 7 |
1.3.3 | Poor Data Quality Lowers Job Satisfaction and Breeds Organizational Mistrust | 9 |
1.3.4 | Poor Data Quality Impacts Decision Making | 9 |
1.3.5 | Poor Data Quality Impedes Re-engineering | 10 |
1.3.6 | Poor Data Quality Hinders Long-Term Business Strategy | 11 |
1.3.7 | Data Fill the White Space on the Organization Chart | 11 |
1.3.8 | The Enabling Role of Information Technology | 12 |
1.4 | Data Quality Can Be a Unique Source of Competitive Advantage | 12 |
1.5 | Summary | 13 |
| References | 14 |
Chapter 2 | Strategies for Improving Data Accuracy | 17 |
2.1 | Introduction | 17 |
2.2 | Background | 19 |
2.2.1 | Quality, Data, and Data Quality | 19 |
2.2.2 | Choice 1: Error Detection and Correction | 22 |
2.2.3 | Process Control and Improvement | 25 |
2.2.4 | Process Design | 27 |
2.3 | Which Data to Improve? | 27 |
2.4 | Improving Data Accuracy for One Database | 29 |
2.5 | Improving Data Accuracy for Two Databases | 30 |
2.6 | Improving Data Accuracy in the Data Warehouse | 32 |
2.7 | Summary | 33 |
| References | 34 |
Chapter 3 | Data Quality Policy | 37 |
3.1 | Introduction | 37 |
3.2 | What Should a Data Policy Cover? | 38 |
3.2.1 | The Data Asset in a Typical Enterprise | 38 |
3.2.2 | What a Data Policy Can Cover | 40 |
3.3 | Needed Background on Data | 41 |
3.3.1 | Differences Between Data and Other Assets | 41 |
3.3.2 | Who Uses the Data | 44 |
3.4 | A Model Data Policy | 46 |
3.4.1 | Model Data Policy | 47 |
3.5 | Deploying the Policy | 49 |
3.6 | Summary | 52 |
| References | 53 |
Chapter 4 | Starting and Nurturing a Data Quality Program | 55 |
4.1 | Introduction | 55 |
4.2 | A Model for Successful Change | 58 |
4.2.1 | Pressure for Change | 58 |
4.2.2 | Clear, Shared Vision | 59 |
4.2.3 | Capacity for Change | 60 |
4.2.4 | Actionable First Steps | 61 |
4.3 | Getting Started | 61 |
4.4 | Growth Stages | 63 |
4.5 | Becoming Part of the Mainstream | 64 |
4.6 | The Role of Senior Management | 66 |
4.7 | Summary | 67 |
| References | 67 |
Chapter 5 | Data Quality and Re-engineering at AT&T | 69 |
5.1 | Introduction | 69 |
5.2 | Background | 70 |
5.3 | First Steps | 73 |
5.3.1 | Improve Bill Verification | 73 |
5.3.2 | Prototype with Cincinnati Bell | 77 |
5.4 | Re-engineering | 77 |
5.4.1 | Business Direction | 78 |
5.4.2 | Program Administration | 79 |
5.4.3 | Management Responsibilities | 80 |
5.4.4 | Operational Plan for Improvement | 81 |
5.5 | Summary | 83 |
| References | 84 |
Chapter 6 | Data Quality Across the Corporation: Telstra's Experiences | 85 |
6.1 | Introduction | 85 |
6.2 | Program Definition | 87 |
6.3 | First Steps | 89 |
6.4 | Full Program | 90 |
6.5 | Results | 94 |
6.6 | Summary | 95 |
| References | 96 |
Part II97 |
Chapter 7 | Managing Information Chains | 99 |
7.1 | Introduction | 99 |
7.2 | Future Performance of Processes | 104 |
7.2.1 | Step 1: Establish a Process Owner and Management Team | 105 |
7.2.2 | Step 2: Describe the Process and Understand Customer Needs | 107 |
7.2.3 | Step 3: Establish a Measurement System | 110 |
7.2.4 | Step 4: Establish Statistical Control and Check Conformance to Requirements | 111 |
7.2.5 | Step 5: Identify Improvement Opportunities | 112 |
7.2.6 | Step 6: Select Opportunities | 113 |
7.2.7 | Step 7: Make and Sustain Improvements | 114 |
7.3 | Summary | 117 |
| References | 118 |
Chapter 8 | Process Representation and the Functions of Information Processing Approach | 119 |
8.1 | Introduction | 119 |
8.2 | Basic Ideas | 120 |
8.3 | The Information Model/The FIP Chart | 122 |
8.3.1 | The FIP Row | 122 |
8.3.2 | The Process Instruction Row | 123 |
8.3.3 | The IIPs/OIPs Rows | 124 |
8.3.4 | The Physical Devices Row | 125 |
8.3.5 | The Person/Organization Row | 125 |
8.3.6 | An Example--an Employee Move | 125 |
8.4 | Enhancements to the Basic Information Model | 129 |
8.4.1 | Pictorial Representation | 130 |
8.4.2 | Exception, Alternative, and Parallel Processes | 131 |
8.5 | Measurement and Improvement Opportunities | 134 |
8.5.1 | Accuracy | 134 |
8.5.2 | Timeliness | 134 |
8.5.3 | Cues for Improvement | 134 |
8.6 | Summary | 136 |
| References | 137 |
Chapter 9 | Data Quality Requirements | 139 |
9.1 | Introduction | 139 |
9.2 | Quality Function Deployment | 140 |
9.3 | Data Quality Requirements for an Existing Information Chain | 141 |
9.3.1 | Step 1: Understand Customers' Requirements | 142 |
9.3.2 | Step 2: Develop a Set of Consistent Customer Requirements | 142 |
9.3.3 | Step 3: Translate Customer Requirements into Technical Language | 145 |
9.3.4 | Step 4: Map Data Quality Requirements into Individual Performance Requirements | 146 |
9.3.5 | Step 5: Establish Performance Specifications for Processes | 148 |
9.3.6 | Summary Remarks | 148 |
9.4 | Data Quality Requirements at the Design Stage | 149 |
9.4.1 | Background and Motivation | 149 |
9.4.2 | The Complete Job--the Entire Data Life Cycle | 150 |
9.4.3 | The Methodology Applied at the Design Stage | 151 |
9.5 | Summary | 152 |
| References | 154 |
Chapter 10 | Statistical Quality Control | 155 |
10.1 | Introduction | 155 |
10.2 | Variation | 158 |
10.2.1 | Sources of Variation | 159 |
10.3 | Stable Processes | 162 |
10.3.1 | Judgment of Stability | 164 |
10.4 | Control Limits: Statistical Theory and Methods of SQC | 165 |
10.4.1 | The Underlying Theory | 165 |
10.4.2 | Formulae | 167 |
10.5 | Interpreting Control Charts | 174 |
10.6 | Conformance to Requirements | 181 |
10.7 | Summary | 181 |
10.8 | Notes on References | 182 |
| References | 182 |
Chapter 11 | Measurement Systems, Data Tracking, and Process Improvement | 185 |
11.1 | Introduction | 185 |
11.2 | Measurement Systems | 186 |
11.3 | Process Requirements | 189 |
11.4 | What to Measure | 190 |
11.5 | The Measuring Device and Protocol: Data Tracking | 191 |
11.5.1 | Philosophy | 191 |
11.5.2 | Step 1: Sampling | 193 |
11.5.3 | Step 2: Tracking | 194 |
11.5.4 | Step 3: Identify Errors and Calculate Process Cycle Times | 194 |
11.5.5 | Step 4: Summarize Results | 196 |
11.6 | Implementation | 209 |
11.7 | Summary | 211 |
| References | 212 |
Part III213 |
Chapter 12 | Just What Is (or Are) Data? | 215 |
12.1 | Introduction | 215 |
12.2 | The Data Life Cycle | 217 |
12.2.1 | Preliminaries | 218 |
12.2.2 | Acquisition Cycle | 219 |
12.2.3 | Usage Cycle | 222 |
12.2.4 | Checkpoints, Feedback Loops, and Data Destruction | 224 |
12.2.5 | Discussion | 225 |
12.3 | Data Defined | 227 |
12.3.1 | Preliminaries | 227 |
12.3.2 | Competing Definitions | 227 |
12.3.3 | A Set of Facts | 228 |
12.3.4 | The Result of Measurement | 228 |
12.3.5 | Raw Material for Information | 228 |
12.3.6 | Surrogates for Real-World Objects | 229 |
12.3.7 | Representable Triples | 229 |
12.3.8 | Discussion | 230 |
12.4 | Management Properties of Data | 232 |
12.4.1 | How Data Differ From Other Resources | 233 |
12.4.2 | Implications for Data Quality | 235 |
12.5 | A Model of an Enterprise's Data Resource | 236 |
12.6 | Information | 237 |
12.7 | Summary | 239 |
| References | 240 |
Chapter 13 | Dimensions of Data Quality | 245 |
13.1 | Introduction | 245 |
13.2 | Quality Dimensions of a Conceptual View | 246 |
13.2.1 | Content | 248 |
13.2.2 | Scope | 249 |
13.2.3 | Level of Detail | 249 |
13.2.4 | Composition | 250 |
13.2.5 | View Consistency | 252 |
13.2.6 | Reaction to Change | 252 |
13.3 | Quality Dimensions of Data Values | 254 |
13.3.1 | Accuracy | 255 |
13.3.2 | Completeness | 256 |
13.3.3 | Currency and Related Dimensions | 258 |
13.3.4 | Value Consistency | 259 |
13.4 | Quality Dimensions of Data Representation | 260 |
13.4.1 | Appropriateness | 261 |
13.4.2 | Interpretability | 261 |
13.4.3 | Portability | 262 |
13.4.4 | Format Precision | 262 |
13.4.5 | Format Flexibility | 262 |
13.4.6 | Ability to Represent Null Values | 262 |
13.4.7 | Efficient Usage of Recording Media | 263 |
13.4.8 | Representation Consistency | 263 |
13.5 | More on Data Consistency | 263 |
13.6 | Summary | 266 |
| References | 267 |
Part IV271 |
Chapter 14 | Summary: Roles and Responsibilities | 273 |
14.1 | Introduction | 273 |
14.2 | Roles for Leaders | 274 |
14.3 | Roles for Process Owners | 277 |
14.4 | Roles for Information Professionals | 281 |
14.4.1 | Design Principle: Process Management | 283 |
14.4.2 | Design Principle: Measurement Systems | 284 |
14.4.3 | Design Principle: Data Architecture | 284 |
14.4.4 | Design Principle: Cycle Time | 285 |
14.4.5 | Design Principle: Data Values | 285 |
14.4.6 | Design Principle: Redundancy in Data Storage | 285 |
14.4.7 | Design Principle: Computerization | 286 |
14.4.8 | Design Principle: Data Transformations and Transcription | 286 |
14.4.9 | Design Principle: Value Creation | 286 |
14.4.10 | Design Principle: Data Destruction | 287 |
14.4.11 | Design Principle: Editing | 287 |
14.4.12 | Design Principle: Coding | 287 |
14.4.13 | Design Principle: Single-Fact Data | 288 |
14.4.14 | Design Principle: Data Dictionaries | 288 |
14.5 | Final Remarks--The Three Most Important Points | 288 |
| Glossary | 289 |
| About the Author | 295 |
| Index | 297 |