The OpenMP Common Core: Making OpenMP Simple Again

The OpenMP Common Core: Making OpenMP Simple Again


Members save with free shipping everyday! 
See details


How to become a parallel programmer by learning the twenty-one essential components of OpenMP.

This book guides readers through the most essential elements of OpenMP—the twenty-one components that most OpenMP programmers use most of the time, known collectively as the “OpenMP Common Core.” Once they have mastered these components, readers with no prior experience writing parallel code will be effective parallel programmers, ready to take on more complex aspects of OpenMP. The authors, drawing on twenty years of experience in teaching OpenMP, introduce material in discrete chunks ordered to support effective learning. OpenMP was created in 1997 to make it as simple as possible for applications programmers to write parallel code; since then, it has grown into a huge and complex system. The OpenMP Common Core goes back to basics, capturing the inherent simplicity of OpenMP.
After introducing the fundamental concepts of parallel computing and history of OpenMP's development, the book covers topics including the core design pattern of parallel computing, the parallel and worksharing-loop constructs, the OpenMP data environment, and tasks. Two chapters on the OpenMP memory model are uniquely valuable for their pedagogic approach. The key for readers is to work through the material, use an OpenMP-enabled compiler, and write programs to experiment with each OpenMP directive or API routine as it is introduced. The book's website, updated continuously, offers a wide assortment of programs and exercises.

Product Details

ISBN-13: 9780262538862
Publisher: MIT Press
Publication date: 11/19/2019
Series: Scientific and Engineering ComputationSeries Series
Pages: 320
Sales rank: 882,657
Product dimensions: 8.06(w) x 9.06(h) x 0.79(d)
Age Range: 18 Years

About the Author

Timothy G. Mattson is Senior Principal Engineer at Intel Corporation.

Yun (Helen) He is a High Performance Computing Consultant at the National Energy Research Supercomputing Center of Lawrence Berkeley National Laboratory.

Alice E. Koniges is a computer scientist and research principal investigator at the University of Hawaii.

Table of Contents

Series Foreword xiii

Foreword xv

Preface xvii

I Setting the Stage

1 Parallel Computing 5

1.1 Fundamental Concepts of Parallel Computing 5

1.2 The Rise of Concurrency 7

1.3 Parallel Hardware 9

1.3.1 Multiprocessor Systems 9

1.3.2 Graphics Processing Units (GPU) 13

1.3.3 Distributed Memory Clusters 15

1.4 Parallel Software for Multiprocessor Computers 16

2 The Language of Performance 21

2.1 The Basics: FLOPS, Speedup, and Parallel Efficiency 21

2.2 Amdahl's Law 23

2.3 Parallel Overhead 25

2.4 Strong Scaling vs. Weak Scaling 27

2.5 Load Balancing 29

2.6 Understanding Hardware with the Roofline Model 31

3 What is OpenMP? 35

3.1 OpenMP History 35

3.2 The Common Core 38

3.3 Major Components of OpenMP 38

II The OpenMP Common Core

4 Threads and the OpenMP Programming Model 47

4.1 Overview of OpenMP 47

4.2 The Structure of OpenMP Programs 47

4.3 Threads and the Fork Join Pattern 50

4.4 Working with Threads 56

4.4.1 The SPMD Design Pattern 58

4.4.2 False Sharing 63

4.4.3 Synchronization 67 Critical 67 Barrier 69

4.5 Closing Comments 72

5 Parallel Loops 75

5.1 Worksharing-Loop Construct 76

5.2 Combined Parallel Worksharing-Loop Construct 79

5.3 Reductions 79

5.4 Loop Schedules 83

5.4.1 The Static Schedule 83

5.4.2 The Dynamic Schedule 84

5.4.3 Choosing a Schedule 86

5.5 Implicit Barriers and the Nowait Clause 90

5.6 Pi Program with Parallel Loop Worksharing 92

5.7 A Loop-Level Parallelism Strategy 94

5.8 Closing Comments 96

6 OpenMP Data Environment 99

6.1 Default Storage Attributes 100

6.2 Modifying Storage Attributes 102

6.2.1 The Shared Clause 103

6.2.2 The Private Clause 105

6.2.3 The Firstprivate Clause 107

6.2.4 The Default Clause 108

6.3 Data Environment Examples 109

6.3.1 A Data Scope Test 109

6.3.2 Mandelbrot Set Area 111

6.3.3 Pi Loop Example Revisited 115

6.4 Arrays and Pointers 116

6.5 Closing Comments 119

7 Tasks in OpenMP 121

7.1 The Need for Tasks 121

7.2 Explicit Tasks 125

7.3 Our First Example: Schrodinger's Program 125

7.4 The Single Construct 128

7.5 Working with Tasks 130

7.5.1 When Do Tasks Complete? 131

7.6 Task Data Environment 132

7.6.1 Default Data Scoping for Tasks 132

7.6.2 Linked List Program Revisited with Tasks 134

7.7 Fundamental Design Patterns with Tasks 135

7.7.1 Divide and Conquer Pattern 137

7.8 Closing Comments 143

8 OpenMP Memory Model 145

8.1 Memory Hierarchies Revisited 146

8.2 The OpenMP Common Core Memory Model 149

8.3 Working with Shared Memory 152

8.4 Closing Comments 156

9 Common Core Recap 159

9.1 Managing Threads 160

9.2 Worksharing Constructs 161

9.3 Parallel Worksharing-Loop Combined Construct 162

9.4 OpenMP Tasks 163

9.5 Synchronization and Memory Consistency Models 164

9.6 Data Environment Clauses 166

9.7 The Reduction Clause 167

9.8 Environment Variables and Runtime Library Routines 168

III Beyond the Common Core

10 Multithreading beyond the Common Core 175

10.1 Additional Clauses for OpenMP Common Core Constructs 175

10.1.1 The Parallel Construct 176

10.1.2 The Worksharing-Loop Construct 178

10.1.3 The Task Construct 186

10.2 Multithreading Functionality Missing from the Common Core 190

10.2.1 Threadprivate 191

10.2.2 Master 195

10.2.3 Atomic 196

10.2.4 OMP_STACKSIZE 197

10.2.5 Runtime Library Routines 199 omp_get_max_threads 199 omp_set-dynamic 199 omp_in_parallel 200

10.3 Closing Comments 201

11 Synchronization and the OpenMP Memory Model 203

11.1 Memory Consistency Models 204

11.2 Pairwise Synchronization 210

11.3 Locks and How to Use Them 217

11.4 The C++ Memory Model and OpenMP 220

11.5 Closing Comments 224

12 Beyond OpenMP Common Core Hardware 225

12.1 Nonuniform Memory Access (NUMA) Systems 226

12.1.1 Working with NUMA Systems 228 Controlling Thread Affinity 232 Managing Data Locality 234

12.1.2 Nested Parallel Constructs 240

12.1.3 Checking the Thread Affinity 243

12.1.4 Summary: Thread Affinity and Data Locality 245

12.2 SIMD 247

12.3 Device Constructs 256

12.4 Closing Comments 262

13 Your Continuing Education in OpenMP 265

13.1 Programmer Resources from the ARB 265

13.2 How to Read the OpenMP Specification 267

13.2.1 OpenMP with All the Formal Jargon 268

13.3 The Structure of the OpenMP Specification 272

13.4 Closing Comments 275

Glossary 277

References 289

Subject Index 291

What People are Saying About This

From the Publisher

“This book fills a major gap in the literature on parallel application development. It offers a practical introduction to the most widely used features in OpenMP and explains how to use them effectively to solve a range of programming problems. I highly recommend it for beginners and intermediate users.”

Barbara Chapman, Professor of Applied Mathematics and Statistics and Computer Science, Stony Brook University

“The OpenMP API is used to program a wide range of parallel systems from basic multicore processors to large shared-memory and heterogeneous systems with coprocessors. To master OpenMP, you need a solid foundation based on the core features of the language. Based on the authors' experience teaching OpenMP for over twenty years, this book is a great tool to help readers build that foundation.”

Michael Klemm, CEO, OpenMP Architecture Review Board

“This book should be on the bookshelf of every OpenMP programmer. This is not only a mandatory read for beginners; even more seasoned OpenMP developers will find certain parts very useful. In particular, I found the coverage of the memory model—one of the hardest parts to master—excellent.”

Ruud van der Pas, Distinguished Engineer, Oracle; coauthor of Using OpenMP:Portable Shared Memory Parallel Programming and Using OpenMP—The Next Step

Customer Reviews