The Art of Concurrency: A Thread Monkey's Guide to Writing Parallel Applications

by Clay Breshears

Overview

If you're looking to take full advantage of multi-core processors with concurrent programming, this practical book provides the knowledge and hands-on experience you need. The Art of Concurrency is one of the few resources to focus on implementing algorithms in the shared-memory model of multi-core processors, rather than just theoretical models or distributed-memory architectures. The book provides detailed explanations and usable samples to help you transform algorithms from serial to parallel code, along with advice and analysis for avoiding mistakes that programmers typically make when first attempting these computations.

Written by an Intel engineer with over two decades of parallel and concurrent programming experience, this book will help you:

• Understand parallelism and concurrency
• Explore differences between programming for shared-memory and distributed-memory
• Learn guidelines for designing multithreaded applications, including testing and tuning
• Discover how to make best use of different threading libraries, including Windows threads, POSIX threads, OpenMP, and Intel Threading Building Blocks
• Explore how to implement concurrent algorithms that involve sorting, searching, graphs, and other practical computations

The Art of Concurrency shows you how to keep algorithms scalable to take advantage of new processors with even more cores. For developing parallel code algorithms for concurrent programming, this book is a must.

Product Details

ISBN-13:	9780596555788
Publisher:	O'Reilly Media, Incorporated
Publication date:	05/07/2009
Sold by:	Barnes & Noble
Format:	eBook
Pages:	304
File size:	4 MB

About the Author

Clay Breshears has been with Intel since September 2000. He started as a Senior Parallel Application Engineer at the Intel Parallel Applications Center in Champaign, IL, implementing multithreaded and distributed solutions in customer applications. Clay is currently a Course Architect for the Intel Software College, specializing in multi-core and multithreaded programming and training. Before joining Intel, Clay was a Research Scientist at Rice University helping Department of Defense researchers make best use ofthe latest High Performance Computing (HPC) platforms and resources.

Clay received his Ph.D. in Computer Science from the University of Tennessee, Knoxville, in 1996, but has been involved with parallel computation and programming for over twenty years; six of those years were spent in academia at Eastern Washington University and The University of Southern Mississippi.

Table of Contents

Preface vii

1 Want to go Faster? Raise Your Hands if You Want to go Faster! 1

Some Questions You May Have 2

Four Steps of a Threading Methodology 7

Background of Parallel Algorithms 12

Shared-Memory Programming Versus Distributed-Memory Programming 15

This Book's Approach to Concurrent Programming 19

2 Concurrent or not Concurrent? 21

Design Models for Concurrent Algorithms 22

What's Not Parallel 42

3 Proving Correctness and Measuring Performance 49

Verification of Parallel Algorithms 50

Example: The Critical Section Problem 53

Performance Metrics (How Am I Doing?) 66

Review of the Evolution for Supporting Parallelism in Hardware 71

4 Eight Simple Rules for Designing Multithreaded Applications 73

Rule 1: Identify Truly Independent Computations 74

Rule 2: Implement Concurrency at the Highest Level Possible 74

Rule 3: Plan Early for Scalability to Take Advantage of Increasing Numbers of Cores 75

Rule 4: Make Use of Thread-Safe Libraries Wherever Possible 76

Rule 5: Use the Right Threading Model 77

Rule 6: Never Assume a Particular Order of Execution 77

Rule 7: Use Thread-Local Storage Whenever Possible or Associate Locks to Specific Data 78

Rule 8: Dare to Change the Algorithm for a Better Chance of Concurrency 79

Summary 80

5 Threading Libraries 81

Implicit Threading 82

Explicit Threading 88

What Else Is Out There? 92

Domain-Specific Libraries 92

6 Parallel Sum and Prefix Scan 95

Parallel Sum 96

Prefix Scan 103

Selection 112

A Final Thought 123

7 Mapreduce 125

Map As a Concurrent Operation 127

Reduce As a Concurrent Operation 129

Applying MapReduce 138

MapReduce As GenericConcurrency 143

8 Sorting 145

Bubblesort 146

Odd-Even Transposition Sort 153

Shellsort 162

Quicksort 169

Radix Sort 182

9 Searching 201

Unsorted Sequence 202

Binary Search 210

10 Graph Algorithms 221

Depth-First Search 224

All-Pairs Shortest Path 240

Minimum Spanning Tree 245

11 Threading Tools 257

Debuggers 258

Performance Tools 260

Anything Else Out There? 262

Go Forth and Conquer 263

Glossary 265

Photo Credits 275

Index 277