Table of Contents

Chapter 1 Introduction: Parallelism = Opportunities + Challenges
The Power and Potential of Parallelism
Examining Sequential and Parallel Programs
A Paradigm Shift
Parallelism Using Multiple Instruction Streams
The Goals: Scalable Performance and Portability
Summary
Historical Context
Exercises
Chapter 2 Parallel Computers And Their Model
Balancing Machine Specifics with Portability
A Look at Five Parallel Computers
The RAM: An Abstraction of a Sequential Computer
The PRAM: A Parallel Computer Model
The CTA: A Practical Parallel Computer Model
Memory Reference Mechanisms
A Closer Look at Communication
Applying the CTA Model
Summary
Historical Perspective
Exercises
Chapter 3 Reasoning about Performance
Introduction
Motivation and Some Basic Concepts
Sources of Performance Loss
Parallel Structure
Reasoning about Performance
Performance Trade-Offs
Measuring Performance
What should we measure?
Summary
Historical Perspective
Exercises
Chapter 4 First Steps Towards Parallel Programming
Task and Data Parallelism
Peril-L
Count 3s Example
Conceptualizing Parallelism
Alphabetizing Example
Comparison of Three Solutions
Summary
Historical Perspective
Exercises
Chapter 5 Scalable Algorithmic Techniques
The Inevitability of Trees
Blocks of Independent Computation
Schwartz’ Algorithm
Assigning Work To Processes Statically
Assigning Work to Processes Dynamically
The Reduce & Scan Abstractions
Trees
Summary
Historical Context
Exercises
Chapter 6 Programming with Threads
POSIX Threads
Thread Creation and Destruction
Mutual Exclusion
Synchronization
Safety Issues
Performance Issues
Open MP
The Count 3s Example
Semantic Limitations on
Reduction
Thread Behavior and Interaction
Sections
Summary of OpenMP
Java Threads
Summary
Historical Perspectives
Exercises
Chapter 7 Local View Programming Languages
MPI: The Message Passing Interface
Getting Started
Safety Issues
Performance Issues
Co-Array Fortran
Unified Parallel C
Titanium
Summary
Exercises
Chapter 8 Global View Programming Languages
The Z-level Programming Language
Basic Concepts of ZPL
Life, An Example
Design Principles
Manipulating Arrays Of Different Ranks
Reordering Data With Remap
Parallel Execution of ZPL
Performance Model
Summary
NESL
Historical Context
Exercises
Chapter 9 Assessing Our Knowledge
Introduction
Evaluating Existing Approaches
Lessons for the Future
Summary
Historical Perspectives
Exercises
Chapter 10 Future Directions in Parallel Programming
Attached Processors
Grid Computing
Transactional Memory
Summary
Exercises
Chapter 11 Capstone Project: Designing a Parallel Program
Introduction
Motivation
Getting Started
Summary
Historical Perspective
Exercises
Appendix 1 More Advanced Concepts