Parallel and Distributed Programming Using C++

Overview

Parallel and Distributed Programming Using C++ provides an up-close look at how to build software that can take advantage of multiprocessor computers. Simple approaches for programming parallel virtual machines are presented, and the basics of cluster application development are explained. Through an easy-to-understand overview of multithreaded programming, this book also shows you how to write software components that work together over a network to solve problems and do work.

...
See more details below
Other sellers (Paperback)
  • All (6) from $42.99   
  • New (4) from $42.99   
  • Used (2) from $46.80   
Sending request ...

Overview

Parallel and Distributed Programming Using C++ provides an up-close look at how to build software that can take advantage of multiprocessor computers. Simple approaches for programming parallel virtual machines are presented, and the basics of cluster application development are explained. Through an easy-to-understand overview of multithreaded programming, this book also shows you how to write software components that work together over a network to solve problems and do work.

Parallel and Distributed Programming Using C++ provides an architectural approach to parallel programming for computer programmers, software developers, designers, researchers, and software architects. It will also be useful for computer science students.

  • Demonstrates how agents and blackboards can be used to make parallel programming easier
  • Shows object-oriented approaches to multitasking and multithreading
  • Demonstrates how the UML is used to document designs that require parallel or distributed programming
  • Contains the new POSIX/UNIX IEEE Standard for the Pthreads library
Read More Show Less

Product Details

  • ISBN-13: 9780321544674
  • Publisher: Addison-Wesley
  • Publication date: 2/26/2008
  • Pages: 720
  • Product dimensions: 6.80 (w) x 9.10 (h) x 1.60 (d)

Meet the Author

CAMERON HUGHES is a senior software engineer for CTEST Laboratories and a staff programmer/analyst at Youngstown State University. He has been involved in software development efforts of all sizes and is currently working in cluster programming of the Proteus V that involves NLP and knowledge representation modeling at CTEST and the Colony at Youngstown.

TRACEY HUGHES is a software engineer at CTEST, where she does graphics programming and simulation in C++ and develops image processing class libraries and virtual worlds.

Together they are the co-authors of numerous books including Linux Rapid Application Development and Object-Oriented Multithreading Using C++.

Read More Show Less

Read an Excerpt

We present an architectural approach to distributed and parallel programming using the C++ language. Particular attention is paid to how the C++ standard library, algorithms, and container classes behave in distributed and parallel environments. Methods for extending the C++ language through class libraries and function libraries to accomplish distributed and parallel programming tasks are explained. Emphasis is placed on how C++ works with the new POSIX and Single UNIX standards for multithreading. Combining C++ executables with other language executables to achieve multilingual solutions to distributed or parallel programming problems is also discussed. Several methods of organizing software that support parallel and distributed programming are introduced.

We demonstrate how to remove the fundamental obstacles to concurrency. The notion of emergent parallelization is explored. Our focus is not on optimization techniques, hardware specifics, performance comparisons, or on trying to apply parallel programming techniques to complex scientific or mathematical algorithms; rather, on how to structure computer programs and software systems to take advantage of opportunities for parallelization.

Furthermore, we acquaint the reader with a multiparadigm approach to solving some of the problems that are inherent with distributed and parallel programming.

Effective solutions to these problems often require a mix of several software design and engineering approaches. For instance, we deploy object-oriented programming techniques to tackle data race and synchronization problems. We use agent-oriented architectures to deal with multiprocess and multithread management. Blackboards are used to minimize communication issues. In addition to object-oriented, agent-oriented, and AI-oriented programming, we use parameterized programming to implement generalized algorithms that are suitable where concurrency is required.

Our experience with the development of software of all sizes and shapes has led us to believe that successful software design and implementation demands versatility. The suggestions, ideas, and solutions we present in this book reflect that experience.

The Challenges

There are three basic challenges to writing parallel or distributed programs:

  1. Identifying the natural parallelism that occurs within the context of a problem domain.
  2. Dividing the software appropriately into two or more tasks that can be performed at the same time to accomplish the required parallelism.
  3. Coordinating those tasks so that the software correctly and efficiently does what it is supposed to do.

These three challenges are accompanied by the following obstacles to concurrency:

Data race
Deadlock detection
Partial failure
Latency
Deadlock
Communication failures
Termination detection
Lack of global state
Multiple clock problem
Protocol mismatch
Localized errors
Lack of centralized resource allocation

This book explains what these obstacles are, why they occur, and how they can be managed.

Finally, several of the mechanisms we use for concurrency use TCP/IP as a protocol. Specifically the MPI (Message Passing Interface) library, PVM (Parallel Virtual Machine) library, and the MICO (CORBA) library. This allows our approaches to be used in an Internet/Intranet environment, which means that programs cooperating in parallel may be executing at different sites on the Internet or a corporate intranet and communicating through message passing. Many of the ideas serve as foundations for infrastructure of Web services. In addition to the MPI and PVM routines, the CORBA objects we use can communicate from different servers accross the Internet. These components can be used to provide a variety of Internet/intranet services.

The Approach

We advocate a component approach to the challenges and obstacles found in distributed and parallel programming. Our primary objective is to use framework classes as building blocks for concurrency. The framework classes are supported by object-oriented mutexes, semaphores, pipes, and sockets. The complexity of task synchronization and communication is significantly reduced through the use of interface classes. We deploy agent-driven threads and processes to facilitate thread and process management. Our primary approach to a global state and its related problems involve the use of blackboards.

We combine agent-oriented and object-oriented architectures to accomplish multiparadigm solutions. Our multiparadigm approach is made possible using the support C++ has for object-oriented programming, parameterized programming, and structured programming.

Why C++?

There are C++ compilers available for virtually every platform and operating environment. The ANSI (American National Standards Institute) and ISO (International Standard Organization) have defined standards for the C++ language and its library. There are robust open-source implementations as well as commercial implementations of the language. The language has been widely adopted by researchers, designers, and professional developers around the world. The C++ language has been used to solve problems of all sizes and shapes from device drivers to large-scale industrial applications.

The language supports a multiparadigm approach to software development and libraries that add parallel and distributed programming capabilities are readily available.

Libraries for Parallel and Distributed Programming

The MPICH, an implementation of MPI, the PVM library, and the Pthreads (POSIX Threads) library, are used to implement parallel programming using C++. MICO, a C++ implementation of the CORBA standard, is used to achieve distrbuted programming. The C++ Standard Library, in combination with CORBA and the Pthreads library, provides the support for agentoriented and blackboard programming concepts that are discussed in this book.

The New Single UNIX Specification Standard

The new Single UNIX Specification Standard, Version 3, a joint effort between IEEE and the Open Group, was finalized and released in December 2001. The new Single UNIX Specification encompasses the POSIX standards and promotes portability for application programmers. It was designed to give software developers a single set of APIs to be supported by every UNIX system. It provides a reliable road map of standards for programmers who need to write multitasking and multithreading applications. In this book we rely on the Single UNIX Specification Standard for our discussions on process creations, process management, the Pthreads library, the new posix_spawn() routines, the POSIX semaphores, and FIFOs. Appendix B in this book contains excerpts from the standard that can be used as a reference to the material that we present.

Who is This Book For?

This book is written for software designers, software developers, application programmers, researchers, educators, and students who need an introduction to parallel and distributed programming using the C++ language. A modest knowledge of the C++ language and standard C++ class libraries is required. This book is not intended as a tutorial on programming in C++ or object-oriented programming. It is assumed that the reader will have a basic understanding of object-oriented programming techniques such as encapsulation, inheritance, and polymorphism. This book introduces the basics of parallel and distributed programming in the context of C++.

Development Environments Supported

The examples and programs presented in this book were developed and tested in the Linux and UNIX environments, specifically with Solaris 8, Aix, and Linux (SuSE, Red Hat). The PVM and MPI code was developed and tested on a 32-node Linux-based cluster. Many of the programs were tested on Sun Enterprise 450s. We used Sun's C++ Workshop, The Portland Group's C++ compiler, and GNU C++. Most examples will run in both the UNIX and Linux environments. In the cases where an example will not run in both environments, this fact is noted in the Program Profiles that are provided for all the complete program examples in the book.

AncillariesUML Diagrams

Many of the diagrams in this book use the UML (Unified Modeling Language) standard. In particular, activity diagrams, deployment diagrams, class diagrams, the state diagrams are used to describe important concurrency architectures and class relationships. Although a knowledge of the UML is not necessary, familarity is helpful. Appendix A contains an explanation and description of the UML symbols and language that we use in this book.

Program Profiles

Each complete program in the book is accompanied by a program profile.

The profile will contain implementation specifics such as headers required, libraries required, compile instructions, and link instructions. The profile also includes a Notes section that will contain any special considerations that need to be taken when executing the program. Code that is not accompanied by a profile is meant for exposition purposes only.

Sidebars

We made every attempt to stay away from notation that is too theoretical for a introductory book such as this one. However, in some cases the theoretical or mathematical notation was unavoidable. In those cases we use the notation but we provide a detailed explanation of the notation in a sidebar.

Testing and Code Reliability

Although all examples and applications in this book were tested to ensure correctness, we make no warranties that the programs contained in this book are free of defects or error, are consistent with any particular standard or merchantability, or will meet your requirement for any particular application.

They should not be relied upon for solving problems whose incorrect solution could result in injury to person or loss of property. The authors and publishers disclaim all liability for direct or consequential damages resulting from your use of the examples, programs, or applications present in this book.

Acknowledgments

We could not have successfully pulled this project off without the help, suggestions, constructive criticisms, and resources of many of our friends and colleagues. In particular, we would like to thank Terry Lewis and Doug Johnson from OSC (Ohio Super-Computing) for providing us with complete access to a 32-node Linux-based cluster. To Mark Welton from YSU for his expertise and help with configuring the cluster to support our PVM and MPI programs. To Sal Sanders from YSU for providing us with access to Power- PCs running Mac OSX and Adobe Illustrator. To Brian Nelson from YSU for allowing us to test many of our multithreaded and distributed programs on multiprocessor Sun E-250s and E-450s. We are also indebted to Mary Ann Johnson and Jeffrey Trimble from YSU MAAG for helping us locate and hold on to the technical references we required. Claudio M. Stanziola, Paulette Goldweber, and Jacqueline Hansson from the IEEE Standards and Licensing and Contracts Office for obtaining permission to reprint parts of the new Single-UNIX/POSIX standard; Andrew Josey and Gene Pierce from The Open Group was also helpful in this regard. Thanks to Trevor Watkins of the Z-Group for all his help with the testing of the program examples; his multi- Linux distribution environment was especially important in the testing process. A special thanks to Steve Tarasweki for agreeing to provide a technical review for this book while it was in its roughest form. To Dr. Eugene Santos for pointing us in the right direction as we explored how categorical data structures could be used with PVMs. To Dr. Mike Crescimanno from the Advanced Computing Work Group at YSU for allowing us to present some of the materials from this book at one of the ACWG meetings. Finally, to Paul Petralia and the production team (especially Gail Cocker-Bogusz) from Prentice Hall who had to put up with all of our missed deadlines and strange UNIX/Linux file formats—we are extremely indebted to their patience, encouragement, enthusiasm, and professionalism.

Read More Show Less

Table of Contents

Preface.

1. The Joys of Concurrent Programming.

What is Concurrency? The Benefits of Parallel Programming. The Benefits of Distributed Programming. The Minimal Effort Required. The Basic Layers of Software Concurrency. No Keyword Support for Parallelism in C++. Programming Environments for Parallel and Distributed Programming. Summary–Toward Concurrency.

2. The Challenges of Parallel and Distributed Programming.

The Big Paradigm Shift. Coordination Challenges. Sometimes Hardware Fails and Software Quits. Too Much Parallelization or Distribution Can Have Negative Consequences. Selecting a Good Architecture Requires Research. Different Techniques for Testing and Debugging are Required. The Parallel or Distributed Design Must Be Communicated. Summary.

3. Dividing C++ Programs into Multiple Tasks.

Process: A Definition. Anatomy of a Process. Process States. Process Scheduling. Context Switching. Creating a Process. Terminating a Process. Process Resources. What are Asynchronous and Synchronous Processes? Dividing the Program into Tasks. Summary.

4. Dividing C++ Programs into Multiple Threads.

Threads: A Definition. The Anatomy of a Thread. Thread Scheduling. Thread Resources. Thread Models. Introduction to the Pthread Library. The Anatomy of a Simple Threaded Program. Creating Threads. Managing Threads. Thread Safety and Libraries. Dividing Your Program into Multiple Threads. Summary.

5. Synchronizing Concurrency between Tasks.

Coordinating Order of Execution. Synchronizing Access to Data. What are Semaphores? Synchronization: An Object-Oriented Approach. Summary.

6. Adding Parallel Programming Capabilities to C++ through the PVM.

The Classic Parallelism Models Supported by PVM. The PVM Library for C++. The Basic Mechanics of the PVM. Accessing Standard Input (stdin) and Standard Output (stdout) within PVM Tasks. Summary.

7. Error Handling, Exceptions, and Software Reliability.

What is Software Reliability? Failures in Software Layers and Hardware Components. Definitions of Defects Depend on Software Specifications. Recognizing Where to Handle Defects versus Where to Handle Exceptions. Software Reliability: A Simple Plan. Using Map Objects in Error Handling. Exception Handling Mechanisms in C++. Event Diagrams, Logic Expressions, and Logic Diagrams. Summary.

8. Distributed Object-Oriented Programming in C++.

Decomposition and Encapsulation of the Work. Accessing Objects in Other Address Spaces. The Anatomy of a Basic CORBA Consumer. The Anatomy of a CORBA Producer. The Basic Blueprint of a CORBA Application. The Naming Service. A Closer Look at Object Adapters. Implementation and Interface Repositories. Simple Distributed Web Services Using CORBA. The Trading Service. The Client/Server Paradigm. Summary.

9. SPMD and MPMD Using Templates and the MPI.

Work Breakdown Structure for the MPI. Using Template Functions to Represent MPI Tasks. Simplifying MPI Communications. Summary.

10. Visualizing Concurrent and Distributed System Design.

Visualizing Structures. Visualizing Concurrent Behavior. Visualizing the Whole System. Summary.

11. Designing Components That Support Concurrency.

Taking Advantage of Interface Classes. A Closer Look at Object-Oriented Mutual Exclusion and Interface Classes. Maintaining the Stream Metaphor. User-Defined Classes Designed to Work with PVM Streams. Object-Oriented Pipes and fifos as Low-Level Building Blocks. Framework Classes Components for Concurrency. Summary.

12. Implementing Agent-Oriented Architectures.

What are Agents? What is Agent-Oriented Programming? Basic Agent Components. Implementing Agents in C++. Multiagent Systems. Summary.

13. Blackboard Architectures Using PVM, Threads, And C++ Components.

The Blackboard Model. Approaches to Structuring the Blackboard. The Anatomy of a Knowledge Source. The Control Strategies for Blackboards. Implementing the Blackboard Using CORBA Objects. Implementing the Blackboard Using Global Objects. Activating Knowledge Sources Using Pthreads. Summary.

Appendix A.

Class and Object Diagrams. Interaction Diagrams. State Diagrams. Package Diagrams.

Appendix B.

Bibliography.

Index.

Read More Show Less

Preface

We present an architectural approach to distributed and parallel programming using the C++ language. Particular attention is paid to how the C++ standard library, algorithms, and container classes behave in distributed and parallel environments. Methods for extending the C++ language through class libraries and function libraries to accomplish distributed and parallel programming tasks are explained. Emphasis is placed on how C++ works with the new POSIX and Single UNIX standards for multithreading. Combining C++ executables with other language executables to achieve multilingual solutions to distributed or parallel programming problems is also discussed. Several methods of organizing software that support parallel and distributed programming are introduced.

We demonstrate how to remove the fundamental obstacles to concurrency. The notion of emergent parallelization is explored. Our focus is not on optimization techniques, hardware specifics, performance comparisons, or on trying to apply parallel programming techniques to complex scientific or mathematical algorithms; rather, on how to structure computer programs and software systems to take advantage of opportunities for parallelization.

Furthermore, we acquaint the reader with a multiparadigm approach to solving some of the problems that are inherent with distributed and parallel programming.

Effective solutions to these problems often require a mix of several software design and engineering approaches. For instance, we deploy object-oriented programming techniques to tackle data race and synchronization problems. We use agent-oriented architectures to deal with multiprocess and multithread management. Blackboards areused to minimize communication issues. In addition to object-oriented, agent-oriented, and AI-oriented programming, we use parameterized programming to implement generalized algorithms that are suitable where concurrency is required.

Our experience with the development of software of all sizes and shapes has led us to believe that successful software design and implementation demands versatility. The suggestions, ideas, and solutions we present in this book reflect that experience.

The Challenges

There are three basic challenges to writing parallel or distributed programs:

  1. Identifying the natural parallelism that occurs within the context of a problem domain.
  2. Dividing the software appropriately into two or more tasks that can be performed at the same time to accomplish the required parallelism.
  3. Coordinating those tasks so that the software correctly and efficiently does what it is supposed to do.

These three challenges are accompanied by the following obstacles to concurrency:

Data race Deadlock detection
Partial failure Latency
Deadlock Communication failures
Termination detection Lack of global state
Multiple clock problem Protocol mismatch
Localized errors Lack of centralized resource allocation

This book explains what these obstacles are, why they occur, and how they can be managed.

Finally, several of the mechanisms we use for concurrency use TCP/IP as a protocol. Specifically the MPI (Message Passing Interface) library, PVM (Parallel Virtual Machine) library, and the MICO (CORBA) library. This allows our approaches to be used in an Internet/Intranet environment, which means that programs cooperating in parallel may be executing at different sites on the Internet or a corporate intranet and communicating through message passing. Many of the ideas serve as foundations for infrastructure of Web services. In addition to the MPI and PVM routines, the CORBA objects we use can communicate from different servers accross the Internet. These components can be used to provide a variety of Internet/intranet services.

The Approach

We advocate a component approach to the challenges and obstacles found in distributed and parallel programming. Our primary objective is to use framework classes as building blocks for concurrency. The framework classes are supported by object-oriented mutexes, semaphores, pipes, and sockets. The complexity of task synchronization and communication is significantly reduced through the use of interface classes. We deploy agent-driven threads and processes to facilitate thread and process management. Our primary approach to a global state and its related problems involve the use of blackboards.

We combine agent-oriented and object-oriented architectures to accomplish multiparadigm solutions. Our multiparadigm approach is made possible using the support C++ has for object-oriented programming, parameterized programming, and structured programming.

Why C++?

There are C++ compilers available for virtually every platform and operating environment. The ANSI (American National Standards Institute) and ISO (International Standard Organization) have defined standards for the C++ language and its library. There are robust open-source implementations as well as commercial implementations of the language. The language has been widely adopted by researchers, designers, and professional developers around the world. The C++ language has been used to solve problems of all sizes and shapes from device drivers to large-scale industrial applications.

The language supports a multiparadigm approach to software development and libraries that add parallel and distributed programming capabilities are readily available.

Libraries for Parallel and Distributed Programming

The MPICH, an implementation of MPI, the PVM library, and the Pthreads (POSIX Threads) library, are used to implement parallel programming using C++. MICO, a C++ implementation of the CORBA standard, is used to achieve distrbuted programming. The C++ Standard Library, in combination with CORBA and the Pthreads library, provides the support for agentoriented and blackboard programming concepts that are discussed in this book.

The New Single UNIX Specification Standard

The new Single UNIX Specification Standard, Version 3, a joint effort between IEEE and the Open Group, was finalized and released in December 2001. The new Single UNIX Specification encompasses the POSIX standards and promotes portability for application programmers. It was designed to give software developers a single set of APIs to be supported by every UNIX system. It provides a reliable road map of standards for programmers who need to write multitasking and multithreading applications. In this book we rely on the Single UNIX Specification Standard for our discussions on process creations, process management, the Pthreads library, the new posix_spawn() routines, the POSIX semaphores, and FIFOs. Appendix B in this book contains excerpts from the standard that can be used as a reference to the material that we present.

Who is This Book For?

This book is written for software designers, software developers, application programmers, researchers, educators, and students who need an introduction to parallel and distributed programming using the C++ language. A modest knowledge of the C++ language and standard C++ class libraries is required. This book is not intended as a tutorial on programming in C++ or object-oriented programming. It is assumed that the reader will have a basic understanding of object-oriented programming techniques such as encapsulation, inheritance, and polymorphism. This book introduces the basics of parallel and distributed programming in the context of C++.

Development Environments Supported

The examples and programs presented in this book were developed and tested in the Linux and UNIX environments, specifically with Solaris 8, Aix, and Linux (SuSE, Red Hat). The PVM and MPI code was developed and tested on a 32-node Linux-based cluster. Many of the programs were tested on Sun Enterprise 450s. We used Sun's C++ Workshop, The Portland Group's C++ compiler, and GNU C++. Most examples will run in both the UNIX and Linux environments. In the cases where an example will not run in both environments, this fact is noted in the Program Profiles that are provided for all the complete program examples in the book.

Ancillaries

UML Diagrams

Many of the diagrams in this book use the UML (Unified Modeling Language) standard. In particular, activity diagrams, deployment diagrams, class diagrams, the state diagrams are used to describe important concurrency architectures and class relationships. Although a knowledge of the UML is not necessary, familarity is helpful. Appendix A contains an explanation and description of the UML symbols and language that we use in this book.

Program Profiles

Each complete program in the book is accompanied by a program profile.

The profile will contain implementation specifics such as headers required, libraries required, compile instructions, and link instructions. The profile also includes a Notes section that will contain any special considerations that need to be taken when executing the program. Code that is not accompanied by a profile is meant for exposition purposes only.

Sidebars

We made every attempt to stay away from notation that is too theoretical for a introductory book such as this one. However, in some cases the theoretical or mathematical notation was unavoidable. In those cases we use the notation but we provide a detailed explanation of the notation in a sidebar.

Testing and Code Reliability

Although all examples and applications in this book were tested to ensure correctness, we make no warranties that the programs contained in this book are free of defects or error, are consistent with any particular standard or merchantability, or will meet your requirement for any particular application.

They should not be relied upon for solving problems whose incorrect solution could result in injury to person or loss of property. The authors and publishers disclaim all liability for direct or consequential damages resulting from your use of the examples, programs, or applications present in this book.

Acknowledgments

We could not have successfully pulled this project off without the help, suggestions, constructive criticisms, and resources of many of our friends and colleagues. In particular, we would like to thank Terry Lewis and Doug Johnson from OSC (Ohio Super-Computing) for providing us with complete access to a 32-node Linux-based cluster. To Mark Welton from YSU for his expertise and help with configuring the cluster to support our PVM and MPI programs. To Sal Sanders from YSU for providing us with access to Power- PCs running Mac OSX and Adobe Illustrator. To Brian Nelson from YSU for allowing us to test many of our multithreaded and distributed programs on multiprocessor Sun E-250s and E-450s. We are also indebted to Mary Ann Johnson and Jeffrey Trimble from YSU MAAG for helping us locate and hold on to the technical references we required. Claudio M. Stanziola, Paulette Goldweber, and Jacqueline Hansson from the IEEE Standards and Licensing and Contracts Office for obtaining permission to reprint parts of the new Single-UNIX/POSIX standard; Andrew Josey and Gene Pierce from The Open Group was also helpful in this regard. Thanks to Trevor Watkins of the Z-Group for all his help with the testing of the program examples; his multi- Linux distribution environment was especially important in the testing process. A special thanks to Steve Tarasweki for agreeing to provide a technical review for this book while it was in its roughest form. To Dr. Eugene Santos for pointing us in the right direction as we explored how categorical data structures could be used with PVMs. To Dr. Mike Crescimanno from the Advanced Computing Work Group at YSU for allowing us to present some of the materials from this book at one of the ACWG meetings. Finally, to Paul Petralia and the production team (especially Gail Cocker-Bogusz) from Prentice Hall who had to put up with all of our missed deadlines and strange UNIX/Linux file formats--we are extremely indebted to their patience, encouragement, enthusiasm, and professionalism.

Read More Show Less

Introduction

We present an architectural approach to distributed and parallel programming using the C++ language. Particular attention is paid to how the C++ standard library, algorithms, and container classes behave in distributed and parallel environments. Methods for extending the C++ language through class libraries and function libraries to accomplish distributed and parallel programming tasks are explained. Emphasis is placed on how C++ works with the new POSIX and Single UNIX standards for multithreading. Combining C++ executables with other language executables to achieve multilingual solutions to distributed or parallel programming problems is also discussed. Several methods of organizing software that support parallel and distributed programming are introduced.

We demonstrate how to remove the fundamental obstacles to concurrency. The notion of emergent parallelization is explored. Our focus is not on optimization techniques, hardware specifics, performance comparisons, or on trying to apply parallel programming techniques to complex scientific or mathematical algorithms; rather, on how to structure computer programs and software systems to take advantage of opportunities for parallelization.

Furthermore, we acquaint the reader with a multiparadigm approach to solving some of the problems that are inherent with distributed and parallel programming.

Effective solutions to these problems often require a mix of several software design and engineering approaches. For instance, we deploy object-oriented programming techniques to tackle data race and synchronization problems. We use agent-oriented architectures to deal with multiprocess and multithread management.Blackboards are used to minimize communication issues. In addition to object-oriented, agent-oriented, and AI-oriented programming, we use parameterized programming to implement generalized algorithms that are suitable where concurrency is required.

Our experience with the development of software of all sizes and shapes has led us to believe that successful software design and implementation demands versatility. The suggestions, ideas, and solutions we present in this book reflect that experience.

The Challenges

There are three basic challenges to writing parallel or distributed programs:

  1. Identifying the natural parallelism that occurs within the context of a problem domain.
  2. Dividing the software appropriately into two or more tasks that can be performed at the same time to accomplish the required parallelism.
  3. Coordinating those tasks so that the software correctly and efficiently does what it is supposed to do.

These three challenges are accompanied by the following obstacles to concurrency:

Data race Deadlock detection
Partial failure Latency
Deadlock Communication failures
Termination detection Lack of global state
Multiple clock problem Protocol mismatch
Localized errors Lack of centralized resource allocation

This book explains what these obstacles are, why they occur, and how they can be managed.

Finally, several of the mechanisms we use for concurrency use TCP/IP as a protocol. Specifically the MPI (Message Passing Interface) library, PVM (Parallel Virtual Machine) library, and the MICO (CORBA) library. This allows our approaches to be used in an Internet/Intranet environment, which means that programs cooperating in parallel may be executing at different sites on the Internet or a corporate intranet and communicating through message passing. Many of the ideas serve as foundations for infrastructure of Web services. In addition to the MPI and PVM routines, the CORBA objects we use can communicate from different servers across the Internet. These components can be used to provide a variety of Internet/intranet services.

The Approach

We advocate a component approach to the challenges and obstacles found in distributed and parallel programming. Our primary objective is to use framework classes as building blocks for concurrency. The framework classes are supported by object-oriented mutexes, semaphores, pipes, and sockets. The complexity of task synchronization and communication is significantly reduced through the use of interface classes. We deploy agent-driven threads and processes to facilitate thread and process management. Our primary approach to a global state and its related problems involve the use of blackboards.

We combine agent-oriented and object-oriented architectures to accomplish multiparadigm solutions. Our multiparadigm approach is made possible using the support C++ has for object-oriented programming, parameterized programming, and structured programming.

Why C++?

There are C++ compilers available for virtually every platform and operating environment. The ANSI (American National Standards Institute) and ISO (International Standard Organization) have defined standards for the C++ language and its library. There are robust open-source implementations as well as commercial implementations of the language. The language has been widely adopted by researchers, designers, and professional developers around the world. The C++ language has been used to solve problems of all sizes and shapes from device drivers to large-scale industrial applications.

The language supports a multiparadigm approach to software development and libraries that add parallel and distributed programming capabilities are readily available.

Libraries for Parallel and Distributed Programming

The MPICH, an implementation of MPI, the PVM library, and the Pthreads (POSIX Threads) library, are used to implement parallel programming using C++. MICO, a C++ implementation of the CORBA standard, is used to achieve distributed programming. The C++ Standard Library, in combination with CORBA and the Pthreads library, provides the support for agentoriented and blackboard programming concepts that are discussed in this book.

The New Single UNIX Specification Standard

The new Single UNIX Specification Standard, Version 3, a joint effort between IEEE and the Open Group, was finalized and released in December 2001. The new Single UNIX Specification encompasses the POSIX standards and promotes portability for application programmers. It was designed to give software developers a single set of APIs to be supported by every UNIX system. It provides a reliable road map of standards for programmers who need to write multitasking and multithreading applications. In this book we rely on the Single UNIX Specification Standard for our discussions on process creations, process management, the Pthreads library, the new posix_spawn() routines, the POSIX semaphores, and FIFOs. Appendix B in this book contains excerpts from the standard that can be used as a reference to the material that we present.

Who is This Book For?

This book is written for software designers, software developers, application programmers, researchers, educators, and students who need an introduction to parallel and distributed programming using the C++ language. A modest knowledge of the C++ language and standard C++ class libraries is required. This book is not intended as a tutorial on programming in C++ or object-oriented programming. It is assumed that the reader will have a basic understanding of object-oriented programming techniques such as encapsulation, inheritance, and polymorphism. This book introduces the basics of parallel and distributed programming in the context of C++.

Development Environments Supported

The examples and programs presented in this book were developed and tested in the Linux and UNIX environments, specifically with Solaris 8, Aix, and Linux (SuSE, Red Hat). The PVM and MPI code was developed and tested on a 32-node Linux-based cluster. Many of the programs were tested on Sun Enterprise 450s. We used Sun's C++ Workshop, The Portland Group's C++ compiler, and GNU C++. Most examples will run in both the UNIX and Linux environments. In the cases where an example will not run in both environments, this fact is noted in the Program Profiles that are provided for all the complete program examples in the book.

Ancillaries

UML Diagrams

Many of the diagrams in this book use the UML (Unified Modeling Language) standard. In particular, activity diagrams, deployment diagrams, class diagrams, the state diagrams are used to describe important concurrency architectures and class relationships. Although a knowledge of the UML is not necessary, familarity is helpful. Appendix A contains an explanation and description of the UML symbols and language that we use in this book.

Program Profiles

Each complete program in the book is accompanied by a program profile.

The profile will contain implementation specifics such as headers required, libraries required, compile instructions, and link instructions. The profile also includes a Notes section that will contain any special considerations that need to be taken when executing the program. Code that is not accompanied by a profile is meant for exposition purposes only.

Sidebars

We made every attempt to stay away from notation that is too theoretical for a introductory book such as this one. However, in some cases the theoretical or mathematical notation was unavoidable. In those cases we use the notation but we provide a detailed explanation of the notation in a sidebar.

Testing and Code Reliability

Although all examples and applications in this book were tested to ensure correctness, we make no warranties that the programs contained in this book are free of defects or error, are consistent with any particular standard or merchantability, or will meet your requirement for any particular application.

They should not be relied upon for solving problems whose incorrect solution could result in injury to person or loss of property. The authors and publishers disclaim all liability for direct or consequential damages resulting from your use of the examples, programs, or applications present in this book.

Acknowledgments

We could not have successfully pulled this project off without the help, suggestions, constructive criticisms, and resources of many of our friends and colleagues. In particular, we would like to thank Terry Lewis and Doug Johnson from OSC (Ohio Super-Computing) for providing us with complete access to a 32-node Linux-based cluster. To Mark Welton from YSU for his expertise and help with configuring the cluster to support our PVM and MPI programs. To Sal Sanders from YSU for providing us with access to Power- PCs running Mac OSX and Adobe Illustrator. To Brian Nelson from YSU for allowing us to test many of our multithreaded and distributed programs on multiprocessor Sun E-250s and E-450s. We are also indebted to Mary Ann Johnson and Jeffrey Trimble from YSU MAAG for helping us locate and hold on to the technical references we required. Claudio M. Stanziola, Paulette Goldweber, and Jacqueline Hansson from the IEEE Standards and Licensing and Contracts Office for obtaining permission to reprint parts of the new Single-UNIX/POSIX standard; Andrew Josey and Gene Pierce from The Open Group was also helpful in this regard. Thanks to Trevor Watkins of the Z-Group for all his help with the testing of the program examples; his multi- Linux distribution environment was especially important in the testing process. A special thanks to Steve Tarasweki for agreeing to provide a technical review for this book while it was in its roughest form. To Dr. Eugene Santos for pointing us in the right direction as we explored how categorical data structures could be used with PVMs. To Dr. Mike Crescimanno from the Advanced Computing Work Group at YSU for allowing us to present some of the materials from this book at one of the ACWG meetings. Finally, to Paul Petralia and the production team (especially Gail Cocker-Bogusz) from Prentice Hall who had to put up with all of our missed deadlines and strange UNIX/Linux file formats--we are extremely indebted to their patience, encouragement, enthusiasm, and professionalism.

Read More Show Less

Customer Reviews

Be the first to write a review
( 0 )
Rating Distribution

5 Star

(0)

4 Star

(0)

3 Star

(0)

2 Star

(0)

1 Star

(0)

Your Rating:

Your Name: Create a Pen Name or

Barnes & Noble.com Review Rules

Our reader reviews allow you to share your comments on titles you liked, or didn't, with others. By submitting an online review, you are representing to Barnes & Noble.com that all information contained in your review is original and accurate in all respects, and that the submission of such content by you and the posting of such content by Barnes & Noble.com does not and will not violate the rights of any third party. Please follow the rules below to help ensure that your review can be posted.

Reviews by Our Customers Under the Age of 13

We highly value and respect everyone's opinion concerning the titles we offer. However, we cannot allow persons under the age of 13 to have accounts at BN.com or to post customer reviews. Please see our Terms of Use for more details.

What to exclude from your review:

Please do not write about reviews, commentary, or information posted on the product page. If you see any errors in the information on the product page, please send us an email.

Reviews should not contain any of the following:

  • - HTML tags, profanity, obscenities, vulgarities, or comments that defame anyone
  • - Time-sensitive information such as tour dates, signings, lectures, etc.
  • - Single-word reviews. Other people will read your review to discover why you liked or didn't like the title. Be descriptive.
  • - Comments focusing on the author or that may ruin the ending for others
  • - Phone numbers, addresses, URLs
  • - Pricing and availability information or alternative ordering information
  • - Advertisements or commercial solicitation

Reminder:

  • - By submitting a review, you grant to Barnes & Noble.com and its sublicensees the royalty-free, perpetual, irrevocable right and license to use the review in accordance with the Barnes & Noble.com Terms of Use.
  • - Barnes & Noble.com reserves the right not to post any review -- particularly those that do not follow the terms and conditions of these Rules. Barnes & Noble.com also reserves the right to remove any review at any time without notice.
  • - See Terms of Use for other conditions and disclaimers.
Search for Products You'd Like to Recommend

Recommend other products that relate to your review. Just search for them below and share!

Create a Pen Name

Your Pen Name is your unique identity on BN.com. It will appear on the reviews you write and other website activities. Your Pen Name cannot be edited, changed or deleted once submitted.

 
Your Pen Name can be any combination of alphanumeric characters (plus - and _), and must be at least two characters long.

Continue Anonymously

    If you find inappropriate content, please report it to Barnes & Noble
    Why is this product inappropriate?
    Comments (optional)