The Art of Multiprocessor Programming, Revised Reprint and over 2 million other books are available for Amazon Kindle . Learn more
  • RRP: £45.99
  • You Save: £10.91 (24%)
FREE Delivery in the UK.
In stock.
Dispatched from and sold by Amazon.
Gift-wrap available.
The Art of Multiprocessor... has been added to your Basket
Trade in your item
Get a £4.75
Gift Card.
Have one to sell?
Flip to back Flip to front
Listen Playing... Paused   You're listening to a sample of the Audible audio edition.
Learn more
See all 2 images

The Art of Multiprocessor Programming, Revised Reprint Paperback – 12 Jun 2012

See all 2 formats and editions Hide other formats and editions
Amazon Price New from Used from
Kindle Edition
"Please retry"
"Please retry"
£26.00 £27.30

Frequently Bought Together

The Art of Multiprocessor Programming, Revised Reprint + Structured Parallel Programming: Patterns for Efficient Computation
Price For Both: £68.37

Buy the selected items together

Trade In this Item for up to £4.75
Trade in The Art of Multiprocessor Programming, Revised Reprint for an Amazon Gift Card of up to £4.75, which you can then spend on millions of items across the site. Trade-in values may vary (terms apply). Learn more

Product details

  • Paperback: 528 pages
  • Publisher: Morgan Kaufmann; Revised edition edition (12 Jun. 2012)
  • Language: English
  • ISBN-10: 0123973376
  • ISBN-13: 978-0123973375
  • Product Dimensions: 23.1 x 18.8 x 3.3 cm
  • Average Customer Review: 4.3 out of 5 stars  See all reviews (3 customer reviews)
  • Amazon Bestsellers Rank: 206,784 in Books (See Top 100 in Books)

More About the Authors

Discover books, learn about writers, and more.

Product Description


"The book could be used for a short course for practitioners looking for solutions to particular problems, a medium course for non-computer science major who would use multiprocessor programming in their own field, or a semester-long course for computer science majors."--Reference and Research Book News, February 2013

About the Author

Maurice Herlihy received an A.B. in Mathematics from Harvard University, and a Ph.D. in Computer Science from M.I.T. He has served on the faculty of Carnegie Mellon University, on the staff of DEC Cambridge Research Lab, and is currently a Professor in the Computer Science Department at Brown University. Maurice Herlihy is an ACM Fellow, and is the recipient of the 2003 Dijkstra Prize in Distributed Computing. He shared the 2004 G del Prize with Nir Shavit, the highest award in theoretical computer science. In 2012 he shared the Edsger W. Dijkstra Prize In Distributed Computing with Nir Shavit. Nir Shavit received a B.A. and M.Sc. from the Technion and a Ph.D. from the Hebrew University, all in Computer Science. From 1999 to 2011 he served as a member of technical staff at Sun Labs and Oracle Labs. He shared the 2004 G del Prize with Maurice Herlihy, the highest award in theoretical computer science. He is a Professor in the Electrical Engineering and Computer Science Department at M.I.T. and the Computer Science Department at Tel-Aviv University. In 2012 he shared the Edsger W. Dijkstra Prize In Distributed Computing with Maurice Herlihy.

Inside This Book (Learn More)
Browse Sample Pages
Front Cover | Copyright | Table of Contents | Excerpt | Index
Search inside this book:

Customer Reviews

4.3 out of 5 stars
5 star
4 star
3 star
2 star
1 star
See all 3 customer reviews
Share your thoughts with other customers

Most Helpful Customer Reviews

4 of 4 people found the following review helpful By Florian on 5 Oct. 2011
Format: Paperback
The book is quite good but it can sometimes be hard to read. Especially the algorithms could use some extra explination in a natural language. The book tends to skip to the formal proof right away which is harder to read if one does not completely understand the algorithm yet. The upside of this is that the book is quite to the point.

The book's examples are all in Java, which is usually no problem for someone who knows a comparable language like C#. There are however a few cases where examples refer to specific Java libraries which are not available for C# (or comparable languages) or which work differently. The book does have an appendix with a few pages about how to do multithreading in different langauges, but it does not go into differences important for the examples.
Comment Was this review helpful to you? Yes No Sending feedback...
Thank you for your feedback. If this review is inappropriate, please let us know.
Sorry, we failed to record your vote. Please try again
3 of 3 people found the following review helpful By Meredydd Williams on 10 Aug. 2013
Format: Kindle Edition Verified Purchase
Very comprehensive, covers all areas of multiprocessing, building from abstract primitives into fully implementable software constructs. Also co-authored by a Djikstra and Turing award winner seen as the leader within this field.
Comment Was this review helpful to you? Yes No Sending feedback...
Thank you for your feedback. If this review is inappropriate, please let us know.
Sorry, we failed to record your vote. Please try again
1 of 1 people found the following review helpful By nt2012 on 22 July 2014
Format: Paperback Verified Purchase
A well written book on multiprocessor programming.
The companion site is very helpful, make sure you check out the errata for this book, as there are quite a few code errors,
Comment Was this review helpful to you? Yes No Sending feedback...
Thank you for your feedback. If this review is inappropriate, please let us know.
Sorry, we failed to record your vote. Please try again

Most Helpful Customer Reviews on (beta) 22 reviews
104 of 108 people found the following review helpful
Soon to be the classic text on multiprocessor programming 7 Dec. 2008
By Justin E. Gottschlich - Published on
Format: Paperback Verified Purchase
The Art of Multiprocessor Programming is an outstanding text that will soon become a classic. I give a chapter by chapter review of it below.

Practitioners that are already well versed in parallel programming can jump directly to Chapter 7, however, I would suggest at least skimming Chapters 2, 3 and 4. Even those programmers who understand shared memory and locking may be shocked at how relaxed memory models or compiler optimizations can reorder operations causing innocent looking code to break.


Chapter 1 - Introduction

Why is this book called "The Art of Multiprocessor Programming" and not "The Art of Parallel Programming?" It is not by accident. There is a directed effort to explain parallel programming concepts as they relate to multi-core (or many-core) architectures. In particular, shared-memory multiprocessors have specific implementation details, such as cache coherence policies, that directly affect parallel software run on such architectures. The introduction gives a brief overview of the direction of the text: principles and practice.


Part 1 - Principles

Chapter 2 - Mutual Exclusion

Mutual exclusion is a key concept to multi-threaded programming, and this chapter is rightly placed at the beginning of the text. This chapter presents some of the foundational concepts in parallel computing, such as, understanding time related to operation interleavings, pessimistic critical sections, forward progress, deadlocks and fairness. In addition, some of the classic algorithms are presented here, such as Lamport's Ticket Locking and Peterson's 2-Threaded Lock.

Chapter 3 - Concurrent Objects

This chapter starts off simple, but gets complex fast. While experts will understand and acknowledge the importance of this chapter, less experienced programmers will find it very challenging to understand and may be turned off: don't give up!

My suggestion to non-experts is to focus on understanding two concepts of this chapter: hardware sequential consistency (3.4) and software linearizibility (3.5). Once you understand both concepts, skim all other sections except section 3.8.

Java programmers may want to pay special attention to the Java Memory Model section (3.8) and ill-formed unsynchronized code. General programmers will also be interested in this section as it is important to understand how the hardware's memory consistency model, the programming language's memory model and the compiler's operation reordering optimizations may interfere with what "looks like" correct code.

Do not be discouraged by the difficult of this chapter. It is one of the most difficult chapter in the text. Get through it and keep reading.

Chapter 4 - Foundations of Shared Memory

This chapter concentrates on understanding shared memory, the cornerstone of all multi-threaded applications. It explains how to implement shared memory that behaves "correctly" without using mutual exclusion. The different types of memory that are discussed are single-reader single-writer (SRSW), multiple-reader single-writer (MRSW) and multiple-reader multiple-writer (MRMW). This is an important chapter for non-experts to think about, as it explains how operation interleavings are not as discrete as we pretend they are and how shared memory should behave in all possible cases.

Chapter 5 - The Relative Power of Primitive Synchronization Operations

This chapter explains the varying strength of different wait-free synchronization primitives. Consensus numbers will undoubtedly confuse novice parallel programmers. In short, the higher the consensus number the better. A high consensus number, say N, for a synchronization primitive means that synchronization primitive can "correctly" solve the consensus problem for N concurrently executing threads. For example, critical sections have an infinite consensus number (e.g. support an infinite number of concurrent threads). Atomic registers have a consensus number of 1, they support only 1 thread's execution that is guaranteed to consistently and validly solve the consensus problem.

The most important point of this chapter (in my opinion) is that compare-and-swap (CAS), or compare-and-set, has an infinite consensus number (section 5.8). This is why modern instruction set architectures (ISAs) all provide CAS: it is critical to supporting an unlimited number of concurrently executing threads. Realizing the importance of CAS is vital for advanced parallel programmers who want to implement nonblocking algorithms.

Chapter 6 - Universality of Consensus

This chapter explains how to build universal consensus for your own concurrent objects. While it will be an interesting chapter for experts, novices may want to skip it.


Part II - Practice

Chapter 7 - Spinlocks and Contention

This chapter explains the important differences between different types of locking. It explains how to implement locks using assembly level operations (test-and-set and test-and-test-and-set), how to reduce bus contention using backoff, how to reduce cache pressure by having threads spin on their local cache memory and how to manage an unknown number of threads using locks.

After reading this chapter, most readers should have an appreciation for the hardware complexity of implementing something as simple as a lock. Some programmers may argue that they should not need to know how hardware behaves. While I would like to agree, the unfortunate state of multi-core programming currently requires a basic understanding of memory consistency models and cache behaviors. Herlihy and Shavit note this and make an effort to address it in a "just what you need to know" fashion, as done in this chapter.

Chapter 8 - Monitors and Blocking Synchronization

This chapter explains monitors, conditions, the differences between readers and writers, and reentrant locks. Java programmers will be especially interested in understanding monitors, while all OO programmers should have an appreciation of synchronizing an entire class. Moreover, the section on reentrant locks is simple but important to preventing deadlocks.


Unofficially, Chapters 9 - 11 focus on achieving parallelism in algorithms that have sequential bottlenecks and are therefore inherently sequential.


Chapter 9 - Linked Lists: The Role of Locking

The chapter explains how to implement ordered linked lists (e.g., IntSets or just sets) in a variety of different ways. The chapter starts out with the most basic implementation and then begins to increase performance by relaxing the strictness of the required thread synchronization.

Chapter 10 - Concurrent Queues and the ABA Problem

The chapter explains how to implement pools, a collection of unordered or ordered items. The chapter then explains the ways to implement pools as different types of queues, containers with first-in-first-out behavior. The chapter also explains a classic parallel problem known as ABA, where thread1 observes x == A, thread2 does x=B and then x=A and thread1 then observes x == A. The ABA problem is a subtle, but important problem.

Chapter 11 - Concurrent Stacks and Elimination

This chapter starts where the last chapter left off; it explains how to implement concurrent stacks, containers with first-in-last-out behavior. The chapter also explains a neat cancellation problem called elimination. Elimination is useful for avoiding overflows, underflows and other types of bounded problems.

Chapter 12 - Counting, Sorting and Distributed Coordination

This chapter explains how to take problems that seem to be inherently sequential and make them parallel. The chapter also explains both combining and diffracting trees, both of which are very interesting ways to make sequential problems parallel. This chapter is one of the more complex chapters of the text. Some readers may want to skim it.


Unofficially, Chapters 13 - 15 focus on achieving parallelism in algorithms that are inherently parallel. Readers will enjoy seeing how easy it is to extract parallelism in these naturally parallel algorithms.


Chapter 13 - Concurrent Hashing and Natural Parallelism

This chapter explains how to build parallel hash tables, with both open and closed addressing. First, the authors explain how to implement hash tables using coarse-grained locks, then with fine-grained locks, then with no locks at all. The chapter also explains how to deal with open-addressed hash tables which are particularly challenging.

Chapter 14 - Skiplists and Balanced Search

Most non-experts that make it this far in the text will be greatly rewarded by this chapter. Chapter 14 explains skiplists, an intriguing way to implement a container that has logarithmic search time and that is inherently parallel. Unlike red-black trees or balanced binary trees that yield logarithmic search time complexity, skiplists do not need to be rebalanced. Skiplists do not need to be rebalanced due to their unique algorithmic layering, making them inherently parallel. As such, skiplists have a notable benefit over their inherently sequential logarithmic search time counterparts.

This is a critically important chapter for practitioners hoping to exploit high parallelism while retaining logarithmic search time.

Chapter 15 - Priority Queues

This chapter explains how to implement priority queues, containers that are queues where each element has an identifiable level of importance. The authors demonstrate how to build priority queues with arrays, trees, heaps and skiplists.

Chapter 16 - Futures, Schedules and Work Distribution

This chapter presents some important aspects in understanding parallelism. In particular, the authors explain how to keep threads busy with work without causing the threads to become busy with "looking for work". The chapter also explains important ideas about the overhead of threads, stealing work, and sharing work. This chapter may cause some confusion to non-experts, but readers should try to understand at least the basic principles conveyed here as they are important to most general parallel programming problems.

Chapter 17 - Barriers

This chapter explains how to use barriers, a synchronization primitive that ensures threads move together through "gates" or "phases". Barriers are important in preventing threads from getting to far ahead or too far behind one another.

Chapter 18 - Transactional Memory

This chapter briefly describes a new parallel programming concept called transactional memory (TM). TM uses optimistic concurrency and greatly simplifies parallel programming. Herlihy and Shavit are responsible for HTM and STM, respectively.

The following ideas are touched on: HTM + cache coherence, composition, contention managers and transactional serialization. TM is currently receiving a lot of research attention and many researchers believe TM will soon become the new way to do parallel programming. Because of this, readers should pay particular attention to this chapter.
78 of 82 people found the following review helpful
The content is brilliant, the code is sometimes misleading 28 May 2008
By Vyacheslav Imameyev - Published on
Format: Paperback
The content is perfect and deserves 5 stars and I agree with the 5 stars comments, but the code deserves the only 3 stars as there are a lot of flaws in it - the code even contradicts its description( both in the book and in the code downloaded from a site ). For example, at chapter 8.3.1 the Readers-Writers ( i.e. multiple-readers-multiple-writers as the name suggests ) implementation is actually a multiple-readers-single-writer as the WriteLock.lock() method doesn't protect from multiple writers( there is a mention about a single writer in the text but the paragraph name suggests multiple writers ). The code at 8.3.2 is just misleading and doesn't match the description - again the WriteLock.lock() is flawed - it frees the lock if readAcquires != readReleases thus allowing the ReadLock.lock() method to acquire the lock and increment the readAcquires counter which results in the writer starvation and lost of fairness( should be FIFO ) and again there is no protection from multiple writers but the "Readers-Writers lock" name suggests that it should be. And as the last blow the code in 8.3.2 suffers from the lost-wakeup problem described two pages before - the WriterLock.unlock() method doesn't wake up the readers waiting in condition.await(). But there is a rehabilitation for the authors - the description for the code doesn't contain the flaws mentioned above - it is absolutely correct! The Chapter 8 drove me mad by its discrepancy between the text and the code!
So, I got suspicious about the code in the book but not about the description.
I rated the book 4 stars as the content and description( including pictures )is brilliant but the code is sometimes wrong and misleading ( I think it was copy-pasted from the old authors's works ), if the code had not contained such bizarre flaws I would have rated 5 stars as the content is really perfect and shows the authors expertise in the field.
32 of 32 people found the following review helpful
An invaluable resource for contemporary programmers 30 May 2008
By Spork - Published on
Format: Paperback Verified Purchase
This book gives programmers the practical and theoretical tools they need to adapt to the proliferation of multi-core machines. It opens with six chapters on theoretical subjects. These chapters are fascinating in their own right as well as directly applicable to my daily work. I thought the most important subjects were wait-free synchronization (every method completes in a finite number of steps), lock-free synchronization (some method completes in a finite number of steps), and some computability proofs. The authors use computability to demonstrate the equivalence of several types of synchronization primitives. They also present some impossibility proofs that show you how to avoid trying to solve unsolvable problems. The computability results and synchronization guarantees combine to give you the tools to determine whether one concurrent algorithm is "better" than another.

The remainder of the book is devoted to practical subjects. These chapters cover locks, a variety of data structures, work scheduling, and some miscellaneous topics. Java's java.util.concurrent package provides production-quality implementations of most of these data structures. The authors know this, and they use the data structures chapters to demonstrate generally applicable techniques while avoiding unnecessary implementation details. The work scheduling chapter is a sobering reminder of the difficulty inherent in fully exploiting highly parallel architectures. The authors show how to use recurrences to analyze the relative speedup an algorithm gains by running on P processors instead of a single processor. Combining this with the discussion of Ahmdal's Law earlier in the book we see that the essential math behind parallelism severely penalizes you for seemingly small sequential portions of your code. I also found the counting networks chapter fascinating, as I had never encountered that material before.

The book also provides appendices aimed at bringing inexperienced readers up to speed. That said, I wouldn't recommend this book for inexperienced programmers. The material is challenging. If you are looking for a gentler introduction to this subject, consider Java Concurrency in Practice. Each chapter ends with a note describing the history of the material and providing pointers to the bibliography. These demonstrate that the authors have been significant contributors to this field. I do agree with the review from Vyacheslav Imameyev - some of the code samples are wrong. I think they missed "volatile" keywords in several places. I don't see this as a cookbook, so I'm still giving the book five stars.

Highly parallel machines are here to stay. Programmers need to adapt to this or suffer competitive disadvantage. This is the book to read in order to meet that challenge.
18 of 19 people found the following review helpful
Challenging 9 Oct. 2008
By J. johnson - Published on
Format: Paperback
This book is challenging. Several occasions I flipped between chapters reviewing earlier notational details I thought I understood only to come to the conclusion the book is inconsistent in usage. An example of this the introduction of the precedence relation operator "->" (a right pointing arrow). -> is a total order on events, partial order on intervals. Events are denoted with lower-case labels while intervals upper-case, unless of course the upper-case label is denoting a thread/memory location/some other entity. The problem is by the time it's important to recall upper-case really does mean interval, there's been numerous examples where it hasn't. The material is challenging enough without this ambiguity. That said the notation introduced in chapters 2 & 3 have me excited for what's to come (I've just completed ch 3).

English description is good at communicating the broad sense while formal notation is needed to be rigorous. Throughout chapter 3 I was wanting more of both.

Finally like another reviewer I've noticed discrepancies between code and text. If you're experiencing a WTF moment, review twice break review again, then hit the errata. I'm hopeful my submitted erratum will help others.

To get the most out of this book do yourself a favor visit the companion site to get the slides and errata. Then hit iTunes U and look up Maurice Herlihy.
14 of 15 people found the following review helpful
At the Border of Theory and Practice 6 Sept. 2009
By Amazon Customer - Published on
Format: Paperback
What this book is not:
- A programmer's cookbook
- A programming manual
- A math text

This is an engineering theory textbook: enough theory to teach you how to reason about practical problems on the problems' own terms. If it were easy, such books would be unnecessary. Learning to reason about these problems is hard and the books are hard. This book is no exception. That said, it is generally very clear, but it is not light reading or easy study. It demands serious study time. Given the increasing importance of concurrency to real-world performance (and the economics of computing) the investment of time seems likely to be profitable.

The book progresses from simple (but still difficult) foundation principles to a variety of established techniques. It is not concerned with other issues of software design and construction but it covers its own purview thoroughly.

The exercises are non-trivial and sometimes geniunely hard. Some call for programming, some for thinking. They indicate that the authors have a deep grasp not only of their subject matter but of how to teach it. The examples are in Java. (A note on doing the exercises: due to the inherent limitations of threading on single-processor/single-core/single-thread machines, errors in code may not show up on such machines; if you don't have true multi-thread hardware, beware.)
Were these reviews helpful? Let us know