Programming Massively Parallel Processors Book


Most parts of the teaching material for beginners is covered by the book:

  • “Programming Massively Parallel Processors, A Hands-on Approach” by David Kirk and Wen-mei Hwu

NVIDIA, Jan. 28,2010: The first textbook of its kind, is authored by Dr. David B. Kirk, NVIDIA Fellow and former chief scientist, and Dr. Wen-mei Hwu, who serves at the University of Illinois at Urbana-Champaign as Chair of Electrical and Computer Engineering in the Coordinated Science Laboratory, co-director of the Universal Parallel Computing Research Center and principal investigator of the CUDA Center of Excellence.

The textbook, which is 256 pages, is the first aimed at teaching advanced students and professionals the basic concepts of parallel programming and GPU architectures. Published by Morgan Kaufmann, it explores various techniques for constructing parallel programs and reviews numerous case studies.

With conventional CPU-based computing no longer scaling in performance and the world’s computational challenges increasing in complexity, the need for massively parallel processing has never been greater. GPUs have hundreds of cores capable of delivering transformative performance increases across a wide range of computational challenges. The rise of these multi-core architectures has raised the need to teach advanced programmers a new and essential skill: how to program massively parallel processors.

“I’d like to personally congratulate David and Wen-mei for writing this landmark book and enabling generations of student programmers to understand and exploit the massively parallel architecture of GPUs,” said Bill Dally, chief scientist at NVIDIA and former chairman of Stanford University’s computer science department. “As a former professor, I have seen firsthand how seminal texts like this can transform a field. I look forward to seeing the transformation of computing as students are inspired and guided to master GPU computing by this book.”

Among the book’s key features:

  • First and only text that teaches how to program within a massively parallel environment
  • Portions of the NVIDIA-provided content have been part of the curriculum at 300 universities worldwide
  • Drafts of sections of the book have been tested and taught by Kirk at the University of Illinois
  • Book utilizes OpenCL™ and CUDA™ C, the NVIDIA® parallel computing language developed specifically for massively parallel environments

For more information:

GPU Computing Gems Book

Another related textbook is the Wen-mei Hwu edited:

  • “GPU Computing Gems”

Graphics Processing Units (GPUs) are designed to be parallel – having hundreds of cores versus traditional CPUs. Increasingly, you can leverage GPU power for many computationally-intense applications, not just for graphics. If you’re facing the challenge of programming systems to effectively use these massively parallel processors to achieve efficiency and performance goals, GPU Computing Gems provides a wealth of tested, proven GPU techniques.

GPU Computing Gems: Emerald Edition is the first volume in this new series from Morgan Kaufmann. Different application domains often pose similar algorithm problems, and researchers from diverse application domains often develop similar algorithmic strategies.

Each edition of GPU Computing Gems covers a carefully-selected subset of domains representing the latest GPU techniques, offering you a window into these diverse application areas, and the opportunity to gain insights from others’ algorithm work you may apply to your own projects.


  • A snapshot of the state of GPU computing in ten critical domains, edited by Wen-mei W. Hwu with experts from NVIDIA and instructors from leading GPU programs worldwide
  • Examples are written to be accessible across industries, and offer insights and ideas as well as practical “hands-on” skills you can immediately put to use

For more information:

textbook.txt · Last modified: 2011/06/08 14:19 by vgarcia