PD_ParallelDecomposition PD_ParallelArchitecture PD_ParallelPerformance TCPP_Architecture TCPP_Programming CS2 DSA Systems visual movement

Oklahoma Supercomputing Center for Educaton and Research (OSCER)

“Supercomputing in Plain English” - “Distributed Multiprocessing”, Slides 19-25.

The following questions from (Neeman2006) can be used to stimulate discussion:

The point is that someone very clever (the instructor) can plan it out ahead of time that everyone is running the same program on different units of data. Together therefore, they can solve a problem much larger (and faster) than any single person can solve.


Activity: It is possible to act this out.

GPUs: The analogy can also be extended to describe processes on GPUs. To act out this analogy, have a collection of chairs organized in a row (no desks), to represent an “archipelago” of smaller islands. Have a person sit in each chair. Each person in a chair represents a GPU core. Next to the row of chairs is a person at a desk (representing the much larger CPU).

This can conclude with a discussion of Flynn’s taxonomy, where we talk about MIMD (distributed, multicore) architectures, and SIMD (GPU) architectures.

CS2013 Knowledge Unit Coverage

Parallel Decomposition (Core Tier 2)

5. Parallelize an algorithm by applying data-parallel decomposition. [Usage]

Parallel Architecture (Core Tier 1, Core Tier 2, Elective)

1. Core Tier 1: Explain the differences between shared and distributed memory. [Familiarity]

3. Core Tier 2: Characterize the kinds of tasks that are a natural match for SIMD machines. [Familiarity]

4. Elective: Describe the advantages and limitations of GPUs vs. CPUs. Familiarity]

5. Elective: Explain the features of each classification in Flynn’s taxonomy. [Familiarity]

8. Elective: Describe the key performance challenges in different memory and distributed system topologies. [Familiarity]

Parallel Performance (Elective)

3. Describe how data distribution/layout can affect an algorithm’s communication costs. [Familiarity]

TCPP Topics Coverage

Architecture Topics

Programming Topics



Unknown. (Neeman2006) describes the different analogies. There is no assessment provided in (Neeman2006) or (Neeman2008).