342.295, KV Parallel Computing, Armin Biere / Wolfgang Schreiner, 2021S

Topic outline

• Select topic Parallel Computing (SS 2021)

  Parallel Computing (SS 2021)

  Collapse all Expand all

  342.295, Time: Tuesday, 8:30-11:00, Start: March 9, 2021
  

  This course introduces basic techniques and models for parallel computing. We will discuss the basic theory, algorithms, programming models, and design principles for building parallel programs on multi-core processors, shared memory multiprocessors, and distributed memory multiprocessors respectively cluster systems. Class presentation will be accompanied by hands-on experience, e.g. on an SGI UV-1000 distributed shared memory multiprocessor.
  

  Background in programming in C/C++ and Java is assumed. Course grades will be based on practical programming assignments.
  

  The course will be given via Zoom meetings respectively video recordings, see the links below.
  

  We have to use two Zoom meetings depending on the lecturer (you can not host two meetings in parallel).

  • Select activity Zoom Meetings [Wolfgang Schreiner] https://jku.zo...

    Zoom Meetings [Wolfgang Schreiner]
    https://jku.zoom.us/j/821203635?pwd=NExrVElZOTUzcWxLeWR1SVVDc0RjZz09
    Meeting-ID: 821 203 635 Password: parallel
    

  • Select activity Zoom Meetings [Armin Biere]https://jku.zoom.us/j/9...

    Zoom Meetings [Armin Biere]
    https://jku.zoom.us/j/92642397616?pwd=Uy80czZ2SU1QT1lvWUVKWmdLcjMvQT09
    Meeting-ID: 926 4239 7616 Password: parallel

• Select topic Topics and Schedule

  Topics and Schedule

  The (preliminary) course schedule is as follows:

  1. March 9 (Schreiner): Parallel Architectures and Performance.
  2. March 16 (Biere): Parallel Algorithms and Complexity.
  3. March 23 (Schreiner): Concurrency in Java and OpenMP.
  4. April 13 (Biere): Shared Memory Programming, Lockless Data Structures.
  5. April 20 (Biere): Shared Memory Programming with Cilk, Lace, Work Stealing
  6. April 27 (Schreiner): Presentations of Solutions 1.
     
  7. May 11 (Schreiner): Message Passing Programming with MPI.
  8. May 18 (Biere):  Shared Memory Programming with PThreads.
     
  9. June 1 (Biere): Presentation of Solutions 2.
  10. June 8  (Schreiner): Designing Parallel Programs.
      
  11. June 15 (Schreiner): Presentations of Solutions 3.
  12. June 29 (Biere): Presentation of Solutions 4.
      

  Individual meetings may be organized for discussing the assignments.
• Select topic Course Materials

  Course Materials
  • Select activity Introduction and Organization

    

    Introduction and Organization File
  • Select activity Video Lecture March 9, 2021

    

    Video Lecture March 9, 2021 URL
  • Select activity Parallel Architectures

    

    Parallel Architectures File
  • Select activity Technical Advances in the SGI® UV Architecture

    

    Technical Advances in the SGI® UV Architecture File
  • Select activity Performance and Productivity Breakthroughs with Very Large Coherent Shared Memory: The SGI® UV Architecture

    

    Performance and Productivity Breakthroughs with Very Large Coherent Shared Memory: The SGI® UV Architecture File
  • Select activity SGI UV Data Sheet

    

    SGI UV Data Sheet File
  • Select activity JKU Data Sheet

    

    JKU Data Sheet File
  • Select activity How to Use the Parallel Computing Server

    

    How to Use the Parallel Computing Server Page
  • Select activity How to Use the Server From Other Networks (Restricted)

    

    How to Use the Server From Other Networks (Restricted) URL
  • Select activity Some More Information (in German)

    

    Some More Information (in German) Page
  • Select activity The Supercomputer "MACH-2"

    

    The Supercomputer "MACH-2" URL
  • Select activity Performance Analysis

    

    Performance Analysis File
  • Select activity Video "Performance Analysis" (Lecture March 9)

    

    Video "Performance Analysis" (Lecture March 9) URL
  • Select activity Algorithms and Complexity

    

    Algorithms and Complexity File

    Zoom recording of first and second part:
    https://jku.zoom.us/rec/share/ISbN2huO6WkPnCH2uf1fGmtXABVy8xOOMx5QXIm2F6PVnbNNnc3G5SYWnyJKZOGB.4j93u-i6ci2LebVp
    Passcode: 8+S$gjU+
    
    The Zoom Video has actually two parts (called 'recordings').  The first is about the motivational slides.
    Press the  "|>" button to go to the more technical part (which starts with the discussion of 'work' and 'span').

    The third part had to be recorded locally and is available on YouTube:
    https://youtu.be/ZLoYUKSl5b4

  • Select activity Concurrency in Java

    

    Concurrency in Java File
  • Select activity Video "Concurrency in Java" (Lecture March 23)

    

    Video "Concurrency in Java" (Lecture March 23) URL
  • Select activity Sources

    

    Sources File
  • Select activity Lesson: Concurrency

    

    Lesson: Concurrency URL
  • Select activity Trail: Custom Networking

    

    Trail: Custom Networking URL
  • Select activity OpenMP

    

    OpenMP File
  • Select activity Video "OpenMP" (Lecture March 23)

    

    Video "OpenMP" (Lecture March 23) URL
  • Select activity Sources

    

    Sources File
  • Select activity OpenMP

    

    OpenMP URL
  • Select activity OpenMP Tutorial

    

    OpenMP Tutorial URL
  • Select activity Intel OpenMP* Support

    

    Intel OpenMP* Support URL
  • Select activity OpenMP* Pragmas Summary

    

    OpenMP* Pragmas Summary URL
  • Select activity Worksharing Using OpenMP*

    

    Worksharing Using OpenMP* URL
  • Select activity Thread Affinity Interface

    

    Thread Affinity Interface URL
  • Select activity GNU libgomp

    

    GNU libgomp URL
  • Select activity Enabling OpenMP

    

    Enabling OpenMP URL
  • Select activity GOMP_CPU_AFFINITY

    

    GOMP_CPU_AFFINITY URL
  • Select activity Shared Memory Programming, Lockless Data Structures

    

    Shared Memory Programming, Lockless Data Structures File

    Zoom Recording 13. April 2021 (Intro, Sequential Consistency, TSO Example)
    https://jku.zoom.us/rec/share/fpHGmNro4tmfjn4ahJK101eZRayb-Gceo7-cvHfY9RYkVJeqyub9ni23VtrFe4SO.K8SW-Q4IOt0mS_q-
    Passcode: 289^C?bZ

    Zoom Recording 20. April 2021 (Linearizability, Wait-Free, Lock-Free, CAS, Treiber Stack, ABA, Tagged Pointers)
    https://jku.zoom.us/rec/share/fpHGmNro4tmfjn4ahJK101eZRayb-Gceo7-cvHfY9RYkVJeqyub9ni23VtrFe4SO.K8SW-Q4IOt0mS_q-
    Passcode: 6pa*7o$X
    

    ABA Problem Black Board Drawing

  • Select activity Shared Memory Examples

    

    Shared Memory Examples File
  • Select activity Video "Data Race Example" (Lecture April 13)

    

    Video "Data Race Example" (Lecture April 13) URL
  • Select activity Video "Peterson Algorithm" (Lecture April 13)

    

    Video "Peterson Algorithm" (Lecture April 13) URL
  • Select activity PerfBook by Paul McKenney

    

    PerfBook by Paul McKenney File

    There might be a more up-to-date version available here:

    https://mirrors.edge.kernel.org/pub/linux/kernel/people/paulmck/perfbook/perfbook.html
    

  • Select activity x86-TSO Paper (Store Buffer, Litmus Test, Lock)

    

    x86-TSO Paper (Store Buffer, Litmus Test, Lock) File
  • Select activity Work StealingZoom Recording 18. May 2021 (Task Par...

    Work Stealing
    

    Zoom Recording 18. May 2021 (Task Parallelism, Leaf Coarsening, Cilk, Lace, Dequeue)
    https://jku.zoom.us/rec/share/1EtanfwpRDD4QWu2bKgG5ol_PBMIVzGAzLOJgYmDRLbUoni9Daq8jemUdbGCpVIK.iVuOjuZxkpb-Eshf
    Passcode: n!*a8&^e

  • Select activity Cilk

    

    Cilk File
  • Select activity Cilk Examples

    

    Cilk Examples File
  • Select activity Lace

    

    Lace File
  • Select activity PThreads

    PThreads

  • Select activity PThreads Introduction

    

    PThreads Introduction File
  • Select activity PThreads Basics

    

    PThreads Basics File
  • Select activity PThreads Synchronization

    

    PThreads Synchronization File
  • Select activity PThreads Examples

    

    PThreads Examples File
  • Select activity MPI

    

    MPI File
  • Select activity Video "MPI" (Lecture May 11)

    

    Video "MPI" (Lecture May 11) URL
  • Select activity Sources

    

    Sources File
  • Select activity MPI Forum

    

    MPI Forum URL
  • Select activity MPI Tutorial

    

    MPI Tutorial URL
  • Select activity Parallel Program Design

    

    Parallel Program Design File
  • Select activity Video "Parallel Program Design" (Lecture June 8)

    

    Video "Parallel Program Design" (Lecture June 8) URL
  • Select activity Ian Foster: Designing and Building Parallel Programs

    

    Ian Foster: Designing and Building Parallel Programs URL
• Select topic Assignments

  Assignments

  We will hand out four assignments that have to be worked out individually. At least three assignments have to be turned in and graded positively to pass the course. Submitters may be asked for presentations of their solutions; be prepared to give at least one such presentation.
  

  • Select activity Assignment 1 (April 20)

    

    Assignment 1 (April 20)
  • Select activity Assignment 2 (May 25)

    

    Assignment 2 (May 25)
  • Select activity Solution 2 (proposed)

    

    Solution 2 (proposed) File
  • Select activity Assignment 3 (June 8)

    

    Assignment 3 (June 8)
  • Select activity Assignment 4 (June 22)

    

    Assignment 4 (June 22)
  • Select activity Solution 4 (proposed)

    

    Solution 4 (proposed) File