How to Use these ResourcesThe resources below offer tutorials and reference information on MPI, its different uses and applications, and distributed-memory parallelism, from beginner to advanced levels. Almost all the resources presume some reasonable familiarity with a compiled language like C, C++, or Fortran. VideosIntroducing MPI (2011) -- short video (~11 mins) giving a broad-strokes overview of what MPI is and why it's used. From Software Carpenty.MPI Quiz -- video (~1 hr) accompanying an interactive MPI Quiz (enter "HPCQUIZ" in all caps as the 'Room Name'). Teaches MPI concepts in Q&A format followed by discussion. Part of the Virtual Tutorial series from ARCHER (the UK National Supercomputing Service)Message-Passing Programming with MPI (2019) -- a video playlist of four recorded sessions (~1.5 - 2 hrs each) from a 2019 online MPI course from ARCHER Advanced MPI Capabilities (2014) -- video recordings along with slides of a three-day workshop (Day 1, Day 2, Day 3 ~2 hrs each) on advanced MPI from the Virtual School of Computational Science and Engineering (VSCSE).MPI for Scalable Computing (2018) -- video recording of a ~5 hr workshop, with slides. Assumes some prior MPI exposure but not a great deal of advanced experience. Focuses more on concepts than coding but does have some hands-on examples. Delivered at Argonne National Lab (ANL) during the Argonne Training Program on Extreme-Scale Computing (ATPESC) 2018. Self-paced online coursesMPI Workshop -- video playlist with recordings of a two-day 2019 XSEDE MPI workshop co-sponsored by the Pittsburgh Supercomputing Center (joint partnership between Carnegie Mellon University and the University of Pittsburgh). Slides and exercises are included, along with a review of the exercises. Includes some coverage of parallel debugging and profiling tools and best practices.Training Course: Message-Passing Programming with MPI (2014) -- a 2014 version of the MPI course from ARCHER more suitable to use as a self-paced tutorial. Videos in the linked playlist are more atomic, and there are accompanying materials to use for reference and exercises.Introduction to MPI -- a self-paced online course from Cyberinfrastructure Tutor at the National Center for Supercomputing Applications (NCSA)Intermediate to MPI -- another self-paced online course from Cyberinfrastructure TutorCollection of MPI Virtual Courses -- the Cornell University Center for Advanced Computing (CAC) offers a sequence of five extremely granular self-paced virtual courses that walk you through the different aspects of MPI:Message Passing InterfaceMPI Point-to-PointMPI Collective CommunicationsMPI Advanced TopicsMPI One-sided Communication Web pages / written online tutorialsMessage Passing with MPI -- video recordings (Part 1 and Part 2, each ~90 mins) of a lecture from the Parallel Programming in Computational Engineering and Science (PPCES) 2014, an HPC workshop held at RWTH Aachen in Germany. Slides are included. The video lecture is followed by a hands-on lab session. There once existed a tarball of files for the hands-on exercises along with instructions for how to use them (note that some references in the exercises are specific to the Aachen computing system used at the workshop, but there should be fairly evident substitutions to make when working on your local laptop or on one of the Princeton systems).MPI Tutorials -- a gateway to other online tutorials from mpitutorial.comMPI Tutorial -- written tutorials with exercises from Lawrence Livermore National Lab (LLNL).Using MPI with C -- introductory level tutorial with examples (in C) on different aspects of MPI. From University of Colorado Boulder Research Computing (CURC).Using MPI with Fortran -- same as above, but with examples in FortranIntro MPI Workshop (2013) -- from Harvard Research Computing. There are two versions of the slides from two different instructors (version 1 and version 2) along with a suite of materials and sample code for exercises. Suitable to follow as a self-paced tutorial. BooksBooks on MPI -- another portal listing favorite published MPI references of the authors of mpitutorial.comParallel Programming In MPI and Open MP -- book by Victor Eijkhout from the Texas Advanced Computing Center (TACC). Available in online (the online version is titled "Parallel Programming for Science & Engineering") and print versions.Effective Computation in Physics: Field Guide to Research with Python -- from O'Reilly media, and applicable to almost any computational discipline, not just physics. Online version (linked) is part of the O'Reilly's Safari Tech Books Online (free usage available through the Princeton University Library with an active NetID). Print version also available. Chapter 12 touches on parallel programming in Python in general, with a section focusing on Python interfaces to MPI.
