High Performance Scientific Computing: an independent textbook rewrite
NPTEL, IIT Bombay · Prof. Shivasubramanian Gopalakrishnan and course team · January 2025 · 31 chapters from 75 recordings · 11 chapter pages.
This reading edition is partial. Chapters 1-11 cover the first 20 recordings (Week 0 to Week 4, Lecture 1) and have recorded independent reviews; chapters 12-31 are not yet written. Final independent sign-offs remain pending for chapters 3, 9, 10 and 11: their last review findings were repaired and checked by the lead, but no later independent zero-blocking verdict is recorded.
Learning route and prerequisites
Start with C programming, elementary linear algebra and calculus, and a working Linux shell (native Linux or WSL 2 on Windows). Chapter 9 uses finite differences for the Laplace equation. The chapter programs are C, Python 3 and Bash; they need gcc with OpenMP support, and the Python programs of chapters 1, 4 and 9 also need NumPy.
- Purpose and the Linux environment, chapters 1–5: what scientific computing needs from hardware, the operating system and kernel, the file system and its permissions, processes, text tools, memory and networking commands, and shell scripts.
- Hardware, memory and concurrency, chapters 6–8: the components of an HPC system and a compute node, the memory hierarchy and scalable architectures, and races, locks, atomic instructions and barriers.
- Why parallel, and what limits it, chapters 9–10: the cost of a numerical model (the serial Jacobi solver used as the running example), speedup laws, memory bottlenecks, the organization of large systems and the Grace Hopper superchip.
- Shared-memory programming, chapter 11: OpenMP threads, parallel regions and the distribution of work.
- Not yet written, chapters 12–31: OpenMP II–III, MPI, domain decomposition, GPU architecture and CUDA, OpenACC, multi-device programs, profiling and debugging, numerical libraries, batch schedulers and visualization, and heterogeneous computing.
Read the chapters in order, then work through the expandable solutions and reproduce the chapter programs. The source map separates lecture material, research extensions, and original teaching constructions.
This navigation guide is an editorial aid. Final scientific review status is recorded separately in the editorial record.
31 chapters shown
01. What high performance scientific computing is for
Reviewed · locally validated02. Operating systems and the Linux kernel
Reviewed · locally validated03. The Linux file system: names, inodes, permission bits and ownership
Written · final review pending04. Processes, text processing, memory and networking from the command line
Reviewed · locally validated05. Shell scripting: processes, input, and control flow
Reviewed · locally validated06. The components of an HPC system and the architecture of a compute node
Reviewed · locally validated07. Hierarchical memory and scalable architectures
Reviewed · locally validated08. Concurrency and synchronization: races, locks, atomic instructions and barriers
Reviewed · locally validated09. Why parallel computing: numerical models and their cost
Written · final review pending10. Memory bottlenecks, the organization of large HPC systems and the Grace Hopper superchip
Written · final review pending11. OpenMP I: shared-memory threads and the distribution of work
Written · final review pending12. OpenMP II: synchronization, parallel loops, and data-sharing attributes
Chapter pending13. OpenMP III: memory model, tasks, and parallelizing example codes
Chapter pending14. MPI I: processes and point-to-point messages
Chapter pending15. MPI II: collective communication
Chapter pending16. Domain decomposition, an MPI solver, and performance analysis
Chapter pending17. GPU computing: motivation, software stack, NVIDIA and Intel architectures
Chapter pending18. GPU throughput arithmetic: FLOPS, bandwidth, SIMD and SIMT
Chapter pending19. The thread hierarchy: grids, blocks, warps, and warp scheduling
Chapter pending20. CUDA programming: tools, kernels, and launch configuration
Chapter pending21. GPU memory types and memory management
Chapter pending22. Thread divergence and memory layouts
Chapter pending23. CUDA compilation and execution
Chapter pending24. OpenACC I: directives, parallel loops, and data management
Chapter pending25. OpenACC II: loop optimization, asynchrony, and multiple devices
Chapter pending26. Multi-device programs: profiling-driven OpenACC, MPI with CUDA, MPI with OpenACC
Chapter pending27. Profiling and debugging serial codes: gprof, Valgrind, GDB
Chapter pending28. Performance analysis and debugging of MPI programs
Chapter pending29. Numerical methods and scientific libraries
Chapter pending30. Batch schedulers and scientific visualization
Chapter pending31. Heterogeneous computing, processor trends, and the road ahead (with revision)
Chapter pending