Accelerated C++ Programming with CUDA Training Course

Foundations of GPU-Accelerated Computing

• Concepts of heterogeneous computing and CPU-GPU architectural differences.
• Understanding CUDA execution models and memory spaces.
• Compiling CUDA C++ code using nvcc and CMake.
• Validating the GPU development environment setup.

Parallel Algorithms with Thrust and CUB

• Implementing GPU-accelerated sorting, reduction, and transformation operations.
• Adapting STL algorithms for efficient GPU execution.
• Utilizing Thrust device vectors and execution policies.
• Applying CUB device-wide primitives for custom processing pipelines.

GPU Memory Architecture and Management

• Differentiating global, constant, and texture memory types.
• Managing explicit device memory allocation and data transfers.
• Using Unified Memory to streamline data access.
• Optimizing memory coalescing and access patterns.

Asynchronous Execution via CUDA Streams

• Establishing and managing CUDA stream workflows.
• Overlapping kernel execution with data transfer operations.
• Employing CUDA events for managing dependencies.
• Tuning stream priorities and concurrency.

Developing Custom CUDA Kernels

• The SIMT programming model and the concept of warp execution.
• Configuring kernel launches and implementing grid-stride loops.
• Managing thread indexing and multidimensional grids.
• Implementing robust error handling and CUDA runtime API validation.

Thread Hierarchy and Execution Model

• Structuring grids, blocks, and threads within device code.
• Utilizing warp-level primitives and ballot operations.
• Implementing block-level synchronization and barriers.
• Analyzing occupancy and resource utilization.

Cooperative Groups for Flexible Parallelism

• Working with the cooperative_groups API and various group types.
• Implementing thread-block tiles and tiled_partition.
• Executing grid-level cooperative launches.
• Managing multi-grid synchronization patterns.

Shared Memory Optimization Techniques

• Understanding shared memory banks and avoiding bank conflicts.
• Applying tiling strategies for matrix operations.
• Using shared memory as a user-managed cache.
• Utilizing cuda::shared_memory_mdspan for multidimensional data views.

Kernel Fusion and Advanced Parallel Patterns

• Merging multiple kernels to minimize launch overhead.
• Implementing scan, reduce-by-key, and segmented algorithms.
• Using atomic operations and lock-free data structures.
• Leveraging warp-aggregated atomics to improve throughput.

Profiling and Optimization with Nsight Systems

• Conducting timeline analysis of CPU and GPU activities.
• Identifying and resolving memory transfer bottlenecks.
• Profiling kernel performance and occupancy levels.
• Performing iterative optimization using Nsight Compute.

Modern C++ Features in CUDA Device Code

• Incorporating lambdas, constexpr, and auto within kernels.
• Using C++17 parallel algorithms and execution policies.
• Applying C++20 concepts and ranges on the device.
• Leveraging C++23 support in nvcc and CCCL 3.x.

CUDA Graphs and Advanced Asynchrony

• Defining and launching CUDA graph structures.
• Capturing graphs from stream execution flows.
• Implementing graph updates and conditional execution nodes.
• Minimizing launch latency for iterative workloads.

Integration Patterns for Existing Applications

• Encapsulating GPU code behind standard C++ interfaces.
• Managing multi-GPU and NUMA system architectures.
• Integrating build systems with CMake and CUDA.
• Debugging device code using cuda-gdb.

Summary and Best Practices

• Evaluating when to use Thrust, CUB, or custom kernels.
• Achieving performance portability across different GPU architectures.
• Organizing code and applying RAII principles for CUDA resources.
• Exploring next steps and advanced pathways in CUDA development.