Fundamentals of Accelerated Computing with CUDA C/C++ Training Course

GPU Computing and CUDA Architecture

• Comparative analysis of CPU and GPU architectural distinctions
• The NVIDIA GPU streaming multiprocessor architecture
• Overview of the CUDA programming paradigm
• Heterogeneous computing and the host-device interaction model

Configuring the CUDA Development Environment

• Installation procedures for CUDA Toolkit 13.x
• Utilizing the NVCC compiler and establishing build workflows
• Environment validation through device querying
• Integration with IDEs and auxiliary development utilities

Development and Execution of CUDA Kernels

• Syntax and qualifiers for kernel functions
• Configuration and initiation of kernel execution
• Vector addition and fundamental data-parallel structures
• Implementation of CUDA error checking macros

CUDA Thread Hierarchy and Execution Dynamics

• Organization of grids, blocks, and threads
• Thread indexing and global identifier computation
• Warp execution mechanics and the SIMT model
• Occupancy metrics and resource efficiency

GPU Memory Architecture and Control

• Memory categories: global, shared, constant, and registers
• Processes for allocating and releasing device memory
• Data transfer mechanisms between host and device
• Utilizing shared memory for intra-block cooperation

Unified Memory and Data Movement

• The unified memory model and managed allocation strategies
• Page migration and on-demand paging mechanisms
• Asynchronous prefetching via cudaMemPrefetchAsync
• Memory advice hints for optimizing access patterns

System-Level Profiling with Nsight Systems

• Timeline analysis using Nsight Systems
• Detection of CPU-GPU synchronization points
• Visualization of kernel execution and memory operations
• Interpretation of system-wide performance metrics

Kernel Optimization via Nsight Compute

• Interactive kernel profiling with Nsight Compute
• Analysis of memory throughput and bandwidth
• Evaluation of compute utilization and warp state statistics
• Guided analysis and application of optimization rules

Concurrent Streams and Asynchronous Execution

• CUDA streams and the concept of the default stream
• Overlapping data transfers with kernel execution
• Stream synchronization techniques and CUDA events
• Design patterns for multi-stream pipelines

Error Management and Debugging Utilities

• CUDA API error codes and recovery protocols
• Memory access validation using compute-sanitizer
• Kernel debugging with cuda-gdb
• Assertions and synchronous error detection methods

Profile-Guided Optimization Strategy

• Methodology for iterative profiling
• Identification and prioritization of bottlenecks
• Performance regression testing procedures
• Documentation of optimization choices

End-to-End Accelerated Application Project

• Architecting a comprehensive GPU-accelerated solution
• Integrating profiling throughout the development lifecycle
• Performance benchmarking and result reporting
• Considerations for production deployment