NVIDIA CUDA library which can be integrated with C/C++

The NVIDIA CUDA library is a parallel computing platform and programming model that allows software developers to use NVIDIA GPUs for general-purpose processing – an approach known as GPGPU. It provides a C/C++ API, including header files and libraries, enabling developers to write code that runs on CUDA-enabled GPUs, leveraging their massive parallel processing capabilities to accelerate computationally intensive tasks. This integration allows for relatively straightforward porting of existing C/C++ code to take advantage of GPU acceleration.

NVIDIA provides several CUDA libraries that can be integrated with C and C++ programs to accelerate computations on NVIDIA GPUs. These libraries are optimized for high-performance computing and span domains like linear algebra, deep learning, image processing, and more.

Key NVIDIA CUDA Libraries

Here’s a breakdown of commonly used CUDA libraries for various purposes:

1. CUDA Runtime API and Driver API

• Purpose: Core APIs for managing memory, kernels, and device synchronization.
• Integration: Provides foundational support for writing GPU programs in C/C++.
• Key Features:
  • Device management
  • Memory allocation (e.g., cudaMalloc, cudaMemcpy)
  • Kernel launches

Example:

#include <cuda_runtime.h>
#include <iostream>

__global__ void addKernel(int *a, int *b, int *c) {
    int idx = threadIdx.x;
    c[idx] = a[idx] + b[idx];
}

int main() {
    const int size = 256;
    int a[size], b[size], c[size];
    int *d_a, *d_b, *d_c;

    cudaMalloc((void **)&d_a, size * sizeof(int));
    cudaMalloc((void **)&d_b, size * sizeof(int));
    cudaMalloc((void **)&d_c, size * sizeof(int));

    cudaMemcpy(d_a, a, size * sizeof(int), cudaMemcpyHostToDevice);
    cudaMemcpy(d_b, b, size * sizeof(int), cudaMemcpyHostToDevice);

    addKernel<<<1, size>>>(d_a, d_b, d_c);

    cudaMemcpy(c, d_c, size * sizeof(int), cudaMemcpyDeviceToHost);

    cudaFree(d_a);
    cudaFree(d_b);
    cudaFree(d_c);

    std::cout << "Addition complete." << std::endl;
    return 0;
}

2. cuBLAS

• Purpose: GPU-accelerated implementation of BLAS (Basic Linear Algebra Subprograms).
• Features:
  • Vector and matrix operations like dot product, matrix multiplication, etc.
  • Highly optimized for GPUs.
• Header: #include <cublas_v2.h>

Example:

#include <cublas_v2.h>
#include <iostream>

int main() {
    cublasHandle_t handle;
    cublasCreate(&handle);

    float alpha = 1.0f, beta = 0.0f;
    float A[6] = {1, 2, 3, 4, 5, 6}; // 2x3 matrix
    float B[3] = {1, 2, 3};          // 3x1 vector
    float C[2] = {0};                // 2x1 result

    float *d_A, *d_B, *d_C;
    cudaMalloc((void **)&d_A, 6 * sizeof(float));
    cudaMalloc((void **)&d_B, 3 * sizeof(float));
    cudaMalloc((void **)&d_C, 2 * sizeof(float));

    cudaMemcpy(d_A, A, 6 * sizeof(float), cudaMemcpyHostToDevice);
    cudaMemcpy(d_B, B, 3 * sizeof(float), cudaMemcpyHostToDevice);

    cublasSgemv(handle, CUBLAS_OP_N, 2, 3, &alpha, d_A, 2, d_B, 1, &beta, d_C, 1);

    cudaMemcpy(C, d_C, 2 * sizeof(float), cudaMemcpyDeviceToHost);

    std::cout << "Result: " << C[0] << ", " << C[1] << std::endl;

    cudaFree(d_A);
    cudaFree(d_B);
    cudaFree(d_C);
    cublasDestroy(handle);

    return 0;
}

3. cuFFT

• Purpose: Fast Fourier Transform (FFT) on NVIDIA GPUs.
• Features:
  • Forward and inverse FFTs
  • Single and double precision
• Header: #include <cufft.h>

Example:

#include <cufft.h>
#include <iostream>

int main() {
    cufftHandle plan;
    cufftComplex *d_data;
    int n = 16;

    cudaMalloc((void **)&d_data, sizeof(cufftComplex) * n);

    cufftPlan1d(&plan, n, CUFFT_C2C, 1);
    cufftExecC2C(plan, d_data, d_data, CUFFT_FORWARD);

    cufftDestroy(plan);
    cudaFree(d_data);

    std::cout << "FFT complete." << std::endl;
    return 0;
}

4. cuDNN

• Purpose: GPU-accelerated deep learning primitives (used in frameworks like TensorFlow and PyTorch).
• Features:
  • Convolutions, activation functions, normalization, etc.
• Header: #include <cudnn.h>

Example: Visit NVIDIA's documentation for in-depth integration examples with neural network layers.

5. NPP (NVIDIA Performance Primitives)

• Purpose: Image and signal processing.
• Features:
  • Filters, resizing, morphology, color conversion.
• Header: #include <nppi.h>

6. Thrust

• Purpose: High-level parallel programming library with STL-like interfaces.
• Header: #include <thrust/sort.h>

Example:

#include <thrust/device_vector.h>
#include <thrust/sort.h>
#include <iostream>

int main() {
    thrust::device_vector<int> d_vec(4);
    d_vec[0] = 10;
    d_vec[1] = 3;
    d_vec[2] = 7;
    d_vec[3] = 1;

    thrust::sort(d_vec.begin(), d_vec.end());

    for (int i = 0; i < d_vec.size(); i++)
        std::cout << d_vec[i] << " ";
    return 0;
}

Getting Started

1. Install CUDA Toolkit: Download from the NVIDIA Developer site.
2. Set Up Environment: Ensure paths for nvcc compiler and libraries are correctly configured.
3. Link Libraries:
   • For compilation: Use flags like -lcublas, -lcufft, etc.
   • Example: nvcc -o example example.cu -lcublas

These libraries allow high-performance computing tasks to harness the power of NVIDIA GPUs directly in your C or C++ applications. Let me know if you'd like specific guidance!