.. _gpu_usage: ========= GPU Usage ========= .. contents:: In this section: :local: :depth: 2 GridUnesp has a dedicated node with high-performance GPUs for accelerating scientific applications, especially in areas such as machine learning, molecular simulations, and image processing. GPU Infrastructure ================== **GPU node:** ``gpunode001`` .. list-table:: GPU node specifications :header-rows: 1 :widths: 30 70 * - Component - Specification * - CPU - AMD EPYC 9224 (48 cores, 96 threads) * - RAM - 1.5 TB * - GPUs - 4 × NVIDIA L40S * - Memory per GPU - 48 GB GDDR6 * - Total GPU memory - 192 GB * - Architecture - Ada Lovelace * - CUDA Cores - 18,176 per GPU * - Tensor Cores - 568 per GPU (4th generation) GPU Partition ============= Jobs that use a GPU must be submitted to the special **gpu** partition: .. code-block:: bash #SBATCH --partition=gpu #SBATCH --gres=gpu:N # N = number of GPUs desired (1-4) **Limits of the gpu partition:** - **Maximum time:** 24 hours - **GPUs per job:** 1 to 4 - **Memory:** Up to 1.5 TB (shared with CPU) Basic GPU Script ================ .. code-block:: bash :caption: simple_gpu_job.sh #!/bin/bash #SBATCH -J gpu_test #SBATCH --partition=gpu #SBATCH --nodes=1 #SBATCH --ntasks=1 #SBATCH --gres=gpu:1 # Requests 1 GPU #SBATCH --cpus-per-task=8 # 8 CPUs to feed the GPU #SBATCH --time=06:00:00 #SBATCH --mem=32G #SBATCH --output=gpu_%j.out export INPUT="input_data/" export OUTPUT="gpu_results/" # Load required modules module load cuda/12.9 job-nanny ./cuda_program Example with PyTorch ==================== Creating the Conda environment ------------------------------ .. code-block:: bash # Load the Conda module module load miniconda/24.4.0-libmamba # Create an environment with Python and PyTorch conda create -n pytorch_env python=3.9 source activate pytorch_env # Install PyTorch with CUDA support conda install pytorch torchvision torchaudio pytorch-cuda=11.8 -c pytorch -c nvidia .. _example_python_code: Example Python code ------------------- .. code-block:: python :caption: training.py import torch import torch.nn as nn import torch.optim as optim from torchvision import datasets, transforms from torch.utils.data import DataLoader import time # Check available GPUs device = torch.device('cuda' if torch.cuda.is_available() else 'cpu') num_gpus = torch.cuda.device_count() print(f"Available GPUs: {num_gpus}") print(f"Using device: {device}") # Define a simple model class SimpleNN(nn.Module): def __init__(self): super(SimpleNN, self).__init__() self.fc1 = nn.Linear(28*28, 512) self.fc2 = nn.Linear(512, 256) self.fc3 = nn.Linear(256, 10) self.relu = nn.ReLU() def forward(self, x): x = x.view(-1, 28*28) x = self.relu(self.fc1(x)) x = self.relu(self.fc2(x)) x = self.fc3(x) return x # Prepare data transform = transforms.Compose([ transforms.ToTensor(), transforms.Normalize((0.1307,), (0.3081,)) ]) train_dataset = datasets.MNIST('./data', train=True, download=True, transform=transform) train_loader = DataLoader(train_dataset, batch_size=64, shuffle=True) # Initialize model model = SimpleNN().to(device) # If there are multiple GPUs, use DataParallel if num_gpus > 1: model = nn.DataParallel(model) criterion = nn.CrossEntropyLoss() optimizer = optim.Adam(model.parameters(), lr=0.001) # Train model.train() start_time = time.time() for epoch in range(5): running_loss = 0.0 for batch_idx, (data, target) in enumerate(train_loader): data, target = data.to(device), target.to(device) optimizer.zero_grad() output = model(data) loss = criterion(output, target) loss.backward() optimizer.step() running_loss += loss.item() if batch_idx % 100 == 99: print(f'Epoch {epoch+1}, Batch {batch_idx+1}, Loss: {running_loss/100:.4f}') running_loss = 0.0 elapsed = time.time() - start_time print(f"Training completed in {elapsed:.2f} seconds") # Save model torch.save(model.state_dict(), 'final_model.pth') Submission script for PyTorch ----------------------------- .. code-block:: bash :caption: job_pytorch.sh #!/bin/bash #SBATCH -J pytorch_gpu #SBATCH --partition=gpu #SBATCH --nodes=1 #SBATCH --ntasks=1 #SBATCH --gres=gpu:2 # Use 2 GPUs #SBATCH --cpus-per-task=16 # 16 CPUs to feed the GPUs #SBATCH --time=12:00:00 #SBATCH --mem=64G #SBATCH --output=pytorch_%j.out export INPUT="training.py" export OUTPUT="final_model.pth data/" module load miniconda/24.4.0-libmamba source activate pytorch_env job-nanny python training.py Example with TensorFlow ======================= Creating the environment ------------------------ .. code-block:: bash module load miniconda/24.4.0-libmamba conda create -n tf_env python=3.9 source activate tf_env pip install tensorflow[and-cuda] Python code ----------- .. code-block:: python :caption: tf_train.py import tensorflow as tf import time # Check GPUs gpus = tf.config.list_physical_devices('GPU') print(f"Available GPUs: {len(gpus)}") if gpus: for gpu in gpus: print(f"GPU: {gpu.name}") # Configure memory growth (optional) for gpu in gpus: tf.config.experimental.set_memory_growth(gpu, True) # Load MNIST data mnist = tf.keras.datasets.mnist (x_train, y_train), (x_test, y_test) = mnist.load_data() x_train, x_test = x_train / 255.0, x_test / 255.0 # Define model model = tf.keras.models.Sequential([ tf.keras.layers.Flatten(input_shape=(28, 28)), tf.keras.layers.Dense(512, activation='relu'), tf.keras.layers.Dense(256, activation='relu'), tf.keras.layers.Dense(10, activation='softmax') ]) # Compile model.compile(optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy']) # Train start_time = time.time() history = model.fit(x_train, y_train, epochs=5, validation_data=(x_test, y_test)) elapsed = time.time() - start_time print(f"Training completed in {elapsed:.2f} seconds") model.save('final_tf_model.h5') Submission script ----------------- .. code-block:: bash :caption: job_tensorflow.sh #!/bin/bash #SBATCH -J tf_gpu #SBATCH --partition=gpu #SBATCH --nodes=1 #SBATCH --ntasks=1 #SBATCH --gres=gpu:1 #SBATCH --cpus-per-task=8 #SBATCH --time=08:00:00 #SBATCH --mem=32G export INPUT="tf_train.py" export OUTPUT="final_tf_model.h5" module load miniconda/24.4.0-libmamba source activate tf_env job-nanny python tf_train.py Example with CUDA (C/C++) ========================= Simple CUDA code ---------------- .. code-block:: cuda :caption: vector_add.cu #include #include // CUDA kernel __global__ void add_vectors(float *a, float *b, float *c, int n) { int i = blockIdx.x * blockDim.x + threadIdx.x; if (i < n) { c[i] = a[i] + b[i]; } } int main() { int n = 1000000; size_t size = n * sizeof(float); // Allocate memory on the host float *h_a = (float*)malloc(size); float *h_b = (float*)malloc(size); float *h_c = (float*)malloc(size); // Initialize data for (int i = 0; i < n; i++) { h_a[i] = i * 1.0f; h_b[i] = i * 2.0f; } // Allocate memory on the device float *d_a, *d_b, *d_c; cudaMalloc(&d_a, size); cudaMalloc(&d_b, size); cudaMalloc(&d_c, size); // Copy data to the device cudaMemcpy(d_a, h_a, size, cudaMemcpyHostToDevice); cudaMemcpy(d_b, h_b, size, cudaMemcpyHostToDevice); // Configure and launch the kernel int threads = 256; int blocks = (n + threads - 1) / threads; add_vectors<<>>(d_a, d_b, d_c, n); cudaDeviceSynchronize(); // Copy the result back cudaMemcpy(h_c, d_c, size, cudaMemcpyDeviceToHost); // Check the result printf("c[0] = %f\n", h_c[0]); printf("c[%d] = %f\n", n-1, h_c[n-1]); // Free memory cudaFree(d_a); cudaFree(d_b); cudaFree(d_c); free(h_a); free(h_b); free(h_c); return 0; } Compilation and execution ------------------------- .. code-block:: bash # Submission script cat > job_cuda.sh << 'EOF' #!/bin/bash #SBATCH -J cuda_test #SBATCH --partition=gpu #SBATCH --gres=gpu:1 #SBATCH --time=00:30:00 export INPUT="vector_add.cu" export OUTPUT="*" # Compile module load cuda/12.9 nvcc -o vector_add vector_add.cu job-nanny ./vector_add EOF sbatch job_cuda.sh GPU Monitoring ============== During the job's execution, you can monitor GPU usage: .. code-block:: bash # After the job starts, submit a new job #!/bin/bash #SBATCH -J check_gpu #SBATCH --partition=gpu #SBATCH --gres=gpu:1 #SBATCH --time=00:30:00 nvidia-smi **Example nvidia-smi output:** .. code-block:: text +-----------------------------------------------------------------------------+ | NVIDIA-SMI 525.60.11 Driver Version: 525.60.11 CUDA Version: 12.9 | |-------------------------------+----------------------+----------------------+ | GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC | | Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. | | | | MIG M. | |===============================+======================+======================| | 0 NVIDIA L40S Off | 00000000:00:00.0 Off | 0 | | 30% 65°C P0 250W / 300W| 10240MiB / 49152MiB | 85% Default | +-------------------------------+----------------------+----------------------+ **Within the script, you can monitor:** .. code-block:: bash # In the submission script nvidia-smi --query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used --format=csv GPU Optimization ================ 1. **Appropriate batch size** - Too small: underutilizes the GPU - Too large: may exceed memory 2. **Efficient data loading** - Use ``DataLoader`` with an appropriate ``num_workers`` (see :ref:`example_python_code`) - Pre-process data in parallel with the CPU 3. **Minimized CPU-GPU communication** - Transfer data in batches - Keep data on the GPU as much as possible 4. **Mixed precision** .. code-block:: python # PyTorch from torch.cuda.amp import autocast, GradScaler scaler = GradScaler() with autocast(): output = model(data) loss = criterion(output, target) **Script with mixed precision:** .. code-block:: bash :caption: job_mixed_precision.sh #!/bin/bash #SBATCH -J mixed_gpu #SBATCH --partition=gpu #SBATCH --gres=gpu:1 #SBATCH --cpus-per-task=8 #SBATCH --time=12:00:00 export INPUT="training_mixed.py" export OUTPUT="mixed_model.pth" module load miniconda/24.4.0-libmamba source activate pytorch_env # Configure for mixed precision export CUDA_VISIBLE_DEVICES=0 export PYTORCH_CUDA_ALLOC_CONF=max_split_size_mb:128 job-nanny python training_mixed.py Common Problems =============== 1. **Out of memory (OOM)** - Reduce batch size - Use gradient accumulation - Free memory with ``torch.cuda.empty_cache()`` 2. **Underutilized GPU** - Increase batch size - Check for I/O bottlenecks - Increase the number of CPUs for data loading 3. **CUDA version error** - Check PyTorch/TensorFlow compatibility with CUDA - Use compatible modules .. code-block:: bash module load cuda/12.4 # Compatible with PyTorch 2.4 or newer versions module load cuda/12.9 # Compatible with PyTorch 2.7 or newer versions .. seealso:: - :ref:`installing_applications` - Installing packages for GPU - :ref:`containers` - Using containers with GPU - :ref:`running_simulations` - Basic submission concepts - :ref:`optimizing_performance` - General optimization