GPU Usage

GridUnesp has a dedicated node with high-performance GPUs for accelerating scientific applications, especially in areas such as machine learning, molecular simulations, and image processing.

GPU Infrastructure

GPU node: gpunode001

GPU node specifications

Component

Specification

CPU

AMD EPYC 9224 (48 cores, 96 threads)

RAM

1.5 TB

GPUs

4 × NVIDIA L40S

Memory per GPU

48 GB GDDR6

Total GPU memory

192 GB

Architecture

Ada Lovelace

CUDA Cores

18,176 per GPU

Tensor Cores

568 per GPU (4th generation)

GPU Partition

Jobs that use a GPU must be submitted to the special gpu partition:

#SBATCH --partition=gpu
#SBATCH --gres=gpu:N    # N = number of GPUs desired (1-4)

Limits of the gpu partition:

  • Maximum time: 24 hours

  • GPUs per job: 1 to 4

  • Memory: Up to 1.5 TB (shared with CPU)

Basic GPU Script

simple_gpu_job.sh
#!/bin/bash
#SBATCH -J gpu_test
#SBATCH --partition=gpu
#SBATCH --nodes=1
#SBATCH --ntasks=1
#SBATCH --gres=gpu:1          # Requests 1 GPU
#SBATCH --cpus-per-task=8     # 8 CPUs to feed the GPU
#SBATCH --time=06:00:00
#SBATCH --mem=32G
#SBATCH --output=gpu_%j.out

export INPUT="input_data/"
export OUTPUT="gpu_results/"

# Load required modules
module load cuda/12.9

job-nanny ./cuda_program

Example with PyTorch

Creating the Conda environment

# Load the Conda module
module load miniconda/24.4.0-libmamba

# Create an environment with Python and PyTorch
conda create -n pytorch_env python=3.9
source activate pytorch_env

# Install PyTorch with CUDA support
conda install pytorch torchvision torchaudio pytorch-cuda=11.8 -c pytorch -c nvidia

Example Python code

training.py
import torch
import torch.nn as nn
import torch.optim as optim
from torchvision import datasets, transforms
from torch.utils.data import DataLoader
import time

# Check available GPUs
device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
num_gpus = torch.cuda.device_count()
print(f"Available GPUs: {num_gpus}")
print(f"Using device: {device}")

# Define a simple model
class SimpleNN(nn.Module):
    def __init__(self):
        super(SimpleNN, self).__init__()
        self.fc1 = nn.Linear(28*28, 512)
        self.fc2 = nn.Linear(512, 256)
        self.fc3 = nn.Linear(256, 10)
        self.relu = nn.ReLU()

    def forward(self, x):
        x = x.view(-1, 28*28)
        x = self.relu(self.fc1(x))
        x = self.relu(self.fc2(x))
        x = self.fc3(x)
        return x

# Prepare data
transform = transforms.Compose([
    transforms.ToTensor(),
    transforms.Normalize((0.1307,), (0.3081,))
])

train_dataset = datasets.MNIST('./data', train=True, download=True, transform=transform)
train_loader = DataLoader(train_dataset, batch_size=64, shuffle=True)

# Initialize model
model = SimpleNN().to(device)

# If there are multiple GPUs, use DataParallel
if num_gpus > 1:
    model = nn.DataParallel(model)

criterion = nn.CrossEntropyLoss()
optimizer = optim.Adam(model.parameters(), lr=0.001)

# Train
model.train()
start_time = time.time()

for epoch in range(5):
    running_loss = 0.0
    for batch_idx, (data, target) in enumerate(train_loader):
        data, target = data.to(device), target.to(device)

        optimizer.zero_grad()
        output = model(data)
        loss = criterion(output, target)
        loss.backward()
        optimizer.step()

        running_loss += loss.item()

        if batch_idx % 100 == 99:
            print(f'Epoch {epoch+1}, Batch {batch_idx+1}, Loss: {running_loss/100:.4f}')
            running_loss = 0.0

elapsed = time.time() - start_time
print(f"Training completed in {elapsed:.2f} seconds")

# Save model
torch.save(model.state_dict(), 'final_model.pth')

Submission script for PyTorch

job_pytorch.sh
#!/bin/bash
#SBATCH -J pytorch_gpu
#SBATCH --partition=gpu
#SBATCH --nodes=1
#SBATCH --ntasks=1
#SBATCH --gres=gpu:2          # Use 2 GPUs
#SBATCH --cpus-per-task=16    # 16 CPUs to feed the GPUs
#SBATCH --time=12:00:00
#SBATCH --mem=64G
#SBATCH --output=pytorch_%j.out

export INPUT="training.py"
export OUTPUT="final_model.pth data/"

module load miniconda/24.4.0-libmamba
source activate pytorch_env

job-nanny python training.py

Example with TensorFlow

Creating the environment

module load miniconda/24.4.0-libmamba
conda create -n tf_env python=3.9
source activate tf_env
pip install tensorflow[and-cuda]

Python code

tf_train.py
import tensorflow as tf
import time

# Check GPUs
gpus = tf.config.list_physical_devices('GPU')
print(f"Available GPUs: {len(gpus)}")

if gpus:
    for gpu in gpus:
        print(f"GPU: {gpu.name}")
    # Configure memory growth (optional)
    for gpu in gpus:
        tf.config.experimental.set_memory_growth(gpu, True)

# Load MNIST data
mnist = tf.keras.datasets.mnist
(x_train, y_train), (x_test, y_test) = mnist.load_data()
x_train, x_test = x_train / 255.0, x_test / 255.0

# Define model
model = tf.keras.models.Sequential([
    tf.keras.layers.Flatten(input_shape=(28, 28)),
    tf.keras.layers.Dense(512, activation='relu'),
    tf.keras.layers.Dense(256, activation='relu'),
    tf.keras.layers.Dense(10, activation='softmax')
])

# Compile
model.compile(optimizer='adam',
              loss='sparse_categorical_crossentropy',
              metrics=['accuracy'])

# Train
start_time = time.time()
history = model.fit(x_train, y_train, epochs=5, validation_data=(x_test, y_test))
elapsed = time.time() - start_time

print(f"Training completed in {elapsed:.2f} seconds")
model.save('final_tf_model.h5')

Submission script

job_tensorflow.sh
#!/bin/bash
#SBATCH -J tf_gpu
#SBATCH --partition=gpu
#SBATCH --nodes=1
#SBATCH --ntasks=1
#SBATCH --gres=gpu:1
#SBATCH --cpus-per-task=8
#SBATCH --time=08:00:00
#SBATCH --mem=32G

export INPUT="tf_train.py"
export OUTPUT="final_tf_model.h5"

module load miniconda/24.4.0-libmamba
source activate tf_env

job-nanny python tf_train.py

Example with CUDA (C/C++)

Simple CUDA code

vector_add.cu
#include <stdio.h>
#include <cuda_runtime.h>

// CUDA kernel
__global__ void add_vectors(float *a, float *b, float *c, int n) {
    int i = blockIdx.x * blockDim.x + threadIdx.x;
    if (i < n) {
        c[i] = a[i] + b[i];
    }
}

int main() {
    int n = 1000000;
    size_t size = n * sizeof(float);

    // Allocate memory on the host
    float *h_a = (float*)malloc(size);
    float *h_b = (float*)malloc(size);
    float *h_c = (float*)malloc(size);

    // Initialize data
    for (int i = 0; i < n; i++) {
        h_a[i] = i * 1.0f;
        h_b[i] = i * 2.0f;
    }

    // Allocate memory on the device
    float *d_a, *d_b, *d_c;
    cudaMalloc(&d_a, size);
    cudaMalloc(&d_b, size);
    cudaMalloc(&d_c, size);

    // Copy data to the device
    cudaMemcpy(d_a, h_a, size, cudaMemcpyHostToDevice);
    cudaMemcpy(d_b, h_b, size, cudaMemcpyHostToDevice);

    // Configure and launch the kernel
    int threads = 256;
    int blocks = (n + threads - 1) / threads;

    add_vectors<<<blocks, threads>>>(d_a, d_b, d_c, n);
    cudaDeviceSynchronize();

    // Copy the result back
    cudaMemcpy(h_c, d_c, size, cudaMemcpyDeviceToHost);

    // Check the result
    printf("c[0] = %f\n", h_c[0]);
    printf("c[%d] = %f\n", n-1, h_c[n-1]);

    // Free memory
    cudaFree(d_a);
    cudaFree(d_b);
    cudaFree(d_c);
    free(h_a);
    free(h_b);
    free(h_c);

    return 0;
}

Compilation and execution

# Submission script
cat > job_cuda.sh << 'EOF'
#!/bin/bash
#SBATCH -J cuda_test
#SBATCH --partition=gpu
#SBATCH --gres=gpu:1
#SBATCH --time=00:30:00

export INPUT="vector_add.cu"
export OUTPUT="*"

# Compile
module load cuda/12.9
nvcc -o vector_add vector_add.cu

job-nanny ./vector_add
EOF

sbatch job_cuda.sh

GPU Monitoring

During the job’s execution, you can monitor GPU usage:

# After the job starts, submit a new job
#!/bin/bash
#SBATCH -J check_gpu
#SBATCH --partition=gpu
#SBATCH --gres=gpu:1
#SBATCH --time=00:30:00

nvidia-smi

Example nvidia-smi output:

+-----------------------------------------------------------------------------+
| NVIDIA-SMI 525.60.11    Driver Version: 525.60.11    CUDA Version: 12.9     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  NVIDIA L40S          Off | 00000000:00:00.0 Off |                    0 |
| 30%   65°C    P0   250W / 300W|  10240MiB / 49152MiB |     85%      Default |
+-------------------------------+----------------------+----------------------+

Within the script, you can monitor:

# In the submission script
nvidia-smi --query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used --format=csv

GPU Optimization

  1. Appropriate batch size

    • Too small: underutilizes the GPU

    • Too large: may exceed memory

  2. Efficient data loading

    • Use DataLoader with an appropriate num_workers (see Example Python code)

    • Pre-process data in parallel with the CPU

  3. Minimized CPU-GPU communication

    • Transfer data in batches

    • Keep data on the GPU as much as possible

  4. Mixed precision

    # PyTorch
from torch.cuda.amp import autocast, GradScaler
scaler = GradScaler()

with autocast():
    output = model(data)
    loss = criterion(output, target)

Script with mixed precision:

job_mixed_precision.sh
#!/bin/bash
#SBATCH -J mixed_gpu
#SBATCH --partition=gpu
#SBATCH --gres=gpu:1
#SBATCH --cpus-per-task=8
#SBATCH --time=12:00:00

export INPUT="training_mixed.py"
export OUTPUT="mixed_model.pth"

module load miniconda/24.4.0-libmamba
source activate pytorch_env

# Configure for mixed precision
export CUDA_VISIBLE_DEVICES=0
export PYTORCH_CUDA_ALLOC_CONF=max_split_size_mb:128

job-nanny python training_mixed.py

Common Problems

  1. Out of memory (OOM)

    • Reduce batch size

    • Use gradient accumulation

    • Free memory with torch.cuda.empty_cache()

  2. Underutilized GPU

    • Increase batch size

    • Check for I/O bottlenecks

    • Increase the number of CPUs for data loading

  3. CUDA version error

    • Check PyTorch/TensorFlow compatibility with CUDA

    • Use compatible modules

module load cuda/12.4  # Compatible with PyTorch 2.4 or newer versions
module load cuda/12.9  # Compatible with PyTorch 2.7 or newer versions

See also