.. _shared_memory: ============= Shared Memory ============= .. contents:: In this section: :local: :depth: 2 Shared memory is a parallelism model in which multiple threads (lines of execution) share the same memory space within a single node (server). Basic Concepts ============== - **Threads:** Multiple lines of execution within the same process - **Shared memory:** All threads see the same memory - **OpenMP:** The most common standard for shared-memory programming - **SMP (Symmetric Multi-Processing):** Architecture with multiple processors sharing memory **Characteristics:** - All cores used are on the **same node** - Access to the same memory area - Communication between threads is very fast - Limited to the resources of a single node OpenMP ====== OpenMP is an API for parallel programming in C/C++ and Fortran, based on compiler directives. Example in C ------------ .. code-block:: c :caption: omp_hello.c /****************************************************************************** * OpenMP Example - Hello World * The master thread creates a parallel region. * All threads obtain their unique number and print. ******************************************************************************/ #include #include #include int main (int argc, char *argv[]) { int nthreads, tid; /* Fork: create a team of threads */ #pragma omp parallel private(nthreads, tid) { /* Get the thread number */ tid = omp_get_thread_num(); printf("Hello World from thread = %d\n", tid); /* Only the master thread does this */ if (tid == 0) { nthreads = omp_get_num_threads(); printf("Number of threads = %d\n", nthreads); } } /* Join: all threads meet here */ return 0; } Compiling --------- On GridUnesp, we recommend the Intel compiler, but GCC is also available: .. code-block:: bash # With Intel module load intel/compilers icc -openmp omp_hello.c -o omp_hello # With GCC module load gcc/10.2.0 gcc -fopenmp omp_hello.c -o omp_hello Submission Script ================= In SLURM, the number of threads is controlled by the ``-c`` (``--cpus-per-task``) directive. .. code-block:: bash :caption: job_openmp.sh #!/bin/bash #SBATCH -J openmp_test #SBATCH -N 1 # 1 node (required for shared memory) #SBATCH -c 8 # 8 threads #SBATCH -t 01:00:00 #SBATCH --mem=16G export INPUT="input.dat" export OUTPUT="output_openmp.dat" # Set the number of threads for OpenMP export OMP_NUM_THREADS=$SLURM_CPUS_PER_TASK # Load the compiler module module load intel/compilers job-nanny srun ./omp_hello Important Variables =================== - **OMP_NUM_THREADS:** Number of threads to be used - **OMP_STACKSIZE:** Stack size per thread - **OMP_SCHEDULE:** Type of loop scheduling .. code-block:: bash export OMP_NUM_THREADS=28 export OMP_STACKSIZE=256M export OMP_SCHEDULE="dynamic" Limitations =========== - Limited to **1 node** (cannot use multiple servers) - Maximum threads per node: **56** (28 physical cores + hyper-threading) - Maximum available threads per node: **52** (26 physical cores + hyper-threading) - Total memory limited to the node's RAM: **128 GB** .. list-table:: Cluster Resource Allocation (Core Specialization) :widths: 34 22 22 22 :align: center * - Node Type - Total Capacity - Reserved for OS - Available for Jobs * - **Regular Nodes** (``node[001-056]``) - 28 Cores / 56 CPUs - 2 Cores / 4 CPUs - 26 Cores / 52 CPUs * - **GPU Node** (``gpunode001``) - 48 Cores / 96 CPUs - 4 Cores / 8 CPUs - 44 Cores / 88 CPUs When to Use Shared Memory ========================= ✅ **Ideal for:** - Problems that fit in the memory of a single node - Heavy loops that can be parallelized - Applications with a lot of communication between threads - When network latency would be a problem ❌ **Not recommended for:** - Problems that do not fit in 128 GB of RAM - Applications that need more than 52 threads - When the problem is naturally distributed Practical Example: Matrix Product ================================= .. code-block:: c :caption: matmul_omp.c #include #include #include #define N 1000 int main() { double A[N][N], B[N][N], C[N][N]; double start, end; // Initialization #pragma omp parallel for for (int i = 0; i < N; i++) { for (int j = 0; j < N; j++) { A[i][j] = i + j; B[i][j] = i - j; C[i][j] = 0.0; } } start = omp_get_wtime(); // Parallel multiplication #pragma omp parallel for for (int i = 0; i < N; i++) { for (int k = 0; k < N; k++) { for (int j = 0; j < N; j++) { C[i][j] += A[i][k] * B[k][j]; } } } end = omp_get_wtime(); printf("Time with %d threads: %f seconds\n", omp_get_max_threads(), end - start); return 0; } Script for a scalability test: .. code-block:: bash :caption: test_omp_scaling.sh #!/bin/bash # Test with 1, 2, 4, 8, 16, 28, 52 threads for threads in 1 2 4 8 16 28 52; do cat > job_${threads}.sh << EOF #!/bin/bash #SBATCH -J omp_${threads} #SBATCH -N 1 #SBATCH -c ${threads} #SBATCH -t 00:30:00 #SBATCH --mem=16G #SBATCH --output=omp_${threads}_%j.out export OMP_NUM_THREADS=${threads} module load intel/compilers ./test_omp EOF sbatch job_${threads}.sh done Optimization Tips ================= 1. **Load balancing:** Use dynamic scheduling for irregular loops .. code-block:: c #pragma omp parallel for schedule(dynamic, 100) 2. **Avoid false sharing:** Place shared variables on different cache lines 3. **Use private variables:** Declare temporary variables as private .. code-block:: c #pragma omp parallel for private(temp) 4. **Nested parallelism:** Be careful when nesting parallel regions .. code-block:: bash export OMP_NESTED=FALSE .. seealso:: - `OpenMP `_ - Shared-memory programming - `SMP `_ - Multiple processors sharing memory - :ref:`distributed_memory` - For problems that do not fit on one node - :ref:`running_simulations` - Basic submission concepts - :ref:`optimizing_performance` - How to find the ideal configuration - :ref:`installed_applications` - Examples with real applications