HPC on Xi's Blog

HPC-3-use-openmp(shared-memory-method)

Tue, 16 Apr 2024 00:05:14 -0400

Introduction to HPC, shared memory parallel using openmp

1 The multicore system

The relationship with L1-L3 cache. The L3 cache is shared, but every core have its own L1-2 cache.

2 Using openmp

1

#include "omp.h"

Before using it, we need to define how many threads we want to use:

In Unix system:

1

export OMP_NUM_THREADS=4

The instruction:

1

#pragma omp parallel

If we put this macro before one line of code or one block, the line or block will be executed $OMP_NUM_TRHREADS times.

HPC-1-divide-and-conquer-block-matrix-algorithmr

Mon, 15 Apr 2024 00:05:14 -0400

week 2 block matrix algorithm

1. BLIS reference high performance implitation v.s. naive methods:

2. With different block size:

This is the MB NB PB = 40.

But if the block size is too small, the performance is not as good as naive PJI.

The front for loop is JIP is not related to the performance of the algorithm because the computer will focus on each implementation in blocking. That means the register will focus on optimize the final for loop: the Gemm_JPI function, but will not paralize and optimize the for loop for block - matrix- matrix - multiplication.

HPC-2-Memory-hierarchy-in-computer

Mon, 15 Apr 2024 00:05:14 -0400

Hierarchy Memory

1. Why use Hierarchy Memory

Because the register memory is much faster than main memory, in fact the difference is about two magnitude. And the performance gap will be larger because the CPU’s speed increase faster than main memory.

In this situation, if we fetch data from the main memory too many times, the expense will be very expensive. But if we create some memory which is faster than main memory but a little bit slower than register memory. We call it cache.