Update benchmarks: add results for Archer2 up to 1024x1024x1024

Author: rfj82982

docs/source/pages/api_domain.rst

@@ -3,7 +3,9 @@
 ===========================
 
 This page explains the key public interfaces of the 2D decomposition library. After reading this section, users should be able to easily build applications using this domain decomposition strategy. 
-The library interface is designed to be very simple. One can refer to the sample applications for a quick start.
+The library interface is designed to be very simple. One can refer to the 
+`example applications <https://github.com/2decomp-fft/2decomp-fft/tree/main/examples>`_ 
+for a quick start.
 
 The 2D Pencil Decomposition API is defined in three Fortran module which should be used by applications as:
 

docs/source/pages/benchmarks.rst

@@ -1,3 +1,71 @@
+.. role:: raw-html(raw)
+    :format: html
+ 
+.. |br| raw:: html
+
+  <br/>
+
 ==========
 Benchmarks
 ==========
+
+This page report the result of some scalability tests which are available in the under 
+`example <https://github.com/2decomp-fft/2decomp-fft/tree/main/examples>`_
+The test being performed are the following: 
+
+#. `Transpose of a real <https://github.com/2decomp-fft/2decomp-fft/blob/main/examples/test2d/timing2d_real.f90>`_ 
+   3D array.
+
+#. `Transpose of a complex <https://github.com/2decomp-fft/2decomp-fft/blob/main/examples/test2d/timing2d_complex.f90>`_ 3D array.
+
+#. 3D FFT trasform 
+   `(fft_r2c_x) <https://github.com/2decomp-fft/2decomp-fft/blob/main/examples/fft_physical_x/fft_r2c_x.f90>`_ 
+   of a **real** 3D array starting from `X` physical direction. 
+   The trasform has both forward and backward to retrieve the inout array.
+
+#. 3D FFT trasform 
+   `(fft_c2c_x) <https://github.com/2decomp-fft/2decomp-fft/blob/main/examples/fft_physical_x/fft_c2c_x.f90>`_ 
+   of a **complex** 3D array starting from `X` physical direction. 
+   The trasform has both forward and backward to retrieve the inout array.
+
+#. 3D FFT trasform 
+   `(fft_r2c_z) <https://github.com/2decomp-fft/2decomp-fft/blob/main/examples/fft_physical_z/fft_r2c_z.f90>`_ 
+   of a **real** 3D array starting from `Z` physical direction. 
+   The trasform has both forward and backward to retrieve the inout array.
+
+#. 3D_FFT_trasform 
+   `(fft_c2c_z) <https://github.com/2decomp-fft/2decomp-fft/blob/main/examples/fft_physical_z/fft_c2c_z.f90>`_ 
+   of a **complex** 3D array starting from `Z` physical direction. 
+   The trasform has both forward and backward to retrieve the inout array.
+
+All timing are collected averaging 50 repetitions of the test with the 0 iteration being discarded. 
+Two resolutions have been tested: 
+
+* ``NX=NY=NZ=512`` which corresponds to rougly 130 million points.
+
+* ``NX=NY=NZ=1024`` which corresponds to rougly 1 billion points.
+
+A **2D** label for the results indicates a 2D (i.e. pencils) decomposition using the optimal automatic configuration
+(that generally corresponds to the closest decomposition to ``NR=NC``).
+A **1D** label for the results indicates a 1D (i.e. slabs) decomposition. With 2DECOMP&FFT this is obatained
+forcing one of the two decomposition direction to 1. If ``N_ROW=1`` an initial ``Z`` slabs 
+(i.e. local memory data are in the ``XY`` plane) is obatained, 
+conversely ``N_COL=1`` start from a ``X`` slabs configuration
+(i.e. local memmory data are in the ``YZ`` plane).
+Generally only one set of slabs data are plotted since performances are relatively similar.
+
+
+The library has been benchmark on the following systems: 
+
+* The UK National Supercomputer service `Archer2 <https://www.archer2.ac.uk>`_ 
+
+* The GPU partition of the EPCC `Cirrus <https://github.com/2decomp-fft/2decomp-fft/tree/main/examples>`_ 
+  service
+
+
+.. toctree::
+   :maxdepth: 1
+   :caption: Here the detailed list of the results:
+
+   benchmarks_archer2.rst
+

docs/source/pages/benchmarks_archer2.rst

@@ -0,0 +1,162 @@
+.. role:: raw-html(raw)
+    :format: html
+ 
+.. |br| raw:: html
+
+  <br/>
+
+========================
+Results for CPU: Archer2 
+========================
+
+Archer2 is the UK National Supercomputer service capable of 28 Pflops/s at peak performance. 
+The systems has 5,860 compute nodes, each with dual AMD EPYCTM 7742 64-core processors at 2.25GHz, 
+giving 750,080 cores in total. 
+
+Resuls are listed below
+
+* Transpose real 3D array
+
+  * Resolution ``NX=NY=NZ=512`` |br| |CPU_0512_TrReal_Scal| |CPU_0512_TrReal_SpeedUp| 
+
+  * Resolution ``NX=NY=NZ=1024`` |br| |CPU_1024_TrReal_Scal| |CPU_1024_TrReal_SpeedUp|
+
+* Transpose complex 3D array 
+  
+  * Resolution ``NX=NY=NZ=512`` |br| |CPU_0512_TrReal_Scal| |CPU_0512_TrReal_SpeedUp| 
+
+  * Resolution ``NX=NY=NZ=1024`` |br| |CPU_1024_TrClx_Scal| |CPU_1024_TrClx_SpeedUp|
+
+* FFT transform of a 3D real array starting from ``X`` physical direction
+
+  * Resolution ``NX=NY=NZ=512`` |br| |CPU_0512_R2CX_Scal| |CPU_0512_R2CX_SpeedUp| 
+
+  * Resolution ``NX=NY=NZ=1024`` |br| |CPU_1024_R2CX_Scal| |CPU_1024_R2CX_SpeedUp|
+
+* FFT transform of a 3D complex array starting from ``X`` physical direction
+
+  * Resolution ``NX=NY=NZ=512`` |br| |CPU_0512_C2CX_Scal| |CPU_0512_C2CX_SpeedUp| 
+
+  * Resolution ``NX=NY=NZ=1024`` |br| |CPU_1024_C2CX_Scal| |CPU_1024_C2CX_SpeedUp|
+
+* FFT transform of a 3D real array starting from ``Z`` physical direction
+
+  * Resolution ``NX=NY=NZ=512`` |br| |CPU_0512_R2CZ_Scal| |CPU_0512_R2CZ_SpeedUp| 
+
+  * Resolution ``NX=NY=NZ=1024`` |br| |CPU_1024_R2CZ_Scal| |CPU_1024_R2CZ_SpeedUp|
+
+* FFT transform of a 3D complex array starting from ``Z`` physical direction
+  
+  * Resolution ``NX=NY=NZ=512`` |br| |CPU_0512_C2CZ_Scal| |CPU_0512_C2CZ_SpeedUp| 
+
+  * Resolution ``NX=NY=NZ=1024`` |br| |CPU_1024_C2CZ_Scal| |CPU_1024_C2CZ_SpeedUp|
+
+Discussion on Archer2 results
+_____________________________
+
+The results above show that the the version 2.0 of 2DECOMP&FFT library keeps on having extremely good 
+scalability performances.
+The transpose tests show no difference between compilers since the tests mainly focus on MPI communication 
+and for all executable CRAY MPICH (Version 8.1.23) has been used. 
+It is interesting to notice that a 1D decomposition, when possible, can give up to a 80% speedup in comparison 
+with the optimal 2D decomposition. This is because of the new feature of the library where a simple copy, 
+avoiding completely MPI communication, is performed when data are all co-located in the local memory.
+This was not the case with the previous version of the library. 
+CRAY and GNU compilers performances using the *generic* FFT tends to differ for a low core count with the GNU
+performing a bit better in some cases (up to 50% performace increase), however results tends to converge with the
+increase of the numbers of nodes. 
+This gives some superlinear behaviour when looking at the speedup. 
+
+The **FFTW** has been tested only with the CRAY compiler and it gives a speed up of about 3 for a low core count 
+decreasing to something in between 1.5 and 2 for the larger number of nodes.
+The speed up with the FFTW is generally very close to the ideal lineat behaviour. 
+
+.. 
+   _Figures for Archer 2
+
+.. |CPU_0512_TrReal_Scal| image:: benchmarks_figs/2023_08_01_Res0512x0512x0512_TrReal_CPU_Archer2_ScalabilityTsec.pdf
+   :width: 35%
+   :alt: Archer2 Scalability transpose real test: Resolution 512^3 
+.. |CPU_0512_TrReal_SpeedUp| image:: benchmarks_figs/2023_08_01_Res0512x0512x0512_TrReal_CPU_Archer2_SpeedUp.pdf
+   :width: 35%
+   :alt: Archer2 SpeedUp transpose real test: Resolution 512^3 
+.. |CPU_1024_TrReal_Scal| image:: benchmarks_figs/2023_08_01_Res1024x1024x1024_TrReal_CPU_Archer2_ScalabilityTsec.pdf
+   :width: 35%
+   :alt: Archer2 Scalability transpose real test: Resolution 1024^3 
+.. |CPU_1024_TrReal_SpeedUp| image:: benchmarks_figs/2023_08_01_Res1024x1024x1024_TrReal_CPU_Archer2_SpeedUp.pdf
+   :width: 35%
+   :alt: Archer2 SpeedUp transpose real test: Resolution 1024^3 
+
+
+.. |CPU_0512_TrClx_Scal| image:: benchmarks_figs/2023_08_01_Res0512x0512x0512_TrClx_CPU_Archer2_ScalabilityTsec.pdf
+   :width: 35%
+   :alt: Archer2 Scalability transpose complex test: Resolution 512^3 
+.. |CPU_0512_TrClx_SpeedUp| image:: benchmarks_figs/2023_08_01_Res0512x0512x0512_TrClx_CPU_Archer2_SpeedUp.pdf
+   :width: 35%
+   :alt: Archer2 SpeedUp transpose complex test: Resolution 512^3 
+.. |CPU_1024_TrClx_Scal| image:: benchmarks_figs/2023_08_01_Res1024x1024x1024_TrClx_CPU_Archer2_ScalabilityTsec.pdf
+   :width: 35%
+   :alt: Archer2 Scalability transpose complex test: Resolution 1024^3 
+.. |CPU_1024_TrClx_SpeedUp| image:: benchmarks_figs/2023_08_01_Res1024x1024x1024_TrClx_CPU_Archer2_SpeedUp.pdf
+   :width: 35%
+   :alt: Archer2 SpeedUp transpose complex test: Resolution 1024^3 
+
+
+.. |CPU_0512_R2CX_Scal| image:: benchmarks_figs/2023_08_01_Res0512x0512x0512_R2CX_CPU_Archer2_ScalabilityTsec.pdf
+   :width: 35%
+   :alt: Archer2 Scalability R2CX test: Resolution 0512^3 
+.. |CPU_0512_R2CX_SpeedUp| image:: benchmarks_figs/2023_08_01_Res0512x0512x0512_R2CX_CPU_Archer2_SpeedUp.pdf
+   :width: 35%
+   :alt: Archer2 SpeedUp R2CX test: Resolution 0512^3 
+.. |CPU_1024_R2CX_Scal| image:: benchmarks_figs/2023_08_01_Res1024x1024x1024_R2CX_CPU_Archer2_ScalabilityTsec.pdf
+   :width: 35%
+   :alt: Archer2 Scalability R2CX test: Resolution 1024^3 
+.. |CPU_1024_R2CX_SpeedUp| image:: benchmarks_figs/2023_08_01_Res1024x1024x1024_R2CX_CPU_Archer2_SpeedUp.pdf
+   :width: 35%
+   :alt: Archer2 SpeedUp R2CX test: Resolution 1024^3 
+
+
+.. |CPU_0512_C2CX_Scal| image:: benchmarks_figs/2023_08_01_Res0512x0512x0512_C2CX_CPU_Archer2_ScalabilityTsec.pdf
+   :width: 35%
+   :alt: Archer2 Scalability R2CX test: Resolution 0512^3 
+.. |CPU_0512_C2CX_SpeedUp| image:: benchmarks_figs/2023_08_01_Res0512x0512x0512_C2CX_CPU_Archer2_SpeedUp.pdf
+   :width: 35%
+   :alt: Archer2 SpeedUp R2CX test: Resolution 0512^3 
+.. |CPU_1024_C2CX_Scal| image:: benchmarks_figs/2023_08_01_Res1024x1024x1024_C2CX_CPU_Archer2_ScalabilityTsec.pdf
+   :width: 35%
+   :alt: Archer2 Scalability R2CX test: Resolution 1024^3 
+.. |CPU_1024_C2CX_SpeedUp| image:: benchmarks_figs/2023_08_01_Res1024x1024x1024_C2CX_CPU_Archer2_SpeedUp.pdf
+   :width: 35%
+   :alt: Archer2 SpeedUp R2CX test: Resolution 1024^3 
+
+
+.. |CPU_0512_R2CZ_Scal| image:: benchmarks_figs/2023_08_01_Res0512x0512x0512_R2CZ_CPU_Archer2_ScalabilityTsec.pdf
+   :width: 35%
+   :alt: Archer2 Scalability R2CZ test: Resolution 0512^3 
+.. |CPU_0512_R2CZ_SpeedUp| image:: benchmarks_figs/2023_08_01_Res0512x0512x0512_R2CZ_CPU_Archer2_SpeedUp.pdf
+   :width: 35%
+   :alt: Archer2 SpeedUp R2CZ test: Resolution 0512^3 
+.. |CPU_1024_R2CZ_Scal| image:: benchmarks_figs/2023_08_01_Res1024x1024x1024_R2CZ_CPU_Archer2_ScalabilityTsec.pdf
+   :width: 35%
+   :alt: Archer2 Scalability R2CZ test: Resolution 1024^3 
+.. |CPU_1024_R2CZ_SpeedUp| image:: benchmarks_figs/2023_08_01_Res1024x1024x1024_R2CZ_CPU_Archer2_SpeedUp.pdf
+   :width: 35%
+   :alt: Archer2 SpeedUp R2CZ test: Resolution 1024^3 
+
+
+.. |CPU_0512_C2CZ_Scal| image:: benchmarks_figs/2023_08_01_Res0512x0512x0512_C2CZ_CPU_Archer2_ScalabilityTsec.pdf
+   :width: 35%
+   :alt: Archer2 Scalability R2CZ test: Resolution 0512^3 
+.. |CPU_0512_C2CZ_SpeedUp| image:: benchmarks_figs/2023_08_01_Res0512x0512x0512_C2CZ_CPU_Archer2_SpeedUp.pdf
+   :width: 35%
+   :alt: Archer2 SpeedUp R2CZ test: Resolution 0512^3 
+.. |CPU_1024_C2CZ_Scal| image:: benchmarks_figs/2023_08_01_Res1024x1024x1024_C2CZ_CPU_Archer2_ScalabilityTsec.pdf
+   :width: 35%
+   :alt: Archer2 Scalability R2CZ test: Resolution 1024^3 
+.. |CPU_1024_C2CZ_SpeedUp| image:: benchmarks_figs/2023_08_01_Res1024x1024x1024_C2CZ_CPU_Archer2_SpeedUp.pdf
+   :width: 35%
+   :alt: Archer2 SpeedUp R2CZ test: Resolution 1024^3 
+
+
+
+