rse,

Intel Parallel Studio XE 2020 update 4

Matthew Alexandrakis Matthew Alexandrakis Apr 22, 2021 · 4 mins read
Intel Parallel Studio XE 2020 update 4
Share this

The Intel Parallel Studio XE Cluster Edition 2020 update 4 suite is now installed and available on Apocrita. This release will be the last of the Parallel Studio releases we support, as Intel is moving to its “oneAPI” model of bundling.

The suite provides releases for Intel’s C, C++, and Fortran compilers. Also included are releases for MKL, MPI, Python, as well as a wide array of tools. The available modules are given below. Note that MPI and Python distributions have their own, distinct, modules.

module load intel/2020.4             # Compilers, Tools
module load intelmpi/2020.4          # MPI
module load intelpython/3.7.7-2020.4 # Python

We have not made these modules for the default for intel or intelmpi, as this version has breaking issues when using Fortran coarrays. You can read more in the “known issues” section. The default modules remain intel/2018.3 and intelmpi/2018.3. If you think you are affected by any issues and would like more specialised help or suggestions, we would encourage you to get in touch with our RSE team.

Release Notes

In this version, we are changing how MPI communications are propagating across the InfiniBand network. In earlier releases of Intel MPI we used the general Unix verbs provider to enable InfiniBand communication across our nodes. Now, the communication will be handled by the mlx provider, which is optimised for our Mellanox network cards, for compatibility and hardware detection considerations.

A complete list of release notes is provided by Intel.

Known Issues

  • The default environment variable I_MPI_HYDRA_TOPOLIB=hwloc can cause bad termination faults through SIGKILL 8/11 signals during compilation, including an empty cpuinfo card on some nodes. Intel’s suggested workaround is to set I_MPI_HYDRA_TOPOLIB=ipl, and this has been applied in the module. There is currently no visible impact to the end user in terms of performance or utility.

  • The compiler option -fast can cause severe issues during compilation. Static linking with optimisations should be avoided. Ideally, dynamic linking should be used along with the optimisation options -O2 and -O3.

  • Benign warnings during program execution can commonly appear and are related to environment variables created by the modules system. They are named by appending _modshare to the original variable. They can be safely ignored. Example:

    [0] MPI startup(): I_MPI_ROOT_modshare environment variable is not supported.
    

Known issues using Fortran coarrays

Several changes implemented since the 2018 releases of the compiler and MPI suites can affect the working of coarrays in Fortran programs as well as their performance. At this time, we recommend avoiding using intel/2020.4 with coarrays. Below is a non-exhaustive list of known issues we have discovered and brought to Intel’s attention.

  • Fortran coarrays runtime is affected by a longstanding bug of the underlying network structure. Running a Fortran coarray program on any sdx and ddy nodes will force a signal 11 segmentation fault. Intel’s suggested workaround is to export the following environment variable before runtime: export MPIR_CVAR_CH4_OFI_ENABLE_RMA=0. You can safely add it in your job submission script, before running the coarray binary.

  • Programs compiled with ifort -coarray=shared can now spawn images across multiple nodes. In previous releases it was used exclusively for shared memory image spawning (single node). Intel has confirmed that this is a feature and not a bug, but still strongly suggests using ifort -coarray=distributed for any multi-node runs.

  • In previous releases, programs compiled with ifort -coarray=distributed required a run-time configuration file. This is no longer needed.

  • The environment variable I_MPI_CAF_RUNTIME has been deprecated in the newer version. However, it has not been completely removed, leading to the following warning (safely ignored):

    [0] MPI startup(): I_MPI_CAF_RUNTIME environment variable is not supported.
    [0] MPI startup(): Similar variables: I_MPI_THREAD_RUNTIME
    

Reproducibility concerns

Reproducibility can be affected by some of the environment variables set in the module files. The module files are supplied with the minimal amount of variables set to enable the compilers to work across as many Apocrita nodes as possible. The exact environment-setting statements used in the module file (and any other module file) can be seen through module show <module name>. For any reproducibility considerations we would encourage you to get in touch with the RSE team.

Appendix - Intel tools

  • Enhanced GNU* Project Debugger (GDB)
  • Intel Data Analytics Acceleration Library (DAAL)
  • Intel Threading Building Blocks (TBB)
  • Intel Integrated Performance Primitives (IPP)
  • Intel Advisor
  • Intel Inspector
  • Intel VTune Profiler
  • Intel Trace Analyzer and Collector (ITAC)
  • Intel Cluster Checker (CLCK)
Matthew Alexandrakis
Written by Matthew Alexandrakis
Research Software Engineer. Lately, it's all about MPI.