The Intel Parallel Studio XE Cluster Edition 2020 update 4 suite is now installed and available on Apocrita. This release will be the last of the Parallel Studio releases we support, as Intel is moving to its “oneAPI” model of bundling.
The suite provides releases for Intel’s C, C++, and Fortran compilers. Also included are releases for MKL, MPI, Python, as well as a wide array of tools. The available modules are given below. Note that MPI and Python distributions have their own, distinct, modules.
module load intel/2020.4 # Compilers, Tools module load intelmpi/2020.4 # MPI module load intelpython/3.7.7-2020.4 # Python
In this version, we are changing how MPI communications are propagating across
the InfiniBand network. In earlier releases of Intel MPI we used the general
verbs provider to enable InfiniBand communication across our nodes.
Now, the communication will be handled by the
mlx provider, which is
optimised for our Mellanox
network cards, for compatibility and hardware detection considerations. We
are currently tracking no issues, and the change will not impact our end users.
A complete list of release notes is provided by Intel.
The default environment variable
I_MPI_HYDRA_TOPOLIB=hwloccan cause bad termination faults through
SIGKILL 8/11signals during compilation, including an empty
cpuinfocard on some nodes. Intel’s suggested workaround is to set
I_MPI_HYDRA_TOPOLIB=ipl, and this has been applied in the module. There is currently no visible impact to the end user in terms of performance or utility.
The compiler option
-fastcan cause severe issues during compilation. Static linking with optimisations should be avoided. Ideally, dynamic linking should be used along with the optimisation options
-O3. The RSE team is available for any further advice and personalised suggestions.
Benign warnings during program execution can commonly appear and are related to environment variables created by the modules system. They are named by appending
_modshareto the original variable. They can be safely ignored. Example:
 MPI startup(): I_MPI_ROOT_modshare environment variable is not supported.
Known issues using Fortran coarrays
Several changes implemented since the 2018 releases of the compiler suite can affect the usability of coarrays in Fortran programs. Below is a non-inclusive list of known issues we have discovered and brought to Intel’s attention.
Coarray programs can now spawn images across multiple nodes with
ifort -coarray=shared, a major departure from previous versions where it was used exclusively for shared memory image spawning (single node). Intel stresses that while the feature is indeed working as intended (ie, not a bug), it should still only be used to spawn images on a single node. For multiple nodes, Intel suggests using
Coarray programs can use multiple nodes when compiled with
ifort -coarray=distributed, a significant departure from previous versions. There is no need to create or specify node configuration files anymore.
The environment variable
I_MPI_CAF_RUNTIMEhas been deprecated in the newer version. However, it has not been completely removed, leading to the following warning (safely ignored):
 MPI startup(): I_MPI_CAF_RUNTIME environment variable is not supported.  MPI startup(): Similar variables: I_MPI_THREAD_RUNTIME
Reproducibility can be affected by some of the environment variables set in
the module files. The module files are supplied with the minimal amount of
variables set to enable the compilers to work across as many Apocrita nodes as
possible. The exact environment-setting statements used in the module file
(and any other module file) can be seen through
module show <module name>.
For any reproducibility considerations we would encourage you to get in touch
with the RSE team.
Appendix - Intel tools
- Enhanced GNU* Project Debugger (GDB)
- Intel Data Analytics Acceleration Library (DAAL)
- Intel Threading Building Blocks (TBB)
- Intel Integrated Performance Primitives (IPP)
- Intel Advisor
- Intel Inspector
- Intel VTune Profiler
- Intel Trace Analyzer and Collector (ITAC)
- Intel Cluster Checker (CLCK)