rse,

Apocrita says Hello, world!: basic use of our cluster

Iain Barrass Iain Barrass Follow Feb 12, 2020 · 11 mins read
Apocrita says Hello, world!: basic use of our cluster
Share this

A common first program to write in a new language is a “Hello world” example where we print a simple line of output. In this tutorial we first look at examples written in C, C++ and Fortran. To run the examples we’ll learn about interactive sessions on compute nodes, modules and compiling source code. We’ll also look at examples in MATLAB, Python and R. For these we’ll see how to use modules to select suitable interpreters.

After completing this tutorial the reader should:

  • understand basic module usage on Apocrita;
  • be able to compile and run simple programs on a compute node;
  • be able to use interpreters on a compute node;
  • know the difference between interactive and batch jobs.

We run all of our jobs on compute nodes on Apocrita, to avoid using excessive resources on the login node. We’re first going to use an interactive session on one of the nodes before creating a job script which can be run when we aren’t logged in.

Using qlogin to create an interactive session

Modules on Apocrita modifying our user environment to give us access to tools. These allow us to switch between different versions of tools or select a particular choice of tool when there are several conflicting options.

For example, we can load a module to select which compiler suite and its version we wish to use. We may also use a module to set up our environment to use a particular version of Python.

Running a C, C++ or Fortran program

On Apocrita, the plain text files that contain a C, C++ or Fortran program cannot be run directly. Instead, these source files must first be compiled to give us an executable program. For example, if we have a file called hello.f90 with the following Fortran program

  print '("Hello, world!")'
end

we need to use a Fortran compiler to process this file.

Apocrita has a number of different compiler suites available, containing compilers for C, C++ or Fortran source. The available GCC compilers can be seen with the module command

$ module avail gcc/
-------- /share/apps/environmentmodules/centos7/devtools ---------
gcc/6.3.0  gcc/7.1.0(default)  gcc/8.2.0

The Intel and PGI compilers may be seen similarly:

$ module avail intel/
-------- /share/apps/environmentmodules/centos7/devtools ---------
intel/2017.1  intel/2017.3  intel/2018.1  intel/2018.3(default)
$ module avail pgi/
-------- /share/apps/environmentmodules/centos7/devtools ---------
pgi/19.4  pgi/19.10(default)

We can choose a particular version of one of the compiler suites with the module load command, giving the name of a module:

module load intel/2018.1

This load command says that we want to access the Intel compiler suite, version “2018.1”.

If we are happy with the “default” version of a compiler suite, we can load that with the command

$ module purge
$ module load gcc
$ module list
Currently Loaded Modulefiles:
 1)gcc/7.1.0(default)

When we load one of these modules, we have access to the compilers of that suite. Although the compiler commands have the same name for each version of a suite, the name will be different for the Intel compiler and one of the others. That is, the Intel Fortran compiler is called ifort, but the GCC Fortran compiler is called gfortran and PGI’s pgfortran. Fortunately, our modules set environment variables to give us a command name our C compiler can be used with ${CC}, our C++ compiler with ${CXX} and our Fortran compiler with ${FC}:

$ module purge
$ module load intel/2018.1
$ echo ${FC}
ifort
$ module switch intel/2018.1 gcc/8.2.0
$ echo ${FC}
/share/apps/centos7/gcc/gcc/4.8.5/8.2.0/bin/gfortran

When we’ve loaded a compiler module, we can compile the Fortran source file we have:

${FC} hello.f90 -o hello

This calls the Fortran compiler (${FC}), asking it to compile the source file hello.f90 and producing an executable program called hello (-o hello). We can see this new file has been created with the command

ls -l hello

Finally, we can run this program with the command:

$ ./hello
Hello, world!

For the C and C++ source files below, we can compile these in the same way using the corresponding compilers:

$ ${CC} hello.c -o hello && ./hello
Hello, world!
$ ${CXX} hello.cpp -o hello && ./hello
Hello, world!

C:

#include <stdio.h>

int main() {
  printf("Hello, world!\n");
  return 0;
}

C++:

#include <iostream>

int main() {
  std::cout<<"Hello, world!"<<std::endl;
}

Once we’ve compiled the program we can run it again as many times as we like without recompiling it. It’s only when we make changes to the source code that we need to go through the compilation step before running.

Running a MATLAB, Python or R program

In contrast to the examples of C, C++ and Fortran above, some programming languages environments on Apocrita may come with interpreters. With the language R, for example, we have an interpreter we can load and then, in an interactive session, type in commands and see the results fed back to us.

As with the compilers, we first load a module to select a language interpreter and its version:

$ module avail R/
--------- /share/apps/environmentmodules/centos7/general ---------
R/3.3.2  R/3.4.0  R/3.4.2  R/3.4.3  R/3.5.1  R/3.5.3  R/3.6.1(default)
$ module load R/3.6.1
Loading R/3.6.1
  Loading requirement: java/1.8.0_171-oracle
$ R

R version 3.6.1 (2019-07-05) -- "Action of the Toes"
...
>

Where the > is now a prompt within the R interpreter, rather than our shell. As we type commands in there, they are interpreted within the R environment:

> print('Hello, world!')
[1] "Hello, world!"
> quit(save='no')
$

Our R statement print('Hello, world!') is run to give the output on the second line. We can then exit the R interpreter, returning to our shell, with the function quit().

Similarly, the Python interpreter can be used:

$ module load python/3.6.3
$ python
Python 3.6.3 (default, Oct  4 2017, 15:04:38)
...
>>> print('Hello, world!')
Hello, world!
>>> exit()
$

MATLAB on Apocrita defaults to the desktop GUI: for simple interactive use we usually want to avoid that:

$ module load matlab/2019a
$ matlab -nodisplay -nojvm

                            < M A T L A B (R) >
                  Copyright 1984-2019 The MathWorks, Inc.
                  R2019a (9.6.0.1072779) 64-bit (glnxa64)
                               March 8, 2019

For online documentation, see https://www.mathworks.com/support
For product information, visit www.mathworks.com.

>> disp 'Hello, world!'
Hello, world!
>> exit
$

While we can use the interpreters mentioned above interactively within an interactive session on a compute node, typing commands in and getting the result back in the terminal, we can also use the interpreters to process whole scripts. If we have the text files with our desired commands in, in the files hello.R, hello.py and hello.m for the R, Python and MATLAB examples respectively we can run them non-interactively.

For R we use the command Rscript to process a script file (instead of R for the interactive session):

$ Rscript hello.R
[1] "Hello, world!"

For Python:

$ python hello.py
Hello, world!

For MATLAB we use the -r hello option to execute commands from the file hello.m (the statement hello in MATLAB will be sourced from a file hello.m found in the search path, which in this case is the current directory):

$ matlab -nodesktop -nodisplay -nosplash -nojvm -r hello

                            < M A T L A B (R) >
                  Copyright 1984-2019 The MathWorks, Inc.
                  R2019a (9.6.0.1072779) 64-bit (glnxa64)
                               March 8, 2019


For online documentation, see https://www.mathworks.com/support
For product information, visit www.mathworks.com.

Hello, world!

Using script files in this way allows us to repeat a whole program without entering each line and waiting for the response. As we’ll see in the next part of the tutorial, such scripts may also be used in a non-interactive mode in a submitted batch job.

Writing and submitting job scripts

In the previous part of the tutorial we used an interactive qlogin session to run programs on a compute node. Here we’ll use a batch job to run a compiled program or script in a non-interactive way.

Running pre-compiled C, C++ or Fortran program

Once we’ve compiled our program source in an interactive session, we have an executable program which we can run whenever we want. To run the program on compute modes we switch to using batch jobs. If we have the program hello in our current directory we create the following job submission script, called hello-compiled.sh, to run it:

#!/bin/sh
#$ -cwd              # Run the job in the current directory
#$ -pe smp 1         # Request a single core on the node
#$ -l h_vmem=0.5G    # Request 0.5GB memory for the job
#$ -l h_rt=0:10:0    # The job should complete in 10 minutes
#$ -N hello          # Call the job "hello"
#$ -o hello.out      # Write output to the file hello.out

# Load the compiler module
module load intel/2018.3

# Run the compiled program
./hello

We load the same module file that we used to compile the program before running it in the job to ensure that the compiler’s run-time environment is available.

The job script can be submitted using the qsub command, and we can see the state of the submitted job in the queue using qstat:

qsub hello-compiled.sh
qstat

Once the job has finished, which can be seen by the job no longer appearing in the qstat output, we can look at the output file created by the job:

$ cat hello.out
Hello, world!

Running in batch a MATLAB, Python or R program

Running a script non-interactively in a batch job is much the same as how we ran it non-interactively in the session earlier. As with the compiled program we create a submission script hello-scripts.sh:

#!/bin/sh
#$ -cwd              # Run the job in the current directory
#$ -pe smp 1         # Request a single core on the node
#$ -l h_vmem=0.5G    # Request 0.5GB memory for the job
#$ -l h_rt=0:10:0    # The job should complete in 10 minutes
#$ -N hello          # Call the job "hello"
#$ -o hello.out      # Write output to the file hello.out

# Load the interpreter modules
module load matlab/2019a python/3.6.3 R/3.6.1

# Run the interpreters with the scripts
# MATLAB
matlab -nodesktop -nodisplay -nosplash -nojvm -r hello

# Python
python hello.py

# R
Rscript hello.R

Conclusion

In this tutorial we’ve looked at how we can run basic programs on the cluster Apocrita. We used modules to set up our environment to gain access to compilers or interpreters. For C, C++ and Fortran we saw how to compile source code into runnable programs and how to run those programs; for MATLAB, Python and R we ran interactive sessions and submitted scripts which ran non-interactively.

Iain Barrass
Written by Iain Barrass
RSE Team Leader. He prefers Fortran, with a background in infectious diseases