Modules Environment

Environment Modules provide a convenient way to dynamically change the users’ environment through modulefiles. This includes easily adding or removing directories to the PATH environment variable. Modulefiles for Library packages provide environment variables (INCLUDE, LD_LIBRARY_PATH, …) that specify where the library and header files can be found .

The module command sets the appropriate environment variable independent of the user’s shell. A user can list the modules loaded by:

$ module list

To find out what modules are available to be loaded a user can do:

$ module avail

To load packages a user simply does:

$ module load package1 package2 ...

To unload packages a user does:

$ module unload package1 package2 ...

A user might wish to change from one compiler to another:

$ module swap gnu intel

The above command is short for:

$ module unload gnu
$ module load intel

A user may wish to go back to an initial set of modules:

$ module restore

This command will also unload all currently loaded modules, including the sticky ones, and then load the user's default collection. See User Collections for more details.

If a module is not available then an error message is produced:

$ module load packageXYZ
Warning: Failed to load: packageXYZ

Modulefiles can contain help messages. To access a modulefile’s help do:

$ module help packageName

To get a list of all the commands that module knows about do:

$ module help

The module avail command has search capabilities:

$ module avail cc

will list for any modulefile where the name contains the string “cc”.

Modulefiles can have a description section known as “whatis”. It is accessed by:

$ module whatis gnu8/8.3.0
gnu8/8.3.0          : Name: GNU Compiler Collection 
gnu8/8.3.0          : Version: 8.3.0 
gnu8/8.3.0          : Category: compiler, runtime support 
gnu8/8.3.0          : Description: GNU Compiler Family (C/C++/Fortran for x86_64) 
gnu8/8.3.0          : URL: http://gcc.gnu.org/ 

Finally, there is a keyword search tool:

$ module keyword word1 word2 ...

This will search any help message or whatis description for the word(s) given on the command line.

Another way to search for modules is with the module spidercommand. This command searches the entire list of possible modules. The difference between module avail and module spider is explained in the Module Hierarchy and Searching for Modules section:

$ module spider

Modules are a way to ask for a certain version of a package. For example we have more versions of git (say versions 2.9.3 and 2.18). So a user may load:

$ module load git

or:

$ module load git/2.9.3

In the second case, module will load git version 2.9.3 where as in the first case module will load the default version of git, the last one (in this case version 2.18).

Libraries built with one compiler need to be linked with applications with the same compiler version. If sites are going to provide libraries, then there will be more than one version of the library, one for each compiler version. Therefore, whether it is the Boost library or an mpi library, there are multiple versions.

There are two main choices for system administrators. For the XYZ library compiled with either the Intel compiler or the GNU compiler, there could be the xyz-intel modulefile and the xyz-gnu module file. This gets much more complicated when there are multiple versions of the XYZ library and different compilers. How does one label the various versions of the library and the compiler? Even if one makes sense of the version labeling, when a user changes compilers, the user will have to remember to unload the intel and the xyz-intel modulefiles when changing to gnu and xyz-gnu. If users have mismatched modules, their programs are going to fail in very mysterious ways.

A much saner strategy is to use a module hierarchy. Each compiler module adds to the MODULEPATH a compiler version modulefile directory. Only modulefiles that exist in that directory are packages that have been built with that compiler. When a user loads a particular compiler, that user only sees modulefile(s) that are valid for that compiler.

Similarly, applications that use libraries depending on MPI implementations must be built with the same compiler - MPI pairing. This leads to modulefile hierarchy. Therefore, as users start with the minimum set of loaded modules, all they will see are compilers, not any of the packages that depend on a compiler. Once they load a compiler they will see the modules that depend on that compiler. After choosing an MPI implementation, the modules that depend on that compiler-MPI pairing will be available. One of the nice features of module is that it handles the hierarchy easily. If a user swaps compilers, then module automatically unloads any modules that depends on the old compiler and reloads those modules that are dependent on the new compiler.

$ module list
Currently Loaded Modules:
1) gnu8/8.3.0   2) hdf5/1.10.5

$ module swap gnu intel

Due to MODULEPATH changes, the following have been reloaded:
  1) hdf5/1.10.5

If a modulefile is not available with the new compiler, then the module is marked as inactive. Every time MODULEPATH changes, module attempts to reload any inactive modules.

When a user enters:

$ module avail

module reports only the modules that are in the current MODULEPATH. Those are the only modules that the user can load. If there is a modulefile hierarchy, then a package the user wants may be available but not with the current compiler version. module offers a new command:

$ module spider

which lists all possible modules and not just the modules that can be seen in the current MODULEPATH. This command has three modes. The first mode is:

$ module spider

...
  boost: boost/1.70.0
    Boost free peer-reviewed portable C++ source libraries 

  cmake: cmake/3.14.3
    CMake is an open-source, cross-platform family of tools designed to build, test and package software. 

  cuda: cuda/9.0, cuda/9.1, cuda/10.0, cuda/10.1
    NVIDIA CUDA Toolkit 10.0 Patch 1 / cuDNN 7.4.2.24
...

This is a compact listing of all the possible modules on the system. The second mode describes a particular module:

$ module spider boost
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
  boost:
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
    Description:
      Boost free peer-reviewed portable C++ source libraries 

     Versions:
        boost/1.70.0

The third mode reports on a particular module version and where it can be found:

----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
  boost: boost/1.70.0
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
    Description:
      Boost free peer-reviewed portable C++ source libraries 


    You will need to load all module(s) on any one of the lines below before the "boost/1.70.0" module is available to load.

      gnu8/8.3.0  mvapich2/2.3.1
      gnu8/8.3.0  openmpi3/3.1.4
      intel/19.0.4.243  mvapich2/2.3.1
      intel/19.0.4.243  openmpi3/3.1.4
Automatically loaded modules may have a negative effect on GNOME graphical login

Users can automatically load a standard set of modules when log in, adding module commands to their personal startup files. To add module commands to users’ startup scripts, Bash users need to place the following in their ~/.bashrc file:

if [ -z "$BASHRC_READ" ]; then
   export BASHRC_READ=1
   # Place any module commands here
   # module load git
fi

By wrapping the module command in an if test, the module commands need only be read in once. Any sub-shell will inherit the PATH and other environment variables automatically.

Csh users need specify the module commands in their ~/.cshrc file:

if ( ! $?CSHRC_READ ) then
   setenv CSHRC_READ 1
   # Place any module command here
   # module load git
endif

If users have created at least a default collection (see User Collections for more details), they can replace all module load … commands with a single module restore. Bash users need to place the following in their ~/.bashrc file:

if [ -z "$BASHRC_READ" ]; then
   export BASHRC_READ=1
   module --initial_load --no_redirect restore
else
   module refresh
fi

Csh users need specify the module restore command in their ~/.cshrc file:

if ( ! $?CSHRC_READ ) then
   setenv CSHRC_READ 1
  module --initial_load restore
else
  module refresh
endif

Tips for graphical login

When using automatically loaded modules, a possible solution to avoid problems on GNOME login could be testing TERM environment variable. In this way, only when users open a graphical terminal (e.g., GNOME Terminal), modules will be loaded. On SISSA linux workstations, GNOME Terminal sets it as xterm-256color. Bash users may add a test on TERM variable in their ~/.bashrc:

if [ "$TERM" == "xterm-256color" -a -z "$BASHRC_READ" ]; then
   export BASHRC_READ=1
   # Place any module commands here
   # module load git
fi

Csh users need to add in their ~/.cshrc file:

if ( "$TERM" == "xterm-256color" && ! $?CSHRC_READ ) then
   setenv CSHRC_READ 1
   # Place any module command here
   # module load git
endif

Users can defined lists (collections) of modules to load with a single command. To define the default collection, users have to load the desired modules, for example:

$ module load cuda/9.0
$ module load intel/19.0.4.243
$ module load hdf5/1.10.5

and then issue:

$ module save

This creates a file called ~/.lmod.d/default which has the list of desired modules. Once this is set-up, a user can issue:

$ module restore

and only the desired modules will be loaded.

If a user doesn’t have a default collection, an error message is produced.

Users can have as many collections as they like. They can save to a named collection with:

$ module save <collection_name>

and restore that named collection with:

$ module restore <collection_name>

A user can print the contents of a collection with:

$ module describe <collection_name>

A user can list the collections they have with:

$ module savelist

Finally a user can disable a collection with:

$ module disable <collection_name>

If no collection_name is given then the default is disabled. Note that the collection is not remove just renamed. If a user disables the foo collection, the file foo is renamed to foo~. To restore the foo collection, a user will have to do the following:

$ cd ~/.lmod.d; mv foo~ foo

Rules for loading modules from a collection

module has rules on what modules to load when restoring a collection. In this guide, we call gnu8/8.3.0 the fullName of the module and gnu8 as the shortName. We also call what the user asked for as the userName which could either be the fullName or the shortName depending on what the user typed on the command line.

  1. module records the fullName and the userName in the collection.
  2. If the userName is the same as the fullName then it loads fullName independent of the default.
  3. if the userName is not the same as the fullName then it loads the default.
  4. Unless LMOD_PIN_VERSIONS=yes then the fullName is always loaded.

In other words if a user does:

$ module --force purge; module load A B C
$ module save

then module restore will load the default A, B, and C. So if the default for module A changed between when the collection was saved and then restored, a new version of A will be loaded. This assumes that LMOD_PIN_VERSIONS is not set. If it is set then if A/1.1, B/2.4 and C/3.3 are the default then those modules will be loaded in the future independent of what the defaults are in the future.

On the other hand:

$ module --force purge; module load A/1.0 B/2.3 C/3.4
$ module save

then module restore will load the A/1.0, B/2.3, and C/3.4 independent of what the defaults are now or in the future.

This website uses cookies for visitor traffic analysis. By using the website, you agree with storing the cookies on your computer.More information