This is an old revision of the document!
Modules Environment
Introduction
Environment Modules provide a convenient way to dynamically change the users’ environment through modulefiles. This includes easily adding or removing directories to the PATH environment variable. Modulefiles for Library packages provide environment variables (INCLUDE, LD_LIBRARY_PATH, …) that specify where the library and header files can be found .
Usage
The module command sets the appropriate environment variable independent of the user’s shell. A user can list the modules loaded by:
$ module list
To find out what modules are available to be loaded a user can do:
$ module avail
To load packages a user simply does:
$ module load package1 package2 ...
To unload packages a user does:
$ module unload package1 package2 ...
A user might wish to change from one compiler to another:
$ module swap gnu intel
The above command is short for:
$ module unload gnu $ module load intel
A user may wish to go back to an initial set of modules:
$ module reset
This will unload all currently loaded modules, including the sticky ones, then load the list of modules specified by LMOD_SYSTEM_DEFAULT_MODULES. There is a related command:
$ module restore
This command will also unload all currently loaded modules, including the sticky ones, and then load the system default unless the user has a default collection. See User Collections for more details.
If a module is not available then an error message is produced:
$ module load packageXYZ Warning: Failed to load: packageXYZ
Modulefiles can contain help messages. To access a modulefile’s help do:
$ module help packageName
To get a list of all the commands that module knows about do:
$ module help
The module avail command has search capabilities:
$ module avail cc
will list for any modulefile where the name contains the string “cc”.
Modulefiles can have a description section known as “whatis”. It is accessed by:
$ module whatis gnu8/8.3.0 gnu8/8.3.0 : Name: GNU Compiler Collection gnu8/8.3.0 : Version: 8.3.0 gnu8/8.3.0 : Category: compiler, runtime support gnu8/8.3.0 : Description: GNU Compiler Family (C/C++/Fortran for x86_64) gnu8/8.3.0 : URL: http://gcc.gnu.org/
Finally, there is a keyword search tool:
$ module keyword word1 word2 ...
This will search any help message or whatis description for the word(s) given on the command line.
Another way to search for modules is with the module spidercommand. This command searches the entire list of possible modules. The difference between module avail and module spider is explained in the “Module Hierarchy” and “Searching for Modules” section.:
$ module spider
Specifying modules to load
Modules are a way to ask for a certain version of a package. For example we have more versions of git (say versions 2.9.3 and 2.18). So a user may load:
$ module load git
or:
$ module load git/2.9.3
In the second case, module will load git version 2.9.3 where as in the first case module will load the default version of git, the last one (in this case version 2.18).
Module Hierarchy
Libraries built with one compiler need to be linked with applications with the same compiler version. If sites are going to provide libraries, then there will be more than one version of the library, one for each compiler version. Therefore, whether it is the Boost library or an mpi library, there are multiple versions.
There are two main choices for system administrators. For the XYZ library compiled with either the Intel compiler or the GNU compiler, there could be the xyz-intel modulefile and the xyz-gnu module file. This gets much more complicated when there are multiple versions of the XYZ library and different compilers. How does one label the various versions of the library and the compiler? Even if one makes sense of the version labeling, when a user changes compilers, the user will have to remember to unload the intel and the xyz-intel modulefiles when changing to gnu and xyz-gnu. If users have mismatched modules, their programs are going to fail in very mysterious ways.
A much saner strategy is to use a module hierarchy. Each compiler module adds to the MODULEPATH a compiler version modulefile directory. Only modulefiles that exist in that directory are packages that have been built with that compiler. When a user loads a particular compiler, that user only sees modulefile(s) that are valid for that compiler.
Similarly, applications that use libraries depending on MPI implementations must be built with the same compiler - MPI pairing. This leads to modulefile hierarchy. Therefore, as users start with the minimum set of loaded modules, all they will see are compilers, not any of the packages that depend on a compiler. Once they load a compiler they will see the modules that depend on that compiler. After choosing an MPI implementation, the modules that depend on that compiler-MPI pairing will be available. One of the nice features of module is that it handles the hierarchy easily. If a user swaps compilers, then module automatically unloads any modules that depends on the old compiler and reloads those modules that are dependent on the new compiler.
$ module list Currently Loaded Modules: 1) gnu8/8.3.0 2) hdf5/1.10.5 $ module swap gnu intel Due to MODULEPATH changes, the following have been reloaded: 1) hdf5/1.10.5
If a modulefile is not available with the new compiler, then the module is marked as inactive. Every time MODULEPATH changes, module attempts to reload any inactive modules.
Searching For Modules
When a user enters:
$ module avail
module reports only the modules that are in the current MODULEPATH. Those are the only modules that the user can load. If there is a modulefile hierarchy, then a package the user wants may be available but not with the current compiler version. module offers a new command:
$ module spider
which lists all possible modules and not just the modules that can be seen in the current MODULEPATH. This command has three modes. The first mode is:
$ module spider
...
boost: boost/1.70.0
Boost free peer-reviewed portable C++ source libraries
cmake: cmake/3.14.3
CMake is an open-source, cross-platform family of tools designed to build, test and package software.
cuda: cuda/9.0, cuda/9.1, cuda/10.0, cuda/10.1
NVIDIA CUDA Toolkit 10.0 Patch 1 / cuDNN 7.4.2.24
...
This is a compact listing of all the possible modules on the system. The second mode describes a particular module:
$ module spider boost
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
boost:
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Description:
Boost free peer-reviewed portable C++ source libraries
Versions:
boost/1.70.0
The third mode reports on a particular module version and where it can be found:
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
boost: boost/1.70.0
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Description:
Boost free peer-reviewed portable C++ source libraries
You will need to load all module(s) on any one of the lines below before the "boost/1.70.0" module is available to load.
gnu8/8.3.0 mvapich2/2.3.1
gnu8/8.3.0 openmpi3/3.1.4
intel/19.0.4.243 mvapich2/2.3.1
intel/19.0.4.243 openmpi3/3.1.4
Controlling Modules During Login
Normally when a user logs in, there are a standard set of modules that are automatically loaded. Users can override and add to this standard set in two ways. The first is adding module commands to their personal startup files. The second way is through the “module save” command.
To add module commands to users’ startup scripts requires a few steps. Bash users can put the module commands in either their ~/.profile file or their ~/.bashrc file. It is simplest to place the following in their ~/.profile file:
if [ -f ~/.bashrc ]; then
. ~/.bashrc
fi and place the following in their ~/.bashrc file:
if [ -z “$BASHRC_READ” ]; then
export BASHRC_READ=1 # Place any module commands here # module load git
fi By wrapping the module command in an if test, the module commands need only be read in once. Any sub-shell will inherit the PATH and other environment variables automatically. On login shells the ~/.profile file is read which, in the above setup, causes the ~/.bashrc file to be read. On interactive non-login shells, the ~/.bashrc file is read instead. Obviously, having this setup means that module commands need only be added in one file and not two.
Csh users need only specify the module commands in their ~/.cshrc file as that file is always sourced:
if ( ! $?CSHRC_READ ) then
setenv CSHRC_READ 1 # Place any module command here # module load git
endif User Collections User defined initial list of login modules:
Assuming that the system administrators have installed Lmod correctly, there is a second way which is much easier to setup. A user logs in with the standard modules loaded. Then the user modifies the default setup through the standard module commands:
$ module unload XYZ $ module swap gcc ucc $ module load git Once users have the desired modules load then they issue:
$ module save This creates a file called ~/.lmod.d/default which has the list of desired modules. Once this is set-up a user can issue:
$ module restore and only the desired modules will be loaded. If Lmod is setup correctly (see Providing A Standard Set Of Modules for all Users) then the default collection will be the user’s initial set of modules.
If a user doesn’t have a default collection, the Lmod purges ALL currently loaded modules, including the sticky ones, and loads the list of module specified by LMOD_SYSTEM_DEFAULT_MODULES just like the module reset command.
Users can have as many collections as they like. They can save to a named collection with:
$ module save <collection_name> and restore that named collection with:
$ module restore <collection_name> A user can print the contents of a collection with:
$ module describe <collection_name> A user can list the collections they have with:
$ module savelist Finally a user can disable a collection with:
$ module disable <collection_name> If no collection_name is given then the default is disabled. Note that the collection is not remove just renamed. If a user disables the foo collection, the file foo is renamed to foo~. To restore the foo collection, a user will have to do the following:
$ cd ~/.lmod.d; mv foo~ foo Rules for loading modules from a collection Lmod has rules on what modules to load when restoring a collection. Remember that userName is what the user asked for, the fullName is the exact module name and shortName is name of the package (e.g. gcc, fftw3).
Lmod records the fullName and the userName in the collection. If the userName is the same as the fullName then it loads fullName independent of the default. if the userName is not the same as the fullName then it loads the default. Unless LMOD_PIN_VERSIONS=yes then the fullName is always loaded. In other words if a user does:
$ module –force purge; module load A B C $ module save then “module restore” will load the default A, B, and C. So if the default for module A changed between when the collection was saved and then restored, a new version of A will be loaded. This assumes that LMOD_PIN_VERSIONS is not set. If it is set or Lmod is configured that way then if A/1.1, B/2.4 and C/3.3 are the default then those modules will be loaded in the future independent of what the defaults are in the future.
On the other hand:
$ module –force purge; module load A/1.0 B/2.3 C/3.4 $ module save then “module restore” will load the A/1.0, B/2.3, and C/3.4 independent of what the defaults are now or in the future.
User Collections on shared home file systems If your site has a shared home file system, then things become a little more complicated. A shared home file system means that your site has a single home file system shared between two or more clusters. See Lmod on Shared Home File Systems for a system administrators point of view.
If you have a collection on one cluster it needs to be independent of another cluster. Your site should set $LMOD_SYSTEM_NAME uniquely for each cluster. Suppose you have cluster A and B. Then $LMOD_SYSTEM_NAME will be either A or B. A default collection will be named “default.A” for the A cluster and “default.B” for the B cluster. The names a user sees will have the extension removed. In other words on the A cluster a user would see:
$ module savelist
1) default
where the default file is named “default.A”.
Usage
The workstations have a variety of compilers, libraries, and other bits of software installed. Some of it exists in multiple versions, and, for example, you may wish to switch between compilers for testing or you may wish to select a specific revision of Matlab.
The purpose of the module package is to simplify this for you.
On all workstations the modules package is available as standard; it provides for the dynamic modification of the user's environment via modulefiles.
Each modulefile contains the information needed to configure the shell for an application; the environment can be modified on a per-module basis using the module command which interprets modulefiles. Typically modulefiles instruct the module command to alter or set shell environment variables such as PATH, MANPATH, etc.
The modulefiles are added to and removed from the current environment by the user. The environment changes contained in a modulefile can be summarized through the module command as well. If no arguments are given, a summary of the module usage and sub-commands are shown:
[user@workstation ~]# module Modules Release Tcl x.x.x ($RCSfile: modulecmd.tcl,v $ $Revision: x.xxx $) Copyright GNU GPL v2 1991 Usage: module [ switches ] [ command ] Switches: -t terse format avail and list -l long format avail and list Commands: list | add|load modulefile [modulefile ...] purge | rm|unload modulefile [modulefile ...] reload | switch|swap [oldmodulefile] newmodulefile | display|show modulefile [modulefile ...] | avail [modulefile [modulefile ...]] | whatis [modulefile [modulefile ...]] | help [modulefile [modulefile ...]] | path modulefile | paths modulefile | use dir [dir ...] | unuse dir [dir ...] | source scriptfile | apropos|keyword string | initlist | initadd modulefile initclear | initprepend modulefile | initrm modulefile
Sub-Commands
avail
With the subcommand avail you get available list of module files (versions or names may change):
[user@workstation ~]# module avail ------------ /u/shared/programs/noarch/common/modulefiles ------------ mathematica/6.0.2(default) mkl/10.0.2.018(default) mathematica/6.0.1 mkl/9.0.1 ------------ /u/shared/programs/x86_64/common/modulefiles ------------ cce/10.1.013(default) intel/10.1.013(default) pgi/7.1-6(default) cce/9.0.1 intel/9.1 pgi/6.1 fce/10.1.013(default) maple/11(default) sm/2.4.34(default) fce/9.1 maple/10 sm/2.2.3 idbe/10.1.013(default) matlab/2007b(default) idbe/9.1 matlab/2007a
The notation for the modulefilename is “name of the application/version and description”.
load
To use a particular module, run module load modulename. You don't need to list the full name of the module, as listed in module avail output, if you only use the first component, it will choose the latest version for you. (It actually chooses the last item alphabetically.)
[user@workstation ~]# module load cce
list
This lists all modules which are currently loaded into the users' environment (versions or names may change).
[user@workstation ~]# module list Currently Loaded Modulefiles: 1) cce/10.1.013
unload
Use this command to unload one or more of the loaded module files (seen in list before).
[user@workstation ~]# module unload cce
whatis
If a module looks interesting, use this command to display the information contained within the given module file (versions or names may change).
[user@workstation ~]# module whatis cce
----------- /u/shared/programs/noarch/common/modulefiles -------------
----------- /u/shared/programs/x86_64/common/modulefiles -------------
cce/10.1.013: loads the Intel(R) C++ Compiler 10.1.013
display
Use this command to see exactly what a given modulefile will do to your environment, such as what will be added to the PATH, MANPATH, etc. environment variables.
[user@workstation ~]# module display cce ------------------------------------------------------------------- /u/shared/programs/x86_64/common/modulefiles/cce/10.1.013: module-whatis loads the Intel(R) C++ Compiler 10.1.013 prepend-path PATH /u/shared/programs/x86_64/common/intel/cce/10.1.013/bin prepend-path LD_LIBRARY_PATH /u/shared/programs/x86_64/common/intel/cce/10.1.013/lib prepend-path MANPATH /u/shared/programs/x86_64/common/intel/cce/10.1.013/man prepend-path INTEL_LICENSE_FILE /u/shared/licenses/intel -------------------------------------------------------------------
Using modules in batch
To automatically load modules when you log in to a system, put the module load modulename command in your shell's startup script.
For example:
- if your shell is
bash, put the command in your~/.bashrc - if your shell is
csh, put it in your~/.cshrc.
If you change shells in your batch script you may need to explicitly load the modules environment (if your script does not use modules commands, there is, of course, no need to do this) by adding as the first line of your bash script:
. /etc/profile.d/modules.sh
or in case your script is csh derived:
source /etc/profile.d/modules.csh