Managing environment variables with Modules

Official website link: http://modules.sourceforge.net/

Official documents: https://modules.readthedocs.io/en/latest/

1, What are Modules

Typically, users initialize their environment at login by setting environment information for each application they will reference during the session. The Environment Modules package is a tool that simplifies shell initialization and allows users to easily modify their environment during sessions using module files.

Each module file contains the information needed to configure the shell for the application. Once the Modules package is initialized, you can use the module command that interprets the module file to modify the environment on a per module basis. Typically, the module file instructs the module command to change or set shell environment variables, such as PATH, MANPATH, and so on. Module files may be shared by many users on the system, and users may have their own collection to supplement or replace the shared module files.

Modules can be loaded and unloaded dynamically and atomically in a clean manner. Support all popular shell s, including bash, ksh, zsh, sh, csh, tcsh, fish, and some scripting languages, such as perl, ruby, tcl, python, cmake, and R.

The module can be used to manage different versions of applications. Modules can also be bundled into meta modules, which will load a complete set of different applications.

Most Linux users manage environment variables by exporting path = / path / to / some / bin: / $Path > > ~ / Bashrc, for a long time, when you echo $PATH, you will find a large number of software paths. In my eyes, reducing the search speed of the system for tools is the second, mainly because it is not very good-looking. For Module, the administrator writes the environment configuration of different software, and then after logging in, the user only needs to use module load xxx to add the required tools in the environment.

 

2, Installation configuration

1. Install tlc first

sudo apt-get install tcl-dev tk-dev mesa-common-dev libjpeg-dev libtogl-dev

2. Installation

• Download the source file and compile it

  curl -LJO https://github.com/cea-hpc/modules/releases/download/v4.7.1/modules-4.7.1.tar.gz
  tar xfz modules-4.7.1.tar.gz
  

  cd modules-4.7.1
  ./configure
  make
  make install
  

• Online installation

  # The default installation version is 3.2.10
  yum install environment-modules
  

3. Initialize configuration

In order to ensure that users can call module when logging in to the server, you have to copy the initialization script of module to / etc / profiled D directory

PREFIX is usually / usr/local/Modules

ln -s PREFIX/init/profile.sh /etc/profile.d/modules.sh
ln -s PREFIX/init/profile.csh /etc/profile.d/modules.csh

Or manually

source PREFIX/init/profile.sh

 

3, Basic use

module avail  # Displays the available modules

module list     # Displays the modules that have been loaded

module load/add <module_name>  # Loading module

module unload/rm <module_name>  # Uninstall module

module switch <module1_name> <module2_name> # Switch from module1 to module2

The module tool relies on the environment variable MODULEPATH to find the configuration information directory, that is, after setting the directory structure and configuring the environment variables, you only need to set the environment variables of this module, and the module tool will automatically find all the configuration information under this path.

up@up-vpc:~$ export | grep MODULEPATH
declare -x MODULEPATH="/usr/local/Modules/modulefiles"
declare -x MODULEPATH_modshare="/usr/local/Modules/modulefiles:1"

up@up-vpc:~$ ls /usr/local/Modules/modulefiles/
dot  module-git  module-info  modules  null  use.own
# There are several default modulefiles in the modulefiles directory

 

4, Modulefile

1. Introduction

Here, MODULEPATH uses the default path, / usr/local/Modules/modulefiles. Generally, we will put the same kind of software, libraries and other contents in the same directory. Assuming that the directory name is modulefiles, the modulefiles directory contains gcc, python, Java and other common tools, while gcc contains versions 4.9.3 and 5.5.0, python contains versions 2.7 and 3.7, and java contains versions 1.5 and 1.8, its directory structure is as follows:

The file of the last version number command is modulefile. We only need to install this directory format and create directories and files.

 

2. Examples

#1. Be sure to write at the beginning of the file
#%Module1.0

This is to identify this file as modulefile Yes, this document will not be recognized without him

#2. Prepend path the path of the environment variable to be modified
prepend-path PATH /soft/gcc/4.8.4/bin
#This command will add the tool path to the front of the environment variable

#3. setenv environment variable name value
setenv GCC_HOME /soft/gcc/4.8.4/
#This command will configure the environment variables you need into the system

#%Module1.0

setenv GCC_HOME /software/gcc/4.8.4/

prepend-path PATH /software/gcc/4.8.4/bin

 

3. modulefile syntax

file: https://modules.readthedocs.io/en/latest/modulefile.html#description

• chdir modify working directory

  chdir directory
  

• Prepend path appends or pre adds values to environment variables

  prepend-path [-d C|--delim C|--delim=C] [--duplicates] variable value...
  # For example: prepend path path / usr / local / soft / bin
  

  • --delim c can reset the separator. By default, it uses: as the separator, for example, PATH=directory:directory:directory
  • --duplicates is set to be repeatable. If it is defined by default, it cannot be repeated

• setenv setting environment variable

  setenv variable value
  # For example: setenv CUS_VARIABLE 123
  

• Module whatis defines the string displayed when the command is invoked

  The string parameter must be in double quotation marks

  module-whatis string
  # For example: module Whatis 	 "sysC version 2.3.1"
  

• system runs string commands through the shell

  On Unix, the command is passed to / bin / sh shell, while on Windows, it is passed to CMD exe. modulecmd.tcl redirects stdout to stderr because stdout will be evaluated for shell resolution. Returns the exit status of executing the command.

  system string
  

 

5, MODULEPATH

As mentioned in the third, MODULEPATH is used to find the configuration information directory. It can also be set to multiple paths

• Note: modulefile s with the same software name and version must not appear in all MODULEPATH paths

export MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/soft:/soft1:/soft2

 

6, Set default version

In module avail able, we can set the default version. For example, I want to set gcc 4.9.3 as the default version. When I load GCC, I don't need to write / 4.9.3 later, so I can directly load version 4.9.3. We can use it Version file to control this default information. Create in the same level directory of modulefile Version file

#%Module1.0
set ModulesVersion "4.9.3"

ls /soft/gcc

# 4.8.4  4.9.3  4.9.7

• Use module avail able to check and find that default is added after version 4.9.3