Install Software HPC

Article author
Jillian Rowe
  • Updated

(Please note that your HPC may have different locations for filesystems. Please consult the articles in your organization for your own specifics.)

Software installation has 3 main levels.

  1. Software Stacks Available to all users as Modules

  2. Software Stacks Available per project

  3. Software Stacks customized per user

Modules

On an HPC system software is typically installed as modules.

module avail

Will give you a list of the possible software modules.

# Load the module
module load my-module
# Unload only one module
module unload my-module
# Unload all modules
module purge

Install new software

The recommended way to install software on the cluster is through Conda. The easiest way to search for conda packages is through google, but you can also use conda search.

module load Miniconda3

You can then install software using conda. It is always a good idea to keep your software environments as small and isolated as possible to keep softwares from clobbering eachother.

Install sofware directly with conda

conda install -p /apps/users/$USER/software-name package-1 package-2
# or by name
conda create -n some-name package-1 package-2

 

Once it’s installed conda will tell you to use conda activate /path/to/env or name-of-env. Don’t do that. Use source activate name-of-env or source activate path-to-env.

Share software environments

The easiest way to share a software environment is to install it to a shared directory such as /scratch/project

 

Clean up

After you’ve installed software it’s a good idea to clean out the package cache.

conda clean -y --all
 

Troubleshoot

The first thing to try is to clean out your package cache.

conda clean -y --all

 

If you are getting errors about packages or package manifests being corrupted, and you are using any modules you do not have write access to (Miniconda3) you will have to ask an admin to clean out the package cache there.

If you are trying to create an environment and it’s being killed, it is probably because the dependency tree of the packages you are trying to solve is too large for that node. Submit a SLURM job for a higher memory node, either through your Jupyterhub or through the command line, and install there.

Was this article helpful?

Comments

0 comments

Article is closed for comments.

Still have questions?

Submit a request