Installing Software with Conda

Conda (via Miniconda) lets you install and manage software in your own user space without needing admin access. This is the recommended way to install tools that are not available system-wide.

• You need a tool that is not installed on the cluster.
• You need a specific version of a tool.
• You need an isolated environment to avoid conflicts between tools.
• You want to reproduce an environment from a collaborator.

If the tool you need is already installed system-wide, you do not need Conda. Check with which <tool_name> first.

Miniconda is a minimal Conda installer. You only need to do this once.

# Download the installer
wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh
 
# Run the installer
bash Miniconda3-latest-Linux-x86_64.sh

During installation:

1. Press Enter to review the license, then type yes to accept.
2. Press Enter to accept the default install location (~/miniconda3), or specify a custom path.
3. When asked “Do you wish the installer to initialize Miniconda3 by running conda init?”, type no.

Important: Do NOT let Conda modify your ~/.bashrc. Conda initialization in bashrc runs every time you log in or open a shell, significantly slowing down SSH logins. Instead, activate Conda manually when you need it (see below).

After installation, verify it works by sourcing it manually:

source ~/miniconda3/etc/profile.d/conda.sh
conda --version
conda list

rm Miniconda3-latest-Linux-x86_64.sh

Since Conda is not in your bashrc, you need to activate it manually each time you want to use it:

source ~/miniconda3/etc/profile.d/conda.sh
conda activate myproject

You can create a short alias in your ~/.bashrc for convenience:

alias conda-init='source ~/miniconda3/etc/profile.d/conda.sh'

Then when you need Conda, just type:

conda-init
conda activate myproject

Always create a separate environment for each project or tool. Do not install everything into the base environment.

# Create a new environment named "myproject" with Python 3.11
conda create -n myproject python=3.11
 
# Activate it
conda activate myproject

Your prompt changes to (myproject), indicating you are in the environment.

# Install from the default channel
conda install numpy pandas matplotlib
 
# Install from the bioconda channel (bioinformatics tools)
conda install -c bioconda -c conda-forge bwa samtools
 
# Install a specific version
conda install -c bioconda gatk4=4.4.0.0

Most bioinformatics tools are available through the bioconda channel. Always include -c conda-forge along with -c bioconda for dependency resolution:

conda install -c bioconda -c conda-forge fastqc multiqc fastp

To search for a tool:

conda search -c bioconda <tool_name>

Or visit bioconda.github.io to browse available packages.

# List all environments
conda env list
 
# Activate an environment
conda activate myproject
 
# Deactivate (go back to base)
conda deactivate
 
# Remove an environment entirely
conda env remove -n myproject
 
# List packages in the current environment
conda list

To share your environment with a collaborator:

# Export to a YAML file
conda env export > environment.yml
 
# The collaborator can recreate it with:
conda env create -f environment.yml

When submitting Slurm jobs, you need to activate the Conda environment inside your job script:

#!/bin/bash
#SBATCH --mem=10gb
#SBATCH --cpus-per-task=4
#SBATCH --output=log/my_job_%j.log
 
# Initialize conda for the batch shell
source ~/miniconda3/etc/profile.d/conda.sh
 
# Activate the environment
conda activate myproject
 
# Now you can use tools from this environment
fastqc -o fastqc_results/ fastq/*.fq.gz
 
echo "Done!"

Important: The source ~/miniconda3/etc/profile.d/conda.sh line is necessary because (a) Slurm jobs start a non-interactive shell, and (b) at ABI we do not add Conda to ~/.bashrc to keep logins fast.

• Do not install tools in the base environment. Create separate environments per project.
• Use conda-forge as a default channel. It has better dependency resolution:

conda config --add channels conda-forge
conda config --set channel_priority strict

• Use mamba for faster installs (optional):

conda install -n base -c conda-forge mamba
mamba install -c bioconda bwa    # Much faster than conda install

• Storage considerations: Conda environments can become large. Your home directory has only ~12G quota. If your environments grow large, consider installing Miniconda under /mnt/nas0/user/<your_username>/miniconda3 instead (which has ~100G). Check your disk usage with:

du -sh ~/miniconda3/
du -sh ~/miniconda3/envs/*/

• Conda Getting Started Guide
• Bioconda -- Bioinformatics packages for Conda
• Software Overview
• Using Slurm – how to use Conda environments in Slurm jobs