Table of Contents
Installing Software with Conda
Conda (via Miniconda) lets you install and manage software in your own user space without needing admin access. This is the recommended way to install tools that are not available system-wide.
When to Use Conda
- You need a tool that is not installed on the cluster.
- You need a specific version of a tool.
- You need an isolated environment to avoid conflicts between tools.
- You want to reproduce an environment from a collaborator.
If the tool you need is already installed system-wide, you do not need Conda. Check with which <tool_name> first.
Step 1: Install Miniconda
Miniconda is a minimal Conda installer. You only need to do this once.
# Download the installer wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh # Run the installer bash Miniconda3-latest-Linux-x86_64.sh
During installation:
- Press Enter to review the license, then type yes to accept.
- Press Enter to accept the default install location (
~/miniconda3), or specify a custom path. - When asked “Do you wish the installer to initialize Miniconda3 by running conda init?”, type
no.
Important: Do NOT let Conda modify your~/.bashrc. Conda initialization in bashrc runs every time you log in or open a shell, significantly slowing down SSH logins. Instead, activate Conda manually when you need it (see below).
After installation, verify it works by sourcing it manually:
source ~/miniconda3/etc/profile.d/conda.sh conda --version conda list
Clean up the installer
rm Miniconda3-latest-Linux-x86_64.sh
Activating Conda
Since Conda is not in your bashrc, you need to activate it manually each time you want to use it:
source ~/miniconda3/etc/profile.d/conda.sh conda activate myproject
You can create a short alias in your ~/.bashrc for convenience:
alias conda-init='source ~/miniconda3/etc/profile.d/conda.sh'
Then when you need Conda, just type:
conda-init conda activate myproject
Step 2: Create an Environment
Always create a separate environment for each project or tool. Do not install everything into the base environment.
# Create a new environment named "myproject" with Python 3.11 conda create -n myproject python=3.11 # Activate it conda activate myproject
Your prompt changes to (myproject), indicating you are in the environment.
Step 3: Install Packages
# Install from the default channel conda install numpy pandas matplotlib # Install from the bioconda channel (bioinformatics tools) conda install -c bioconda -c conda-forge bwa samtools # Install a specific version conda install -c bioconda gatk4=4.4.0.0
Bioconda
Most bioinformatics tools are available through the bioconda channel. Always include -c conda-forge along with -c bioconda for dependency resolution:
conda install -c bioconda -c conda-forge fastqc multiqc fastp
To search for a tool:
conda search -c bioconda <tool_name>
Or visit bioconda.github.io to browse available packages.
Managing Environments
# List all environments conda env list # Activate an environment conda activate myproject # Deactivate (go back to base) conda deactivate # Remove an environment entirely conda env remove -n myproject # List packages in the current environment conda list
Export and share an environment
To share your environment with a collaborator:
# Export to a YAML file conda env export > environment.yml # The collaborator can recreate it with: conda env create -f environment.yml
Using Conda with Slurm
When submitting Slurm jobs, you need to activate the Conda environment inside your job script:
#!/bin/bash #SBATCH --mem=10gb #SBATCH --cpus-per-task=4 #SBATCH --output=log/my_job_%j.log # Initialize conda for the batch shell source ~/miniconda3/etc/profile.d/conda.sh # Activate the environment conda activate myproject # Now you can use tools from this environment fastqc -o fastqc_results/ fastq/*.fq.gz echo "Done!"
Important: The source ~/miniconda3/etc/profile.d/conda.sh line is necessary because (a) Slurm jobs start a non-interactive shell, and (b) at ABI we do not add Conda to ~/.bashrc to keep logins fast.
Tips
- Do not install tools in the base environment. Create separate environments per project.
- Use
conda-forgeas a default channel. It has better dependency resolution:
conda config --add channels conda-forge conda config --set channel_priority strict
- Use
mambafor faster installs (optional):
conda install -n base -c conda-forge mamba mamba install -c bioconda bwa # Much faster than conda install
- Storage considerations: Conda environments can become large. Your home directory has only ~12G quota. If your environments grow large, consider installing Miniconda under
/mnt/nas0/user/<your_username>/miniconda3instead (which has ~100G). Check your disk usage with:
du -sh ~/miniconda3/ du -sh ~/miniconda3/envs/*/
Troubleshooting
| Problem | Solution |
|---|---|
conda: command not found | Run source ~/miniconda3/etc/profile.d/conda.sh to activate Conda for this session. |
(base) not showing in prompt | This is expected at ABI since we do not add Conda to bashrc. Run source ~/miniconda3/etc/profile.d/conda.sh first. |
| Package conflict / solver hangs | Try using mamba instead of conda. Or create a fresh environment. |
| Disk quota exceeded | Clean unused packages: conda clean –all. Remove old environments. |
Tool not found after conda activate | Make sure you installed the tool in the correct environment: conda list | grep <tool> |
See Also
- Using Slurm – how to use Conda environments in Slurm jobs
