Installing Anaconda packages more quickly using Mamba¶

The ITSR support team often receive tickets from Anaconda users concerned that creating environments and installing packages is taking quite a long time. We recently installed Miniconda as a module on Apocrita, which enables users to install packages using the Mamba libsolver.

What is Mamba?¶

Anaconda has traditionally used conda as its package manager, but over time it has become quite slow at resolving dependencies when installing packages. mamba is a re-implementation of conda written in C++ offering the following benefits:

parallel downloading of repository data and package files using multi-threading
libsolv for much faster dependency solving, a state of the art library used in the RPM package manager of Red Hat, Fedora and OpenSUSE
core parts of mamba are implemented in C++ for maximum efficiency

At the same time, mamba utilises the same command line parser, package installation code and transaction verification routines as conda to stay as compatible as possible.

Why Mamba?¶

Put simply, in a lot of cases mamba is a lot faster at installing packages when compared to conda:

To give another example, recently some Apocrita users were attempting to install the busco package using Anaconda and the conda package manager, and after two hours it still hadn't resolved all the required dependencies.

By using mamba instead, busco takes less than five minutes to install.

Installing packages with Mamba¶

So, to use mamba to install busco:

1) Load the miniconda module:

module load miniconda

2) Create a new conda env (I've called mine busco but it can be called anything you like):

mamba create --quiet --yes --name busco

3) Activate the environment:

mamba activate busco

4) Install busco (version 5.1.1) using mamba:

mamba install -c conda-forge -c bioconda busco=5.1.1

mamba will now rapidly pull all required dependencies and ask you to confirm the package installation list (which you can do by typing y and pressing Enter). The installation should then complete within just a few minutes, and busco will be available in your env:

(busco) $ busco -v
BUSCO 5.1.1
(busco) $ busco -h
usage: busco -i [SEQUENCE_FILE] -l [LINEAGE] -o [OUTPUT_NAME] -m [MODE] [OTHER OPTIONS]
...

Can I use Mamba with the Anaconda modules on Apocrita?¶

Sometimes the miniconda Apocrita module doesn't offer users all of the things the anaconda3 module does. Using mamba with anaconda3 is possible, but only if you use the most recent anaconda3/2022.05 module (mamba isn't available in older versions). You can't run mamba commands directly using this module, but instead you have to add --experimental-solver=libmamba to your conda install commands. Future versions of Anaconda may integrate mamba so that it can be used directly - we'll update this blog page if/when Anaconda natively supports mamba without the --experimental-solver=libmamba extra switch.

So, once again using the busco package as an example:

1) Load the anaconda3/2022.05 module:

module load anaconda3/2022.05

2) Create a new conda env (I've called mine busco but it can be called anything you like):

conda create --quiet --yes --name busco

3) Activate the environment:

conda activate busco

4) Install busco (version 5.1.1) using conda and the --experimental-solver=libmamba flag:

conda install -c conda-forge -c bioconda busco=5.1.1 --experimental-solver=libmamba

mamba will now rapidly pull all required dependencies and ask you to confirm the package installation list (which you can do by typing y and pressing Enter). The installation should then complete within just a few minutes, and busco will be available in your env:

(busco) $ busco -v
BUSCO 5.1.1
(busco) $ busco -h
usage: busco -i [SEQUENCE_FILE] -l [LINEAGE] -o [OUTPUT_NAME] -m [MODE] [OTHER OPTIONS]
...

You can find more information about mamba on the project's official docs page, as well as on the QMUL Miniconda docs page.

We hope you find these tips useful. As usual, you can ask a question on our Slack channel (QMUL users only), or by sending an email to its-research-support@qmul.ac.uk which is handled directly by staff with relevant expertise.

Title image: Generated using Stable Diffusion
Comparison video: Taken from the Anaconda Blog