Reproducible Software Environments: venv, conda and renv
2026-11-23
The course book goes deeper on everything in this lecture. Read before or after class:
venv (Python), conda (anything), renv (R) - and the record file each one leavesconda in depth: channels, Miniforge, environment.yml| Tool | For | Environments live… | The record you share |
|---|---|---|---|
venv (+ pip) |
Python | in the project folder (.venv/) |
requirements.txt |
conda / mamba |
Python plus compiled tools (samtools, even R) |
centrally, by name (conda env list) |
environment.yml |
renv |
R | in the project folder | renv.lock |
venv is the Python default on your laptop - you have used it since the bootcampconda is the standard in bioinformatics and on Talapas (next week’s cluster) - one tool for any softwarerenv for R projects: renv::init() once, renv::snapshot() to record, renv::restore() to rebuildconda in practice - on your own computer# Install once: Miniforge, from https://conda-forge.org/download/
conda create -n rnaseq python=3.12 pandas matplotlib # make a named environment - once
conda activate rnaseq # turn it on - every session
conda install -c bioconda samtools # compiled genomics tools too!
conda env export > environment.yml # record it, to rebuild anywhere
conda deactivate # turn it offconda env list shows them allconda-forge (general) and bioconda (genomics)conda-forgemamba is its older drop-in cousin, still seen in cluster docsenvironment.yml rebuilds this environment on Talapas: conda env create -f environment.ymlvenv in practice - the Python defaultpython3 -m venv .venv # create, once - lives INSIDE the project folder
source .venv/bin/activate # turn it on - every session; prompt shows (.venv)
pip install pandas matplotlib # installs land in the environment, nowhere else
pip freeze > requirements.txt # record it - the recipe a collaborator rebuilds from
deactivate # turn it off.venv/ folder - delete the folder, the environment is gonerequirements.txt is the pip-world counterpart of environment.yml: pip install -r requirements.txt rebuildssamtools, bwa), you need condaenvironment.ymlWith the environment active:
--from-history records only what you asked for, not all ~200 resolved dependencies - which keeps the file short and portable across operating systemssed drops the machine-specific prefix: line - this is the exact form the Talapas documentation recommends for sharingenvironment.yml in each project’s repository, next to the code. It is your collaborator instructions, your Talapas deployment recipe, and the environment section of your methods - all one filerenv: the same idea for R projectsvenv - the library and renv.lock live in the project folderrenv::init() records exact package versions in the project foldermodule load miniconda3/20240410
# Best: rebuild the environment you recorded on your laptop (previous slides)
conda env create -f environment.yml # once
conda activate rnaseq # every session / job
# or create one from scratch on the cluster:
conda create -n my_env python=3.11 pandas matplotlib biopython
# or, with the plain python3 module and a virtual environment:
module load python3/3.11.4
python3 -m venv ~/envs/my_env && source ~/envs/my_env/bin/activateconda (with the bioconda channel) installs compiled bioinformatics tools too - conda install -c bioconda samtoolsmodule load + conda activate lines inside your sbatch script - every batch job starts with a clean shellconda env list to see them, conda clean -a to reclaim cache space, and mind your quotaThe whole reason this lecture exists:
conda env export --from-history | sed '/prefix:/d' > environment.yml; commit it to the project repomodule load miniconda3, then conda env create -f environment.yml#!/bin/bash
#SBATCH --job-name=rnaseq_run
#SBATCH --account=<your_pirg>
#SBATCH --partition=compute
#SBATCH --time=02:00:00
#SBATCH --mem=8G
module load miniconda3/20240410 # every batch job starts with a clean shell,
conda activate rnaseq # so BOTH lines go inside the script
python3 scripts/analysis.py data/Stickle_RNAseq.tsv results/Tip
Last week’s rule still holds: request only what you need. This week’s addition: record what you ran it with. Together they are the recipe for results you can defend.
| Tool | Family | One-line verdict |
|---|---|---|
| mamba | conda | mostly obsolete - modern conda uses its fast solver |
| micromamba | conda | conda as one tiny binary, for containers and CI |
| pixi | conda | per-project conda with a lockfile - watch this one |
| uv | PyPI-only | very fast venv+pip replacement, pure Python only |
samtools? Only the conda family can| Layer | Isolates | Travels between clusters? |
|---|---|---|
| Environment (venv/conda/renv) | packages + interpreter | yes, via the record file (rebuilt) |
| Container (Apptainer/Docker) | an entire operating system + software stack | yes, as one image file (bit-identical) |
environment.yml, destroy it, and rebuild it from the file (Ex 9 walks through every step) - then commit the file to last week’s repositorypandas.__version__ from a compute node (Ex 10 closes the loop)environment.yml, and do all remaining project computing inside itvenv + requirements.txt for pure Python; conda + environment.yml for anything (and everything bioinformatic); renv + renv.lock for Renvironment.yml works on your laptop and on Talapas - that loop is reproducible computingBioE Computational Tools 2026 - Knight Campus