Tutorial 11 — Raw Data Download + Cell Ranger `count` (bonus, upstream)

BCL → FASTQ → counts on a SLURM cluster — the raw-data step that feeds the core pipeline

Author

Single Cell RNA-seq Workshop

Running the code in this tutorial

Every code chunk here is tagged with #| eval: false, so the published page shows the code without running it. To run it yourself, open the downloaded .qmd in RStudio:

Run one chunk: click the green ▶ (Run Current Chunk) at the chunk’s top-right, or press Ctrl/Cmd + Enter. This runs the chunk regardless of its eval setting — the easiest way to work through the tutorial interactively.
Run a chunk on render: change that chunk’s #| eval: false to #| eval: true (or delete the line) so it executes when the document is rendered.
Render the whole tutorial: click the Render button in RStudio (or run quarto render in a terminal) to execute the chunks top-to-bottom and knit a finished HTML. To run everything on render, set eval: true once in the YAML header at the top.

Important

This is a bonus upstream Talapas exercise, not a laptop tutorial. It runs before the core pipeline — it produces the counts matrix the core Module 01 consumes. Cell Ranger requires a ~30 GB human reference genome, multi-GB FASTQs, and a multi-core machine with 64+ GB RAM. Do not try to run it on your laptop. Work through the laptop sequence (core tutorials 01–08) first; come here when you have your own raw 10x data and need to turn it into a counts matrix on Talapas.

About this tutorial

The companion to scNotebooks Module 03 (their Cell Ranger walkthrough), adapted for SLURM on Talapas. You’ll learn:

Note

Companion lecture: Lecture 11 — Cell Ranger: Raw Reads to Counts · Companion reading: Chapter 11 — Cell Ranger: From Raw Reads to a Counts Matrix

How a 10x sequencing run goes from BCL → FASTQ (cellranger mkfastq) and FASTQ → counts (cellranger count)
How to install Cell Ranger and the human reference on Talapas
How to write a batch SLURM script to run cellranger count on a single sample
How to scale to many samples with a SLURM array job
How to inspect the Cell Ranger outputs (the same filtered_feature_bc_matrix/ directory that Read10X() consumes)
How to feed the outputs into the workshop’s R-based downstream tutorials

Why this matters

Most workshop participants will start from filtered_feature_bc_matrix/, because that’s what your sequencing core or your collaborator hands you. But the moment you need to:

Re-run on a newer reference genome,
Add a missing sample to the same study,
Process a non-standard chemistry (e.g. 5’-seq, multiome, fixed-cell),
Demultiplex CMO-multiplexed samples,

you’ll be running Cell Ranger yourself. This exercise gives you the playbook so the first time isn’t on a deadline.

Prerequisites

A Talapas account (request one) and basic familiarity with SLURM (see the SLURM Basics tutorial)
~50 GB of free space in your /projects/<PIRG>/ quota for the human reference + intermediate files
Either your own 10x FASTQs or the small public test dataset linked below

Cell Ranger overview

BCL (Illumina raw)
   │
   │  cellranger mkfastq   (uses bcl2fastq under the hood)
   ▼
FASTQ (per lane, per sample)
   │
   │  cellranger count     (this is the heavy lifting)
   ▼
Cell Ranger outs/
   ├── web_summary.html              ← QC at a glance
   ├── metrics_summary.csv           ← QC numbers
   ├── filtered_feature_bc_matrix/   ← what the core pipeline's `01_qc_preprocessing.R` (Read10X) loads
   ├── raw_feature_bc_matrix/        ← what SoupX needs
   ├── possorted_genome_bam.bam      ← aligned reads
   └── molecule_info.h5              ← per-UMI info, used by aggr

For most users, the sequencing core runs cellranger mkfastq and hands you the FASTQs. This tutorial focuses on cellranger count, which is where you start.

Step 1 — Install Cell Ranger on Talapas

# In your projects directory (NOT in $HOME — Cell Ranger is ~3 GB)
cd /projects/<PIRG>/<your-username>

mkdir -p software && cd software

# Get the latest Cell Ranger from 10x (the URL has a temporary signed token,
# so generate it fresh from https://www.10xgenomics.com/support/software/cell-ranger/downloads).
# Replace the URL below with the freshly-generated one.
wget -O cellranger-9.0.0.tar.gz "https://cf.10xgenomics.com/releases/cell-exp/cellranger-9.0.0.tar.gz?<token>"
tar -xzf cellranger-9.0.0.tar.gz

# Add to PATH (put this in your ~/.bashrc so future shells find it)
export PATH=/projects/<PIRG>/<your-username>/software/cellranger-9.0.0:$PATH
which cellranger
cellranger --version

Tip

On Talapas, cellranger may already be available as a module — check with module avail cellranger. If so, skip the manual install: module load cellranger/9.0.0.

Step 2 — Download the reference genome

cd /projects/<PIRG>/<your-username>/refs

# Human GRCh38 (current release)
wget https://cf.10xgenomics.com/supp/cell-exp/refdata-gex-GRCh38-2024-A.tar.gz
tar -xzf refdata-gex-GRCh38-2024-A.tar.gz

# For mouse: refdata-gex-GRCm39-2024-A.tar.gz
# For Drosophila / other species: build a custom reference with `cellranger mkref`
ls refdata-gex-GRCh38-2024-A/
# fasta/  genes/  reference.json  star/

Think about it

The reference is ~12 GB compressed and ~30 GB extracted. Why would you NOT put it in your $HOME?

Show answers

$HOME on Talapas has a small per-user quota. Reference data goes in /projects/<PIRG>/... where your group’s PI has shared quota. Keep one canonical copy of each reference per group; multiple users sharing one reference is the norm.

Step 3 — A test dataset to practice on

10x Genomics ships small public datasets for exactly this — test-running the pipeline. The smallest is PBMC 1k v3 (~5 GB FASTQ, runs in ~30 minutes):

cd /projects/<PIRG>/<your-username>/test_data

wget https://s3-us-west-2.amazonaws.com/10x.files/samples/cell-exp/3.0.0/pbmc_1k_v3/pbmc_1k_v3_fastqs.tar
tar -xf pbmc_1k_v3_fastqs.tar
ls pbmc_1k_v3_fastqs/
# pbmc_1k_v3_S1_L001_I1_001.fastq.gz
# pbmc_1k_v3_S1_L001_R1_001.fastq.gz
# pbmc_1k_v3_S1_L001_R2_001.fastq.gz
# pbmc_1k_v3_S1_L002_I1_001.fastq.gz   ...

The naming is strict. Cell Ranger uses the pattern <SAMPLE>_S<num>_L<lane>_<RNA-or-index>_001.fastq.gz. Don’t rename the files.

Step 4 — A SLURM batch script for one sample

Create run_cellranger_count.sbatch:

#!/bin/bash
#SBATCH --job-name=cellranger_count_pbmc1k
#SBATCH --partition=compute
#SBATCH --account=<your_PIRG>
#SBATCH --time=04:00:00
#SBATCH --nodes=1
#SBATCH --cpus-per-task=16
#SBATCH --mem=128G
#SBATCH --output=logs/cellranger_%j.out
#SBATCH --error=logs/cellranger_%j.err
#SBATCH --mail-type=END,FAIL
#SBATCH --mail-user=<your_email>@uoregon.edu

# Load the module if Talapas provides one, OR add Cell Ranger to PATH manually:
# module load cellranger/9.0.0
export PATH=/projects/<PIRG>/<your-username>/software/cellranger-9.0.0:$PATH

# Change into the parent directory where the run-output should be written
cd /projects/<PIRG>/<your-username>/cellranger_runs

cellranger count \
  --id=pbmc_1k_v3 \
  --transcriptome=/projects/<PIRG>/<your-username>/refs/refdata-gex-GRCh38-2024-A \
  --fastqs=/projects/<PIRG>/<your-username>/test_data/pbmc_1k_v3_fastqs \
  --sample=pbmc_1k_v3 \
  --create-bam=true \
  --localcores=16 \
  --localmem=120

Submit it:

mkdir -p logs
sbatch run_cellranger_count.sbatch

# Watch the job
squeue -u $USER
tail -f logs/cellranger_*.out

Think about it

Why do we tell SLURM --mem=128G but tell Cell Ranger --localmem=120?
Why --cpus-per-task=16 not --ntasks=16?

Show answers

Cell Ranger’s --localmem is the budget it allocates internally. Always set it slightly lower than the SLURM allocation so that I/O buffers, pre-loaded indexes, and brief spikes don’t push the job above its hard SLURM limit and trigger an OOM-kill.
Cell Ranger is a single-process multi-threaded program (Rust + Python). It scales by threads inside one task, not by separate MPI tasks. Use --cpus-per-task to give it cores; --ntasks would request multiple MPI processes that Cell Ranger doesn’t use.

Step 5 — Inspect the output

When the job finishes (~30–60 min for PBMC 1k):

cd /projects/<PIRG>/<your-username>/cellranger_runs/pbmc_1k_v3/outs
ls -la
# web_summary.html          ← open in a browser via SSH-tunneled X11 or scp it down
# metrics_summary.csv       ← cat or column -t
# filtered_feature_bc_matrix/
# raw_feature_bc_matrix/
# possorted_genome_bam.bam (+ .bai)
# molecule_info.h5
# analysis/                 ← Cell Ranger's automatic clustering (rough, ignore)

cat metrics_summary.csv | column -t -s,

The web_summary.html is the first thing to look at. Key red flags:

Metric	Worry threshold
Estimated number of cells	Outside expected target (over-loading or mis-calling)
Mean reads per cell	Below ~20,000 → undersequenced
Median genes per cell	Below ~1,000 → poor capture or dying cells
Q30 bases in barcode	Below 90% → poor sequencing
Q30 bases in RNA read	Below 70% → poor sequencing
Reads mapped confidently to transcriptome	Below 60% → wrong reference or contamination

Tip

The web_summary.html from a run is the single most informative QC document in the entire pipeline. Save it alongside your raw data; reviewers will ask for it.

Step 6 — Hand off to the workshop tutorials

Copy the filtered_feature_bc_matrix/ directory into a place the laptop tutorials can find it:

# From your local machine:
scp -r <username>@login.talapas.uoregon.edu:/projects/<PIRG>/<your-username>/cellranger_runs/pbmc_1k_v3/outs/filtered_feature_bc_matrix \
        ~/Documents/scrnaseq_workshop/data/

Now from R:

library(Seurat)
counts <- Read10X("~/Documents/scrnaseq_workshop/data/filtered_feature_bc_matrix/")
seu    <- CreateSeuratObject(counts, project = "pbmc_1k_v3",
                              min.cells = 3, min.features = 200)
# … pick up at the core pipeline (Module 10), script 01 (QC metrics)

Step 7 — Scale to many samples with a SLURM array

When you have N samples (one FASTQ folder per sample), don’t write N scripts. Use a SLURM array job:

#!/bin/bash
#SBATCH --job-name=cellranger_count_array
#SBATCH --partition=compute
#SBATCH --account=<your_PIRG>
#SBATCH --time=06:00:00
#SBATCH --nodes=1
#SBATCH --cpus-per-task=16
#SBATCH --mem=128G
#SBATCH --output=logs/cr_%A_%a.out
#SBATCH --array=1-8         # one sub-job per sample (edit to your N)

export PATH=/projects/<PIRG>/<your-username>/software/cellranger-9.0.0:$PATH

# samples.txt has one sample ID per line, in the same order you'd want them processed
SAMPLE=$(sed -n "${SLURM_ARRAY_TASK_ID}p" samples.txt)

cd /projects/<PIRG>/<your-username>/cellranger_runs

cellranger count \
  --id="$SAMPLE" \
  --transcriptome=/projects/<PIRG>/<your-username>/refs/refdata-gex-GRCh38-2024-A \
  --fastqs=/projects/<PIRG>/<your-username>/raw_fastqs/"$SAMPLE" \
  --sample="$SAMPLE" \
  --create-bam=true \
  --localcores=16 \
  --localmem=120

samples.txt:

sample_01
sample_02
sample_03
...
sample_08

Submit with sbatch run_cellranger_count_array.sbatch. SLURM runs all 8 sub-jobs (subject to your group’s resource limits, often 4–8 in parallel).

Step 8 — Aggregate multiple runs (optional, when needed)

If you have multiple Cell Ranger runs from the same library prep that need to be count-matrix-merged (not the same as Seurat’s downstream merge), use cellranger aggr:

# aggr_csv.csv:
# library_id,molecule_h5
# sample_01,/projects/.../sample_01/outs/molecule_info.h5
# sample_02,/projects/.../sample_02/outs/molecule_info.h5

cellranger aggr \
  --id=combined \
  --csv=aggr_csv.csv \
  --normalize=mapped

For most workshop-style analyses you do not want aggr — Seurat’s merge() + integration (Tutorial 05) gives you finer control. Use aggr only when you have technical duplicates of one library and need a depth-normalized merge before any analysis.

Common gotchas

FASTQ filename mismatch. Cell Ranger requires the exact <SAMPLE>_S<n>_L<lane>_R<1or2>_001.fastq.gz pattern. Don’t rename. If your core hands you renamed files, symlink them back.
Mixed chemistries in one folder. Cell Ranger refuses to run if --fastqs contains files from different chemistries (3’ v2 + v3). Separate them.
The “killed” exit with no error message. Almost always SLURM-killed for OOM. Bump --mem and --localmem.
Reference / chemistry mismatch. Running 5’ chemistry through a 3’ reference works but gives garbage stats. Match --chemistry to your library if you’re not auto-detected.