Lecture 11 — Cell Ranger: Raw Reads to Counts

Bill Cresko

Lecture 11: Cell Ranger — Raw Reads to Counts

Where this lecture fits

This is a bonus upstream module: it runs before the core pipeline, not after it
Most participants start from a counts matrix; this lecture explains where that matrix comes from
You arrive here when you have raw 10x data and must turn it into counts yourself
The hand-off target is Module 01 — the QC/preprocessing step that loads filtered_feature_bc_matrix/
Reading: Chapter 11 · Tutorial: Tutorial 11 · ATAC companion: Lecture 12

Goals of this lecture

Trace the path from sequencer to counts: BCL → FASTQ → matrix
Distinguish cellranger mkfastq from cellranger count
Understand the reference genome and why it lives in shared storage
Read the web_summary.html QC report critically
Know exactly which output file feeds Module 01
Scale one sample to many with SLURM arrays

Part 1 — Where raw data comes from

The sequencer doesn’t make FASTQs

An Illumina run writes BCL files — raw per-cycle base-call intensities, not reads
BCLs are organized by lane and cycle, not by sample or by molecule
Two things still have to happen before any biology:
- Demultiplex — split reads back to the sample they came from (by index)
- Assemble — stitch per-cycle calls into per-read FASTQ records

The full chain in one picture

BCL (Illumina raw intensities)
   │  cellranger mkfastq      (wraps bcl2fastq / BCL Convert)
   ▼
FASTQ (per lane, per sample)
   │  cellranger count        (the heavy lifting)
   ▼
outs/
   ├── web_summary.html              QC at a glance
   ├── metrics_summary.csv           QC numbers
   ├── filtered_feature_bc_matrix/   feeds Module 01 (Read10X)
   ├── raw_feature_bc_matrix/        feeds SoupX
   ├── possorted_genome_bam.bam      aligned reads
   └── molecule_info.h5              per-UMI info, used by aggr

Who runs what

In practice your sequencing core usually runs mkfastq and hands you FASTQs
So the step you own is almost always cellranger count
This lecture focuses there, but you should recognize the whole chain when something upstream breaks

Installation on Talapas

# Check the cluster module first
module avail cellranger      # e.g.  cellranger/9.0.0

# If not provided, install manually into /projects (NOT $HOME)
cd /projects/<PIRG>/<user>/software
wget -O cellranger-9.0.0.tar.gz "https://cf.10xgenomics.com/releases/cell-exp/cellranger-9.0.0.tar.gz?<token>"
tar -xzf cellranger-9.0.0.tar.gz
export PATH=/projects/<PIRG>/<user>/software/cellranger-9.0.0:$PATH

Always check module avail cellranger first — saves 3 GB of manual install
If installing manually, add the export PATH= line to ~/.bashrc so batch jobs find it

Part 2 — mkfastq vs count

`cellranger mkfastq`

Input: the BCL run folder + a sample sheet mapping index → sample
Job: demultiplex and convert to FASTQ
Output: FASTQs in the strict 10x naming pattern
<SAMPLE>_S<num>_L<lane>_<R1/R2/I1>_001.fastq.gz
The naming is not cosmetic — count parses sample and lane from it. Never rename.

`cellranger count`

Input: FASTQs + a reference transcriptome
Job: barcode correction → alignment → UMI dedup → cell calling → count matrix
Output: the outs/ directory, including the matrix Module 01 consumes
Single-process, multi-threaded (Rust + Python) — scale with cores, not MPI tasks

What `count` actually does inside

Barcode correction — fix 1-base errors against the known 10x whitelist
Alignment — STAR maps cDNA reads to the transcriptome
UMI deduplication — collapse PCR duplicates so each molecule counts once
Cell calling — separate real cells from empty droplets (knee + EmptyDrops-style)
Counting — build the genes × barcodes sparse matrix

Part 3 — The reference genome

What the reference is

A pre-built bundle: genome FASTA, gene annotation (GTF), and a STAR index
10x ships refdata-gex-GRCh38-... (human), refdata-gex-GRCm39-... (mouse)
Non-model species → build your own with cellranger mkref
Note the gex tag: this is the gene-expression reference (distinct from the ATAC one)

Where it lives — and why it matters

# Human GRCh38 reference (~12 GB compressed, ~30 GB extracted)
cd /projects/<PIRG>/<user>/refs
wget https://cf.10xgenomics.com/supp/cell-exp/refdata-gex-GRCh38-2024-A.tar.gz
tar -xzf refdata-gex-GRCh38-2024-A.tar.gz

Never in $HOME — small per-user quota on Talapas
Keep one canonical copy per group in /projects/<PIRG>/...; everyone shares it
Check module avail cellranger first — the cluster may already provide it

Part 4 — Key `count` parameters

The core invocation

cellranger count \
  --id=pbmc_1k_v3 \
  --transcriptome=/projects/<PIRG>/<user>/refs/refdata-gex-GRCh38-2024-A \
  --fastqs=/projects/<PIRG>/<user>/test_data/pbmc_1k_v3_fastqs \
  --sample=pbmc_1k_v3 \
  --create-bam=true \
  --localcores=16 \
  --localmem=120

Parameter by parameter

--id — names the output folder; pick something stable and traceable
--transcriptome — the reference directory (not the tarball)
--fastqs / --sample — where the reads are and which sample prefix to use
--create-bam — keep the aligned BAM (needed for some QC / re-analysis)
--localcores / --localmem — the internal compute budget

SLURM-vs-Cell-Ranger memory

SLURM --mem=128G is the hard ceiling; exceed it and the job is OOM-killed
--localmem=120 is Cell Ranger’s self-imposed budget
Set --localmem slightly below the SLURM allocation
The gap absorbs I/O buffers, index loading, and brief spikes
A silent “Killed” with no error message is almost always this

Threads, not tasks

Cell Ranger is one multi-threaded process — use --cpus-per-task=16
Not --ntasks=16 — that requests MPI processes Cell Ranger never uses
Match --localcores to --cpus-per-task

Part 5 — Reading the QC

`web_summary.html` is the single best QC doc

Open it first, before any R
It summarizes sequencing quality, mapping, cell calling, and the barcode-rank plot
Save it next to your raw data — reviewers will ask for it
metrics_summary.csv has the same numbers in machine-readable form

Red flags to scan for

Metric	Worry threshold
Estimated number of cells	Far off your target (over-loading / mis-call)
Mean reads per cell	< ~20,000 → undersequenced
Median genes per cell	< ~1,000 → poor capture or dying cells
Q30 in barcode	< 90% → poor sequencing
Q30 in RNA read	< 70% → poor sequencing
Reads mapped to transcriptome	< 60% → wrong reference / contamination

The barcode-rank (“knee”) plot

X = barcodes ranked by UMI count; Y = UMIs (log–log)
A clean run shows a sharp knee: real cells (high) drop steeply to empties (low)
A soft, gradual slope means ambiguous cell calling — inspect carefully
This plot is the visual story behind “Estimated number of cells”

Part 6 — The hand-off

The one directory that matters downstream

outs/filtered_feature_bc_matrix/
  ├── barcodes.tsv.gz
  ├── features.tsv.gz
  └── matrix.mtx.gz

This is exactly what Read10X() loads in Module 01:

library(Seurat)
counts <- Read10X("data/filtered_feature_bc_matrix/")
seu <- CreateSeuratObject(counts, project = "pbmc_1k_v3",
                          min.cells = 3, min.features = 200)
# … continue at Module 01 (QC metrics)

raw_feature_bc_matrix/ (all barcodes, pre-calling) is what SoupX needs
Everything downstream in the workshop begins from these three files

Part 7 — Scaling up

Don’t write N scripts — use a SLURM array

#SBATCH --array=1-8        # one sub-job per sample

# samples.txt: one sample ID per line
SAMPLE=$(sed -n "${SLURM_ARRAY_TASK_ID}p" samples.txt)

cellranger count \
  --id="$SAMPLE" \
  --transcriptome=/projects/<PIRG>/<user>/refs/refdata-gex-GRCh38-2024-A \
  --fastqs=/projects/<PIRG>/<user>/raw_fastqs/"$SAMPLE" \
  --sample="$SAMPLE" \
  --create-bam=true --localcores=16 --localmem=120

One row in samples.txt per sample; SLURM fans them out in parallel
Same script, scales from 1 to dozens of samples
cellranger aggr is a separate, rarely-needed depth-normalized merge — prefer Seurat merge() + integration

Common gotchas

FASTQ filename mismatch — keep the exact _S_L_R_001 pattern; symlink if renamed
Mixed chemistries in one --fastqs folder — Cell Ranger refuses; separate them
Silent “Killed” — almost always OOM; raise --mem and --localmem
Reference / chemistry mismatch — 5’ reads through a 3’ reference “run” but give garbage stats

Summary

Design considerations

Reference genome version matters — re-run with a newer GTF/genome if your annotation is out of date; old references reduce detected genes.
cellranger aggr vs Seurat merge — aggr depth-normalizes before combining; Seurat merge() + integration gives finer control for most workshop-style analyses. Prefer the Seurat path unless you specifically need aggr’s normalization.
FASTQ naming is non-negotiable — the <SAMPLE>_S<n>_L<lane>_<R/I>_001.fastq.gz pattern is parsed by count; symlink if your core handed you renamed files.
raw_feature_bc_matrix/ is what ambient-RNA tools like SoupX need; keep it alongside filtered_.

What to take away

BCL → FASTQ (mkfastq) → counts (count) — you usually own the last step
The reference is big, shared, and gex-tagged; keep it in /projects
--localmem < SLURM --mem; cores not tasks
Read web_summary.html first; trust the knee plot
filtered_feature_bc_matrix/ is the hand-off to Module 01
Scale with a SLURM array, not N scripts

Next steps

Read: Chapter 11 — From Raw Reads to a Counts Matrix
Do: Tutorial 11 — Cell Ranger count on Talapas
Then: pick up the core pipeline at Module 01
ATAC next: Lecture 12 — Raw scATAC-seq

Lecture 11 — Cell Ranger: Raw Reads to Counts

Lecture 11: Cell Ranger — Raw Reads to Counts

Where this lecture fits

Goals of this lecture

Part 1 — Where raw data comes from

The sequencer doesn’t make FASTQs

The full chain in one picture

Who runs what

Installation on Talapas

Part 2 — mkfastq vs count

cellranger mkfastq

cellranger count

What count actually does inside

Part 3 — The reference genome

What the reference is

Where it lives — and why it matters

Part 4 — Key count parameters

The core invocation

Parameter by parameter

SLURM-vs-Cell-Ranger memory

Threads, not tasks

Part 5 — Reading the QC

web_summary.html is the single best QC doc

Red flags to scan for

The barcode-rank (“knee”) plot

Part 6 — The hand-off

The one directory that matters downstream

Part 7 — Scaling up

Don’t write N scripts — use a SLURM array

Common gotchas

Summary

Design considerations

What to take away

Next steps

`cellranger mkfastq`

`cellranger count`

What `count` actually does inside

Part 4 — Key `count` parameters

`web_summary.html` is the single best QC doc