Appendix F: Common Bioinformatics File Formats

A field guide to the text and binary formats you will meet in bulk and single-cell analyses

Author

Single Cell RNA-seq Workshop

Overview

Almost every bioinformatics tool you will run is a format converter. A sequencer emits FASTQ, an aligner turns FASTQ into SAM/BAM, a variant caller turns BAM into VCF, a counter turns BAM + GTF into a counts matrix, a peak caller turns BAM into narrowPeak/BigWIG. Read errors, misalignments, and silent data loss almost always trace back to one of these conversions.

This appendix is a reference for the dozen or so formats you will see most often. For each format the structure is the same:

A head of a real example, with the tags and flag fields visible.
A prose description of what the format is for, when it appears in a pipeline, and when not to use it.
A bullet-pointed walk-through of the fields, keyed to specific lines or columns in the example.

Tip

How to read this appendix. You do not need to memorize every field. You do need to know (a) which format goes with which step of the pipeline, (b) what gets lost when you convert between formats, and (c) where in the file the metadata lives so you can grep / awk a quick sanity check.

Note

Companion reading. Appendix E covers the single-cell-specific container formats (.h5, .h5ad, .rds, Matrix Market) in the context of public databases. This appendix overlaps slightly but is organized around the file format itself rather than the repository.

A short taxonomy

Layer	Format	Purpose
Raw sequence	FASTA, FASTQ	Sequences (with / without quality scores)
Alignments	SAM, BAM, CRAM	Reads aligned to a reference
Annotation	GFF3, GTF, BED	Genomic features (genes, exons, peaks, regions)
Variants	VCF, BCF	Per-position variant calls and genotypes
Signal tracks	Wig, BedGraph, BigWIG, bigBed, narrowPeak	Per-base or per-interval quantitative tracks
Count matrices	TSV/CSV, MatrixMarket (.mtx), HDF5 (.h5), .h5ad, .loom, .rds	Gene × sample or gene × cell count tables
Single-cell specific	fragments.tsv.gz, 10x feature-barcode matrix	scATAC fragments; 10x triplet directory

Most pipelines move data left-to-right and top-to-bottom across this table.

1. FASTA — reference sequences

FASTA is the lowest-common-denominator format for biological sequence: a header line and the sequence itself. It carries no quality information, so it is used for references (genomes, transcriptomes, proteomes) and for assembled / consensus sequences, not for raw reads.

Example

>chr1 dna:chromosome chromosome:GRCh38:1:1:248956422:1 REF
NNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNN
NNNNNNNNNNTAACCCTAACCCTAACCCTAACCCTAACCCTAACCCTAACCCTAACCCTA
ACCCTAACCCTAACCCTAACCCTAACCCTAACCCTAACCCTAACCCTAACCCTAACCCTA
>chr2 dna:chromosome chromosome:GRCh38:2:1:242193529:1 REF
NNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNN
CGTATCCCACACACCACACCCACACACCACACCCACACACACCACACCCACACACACACA
>ENST00000456328.2 cdna chromosome:GRCh38:1:11869:14409:1
GTTAACTTGCCGTCAGCCTTTTCTTTGACCTCTTCTTTCTGTTCATGTGTATTTGCTGTC
TCTTAGCCCAGACTTCCCGTGTCATTTTGCCCCAGGGGTCAATATTGTCTGGGCTCCGAA

What it is and when you see it

Reference genomes (GRCh38.primary_assembly.genome.fa) for read alignment.
Reference transcriptomes (gencode.v44.transcripts.fa) for salmon / kallisto quantification.
Protein databases for BLAST.
Assembly outputs (assembly.fasta).

Field walk-through

Line 1 (>chr1 ...) — the header line. Every header starts with > in the first column. The first whitespace-delimited token (chr1) is the sequence ID; the rest is free-text description. Tools that index a FASTA (e.g. samtools faidx) use only the ID, so make sure IDs are unique and short.
Lines 2–4 — the sequence, hard-wrapped (here at 60 characters). Whitespace and line breaks are not significant; tools concatenate the lines internally. Characters are IUPAC codes (ACGTN plus ambiguity codes like R, Y, W).
N runs represent unknown bases — gaps in the assembly, masked regions, etc.
Line 4 (>chr2 ...) — start of the next record. There is no end-of-record marker; the next > is the marker.
Last header (>ENST00000456328.2 ...) — same format used for transcripts; the ID is now an Ensembl transcript ID and the description encodes coordinates on the parent chromosome.

Warning

FASTA IDs are sticky. The exact ID in your reference FASTA must match the chromosome names in your GTF/GFF, VCF, and BAM @SQ headers. Mixing chr1 and 1 (UCSC vs Ensembl naming) is the single most common cause of “no reads aligned” errors.

Companion: `.fai` index

samtools faidx ref.fa creates ref.fa.fai, a five-column TSV (name, length, offset, linebases, linewidth) that lets tools jump to any chromosome in O(1) without reading the file.

2. FASTQ — raw sequencing reads

FASTQ is FASTA + per-base quality scores. It is the format your sequencer emits and the input to every aligner, trimmer, and pseudo-aligner. Always gzipped in practice (*.fastq.gz).

Example

@A00228:279:HGNJ7DSXY:1:1101:2374:1000 1:N:0:ATCACG+CGATGT
NCAGTGATCTTTGCTGTGGGAATTGGGGAGAGCGTCTGGAGGAGAACATC
+
#FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF
@A00228:279:HGNJ7DSXY:1:1101:2374:1000 2:N:0:ATCACG+CGATGT
CACTCAGCACCATGGCCTGAACTCCTGTCTGCATGTGACTCAGGAATTAA
+
FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF:FFFFFFF

What it is and when you see it

Illumina, PacBio, Oxford Nanopore, Element, and 10x Chromium all emit FASTQ.
A 10x Chromium scRNA-seq run produces three FASTQs per lane: _I1 (sample index), _R1 (cell barcode + UMI), _R2 (cDNA insert).
Inputs to STAR, STARsolo, Cell Ranger, salmon, kallisto, bowtie2, bwa, minimap2.

Field walk-through (per read, exactly 4 lines)

Line 1 — header, starts with @. The Illumina convention is @<instrument>:<run#>:<flowcell>:<lane>:<tile>:<x>:<y> <read#>:<filter>:<control#>:<index>.
- A00228:279:HGNJ7DSXY — instrument, run, flowcell.
- 1:1101:2374:1000 — lane, tile, x-coordinate, y-coordinate.
- 1:N:0:ATCACG+CGATGT — read 1 of the pair, N = passed chastity filter, 0 = control bits, then the dual sample index.
Line 2 — sequence, IUPAC bases. Same length as line 4. An N indicates the basecaller could not assign a base.
Line 3 — +, optionally followed by the header again. Just a separator.
Line 4 — quality string, one ASCII character per base. Phred+33 encoding: Q = ord(char) − 33. So F = ASCII 70 = Q37 (very good); # = ASCII 35 = Q2 (terrible — Illumina’s “no-call” placeholder).

Tip

Quality cheatsheet. Q20 = 1 % error; Q30 = 0.1 % error. Illumina chemistry now reports binned qualities, so you typically see runs of F (Q37) and a sprinkle of lower bins.

Paired-end files come in matched pairs (*_R1.fastq.gz, *_R2.fastq.gz) with read \(i\) in R1 being the mate of read \(i\) in R2. Order matters; never re-sort one without the other.

3. SAM / BAM / CRAM — aligned reads

SAM (“Sequence Alignment / Map”) is the universal output of read aligners¹. BAM is the binary, block-compressed form of the same data — smaller, indexable, and the only form tools should consume in practice. CRAM is a more aggressively compressed form that stores reads relative to the reference. The samtools/bcftools toolkit that reads and writes all of these is the de-facto standard for manipulating them².

Example (SAM)

@HD     VN:1.6  SO:coordinate
@SQ     SN:chr1 LN:248956422
@SQ     SN:chr2 LN:242193529
@RG     ID:Sample01     SM:Sample01     LB:Lib01        PL:ILLUMINA
@PG     ID:STAR PN:STAR VN:2.7.10a      CL:STAR --runMode alignReads --genomeDir ...
r0001   99      chr1    10003   60      50M     =       10101   148     CTAACCCTAACCCTAACCCTAACCCTAACCCTAACCCTAACCCTAACCCT      FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF      NH:i:1  HI:i:1  AS:i:96 nM:i:1  CB:Z:AAACCTGAGAAACCAT-1 UB:Z:TACGCATGAC
r0001   147     chr1    10101   60      50M     =       10003   -148    AACCCTAACCCTAACCCTAACCCTAACCCTAACCCTAACCCTAACCCTAA      FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF      NH:i:1  HI:i:1  AS:i:96 nM:i:1
r0002   4       *       0       0       *       *       0       0       NNCAGTGATCTTTGCTGTGGGAATTGGGGAGAGCGTCTGGAGGAGAACATC     ##FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF

What it is and when you see it

Output of every short-read aligner: STAR, bwa-mem, bowtie2, minimap2, Cell Ranger’s internal aligner.
Input to variant callers (GATK, bcftools, DeepVariant), counters (featureCounts, htseq-count), peak callers (MACS2), and visualization tools (IGV).
The .bai (or .csi) sidecar file indexes the BAM for random access. Without it, you cannot jump to a region.

Header section walk-through (lines starting with `@`)

@HD VN:1.6 SO:coordinate — file header. VN is the SAM spec version; SO is the sort order (unsorted, queryname, or coordinate). Most downstream tools require coordinate and a .bai.
@SQ SN:chr1 LN:248956422 — one line per reference sequence. SN = sequence name (must match your FASTA), LN = length. These define the coordinate system.
@RG ID:Sample01 SM:Sample01 ... — read group. ID is the unique tag attached to every read in the alignment section; SM is the sample name (used by variant callers to assign genotypes); LB is the library; PL is the platform. Missing @RG lines break GATK.
@PG — program record. Tells you which aligner, which version, and the command line. Invaluable when reproducing someone else’s BAM.

Alignment section walk-through (one row per read)

Eleven mandatory tab-delimited columns, then optional TAG:TYPE:VALUE triplets.

Col 1 QNAME (r0001) — read name; identical for the two mates of a pair.
Col 2 FLAG (99) — a bitwise integer encoding alignment properties. Decode it field-by-field, never just compare integers. Common bits:
- 1 paired, 2 proper pair, 4 unmapped, 8 mate unmapped,
- 16 reverse strand, 32 mate reverse strand,
- 64 first in pair, 128 second in pair,
- 256 secondary, 512 failed QC, 1024 PCR/optical duplicate, 2048 supplementary.
- 99 = 1 + 2 + 32 + 64 → paired, proper pair, mate on reverse strand, first in pair.
- 147 = 1 + 2 + 16 + 128 → second mate, on reverse strand.
- 4 → unmapped read (the r0002 row).
- Use samtools flagstat and https://broadinstitute.github.io/picard/explain-flags.html rather than memorizing.
Col 3 RNAME (chr1) — reference sequence the read aligned to. * means unmapped.
Col 4 POS (10003) — 1-based leftmost mapping position.
Col 5 MAPQ (60) — Phred-scaled mapping quality. 255 means “not available”, 0 means “maps equally well to multiple locations”. 10x / STAR use 255 for unique alignments by default.
Col 6 CIGAR (50M) — a compressed alignment string. M = alignment match (could be sequence match or mismatch), = exact match, X mismatch, I insertion to ref, D deletion from ref, N skipped region (used for splice junctions), S soft clip, H hard clip, P padding. A typical spliced RNA-seq read might look like 38M2156N12M.
Cols 7–9 RNEXT POS NEXT TLEN — coordinates of the mate (= means same chromosome) and the inferred fragment length (template length, signed by orientation).
Col 10 SEQ — the read sequence, reverse-complemented if the read is on the reverse strand (so it always reads 5′→3′ relative to the reference).
Col 11 QUAL — the corresponding quality string, in Phred+33.
Optional tags (everything after column 11):
- NH:i:1 — number of hits (1 = uniquely mapped).
- HI:i:1 — which hit this row represents.
- AS:i:96 — alignment score from the aligner.
- nM:i:1 — number of mismatches.
- CB:Z:AAACCTGAGAAACCAT-1 — corrected cell barcode (10x convention).
- UB:Z:TACGCATGAC — corrected UMI (10x convention).
- XS:A:+ — predicted transcript strand (STAR / HISAT2).
- MD:Z:50 — exact mismatch positions (used to reconstruct the reference from the read).

Tip

A useful one-liner. samtools view -F 4 -q 30 file.bam chr1:1000000-1100000 | wc -l counts uniquely mapped (-q 30), mapped (-F 4) reads in a region. That is 90 % of the BAM operations you will ever need.

BAM vs SAM vs CRAM

SAM is the human-readable text form. Almost never written to disk in practice — too large.
BAM is the same data in BGZF-compressed binary. ~5× smaller than SAM; randomly indexable via .bai. The de facto standard.
CRAM is reference-based compression. Stores the differences from the reference rather than the full sequence, so it requires the original FASTA to decode. ~30–50 % smaller than BAM. Used by archives (EGA, ENA) but less convenient day-to-day.

4. GFF3 and GTF — genomic feature annotation

Both are tab-delimited, 9-column formats describing features (genes, transcripts, exons, regulatory elements) on a reference. They differ mostly in how column 9 is structured. GTF is older and still standard for RNA-seq tooling (STAR, featureCounts, Cell Ranger); GFF3 is the modern, fully-specified format used by Ensembl, NCBI, and most non-model-organism resources.

GTF example

##description: evidence-based annotation of the human genome (GRCh38), version 44
##provider: GENCODE
##format: gtf
##date: 2023-05-04
chr1    HAVANA  gene            11869   14409   .       +       .       gene_id "ENSG00000223972.6"; gene_type "transcribed_unprocessed_pseudogene"; gene_name "DDX11L1"; level 2;
chr1    HAVANA  transcript      11869   14409   .       +       .       gene_id "ENSG00000223972.6"; transcript_id "ENST00000456328.2"; gene_type "transcribed_unprocessed_pseudogene"; gene_name "DDX11L1"; transcript_type "processed_transcript"; transcript_name "DDX11L1-202"; level 2;
chr1    HAVANA  exon            11869   12227   .       +       .       gene_id "ENSG00000223972.6"; transcript_id "ENST00000456328.2"; exon_number 1; exon_id "ENSE00002234944.1"; gene_name "DDX11L1"; level 2;
chr1    HAVANA  exon            12613   12721   .       +       .       gene_id "ENSG00000223972.6"; transcript_id "ENST00000456328.2"; exon_number 2; exon_id "ENSE00003582793.1"; gene_name "DDX11L1"; level 2;
chr1    HAVANA  exon            13221   14409   .       +       .       gene_id "ENSG00000223972.6"; transcript_id "ENST00000456328.2"; exon_number 3; exon_id "ENSE00002312635.1"; gene_name "DDX11L1"; level 2;

GFF3 example

##gff-version 3
##sequence-region chr1 1 248956422
chr1    HAVANA  gene            11869   14409   .       +       .       ID=gene:ENSG00000223972;Name=DDX11L1;biotype=transcribed_unprocessed_pseudogene
chr1    HAVANA  mRNA            11869   14409   .       +       .       ID=transcript:ENST00000456328;Parent=gene:ENSG00000223972;Name=DDX11L1-202;biotype=processed_transcript
chr1    HAVANA  exon            11869   12227   .       +       .       ID=exon:ENSE00002234944;Parent=transcript:ENST00000456328
chr1    HAVANA  exon            12613   12721   .       +       .       ID=exon:ENSE00003582793;Parent=transcript:ENST00000456328
chr1    HAVANA  exon            13221   14409   .       +       .       ID=exon:ENSE00002312635;Parent=transcript:ENST00000456328

What they are and when you see them

The annotation half of every reference bundle. STAR --genomeDir needs the FASTA and the GTF.
featureCounts, htseq-count, salmon, and Cell Ranger all consume a GTF to assign reads to genes.
Ensembl ships GFF3; GENCODE³ ships both; UCSC ships GTF / GenePred.

Shared field walk-through (columns 1–8)

Col 1 seqid (chr1) — chromosome / scaffold. Must match the FASTA and BAM.
Col 2 source (HAVANA) — the program or curation group that produced this row.
Col 3 type (gene, transcript / mRNA, exon, CDS, five_prime_UTR, three_prime_UTR, …) — the SO term naming the feature kind. GTF requires gene / transcript / exon; GFF3 uses mRNA instead of transcript.
Col 4 start — 1-based, inclusive start coordinate.
Col 5 end — 1-based, inclusive end coordinate. GTF/GFF3 are 1-based and inclusive on both ends — different from BED (see below).
Col 6 score (.) — usually unused for annotation.
Col 7 strand — +, -, . (unstranded), or ? (unknown).
Col 8 phase — for CDS features only: 0, 1, or 2, the number of bases to skip to reach the next codon. . elsewhere.

Column 9 — the difference between GTF and GFF3

GTF uses key "value"; key "value"; pairs. Required keys: gene_id, transcript_id (on transcript / exon / CDS rows). GENCODE adds gene_name, gene_type, exon_number, level.
GFF3 uses key=value;key=value pairs. Parent–child relationships are explicit: an exon row has Parent=transcript:ENST..., and the transcript has Parent=gene:ENSG.... The same exon can list multiple parents.

Warning

Chromosome naming. Ensembl annotation uses 1, 2, MT. UCSC and most 10x references use chr1, chr2, chrM. If your aligner reports “0 reads assigned to features”, check column 1 of the GTF against the @SQ lines of the BAM first.

5. BED — interval lists

BED is the canonical format for “lists of regions” — peaks, blacklists, windows, target capture intervals, ATAC fragments-as-regions, gene bodies. It is tab-delimited with 0-based, half-open coordinates (start inclusive, end exclusive). The first three columns are required; up to nine more are defined.

Example (BED6)

chr1    9999    10468   peak_1  234     +
chr1    16110   16414   peak_2   78     +
chr1    28903   29370   peak_3  155     -
chr1    629390  629902  peak_4  482     +
chr2    9991    10800   peak_5  340     +

What it is and when you see it

Output of peak callers (MACS2, Genrich) — see also narrowPeak / broadPeak below.
Region inputs to bedtools intersect, bedtools coverage, IGV tracks.
Blacklists (ENCODE), exome capture targets, promoter windows.

Field walk-through

Col 1 chrom — chromosome. Naming must match the alignment / reference.
Col 2 chromStart — 0-based, inclusive. Position 0 is the first base.
Col 3 chromEnd — 0-based, exclusive. So 9999–10468 spans 469 bases (10468 − 9999), and a single-base feature at position 100 is 100 101.
Col 4 name — feature name.
Col 5 score — integer 0–1000, often used to control display shading.
Col 6 strand — +, -, or ..
Cols 7–12 (BED12) add thickStart, thickEnd, itemRgb, blockCount, blockSizes, blockStarts — used to draw multi-exon transcripts as a single row in genome browsers.

Warning

0-based vs 1-based. BED is 0-based, half-open. GFF/GTF/VCF/SAM-POS are 1-based, fully-closed. samtools view chr1:100-200 will return slightly different reads than bedtools intersect -b a.bed -a "chr1 99 200". Get this wrong and you will off-by-one for the rest of the project.

narrowPeak (BED6+4) — ChIP-seq / ATAC-seq peaks

chr1    9990    10468   peak_1  234   .  4.85    7.62   5.10   239
chr1    16110   16414   peak_2   78   .  2.10    3.41   1.88   152
chr1    28903   29370   peak_3  155   .  3.78    5.55   3.21   220

Same first six columns as BED6; the four extra columns are MACS-specific: signalValue (fold-enrichment), pValue (-log10), qValue (-log10, FDR-adjusted), and peak (0-based offset from chromStart to the summit). broadPeak drops the summit column.

6. VCF / BCF — variant calls

VCF (“Variant Call Format”) records differences from a reference at specific positions: SNVs, indels, structural variants, copy-number changes. BCF is the binary, indexable form (analogous to BAM ↔︎ SAM).

Example

##fileformat=VCFv4.2
##reference=file:///refs/GRCh38.primary_assembly.genome.fa
##contig=<ID=chr1,length=248956422>
##FILTER=<ID=PASS,Description="All filters passed">
##FILTER=<ID=LowQual,Description="QUAL below 30">
##INFO=<ID=DP,Number=1,Type=Integer,Description="Approximate read depth">
##INFO=<ID=AF,Number=A,Type=Float,Description="Allele frequency in called genotypes">
##INFO=<ID=AN,Number=1,Type=Integer,Description="Total alleles called">
##FORMAT=<ID=GT,Number=1,Type=String,Description="Genotype">
##FORMAT=<ID=AD,Number=R,Type=Integer,Description="Allelic depths (ref,alt)">
##FORMAT=<ID=DP,Number=1,Type=Integer,Description="Read depth">
##FORMAT=<ID=GQ,Number=1,Type=Integer,Description="Genotype quality">
#CHROM  POS     ID              REF  ALT  QUAL    FILTER  INFO                    FORMAT          Sample01            Sample02
chr1    10177   rs367896724     A    AC   285.4   PASS    DP=42;AF=0.50;AN=4      GT:AD:DP:GQ     0/1:18,17:35:99     0/1:12,9:21:75
chr1    14653   .               C    T    34.8    LowQual DP=8;AF=0.25;AN=4       GT:AD:DP:GQ     0/0:8,0:8:24        0/1:3,2:5:18
chr1    16495   rs141130360     G    C    488.1   PASS    DP=55;AF=1.00;AN=4      GT:AD:DP:GQ     1/1:0,28:28:84      1/1:0,27:27:81

What it is and when you see it

Output of GATK HaplotypeCaller, bcftools call, DeepVariant, Strelka, somatic callers (Mutect2).
Input to annotators (VEP, SnpEff, ANNOVAR), filtering tools, and per-cell variant assignment in scRNA / scDNA workflows.
A single VCF can hold dozens to thousands of samples in one file (columns 10+).

Meta-header walk-through (`##` lines, before the column header)

##fileformat=VCFv4.2 — required first line; declares the spec version.
##reference=... — the FASTA the calls were made against. Critical: positions are only meaningful relative to this reference build.
##contig=<ID=chr1,length=...> — one line per reference contig; matches the BAM @SQ lines.
##FILTER=<ID=...,Description=...> — every value that may appear in column 7 must be declared here, plus the special PASS.
##INFO=<ID=...,Number=...,Type=...,Description=...> — declares each key that may appear in column 8. Number=A means “one value per alternate allele”; Number=R means “one per allele including ref”; Number=1 is a single value; Number=. is variable.
##FORMAT=<...> — same idea, for keys that appear in the per-sample genotype columns.

Column header (single `#CHROM` line)

The mandatory eight columns, then FORMAT, then one column per sample.

Body walk-through

Col 1 CHROM (chr1) — contig.
Col 2 POS (10177) — 1-based position of the first base of REF.
Col 3 ID (rs367896724 or .) — dbSNP rsID or . if novel.
Col 4 REF (A) — reference allele at POS. For indels, this is the base before the event plus the event itself (the “anchor base” convention).
Col 5 ALT (AC) — comma-separated alternate alleles. A → AC at position 10177 is a single-base insertion of C after position 10177.
Col 6 QUAL (285.4) — Phred-scaled probability that the site is not variant. Higher is better.
Col 7 FILTER (PASS / LowQual) — PASS if the site passes all filters; otherwise a semicolon-separated list of failed filter IDs declared in the header.
Col 8 INFO (DP=42;AF=0.50;AN=4) — site-level annotation. Decoded against the ##INFO definitions: depth 42, alt allele frequency 0.50, 4 total alleles called.
Col 9 FORMAT (GT:AD:DP:GQ) — colon-separated list of keys that describe each sample column.
Cols 10+ per sample (0/1:18,17:35:99) — values in the order declared by FORMAT. Decoding for Sample01 at row 1:
- GT = 0/1 — heterozygous (one ref allele, one alt). / = unphased, | = phased. 0/0 = hom-ref, 1/1 = hom-alt, ./. = missing.
- AD = 18,17 — 18 reads supporting REF, 17 supporting ALT.
- DP = 35 — total reads at site (may differ from AD sum due to filtered reads).
- GQ = 99 — genotype quality (Phred-scaled).

Tip

Indexing. bgzip my.vcf && tabix -p vcf my.vcf.gz produces my.vcf.gz.tbi. Without it you cannot do bcftools view -r chr1:1000000-2000000.

7. Wig / BedGraph / BigWIG — quantitative signal tracks

A signal track is a per-base or per-interval numerical value along the genome: read coverage, methylation rate, conservation score, ATAC signal, ChIP-seq fold-enrichment. Wig and BedGraph are plain text; BigWIG is an indexed binary version of either.

Wig (variableStep) example

track type=wiggle_0 name="Sample01 coverage" description="STAR uniquely mapped"
variableStep chrom=chr1 span=10
9990    0
10000  12
10010  35
10020  47
10030  41
10040  22

BedGraph example

track type=bedGraph name="Sample01 fold-change"
chr1   9990   10000   0.0
chr1   10000  10100   12.4
chr1   10100  10200   18.7
chr1   10200  10300   24.1
chr1   10300  10400   17.3

What it is and when you see it

bamCoverage (from deepTools) produces a BigWIG of read coverage from a BAM.
MACS2 --bdg produces BedGraph signal tracks alongside narrowPeak.
ENCODE distributes BigWIGs for almost every assay (ChIP-seq, ATAC-seq, RNA-seq, DNase).
Genome browsers (IGV, UCSC, JBrowse) consume BigWIG directly without downloading the whole file.

Wig walk-through

Line 1 track type=wiggle_0 ... — optional UCSC track header. Sets display name, color, viewLimits.
Line 2 variableStep chrom=chr1 span=10 — the declaration. variableStep means the data lines specify positions; fixedStep would imply uniform spacing. span=10 means each value applies to 10 bases.
Data lines — for variableStep, two columns: 1-based start position and value. For fixedStep, just values.

BedGraph walk-through

Four columns per row: chrom, start (0-based), end (exclusive), value. Same coordinate convention as BED.
More verbose than Wig but easier to manipulate with awk / bedtools.

BigWIG (and bigBed)

Binary, indexed forms of Wig/BedGraph (and BED) introduced by Jim Kent. Random-access — IGV streams only the portion you are viewing.
Convert with wigToBigWig file.wig chrom.sizes file.bw or bedGraphToBigWig file.bg chrom.sizes file.bw. chrom.sizes is a 2-column TSV of chrom length (cut -f1,2 genome.fa.fai > chrom.sizes).
Cannot be read with head — use bigWigInfo, bigWigToBedGraph, or pyBigWig.

8. Count matrices — bulk and single-cell

The output of an RNA-seq pipeline is, eventually, a gene × sample (bulk) or gene × cell (single-cell) matrix of integer counts. Several serializations exist; they all encode the same logical object.

8a. Plain-text TSV / CSV (bulk)

gene_id              Sample01  Sample02  Sample03  Sample04
ENSG00000000003.15      1247       982      1531      1102
ENSG00000000005.6          0         2         0         1
ENSG00000000419.13       845       712       901       788
ENSG00000000457.14       312       298       351       277
ENSG00000000460.17       189       176       210       162

Row 1 — header with one column name per sample.
Col 1 — feature ID (Ensembl gene ID, often with version suffix .NN).
Cols 2+ — integer counts. featureCounts, htseq-count, Salmon merge, and tximport::summarizeToGene() all eventually land here. Input to DESeq2, edgeR, limma-voom.

8b. Matrix Market `.mtx` triplet (single-cell)

%%MatrixMarket matrix coordinate integer general
%metadata_json: {"software_version": "cellranger-7.1.0", "format_version": 2}
33538 6794880 9281379
33509 1 1
33514 1 4
33538 1 1
12834 2 2
13987 2 1

Line 1 — magic header: data type is a sparse coordinate matrix of integers, no symmetry.
Line 2 — comment with provenance.
Line 3 — the dimensions: 33,538 features × 6,794,880 cells × 9,281,379 non-zero entries.
Data lines — feature_index cell_index count, all 1-based. So 33509 1 1 means feature 33,509 has 1 count in cell 1.
This file ships in a directory alongside features.tsv.gz (one feature ID and name per row) and barcodes.tsv.gz (one cell barcode per row). Read10X() / scanpy.read_10x_mtx() consume the directory.

8c. HDF5 `.h5` (10x)

filtered_feature_bc_matrix.h5 is a single binary HDF5 file with the same matrix and metadata as the three-file triplet, just packaged together. Internally it has groups like /matrix/data, /matrix/indices, /matrix/indptr (CSC sparse layout) and /matrix/features/name. Read with Read10X_h5() (Seurat) or scanpy.read_10x_h5().

8d. AnnData `.h5ad` (the scRNA-seq archival format)

Also HDF5 under the hood, but with a fixed schema:

/X — the main counts matrix (cells × genes).
/obs — per-cell metadata (DataFrame).
/var — per-gene metadata (DataFrame).
/obsm — multi-dimensional per-cell arrays (X_pca, X_umap, …).
/varm — multi-dimensional per-gene arrays.
/layers — alternative matrices (spliced, unspliced, raw).
/uns — unstructured annotations (cluster colors, parameter dictionaries).

This is the format CELLxGENE requires for submission and the most portable single-cell container.

8e. `.loom`

Another HDF5-based single-cell format (Linnarsson lab / loompy). Layout is matrix, col_attrs (cells), row_attrs (genes), layers. Slightly older than AnnData; still used by RNA-velocity tools (velocyto, scVelo).

8f. `.rds` (R serialized object)

Native R serialization. A Seurat object saved with saveRDS() produces a .rds that holds the counts, embeddings, metadata, clusters, and graphs in one file. Compact for R users — but:

Not cross-language (Python cannot read it).
Not stable across major Seurat versions (a v4 .rds may need migration to load in v5).
Not a long-term archival format. Use .h5ad or a SingleCellExperiment saved as HDF5 for archiving.

9. scATAC fragments — `fragments.tsv.gz`

Cell Ranger ATAC and cellranger-arc emit a fragments file: one row per Tn5 cut-pair that survived QC, with the cell barcode attached.

Example

#description: ATAC-seq fragments
#cellranger-arc-version: 2.0.2
#reference: GRCh38-2020-A
chr1    10074  10309  AAACAGCCAAGGAATC-1  1
chr1    10080  10300  AAACAGCCATAGCTTG-1  1
chr1    10100  10455  AAACATGCAACGTGCT-1  2
chr1    10120  10311  AAACGAACATCTACTC-1  1
chr1    10150  10405  AAACAGCCAAGGAATC-1  1

Header lines (#) — provenance.
Col 1 chrom, Col 2 start (0-based), Col 3 end (exclusive). Same coordinate convention as BED.
Col 4 — the 10x cell barcode (with the -1 GEM-well suffix). This is what makes the file single-cell-aware.
Col 5 — read support count (PCR duplicates of the same fragment collapsed to one row).
Always bgzipped and tabix-indexed (fragments.tsv.gz.tbi) for random-access queries by Signac / ArchR / scanpy-snap.

10. Other formats you will run into

Format	Where it shows up	One-line summary
`.fai`	Beside every reference FASTA	5-column index produced by `samtools faidx`
`.bai` / `.csi`	Beside every BAM	Binary index for random-access reads
`.tbi`	Beside any bgzipped tab-delimited file	Tabix index for region queries
`.dict`	Beside reference FASTAs for GATK	Sequence dictionary (same info as `@SQ` lines)
`.gbk` (GenBank)	NCBI nucleotide records	Sequence + rich feature annotation, single file
`.embl`	EBI nucleotide records	European cousin of GenBank
`.bcf`	bcftools, large cohorts	Binary, indexable VCF
`.bigBed`	UCSC / IGV tracks	Indexed binary BED
`.hic` / `.cool`	Hi-C contact matrices	Sparse 3D-genome interaction storage
`.pdb` / `.cif`	Structural biology	3D atomic coordinates
`.parquet`, `.feather`, `.zarr`	Modern data-science pipelines	Column-store / chunked binary, increasingly used for very large single-cell objects

11. Quick reference — coordinate conventions

Format	Indexing	End	Notes
FASTA position	1-based	inclusive	`samtools faidx chr1:100-110` returns 11 bases
FASTQ	n/a	n/a	reads, not coordinates
SAM `POS`	1-based	leftmost only	end inferred from CIGAR
GFF3 / GTF	1-based	inclusive	both endpoints in the feature
BED	0-based	exclusive	end is one past the last base
VCF `POS`	1-based	inclusive	indels include an anchor base
Wig / BedGraph	Wig 1-based, BedGraph 0-based	matches its parent	mind the difference

Off-by-one errors live in this table. When in doubt, compute the length of a feature both ways and see whether it matches the underlying FASTA slice.

Overview

A short taxonomy

1. FASTA — reference sequences

Example

What it is and when you see it

Field walk-through

Companion: .fai index

2. FASTQ — raw sequencing reads

Example

What it is and when you see it

Field walk-through (per read, exactly 4 lines)

3. SAM / BAM / CRAM — aligned reads

Example (SAM)

What it is and when you see it

Header section walk-through (lines starting with @)

Alignment section walk-through (one row per read)

BAM vs SAM vs CRAM

4. GFF3 and GTF — genomic feature annotation

GTF example

GFF3 example

What they are and when you see them

Shared field walk-through (columns 1–8)

Column 9 — the difference between GTF and GFF3

5. BED — interval lists

Example (BED6)

What it is and when you see it

Field walk-through

narrowPeak (BED6+4) — ChIP-seq / ATAC-seq peaks

6. VCF / BCF — variant calls

Example

What it is and when you see it

Meta-header walk-through (## lines, before the column header)

Column header (single #CHROM line)

Body walk-through

7. Wig / BedGraph / BigWIG — quantitative signal tracks

Wig (variableStep) example

BedGraph example

What it is and when you see it

Wig walk-through

BedGraph walk-through

BigWIG (and bigBed)

8. Count matrices — bulk and single-cell

8a. Plain-text TSV / CSV (bulk)

8b. Matrix Market .mtx triplet (single-cell)

8c. HDF5 .h5 (10x)

8d. AnnData .h5ad (the scRNA-seq archival format)

8e. .loom

8f. .rds (R serialized object)

9. scATAC fragments — fragments.tsv.gz

Example

10. Other formats you will run into

11. Quick reference — coordinate conventions

See also

References

Companion: `.fai` index

Header section walk-through (lines starting with `@`)

Meta-header walk-through (`##` lines, before the column header)

Column header (single `#CHROM` line)

8b. Matrix Market `.mtx` triplet (single-cell)

8c. HDF5 `.h5` (10x)

8d. AnnData `.h5ad` (the scRNA-seq archival format)

8e. `.loom`

8f. `.rds` (R serialized object)

9. scATAC fragments — `fragments.tsv.gz`