Chapter P1 — Computer Systems & the Command Line
Single-cell RNA-seq analysis pushes against the limits of a laptop almost immediately: datasets are large, the software stack is deep, and many steps run faster (or only run at all) on shared research computing infrastructure. This chapter builds the mental model you need — how a computer actually executes work, how an operating system manages resources, how to use VS Code as a local editor, and how to navigate and manipulate the file system from the command line. The University of Oregon’s Talapas HPC cluster, SSH login, modules, and SLURM scheduling are covered in Chapter 9 — VS Code & SLURM Basics on Talapas (the Friday track).
This is the reading for P1 — Computer Systems & the Command Line, the first hour of the optional two-hour Tuesday refresher. It pairs with the Module P1 lecture slides. Nothing here is single-cell-specific yet; it is the computing foundation that the rest of the workshop assumes. Connecting to and working on Talapas — VS Code Remote-SSH, Lmod modules, file systems, interactive sessions, and reproducible environments — is covered on Friday in Chapter 9.
Keep a cheat sheet open. The Cheat Sheets tab collects one-page Unix shell and VS Code references that pair with this chapter; you will lean on the Unix shell sheet again every time you work in a terminal — locally now, and on Talapas during the Friday track (Modules 09–10). See also the Software Setup page for installation instructions if you have not yet installed VS Code, R, or the workshop repository.
By the end of this chapter you should be able to:
- Explain how a computer executes work (CPU/GPU, memory vs. storage) and what an operating system does
- Decide where a given analysis should run — laptop vs. HPC cluster
- Open and work in a project folder in VS Code as a local editor
- Navigate the file system from the command line and tell absolute from relative paths
- Create, move, copy, and remove files and directories safely from the shell
- Use wildcards, pipes, and redirection to combine commands, and download data with
wget/curl
How computers and operating systems work
What is a computer system?
A computer system is the combination of four things working together:
- Hardware: the physical components (CPU, memory, storage, I/O devices)
- Software: the programs and instructions that run on the hardware
- Firmware: low-level software embedded in hardware
- Data: the information being processed
The fetch-decode-execute cycle
Every operation a computer performs reduces to a small cycle repeated billions of times per second:
- Fetch: retrieve an instruction from memory
- Decode: interpret what the instruction means
- Execute: perform the operation
- Store: save results to memory or a register
The classic name is the three-phase “fetch–decode–execute” cycle; the store (write-back) phase is often counted as part of execute, which is why you will see it described as either three or four steps.
Hardware components
Central Processing Unit (CPU)
Core components
- Control Unit (CU): directs operations
- Arithmetic Logic Unit (ALU): performs calculations
- Registers: ultra-fast temporary storage
- Cache: high-speed memory buffer
Key concepts
- Clock speed: cycles per second (GHz)
- Cores: independent processing units
- Threads: virtual cores for parallel processing
- Instruction set: the CPU’s language (x86, ARM)
Graphics Processing Unit (GPU)
Core components
- Streaming Multiprocessors (SMs): processing clusters
- CUDA cores / stream processors: parallel compute units
- Video Memory (VRAM): dedicated high-bandwidth memory
- Memory controller: manages data flow to/from VRAM
Key concepts
- Parallel architecture: thousands of cores for simultaneous tasks
- Memory bandwidth: data throughput (GB/s)
- Compute units: groups of processing cores
- APIs: programming interfaces (CUDA, OpenCL, Vulkan)
CPU vs GPU
CPU characteristics
- Design focus: sequential processing & complex tasks
- Core count: 4–64 powerful cores
- Architecture: optimized for single-thread performance
- Best for: operating systems & general computing, complex branching logic, serial tasks
GPU characteristics
- Design focus: parallel processing & high throughput
- Core count: thousands of smaller cores
- Architecture: optimized for massive parallelism
- Best for: graphics rendering, machine learning/AI, scientific simulations, parallel computations
Memory and storage
Random Access Memory (RAM)
- DRAM: main system memory; needs constant refresh; cheaper, higher density (DDR4, DDR5)
- SRAM: used in CPU cache; no refresh needed; faster but expensive
- Volatile: data is lost when power is off
- Random access: any location is equally fast
Persistent storage
- HDD: mechanical spinning platters; large capacity, low cost; ~100–200 MB/s; higher latency (~10 ms)
- SSD: no moving parts (NAND flash); faster access (~0.1 ms); 500–7000 MB/s; more expensive per GB
These three are often confused but serve entirely different roles:
- CPU (processor): does the actual computation — arithmetic, logic, calling functions. More cores = more things happening in parallel; higher clock speed = each thing happens faster.
- RAM (memory): holds the data and code the CPU is actively working with. Fast but small and volatile — it empties when you power off. A Seurat object for 50,000 cells can easily occupy 8–16 GB of RAM.
- Disk (storage): holds files permanently. Much slower than RAM but far larger and persistent. Reading a
.rdsfile moves data from disk into RAM; saving it moves data from RAM back to disk.
The most common reason a single-cell analysis crashes on a laptop is running out of RAM, not disk space or CPU. Knowing which resource is the bottleneck guides every decision about whether to work locally or on a cluster.
Input/output (I/O) systems connect the computer to the outside world: keyboards, mice, touchscreens, cameras and sensors on the input side; monitors, printers, speakers and actuators on the output side; and network interfaces in both directions.
Operating systems
An operating system (OS) is system software that manages hardware and software resources and provides common services for programs.
Key roles
- Resource manager
- Extended machine
- User interface provider
- Security enforcer
Common operating systems
- Desktop: Windows, macOS, Unix, Linux
- Mobile: Android, iOS
- Server: Unix, Linux, Windows Server
- Specialized: supercomputers, IoT
Research computing clusters — including the University of Oregon’s Talapas cluster covered in Chapter 9 — run Linux, which is why the workshop spends time on the Unix command line.
Core OS functions
Process management
- Process creation & termination
- Process scheduling: CPU time allocation
- Inter-process communication (IPC)
- Synchronization & deadlock handling
File system management
- File operations: create, read, write, delete
- Directory structure
- Access control & permissions
Memory management
- Memory allocation/deallocation
- Virtual memory, paging & segmentation
- Memory protection
- Swap space management
Device management
- Device drivers: hardware abstraction
- I/O scheduling, buffering & caching
- Interrupt handling
The OS also decides which process runs next on a CPU (scheduling algorithms such as First-Come-First-Served, Shortest Job First, Round Robin, and Priority scheduling) and enforces security through access control lists, user/group permissions, memory protection, user-vs-kernel CPU modes, and sandboxing. The guiding principle:
“Every program and user should operate using the least amount of privilege necessary.”
This same scheduling-and-fairness problem reappears at cluster scale — that is exactly what SLURM solves on Talapas.
Coding and scripting
Why work at the command line and in code rather than point-and-click GUIs?
- It is fast and powerful, particularly for repeated actions — thousands of “clicks” in a single command.
- It can analyze datasets far too large for Excel or other GUIs.
- It gives access to thousands of free programs made for and by scientists.
- Commands work almost identically across platforms.
- It is the only practical way to use a computer cluster (see Chapter 9 for the Talapas-specific details).
There is a loose distinction between coding (compiled languages such as C++, Fortran — faster but less flexible) and scripting (interpreted languages such as Unix shell, Python, R, Julia — flexible but slower). The line has blurred, and most modern analytical pipelines mix both.
Where computation can happen
Before reaching for a cluster, it helps to see the landscape of where computation can happen.
- Local computing — resources physically present and directly controlled by you (laptop, workstation, on-prem server).
- Cluster computing — many interconnected computers working together as one system (this is what Talapas is).
- Cloud computing — on-demand resources delivered over the internet with pay-as-you-go pricing (AWS, Azure, Google Cloud).
A laptop is fine for learning and for small single-cell datasets, but it runs out of memory and time quickly once you scale to full experiments. The University of Oregon’s Talapas HPC cluster is free to UO researchers, has large-memory nodes and GPUs, and is where the Friday modules run the real single-cell pipeline. Everything you need to know about connecting to Talapas, loading software, managing files, and submitting jobs is in Chapter 9 — VS Code & SLURM Basics on Talapas.
VS Code on your laptop
Visual Studio Code is a free, cross-platform editor from Microsoft that you will use throughout the workshop — first locally on your laptop, and later to connect to Talapas over SSH (covered in Chapter 9). This section covers the local laptop setup only.
Installing VS Code
Install it from https://code.visualstudio.com/download. Run the installer for your operating system (Windows, macOS, or Linux). On first launch, allow any requested permissions.
Take a moment to find the Activity Bar down the left edge — the icons there switch between the Explorer, Search, Source Control, Extensions, and (after the next step) Remote Explorer panels. The Command Palette (Ctrl+Shift+P on Windows/Linux, Cmd+Shift+P on macOS) searches every VS Code command by name; you will use it constantly.
Useful extensions
In the left sidebar, click the Extensions tab (the four-squares icon) and search for the following:
- Quarto — edit, preview, and run
.qmdfiles (the workshop chapters and tutorials). With this extension installed, a Preview button appears in the editor’s top-right corner; clicking it (or pressingCtrl+Shift+K/Cmd+Shift+K) renders a live HTML preview in a split pane that refreshes on save. - R — syntax highlighting, autocompletion, and integration with the R language server.
- R Debugger — step-through debugging for R scripts.
- vscode-pdf (PDF Viewer) — open plots and rendered documents without leaving the editor.
- Excel Viewer (CSV Editor/Viewer) — renders
.csvfiles as a formatted table. - GitLens or Git Graph — visualize the repository history alongside your code.
After installing, restart VS Code if prompted. The Remote — SSH and Remote Explorer extensions (for connecting VS Code to Talapas) are installed and configured in Chapter 9 — no need to install them today.
Opening a project folder
VS Code is folder-centric: almost everything works better when you open the root folder of a project rather than individual files.
- Download or clone the workshop repository so it is somewhere on your laptop (e.g.
~/scRNAseq_tutorial). - In VS Code choose File → Open Folder… (macOS: File → Open…) and navigate to that folder.
- The Explorer panel on the left now shows the directory tree. Click on any file to open it.
- VS Code may ask if you trust the authors of the files in this folder — click Yes, I trust the authors to enable full functionality.
A folder opened directly is the simplest way to work. A workspace (.code-workspace file) lets you group multiple root folders — handy when a project spans several directories, but unnecessary for this workshop. Stick with Open Folder for now.
Using VS Code as a local IDE
Once you have a project folder open, VS Code behaves as a full local IDE:
- The Explorer panel (left sidebar) shows the directory tree; click a file to open it in the editor.
- The integrated terminal (Terminal → New Terminal, or
Ctrl+`) opens a shell in the project directory — use it to run R scripts, shell commands, or Git commands without leaving VS Code. PressCtrl+`to toggle it open and closed. - Comment / uncomment selected lines with
Ctrl + /(macOSCmd + /). - Search across all files in the folder with
Ctrl + Shift + F(macOSCmd + Shift + F). - Split-pane editing lets you view a script and its output log side by side (
Ctrl+\/Cmd+\). - Add more terminal tabs with the + at the top-right of the terminal pane; you can have multiple shells open simultaneously.
- Run R locally: the integrated terminal is a full shell — start an interactive R session with
R, or run a script withRscript my_script.R, exactly as you would in any terminal.
Useful VS Code shortcuts
| What | How (Win/Linux) | How (macOS) |
|---|---|---|
| Toggle word wrap | Alt+Z | Alt+Z |
| Find in current file | Ctrl+F | Cmd+F |
| Find across all files | Ctrl+Shift+F | Cmd+Shift+F |
| Go to a line number | Ctrl+G | Cmd+G |
| Format a file | Shift+Alt+F | Shift+Option+F |
| Toggle integrated terminal | Ctrl+| Ctrl+ |
|
| Open new terminal | Ctrl+Shift+| Cmd+Shift+ |
|
| Split editor pane | Ctrl+\ | Cmd+\ |
| Open Command Palette | Ctrl+Shift+P | Cmd+Shift+P |
| Preview Quarto document | Ctrl+Shift+K | Cmd+Shift+K |
For the early modules (Modules 01–08) you work entirely locally — open the workshop repository folder in VS Code and run everything from the integrated terminal or from RStudio. When you move to the Friday Talapas modules, VS Code’s Remote-SSH extension connects the same editor interface directly to the cluster; the experience feels nearly identical, but your files and terminal are on Talapas. That setup is in Chapter 9.
The command line
The shell is a text-based interface to your computer’s operating system. You type a command, the shell passes it to the OS, the OS executes it, and the result comes back as text. This is not a workaround for an absence of GUI tools — it is a deliberate design: small, focused programs that each do one thing well, combined in flexible ways.
The shell is both a command interpreter (it reads what you type and runs the corresponding programs) and a scripting language (you can save a sequence of commands in a file and run it repeatedly). Every bioinformatics tool you will encounter — Cell Ranger, STAR, samtools, salmon — is run from the shell. Seurat workflows on a cluster are submitted as shell scripts. Knowing the shell is not optional for serious genomics work.
The most common shell on macOS and Linux is bash (Bourne-Again SHell); modern macOS defaults to zsh, which is nearly identical for interactive use. On Windows, WSL (Windows Subsystem for Linux) gives a full Linux environment; the RStudio Terminal tab also works for the commands in this chapter. All examples here run the same on macOS, Linux, and WSL.
Absolute vs relative paths
Every file on a Unix-like system has an absolute path — the full address from the root / of the file system — and you can also refer to files by a relative path from wherever you currently are.
/. An absolute path traces the route from /; a relative path starts from your current working directory.cd / # jump to the root of the whole file system (absolute)
ls # the top-level directories of the machine
cd ~ # back home (absolute, via the ~ shortcut)
cd .. # up one level (relative — .. means "parent directory")
cd - # back to where you just were (toggles between last two dirs)
cd ~/Desktop # absolute path built from home; adjust if Desktop does not exist
pwdAn absolute path starts at the filesystem root / and is unambiguous from anywhere on the machine: /home/wcresko/projects/scrnaseq/data/sample01.csv. A relative path is interpreted from the current working directory: ../data/sample01.csv means “go up one level, then into data/.”
Special path shortcuts:
| Shortcut | Meaning |
|---|---|
. |
The current directory |
.. |
The parent directory (one level up) |
~ |
Your home directory |
- |
The previous directory (where you just were) |
Relative paths are shorter and survive moving an entire project to a new location; absolute paths are unambiguous but brittle if anything above them moves. In analysis scripts, relative paths anchored to the project root (e.g. data/raw/sample01.fastq.gz) are best for reproducibility.
Making, moving, copying, and removing files
Create a scratch workspace and practice the core file operations.
cd ~
mkdir p1_practice # make a new directory
cd p1_practice
touch notes.txt # create an empty file
echo "single cell RNA-seq" > notes.txt # write one line (> overwrites)
echo "day 0 refresher" >> notes.txt # append a second line (>> appends)
cp notes.txt notes_backup.txt # copy a file
mv notes.txt readme.txt # rename (move within the same directory)
mkdir archive
mv notes_backup.txt archive/ # move a file into a subdirectory
ls -F # see what you have made
ls archive/Now clean up selectively:
rm archive/notes_backup.txt # remove a single file
rmdir archive # remove an *empty* directory
# rm -r somedir # remove a directory AND everything inside itA few rules the shell enforces that can surprise newcomers: the shell trusts you — it deletes what you tell it to and overwrites same-named files without asking. There is no Recycle Bin. Avoid spaces in filenames; stick to letters, numbers, ., -, and _.
| Command | Effect |
|---|---|
rm file |
Delete a single file (irreversible — no Recycle Bin) |
rm -r dir/ |
Delete a directory and all its contents recursively |
rm -i file |
Interactive: ask before each deletion |
cp src dst |
Copy a single file |
cp -r src/ dst/ |
Copy a directory and all its contents recursively |
mv src dst |
Move or rename a file or directory |
rm -r is powerful and irreversible. Always double-check the path before pressing Enter, and never combine it with wildcards (rm -r *) without careful thought.
Viewing file contents
You rarely open genomics data files in a GUI; you inspect them directly in the shell. Build a test file, then look at it several ways:
cd ~/p1_practice
# write 100 numbered lines into a file
for i in {1..100}; do echo "line $i"; done > big.txt
cat readme.txt # dump an entire (small) file to the screen
head big.txt # first 10 lines (default)
head -n 3 big.txt # first 3 lines
tail big.txt # last 10 lines
tail -n 3 big.txt # last 3 lines
wc -l big.txt # count lines in a file
less big.txt # page through interactively (SPACE=down, b=up, /pattern=search, q=quit)cat dumps the entire file to the screen. That is fine for a 10-line README, but a 3 GB genome FASTA or a 50,000-cell expression matrix will flood your terminal and can stall your session. For big files always use head, tail, less, or streaming tools — they never load the whole file into memory at once.
| Command | What it does |
|---|---|
cat file |
Print entire file to stdout |
head -n N file |
Print first N lines (default 10) |
tail -n N file |
Print last N lines (default 10) |
wc -l file |
Count lines; -w = words, -c = bytes |
less file |
Interactive pager: scroll, search, quit with q |
zcat file.gz |
Stream a gzip-compressed file without unpacking |
zless file.gz |
Page through a gzip-compressed file |
Wildcards
Wildcards let one command act on many files at once. The shell expands them before passing the result to the command.
cd ~/p1_practice
touch sample_01.fastq sample_02.fastq sample_03.fastq control.fastq notes.md
ls *.fastq # everything ending in .fastq
ls sample_*.fastq # the three sample files (prefix match)
ls sample_0?.fastq # ? matches exactly one character
ls sample_0[12].fastq # character class: only sample_01 and sample_02
ls *.{fastq,md} # brace expansion: match .fastq OR .md| Pattern | Matches |
|---|---|
* |
Any string of characters (including none) |
? |
Exactly one character |
[abc] |
Any single character in the set |
[a-z] |
Any single character in the range |
{foo,bar} |
Either foo or bar (brace expansion, not a true wildcard) |
Wildcards are expanded by the shell before the command receives them. If no file matches, many shells pass the literal pattern to the command, which will usually error — a useful signal that your pattern is wrong or you are in the wrong directory.
Pipes and redirection
Two mechanisms power most command-line work:
Redirection routes a command’s input or output to/from a file:
>writes stdout to a file (overwrites if it exists)>>appends stdout to a file<reads stdin from a file2>redirects stderr (error messages) to a file
Pipes (|) connect the stdout of one command directly to the stdin of the next — no temporary file needed. This is the Unix philosophy: small tools that each do one thing well, chained together into powerful pipelines.
cd ~/p1_practice
# --- redirection ---
ls -l > listing.txt # write a directory listing to a file (overwrites)
echo "generated $(date)" >> listing.txt # append a timestamp line
wc -l < big.txt # feed big.txt as stdin; counts its lines
# --- pipes ---
cat big.txt | wc -l # count lines via a pipe (same result)
head -n 50 big.txt | tail -n 5 # lines 46–50: take first 50, then last 5 of those
ls *.fastq | wc -l # how many .fastq files are there?
# --- a longer pipeline ---
# count and rank the most common words in readme.txt
cat readme.txt | tr ' ' '\n' | sort | uniq -c | sort -rn | headEvery program has three standard streams: stdin (0, input), stdout (1, normal output), and stderr (2, error messages). You can redirect each independently — for example, command 2> errors.log keeps error messages out of your results file.
A pipe | makes the output of one program the input of the next — all in memory, with no temporary files written to disk. This lets you chain simple, specialized tools into powerful one-liners. For example:
grep "^>" sequences.fasta | sort | uniq | wc -lreads a FASTA file, keeps only header lines (starting with >), sorts them, removes duplicates, and counts the result — four operations in one readable command.
Pipes and redirection can be combined freely: ls | wc -l > count.txt pipes the listing into a counter, then saves the count to a file.
For deeper treatment of grep, regular expressions, and shell scripting see Appendix C — grep, regex & shell scripting.
Downloading data with wget and curl
Genomic references, annotation files, and datasets are usually distributed by URL. wget and curl fetch them directly to your machine or the cluster — no browser required.
cd ~/p1_practice
# wget: download a file to the current directory
wget https://raw.githubusercontent.com/githubtraining/hellogitworld/master/README.txt
# curl: -o specifies the output filename
curl -o readme_copy.txt \
https://raw.githubusercontent.com/githubtraining/hellogitworld/master/README.txt
ls -lh README.txt # -h = human-readable size (e.g. 1.2K instead of 1234)
head README.txt| Flag | Tool | Effect |
|---|---|---|
-O filename |
wget | Save to a specific filename instead of the URL’s basename |
-q |
wget | Quiet mode — suppress progress output |
-o filename |
curl | Write output to a named file |
-L |
curl | Follow HTTP redirects (important for some data portals) |
-C - |
wget | Resume a partially downloaded file |
The backslash \ at the end of a line continues the command on the next line — useful for long URLs or multi-flag commands. Compressed files ending in .gz can be streamed without unpacking: wget -qO- url | zcat | head pipes the download straight into zcat for inspection.
When you finish practicing, clean up the scratch folder:
cd ~
rm -r p1_practice # deletes the folder and everything in it — intentional hereQuick reference: the most-used shell commands
| Category | Command | What it does |
|---|---|---|
| Navigation | pwd |
Print current directory |
ls [-la] |
List directory contents | |
cd path |
Change directory | |
| File ops | touch file |
Create empty file |
cp src dst |
Copy file or directory (-r for directories) |
|
mv src dst |
Move or rename | |
rm file |
Delete file (-r for directories) |
|
mkdir dir |
Create directory | |
| Viewing | cat file |
Print file contents |
head / tail |
First/last lines | |
less file |
Page through interactively | |
wc -l file |
Count lines | |
| Streams | > / >> |
Redirect stdout (overwrite / append) |
\| |
Pipe stdout to next command’s stdin | |
2> |
Redirect stderr | |
| Download | wget url |
Download file from URL |
curl -o file url |
Download to named file |
For advanced shell topics — grep pattern matching, regular expressions, sed, awk, loops, and writing shell scripts — see Appendix C — grep, regex & shell scripting. The Cheat Sheets page has a printable one-page Unix shell reference.
- A computer is hardware + OS + software + data; the CPU runs the work, memory is fast and volatile, storage is slower and persistent, and the GPU accelerates massively parallel work.
- Where to compute is a deliberate choice: a laptop for small/interactive work, an HPC cluster for large or long jobs (Talapas — the Friday track, Chapter 9).
- VS Code is your local editor/IDE; a project folder keeps scripts, data, and outputs organized.
- On the command line:
pwd/ls/cdto orient, relative paths (../data/raw/) for portability,mkdir/cp/mv/rmto manage files (rmhas no undo), and wildcards + pipes + redirection to compose commands. - Next, Chapter P2 builds the R foundation; the Friday track (Chapters 9–10) takes these same shell skills onto Talapas.
Best practices and getting help
Getting help in the shell
The shell has several built-in ways to learn about a command:
man ls # the full manual page (q to quit)
ls --help # brief usage summary (works on Linux; not all macOS tools support it)
whatis ls # one-line descriptionMost commands follow the pattern command --help or man command. Reading a man page for any command you are about to use on production data is a good habit.
Getting help with VS Code
- Command Palette:
Ctrl + Shift + P(macOSCmd + Shift + P) searches all available VS Code commands by name. - Official docs: code.visualstudio.com/docs — thorough reference for every feature and extension.
- Extension pages: each extension’s page in the Marketplace explains its keybindings and settings.
- Settings Sync: if you use VS Code on multiple machines, File → Preferences → Turn on Settings Sync… keeps your extensions and keybindings in sync automatically.
Getting help with Talapas
Everything Talapas-related — accounts, SSH login, modules, SLURM, file systems, and reproducible environments — is covered in Chapter 9. Quick links for when you need UO support directly:
- Knowledge Base: uoracs.github.io/talapas2-knowledge-base
- Service Desk: hpcrcf.atlassian.net/servicedesk
- Email: racs@uoregon.edu
Additional resources
- Appendix C — grep, regex & shell scripting — deeper treatment of pattern matching, sed, awk, and writing reusable shell scripts.
- Cheat Sheets — one-page Unix shell and VS Code references.
- Software Setup — step-by-step installation instructions for VS Code, R, and RStudio.
- Software Carpentry — The Unix Shell — a well-regarded self-paced introduction that covers everything in this chapter and more with hands-on exercises.