4  Computer Systems Architecture

NoteLearning Objectives

After completing this chapter, you will be able to:

  • Describe the major components of a computer system
  • Explain the fetch-decode-execute cycle
  • Compare CPU and GPU architectures and their appropriate use cases
  • Distinguish between different types of memory and storage
  • Understand the role of operating systems
  • Compare local, cluster, and cloud computing approaches

Understanding how computers work at a fundamental level will help you make better decisions about computational resources, write more efficient code, and troubleshoot problems effectively. This chapter provides the conceptual foundation for everything that follows.

4.1 What Is a Computer System?

A computer system consists of four major components working together:

Hardware. The physical components you can touch—processors, memory chips, storage devices, and input/output peripherals.

Software. Programs and instructions that tell the hardware what to do. This includes operating systems, applications, and your own scripts.

Firmware. Low-level software embedded directly in hardware components, providing basic control functions.

Data. The information being processed, stored, and transmitted by the system.

Figure 4.1 shows how data flows between these components.

Figure 4.1: Basic computer system architecture showing the flow of data between input devices, the CPU, memory, storage, and output devices

The hardware on the edges of that diagram is the input/output (I/O) system: keyboards, mice, touchscreens, cameras, microphones, and laboratory sensors bring data in; displays, printers, speakers, and actuators send results out. A network interface is both, which is why a cluster or cloud machine you never touch physically can still be a full computer system.

4.2 Bits, Bytes, and the Units of Data

Everything a computer stores—your thesis, a genome, a confocal stack—is ultimately a sequence of bits, each a single 0 or 1. Bits are grouped into bytes of eight, and one byte is enough to hold one character of ordinary text, so it is a useful mental unit: a plain-text file’s size in bytes is roughly its length in characters. The prefixes then climb by factors of about a thousand (Table 4.1), and it pays to have a feel for what lives at each scale.

Table 4.1: Units of digital data, with a biological sense of scale
Unit Size Holds, roughly
Byte (B) 8 bits One character of text
Kilobyte (KB) ~1,000 B A paragraph; a short email
Megabyte (MB) ~1,000 KB A photo; the entire E. coli genome as text (4.6 million bases)
Gigabyte (GB) ~1,000 MB An RNA-seq FASTQ file; the human genome as text is about 3 GB
Terabyte (TB) ~1,000 GB A laptop’s disk; a high-content imaging screen
Petabyte (PB) ~1,000 TB Talapas’s shared storage, about 3 PB (Chapter 3)

Numbers in memory are stored in fixed-size slots: a typical integer takes 4 bytes and a double-precision decimal takes 8. That makes memory needs easy to estimate. The course’s stickleback dataset—600 genes measured in 80 fish—is 48,000 numbers, well under a megabyte, and will open instantly on any laptop. A single-cell experiment with 20,000 genes and 100,000 cells is 2 billion numbers, roughly 16 GB as a dense matrix: more than most laptops’ RAM, and precisely why sparse matrix formats and machines like Talapas exist.

Note1,000 or 1,024?

Two conventions coexist. Decimal units (KB, MB, GB) use powers of 1,000; binary units (KiB, MiB, GiB) use powers of 1,024. Disk manufacturers use decimal, operating systems historically used binary while labeling it “GB,” and the mismatch compounds with size—which is why a “1 TB” drive shows up as about 931 “GB.” Nothing is missing; the two sides are counting in different units.

Files, in turn, come in two broad families. Plain-text files—scripts, CSV and TSV tables, FASTA and FASTQ sequence files, Quarto documents—contain nothing but characters: any editor can open them, the shell tools of Chapter 5 and Chapter 7 operate on them line by line, and Git can show exactly what changed between versions. Binary files—images, .docx and .xlsx documents, compressed archives, and specialized scientific formats such as BAM and HDF5—encode data more compactly but are readable only by software that knows the format. Much of this course is about the leverage that comes from keeping data in plain text as long as possible; Appendix C tours the formats you will meet in practice.

At the heart of every computation is the fetch-decode-execute cycle, which the CPU performs billions of times per second.

4.3 The Fetch-Decode-Execute Cycle

Every instruction your programs execute follows the same fundamental pattern (Figure 4.2):

Figure 4.2: The fundamental cycle underlying all computer operations: fetch, decode, execute, store, and repeat
  1. Fetch: The CPU retrieves the next instruction from memory
  2. Decode: The CPU interprets what the instruction means
  3. Execute: The CPU performs the specified operation
  4. Store: Results are saved to memory or a register

This cycle repeats continuously, with modern processors executing billions of cycles per second. Understanding this cycle helps explain why some operations are fast (data already in registers) while others are slow (waiting for data from disk).

4.4 The Central Processing Unit (CPU)

The CPU is often called the “brain” of the computer, though this metaphor oversimplifies its function. The CPU is really a very fast calculator that follows instructions precisely.

CPU Components

The CPU contains several critical components:

Control Unit (CU). Directs the operation of the processor, managing the fetch-decode-execute cycle.

Arithmetic Logic Unit (ALU). Performs mathematical calculations and logical comparisons.

Registers. Ultra-fast temporary storage locations (typically 64 bits each) for data being actively processed.

Cache. High-speed memory buffer that stores frequently accessed data to reduce trips to main memory.

Key CPU Concepts

Several metrics describe CPU performance:

Clock Speed. Measured in gigahertz (GHz), this indicates how many cycles the CPU can perform per second. A 3.5 GHz processor performs 3.5 billion cycles per second.

Cores. Modern CPUs contain multiple independent processing units (cores). An 8-core processor can theoretically perform 8 operations simultaneously.

Threads. Virtual cores that allow better utilization of physical cores through simultaneous multithreading (SMT) or hyper-threading.

Instruction Set. The low-level language the CPU understands. Common architectures include x86-64 (Intel/AMD) and ARM (Apple Silicon, most phones).

Note

More cores and higher clock speeds don’t always mean faster performance for your specific task. Many analyses are limited by memory access speed or storage I/O rather than raw CPU power.

4.5 Graphics Processing Units (GPUs)

While CPUs are designed for complex sequential tasks, GPUs excel at simple parallel operations. Originally designed for rendering graphics (where millions of pixels need similar calculations), GPUs have become essential for:

  • Machine learning and deep neural networks
  • Scientific simulations
  • Cryptocurrency mining
  • Image and video processing

GPU Architecture

GPUs differ fundamentally from CPUs:

Streaming Multiprocessors (SMs). Processing clusters containing many smaller cores.

CUDA Cores / Stream Processors. Thousands of simple processing units optimized for parallel computation.

Video Memory (VRAM). Dedicated high-bandwidth memory separate from system RAM.

CPU vs. GPU: When to Use Each

Table 4.2 contrasts the two processor types.

Table 4.2: Comparison of CPU and GPU characteristics
Characteristic CPU GPU
Design Focus Sequential, complex tasks Parallel, simple tasks
Core Count 4-64 powerful cores Thousands of smaller cores
Memory Large cache, lower bandwidth Smaller cache, higher bandwidth
Best For Operating systems, complex logic, serial tasks Graphics, ML/AI, simulations
Tip

If your computation involves applying the same operation to millions of data points (like image processing or matrix multiplication), GPUs can provide dramatic speedups. If your computation requires complex branching logic and sequential dependencies, CPUs are more appropriate.

4.6 Types of Computer Memory

Understanding the memory hierarchy helps explain performance characteristics of different operations.

Random Access Memory (RAM)

RAM provides temporary working space for programs and data currently in use. Key characteristics:

Volatile. Contents are lost when power is removed.

Random Access. Any memory location can be accessed with equal speed (unlike sequential access).

Bandwidth and Latency. Two different numbers describe how fast memory is: bandwidth is how much data can move per second (tens of gigabytes per second for modern DDR4/DDR5 memory), and latency is the delay before the first byte arrives (on the order of 100 nanoseconds). Running two memory modules in dual channel doubles the available bandwidth, which is why laptops are usually sold with matched pairs.

Types. There are two main kinds of RAM:

  • DRAM (Dynamic RAM): Main system memory (the DDR4 and DDR5 modules in a laptop or server); cheap and dense, but must be refreshed constantly
  • SRAM (Static RAM): Used in CPU caches; needs no refresh and is much faster, but far more expensive per byte

Modern systems typically have 8-64 GB of RAM, though scientific workstations and servers may have much more. For a concrete sense of scale, Table 3.4 in Chapter 3 compares a typical laptop (8-16 GB) with a single Talapas node (512 GB) and the cluster’s large-memory nodes (up to 4 TB).

TipThe usual bottleneck is RAM

For data analysis, the most common limit is not processor speed but memory: your data must fit in RAM while you work on it. When a program dies with “out of memory,” or your laptop grinds to a halt swapping to disk, the remedy is more RAM or a move to the cluster (Chapter 20), not a faster CPU. A workable rule of thumb: plan on RAM being three to five times the size of your data on disk, because a file expands when parsed into in-memory structures and analysis steps make copies. A 1 GB CSV is comfortable on a 16 GB laptop; a 10 GB one is not.

Persistent Storage

Unlike RAM, storage devices retain data without power:

Hard Disk Drives (HDDs)

  • Mechanical spinning platters with magnetic storage
  • Large capacities at low cost (many terabytes)
  • Slower access (~10ms latency, 100-200 MB/s throughput)
  • Susceptible to physical damage

Solid State Drives (SSDs)

  • No moving parts (NAND flash memory)
  • Much faster access (~0.1ms latency, 500-7000 MB/s throughput)
  • More expensive per gigabyte than HDDs
  • More durable and energy-efficient
Important

The memory hierarchy creates significant performance differences. Data in CPU registers can be accessed in nanoseconds, data in RAM in tens of nanoseconds, but data on disk takes milliseconds—a difference of six orders of magnitude!

The Memory Hierarchy

Figure 4.3 illustrates the tradeoffs between speed, capacity, and cost:

Figure 4.3: The memory hierarchy: faster storage is smaller and more expensive

Another way to see the same tradeoff is as a pyramid (Figure 4.4): each step down gains capacity but gives up speed.

Figure 4.4: The memory hierarchy as a pyramid. Moving up buys speed at the cost of capacity and price per byte: CPU registers respond in nanoseconds and RAM in tens of nanoseconds, while an SSD takes about 0.1 ms and a spinning hard disk about 10 ms per access.

4.7 Operating Systems

An operating system (OS) is the software layer that manages hardware resources and provides services to applications. Think of it as a translator between your programs and the physical computer.

How an Operating System Is Organized

Figure 4.5 shows the layers. Your applications (R, a web browser, a shell) never touch the hardware directly. They make system calls—requests such as “open this file” or “give me more memory”—to the kernel, the core of the OS that runs with full privileges. The kernel’s subsystems handle process management, memory management, the file system, and I/O, and talk to the physical hardware through device drivers, small pieces of code that know the details of one particular disk, network card, or GPU. This layering is what lets the same R script run on a laptop and a cluster node with completely different hardware.

Figure 4.5: Operating system architecture: applications reach the hardware only through system calls, the kernel, and device drivers

Core Functions

Operating systems handle:

Process Management. Creating, scheduling, and terminating programs. Managing how CPU time is shared among competing processes.

Memory Management. Allocating RAM to programs, implementing virtual memory, protecting programs from interfering with each other’s memory.

File System Management. Organizing data on storage devices, managing directories and permissions, handling read/write operations.

Device Management. Providing standardized interfaces to diverse hardware through device drivers.

Security. User authentication, access control, protection against malicious software.

CPU Scheduling

A computer runs far more processes than it has cores, so the OS must decide which process gets a core next. This is CPU scheduling, and the same ideas reappear at cluster scale when SLURM decides which job runs next (Section 20.6.1). The classic strategies are:

  • First-come, first-served (FCFS): simple, but one long process can make everything behind it wait
  • Shortest job first (SJF): gives the best average waiting time, if job lengths are known in advance
  • Round robin (RR): every runnable process gets a short time slice in turn, which keeps interactive programs responsive
  • Priority scheduling: important processes go first, usually with some aging so low-priority work eventually runs

Schedulers are judged by CPU utilization (keep the cores busy), throughput (jobs finished per unit time), turnaround time (submission to completion), waiting time (time spent in the ready queue), and response time (delay before the first visible output). Desktop operating systems weight responsiveness heavily; a cluster scheduler weights throughput and fairness instead.

Security and Protection

The OS also keeps programs and users from interfering with one another. The CPU itself runs in two modes: ordinary programs execute in user mode, and only the kernel executes in kernel mode, where it can touch hardware directly; a user-mode program that tries to access memory that is not its own is stopped by memory protection. On top of that, file permissions and access control lists decide who may read or write what (Section 6.2), user authentication verifies who you are, and sandboxing isolates untrusted processes (containers, discussed in Section 20.11.3, are a modern form of this).

Common Operating Systems

For scientific computing, you’ll encounter:

Linux. Open-source Unix-like OS dominant in servers and clusters, and heir to the design philosophy of the original Unix (Kernighan & Pike, 1984). Most bioinformatics tools assume Linux.

macOS. Apple’s Unix-based desktop OS. Provides native access to Unix tools.

Windows. Microsoft’s desktop OS. Now supports Linux tools through WSL (see Section 1.6.3).

NoteThe Principle of Least Privilege

A fundamental security concept: every program and user should operate using the minimum permissions necessary to accomplish their task. This limits the damage that can occur from mistakes or malicious actions.

4.8 Computing Environments

Research computing happens across a spectrum of environments, from your laptop to massive cloud data centers.

Local Computing

Local computing refers to resources physically present and directly controlled by you—your laptop, desktop, or lab workstation. Figure 4.6 shows the software stack on such a machine: applications sit on the operating system, which reaches the CPU, memory, storage, GPU, and network card through a hardware abstraction layer.

Figure 4.6: Local computing architecture: one machine, one operating system, all hardware reached through the OS

The appeal of local computing is control. The machine is yours, the software on it is whatever you choose to install, the data never leaves your desk, nothing depends on a network connection, and performance is predictable because no one else is using it. The costs are the mirror image of those strengths: the scale is fixed by the hardware you bought, you are responsible for maintenance and backups, a powerful workstation is a large one-time expense, and for most of every day and all of every night the machine sits idle.

Cluster Computing

A computing cluster is a collection of interconnected computers (nodes) working together as a single system. The University of Oregon’s Talapas is an example; Figure 4.7 shows the basic layout, and Chapter 20 covers using it.

Figure 4.7: Simplified cluster architecture with a head node, compute nodes, and shared storage

Key components include:

Head/Login Node. Where users log in and submit jobs. Not for running computations!

Compute Nodes. Worker machines that actually run your analyses.

Interconnect. A high-speed network (InfiniBand or 10+ gigabit Ethernet) linking the nodes to each other and to storage. It is much faster than an ordinary office network, and it is what makes a cluster behave like one machine rather than many.

Shared Storage. Parallel file systems such as Lustre or IBM’s GPFS (the one Talapas uses; Section 20.5) that every node sees identically, so a file written on the login node is immediately available to a job on any compute node.

Resource Manager. Software that schedules jobs and allocates resources fairly. Several exist (Table 4.3); Talapas uses SLURM (Yoo et al., 2003), for which Appendix I is a command reference.

Table 4.3: Resource managers used to schedule jobs on clusters
System Typical use Key features
SLURM Academic and national-lab HPC (Talapas) Fair-share scheduling, backfill, job arrays, power management
PBS / Torque Traditional HPC Mature, stable, widely supported
SGE / UGE (Grid Engine) Mixed workloads Flexible queue policies
Kubernetes Containerized services and cloud Microservices, auto-scaling

A cluster trades that control for scale. Because nodes can be added, a cluster grows to a size no single machine can reach, and because it is shared among many users and projects, it can justify specialized hardware such as GPUs and very-large-memory nodes that no individual lab would buy. It is professionally managed and maintained, and it tolerates failure: if one node dies, the others keep working. In return you must learn the job-scheduling system, accept that jobs may wait in a queue, compete with other users for resources, and live with less control over the software environment than on your own machine. Somebody also has to fund the space, power, and cooling, which is why access is organized through PIRGs and allocations (Chapter 3).

Cloud Computing

Cloud computing provides on-demand computing resources over the internet, with pay-as-you-go pricing. Major providers include Amazon Web Services (AWS, the market leader with hundreds of services), Microsoft Azure (strong enterprise integration), and Google Cloud Platform (GCP, with an AI/ML focus). Figure 4.8 sketches how a provider organizes its resources: each geographic region contains several independent availability zones, each with its own compute, storage, and networking, so that a failure in one zone does not take down the others.

Figure 4.8: Cloud computing architecture: regions, availability zones, and the compute, storage, and networking inside each

The cloud’s distinctive strength is elasticity: resources can be added or removed in minutes, there is no hardware to buy up front, the same services are available worldwide, and managed offerings such as hosted databases and notebooks remove much of the operational burden, with redundancy and disaster recovery built in. Its weaknesses are financial and contractual as much as technical. Pay-as-you-go costs can exceed those of an on-premises system for steady workloads; proprietary services create vendor lock-in; everything depends on a reliable internet connection; moving data out is billed as egress and can be surprisingly expensive; you have less control than on a local system and must answer compliance questions about where data may reside; and performance can vary with “noisy neighbors” sharing the same physical hardware.

Choosing the Right Environment

Table 4.4 summarizes the tradeoffs between the three environments.

Table 4.4: Comparison of computing environments
Factor Local Cluster Cloud
Scale Small Large Unlimited
Control Complete Moderate Limited
Cost Model Capital expense Shared allocation Pay-per-use
Learning Curve Low Moderate Moderate-High
Best For Development, small analyses Large batch jobs Burst computing, web services
Tip

A common workflow: develop and test locally, run production analyses on a cluster, use cloud resources for specialized needs or when cluster queues are full.

4.9 The Evolution of Scientific Computing

Computing infrastructure has evolved dramatically over the past several decades (Figure 4.9): from centralized mainframes in the 1960s and 70s, through personal computers in the 1980s and 90s and client-server systems in the 1990s and 2000s, to clusters and grids in the 2000s, cloud computing from about 2006 on, and today’s edge and hybrid arrangements.

Figure 4.9: Evolution of computing paradigms from mainframes to modern hybrid approaches

Key drivers of this evolution include:

  • Performance Needs: Ever-increasing computational demands from research
  • Economics: Pursuit of cost optimization and economies of scale
  • Connectivity: Internet bandwidth improvements enabling remote computing
  • Virtualization: Technologies that abstract hardware from software

Modern research computing increasingly uses hybrid approaches, combining local development with cluster and cloud resources based on the specific needs of each project.

4.10 Summary

This chapter provided foundational knowledge about computer systems:

  • Computers consist of hardware, software, firmware, and data working together
  • The CPU executes instructions through the fetch-decode-execute cycle
  • GPUs excel at parallel tasks; CPUs excel at sequential, complex operations
  • Memory hierarchy creates significant performance differences
  • Operating systems manage resources and provide services to applications
  • Computing environments range from local to cluster to cloud, each with tradeoffs

Understanding these concepts will help you choose appropriate resources for your analyses, interpret performance characteristics, and communicate effectively with system administrators.

Additional Reading

To deepen your understanding of computer systems architecture:

  • “Code: The Hidden Language of Computer Hardware and Software” by Charles Petzold — An accessible introduction to how computers work from first principles
  • “Computer Organization and Design” by Patterson and Hennessy — The classic textbook on computer architecture
  • Putting the “You” in CPU — A free, interactive online guide to how processors work
  • How Computers Really Work by Matthew Justice — A practical guide covering hardware, software, and everything in between
  • AWS, GCP, and Azure Documentation — Each cloud provider offers extensive free training on cloud computing concepts

For GPU computing and parallel processing:

Exercises

TipPractice Exercises
  1. System Exploration: On your computer, find out:

    • How many CPU cores does your system have?
    • How much RAM is installed?
    • What type of storage (SSD or HDD) do you have?
    • What operating system and version are you running?
  2. Memory Hierarchy: Explain why reading data from disk is so much slower than reading from RAM. What implications does this have for analyzing large datasets?

  3. Computing Environment Selection: For each scenario, which computing environment (local, cluster, or cloud) would you recommend and why?

    • Testing a new analysis script on a small dataset
    • Running a genome assembly requiring 256 GB of RAM
    • Hosting a web application for your lab’s research tools
    • Training a deep learning model requiring multiple GPUs for two weeks