Tutorial 09 — VS Code & SLURM Basics on Talapas

Connect to Talapas with VS Code Remote-SSH, explore the cluster interactively, and run your first SLURM jobs

Author

Single Cell RNA-seq Workshop

NoteRunning the code in this tutorial

Every code chunk here is tagged with #| eval: false, so the published page shows the code without running it. To run it yourself, open the downloaded .qmd in RStudio:

  • Run one chunk: click the green ▶ (Run Current Chunk) at the chunk’s top-right, or press Ctrl/Cmd + Enter. This runs the chunk regardless of its eval setting — the easiest way to work through the tutorial interactively.
  • Run a chunk on render: change that chunk’s #| eval: false to #| eval: true (or delete the line) so it executes when the document is rendered.
  • Render the whole tutorial: click the Render button in RStudio (or run quarto render in a terminal) to execute the chunks top-to-bottom and knit a finished HTML. To run everything on render, set eval: true once in the YAML header at the top.

About this tutorial

This is the first of the two Friday-morning Talapas modules (the second is Tutorial 10 — the analysis pipeline). It has two parts: first you connect VS Code to Talapas over Remote-SSH, explore the Lmod module system, and navigate the file systems; then you learn the basics of SLURM (the job scheduler on the University of Oregon’s Talapas HPC cluster) — both in interactive mode and through batch scripts — and end with an example batch script that runs the whole 01–08 analysis series. It is aimed at workshop participants who have a Duck ID and Talapas account and are connecting for the first time on Friday morning.

Note

Companion lecture: Lecture 09 — VS Code & SLURM Basics on Talapas · Companion reading: Chapter 9 — VS Code & SLURM Basics on Talapas

All examples are specific to UO and Talapas. The VS Code Remote-SSH connection to Talapas is the first step of this module (Section 2 below) — you do not need any prior cluster login to start here. Material is adapted from the Talapas Quick Start Guide and the How-To articles in the Talapas Knowledge Base, and from Shannon Snyder’s Talapas Basics workshop guide (the hello_world.R / salloc / job-monitoring walkthroughs below).

TipHow to use this page

Read this page top to bottom the first time. Later, treat it as a reference — the commands here cover ~95% of what you will do in the workshop.

1. Why SLURM?

Talapas is a shared cluster with thousands of CPU cores, hundreds of GPUs, and many terabytes of RAM. Dozens of users compete for those resources at any given time. SLURM (Simple Linux Utility for Resource Management) is the scheduler that decides when, where, and with what resources each job runs.

At UO, this means:

  • You never run heavy computation directly on a login node (login.talapas.uoregon.edu). Login nodes are for editing, short commands, and submitting jobs only.
  • Instead, you ask SLURM for resources — CPUs, GPUs, memory, time — and SLURM allocates a compute node for you.
  • Every job at UO must specify an account (your PIRG, Principal Investigator Research Group) for correct service charging.

2. Prerequisites

Before you start this module, you need:

  1. The P1 and P2 (Tuesday) material done on your laptop — P1 — Computer Systems & the Command Line and P2 — R & RStudio. You should have VS Code installed, the Quarto and R extensions installed, and have run R locally.
  2. A UO Duck ID and your University-wide UO password.
  3. A Talapas account tied to a PIRG (Principal Investigator Research Group). Ask your PI to add you to their PIRG, or request access at racs.uoregon.edu/request-access.
  4. If you are off campus, connect to the UO VPN first — the Talapas login nodes are only reachable from the campus network or the VPN.

Throughout this module, replace <yourduckid> with your Duck ID and <myPIRG> with your PIRG name. If you are not sure which PIRG you belong to, you will find out in Section 4 by running groups once you are logged in.

NoteTalapas account not set up yet?

If you do not yet have a Talapas account, request one at racs.uoregon.edu/request-access before Friday morning. Account provisioning can take 1–2 business days.

3. Connect VS Code to Talapas

3.1 Install the Remote-SSH extension

In VS Code’s Activity Bar (left edge), click the Extensions icon (four-squares) and search for and install:

  • Remote — SSH — lets you open folders and run terminals on Talapas directly from VS Code.
  • Remote Explorer — the sidebar panel for managing SSH hosts.

After installing, a new Remote Explorer icon appears in the Activity Bar.

NoteQuarto and R extensions from P1

You installed the Quarto and R extensions on your laptop in the Chapter P1 session. VS Code installs extensions per host, so once you connect to Talapas you will be prompted to install them on the remote side as well — do so when asked.

3.2 Add the Talapas SSH host

  1. Open Remote Explorer from the Activity Bar.
  2. Make sure the dropdown at the top is set to SSH (sometimes labelled SSH Targets).
  3. Click the + (“Add New”) icon, and in the prompt that appears type the SSH command below, replacing <yourduckid> with your Duck ID:
#| label: M9-ssh_yourduckid_login_talapas
#| eval: false
ssh <yourduckid>@login.talapas.uoregon.edu
  1. Press Enter, then Enter again to save it to the default SSH config file (~/.ssh/config).

login.talapas.uoregon.edu now appears as a host under SSH in the Remote Explorer.

3.3 Connect (Duck ID + Duo)

  1. In Remote Explorer, find login.talapas.uoregon.edu and click the → (Connect) arrow — choose to connect in the current window or a new one.
  2. If asked about the platform of the remote host, choose Linux.
  3. Enter your Duck ID password when prompted. For security, no characters appear as you type — that is expected.
  4. When prompted for Duo, type 1 and press Enter to receive a Duo push (typing 2 sends a text, 3 calls you). Approve the push on your phone.
  5. Once connected, a green banner reading SSH: login.talapas.uoregon.edu appears in the bottom-left corner of the window.

If the connection hangs or is refused: confirm you are on the UO VPN (if off campus), that your Duck ID password is current, and that your Talapas account is active.

ImportantThink about it

You are now connected, and the green banner says login.talapas.uoregon.edu. Which kind of node are you on — and what are you allowed to do here?

Show answers

You are on a login (head) node. Login nodes are shared by everyone and are for light tasks only: browsing and editing files, managing data, and submitting jobs. They are not for running real analysis — heavy processes on a login node will be throttled or killed, and they slow the node down for every other user. To do actual computing you must move onto a compute node, which you do through SLURM in Sections 7–8.

4. Open a terminal and explore the cluster

4.1 Identity and groups

Open VS Code’s integrated terminal with Terminal → New Terminal (or Ctrl+`). The prompt will show one of the Talapas login nodes (e.g. login1).

#| label: M9-whoami_duck_id
#| eval: false
whoami                 # your Duck ID
groups                 # your PIRG(s) appear alongside system groups
pwd                    # where am I? -> /home/<yourduckid>
ImportantThink about it

Why do you need a PIRG just to run a job? What is it used for?

Show answers

A PIRG is your research group’s account on Talapas. SLURM uses it to track and charge resource usage to the right group, and to enforce each group’s share of the cluster. Every job you submit — interactive or batch — must name an account with --account=<myPIRG>, or it will be rejected.

4.2 Lmod modules

Talapas manages centrally installed software with Lmod modules. Software is not available until you module load it — and modules are loaded per shell, so you reload them in every new session and in every job script.

#| label: M9-module_avail_list_modules
#| eval: false
module avail           # list modules you can load right now
module spider R        # search for R and see which versions exist
module load R          # load the default R
module list            # show what is currently loaded
R --version            # confirm R is now on your PATH
module purge           # unload everything (reset) when you want a clean slate
ImportantThink about it

You ran R --version before module load R and got “command not found”, but after loading it worked. Why isn’t R just always available?

Show answers

On a shared cluster, many versions of many tools coexist, and different projects need different (sometimes conflicting) versions. Rather than putting everything on everyone’s PATH at once — which would cause clashes — Talapas keeps software modular: nothing analysis-related is loaded by default, and you opt in to exactly what you need with module load. Because this happens per shell, a fresh terminal (or a new job) starts clean, so you must load your modules again. For reproducibility, pin the version — module load R/4.4.1 rather than bare R — so you get the same R every time.

5. Key SLURM commands at a glance

Command Purpose
sinfo Show partitions and node states.
squeue -u $USER Show your queued/running jobs.
sbatch script.sh Submit a batch job.
srun ... --pty bash Start an interactive session on a compute node.
scancel <JOBID> Cancel a queued or running job.
sacct -j <JOBID> Show accounting info (state, memory, runtime) for past jobs.
seff <JOBID> Friendly summary of resources used by a finished job.
NoteKey SLURM flags you will use constantly
  • --account=<myPIRG> — PIRG / account for charging (required).
  • --partition=<name> — queue (e.g. compute, gpu, interactive).
  • --time=<HH:MM:SS> or --time=<minutes> — wall-clock time limit.
  • --mem=<size> — total memory per node (e.g. 8G, 100M).
  • --mem-per-cpu=<size> — memory per core.
  • --cpus-per-task=<N> — cores for a multi-threaded task.
  • --gpus=<N> — request GPUs (paired with --partition=gpu).
  • --constraint=gpu-10gb / gpu-40gb / gpu-80gb — GPU memory tier.

6. Talapas partitions (queues)

Every job must target a partition. Common ones at UO:

Partition What it’s for Max time
compute Standard CPU jobs. 1 day
computelong Long-running CPU jobs. 14 days
gpu / gpulong GPU jobs, short / long. 1 / 14 d
memory / memorylong High-memory CPU jobs (up to 4 TB RAM). 1 / 14 d
interactive / interactivegpu Short interactive sessions. 12 / 8 h
preempt All nodes; your job may be preempted by its owner. Variable

Run sinfo to see the current state of each partition on the cluster.

7. Interactive mode — srun

Use an interactive session when you want a shell on a compute node — for testing scripts, exploring data, or running applications like RStudio or Python interactively. This is the mode you will use most often during the workshop.

NoteInteractive vs. batch — which one?

Use an interactive session (srun / salloc) when you’ll sit and watch something run: testing or debugging a script, running a few commands by hand, exploring data, or running something very short. Use a batch job (sbatch, §8) for anything long-running, or anything you want to start and walk away from — a batch job keeps running after you log out.

7.1 A minimum interactive session

From a Talapas login node, run:

#| label: M9-srun_account_mypirg
#| eval: false
srun --account=<myPIRG> \
     --partition=compute \
     --time=1:00:00 \
     --mem=4G \
     --cpus-per-task=2 \
     --pty bash

What each flag does:

  • --account — your PIRG (required on Talapas; omit if you set SLURM_ACCOUNT in your profile).
  • --partition=compute — standard CPU queue.
  • --time=1:00:00 — 1 hour; after this the session is killed.
  • --mem=4G — 4 GB of RAM total on that node.
  • --cpus-per-task=2 — two CPU cores.
  • --pty bash — give me a pseudo-terminal running bash.

When the resources are available, your prompt will change to show the hostname of a compute node (e.g. n0123). You are now running on a compute node. When you’re done, type exit to release the resources — do not leave idle sessions running.

salloc — the close cousin of srun --pty bash. Run salloc from a login node to request an interactive allocation; when it’s granted, your shell prompt changes from the login node to the compute node (e.g. n0135) and you work there directly:

#| label: M9-allocation_granted_prompt_changes
#| eval: false
salloc --account=<myPIRG> --partition=compute --time=1:00:00 --mem=4G --cpus-per-task=2
# allocation granted → prompt changes, e.g.  [duckid@n0135 ~]$

srun --pty bash does the request and hands you a shell in one step; salloc is handy when you want a persistent allocation you can launch several srun steps into. Either way, exit releases the node.

7.2 Using the short interactive partition

For short, quick sessions the interactive partition is usually faster to start:

#| label: M9-srun_account_mypirg_partition
#| eval: false
srun --account=<myPIRG> --partition=interactive --time=2:00:00 --mem=8G --pty bash

7.3 Interactive GPU session

#| label: M9-srun_account_mypirg_2
#| eval: false
srun --account=<myPIRG> \
     --partition=interactivegpu \
     --time=1:00:00 \
     --nodes=1 --ntasks=1 --cpus-per-task=2 \
     --mem=16G \
     --gpus=1 --constraint=gpu-10gb \
     --pty bash

--constraint=gpu-10gb (or gpu-40gb / gpu-80gb) pins the job to a GPU of that memory tier.

7.4 Graphical interactive jobs

If you need a GUI (e.g. RStudio, MATLAB), two options:

  1. X11 forwarding — SSH in with ssh -Y and then:

    #| eval: false
srun --account=<myPIRG> --x11 --partition=compute \
     --time=2:00:00 --mem=4G --pty bash

    You need an X server running on your laptop (XQuartz on macOS, VcXsrv on Windows).

  2. Open OnDemand (recommended and simpler) — open https://ondemand.talapas.uoregon.edu/ in Chrome or Firefox, sign in with your Duck ID, and launch an RStudio, Jupyter, or full desktop session from the web portal. No X server needed.

WarningDon’t hoard nodes

Interactive jobs reserve their resources even when you are not actively typing. As soon as you are done, exit the shell so other users can use the node.

7.5 Hands-on: run hello_world.R on an interactive node

The Talapas scripts folder ships a tiny demo, hello_world.R — open it in VS Code to read it (note the #!/usr/bin/env Rscript shebang on the first line). It uses only base R, prints which compute node it ran on, and writes a PDF plot and a summary CSV.

From an interactive session (above), load R and run it:

#| label: M9-module_load_r
#| eval: false
module load R
Rscript hello_world.R

It reports the node it ran on and writes two files in the directory you ran it from:

hello_world_plot.pdf
hello_world_summary.csv
WarningBe in the script’s directory — or give the full path

Rscript hello_world.R only finds the script if you are in the same directory as it. Otherwise supply the path, e.g. Rscript /projects/<myPIRG>/<yourduckid>/scrnaseq/scripts/hello_world.R. The same is true for sbatch <script>.

TipView the outputs in VS Code

With VS Code connected to Talapas (Section 3 above) and the PDF/CSV viewer extensions installed, double-click hello_world_plot.pdf or hello_world_summary.csv in the file explorer to open them right in the editor — no download needed.

ImportantThink about it

The output says Hello from n0123. On your laptop earlier (in the P1 session), the same script said Hello from <your-laptop-name>. Why does the hostname tell you where the job actually ran — and why does confirming a compute-node name matter?

Show answers

The script prints Sys.info()[["nodename"]], so the hostname is proof of execution location. Seeing a compute node (n0123) confirms you ran on the node SLURM allocated — not on the login node. Running heavy analysis on the login node is the golden rule violation: it slows the machine for every other user and gets your process killed. The hostname test is a quick sanity check every time you run a job.

End the session. When you are done, exit the interactive shell to release the node back to the cluster:

#| label: M9-exit_interactive_session
#| eval: false
exit          # ends the srun/salloc session; prompt returns to the login node

8. Batch mode — sbatch

For longer-running analyses (which is most real workshop work), write a batch script and submit it with sbatch. The script runs without an interactive terminal: you can log out and the job keeps running.

ImportantCreate the logs/ directory first (one time)

The shipped scripts (slurm_demo.sbatch, run_rscript.sbatch, and the analysis-step examples below) write their .out/.err logs to a logs/ subdirectory via #SBATCH --output=logs/%x_%j.out. SLURM does not create this directory for you — if it is missing, the job is rejected (or dies instantly with no log to explain why). From the scripts/ folder you submit jobs from, run this once:

#| label: M9-mkdir_p_logs
#| eval: false
mkdir -p logs

A batch job’s lifecycle: you sbatch from a login node; SLURM queues it (PD), runs it on a compute node (R), and writes your results and .out/.err logs. Monitor with squeue, cancel with scancel.

8.1 A first batch script

Save the following as hostname.sbatch:

#| label: M9-bin_bash
#| eval: false
#!/bin/bash
#SBATCH --account=<myPIRG>        ### PIRG / account for charging
#SBATCH --partition=compute       ### queue
#SBATCH --job-name=hello          ### visible in squeue
#SBATCH --output=hello_%j.out     ### %j = job ID
#SBATCH --error=hello_%j.err
#SBATCH --time=00:05:00           ### HH:MM:SS
#SBATCH --nodes=1
#SBATCH --ntasks-per-node=1
#SBATCH --cpus-per-task=1
#SBATCH --mem=100M

echo "Hello from $(hostname) at $(date)"

Submit it:

#| label: M9-sbatch_hostname_sbatch
#| eval: false
sbatch hostname.sbatch

SLURM prints a job ID:

Submitted batch job 123456

Check its status:

#| label: M9-or
#| eval: false
squeue -j 123456
# or
squeue -u $USER

State codes you’ll see: PD (pending), R (running), CG (completing). Once it finishes, two files appear in the directory you submitted from: hello_123456.out (stdout) and hello_123456.err (stderr).

NoteThis exact demo ships with the workshop

You don’t have to type it out: the Talapas scripts folder includes slurm_demo.sbatch (the same idea). Open it in VS Code, set --account, and sbatch slurm_demo.sbatch. Because it’s a very short job, it may finish before you can catch it in squeue.

8.2 Anatomy of #SBATCH directives

All #SBATCH lines must be at the top of the script, before any other commands. A few additional flags worth knowing:

#| label: M9-sbatch_mem_per_cpu
#| eval: false
#SBATCH --mem-per-cpu=4G          ### per-core memory (alternative to --mem)
#SBATCH --mail-type=END,FAIL      ### email on end/failure
#SBATCH --mail-user=you@uoregon.edu
#SBATCH --array=1-10              ### job array: 10 independent tasks

8.3 A realistic scRNA-seq job

A batch script that loads R and runs a Seurat QC pipeline looks like this:

#| label: M9-bin_bash_2
#| eval: false
#!/bin/bash
#SBATCH --account=<myPIRG>
#SBATCH --partition=compute
#SBATCH --job-name=seurat_qc
#SBATCH --output=logs/seurat_qc_%j.out
#SBATCH --error=logs/seurat_qc_%j.err
#SBATCH --time=04:00:00
#SBATCH --nodes=1
#SBATCH --ntasks-per-node=1
#SBATCH --cpus-per-task=8
#SBATCH --mem=64G

set -euo pipefail

# Load environment
module purge
module load R/4.3.2

# Run the pipeline
cd /projects/<myPIRG>/<yourduckid>/scrnaseq
Rscript scripts/01_qc_preprocessing.R

Note how module purge and module load happen inside the script — modules don’t survive from the login node into the job.

8.4 GPU batch job

#| label: M9-bin_bash_3
#| eval: false
#!/bin/bash
#SBATCH --account=<myPIRG>
#SBATCH --partition=gpu
#SBATCH --job-name=scvi
#SBATCH --output=logs/scvi_%j.out
#SBATCH --error=logs/scvi_%j.err
#SBATCH --time=12:00:00
#SBATCH --nodes=1
#SBATCH --ntasks-per-node=1
#SBATCH --cpus-per-task=8
#SBATCH --mem=64G
#SBATCH --gpus=1
#SBATCH --constraint=gpu-40gb

module purge
module load python/3.11
source ~/envs/scvi/bin/activate

python scripts/train_scvi.py

8.5 Running an R script as a batch job (the wrapper)

You don’t need a separate .sbatch file for every R script. The folder ships a generic wrapper, run_rscript.sbatch, whose last line is simply:

#| label: M9-rscript_script
#| eval: false
Rscript "${SCRIPT}"

Pass the R script as an argument and override the job name per run:

#| label: M9-or_real_analysis_step
#| eval: false
sbatch --job-name=hello run_rscript.sbatch hello_world.R
# or a real analysis step:
sbatch --job-name=qc    run_rscript.sbatch 01_qc_preprocessing.R

Then watch the queue and, when it finishes, read the logs (or open them in VS Code):

#| label: M9-squeue_u_user
#| eval: false
squeue -u $USER
cat logs/hello_*.out      # stdout
cat logs/hello_*.err      # stderr / R errors

This is exactly how the rest of the Talapas track is run — every analysis step (the numbered pipeline scripts in Tutorial 10 — the analysis pipeline) is an R script submitted through this wrapper.

8.6 Example: run the whole 01–08 series in one job

When you want to reproduce the entire core analysis (Modules 01–08) end-to-end rather than one step at a time, the folder also ships run_core_pipeline.sbatch. It runs the eight numbered scripts in order inside a single allocation — each step’s .rds/.csv output is the next step’s input:

#| label: M9-sbatch_one_allocation_12
#| eval: false
#SBATCH ... (one allocation, ~12 h, 96 GB)
STEPS=(01_qc_preprocessing.R 02_dimreduction_clustering.R … 08_differential_abundance.R)
for step in "${STEPS[@]}"; do
  echo "==== RUNNING ${step} ===="
  Rscript "${step}"
done

Submit it once (after mkdir -p logs and pre-caching ifnb on a login node):

#| label: M9-sbatch_core_pipeline_sbatch
#| eval: false
sbatch run_core_pipeline.sbatch
squeue -u $USER
seff <jobid>                 # confirm State: COMPLETED
tail -f logs/ifnb_pipeline_*.out

This is the simplest way to “run the series of R scripts” on the cluster. For finer control — running one step at a time, or right-sizing memory per step — use run_rscript.sbatch (above) as documented in Module 10.

9. Choosing the right resources

  • Smaller requests run sooner. SLURM finds small gaps in the schedule more easily, so over-requesting memory or time will delay your job.
  • But specify enough. If your job exceeds its --mem or --time, SLURM kills it (OUT_OF_MEMORY or TIMEOUT). Err slightly on the high side the first time you run a new pipeline.
  • The default per-core memory is about 3.7 GB. Request --mem-per-cpu or --mem explicitly if you need more.
  • The compute partition default wall time is 24 hours; computelong defaults to two weeks. Both cap at those values.

After a job finishes, check what it actually used:

#| label: M9-seff_123456
#| eval: false
seff 123456

or:

#| label: M9-sacct_j_123456_format
#| eval: false
sacct -j 123456 --format=JobID,JobName,ReqMem,MaxRSS,Elapsed,State,ExitCode

Use MaxRSS (peak memory) and Elapsed (wall time) to right-size future submissions.

10. Monitoring, cancelling, and troubleshooting

#| label: M9-see_jobs
#| eval: false
# See your jobs
squeue -u $USER

# Watch a specific job
squeue -j 123456

# Cancel a job
scancel 123456

# Cancel all your pending jobs
scancel -u $USER -t PENDING

# Summary of a finished job
seff 123456

# Detailed accounting (multiple steps)
sacct -j 123456 --format=JobID,State,ExitCode,Elapsed,MaxRSS,ReqMem

Common reasons a job stays PD (pending):

  • Priority / Resources — normal queueing; too many users ahead of you, or the resources you asked for are not free yet.

  • QOSMaxJobsPerUserLimit — you have hit a per-user submission limit; let earlier jobs finish.

  • ReqNodeNotAvail, Reserved for maintenance — your --time would run into an upcoming quarterly maintenance window. Shorten --time, or add --time-min= to let SLURM start the job sooner with a reduced limit:

    #| eval: false
#SBATCH --time=0-08:00:00
#SBATCH --time-min=0-01:00:00

See also the Knowledge Base FAQ Why is my SLURM job not running?.

11. Being a good cluster citizen at UO

Talapas is shared across all of UO’s research community. Please:

  • Never run intensive work on login nodes; use srun or sbatch.
  • Don’t leave idle interactive sessions open. exit when you’re done.
  • Don’t request more CPUs, memory, or GPUs than you will actually use.
  • Put large output (caches, intermediate files) in /scratch/<PIRG>/, not in /home/.
    • Remember that /scratch purges files not accessed in 90 days.
  • Back up anything you care about. RACS does not back up Talapas.
  • If you have a deadline and need special scheduling, email RACS rather than trying to game the queue.

12. Where to go next