Week 2 Exercises - Files, Pipes, and First Shell Scripts

Short, ungraded practice tasks - about 30-40 minutes. Companion reading: Ch. 6 and Ch. 7, with Ch. 5 as review.

TipAdjust as you go

Commands copied from the lecture slides may need small tweaks to run in your project - most often a file path: the slides often use data/, while your Week 2 skeleton keeps the raw file in data/raw/. Spotting and fixing these little mismatches is not busywork; it is a core skill, and the error messages are your friend.

Files, directories, and pipes

  1. Inside ~/projects/comptools, build a project skeleton in one command: mkdir -p data/raw data/processed scripts output/figures output/tables docs - the structure from Lecture 01.
  2. Run ls -F and confirm the directories are there (the / suffix marks directories).
  3. Create a README.md with nano: type one sentence describing the project, save (Ctrl+O, Enter), and exit (Ctrl+X). Print it with cat.
  4. Download the course dataset into data/raw (the command is on the L02 slides and in Ch. 1). If you already downloaded it, cp or mv it into data/raw.
  5. How many lines does Stickle_RNAseq.tsv have (wc -l)? Given 80 fish, why that number?
  6. Look at just the first three lines with head -n 3. Too wide to read? Try cut -f1-4 data/raw/Stickle_RNAseq.tsv | head -n 3 to see only the first four columns.
  7. In a scratch directory, create file_1.txt, file_2.txt, and file_3.txt with one touch command, then list them with the wildcard file_?.txt.
  8. Save a directory listing to a file with redirection: ls -lh data/raw > output/data_inventory.txt, then cat it.

Shell scripts and permissions

  1. In scripts/, use nano to create hello.sh containing the two lines from the L02 slides (the #!/usr/bin/env bash shebang and an echo). Try ./hello.sh - read the error out loud.
  2. Run ls -l hello.sh and decode the ten-character permission string: who can read it? Write it? Run it?
  3. Fix it with chmod u+x hello.sh, check ls -l again to see what changed, and run it. Then try bash hello.sh - why does that work even without x?
  4. Download the five example scripts from the Example Files page into scripts/ and make them all executable with one command: chmod +x *.sh.
  5. Run ./file_inventory.sh ~/projects/comptools inventory.txt and open the report. Which file type takes the most space in your project?
  6. Try the two-script pipeline on the dataset: ./filter_rows.sh Genotype A < ../data/raw/Stickle_RNAseq.tsv | ./total_counts.sh | sort -t$'\t' -k2,2nr | head -5. Which fish has the most reads? Rerun for genotype B.
  7. Scripts are just text, so modify one and run it again: open total_counts.sh in nano, change the header it prints from Total_counts to Total_reads, save, and rerun the pipeline from the previous task. Two things to notice: your change shows up immediately, and no new chmod was needed - the execute permission belongs to the file, not to its contents.
  8. Make a dated snapshot of your whole project: cd ~/projects && comptools/scripts/backup_project.sh comptools. List the archive’s contents with tar -tzf <archive-name> - your output/ folder should be missing while everything else is there. Read the script’s comments to see why output/ is treated as expendable.
  9. Protect your raw data the Unix way: chmod a-w data/raw/*, then try echo "oops" >> data/raw/Stickle_RNAseq.tsv and watch the shell refuse. (Undo later with chmod u+w if you ever need to - but should you?)

Done early? Run ./make_project.sh practice_project and compare its layout to the skeleton you built by hand in task 1, then read the script’s comments to see how it works.