Week 2 Exercises - Files, Pipes, and First Shell Scripts
Short, ungraded practice tasks - about 30-40 minutes. Companion reading: Ch. 6 and Ch. 7, with Ch. 5 as review.
TipAdjust as you go
Commands copied from the lecture slides may need small tweaks to run in your project - most often a file path: the slides often use data/, while your Week 2 skeleton keeps the raw file in data/raw/. Spotting and fixing these little mismatches is not busywork; it is a core skill, and the error messages are your friend.
Files, directories, and pipes
- Inside
~/projects/comptools, build a project skeleton in one command:mkdir -p data/raw data/processed scripts output/figures output/tables docs- the structure from Lecture 01. - Run
ls -Fand confirm the directories are there (the/suffix marks directories). - Create a
README.mdwithnano: type one sentence describing the project, save (Ctrl+O, Enter), and exit (Ctrl+X). Print it withcat. - Download the course dataset into
data/raw(the command is on the L02 slides and in Ch. 1). If you already downloaded it,cpormvit intodata/raw. - How many lines does
Stickle_RNAseq.tsvhave (wc -l)? Given 80 fish, why that number? - Look at just the first three lines with
head -n 3. Too wide to read? Trycut -f1-4 data/raw/Stickle_RNAseq.tsv | head -n 3to see only the first four columns. - In a scratch directory, create
file_1.txt,file_2.txt, andfile_3.txtwith onetouchcommand, then list them with the wildcardfile_?.txt. - Save a directory listing to a file with redirection:
ls -lh data/raw > output/data_inventory.txt, thencatit.
Shell scripts and permissions
- In
scripts/, usenanoto createhello.shcontaining the two lines from the L02 slides (the#!/usr/bin/env bashshebang and anecho). Try./hello.sh- read the error out loud. - Run
ls -l hello.shand decode the ten-character permission string: who can read it? Write it? Run it? - Fix it with
chmod u+x hello.sh, checkls -lagain to see what changed, and run it. Then trybash hello.sh- why does that work even withoutx? - Download the five example scripts from the Example Files page into
scripts/and make them all executable with one command:chmod +x *.sh. - Run
./file_inventory.sh ~/projects/comptools inventory.txtand open the report. Which file type takes the most space in your project? - Try the two-script pipeline on the dataset:
./filter_rows.sh Genotype A < ../data/raw/Stickle_RNAseq.tsv | ./total_counts.sh | sort -t$'\t' -k2,2nr | head -5. Which fish has the most reads? Rerun for genotype B. - Scripts are just text, so modify one and run it again: open
total_counts.shinnano, change the header it prints fromTotal_countstoTotal_reads, save, and rerun the pipeline from the previous task. Two things to notice: your change shows up immediately, and no newchmodwas needed - the execute permission belongs to the file, not to its contents. - Make a dated snapshot of your whole project:
cd ~/projects && comptools/scripts/backup_project.sh comptools. List the archive’s contents withtar -tzf <archive-name>- youroutput/folder should be missing while everything else is there. Read the script’s comments to see whyoutput/is treated as expendable. - Protect your raw data the Unix way:
chmod a-w data/raw/*, then tryecho "oops" >> data/raw/Stickle_RNAseq.tsvand watch the shell refuse. (Undo later withchmod u+wif you ever need to - but should you?)
Done early? Run ./make_project.sh practice_project and compare its layout to the skeleton you built by hand in task 1, then read the script’s comments to see how it works.