Week 4 Exercises - GREP in Practice, Shell Scripts, sed and awk
Short, ungraded practice tasks - about 25-35 minutes. Companion reading: Ch. 10 and Ch. 9 - review the grep/regex sections, and read the sed and awk sections carefully: in class we only learned them by example.
TipAdjust as you go
Commands copied from the lecture slides may need small tweaks to run in your project - most often a file path: the slides often use data/, while your Week 2 skeleton keeps the raw file in data/raw/. Spotting and fixing these little mismatches is not busywork; it is a core skill, and the error messages are your friend.
GREP on real data
- Warm-up from the L04 reminder: run
grep -c "mono" data/raw/Stickle_RNAseq.tsvandgrep -cw "mono" data/raw/Stickle_RNAseq.tsv. Both give 40 - work out which column each one matches on, then construct a pattern where adding-wwould change the answer. - Download human chromosome 21 into
data/raw/(thewget/curlcommand is on the L04 slides) and decompress a copy withgunzip -k. Then finish the lecture’s “Your turn” tasks if we ran out of time in class: total bases and GC content ofchr21.fa. Save your working pipelines inscripts/- you will want them for the homework.
Shell scripts
- In Week 2 you made
scripts/hello.sh(if you skipped that, the recipe is on the L04 “mechanics” slide). Make it greet whoever is passed on the command line using$1:./hello.sh Billshould print “Hello, Bill”. - Write a
forloop (in a script or at the prompt) that prints the name of every.tsvfile underdata/. - Add a safety check to a script:
if [ ! -f "$1" ]; then echo "no such file" >&2; exit 1; fi. Test it with a real and a fake filename. - Add
set -euo pipefailnear the top of one of your scripts, then introduce a typo in a command and observe how the script now stops instead of plowing on. - Write a function
count_groupthat takes a file and a group name and prints how many fish are in that group (the L04 functions slide is your template). Call it in a loop overconvA convB monoA monoB- do you get 20 of each?
sed and awk
- Fix the awkward column name exactly as in lecture, but matched to your project layout:
sed '1s/Geno&Micro/Geno_Micro/' data/raw/Stickle_RNAseq.tsv > data/processed/stickle_clean.tsv. Verify withhead -n 1 data/processed/stickle_clean.tsv | cut -f1-4. - Use
awkto print the Individual and Genotype columns (awk -F'\t' '{print $1, $2}' data/processed/stickle_clean.tsv | head), then compute the mean ofGene1(column 5) - the command is on the L04 slides. - Copy
gene_means.shfrom the lecture intoscripts/, change itsinfile=line to point at yourdata/processed/stickle_clean.tsv, make it executable, and run it for column 5. Then loop it over columns 5-7. (Annoyed that you had to edit a hardcoded path? That is best practice #6 from the last slide.) - Re-run one of your scripts from a different working directory. Does it still work? If not, explain why (hint: relative paths).