Week 3 Exercises - Tidy Data, GREP, and Regular Expressions
Short, ungraded practice tasks - about 20-30 minutes. Companion reading: Ch. 8 and Ch. 9, with Ch. 7 as review; reference: App. D. All tasks use data/raw/Stickle_RNAseq.tsv from last week. (We finish grep - and take it to a whole human chromosome - at the start of Week 4, so this week’s practice is the foundation.)
TipAdjust as you go
Commands copied from the lecture slides may need small tweaks to run in your project - most often a file path: the slides often use data/, while your Week 2 skeleton keeps the raw file in data/raw/. Spotting and fixing these little mismatches is not busywork; it is a core skill, and the error messages are your friend.
- Count the fish raised with the “mono” microbiota:
grep -c "mono" data/raw/Stickle_RNAseq.tsv. Does the answer match the experimental design (Ch. 1)? - Print only the Genotype column and tabulate it:
cut -f2 data/raw/Stickle_RNAseq.tsv | sort | uniq -c. What do you get, and why is there one odd line? - Use
grep -nto find which line number fishBM7is on. - Use
grep -vto print every line that is not genotype A (careful: think about what string you match), and count them by piping towc -l. - Chain three commands with two pipes: extract column 3, sort it, and count unique values in one line.
- Compress a copy of the dataset with
gzip -kand compare sizes withls -lh. Then read the compressed file without uncompressing it:zcat(Linux) orgzcat(macOS) piped intohead. - Use an extended regular expression to match gene column names:
head -n 1 data/raw/Stickle_RNAseq.tsv | tr '\t' '\n' | grep -Ec "Gene[0-9]+". How many gene columns are there? - Page through the file with
less, search formonowith/mono, jump between matches withn, and quit withq. - In
man grep, find what-idoes, then demonstrate it by matchingMONO. - Write one pipeline of your own that answers a question about the file, and save it in a text file in
scripts/so you have it for the homework.
Is it tidy?
- Look at the data the way a statistician would:
cut -f1-6 data/raw/Stickle_RNAseq.tsv | head -n 3. One row per fish, one column per variable, a header line of descriptive names - which of the tidy-data rules of thumb from L03 (and Ch. 8) does this file already satisfy? - Now find the flaw: print columns 2-4 with
cut -f2-4 | head. TheGeno&Microcolumn just glues together the two columns before it - which tidy rule does that break? And tryecho Geno&Microin your shell to see why that&in a name will cause trouble far beyond R. - Wide or long? Count how many values one fish’s row holds:
head -n 1 data/raw/Stickle_RNAseq.tsv | tr '\t' '\n' | wc -l. For per-gene questions each gene needs to be a row, not a column - how many rows would the long version of this table have (80 fish x 600 genes)? We will do that reshape, in one line of R, in Week 6.