Hackerrank Linux Text Processing
Paste
Good resource: http://www.theunixschool.com/2012/07/10-examples-of-paste-command-usage-in.html
Source: https://www.hackerrank.com/challenges/paste-2/problem
Task: "You are given a CSV file where each row contains the name of a city and its state separated by a comma. Your task is to restructure the file so that three consecutive rows are folded into one line and are separated by semicolons."
You can introduce repeating delimiters; paste is smart enough to fit this delimiter pattern:
paste -s -d ';;\n'
But I learned from the discussion that - reads one line from standard input, so this also works:
paste -d ';' - - -
Another assignment asked for replacing newlines with tabs using space. But it seems this is really not the primary use of paste. Using translate seems more appropriate:
Use ANSI escaped strings to expand \n and \t:
tr $'\n' $'\t
Get the middle of a text file
First cut of the tail with head, then cut of the head of the head with tail. From line 12 tot 22:
head -22 | tail -1
Translate tr
Source: https://www.hackerrank.com/challenges/text-processing-tr-1/problem
Input:
int i=(int)5.8
(23 + 5)*2
Output:
int i=[int]5.8
[23 + 5]*2
Solution:
tr "()" "[]
Sort numerical second column of tab-separated file
Source: https://www.hackerrank.com/challenges/text-processing-sort-5/problem
I learned how to use ANSI escapes to expand '\t'. See https://www.gnu.org/savannah-checkouts/gnu/bash/manual/bash.html#ANSI_002dC-Quotin
Solution:
sort -nr -t $'\t' -k 2
Count unique lines and format output
Source: https://www.hackerrank.com/challenges/text-processing-in-linux-the-uniq-command-2/problem
So the problem formulation is a bit shitty. The assignment is not really about uniq but about string formatting My solution using tools used in the challenges so far:
uniq -c $1 | tr -s [' '] | cut -c2-
I did learn about the xargs command though! But does not run here because ...?
uniq -c | xargs -L
For nice blog, see: https://shapeshed.com/unix-xargs/
Common use is with find:
find /tmp -mtime +14 | xargs rm
Is way faster than using -exec
grep
Match 'the' exactly:
# Most straightforward: match spaces as word boundaries
grep ' the '
# Explicit word bounary
grep '\bthe\b'
# Indicate exact match with flag
grep -w 'the'
# Show everything that does NOT match
grep -i -v -w 'the'
# | is extended syntax, so instead of -e we need to use -E for extended regular expressions (POSIX ERE).
grep -i -E -w "the|that|then|those
Don't forget that in bash brackets have special meanings so need to be escaped, e.g. :
grep -e "\([0-9]\) *\1
sed
These two are equivalent:
sed "s/\bthe\b/this/"
sed -e s/\\bthe\\b/this
The difference is that if you do not explicitly provide a pattern string with the -e flag, you are typing straight into bash, which requires you to provide further escapes for \b.
Flags are provided in the pattern itself, e.g. g for global, I for case insensitive:
sed -e "s/\bthy\b/your/gI
& corresponds to the found pattern (so basically a single backreference for the whole match, not just a capture group): For example, two wrap whatever match you find in brackets:
sed -e "s/\bthy\b/{&}/gI
Always note that for some things you need extended syntax with -E or -r flag:
# Assumes there is not accidentally a space at the end
# I would normally check this with another script
# {4} repetition needs ERE so -E flag is necessary
sed -E "s/[0-9]{4} /**** /g