Hackerrank Linux Text Processing

Paste

Good resource: http://www.theunixschool.com/2012/07/10-examples-of-paste-command-usage-in.html

Source: https://www.hackerrank.com/challenges/paste-2/problem

Task: "You are given a CSV file where each row contains the name of a city and its state separated by a comma. Your task is to restructure the file so that three consecutive rows are folded into one line and are separated by semicolons."

You can introduce repeating delimiters; paste is smart enough to fit this delimiter pattern:

paste -s -d ';;\n' 

But I learned from the discussion that - reads one line from standard input, so this also works:

paste -d ';' - - - 

Another assignment asked for replacing newlines with tabs using space. But it seems this is really not the primary use of paste. Using translate seems more appropriate:

Use ANSI escaped strings to expand \n and \t:

tr $'\n' $'\t

Get the middle of a text file

First cut of the tail with head, then cut of the head of the head with tail. From line 12 tot 22:

head -22 | tail -1

Translate tr

Source: https://www.hackerrank.com/challenges/text-processing-tr-1/problem

Input:

int i=(int)5.8
(23 + 5)*2

Output:

int i=[int]5.8
[23 + 5]*2

Solution:

tr  "()" "[]

Sort numerical second column of tab-separated file

Source: https://www.hackerrank.com/challenges/text-processing-sort-5/problem

I learned how to use ANSI escapes to expand '\t'. See https://www.gnu.org/savannah-checkouts/gnu/bash/manual/bash.html#ANSI_002dC-Quotin

Solution:

sort -nr -t $'\t' -k 2

Count unique lines and format output

Source: https://www.hackerrank.com/challenges/text-processing-in-linux-the-uniq-command-2/problem

So the problem formulation is a bit shitty. The assignment is not really about uniq but about string formatting My solution using tools used in the challenges so far:

uniq -c $1 | tr -s [' '] | cut -c2-

I did learn about the xargs command though! But does not run here because ...?

uniq -c | xargs -L

For nice blog, see: https://shapeshed.com/unix-xargs/

Common use is with find:

find /tmp -mtime +14 | xargs rm

Is way faster than using -exec

grep

Match 'the' exactly:

# Most straightforward: match spaces as word boundaries
grep ' the '

# Explicit word bounary
grep '\bthe\b' 

# Indicate exact match with flag
grep -w 'the'

# Show everything that does NOT match
grep -i -v -w 'the'

# | is extended syntax, so instead of -e we need to use -E for extended regular expressions (POSIX ERE). 
grep -i -E -w "the|that|then|those

Don't forget that in bash brackets have special meanings so need to be escaped, e.g. :

grep -e "\([0-9]\) *\1

sed

These two are equivalent:

sed "s/\bthe\b/this/"
sed -e s/\\bthe\\b/this

The difference is that if you do not explicitly provide a pattern string with the -e flag, you are typing straight into bash, which requires you to provide further escapes for \b.

Flags are provided in the pattern itself, e.g. g for global, I for case insensitive:

sed -e "s/\bthy\b/your/gI

& corresponds to the found pattern (so basically a single backreference for the whole match, not just a capture group): For example, two wrap whatever match you find in brackets:

sed -e "s/\bthy\b/{&}/gI

Always note that for some things you need extended syntax with -E or -r flag:

# Assumes there is not accidentally a space at the end
# I would normally check this with another script
# {4} repetition needs ERE so -E flag is necessary
sed -E "s/[0-9]{4} /**** /g