Awesome examples for Perl one-liner

Good coders code, 

great coders reuse



1) Let's look at several examples to get more familiar with one-liners. Here is one:

perl -pi -e 's/you/me/g' file

This one-liner replaces all occurrences of the text you with me in the file file. Very useful if you ask me. Imagine you are on a remote server and have this file and you need to do the replacement. You can either open it in text editor and execute find-replace or just do it through command line and, bam, be done with it.

The -e argument is the best argument. It allows you to specify the Perl code to be executed right on the command line. In this one-liner the code says, do the substitution (s/find/replace/flags command) and replace you with me globally (g flag). The -p argument makes sure the code gets executed on every line, and that the line gets printed out after that. The -i argument makes sure that file gets edited in-place, meaning Perl opens the file, executes the substitution for each line, prints the output to a temporary file, and then replaces the original file.

How about doing the same replacement in multiple files? Just specify them on the command line!

perl -pi -e 's/you/me/g' file1 file2 file3

Now let's do the same replacement only on lines that match we. It's as simple as this:

perl -pi -e 's/you/me/g if /we/' file

Here we use the conditional if /we/. This makes sure that s/you/me/g gets executed only on lines that match the regular expression /we/. The regular expression here can be anything. Let's say you want to execute the substitution only on lines that have digits in them. You can then use the /\d/ regular expression that matches numbers:

perl -pi -e 's/you/me/g if /\d/' file

2) How to find all repeated lines in a file?

perl -ne 'print if $a{$_}++' file

3) How about numbering lines? Super simple! Perl has the $. special variable that automatically maintains the current line number. You can just print it out together with the line:

cat someFile.txt | perl -ne 'print "$. $_"'

4) How about finding all repeated lines in a file?

perl -ne 'print if $a{$_}++' file

5) How about we combine the previous two one-liners and create one that numbers repeated lines? Here we go:

perl -ne 'print "$. $_" if $a{$_}++' file

6)  How about generating an 8 letter password? Again, solvable elegantly with a one-liner:

perl -le 'print map { (a..z)[rand 26] } 1..8'

The a..z generates a list of letters from a to z (total 26). Then we randomly choose one of the characters through generating a random number in range 0-25, and then we repeat this whole process 8 times!

7) Let's find the sum of the numbers in the first column:

BEGIN { ... } and END { ... } let you put code that gets run entirely before or after the loop over the lines.

For example, I could sum the values in the second column of a CSV file using:

perl -F, -lane '$t += $F[1]; END { print $t }'

cat SomeFile.txt | perl -lane '$sum += $F[0]; END { print $sum }'

8) How about getting a list of the names of all users on the system?

perl -a -F: -lne 'print $F[4]' /etc/passwd

it is the same as:

perl -a -F':' -lne 'print $F[4]' /etc/passwd

9) .. operator:  It is a stateful operator -- it remembers state between evaluations. 


Perl's .. operator is a stateful operator -- it remembers state between evaluations. As long as its left operand is false, it returns false; Once the left hand returns true, it starts evaluating the right-hand operand until that becomes true, at which point, on the next iteration it resets to false and starts testing the other operand again.

What does that mean in practice? It's a range operator: It can be easily used to act on a range of lines in a file. For instance, I can extract all GPG public keys from a file using:

perl -ne 'print if /-----BEGIN PGP PUBLIC KEY BLOCK-----/../-----END PGP PUBLIC KEY BLOCK-----/' FILE

Example 2: (The following command print out line 2 through 4 (inclusive))

cat someFile.txt | perl -lane 'print "$_" if $.=~m/^2/..$.=~/^4/'


10)  $ENV{}: bridge between environment and Perl
When you're writing a one-liner using -e in the shell, you generally want to quote it with ', so that dollar signs inside the one-liner aren't expanded by the shell. But that makes it annoying to use a ' inside your one-liner, since you can't escape a single quote inside of single quotes, in the shell.

Let's suppose we wanted to print the username of anyone in /etc/passwd whose name included an apostrophe. One option would be to use a standard shell-quoting trick to include the ':

perl -F: -lane 'print $F[0] if $F[4] =~ /'"'"'/' /etc/passwd
But counting apostrophes and backslashes gets old fast. A better option, in my opinion, is to use the environment to pass the regex into perl, which lets you dodge a layer of parsing entirely:

env re="'" perl -F: -lane 'print $F[0] if $F[4] =~ /$ENV{re}/' /etc/passwd
We use the env command to place the regex in a variable called re, which we can then refer to from the perl script through the %ENV hash. This way is slightly longer, but I find the savings in counting backslashes or quotes to be worth it, especially if you need to end up embedding strings with more than a single metacharacter.


Reference:

Introduction to Perl one-liners - good coders code, great coders reuse

http://www.catonmat.net/blog/introduction-to-perl-one-liners/