Awk

awk

Introduction

 * A file is treated as a sequence of lines; each line is a sequence of fields. Each pattern that matches, the associated action is executed
 * A complete programming language, but if you’re doing more than one or two lines in it, you should probably be using Perl instead
 * Mostly used for extracting or transforming text, such as producing formatted reports

Syntax
awk 'BEGIN{ print "start" } pattern { commands } END{ print "end" }' file

Some special variables

 * NR: current line number
 * NF: current number of fields
 * $NF: value of the last field
 * $0: current line content
 * $n: content of the n th  field

How it works
Count number of lines in a file

awk 'BEGIN{ i=0 } { i++ } END{ print i}' sample.log or awk 'END{ print NR }' sample.log

Command "print"
print var1,var2,var3;          => Output: var1 var2 var3 print var1 "-" var2 "-" var3;  => Output: var1-var2-var3

Set delimiter different from default space " "
awk -F: '{ print $NF }' /etc/passwd cat /etc/passwd | awk -F: '{if ($7 ~ /tcsh/) print $1;}'

Examples
Count number of users who is using /bin/bash shell awk -F ':' '$NF ~ /\/bin\/bash/ { n++ }; END { print n }' /etc/passwd Count the size of all *.log file in current directory ls -l *.log | awk '{s+=$5} END {print "Total size: " s}' Show host IP and request time in apache log file (host & reqtime_microsec) tail -f web.acc.20130925 | awk -F'\t' '{print $2 " " $10}' cat web.acc.20131121 | awk -F'\t' '{print $10}' | awk -F':' '{if ($2 >= 500000) print $2}' | sort -n Show 20 most-visited requests in apache log file cat web.acc.20130925 | awk '{print $6}' | sort | uniq -c | sort -rn | head -20 Print the line having CPU wait is max in log awk 'BEGIN {max=0; line="";} /12:30:/ {if(max<$18) {max=$18; line=$0}} END {print line}' vmlog.20131001 awk 'BEGIN {max=0; line="";} /12:30:/ {if(max<$NF) {max=$NF; line=$0}} END {print line}' iolog.20131001 Find min value awk 'BEGIN {i=0; ratemin=100; totals=0; accepts=0;} { if (i % 2 == 0) {accept=$4} else {total=$4; rate=accept/total; if (rate<ratemin) {ratemin=rate; totals=total; accepts=accept} }; i=i+1; print i" "accept" "total; } END {print ratemin" "accepts" "totals}'