On this page 9 sections
  1. Think in records and fields
  2. Choose a delimiter
  3. Filter records
  4. Pattern matching
  5. Built-in variables
  6. BEGIN and END
  7. Calculate totals
  8. Analyse web logs
  9. Custom output formatting
Lab use only. Apply these techniques only to systems you own or have explicit permission to test.
01

Think in records and fields

awk processes input one record at a time. By default a record is one line and fields are separated by whitespace. $1 represents the first field, $2 the second and $0 the complete line.

Print first field
awk '{print $1}' file.txt
Print first and third fields
awk '{print $1, $3}' file.txt
02

Choose a delimiter

-F sets the input field separator. This makes awk useful for colon-separated system files, CSV-like input and structured logs.

List users from passwd
awk -F: '{print $1}' /etc/passwd
Username and shell
awk -F: '{print $1, $7}' /etc/passwd
03

Filter records

A condition placed before the action controls which lines are processed.

Users with UID 0
awk -F: '$3 == 0 {print $1}' /etc/passwd
Processes using more than 10 percent CPU
ps aux | awk '$3 > 10 {print $2, $3, $11}'
04

Pattern matching

awk supports regular expressions directly in conditions.

Lines containing ERROR
awk '/ERROR/ {print}' application.log
Exclude comments
awk '!/^#/ {print}' config.txt
05

Built-in variables

NF is the number of fields on the current line and NR is the current record number. $NF therefore represents the last field.

Print last field
awk '{print $NF}' file.txt
Add line numbers
awk '{print NR, $0}' file.txt
Print field count
awk '{print NF, $0}' file.txt
06

BEGIN and END

BEGIN executes before any input is processed. END executes after all records have been processed. They are useful for headers, initialisation and final calculations.

Count lines
awk 'END {print NR}' file.txt
Custom header
awk 'BEGIN {print "USER SHELL"} {print $1, $2}' file.txt
07

Calculate totals

awk variables make it easy to aggregate numerical fields.

Sum values in second column
awk '{sum += $2} END {print sum}' values.txt
Average
awk '{sum += $2} END {if (NR) print sum/NR}' values.txt
08

Analyse web logs

For common web log formats, the first field often contains the client IP. awk can extract it before sort and uniq are used to count repeated clients.

Most common client IPs
awk '{print $1}' access.log | sort | uniq -c | sort -nr | head
09

Custom output formatting

printf provides precise formatting compared with print.

Formatted passwd output
awk -F: '{printf "%-20s %s\n", $1, $7}' /etc/passwd