AWK Command Examples: Fields, Filters, Counts and Log Analysis
awk is a powerful Linux command for working with columns, fields and structured text. It is especially useful for log analysis, quick reports and turning messy command output into something useful.
What is AWK?
awk reads text line by line, splits each line into fields, and then lets you print, filter, count or calculate with those fields. It is often used with log files, CSV-style files and command output.
awk '{print $1}' file.txtThis prints the first field from each line.
AWK fields and built-in variables
| Variable | Meaning |
|---|---|
$0 | The whole line. |
$1 | The first field. |
$2 | The second field. |
$NF | The last field. |
NF | Number of fields. |
NR | Current line number. |
awk '{print NR, $1, $NF}' file.txtUsing delimiters
By default, AWK splits on whitespace. Use -F to choose a delimiter.
Colon-delimited passwd file
awk -F: '{print $1, $7}' /etc/passwdCSV-style file
awk -F, '{print $1, $3}' data.csvPipe-delimited file
awk -F'|' '{print $2}' data.txtFiltering with AWK
Status code 500
awk '$9 == 500 {print $1, $7, $9}' access.logLarge response size
awk '$10 > 1000000 {print $1, $7, $10}' access.logRegex match
awk '$7 ~ /wp-login.php/ {print $1}' access.logRegex not match
awk '$7 !~ /\.(css|js|png|jpg)$/ {print $7}' access.logCounting, sums and averages
Count lines
awk 'END {print NR}' file.txtSum column 3
awk '{sum += $3} END {print sum}' numbers.txtAverage column 3
awk '{sum += $3} END {print sum/NR}' numbers.txtCount first-column values
awk '{count[$1]++} END {for (item in count) print count[item], item}' file.txtAWK log analysis examples
These examples are useful for Apache/cPanel domlogs and many standard web access logs.
Top IPs
awk '{print $1}' access.log | sort | uniq -c | sort -nr | headTop URLs
awk '{print $7}' access.log | sort | uniq -c | sort -nr | headStatus code counts
awk '{print $9}' access.log | sort | uniq -c | sort -nrBandwidth by IP
awk '{bytes[$1]+=$10} END {for (ip in bytes) print bytes[ip], ip}' access.log | sort -nr | headExample output
1243 203.0.113.10
842 198.51.100.25
411 192.0.2.44Common AWK mistakes
- Forgetting to quote the AWK program: use
'{print $1}'. - Using the wrong field number for a log format.
- Assuming every file is whitespace-delimited. Use
-Fwhen needed. - Forgetting that shell variables and AWK variables are different.
AWK workflows for web server logs
AWK is especially useful when logs have predictable columns. The examples below assume a common Apache-style access log.
# Count requests per IP
awk '{count[$1]++} END {for (ip in count) print count[ip], ip}' access.log | sort -nr | head
# Count HTTP status codes
awk '{codes[$9]++} END {for (code in codes) print code, codes[code]}' access.log | sort
# Show slow-looking large responses
awk '$10 > 1000000 {print $1, $7, $9, $10}' access.log1842 203.0.113.51
923 198.51.100.24
311 192.0.2.19
200 2511
301 42
404 188
500 9AWK mistakes to watch for
- Using the wrong field number because the log format is different.
- Forgetting
-Fwhen the file uses commas, colons or tabs instead of spaces. - Assuming every line is valid. Real logs often contain odd or partial lines.
- Not sorting the output when using arrays, because AWK array order is not designed for reports.
For more log-focused examples, read Search Logs for Errors on Linux and the Linux Troubleshooting Hub.
Frequently Asked Questions
What is AWK best used for?
AWK is best for field-based text processing, such as extracting columns, filtering rows, counting values and summarising logs.
How do I choose a delimiter in AWK?
Use -F followed by the delimiter. For example, awk -F: '{print $1}' /etc/passwd uses a colon as the field separator.
How do I count unique values with AWK?
Use an associative array, such as awk '{count[$1]++} END {for (item in count) print item, count[item]}' file.
Is AWK good for log analysis?
Yes. AWK is excellent for quick log summaries, especially when combined with sort, uniq, head and grep.