pattern{action statements;..}

awk working process

Step 1: Execute BEGIN{action;... } Statements in a statement block

Step 2: import from file or standard (stdin) Read a row and execute Pattern {action;...} statement block, which scans the file line by line and repeats the process from the first line to the last line until all the files are read.

Step 3: when reading to the end of the input stream, execute END{action;...} Statement block

BEGIN Statement block in awk is executed before reading rows from the input stream. This is an optional statement block, such as variable initialization, printout table header and so on. Statements can usually be written in BEGIN In a statement block

END Statement block in awk is executed after reading all rows from the input stream, such as printing the analysis results of all rows. Such information is summarized in END Statement block, which is also an optional statement block

pattern The general commands in the statement block are the most important and optional. If not provided pattern Statement block, execute {print} by default That is, print each read line, awk The statement block is executed for each row read

Delimiters, fields, and records

• Fields separated by separators (column, field) are marked with $1, ...$n is called domain ID, and $0 is all domains. Note: the meaning of variable $in shell is different from that of variable $in shell
• Each line of the file is called record
• If action is omitted, print $0 will be executed by default

frequently-used action classification

• output statements: print,printf
• Expressions: arithmetic, comparison expressions, etc
• Compound statements: compound statements
• Control statements: if, while, etc
• input statements

awk Control statement

• {statements;...} combined statements
• if(condition) {statements;...}
• if(condition) {statements;...} else {statements;...}
• while(conditon) {statments;...}
• do {statements;...} while(condition)
• for(expr1;expr2;expr3) {statements;...}
• break
• continue
• exit

2. Action print

format

print item1, item2, ...

explain:

• GNU sed
• The output item can be a string or a numeric value; The expression for the field, variable, or awk of the current record
• If item is omitted, it is equivalent to print $0
• Fixed characters need to be enclosed by "", while variables and numbers do not

Example: take out the top one with the largest number of website visits 3 individual IP

[root@VM_0_10_centos logs]# awk '{print $1}' nginx.access.log-20200428|sort | 
uniq -c |sort -nr|head -3
   5498 122.51.38.20
   2161 117.157.173.214
    953 211.159.177.120
[root@centos8 ~]#awk '{print $1}' access_log |sort |uniq -c|sort -nr|head 
   4870 172.20.116.228
   3429 172.20.116.208
   2834 172.20.0.222
   2613 172.20.112.14
   2267 172.20.0.227
   2262 172.20.116.179
   2259 172.20.65.65
   1565 172.20.0.76
   1482 172.20.0.200
   1110 172.20.28.145

Example: fetch partition utilization

[root@centos8 ~]# df | awk -F"[[:space:]]+|%" '{print $5}'
Use
0
0
1
0
3
19
1
0

Example: take the IP address in the ifconfig output result

[root@centos8 ~]# ifconfig eth0
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 10.0.0.85  netmask 255.255.255.0  broadcast 10.0.0.255
        inet6 fe80::20c:29ff:fe3d:d1e7  prefixlen 64  scopeid 0x20<link>
        ether 00:0c:29:3d:d1:e7  txqueuelen 1000  (Ethernet)
        RX packets 24590  bytes 25224965 (24.0 MiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 12793  bytes 4232673 (4.0 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

[root@centos8 ~]# ifconfig eth0 | sed -n "2p"
        inet 10.0.0.85  netmask 255.255.255.0  broadcast 10.0.0.255
[root@centos8 ~]# ifconfig eth0 | sed -n "2p" | awk '{print $2}'
10.0.0.85

[root@centos8 ~]# ifconfig eth0 | awk '/netmask/{print $2}'
10.0.0.85

[root@centos8 ~]# ifconfig eth0 | awk 'NR==2{print $2}'
10.0.0.85

3.awk variables

awk -v RS=' ' -v ORS='###'  '{print $0}' /etc/passwd

[root@centos8 ~]#awk '{print NR,$0}' /etc/issue /etc/centos-release
1 \S
2 Kernel \r on an \m
34 CentOS Linux release 8.1.1911 (Core)

[root@centos8 ~]#awk '{print ARGC}' /etc/issue /etc/redhat-release 
3
3
3
3
[root@centos8 ~]#awk 'BEGIN{print ARGC}' /etc/issue /etc/redhat-release 
3

[root@centos8 ~]#awk 'BEGIN{print ARGV[0]}' /etc/issue /etc/redhat-release 
awk
[root@centos8 ~]#awk 'BEGIN{print ARGV[1]}' /etc/issue /etc/redhat-release 
/etc/issue
[root@centos8 ~]#awk 'BEGIN{print ARGV[2]}' /etc/issue /etc/redhat-release 
/etc/redhat-release
[root@centos8 ~]#awk 'BEGIN{print ARGV[3]}' /etc/issue /etc/redhat-release 
[root@centos8 ~]#

printf "FORMAT", item1, item2, ...

awk -F:   '{printf "%s",$1}' /etc/passwd
awk -F:   '{printf "%s\n",$1}' /etc/passwd
awk -F:   '{printf "%20s\n",$1}' /etc/passwd
awk -F:   '{printf "%-20s\n",$1}' /etc/passwd
awk -F:   '{printf "%-20s %10d\n",$1,$3}' /etc/passwd
awk -F:   '{printf "Username: %s\n",$1}' /etc/passwd
awk -F:   '{printf "Username: %sUID:%d\n",$1,$3}' /etc/passwd
awk -F:   '{printf "Username: %25sUID:%d\n",$1,$3}' /etc/passwd
awk -F:   '{printf "Username: %-25sUID:%d\n",$1,$3}'

5. Operator

Arithmetic operators:

[root@centos8 ~]#awk 'BEGIN{i=0;print i++,i}'
0 1
[root@centos8 ~]#awk 'BEGIN{i=0;print ++i,i}'
1 1

[root@centos8 ~]#seq 10 | awk 'NR%2==0'
2
4
6
8
10
[root@centos8 ~]#seq 10 | awk 'NR%2==1'
1
3
5
7
9

[root@centos8 ~]#awk -F: '$0 ~ /root/{print $1}' /etc/passwd
[root@centos8 ~]#awk -F: '$0 ~ "^root"{print $1}' /etc/passwd
[root@centos8 ~]#awk '$0 !~ /root/'   /etc/passwd
[root@centos8 ~]#awk '/root/'   /etc/passwd
[root@centos8 ~]#awk -F: '/r/' /etc/passwd
[root@centos8 ~]#awk -F: '$3==0'     /etc/passwd
[root@centos8 ~]#df | awk -F"[[:space:]]+|%" '$0 ~ /^\/dev\/sd/{print $5}'
51
92
[root@centos8 ~]#ifconfig eth0 | awk 'NR==2{print $2}'
10.0.0.8