Linux command line and Shell scripting Daquan-sed legendary techniques

Contents of this chapter:

  • sed Editor's Popular Skills
# tail /etc/services
nimgtw         48003/udp               # Nimbus Gateway
3gpp-cbsp       48049/tcp               # 3GPP Cell Broadcast Service Protocol
isnetserv       48128/tcp               # Image Systems Network Services
isnetserv       48128/udp               # Image Systems Network Services
blp5           48129/tcp               # Bloomberg locator
blp5           48129/udp               # Bloomberg locator
com-bardac-dw     48556/tcp               # com-bardac-dw
com-bardac-dw     48556/udp               # com-bardac-dw
iqobject        48619/tcp               # iqobject
iqobject        48619/udp               # iqobject
Print odd lines
#seq 10 | sed -n '1~2p'
1
3
5
7
9
Print the matching line and the following line
# tail /etc/services |sed -n '/blp5/,+1p'
blp5            48129/tcp               # Bloomberg locator
blp5           48129/udp               # Bloomberg locator
Do not print the last line
# tail /etc/services |sed -n '$!p'
3gpp-cbsp       48049/tcp               # 3GPP Cell Broadcast Service Protocol
isnetserv       48128/tcp               # Image Systems Network Services
isnetserv       48128/udp               # Image Systems Network Services
blp5            48129/tcp               # Bloomberg locator
blp5            48129/udp               # Bloomberg locator
com-bardac-dw   48556/tcp               # com-bardac-dw
com-bardac-dw   48556/udp               # com-bardac-dw
iqobject        48619/tcp               # iqobject
iqobject        48619/udp               # iqobject

#The exclamation mark is the opposite of the following command.
Matching range
# tail /etc/services  |sed -n '/^blp5/,/^com/p'
blp5            48129/tcp               # Bloomberg locator
blp5            48129/udp               # Bloomberg locator
com-bardac-dw   48556/tcp               # com-bardac-dw

#Choose a range by separating two styles with commas
Refer to system variables with quotation marks
 a=2
 # tail /etc/services |sed -n ''$a',3p'
//or
# tail /etc/services |sed -n "$a,3p"
Use & command to reference matching content and replace it
# tail /etc/services |sed 's/48049/&.0/'
3gpp-cbsp       48049.0/tcp               # 3GPP Cell Broadcast Service Protocol
isnetserv       48128/tcp               # Image Systems Network Services
isnetserv       48128/udp               # Image Systems Network Services
blp5            48129/tcp               # Bloomberg locator
blp5            48129/udp               # Bloomberg locator
com-bardac-dw   48556/tcp               # com-bardac-dw
com-bardac-dw   48556/udp               # com-bardac-dw
iqobject        48619/tcp               # iqobject
iqobject        48619/udp               # iqobject
matahari        49000/tcp               # Matahari Broker

IP Add single quotation marks:
# echo '10.10.10.1 10.10.10.2 10.10.10.3' |sed -r 's/[^ ]+/"&"/g'
"10.10.10.1" "10.10.10.2" "10.10.10.3"
Group use, adding 123 after each string
# tail /etc/services |sed -r 's/(.*) (.*)(#.*)/\1\2test \3/'
3gpp-cbsp       48049/tcp              test # 3GPP Cell Broadcast Service Protocol
isnetserv       48128/tcp              test # Image Systems Network Services
isnetserv       48128/udp              test # Image Systems Network Services
blp5            48129/tcp              test # Bloomberg locator
blp5            48129/udp              test # Bloomberg locator
com-bardac-dw   48556/tcp              test # com-bardac-dw
com-bardac-dw   48556/udp              test # com-bardac-dw
iqobject        48619/tcp              test # iqobject
iqobject        48619/udp              test # iqobject
matahari        49000/tcp              test # Matahari Broker
Packet usage, switching protocol to port number position
# tail /etc/services |sed -r 's/(.*)(\<[0-9]+\>)\/(tcp|udp)(.*)/\1\3\/\2\4/'
3gpp-cbsp       tcp/48049               # 3GPP Cell Broadcast Service Protocol
isnetserv       tcp/48128               # Image Systems Network Services
isnetserv       udp/48128               # Image Systems Network Services
blp5            tcp/48129               # Bloomberg locator
blp5            udp/48129               # Bloomberg locator
com-bardac-dw   tcp/48556               # com-bardac-dw
com-bardac-dw   udp/48556               # com-bardac-dw
iqobject        tcp/48619               # iqobject
iqobject        udp/48619               # iqobject
matahari        tcp/49000               # Matahari Broker
How many lines after the comment matches the line
# seq 10 |sed '/5/,+3s/^/#/'
1
2
3
4
#5
#6
#7
#8
9
10
Remove start and end spaces or tabs
# echo "  1 2 3  " |sed 's/^[ \t]*//;s/[ \t]*$//'
1 2 3
Add new content (a, i and c)
#Add test on the line of blp5
# tail /etc/services |sed '/blp5/i \test'
3gpp-cbsp       48049/tcp               # 3GPP Cell Broadcast Service Protocol
isnetserv       48128/tcp               # Image Systems Network Services
isnetserv       48128/udp               # Image Systems Network Services
test
blp5            48129/tcp               # Bloomberg locator
test
blp5            48129/udp               # Bloomberg locator
com-bardac-dw   48556/tcp               # com-bardac-dw
com-bardac-dw   48556/udp               # com-bardac-dw
iqobject        48619/tcp               # iqobject
iqobject        48619/udp               # iqobject
matahari        49000/tcp               # Matahari Broker

#Add test on the next line of blp5
# tail /etc/services |sed '/blp5/a \test'
3gpp-cbsp       48049/tcp               # 3GPP Cell Broadcast Service Protocol
isnetserv       48128/tcp               # Image Systems Network Services
isnetserv       48128/udp               # Image Systems Network Services
blp5            48129/tcp               # Bloomberg locator
test
blp5            48129/udp               # Bloomberg locator
test
com-bardac-dw   48556/tcp               # com-bardac-dw
com-bardac-dw   48556/udp               # com-bardac-dw
iqobject        48619/tcp               # iqobject
iqobject        48619/udp               # iqobject
matahari        49000/tcp               # Matahari Broker

#Replace blp5 with a new line
# tail /etc/services |sed '/blp5/c \test'
3gpp-cbsp       48049/tcp               # 3GPP Cell Broadcast Service Protocol
isnetserv       48128/tcp               # Image Systems Network Services
isnetserv       48128/udp               # Image Systems Network Services
test
test
com-bardac-dw   48556/tcp               # com-bardac-dw
com-bardac-dw   48556/udp               # com-bardac-dw
iqobject        48619/tcp               # iqobject
iqobject        48619/udp               # iqobject
matahari        49000/tcp               # Matahari Broker

#Add a row next to the specified line
# tail /etc/services |sed '2a \test'     
3gpp-cbsp       48049/tcp               # 3GPP Cell Broadcast Service Protocol
isnetserv       48128/tcp               # Image Systems Network Services
test
isnetserv       48128/udp               # Image Systems Network Services
blp5            48129/tcp               # Bloomberg locator
blp5            48129/udp               # Bloomberg locator
com-bardac-dw   48556/tcp               # com-bardac-dw
com-bardac-dw   48556/udp               # com-bardac-dw
iqobject        48619/tcp               # iqobject
iqobject        48619/udp               # iqobject
matahari        49000/tcp               # Matahari Broker
Add a line before and after the specified line
# seq 5 |sed '3s/.*/txt\n&/' 
1
2
txt
3
4
5
# seq 5 |sed '3s/.*/&\ntxt/'
1
2
3
txt
4
5
Read the next line (n and N)
#Print the next line of matching
# seq 5 |sed -n '/3/{n;p}'                             
4

#Print even numbers
# seq 6 |sed -n 'n;p'  
2
4
6
#Sed reads the first line 1, executes the n command, and gets the next line 2. At this time, the mode space is 2, executes the p command, and prints the mode space. Now the mode space is 2, sed reads 3, executes n commands, gets the next line 4, then the mode space is 4, executes p commands, and so on.

#Print odd numbers
seq 6 |sed -n 'n;d'
1
3
5
#sed first reads the first line 1, then prints the mode space 1, executes the n command, obtains the next line 2, executes the d command, deletes the 2 of the mode space, and then reads the 3. At this time, the mode space is 3, and prints the mode space, then executes the n command, obtains the next line 4, executes the d command, deletes the 3 of the mode space, in order to And so on.
Execute p command every three lines
# seq 6 |sed 'n;n;p'   
1
2
3
3
4
5
6
6
#sed reads the first line 1, prints mode space 1, executes n commands, gets the next line 2, and prints mode space 2, then executes n commands, obtains the next line 3, executes p commands, and prints mode space 3. sed reads the next line 3 and prints mode space 3, and so on.

#Replace every three lines
# seq 6 |sed 'n;n;s/^/=/;s/$/=/'
1
2
=3=
4
5
=6=

#When multiple sed commands are executed, they sometimes interact with each other. We can enclose them in braces {}.
# seq 6 |sed '3~3{s/^/=/;s/$/=/}'
1
2
=3=
4
5
=6=

#Look again at the function of the N command
# seq 6 |sed 'N;q'
1
2
//Merge the two rows into one row:
# seq 6 |sed 'N;s/\n//'
12
34
56


# seq 5 |sed -n 'N;p'
1
2
3
4
# seq 6 |sed -n 'N;p'
1
2
3
4
5
6

#Why didn't the first print 5?
#Because the N command reads the next line and appends it to the current line read by sed, when N reads the next line without content, it exits and does not print the current line by executing the p command.
#When the number of lines is even, N always reads the next line, so it also executes the p command.

#The last line when printing odd rows
# seq 5 |sed -n '$!N;p'           
1
2
3
4
5

#Print odd numbers
# seq 6 |sed -n 'N;P'
1
3
5

#Keep the last line
# seq 6 |sed 'N;D'       
6
#Read the first line 1 and execute the N command to read the next line and append it to the mode space. At this time, the mode space is 1n2. Execute the D command to delete the first line 1 of the mode space and the remaining 2.
#Read the second line and execute the N command. At this time, the mode space is 3 n4. Execute the D command to delete the first line 3 and the remaining 4 of the mode space.
#By analogy, when reading the last line of printing, N exits without getting the next line, and no longer executes D, so only the remaining 6 of the mode space is printed.
Keep Spatial Operations (h and H, g and G and x)

The h command copies the content of the schema space to the holding space (coverage).
The role of the H command is to copy the schema space content and append it to the holding space.
The g command replicates and holds spatial content to schema space (coverage).
The G command replicates and keeps spatial content appended to the schema space.
The function of the x command is to interchange schema space and preserve space content.

# Overlay the matched content to another match
# seq 6 |sed -e '/3/{h;d}' -e '/5/g'
1
2
4
3
6
 # The h command copies the matched 3 into the hold space, and the d command deletes the 3 in the schema space. The latter command matches 5 to the pattern space and covers the pattern space 5 with the g command.

# Put the matching content to the end
# seq 6 |sed -e '/3/{h;d}' -e '$G'
1
2
4
5
6
3

# Exchange Mode Space and Maintenance Space
# seq 6 |sed -e '/3/{h;d}' -e '/5/x' -e '$G'
1
2
4
3
6
5

# Retrospective output
# seq 5 |sed '1!G;h;$!d'
5
4
3
2
1

1! The first line of G does not execute to append the holding space content to the schema space, because there is no data in the holding space now.
h Put the mode space in the holding space for temporary storage.
The last line of $! d does not execute deleting the contents of the schema space.
When reading the first line 1, skip G command, execute h to copy mode space 1 to hold space, and execute d command to delete mode space 1.
When the second line 2 is read, the mode space is 2. Execute G command to append the holding space 1 to the mode space. At this time, the mode space is 2n1. Execute h to cover the holding space with 2n1, and delete the mode space with d.
When reading the third line 3, the mode space is 3. Execute G command to append the holding space 2 N1 to the mode space. At this time, the mode space is 3 n2 n1. Execute h to copy the content of the mode space to the holding space, and d to delete the mode space.
By analogy, when you read the fifth line, the mode space is 5, execute the G command, append the mode space of 4 n3 n2 n1, and copy it to the mode space, 5 n4 n3 n2 n1, do not execute d, save the mode space, output.
Thus, each read line is first placed in the mode space, then copied to the holding space. The d command deletes the content of the mode space, prevents the output, and then appends to the mode space, because appending to the mode space appends to the back of the newly read line. By doing this, all rows and rows are appended to the new read line. Later, a flashback is formed.

# Add a new blank line after each line
# seq 10 |sed G
1

2

3

4

5
Label

Label can control flow and realize branch judgment.
lable name defines labels
b lable jumps to the specified label, if no label, to the end of the script
t lable jumps to the specified label on condition that the s/// command executes successfully

#Replace line breaks with commas
# seq 6 |sed ':a;N;s/\n/,/;b a'           
1,2,3,4,5,6
#Look at the tag usage here: A is the defined tag name, B is the jump to a position.
#sed reads the first line 1, N command reads the next line 2, when the mode space is 1 N2 $, and performs substitution. At this time, the mode space is 1,2 $. Execute b command and jump to label a to continue executing N command. Read the next line 3 and add it to the mode space. At this time, the mode space is 1,2 N3 $, and then replace it, and so on, add and replace it continuously. Until the last line N cannot read the next line to exit.

# seq 6 |sed ':a;N;$!b a;s/\n/,/g'
1,2,3,4,5,6

#Add a comma to every three digits
# echo "123456789" |sed -r 's/([0-9]+)([0-9]+{3})/\1,\2/'
123456,789
# echo "123456789" |sed -r ':a;s/([0-9]+)([0-9]+{3})/\1,\2/;t a'
123,456,789
# echo "123456789" |sed -r ':a;s/([0-9]+)([0-9]+{2})/\1,\2/;t a'
1,23,45,67,89
Ignore case matching
# echo -e "a\nA\nb\nc" |sed 's/a/1/Ig'
1
1
b
c
Get the total number of rows
# seq 10 |sed -n '$='

Keywords: network

Added by viraj on Wed, 17 Jul 2019 00:44:22 +0300