10.6 Monitor io performance
iostat command
The iostat command is used to monitor the usage of system input and output devices and CPUs.It features a report on disk activity statistics as well as CPU usage.Like vmstat, iostat has a disadvantage that it cannot analyze a process in depth, only the overall situation of the system.
- iostat -x
[root@adai003 ~]# iostat -x Linux 3.10.0-514.el7.x86_64 (adai003) 2017 July 11, 2000 _x86_64_ (1 CPU) avg-cpu: %user %nice %system %iowait %steal %idle 0.12 0.00 0.24 0.25 0.00 99.39 Device: rrqm/s wrqm/s r/s w/s rkB/s wkB/s avgrq-sz avgqu-sz await r_await w_await svctm %util sda 0.00 0.06 0.45 0.30 16.50 4.06 54.39 0.02 31.35 32.52 29.60 6.25 0.47 scd0 0.00 0.00 0.00 0.00 0.00 0.00 8.00 0.00 170.45 170.45 0.00 170.45 0.01
Description: util%: Indicates the proportion of io waiting. Normally this value is proportional to disk read/write (r/w). If the value is large and the read/write value is small, the disk is having problems and the system performance will be affected!
iotop command
The iotop command is a top class tool (dynamic) that monitors disk I/O usage.Iotop has a UI similar to top, including information about PID, user, I/O, process, and so on.Most of the IO statistics tools under Linux, such as iostat and nmon, can only count the read and write of per devices. If you want to know how IO is used by each process, it is more cumbersome to use the iotop command, which can be easily viewed.
[root@adai003 ~]# iotop Total DISK READ : 0.00 B/s | Total DISK WRITE : 0.00 B/s Actual DISK READ: 0.00 B/s | Actual DISK WRITE: 0.00 B/s TID PRIO USER DISK READ DISK WRITE SWAPIN IO> COMMAND 3694 be/4 root 0.00 B 0.00 B 0.00 % 0.02 % [kworker/0:0] 1 be/4 root 0.00 B 0.00 B 0.00 % 0.00 % systemd --swit~-deserialize 21 2 be/4 root 0.00 B 0.00 B 0.00 % 0.00 % [kthreadd] 3 be/4 root 0.00 B 0.00 B 0.00 % 0.00 % [ksoftirqd/0] 516 be/4 root 0.00 B 0.00 B 0.00 % 0.00 % crond -n 517 be/4 root 0.00 B 0.00 B 0.00 % 0.00 % login -- root ......
Note: Focus on the'IO>'column here!
-
Shortcut keys commonly used in iotop
- Left and right arrows: change the sort order, default is to sort by IO
- r: Change the sort order.
- o: Only processes with IO output are displayed.
- p: Switching the way a process/thread is displayed.
- a: Display cumulative usage.
- q: Exit.
10.7 free command
The free command displays the number of unused and used memory on the current system, as well as the memory buffers used by the kernel.
Syntax: free [options]
Options:
-b/k/m/g: display in byte, KB, M, G (default is KB)
-h: the appropriate units are displayed
-t: Display the sum of memory
[root@adai003 ~]# free total used free shared buff/cache available Mem: 1008392 116072 586356 6864 305964 724644 Swap: 2097148 0 2097148 [root@adai003 ~]# free -h total used free shared buff/cache available Mem: 984M 113M 572M 6.7M 298M 707M Swap: 2.0G 0B 2.0G
Explain:
"total=used+free+buff/cache"
"available=free+buff/cache (idle part)"
Buffer: When the CPU writes data to the disk, because the disk storage rate is lower than the CPU, the written data is stored in memory first when the CPU works, which is the buff er memory.
Cache: When the CPU reads data from the disk, the disk output rate is lower than the CPU read speed, so the disk data is pre-stored in memory, which is the cache memory.
10.8 ps command
The > ps command reports the process status of the current system.You can interrupt or delete unnecessary programs at any time with the kill command.The ps command is the most basic and powerful process view command that can be used to determine which processes are running and running, whether they end, whether they are dead, which processes are consuming too much resources, etc. Most of the information can be obtained by executing the command.
usage
Syntax: ps [options]
Options:
a: Displays all programs under the current terminal, including those of other users.
u: Display the system status in a user-centric format.
x: Displays all programs, including historical processes.
-e: Show all processes (same as a)
-f: Display UID, PPIP, C and STIME columns
-l: Display process details
- ps aux
[root@adai003 ~]# ps aux USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND root 1 0.0 0.3 125100 3628 ? Ss 10:28 0:01 /usr/lib/systemd/systemd root 2 0.0 0.0 0 0 ? S 10:28 0:00 [kthreadd] ......
Description: STAT represents the state of the process.
-
Process status:
- D: Uninterrupted processes
- Process in R:run state
- Processes in S:sleep state
- s: Main process
- T: Suspended process
- Z: Zombie Process
- <: High priority process
- N: Low priority process
- L: Paging locked in memory
- l: Multithreaded process
- +: Foreground process
- ps -elf
[root@adai003 ~]# ps -elf F S UID PID PPID C PRI NI ADDR SZ WCHAN STIME TTY TIME CMD 4 S root 1 0 0 80 0 - 31275 ep_pol 10:28 ? 00:00:01 /usr/lib/system 1 S root 2 0 0 80 0 - 0 kthrea 10:28 ? 00:00:00 [kthreadd]
Process/Thread
process
>Process is a running activity of a program on a data collection in a computer, the basic unit for resource allocation and scheduling of the system, and the basis of the structure of the operating system.In the early process-oriented computer architecture, processes were the basic executing entities of programs; in the current thread-oriented computer architecture, processes were containers of threads.Programs are descriptions of instructions, data, and their organizational forms, and processes are entities of programs.
State Classification
The discontinuity of process execution determines that a process may have multiple states.In fact, a running process may have three basic states:
- Ready state: The process has acquired the required resources other than the processor, waiting to allocate the processor resources; it can execute as long as the processor process is allocated.Ready processes can divide queues by multiple priorities.For example, when a process enters a ready state due to exhaustion of time slices, it is queued at a lower priority; when a process enters a ready state due to I/O operations, it is queued at a higher priority.
- Running state: Processes consume processor resources; the number of processes in this state is less than or equal to the number of processors.When no other process can execute, such as when all processes are blocked, idle processes of the system are usually executed automatically.
- Blocked state: A process cannot continue execution until the condition is met because it waits for a condition, such as an I/O operation or process synchronization.Even if the processor is assigned to the process before this event occurs, it will not run.
thread
>Threads, sometimes referred to as lightweight processes (LWPs), are the smallest unit of program execution flow.A standard thread consists of a thread ID, a current instruction pointer (PC), a register collection, and a stack.In addition, threads are an entity in the process and are the basic units that are independently dispatched and assigned by the system. Threads do not own system resources and only have a little of the resources necessary for running, but they can share all the resources owned by the process with other threads belonging to the same process.One thread can create and undo another thread, and multiple threads in the same process can execute concurrently.Threads exhibit interruptions in operation due to interdependencies between threads.Threads also have three basic states: ready, blocked, and running.Every program has at least one thread. If a program has only one thread, it is the program itself.
Running multiple threads simultaneously in a single program to accomplish different tasks is called multithreading.
classification
- User-level threads: The management process is completely completed by the user program, and the core of the operating system only manages the process.
- System-level threads (core-level threads): Managed by the operating system kernel.The operating system kernel provides applications with appropriate system calls and application interface API s so that user programs can create, execute, and undo threads.
Process and Thread Relationships:
10.9 View network status
netstat command
>The netstat command is used to print the status information of the network system in Linux, allowing you to know the network condition of the entire Linux system.
Syntax: netstat [options]
Options:
-a:=all Displays socket s in all connections
-l:=listening Displays socket s for servers in monitoring
-n:=numeric uses IP address directly
-p:=programs Displays the program ID and name of the socket being used
-t:=tcp Displays the connection status of the TCP transport protocol
- Netstat-lnp View Listening Port
[root@adai003 ~]# netstat -lnp Active Internet connections (only servers) Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 1839/sshd tcp 0 0 127.0.0.1:25 0.0.0.0:* LISTEN 1979/master tcp6 0 0 :::22 :::* LISTEN 1839/sshd tcp6 0 0 ::1:25 :::* LISTEN 1979/master udp 0 0 127.0.0.1:323 0.0.0.0:* 488/chronyd udp6 0 0 ::1:323 :::* 488/chronyd raw6 0 0 :::58 :::* 7 543/NetworkManager Active UNIX domain sockets (only servers) Proto RefCnt Flags Type State I-Node PID/Program name Path unix 2 [ ACC ] SEQPACKET LISTENING 11807 1/systemd /run/udev/control unix 2 [ ACC ] STREAM LISTENING 17879 1979/master private/rewrite : proto by unix Is within the system socket File. ......
- Netstat-an View System Network Connection Status
[root@adai003 ~]# netstat -an Active Internet connections (servers and established) Proto Recv-Q Send-Q Local Address Foreign Address State tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN tcp 0 0 127.0.0.1:25 0.0.0.0:* LISTEN tcp 0 52 192.168.8.125:22 192.168.8.1:61445 ESTABLISHED tcp6 0 0 :::22 :::* LISTEN tcp6 0 0 ::1:25 :::* LISTEN udp 0 0 127.0.0.1:323 0.0.0.0:* udp6 0 0 ::1:323 :::* raw6 0 0 :::58 :::* 7 Active UNIX domain sockets (servers and established) Proto RefCnt Flags Type State I-Node Path unix 2 [ ACC ] SEQPACKET LISTENING 11807 /run/udev/control unix 2 [ ACC ] STREAM LISTENING 17879 private/rewrite ......
- Netstat-lntp only looks at tcp protocol connections, not socket s
[root@adai003 ~]# netstat -lntp Active Internet connections (only servers) Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 1839/sshd tcp 0 0 127.0.0.1:25 0.0.0.0:* LISTEN 1979/master tcp6 0 0 :::22 :::* LISTEN 1839/sshd tcp6 0 0 ::1:25 :::* LISTEN 1979/master
Supplement: Command to view tcp protocol status:
[root@adai003 ~]# netstat -an | awk '/^tcp/ {++sta[$NF]} END {for(key in sta) print key,"\t",sta[key]}' LISTEN 4 ESTABLISHED 1
ss command
The > SS command displays active socket information.The SS command can be used to obtain socket statistics, which can display content similar to netstat.The advantage of ss, however, is that it can display more detailed information about TCP and connection status, and is faster and more efficient than netstat. The disadvantage is that the process name is not displayed.
Syntax: ss [options]
Options:
-a: Show all socket s
-n: do not resolve the server name, display it digitally
[root@adai003 ~]# ss -an Netid State Recv-Q Send-Q Local Address:Port Peer Address:Port nl UNCONN 0 0 0:0 * nl UNCONN 0 0 0:-1442840033 *
Snap packages on 10.10 Linux
tcpdump command
The >tcpdump command is an sniffer tool that can print the header information of all packets that pass through the network interface or save the packets to a file with the -w option for later analysis.
Syntax: tcpdump [options]
Options:
-i: Specify the network card name to send packets using the specified network
-c: specified quantity
-w: Specify storage location
-r:=read, view package data from specified file
usage
- Tcpdump-n n-i ens33 (the first n indicates that IP is displayed digitally, if not, as the host name)
- Tcpdump-nn ens33 port 22 (not port 22) specifies port 22 (not 22)
- Tcpdump-nn ens33 port 22 and host 192.168.8.1 specify multiple conditions (host: host followed by host name or IP)
- Tcpdump-nn-i ens33-c 10-w/tmp/1.cap Specifies the number of grabs and storage location
[root@adai003 ~]# tcpdump -nn -i ens33 -c 10 -w /tmp/1.cap tcpdump: listening on ens33, link-type EN10MB (Ethernet), capture size 65535 bytes 10 packets captured 12 packets received by filter 0 packets dropped by kernel //View 1.cap file information: [root@adai003 ~]# file /tmp/1.cap /tmp/1.cap: tcpdump capture file (little-endian) - version 2.4 (Ethernet, capture length 65535)
Note: 1. The cap content cannot be viewed with the cat command, but can be viewed with the tcpdump-r command.
- Tcpdump-r 1.cap View specified packet contents
[root@adai003 ~]# tcpdump -r /tmp/1.cap reading from file /tmp/1.cap, link-type EN10MB (Ethernet) 18:42:15.311230 IP adai003.ssh > 192.168.8.1.61445: Flags [P.], seq 1594109651:1594109799, ack 208567947, win 295, length 148 18:42:15.311978 IP 192.168.8.1.61445 > adai003.ssh: Flags [.], ack 148, win 16316, length 0 18:42:16.296782 IP adai003.ssh > 192.168.8.1.61445: Flags [.], seq 148:3068, ack 1, win 295, length 2920
Description: The contents of the package are data packaged using tcpdump.
tshark command
>This command is also used to grab packages.
The tool'wireshark'needs to be installed before use:
[root@adai003 ~]# yum install -y wireshark
usage
- View access to a web service on port 80 of the specified network card (similar to the web access log):
[Command: tshark-n-t a-R http.request-T fields-e "frame.time" -e "ip.src" -e "http.host" -e "http.request.method" -e "http.request.uri"]
[root@adai003 ~]# tshark -n -t a -R http.request -T fields -e "frame.time" -e "ip.src" -e "http.host" -e "http.request.method" -e "http.request.uri"
- Grab mysql's query
Method 1: [root@adai003 ~]# tshark -n -i ens33 -R 'mysql.query' -T fields -e "ip.src" -e "mysql.query" tshark: -R without -2 is deprecated. For single-pass filtering use -Y. Running as user "root" and group "root". This could be dangerous. Capturing on 'ens33' 0 packets captured //Method 2: [root@adai003 ~]# tshark -i ens33 port 3307 -d tcp.port==3307,mysql -z "proto,colinfo,mysql.query,mysql.query"
Note: Nothing happens because there are no related processes running!
- Grab MySQL queries of specified type
[root@adai003 ~]# tshark -n -i ens33 -R 'mysql matches "SELECT|INSERT|DELETE|UPDATE"' -T fields -e "ip.src" -e "mysql.query" tshark: -R without -2 is deprecated. For single-pass filtering use -Y. Running as user "root" and group "root". This could be dangerous. Capturing on 'ens33' 8 packets dropped 0 packets captured
- Statistics http status
[root@adai003 ~]# tshark -n -q -z http,stat, -z http,tree Running as user "root" and group "root". This could be dangerous. Capturing on 'nflog' ......
Note: This command will not show results until you ctrl + c!
- tshark Add Time Label
[root@adai003 ~]# tshark -t ad [root@adai003 ~]# tshark -t a