Linux command artifact: lsof

brief introduction

lsof(list open files) is a tool that lists the open files of the current system. In the linux environment, everything exists in the form of files. Through files, you can access not only conventional data, but also network connections and hardware. Therefore, for example, transmission control protocol (TCP) and user datagram protocol (UDP) sockets, the system assigns a file descriptor to the application in the background. Regardless of the nature of the file, the file descriptor provides a general interface for the interaction between the application and the basic operating system. Because the descriptor list of the application open file provides a lot of information about the application itself, it will be very helpful for system monitoring and troubleshooting to view this list through lsof tool.

Meaning of output information

Input lsof in the terminal to display the files opened by the system. Because lsof needs to access core memory and various files, it must be run as root user to give full play to its functions.

Direct input lsof part output is:

java       32172  32317           root  138r      REG              253,0    121446  101907647 /export/server/hadoop-2.7.5/share/hadoop/yarn/hadoop-yarn-registry-2.7.5.jar
java       32172  32317           root  139r      REG              253,0    218316  100671584 /export/server/hadoop-2.7.5/share/hadoop/yarn/hadoop-yarn-server-applicationhistoryservice-2.7.5.jar
java       32172  32317           root  140r      REG              253,0     62054  100671585 /export/server/hadoop-2.7.5/share/hadoop/yarn/hadoop-yarn-server-tests-2.7.5.jar
java       32172  32317           root  141r      REG              253,0   1681766  100671586 /export/server/hadoop-2.7.5/share/hadoop/yarn/hadoop-yarn-common-2.7.5.jar
java       32172  32317           root  142r      REG              253,0     69277  100671587 /export/server/hadoop-2.7.5/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-2.7.5.jar
java       32172  32317           root  143r      REG              253,0   2033547  100671588 /export/server/hadoop-2.7.5/share/hadoop/yarn/hadoop-yarn-api-2.7.5.jar
java       32172  32317           root  144r      REG              253,0    717440  100671589 /export/server/hadoop-2.7.5/share/hadoop/yarn/hadoop-yarn-server-nodemanager-2.7.5.jar
java       32172  32317           root  145r      REG              253,0   1281734  100671590 /export/server/hadoop-2.7.5/share/hadoop/yarn/hadoop-yarn-server-resourcemanager-2.7.5.jar
java       32172  32317           root  146r      REG              253,0    386484  100671591 /export/server/hadoop-2.7.5/share/hadoop/yarn/hadoop-yarn-server-common-2.7.5.jar
java       32172  32317           root  147r      REG              253,0     58579  100671592 /export/server/hadoop-2.7.5/share/hadoop/yarn/hadoop-yarn-server-web-proxy-2.7.5.jar
java       32172  32317           root  148r      REG              253,0    165508  100671593 /export/server/hadoop-2.7.5/share/hadoop/yarn/hadoop-yarn-client-2.7.5.jar
java       32172  32317           root  149r      REG              253,0     76558  100671594 /export/server/hadoop-2.7.5/share/hadoop/yarn/hadoop-yarn-server-sharedcachemanager-2.7.5.jar
java       32172  32317           root  150r      REG              253,0    245039   68534626 /export/server/hadoop-2.7.5/share/hadoop/mapreduce/lib/junit-4.11.jar
java       32172  32317           root  151r      REG              253,0     65012   68534627 /export/server/hadoop-2.7.5/share/hadoop/mapreduce/lib/guice-servlet-3.0.jar
java       32172  32317           root  152r      REG              253,0     41056   68534628 /export/server/hadoop-2.7.5/share/hadoop/mapreduce/lib/hadoop-annotations-2.7.5.jar
java       32172  32317           root  153r      REG              253,0      2497   68534629 /export/server/hadoop-2.7.5/share/hadoop/mapreduce/lib/javax.inject-1.jar
java       32172  32317           root  154r      REG              253,0     29555   68534630 /export/server/hadoop-2.7.5/share/hadoop/mapreduce/lib/paranamer-2.3.jar
java       32172  32317           root  155r      REG              253,0    780664   68534631 /export/server/hadoop-2.7.5/share/hadoop/mapreduce/lib/jackson-mapper-asl-1.9.13.jar
java       32172  32317           root  156r      REG              253,0      4467   68534632 /export/server/hadoop-2.7.5/share/hadoop/mapreduce/lib/aopalliance-1.0.jar
java       32172  32317           root  157r      REG              253,0     45024   68534633 /export/server/hadoop-2.7.5/share/hadoop/mapreduce/lib/hamcrest-core-1.3.jar
java       32172  32317           root  158r      REG              253,0    533455   68534634 /export/server/hadoop-2.7.5/share/hadoop/mapreduce/lib/protobuf-java-2.5.0.jar
java       32172  32317           root  159r      REG              253,0    713089   68534635 /export/server/hadoop-2.7.5/share/hadoop/mapreduce/lib/jersey-server-1.9.jar
java       32172  32317           root  160r      REG              253,0     14786   68534636 /export/server/hadoop-2.7.5/share/hadoop/mapreduce/lib/jersey-guice-1.9.jar

Each line displays an open file. If no condition is specified, all files opened by all processes will be displayed by default.

The meaning of each column of information output by lsof is as follows:

COMMAND: name of the process PID: process identifier

USER: process owner

FD: file descriptor, which is used by the application to identify the file. Such as cwd, txt, etc. TYPE: file TYPE, such as DIR, REG, etc

DEVICE: Specifies the name of the disk

SIZE: the SIZE of the file

NODE: index NODE (the identification of the file on the disk)

NAME: the exact NAME of the open file

The file descriptor cwd value in the FD column indicates the current working directory of the application, which is the directory where the application starts, unless it changes this directory. txt type files are program code, such as the application binary itself or shared library, such as the / sbin/init program shown in the above table.

Next, the value represents the file descriptor of the application, which is an integer returned when the file is opened. In the last line above, the file / dev/initctl has a file descriptor of 10. u indicates that the file is open and in read / write mode, not read-only ® Or write only (w) mode. At the same time, a capital w indicates that the application has a write lock on the whole file. This file descriptor is used to ensure that only one application instance can be opened at a time. When each application is initially opened, it has three file descriptors, from 0 to 2, representing standard input, output, and error streams. Therefore, the FD of files opened by most applications starts from 3.

The Type column is more intuitive than the FD column. The files and directories are called REG and DIR, respectively. CHR and BLK represent character and block devices respectively; Or UNIX, FIFO and IPv4, representing UNIX domain socket, first in first out (FIFO) queue and Internet Protocol (IP) socket respectively.

Common parameters

The lsof syntax format is:
lsof [options] filename

lsof abc.txt Show open files abc.txt Process of
lsof -c abc display abc The file that the process now opens
lsof -c -p 1234 Lists the files opened by the process with process number 1234
lsof -g gid Display attribution gid Progress of
lsof +d /usr/local/ Displays the files opened by the process in the directory
lsof +D /usr/local/ The same as above, but it will search the directory under the directory for a long time
lsof -d 4 Display use fd Process for 4
lsof -i Used to display the process conditions that meet the conditions
lsof -i[46] [protocol][@hostname|hostaddr][:service|port]
  46 --> IPv4 or IPv6
  protocol --> TCP or UDP
  hostname --> Internet host name
  hostaddr --> IPv4 address
  service --> /etc/service Medium service name (There can be more than one)
  port --> Port number (There can be more than one)

lsof usage instance
Find out who is using the file system

When unmounting a file system, if there are any open files in the file system, the operation will usually fail. Through lsof, you can find out which processes are using the file system to be unloaded, as follows:

lsof /GTES11/

COMMAND PID USER FD TYPE DEVICE SIZE NODE NAME
bash 4208 root cwd DIR 3,1 4096 2 /GTES11/
vim 4230 root cwd DIR 3,1 4096 2 /GTES11/
In this example, the user root is doing something in his / GTES11 directory. One bash shows that the instance is running and its current directory is / GTES11. The other shows that vim is editing the file under / GTES11. To successfully uninstall / GTES11, these processes should be aborted after notifying the user to ensure that things are normal. This example shows that the current working directory of the application is very important because it still maintains file resources and prevents the file system from being unmounted. This is why most daemons (daemons) change their directories to root or service specific directories (such as / var/spool/mqueue in the sendmail example) to avoid the daemons preventing unmounting irrelevant file systems.

Utility command

lsof `which httpd` //That process is using the apache executable
lsof /etc/passwd //That process is occupying / etc/passwd
lsof /dev/hda6 //That process is occupying hda6
lsof /dev/cdrom //That process is occupying the optical drive
lsof -c sendmail //View the file usage of sendmail process
lsof -c courier -u ^zahn //It shows that those files are opened by the process starting with courier, but do not belong to the user zahn
lsof -p 30297 //Displays which files were opened by a process with pid 30297
lsof -D /tmp Show all in/tmp Open in folder instance And files. however symbol The file is not listed

lsof -u1000 //View the file usage of the user's process whose uid is 100
lsof -utony //View the file usage of user tony's process
lsof -u^tony //View the file usage of processes other than user tony (^ means reverse)
lsof -i //Show all open ports
lsof -i:80 //Displays all processes with port 80 open
lsof -i -U //Displays all open ports and UNIX domain files
lsof -i UDP@[url]www.akadia.com:123 //Show which processes are open to www.akadia.com Com UDP 123(ntp) port link
lsof -i tcp@ohaha.ks.edu.tw:ftp -r //Keep checking the current ftp connection (- r, lsof will always execute until the interrupt signal is received, + r, lsof will always execute until no file is displayed, and the default is 15s refresh)
lsof -i tcp@ohaha.ks.edu.tw:ftp -n //lsof -n does not convert IP to hostname. By default, the - n parameter is not added

Keywords: Linux Operation & Maintenance Hadoop

Added by nubby on Tue, 04 Jan 2022 10:40:20 +0200