There is A blog Redirection was introduced briefly, but it was very shallow at that time. Now let's get a better understanding of redirection.
1 Standard File Descriptor
Linux uses a file descriptor to identify each file object.
The file descriptor is a non-negative integer that uniquely identifies files opened by a session. Each process can have up to nine file descriptors at a time.
The bash shell retains the first three file descriptors (0, 1, 2):
| File Descriptor | Abbreviation | describe |
|---|---|---|
| 0 | STDIN | Standard Input |
| 1 | STDOUT | standard output |
| 2 | STDERR | Standard error |
The shell uses the three file descriptors above to direct the shell's default inputs and outputs (including errors) to the location of the response.
1.1 STDIN
File descriptor 0 represents the standard input for the shell. For terminals, the standard input is the keyboard.
When we need input, we can use the input redirection symbol (<) to replace STDIN with the file specified by the redirection.
Many bash commands accept STDIN input, such as cat, which reads STDIN if no file is specified:
$ cat test1 test1 test2 test2
Enter cat only on the command line, it will accept input from STDIN, enter a line cat to display a line.
We can have cat accept input from a file by entering a redirection symbol (<):
$ cat test.txt Rave Wave hotel california nothing's gonna change my love for you $ cat < test.txt Rave Wave hotel california nothing's gonna change my love for you
1.2 STDOUT
File descriptor 1 represents the standard output of the shell. For terminals, the standard output points to the display.
By default, almost all commands direct output to STDOUT (i.e., the display).
We can use output redirection symbols (>, >) to direct output elsewhere, such as to a file.
1.3 STDERR
File descriptor 2 represents the standard error of the shell, and for terminals, the standard error points to the display as well.
By default, STDERR and STDOUT point to the same place, although the file descriptors assigned to them are different.
1.4 Redirection error
<and<< are standard input (STDIN) redirection symbols;
>and >> are standard output (STDOUT) redirection symbols;
What is the standard error (STDERR) redirection symbol?
We already know that the STDERR file descriptor is 2. In fact, the standard error (STDERR) redirection symbol is to place 2 before the redirection symbol >.
So the standard error (STDERR) redirection symbol is 2>.
Be careful:
There is no space between 2 > medium 2 and >.
Take an example of redirecting STDERR and STDOUT:
$ ls test1 no_this_file ls: cannot access no_this_file: No such file or directory test1 $ ls test1 no_this_file 2> error.log test1 $ ls test1 no_this_file 2> std_error.log > std_out.log $ cat std_error.log ls: cannot access no_this_file: No such file or directory $ cat std_out.log test1
In the above command, Standard Output Redirection Symbol > can also be written as 1 >. Because STDOUT file descriptor is 1, > can be seen as a shorthand form of 1 >.
There is a special redirection symbol, &>, that directs all output generated by the command, including STDOUT and STDERR, to the same location:
$ ls test1 no_this_file &> out $ cat out ls: cannot access no_this_file: No such file or directory test1
2 Redirect output in script
There are two ways to redirect output in a script:
- Temporary redirect output (a command is redirected once)
- Permanently redirect all commands in the script (once opened, all commands are redirected indifferently)
2.1 Temporary Redirection
If you want to redirect STDOUT to STDERR you can use >&2, you must add an &. before the file STDERR file descriptor 2. If you don't add &, > 2 redirects STDOUT to a file named 2.
Similarly, STDERR can be redirected to STDOUT using 2>&1.
Take an example 2>&1:
$ ls no_this_file > out ls: cannot access no_this_file: No such file or directory $ ls no_this_file > out 2>&1 $ cat out ls: cannot access no_this_file: No such file or directory
File no_this_file does not exist, so ls no_this_file produces STDERR, but STDERR cannot be redirected with >, so the error message is displayed on the screen;
Ls no_This_File > out 2>&1 redirects STDOUT to out file and STDERR to STDOUT at the same time, so the error information is written to out file.
Be careful:
Ls no_This_Write file 2 >&1 > out, and the error message will still be displayed on the screen.
2.2 Permanent redirection
Use the exec command to redirect a specific file descriptor during script execution.
Redirect STDOUT to file:
$ cat temp.sh #!/bin/bash exec 1>out echo "test redirecting" echo "=======" echo "author: miyan" $ ./temp.sh $ cat out test redirecting ======= author: miyan
Since exec 1>out redirects STDOUT to the out file, when the script is executed by. /temp.sh, there is no output, and all three lines of output information are in the out file.
3 Redirect input in script
Use the exec command to redirect a specific file descriptor during script execution.
Redirect STDIN to file:
$ cat out
test redirecting
=======
author: miyan
$ cat temp.sh
#!/bin/bash
exec 0< out
while read line; do
echo "$line"
done
$ ./temp.sh
test redirecting
=======
author: miyan
4 List open file descriptors
lsof - list open files.
The lsof command lists all file descriptors open on the Linux system.
By default, lsof produces a lot of output and displays information about each file opened on Linux, including all processes running in the background and all users logged in to the system.
You can use -p to specify the PID (process ID), -d to specify the file descriptor number, and -a to intersect the specified options (AND operation):
$ lsof -a -p $$ -d 0,1,2
lsof: WARNING: can't stat() proc file system /run/docker/netns/default
Output information may be incomplete.
lsof: WARNING: can't stat() proc file system /run/docker/netns/2ab0329edc51
Output information may be incomplete.
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
zsh 27644 miyan 0u CHR 136,6 0t0 9 /dev/pts/6
zsh 27644 miyan 1u CHR 136,6 0t0 9 /dev/pts/6
zsh 27644 miyan 2u CHR 136,6 0t0 9 /dev/pts/6
Special environment variable $$denotes the current process ID.
The output above shows:
Process 27644 has three file descriptors open (0, 1, 2).
COMMAND: The name of the running command;
PID;
USER: The login name of the process owner;
FD: file descriptor and access type (r for read, w for write, u for read and write);
TYPE: File type (CHR character type, BLK block type, DIR directory, REG regular file);
DEVICE: Device number;
SIZE: If there is one, it indicates the size of the file;
NODE: Node number of local file;
NAME: File name.
5 Prevent command output
You can redirect the output to a special file called a null file. There is nothing in the null file. Any data entered into the null file will not be saved and all will be discarded.
The standard location for null files is/dev/null.
6 Log messages
The tee command sends the output to both the display and the log file.
Tee is equivalent to the T-joint of a pipe. It sends input from STDIN to two locations simultaneously. One is STDOUT and the other is the file name specified on the tee command line.
Example:
$ whoami | tee out miyan $ cat out miyan
7 instances
Convert csv file to sql statement:
$ cat member.csv
mitre,boy,23
rosie,girl,21
$ cat temp.sh
#!/bin/bash
if (( $# != 1 )); then
echo "usage: $0 csvfile"
exit 1
fi
outfile="sqlfile"
IFS=","
while read name gender age; do
cat >> $outfile << EOF
insert into member (name, gender, age) values ('$name', '$gender', '$age');
EOF
done < $1
$ ./temp.sh member.csv
$ cat sqlfile
insert into member (name, gender, age) values ('mitre', 'boy', '23');
insert into member (name, gender, age) values ('rosie', 'girl', '21');
Be careful:
Why should I write the statement cat >> $outfile << EOF?
Since we can't enter content directly into a file via << EOF, we can only redirect content to a file via >>, so we did a transfer with cat here:
Content input is redirected to cat first, and then to file by cat output.