Linux System Fundamentals (9) – file transfer

Experimental environment:

Two virtual machines in the same network segment are required:
client : 172.25.254.10
server : 172.25.254.20

I SCP command

The main function of SCP command is to realize file transfer between Linux and Linux system.
The SCP command is based on SSH protocol, so the sshd services of the two servers must be turned on, otherwise the upload and download operations cannot be completed.

#1. Upload files
scp linux Local file path remote user name@linux Host address:Remote path
#2. Download files
scp Remote user name@linux Host address:Resource path  linux Local file path

Common options:
-r		#Recursive operation
-q		#Silent transmission

Case: upload the / mnt/test file of the client server to the / root directory of the server server

#On the client side
scp /mnt/test root@172.25.254.20:/root/

Case: download the / mnt/test file of the server to the / root directory of the client server

#On the client side
scp root@172.25.254.20:/mnt/test /root/

II rsync command

rsync command is a remote data synchronization tool, which can quickly synchronize files between multiple hosts through LAN/WAN. rsync transfers only different parts of two files, rather than the whole file every time, so it is quite fast. However, when there are many small files, rsync will lead to very high hard disk I/O.

rsync linux Local file path remote user name@linux Host address:Remote path
rsync Remote user name@linux Host address:Resource path linux Local file path
 Common options:
-r			#Recursive replication
-l          #Copy link
-p          #Copy permissions
-t          #Copy timestamp
-o          #Copy owner
-g          #Replication owning group
-D          #Copy device files

III Comparison between scp and rsync

dd command:

dd if=/dev/zero of=File name bs=1M count=1
#explain:
if Represents the input file
of Represents the output file
bs Represents the block size in bytes.
count Represents the copied block.
among/dev/zero Is a character device that continuously returns 0 value bytes.

1. Establish experimental materials

dd if=/dev/zero of=/mnt/file1 bs=1M count=10
dd if=/dev/zero of=/mnt/file2 bs=1M count=20
dd if=/dev/zero of=/mnt/file3 bs=1M count=30

The above three files are established, with the size of 10m, 20m and 30m respectively

1. Configure ssh secret free connection in server

ssh-keygen
ssh-copy-id -i /root/.ssh/id_rsa.pub. root@172.25.254.20

1. Test transfer time using scripts

vim /mnt/test.sh
time scp -qr /root/Desktop root@172.25.254.20:/root/Desktop
time scp -qr /root/Desktop root@172.25.254.20:/root/Desktop
time scp -qr /root/Desktop root@172.25.254.20:/root/Desktop

vim /mnt/test1.sh
time rsync -raCq /root/Desktop root@172.25.254.20:/root/Desktop
time rsync -raCq /root/Desktop root@172.25.254.20:/root/Desktop
time rsync -raCq /root/Desktop root@172.25.254.20:/root/Desktop

1. After executing the above script, you will find that the time of using the scp command three times is almost the same, while the time of using the rsync command two times is much less than the first time.
   This is because scp is equivalent to copying, pasting, if any, covering, which is time-consuming and not intelligent. rsync synchronizes all the files for the first time. When the files are modified, only the modified files are synchronized.

IV File archiving and compression

1. Concept of packaging and archiving

Packaging: by default, the compression concept of Linux can compress only one file at a time. Direct compression is not possible for multiple files or folders. Therefore, multiple files or folders need to be packaged in advance so that compression can be performed.

tar [option]  Packaged name.tar  Multiple files or folders
[option]: 
-c				#pack
-v				#Displays the progress of the package
-f				#Specifies the packaged file name

-t				#View packaged file contents

-r				#Add files to archive

-x				#De filing

-u				#update abbreviation, which updates the files in the original package file

--get			#Unpack specified file
--delete		#Delete specified file
-C				#Specify the de filing path

Case: package a.txt, b.txt and c.txt files into abc.txt Tar file

tar -cvf abc.tar a.txt b.txt c.txt

Case: package the etc folder, etc.tar

tar -cvf etc.tar /etc/

Case: view the contents in etc.tar

tar -tf etc.tar

Case: adding files to etc.tar

tar -rf etc.tar test1         

Case: unpack etc.tar

tar -xf etc.tar 

Case: unpacking specified files

tar -f etc.tar --get test1

Case: deleting a specified file

tar -f etc.tar --delete test1

Case: specify the de filing path

tar -xf etc.tar -C /root/Desktop

2. Compressed file

zip format compression

zip -r etc.tar.zip mnt.tar		#zip format compression -r recursion
unzip etc.tar.zip				#zip format decompression
 Optional:
-d			#Unzip to the specified path

Tips: zip format can be used in both Windows and Linux.

gzip

gzip etc.tar 					#gzip format compression
gunzip etc.tar.gz				#gzip format decompression

bzip

bzip2 etc.tar					#bzip2 format compression
bunzip2 etc.tar.bz2				#bzip2 format decompression

xz

xz etc.tar						#xz format compression
unxz etc.tar.xz					#xz format decompression

Compression speed: gzip > bzip2 > XZ
Compression ratio: gzip < bzip2 < XZ

3. Packing + compression

-z	: use gzip The compression tool compresses the packaged file into.gz
-j  : use bzip2 The compression tool compresses the packaged file into.bz2
-J	: use xz The compression tool compresses the packaged file into.xz

#gzip
tar -zcf etc.tar.gz /etc
tar -zxf etc.tar.gz
 
#bzip2
tar -jcf etc.tar.bz2 /etc
tar -jxf etc.tar.bz2
 
#xz
tar -Jcf etc.tar.xz /etc
tar -Jxf etc.tar.xz