Hadoop HDFS operation command

Hadoop HDFS operation command

View all commands supported by Hadoop HDFS
hadoop fs
List directory and file information
hadoop fs -ls
Circular listing of directory, subdirectory and file information
hadoop fs -lsr
Copy the test.txt of the local file system to the / user / sunlight directory of the HDFS file system
hadoop fs -put test.txt /user/sunlightcs
Copy test.txt from HDFS to the local file system, as opposed to the - put command
hadoop fs -get /user/sunlightcs/test.txt
View the contents of test.txt in HDFS file system
hadoop fs -cat /user/sunlightcs/test.txt
View last 1KB
hadoop fs -tail /user/sunlightcs/test.txt
Delete the test.txt file from the HDFS file system, and the rm command can also delete the empty directory
hadoop fs -rm /user/sunlightcs/test.txt
Delete the / user / sunlight directory and all subdirectories
hadoop fs -rmr /user/sunlightcs
Copying files from the local file system to the HDFS file system is equivalent to the put command
hadoop fs -copyFromLocal test.txt /user/sunlightcs/test.txt
Copying files from the HDFS file system to the local file system is equivalent to the get command
hadoop fs -copyToLocal /user/sunlightcs/test.txt test.txt
Modify the group of the / user / sunlight directory in the HDFS system, and execute the option - R recursively, just like the linux command
hadoop fs -chgrp [-R] /user/sunlightcs
Modify the owner of the / user/sunlightcs directory in the HDFS system, option - R recursively
hadoop fs -chown [-R] /user/sunlightcs
Modify the permissions of / user / sunlight CS directory in HDFS system. MODE can be 3 digits of the corresponding permissions or + / {rwx}. Option - R is executed recursively
hadoop fs -chmod [-R] MODE /user/sunlightcs
View the number of subdirectories, files, file size, file name / directory name in the PATH directory
hadoop fs -count [-q] PATH
Copy files from SRC to DST, if more than one SRC is specified, DST must be a directory
hadoop fs -cp SRC [SRC ...] DST
Displays the size of each file or directory in the directory
hadoop fs -du PATH
Similar to du, when PATH is a directory, the total size of the directory is displayed
hadoop fs -dus PATH
Empty the recycle bin. When a file is deleted, it will first be moved to the temporary directory. Trash /. When the delay time is exceeded, the file will be permanently deleted
hadoop fs -expunge
Gets all the files specified by SRC, merges them into a single file, and writes them to LOCALDST in the local file system. The option addnl adds a line break at the end of each file
hadoop fs -getmerge SRC [SRC ...] LOCALDST [addnl]
View help documents for a [ls] Command
hadoop fs -help ls
Create directory
hadoop fs -mkdir /user/hadoop/examples (directory / directory name)
View block information
hadoop fsck / -files -blocks
View the occupied capacity of the directory
sudo -u hdfs hadoop fs -du -h  /user/hive/
Modify file directory owner and its group - R: recursively modify all file owners in the directory together
sudo -u hdfs hadoop fs -chown -R hive:hive /user/hive/warehouse/tmp_tanjin.db/tmp_startup_c
Find HDFS Hive snapshot capacity
hdfs dfs -du -h /user/hive/warehouse/.snapshot/   
Create Snapshot
sudo -uhdfs hdfs dfs -createSnapshot $dir $snapshotname 

$dir: Snapshot creation directory, $snapshotname: Create snapshot name
Delete snapshot
sudo -uhdfs hdfs dfs -deleteSnapshot $dir $snapshotname

$dir  Directory path not required .snapshot  This level
111 original articles published, 94 praised, 510000 visitors+
Private letter follow

Keywords: Hadoop snapshot hive sudo

Added by True`Logic on Mon, 27 Jan 2020 14:52:31 +0200