1, Recycle bin for HDFS
There is a recycle bin in our windows system. If you want to restore deleted files, you can restore them here. HDFS also has a recycle bin.
HDFS will create a recycle bin directory for each user: / user / user name / Trash /, every file / directory deleted by the user on the Shell command line will enter the corresponding recycle bin directory. The data in the recycle bin has a life cycle, that is, when the file / directory in the recycle bin is not recovered by the user within a period of time, HDFS will automatically delete the file / directory completely, and then, The user will never find this file / directory.
By default, the recycle bin of hdfs is not enabled. It needs to be enabled through a configuration in the core site The following configuration is added to the XML. The unit of value is minutes, and 1440 minutes represents the life cycle of a day
<property> <name>fs.trash.interval</name> <value>1440</value> </property>
Verify the deletion operation before modifying the configuration information. It shows that it has been deleted directly.
[root@bigdata01 hadoop-3.2.0]# hdfs dfs -rm -r /NOTICE.txt Deleted /NOTICE.txt
Modify the recycle bin configuration, first operate on bigdata01, then synchronize to the other two nodes, and stop the cluster first
[root@bigdata01 hadoop-3.2.0]# sbin/stop-all.sh [root@bigdata01 hadoop-3.2.0]# vi etc/hadoop/core-site.xml <configuration> <property> <name>fs.defaultFS</name> <value>hdfs://bigdata01:9000</value> </property> <property> <name>hadoop.tmp.dir</name> <value>/data/hadoop_repo</value> </property> <property> <name>fs.trash.interval</name> <value>1440</value> </property> </configuration> [root@bigdata01 hadoop-3.2.0]# scp -rq etc/hadoop/core-site.xml bigdata02:/data/soft/hadoop-3.2.0/etc/hadoop/ [root@bigdata01 hadoop-3.2.0]# scp -rq etc/hadoop/core-site.xml bigdata03:/data/soft/hadoop-3.2.0/etc/hadoop/
Start the cluster and then delete it
[root@bigdata01 hadoop-3.2.0]# sbin/start-all.sh [root@bigdata01 hadoop-3.2.0]# hdfs dfs -rm -r /README.txt 2020-04-09 11:43:47,664 INFO fs.TrashPolicyDefault: Moved: 'hdfs://bigdata01:9000/README.txt' to trash at: hdfs://bigdata01:9000/user/root/.Trash/Current/README.txt
At this time, you see the prompt message that the deleted file has been moved to the specified directory, which is actually moved to the recycle bin directory of the current user.
Recycle bin files can also be downloaded locally. In fact, the recycle bin is just an HDFS directory with special meaning.
Note: if the deleted file is too large and exceeds the size of the recycle bin, you will be prompted that the deletion fails Parameters need to be specified -skipTrash ，Specifying this parameter means that deleted files will not go to the recycle bin [root@bigdata01 hadoop-3.2.0]# hdfs dfs -rm -r -skipTrash /user.txt Deleted /user.txt
2, Security mode of HDFS
When you are operating HDFS at ordinary times, you may sometimes encounter this problem. Especially when you upload or delete files when you start the cluster, you will find an error and prompt that the NameNode is in safe mode.
This belongs to the security mode of HDFS. Every time the cluster restarts, HDFS will check whether the file information in the cluster is complete, such as whether the replica is missing. Therefore, it is not allowed to modify the cluster during this time period. If this situation occurs, you can wait a little while and exit the security mode automatically after the HDFS self-test is completed.
[root@bigdata01 hadoop-3.2.0]# hdfs dfs -rm -r /hadoop-3.2.0.tar.gz 2020-04-09 12:00:36,646 WARN fs.TrashPolicyDefault: Can't create trash directory: hdfs://bigdata01:9000/user/root/.Trash/Current org.apache.hadoop.hdfs.server.namenode.SafeModeException: Cannot create directory /user/root/.Trash/Current. Name node is in safe mode.
At this time, access the web ui interface of HDFS, and you can see the following information: on indicates that it is in safe mode, and off indicates that the safe mode has ended
Or you can view the current status through the hdfs command
[root@bigdata01 hadoop-3.2.0]# hdfs dfsadmin -safemode get Safe mode is ON
If you want to leave the safe mode quickly, you can force it to leave by command. Under normal circumstances, it is recommended to exit automatically after HDFS self-test is completed
[root@bigdata01 hadoop-3.2.0]# hdfs dfsadmin -safemode leave Safe mode is OFF
At this point, you can operate the files in HDFS again.