What is FastDFS?
FastDFS is an open source distributed file system. It manages files. Its functions include file storage, file synchronization, file access (file upload, file download), etc. It solves the problems of mass storage and load balancing. It is especially suitable for online services based on documents, such as photo album websites, video websites and so on.
FastDFS is an application level distributed file storage service. It adopts a central structure (similar to GFS, HDFS, TFS, etc.) and is mainly used for storing resource files in large and medium-sized websites. FastDFS has the advantages of lightweight, supporting high and distributed access, load balancing, scalability and so on. The biggest highlight of FastDFS is its good storage performance for small files, which mainly comes from its file name strategy.
1. Small file storage performance optimization
The performance bottleneck of small files mainly comes from accessing metadata servers (such as TrackerServer in FastDFS or NameServer in TFS). When the size of the file itself is very small, the proportion of metadata storage space to file content storage space becomes larger, and the proportion of resources consumed by accessing metadata and accessing file content also becomes larger. Therefore, there are two main ways to optimize small file storage: one is to reduce the number of metadata accesses, such as Cache prefetching; The second is to reduce the storage space occupied by metadata, such as the file name policy used by FastDFS.
2. FastDFS file name policy
The file name in FastDFS is specified by the system when storing files to StorageServer. The file name includes VolumeID and FileID. That is, when a customer wants to read a file, it can know which Volume the file is stored on and its FileID in the StorageServer by parsing the file name on the client. However, at this time, the user cannot read the file because he does not know the ip address of each StorageServer in the Volume or which StorageServer in the Volume he should read from. Therefore, the user needs to hold the VolumeID of the file to be accessed and ask the TrackerServer. TrackerServe will balance the Current IO load of each StorageServer and return an optimal ip address of the StorageServer. Finally, the user connects to the StorageServer and shows the FileID of the file to be accessed. A table of the offset corresponding to the FileID will be maintained on the StorageServer to obtain the offset of the file to be accessed.
It can be seen that the file name policy of FastDFS hides the file storage location information in the file name, thus reducing the amount of metadata and optimizing the performance of small file storage.
The FastDFS server has two roles: tracker and storage node. The tracker mainly does scheduling work and plays the role of load balancing in access.
Upload interactive process editing
-
The client asks about the storage uploaded by the tracker without additional parameters;
-
The tracker returns an available storage;
-
client communicates directly with storage to complete file upload.
FastDFS file download
Download interactive process editing
-
The client asks the tracker to download the storage of the file. The parameter is the file ID (volume name and file name);
-
The tracker returns an available storage;
-
client communicates directly with storage to complete file download.
It should be noted that the client is the caller using FastDFS service, and the client should also be a server. Its calls to tracker and storage are calls between servers.
The storage node stores files and completes all functions of file management: storage, synchronization and providing access interfaces. FastDFS manages the meta data of files at the same time. The so-called meta data of a file is the related attributes of the file, expressed in the form of key value pairs, such as: width=1024, where key is width and value is 1024. File meta data is a list of file attributes and can contain multiple key value pairs.
The FastDFS system structure is shown in the following figure:
Have four files ready
Download address:
https://github.com/happyfish100
CentOS 7 FastDFS setup
Install libfastcommon
Upload to the server and unzip:
[root@localhost home]# unzip libfastcommon-master.zip
Enter the libfastcommon-1.0.36 Directory:
[root@localhost home]# cd libfastcommon-master/ [root@localhost libfastcommon-master]# ll total 32 drwxr-xr-x. 2 root root 114 Dec 6 11:49 doc -rw-r--r--. 1 root root 10179 Dec 6 11:49 HISTORY -rw-r--r--. 1 root root 674 Dec 6 11:49 INSTALL -rw-r--r--. 1 root root 1607 Dec 6 11:49 libfastcommon.spec -rwxr-xr-x. 1 root root 3253 Dec 6 11:49 make.sh drwxr-xr-x. 2 root root 191 Dec 6 11:49 php-fastcommon -rw-r--r--. 1 root root 2776 Dec 6 11:49 README drwxr-xr-x. 3 root root 4096 Dec 6 11:49 src [root@localhost libfastcommon-master]#
Install gcc using yum or up2date:
The Yum command is quite easy to use. RedHad and CentOS download RPM packages from the specified server and install them automatically. I personally prefer it.
[root@localhost libfastcommon-master]# yum -y install gcc-c++
Complete! Execution is complete!
At this time, execute. / make.sh and. / make.sh install respectively. It can succeed under normal conditions.
[root@localhost libfastcommon-master]# ./make.sh cc -Wall -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -g -O3 -c -o hash.o hash.c cc -Wall -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -g -O3 -c -o chain.o chain.c cc -Wall -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -g -O3 -c -o shared_func.o shared_func.c cc -Wall -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -g -O3 -c -o ini_file_reader.o ini_file_reader.c cc -Wall -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -g -O3 -c -o logger.o logger.c cc -Wall -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -g -O3 -c -o sockopt.o sockopt.c cc -Wall -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -g -O3 -c -o base64.o base64.c cc -Wall -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -g -O3 -c -o sched_thread.o sched_thread.c cc -Wall -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -g -O3 -c -o http_func.o http_func.c cc -Wall -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -g -O3 -c -o md5.o md5.c cc -Wall -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -g -O3 -c -o pthread_func.o pthread_func.c cc -Wall -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -g -O3 -c -o local_ip_func.o local_ip_func.c cc -Wall -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -g -O3 -c -o avl_tree.o avl_tree.c cc -Wall -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -g -O3 -c -o ioevent.o ioevent.c cc -Wall -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -g -O3 -c -o ioevent_loop.o ioevent_loop.c cc -Wall -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -g -O3 -c -o fast_task_queue.o fast_task_queue.c cc -Wall -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -g -O3 -c -o fast_timer.o fast_timer.c cc -Wall -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -g -O3 -c -o process_ctrl.o process_ctrl.c cc -Wall -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -g -O3 -c -o fast_mblock.o fast_mblock.c cc -Wall -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -g -O3 -c -o connection_pool.o connection_pool.c cc -Wall -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -g -O3 -c -o fast_mpool.o fast_mpool.c cc -Wall -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -g -O3 -c -o fast_allocator.o fast_allocator.c cc -Wall -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -g -O3 -c -o fast_buffer.o fast_buffer.c cc -Wall -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -g -O3 -c -o multi_skiplist.o multi_skiplist.c cc -Wall -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -g -O3 -c -o flat_skiplist.o flat_skiplist.c cc -Wall -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -g -O3 -c -o system_info.o system_info.c cc -Wall -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -g -O3 -c -o fast_blocked_queue.o fast_blocked_queue.c cc -Wall -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -g -O3 -c -o id_generator.o id_generator.c cc -Wall -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -g -O3 -c -o char_converter.o char_converter.c cc -Wall -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -g -O3 -c -o char_convert_loader.o char_convert_loader.c cc -Wall -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -g -O3 -c -o common_blocked_queue.o common_blocked_queue.c cc -Wall -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -g -O3 -c -o multi_socket_client.o multi_socket_client.c cc -Wall -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -g -O3 -c -o skiplist_set.o skiplist_set.c cc -Wall -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -g -O3 -c -o json_parser.o json_parser.c cc -Wall -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -g -O3 -c -fPIC -o hash.lo hash.c cc -Wall -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -g -O3 -c -fPIC -o chain.lo chain.c cc -Wall -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -g -O3 -c -fPIC -o shared_func.lo shared_func.c cc -Wall -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -g -O3 -c -fPIC -o ini_file_reader.lo ini_file_reader.c cc -Wall -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -g -O3 -c -fPIC -o logger.lo logger.c cc -Wall -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -g -O3 -c -fPIC -o sockopt.lo sockopt.c cc -Wall -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -g -O3 -c -fPIC -o base64.lo base64.c cc -Wall -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -g -O3 -c -fPIC -o sched_thread.lo sched_thread.c cc -Wall -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -g -O3 -c -fPIC -o http_func.lo http_func.c cc -Wall -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -g -O3 -c -fPIC -o md5.lo md5.c cc -Wall -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -g -O3 -c -fPIC -o pthread_func.lo pthread_func.c cc -Wall -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -g -O3 -c -fPIC -o local_ip_func.lo local_ip_func.c cc -Wall -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -g -O3 -c -fPIC -o avl_tree.lo avl_tree.c cc -Wall -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -g -O3 -c -fPIC -o ioevent.lo ioevent.c cc -Wall -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -g -O3 -c -fPIC -o ioevent_loop.lo ioevent_loop.c cc -Wall -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -g -O3 -c -fPIC -o fast_task_queue.lo fast_task_queue.c cc -Wall -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -g -O3 -c -fPIC -o fast_timer.lo fast_timer.c cc -Wall -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -g -O3 -c -fPIC -o process_ctrl.lo process_ctrl.c cc -Wall -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -g -O3 -c -fPIC -o fast_mblock.lo fast_mblock.c cc -Wall -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -g -O3 -c -fPIC -o connection_pool.lo connection_pool.c cc -Wall -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -g -O3 -c -fPIC -o fast_mpool.lo fast_mpool.c cc -Wall -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -g -O3 -c -fPIC -o fast_allocator.lo fast_allocator.c cc -Wall -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -g -O3 -c -fPIC -o fast_buffer.lo fast_buffer.c cc -Wall -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -g -O3 -c -fPIC -o multi_skiplist.lo multi_skiplist.c cc -Wall -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -g -O3 -c -fPIC -o flat_skiplist.lo flat_skiplist.c cc -Wall -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -g -O3 -c -fPIC -o system_info.lo system_info.c cc -Wall -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -g -O3 -c -fPIC -o fast_blocked_queue.lo fast_blocked_queue.c cc -Wall -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -g -O3 -c -fPIC -o id_generator.lo id_generator.c cc -Wall -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -g -O3 -c -fPIC -o char_converter.lo char_converter.c cc -Wall -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -g -O3 -c -fPIC -o char_convert_loader.lo char_convert_loader.c cc -Wall -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -g -O3 -c -fPIC -o common_blocked_queue.lo common_blocked_queue.c cc -Wall -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -g -O3 -c -fPIC -o multi_socket_client.lo multi_socket_client.c cc -Wall -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -g -O3 -c -fPIC -o skiplist_set.lo skiplist_set.c cc -Wall -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -g -O3 -c -fPIC -o json_parser.lo json_parser.c cc -Wall -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -g -O3 -o libfastcommon.so -shared hash.lo chain.lo shared_func.lo ini_file_reader.lo logger.lo sockopt.lo base64.lo sched_thread.lo http_func.lo md5.lo pthread_func.lo local_ip_func.lo avl_tree.lo ioevent.lo ioevent_loop.lo fast_task_queue.lo fast_timer.lo process_ctrl.lo fast_mblock.lo connection_pool.lo fast_mpool.lo fast_allocator.lo fast_buffer.lo multi_skiplist.lo flat_skiplist.lo system_info.lo fast_blocked_queue.lo id_generator.lo char_converter.lo char_convert_loader.lo common_blocked_queue.lo multi_socket_client.lo skiplist_set.lo json_parser.lo -lm -ldl -lpthread ar rcs libfastcommon.a hash.o chain.o shared_func.o ini_file_reader.o logger.o sockopt.o base64.o sched_thread.o http_func.o md5.o pthread_func.o local_ip_func.o avl_tree.o ioevent.o ioevent_loop.o fast_task_queue.o fast_timer.o process_ctrl.o fast_mblock.o connection_pool.o fast_mpool.o fast_allocator.o fast_buffer.o multi_skiplist.o flat_skiplist.o system_info.o fast_blocked_queue.o id_generator.o char_converter.o char_convert_loader.o common_blocked_queue.o multi_socket_client.o skiplist_set.o json_parser.o [root@localhost libfastcommon-master]# ./make.sh install mkdir -p /usr/lib64 mkdir -p /usr/lib mkdir -p /usr/include/fastcommon install -m 755 libfastcommon.so /usr/lib64 install -m 644 common_define.h hash.h chain.h logger.h base64.h shared_func.h pthread_func.h ini_file_reader.h _os_define.h sockopt.h sched_thread.h http_func.h md5.h local_ip_func.h avl_tree.h ioevent.h ioevent_loop.h fast_task_queue.h fast_timer.h process_ctrl.h fast_mblock.h connection_pool.h fast_mpool.h fast_allocator.h fast_buffer.h skiplist.h multi_skiplist.h flat_skiplist.h skiplist_common.h system_info.h fast_blocked_queue.h php7_ext_wrapper.h id_generator.h char_converter.h char_convert_loader.h common_blocked_queue.h multi_socket_client.h skiplist_set.h fc_list.h json_parser.h /usr/include/fastcommon if [ ! -e /usr/lib/libfastcommon.so ]; then ln -s /usr/lib64/libfastcommon.so /usr/lib/libfastcommon.so; fi [root@localhost libfastcommon-master]#
Libfastcommon will be installed in / usr/lib64/libfastcommon.so by default, but the main program of FastDFS is in / usr/local/lib directory
At this time, we will establish a soft link, which is actually equivalent to a shortcut on windows.
ln -s /usr/lib64/libfastcommon.so /usr/local/lib/libfastcommon.so
ln -s /usr/lib64/libfastcommon.so /usr/lib/libfastcommon.so
ln -s /usr/lib64/libfdfsclient.so /usr/local/lib/libfdfsclient.so
ln -s /usr/lib64/libfdfsclient.so /usr/lib/libfdfsclient.so
[root@localhost libfastcommon-master]# ln -s /usr/lib64/libfastcommon.so /usr/local/lib/libfastcommon.so [root@localhost libfastcommon-master]# ln -s /usr/lib64/libfastcommon.so /usr/lib/libfastcommon.so ln: failed to create symbolic link '/usr/lib/libfastcommon.so': File exists [root@localhost libfastcommon-master]# ln -s /usr/lib64/libfdfsclient.so /usr/local/lib/libfdfsclient.so [root@localhost libfastcommon-master]# ln -s /usr/lib64/libfdfsclient.so /usr/lib/libfdfsclient.so [root@localhost libfastcommon-master]#
Installing FastDFS
Unzip the FastDFS installation package
[root@localhost home]# unzip fastdfs-master.zip
After decompression, you can see:
[root@localhost home]# ll total 2144 drwx------. 2 dev1 dev1 62 Apr 11 2018 dev1 drwxr-xr-x. 12 root root 4096 Dec 8 10:17 fastdfs-master -rw-r--r--. 1 root root 905173 Dec 10 20:24 fastdfs-master.zip -rw-r--r--. 1 root root 22492 Dec 10 20:24 fastdfs-nginx-module-master.zip drwxr-xr-x. 5 root root 153 Dec 10 20:37 libfastcommon-master -rw-r--r--. 1 root root 218881 Dec 10 20:24 libfastcommon-master.zip -rw-r--r--. 1 root root 1037527 Dec 10 20:24 nginx-1.17.6.tar.gz
Go to the directory you just extracted
[root@localhost home]# cd fastdfs-master/ [root@localhost fastdfs-master]# ./make.sh [root@localhost fastdfs-master]# ./make.sh install
If there is no error, it will succeed. The installation log will prompt that FastDFS has been installed in the / etc/fdfs directory.
View the installation directory after success:
[root@localhost fastdfs-master]# cd /etc/fdfs/ [root@localhost fdfs]# ll total 28 -rw-r--r--. 1 root root 1909 Dec 10 20:40 client.conf.sample -rw-r--r--. 1 root root 10246 Dec 10 20:40 storage.conf.sample -rw-r--r--. 1 root root 620 Dec 10 20:40 storage_ids.conf.sample -rw-r--r--. 1 root root 8128 Dec 10 20:40 tracker.conf.sample [root@localhost fdfs]#
We need to make a copy of these three sample files and remove. Sample.
[root@localhost fdfs]# cp client.conf.sample client.conf [root@localhost fdfs]# cp storage.conf.sample storage.conf [root@localhost fdfs]# cp tracker.conf.sample tracker.conf [root@localhost fdfs]# ll total 52 -rw-r--r--. 1 root root 1909 Dec 10 20:40 client.conf -rw-r--r--. 1 root root 1909 Dec 10 20:40 client.conf.sample -rw-r--r--. 1 root root 10246 Dec 10 20:40 storage.conf -rw-r--r--. 1 root root 10246 Dec 10 20:40 storage.conf.sample -rw-r--r--. 1 root root 620 Dec 10 20:40 storage_ids.conf.sample -rw-r--r--. 1 root root 8128 Dec 10 20:40 tracker.conf -rw-r--r--. 1 root root 8128 Dec 10 20:40 tracker.conf.sample [root@localhost fdfs]#
FastDFS installation ended.
Install tracker
Create tracker working directory
This directory can be customized to save the tracker's data and log
According to my personal habits, I created the following directory:
[root@localhost ~]# mkdir /data [root@localhost ~]# cd /data/ [root@localhost data]# mkdir fastdfs [root@localhost data]# cd fastdfs/ [root@localhost fastdfs]# mkdir fastdfs_tracker [root@localhost fastdfs]# cd fastdfs_tracker/ [root@localhost fastdfs_tracker]# pwd /data/fastdfs/fastdfs_tracker [root@localhost fastdfs_tracker]#
Configure tracker
[root@localhost fastdfs-5.11]# cd /etc/fdfs/ [root@localhost fastdfs-5.11]# vim tracker.conf [root@localhost fastdfs-5.11]#
The minimized CentOS7 does not have VIM installed. You can change the vim tracker.conf command to vi tracker.conf, or download a vim
yum -y install vim
After opening, focus on the following four configurations:
disabled = false port=22122 base_path=/data/fastdfs/fastdfs_tracker http.server_port=6666 # Default 8080 9901
Start tracker
Start the tracker after saving the configuration. The command is as follows:
service fdfs_trackerd start
If it cannot be started or prompted to use systemctl, you can use the command instead:
systemctl start fdfs_trackerd
After success, you should see:
[root@localhost fdfs]# service fdfs_trackerd start Reloading systemd: [ OK ] Starting fdfs_trackerd (via systemctl): [ OK ] [root@localhost fdfs]#
The tracker directory just created is found to contain two more directories: data and log
[root@localhost fdfs]# cd /data/fastdfs/fastdfs_tracker/ [root@localhost fastdfs_tracker]# ll total 0 drwxr-xr-x. 2 root root 83 Dec 10 20:45 data drwxr-xr-x. 2 root root 26 Dec 10 20:44 logs [root@localhost fastdfs_tracker]#
Finally, we need to add boot to the tracker
[root@localhost fastdfs_tracker]# ll /etc/rc.d/rc.local -rw-r--r--. 1 root root 473 Feb 20 2019 /etc/rc.d/rc.local [root@localhost fastdfs_tracker]#
If you find that you do not have execution permission, you need to add the following:
[root@localhost fastdfs_tracker]# chmod +x /etc/rc.d/rc.local [root@localhost fastdfs_tracker]#
After adding, it should be like this:
[root@localhost fastdfs_tracker]# ll /etc/rc.d/rc.local -rwxr-xr-x. 1 root root 473 Feb 20 2019 /etc/rc.d/rc.local [root@localhost fastdfs_tracker]#
Modify rc.local
[root@localhost fastdfs_tracker]# vim /etc/rc.d/rc.local #!/bin/bash # THIS FILE IS ADDED FOR COMPATIBILITY PURPOSES # # It is highly advisable to create own systemd services or udev rules # to run scripts during boot instead of using this file. # # In contrast to previous versions due to parallel execution during boot # this script will NOT be run after all other services. # # Please note that you must run 'chmod +x /etc/rc.d/rc.local' to ensure # that this script will be executed during boot. touch /var/lock/subsys/local service fdfs_trackerd start
Check the port listening of the tracker
[root@localhost fastdfs_tracker]# netstat -unltp|grep fdfs tcp 0 0 0.0.0.0:22122 0.0.0.0:* LISTEN 20553/fdfs_trackerd [root@localhost fastdfs_tracker]#
Port 22122 successfully listens.
Install storage
The installation of storage is similar to that of tracker.
Configure working directory for storage
Unlike the tracker, the storage also needs a directory to store data, so I built an additional fasdfs_storage_data
Here is my directory structure:
[root@localhost fastdfs]# cd /data/fastdfs [root@localhost fastdfs]# mkdir fastdfs_storage [root@localhost fastdfs]# mkdir fastdfs_storage_data [root@localhost fastdfs]# ll total 0 drwxr-xr-x. 2 root root 6 Dec 10 20:43 fastdfs_storage drwxr-xr-x. 2 root root 6 Dec 10 20:43 fastdfs_storage_data drwxr-xr-x. 4 root root 30 Dec 10 20:42 fastdfs_tracker [root@localhost fastdfs]#
Modify storage profile
Modify storage.conf
[root@localhost fastdfs]# vim /etc/fdfs/storage.conf disabled=false group_name=group1 #Group name, modified according to the actual situation port=23000 #Set the storage port number, which is 23000 by default. The storage port numbers of the same group must be the same base_path=/data/fastdfs/fastdfs_storage #Set the storage data file and log directory store_path_count=1 #Number of storage paths, and store_ The number of paths matches >>>>>store_path0=/data/fastdfs/fastdfs_storage_data #Actual file storage path >>>>>base_path0=/data/fastdfs/fastdfs_storage_data #Actual file storage path tracker_server=192.168.31.100:22122 #My CentOS7 ip address http.server_port=8888 #Set http port number [root@localhost fastdfs]#
Create soft reference after saving changes
[root@localhost fastdfs]# ln -s /usr/bin/fdfs_storaged /usr/local/bin [root@localhost fastdfs]#
Start storage
service fdfs_storaged start
If it cannot be started or prompted to use systemctl, you can use the command instead:
systemctl start fdfs_storaged
After success, you should see:
[root@localhost fastdfs]# service fdfs_storaged start Starting fdfs_storaged (via systemctl): [ OK ] [root@localhost fastdfs]#
Similarly, set startup:
Modify rc.local
#!/bin/bash # THIS FILE IS ADDED FOR COMPATIBILITY PURPOSES # # It is highly advisable to create own systemd services or udev rules # to run scripts during boot instead of using this file. # # In contrast to previous versions due to parallel execution during boot # this script will NOT be run after all other services. # # Please note that you must run 'chmod +x /etc/rc.d/rc.local' to ensure # that this script will be executed during boot. touch /var/lock/subsys/local service fdfs_trackerd start service fdfs_storaged start
Check whether the service is started
[root@localhost fastdfs]# netstat -unltp | grep fdfs tcp 0 0 0.0.0.0:22122 0.0.0.0:* LISTEN 20553/fdfs_trackerd tcp 0 0 0.0.0.0:23000 0.0.0.0:* LISTEN 20817/fdfs_storaged [root@localhost fastdfs]#
Verification integration
At this point, fastdfs has been installed. Finally, we need to make sure whether the storage is registered in the tracker.
View command:
[root@localhost fastdfs]# /usr/bin/fdfs_monitor /etc/fdfs/storage.conf
[2019-12-10 21:04:31] DEBUG - base_path=/data/fastdfs/fastdfs_storage, connect_timeout=5, network_timeout=60, tracker_server_count=1, anti_steal_token=0, anti_steal_secret_key length=0, use_connection_pool=1, g_connection_pool_max_idle_time=3600s, use_storage_id=0, storage server id count: 0
server_count=1, server_index=0
tracker server is 192.168.31.100:22122
group count: 1
Group 1:
group name = group1
disk total space = 51175 MB
disk free space = 48527 MB
trunk free space = 0 MB
storage server count = 1
active server count = 1
storage server port = 23000
storage HTTP port = 8888
store path count = 1
subdir count per path = 256
current write server index = 0
current trunk file id = 0
Storage 1:
id = 10.254.193.118
ip_addr = 10.254.193.118 (anantes-651-1-49-net.w2-0.abo.wanadoo.fr) ACTIVE
http domain =
version = 6.04
join time = 2019-12-10 21:01:47
up time = 2019-12-10 21:01:47
total storage = 51175 MB
free storage = 48527 MB
upload priority = 10
store_path_count = 1
subdir_count_per_path = 256
storage_port = 23000
storage_http_port = 8888
current_write_path = 0
source storage id =
if_trunk_server = 0
connection.alloc_count = 256
connection.current_count = 0
connection.max_count = 0
total_upload_count = 0
success_upload_count = 0
total_append_count = 0
success_append_count = 0
total_modify_count = 0
success_modify_count = 0
total_truncate_count = 0
success_truncate_count = 0
total_set_meta_count = 0
success_set_meta_count = 0
total_delete_count = 0
success_delete_count = 0
total_download_count = 0
success_download_count = 0
total_get_meta_count = 0
success_get_meta_count = 0
total_create_link_count = 0
success_create_link_count = 0
total_delete_link_count = 0
success_delete_link_count = 0
total_upload_bytes = 0
success_upload_bytes = 0
total_append_bytes = 0
success_append_bytes = 0
total_modify_bytes = 0
success_modify_bytes = 0
stotal_download_bytes = 0
success_download_bytes = 0
total_sync_in_bytes = 0
success_sync_in_bytes = 0
total_sync_out_bytes = 0
success_sync_out_bytes = 0
total_file_open_count = 0
success_file_open_count = 0
total_file_read_count = 0
success_file_read_count = 0
total_file_write_count = 0
success_file_write_count = 0
last_heart_beat_time = 2019-12-10 21:04:20
last_source_update = 1970-01-01 08:00:00
last_sync_update = 1970-01-01 08:00:00
last_synced_timestamp = 1970-01-01 08:00:00
[root@localhost fastdfs]#
test
The installation and configuration of FastDFS have been explained in detail. The basic modules of FastDFS have been set up. Now start testing and downloading.
Configure client
Similarly, you need to modify the configuration file of the client:
[root@localhost fastdfs]# vim /etc/fdfs/client.conf base_path = /data/fastdfs/fastdfs_tracker tracker_server = 192.168.31.100:22122 http.tracker_server_port = 6666 # Default port 80
Upload pictures to CentOS via ftp:
On my windows, I casually dragged a picture to it.
[root@localhost fastdfs]# cd /home/ [root@localhost home]# ll total 1148 -rw-r--r--. 1 root root 18832 Oct 12 12:58 123.png drwx------. 2 dev1 dev1 62 Apr 11 2018 dev1 drwxr-xr-x. 12 root root 4096 Dec 8 10:17 fastdfs-5.11 -rw-r--r--. 1 root root 905173 Dec 10 11:38 fastdfs-5.11.zip -rw-r--r--. 1 root root 22492 Dec 10 11:38 fastdfs-nginx-module-master.zip drwxr-xr-x. 5 root root 153 Dec 10 19:48 libfastcommon-1.0.36 -rw-r--r--. 1 root root 218881 Dec 10 11:36 libfastcommon-1.0.36.zip [root@localhost home]#
Simulation upload
After determining the location of the picture, we enter the upload picture command:
[root@localhost home]# /usr/bin/fdfs_upload_file /etc/fdfs/client.conf /home/123.jpg group1/M00/00/00/wKgfZF3vl6WAJDW5AAvWFlS1kOw230.jpg [root@localhost home]#
The path of the picture will be returned after success:
[root@localhost~]# /usr/bin/fdfs_upload_file /etc/fdfs/client.conf /home/123.png
group1/M00/00/00/wKgfZF3vl6WAJDW5AAvWFlS1kOw230.jpg
Group name: group1
Disk: M00
Catalog: 00 / 00
File name: wKgfZF3vl6WAJDW5AAvWFlS1kOw230.png
The pictures we upload will be uploaded to the storage we create_ Under the data directory, let's take a look:
[root@localhost fastdfs_storage_data]# cd /data/fastdfs/fastdfs_storage_data/data/ [root@localhost data]# ls 00 06 0C 12 18 1E 24 2A 30 36 3C 42 48 4E 54 5A 60 66 6C 72 78 7E 84 8A 90 96 9C A2 A8 AE B4 BA C0 C6 CC D2 D8 DE E4 EA F0 F6 FC 01 07 0D 13 19 1F 25 2B 31 37 3D 43 49 4F 55 5B 61 67 6D 73 79 7F 85 8B 91 97 9D A3 A9 AF B5 BB C1 C7 CD D3 D9 DF E5 EB F1 F7 FD 02 08 0E 14 1A 20 26 2C 32 38 3E 44 4A 50 56 5C 62 68 6E 74 7A 80 86 8C 92 98 9E A4 AA B0 B6 BC C2 C8 CE D4 DA E0 E6 EC F2 F8 FE 03 09 0F 15 1B 21 27 2D 33 39 3F 45 4B 51 57 5D 63 69 6F 75 7B 81 87 8D 93 99 9F A5 AB B1 B7 BD C3 C9 CF D5 DB E1 E7 ED F3 F9 FF 04 0A 10 16 1C 22 28 2E 34 3A 40 46 4C 52 58 5E 64 6A 70 76 7C 82 88 8E 94 9A A0 A6 AC B2 B8 BE C4 CA D0 D6 DC E2 E8 EE F4 FA 05 0B 11 17 1D 23 29 2F 35 3B 41 47 4D 53 59 5F 65 6B 71 77 7D 83 89 8F 95 9B A1 A7 AD B3 B9 BF C5 CB D1 D7 DD E3 E9 EF F5 FB [root@localhost data]# cd 00/ [root@localhost 00]# ls 00 06 0C 12 18 1E 24 2A 30 36 3C 42 48 4E 54 5A 60 66 6C 72 78 7E 84 8A 90 96 9C A2 A8 AE B4 BA C0 C6 CC D2 D8 DE E4 EA F0 F6 FC 01 07 0D 13 19 1F 25 2B 31 37 3D 43 49 4F 55 5B 61 67 6D 73 79 7F 85 8B 91 97 9D A3 A9 AF B5 BB C1 C7 CD D3 D9 DF E5 EB F1 F7 FD 02 08 0E 14 1A 20 26 2C 32 38 3E 44 4A 50 56 5C 62 68 6E 74 7A 80 86 8C 92 98 9E A4 AA B0 B6 BC C2 C8 CE D4 DA E0 E6 EC F2 F8 FE 03 09 0F 15 1B 21 27 2D 33 39 3F 45 4B 51 57 5D 63 69 6F 75 7B 81 87 8D 93 99 9F A5 AB B1 B7 BD C3 C9 CF D5 DB E1 E7 ED F3 F9 FF 04 0A 10 16 1C 22 28 2E 34 3A 40 46 4C 52 58 5E 64 6A 70 76 7C 82 88 8E 94 9A A0 A6 AC B2 B8 BE C4 CA D0 D6 DC E2 E8 EE F4 FA 05 0B 11 17 1D 23 29 2F 35 3B 41 47 4D 53 59 5F 65 6B 71 77 7D 83 89 8F 95 9B A1 A7 AD B3 B9 BF C5 CB D1 D7 DD E3 E9 EF F5 FB [root@localhost 00]# cd 00 [root@localhost 00]# ls wKgfZF3vl6WAJDW5AAvWFlS1kOw230.png [root@localhost 00]#
Sure enough, through the path we just returned, we successfully found the picture.
Let's take a closer look. There are created multi-level directories under the actual file storage path. There are 256 level-1 directories under data, and 256 level-2 subdirectories under each level of directory, with a total of 65536 files. The newly written files will be hash ed to one of the subdirectories, and then the file data will be directly stored in the directory as a local file.
HTTP access file
Let's go to the browser and use http to access the picture just now:
Write a picture description here
We found that http can not directly access images. Why.
I went to the official website and looked at the original code. In HISTORY, I found that I had removed embedded HTTP support as early as 4.05
Version 4.05 2012-12-30
* client/fdfs_upload_file.c can specify storage ip port and store path index
* add connection pool
* client load storage ids config
* common/ini_file_reader.c does NOT call chdir
* keep the mtime of file same
* use g_current_time instead of call time function
* remove embed HTTP support
The reason why the HTTP request cannot access the file
When using FastDFS to deploy a distributed file system, we upload, download and delete files through FastDFS client API. At the same time, HTTP services are provided through the HTTP server of FastDFS. However, the HTTP service of FastDFS is relatively simple and cannot provide high-performance services such as load balancing. Therefore, Yu Qing, the architect of Taobao, the developer of FastDFS, provided us with the FastDFS module used on Nginx (also known as the Nginx module of FastDFS).
FastDFS stores files on the Storage server through the Tracker server, but servers in the same group need to copy files, which has a delay problem. Suppose the Tracker server uploads files to 192.168.128.131 and the file ID has been returned to the client. At this time, the background will copy the file to 192.168.128.131. If the replication is not completed, The client uses this ID to get the file at 192.168.128.131, and there will be an error. This FastDFS nginx module can redirect the connection to the source server to fetch files, so as to avoid errors on the client due to replication delay.
Just like this, FastDFS needs to be combined with nginx, so the original direct support for HTTP is cancelled.
Installation of nginx module for FastDFS
Installation nginx preparation
Upload nginx-1.17.6.tar.gz to linux
[root@localhost 00]# cd /home/ [root@lml74xunyuanfuwuqi home]# ll Total consumption 2164 -rw-r--r-- 1 root root 18832 10 December 12:58 123.png drwx------ 2 dceq dceq 62 11 June 15-18:14 dceq drwxr-xr-x 12 root root 4096 12 August 10:17 fastdfs-master -rw-r--r-- 1 root root 905173 12 October 11:38 fastdfs-master.zip -rw-r--r-- 1 root root 22492 12 October 11:38 fastdfs-nginx-module-master.zip drwxr-xr-x 5 root root 153 12 November 15:07 libfastcommon-master -rw-r--r-- 1 root root 218881 12 October 11:36 libfastcommon-master.zip -rw-r--r-- 1 root root 1037527 12 October 13:38 nginx-1.17.6.tar.gz drwxr-xr-x 3 root root 21 12 November 9:03 work [root@localhost home]#
Before installing nginx, install the dependent lib required by nginx:
[root@localhost home]# yum -y install pcre pcre-devel [root@localhost home]# yum -y install zlib zlib-devel [root@localhost home]# yum -y install openssl openssl-devel
Install nginx and add fastdfs nginx module
Unzip nginx and fastdfs nginx module:
[root@localhost home]# tar -zxvf nginx-1.17.6.tar.gz [root@localhost home]# unzip fastdfs-nginx-module-master.zip
After decompression, enter the nginx directory, compile and install nginx, and add fastdfs nginx module:
[root@lml74xunyuanfuwuqi home]# cd nginx-1.17.6 [root@localhost nginx-1.17.6]# ./configure --prefix=/usr/local/nginx --add-module=/home/fastdfs-nginx-module-master/src #Location of fastdfs nginx module after decompression
If the configuration does not report an error, start compiling:
[root@localhost nginx-1.17.6]# make [root@localhost nginx-1.17.6]# make install
If an error is reported, it may be due to the version. In my second blog post, I provided a download of the version that I tested successfully without an error.
The default directory of nginx is / usr/local/nginx. After successful installation, view:
[root@localhost nginx-1.17.6]# cd /usr/local/nginx/ [root@localhost nginx]# ll total 4 drwxr-xr-x. 2 root root 4096 Dec 10 21:06 conf drwxr-xr-x. 2 root root 40 Dec 10 21:06 html drwxr-xr-x. 2 root root 6 Dec 10 21:06 logs drwxr-xr-x. 2 root root 19 Dec 10 21:06 sbin [root@lml74xunyuanfuwuqi nginx]# cd conf/ [root@localhost nginx]# [root@lml74xunyuanfuwuqi conf]# vi nginx.conf
Configure storage nginx
Modify nginx.conf:
Modify the listening port listen 9999 and add location
[root@localhost nginx]# cd conf/
[root@localhost conf]# ll
total 68
-rw-r--r--. 1 root root 1077 Dec 10 21:06 fastcgi.conf
-rw-r--r--. 1 root root 1077 Dec 10 21:06 fastcgi.conf.default
-rw-r--r--. 1 root root 1007 Dec 10 21:06 fastcgi_params
-rw-r--r--. 1 root root 1007 Dec 10 21:06 fastcgi_params.default
-rw-r--r--. 1 root root 2837 Dec 10 21:06 koi-utf
-rw-r--r--. 1 root root 2223 Dec 10 21:06 koi-win
-rw-r--r--. 1 root root 5231 Dec 10 21:06 mime.types
-rw-r--r--. 1 root root 5231 Dec 10 21:06 mime.types.default
-rw-r--r--. 1 root root 2656 Dec 10 21:06 nginx.conf
-rw-r--r--. 1 root root 2656 Dec 10 21:06 nginx.conf.default
-rw-r--r--. 1 root root 636 Dec 10 21:06 scgi_params
-rw-r--r--. 1 root root 636 Dec 10 21:06 scgi_params.default
-rw-r--r--. 1 root root 664 Dec 10 21:06 uwsgi_params
-rw-r--r--. 1 root root 664 Dec 10 21:06 uwsgi_params.default
-rw-r--r--. 1 root root 3610 Dec 10 21:06 win-utf
[root@localhost conf]# vi nginx.conf
#user nobody;
worker_processes 1;
#error_log logs/error.log;
#error_log logs/error.log notice;
#error_log logs/error.log info;
#pid logs/nginx.pid;
events {
worker_connections 1024;
}
http {
include mime.types;
default_type application/octet-stream;
#log_format main '$remote_addr - $remote_user [$time_local] "$request" '
# '$status $body_bytes_sent "$http_referer" '
# '"$http_user_agent" "$http_x_forwarded_for"';
#access_log logs/access.log main;
sendfile on;
#tcp_nopush on;
#keepalive_timeout 0;
keepalive_timeout 65;
#gzip on;
server {
listen 9999;
server_name localhost;
#charset koi8-r;
#access_log logs/host.access.log main;
location / {
root html;
index index.html index.htm;
}
location ~/group1/M00 {
root /data/fastdfs/fastdfs_storage_data/data;
ngx_fastdfs_module;
}
#error_page 404 /404.html;
# redirect server error pages to the static page /50x.html
#
error_page 500 502 503 504 /50x.html;
location = /50x.html {
root html;
}
# proxy the PHP scripts to Apache listening on 127.0.0.1:80
#
#location ~ \.php$ {
# proxy_pass http://127.0.0.1;
#}
# pass the PHP scripts to FastCGI server listening on 127.0.0.1:9000
#
#location ~ \.php$ {
# root html;
# fastcgi_pass 127.0.0.1:9000;
# fastcgi_index index.php;
# fastcgi_param SCRIPT_FILENAME /scripts$fastcgi_script_name;
# include fastcgi_params;
#}
# deny access to .htaccess files, if Apache's document root
# concurs with nginx's one
#
#location ~ /\.ht {
# deny all;
#}
}
# another virtual host using mix of IP-, name-, and port-based configuration
#
#server {
# listen 8000;
# listen somename:8080;
# server_name somename alias another.alias;
# location / {
# root html;
# index index.html index.htm;
# }
#}
# HTTPS server
#
#server {
# listen 443 ssl;
# server_name localhost;
# ssl_certificate cert.pem;
# ssl_certificate_key cert.key;
# ssl_session_cache shared:SSL:1m;
# ssl_session_timeout 5m;
# ssl_ciphers HIGH:!aNULL:!MD5;
# ssl_prefer_server_ciphers on;
# location / {
# root html;
# index index.html index.htm;
# }
#}
}
[root@localhost conf]#
[root@localhost conf]#
Then enter the extracted directory during FastDFS installation, and copy http.conf and mime.types to the / etc/fdfs Directory:
[root@localhost conf]# cd /home/fastdfs-master/conf/ [root@localhost conf]# ll total 88 -rw-r--r--. 1 root root 23981 Dec 8 10:17 anti-steal.jpg -rw-r--r--. 1 root root 1909 Dec 8 10:17 client.conf -rw-r--r--. 1 root root 965 Dec 8 10:17 http.conf -rw-r--r--. 1 root root 31172 Dec 8 10:17 mime.types -rw-r--r--. 1 root root 10246 Dec 8 10:17 storage.conf -rw-r--r--. 1 root root 620 Dec 8 10:17 storage_ids.conf -rw-r--r--. 1 root root 8128 Dec 8 10:17 tracker.conf [root@localhost conf]# [root@localhost conf]# cp http.conf /etc/fdfs/ [root@localhost conf]# cp mime.types /etc/fdfs/ [root@localhost conf]#
In addition, you need to install fastdfs nginx module in the src directory of the installation directory_ Fastdfs.conf is also copied to the / etc/fdfs Directory:
[root@localhost conf]# cp /home/fastdfs-nginx-module-master/src/mod_fastdfs.conf /etc/fdfs/ [root@localhost conf]#
For the mod just copied_ Modify the fastdfs.conf file:
[root@localhost conf]# vim /etc/fdfs/mod_fastdfs.conf ! base_path=/data/fastdfs/fastdfs_storage #Save log directory ! tracker_server=192.168.31.100:22122 #The IP address and port number of the tracker server ! storage_server_port=23000 #Port number of the storage server ! url_have_group_name = true #Is there a group name in the file url ! store_path0=/data/fastdfs/fastdfs_storage_data #Storage path group_count = 3 #Set the number of groups. In fact, only group1 is used this time At the end of the file, set group [group1] group_name=group1 storage_server_port=23000 store_path_count=1 store_path0=/data/fastdfs/fastdfs_storage_data store_path1=/data/fastdfs/fastdfs_storage_data # group settings for group #2 # # since v1.14 # # when support multi-group, uncomment following section as neccessary [group2] group_name=group2 storage_server_port=23000 store_path_count=1 store_path0=/data/fastdfs/fastdfs_storage_data [group3] group_name=group3 storage_server_port=23000 store_path_count=1 store_path0=/data/fastdfs/fastdfs_storage_data [root@localhost conf]#
Create a symbolic connection from M00 to the storage directory:
[root@localhost conf]# ln -s /data/fastdfs/fastdfs_storage_data/data/ /data/fastdfs/fastdfs_storage_data/data/M00 [root@localhost conf]#
Start nginx:
[root@localhost conf]# /usr/local/nginx/sbin/nginx ngx_http_fastdfs_set pid=31687 [root@localhost conf]#
Successfully started:
[root@localhost conf]# /usr/local/nginx/sbin/nginx ngx_http_fastdfs_set pid=31687 [root@localhost conf]#
Congratulations, nginx of storage has been configured successfully. Next, we will continue to configure nginx of tracker.
Configure tracker nginx
Unzip another nginx:
I built another nginx2 under my own work and decompressed another copy of the original nginx-1.12.0.tar.gz into it
[root@localhost home]# mkdir nginx2 [root@localhost home]# tar -zxvf nginx-1.17.6.tar.gz -C nginx2/ [root@localhost home]# cd nginx2/nginx-1.17.6/ [root@localhost nginx-1.17.6]# ./configure --prefix=/usr/local/nginx2 --add-module=/home/fastdfs-nginx-module-master/src [root@localhost nginx-1.17.6]# make && make install
Next, the same thing is to modify nginx.conf. The port number can be 80 instead of changing. You need to point the upstream to the nginx address of the tracker.
[root@localhost nginx-1.17.6]# vim /usr/local/nginx2/conf/nginx.conf
upstream fdfs_group1 {
server 127.0.0.1:9999;
}
server {
listen 80;
server_name localhost;
#charset koi8-r;
#access_log logs/host.access.log main;
location /group1/M00 {
proxy_pass http://fdfs_group1;
}
#location / {
# root html;
# index index.html index.htm;
#}
#error_page 404 /404.html;
# redirect server error pages to the static page /50x.html
#
error_page 500 502 503 504 /50x.html;
location = /50x.html {
root html;
}
}
Start nginx2:
[root@localhost nginx-1.17.6]# /usr/local/nginx2/sbin/nginx
Can be successfully accessed.
Firewall port settings
If you succeed, why talk about this. Because some students are still unable to access here, it is likely that the firewall does not open the corresponding port. I suggest you don't close the firewall, although it's a little troublesome.
View open ports:
[root@localhost nginx-1.13.8]# firewall-cmd --zone=public --list-ports 20880/tcp 80/tcp 2181/tcp 23000/tcp 22122/tcp 9999/tcp [root@localhost nginx-1.13.8]#
On my CentOS, these ports are open.
storage:20880
tracker:23000
These two ports should be opened. In the next article, fastdfs client Javas may cause failure to connect.
9999 and 80 ports are provided for nginx access.
Open port number command: – permanent means permanent. If it is not added, it will not take effect after restart
firewall-cmd --zone=public --add-port=23000/tcp --permanent #Account opening port number
CentOS7 firewall related commands:
systemctl enable firewalld.service #Turn on the firewall systemctl stop firewalld.service #Turn off the firewall (boot will still start) systemctl disable firewalld.service #Disable firewall (do not start after power on)
HTTP test
Now let's visit the files we uploaded:
This time has been able to visit successfully.