Overview of fastDFS
_FastDFS is an open source, lightweight, distributed file system, which manages files, including file storage, file synchronization, file access (file upload, file download), etc. It solves the problems of mass storage and load balancing.It is especially suitable for online services with files as carriers, such as photo album websites, video websites, etc.
_FastDFS is tailored for the Internet, taking full account of redundant backup, load balancing, linear expansion and other mechanisms, and focusing on high availability, high performance and other indicators, it is easy to set up a high-performance file server cluster using FastDFS to provide file upload, download and other services.
advantage
(1) Mass storage: master-slave distributed storage, easy to expand storage space, (2) fastDFS hash the contents of files to avoid duplicate files (3) fastDFS then integrates with Nginx to provide site efficiency
Composition structure
1. tracker: tracker mainly does scheduling work; 2. storage nodes: load balancing on access
Experimental environment
tracker server 192.168.13.128 (libfastcommon, fastd,) storage+Nginx server 192.168.13.129 (libfastcommon)
1, Install libfastcommon service on all node servers
[root@tracker ~]# yum -y install libevent \ ##Install the necessary components of the environment libevent-devel perl make gcc zlib zlib-devel \ pcre pcre-devel gcc-c++ openssl-devel [root@tracker ~]# wget https://github.com/happyfish100/libfastcommon/archive/V1.0.38.tar.gz ##Download the libfastcommon package from the website [root@tracker ~]# tar zxvf V1.0.38.tar.gz -C /opt/ ##decompression [root@tracker ~]# cd /opt/libfastcommon-1.0.38/ [root@tracker libfastcommon-1.0.38]# ./make.sh && ./make.sh install ##Compile Installation ##Establish soft connections for system identification [root@tracker libfastcommon-1.0.38]# ln -s /usr/lib64/libfastcommon.so /usr/local/lib/libfastcommon.so [root@tracker libfastcommon-1.0.38]# ln -s /usr/lib64/libfdfsclient.so /usr/local/lib/libfdfsclient.so [root@tracker libfastcommon-1.0.38]# ln -s /usr/lib64/libfdfsclient.so /usr/lib/libfdfsclient.so
2, Install fastDFS service on all node servers
[root@storage ~]# wget https://github.com/happyfish100/fastdfs/archive/V5.11.tar.gz ##Download Compressed Pack from Web Site [root@storage ~]# tar zxf V5.11.tar.gz -C /opt/ ##decompression [root@storage ~]# cd /opt/fastdfs-5.11/ [root@storage fastdfs-5.11]# ./make.sh && ./make.sh install ##Compile Installation [root@storage fastdfs-5.11]# cd /etc/fdfs/ ##Copy Profile Template [root@storage fdfs]# cp tracker.conf.sample tracker.conf [root@storage fdfs]# cp storage.conf.sample storage.conf [root@storage fdfs]# cp client.conf.sample client.conf
3, Configure the tracker server
[root@tracker fdfs]# mkdir -m 755 -p /opt/fastdfs ##Create data files, log files in directories [root@tracker fdfs]# vim /etc/fdfs/tracker.conf ##Modify the tracker configuration file port=22122 //Default port 22122 for tracker service is sufficient base_path=/opt/fastdfs //tracker stores the following paths of data and log, which must be created in advance http.server_port=8080 //Start the http service process on the tracker server without installing Ignore [root@tracker fdfs]# fdfs_trackerd /etc/fdfs/tracker.conf start ##Open Service [root@tracker fdfs]# netstat -natp | grep 22122 tcp 0 0 0.0.0.0:22122 0.0.0.0:* LISTEN 2678/fdfs_trackerd [root@tracker fdfs]# vim /etc/rc.local ##Set boot-up self-start ##Last Line Add fdfs_trackerd /etc/fdfs/tracker.conf start [root@tracker fdfs]# systemctl stop firewalld.service ##Close Firewall [root@tracker fdfs]# setenforce 0
4, Configure storage Server
[root@storage fdfs]# mkdir -m 755 -p /opt/fastdfs ##Create data and log file directories [root@storage fdfs]# vim /etc/fdfs/storage.conf ##Modify storage Profile group_name=group1 //Default group name, modified as appropriate port=23000 //storge default 23000, storage port number must be the same for the same group base_path=/opt/fastdfs //Root path of storage log file store_path_count=1 //Same number of paths as below, default is 1 store_path0=/opt/fastdfs //Provided storage path (stored with log files by default) tracker_server=192.168.13.128:22122 //Own tracker server IP (focus!!!) http.server_port=80 //The default port for http access files is 8888, and the listening ports configured in nginx are consistent [root@storage fdfs]# fdfs_storaged /etc/fdfs/storage.conf start ##Open Service [root@storage fdfs]# netstat -ntap | grep 23000 tcp 0 0 0.0.0.0:23000 0.0.0.0:* LISTEN 2687/fdfs_storaged [root@storage fdfs]# echo "fdfs_storaged /etc/fdfs/storage.conf start" >> /etc/rc.local ##Start-Up Self-Starting [root@storage fdfs]# systemctl stop firewalld.service ##Close Firewall [root@storage fdfs]# setenforce 0 [root@storage fdfs]# fdfs_monitor /etc/fdfs/storage.conf ##Check the success of tracker Association Storage 1: id = 192.168.13.129 ip_addr = 192.168.13.129 (storage) ACTIVE ...
5. Install Nginx on the storage server
##To reduce the number of virtual machine opens, install nginx on the storage side
[root@storage fdfs]# wget https://github.com/happyfish100/fastdfs-nginx-module/archive/V1.20.tar.gz
##Download Module Compression Pack
[root@storage fdfs]# tar zxf V1.20.tar.gz -C /opt/ ##decompression
[root@storage fdfs]# mount.cifs //192.168.100.3/LNMP-C7 /mnt/
Password for root@//192.168.100.3/LNMP-C7:
[root@storage fdfs]# cd /mnt/
[root@storage mnt]# tar zxvf nginx-1.12.2.tar.gz -C /opt/ ##decompression
[root@storage mnt]# cd /opt/nginx-1.12.2
[root@storage nginx-1.12.2]# ./configure \
--prefix=/usr/local/nginx \
--add-module=/opt/fastdfs-nginx-module-1.20/src/ ##fastdfs-nginx-module module
[root@storage nginx-1.12.2]# make && make install
##Possible Errors##
/usr/include/fastdfs/fdfs_define.h:15:27: fatal error: common_define.h: No such file or directory
##Solution##
//Modify the fastdfs-nginx-module-1.20/src/config file and reconfigure the compile installation
ngx_module_incs="/usr/include/fastdfs /usr/include/fastcommon/"
CORE_INCS="$CORE_INCS /usr/include/fastdfs /usr/include/fastcommon/"
[root@storage nginx-1.12.2]# cd ../fastdfs-nginx-module-1.20/src/
[root@storage src]# cp mod_fastdfs.conf /etc/fdfs/
[root@storage src]# cd /etc/fdfs/
[root@storage fdfs]# vim mod_fastdfs.conf ##Configure fastdfs-nginx-module module
#Check Configuration
base_path=/opt/fastdfs //Path to store data files, logs
tracker_server=192.168.13.128:22122 //Address of tracker end (key!!!)
url_have_group_name = true //Does url contain group name
storage_server_port=23000 //Requires the same configuration as storage
store_path_count=1 //Number of storage paths, need to match number of store_path s
store_path0=/opt/fastdfs //Location of file storage
[root@storage fdfs]# vim /usr/local/nginx/conf/nginx.conf ##Configure Nginx Profile
#Add blank lines, do not do this may result in errors
location ~/M00 {
root /opt/fastdfs/data;
ngx_fastdfs_module;
}
[root@storage fdfs]# cd /opt/fastdfs-5.11/conf/
[root@storage conf]# cp mime.types http.conf /etc/fdfs/
##Copy fastdfs to extract http.conf and mime.types from directory6, modify the client client client (used here on the storage server)
[root@storage ~]# vim /etc/fdfs/client.conf #Check the following configuration base_path=/opt/fastdfs ##tracker Server File Path tracker_server=192.168.13.128:22122 ##tracker server IP address and port number http.tracker_server_port=8080 ##The http port number of the tracker server, which must correspond to the settings of the tracker
7, Common commands for fastDFS
Upload file:
/usr/bin/fdfs_upload_file <config_file> <local_filename>
Download file:
/usr/bin/fdfs_download_file <config_file> <file_id> [local_filename]
Delete files:
/usr/bin/fdfs_delete_file <config_file> <file_id>