Introduction: Hello, I'm brother Feng, 🌟 IT migrant workers on the front-line Internet 📝 Senior interviewer 🌹 Founder of Java flea classroom. He has many years of front-line R & D experience and has worked in iFLYTEK, meituan, Ping An and other companies. There is a sideline team formed by its own small partners in Shanghai. At present, IT focuses on Java technology sharing, spring / autumn / social recruitment / job hopping, one-to-one learning assistance and project development in its spare time.
🎉🎉 Scan the QR code on the left, join the group chat, learn together and make progress together!
🌟 Welcome to pay attention 🌟 Collection 🌹 Leaving a message. 🌹
🍊🍊: Welfare at the end of the article
catalogue
5. Start minio with custom script
preface
1. Someone will ask, why not use FastDFS? As we all know, the native installation of FastDFS is very complex. People with installation experience generally understand that although you can use other people's docker s to install directly, there may be many inexplicable problems in the real use process;
2. Others will ask, why not use oss or other existing cloud products? The reason is very simple. You can't guarantee that the projects owned by your company will go to the cloud. According to my personal understanding, most companies either rely on Party A to use the intranet server or build it internally. For example, our company relies on the hospital's own server. All deployments are security first, and can only build their own internal file server;
3.Minio is developed in GO language, with good performance and simple installation. It can store massive pictures, audio, video and other files in a distributed manner, and has its own management background interface, which is very friendly.
4. I have a habit of observing what technologies will be added and removed by training institutions with large traffic every year. Although Minio has been out for some time, lecturers from well-known institutions have begun to introduce Minio in the past two years, which means that the acceptance of this product is increasing. As the personnel trained by training institutions spread to various IT companies, the acceptance will only be higher and higher.
Therefore, I don't know when to stay at this time~
minio official website address: MinIO | The MinIO Quickstart Guide
minio Chinese website address: MinIO Quickstart Guide| Minio Chinese document
Special note: most of the content can be viewed directly from the Chinese website, but when downloading minio, it is best to look at the official website, because the version of minio is updated very quickly, and the directory of resource files is often changed. At the same time, the address of the Chinese website may become invalid, resulting in download failure.
Build Minio
It is divided into download, installation, startup, access, custom startup script and setting permanent access links.
1. Download minio
1). Manual Download: MinIO | The MinIO Quickstart Guide
find this location and download the version you need. I use Linux here, so download the first one.
2). Remote pull
create your own minio directory
remote pull: wget http://dl.minio.org.cn/server/minio/release/darwin-amd64/minio
2. Install minio
1) assign permission to the minio binary file, otherwise it cannot be executed: chmod +x minio
2). Execute in the directory where the binary file is located/ minio, you can see the version number at the bottom after success. I install the latest version here.
3. Start minio
1) create a new data directory in the minio installation directory to store minio data: mkdir data;
2) start Minio in the background process:/ minio server /data/minio/data > /data/minio/minio. log 2>&1 &
View background operation log: Tail - f Mini log
Special note: it can be seen from the log here that the new version of minio is different from the old version. The address behind the API is port 9000, and the port of console, that is, the port of console address, is 33587. In addition, the last WARNING prompts that the console port is generated dynamically. Please use the command to select a static port, which means that if you restart, Then the port of the console will change again. You need to set a fixed static port yourself. The specific setting method can be set according to the prompted command.
The command is as follows: (note that when restarting, execute {kill -9 [process number] to kill the minio started by the previous background process)
# The specified background port is 9999 ./minio server --console-address 0.0.0.0:9999 /data/minio/data > /data/minio/minio.log 2>&1 &
4. Visit minio
after setting the fixed static port, the access address prompted by the log is http://127.0.0.1:9999 Here, we can replace it with the ip address of our own server. I use Tencent cloud server here.
access address: http://42.193.14.144:9999
the effect is as follows, which is different from the interface of the old version:
default account password: minioadmin minioadmin
5. Start minio with custom script
1). Create a new shell script and put in the commands that need to be set during startup. The command of setting account password is added here. The previous default account password minioadmin is no longer used.
New shell script: VIM Mini start sh
# Set account and password export MINIO_ACCESS_KEY=rootexport MINIO_SECRET_KEY=123456 # Background process start minio./minio server --console-address 0.0.0.0:9999 /data/minio/data > /data/minio/minio.log 2>&1 &
2). Give the script execution permission: Chmod + X Minio start sh
3). Execute script to start Minio:/ minio-start.sh
the final effect is the same as above!
6. Use minio
After entering the background, you can simply use minio to upload files, preview and share URL s to try the beauty brought by minio.
For many configurations, it's good to use the default one. If you don't understand more, you'll soon understand it. The only thing to understand is the Bucket concept, because it is often used when calling minio's API. In a simple way, it can be understood as the basket for storing eggs (the directory for storing files). after entering the background, you can simply use minio to upload files, preview and share URL s to try the beauty brought by minio.
For many configurations, it's good to use the default one. If you don't understand more, you'll soon understand it. The only thing to understand is the Bucket concept, because it is often used when calling minio's API. In a simple way, it can be understood as the basket for storing eggs (the directory for storing files).
PS: students who are just beginning to use it may be used to clicking the share button among the buttons on the right side of the file, copy the file link generated in the background, and then paste it into the browser to open it. Basically, they will encounter the situation that they can't open it, because when you look at the link carefully, you will find that the ip port of the link address is wrong, which is a misunderstanding, We usually use minio to execute commands through the mc client for some configuration, so as to achieve the effect of permanently accessing files and directly downloading files.
7. Set permanent access links
1) install mc client
you can refer to the official website and write it in detail: MinIO | The complete guide to the MinIO client
you can also refer to the Chinese website: MinIO Client complete guide | Minio Chinese document
When you open the document and read it for a while, you will find that it is well written, but you can't understand it. It doesn't matter. Many people who have stepped on the pit have cleared the obstacles.
Install MC client:
wget https://dl.min.io/client/mc/release/linux-amd64/mc
This confirms what I said earlier. It's best to see the official website for the installation of minio related files. The Chinese website address here can't be downloaded, so it can't be installed successfully.
Official website mc installation address:
Chinese network mc installation address: (this has expired when I use it)
similarly, give permission to the mc execution file, otherwise the error of insufficient permission will be prompted.
chmod +x mc
set permanent access links. It's not clear on the official website and Chinese website here. Personally, I think this is to set an accessible prefix address, so that you can directly access the pictures after opening the bucket permission. It's convenient to understand that you can imagine acting as an agent for nginx.
Set the configuration name to minio and the access prefix to http://42.193.14.144 , this is my Tencent cloud server address mentioned earlier. The port is set to 9000. Of course, it can also be set to something else. I set it to 9000 here because the rules of Tencent cloud security group already exist at 9000 port. I don't need to add the rules again. The root and 123456 here are the account and password set by the user-defined startup script. Just change them to your own. Nothing else needs to be changed.
./mc config host add minio http://42.193.14.144:9000 root 123456 --api S3v4
Special note: remember to set the port here. If the local virtual machine is used, either close the firewall or open the port you set; If you use a cloud server like me, no matter whether the firewall is turned on or not, you should add rules in the background management of the cloud server to open this port, otherwise you still can't open the file.
Set the permission that files in a bucket (i.e. file directory) can be downloaded directly:/ mc policy set download minio/hospitalimages
The hospitalimages here is the bucket I built to store Internet hospital files. Remember to add the previous mini, which is the configuration name set in the previous command.
After executing the command, the files under the bucket can be accessed directly.
After the command of setting permanent access link and download permission is executed, the final effect is as follows:
Can pass http: / / server ip : port / bucket name / file name directly accessed!
SpringBoot integrates Minio
Special note: there are great differences in the dependent use process of different versions introduced by minio. For example, there are great differences between 7 and 8. I have also stepped on many pits and searched many materials. Although version 7 is used, the current version is relatively new, so version 8 is used, and there are many pits in 8. Later, I finally found a usable scheme from Fengjian yingyue teacher on a website, It has also been used in the company's projects and is shared directly here.
1. Introduce dependency
<!-- MinIO -->
<dependency>
<groupId>io.minio</groupId>
<artifactId>minio</artifactId>
<version>8.2.1</version>
</dependency>2. MinioUtils tool class
import io.minio.*;
import io.minio.http.Method;
import io.minio.messages.Bucket;
import io.minio.messages.DeleteObject;
import io.minio.messages.Item;
import lombok.extern.slf4j.Slf4j;
import org.springframework.web.multipart.MultipartFile;
import java.io.ByteArrayInputStream;
import java.io.InputStream;
import java.io.UnsupportedEncodingException;
import java.net.URLDecoder;
import java.util.ArrayList;
import java.util.LinkedList;
import java.util.List;
import java.util.Optional;
/**
* MinIO Tool class
*
* @author guoj
* @date 2021/12/14 19:30
*/
@Slf4j
public class MinIOUtils {
private static MinioClient minioClient;
private static String endpoint;
private static String bucketName;
private static String accessKey;
private static String secretKey;
private static Integer imgSize;
private static Integer fileSize;
private static final String SEPARATOR = "/";
public MinIOUtils() {
}
public MinIOUtils(String endpoint, String bucketName, String accessKey, String secretKey, Integer imgSize, Integer fileSize) {
MinIOUtils.endpoint = endpoint;
MinIOUtils.bucketName = bucketName;
MinIOUtils.accessKey = accessKey;
MinIOUtils.secretKey = secretKey;
MinIOUtils.imgSize = imgSize;
MinIOUtils.fileSize = fileSize;
createMinioClient();
}
/**
* Create a Java based MinioClient
*/
public void createMinioClient() {
try {
if (null == minioClient) {
log.info("Start creating MinioClient...");
minioClient = MinioClient
.builder()
.endpoint(endpoint)
.credentials(accessKey, secretKey)
.build();
createBucket(bucketName);
log.info("Creation complete MinioClient...");
}
} catch (Exception e) {
log.error("[Minio Tool class]>>>> MinIO Server exception:", e);
}
}
/**
* Get the prefix path of the uploaded file
* @return
*/
public static String getBasisUrl() {
return endpoint + SEPARATOR + bucketName + SEPARATOR;
}
/****************************** Operate Bucket Start ******************************/
/**
* Initialize the Bucket when starting the SpringBoot container
* If there is no Bucket, create it
* @throws Exception
*/
private static void createBucket(String bucketName) throws Exception {
if (!bucketExists(bucketName)) {
minioClient.makeBucket(MakeBucketArgs.builder().bucket(bucketName).build());
}
}
/**
* Judge whether the Bucket exists. true: exists, false: does not exist
* @return
* @throws Exception
*/
public static boolean bucketExists(String bucketName) throws Exception {
return minioClient.bucketExists(BucketExistsArgs.builder().bucket(bucketName).build());
}
/**
* Strategies for obtaining buckets
* @param bucketName
* @return
* @throws Exception
*/
public static String getBucketPolicy(String bucketName) throws Exception {
return minioClient
.getBucketPolicy(
GetBucketPolicyArgs
.builder()
.bucket(bucketName)
.build()
);
}
/**
* Get a list of all buckets
* @return
* @throws Exception
*/
public static List<Bucket> getAllBuckets() throws Exception {
return minioClient.listBuckets();
}
/**
* Get relevant information according to bucket name
* @param bucketName
* @return
* @throws Exception
*/
public static Optional<Bucket> getBucket(String bucketName) throws Exception {
return getAllBuckets().stream().filter(b -> b.name().equals(bucketName)).findFirst();
}
/**
* Delete a Bucket according to the Bucket name. true: the deletion is successful; false: deletion failed. The file or no longer exists
* @param bucketName
* @throws Exception
*/
public static void removeBucket(String bucketName) throws Exception {
minioClient.removeBucket(RemoveBucketArgs.builder().bucket(bucketName).build());
}
/****************************** Operate Bucket End ******************************/
/****************************** Operate Files Start ******************************/
/**
* Determine whether the file exists
* @param bucketName Storage bucket
* @param objectName file name
* @return
*/
public static boolean isObjectExist(String bucketName, String objectName) {
boolean exist = true;
try {
minioClient.statObject(StatObjectArgs.builder().bucket(bucketName).object(objectName).build());
} catch (Exception e) {
log.error("[Minio Tool class]>>>> Determine whether the file exists, Exception:", e);
exist = false;
}
return exist;
}
/**
* Determine whether the folder exists
* @param bucketName Storage bucket
* @param objectName Folder name
* @return
*/
public static boolean isFolderExist(String bucketName, String objectName) {
boolean exist = false;
try {
Iterable<Result<Item>> results = minioClient.listObjects(
ListObjectsArgs.builder().bucket(bucketName).prefix(objectName).recursive(false).build());
for (Result<Item> result : results) {
Item item = result.get();
if (item.isDir() && objectName.equals(item.objectName())) {
exist = true;
}
}
} catch (Exception e) {
log.error("[Minio Tool class]>>>> Judge whether the folder exists, exception:", e);
exist = false;
}
return exist;
}
/**
* Query files according to file prefix
* @param bucketName Storage bucket
* @param prefix prefix
* @param recursive Whether to use recursive query
* @return MinioItem list
* @throws Exception
*/
public static List<Item> getAllObjectsByPrefix(String bucketName,
String prefix,
boolean recursive) throws Exception {
List<Item> list = new ArrayList<>();
Iterable<Result<Item>> objectsIterator = minioClient.listObjects(
ListObjectsArgs.builder().bucket(bucketName).prefix(prefix).recursive(recursive).build());
if (objectsIterator != null) {
for (Result<Item> o : objectsIterator) {
Item item = o.get();
list.add(item);
}
}
return list;
}
/**
* Get file stream
* @param bucketName Storage bucket
* @param objectName file name
* @return Binary stream
*/
public static InputStream getObject(String bucketName, String objectName) throws Exception {
return minioClient.getObject(GetObjectArgs.builder().bucket(bucketName).object(objectName).build());
}
/**
* Breakpoint Download
* @param bucketName Storage bucket
* @param objectName File name
* @param offset Start byte position
* @param length Length to read
* @return Binary stream
*/
public InputStream getObject(String bucketName, String objectName, long offset, long length)throws Exception {
return minioClient.getObject(
GetObjectArgs.builder()
.bucket(bucketName)
.object(objectName)
.offset(offset)
.length(length)
.build());
}
/**
* Get the file list under the path
* @param bucketName Storage bucket
* @param prefix File name
* @param recursive Recursive search or not, false: simulate folder structure search
* @return Binary stream
*/
public static Iterable<Result<Item>> listObjects(String bucketName, String prefix,
boolean recursive) {
return minioClient.listObjects(
ListObjectsArgs.builder()
.bucket(bucketName)
.prefix(prefix)
.recursive(recursive)
.build());
}
/**
* Upload files using MultipartFile
* @param bucketName Storage bucket
* @param file file name
* @param objectName Object name
* @param contentType type
* @return
* @throws Exception
*/
public static ObjectWriteResponse uploadFile(String bucketName, MultipartFile file,
String objectName, String contentType) throws Exception {
InputStream inputStream = file.getInputStream();
return minioClient.putObject(
PutObjectArgs.builder()
.bucket(bucketName)
.object(objectName)
.contentType(contentType)
.stream(inputStream, inputStream.available(), -1)
.build());
}
/**
* Upload local files
* @param bucketName Storage bucket
* @param objectName Object name
* @param fileName Local file path
*/
public static ObjectWriteResponse uploadFile(String bucketName, String objectName,
String fileName) throws Exception {
return minioClient.uploadObject(
UploadObjectArgs.builder()
.bucket(bucketName)
.object(objectName)
.filename(fileName)
.build());
}
/**
* Upload files via streaming
*
* @param bucketName Storage bucket
* @param objectName File object
* @param inputStream File stream
*/
public static ObjectWriteResponse uploadFile(String bucketName, String objectName, InputStream inputStream) throws Exception {
return minioClient.putObject(
PutObjectArgs.builder()
.bucket(bucketName)
.object(objectName)
.stream(inputStream, inputStream.available(), -1)
.build());
}
/**
* Create a folder or directory
* @param bucketName Storage bucket
* @param objectName Directory path
*/
public static ObjectWriteResponse createDir(String bucketName, String objectName) throws Exception {
return minioClient.putObject(
PutObjectArgs.builder()
.bucket(bucketName)
.object(objectName)
.stream(new ByteArrayInputStream(new byte[]{}), 0, -1)
.build());
}
/**
* Get the file information. If an exception is thrown, it indicates that the file does not exist
*
* @param bucketName Storage bucket
* @param objectName File name
*/
public static String getFileStatusInfo(String bucketName, String objectName) throws Exception {
return minioClient.statObject(
StatObjectArgs.builder()
.bucket(bucketName)
.object(objectName)
.build()).toString();
}
/**
* Copy file
*
* @param bucketName Storage bucket
* @param objectName file name
* @param srcBucketName Target bucket
* @param srcObjectName Destination file name
*/
public static ObjectWriteResponse copyFile(String bucketName, String objectName,
String srcBucketName, String srcObjectName) throws Exception {
return minioClient.copyObject(
CopyObjectArgs.builder()
.source(CopySource.builder().bucket(bucketName).object(objectName).build())
.bucket(srcBucketName)
.object(srcObjectName)
.build());
}
/**
* Delete file
* @param bucketName Storage bucket
* @param objectName File name
*/
public static void removeFile(String bucketName, String objectName) throws Exception {
minioClient.removeObject(
RemoveObjectArgs.builder()
.bucket(bucketName)
.object(objectName)
.build());
}
/**
* Batch delete files
* @param bucketName Storage bucket
* @param keys List of files to be deleted
* @return
*/
public static void removeFiles(String bucketName, List<String> keys) {
List<DeleteObject> objects = new LinkedList<>();
keys.forEach(s -> {
objects.add(new DeleteObject(s));
try {
removeFile(bucketName, s);
} catch (Exception e) {
log.error("[Minio Tool class]>>>> Delete files in batch, exception:", e);
}
});
}
/**
* Get file external chain
* @param bucketName Storage bucket
* @param objectName file name
* @param expires Expiration time < = 7 seconds (effective time of external chain (unit: seconds))
* @return url
* @throws Exception
*/
public static String getPresignedObjectUrl(String bucketName, String objectName, Integer expires) throws Exception {
GetPresignedObjectUrlArgs args = GetPresignedObjectUrlArgs.builder().expiry(expires).bucket(bucketName).object(objectName).build();
return minioClient.getPresignedObjectUrl(args);
}
/**
* Obtain external chain of documents
* @param bucketName
* @param objectName
* @return url
* @throws Exception
*/
public static String getPresignedObjectUrl(String bucketName, String objectName) throws Exception {
GetPresignedObjectUrlArgs args = GetPresignedObjectUrlArgs.builder()
.bucket(bucketName)
.object(objectName)
.method(Method.GET).build();
return minioClient.getPresignedObjectUrl(args);
}
/**
* Convert urlcoder code to UTF8
* @param str
* @return
* @throws UnsupportedEncodingException
*/
public static String getUtf8ByURLDecoder(String str) throws UnsupportedEncodingException {
String url = str.replaceAll("%(?![0-9a-fA-F]{2})", "%25");
return URLDecoder.decode(url, "UTF-8");
}
/****************************** Operate Files End ******************************/
}summary
This actually completes the integration, isn't it So Easy? Kaka, go through minioutils where you need it The XXX () method can be called. For example, minioutils is what I use in the company's project Getpreseignedobjecturl () is a method to obtain the external chain of files, because most of the time you do not need to modify or delete the file itself. Normally, you only need to upload and query files. Many product teachers will also avoid this risk in design. In addition, the parameters such as endpoint, bucketName, accessKey and ecretKey passed in the tool class can be obtained in the minio background. If not, you can set them yourself.
🌹 🌹 Thank you for reading. Since you have chosen this road, come on and learn together! If you need learning resources, practical interview materials and project resources. Pay attention to the official account: IT Maple fighter. 🌟 Receive corresponding data benefits according to keywords 🌟! Consultation and solving problems. Official account is for private brother Feng.
🍊 Reply: java full set of learning resources
🍊 Reply: interview materials
🍊 Resume of Fengge
🍊 Reply: programmer confesses artifact
(say goodbye to the programmer's single dog from now on!)
🍊 Reply: programmer part-time website
🍊 Reply: Fengge 666
(obtain 66 sets of project actual combat materials and interview videos of large factories)