Implementation of File Server Based on MongoDB and Spring Boot

MongoDB is a product between relational database and non-relational database, which has the most abundant functions and resembles relational database. It aims to provide a scalable and high-performance data storage solution for WEB applications. It supports a very loose data structure, is similar to JSON BSON format, so it can store more complex data types.

This article will introduce the implementation of a file server MongoDB File Server by storing binary files through MongoDB.

Requirements for File Servers

This file server is dedicated to the storage of small files, such as blog pictures, ordinary documents and so on. Because MongoDB supports the storage of many data formats, it is not necessary for binary storage, so it can be used to store files easily. Because MongoDB's BSON document limits the size of data (each document does not exceed 16M), this file server is mainly for the storage of small files. For storage of large files (over 16M, for example), MongoDB has officially provided mature products. GridFS Reader friends can understand it by themselves.

This article will not introduce the concept and basic usage of MongoDB too much. Interested friends can refer to other documents by themselves, such as the author's works. Common Technologies and Case Analysis of Distributed Systems A book, MongoDB also has some ink.

Required environment

The development environment used in this example is as follows:

MongoDB 3.4.4
Spring Boot 1.5.3.RELEASE
Thymeleaf 3.0.3.RELEASE
Thymeleaf Layout Dialect 2.2.0
Embedded MongoDB 2.0.0
Gradle 3.5

Spring Boot is used to quickly build an independent Java project; Thymeleaf is used as a front-end page template to display data; Embedded MongoDB is an embedded MongoDB produced by Organization Flapdoodle OSS, which can test the MongoDB interface without starting the MongoDB server. He Maven's concept is a new generation of project automation building tools.

For more information on Spring Boot, you can refer to the author's open source book.< Spring Boot Tutorial " For more information on Thymeleaf, you can refer to the author's open source book.< Thymeleaf tutorial " For Gradle's content, you can refer to the author's open source book.< Gradle 3 User Guide>.

build.gradle

The project demonstrated in this paper is organized and constructed by Gradle. If you are not familiar with Gradle, you can turn the project into Maven project by yourself.

The build.gradle file is as follows:

// Priority of script execution in buildscript code block
buildscript {

    // ext is used to define dynamic properties
    ext {
        springBootVersion = '1.5.3.RELEASE'
    }

    // Customize versions of Thymeleaf and Thymeleaf Layout Dialect
    ext['thymeleaf.version'] = '3.0.3.RELEASE'
    ext['thymeleaf-layout-dialect.version'] = '2.2.0'
    // Customize Embedded MongoDB dependencies
    ext['embedded-mongo.version'] = '2.0.0'

    // Maven's central warehouse is used (you can also specify other warehouses)
    repositories {
        //mavenCentral()
        maven {
            url 'http://maven.aliyun.com/nexus/content/groups/public/'
        }
    }

    // Dependency
    dependencies {
        // The classpath declaration illustrates that ClassLoader can use these dependencies when executing the rest of the scripts
        classpath("org.springframework.boot:spring-boot-gradle-plugin:${springBootVersion}")
    }
}

// Using plug-ins
apply plugin: 'java'
apply plugin: 'eclipse'
apply plugin: 'org.springframework.boot'

// The packaging type is jar and the version is specified
version = '1.0.0'

// Specify the JDK version of the compiled. java file
sourceCompatibility = 1.8

// Maven's central warehouse is used by default. Change to a custom mirror Library
repositories {
    //mavenCentral()
    maven {
        url 'http://maven.aliyun.com/nexus/content/groups/public/'
    }
}

// Dependency
dependencies {
    // This dependency is necessary for compilation and distribution
    compile('org.springframework.boot:spring-boot-starter-web')

    // Adding Thymeleaf dependencies
    compile('org.springframework.boot:spring-boot-starter-thymeleaf')

    // Adding dependencies to Spring Data Mongodb
    compile 'org.springframework.boot:spring-boot-starter-data-mongodb'

    // Adding Embedded MongoDB dependencies for testing
    compile('de.flapdoodle.embed:de.flapdoodle.embed.mongo')

    // This dependency is necessary for compilation testing and includes compiled product dependencies and compilation-time dependencies by default.
    testCompile('org.springframework.boot:spring-boot-starter-test')
}

The annotations for the configuration items in the build.gradle file are exhaustive, so the meaning of the configuration items will not be repeated here.

Domain object

Document class File

Document classes are concepts similar to entities in JPA.

import org.springframework.data.mongodb.core.mapping.Document;

@Document
public class File {
    @Id  // Primary key
    private String id;
    private String name; // File name
    private String contentType; // file type
    private long size;
    private Date uploadDate;
    private String md5;
    private byte[] content; // Document content
    private String path; // File path

    ...
    // getter/setter 
    ...

    protected File() {
    }

    public File(String name, String contentType, long size,byte[] content) {
        this.name = name;
        this.contentType = contentType;
        this.size = size;
        this.uploadDate = new Date();
        this.content = content;
    }

    @Override
    public boolean equals(Object object) {
        if (this == object) {
            return true;
        }
        if (object == null || getClass() != object.getClass()) {
            return false;
        }
        File fileInfo = (File) object;
        return java.util.Objects.equals(size, fileInfo.size)
                && java.util.Objects.equals(name, fileInfo.name)
                && java.util.Objects.equals(contentType, fileInfo.contentType)
                && java.util.Objects.equals(uploadDate, fileInfo.uploadDate)
                && java.util.Objects.equals(md5, fileInfo.md5)
                && java.util.Objects.equals(id, fileInfo.id);
    }

    @Override
    public int hashCode() {
        return java.util.Objects.hash(name, contentType, size, uploadDate, md5, id);
    }

    @Override
    public String toString() {
        return "File{"
                + "name='" + name + '\''
                + ", contentType='" + contentType + '\''
                + ", size=" + size
                + ", uploadDate=" + uploadDate
                + ", md5='" + md5 + '\''
                + ", id='" + id + '\''
                + '}';
    }
}

Document class, mainly using the annotations in Spring Data MongoDB, is used to identify this is a document concept in NoSQL.

Repository FileRepository

Repositories are used to provide common data access interfaces for dealing with databases. The FileRepository interface inherits from org. spring framework. data. mongodb. repository. MongoRepository. It does not need to implement the functions of the interface itself.
Spring Data MongoDB automatically implements the methods in the interface.

import org.springframework.data.mongodb.repository.MongoRepository;
import com.waylau.spring.boot.fileserver.domain.File;

public interface FileRepository extends MongoRepository<File, String> {
}

Service Interface and Implementation Class

FileService interface defines the CURD operation for files, in which the query file interface is paginated to effectively improve query performance.

public interface FileService {
    /**
     * Save files
     * @param File
     * @return
     */
    File saveFile(File file);

    /**
     * Delete files
     * @param File
     * @return
     */
    void removeFile(String id);

    /**
     * Obtain files based on id
     * @param File
     * @return
     */
    File getFileById(String id);

    /**
     * Paging queries, in descending order by upload time
     * @param pageIndex
     * @param pageSize
     * @return
     */
    List<File> listFilesByPage(int pageIndex, int pageSize);
}

FileService Impl implements all interfaces in FileService.

@Service
public class FileServiceImpl implements FileService {

    @Autowired
    public FileRepository fileRepository;

    @Override
    public File saveFile(File file) {
        return fileRepository.save(file);
    }

    @Override
    public void removeFile(String id) {
        fileRepository.delete(id);
    }

    @Override
    public File getFileById(String id) {
        return fileRepository.findOne(id);
    }

    @Override
    public List<File> listFilesByPage(int pageIndex, int pageSize) {
        Page<File> page = null;
        List<File> list = null;

        Sort sort = new Sort(Direction.DESC,"uploadDate"); 
        Pageable pageable = new PageRequest(pageIndex, pageSize, sort);

        page = fileRepository.findAll(pageable);
        list = page.getContent();
        return list;
    }
}

Control Layer/API Resource Layer

FileController Controller acts as the provider of API to receive requests and responses from users. The definition of API conforms to RESTful style. Readers of REST-related knowledge can refer to the author's open source book, [REST Practice]. https://github.com/waylau/rest-in-action).

@CrossOrigin(origins = "*", maxAge = 3600)  // Allow access to all domain names
@Controller
public class FileController {

    @Autowired
    private FileService fileService;

    @Value("${server.address}")
    private String serverAddress;

    @Value("${server.port}")
    private String serverPort;

    @RequestMapping(value = "/")
    public String index(Model model) {
        // Show the latest 20 data
        model.addAttribute("files", fileService.listFilesByPage(0,20)); 
        return "index";
    }

    /**
     * Paging query file
     * @param pageIndex
     * @param pageSize
     * @return
     */
    @GetMapping("files/{pageIndex}/{pageSize}")
    @ResponseBody
    public List<File> listFilesByPage(@PathVariable int pageIndex, @PathVariable int pageSize){
        return fileService.listFilesByPage(pageIndex, pageSize);
    }

    /**
     * Getting File Slice Information
     * @param id
     * @return
     */
    @GetMapping("files/{id}")
    @ResponseBody
    public ResponseEntity<Object> serveFile(@PathVariable String id) {

        File file = fileService.getFileById(id);

        if (file != null) {
            return ResponseEntity
                    .ok()
                    .header(HttpHeaders.CONTENT_DISPOSITION, "attachment; fileName=\"" + file.getName() + "\"")
                    .header(HttpHeaders.CONTENT_TYPE, "application/octet-stream" )
                    .header(HttpHeaders.CONTENT_LENGTH, file.getSize()+"")
                    .header("Connection",  "close") 
                    .body( file.getContent());
        } else {
            return ResponseEntity.status(HttpStatus.NOT_FOUND).body("File was not fount");
        }

    }

    /**
     * Online Display Files
     * @param id
     * @return
     */
    @GetMapping("/view/{id}")
    @ResponseBody
    public ResponseEntity<Object> serveFileOnline(@PathVariable String id) {

        File file = fileService.getFileById(id);

        if (file != null) {
            return ResponseEntity
                    .ok()
                    .header(HttpHeaders.CONTENT_DISPOSITION, "fileName=\"" + file.getName() + "\"")
                    .header(HttpHeaders.CONTENT_TYPE, file.getContentType() )
                    .header(HttpHeaders.CONTENT_LENGTH, file.getSize()+"")
                    .header("Connection",  "close") 
                    .body( file.getContent());
        } else {
            return ResponseEntity.status(HttpStatus.NOT_FOUND).body("File was not fount");
        }

    }

    /**
     * upload
     * @param file
     * @param redirectAttributes
     * @return
     */
    @PostMapping("/")
    public String handleFileUpload(@RequestParam("file") MultipartFile file,
                                   RedirectAttributes redirectAttributes) {

        try {
            File f = new File(file.getOriginalFilename(),  file.getContentType(), file.getSize(), file.getBytes());
            f.setMd5( MD5Util.getMD5(file.getInputStream()) );
            fileService.saveFile(f);
        } catch (IOException | NoSuchAlgorithmException ex) {
            ex.printStackTrace();
            redirectAttributes.addFlashAttribute("message",
                    "Your " + file.getOriginalFilename() + " is wrong!");
            return "redirect:/";
        }

        redirectAttributes.addFlashAttribute("message",
                "You successfully uploaded " + file.getOriginalFilename() + "!");

        return "redirect:/";
    }

    /**
     * Upload interface
     * @param file
     * @return
     */
    @PostMapping("/upload")
    @ResponseBody
    public ResponseEntity<String> handleFileUpload(@RequestParam("file") MultipartFile file) {
        File returnFile = null;
        try {
            File f = new File(file.getOriginalFilename(),  file.getContentType(), file.getSize(),file.getBytes());
            f.setMd5( MD5Util.getMD5(file.getInputStream()) );
            returnFile = fileService.saveFile(f);
            String path = "//"+ serverAddress + ":" + serverPort + "/view/"+returnFile.getId();
            return ResponseEntity.status(HttpStatus.OK).body(path);

        } catch (IOException | NoSuchAlgorithmException ex) {
            ex.printStackTrace();
            return ResponseEntity.status(HttpStatus.INTERNAL_SERVER_ERROR).body(ex.getMessage());
        }

    }

    /**
     * Delete files
     * @param id
     * @return
     */
    @DeleteMapping("/{id}")
    @ResponseBody
    public ResponseEntity<String> deleteFile(@PathVariable String id) {

        try {
            fileService.removeFile(id);
            return ResponseEntity.status(HttpStatus.OK).body("DELETE Success!");
        } catch (Exception e) {
            return ResponseEntity.status(HttpStatus.INTERNAL_SERVER_ERROR).body(e.getMessage());
        }
    }
}

Where the @CrossOrigin(origins = "*", maxAge = 3600) annotation identifies that the API can be requested across domains. In order to enable this annotation, security configuration class support is still needed.

Security Configuration

To support cross-domain requests, we set up the security configuration class SecurityConfig:

@Configuration
@EnableWebMvc
public class SecurityConfig extends WebMvcConfigurerAdapter {

    @Override
    public void addCorsMappings(CorsRegistry registry) {
        registry.addMapping("/**").allowedOrigins("*") ; // Allow cross-domain requests
    }
}

Function

There are many ways to run Gradle's Java project. Running with Spring Boot Gradle Plugin plug-in is a relatively simple way, requiring only execution:

$ gradlew bootRun

For other modes of operation, please refer to the author's open source book.< Spring Boot Tutorial>

After the project runs successfully, it is accessed through browser http://localhost:8081 All right. Home page provides a demonstration interface for uploading, after uploading, you can see the details of uploaded files:

The relevant upload interface is exposed http://localhost:8081/ Among them,

GET/files/{pageIndex}/{pageSize}: Paging queries for uploaded files
GET/files/{id}: Download a file
GET/view/{id}: Preview a file online. For example, display pictures
POST/upload: Upload files
DELETE /{id}: Delete files

Source code

MongoDB File Server is an open source product. See the complete project source code. https://github.com/waylau/mongodb-file-server.

Reference

The original text is synchronized to: https://waylau.com/mogodb-file-server-with-spring-boot/
<https://docs.mongodb.com/manual/core/gridfs/
"Common Technologies and Case Analysis of Distributed Systems": https://github.com/waylau/distributed-systems-technologies-and-cases-analysis
Spring Book Course: https://github.com/waylau/spring-boot-tutorial
Thymeleaf Course: https://github.com/waylau/thymeleaf-tutorial
Gradle 3 User Guide: https://github.com/waylau/gradle-3-user-guide
REST Actual Warfare https://github.com/waylau/rest-in-action
https://github.com/waylau/mongodb-file-server

Keywords: MongoDB Spring Thymeleaf Gradle

Added by azul85 on Sun, 23 Jun 2019 01:26:02 +0300

Programming VIP