The video has been successfully uploaded to our server, and the ID of background music has been selected. Now we need to merge the video and background music, and we need to use a tool ffmpeg. Source: wx-spring boot and No.15 in https://github.com/limingios/wxProgram.git
ffmpeg
introduce
> Video and Audio Processing Tools, Cross-platform Video and Audio Processing Solutions, Home Page: http://ffmpeg.org
Application scenarios
Player: Sagittarius Player, Storm Video, Thunderbolt Player...
Conversion Tools: Format Factory, Clipping Tools...
Live broadcast, video coding, filter, watermarking, special effects...
download
According to your needs, I choose to download on windows platform.
test
C:UsersAdministrator>D: D:>cd D:Program Filesffmpeg D:Program Filesffmpeg>cd bin D:Program Filesffmpegbin>ffmpeg.exe -i shanzhu.mp4 shanzhu.avi ffmpeg version N-91949-g6304268e39 Copyright (c) 2000-2018 the FFmpeg developers built with gcc 8.2.1 (GCC) 20180813 configuration: --enable-gpl --enable-version3 --enable-sdl2 --enable-fontconfig --enable-gnutls --enable-iconv --enable-libass --enable-libbluray --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libtheora --enable-libtwolame --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libzimg --enable-lzma --enable-zlib --enable-gmp --enable-libvidstab --enable-libvorbis --enable-libvo-amrwbenc --enable-libmysofa --enable-libspeex --enable-libxvid --enable-libaom --enable-libmfx --enable-amf --enable-ffnvcodec --enable-cuvid --enable-d3d11va --enable-nvenc --enable-nvdec --enable-dxva2 --enable-avisynth libavutil 56. 19.101 / 56. 19.101 libavcodec 58. 30.100 / 58. 30.100 libavformat 58. 18.100 / 58. 18.100 libavdevice 58. 4.103 / 58. 4.103 libavfilter 7. 31.100 / 7. 31.100 libswscale 5. 2.100 / 5. 2.100 libswresample 3. 2.100 / 3. 2.100 libpostproc 55. 2.100 / 55. 2.100 Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'shanzhu.mp4': Metadata: major_brand : mp42 minor_version : 1 compatible_brands: mp41mp42isom creation_time : 2018-09-16T05:29:47.000000Z Duration: 00:00:09.34, start: 0.000000, bitrate: 1210 kb/s Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, bt709), 544x960, 1159 kb/s, 29.97 fps, 29.97 tbr, 600 tbn, 1200 tbc (default) Metadata: creation_time : 2018-09-16T05:29:47.000000Z handler_name : Core Media Video Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, mono, fltp, 46 kb/s (default) Metadata: creation_time : 2018-09-16T05:29:47.000000Z handler_name : Core Media Audio Stream mapping: Stream #0:0 -> #0:0 (h264 (native) -> mpeg4 (native)) Stream #0:1 -> #0:1 (aac (native) -> mp3 (libmp3lame)) Press [q] to stop, [?] for help Output #0, avi, to 'shanzhu.avi': Metadata: major_brand : mp42 minor_version : 1 compatible_brands: mp41mp42isom ISFT : Lavf58.18.100 Stream #0:0(und): Video: mpeg4 (FMP4 / 0x34504D46), yuv420p, 544x960, q=2-31, 200 kb/s, 29.97 fps, 29.97 tbn, 29.97 tbc (default) Metadata: creation_time : 2018-09-16T05:29:47.000000Z handler_name : Core Media Video encoder : Lavc58.30.100 mpeg4 Side data: cpb: bitrate max/min/avg: 0/0/200000 buffer size: 0 vbv_delay: -1 Stream #0:1(und): Audio: mp3 (libmp3lame) (U[0][0][0] / 0x0055), 44100 Hz, mono, fltp (default) Metadata: creation_time : 2018-09-16T05:29:47.000000Z handler_name : Core Media Audio encoder : Lavc58.30.100 libmp3lame frame= 280 fps=0.0 q=31.0 Lsize= 1096kB time=00:00:09.34 bitrate= 961.3kbits/s speed=16.7x video:999kB audio:72kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 2.338377%
java program calls cmd to convert video
Add FFMpegTest to spring boot-common
package com.idig8.utils;
import java.io.BufferedReader;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.util.ArrayList;
import java.util.List;
public class FFMpegTest {
private String ffmpegEXE;
public FFMpegTest(String ffmpegEXE) {
super();
this.ffmpegEXE = ffmpegEXE;
}
public void convertor(String videoInputPath, String videoOutputPath) throws Exception {
// ffmpeg -i input.mp4 -y output.avi
List<String> command = new ArrayList<>();
command.add(ffmpegEXE);
command.add("-i");
command.add(videoInputPath);
command.add("-y");
command.add(videoOutputPath);
for (String c : command) {
System.out.print(c + " ");
}
ProcessBuilder builder = new ProcessBuilder(command);
Process process = builder.start();
InputStream errorStream = process.getErrorStream();
InputStreamReader inputStreamReader = new InputStreamReader(errorStream);
BufferedReader br = new BufferedReader(inputStreamReader);
String line = "";
while ( (line = br.readLine()) != null ) {
}
if (br != null) {
br.close();
}
if (inputStreamReader != null) {
inputStreamReader.close();
}
if (errorStream != null) {
errorStream.close();
}
}
public static void main(String[] args) {
FFMpegTest ffmpeg = new FFMpegTest("D:\Program Files\ffmpeg\bin\ffmpeg.exe");
try {
ffmpeg.convertor("D:\Program Files\ffmpeg\bin\shanzhu.mp4", "D:\Program Files\ffmpeg\bin\shanzhu.avi");
} catch (Exception e) {
e.printStackTrace();
}
}
}Video and Audio Merge Command
> In this way, we can combine our mp4 with our mp3.
ffmpeg.exe-i shanzhu.avi-i music.mp3-t 10-y combines music and video.avi
D:Program Filesffmpegbin>ffmpeg.exe -i shanzhu.avi -i music.mp3 -t 10 -y Combine music and video.avi ffmpeg version N-91949-g6304268e39 Copyright (c) 2000-2018 the FFmpeg developers built with gcc 8.2.1 (GCC) 20180813 configuration: --enable-gpl --enable-version3 --enable-sdl2 --enable-fontconfig --enable-gnutls --enable-iconv --enable-libass --enable-libbluray --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libtheora --enable-libtwolame --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libzimg --enable-lzma --enable-zlib --enable-gmp --enable-libvidstab --enable-libvorbis --enable-libvo-amrwbenc --enable-libmysofa --enable-libspeex --enable-libxvid --enable-libaom --enable-libmfx --enable-amf --enable-ffnvcodec --enable-cuvid --enable-d3d11va --enable-nvenc --enable-nvdec --enable-dxva2 --enable-avisynth libavutil 56. 19.101 / 56. 19.101 libavcodec 58. 30.100 / 58. 30.100 libavformat 58. 18.100 / 58. 18.100 libavdevice 58. 4.103 / 58. 4.103 libavfilter 7. 31.100 / 7. 31.100 libswscale 5. 2.100 / 5. 2.100 libswresample 3. 2.100 / 3. 2.100 libpostproc 55. 2.100 / 55. 2.100 Input #0, avi, from 'shanzhu.avi': Metadata: encoder : Lavf58.18.100 Duration: 00:00:09.38, start: 0.000000, bitrate: 957 kb/s Stream #0:0: Video: mpeg4 (Simple Profile) (FMP4 / 0x34504D46), yuv420p, 544x960 [SAR 1:1 DAR 17:30], 876 kb/s, 29.97 fps, 29.97 tbr, 29.97 tbn, 30k tbc Stream #0:1: Audio: mp3 (U[0][0][0] / 0x0055), 44100 Hz, mono, fltp, 64 kb/s [mp3 @ 000001fcfa57d6c0] Estimating duration from bitrate, this may be inaccurate Input #1, mp3, from 'music.mp3': Duration: 00:00:33.47, start: 0.000000, bitrate: 320 kb/s Stream #1:0: Audio: mp3, 48000 Hz, stereo, fltp, 320 kb/s Stream mapping: Stream #0:0 -> #0:0 (mpeg4 (native) -> mpeg4 (native)) Stream #1:0 -> #0:1 (mp3 (mp3float) -> mp3 (libmp3lame)) Press [q] to stop, [?] for help Output #0, avi, to `Avi': Metadata: ISFT : Lavf58.18.100 Stream #0:0: Video: mpeg4 (FMP4 / 0x34504D46), yuv420p(progressive), 544x960 [SAR 1:1 DAR 17:30], q=2-31, 200 kb/s, 29.97 fps, 29.97 tbn, 29.97 tbc Metadata: encoder : Lavc58.30.100 mpeg4 Side data: cpb: bitrate max/min/avg: 0/0/200000 buffer size: 0 vbv_delay: -1 Stream #0:1: Audio: mp3 (libmp3lame) (U[0][0][0] / 0x0055), 48000 Hz, stereo, fltp Metadata: encoder : Lavc58.30.100 libmp3lame frame= 280 fps=274 q=31.0 Lsize= 1076kB time=00:00:10.00 bitrate= 880.4kbits/s speed=9.79x video:892kB audio:157kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 2.510280%
java program calls cmd to convert video to add audio
package com.idig8.utils;
import java.io.BufferedReader;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.util.ArrayList;
import java.util.List;
public class MergeVideoMp3 {
private String ffmpegEXE;
public MergeVideoMp3(String ffmpegEXE) {
super();
this.ffmpegEXE = ffmpegEXE;
}
public void convertor(String videoInputPath, String mp3InputPath,
double seconds, String videoOutputPath) throws Exception {
//ffmpeg.exe -i big pants underpants.mp4 -i bgm.mp3 -t 7 -y new video.mp4
List<String> command = new ArrayList<>();
command.add(ffmpegEXE);
command.add("-i");
command.add(videoInputPath);
command.add("-i");
command.add(mp3InputPath);
command.add("-t");
command.add(String.valueOf(seconds));
command.add("-y");
command.add(videoOutputPath);
// for (String c : command) {
// System.out.print(c + " ");
// }
ProcessBuilder builder = new ProcessBuilder(command);
Process process = builder.start();
InputStream errorStream = process.getErrorStream();
InputStreamReader inputStreamReader = new InputStreamReader(errorStream);
BufferedReader br = new BufferedReader(inputStreamReader);
String line = "";
while ( (line = br.readLine()) != null ) {
}
if (br != null) {
br.close();
}
if (inputStreamReader != null) {
inputStreamReader.close();
}
if (errorStream != null) {
errorStream.close();
}
}
public static void main(String[] args) {
MergeVideoMp3 ffmpeg = new MergeVideoMp3("D:\Program Files\ffmpeg\bin\ffmpeg.exe");
try {
ffmpeg.convertor("D:\Program Files\ffmpeg\bin\shanzhu.mp4", "D:\Program Files\ffmpeg\bin\music.mp3", 7.1, "D:\Program Files\ffmpeg\bin\Combine music and video.avi");
} catch (Exception e) {
e.printStackTrace();
}
}
}Development function
After uploading the video, ffmpge.exe is called through java to complete the combination of music and video, and the video information is saved in the database.
Development of Video service
VideoService
package com.idig8.service;
import com.idig8.pojo.Videos;
public interface VideoService {
/**
* Save Video Information
* @param Id
* @return
*/
public void saveVideo(Videos video);
}VideoServiceImpl
package com.idig8.service.Impl;
import java.util.List;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Service;
import org.springframework.transaction.annotation.Propagation;
import org.springframework.transaction.annotation.Transactional;
import com.idig8.mapper.BgmMapper;
import com.idig8.pojo.Bgm;
import com.idig8.service.BgmService;
@Service
public class BgmServiceImpl implements BgmService {
@Autowired
private BgmMapper bgmMapper;
@Transactional(propagation =Propagation.SUPPORTS)
@Override
public List<Bgm> queryBgmList(){
return bgmMapper.selectAll();
}
@Transactional(propagation =Propagation.SUPPORTS)
@Override
public Bgm queryBgmById(String Id){
return bgmMapper.selectByPrimaryKey(Id);
}
}BgmService
package com.idig8.service;
import java.util.List;
import com.idig8.pojo.Bgm;
public interface BgmService {
/**
* Get all Bgm lists
* @return
*/
public List<Bgm> queryBgmList();
/**
* Obtaining Bgm objects through bgmId
* @param Id
* @return
*/
public Bgm queryBgmById(String Id);
}BgmServiceImpl
package com.idig8.service.Impl;
import java.util.List;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Service;
import org.springframework.transaction.annotation.Propagation;
import org.springframework.transaction.annotation.Transactional;
import com.idig8.mapper.BgmMapper;
import com.idig8.pojo.Bgm;
import com.idig8.service.BgmService;
@Service
public class BgmServiceImpl implements BgmService {
@Autowired
private BgmMapper bgmMapper;
@Transactional(propagation =Propagation.SUPPORTS)
@Override
public List<Bgm> queryBgmList(){
return bgmMapper.selectAll();
}
@Transactional(propagation =Propagation.SUPPORTS)
@Override
public Bgm queryBgmById(String Id){
return bgmMapper.selectByPrimaryKey(Id);
}
}VideoController
package com.idig8.controller;
import java.io.File;
import java.util.Date;
import java.util.UUID;
import org.apache.commons.lang3.StringUtils;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;
import org.springframework.web.multipart.MultipartFile;
import com.idig8.pojo.Bgm;
import com.idig8.pojo.Videos;
import com.idig8.service.BgmService;
import com.idig8.service.VideoService;
import com.idig8.utils.JSONResult;
import com.idig8.utils.MergeVideoMp3;
import com.idig8.utils.enums.VideoStatusEnum;
import com.idig8.utils.file.FileUtil;
import io.swagger.annotations.Api;
import io.swagger.annotations.ApiImplicitParam;
import io.swagger.annotations.ApiImplicitParams;
import io.swagger.annotations.ApiOperation;
import io.swagger.annotations.ApiParam;
@RestController
@Api(value="Interface of Video-related Services", tags= {"Video related services controller"})
@RequestMapping("/video")
public class VideoController extends BasicController {
@Autowired
private BgmService bgmService;
@Autowired
private VideoService videosService;
@Value("${server.file.path}")
private String fileSpace;
@Value("${server.ffmpeg.path}")
private String ffmpegexe;
@ApiOperation(value="Upload video", notes="Interface for uploading video")
@ApiImplicitParams({
@ApiImplicitParam(name="userId", value="user id", required=true,
dataType="String", paramType="form"),
@ApiImplicitParam(name="bgmId", value="Background music id", required=false,
dataType="String", paramType="form"),
@ApiImplicitParam(name="videoSeconds", value="Background music playback length", required=true,
dataType="String", paramType="form"),
@ApiImplicitParam(name="videoWidth", value="Video width", required=true,
dataType="String", paramType="form"),
@ApiImplicitParam(name="videoHeight", value="Video height", required=true,
dataType="String", paramType="form"),
@ApiImplicitParam(name="desc", value="Video description", required=false,
dataType="String", paramType="form")
})
@PostMapping(value="/upload", headers="content-type=multipart/form-data")
public JSONResult upload(String userId,
String bgmId, double videoSeconds,
int videoWidth, int videoHeight,
String desc,
@ApiParam(value="Short video", required=true)
MultipartFile file) throws Exception {
if (StringUtils.isBlank(userId)) {
return JSONResult.errorMsg("user id Can not be empty...");
}
//Namespaces for file storage
String fileName = file.getOriginalFilename();
//Relative paths saved to the database
String path = "";
String videOutPath = "";
try {
path = FileUtil.uploadFile(file.getBytes(), fileSpace, fileName);
} catch (Exception e) {
e.getStackTrace();
return JSONResult.errorMsg(e.getMessage());
}
if(StringUtils.isNotBlank(bgmId)){
Bgm bgm = bgmService.queryBgmById(bgmId);
String mp3BgmPath = fileSpace + bgm.getPath();
MergeVideoMp3 mergeVideoMp3 = new MergeVideoMp3(ffmpegexe);
String videOutPathName = UUID.randomUUID().toString()+".mp4";
File targetFile = new File(fileSpace + userId);
if (!targetFile.exists()) {
targetFile.mkdirs();
}
videOutPath = "/"+userId+"/"+videOutPathName;
String videoInput = fileSpace +path;
mergeVideoMp3.convertor(videoInput, mp3BgmPath, videoSeconds, fileSpace +videOutPath);
}
Videos videos = new Videos();
videos.setAudioId(bgmId);
videos.setCreateTime(new Date());
videos.setVideoDesc(desc);
videos.setId(UUID.randomUUID().toString());
videos.setUserId(userId);
videos.setVideoHeight(videoHeight);
videos.setVideoWidth(videoWidth);
videos.setVideoPath(videOutPath);
videos.setStatus(VideoStatusEnum.SUCCESS.value);
videosService.saveVideo(videos);
return JSONResult.ok(path);
}
}
PS: Video upload has been completed, and the relevant information is saved in the database.