Valentine's Day Special: do what you want. Let love beans talk to you and celebrate Versailles Valentine's day~

I preface

Valentine's day, also known as St. Valentine's day or St. Valentine's day, is one of the traditional festivals in western countries and originated from Christianity. Nowadays, it has become a famous romantic festival all over the world, but people in different countries express their love in different ways.
Valentine's Day is a festival about love, romance, flowers, chocolates and greeting cards. Men and women send gifts to each other on this day to express love or friendship. Dinner dates on Valentine's Day usually represent the key to the development of relationships.
Valentine's day has become a favorite festival for young people in Europe and the United States, and it has become popular in other countries. China regards the traditional festival Tanabata as Valentine's day. Because it can express the common human feelings, all countries have explored their own "Valentine's Day".

On Valentine's day, ADU actually said "love words" to me!? ̊ ଳ ̊

Valentine's Day is coming. While others are still thinking about how to prepare Valentine's Day gifts, I have received the video of love words from my favorite love bean to me ~ whether it's a goddess or a male god, you can have everything you like!

Not only will love beans fulfill your dreams, but you will also enjoy love in real life 💏 Sweet!!

Are you still shy: grab your ears and disturb your cheeks to surprise your partner ⭐？ Rack one's brains on how to directly attack the person in secret love ❤️ Your confession?

Let's take a look at how to use our skills to impress the goddess and male god. Maybe we will flirt and have an object in the end. If you have an object, it's also OK to express love with the object.

II Project introduction

This project uses paddlepaddle 2.. X framework, based on the video lip synchronization model Wav2lip implemented by paddegan, which realizes the voice synchronization of love bean mouth and advertising lines. For example:

Wav2lip model can also convert your own picture or video to lip shape. The audio is used to drive lip shape synthesis, and the characters in the picture / video can synthesize lip shape according to this audio. Output the video matching with the target voice. The key to the accurate synchronization of lip shape and voice in wav2lip is that it uses a lip shape discriminator to force the generator to produce accurate and realistic lip motion. From then on, touching love words are no longer shy to say, and you can also become a charming emotion master.
At the same time, Wav2Lip model uses multiple continuous frames in the discriminator to improve the visual quality through the loss of visual quality, which solves the problem of time correlation.
In addition, this project supports you to upload your prepared love bean / your own face material video and advertising line audio, and synthesize any desired love words / love song advertising video!!! Just change the face parameter and audio parameter in the following command into your own face material video and advertising line audio path respectively, and then run the following command to generate Aidou advertising video synchronized with the line audio. After the program runs, the Adu advertising video file with the file name specified by the outfile parameter will be generated in the current folder, which is the Adu advertising video file synchronized with the line audio.

This project provides the Aidou face video and advertising lines audio files used in the demo display. The specific parameters are as follows:

Face: the original love bean face video. The lip shape of love bean in the video will be synthesized according to the line audio.
Audio: audio of advertising lines driving lip synthesis.
outfile: Specifies the save path and file name of the generated video file

III code implementation

The code is written based on BML CodeLab of flying propeller

# Download PaddlePaddle installation package
%cd /home/aistudio/work
# Clone paddegan code from github (if the download speed is too slow, use gitee source)
!git clone https://gitee.com/PaddlePaddle/PaddleGAN

# Install the required installation package
!mkdir sndfile
%cd sndfile
!wget http://www.mega-nerd.com/libsndfile/files/libsndfile-1.0.28.tar.gz
!tar xzvf libsndfile-1.0.28.tar.gz

%cd libsndfile-1.0.28
!./configure --prefix=/home/aistudio/build_libs CFLAGS=-fPIC --enable-shared 
!make
!make install

%cd /home/aistudio/work/PaddleGAN
!pip install -r requirements.txt
%cd applications/

!export PYTHONPATH=$PYTHONPATH:/home/aistudio/work/PaddleGAN && python tools/wav2lip.py --face /home/aistudio/work/jieyi13s.mp4 --audio /home/aistudio/work/gaobai12s.m4a --outfile pp_gaobai_jieyi.mp4

IV Project results

If you want to save the video, you can directly go to the above path work / paddegan / applications / PP_ gaobai_ jieyi. Find the file under MP4 and download it locally.

# display the output video
import cv2
import imageio
import numpy as np
import matplotlib.pyplot as plt
import matplotlib.animation as animation
from IPython.display import HTML
import warnings

# video display function
def display(driving, fps, size=(8, 6)):
    fig = plt.figure(figsize=size)
    ims = []
    for i in range(len(driving)):
        cols = []
        cols.append(driving[i])
        im = plt.imshow(np.concatenate(cols, axis=1), animated=True)
        plt.axis('off')
        ims.append([im])
    video = animation.ArtistAnimation(fig, ims, interval=1000.0/fps, repeat_delay=1000)
    plt.close()
    return video
    
# Display the output video
# If the video is relatively long, the running time will be slightly longer. It is recommended to download the video to the local preview, and the video saving path is' / home / aistudio / work / paddegan / applications'
video_path = 'pp_gaobai_jieyi.mp4'
video_frames = imageio.mimread(video_path, memtest=False)
# Get the original resolution of the video
cap = cv2.VideoCapture(video_path)
fps = cap.get(cv2.CAP_PROP_FPS)
HTML(display(video_frames, fps).to_html5_video())

V summary

Paddegan can not only realize lip synchronization, but also realize many other functions, such as style migration. Welcome to try.
You can try it wav2lipHD Model, which is based on wav2lip and adds high resolution.
Project based Flying oar_ PaddleGAN Published items Idol speaks to me in love – paddegan takes you to Versailles Valentine's Day , you can refer to and extend the application on this basis.
Come to AI Studio to have fun with each other. Wait for you. Come to have fun with each other ~ please click - > non caviar
Welcome to collect comments for three times. Friends who are interested can also pay attention to each other~

Keywords: Python AI paddlepaddle

Added by John_wilson on Sat, 12 Feb 2022 04:44:17 +0200

Programming VIP