Artificial intelligence crosstalk generation based on PaddleHUB

๐Ÿงก Fast dialog generation based on PaddleHUB -- Taking crosstalk support generator as an example

Link from AI Studio project

stay Generate military training impressions using PaddleHub In the project, the generation of style text is realized based on PaddleHub, which is based on paddle2 Style text generation in version 0 +. (error again due to version upgrade)

๐Ÿ“– 0. Project background

project Valentine's Day reverse operation: a love word, a hate, what? Spring Festival couplets AI is played into gossip? This paper introduces the case of natural language processing generative task, but lacks the detailed introduction and demonstration of training using their own data. In addition, due to the version upgrade, similar projects and basic failure, the latest PaddleHUB sample project is necessary.

This project completes the task of text dialogue generation based on paddlehub, and introduces how to use your own data set to complete the task of matching text generation. It can help you quickly use paddlehub to realize your ideas.

The effects of the project are as follows:

Funny input: eat a candy cake and stick your teeth

reference material:


๐Ÿ“– 1.PaddleHub introduction

Through API, we can get the pre training model of PaddlePaddle ecosystem conveniently, and complete the management and one key prediction of the model. With the use of fine tune API, the migration learning can be quickly completed based on the large-scale pre training model, so that the pre training model can better serve the application of users' specific scenes.

  • Based on excellent community ecology and continuous investment, paddlepaddle has a number of models that can be directly invoked to quickly realize ideas.

  • These models are stored in different suites according to different directions, such as paddelnlp based on NLP, paddegan based on GAN, etc

If you encounter key point detection, segmentation, target recognition and other tasks, you might as well take a look first PaddleHub model library

โฒ๏ธ 1.1 installation instructions

  • PaddlePaddle installation

    This project relies on PaddlePaddle 2.2 and above. Please refer to Installation guide Install

  • PaddleNLP installation

    pip install --upgrade paddlenlp -i

  • Environmental dependence

    Python version requires 3.7+

# paddlenlp upgrade required
!pip install --upgrade pip --user
!pip install --upgrade paddlenlp -i

# Decompress data set
!unzip  -o work/ -d /home/aistudio/work/

๐Ÿ“– 2. Data set introduction

Multiple collected crosstalk texts come from the network and are stored as TXT after processing. For details, see Crosstalk dataset

Dataset production:

(1). The matching data set is constructed with comma as input and bang as output.

(2). Modify the return value of yield in the read function. It is required that the return value is a dictionary containing first and second. The value of first is the input and the value of second is the corresponding output.

(3). The value of first/second is the data after word segmentation, which is connected at an interval of '\ x02', data_path is the parameter of input read.

In this project, multiple txt files are synthesized into a train Txt, and then use jieba word segmentation for processing.

  • Examples of generated data: {'first': 'especially this time \ x02 this \ x02 disease \ x02 is very serious \ x02,\x02 all \ x02 broken \ x02 phase \ x02', 'second': 'why \ x02 broken phase \ x02?'}
# Data set processing
import glob
path_ ='work/data_crosstalk/*.txt'
txt_list_ = glob.glob(path_)
with open("work/train.txt","w+") as save_f:
    for txt_ in txt_list_:
        f = open(txt_, encoding='gbk')
        lines = f.readlines()
        for i in range(len(lines)-1):
            if 'Funny:' in lines[i] and 'Hold:' in lines[i+1]:
                # Input and output
                text = lines[i].replace('\n','')[2:] + '\t' + lines[i+1].replace('\n','')[2:] + '\n'
from paddlenlp.datasets import load_dataset
import pandas as pd
import jieba

def read(data_path):
    data_ori = pd.read_csv(data_path, names=['first','second'], header=None,sep='\t')
    temp = {}
    for i in range(len(data_ori)):
        temp_data_0  = jieba.lcut(data_ori.iloc[i,0]) # 0 is funny
        temp_data_1  = jieba.lcut(data_ori.iloc[i,1]) # 1 is holding
        yield {'first': '\x02'.join(temp_data_0), 'second': '\x02'.join(temp_data_1)}

# data_path is the parameter of the read() method
train_ds = load_dataset(read, data_path='work/train.txt',lazy=False)
for idx in range(2):
{'first': 'thank\x02 to\x02 teacher\x02,\x02 Sick\x02 come\x02 participate in\x02 show\x02.\x02 this\x02 Period of time\x02 you\x02 body\x02 Not good', 'second': 'no\x02 comfortable'}

{'first': 'especially\x02 this time\x02 this\x02 illness\x02 Very heavy\x02,\x02 all\x02 broken\x02 Yes\x02 mutually\x02 Yes', 'second': 'Yes?\x02 Disfigurement\x02 Yes\x02?'}
# View the first five data sets
data_ori = pd.read_csv('work/train.txt', names=['first','second'], header=None,sep='\t')
0Thank teacher Yu for coming to the performance with illness You are not in good health during this perioduncomfortable
1Especially this time, the disease is very serious and has broken its appearanceWhat's wrong?
2hemorrhoidsHemorrhoids disfigured?
3IncompleteNever heard of it
4At first glance, it doesn't come outNo, I didn't
# View the length distribution of data and modify the maximum length entered
%matplotlib inline
%config InlineBackend.figure_format = 'retina'
len_list = []
for i in range(len(data_ori)):
    temp_data  = data_ori.iloc[i,0] # 0 is funny and 1 is flattering
import seaborn as sns
import matplotlib.pyplot as plt


๐Ÿ“– 3. Model implementation and training

import paddlenlp

# Set model name
MODEL_NAME = 'unimo-text-1.0'
tokenizer = paddlenlp.transformers.UNIMOTokenizer.from_pretrained(MODEL_NAME)
[2022-02-20 16:52:01,087] [    INFO] - Already cached /home/aistudio/.paddlenlp/models/unimo-text-1.0/unimo-text-1.0-vocab.txt
from functools import partial
from utils import convert_example

train_trans_func = partial(
    mode='train'), lazy=False, num_workers=4)
<paddlenlp.datasets.dataset.MapDataset at 0x7fb262505690>
idx = 1 # View output
import paddle
from utils import batchify_fn

batch_size = 32

# Define BatchSampler
train_batch_sampler =
        train_ds, batch_size=batch_size, shuffle=True)

# Define batchify_fn
train_collate_fn = partial(batchify_fn, pad_val=0, mode='train')

# Construct DataLoader
train_data_loader =
from paddlenlp.transformers import UNIMOLMHeadModel
model = UNIMOLMHeadModel.from_pretrained(MODEL_NAME)
[2022-02-20 19:14:21,134] [    INFO] - Already cached /home/aistudio/.paddlenlp/models/unimo-text-1.0/unimo-text-1.0.pdparams
# Maximum learning rate during training
learning_rate = 1e-4 

# Training rounds
epochs = 20

# Learning rate preheating ratio
warmup_proportion = 0.02

# The weight attenuation coefficient is similar to the regular term strategy of the model to avoid over fitting of the model
weight_decay = 0.01

num_training_steps = len(train_data_loader) * epochs

# Learning rate attenuation strategy
lr_scheduler = paddlenlp.transformers.LinearDecayWithWarmup(learning_rate, num_training_steps, warmup_proportion)

decay_params = [ for n, p in model.named_parameters()
    if not any(nd in n for nd in ["bias", "norm"])
optimizer = paddle.optimizer.AdamW(
    apply_decay_param_fun=lambda x: x in decay_params)
from utils import evaluation
import paddle.nn.functional as F
from tqdm import tqdm
for epoch in range(0, epochs):
    for batch in tqdm(train_data_loader,desc='epoch:'+str(epoch+1)):
        labels = batch[-1]
        logits = model(*batch[:-1])
        labels = paddle.nn.functional.one_hot(labels, num_classes=logits.shape[-1])
        labels = paddle.nn.functional.label_smooth(labels)
        loss = F.cross_entropy(logits, labels, soft_label=True)


    ppl = paddle.exp(loss)
    print(" epoch: %d, ppl: %.4f, loss: %.5f" % (epoch + 1, ppl, loss))

from utils import post_process_sum

num_return_sequences = 8 #
inputs = 'The first sentence of getting up every day'

inputs_ids = tokenizer.gen_encode(

# Call the generation api and specify the decoding strategy as beam_search
outputs, scores = model.generate(**inputs_ids, decode_strategy='beam_search', num_beams=8,num_return_sequences=num_return_sequences)
print("Result:\n" + 100 * '-')
for i in range(num_return_sequences):
='beam_search', num_beams=8,num_return_sequences=num_return_sequences)
print("Result:\n" + 100 * '-')
for i in range(num_return_sequences):
    print(i, 'Funny:', inputs, 'Hold:', ''.join(post_process_sum(outputs[i].numpy(), tokenizer)[1]))
0 Funny: get up every day, the first sentence is: HMM
1 Tease: get up every day first sentence: hold: first sentence
2 Funny: get up every day the first sentence: ah
3 Funny: get up every day, the first sentence: ah
4 Funny: get up every day the first sentence: ah?
5 Tease: get up every day, the first sentence: hold: the first sentence
6 Funny: get up every day, the first sentence: Oh
7 Funny: get up every day, the first sentence: hold the first sentence

๐Ÿฑ 4 project summary

Special note: if you need to modify the dataset, you can run it with one click after modifying it according to the project description.

If you have any questions, please leave a message in the comment area.

Keywords: AI Deep Learning paddlepaddle

Added by gkelley091565 on Fri, 25 Feb 2022 17:34:36 +0200