Intelligent transportation combination fist - Propeller realizes vehicle category / license plate / speed detection, cross-border head tracking, traffic flow density detection and retrograde detection

Intelligent transportation combination fist - flying paddle realizes vehicle category / license plate / speed detection, cross-border head tracking, traffic flow density detection and retrograde detection

General introduction

All the codes and files of this project have been uploaded to Baidu aistudio. Partners in need can search the following link to find it. All the environment, required files and hardware have been configured there.

The items here are:
1. Traffic flow + congestion
2. License plate detection
3. Vehicle type
4. Vehicle visual speed measurement

This project is the integration and coordinated use of some models of traffic vehicle detection after collecting a large number of aistudio, BML open source projects and official technical documents. Several models and data sets are used here. The models used are all common model architectures of flying oars for training. Please prepare enough computing power time or gpuhhh before starting.

Firstly, these detection, recognition and tracking are in the charge of different models. We need to train multiple models of different detection separately for the purpose of detection, and then coordinate these models reasonably according to a certain priority, so as to realize traffic monitoring. In order to shorten the overall demand for GPU resource computing power, I will read the license plate in paddeocr. In this part, I will use GPU to pre train the whole process training model of the model, and the rest will be developed using paddle's adult model as much as possible to reduce the development of the overall project.

At the same time, these models have their own characteristics and different requirements, so the frequency used in the pipeline is also different. I guess the demand of intelligent transportation is like this according to the detection demand in reality: now there are multiple cameras, which can be recognized across lenses. When the vehicle comes like us, we only need to perform one detection to identify the type, color, driving direction and license plate of the vehicle, and then use the tracking model to deal with the vehicle. At this time, the demand becomes the statistics of vehicle speed measurement and traffic congestion.

If time permits, I will publish the second article, which will export these models into ONNX models through padde2ONNX, and then convert them into IR models through openVINO to realize the detection of multi-channel video streams only in the CPU

For the following six practical situations:
1. Vehicle type identification
2. Vehicle license plate recognition
3. Cross border vehicle density
4. Traffic density detection
5. Retrograde detection
6. Vehicle speed detection
The following four models will be used:
4, YOLO v3/2

I believe many friends here will wonder why we don't use these high-precision models such as YOLO4/5, fast RCNN and transformer, and why we use these relatively backward models. The reason is that I consider the actual traffic. Firstly, the performance of YOLO v2/v3 models is fairly good after improvement. Secondly, the real-time requirements of our cameras. A video stream needs to call multiple models to calculate the vehicle image of multiple vehicles.

There are many areas to be improved. Interested friends can also take a look at the following projects and documents I mainly refer to in this process

Model introduction

1. Introduction to PP tracking

PP tracking is the industry's first open source real-time tracking system based on the propeller deep learning framework. PP tracking has built-in capabilities and industrial applications such as pedestrian and vehicle tracking, cross camera tracking, multi category tracking, small target tracking and traffic counting. It integrates multi-target tracking, target detection and ReID lightweight algorithms in the model to further improve the deployment performance of PP tracking on the server. It also supports python and C + + deployment and adapts to Linux, NVIDIA and Jetson multi platform environment.

According to the above figure, let me talk about the target detection under the characteristics of multiple cameras. Let's start with a camera. As for a camera, it will track all the dynamic targets in the video stream, and then match these targets with the features we have trained in advance.

If there are multiple cameras, it can be regarded as a combination of multiple single cameras. Each camera performs the above detection, matches the features, and identifies the type of target. Then what we do is find this category in each camera.

For example, the license plate we want to track is 88888. When a camera detects 88888, the vehicle will not stay in one camera. It may appear in each camera, and the track of 88888 may be captured by each camera.

2. PaddleOCR introduction

PaddleOCR is an OCR algorithm suite recently opened by the flying propeller. OCR refers to a kind of model that checks characters, determines its shape by detecting dark and bright patterns, and then translates the shape into computer text by character recognition method. PP OCR includes various mainstream detection and recognition algorithms, such as DB, CRNN, etc., which have excellent detection results.

3. Introduction to paddedetection

Paddedetection is a unified framework for object detection launched by the propeller. Support existing RCNN, SSD, YOLO and other series models, and support backbone networks such as ResNet, ResNet VD, ResNeXt, ResNeXt VD, SENet, MobileNet and DarkNet. For different business scenarios (performance, target size, accuracy, etc.), you can call different parameters (or parameter files) under the framework to realize the task. Compared with the object of tensorflow_ One of the advantages of detection is to integrate YOLO, a fast algorithm for target detection, into the framework.

4. Introduction to YOLO v3/2

YOLO series is the most commonly used algorithm for target detection, with good effect and performance.
Thesis address:
Thesis: YOLOv3: An Incremental Improvement
And interested friends can directly read the author's paper, which has been written in great detail.

Model establishment

1. Cross border traffic density statistics -- PP tracking

The dataset is downloaded here:

  1. Environment preparation: Download paddedetection
!git clone -b develop

Let's do the following three steps with three pieces of code
Image upgrade

!pip install --upgrade pip -i

Install paddlepaddle-gpu2 two

!pip install paddlepaddle-gpu==2.2.0rc0.post101 -f -i

Install the dependencies related to paddedetection

!cd PaddleDetection/ && pip install -r requirements.txt && python install

1. Download the prediction deployment model. Here, in order to get started quickly and save computing power, we directly use the training model provided by the official. First, we download [wegt] target detection and ReID prediction model, download the address, and then put [mv] in ~ / paddedetection / output_ Under information [CD], then unzip [tar]



!cd ~/PaddleDetection/ && mkdir -p output_inference

!mv ppyolov2_r50vd_dcn_365e_aic21mtmct_vehicle.tar ~/PaddleDetection/output_inference

!mv deepsort_pplcnet_vehicle.tar ~/PaddleDetection/output_inference

!cd ~/PaddleDetection/output_inference && tar -xvf ppyolov2_r50vd_dcn_365e_aic21mtmct_vehicle.tar && tar -xvf deepsort_pplcnet_vehicle.tar

2. Cross mirror tracking and prediction

After downloading the model, you need to modify the mtmct in the paddedetection / deploy / pptracking / Python path_ cfg. YML, this configuration file contains the relevant parameters of trajectory fusion in cross mirror tracking. First, you need to determine the cameras_ The corresponding name in bias corresponds to the input video name; Secondly, in this project, we use the general method in trajectory fusion to set the methods related to zone and camera to False.
The setting parameters are as follows:

# config for MTMCT
  c003: 0
  c004: 0
# releated parameters
use_zone: False #True
zone_path: dataset/mot/aic21mtmct_vehicle/S06/zone
# 2.tricks parameters, can be used for other mtmct dataset
use_ff: True
use_rerank: False #True
# releated parameters
use_camera: False #True
use_st_filter: False
# releated parameters
use_roi: False #True
roi_dir: dataset/mot/aic21mtmct_vehicle/S06

3. After the configuration is completed, run the following command, and the input video is C004 MP4 and C003 Mp4 camera shooting results from two different perspectives

Cross mirror tracking output video is saved in paddedetection / output / mtmct vis

!wget && mv mtmct-demo.tar ~/PaddleDetection && cd ~/PaddleDetection && tar xvf mtmct-demo.tar 

!cd ~/PaddleDetection && python deploy/pptracking/python/ --model_dir=output_inference/ppyolov2_r50vd_dcn_365e_aic21mtmct_vehicle/ --reid_model_dir=output_inference/deepsort_pplcnet_vehicle/ --mtmct_dir=./mtmct-demo --device=GPU --mtmct_cfg=deploy/pptracking/python/mtmct_cfg.yml --scaled=True --save_mot_txts --save_images

txt file. The results are saved in output/mot_results/img1.txt, the output format is expressed as frame_id (number of frames), ID (number of cars), bbox_left, bbox_top, bbox_width, bbox_height (four coordinates), score (score), x, y, z. it is necessary to master this output. You need to confirm the number of vehicles by outputting the detected ID. you can open it and see the results of the video, as shown in the figure below

In real-time detection, the congestion situation needs to be judged according to the actual lane. The judgment function is given here. The function here is not a visual model, but a congestion judgment model that inputs the number of vehicles and lanes. To detect congestion or not, you need to input the results of the above model and the number of lanes into the following function. For real-time video, you only need to adjust the input format to get congestion automatic test. For example, the following six chapters and pictures represent several situations to substitute into the detection function:
{'index': 1, 'detector_count': 41, 'pic_name': '7.png'},
{'index': 2, 'detector_count': 47, 'pic_name': '4.png'},
{'index': 3, 'detector_count': 19, 'pic_name': '12.png'},
{'index': 4, 'detector_count': 20, 'pic_name': '11.png'},
{'index': 5, 'detector_count': 39, 'pic_name': '1.png'},
{'index': 6, 'detector_count': 1, 'pic_name': '5.png'}

#Threshold setting
#N number of instantaneous vehicles in the observation section, unit: Vehicle
#L number of lanes, the default setting is 4 lanes
L = 4
limit = 10  #Set the maximum number of vehicles per lane in the figure as 10 
data_list = [{'index': 1, 'detector_count': 41, 'pic_name': '7.png'}, {'index': 2, 'detector_count': 47, 'pic_name': '4.png'}, {'index': 3, 'detector_count': 19, 'pic_name': '12.png'}, {'index': 4, 'detector_count': 20, 'pic_name': '11.png'}, {'index': 5, 'detector_count': 39, 'pic_name': '1.png'}, {'index': 6, 'detector_count': 1, 'pic_name': '5.png'}]
#Density calculation
def traffic_density(N):
    global L
    K = N / L #Calculation formula 
    return K 
#Get traffic conditions
def Get_traffic_situation():

    for i in data_list:
        N = i.setdefault('detector_count')
        density = traffic_density(N)
        if density >= limit-5 and density < limit:
            print("index by",i.setdefault('index'),"The traffic flow of the road section is about to cause traffic congestion, which needs to be regulated in time!")
        elif density >= limit:
            print("index by",i.setdefault('index'),"The road is in a traffic jam!")
            print("index by",i.setdefault('index'),"The traffic condition of the road section is good!")

2. paddleOCR training license plate recognition model

Data sets can be downloaded from this channel: Then download the dataset

!unzip -q /home/work/

Generate label document

The data set used this time is CCPD2019 license plate data set
The data set was collected in the parking lot of Hefei from 7:30 a.m. to 10:00 p.m. The collection personnel in the parking lot take photos of the vehicles in the parking lot with Android POS machine and manually mark the license plate position. The license plate photos taken involve a variety of complex environments, including blur, tilt, rainy days, snowy days and so on. The CCPD dataset contains nearly 300000 images, each of which is 720x1160x3 in size. There are 8 items in total, as follows:

There is no special label file in CCPD dataset, and the file name of each image is the corresponding data label
For example: 025-95_ 113-154&383_ 386&473-386&473_ 177&454_ 154&383_ 363&402-0_ 0_ 22_ 27_ 27_ 33_ 16-37-15. Jpg is divided into several parts by the separator '-'
025 is area
95_ 113 corresponds to two angles, 95 ° horizontal and 113 ° vertical
154&383_ 386-473 corresponding bounding box coordinates: upper left (154, 383), lower right (386, 473)
386&473_ 177&454_ 154&383_ 363-402 corresponds to four corner coordinates
0_ 0_ 22_ 27_ 27_ 33_ 16 is the license plate number, and the mapping relationship is as follows: the first is province 0, corresponding to province dictionary Wan, followed by letters and words. Check ads dictionary If 0 is A, 22 is Y
Only the normal license plate in the dataset, i.e. CCPD, is used_ Base data

import os, cv2
import random
words_list = [
    "A", "B", "C", "D", "E",
    "F", "G", "H", "J", "K", 
    "L", "M", "N", "P", "Q", 
    "R", "S", "T", "U", "V", 
    "W", "X", "Y", "Z", "0", 
    "1", "2", "3", "4", "5", 
    "6", "7", "8", "9" ]

con_list = [
    "Wan", "Shanghai", "Jin", "Chongqing", "Hope",
    "Jin", "Meng", "Liao", "luck", "black",
    "Soviet", "Zhejiang", "Beijing", "Min", "short name for Jiangxi province",
    "Lu", "Yu", "Hubei", "Hunan", "Guangdong",
    "Cinnamon", "Joan", "Chuan", "expensive", "cloud",
    "west", "Shaanxi", "Sweet", "young", "Rather",

count = 0
total = []
paths = os.listdir('/home/work/chepaishibie')#Real dataset path
#for path in paths:
for item in os.listdir(os.path.join('/home/work/chepaishibie')):#Real dataset path
    if item[-3:] =='jpg':
        new_path = os.path.join('/home/work/chepaishibie', item) #Training image path
        _, _, bbox, points, label, _, _ = item.split('-')
        points = points.split('_')
        points = [_.split('&') for _ in points]
        tmp = points[-2:]+points[:2]
        points = []
        for point in tmp:
            points.append([int(_) for _ in point])
        label = label.split('_')
        con = con_list[int(label[0])]
        words = [words_list[int(_)] for _ in label[1:]]
        label = con+''.join(words)
        line = new_path+'\t'+'[{"transcription": "%s", "points": %s}]' % (' ', str(points))
        line = line[:]+'\n'

#Note: Here's train Txt / dev.txt was not available before. Now you can create a folder to open it
with open('/home/work/data/train.txt', 'w', encoding='UTF-8') as f:
    for line in total[:-200]:

with open('/home/work/data/dev.txt', 'w', encoding='UTF-8') as f:
    for line in total[-200:]:

Check the two txt files under data

Configuration environment

1. upgrade PaddlePaddle

!pip install paddlepaddle-gpu==2.2.1.post101 -f

2. Download PaddleOCR

!git clone

3. Download the pre training model

!cd /home/work/PaddleOCR
!wget -P ./pretrain_models/

!wget -P ./pretrain_models/

Note: look where your pre training model folder is. I'm under home/work. I've messed up here and switched the path to other places. Just understand that the following operation is to extract the downloaded pre training file

!cd /home/work/pretrain_models
!tar -xf ch_det_mv3_db.tar && rm -rf ch_det_mv3_db.tar
!tar -xf rec_mv3_tps_bilstm_attn.tar && rm -rf rec_mv3_tps_bilstm_attn.tar

cd switching path

cd /home/work/PaddleOCR

Install the dependent files related to PaddleOCR 1 Modify the requirements file

Delete opencv contrib python = =

2. Execute the following two installation commands:

!pip install -r requirements.txt
!pip install opencv-contrib-python==

Step 3: modify the yml file of model training
Enter the path PaddleOCR/configs/det/
Open det_mv3_db.yml

1. Modify model path and save path

2. Modify the path of training data
Training set:

Validation set:

3. Set training parameters
Number of epoch s:


Step 4: model training

The model training here can be completed in about half an hour according to the performance of V100

!python3 tools/ -c configs/det/det_mv3_db.yml

Step 5: export the text detection model

!python3 /home/work/PaddleOCR/tools/ \
        -c /home/work/PaddleOCR/configs/det/det_mv3_db.yml \
        -o Global.checkpoints=/home/work/PaddleOCR/output/db_mv3/best_accuracy \

Here, the training of a model is completed. Of course, the step of model evaluation is still missing, but the training time of this model configuration is too little, so we know it can't be done without evaluation, so we'll download one next. Step 3: Download and install the character recognition model

!tar -xf ch_ppocr_server_v2.0_rec_infer.tar -C inference

Step 6: model prediction

1. Upload a license plate picture as a prediction picture

!python /home/work/PaddleOCR/tools/infer/ \
    --image_dir="/home/work/test.jpg" \
    --det_model_dir="/home/work/PaddleOCR/inference/" \
    --rec_model_dir="/home/work/PaddleOCR/inference/ch_ppocr_server_v2.0_rec_infer/" \
    --use_gpu False

You can use any picture in the dataset and enter the path yourself to adjust it. The path is test Jpg that

Detect vehicle type, color and driving direction

The environment here is too difficult to configure. After configuration, I simply compress the required environment file into a compressed package. Just unzip the file and put it in the same directory as paddleOCR and paddle Detection. These models are trained models, which can be used after being opened without many GPU resources. This package has been uploaded to my CSDN: Password: zw9v, if the package is lost, you can send me CSDN Personal blog To maintain.
The decompression directory is shown in the figure:

1. Unzip folder


Import the library we need. As long as what we did in the previous step is correct, there will be no error in this step.

import cv2
from paddle_model_cls.model_with_code.model import x2paddle_net

import argparse
import functools
import numpy as np
import paddle.fluid as fluid
import matplotlib.pyplot as plt

The following is the image preprocessing function: this function is mainly used to standardize the image and zoom the image to 224 × 224 pixels

def process_img(img, image_shape=[3, 224, 224]):

    mean = [0.485, 0.456, 0.406]
    std = [0.229, 0.224, 0.225]

    img = cv2.resize(img, (image_shape[1], image_shape[2]))
    #img = cv2.resize(img,(256,256))
    #img = crop_image(img, image_shape[1], True)

    # RBG img [224,224,3]->[3,224,224]
    img = img[:, :, ::-1].astype('float32').transpose((2, 0, 1)) / 255
    #img = img.astype('float32').transpose((2, 0, 1)) / 255
    img_mean = np.array(mean).reshape((3, 1, 1))
    img_std = np.array(std).reshape((3, 1, 1))
    img -= img_mean
    img /= img_std

    img = img.astype('float32')
    img = np.expand_dims(img, axis=0)

    return img

Define prediction function:

The function is to convert the label represented by one hot into the index of color, orientation and category

def get_predict(output):
    output = np.squeeze(output)
    pred_color = output[:9]
    pred_direction = output[9:11]
    pred_type = output[11:]

    color_idx = np.argmax(pred_color)
    direction_idx = np.argmax(pred_direction)
    type_idx = np.argmax(pred_type)

    return color_idx, direction_idx, type_idx

Define classifier: a class of classifier is defined here, which is used to obtain image input and output prediction; You can see the following_ GPU = false, which means that in the CPU environment, if you use GPU, you can change it to yes

use_gpu = False
class CarClassifier(object):

    def __init__(self):

        self.color_attrs = ['Black', 'Blue', 'Brown',
                            'Gray', 'Green', 'Pink',
                            'Red', 'White', 'Yellow']  # Car body color

        self.direction_attrs = ['Front', 'Rear']  # Shooting position

        self.type_attrs = ['passengerCar', 'saloonCar',
                           'shopTruck', 'suv', 'trailer', 'truck', 'van', 'waggon']  # Vehicle type


    def inference(self, img):
        fetch_list = []

        output =,
                              feed={'image': img})
        color_idx, direction_idx, type_idx = get_predict(np.array(output))

        color_name = self.color_attrs[color_idx]
        direction_name = self.direction_attrs[direction_idx]
        type_name = self.type_attrs[type_idx]

        return color_name, direction_name, type_name

    def init_params(self):

        # Attack graph
        adv_program = fluid.Program()

        # Complete initialization
        with fluid.program_guard(adv_program):
            input_layer =
                name='image', shape=[3, 224, 224], dtype='float32')
            # Set to calculate the gradient
            input_layer.stop_gradient = False

            # model definition
            _, out_logits = x2paddle_net(inputs=input_layer)
            self.out = fluid.layers.softmax(out_logits[0])

            place = fluid.CUDAPlace(0) if use_gpu else fluid.CPUPlace()
            self.exe = fluid.Executor(place)

            # Record model parameters
                self.exe, './paddle_model_cls/model_with_code/')

        # Create evaluation mode for testing
        self.eval_program = adv_program.clone(for_test=True)

    def predict(self, im):

        im_input = process_img(im)

        color_name, direction_name, type_name = self.inference(im_input)

        label = 'Color:{}\n orientation:{}\n Type:{}'.format(
            color_name, direction_name, type_name)

        return label

%matplotlib inline

I'll change the name of this classifier to make it easier to type it later

classifier = CarClassifier()

Import other packages

Mainly some drawing tools (PIL) and paddedetection tools (ppdet). Don't worry about being too strange here. All the environments are equipped here. You just need to unzip the compressed package and put multiple folders in the same directory of this file

import ppdet.utils.checkpoint as checkpoint
from ppdet.utils.cli import ArgsParser
from ppdet.utils.eval_utils import parse_fetches
from ppdet.core.workspace import load_config, create
from paddle import fluid
import os
import cv2
import glob

from ppdet.utils.coco_eval import bbox2out, mask2out, get_category_info

import numpy as np
from PIL import Image
from PIL import ImageFont, ImageDraw

Define drawing functions

Using PIL library to solve the problem that opencv does not support Chinese

font_path = r'./simsun.ttc' # NSimSun 
font = ImageFont.truetype(font_path, 16)

def putText(img, text, x, y, color=(0, 0, 255)):

    img_pil = Image.fromarray(img)
    draw = ImageDraw.Draw(img_pil)
    b, g, r = color
    a = 0
    draw.text((x, y), text, font=font, fill=(b, g, r, a))
    img = np.array(img_pil)
    return img

What does this set of functions do? How is it affected by the type and direction of the car? It's like this. There is a classifier function, which I defined once. Through the detection function, you can detect the color of the vehicle, the model (car, truck, bus, taix, SUV, etc.) and the positive and negative surfaces of the vehicle. If the negative of the vehicle is detected in the forward driving channel, it is a violation.

right_direct = 'Rear'

class new_Det(VehicleDetector):

    def __init__(self):

        self.size = 608

        self.draw_threshold = 0.1

        self.cfg = load_config('./configs/vehicle_yolov3_darknet.yml') = fluid.CUDAPlace(
            0) if use_gpu else fluid.CPUPlace()
        # = fluid.CPUPlace()
        self.exe = fluid.Executor(

        self.model = create(self.cfg.architecture)

        self.classifier = classifier


    def init_params(self):

        startup_prog = fluid.Program()
        infer_prog = fluid.Program()
        with fluid.program_guard(infer_prog, startup_prog):
            with fluid.unique_name.guard():
                inputs_def = self.cfg['TestReader']['inputs_def']
                inputs_def['iterable'] = True
                feed_vars, loader = self.model.build_inputs(**inputs_def)
                test_fetches = self.model.test(feed_vars)
        infer_prog = infer_prog.clone(True)
        if self.cfg.weights:
            checkpoint.load_params(self.exe, infer_prog, self.cfg.weights)

        extra_keys = ['im_info', 'im_id', 'im_shape']
        self.keys, self.values, _ = parse_fetches(
            test_fetches, infer_prog, extra_keys)
        dataset = self.cfg.TestReader['dataset']
        anno_file = dataset.get_anno()
        with_background = dataset.with_background
        use_default_label = dataset.use_default_label

        self.clsid2catid, self.catid2name = get_category_info(anno_file, with_background,

        is_bbox_normalized = False
        if hasattr(self.model, 'is_bbox_normalized') and \
            is_bbox_normalized = self.model.is_bbox_normalized()

        self.is_bbox_normalized = is_bbox_normalized

        self.infer_prog = infer_prog

    def process_img(self, img):

        mean = [0.485, 0.456, 0.406]
        std = [0.229, 0.224, 0.225]
        shape = img.shape[:2]

        img = cv2.resize(img, (self.size, self.size))

        # RBG img [224,224,3]->[3,224,224]
        img = img[:, :, ::-1].astype('float32').transpose((2, 0, 1)) / 255
        img_mean = np.array(mean).reshape((3, 1, 1))
        img_std = np.array(std).reshape((3, 1, 1))
        img -= img_mean
        img /= img_std

        img = img.astype('float32')
        img = np.expand_dims(img, axis=0)

        shape = np.expand_dims(np.array(shape), axis=0)
        im_id = np.zeros((1, 1), dtype=np.int64)

        return img, im_id, shape

    def detect(self, img):

        # img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
        raw = img.copy()
        img, im_id, shape = self.process_img(img=img)
        outs =,
                            feed={'image': img, 'im_size': shape.astype(np.int32), 'im_id': im_id},
        res = {
            k: (np.array(v), v.recursive_sequence_lengths())
            for k, v in zip(self.keys, outs)

        bbox_results = bbox2out(
            [res], self.clsid2catid, self.is_bbox_normalized)

        result = self.draw_bbox(raw, self.catid2name,
                                bbox_results, self.draw_threshold)

        return result
    def draw_bbox(self, image, catid2name, bboxes, threshold):

        raw = image.copy()

        for dt in np.array(bboxes):

            catid, bbox, score = dt['category_id'], dt['bbox'], dt['score']
            if score < threshold or catid == 6:
                # if score < threshold:

            xmin, ymin, w, h = bbox
            xmin = int(xmin)
            ymin = int(ymin)
            xmax = int(xmin + w)
            ymax = int(ymin + h)
            roi = raw[ymin:ymax, xmin:xmax].copy()
            label = self.classifier.predict(roi)
            if right_direct in label:
                color = (0, 255, 0)
                color = (0, 0, 255)
                print('Illegal retrograde detected!')

            cv2.rectangle(image, (xmin, ymin), (xmax, ymax), color, 4)

        return image

When you use it, you can directly copy the above reference code and function code, and then I you only need to use a det to detect the situation of the car.

det = new_Det()
In [8]
im = cv2.imread('./demo.png')
result = det.detect(im)

plt.imshow(result[:, :, [2, 1, 0]])

Visual detection of vehicle speed

1, Install and import the package. If you have installed it according to the previous steps, you can directly reinstall the following libraries and run the supplementary installation. If not, you also need to install paddedetectino and installation environment variables

!pip install --upgrade pip  
!pip install paddledet 
!pip install cmake 
!pip install boost 
!pip install dlib 
!pip install pycocotools 
import os  #File operation Library
import ppdet
import dlib  #Machine learning algorithm library
import time
import threading  #Multithreading Library
import math  #Mathematical Library
import cv2   #Import CV2 library and process pictures or videos. The version number is
import paddle  
import paddle.fluid as fluid
import numpy as np  #Mathematical function library
from PIL import ImageFont, ImageDraw, Image  #Image library
import fnmatch
import random  

Here I provide an example video, which is placed under the work path of the project. You can add the video according to your own path in vediopath, and then OS MKDIR is to create a folder for later work

videopath = '/home/work/test/0831_1.mp4'
#Vehicle speed video recognition
#Vehicle speed detection function
#Load vehicle identification file
#Load classifier for vehicle recognition
carCascade = cv2.CascadeClassifier('/home/work/myhaar1.xml')
#Read video file
video = cv2.VideoCapture(videopath)

carWidht = 1.85

# def carNumber(carNum, cID):
	# time.sleep(2)
	# carNum[cID] = 'Car ' + str(cID)

def estimateSpeed(location1, location2, mySpeed,fps):
#Calculate pixel distance
	d_pixels = math.sqrt(math.pow(location2[0] - location1[0], 2) + math.pow(location2[1] - location1[1], 2))
	ppm = location2[2] / carWidht
	d_meters = d_pixels / ppm
	speed = mySpeed + d_meters * fps
	return speed
def trackMultipleObjects():
	rectangleColor = (0, 0, 255)
	currentCarID = 0
	fps = 0
	global frameCounter
	frameCounter = 0

	carTracker = {}
	carNumbers = {}
	carLocation1 = {}
	carLocation2 = {}

	while True:
		start_time = time.time()
        #Read frame
		rc, image =
		if(frameCounter == 0):  #Read video resolution
			global WIDTH
			global HEIGHT
			WIDTH = len(image[0])
			HEIGHT = len(image)
        #Check whether the end of the video file is reached
		if type(image) == type(None):
		#Convert the size of the frame to speed up processing
		#image = cv2.resize(image, (WIDTH, HEIGHT))
		resultImage = image.copy()
		frameCounter = frameCounter + 1
		carIDtoDelete = []
		for carID in carTracker.keys():
			trackingQuality = carTracker[carID].update(image)
			if trackingQuality < 7:
		for carID in carIDtoDelete:
			#print('Removing carID ' + str(carID) + ' from list of trackers.')
			#print('Removing carID ' + str(carID) + ' previous location.')
			#print('Removing carID ' + str(carID) + ' current location.')
			carTracker.pop(carID, None)
			carLocation1.pop(carID, None)
			carLocation2.pop(carID, None)
		# Execute when frameCounter is a multiple of 10
		if not (frameCounter % 10):
			#Convert image to grayscale image
			gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
			#Detect the vehicle in the video, and save the coordinates and size of the vehicle with vector (represented by rectangle)
			# x. Y represents the central abscissa and ordinate position of the circumscribed rectangle of the ith moving target in frame n, which can roughly describe the position of the vehicle target.
			# w. H represents the width and length of the circumscribed rectangle of the ith moving target in frame n, which can describe the size of the vehicle target
			cars = carCascade.detectMultiScale(gray, 1.1, 13, 0, (24, 24))
            #Vehicle detection
			for (_x, _y, _w, _h) in cars:
				x = int(_x)
				y = int(_y)
				w = int(_w)
				h = int(_h)
				x_bar = x + 0.5 * w
				y_bar = y + 0.5 * h
				matchCarID = None
				for carID in carTracker.keys():
					trackedPosition = carTracker[carID].get_position()
					t_x = int(trackedPosition.left())
					t_y = int(
					t_w = int(trackedPosition.width())
					t_h = int(trackedPosition.height())
					t_x_bar = t_x + 0.5 * t_w
					t_y_bar = t_y + 0.5 * t_h
					if ((t_x <= x_bar <= (t_x + t_w)) and (t_y <= y_bar <= (t_y + t_h)) and (x <= t_x_bar <= (x + w)) and (y <= t_y_bar <= (y + h))):
						matchCarID = carID
				if matchCarID is None:
					print('Creating new tracker ' + str(currentCarID))
					#Construct tracker
					tracker = dlib.correlation_tracker()
					# Set the initial position of the tracker
					#If the vehicle is identified, it will return to the position of the vehicle in the form of Rect(x,y,w,h), and then we can identify the vehicle with a rectangular grid
					tracker.start_track(image, dlib.rectangle(x, y, x + w, y + h))
					carTracker[currentCarID] = tracker
					# Rectangle used to generate tracker [(startX, startY), (endX, endY)]
					carLocation1[currentCarID] = [x, y, w, h]
					# t = threading.Thread(target = carNum, args = (carNumbers, currentCarID))
					# t.start()
					currentCarID = currentCarID + 1
		for carID in carTracker.keys():
			# Get the current position of the tracker
			trackedPosition = carTracker[carID].get_position()
			t_x = int(trackedPosition.left())
			t_y = int(
			t_w = int(trackedPosition.width())
			t_h = int(trackedPosition.height())
			#cv2.rectangle(resultImage, (t_x, t_y), (t_x + t_w, t_y + t_h), rectangleColor, 2)
			# speed estimation
			carLocation2[carID] = [t_x, t_y, t_w, t_h]
			# if carID in carNumbers.keys():
				# cv2.putText(resultImage, carNumbers[carID], (int(t_x + t_w/2), int(t_y)),cv2.FONT_HERSHEY_SIMPLEX, 0.5, (255, 255, 255), 2)
			# else:
				# cv2.putText(resultImage, 'Detecting...', (int(t_x + t_w/2), int(t_y)),cv2.FONT_HERSHEY_SIMPLEX, 0.5, (255, 255, 255), 2)
		end_time = time.time()
		if not (end_time == start_time):
			fps = 1.0/(end_time - start_time)
		cv2.putText(resultImage, 'FPS: ' + str(int(fps)), (620, 30),cv2.FONT_HERSHEY_SIMPLEX, 0.75, (0, 0, 255), 2)
		for i in carLocation1.keys():
			if frameCounter % 10 == 0:
				[x1, y1, w1, h1] = carLocation1[i]
				[x2, y2, w2, h2] = carLocation2[i]
				#print ('previous location: ' + str(carLocation1[i]) + ', current location: ' + str(carLocation2[i]))
				carLocation1[i] = [x2, y2, w2, h2]
				#print('new previous location: ' + str(carLocation1[i]))
				if [x1, y1, w1, h1] != [x2, y2, w2, h2]:
					speed = estimateSpeed([x1, y1, w1, h1], [x2, y2, w2, h2], 100, fps)
					#speed = (speed-100)*100
					print('CarID ' + str(i) + ' speed is ' + str("%.2f" % round(speed, 2)) + ' km/h')
					cv2.putText(resultImage, str("%.2f" % round(speed, 2)) + ' km/h' , (t_x, t_y), cv2.FONT_HERSHEY_SIMPLEX, 1, (0, 255, 0), 2)
				cv2.putText(resultImage, str("%.2f" % round(speed, 2)) + ' km/h' , (t_x, t_y), cv2.FONT_HERSHEY_SIMPLEX, 1, (0,255, 0), 2)	
		#%cd /home/aistudio/
		str_name = '/home/work/test/video1/frame'+str(frameCounter)+'.png'
		cv2.imwrite(str_name, resultImage)
		vediowriter =cv2.VideoWriter('/home/work/test/test1.mp4',cv2.VideoWriter_fourcc(*'mp4v'),30,(WIDTH,HEIGHT))
		for i in range(frameCounter):
			new_image_name = '/home/work/test/video1/frame'+str(i)+'.png'
			img =cv2.imread(new_image_name)


You can see CV2 above Videowriter, this is the address where the video is saved. You can adjust and view it according to your needs. I cut a picture

OK, so far we have made all the models that need to be used. You can get the desired output results only by a simple sentence of function and defining the input video stream in advance. How to match these function calls to make an intelligent transportation scheme has to wait until the next project hhh.

Keywords: paddlepaddle paddle

Added by Chizzad on Sat, 05 Mar 2022 05:38:58 +0200