Python multitasking - process

Advanced programming skills learning notes

I. progress

1.1 process and procedure

  • Process: executing code + resources used

  • Program: no executed code, it is a static exe file

1.2 process status

1.3 use process to realize multi task

Multiprocessing module is a cross platform multiprocessing module. It provides a Process class to represent a Process object. This object can be understood as an independent Process and can perform other things.

# Creating a process is very similar to creating a thread
# You can execute threads and processes separately, and check the differences in task manager

import multiprocessing
import time
import threading

def demo():
    while True:

def demo1():
    while True:

def main():
    p1 = multiprocessing.Process(target=demo)
    p2 = multiprocessing.Process(target=demo1)
    # t1 = threading.Thread(target=demo)
    # t2 = threading.Thread(target=demo1)

    # t1.start()
    # t2.start()

if __name__ == '__main__':

  • Thread implements multitasking, which is executed in a code file

  • A process implements multitasking, which is equivalent to copying a whole code file to execute a subprocess

1.4 comparison between processes and threads

  • Process: be able to complete multiple tasks and run multiple QQ S on one computer at the same time

  • Thread: capable of multitasking, multiple chat windows in a QQ (process first, thread second)

  • Fundamental difference: process is the basic unit of operating system resource allocation, while thread is the basic unit of task scheduling and execution

2, Interprocess communication - Queue

2.1. Queue

  • FIFO
from multiprocessing import Queue

# Create a queue to hold up to 3 pieces of data
q = Queue(3)

# Store data
q.put([11, 22])

# Queue size
print(f'Queue size: {q.qsize()}')
# Determine whether the queue is full
print(f'Is the queue full: {q.full()}')

# Queue full, blocking
# q.put({"name": "juran"})
# No blocking, throw an exception directly
# q.put_nowait({"name": "juran"})

# Fetch data
print('\n Remove data: ')

# No data in the queue, blocking
# print(q.get())
# No blocking, throw an exception directly
# print(q.get_nowait())

# Judge whether the queue is full
print(f'Is the queue full: {q.full()}')
# Judge whether the queue is empty
print(f'Whether the queue is empty: {q.empty()}')

2.2 use queue to realize inter process communication

  • Save while downloading
import multiprocessing
import time

def download(q):
    """Download data"""
    lis = [11, 22, 33]
    for item in lis:

    print("Download to queue complete, Start saving...\n")

def analysis(q):
    """data processing"""
    analysis_data = list()

    # Don't know the size of the queue, until the end of fetching
    while True:
        data = q.get()
        print(f'Add to {data}')

        # Exit if the queue is empty
        if q.empty():

    print(f'Save finished: {analysis_data}')

def main():
    # Create a queue to communicate across processes
    # q = multiprocessing.Queue(2)
    # The number of data downloaded to the queue is uncertain, and no value can be filled in
    q = multiprocessing.Queue()

    t1 = multiprocessing.Process(target=download, args=(q, ))
    t2 = multiprocessing.Process(target=analysis, args=(q, ))


if __name__ == '__main__':

2.3. Note that there is another queue

  • from queue import Queue
  • from multiprocessing import Queue
from queue import Queue
import multiprocessing

def demo1(q):

def demo2(q):
    data = q.get()

if __name__ == '__main__':
    # Normal queue
    q = Queue()
    # Implementing multi process queues
    # q = multiprocessing.Queue()

    t1 = multiprocessing.Process(target=demo1, args=(q,))
    t2 = multiprocessing.Process(target=demo2, args=(q,))

    # Report wrong
    # If you want the code to run, you have to use run(), but this is not multithreading

3, Multi process sharing global variables

  • Processes are not shared, threads are shared
import multiprocessing
import threading

a = 1

def demo1():
    global a
    a += 1

def demo2():

if __name__ == '__main__':
    # process
    t1 = multiprocessing.Process(target=demo1)
    t2 = multiprocessing.Process(target=demo2)
    # thread
    # t1 = threading.Thread(target=demo1)
    # t2 = threading.Thread(target=demo2)

Published 85 original articles, won praise 0, visited 1169
Private letter follow

Keywords: Programming Windows

Added by JukEboX on Fri, 31 Jan 2020 05:57:38 +0200