Advanced programming skills learning notes

I. progress

1.1 process and procedure

• Process: executing code + resources used

• Program: no executed code, it is a static exe file
  
  

1.2 process status

1.3 use process to realize multi task

Multiprocessing module is a cross platform multiprocessing module. It provides a Process class to represent a Process object. This object can be understood as an independent Process and can perform other things.

# Creating a process is very similar to creating a thread
# You can execute threads and processes separately, and check the differences in task manager

import multiprocessing
import time
import threading


def demo():
    while True:
        print("--1--")
        time.sleep(1)


def demo1():
    while True:
        print("--2--")
        time.sleep(1)


def main():
    p1 = multiprocessing.Process(target=demo)
    p2 = multiprocessing.Process(target=demo1)
    # t1 = threading.Thread(target=demo)
    # t2 = threading.Thread(target=demo1)

    p1.start()
    p2.start()
    # t1.start()
    # t2.start()


if __name__ == '__main__':
    main()

• Thread implements multitasking, which is executed in a code file

• A process implements multitasking, which is equivalent to copying a whole code file to execute a subprocess

1.4 comparison between processes and threads

• Process: be able to complete multiple tasks and run multiple QQ S on one computer at the same time

• Thread: capable of multitasking, multiple chat windows in a QQ (process first, thread second)

• Fundamental difference: process is the basic unit of operating system resource allocation, while thread is the basic unit of task scheduling and execution

2, Interprocess communication - Queue

2.1. Queue

• FIFO

from multiprocessing import Queue


# Create a queue to hold up to 3 pieces of data
q = Queue(3)


# Store data
q.put(1)
q.put("juran")
q.put([11, 22])


# Queue size
print(f'Queue size: {q.qsize()}')
# Determine whether the queue is full
print(f'Is the queue full: {q.full()}')


# Queue full, blocking
# q.put({"name": "juran"})
# No blocking, throw an exception directly
# q.put_nowait({"name": "juran"})


# Fetch data
print('\n Remove data: ')
print(q.get())
print(q.get())
print(q.get())
print()


# No data in the queue, blocking
# print(q.get())
# No blocking, throw an exception directly
# print(q.get_nowait())


# Judge whether the queue is full
print(f'Is the queue full: {q.full()}')
# Judge whether the queue is empty
print(f'Whether the queue is empty: {q.empty()}')

2.2 use queue to realize inter process communication

• Save while downloading

import multiprocessing
import time


def download(q):
    """Download data"""
    lis = [11, 22, 33]
    for item in lis:
        q.put(item)

    print("Download to queue complete, Start saving...\n")


def analysis(q):
    """data processing"""
    analysis_data = list()

    # Don't know the size of the queue, until the end of fetching
    while True:
        data = q.get()
        print(f'Add to {data}')
        analysis_data.append(data)

        # Exit if the queue is empty
        if q.empty():
            break

    print(f'Save finished: {analysis_data}')

def main():
    # Create a queue to communicate across processes
    # q = multiprocessing.Queue(2)
    # The number of data downloaded to the queue is uncertain, and no value can be filled in
    q = multiprocessing.Queue()

    t1 = multiprocessing.Process(target=download, args=(q, ))
    t2 = multiprocessing.Process(target=analysis, args=(q, ))

    t1.start()
    time.sleep(1)
    t2.start()


if __name__ == '__main__':
    main()

2.3. Note that there is another queue

• from queue import Queue
• from multiprocessing import Queue

from queue import Queue
import multiprocessing


def demo1(q):
    q.put('a')


def demo2(q):
    data = q.get()
    print(data)
    

if __name__ == '__main__':
    # Normal queue
    q = Queue()
    # Implementing multi process queues
    # q = multiprocessing.Queue()

    t1 = multiprocessing.Process(target=demo1, args=(q,))
    t2 = multiprocessing.Process(target=demo2, args=(q,))

    # Report wrong
    t1.start()
    t2.start()
    # If you want the code to run, you have to use run(), but this is not multithreading
    # t1.run()
    # t2.run()

3, Multi process sharing global variables

• Processes are not shared, threads are shared

import multiprocessing
import threading

a = 1

def demo1():
    global a
    a += 1


def demo2():
    print(a)


if __name__ == '__main__':
    # process
    t1 = multiprocessing.Process(target=demo1)
    t2 = multiprocessing.Process(target=demo2)
    
    # thread
    # t1 = threading.Thread(target=demo1)
    # t2 = threading.Thread(target=demo2)

    t1.start()
    t2.start()