Concurrent programming

Concurrent and Serial

1. What is serial?

   Serial means that a program is executed from top to bottom, one line at a time, and the current task must be executed before the next program can be executed.

   Question:

   • Inefficient program execution
   • Occupy memory resources

2. What is concurrency?

   Concurrency means that multiple programs can be executed simultaneously (essentially fast switching execution between different processes and saving program state). The summary is switching + saving

   Both serial and concurrent are the way programs handle tasks

Why use concurrency?

• Multiple programs can be executed at the same time, which improves productivity
• Increase the utilization of memory resources

Ways to achieve concurrency

• Multiprocess
• Multithreaded
• Protocol

What is a process?

A process is a running program and is the basic unit for operating system scheduling and resource allocation (a process is a resource unit)

1. How did the process come about?

   When a program is read from the hard disk into memory, a process occurs

2. What is multi-process?

   Multiple programs are read into memory and executed at the same time, that is, multiple processes

Processes come from the operating system, which schedules and allocates resources

The principle of multi-process implementation is essentially the principle of operating system scheduling processes

Multichannel Technology (Key)

Implementation principle:

1. Spatial reuse:

   Partition memory into multiple zones, place multiple programs in memory, and isolate memory zones from each other (physical isolation to ensure data security)

2. Time Reuse: (Switch+Save)

   Operating system switches execution between processes

   There are two scenarios for switching tasks:

   • Automatically switch when a process encounters an IO operation
   • Force switching when a task execution time exceeds the threshold

   The state must be saved before switching for subsequent resumption of execution

Important concepts in concurrent programming (focus)

1. Serial: Programs are executed top-down, one by one in sequence

2. Concurrency: Quick switching between different people

3. Parallel: True simultaneous execution of multiple programs requires a multi-core CPU

   All three are ways to handle tasks

4. Blocking: A state in which a program encounters IO operations and cannot continue executing code

5. Non-blocking: A state in which a program does not encounter IO operations

6. Three states of the process

   • Blocking: Release IO operation, convert to ready
   • Run: Convert to blocking when IO operation and run timeout occurs; Release IO operation is converted to ready
   • Ready: CPU call, converted to run

​

There are two ways to use multiprocesses in python:

1. Import the Process class in multiprocessing, instantiate it, and specify the task program target to execute

   import os
   from multiprocessing import Process
   """
   Process Represents a process
   //Why start a process
   """
   
   def task():
       print("this is sub process")
       print("sub process id %s" % os.getpid())
   
   
   if __name__ == '__main__':
       # ######Note that the code to start the process must be placed below --- main --- judgment
       #  Instantiate a process object and specify what it does with a function
       p = Process(target=task)
       p.start() # Send a message to the operating system to start the process
       print("this is parent process")
       print("parent process is: %s" % os.getpid())
       print("over")

2. Import the Process class in multiprocess, inherit it, override the run method, put the task to be executed into run, and start the process will automatically execute the function

   from multiprocessing import Process
   import os
   
   
   class Downloader(Process):
   
       # def __init__(self,url,size,name):
       #     super().__init__()
       #     self.url = url
       #     self.size = size
       #     self.name = name
   
       def run(self):
           print(os.getpid())
           pass
   
   if __name__ == '__main__':
       m = Downloader()
       m.start()
       print("parent over",os.getpid())

join function (emphasis)

Priority of the child process is raised so that when the child process finishes executing, the parent process executes

from multiprocessing import Process
import time
def task1(name):
    for i in range(10000):
        print("%s run" % name)

def task2(name):
    for i in range(100):
        print("%s run" % name)

if __name__ == '__main__': # args is a parameter passed to a child process and must be a tuple
    p1 = Process(target=task1,args=("p1",))
    p1.start()  # Send instructions to the operating system
    # p1.join()   # Let the main process wait for the child process to finish executing before continuing

    p2 = Process(target=task2,args=("p2",))
    p2.start()  # Send instructions to the operating system

    p2.join()  # Let the main process wait for the child process to finish executing before continuing
    p1.join()


    #The effect to be achieved is that the two subprocesses must be executed concurrently and the over run must be executed after all tasks have been executed
    print("over")

Common properties of process objects (Understanding)

 # p.daemon #Daemon
    # p.join()
    # print(p.exitcode) # Getting the exit code of a process is the value passed in from the exit() function
    # print(p.is_alive())  # Check to see if the process is alive
    # print("zi",p.pid) # Get process id
    # print(os.getpid()) #Get the number of the current process
    # p.terminate()  #Terminating a process like strat does not terminate immediately because the operating system has a lot to do  
    # print(p.is_alive()) to determine if a process is alive

Zombie and Orphan Processes

An orphan process When the parent process has ended and the child process is still running, it is called an orphan process. It is especially necessary and has no adverse effects.

The zombie process When a process has ended, but it still has some data, which is now called the zombie process

In linux, there is a mechanism whereby a parent process can get some data from a child process at any time

When the child process task finishes executing, it does end, but some data remains to allow the parent process to get this information

waitpid can be called in linux to completely clean up residual information from child processes

python has encapsulated operations to handle zombie processes, so you don't need to worry about them