Concurrent and Serial
What is serial?
Serial means that a program is executed from top to bottom, one line at a time, and the current task must be executed before the next program can be executed.
- Inefficient program execution
- Occupy memory resources
What is concurrency?
Concurrency means that multiple programs can be executed simultaneously (essentially fast switching execution between different processes and saving program state). The summary is switching + saving
Both serial and concurrent are the way programs handle tasks
Why use concurrency?
- Multiple programs can be executed at the same time, which improves productivity
- Increase the utilization of memory resources
Ways to achieve concurrency
What is a process?
A process is a running program and is the basic unit for operating system scheduling and resource allocation (a process is a resource unit)
How did the process come about?
When a program is read from the hard disk into memory, a process occurs
What is multi-process?
Multiple programs are read into memory and executed at the same time, that is, multiple processes
Processes come from the operating system, which schedules and allocates resources
The principle of multi-process implementation is essentially the principle of operating system scheduling processes
Multichannel Technology (Key)
Partition memory into multiple zones, place multiple programs in memory, and isolate memory zones from each other (physical isolation to ensure data security)
Time Reuse: (Switch+Save)
Operating system switches execution between processes
There are two scenarios for switching tasks:
- Automatically switch when a process encounters an IO operation
- Force switching when a task execution time exceeds the threshold
The state must be saved before switching for subsequent resumption of execution
Important concepts in concurrent programming (focus)
Serial: Programs are executed top-down, one by one in sequence
Concurrency: Quick switching between different people
Parallel: True simultaneous execution of multiple programs requires a multi-core CPU
All three are ways to handle tasks
Blocking: A state in which a program encounters IO operations and cannot continue executing code
Non-blocking: A state in which a program does not encounter IO operations
Three states of the process
- Blocking: Release IO operation, convert to ready
- Run: Convert to blocking when IO operation and run timeout occurs; Release IO operation is converted to ready
- Ready: CPU call, converted to run
There are two ways to use multiprocesses in python:
Import the Process class in multiprocessing, instantiate it, and specify the task program target to execute
import os from multiprocessing import Process """ Process Represents a process //Why start a process """ def task(): print("this is sub process") print("sub process id %s" % os.getpid()) if __name__ == '__main__': # ######Note that the code to start the process must be placed below --- main --- judgment # Instantiate a process object and specify what it does with a function p = Process(target=task) p.start() # Send a message to the operating system to start the process print("this is parent process") print("parent process is: %s" % os.getpid()) print("over")
Import the Process class in multiprocess, inherit it, override the run method, put the task to be executed into run, and start the process will automatically execute the function
from multiprocessing import Process import os class Downloader(Process): # def __init__(self,url,size,name): # super().__init__() # self.url = url # self.size = size # self.name = name def run(self): print(os.getpid()) pass if __name__ == '__main__': m = Downloader() m.start() print("parent over",os.getpid())
join function (emphasis)
Priority of the child process is raised so that when the child process finishes executing, the parent process executes
from multiprocessing import Process import time def task1(name): for i in range(10000): print("%s run" % name) def task2(name): for i in range(100): print("%s run" % name) if __name__ == '__main__': # args is a parameter passed to a child process and must be a tuple p1 = Process(target=task1,args=("p1",)) p1.start() # Send instructions to the operating system # p1.join() # Let the main process wait for the child process to finish executing before continuing p2 = Process(target=task2,args=("p2",)) p2.start() # Send instructions to the operating system p2.join() # Let the main process wait for the child process to finish executing before continuing p1.join() #The effect to be achieved is that the two subprocesses must be executed concurrently and the over run must be executed after all tasks have been executed print("over")
Common properties of process objects (Understanding)
# p.daemon #Daemon # p.join() # print(p.exitcode) # Getting the exit code of a process is the value passed in from the exit() function # print(p.is_alive()) # Check to see if the process is alive # print("zi",p.pid) # Get process id # print(os.getpid()) #Get the number of the current process # p.terminate() #Terminating a process like strat does not terminate immediately because the operating system has a lot to do # print(p.is_alive()) to determine if a process is alive
Zombie and Orphan Processes
An orphan process When the parent process has ended and the child process is still running, it is called an orphan process. It is especially necessary and has no adverse effects.
The zombie process When a process has ended, but it still has some data, which is now called the zombie process
In linux, there is a mechanism whereby a parent process can get some data from a child process at any time
When the child process task finishes executing, it does end, but some data remains to allow the parent process to get this information
waitpid can be called in linux to completely clean up residual information from child processes
python has encapsulated operations to handle zombie processes, so you don't need to worry about them