Thread,Process,Line

In the programmer's life, we must encounter the problem of too long calculation time, at this time, it is necessary to improve the efficiency of the algorithm.There are several methods: algorithm optimization, the power to stimulate CPUGPU, and so on.
I'm now sharing my thoughts on python's multi-threaded, multi-process approach to boosting computer performance.

Thread

from threading import Thread

Any optimization method has its own limitations, python3 does not work well in most cases due to the GIL locking mechanism.But for network programming, multithreading is a very good optimization method!Attention to the above sentence!!!!!!!!!!!
Opening Task Manager during multithreaded use can see a large amount of memory consumption.
How to use:

temp=Thread(target=function,args)
temp.start()

Process

from multiprocessing import  Process

Multi-threaded is a table surrounded by several people, while multi-threaded is a table surrounded by many people.Obviously, multiple processes depend on the performance of the computer CPU, that is, the number of cores.When you open Task Manager, you can see that the CPU is in 100% state.
How to use:

temp=Process(target=function,args)
temp.start()

Line

Common sequential execution of code

Compare Three

To more intuitively understand the differences and advantages of the three, for example, feel the differences from the results.

# Three Test Functions
#(1) Define CPU-intensive computing functions
def count(x,y):
    #Make 1.5 million calculations with the program
    c=0
    while c<500000:
        c+=1
        x+=1
        y+=1

#(2) Define IO-intensive file read-write functions
def write():
    f=open('test.txt','w')
    for x in range(500000):
        f.write("testwrite\n")
    f.close()
def read():
    f=open('test.txt','r')
    f.readlines()
    f.close()
def io():
    write()
    read()
#(3) Define the network request function
#head
head={}
head['User-Agent']=r'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/60.0.3112.101 Safari/537.36'
#url
url=r"http://www.tieba.com"
def http_request():
    try:
        webPage=requests.get(url,head)
        html=webPage.text
        return {"context":html}
    except Exception as e:
        return {"error":e}

Be aware of the significance of CPU-intensive, IO-intensive, and network operations, which correspond to most project needs.

To reduce the amount of code and define test functions, you can understand the idea of generic programming.

#(x) Define test functions

def test(testname=None,number=None,way=None,function=None,args=()):
    counts=[]
    t=time.time()
    #Guarantee defaults
    if testname==None:
        testname='Thread'
    if number==None:
        number=10
    if way==None:
        way=Thread
    if function==None:
        def die():
            i=1000
            return i
        function=die
        pass
    #Open multithreaded or multiprocess
    for x in range(number):
        temp=way(target=function,args=args)
        counts.append(temp)
        temp.start()
    #Wait for the algorithm to stop, note that this is a bad way to force kill
    e = counts.__len__()
    while True:
        for a in counts:
            if not a.is_alive():
                e-=1
        if e<=0:
            break
    #Print out algorithm run time
    print("%s %s :    %s"%(way.__name__,testname,(time.time()-t)))

Readiness is complete and testing begins

#(5) Testing the time required to perform IO-intensive, CPU-intensive, and network request-intensive operations linearly
if __name__=='__main__':
    #IO intensive operations
    t=time.time()
    for x in range(50):
        io()
    print("line IO:    %s"%(time.time()-t))
    #CPU intensive operation
    t=time.time()
    for x in range(50):
        count(1,1)
    print("line CPU:    %s"%(time.time()-t))
    #Network Request Intensive Operations
    t=time.time()
    for x in range(50):
        http_request()
    print("line http Request:    %s"%(time.time()-t))
    pass

#(6) Testing the time required for multi-threaded IO-intensive operations, CPU-intensive operations, and network request-intensive operations
if __name__=='__main__':
    test('CPU',50,function=count,way=Thread,args=(1,1))
    test('IO',50,function=io,way=Thread)
    test('http request',50,way=Thread,function=http_request)
    pass

#(7) Testing the time required for multi-process IO-intensive operations, CPU-intensive operations, and network request-intensive operations
if __name__=='__main__':
    test("CPU",50,Process,count,(1,1))
    test("IO",50,Process,io)
    test("http request",50,way=Process,function=http_request)
    os.remove("test.txt")
    pass

Result comparison

line IO:               19.93960452079773
line CPU:               4.375457525253296
line http Request:      17.290654182434082
Thread IO :             19.752081871032715
Thread CPU :            4.031674861907959
Thread http request :   1.578291654586792
Process IO :            6.2662858963012695
Process CPU :           3.2815961837768555
Process http request :  8.39150881767273

Clearly, you can see the advantages of python multithreading in network requests, as well as the characteristics of multiprocesses in IO and CPU.Materialistic dialectical thinking tells us to analyze a specific problem in a specific way~~So after learning the above ways, it will be interesting to choose a reasonable solution based on the needs of the project!

D.