1, Code optimization principle

(collated from) https://mp.weixin.qq.com/s/zBjfGihZYY6XcVdGkyQfww )

Before introducing the specific optimization rules, summarize three basic principles.

1. Don't optimize too early

The premise of code optimization is that the code can work normally. Always remember that it's much easier to make the right program faster than to make the fast program right.

2. Weigh the cost of optimization

There is no perfect program. The usual optimization strategy is time for space and space for time.

3. Do not optimize unimportant parts

If you optimize every part of the code, the end result is that you will write an extremely obscure program. Therefore, only those parts that take a long time to optimize, usually internal loops.

2, Code optimization suggestions

1. Avoid global variables

# The writing method is not recommended. The running time is 27.04s
import math
size = 10000

for x in range(size):
	for y in range(size):
		z = math.sqrt(x) + math.sqrt(y)

This code defines a global variable size. We try to encapsulate this code into a function and look at the running time.

# Recommended writing operation time: 22.01s
import math

def main():

	size = 10000
	for x in range(size):
		for y in range(size):
			z = math.sqrt(x) + math.sqrt(y)

main()

Program running time is shortened!

2. Avoid

2.1 avoid module and function attribute access

For example, math in this example Sqrt() and result append(). Every visit Specific methods are triggered, such as__ getattribute__ () and__ getattr__ (), these methods will perform dictionary related operations, which will bring additional time overhead.

# Running time: 15.92s
import math

def computeSqrt(size:int):
	result = []
	for i in range(size):
		result.append(math.sqrt(i))
	return result

def main():
	size = 10000
	for _ in range(size):
		result = computeSqrt(size)

main()

It can be seen that the access of module and function attributes is written in the for loop. Therefore, the efficiency of program operation can be improved. First, we use the from import statement to eliminate module attribute access.

# Running time: 13.96s
from math import sqrt

def computeSqrt(size: int):
    result = []
    for i in range(size):
        result.append(sqrt(i))  # Avoid math Use of sqrt
    return result

def main():
    size = 10000
    for _ in range(size):
        result = computeSqrt(size)

main()

You can see that the running time is shortened. Further, we convert sqrt from global variable to local variable according to the first rule.

# Running time: 12.62s
import math

def computeSqrt(size: int):
    result = []
    sqrt = math.sqrt  # Assign to local variable
    for i in range(size):
        result.append(sqrt(i))  # Avoid math Use of sqrt
    return result

def main():
    size = 10000
    for _ in range(size):
        result = computeSqrt(size)

main()

Code run time is further reduced. But there's more in the for loop The existence of, i.e. result Append(), let's optimize this part again.

# Running time: 8.10s
import math

def computeSqrt(size: int):
    result = []
    append = result.append
    sqrt = math.sqrt    # Assign to local variable
    for i in range(size):
        append(sqrt(i))  # Avoid result Append and math Use of sqrt
    return result

def main():
    size = 10000
    for _ in range(size):
        result = computeSqrt(size)

main()

The running time is greatly reduced. Therefore, in terms of program running time, this method is the best. However, it should be understood that this writing method sacrifices part of the simplicity and readability of the program. Therefore, when using, you can consider whether this extreme optimization is necessary.

2.2 avoid intra class attribute access

Avoid The same principle applies to in class properties, accessing self_ Value is slower than accessing a local variable. If you need to frequently access an attribute in a class, you can assign it to a local variable first. For example:

# Frequently visited self_ value.  Code time: 11.77s
import math
from typing import List

class DemoClass:
    def __init__(self, value: int):
        self._value = value
    
    def computeSqrt(self, size: int) -> List[float]:
        result = []
        append = result.append
        sqrt = math.sqrt
        for _ in range(size):
            append(sqrt(self._value))
        return result

def main():
    size = 10000
    for _ in range(size):
        demo_instance = DemoClass(size)
        result = demo_instance.computeSqrt(size)

main()

After optimization

# Recommended writing. Code time: 8.07s
import math
from typing import List

class DemoClass:
    def __init__(self, value: int):
        self._value = value
    
    def computeSqrt(self, size: int) -> List[float]:
        result = []
        append = result.append
        sqrt = math.sqrt
        value = self._value
        for _ in range(size):
            append(sqrt(value))  # Avoid self_ Use of value
        return result

def main():
    size = 10000
    for _ in range(size):
        demo_instance = DemoClass(size)
        demo_instance.computeSqrt(size)

main()

Code running time is greatly reduced.

3. Splice strings with join instead of+

String in python is an immutable object, so when using a+b, another memory space will be applied, and a and b will be copied into the newly applied memory space respectively. Therefore, if n strings need to be spliced, n-1 intermediate results will be generated. Each intermediate result will apply for and copy memory, which will greatly affect the operation efficiency. When using join() to splice strings, you will first calculate the total memory space to be applied for, then apply for it in place at one time, and copy each string to be spliced into the memory.

4. Use the short circuit characteristics of if

The short circuit characteristic of if refers to:

1. For if a and b, when a is False, it directly returns False and b is no longer calculated.
2. For if a or b, when a is True, it directly returns True and b is no longer calculated.

Therefore, using this feature, for or statements, statements with a higher probability of True should be placed in front, while for and statements, statements with a higher probability of False should be placed in front.

5. Cycle optimization

1. The for loop is better than the while loop
2. An implicit loop replaces a display loop. Example sum(range(a))

6. Select the appropriate data structure

python's built-in data structures such as STR, tuple, list, set and dict are implemented in c, which is very fast. It is almost impossible for our newly implemented data structure to achieve the built-in speed in performance. Therefore, it is very important to understand the characteristics of these data structures and use them well.
For example: list is a dynamic array, which will be pre allocated memory space during initialization. When you need to add elements to it, you need to apply for a larger space, then copy the original elements of the list, destroy the original space, and finally insert new elements. If you add and delete frequently and add or delete a large number of elements, the efficiency of the list is not high. At this point, you can use other data structures, such as collections Deque dual end queue has the characteristics of stack and queue at the same time, and can insert and delete with O(1) complexity at both ends.

7. Exchange element values

a,b = b,a #No intermediate variables need to be created