demand

When python is multithreaded, the sub thread is abnormal, and the main thread still executes downward normally, which obviously does not comply with the engineering code. In order to solve the problem of sub thread exceptions, today's content is added: capture sub thread exceptions.

problem

Look directly at the code:

def thread_text(i):
    time.sleep(i)
    raise Exception(u'error')
    
def main():
    t = threading.Thread(target=thread_text, args=(1,))
    t.start()
    t.join()
    print(u'end')
if __name__ =='__main__':
    main()

This is a simple test. An exception is returned in the child thread. The output is as follows:

end

Exception in thread Thread-1:
Traceback (most recent call last):
  File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.6/threading.py", line 864, in run
    self._target(*self._args, **self._kwargs)
  File "/usercode/file.py", line 8, in thread_text
    raise Exception(u'error')
Exception: error

It can be seen that although the sub thread reported an exception, the main method still printed 'end' normally, which is obviously not what we expected. Next, we began to solve it

queue introduction

Queue is a built-in module of Python 3. It creates a stack queue to handle multi-threaded communication. The main methods are as follows:

• queue.Queue(maxsize=0)
  First in first out (FIFO) queue,
  The input parameter maxsize is an integer used to set the maximum length of the queue. Once the queue reaches the upper limit, the inserted data will be blocked until there is data out of the queue. If maxsize is set to less than or equal to zero, there is no limit on the length of the queue.

import queue
q = queue.Queue()  # Create Queue queue
for i in range(3):
    q.put(i)  # Insert 0, 1, and 2 elements into the queue
for i in range(3):
    print(q.get())  # Take out the inserted elements from the queue in turn, and the output order of data elements is 0, 1 and 2

• queue.LifoQueue(maxsize=0)
  Last in first out (LIFO) queue. The data that enters the queue last has the priority of leaving the queue, just like the stack.

• PriorityQueue(maxsize=0)
  Priority queue: compare the size of each data in the queue. The data with the smallest value has the priority out of the queue. Data is usually inserted in the form of tuples. The typical form is (priority_number, data). If the data in the queue is not comparable, the data will be wrapped in a class, ignoring the data value and only comparing the priority number.

We only need the priority Queue here. After all, we just need to know whether there is an exception. The methods are as follows:

• Queue.qsize()
  Returns the number of data elements in the queue.
• Queue.empty()
  Returns True if the queue is empty, False otherwise.
• Queue.full()
  If the number of elements in the queue reaches the upper limit, return True; otherwise, return False.
• Queue.put(item, block=True, timeout=None)
  item, the data element put into the queue.
  Block, when the number of elements in the queue reaches the upper limit and continues to put data in: if block=False, a queue. Full exception is directly thrown; If block=True and timeout=None, wait until there is data out of the queue; If block=True, timeout=N, and N is a positive integer, wait for N seconds. If there is no place to put data in the queue, a queue.Full exception will be thrown.
  Timeout to set the timeout.
• Queue.put_nowait(item)
  Equivalent to Queue.put(item, block=False). When the number of elements in the queue reaches the upper limit and continue to put data, the queue.Full exception is directly thrown.
• Queue.get(block=True, timeout=None)
  Take the data from the queue and return the data content.
  Block, when no data element in the queue continues to fetch data: if block=False, a queue.Empty exception is directly thrown; If block=True and timeout=None, wait until there is data in the queue, and the data can be taken out; If block=True, timeout=N, and N is a positive integer, wait for N seconds. If there is no data in the queue, a queue.Empty exception will be thrown.
  Timeout to set the timeout.
• Queue.get_nowait()
  It is equivalent to Queue.get(block=False)block. When no data element in the queue continues to fetch data, the queue.Empty exception is directly thrown.

We mainly use queue.get/queue.put/queue.empty
The idea is: when the child thread is abnormal, add a piece of data to the queue, which can be any data. The main method detects whether the queue is empty. If it is empty, it indicates that the child thread has no exception. On the contrary, it is abnormal, and then determine whether the child thread has completed execution by detecting the number of active threads.

Detect thread related code

threading.enumerate()  # list the currently active threads, including the main thread
threading.active_count()  # int current number of active threads, including the main thread

The output is as follows

[<_MainThread(MainThread, started 8628338176)>]
1

Well, the ideas and modules used have been introduced. Go directly to the code

import queue
import threading
import sys
def thread_text(q, i):
    try:
        time.sleep(i)
        raise Exception(u'error')
    except:
        q.put(sys.exc_info())
    # time.sleep(1)
    
if __name__ =='__main__':
    q = queue.Queue()
    t = threading.Thread(target=thread_text, args=(q, 1))
    t.start()
    while True:
        
        if q.empty():
            if threading.active_count() == 1:
                break
        else:
            print(q.get())
            raise Exception(u'main error')
    print(u'end')

The output is as follows:

(<class 'NameError'>, NameError("name 'time' is not defined",), <traceback object at 0x7f7a1b2625c8>)

		
Traceback (most recent call last):
  File "/usercode/file.py", line 25, in <module>
    raise Exception(u'main error')
Exception: main error

You can see that instead of outputting 'end', it returns' main error ', indicating that the main thread has successfully captured the child thread,
Here, I queue.put the exc of sys module in the sub thread_ Info (), which makes it easier to check and correct errors.

Here the problem is solved.

END