background

After receiving a user request, the backend needs to carry the trace in the user request when requesting other third-party interfaces_ ID to sort out the request link and location problems later.

The original code is developed based on the Django framework of Python. Different user requests are thread isolated, and different user requests are traced_ ID is naturally different. In addition, to facilitate reading trace_id so it is stored as a global variable. Based on these two points, threading is used in the original code Local () to store trace_id.

In order to speed up the request speed, it is necessary to request the third-party interface concurrently through multiple threads (PS: you can also use coprocessors, but the original interface call encapsulation does not support asynchrony, so the workload is too heavy to change). However, because threading.local() is thread isolated, Therefore, the child thread cannot get the trace_id of the parent thread (request thread) at all, and then an error will appear: 'U thread._local' object has no attribute 'trace_id'.

After a simple search, it is found that InheritableThreadLocal comes with Java (unlike ThreadLocal, the properties of variables created with InheritableThreadLocal can be inherited by child threads to continue to be used). However, it is unclear why Python does not (PS: in fact, it is also available on stackoverflow) put questions to , but I don't know why I didn't answer), so I made a simple implementation.

code implementation

#!/usr/bin/env python3
# -*- coding: utf-8 -*-

import copy
from _threading_local import local
from threading import current_thread, Thread


class InheritableThread(Thread):
    def __init__(self, *args, **kwargs):
        self.parent = current_thread()
        Thread.__init__(self, *args, **kwargs)


def copy_parent_attribute(self):
    """
    :raise AttributeError: parent may not exist
    """

    parent = current_thread().parent
    parent_local = self._local__impl.dicts.get(id(parent))
    if parent_local is not None:
        parent_dicts = parent_local[1]
        for key, value in parent_dicts.items():
            self.__setattr__(key, copy.deepcopy(value))


class InheritableLocal(local):
    """
    Please use InheritableThread to create threads, if you want your son threads can inherit parent threads' local
    """

    def __new__(cls, *args, **kwargs):
        self = local.__new__(cls, *args, **kwargs)
        try:
            copy_parent_attribute(self)
        except AttributeError:
            # Some threads may not be created by InheritableThread
            pass
        return self

    def __getattribute__(self, item):
        try:
            return local.__getattribute__(self, item)
        except AttributeError as e:
            try:
                copy_parent_attribute(self)
                return local.__getattribute__(self, item)
            except AttributeError:
                raise e

    def __delattr__(self, item):
        try:
            return local.__delattr__(self, item)
        except AttributeError as e:
            try:
                copy_parent_attribute(self)
                return local.__delattr__(self, item)
            except AttributeError:
                raise e

1. In order to access the parent thread in the child thread, you need to record the parent thread when creating the child thread: self parent = current_ thread()
2. When the child thread builds InheritableLocal(), it copies the attributes of InheritableLocal() in the parent thread according to the recorded parent thread, because InheritableLocal() is based on threading Local (), so you need to be clear about threading Structure of local()

threading. Structure of local():

usage method

thread_ctx = InheritableLocal()
t = InheritableThread(target=function, args=(arg1, arg2, arg3))
t.start()
t.join()

1. Use InheritableLocal() instead of threading Local () to create a thread environment variable without other side effects
2. Using InheritableThread() instead of Thread() to create a thread also has no side effects

Then the created child thread can inherit the environment variables of the parent thread. Of course, the environment variables between the two threads are isolated from each other. Inheritance will only occur when the child thread is created.

This article is composed of blog one article multi posting platform OpenWrite release!