Hot update of Python in game


Hot update is to add new functions to the game or fix bug code without restarting the server. The fast iteration speed of game update has given birth to the demand for hotter technology. In the game projects I have experienced, whether it is the server or the client, the version update is around hotter update. Especially now the game is prone to several G's, and it is unrealistic for players to download complete packages every time. Casual games must support hotter. Let's talk about the processing of Python hot update on the client side.


1. Standard import

As we all know, python provides import, which can import a standard Python module, load the module into memory and add it to sys.modules. But many import modules the first mock exam, but only one module is imported into the current Local namespace, that is, a module will not be loaded repeatedly, so it is not possible to rely on this feature. This road is impassable. Please change your mind.

2.reload function

The reload() function can reload the imported module, so it seems that Python code can be hot updated. However, Python's native reload function is too simple to support the hot update requirements of the game. There are several main reasons: the reload reloaded module will not replace the old version of the module, that is, the referenced old module cannot be updated. Similarly, because the reference of the old object cannot be used, the module referenced by the from... Import... Method cannot be updated after reloas(m), Class and its derived class instance objects still use the old class definition. At the same time, when the module fails to load, there is no rollback mechanism, resulting in the need to re import the module. Therefore, combined with the hot update requirements of the game, customize the appropriate reload. The purpose of the new custom reload is to enable the program to dynamically load the changed code without the end of the original program. We mainly want to achieve the following two points: improve development efficiency and repair emergency bugs without restarting the game


The core requirement of hot update is to let the python interpreter execute the latest code and ensure that other related modules will not have problems. For the refresh function, the method defined in the class is easier to implement, but for the variables defined in the refresh module, the variables defined in the class, and the newly added member variables, there needs to be a unified convention. Therefore, in the process of implementing hot update, we need to consider code update and data update. Here are the features of the new reload:

-1. Update code definition

(function/method/static_method/class_method) the data is not updated (all types except code definitions are regarded as data) and reload is agreed in the module_ Module interface, and reload is agreed in class_ Class interface. In these two interfaces, data updates are handled manually. There are more conventions and interfaces to be completed to replace the content of function objects

# Update the contents of the old function object with the contents of the new function object, and keep the address of the function object unchanged  
def update_function(oldobj, newobj, depth=0):  
    setattr(oldobj, "func_code", newobj.func_code)  
    setattr(oldobj, "func_defaults", newobj.func_defaults)  
    setattr(oldobj, "func_doc", newobj.func_doc)

2. Contents of replacement class

# Update the content of the old class with the content of the new class and keep the address of the old class unchanged  
def _update_new_style_class(oldobj, newobj, depth):  
    handlers = get_valid_handlers()  
    for k, v in newobj.__dict__.iteritems():  
        # If the new key is not in the old class, add it  
        if k not in oldobj.__dict__:  
            setattr(oldobj, k, v)  
            _log("[A] %s : %s"%(k, _S(v)), depth)  
        oldv = oldobj.__dict__[k]  
        # If the key object type is different between the old and new classes, the object of the old class is retained  
        if type(oldv) != type(v):  
            _log("[RD] %s : %s"%(k, _S(oldv)), depth)  
        # Update objects that currently support updates  
        v_type = type(v)  
        handler = handlers.get(v_type)  
        if handler:  
            _log("[U] %s : %s"%(k, _S(v)), depth)  
            handler(oldv, v, depth + 1)  
            # Since the content of oldv is changed directly, there is no need to set attr.  
            _log("[RC] %s : %s : %s"%(k, type(oldv), _S(oldv)), depth)  
    # Call the agreed reload_class interface, which handles the replacement logic of class variables  
    object_list = gc.get_referrers(oldobj)  
    for obj in object_list:  
        # Only those of the same type are instance objects of the class  
        if obj.__class__.__name__ != oldobj.__name__:  
        if hasattr(obj, "x_reload_class"):  


def _update_staticmethod(oldobj, newobj, depth):  
    # A staticmethod object, its SM__ get__ (object) is the function object  
    oldfunc = oldobj.__get__(object)  
    newfunc = newobj.__get__(object)  
    update_function(oldfunc, newfunc, depth)

-4.classmethod class method

def _update_classmethod(oldobj, newobj, depth):  
    oldfunc = oldobj.__get__(object).im_func  
    newfunc = newobj.__get__(object).im_func  
    update_function(oldfunc, newfunc, depth)

The module updates are similar, so they are not pasted one by one. They are only improved on the basis of the original reload. For the module hot update, a reload is also agreed_ Module interface, you can customize the update of data. Add some use cases below:

def x_reload_class(self):  
    """ After hot update, each instance of the re object executes this function 
    Since the replacement of new and old objects will not call the constructor again, it is necessary to perform initialization logic on hot updated class objects 
    Handle the repair of new and old variables and the repair of function execution environment 
    self._new_var = 5000    # Initialization of new variables  
    self.runLogic()      # Newly repaired logic

Just made some customization on the basic reload module to make the hot update more suitable for the development rhythm of the game, rather than the simple and violent reload module

Keywords: Python Operation & Maintenance Back-end server

Added by shergar1983 on Sat, 23 Oct 2021 11:12:35 +0300