XLua source learning: calling CS in Lua

In the process of using lua code development, a very important part is the call to C# code. Next, we will understand its implementation in combination with source code analysis:

In lua, you can call C#'s code using code such as:

CS.UnityEngine.Debug.Log('hello world')

CS is a global Table, so CS UnityEngine can be regarded as querying the value named UnityEngine in a Table named CS. When creating a virtual machine, that is, LuaEnv, the following code will be called to initialize the CS Table:

DoString(init_xlua, "Init");

The initialization code is intercepted as follows:

local metatable = {}
local rawget = rawget
local setmetatable = setmetatable
local import_type = xlua.import_type
local import_generic_type = xlua.import_generic_type
local load_assembly = xlua.load_assembly

function metatable:__index(key)
     --obtain key by".fqn"Value of
    local fqn = rawget(self, ".fqn")
    --Splicing".fqn"And the value of this call key
    fqn = ((fqn and fqn .. ".") or "") .. key
    --query CS type
    local obj = import_type(fqn)

    if obj == nil then
        -- It might be an assembly, so we load it too.
        obj = {[".fqn"] = fqn}
        setmetatable(obj, metatable)
    elseif obj == true then
        return rawget(self, key)
    end

    -- Cache this lookup
    rawset(self, key, obj)
    return obj
end

CS = CS or {}
setmetatable(CS, metatable)

This section describes the CS table__ The implementation of the index meta method will call this function when we call a non-existent field in the CS table.

We use CS UnityEngine. Take debug as an example to explain how this function is executed:

1) first, CS is a global empty table, and access CS UnityEngine calls this function directly because the UnityEngine field does not exist. The first line obtains the value with key ". fqn", and fqn is empty, so fqn=UnityEngine at the end of the second line

2) call import_type to query the lua table corresponding to the type UnityEngine.

3) judge the return value obj. UnityEngine is obviously not a type, so take the branch of = = nil, create a table and set ". fqn"="UnityEngine", and set this metatable as the meta table of this table

4) set obj in CS table to "UnityEngine".

5) next, visit CS UnityEngine.Debug is equivalent to from CS Access the debug field in unityengine. Since there is no debug field, it will also be called__ In the index function, fqn will have a value at this time. After the end of the second line, the value of fqn is "UnityEngine.Debug"

6) import at this time_ When the type function queries this type, it will have a value. It will take the branch of obj==true and import_ The type function actually has CS UnityEngine. After the debug value is set, the lua table is returned through the rawget function

The above should clearly explain CS XXX is written to implement the process of calling C# code, while import_ How to find C# type in type is described below:

(Note: the following code contains a large number of parts about LuaAPI. For further understanding, please refer to the contents after chapter 27 of lua programming (4th Edition))

        import_type = xlua.import_type. The xlua is a global table, which is declared in the C code at build / xlua c:

xlua.c: 
LUA_API void luaopen_xlua(lua_State *L) {
	luaL_openlibs(L);
	
#if LUA_VERSION_NUM >= 503
	luaL_newlib(L, xlualib);
	lua_setglobal(L, "xlua");
#else
	luaL_register(L, "xlua", xlualib);
    lua_pop(L, 1);
#endif
}

This code belongs to xlua DLL, which will be called when creating luaenv to set a global table xlua. At the same time, luaenv will register import when it is created_ The type function, when xlua. XML is called import_type will be called on the corresponding C# delegate:

ObjectTranslator.cs: 
public void OpenLib(RealStatePtr L) {
    if (0 != LuaAPI.xlua_getglobal(L, "xlua")){  throw new Exception("call xlua_getglobal fail!" + LuaAPI.lua_tostring(L, -1));} 
    LuaAPI.xlua_pushasciistring(L, "import_type");
    LuaAPI.lua_pushstdcallcfunction(L,importTypeFunction);
    LuaAPI.lua_rawset(L, -3); 
    ...
}

Next, find the importTypeFunction delegate, and you will find that it is finally called to this function:

StaticLuaCallbacks.cs: 
public static int ImportType(RealStatePtr L)
{
    try
    {
        ObjectTranslator translator = ObjectTranslatorPool.Instance.Find(L);
        //Class name to query
        string className = LuaAPI.lua_tostring(L, 1);
        //Find C# corresponding type (the table corresponding to Lua has not been found here)
        Type type = translator.FindType(className);
        if (type != null)
        {
            //This sentence looks up the lua table corresponding to the Type
            if (translator.GetTypeId(L, type) >= 0)
            {
                LuaAPI.lua_pushboolean(L, true);
            }
            else
            {
                return LuaAPI.luaL_error(L, "can not load type " + type);
            }
        }
        else
        {
            LuaAPI.lua_pushnil(L);
        }
             
        return 1;
    }
}

Through the above code, you will find that, If the corresponding type is not found, it is directly to the lua virtual stack (the main component of communication between lua and other languages), push a nil. Otherwise, go to find the corresponding lua table. If there is one, push a true, otherwise push an error code. Finally, return 1 means that the number of returned parameters is 1. In a word, the obj returned by the xlua.import_type function in the above lua code can only be true/nil / error code, and there is no special error code Handled.

And translator The internal details of gettypeid are the key. It contains two ways for x lua to call C# Code: one is to generate adaptation code, and the other is to call through reflection:

        

ObjectTranslator.cs
internal int getTypeId(RealStatePtr L, Type type, out bool is_first, LOGLEVEL log_level = LOGLEVEL.WARN)
{
    int type_id;
    is_first = false;
    //Query whether there is Lua table corresponding to Type in the cache. If so, it will be returned directly
    if (!typeIdMap.TryGetValue(type, out type_id)) // no reference
    {
        ...
        is_first = true;
        Type alias_type = null;
        aliasCfg.TryGetValue(type, out alias_type);
        //Check the meta table corresponding to Type from the registry
        LuaAPI.luaL_getmetatable(L, alias_type == null ? type.FullName : alias_type.FullName);
        //If the meta table is empty, follow the relevant registration logic
        if (LuaAPI.lua_isnil(L, -1)) //no meta yet, try to use reflection meta
        {
            LuaAPI.lua_pop(L, 1);
            //Here we will check whether to use reflection or generate adaptation code
            if (TryDelayWrapLoader(L, alias_type == null ? type : alias_type))
            {
                LuaAPI.luaL_getmetatable(L, alias_type == null ? type.FullName : alias_type.FullName);
            }
            else
            {
                throw new Exception("Fatal: can not load metatable of type:" + type);
            }
        }
        //Circular dependency. It depends on its own class. For example, it has a static readonly object of its own type.
        if (typeIdMap.TryGetValue(type, out type_id))
        {
            LuaAPI.lua_pop(L, 1);
        }
        else
        {
            ...
            LuaAPI.lua_pop(L, 1);
            //Cache type and its corresponding table in lua
            typeIdMap.Add(type, type_id);
        }
    }
    return type_id;
}

public bool TryDelayWrapLoader(RealStatePtr L, Type type)
{
    if (loaded_types.ContainsKey(type)) return true;
    loaded_types.Add(type, true);
    LuaAPI.luaL_newmetatable(L, type.FullName); 
    LuaAPI.lua_pop(L, 1);
    Action<RealStatePtr> loader;
    int top = LuaAPI.lua_gettop(L);.
    //This delayWrap dictionary is in the code generated by Xlua (in the Xlua_gen_initiator_register_class)
    //When instantiating), register each type of wrap in it
    if (delayWrap.TryGetValue(type, out loader))
    {
        delayWrap.Remove(type);
        //Register classes, methods, fields, members, etc
        loader(L);
    }
    //This is the logic of reflection
    else
    {
        ...
        //Register classes, methods, fields, members, etc. with reflection
        Utils.ReflectionWrap(L, type, privateAccessibleFlags.Contains(type));
        ...
    }
    ...
    ...
    return true;
}

First, let's see how the generated adaptation code is executed, that is, the execution of loader(L) in the above code. Taking Vector3 as an example, the following code is actually executed:

public static void __Register(RealStatePtr L)
{
    ObjectTranslator translator = ObjectTranslatorPool.Instance.Find(L);
	System.Type type = typeof(UnityEngine.Vector3);
	Utils.BeginObjectRegister(type, L, translator, 6, 6, 6, 3);
    //Register some methods related to operators such as + / / = = here
	Utils.RegisterFunc(L, Utils.OBJ_META_IDX, "__add", __AddMeta);
    ...
    Utils.RegisterFunc(L, Utils.OBJ_META_IDX, "__eq", __EqMeta);
   
    //What is registered here is the member function                  
    Utils.RegisterFunc(L, Utils.METHOD_IDX, "Set", _m_Set);
    Utils.RegisterFunc(L, Utils.METHOD_IDX, "GetHashCode", _m_GetHashCode);
    ...
    Utils.RegisterFunc(L, Utils.METHOD_IDX, "Equals", _m_Equals);
    Utils.RegisterFunc(L, Utils.METHOD_IDX, "ToString", _m_ToString);
	
    //What is registered here is the get of member variables   					
    Utils.RegisterFunc(L, Utils.GETTER_IDX, "normalized", _g_get_normalized);
    ...
    Utils.RegisterFunc(L, Utils.GETTER_IDX, "y", _g_get_y);
    Utils.RegisterFunc(L, Utils.GETTER_IDX, "z", _g_get_z);
    
    //The set of the member variable is registered here          
    Utils.RegisterFunc(L, Utils.SETTER_IDX, "x", _s_set_x);
    Utils.RegisterFunc(L, Utils.SETTER_IDX, "y", _s_set_y);
    Utils.RegisterFunc(L, Utils.SETTER_IDX, "z", _s_set_z);
            		
    Utils.EndObjectRegister(type, L, translator, __CSIndexer, __NewIndexer,
		null, null, null);

    Utils.BeginClassRegister(type, L, __CreateInstance, 26, 10, 0);
    //Static functions are registered here 
    Utils.RegisterFunc(L, Utils.CLS_IDX, "Slerp", _m_Slerp_xlua_st_);
    ...  
    Utils.RegisterObject(L, translator, Utils.CLS_IDX, "kEpsilonNormalSqrt", UnityEngine.Vector3.kEpsionNormalSqrt);
    //get and set of static variables are registered here (Vector3 has no set of static variables, so there is no set here)         
    Utils.RegisterFunc(L, Utils.CLS_GETTER_IDX, "zero", _g_get_zero);
    Utils.RegisterFunc(L, Utils.CLS_GETTER_IDX, "one", _g_get_one);
    Utils.RegisterFunc(L, Utils.CLS_GETTER_IDX, "forward", _g_get_forward);
    Utils.RegisterFunc(L, Utils.CLS_GETTER_IDX, "back", _g_get_back);
    Utils.RegisterFunc(L, Utils.CLS_GETTER_IDX, "up", _g_get_up);
    Utils.RegisterFunc(L, Utils.CLS_GETTER_IDX, "down", _g_get_down);
    Utils.RegisterFunc(L, Utils.CLS_GETTER_IDX, "left", _g_get_left);
    Utils.RegisterFunc(L, Utils.CLS_GETTER_IDX, "right", _g_get_right);
    Utils.RegisterFunc(L, Utils.CLS_GETTER_IDX, "positiveInfinity", _g_get_positiveInfinity);
    Utils.RegisterFunc(L, Utils.CLS_GETTER_IDX, "negativeInfinity", _g_get_negativeInfinity);
            			
    Utils.EndClassRegister(type, L, translator);
}

As can be seen from the above code, it is mainly divided into two parts: static and object.

Let's start with the static part:

        Utils. In the beginclassregister function, four lua tables are created, namely CLS_ table,cls_ metatable,cls_getter_table,cls_setter_table is placed at the position of - 4, - 3, - 2, - 1 in the stack (- 1 refers to the top of the stack, - 2 refers to the next position at the top of the stack, and so on, and 1 at the bottom of the stack). cls_table is the lua table corresponding to this type, cls_metatable is the meta table of this lua table, and getter table and Setter table are used to assist in the implementation of _indexand _newindex meta methods.

And utils Registerfunc registers the corresponding methods in these tables:

Utils.cs
public static void RegisterFunc(RealStatePtr L, int idx, string name, LuaCSFunction func)
{
    idx = abs_idx(LuaAPI.lua_gettop(L), idx);
    LuaAPI.xlua_pushasciistring(L, name);
    LuaAPI.lua_pushstdcallcfunction(L, func);
    LuaAPI.lua_rawset(L, idx);
}

Use the code "utils. Registerfunc (L, utils. Cls_idx," SLERP ", _m_Slerp_xlua_st_);" For example, it is equivalent to in CLS_ The SLERP field is registered in table. When vector3.0 is called Called when SLERP_ m_ Slerp_ xlua_ st_ Function.

When everything in the type is registered, in the function utils In endclassregister__ index meta method and__ The newindex meta method is implemented and set in the meta table:

public static void EndClassRegister(Type type, RealStatePtr L, ObjectTranslator translator)
{
    //Get the stack top index and the indexes of the four lua tables
    int top = LuaAPI.lua_gettop(L);
    int cls_idx = abs_idx(top, CLS_IDX);
    int cls_getter_idx = abs_idx(top, CLS_GETTER_IDX);
    int cls_setter_idx = abs_idx(top, CLS_SETTER_IDX);
    int cls_meta_idx = abs_idx(top, CLS_META_IDX);

    //begin cls index
    LuaAPI.xlua_pushasciistring(L, "__index");//Push "_index" into the stack
    LuaAPI.lua_pushvalue(L, cls_getter_idx);//Press in cls_get_table
    LuaAPI.lua_pushvalue(L, cls_idx);//Press in cls_table
    translator.Push(L, type.BaseType());//Press in the address of BaseType
    //Press in storage__ The Key of the table of the index meta method, which stores all types of data__ Index meta method
    LuaAPI.xlua_pushasciistring(L, LuaClassIndexsFieldName);
    //Take the _index set table from the registry (which can be regarded as the place where global variables are stored)
    LuaAPI.lua_rawget(L, LuaIndexes.LUA_REGISTRYINDEX);
    //A closure function CLS will be created here_ Indexer and press it into the stack, associate and pop up the value other than "_index" pressed above as the upper value
    //This closure function implements the__ Function of index meta method
    //After this step is executed, the stack is: "_index"|cls_indexer
    LuaAPI.gen_cls_indexer(L);

    //Press in LuaClassIndexsFieldName
    LuaAPI.xlua_pushasciistring(L, LuaClassIndexsFieldName);
    //Pop up key and press in__ index set table
    LuaAPI.lua_rawget(L, LuaIndexes.LUA_REGISTRYINDEX);//store in lua indexs function tables
    //Press in current type
    translator.Push(L, type);
    //At this time, the position of - 3 is cls_indexer means to copy a copy and press it into the top of the stack
    LuaAPI.lua_pushvalue(L, -3);
    //Put this cls_indexer deposit__ Set table of index,
    LuaAPI.lua_rawset(L, -3);
    //__ index will pop up the set table
    LuaAPI.lua_pop(L, 1);
    //Set cls_metatable[__index] = cls_indexer and pop up keys and values
    LuaAPI.lua_rawset(L, cls_meta_idx);
    //end cls index

    //begin cls newindex
    LuaAPI.xlua_pushasciistring(L, "__newindex");
    //__ Nex index and__ The steps are the same and will not be described
    ...
    //Set cls_metatable[__newindex] = cls_newindexer
    LuaAPI.lua_rawset(L, cls_meta_idx);
    //end cls newindex

    LuaAPI.lua_pop(L, 4);
}

The code of object and static implementation are basically the same, in utils Beginobjectregister creates four tables: obj_meta,obj_method,obj_get,obj_set. Then in utils Set and implement meta table in endobjectregister__ index and__ newindex, when used each time, look up the meta table in the global variable according to the type to complete the operation mapping.

When a type is not generated with adaptation code, it can actually be accessed by lua. At this time, xlua uses a reflection mechanism, and the reflected branches can be accessed from objecttranslator Trydelaywraploader sees:

public bool TryDelayWrapLoader(RealStatePtr L, Type type)
{
    ...
    if (delayWrap.TryGetValue(type, out loader))
    {
        ...
    }
    //This else branch is taken when the Type has no adaptation code
    else
    {
        ...
        //Register classes, methods, fields, members, etc. with reflection
        Utils.ReflectionWrap(L, type, privateAccessibleFlags.Contains(type));
        ...
    }
    ...
    ...
    return true;
}

         Utils. The reflectionwrap function actually contains the registration logic related to the above adaptation code. For the field processing in the type:

Utils.cs
static void makeReflectionWrap(RealStatePtr L, Type type, int cls_field, int cls_getter, int cls_setter,
			int obj_field, int obj_getter, int obj_setter, int obj_meta, out LuaCSFunction item_getter, out LuaCSFunction item_setter, BindingFlags access)
{
     ...   
     for (int i = 0; i < fields.Length; ++i)
     {
         ...
        if (field.IsStatic && (field.IsInitOnly || field.IsLiteral))
        {
        //For static fields, save the field name = value in CLS directly_ Field table (equivalent to the previous cls_table)
        LuaAPI.xlua_pushasciistring(L, fieldName);
        translator.PushAny(L, field.GetValue(null));
        LuaAPI.lua_rawset(L, cls_field);
        }
        else
        {
            //The member field is saved in the table as a closure function,
            //Staticluacallbacks mapped to the C# side Fixcsfunctionwrapper this delegate
            LuaAPI.xlua_pushasciistring(L, fieldName);
           translator.PushFixCSFunction(L, genFieldGetter(type, field));
           LuaAPI.lua_rawset(L, field.IsStatic ? cls_getter : obj_getter);

            LuaAPI.xlua_pushasciistring(L, fieldName);
            translator.PushFixCSFunction(L, genFieldSetter(type, field));
            LuaAPI.lua_rawset(L, field.IsStatic ? cls_setter : obj_setter);
        }
    }
    ...
    
    //Event mapped to fixcsfunctionwrapper delegate
    EventInfo[] events = type.GetEvents(flag);
	for (int i = 0; i < events.Length; ++i)
	{
    	EventInfo eventInfo = events[i];
		LuaAPI.xlua_pushasciistring(L, eventInfo.Name);
		translator.PushFixCSFunction(L, translator.methodWrapsCache.GetEventWrap(type, eventInfo.Name));
		bool is_static = (eventInfo.GetAddMethod(true) != null) ? eventInfo.GetAddMethod(true).IsStatic : eventInfo.GetRemoveMethod(true).IsStatic;
		LuaAPI.lua_rawset(L, is_static ? cls_field : obj_field);
	}

    ...
    //Method to the fixcsfunctionwrapper delegate
    foreach (var kv in pending_methods)
	{
		if (kv.Key.Name.StartsWith("op_")) // Operator
		{
               LuaAPI.xlua_pushasciistring(L, InternalGlobals.supportOp[kv.Key.Name]);
               translator.PushFixCSFunction(L,new LuaCSFunction(translator.methodWrapsCache._GenMethodWrap(type, kv.Key.Name, kv.Value.ToArray()).Call));
               LuaAPI.lua_rawset(L, obj_meta);
		}
		else
		{
		    LuaAPI.xlua_pushasciistring(L, kv.Key.Name);
		    translator.PushFixCSFunction(L,new LuaCSFunction(translator.methodWrapsCache._GenMethodWrap(type, kv.Key.Name, kv.Value.ToArray()).Call));
		    LuaAPI.lua_rawset(L, kv.Key.IsStatic ? cls_field : obj_field);
		}
	}
}

In addition, the getter and setter of the property are also mapped to the fixcsfunctionwrapper delegate, and the code is in utils Reflectionwrap is in this function, but it is scattered in different places, so it will not be pasted. About how this PushFixCSFunction is mapped:

internal void PushFixCSFunction(RealStatePtr L, LuaCSFunction func)
{
    if (func == null)
    {
        LuaAPI.lua_pushnil(L);
    }
    else
    {
        //Press in an index here
        LuaAPI.xlua_pushinteger(L, fix_cs_functions.Count);
        //Cache functions in fix_ cs_ In functions
        fix_cs_functions.Add(func);
        //Create a closure function and push it onto the stack, taking the above index as upvalue
        LuaAPI.lua_pushstdcallcfunction(L, metaFunctions.FixCSFunctionWraper, 1);
   }
}

Whenever we call through reflection, we will eventually call the fixcsfunctionwrapper delegate:

StaticLuaCallbacks.cs
static int FixCSFunction(RealStatePtr L)
{
    try
    {
        ObjectTranslator translator = ObjectTranslatorPool.Instance.Find(L);
        int idx = LuaAPI.xlua_tointeger(L, LuaAPI.xlua_upvalueindex(1));
        LuaCSFunction func = (LuaCSFunction)translator.GetFixCSFunction(idx);
        return func(L);
   }
   catch (Exception e)
   {
        return LuaAPI.luaL_error(L, "c# exception in FixCSFunction:" + e);
    }
}

As can be seen from the above code, the index in upvalue will be taken out each time, and then from fix_ cs_ Take the function from the functions list and call it.

At this point, the whole call is finished. It can be seen from the above that the first call of each type will execute a series of registration logic. This part of logic has a large number of GC allocs. In our actual development, we should try to execute these logic in a position where performance is not tight in advance.

In addition, the CS type used in lua should also ensure to generate the adaptation code, otherwise it will follow the reflection logic, and the reflection efficiency is not high, and a small amount of GC is generated in the call of the reflection API.

Keywords: lua

Added by yendor on Tue, 04 Jan 2022 00:12:28 +0200