Analysis of std::function source code of C++11

1. Source code preparation

This article is based on the analysis of the source code of gcc-4.9.0. std::function is added to the standard only after C++11. Therefore, the lower version of GCC source code does not have std::function. It is recommended to choose 4.9.0 or later version to learn. The difference between different versions of GCC source code should not be small, but the principle and design idea are the same.
Source code download address: http://ftp.gnu.org/gnu/gcc

2. Introduction to std::function

Class template std::function is a universal wrapper for polymorphic functions. The instance of std::function can store, copy, and call any target entity that can be called, including ordinary functions, Lambda expressions, function pointers, and other function objects. std::function object is a type safe package for existing callable entities in C + + (we know that callable entities such as function pointers are type unsafe).
Generally, std::function is a function object class that wraps any other function object. The wrapped function object has N parameters of type T1,..., TN, and returns a value that can be converted to type R. std::function uses the template conversion constructor to receive the wrapped function object; In particular, closure types can be implicitly converted to std::function.
The simplest understanding is to encapsulate various callable entities (ordinary functions, Lambda expressions, function pointers, and other function objects) in C + + through std::function to form a new callable std::function object, so that we don't tangle with so many callable entities.

3. Source code analysis

3.1. std::function parsing

std::function is located in libstdc++-v3\include\std\functional

template<typename _Res, typename... _ArgTypes>
class function<_Res(_ArgTypes...)> : public _Maybe_unary_or_binary_function<_Res, _ArgTypes...>, private _Function_base
{
    typedef _Res _Signature_type(_ArgTypes...);

    template<typename _Functor>
    using _Invoke = decltype(__callable_functor(std::declval<_Functor&>())(std::declval<_ArgTypes>()...) );

    template<typename _Functor>
    using _Callable = __check_func_return_type<_Invoke<_Functor>, _Res>;

    template<typename _Cond, typename _Tp>
    using _Requires = typename enable_if<_Cond::value, _Tp>::type;

public:
    typedef _Res result_type;

    function() noexcept
        :_Function_base()
    {
    }

    function(nullptr_t) noexcept
        :_Function_base()
    {
    }

    template<typename _Res, typename... _ArgTypes>
    function(const function& __x)
        :_Function_base()
    {
        if (static_cast<bool>(__x))
        {
            _M_invoker = __x._M_invoker;
            _M_manager = __x._M_manager;
            __x._M_manager(_M_functor, __x._M_functor, __clone_functor);
        }
    }

    function(function&& __x)
        :_Function_base()
    { __x.swap(*this); }

    template<typename _Functor, typename = _Requires<_Callable<_Functor>, void>>
    function(_Functor __f)
    {
        typedef _Function_handler<_Signature_type, _Functor> _My_handler;

        if (_My_handler::_M_not_empty_function(__f))
        {
            _My_handler::_M_init_functor(_M_functor, std::move(__f));
            _M_invoker = &_My_handler::_M_invoke;
            _M_manager = &_My_handler::_M_manager;
        }
    }

    function& operator=(const function& __x)
    {
        function(__x).swap(*this);
        return *this;
    }

    function& operator=(function&& __x)
    {
        function(std::move(__x)).swap(*this);
        return *this;
    }

    function& operator=(nullptr_t)
    {
        if (_M_manager)
        {
            _M_manager(_M_functor, _M_functor, __destroy_functor);
            _M_manager = 0;
            _M_invoker = 0;
        }
        return *this;
    }

    template<typename _Functor>
    _Requires<_Callable<_Functor>, function&> operator=(_Functor&& __f)
    {
        function(std::forward<_Functor>(__f)).swap(*this);
        return *this;
    }

    template<typename _Functor>
    function& operator=(reference_wrapper<_Functor> __f) noexcept
    {
        function(__f).swap(*this);
        return *this;
    }

    void swap(function& __x)
    {
        std::swap(_M_functor, __x._M_functor);
        std::swap(_M_manager, __x._M_manager);
        std::swap(_M_invoker, __x._M_invoker);
    }

     explicit operator bool() const noexcept
     { return !_M_empty(); }

    _Res operator()(_ArgTypes... __args) const;
    {
        if (_M_empty())
            __throw_bad_function_call();
        return _M_invoker(_M_functor, std::forward<_ArgTypes>(__args)...);
    }

private:
    typedef _Res (*_Invoker_type)(const _Any_data&, _ArgTypes...);
    _Invoker_type _M_invoker;

The following information can be seen from the source code:

  1. This class is a variable parameter template class
  2. This class inherits from_ Maybe_unary_or_binary_function and_ Function_base, class member only_ M_invoker is a standard function pointer, which can be seen from the definition
  3. First, analyze the operator() method. The overloaded bracket operator must be the most commonly used std::function in normal development. After all, it should be called as a function pointer in the end. You can see that the operator() function is called_ M_ The invoker function has no special handling
  4. Since_ M_ If invoker can be called, it must have been initialized. Judging from the parameters passed to him during the call, there is one more parameter that doesn't know what it is_ M_ Function, so we can guess_ M_invoker does not directly point to the function pointer taken over by std::function, but something similar to the middle layer. In_ M_ The real function pointer we need to execute is called in the invoker implementation
  5. Only constructor function (_functor _f) pairs_ M_invoker is initialized, and STD is used:_ Function_ Handler to initialize_ M_ Of invoker, STD::_ Function_ The implementation of handler will be discussed later
  6. Let's look at the constructor function (_functor _f), because the purpose of std::function at the beginning must be to take over the function pointer. The function pointer here can be ordinary function pointer, class member function pointer, or even Lambda expression. We can see that in the std::function class, there is no direct hosting function pointer, but calling_ My_ handler::_ M_ init_ Functor (_m_functor, STD:: move (_f)), presumably by_ Function_base to host function pointers

3.2,std::_Function_handler parsing

std::_ Function_ The handler is located in libstdc++-v3\include\std\functional

template<typename _Res, typename _Functor, typename... _ArgTypes>
class _Function_handler<_Res(_ArgTypes...), _Functor> : public _Function_base::_Base_manager<_Functor>
{
    typedef _Function_base::_Base_manager<_Functor> _Base;

public:
    static _Res _M_invoke(const _Any_data& __functor, _ArgTypes... __args)
    {
        return (*_Base::_M_get_pointer(__functor))(std::forward<_ArgTypes>(__args)...);
    }
};

template<typename _Functor, typename... _ArgTypes>
class _Function_handler<void(_ArgTypes...), _Functor> : public _Function_base::_Base_manager<_Functor>
{
    typedef _Function_base::_Base_manager<_Functor> _Base;

public:
    static void _M_invoke(const _Any_data& __functor, _ArgTypes... __args)
    {
        (*_Base::_M_get_pointer(__functor))(std::forward<_ArgTypes>(__args)...);
    }
};

template<typename _Res, typename _Functor, typename... _ArgTypes>
class _Function_handler<_Res(_ArgTypes...), reference_wrapper<_Functor> > : public _Function_base::_Ref_manager<_Functor>
{
    typedef _Function_base::_Ref_manager<_Functor> _Base;

public:
    static _Res _M_invoke(const _Any_data& __functor, _ArgTypes... __args)
    {
        return __callable_functor(**_Base::_M_get_pointer(__functor))(std::forward<_ArgTypes>(__args)...);
    }
};

template<typename _Functor, typename... _ArgTypes>
class _Function_handler<void(_ArgTypes...), reference_wrapper<_Functor> > : public _Function_base::_Ref_manager<_Functor>
{
    typedef _Function_base::_Ref_manager<_Functor> _Base;

public:
    static void _M_invoke(const _Any_data& __functor, _ArgTypes... __args)
    {
        __callable_functor(**_Base::_M_get_pointer(__functor))(std::forward<_ArgTypes>(__args)...);
    }
};

template<typename _Class, typename _Member, typename _Res, typename... _ArgTypes>
class _Function_handler<_Res(_ArgTypes...), _Member _Class::*> : public _Function_handler<void(_ArgTypes...), _Member _Class::*>
{
    typedef _Function_handler<void(_ArgTypes...), _Member _Class::*> _Base;

public:
    static _Res _M_invoke(const _Any_data& __functor, _ArgTypes... __args)
    {
        return std::mem_fn(_Base::_M_get_pointer(__functor)->__value)(std::forward<_ArgTypes>(__args)...);
    }
};

template<typename _Class, typename _Member, typename... _ArgTypes>
class _Function_handler<void(_ArgTypes...), _Member _Class::*> : public _Function_base::_Base_manager<_Simple_type_wrapper< _Member _Class::* > >
{
    typedef _Member _Class::* _Functor;
    typedef _Simple_type_wrapper<_Functor> _Wrapper;
    typedef _Function_base::_Base_manager<_Wrapper> _Base;

public:
    static bool _M_manager(_Any_data& __dest, const _Any_data& __source, _Manager_operation __op)
    {
        switch (__op)
        {
            #ifdef __GXX_RTTI
            case __get_type_info:
                __dest._M_access<const type_info*>() = &typeid(_Functor);
                break;
            #endif

            case __get_functor_ptr:
                __dest._M_access<_Functor*>() = &_Base::_M_get_pointer(__source)->__value;
                break;

            default:
                _Base::_M_manager(__dest, __source, __op);
        }
        return false;
    }

    static void _M_invoke(const _Any_data& __functor, _ArgTypes... __args)
    {
        std::mem_fn(_Base::_M_get_pointer(__functor)->__value)(std::forward<_ArgTypes>(__args)...);
    }
};

The following information can be seen from the source code:

  1. The first and second classes inherit from STD:_ Function_ base::_ Base_ Manager, these two classes are used to handle the case that std::function takes over an ordinary function. The difference between them is that one handles the case with a return value and the other handles the case without a return value
  2. The third and fourth classes inherit from STD:_ Function_ base::_ Ref_ Manager, you can see that it is a partial specialized version of the first two classes, when the second template parameter is STD:: reference_ The two partial specialized versions are called when the wrapper wraps the reference. The difference between them is that one handles the case with a return value and the other handles the case without a return value

Because the parameter passed at this time is std::reference_wrapper object, we cannot call this function directly, because at this time, we are not sure whether the wrapped function is an ordinary function or a class member function. If it is a class member function, it cannot be used directly, and STD:: MEM must be called_ FN to infer the type of function. STD used in the class:__ callable_ The functor function is also through std::mem_fn.
About STD:: reference mentioned above_ Wrapper and std::mem_fn, if you can't understand it, you must read the following two articles. Otherwise, it's impossible to understand the content of std::function just as you can't learn English words

  1. The fifth and sixth classes are similar to the previous ones. These two are also specialized versions. They are used to deal with the case where the second template parameter is a class member function. Here you can see that STD:: men is called directly_ FN function so that we can directly use class member functions. From this point, we can also see STD:: men_ The importance of FN function. If you don't understand it, you must read the previous two articles.
  2. In each class_ M_ The invoke function is used_ M_ get_ It's not difficult to see from the logic of the code that the source is not all the same_ M_ get_ The pointer function is used to pass in parameters from the first one__ Take out a function pointer from the function (ordinary function pointer or class member function pointer can be used, because there is a special version, and the class member function pointer can also be called directly through the std::mem_fn function). Then pass the following variable parameters to the function pointer and run it. Is this function a little familiar? Yes, This is how we usually call function pointers, that is, the function execution function of std::function is realized here
  3. From the code, we can see that these classes are and STD:_ Function_ Base related, and I still don't know_ M_ What exactly is a function? Next, analyze STD:_ Function_ Base, let's see what functions are implemented in it

3.3,_ Any_data parsing

_ Any_data is located in libstdc++-v3\include\std\functional

union _Nocopy_types
{
    void*       _M_object;
    const void* _M_const_object;
    void (*_M_function_pointer)();
    void (_Undefined_class::*_M_member_pointer)();
};

union _Any_data
{
    void*       _M_access()       { return &_M_pod_data[0]; }
    const void* _M_access() const { return &_M_pod_data[0]; }

    template<typename _Tp>
    _Tp& _M_access()
    { return *static_cast<_Tp*>(_M_access()); }

    template<typename _Tp>
    const _Tp& _M_access() const
    { return *static_cast<const _Tp*>(_M_access()); }

    _Nocopy_types _M_unused;
    char _M_pod_data[sizeof(_Nocopy_types)];
};

Look at STD:_ Function_ Look at an important consortium before base_ Any_data, which has appeared many times before, has not introduced what it is. Here is a brief analysis:

  1. There are two consortium members, one is_ M_unused, one is_ M_pod_data, the two occupy the same memory. I won't talk about the specific reasons. You can use sizeof to try it yourself
  2. There are four in it_ M_access function, the first two are directly_ M_ pod_ The address of data is returned without any conversion, and the last two can be_ M_pod_data is converted to any type and returned. You can see this from here_ Any_ The function of data is to take over the function pointer and put a shell on the function pointer we want to take over

3.4,std::_Function_base parsing

std::_ Function_ The implementation of base is located in libstdc++-v3\include\std\functional

class _Function_base
{
public:
    static const std::size_t _M_max_size = sizeof(_Nocopy_types);
    static const std::size_t _M_max_align = __alignof__(_Nocopy_types);

    template<typename _Functor>
    class _Base_manager
    {
    protected:
        static const bool __stored_locally =
            (__is_location_invariant<_Functor>::value
            && sizeof(_Functor) <= _M_max_size
            && __alignof__(_Functor) <= _M_max_align
            && (_M_max_align % __alignof__(_Functor) == 0));

        typedef integral_constant<bool, __stored_locally> _Local_storage;

        static _Functor* _M_get_pointer(const _Any_data& __source)
        {
            const _Functor* __ptr = __stored_locally? std::__addressof(__source._M_access<_Functor>()) : __source._M_access<_Functor*>();
            return const_cast<_Functor*>(__ptr);
        }

        static void _M_clone(_Any_data& __dest, const _Any_data& __source, true_type)
        {
            new (__dest._M_access()) _Functor(__source._M_access<_Functor>());
        }

        static void _M_clone(_Any_data& __dest, const _Any_data& __source, false_type)
        {
            __dest._M_access<_Functor*>() = new _Functor(*__source._M_access<_Functor*>());
        }

        static void _M_destroy(_Any_data& __victim, true_type)
        {
            __victim._M_access<_Functor>().~_Functor();
        }

        static void _M_destroy(_Any_data& __victim, false_type)
        {
            delete __victim._M_access<_Functor*>();
        }

    public:
        static bool _M_manager(_Any_data& __dest, const _Any_data& __source, _Manager_operation __op)
        {
            switch (__op)
            {
                case __get_functor_ptr:
                    __dest._M_access<_Functor*>() = _M_get_pointer(__source);
                    break;

                case __clone_functor:
                    _M_clone(__dest, __source, _Local_storage());
                    break;

                case __destroy_functor:
                    _M_destroy(__dest, _Local_storage());
                    break;
            }
            return false;
        }

        static void _M_init_functor(_Any_data& __functor, _Functor&& __f)
        { _M_init_functor(__functor, std::move(__f), _Local_storage()); }

        template<typename _Signature>
        static bool _M_not_empty_function(const function<_Signature>& __f)
        { return static_cast<bool>(__f); }

        template<typename _Tp>
        static bool _M_not_empty_function(_Tp* const& __fp)
        { return __fp; }

        template<typename _Class, typename _Tp>
        static bool _M_not_empty_function(_Tp _Class::* const& __mp)
        { return __mp; }

        template<typename _Tp>
        static bool _M_not_empty_function(const _Tp&)
        { return true; }

    private:
        static void _M_init_functor(_Any_data& __functor, _Functor&& __f, true_type)
        { new (__functor._M_access()) _Functor(std::move(__f)); }

        static void _M_init_functor(_Any_data& __functor, _Functor&& __f, false_type)
        { __functor._M_access<_Functor*>() = new _Functor(std::move(__f)); }
    };

    template<typename _Functor>
    class _Ref_manager : public _Base_manager<_Functor*>
    {
        typedef _Function_base::_Base_manager<_Functor*> _Base;

    public:
        static bool _M_manager(_Any_data& __dest, const _Any_data& __source, _Manager_operation __op)
        {
            switch (__op)
            {
                case __get_functor_ptr:
                    __dest._M_access<_Functor*>() = *_Base::_M_get_pointer(__source);
                    return is_const<_Functor>::value;
                    break;

                default:
                    _Base::_M_manager(__dest, __source, __op);
            }
            return false;
        }

        static void _M_init_functor(_Any_data& __functor, reference_wrapper<_Functor> __f)
        {
            _Base::_M_init_functor(__functor, std::__addressof(__f.get()));
        }
    };

    _Function_base() : _M_manager(0) { }

    ~_Function_base()
    {
        if (_M_manager)
            _M_manager(_M_functor, _M_functor, __destroy_functor);
    }

    bool _M_empty() const { return !_M_manager; }

    typedef bool (*_Manager_type)(_Any_data&, const _Any_data&, _Manager_operation);

    _Any_data     _M_functor;
    _Manager_type _M_manager;
};

The following information can be seen from the source code:

  1. This class has two class members:_ M_functor and_ M_manager,_ M_ There is no need to say more about function. It has been mentioned before. Its type_ Any_data was also mentioned in the previous section. And_ M_manager is a function pointer. I can't see its use for the time being.
  2. Take a look here_ Function_ base::_ Base_ Various methods in the manager class
  • _ M_ init_ Function: this function was mentioned earlier. At that time, we saw that the function pointer taken over by std::function was passed to this function, and we know from the above_ Any_ Data type data can take over all types of function pointers, so it can be obtained by synthesis_ M_ init_ The function of function is to store the second parameter (function pointer) passed to it into the first parameter (_Any_data type data), so as to achieve the function of taking over the function pointer
  • _ M_get_pointer function: this function was also used earlier. At that time, we only knew through calling_ M_get_pointer can get the pointer we took over, but we don't know how to implement it. Here we can see that its implementation is very simple, that is, take out the function pointer from its incoming parameter (_Any_data type data) and force the type to be converted to the type we need. std::__addressof is used to obtain the address of the variable even if the target variable (class) is overloaded with operator &.
    About STD:__ The details of addressof can be found in this article< Source code analysis of std::addressof in C++11>
  • _ M_manager function: this function determines the execution of different functions according to the third parameter passed in. The other functions are nothing more than memory allocation and release, which has little impact on our understanding of std::function. I won't expand here
  1. Let's take a look_ Function_base::_Ref_manager class
  • You can see that this class inherits from_ Function_base::_Base_manager class, it can be seen that it has_ Function_ base::_ Base_ All functions implemented by the manager class
  • This class handles the case when std::function takes over a reference wrapper class. Why should this case be handled separately? Because if we directly take the address of the incoming parameter, we will get the address of the reference wrapper class instead of the address of the function pointer we want to take over, so we can only make a special version, like_ Function_ base::_ Ref_ As the manager class does, the_ M_ init_ The functor function uses reference_ The get method of wrapper obtains the real address of the function pointer taken over
  1. _ Function_ We won't talk about other methods in the base class. Let's have a look for ourselves. Other implementations are basically based on the contents mentioned above, which is relatively simple

4. Summary

This paper first briefly introduces the purpose of std::function (encapsulating various callable entities in C + +), and then through a detailed analysis of the source code, we know how std::function encapsulates callable entities. The content of the source code will be more complex, but the design idea is worth learning from, especially with std::reference_wrapper and std::mem_fn with that part of the code is exquisite.
If the reader is not clear about std::reference_wrapper and std::mem_fn, it is impossible to fully understand the source code of std::function. Therefore, it is recommended that you read the following two articles before learning about std::function:

Finally, if you think this article is well written, please praise and collect it. Thank you. You can also pay attention to this column. There will be more useful articles output in the future.

Keywords: C++ Functional Programming C++11

Added by jaybones on Sat, 29 Jan 2022 20:10:00 +0200