Nonlinear optimization involves the derivation of the objective function and iterative optimization.
Ceres Solver provides three derivation methods: automatic derivation, numerical derivation and analytical derivation.
1. Automatic derivation
Automatic derivation is to define an imitation function and then pass it to AutoDiffCostFunction, so Ceres can derive it by himself.
1.1 definition of affine function
The so-called imitation function is actually a class, but this class acts like a function, so it is called imitation function. The principle is that the class implements the operator() function.
struct CostFunctor {
template <typename T>
bool operator()(const T* const x, T* residual) const {
residual[0] = T(10.0) - x[0];
return true;
}
};
The operator() function implemented by the automatic derivation imitation function must be a template function, because the internal derivation of Ceres needs to be used.
T can be directly understood as double.
1.2 construct CostFunction
CostFunction* cost_function = new AutoDiffCostFunction<CostFunctor, 1, 1>(new CostFunctor);
Ceres constructs the nonlinear least squares problem by first defining the cost function, that is, the above CostFunction, and then through problem AddResidualBlock(cost_function, NULL, &x); To construct the problem for solution.
Template parameters of AutoDiffCostFunction:
- The first parameter is an affine function
- The second parameter is the number of residuals in the residual block
- The third parameter is the number of parameters in the first parameter block
- If there are multiple parameter blocks, write the number of parameters in each parameter block in turn
The parameters are described as follows:
CostFunction* cost_function
= new AutoDiffCostFunction<CostFunctor, 1, 1>(new CostFunctor(1.0));
^ ^
| |
Dimension of residual ------+ |
Dimension of x ----------------+
2. Numerical derivation
Sometimes, it is impossible to define template imitation functions for automatic derivation. For example, the estimation of parameters calls uncontrollable library functions or external functions.
In this case, automatic derivation cannot be used, and numerical derivation can be used.
The use of numerical derivation is similar. First define the imitation function, then pass it to the NumericDiffCostFunction, and then construct the problem solution.
2.1 definition of affine function
struct NumericDiffCostFunctor {
bool operator()(const double* const x, double* residual) const {
residual[0] = 10.0 - x[0];
return true;
}
};
Different from the inverse function of automatic derivation, the operator() function of numerical derivation is not a template function, but directly uses double
2.2 construct CostFunction
CostFunction* cost_function =
new NumericDiffCostFunction<NumericDiffCostFunctor, ceres::CENTRAL, 1, 1>(
new NumericDiffCostFunctor);
Note the template parameters of NumericDiffCostFunction:
- The first parameter is an affine function
- The second parameter is the way to get the value. CENTRAL, FORWARD, RIDDERS, etc. are selected here
- The third parameter is the number of residuals in the residual block
- The fourth parameter is the number of parameters in the first parameter block
- If there are multiple parameter blocks, write the number of parameters in each parameter block in turn
The parameters are described as follows:
CostFunction* cost_function
= new NumericDiffCostFunction<NumericDiffCostFunctor, CENTRAL, 1, 1>(
new NumericDiffCostFunctor; ^ ^ ^
| | |
Finite Differencing Scheme -+ | |
Dimension of residual ------------+ |
Dimension of x ----------------------+
3. Analytical derivation
In some cases, the computational efficiency will be higher if you write your own derivative analytical formula.
If analytical derivation is used, the residual and Jacobian must be calculated by ourselves.
3.1 defining cost function classes
The cost function takes $f(x)=10-x $as an example
class QuadraticCostFunction : public ceres::SizedCostFunction<1, 1> {
public:
virtual ~QuadraticCostFunction() {}
virtual bool Evaluate(double const* const* parameters,
double* residuals,
double** jacobians) const {
const double x = parameters[0][0];
residuals[0] = 10 - x;
<span class="token comment">// Compute the Jacobian if asked for.</span>
<span class="token keyword">if</span> <span class="token punctuation">(</span>jacobians <span class="token operator">!=</span> <span class="token constant">NULL</span> <span class="token operator">&&</span> jacobians<span class="token punctuation">[</span><span class="token number">0</span><span class="token punctuation">]</span> <span class="token operator">!=</span> <span class="token constant">NULL</span><span class="token punctuation">)</span> <span class="token punctuation">{</span>
jacobians<span class="token punctuation">[</span><span class="token number">0</span><span class="token punctuation">]</span><span class="token punctuation">[</span><span class="token number">0</span><span class="token punctuation">]</span> <span class="token operator">=</span> <span class="token operator">-</span><span class="token number">1</span><span class="token punctuation">;</span>
<span class="token punctuation">}</span>
<span class="token keyword">return</span> <span class="token boolean">true</span><span class="token punctuation">;</span>
}
};
The custom cost function class should inherit from CostFunction or SizedCostFunction. In fact, SizedCostFunction inherits from CostFunction and only determines the size (the number of parameter blocks).
The residual and Jacobian are calculated in the Evaluate() function
3.2 construct CostFunction
CostFunction* cost_function = new QuadraticCostFunction;
3.3 when to use analytic derivation
According to the official instructions, analytical derivation can be used in the following cases
- The function formula is simple and easy to find the analytical formula of derivative
- The analytical formula of derivative can be obtained by using mathematical software such as MATLAB Maple mathematical symphony
- Ultimate performance requirements
- There is no other good way to find the derivative
- I like to calculate derivatives by hand
4. Summary
To sum up, it is recommended to give priority to automatic derivation and numerical derivation, and analytical derivation can be considered for those who are good at Jacobian calculation and those who pursue extreme performance.