This article contains the following contents
1. x264 bit rate control principle and corresponding source code analysis
Basic concepts
For readers who have previously understood the rate control principle of H.264 (if not, you can also read my article H. Research on 264 rate control algorithm and analysis of JM corresponding code )Firstly, it should be noted that the rate control algorithm adopted by x264 does not use Lagrange cost function to control the coding, but uses a simpler method, that is, using the SATD of half precision frame as the basis for the selection of quantization level parameters. SATD is about 4% of the residuals transformed by Hardman × The sum of the absolute values of the prediction residuals of the four blocks can be regarded as a simple time-frequency transform, and its value can reflect the size of the generated code stream to a certain extent.
There are explicit rate control methods for CQF, CQF, and CQF The default method is CRF.
See x264 for details_ param_ In the default() method
param->rc.i_rc_method = X264_RC_CRF; param->rc.i_bitrate = 0; param->rc.f_rate_tolerance = 1.0; param->rc.i_vbv_max_bitrate = 0; param->rc.i_vbv_buffer_size = 0; param->rc.f_vbv_buffer_init = 0.9; param->rc.i_qp_constant = 23 + QP_BD_OFFSET; param->rc.f_rf_constant = 23;
For bitrate and QP, once they are set, it means to encode according to the corresponding rate control method. CRF has a default value of 23. If there is no setting on coding control, it will be encoded according to the default value of CRF 23.
See x264 for details_ param_ In parse() method
OPT("bitrate")
{
p->rc.i_bitrate = atoi(value);
p->rc.i_rc_method = X264_RC_ABR;
}
OPT2("qp", "qp_constant")
{
p->rc.i_qp_constant = atoi(value);
p->rc.i_rc_method = X264_RC_CQP;
}
OPT("crf")
{
p->rc.f_rf_constant = atof(value);
p->rc.i_rc_method = X264_RC_CRF;
} For a more detailed description of these three methods, see here . The introduction of some command line parameters related to bit rate control can be seen This article.
Source code analysis
x264 all contents related to rate control are in ratecontrol h ratecontrol. C in these two files, the basic call flow is as follows. The leftmost functions in the figure represent where the content of rate control is called. Readers who are not clear about these functions can refer to my last article
First of all, we need to know that the key structure in the whole rate control process is x264_ratecontrol_t is defined as follows
struct x264_ratecontrol_t
{
/* constants */
int b_abr;
/* 2pass Is a key technology of variable bit rate (VBR) compression, which means to compress a file through twice analysis.
* For the first time, analyze the movement speed of each part of the film, allocate the corresponding bit rate and make corresponding records,
* The second time, compress according to the record of the first time, so that the code rate can be optimally allocated,
* So as to maximize the video quality within the specified file size or specified code stream.*/
int b_2pass;
int b_vbv; //Video buffer checker
int b_vbv_min_rate;
double fps;
double bitrate;
double rate_tolerance;
double qcompress;
int nmb; /* number of macroblocks in a frame */
int qp_constant[3];
/* current frame */
ratecontrol_entry_t *rce; //Relevant contents of current frame rate control
int qp; /* qp for current frame */
float qpm; /* qp for current macroblock: precise float for AQ */
float qpa_rc; /* average of macroblocks' qp before aq */
float qpa_rc_prev;
int qpa_aq; /* average of macroblocks' qp after aq */
int qpa_aq_prev;
float qp_novbv; /* QP for the current frame if 1-pass VBV was disabled. */
/* VBV stuff */
double buffer_size;
int64_t buffer_fill_final; /* The buffer size actually used by the last completed frame */
int64_t buffer_fill_final_min;
double buffer_fill; /* planned buffer, if all in-progress frames hit their bit budget */
/* The planned buffer size is the maximum possible buffer usage for all frames processed */
double buffer_rate; /* # of bits added to buffer_fill after each frame */
/* After each frame, it should be added to the buffer_ Number of bits on fill */
double vbv_max_rate; /* # of bits added to buffer_fill per second */
predictor_t *pred; /* predict frame size from satd */
int single_frame_vbv;
float rate_factor_max_increment; /* Don't allow RF above (CRF + this value). */
/* ABR stuff */
int last_satd;
double last_rceq;//last rc estimated qscale
double cplxr_sum; /* sum of bits*qscale/rceq */
double expected_bits_sum; /* sum of qscale2bits after rceq, ratefactor, and overflow, only includes finished frames */
int64_t filler_bits_sum; /* sum in bits of finished frames' filler data */
double wanted_bits_window; /* target bitrate * window */
double cbr_decay;//It's not clear what it does
double short_term_cplxsum;
double short_term_cplxcount;
double rate_factor_constant;
double ip_offset;
double pb_offset;
/* 2pass stuff */
FILE *p_stat_file_out;
char *psz_stat_file_tmpname;
FILE *p_mbtree_stat_file_out;
char *psz_mbtree_stat_file_tmpname;
char *psz_mbtree_stat_file_name;
FILE *p_mbtree_stat_file_in;
int num_entries; /* number of ratecontrol_entry_ts */
ratecontrol_entry_t *entry; /* FIXME: copy needed data and free this once init is done */
double last_qscale;
double last_qscale_for[3]; /* last qscale for a specific pict type, used for max_diff & ipb factor stuff */
int last_non_b_pict_type;
double accum_p_qp; /* for determining I-frame quant;accumulation of p frame qp*/
/* Used to determine the quantization coefficient of I frame */
double accum_p_norm;
double last_accum_p_norm;
double lmin[3]; /* min qscale by frame type */
double lmax[3];
double lstep; /* max change (multiply) in qscale per frame */
struct
{
uint16_t *qp_buffer[2]; /* Global buffers for converting MB-tree quantizer data. */
int qpbuf_pos; /* In order to handle pyramid reordering, QP buffer acts as a stack.
* This value is the current position (0 or 1). */
int src_mb_count;
/* For rescaling */
int rescale_enabled;
float *scale_buffer[2]; /* Intermediate buffers */
int filtersize[2]; /* filter size (H/V) */
float *coeffs[2];
int *pos[2];
int srcdim[2]; /* Source dimensions (W/H) */
} mbtree;
/* MBRC stuff */
float frame_size_estimated; /* Access to this variable must be atomic: double is
* not atomic on all arches we care about */
double frame_size_maximum; /* Maximum frame size due to MinCR */
double frame_size_planned;
double slice_size_planned;
predictor_t *row_pred;//Predicting a set of coefficients of framesize with satd and qscale
predictor_t row_preds[3][2];//3 stands for IPB frame, but what 2 stands for is not clear
predictor_t *pred_b_from_p; /* predict B-frame size from P-frame satd */
int bframes; /* # consecutive B-frames before this P-frame */
int bframe_bits; /* total cost of those frames */
int i_zones;
x264_zone_t *zones;
x264_zone_t *prev_zone;
/* hrd stuff */
int initial_cpb_removal_delay;
int initial_cpb_removal_delay_offset;
double nrt_first_access_unit; /* nominal removal time */
double previous_cpb_final_arrival_time;
uint64_t hrd_multiply_denom;
};According to my learning experience, after reading this structure, I will be curious about what some strange variables do. At this time, it is necessary to first understand the theoretical method of x264 bit rate control. In this regard, I suggest you search some papers to learn and give a better one I think.
There are many empirical formulas in the steps, and there should also be some discussion articles in this regard, which will be supplemented when there is time in the future.
The following is to analyze the code according to the previous flow chart and theoretical steps. Analyze each key function in order, the function of each function and the content of detailed code analysis are written in the comments. I hope the combination of the three can help you understand the rate control of x264. If there are any problems or errors, you are welcome to point out that there are still some places I don't understand in code analysis, and I hope to know more friends for advice.
x264_ratecontrol_new
//call by x264_encoder_open
//It is used for various assignment and initialization, and manages the input and output files related to 2pass rate control
int x264_ratecontrol_new( x264_t *h )
{
x264_ratecontrol_t *rc;
x264_emms();
CHECKED_MALLOCZERO( h->rc, h->param.i_threads * sizeof(x264_ratecontrol_t) );
rc = h->rc;
//Judgment rate control method
rc->b_abr = h->param.rc.i_rc_method != X264_RC_CQP && !h->param.rc.b_stat_read;
rc->b_2pass = h->param.rc.i_rc_method == X264_RC_ABR && h->param.rc.b_stat_read;
/* FIXME: use integers */
if( h->param.i_fps_num > 0 && h->param.i_fps_den > 0 )
rc->fps = (float) h->param.i_fps_num / h->param.i_fps_den;
else
rc->fps = 25.0; //default fps
// macroblock tree: determines the qp value of which size the MB is quantized. For each MB,
// Predict a certain number of frames forward and record the reference of the MB. The size of qp is inversely proportional to the number of references.
// And mb_tree related parameters: RC lookahead determines MB_ Number of frames predicted forward by tree
if( h->param.rc.b_mb_tree )
{
h->param.rc.f_pb_factor = 1;
// Qcomp (Quantizer Curve Compression): controls the bit rate fluctuation of the whole video,
// It affects MB_ Strength of tree (Higher qcomp = weaker mbtree)
// 0% is equivalent to CBR mode and 100% is equivalent to CQP mode
rc->qcompress = 1;
}
else
rc->qcompress = h->param.rc.f_qcompress; // 0.0 => cbr, 1.0 => constant qp
//The code rate unit is generally 1000
rc->bitrate = h->param.rc.i_bitrate * (h->param.i_avcintra_class ? 1024. : 1000.);
rc->rate_tolerance = h->param.rc.f_rate_tolerance;
rc->nmb = h->mb.i_mb_count; //number of mbs in a frame
rc->last_non_b_pict_type = -1;
rc->cbr_decay = 1.0;
if( h->param.rc.i_rc_method == X264_RC_CRF && h->param.rc.b_stat_read )
{
x264_log( h, X264_LOG_ERROR, "constant rate-factor is incompatible with 2pass.\n" );
return -1;
}
x264_ratecontrol_init_reconfigurable( h, 1 );
if( h->param.i_nal_hrd )
{
uint64_t denom = (uint64_t)h->sps->vui.hrd.i_bit_rate_unscaled * h->sps->vui.i_time_scale;
uint64_t num = 90000;
x264_reduce_fraction64( &num, &denom );
rc->hrd_multiply_denom = 90000 / num;
//Calculate the number of bits required
double bits_required = log2( 90000 / rc->hrd_multiply_denom )
+ log2( h->sps->vui.i_time_scale )
+ log2( h->sps->vui.hrd.i_cpb_size_unscaled );
if( bits_required >= 63 )
{
x264_log( h, X264_LOG_ERROR, "HRD with very large timescale and bufsize not supported\n" );
return -1;
}
}
if( rc->rate_tolerance < 0.01 )
{
x264_log( h, X264_LOG_WARNING, "bitrate tolerance too small, using .01\n" );
rc->rate_tolerance = 0.01;
}
/* whether qp is allowed to vary per macroblock */
h->mb.b_variable_qp = rc->b_vbv || h->param.rc.i_aq_mode;
if( rc->b_abr )
{
/* FIXME ABR_INIT_QP is actually used only in CRF */
#define ABR_INIT_QP (( h->param.rc.i_rc_method == X264_RC_CRF ? h->param.rc.f_rf_constant : 24 ) + QP_BD_OFFSET)
rc->accum_p_norm = .01;
rc->accum_p_qp = ABR_INIT_QP * rc->accum_p_norm;
/* estimated ratio that produces a reasonable QP for the first I-frame */
// Estimated initial complexity = 0.01 * (700000 ^ qcomp) * (total number of macroblocks ^ 0.5)
rc->cplxr_sum = .01 * pow( 7.0e5, rc->qcompress ) * pow( h->mb.i_mb_count, 0.5 );
//Bits per frame
rc->wanted_bits_window = 1.0 * rc->bitrate / rc->fps;
rc->last_non_b_pict_type = SLICE_TYPE_I;
}
rc->ip_offset = 6.0 * log2f( h->param.rc.f_ip_factor );
rc->pb_offset = 6.0 * log2f( h->param.rc.f_pb_factor );
//get qp for p frame at first, then use offset to calculate values for i framne and b frame
rc->qp_constant[SLICE_TYPE_P] = h->param.rc.i_qp_constant;
rc->qp_constant[SLICE_TYPE_I] = x264_clip3( h->param.rc.i_qp_constant - rc->ip_offset + 0.5, 0, QP_MAX );
rc->qp_constant[SLICE_TYPE_B] = x264_clip3( h->param.rc.i_qp_constant + rc->pb_offset + 0.5, 0, QP_MAX );
h->mb.ip_offset = rc->ip_offset + 0.5;
//Maximum change multiple of qscale per frame = qp_step/6
rc->lstep = pow( 2, h->param.rc.i_qp_step / 6.0 );
rc->last_qscale = qp2qscale( 26 );
int num_preds = h->param.b_sliced_threads * h->param.i_threads + 1;
//The prediction error is calculated by satd
CHECKED_MALLOC( rc->pred, 5 * sizeof(predictor_t) * num_preds );
CHECKED_MALLOC( rc->pred_b_from_p, sizeof(predictor_t) );
for( int i = 0; i < 3; i++ )
{//Corresponding P B I frame
rc->last_qscale_for[i] = qp2qscale( ABR_INIT_QP );
rc->lmin[i] = qp2qscale( h->param.rc.i_qp_min );
rc->lmax[i] = qp2qscale( h->param.rc.i_qp_max );
for( int j = 0; j < num_preds; j++ )
{
rc->pred[i+j*5].coeff_min = 2.0 / 4;
rc->pred[i+j*5].coeff = 2.0;
rc->pred[i+j*5].count = 1.0;
rc->pred[i+j*5].decay = 0.5;
rc->pred[i+j*5].offset = 0.0;
}
for( int j = 0; j < 2; j++ )
{
rc->row_preds[i][j].coeff_min = .25 / 4;
rc->row_preds[i][j].coeff = .25;
rc->row_preds[i][j].count = 1.0;
rc->row_preds[i][j].decay = 0.5;
rc->row_preds[i][j].offset = 0.0;
}
}
*rc->pred_b_from_p = rc->pred[0];
// Used to manually assign a low or high bit rate to a specific part of the video
if( parse_zones( h ) < 0 )
{
x264_log( h, X264_LOG_ERROR, "failed to parse zones\n" );
return -1;
}......2pass Omit the relevant content and don't read itThe called x264_ratecontrol_init_reconfigurable is mainly used to initialize vbv. If you don't know vbv, see This article , very vivid.
x264_ratecontrol_start
/* Before encoding a frame, choose a QP for it */
//called by x264_encode_encode
//Calculate the QP value of a frame and control the frame layer bit rate. At this step, all macroblocks in a frame are unified QP
void x264_ratecontrol_start( x264_t *h, int i_force_qp, int overhead )
{
x264_ratecontrol_t *rc = h->rc;
ratecontrol_entry_t *rce = NULL; //used for 2pass
x264_zone_t *zone = get_zone( h, h->fenc->i_frame );
float q;
x264_emms();
if( zone && (!rc->prev_zone || zone->param != rc->prev_zone->param) )
x264_encoder_reconfig_apply( h, zone->param );
rc->prev_zone = zone;
if( h->param.rc.b_stat_read )
{
int frame = h->fenc->i_frame;
assert( frame >= 0 && frame < rc->num_entries );
rce = h->rc->rce = &h->rc->entry[frame];
if( h->sh.i_type == SLICE_TYPE_B
&& h->param.analyse.i_direct_mv_pred == X264_DIRECT_PRED_AUTO )
{
h->sh.b_direct_spatial_mv_pred = ( rce->direct_mode == 's' );
h->mb.b_direct_auto_read = ( rce->direct_mode == 's' || rce->direct_mode == 't' );
}
}
if( rc->b_vbv )
{
// Initialize several parameters of the reconstructed frame, which are mainly used to determine the adaptive B frame
memset( h->fdec->i_row_bits, 0, h->mb.i_mb_height * sizeof(int) );
memset( h->fdec->f_row_qp, 0, h->mb.i_mb_height * sizeof(float) );
memset( h->fdec->f_row_qscale, 0, h->mb.i_mb_height * sizeof(float) );
rc->row_pred = rc->row_preds[h->sh.i_type];
rc->buffer_rate = h->fenc->i_cpb_duration * rc->vbv_max_rate * h->sps->vui.i_num_units_in_tick / h->sps->vui.i_time_scale;
// Update VBV temporarily according to the estimated size of all current frames, that is, update H - > RC - > buffer_ Fill is to update the buffer
update_vbv_plan( h, overhead );
//h264 level
const x264_level_t *l = x264_levels;
while( l->level_idc != 0 && l->level_idc != h->param.i_level_idc )
l++;
// Obtain the minimum compression ratio of this h264 level, for example: level 3.1, mincr value is 4
int mincr = l->mincr;
// Blu ray disc compression, the minimum compression ratio is 4
if( h->param.b_bluray_compat )
mincr = 4;
/* Profiles above High don't require minCR, so just set the maximum to a large value. */
// The level of profile higher than high does not need mincr, so set the maximum value of the frame to a very large value
// Various values of profile are defined in the file common / set H medium
if( h->sps->i_profile_idc > PROFILE_HIGH )
rc->frame_size_maximum = 1e9;
else
{
/* The spec has a bizarre special case for the first frame. */
if( h->i_frame == 0 )
{
//384 * ( Max( PicSizeInMbs, fR * MaxMBPS ) + MaxMBPS * ( tr( 0 ) - tr,n( 0 ) ) ) / MinCR
double fr = 1. / 172;
int pic_size_in_mbs = h->mb.i_mb_width * h->mb.i_mb_height;
// Maximum frame size due to MinCR, calculate the maximum size of each frame
rc->frame_size_maximum = 384 * BIT_DEPTH * X264_MAX( pic_size_in_mbs, fr*l->mbps ) / mincr;
}
else
{
//384 * MaxMBPS * ( tr( n ) - tr( n - 1 ) ) / MinCR
rc->frame_size_maximum = 384 * BIT_DEPTH * ((double)h->fenc->i_cpb_duration * h->sps->vui.i_num_units_in_tick / h->sps->vui.i_time_scale) * l->mbps / mincr;
}
}
}//if(rc->vbv)
if( h->sh.i_type != SLICE_TYPE_B )
rc->bframes = h->fenc->i_bframes;
// The qp value is calculated according to different rate control methods
if( rc->b_abr )
{
// update qscale for 1 frame based on actual bits used so far
q = qscale2qp( rate_estimate_qscale( h ) );
}
else if( rc->b_2pass )
{
rce->new_qscale = rate_estimate_qscale( h );
q = qscale2qp( rce->new_qscale );
}
else /* CQP */
{
if( h->sh.i_type == SLICE_TYPE_B && h->fdec->b_kept_as_ref )
q = ( rc->qp_constant[ SLICE_TYPE_B ] + rc->qp_constant[ SLICE_TYPE_P ] ) / 2;
else
q = rc->qp_constant[ h->sh.i_type ];
if( zone )
{
if( zone->b_force_qp )
q += zone->i_qp - rc->qp_constant[SLICE_TYPE_P];
else
q -= 6*log2f( zone->f_bitrate_factor );
}
}
if( i_force_qp != X264_QP_AUTO )
q = i_force_qp - 1;
q = x264_clip3f( q, h->param.rc.i_qp_min, h->param.rc.i_qp_max );
// Set the qp of the current frame and the average qp of macroblocks before and after aq
rc->qpa_rc = rc->qpa_rc_prev =
rc->qpa_aq = rc->qpa_aq_prev = 0;
rc->qp = x264_clip3( q + 0.5f, 0, QP_MAX );
h->fdec->f_qp_avg_rc =
h->fdec->f_qp_avg_aq =
rc->qpm = q;// qp value of the current macroblock
if( rce )
rce->new_qp = rc->qp;
//Update H - > RC - > Accum_ p_ Value of QP
accum_p_qp_update( h, rc->qpm );
// Record the type of the last non-B frame
if( h->sh.i_type != SLICE_TYPE_B )
rc->last_non_b_pict_type = h->sh.i_type;
}rate_estimate_qscale
// update qscale for 1 frame based on actual bits used so far
//Get the QP and satd values of a frame
/*One of the cores of the rate control part, the other is get_scale
* 0,Calculate the fuzzy complexity of SATD and image
* 1,In get_ In scale, the exponential model is adopted according to the complexity to obtain qscale
* rcc->last_qscale = pow( rce->blurred_complexity, 1 - rcc->qcompress )/rate_factor
* 2,In get_ In scale, qp is adjusted according to the complexity and the number of target bits
* q /= rate_factor; //rcc->wanted_bits_window / rcc->cplxr_sum
* 3,The qp is adjusted again according to the deviation between the actual number of bits of the encoded frame and the target number of bits
* overflow = x264_clip3f( 1.0 + (total_bits - wanted_bits) / abr_buffer, .5, 2 );
q *= overflow;
*/
static float rate_estimate_qscale( x264_t *h )
{
float q;
x264_ratecontrol_t *rcc = h->rc;
ratecontrol_entry_t rce = {0};
int pict_type = h->sh.i_type;
//zhanghui:will there be total_bits if not 2pass，no
int64_t total_bits = 8*(h->stat.i_frame_size[SLICE_TYPE_I]
+ h->stat.i_frame_size[SLICE_TYPE_P]
+ h->stat.i_frame_size[SLICE_TYPE_B])
- rcc->filler_bits_sum;
if( rcc->b_2pass )
{
rce = *rcc->rce;
if( pict_type != rce.pict_type )
{
x264_log( h, X264_LOG_ERROR, "slice=%c but 2pass stats say %c\n",
slice_type_to_char[pict_type], slice_type_to_char[rce.pict_type] );
}
}
if( pict_type == SLICE_TYPE_B )
{
/* B-frames don't have independent ratecontrol, but rather get the
* average QP of the two adjacent P-frames + an offset */
//The QP of frame B is calculated from the QP values of two adjacent reference frames and averaged
int i0 = IS_X264_TYPE_I(h->fref_nearest[0]->i_type);
int i1 = IS_X264_TYPE_I(h->fref_nearest[1]->i_type);
int dt0 = abs(h->fenc->i_poc - h->fref_nearest[0]->i_poc);
int dt1 = abs(h->fenc->i_poc - h->fref_nearest[1]->i_poc);
float q0 = h->fref_nearest[0]->f_qp_avg_rc;
float q1 = h->fref_nearest[1]->f_qp_avg_rc;
if( h->fref_nearest[0]->i_type == X264_TYPE_BREF )
q0 -= rcc->pb_offset/2;
if( h->fref_nearest[1]->i_type == X264_TYPE_BREF )
q1 -= rcc->pb_offset/2;
if( i0 && i1 )
q = (q0 + q1) / 2 + rcc->ip_offset;
else if( i0 ) //Take the one that is not I frame
q = q1;
else if( i1 )
q = q0;
else //If none of them are I frames
q = (q0*dt1 + q1*dt0) / (dt0 + dt1);
if( h->fenc->b_kept_as_ref )
q += rcc->pb_offset/2;
else
q += rcc->pb_offset;
//Estimate frame_size
if( rcc->b_2pass && rcc->b_vbv )
rcc->frame_size_planned = qscale2bits( &rce, qp2qscale( q ) );
else
rcc->frame_size_planned = predict_size( rcc->pred_b_from_p, qp2qscale( q ), h->fref[1][h->i_ref[1]-1]->i_satd );
/* Limit planned size by MinCR */
if( rcc->b_vbv )
rcc->frame_size_planned = X264_MIN( rcc->frame_size_planned, rcc->frame_size_maximum );
h->rc->frame_size_estimated = rcc->frame_size_planned;
/* For row SATDs */
if( rcc->b_vbv )
//Get the calculated satd value from slice decision
rcc->last_satd = x264_rc_analyse_slice( h );
rcc->qp_novbv = q;
return qp2qscale( q );
}//if( pict_type == SLICE_TYPE_B )
else
{
//Buffer initial value
double abr_buffer = 2 * rcc->rate_tolerance * rcc->bitrate;
if( rcc->b_2pass )
{
double lmin = rcc->lmin[pict_type];
double lmax = rcc->lmax[pict_type];
int64_t diff;
int64_t predicted_bits = total_bits;
if( rcc->b_vbv )
{
if( h->i_thread_frames > 1 )
{
int j = h->rc - h->thread[0]->rc;
for( int i = 1; i < h->i_thread_frames; i++ )
{
x264_t *t = h->thread[ (j+i)%h->i_thread_frames ];
double bits = t->rc->frame_size_planned;
if( !t->b_thread_active )
continue;
bits = X264_MAX(bits, t->rc->frame_size_estimated);
predicted_bits += (int64_t)bits;
}
}
}
else
{
if( h->i_frame < h->i_thread_frames )
predicted_bits += (int64_t)h->i_frame * rcc->bitrate / rcc->fps;
else
predicted_bits += (int64_t)(h->i_thread_frames - 1) * rcc->bitrate / rcc->fps;
}
/* Adjust ABR buffer based on distance to the end of the video. */
if( rcc->num_entries > h->i_frame )
{
double final_bits = rcc->entry[rcc->num_entries-1].expected_bits;
double video_pos = rce.expected_bits / final_bits;
double scale_factor = sqrt( (1 - video_pos) * rcc->num_entries );
abr_buffer *= 0.5 * X264_MAX( scale_factor, 0.5 );
}
diff = predicted_bits - (int64_t)rce.expected_bits;
q = rce.new_qscale;
q /= x264_clip3f((double)(abr_buffer - diff) / abr_buffer, .5, 2);
if( ((h->i_frame + 1 - h->i_thread_frames) >= rcc->fps) &&
(rcc->expected_bits_sum > 0))
{
/* Adjust quant based on the difference between
* achieved and expected bitrate so far */
double cur_time = (double)h->i_frame / rcc->num_entries;
double w = x264_clip3f( cur_time*100, 0.0, 1.0 );
q *= pow( (double)total_bits / rcc->expected_bits_sum, w );
}
rcc->qp_novbv = qscale2qp( q );
if( rcc->b_vbv )
{
/* Do not overflow vbv */
double expected_size = qscale2bits( &rce, q );
double expected_vbv = rcc->buffer_fill + rcc->buffer_rate - expected_size;
double expected_fullness = rce.expected_vbv / rcc->buffer_size;
double qmax = q*(2 - expected_fullness);
double size_constraint = 1 + expected_fullness;
qmax = X264_MAX( qmax, rce.new_qscale );
if( expected_fullness < .05 )
qmax = lmax;
qmax = X264_MIN(qmax, lmax);
while( ((expected_vbv < rce.expected_vbv/size_constraint) && (q < qmax)) ||
((expected_vbv < 0) && (q < lmax)))
{
q *= 1.05;
expected_size = qscale2bits(&rce, q);
expected_vbv = rcc->buffer_fill + rcc->buffer_rate - expected_size;
}
rcc->last_satd = x264_rc_analyse_slice( h );
}
q = x264_clip3f( q, lmin, lmax );
}//if( rcc->b_2pass )
else /* 1pass ABR */
{
/* Calculate the quantizer which would have produced the desired
* average bitrate if it had been applied to all frames so far.
* Then modulate that quant based on the current frame's complexity
* relative to the average complexity so far (using the 2pass RCEQ).
* Then bias the quant up or down if total size so far was far from
* the target.
* 1,Calculate a Qp value. If this value is applied to all current frames, can the target average rate be obtained
* 2,Adjust the QP according to the difference between the complexity of the current frame and the average complexity
* 3,If the total size is too different from the target, adjust the QP again
* Result: Depending on the value of rate_tolerance, there is a
* tradeoff between quality and bitrate precision. But at large
* tolerances, the bit distribution approaches that of 2pass. */
double wanted_bits, overflow = 1;
//First, calculate the SATD of the current frame
rcc->last_satd = x264_rc_analyse_slice( h );
//Calculate cplxsum (cumulative complexity) and cplxcount (weighted cumulative frames)
rcc->short_term_cplxsum *= 0.5;
rcc->short_term_cplxcount *= 0.5;
rcc->short_term_cplxsum += rcc->last_satd / (CLIP_DURATION(h->fenc->f_duration) / BASE_FRAME_DURATION);
rcc->short_term_cplxcount ++;
rce.tex_bits = rcc->last_satd;
//Calculate the fuzzy complexity of the image
rce.blurred_complexity = rcc->short_term_cplxsum / rcc->short_term_cplxcount;
rce.mv_bits = 0;
rce.p_count = rcc->nmb;
rce.i_count = 0;
rce.s_count = 0;
rce.qscale = 1;
rce.pict_type = pict_type;
rce.i_duration = h->fenc->i_duration;
if( h->param.rc.i_rc_method == X264_RC_CRF )
{
q = get_qscale( h, &rce, rcc->rate_factor_constant, h->fenc->i_frame );
}
else
{
q = get_qscale( h, &rce, rcc->wanted_bits_window / rcc->cplxr_sum, h->fenc->i_frame );
/* ABR code can potentially be counterproductive in CBR, so just don't bother.
* Don't run it if the frame complexity is zero either. */
if( !rcc->b_vbv_min_rate && rcc->last_satd )
{
// FIXME is it simpler to keep track of wanted_bits in ratecontrol_end?
int i_frame_done = h->i_frame + 1 - h->i_thread_frames;
double time_done = i_frame_done / rcc->fps;
if( h->param.b_vfr_input && i_frame_done > 0 )
time_done = ((double)(h->fenc->i_reordered_pts - h->i_reordered_pts_delay)) * h->param.i_timebase_num / h->param.i_timebase_den;
wanted_bits = time_done * rcc->bitrate;
if( wanted_bits > 0 )
{
//Buffer growth function
abr_buffer *= X264_MAX( 1, sqrt( time_done ) );
overflow = x264_clip3f( 1.0 + (total_bits - wanted_bits) / abr_buffer, .5, 2 );
q *= overflow;
}
}
}
//If it's not the first I frame
if( pict_type == SLICE_TYPE_I && h->param.i_keyint_max > 1
/* should test _next_ pict type, but that isn't decided yet */
&& rcc->last_non_b_pict_type != SLICE_TYPE_I )
{
q = qp2qscale( rcc->accum_p_qp / rcc->accum_p_norm );
q /= fabs( h->param.rc.f_ip_factor );
}
//Otherwise, if it's not the first frame
else if( h->i_frame > 0 )
{
if( h->param.rc.i_rc_method != X264_RC_CRF )
{
/* Asymmetric clipping, because symmetric would prevent
* overflow control in areas of rapidly oscillating complexity */
double lmin = rcc->last_qscale_for[pict_type] / rcc->lstep;
double lmax = rcc->last_qscale_for[pict_type] * rcc->lstep;
if( overflow > 1.1 && h->i_frame > 3 )
lmax *= rcc->lstep;
else if( overflow < 0.9 )
lmin /= rcc->lstep;
q = x264_clip3f(q, lmin, lmax);
}
}
else if( h->param.rc.i_rc_method == X264_RC_CRF && rcc->qcompress != 1 )
{
q = qp2qscale( ABR_INIT_QP ) / fabs( h->param.rc.f_ip_factor );
}
rcc->qp_novbv = qscale2qp( q );
//FIXME use get_diff_limited_q() ?
q = clip_qscale( h, pict_type, q );
}
rcc->last_qscale_for[pict_type] =
rcc->last_qscale = q;
//Predict the frame size according to the final qscale
if( !(rcc->b_2pass && !rcc->b_vbv) && h->fenc->i_frame == 0 )
rcc->last_qscale_for[SLICE_TYPE_P] = q * fabs( h->param.rc.f_ip_factor );
if( rcc->b_2pass && rcc->b_vbv )
rcc->frame_size_planned = qscale2bits(&rce, q);
else
rcc->frame_size_planned = predict_size( &rcc->pred[h->sh.i_type], q, rcc->last_satd );
/* Always use up the whole VBV in this case. */
if( rcc->single_frame_vbv )
rcc->frame_size_planned = rcc->buffer_rate;
/* Limit planned size by MinCR */
if( rcc->b_vbv )
rcc->frame_size_planned = X264_MIN( rcc->frame_size_planned, rcc->frame_size_maximum );
h->rc->frame_size_estimated = rcc->frame_size_planned;
return q;
}
}get_qscale
/**
* modify the bitrate curve from pass1 for one frame
* One of the cores of the rate control part, the other is rate_estimate_qscale
* 1,According to the complexity, the exponential model is used to obtain qscale
* rcc->last_qscale = pow( rce->blurred_complexity, 1 - rcc->qcompress )/rate_factor
* 2,Adjust qp according to complexity and target bits
* q /= rate_factor; //rcc->wanted_bits_window / rcc->cplxr_sum
* 3,At rate_ estimate_ In Qscale, qp is adjusted again according to the deviation between the actual number of bits of the encoded frame and the target number of bits
*/
static double get_qscale(x264_t *h, ratecontrol_entry_t *rce, double rate_factor, int frame_num)
{
x264_ratecontrol_t *rcc= h->rc;
x264_zone_t *zone = get_zone( h, frame_num );
double q;
if( h->param.rc.b_mb_tree )
{
double timescale = (double)h->sps->vui.i_num_units_in_tick / h->sps->vui.i_time_scale;
q = pow( BASE_FRAME_DURATION / CLIP_DURATION(rce->i_duration * timescale), 1 - h->param.rc.f_qcompress );
}
else
q = pow( rce->blurred_complexity, 1 - rcc->qcompress );
// avoid NaN's in the rc_eq
if( !isfinite(q) || rce->tex_bits + rce->mv_bits == 0 )
q = rcc->last_qscale_for[rce->pict_type];
else
{
rcc->last_rceq = q;
q /= rate_factor; //rcc->wanted_bits_window / rcc->cplxr_sum
rcc->last_qscale = q;
}
if( zone )
{
if( zone->b_force_qp )
q = qp2qscale( zone->i_qp );
else
q /= zone->f_bitrate_factor;
}
return q;
}clip_qscale
// apply VBV constraints and clip qscale to between lmin and lmax
//According to the first-order model, the bits allocated are estimated according to satd, and the qscale is modified combined with the vbv limit
static double clip_qscale( x264_t *h, int pict_type, double q )
{
x264_ratecontrol_t *rcc = h->rc;
double lmin = rcc->lmin[pict_type];
double lmax = rcc->lmax[pict_type];
if( rcc->rate_factor_max_increment )
lmax = X264_MIN( lmax, qp2qscale( rcc->qp_novbv + rcc->rate_factor_max_increment ) );
double q0 = q;
/* B-frames are not directly subject to VBV,
* since they are controlled by the P-frames' QPs. */
if( rcc->b_vbv && rcc->last_satd > 0 )
{
double fenc_cpb_duration = (double)h->fenc->i_cpb_duration *
h->sps->vui.i_num_units_in_tick / h->sps->vui.i_time_scale;
/* Lookahead VBV: raise the quantizer as necessary such that no frames in
* the lookahead overflow and such that the buffer is in a reasonable state
* by the end of the lookahead. */
if( h->param.rc.i_lookahead )
{
int terminate = 0;
/* Avoid an infinite loop. */
for( int iterations = 0; iterations < 1000 && terminate != 3; iterations++ )
{
double frame_q[3];
double cur_bits = predict_size( &rcc->pred[h->sh.i_type], q, rcc->last_satd );
double buffer_fill_cur = rcc->buffer_fill - cur_bits;
double target_fill;
double total_duration = 0;
double last_duration = fenc_cpb_duration;
frame_q[0] = h->sh.i_type == SLICE_TYPE_I ? q * h->param.rc.f_ip_factor : q;
frame_q[1] = frame_q[0] * h->param.rc.f_pb_factor;
frame_q[2] = frame_q[0] / h->param.rc.f_ip_factor;
/* Loop over the planned future frames. */
for( int j = 0; buffer_fill_cur >= 0 && buffer_fill_cur <= rcc->buffer_size; j++ )
{//If the next few frames are coded according to this qscale setting, how many buffer s will be used after all coding
total_duration += last_duration;
buffer_fill_cur += rcc->vbv_max_rate * last_duration;
int i_type = h->fenc->i_planned_type[j];
int i_satd = h->fenc->i_planned_satd[j];
if( i_type == X264_TYPE_AUTO )
break;
i_type = IS_X264_TYPE_I( i_type ) ? SLICE_TYPE_I : IS_X264_TYPE_B( i_type ) ? SLICE_TYPE_B : SLICE_TYPE_P;
cur_bits = predict_size( &rcc->pred[i_type], frame_q[i_type], i_satd );
buffer_fill_cur -= cur_bits;
last_duration = h->fenc->f_planned_cpb_duration[j];
}
/* Try to get to get the buffer at least 50% filled, but don't set an impossible goal. */
target_fill = X264_MIN( rcc->buffer_fill + total_duration * rcc->vbv_max_rate * 0.5, rcc->buffer_size * 0.5 );
if( buffer_fill_cur < target_fill )
{//Zhang Hui: if you can't meet your expectations, increase qscale?? Why, wouldn't it be smaller?
q *= 1.01;
terminate |= 1;
continue;
}
/* Try to get the buffer no more than 80% filled, but don't set an impossible goal. */
target_fill = x264_clip3f( rcc->buffer_fill - total_duration * rcc->vbv_max_rate * 0.5, rcc->buffer_size * 0.8, rcc->buffer_size );
if( rcc->b_vbv_min_rate && buffer_fill_cur > target_fill )
{//Zhang Hui: if it exceeds the expectation, reduce qscale?? Why? Wouldn't it be bigger?
q /= 1.01;
terminate |= 2;
continue;
}
break;
}
}
/* Fallback to old purely-reactive algorithm: no lookahead. */
//if( h->param.rc.i_lookahead )
else
{
if( ( pict_type == SLICE_TYPE_P ||
( pict_type == SLICE_TYPE_I && rcc->last_non_b_pict_type == SLICE_TYPE_I ) ) &&
rcc->buffer_fill/rcc->buffer_size < 0.5 )
{
q /= x264_clip3f( 2.0*rcc->buffer_fill/rcc->buffer_size, 0.5, 1.0 );
}
/* Now a hard threshold to make sure the frame fits in VBV.
* This one is mostly for I-frames. */
double bits = predict_size( &rcc->pred[h->sh.i_type], q, rcc->last_satd );
/* For small VBVs, allow the frame to use up the entire VBV. */
double max_fill_factor = h->param.rc.i_vbv_buffer_size >= 5*h->param.rc.i_vbv_max_bitrate / rcc->fps ? 2 : 1;
/* For single-frame VBVs, request that the frame use up the entire VBV. */
double min_fill_factor = rcc->single_frame_vbv ? 1 : 2;
if( bits > rcc->buffer_fill/max_fill_factor )
{
double qf = x264_clip3f( rcc->buffer_fill/(max_fill_factor*bits), 0.2, 1.0 );
q /= qf;
bits *= qf;
}
if( bits < rcc->buffer_rate/min_fill_factor )
{
double qf = x264_clip3f( bits*min_fill_factor/rcc->buffer_rate, 0.001, 1.0 );
q *= qf;
}
q = X264_MAX( q0, q );
}
/* Check B-frame complexity, and use up any bits that would
* overflow before the next P-frame. */
if( h->sh.i_type == SLICE_TYPE_P && !rcc->single_frame_vbv )
{
int nb = rcc->bframes;
double bits = predict_size( &rcc->pred[h->sh.i_type], q, rcc->last_satd );
double pbbits = bits;
double bbits = predict_size( rcc->pred_b_from_p, q * h->param.rc.f_pb_factor, rcc->last_satd );
double space;
double bframe_cpb_duration = 0;
double minigop_cpb_duration;
for( int i = 0; i < nb; i++ )
bframe_cpb_duration += h->fenc->f_planned_cpb_duration[i];
if( bbits * nb > bframe_cpb_duration * rcc->vbv_max_rate )
nb = 0;
pbbits += nb * bbits;
minigop_cpb_duration = bframe_cpb_duration + fenc_cpb_duration;
space = rcc->buffer_fill + minigop_cpb_duration*rcc->vbv_max_rate - rcc->buffer_size;
if( pbbits < space )
{
q *= X264_MAX( pbbits / space, bits / (0.5 * rcc->buffer_size) );
}
q = X264_MAX( q0/2, q );
}
/* Apply MinCR and buffer fill restrictions */
double bits = predict_size( &rcc->pred[h->sh.i_type], q, rcc->last_satd );
double frame_size_maximum = X264_MIN( rcc->frame_size_maximum, X264_MAX( rcc->buffer_fill, 0.001 ) );
if( bits > frame_size_maximum )
q *= bits / frame_size_maximum;
if( !rcc->b_vbv_min_rate )
q = X264_MAX( q0, q );
}
if( lmin==lmax )
return lmin;
else if( rcc->b_2pass )
{
double min2 = log( lmin );
double max2 = log( lmax );
q = (log(q) - min2)/(max2-min2) - 0.5;
q = 1.0/(1.0 + exp( -4*q ));
q = q*(max2-min2) + min2;
return exp( q );
}
else
return x264_clip3f( q, lmin, lmax );
}accum_p_qp_update
//Update H - > RC - > Accum_ p_ Value of QP
static void accum_p_qp_update( x264_t *h, float qp )
{
x264_ratecontrol_t *rc = h->rc;
rc->accum_p_qp *= .95;
rc->accum_p_norm *= .95;
rc->accum_p_norm += 1;
if( h->sh.i_type == SLICE_TYPE_I )
rc->accum_p_qp += qp + rc->ip_offset;
else
rc->accum_p_qp += qp;
}x264_ratecontrol_qp
//Confirm that QP is within the specified range
int x264_ratecontrol_qp( x264_t *h )
{
x264_emms();
return x264_clip3( h->rc->qpm + 0.5f, h->param.rc.i_qp_min, h->param.rc.i_qp_max );
}x264_ratecontrol_mb_qp
//The role of adaptive quantization is reflected here
int x264_ratecontrol_mb_qp( x264_t *h )
{
x264_emms();
float qp = h->rc->qpm;
if( h->param.rc.i_aq_mode )
{
/* MB-tree currently doesn't adjust quantizers in unreferenced frames. */
float qp_offset = h->fdec->b_kept_as_ref ? h->fenc->f_qp_offset[h->mb.i_mb_xy] : h->fenc->f_qp_offset_aq[h->mb.i_mb_xy];
/* Scale AQ's effect towards zero in emergency mode. */
if( qp > QP_MAX_SPEC )
qp_offset *= (QP_MAX - qp) / (QP_MAX - QP_MAX_SPEC);
qp += qp_offset;
}
return x264_clip3( qp + 0.5f, h->param.rc.i_qp_min, h->param.rc.i_qp_max );
}x264_ratecontrol_mb
/* TODO:
* eliminate all use of qp in row ratecontrol: make it entirely qscale-based.
* make this function stop being needlessly O(N^2)
* update more often than once per row? */
//Macroblock level rate control? I think it's more about increasing or decreasing according to the buffer and framesize, which is based on a line of macroblocks
//Why in the last part of entropy coding, we should give guidance to the next line of coding, which is equivalent to updating the model
int x264_ratecontrol_mb( x264_t *h, int bits )
{
x264_ratecontrol_t *rc = h->rc;
const int y = h->mb.i_mb_y;
//qp in this line
h->fdec->i_row_bits[y] += bits;
//Add this qp to calculate the average qp. This qp is used by aq
rc->qpa_aq += h->mb.i_qp;
//Before the end of this line, return
if( h->mb.i_mb_x != h->mb.i_mb_width - 1 )
return 0;
x264_emms();
//Before aq, all QPS are the same, so this calculation is used to calculate the average qp
rc->qpa_rc += rc->qpm * h->mb.i_mb_width;
//No vbv, return
if( !rc->b_vbv )
return 0;
float qscale = qp2qscale( rc->qpm );
//Reconstruct the frame. The qp and qscale of this line are the same
h->fdec->f_row_qp[y] = rc->qpm;
h->fdec->f_row_qscale[y] = qscale;
//Update predictor according to satd, number of bits and qscale in this line
update_predictor( &rc->row_pred[0], qscale, h->fdec->i_row_satd[y], h->fdec->i_row_bits[y] );
if( h->sh.i_type == SLICE_TYPE_P && rc->qpm < h->fref[0][0]->f_row_qp[y] )
update_predictor( &rc->row_pred[1], qscale, h->fdec->i_row_satds[0][0][y], h->fdec->i_row_bits[y] );
/* update ratecontrol per-mbpair in MBAFF */
if( SLICE_MBAFF && !(y&1) )
return 0;
/* FIXME: We don't currently support the case where there's a slice
* boundary in between.
* slice It must be the whole line. In fact, it is usually the whole frame*/
int can_reencode_row = h->sh.i_first_mb <= ((h->mb.i_mb_y - SLICE_MBAFF) * h->mb.i_mb_stride);
/* tweak quality based on difference from predicted size */
//Adjust the quality according to the difference from the predicted size
float prev_row_qp = h->fdec->f_row_qp[y];
float qp_absolute_max = h->param.rc.i_qp_max;
if( rc->rate_factor_max_increment )
qp_absolute_max = X264_MIN( qp_absolute_max, rc->qp_novbv + rc->rate_factor_max_increment );
float qp_max = X264_MIN( prev_row_qp + h->param.rc.i_qp_step, qp_absolute_max );
float qp_min = X264_MAX( prev_row_qp - h->param.rc.i_qp_step, h->param.rc.i_qp_min );
float step_size = 0.5f;
float buffer_left_planned = rc->buffer_fill - rc->frame_size_planned;
float slice_size_planned = h->param.b_sliced_threads ? rc->slice_size_planned : rc->frame_size_planned;
float max_frame_error = X264_MAX( 0.05f, 1.0f / h->mb.i_mb_height );
float size_of_other_slices = 0;
if( h->param.b_sliced_threads )
{
float size_of_other_slices_planned = 0;
for( int i = 0; i < h->param.i_threads; i++ )
if( h != h->thread[i] )
{
size_of_other_slices += h->thread[i]->rc->frame_size_estimated;
size_of_other_slices_planned += h->thread[i]->rc->slice_size_planned;
}
float weight = rc->slice_size_planned / rc->frame_size_planned;
size_of_other_slices = (size_of_other_slices - size_of_other_slices_planned) * weight + size_of_other_slices_planned;
}
if( y < h->i_threadslice_end-1 )
{//Not to the last line
/* B-frames shouldn't use lower QP than their reference frames. */
if( h->sh.i_type == SLICE_TYPE_B )
{
qp_min = X264_MAX( qp_min, X264_MAX( h->fref[0][0]->f_row_qp[y+1], h->fref[1][0]->f_row_qp[y+1] ) );
rc->qpm = X264_MAX( rc->qpm, qp_min );
}
/* More threads means we have to be more cautious in letting ratecontrol use up extra bits. */
float rc_tol = buffer_left_planned / h->param.i_threads * rc->rate_tolerance;
float b1 = predict_row_size_sum( h, y, rc->qpm ) + size_of_other_slices;//Predict the framesize encoded with the current qp
/* Don't increase the row QPs until a sufficent amount of the bits of the frame have been processed, in case a flat */
/* area at the top of the frame was measured inaccurately. */
if( row_bits_so_far( h, y ) < 0.05f * slice_size_planned )
qp_max = qp_absolute_max = prev_row_qp;
//Not I slice
if( h->sh.i_type != SLICE_TYPE_I )
rc_tol *= 0.5f;
if( !rc->b_vbv_min_rate )
qp_min = X264_MAX( qp_min, rc->qp_novbv );
while( rc->qpm < qp_max //Maximum qp not exceeded
&& ((b1 > rc->frame_size_planned + rc_tol) || //framesize is expected to exceed the standard
(rc->buffer_fill - b1 < buffer_left_planned * 0.5f) || //Insufficient buffer expected
(b1 > rc->frame_size_planned && rc->qpm < rc->qp_novbv)) ) //It is estimated that framesize exceeds &&qp and does not exceed qp_novbv
{
rc->qpm += step_size;
b1 = predict_row_size_sum( h, y, rc->qpm ) + size_of_other_slices;
}
while( rc->qpm > qp_min //Not less than minimum qp
&& (rc->qpm > h->fdec->f_row_qp[0] || rc->single_frame_vbv)
&& ((b1 < rc->frame_size_planned * 0.8f && rc->qpm <= prev_row_qp) //framesize is much smaller than expected, and the qp of the current line is smaller than that of the previous line
|| b1 < (rc->buffer_fill - rc->buffer_size + rc->buffer_rate) * 1.1f) )
{
rc->qpm -= step_size;
b1 = predict_row_size_sum( h, y, rc->qpm ) + size_of_other_slices;
}
/* avoid VBV underflow or MinCR violation */
while( (rc->qpm < qp_absolute_max)
&& ((rc->buffer_fill - b1 < rc->buffer_rate * max_frame_error) ||
(rc->frame_size_maximum - b1 < rc->frame_size_maximum * max_frame_error)))
{
rc->qpm += step_size;
b1 = predict_row_size_sum( h, y, rc->qpm ) + size_of_other_slices;
}
//Repeatedly adjusted expected framesize
h->rc->frame_size_estimated = b1 - size_of_other_slices;
/* If the current row was large enough to cause a large QP jump, try re-encoding it. */
//The qp value after various adjustments is too large. Try to re encode
if( rc->qpm > qp_max && prev_row_qp < qp_max && can_reencode_row )
{
/* Bump QP to halfway in between... close enough. */
rc->qpm = x264_clip3f( (prev_row_qp + rc->qpm)*0.5f, prev_row_qp + 1.0f, qp_max );
rc->qpa_rc = rc->qpa_rc_prev;
rc->qpa_aq = rc->qpa_aq_prev;
h->fdec->i_row_bits[y] = 0;
h->fdec->i_row_bits[y-SLICE_MBAFF] = 0;
return -1;
}
}
else
{//To the last line
h->rc->frame_size_estimated = predict_row_size_sum( h, y, rc->qpm );
/* Last-ditch attempt: if the last row of the frame underflowed the VBV,
* try again.
* If the last line causes VBV underflow, try recoding*/
if( (h->rc->frame_size_estimated + size_of_other_slices) > (rc->buffer_fill - rc->buffer_rate * max_frame_error) &&
rc->qpm < qp_max && can_reencode_row )
{
rc->qpm = qp_max;
rc->qpa_rc = rc->qpa_rc_prev;
rc->qpa_aq = rc->qpa_aq_prev;
h->fdec->i_row_bits[y] = 0;
h->fdec->i_row_bits[y-SLICE_MBAFF] = 0;
return -1;
}
}
rc->qpa_rc_prev = rc->qpa_rc;
rc->qpa_aq_prev = rc->qpa_aq;
return 0;
}update_predictor
//Update the coefficients of the frame size prediction model
static void update_predictor( predictor_t *p, float q, float var, float bits )
{
float range = 1.5;
if( var < 10 )
return;
float old_coeff = p->coeff / p->count;
float new_coeff = X264_MAX( bits*q / var, p->coeff_min );
float new_coeff_clipped = x264_clip3f( new_coeff, old_coeff/range, old_coeff*range );
float new_offset = bits*q - new_coeff_clipped * var;
if( new_offset >= 0 )
new_coeff = new_coeff_clipped;
else
new_offset = 0;
p->count *= p->decay;
p->coeff *= p->decay;
p->offset *= p->decay;
p->count ++;
p->coeff += new_coeff;
p->offset += new_offset;
}x264_ratecontrol_end
/* After encoding one frame, save stats and update ratecontrol state */
//After encoding a frame, save the data and update the rc status
int x264_ratecontrol_end( x264_t *h, int bits, int *filler )
{
x264_ratecontrol_t *rc = h->rc;
const int *mbs = h->stat.frame.i_mb_count;
x264_emms();
//Frame count of each type
h->stat.frame.i_mb_count_skip = mbs[P_SKIP] + mbs[B_SKIP];
h->stat.frame.i_mb_count_i = mbs[I_16x16] + mbs[I_8x8] + mbs[I_4x4];
h->stat.frame.i_mb_count_p = mbs[P_L0] + mbs[P_8x8];
for( int i = B_DIRECT; i < B_8x8; i++ )
h->stat.frame.i_mb_count_p += mbs[i];
//qp mean
h->fdec->f_qp_avg_rc = rc->qpa_rc /= h->mb.i_mb_count;
h->fdec->f_qp_avg_aq = (float)rc->qpa_aq / h->mb.i_mb_count;
h->fdec->f_crf_avg = h->param.rc.f_rf_constant + h->fdec->f_qp_avg_rc - rc->qp_novbv;
//Output stat
if( h->param.rc.b_stat_write )
{
char c_type = h->sh.i_type==SLICE_TYPE_I ? (h->fenc->i_poc==0 ? 'I' : 'i')
: h->sh.i_type==SLICE_TYPE_P ? 'P'
: h->fenc->b_kept_as_ref ? 'B' : 'b';
int dir_frame = h->stat.frame.i_direct_score[1] - h->stat.frame.i_direct_score[0];
int dir_avg = h->stat.i_direct_score[1] - h->stat.i_direct_score[0];
char c_direct = h->mb.b_direct_auto_write ?
( dir_frame>0 ? 's' : dir_frame<0 ? 't' :
dir_avg>0 ? 's' : dir_avg<0 ? 't' : '-' )
: '-';
if( fprintf( rc->p_stat_file_out,
"in:%d out:%d type:%c dur:%"PRId64" cpbdur:%"PRId64" q:%.2f aq:%.2f tex:%d mv:%d misc:%d imb:%d pmb:%d smb:%d d:%c ref:",
h->fenc->i_frame, h->i_frame,
c_type, h->fenc->i_duration,
h->fenc->i_cpb_duration,
rc->qpa_rc, h->fdec->f_qp_avg_aq,
h->stat.frame.i_tex_bits,
h->stat.frame.i_mv_bits,
h->stat.frame.i_misc_bits,
h->stat.frame.i_mb_count_i,
h->stat.frame.i_mb_count_p,
h->stat.frame.i_mb_count_skip,
c_direct) < 0 )
goto fail;
/* Only write information for reference reordering once. */
int use_old_stats = h->param.rc.b_stat_read && rc->rce->refs > 1;
for( int i = 0; i < (use_old_stats ? rc->rce->refs : h->i_ref[0]); i++ )
{
int refcount = use_old_stats ? rc->rce->refcount[i]
: PARAM_INTERLACED ? h->stat.frame.i_mb_count_ref[0][i*2]
+ h->stat.frame.i_mb_count_ref[0][i*2+1]
: h->stat.frame.i_mb_count_ref[0][i];
if( fprintf( rc->p_stat_file_out, "%d ", refcount ) < 0 )
goto fail;
}
if( h->param.analyse.i_weighted_pred >= X264_WEIGHTP_SIMPLE && h->sh.weight[0][0].weightfn )
{
if( fprintf( rc->p_stat_file_out, "w:%d,%d,%d",
h->sh.weight[0][0].i_denom, h->sh.weight[0][0].i_scale, h->sh.weight[0][0].i_offset ) < 0 )
goto fail;
if( h->sh.weight[0][1].weightfn || h->sh.weight[0][2].weightfn )
{
if( fprintf( rc->p_stat_file_out, ",%d,%d,%d,%d,%d ",
h->sh.weight[0][1].i_denom, h->sh.weight[0][1].i_scale, h->sh.weight[0][1].i_offset,
h->sh.weight[0][2].i_scale, h->sh.weight[0][2].i_offset ) < 0 )
goto fail;
}
else if( fprintf( rc->p_stat_file_out, " " ) < 0 )
goto fail;
}
if( fprintf( rc->p_stat_file_out, ";\n") < 0 )
goto fail;
/* Don't re-write the data in multi-pass mode. */
if( h->param.rc.b_mb_tree && h->fenc->b_kept_as_ref && !h->param.rc.b_stat_read )
{
uint8_t i_type = h->sh.i_type;
/* Values are stored as big-endian FIX8.8 */
for( int i = 0; i < h->mb.i_mb_count; i++ )
rc->mbtree.qp_buffer[0][i] = endian_fix16( h->fenc->f_qp_offset[i]*256.0 );
if( fwrite( &i_type, 1, 1, rc->p_mbtree_stat_file_out ) < 1 )
goto fail;
if( fwrite( rc->mbtree.qp_buffer[0], sizeof(uint16_t), h->mb.i_mb_count, rc->p_mbtree_stat_file_out ) < h->mb.i_mb_count )
goto fail;
}
}
if( rc->b_abr )
{
//Update cplxr_sum and wanted_bits_window
if( h->sh.i_type != SLICE_TYPE_B )
rc->cplxr_sum += bits * qp2qscale( rc->qpa_rc ) / rc->last_rceq;
else
{
/* Depends on the fact that B-frame's QP is an offset from the following P-frame's.
* Not perfectly accurate with B-refs, but good enough. */
rc->cplxr_sum += bits * qp2qscale( rc->qpa_rc ) / (rc->last_rceq * fabs( h->param.rc.f_pb_factor ));
}
rc->cplxr_sum *= rc->cbr_decay;
rc->wanted_bits_window += h->fenc->f_duration * rc->bitrate;
rc->wanted_bits_window *= rc->cbr_decay;
}
if( rc->b_2pass )
rc->expected_bits_sum += qscale2bits( rc->rce, qp2qscale( rc->rce->new_qp ) );
if( h->mb.b_variable_qp )
{
if( h->sh.i_type == SLICE_TYPE_B )
{
rc->bframe_bits += bits;
if( h->fenc->b_last_minigop_bframe )
{
update_predictor( rc->pred_b_from_p, qp2qscale( rc->qpa_rc ),
h->fref[1][h->i_ref[1]-1]->i_satd, rc->bframe_bits / rc->bframes );
rc->bframe_bits = 0;
}
}
}
//Update vbv
*filler = update_vbv( h, bits );
rc->filler_bits_sum += *filler * 8;
if( h->sps->vui.b_nal_hrd_parameters_present )
{
if( h->fenc->i_frame == 0 )
{
// access unit initialises the HRD
h->fenc->hrd_timing.cpb_initial_arrival_time = 0;
rc->initial_cpb_removal_delay = h->initial_cpb_removal_delay;
rc->initial_cpb_removal_delay_offset = h->initial_cpb_removal_delay_offset;
h->fenc->hrd_timing.cpb_removal_time = rc->nrt_first_access_unit = (double)rc->initial_cpb_removal_delay / 90000;
}
else
{
h->fenc->hrd_timing.cpb_removal_time = rc->nrt_first_access_unit + (double)(h->fenc->i_cpb_delay - h->i_cpb_delay_pir_offset) *
h->sps->vui.i_num_units_in_tick / h->sps->vui.i_time_scale;
if( h->fenc->b_keyframe )
{
rc->nrt_first_access_unit = h->fenc->hrd_timing.cpb_removal_time;
rc->initial_cpb_removal_delay = h->initial_cpb_removal_delay;
rc->initial_cpb_removal_delay_offset = h->initial_cpb_removal_delay_offset;
}
double cpb_earliest_arrival_time = h->fenc->hrd_timing.cpb_removal_time - (double)rc->initial_cpb_removal_delay / 90000;
if( !h->fenc->b_keyframe )
cpb_earliest_arrival_time -= (double)rc->initial_cpb_removal_delay_offset / 90000;
if( h->sps->vui.hrd.b_cbr_hrd )
h->fenc->hrd_timing.cpb_initial_arrival_time = rc->previous_cpb_final_arrival_time;
else
h->fenc->hrd_timing.cpb_initial_arrival_time = X264_MAX( rc->previous_cpb_final_arrival_time, cpb_earliest_arrival_time );
}
int filler_bits = *filler ? X264_MAX( (FILLER_OVERHEAD - h->param.b_annexb), *filler )*8 : 0;
// Equation C-6
h->fenc->hrd_timing.cpb_final_arrival_time = rc->previous_cpb_final_arrival_time = h->fenc->hrd_timing.cpb_initial_arrival_time +
(double)(bits + filler_bits) / h->sps->vui.hrd.i_bit_rate_unscaled;
h->fenc->hrd_timing.dpb_output_time = (double)h->fenc->i_dpb_output_delay * h->sps->vui.i_num_units_in_tick / h->sps->vui.i_time_scale +
h->fenc->hrd_timing.cpb_removal_time;
}
return 0;
fail:
x264_log( h, X264_LOG_ERROR, "ratecontrol_end: stats file could not be written to\n" );
return -1;
}update_vbv
// update VBV after encoding a frame
static int update_vbv( x264_t *h, int bits )
{
int filler = 0;
int bitrate = h->sps->vui.hrd.i_bit_rate_unscaled;
x264_ratecontrol_t *rcc = h->rc;
x264_ratecontrol_t *rct = h->thread[0]->rc;
int64_t buffer_size = (int64_t)h->sps->vui.hrd.i_cpb_size_unscaled * h->sps->vui.i_time_scale;
if( rcc->last_satd >= h->mb.i_mb_count ) //Why compare the two
update_predictor( &rct->pred[h->sh.i_type], qp2qscale( rcc->qpa_rc ), rcc->last_satd, bits );
if( !rcc->b_vbv )
return filler;
uint64_t buffer_diff = (uint64_t)bits * h->sps->vui.i_time_scale;
rct->buffer_fill_final -= buffer_diff;
rct->buffer_fill_final_min -= buffer_diff;
if( rct->buffer_fill_final_min < 0 )
{//underflow happens
double underflow = (double)rct->buffer_fill_final_min / h->sps->vui.i_time_scale;
if( rcc->rate_factor_max_increment && rcc->qpm >= rcc->qp_novbv + rcc->rate_factor_max_increment )
x264_log( h, X264_LOG_DEBUG, "VBV underflow due to CRF-max (frame %d, %.0f bits)\n", h->i_frame, underflow );
else
x264_log( h, X264_LOG_WARNING, "VBV underflow (frame %d, %.0f bits)\n", h->i_frame, underflow );
rct->buffer_fill_final =
rct->buffer_fill_final_min = 0;
}
if( h->param.i_avcintra_class )
buffer_diff = buffer_size;
else
buffer_diff = (uint64_t)bitrate * h->sps->vui.i_num_units_in_tick * h->fenc->i_cpb_duration;
rct->buffer_fill_final += buffer_diff;
rct->buffer_fill_final_min += buffer_diff;
if( rct->buffer_fill_final > buffer_size )
{
if( h->param.rc.b_filler )
{
int64_t scale = (int64_t)h->sps->vui.i_time_scale * 8;
filler = (rct->buffer_fill_final - buffer_size + scale - 1) / scale;
bits = h->param.i_avcintra_class ? filler * 8 : X264_MAX( (FILLER_OVERHEAD - h->param.b_annexb), filler ) * 8;
buffer_diff = (uint64_t)bits * h->sps->vui.i_time_scale;
rct->buffer_fill_final -= buffer_diff;
rct->buffer_fill_final_min -= buffer_diff;
}
else
{
rct->buffer_fill_final = X264_MIN( rct->buffer_fill_final, buffer_size );
rct->buffer_fill_final_min = X264_MIN( rct->buffer_fill_final_min, buffer_size );
}
}
return filler;
}Last x264_ratecontrol_summary and x264_ratecontrol_delete is omitted.
Some supplementary contents
1. What is Macroblock Tree Macroblock Tree is a qp control method based on macroblock. The working principle of MB Tree is similar to that of classical qp compression, except that the object processed by qcomp is the whole frame, while MB Tree is processed for each MB. The working process is simple. For each MB, predict the reference of the MB in a certain number of frames (the number is determined by the smaller value of RC lookahead and keyint), and determine the size of qp used for the MB according to the number of references. The size of qp is inversely proportional to the number of references, that is, for MB with a large number of references, the decoder of 264 believes that this corresponds to a slowly changing scene, so it gives relatively high quality (relatively low qp value). As for the relationship between the change rate of video and the perception ability of human eyes, this is an empirical result based on subjective test: the higher the change rate of video, the lower the sensitivity of human eyes. In other words, human eyes can tolerate some defects in rapidly changing scenes, but relatively speaking, human eyes are quite sensitive to the defects of some smooth scenes. Note that the smoothing mentioned here refers to the change frequency of the scene along the time dimension, rather than the scene in the pixel domain in the general sense. 2,MBTree File This is a temporary file that records the reference of each MB in each P frame. 3. Processing of tree objects It is said that at present, mbtree only deals with the mb of p frames and does not support bpyramid. I haven't looked closely yet. 4. Parameters related to Mbtree --qcomp qcomp tends to weaken the strength of mbtree. Specifically, the closer the value of qcomp is to 1 (Constant Quantizer), the worse the effectiveness of mbtree. --RC lookahead determines the number of frames predicted forward by mbtree. 5. There are two ways to adjust the qp of each mb in x264 (1) Start aq --- this method will slightly disturb the code rate control; (2) Enable vbv --- both vbv and abr are close to cbr mode. vbv will adjust qp line by line, and the bit rate control will be more accurate;
6,rate_ What is the unit of tolerance? No unit, just a simple multiplication coefficient, double ABR_ buffer = 2 * rcc->rate_tolerance * rcc->bitrate;
7,rate_ estimate_ Total in Qscale_ Does the bits thing always happen? No, it can only be read through the stat file when 2pass is used