xCheckRDCostMerge2Nx2N function is used to select the best candidate MV in Merge mode, and select the best motion information according to the predicted value and original value SATD Cost generated by conventional Merge mode, Merge mode with motion vector difference (MMVD) and CIIP mode.
It is mainly divided into the following processes:
① Get Merge mode candidate list
Firstly, the candidate lists of the conventional Merge mode and MMVD mode are obtained through the getInterMergeCandidates function and getInterMergeCandidates function respectively. The candidate lists of the conventional Merge are derived for reference: Extended Merge prediction , candidate list derivation reference of MMVD mode: Merge mode with MVD
② Initializes the list of mode information for RD selection
The definition of mode information is as follows, including the index of Merge candidate modes, whether it is a conventional Merge mode, MMVD mode and CIIP mode.
struct ModeInfo
{
uint32_t mergeCand;
bool isRegularMerge;
bool isMMVD;
bool isCIIP;
ModeInfo() : mergeCand(0), isRegularMerge(false), isMMVD(false), isCIIP(false) {}
ModeInfo(const uint32_t mergeCand, const bool isRegularMerge, const bool isMMVD, const bool isCIIP) :
mergeCand(mergeCand), isRegularMerge(isRegularMerge), isMMVD(isMMVD), isCIIP(isCIIP) {}
};The total number of available candidate modes is the number of available conventional Merge candidates + the number of MMVD candidates.
// The total number of available candidates is: the number of available conventional Merge candidates + the number of MMVD candidates const int candNum = mergeCtx.numValidMergeCand + (tempCS->sps->getUseMMVD() ? std::min<int>(MMVD_BASE_MV_NUM, mergeCtx.numValidMergeCand) * MMVD_MAX_REFINE_NUM : 0);
Add the candidate Merge mode information to the RD mode list RdModeList
③ If fast Merge mode selection is enabled or CIIP mode is available, reduce the number of modes for RD Cost fine selection
This step is to reduce the number of subsequent fine selected modes by calculating the predicted value and original value SAD of the candidate Merge mode and the number of bits of the prediction mode (at this time, it is not necessary to transform and quantify the residual)
First obtain the best encoding mode of the current block in the cache (I don't know much about????), Reduce the number of fine selection modes according to its best mode
1. If the best mode is not Skip mode, reduce the number of fine selection modes according to the SATD Cost of candidate modes
Traverse the conventional Merge candidate mode, perform motion compensation to obtain the corresponding predicted value, calculate SAD according to the brightness component of the predicted value, calculate RD Cost according to the bit number of SAD and prediction mode, update the RD mode list, and move the mode with low cost to the front of the list
If the CIIP mode is available, traverse the first few modes in the RD mode list (up to 4 modes), calculate the CIIP prediction pixels, calculate the distortion SAD and the number of mode bits R, calculate RD Cost, update the RD mode list, and move the mode with low cost to the front of the list
Traverse the MMVD candidate mode, perform motion compensation to obtain the corresponding predicted value, calculate SAD according to the brightness component of the predicted value, calculate RD Cost according to the bit number of SAD and prediction mode, update the RD mode list, and move the mode with low cost to the front of the list
Reduce the number of fine selection modes according to the candidate cost list: if the cost of the ith mode > the cost of the first mode * MRG_FAST_RATIO(1.25), set uiNumMrgSATDCand to i
2. If the best mode is MMVD Skip mode, the number of modes for fine selection is the number of available conventional Merge modes + the number of MMVD modes
3. Otherwise (if the best mode is Skip mode), the number of modes for fine selection is the available conventional Merge mode
④ After traversing the list of RD prediction modes, the best Skip and cost are calculated respectively
The code and notes are as follows;
void EncCu::xCheckRDCostMerge2Nx2N( CodingStructure *&tempCS, CodingStructure *&bestCS, Partitioner &partitioner, const EncTestMode& encTestMode )
{
const Slice &slice = *tempCS->slice;
CHECK( slice.getSliceType() == I_SLICE, "Merge modes not available for I-slices" );
tempCS->initStructData( encTestMode.qp );
MergeCtx mergeCtx;
const SPS &sps = *tempCS->sps;
if (sps.getSbTMVPEnabledFlag())
{
Size bufSize = g_miScaling.scale( tempCS->area.lumaSize() );
mergeCtx.subPuMvpMiBuf = MotionBuf( m_SubPuMiBuf, bufSize );
}
Mv refinedMvdL0[MAX_NUM_PARTS_IN_CTU][MRG_MAX_NUM_CANDS];
setMergeBestSATDCost( MAX_DOUBLE ); //Set the RD Cost of the best Merge mode to MAX_DOUBLE
{
// first get merge candidates
CodingUnit cu( tempCS->area );
cu.cs = tempCS;
cu.predMode = MODE_INTER;
cu.slice = tempCS->slice;
cu.tileIdx = tempCS->pps->getTileIdx( tempCS->area.lumaPos() );
PredictionUnit pu( tempCS->area );
pu.cu = &cu;
pu.cs = tempCS;
PU::getInterMergeCandidates(pu, mergeCtx
, 0
); //Get the general Merge selected list
PU::getInterMMVDMergeCandidates(pu, mergeCtx);//Get MMVD Merge candidate list
pu.regularMergeFlag = true;
}
bool candHasNoResidual[MRG_MAX_NUM_CANDS + MMVD_ADD_NUM]; //The total number of candidate modes without residuals is 6 (Merge) + 64 (MMVD Merge)
for (uint32_t ui = 0; ui < MRG_MAX_NUM_CANDS + MMVD_ADD_NUM; ui++)
{
candHasNoResidual[ui] = false;
}
bool bestIsSkip = false;//The best mode is Skip mode
bool bestIsMMVDSkip = true;//The best mode is MMVD Skip mode
PelUnitBuf acMergeBuffer[MRG_MAX_NUM_CANDS];//Store normal Merge mode prediction pixels
PelUnitBuf acMergeTmpBuffer[MRG_MAX_NUM_CANDS];//Temporary space used to store pixels predicted by conventional Merge mode
PelUnitBuf acMergeRealBuffer[MMVD_MRG_MAX_RD_BUF_NUM];
PelUnitBuf * acMergeTempBuffer[MMVD_MRG_MAX_RD_NUM];//Temporary space for storing MMVD Merge mode
PelUnitBuf * singleMergeTempBuffer;
int insertPos;
// The number of modes for SATD Cost calculation is initialized to the number of available conventional merges + 64, which will be reduced later
unsigned uiNumMrgSATDCand = mergeCtx.numValidMergeCand + MMVD_ADD_NUM;
struct ModeInfo
{
uint32_t mergeCand;
bool isRegularMerge;
bool isMMVD;
bool isCIIP;
ModeInfo() : mergeCand(0), isRegularMerge(false), isMMVD(false), isCIIP(false) {}
ModeInfo(const uint32_t mergeCand, const bool isRegularMerge, const bool isMMVD, const bool isCIIP) :
mergeCand(mergeCand), isRegularMerge(isRegularMerge), isMMVD(isMMVD), isCIIP(isCIIP) {}
};
static_vector<ModeInfo, MRG_MAX_NUM_CANDS + MMVD_ADD_NUM> RdModeList; //RD mode list 6 + 64, save mode information
bool mrgTempBufSet = false;
// The total number of available candidates is: the number of available conventional Merge candidates + the number of MMVD candidates
const int candNum = mergeCtx.numValidMergeCand + (tempCS->sps->getUseMMVD() ? std::min<int>(MMVD_BASE_MV_NUM, mergeCtx.numValidMergeCand) * MMVD_MAX_REFINE_NUM : 0);
// Traverse all available candidate modes and add them to the RD mode list
for (int i = 0; i < candNum; i++)
{
if (i < mergeCtx.numValidMergeCand)
{
RdModeList.push_back(ModeInfo(i, true, false, false)); //Add general candidate mode
}
else
{
// Add MMVD mode
RdModeList.push_back(ModeInfo(std::min(MMVD_ADD_NUM, i - mergeCtx.numValidMergeCand), false, true, false));
}
}
// Initialize an area of the same size as the current CU
const UnitArea localUnitArea(tempCS->area.chromaFormat, Area(0, 0, tempCS->area.Y().width, tempCS->area.Y().height));
for (unsigned i = 0; i < MMVD_MRG_MAX_RD_BUF_NUM; i++)
{
acMergeRealBuffer[i] = m_acMergeBuffer[i].getBuf(localUnitArea);
if (i < MMVD_MRG_MAX_RD_NUM)
{
acMergeTempBuffer[i] = acMergeRealBuffer + i;
}
else
{
singleMergeTempBuffer = acMergeRealBuffer + i;
}
}
bool isIntrainterEnabled = sps.getUseCiip(); //Intra and inter joint coding use flag
if (bestCS->area.lwidth() * bestCS->area.lheight() < 64 || bestCS->area.lwidth() >= MAX_CU_SIZE || bestCS->area.lheight() >= MAX_CU_SIZE)
{
isIntrainterEnabled = false;
}
bool isTestSkipMerge[MRG_MAX_NUM_CANDS]; // record if the merge candidate has tried skip mode
for (uint32_t idx = 0; idx < MRG_MAX_NUM_CANDS; idx++)
{
isTestSkipMerge[idx] = false;
}
if( m_pcEncCfg->getUseFastMerge() || isIntrainterEnabled) //Fast Merge mode or CIIP mode is available
{
uiNumMrgSATDCand = NUM_MRG_SATD_CAND; //Number of SATD candidate modes: 4
if (isIntrainterEnabled)
{
uiNumMrgSATDCand += 1; //Add a CIIP mode
}
bestIsSkip = false;
if( auto blkCache = dynamic_cast< CacheBlkInfoCtrl* >( m_modeCtrl ) )
{
if (slice.getSPS()->getIBCFlag())
{
ComprCUCtx cuECtx = m_modeCtrl->getComprCUCtx();
bestIsSkip = blkCache->isSkip(tempCS->area) && cuECtx.bestCU;
}
else
bestIsSkip = blkCache->isSkip( tempCS->area );
bestIsMMVDSkip = blkCache->isMMVDSkip(tempCS->area);
}
if (isIntrainterEnabled) // always perform low complexity check
{
bestIsSkip = false;
}
static_vector<double, MRG_MAX_NUM_CANDS + MMVD_ADD_NUM> candCostList; //Candidate cost list, save RD Cost
// 1. Pass: get SATD-cost for selected candidates and reduce their count
// 1. Pass: obtain the SATD cost of the selected candidate mode and reduce its number
if( !bestIsSkip )//If the best mode is not Skip mode
{
RdModeList.clear();
mrgTempBufSet = true;
const TempCtx ctxStart(m_CtxCache, m_CABACEstimator->getCtx());
CodingUnit &cu = tempCS->addCU( tempCS->area, partitioner.chType );
const double sqrtLambdaForFirstPassIntra = m_pcRdCost->getMotionLambda( ) * FRAC_BITS_SCALE;
partitioner.setCUData( cu );
cu.slice = tempCS->slice;
cu.tileIdx = tempCS->pps->getTileIdx( tempCS->area.lumaPos() );
cu.skip = false;
cu.mmvdSkip = false;
cu.geoFlag = false;
//cu.affine
cu.predMode = MODE_INTER;
//cu.LICFlag
cu.chromaQpAdj = m_cuChromaQpOffsetIdxPlus1;
cu.qp = encTestMode.qp;
//cu.emtFlag is set below
PredictionUnit &pu = tempCS->addPU( cu, partitioner.chType );
DistParam distParam;
const bool bUseHadamard = !tempCS->slice->getDisableSATDForRD(); //Using Hadamard Transform
m_pcRdCost->setDistParam (distParam, tempCS->getOrgBuf().Y(), m_acMergeBuffer[0].Y(), sps.getBitDepth (CHANNEL_TYPE_LUMA), COMPONENT_Y, bUseHadamard);
const UnitArea localUnitArea( tempCS->area.chromaFormat, Area( 0, 0, tempCS->area.Y().width, tempCS->area.Y().height) );
// Traverse the general available Merge candidates, calculate SAD and update the candidate list
for( uint32_t uiMergeCand = 0; uiMergeCand < mergeCtx.numValidMergeCand; uiMergeCand++ )
{
mergeCtx.setMergeInfo( pu, uiMergeCand );
PU::spanMotionInfo( pu, mergeCtx );
pu.mvRefine = true;
distParam.cur = singleMergeTempBuffer->Y();
acMergeTmpBuffer[uiMergeCand] = m_acMergeTmpBuffer[uiMergeCand].getBuf(localUnitArea);
// motion compensation
m_pcInterSearch->motionCompensation(pu, *singleMergeTempBuffer, REF_PIC_LIST_X, true, true, &(acMergeTmpBuffer[uiMergeCand]));
acMergeBuffer[uiMergeCand] = m_acRealMergeBuffer[uiMergeCand].getBuf(localUnitArea);
acMergeBuffer[uiMergeCand].copyFrom(*singleMergeTempBuffer);
pu.mvRefine = false;
if( mergeCtx.interDirNeighbours[uiMergeCand] == 3 && mergeCtx.mrgTypeNeighbours[uiMergeCand] == MRG_TYPE_DEFAULT_N )
{
mergeCtx.mvFieldNeighbours[2*uiMergeCand].mv = pu.mv[0];
mergeCtx.mvFieldNeighbours[2*uiMergeCand+1].mv = pu.mv[1];
{
int dx, dy, i, j, num = 0;
dy = std::min<int>(pu.lumaSize().height, DMVR_SUBCU_HEIGHT);
dx = std::min<int>(pu.lumaSize().width, DMVR_SUBCU_WIDTH);
if (PU::checkDMVRCondition(pu))
{
for (i = 0; i < (pu.lumaSize().height); i += dy)
{
for (j = 0; j < (pu.lumaSize().width); j += dx)
{
refinedMvdL0[num][uiMergeCand] = pu.mvdL0SubPu[num];
num++;
}
}
}
}
}
Distortion uiSad = distParam.distFunc(distParam); //Calculate SAD
m_CABACEstimator->getCtx() = ctxStart;
uint64_t fracBits = m_pcInterSearch->xCalcPuMeBits(pu);
double cost = (double)uiSad + (double)fracBits * sqrtLambdaForFirstPassIntra; //Calculate Cost
insertPos = -1;
// Update cost list
updateCandList(ModeInfo(uiMergeCand, true, false, false), cost, RdModeList, candCostList, uiNumMrgSATDCand, &insertPos);
if (insertPos != -1)
{
if (insertPos == RdModeList.size() - 1)
{
swap(singleMergeTempBuffer, acMergeTempBuffer[insertPos]);
}
else
{
for (uint32_t i = uint32_t(RdModeList.size()) - 1; i > insertPos; i--)
{
swap(acMergeTempBuffer[i - 1], acMergeTempBuffer[i]);
}
swap(singleMergeTempBuffer, acMergeTempBuffer[insertPos]);
}
}
CHECK(std::min(uiMergeCand + 1, uiNumMrgSATDCand) != RdModeList.size(), "");
} // for( uint32_t uiMergeCand = 0; uiMergeCand < mergeCtx.numValidMergeCand; uiMergeCand++ )
if (isIntrainterEnabled) //CIIP mode available
{
// prepare for Intra bits calculation
pu.ciipFlag = true;
// Save the to be tested merge candidates
uint32_t CiipMergeCand[NUM_MRG_SATD_CAND];
for (uint32_t mergeCnt = 0; mergeCnt < std::min(NUM_MRG_SATD_CAND, (const int)mergeCtx.numValidMergeCand); mergeCnt++)
{
CiipMergeCand[mergeCnt] = RdModeList[mergeCnt].mergeCand;
}
// Traversing the CIIP mode requires testing the candidate Merge mode
for (uint32_t mergeCnt = 0; mergeCnt < std::min(std::min(NUM_MRG_SATD_CAND, (const int)mergeCtx.numValidMergeCand), 4); mergeCnt++)
{
uint32_t mergeCand = CiipMergeCand[mergeCnt];
acMergeTmpBuffer[mergeCand] = m_acMergeTmpBuffer[mergeCand].getBuf(localUnitArea);
// Estimate Merge bit s
mergeCtx.setMergeInfo(pu, mergeCand);
// first round
pu.intraDir[0] = PLANAR_IDX;//Intra Planar mode
uint32_t intraCnt = 0;
// Generate intra y prediction generates intra inter joint prediction pixels
if (mergeCnt == 0)
{//Planar prediction pixels are calculated only once in the first Merge candidate mode
m_pcIntraSearch->initIntraPatternChType(*pu.cu, pu.Y());
m_pcIntraSearch->predIntraAng(COMPONENT_Y, pu.cs->getPredBuf(pu).Y(), pu);
m_pcIntraSearch->switchBuffer(pu, COMPONENT_Y, pu.cs->getPredBuf(pu).Y(), m_pcIntraSearch->getPredictorPtr2(COMPONENT_Y, intraCnt));
}
pu.cs->getPredBuf(pu).copyFrom(acMergeTmpBuffer[mergeCand]); //Load the inter prediction pixels generated by the corresponding Merge mode
if (pu.cs->slice->getLmcsEnabledFlag() && m_pcReshape->getCTUFlag())
{
pu.cs->getPredBuf(pu).Y().rspSignal(m_pcReshape->getFwdLUT());
}
// Calculate CIIP prediction pixels
m_pcIntraSearch->geneWeightedPred(COMPONENT_Y, pu.cs->getPredBuf(pu).Y(), pu, m_pcIntraSearch->getPredictorPtr2(COMPONENT_Y, intraCnt));
// calculate cost
if (pu.cs->slice->getLmcsEnabledFlag() && m_pcReshape->getCTUFlag())
{
pu.cs->getPredBuf(pu).Y().rspSignal(m_pcReshape->getInvLUT());
}
distParam.cur = pu.cs->getPredBuf(pu).Y();
Distortion sadValue = distParam.distFunc(distParam);
if (pu.cs->slice->getLmcsEnabledFlag() && m_pcReshape->getCTUFlag())
{
pu.cs->getPredBuf(pu).Y().rspSignal(m_pcReshape->getFwdLUT());
}
m_CABACEstimator->getCtx() = ctxStart;
pu.regularMergeFlag = false;
uint64_t fracBits = m_pcInterSearch->xCalcPuMeBits(pu);
double cost = (double)sadValue + (double)fracBits * sqrtLambdaForFirstPassIntra;
insertPos = -1;
// Update candidate mode list
updateCandList(ModeInfo(mergeCand, false, false, true), cost, RdModeList, candCostList, uiNumMrgSATDCand, &insertPos);
if (insertPos != -1)
{
for (int i = int(RdModeList.size()) - 1; i > insertPos; i--)
{
swap(acMergeTempBuffer[i - 1], acMergeTempBuffer[i]);
}
swap(singleMergeTempBuffer, acMergeTempBuffer[insertPos]);
}
}
pu.ciipFlag = false;
}// if (isIntrainterEnabled) CIIP mode
if ( pu.cs->sps->getUseMMVD() ) //Using MMVD mode, the best Merge candidate of MMVD is selected here (initial MV + search direction + search step)
{
cu.mmvdSkip = true;
pu.regularMergeFlag = true; //The common Merge list is still available, because the initial MV still needs to be obtained from the common Merge
const int tempNum = (mergeCtx.numValidMergeCand > 1) ? MMVD_ADD_NUM : MMVD_ADD_NUM >> 1;
//For the cyclic traversal of MMVD candidates, since each initial MV has 4 search directions and 8 search steps, that is, each initial MV produces 32 refined MVs
for (int mmvdMergeCand = 0; mmvdMergeCand < tempNum; mmvdMergeCand++)
{
int baseIdx = mmvdMergeCand / MMVD_MAX_REFINE_NUM;//The initial MV index is either 0 or 1 (here is the starting point of the two initial MVs. Each initial MV has 32 combinations of 4 * 8 steps + directions)
int refineStep = (mmvdMergeCand - (baseIdx * MMVD_MAX_REFINE_NUM)) / 4;//Represents 8 steps
if (refineStep >= m_pcEncCfg->getMmvdDisNum())
continue;
//Set the information of MMVD candidates to obtain the specific search initial point, each direction and corresponding step size of each extended MV
mergeCtx.setMmvdMergeCandiInfo(pu, mmvdMergeCand);
PU::spanMotionInfo(pu, mergeCtx);
pu.mvRefine = true;
distParam.cur = singleMergeTempBuffer->Y();
pu.mmvdEncOptMode = (refineStep > 2 ? 2 : 1);
CHECK(!pu.mmvdMergeFlag, "MMVD merge should be set");
// Don't do chroma MC here
//The Merge candidate motion compensation of MMVD calculates the predicted value
m_pcInterSearch->motionCompensation(pu, *singleMergeTempBuffer, REF_PIC_LIST_X, true, false);
pu.mmvdEncOptMode = 0;
pu.mvRefine = false;
Distortion uiSad = distParam.distFunc(distParam);//Distortion function
m_CABACEstimator->getCtx() = ctxStart;
uint64_t fracBits = m_pcInterSearch->xCalcPuMeBits(pu);//Calculate bit rate
double cost = (double)uiSad + (double)fracBits * sqrtLambdaForFirstPassIntra;//Calculate RD Cost
insertPos = -1;
//Update candidate list
updateCandList(ModeInfo(mmvdMergeCand, false, true, false), cost, RdModeList, candCostList, uiNumMrgSATDCand, &insertPos);
if (insertPos != -1)
{
for (int i = int(RdModeList.size()) - 1; i > insertPos; i--)
{
swap(acMergeTempBuffer[i - 1], acMergeTempBuffer[i]);
}
swap(singleMergeTempBuffer, acMergeTempBuffer[insertPos]);
}
}
} // MMVD
// Try to limit number of candidates using SATD-costs
// Try to limit the number of modes using SATD cost
// If the Cost of a mode is greater than the Cost*1.25 of the first mode in the candidate list, SATD Cost calculation will not be performed for the following modes
for( uint32_t i = 1; i < uiNumMrgSATDCand; i++ )
{
if( candCostList[i] > MRG_FAST_RATIO * candCostList[0] )
{
uiNumMrgSATDCand = i;
break;
}
}
setMergeBestSATDCost( candCostList[0] ); //Set the best SATD Cost for Merge mode
if (isIntrainterEnabled && isChromaEnabled(pu.cs->pcv->chrFormat))//CIIP mode is available and chroma components exist
{
pu.ciipFlag = true;
for (uint32_t mergeCnt = 0; mergeCnt < uiNumMrgSATDCand; mergeCnt++)
{
if (RdModeList[mergeCnt].isCIIP) //The Planar prediction pixel that calculates the chromaticity component for the CIIP mode in the candidate list
{
pu.intraDir[0] = PLANAR_IDX;
pu.intraDir[1] = DM_CHROMA_IDX;
if (pu.chromaSize().width == 2)
continue;
uint32_t bufIdx = 0;
m_pcIntraSearch->initIntraPatternChType(*pu.cu, pu.Cb());
m_pcIntraSearch->predIntraAng(COMPONENT_Cb, pu.cs->getPredBuf(pu).Cb(), pu);
m_pcIntraSearch->switchBuffer(pu, COMPONENT_Cb, pu.cs->getPredBuf(pu).Cb(), m_pcIntraSearch->getPredictorPtr2(COMPONENT_Cb, bufIdx));
m_pcIntraSearch->initIntraPatternChType(*pu.cu, pu.Cr());
m_pcIntraSearch->predIntraAng(COMPONENT_Cr, pu.cs->getPredBuf(pu).Cr(), pu);
m_pcIntraSearch->switchBuffer(pu, COMPONENT_Cr, pu.cs->getPredBuf(pu).Cr(), m_pcIntraSearch->getPredictorPtr2(COMPONENT_Cr, bufIdx));
}
}
pu.ciipFlag = false;
}
tempCS->initStructData( encTestMode.qp );
m_CABACEstimator->getCtx() = ctxStart;
}
else
{
if (bestIsMMVDSkip) //If the best mode is MMVD Skip mode
{
uiNumMrgSATDCand = mergeCtx.numValidMergeCand + ((mergeCtx.numValidMergeCand > 1) ? MMVD_ADD_NUM : MMVD_ADD_NUM >> 1);
}
else // The best mode is Skip mode
{
uiNumMrgSATDCand = mergeCtx.numValidMergeCand;
}
}
}
m_bestModeUpdated = tempCS->useDbCost = bestCS->useDbCost = false;
uint32_t iteration;
uint32_t iterationBegin = 0;
iteration = 2;
// Two iterations, one is Skip mode, skipping residual coding; The other is the regular Merge mode
for (uint32_t uiNoResidualPass = iterationBegin; uiNoResidualPass < iteration; ++uiNoResidualPass)
{
// Traverse SATD candidate Merge modes
for( uint32_t uiMrgHADIdx = 0; uiMrgHADIdx < uiNumMrgSATDCand; uiMrgHADIdx++ )
{
uint32_t uiMergeCand = RdModeList[uiMrgHADIdx].mergeCand;//The corresponding Merge candidate mode in the current HAD list
if (uiNoResidualPass != 0 && RdModeList[uiMrgHADIdx].isCIIP) // Intranet does not support Skip mode CIIP does not support Skip mode
{
if (isTestSkipMerge[uiMergeCand])
{
continue;
}
}
if (((uiNoResidualPass != 0) && candHasNoResidual[uiMrgHADIdx])
|| ( (uiNoResidualPass == 0) && bestIsSkip ) )
{
continue;
}
// first get merge candidates
CodingUnit &cu = tempCS->addCU( tempCS->area, partitioner.chType );
partitioner.setCUData( cu );
cu.slice = tempCS->slice;
cu.tileIdx = tempCS->pps->getTileIdx( tempCS->area.lumaPos() );
cu.skip = false;
cu.mmvdSkip = false;
cu.geoFlag = false;
//cu.affine
cu.predMode = MODE_INTER;
//cu.LICFlag
cu.chromaQpAdj = m_cuChromaQpOffsetIdxPlus1;
cu.qp = encTestMode.qp;
PredictionUnit &pu = tempCS->addPU( cu, partitioner.chType );
// Normal Merge mode and CIIP mode
if (uiNoResidualPass == 0 && RdModeList[uiMrgHADIdx].isCIIP)
{
cu.mmvdSkip = false;
mergeCtx.setMergeInfo(pu, uiMergeCand);
pu.ciipFlag = true;
pu.regularMergeFlag = false;
pu.intraDir[0] = PLANAR_IDX;
CHECK(pu.intraDir[0]<0 || pu.intraDir[0]>(NUM_LUMA_MODE - 1), "out of intra mode");
pu.intraDir[1] = DM_CHROMA_IDX;
}
else if (RdModeList[uiMrgHADIdx].isMMVD) //The mode to be tested is MMVD mode
{
cu.mmvdSkip = true;
pu.regularMergeFlag = true;
mergeCtx.setMmvdMergeCandiInfo(pu, uiMergeCand);//Set MMVD candidate information
}
else //General merge mode
{
cu.mmvdSkip = false;
pu.regularMergeFlag = true;
mergeCtx.setMergeInfo(pu, uiMergeCand); //Set Merge mode information
}
PU::spanMotionInfo( pu, mergeCtx );
if( m_pcEncCfg->getMCTSEncConstraint() )
{
bool isDMVR = PU::checkDMVRCondition( pu );
if( ( isDMVR && MCTSHelper::isRefBlockAtRestrictedTileBoundary( pu ) ) || ( !isDMVR && !( MCTSHelper::checkMvBufferForMCTSConstraint( pu ) ) ) )
{
// Do not use this mode
tempCS->initStructData( encTestMode.qp );
continue;
}
}
if( mrgTempBufSet ) //Set temporary Buffer for Merge prediction
{
{
int dx, dy, i, j, num = 0;
dy = std::min<int>(pu.lumaSize().height, DMVR_SUBCU_HEIGHT);
dx = std::min<int>(pu.lumaSize().width, DMVR_SUBCU_WIDTH);
if (PU::checkDMVRCondition(pu)) //Check DMVR conditions
{
for (i = 0; i < (pu.lumaSize().height); i += dy)
{
for (j = 0; j < (pu.lumaSize().width); j += dx)
{
pu.mvdL0SubPu[num] = refinedMvdL0[num][uiMergeCand];
num++;
}
}
}
}
if (pu.ciipFlag) //CIIP mode
{
uint32_t bufIdx = 0;
PelBuf tmpBuf = tempCS->getPredBuf(pu).Y(); //Store predictions in tmpBuf
tmpBuf.copyFrom(acMergeTmpBuffer[uiMergeCand].Y());
if (pu.cs->slice->getLmcsEnabledFlag() && m_pcReshape->getCTUFlag())
{
tmpBuf.rspSignal(m_pcReshape->getFwdLUT());
}
// Obtain weighted predictions
m_pcIntraSearch->geneWeightedPred(COMPONENT_Y, tmpBuf, pu, m_pcIntraSearch->getPredictorPtr2(COMPONENT_Y, bufIdx));
if (isChromaEnabled(pu.chromaFormat))
{
if (pu.chromaSize().width > 2)
{
tmpBuf = tempCS->getPredBuf(pu).Cb();
tmpBuf.copyFrom(acMergeTmpBuffer[uiMergeCand].Cb());
m_pcIntraSearch->geneWeightedPred(COMPONENT_Cb, tmpBuf, pu, m_pcIntraSearch->getPredictorPtr2(COMPONENT_Cb, bufIdx));
tmpBuf = tempCS->getPredBuf(pu).Cr();
tmpBuf.copyFrom(acMergeTmpBuffer[uiMergeCand].Cr());
m_pcIntraSearch->geneWeightedPred(COMPONENT_Cr, tmpBuf, pu, m_pcIntraSearch->getPredictorPtr2(COMPONENT_Cr, bufIdx));
}
else
{
tmpBuf = tempCS->getPredBuf(pu).Cb();
tmpBuf.copyFrom(acMergeTmpBuffer[uiMergeCand].Cb());
tmpBuf = tempCS->getPredBuf(pu).Cr();
tmpBuf.copyFrom(acMergeTmpBuffer[uiMergeCand].Cr());
}
}
} //pu.ciipFlag
else
{
if (RdModeList[uiMrgHADIdx].isMMVD) //MMVD mode
{
pu.mmvdEncOptMode = 0;
m_pcInterSearch->motionCompensation(pu); //motion compensation
}
else if (uiNoResidualPass != 0 && RdModeList[uiMrgHADIdx].isCIIP)
{
tempCS->getPredBuf().copyFrom(acMergeBuffer[uiMergeCand]);
}
else
{
tempCS->getPredBuf().copyFrom(*acMergeTempBuffer[uiMrgHADIdx]);
}
}
} // if (mrgTempBufSet)
else
{
pu.mvRefine = true;
m_pcInterSearch->motionCompensation( pu ); //motion compensation
pu.mvRefine = false;
}
if (!cu.mmvdSkip && !pu.ciipFlag && uiNoResidualPass != 0)
{
CHECK(uiMergeCand >= mergeCtx.numValidMergeCand, "out of normal merge");
isTestSkipMerge[uiMergeCand] = true; //Skip mode is checked for this Merge mode
}
// Coding residual
xEncodeInterResidual( tempCS, bestCS, partitioner, encTestMode, uiNoResidualPass, uiNoResidualPass == 0 ? &candHasNoResidual[uiMrgHADIdx] : NULL );
if( m_pcEncCfg->getUseFastDecisionForMerge() && !bestIsSkip && !pu.ciipFlag)
{
bestIsSkip = !bestCS->cus.empty() && bestCS->getCU( partitioner.chType )->rootCbf == 0;
}
tempCS->initStructData( encTestMode.qp );
}// end loop uiMrgHADIdx
if( uiNoResidualPass == 0 && m_pcEncCfg->getUseEarlySkipDetection() )
{
const CodingUnit &bestCU = *bestCS->getCU( partitioner.chType );
const PredictionUnit &bestPU = *bestCS->getPU( partitioner.chType );
if( bestCU.rootCbf == 0 )
{
if( bestPU.mergeFlag )
{
m_modeCtrl->setEarlySkipDetected();
}
else if( m_pcEncCfg->getMotionEstimationSearchMethod() != MESEARCH_SELECTIVE )
{
int absolute_MV = 0;
for( uint32_t uiRefListIdx = 0; uiRefListIdx < 2; uiRefListIdx++ )
{
if( slice.getNumRefIdx( RefPicList( uiRefListIdx ) ) > 0 )
{
absolute_MV += bestPU.mvd[uiRefListIdx].getAbsHor() + bestPU.mvd[uiRefListIdx].getAbsVer();
}
}
if( absolute_MV == 0 )
{
m_modeCtrl->setEarlySkipDetected();
}
}
}
}
}
if ( m_bestModeUpdated && bestCS->cost != MAX_DOUBLE )
{
xCalDebCost( *bestCS, partitioner );
}
}