JPEG codec principle and decoder debugging
1, Experimental purpose
Master the basic principle of JPEG codec system. Preliminarily master the implementation of complex data compression algorithm, and be able to output the corresponding data according to the needs of theoretical analysis.
2, JPEG codec principle
1. Coding principle
(1) 0 bias level down
For example, from (0255) to (- 128127), for pixels with gray level of 2^n, by subtracting 2^n-1 unsigned integer to signed number, the probability of the number with large absolute value is reduced and the coding efficiency is improved.
(2) 8*8DCT conversion
The minimum coding unit MCU is set as 8X8 blocks, and each individual component image (Y, U, V) is divided accordingly. Each block is subjected to two-dimensional discrete cosine DCT transform, which is to concentrate energy, remove redundancy and decorrelation, and improve coding efficiency**
(3) Quantification
The medium flat uniform quantizer is adopted. According to the sensitive characteristics of human vision, it is divided into two quantization tables: brightness quantization value and color difference quantization value. The low-frequency part is finely quantized and the high-frequency part is roughly quantized. This process will produce certain errors.
Give the suggested quantization table: real quantization table = scaling factor * Basic quantization table
(4) DC coefficient differential coding
After DCT transformation, the energy concentration of each block leads to very large DC coefficient; And the DC coefficient of adjacent image blocks changes little, that is, there is redundancy.
Therefore, the differential pulse modulation coding DPCM technology is used to encode the difference DIFF of quantized DC coefficients between adjacent image blocks:
(5) AC AC coefficient Z-scan and RLE
After DCT transformation, the coefficients are concentrated in the low-frequency component area in the upper left corner. Therefore, the Z-shape is used to read out according to the high and low frequency. Run length code RLE can be used, which are all zero. Finally, EOB can be given.
(6) Huffman coding
- Use Huffman code for DIFF:
It is divided into categories, and the category ID is the unary code code; Fixed length code is adopted for intra class index
- AC coefficient run length coding (run,level) combined with Huffman coding:
Fixed length code is adopted for intra class index
2. Decoding principle
That is, the reverse process of coding:
- Decoding Huffman coding
- Decoded DC difference
- Reconstructed quantized coefficient
- DCT inverse transform
- Discard filled rows and columns
- Reverse 0 bias
- Difference of CbCr component lost (inverse process of down sampling)
3, JPEG file format
SOI, Start of Image
APP0, Application, Application retention flag 0
DQT, Define Quantization Table
SOF0, Start of Frame
DHT, Define Huffman Table
SOS, Start of Scan, 12 bytes
EOI, End of Image, 2 bytes
1. Formal organization of segment
JPEG files are organized in the form of segments, where each segment starts with a marker, and each marker starts with 0xFF and a marker identifier, followed by a 2-byte marker length (excluding the first two bytes of the marker) and the corresponding payload (SOI and EOI marker s have only 2-byte identifiers). Consecutive 0xFF bytes are not the starting mark of marker, but special characters used to fill in.
2. Format example analysis
test.jpg 1024*1024
Use FlexHEX for parsing:
1,SOI&EOI
- SOI, start of image
Tag code 2 byte fixed value 0xFFD8 - EOI, End of Image, 2 bytes
Tag code 2 byte fixed value 0xFFD9
2,APP0
Tag code 2 byte fixed value 0xFFE0
Contains 9 specific fields:
① Data length 2 bytes ① ~ ⑨ total length of 9 fields
② Identifier 5-byte fixed value 0x4A46494600, i.e. string "JFIF0"
③ The 2 bytes of version number are usually 0x0102, indicating the version number of JFIF is 1.2
④ The density units of X and Y are 1 byte, and only three values are available
0: No unit; 1: Points / inch; 2: Points / cm
⑤ The value range of 2 bytes of pixel density in X direction is unknown
⑥ The value range of 2 bytes of pixel density in Y direction is unknown
⑦ The number of thumbnail horizontal pixels is 1 byte, and the value range is unknown
⑧ The number of vertical pixels of thumbnail is 1 byte, and the value range is unknown
⑨ Thumbnail RGB bitmap length may be a multiple of 3 thumbnail RGB bitmap data
The total length of "00 10" data is 16 bytes; "4A 46 49 46 00" identifier corresponds to string "JFIF0"; version number 0x0101; "00" the density unit of X and Y is 0; " 00 01 00 01 "pixel density in X and Y directions;" 00 "the number of horizontal and vertical pixels of the thumbnail is 0
3,DQT
Tag code 2 byte fixed value 0xFFDB
Contains 9 specific fields:
① Data length is the total length of 2-byte field ① and multiple fields ②
② Quantization table data length - 2 bytes
a) Precision and quantization table ID 1 byte
High 4 bits: precision, only two optional values 0:8 bits; 1: 16 bit
Lower 4 bits: quantization table ID, with a value range of 0 ~ 3
b) Table entry (64) × (precision + 1) byte
For example, a quantization table with 8-bit accuracy has a table item length of 64 × (0 + 1) = 64 bytes
In this marking segment, field ② can appear repeatedly, indicating multiple quantization tables, but it can only appear 4 times at most
There are two quantization tables marked with "FF DB", with a length of 67 bytes, a quantization accuracy of 8 bits, a quantization table ID of 0 and 1, and a table item length of 64 bytes.
4,SOF0
Tag code 2 byte fixed value 0xFFC0
Contains 9 specific fields:
① Data length 2 bytes ① ~ ⑥ total length of six fields
② Precision 1 byte the number of bits per data sample
It is usually 8 bits, and general software does not support 12 bits and 16 bits
③ Image height 2 bytes image height (unit: pixel)
④ Image width 2 bytes image width (unit: pixel)
⑤ The number of color components is 1 byte, and only 3 values can be selected
1: Gray scale image; 3: YCrCb or YIQ; 4: CMYK
YCrCb is used in JFIF, so the number of color components here is always 3
⑥ Color component information number of color components × 3 bytes (usually 9 bytes)
a) Color component ID 1 byte
b) Horizontal / vertical sampling factor 1 byte
High 4 bits: horizontal sampling factor
Lower 4 bits: vertical sampling factor
c) Quantization table 1 byte
ID of the quantization table used by the current component
"00 11" indicates the total length of 11 bytes; "08" each data sample is 8 bits; The height and width of "04 00 04 00" image is 1024 * 1024; "03" color component number YCrCb or YIQ; The quantized component table with ID of Cr as 1 is used.
5,DHT
Tag code 2 byte fixed value 0xFFC4
Contains 2 specific fields:
① Data length 2 bytes
② huffman table data length - 2 bytes
- Table ID and table type 1 byte
High 4 bits: type, only two values are optional
0: DC; 1: AC AC
Lower 4 bits: Huffman table ID,
Note that the DC table and the AC table are coded separately - The number of codewords with different bits is 16 bytes
- Sum of the number of codewords with 16 different bits of encoded content (bytes)
In this marking segment, field ② can appear repeatedly (usually 4 times) or only once.
The length of the four Huffman tables is 29 bytes, 62 bytes, 30 bytes and 47 bytes respectively; The table type and ID are "00" DC table 0, "10" AC table 0, "01" DC Table 1, "11" AC Table 1; " 00 03 01 01 01 01 00... 00 "indicates 3 2-bit codewords, 1 3-bit-9-bit codeword in DC table 0, and no codeword in other bits." 04 05... 07 08 "indicates that 10 leaf nodes are arranged from small to large, and their weights are 04, 05, 06, 03, 02, 01, 00, 09, 07 and 08 in turn
6,SOS
Tag code 2 byte fixed value 0xFFDA
Contains 2 specific fields:
① The data length is 2 bytes. The total length of the two fields ① ~ ④
② The number of color components 1 byte should be the same as the value of field ⑤ in SOF, that is:
1: Gray scale image is; 3: YCrCb or YIQ; 4: CMYK.
③ Color component information
a) Color component ID 1 byte
b) DC / AC coefficient table No. 1 byte
High 4 bits: Huffman tree number used for DC component
Lower 4 digits: Huffman tree number used for AC component
④ Compressed image data
a) Spectrum selection start 1 byte fixed value 0x00
b) Spectrum selection end 1 byte fixed value 0x3F
c) 1 byte of spectrum selection is always 00 in basic JPEG
"00 0C" means the length is 12 bytes, "03" means the number of color components YCrCb or YIQ; The DC and AC components of "00", "11" and "11" correspond to Huffman tree numbers respectively.
4, JPEG decoding process
int convert_one_image(const char *infilename, const char *outfilename, int output_format)
{
FILE *fp;
unsigned int length_of_file;
unsigned int width, height;
unsigned char *buf;
struct jdec_private *jdec;
unsigned char *components[3];
/* Load the Jpeg into memory */
fp = fopen(infilename, "rb");
if (fp == NULL)
exitmessage("Cannot open filename\n");
length_of_file = filesize(fp);
buf = (unsigned char *)malloc(length_of_file + 4);
if (buf == NULL)
exitmessage("Not enough memory for loading file\n");
fread(buf, length_of_file, 1, fp);
fclose(fp);
/* Decompress it */
jdec = tinyjpeg_init();
if (jdec == NULL)
exitmessage("Not enough memory to alloc the structure need for decompressing\n");
if (tinyjpeg_parse_header(jdec, buf, length_of_file)<0)
exitmessage(tinyjpeg_get_errorstring(jdec));
/* Get the size of the image */
tinyjpeg_get_size(jdec, &width, &height);
snprintf(error_string, sizeof(error_string),"Decoding JPEG image...\n");
if (tinyjpeg_decode(jdec, output_format) < 0)
exitmessage(tinyjpeg_get_errorstring(jdec));
/*
* Get address for each plane (not only max 3 planes is supported), and
* depending of the output mode, only some components will be filled
* RGB: 1 plane, YUV420P: 3 planes, GREY: 1 plane
*/
tinyjpeg_get_components(jdec, components);
/* Save it */
switch (output_format)
{
case TINYJPEG_FMT_RGB24:
case TINYJPEG_FMT_BGR24:
write_tga(outfilename, output_format, width, height, components);
break;
case TINYJPEG_FMT_YUV420P:
write_yuv(outfilename, width, height, components);
break;
case TINYJPEG_FMT_GREY:
write_pgm(outfilename, width, height, components);
break;
}
/* Only called this if the buffers were allocated by tinyjpeg_decode() */
tinyjpeg_free(jdec);
/* else called just free(jdec); */
free(buf);
return 0;
}
Parsing JPEG file headers:
int tinyjpeg_parse_header(struct jdec_private *priv, const unsigned char *buf, unsigned int size)
{
int ret;
/* Identify the file */
if ((buf[0] != 0xFF) || (buf[1] != SOI))
snprintf(error_string, sizeof(error_string),"Not a JPG file ?\n");
priv->stream_begin = buf+2;
priv->stream_length = size-2;
priv->stream_end = priv->stream_begin + priv->stream_length;
ret = parse_JFIF(priv, priv->stream_begin);
return ret;
}
Create huffman table:
static void build_huffman_table(const unsigned char *bits, const unsigned char *vals, struct huffman_table *table)
{
unsigned int i, j, code, code_size, val, nbits;
unsigned char huffsize[HUFFMAN_BITS_SIZE+1], *hz;
unsigned int huffcode[HUFFMAN_BITS_SIZE+1], *hc;
int next_free_entry;
/*
* Build a temp array
* huffsize[X] => numbers of bits to write vals[X]
*/
hz = huffsize;
for (i=1; i<=16; i++)
{
for (j=1; j<=bits[i]; j++)
*hz++ = i;
}
*hz = 0;
memset(table->lookup, 0xff, sizeof(table->lookup));
for (i=0; i<(16-HUFFMAN_HASH_NBITS); i++)
table->slowtable[i][0] = 0;
/* Build a temp array
* huffcode[X] => code used to write vals[X]
*/
code = 0;
hc = huffcode;
hz = huffsize;
nbits = *hz;
while (*hz)
{
while (*hz == nbits)
{
*hc++ = code++;
hz++;
}
code <<= 1;
nbits++;
}
/*
* Build the lookup table, and the slowtable if needed.
*/
next_free_entry = -1;
for (i=0; huffsize[i]; i++)
{
val = vals[i];
code = huffcode[i];
code_size = huffsize[i];
#if TRACE
fprintf(p_trace,"val=%2.2x code=%8.8x codesize=%2.2d\n", val, code, code_size);
fflush(p_trace);
#endif
table->code_size[val] = code_size;
if (code_size <= HUFFMAN_HASH_NBITS)
{
/*
* Good: val can be put in the lookup table, so fill all value of this
* column with value val
*/
int repeat = 1UL<<(HUFFMAN_HASH_NBITS - code_size);
code <<= HUFFMAN_HASH_NBITS - code_size;
while ( repeat-- )
table->lookup[code++] = val;
}
else
{
/* Perhaps sorting the array will be an optimization */
uint16_t *slowtable = table->slowtable[code_size-HUFFMAN_HASH_NBITS-1];
while(slowtable[0])
slowtable+=2;
slowtable[0] = code;
slowtable[1] = val;
slowtable[2] = 0;
/* TODO: NEED TO CHECK FOR AN OVERFLOW OF THE TABLE */
}
}
}
"Resolve marker ID":
static int parse_JFIF(struct jdec_private *priv, const unsigned char *stream)
{
int chuck_len;
int marker;
int sos_marker_found = 0;
int dht_marker_found = 0;
const unsigned char *next_chunck;
/* Parse marker */
while (!sos_marker_found)
{
if (*stream++ != 0xff)
goto bogus_jpeg_format;
/* Skip any padding ff byte (this is normal) */
while (*stream == 0xff)
stream++;
marker = *stream++;
chuck_len = be16_to_cpu(stream);
next_chunck = stream + chuck_len;
switch (marker)
{
case SOF:
if (parse_SOF(priv, stream) < 0)
return -1;
break;
case DQT:
if (parse_DQT(priv, stream) < 0)
return -1;
break;
case SOS:
if (parse_SOS(priv, stream) < 0)
return -1;
sos_marker_found = 1;
break;
case DHT:
if (parse_DHT(priv, stream) < 0)
return -1;
dht_marker_found = 1;
break;
case DRI:
if (parse_DRI(priv, stream) < 0)
return -1;
break;
default:
#if TRACE
fprintf(p_trace,"> Unknown marker %2.2x\n", marker);
fflush(p_trace);
#endif
break;
}
stream = next_chunck;
}
if (!dht_marker_found) {
#if TRACE
fprintf(p_trace,"No Huffman table loaded, using the default one\n");
fflush(p_trace);
#endif
build_default_huffman_tables(priv);
}
#ifdef SANITY_CHECK
if ( (priv->component_infos[cY].Hfactor < priv->component_infos[cCb].Hfactor)
|| (priv->component_infos[cY].Hfactor < priv->component_infos[cCr].Hfactor))
snprintf(error_string, sizeof(error_string),"Horizontal sampling factor for Y should be greater than horitontal sampling factor for Cb or Cr\n");
if ( (priv->component_infos[cY].Vfactor < priv->component_infos[cCb].Vfactor)
|| (priv->component_infos[cY].Vfactor < priv->component_infos[cCr].Vfactor))
snprintf(error_string, sizeof(error_string),"Vertical sampling factor for Y should be greater than vertical sampling factor for Cb or Cr\n");
if ( (priv->component_infos[cCb].Hfactor!=1)
|| (priv->component_infos[cCr].Hfactor!=1)
|| (priv->component_infos[cCb].Vfactor!=1)
|| (priv->component_infos[cCr].Vfactor!=1))
snprintf(error_string, sizeof(error_string),"Sampling other than 1x1 for Cr and Cb is not supported");
#endif
return 0;
bogus_jpeg_format:
#if TRACE
fprintf(p_trace,"Bogus jpeg format\n");
fflush(p_trace);
#endif
return -1;
}
Establish quantitative table:
static void build_quantization_table(float *qtable, const unsigned char *ref_table)
{
/* Taken from libjpeg. Copyright Independent JPEG Group's LLM idct.
* For float AA&N IDCT method, divisors are equal to quantization
* coefficients scaled by scalefactor[row]*scalefactor[col], where
* scalefactor[0] = 1
* scalefactor[k] = cos(k*PI/16) * sqrt(2) for k=1..7
* We apply a further scale factor of 8.
* What's actually stored is 1/divisor so that the inner loop can
* use a multiplication rather than a division.
*/
int i, j;
static const double aanscalefactor[8] = {
1.0, 1.387039845, 1.306562965, 1.175875602,
1.0, 0.785694958, 0.541196100, 0.275899379
};
const unsigned char *zz = zigzag;
for (i=0; i<8; i++) {
for (j=0; j<8; j++) {
*qtable++ = ref_table[*zz++] * aanscalefactor[i] * aanscalefactor[j];
}
}
}
Analytical quantization table DQT:
static int parse_DQT(struct jdec_private *priv, const unsigned char *stream)
{
int qi;
float *table;
const unsigned char *dqt_block_end;
#if TRACE
fprintf(p_trace,"> DQT marker\n");
fflush(p_trace);
#endif
dqt_block_end = stream + be16_to_cpu(stream);
stream += 2; /* Skip length */
while (stream < dqt_block_end)
{
qi = *stream++;
#if SANITY_CHECK
if (qi>>4)
snprintf(error_string, sizeof(error_string),"16 bits quantization table is not supported\n");
if (qi>4)
snprintf(error_string, sizeof(error_string),"No more 4 quantization table is supported (got %d)\n", qi);
#endif
table = priv->Q_tables[qi];
build_quantization_table(table, stream);
stream += 64;
}
#if TRACE
fprintf(p_trace,"< DQT marker\n");
fflush(p_trace);
#endif
return 0;
}
Parsing DHT:
static int parse_DHT(struct jdec_private *priv, const unsigned char *stream)
{
unsigned int count, i;
unsigned char huff_bits[17];
int length, index;
length = be16_to_cpu(stream) - 2;
stream += 2; /* Skip length */
#if TRACE
fprintf(p_trace,"> DHT marker (length=%d)\n", length);
fflush(p_trace);
#endif
while (length>0) {
index = *stream++;
/* We need to calculate the number of bytes 'vals' will takes */
huff_bits[0] = 0;
count = 0;
for (i=1; i<17; i++) {
huff_bits[i] = *stream++;
count += huff_bits[i];
}
#if SANITY_CHECK
if (count >= HUFFMAN_BITS_SIZE)
snprintf(error_string, sizeof(error_string),"No more than %d bytes is allowed to describe a huffman table", HUFFMAN_BITS_SIZE);
if ( (index &0xf) >= HUFFMAN_TABLES)
snprintf(error_string, sizeof(error_string),"No more than %d Huffman tables is supported (got %d)\n", HUFFMAN_TABLES, index&0xf);
#if TRACE
fprintf(p_trace,"Huffman table %s[%d] length=%d\n", (index&0xf0)?"AC":"DC", index&0xf, count);
fflush(p_trace);
#endif
#endif
if (index & 0xf0 )
build_huffman_table(huff_bits, stream, &priv->HTAC[index&0xf]);
else
build_huffman_table(huff_bits, stream, &priv->HTDC[index&0xf]);
length -= 1;
length -= 16;
length -= count;
stream += count;
}
#if TRACE
fprintf(p_trace,"< DHT marker\n");
fflush(p_trace);
#endif
return 0;
}
static int parse_DRI(struct jdec_private *priv, const unsigned char *stream)
{
unsigned int length;
#if TRACE
fprintf(p_trace,"> DRI marker\n");
fflush(p_trace);
#endif
length = be16_to_cpu(stream);
#if SANITY_CHECK
if (length != 4)
snprintf(error_string, sizeof(error_string),"Length of DRI marker need to be 4\n");
#endif
priv->restart_interval = be16_to_cpu(stream+2);
#if TRACE
fprintf(p_trace,"Restart interval = %d\n", priv->restart_interval);
fprintf(p_trace,"< DRI marker\n");
fflush(p_trace);
#endif
return 0;
}
Resolve SOS:
static int parse_SOS(struct jdec_private *priv, const unsigned char *stream)
{
unsigned int i, cid, table;
unsigned int nr_components = stream[2];
#if TRACE
fprintf(p_trace,"> SOS marker\n");
fflush(p_trace);
#endif
#if SANITY_CHECK
if (nr_components != 3)
snprintf(error_string, sizeof(error_string),"We only support YCbCr image\n");
#endif
stream += 3;
for (i=0;i<nr_components;i++) {
cid = *stream++;
table = *stream++;
#if SANITY_CHECK
if ((table&0xf)>=4)
snprintf(error_string, sizeof(error_string),"We do not support more than 2 AC Huffman table\n");
if ((table>>4)>=4)
snprintf(error_string, sizeof(error_string),"We do not support more than 2 DC Huffman table\n");
if (cid != priv->component_infos[i].cid)
snprintf(error_string, sizeof(error_string),"SOS cid order (%d:%d) isn't compatible with the SOF marker (%d:%d)\n",
i, cid, i, priv->component_infos[i].cid);
#if TRACE
fprintf(p_trace,"ComponentId:%d tableAC:%d tableDC:%d\n", cid, table&0xf, table>>4);
fflush(p_trace);
#endif
#endif
priv->component_infos[i].AC_table = &priv->HTAC[table&0xf];
priv->component_infos[i].DC_table = &priv->HTDC[table>>4];
}
priv->stream = stream+3;
#if TRACE
fprintf(p_trace,"< SOS marker\n");
fflush(p_trace);
#endif
return 0;
}
Resolve SOF:
static int parse_SOF(struct jdec_private *priv, const unsigned char *stream)
{
int i, width, height, nr_components, cid, sampling_factor;
int Q_table;
struct component *c;
#if TRACE
fprintf(p_trace,"> SOF marker\n");
fflush(p_trace);
#endif
print_SOF(stream);
height = be16_to_cpu(stream+3);
width = be16_to_cpu(stream+5);
nr_components = stream[7];
#if SANITY_CHECK
if (stream[2] != 8)
snprintf(error_string, sizeof(error_string),"Precision other than 8 is not supported\n");
if (width>JPEG_MAX_WIDTH || height>JPEG_MAX_HEIGHT)
snprintf(error_string, sizeof(error_string),"Width and Height (%dx%d) seems suspicious\n", width, height);
if (nr_components != 3)
snprintf(error_string, sizeof(error_string),"We only support YUV images\n");
if (height%16)
snprintf(error_string, sizeof(error_string),"Height need to be a multiple of 16 (current height is %d)\n", height);
if (width%16)
snprintf(error_string, sizeof(error_string),"Width need to be a multiple of 16 (current Width is %d)\n", width);
#endif
stream += 8;
for (i=0; i<nr_components; i++) {
cid = *stream++;
sampling_factor = *stream++;
Q_table = *stream++;
c = &priv->component_infos[i];
#if SANITY_CHECK
c->cid = cid;
if (Q_table >= COMPONENTS)
snprintf(error_string, sizeof(error_string),"Bad Quantization table index (got %d, max allowed %d)\n", Q_table, COMPONENTS-1);
#endif
c->Vfactor = sampling_factor&0xf;
c->Hfactor = sampling_factor>>4;
c->Q_table = priv->Q_tables[Q_table];
#if TRACE
fprintf(p_trace,"Component:%d factor:%dx%d Quantization table:%d\n",
cid, c->Hfactor, c->Hfactor, Q_table );
fflush(p_trace);
#endif
}
priv->width = width;
priv->height = height;
#if TRACE
fprintf(p_trace,"< SOF marker\n");
fflush(p_trace);
#endif
return 0;
}
Parsing JPEG actual data:
int tinyjpeg_decode(struct jdec_private *priv, int pixfmt)
{
unsigned int x, y, xstride_by_mcu, ystride_by_mcu;
unsigned int bytes_per_blocklines[3], bytes_per_mcu[3];
decode_MCU_fct decode_MCU;
const decode_MCU_fct *decode_mcu_table;
const convert_colorspace_fct *colorspace_array_conv;
convert_colorspace_fct convert_to_pixfmt;
if (setjmp(priv->jump_state))
return -1;
/* To keep gcc happy initialize some array */
bytes_per_mcu[1] = 0;
bytes_per_mcu[2] = 0;
bytes_per_blocklines[1] = 0;
bytes_per_blocklines[2] = 0;
decode_mcu_table = decode_mcu_3comp_table;
switch (pixfmt) {
case TINYJPEG_FMT_YUV420P:
colorspace_array_conv = convert_colorspace_yuv420p;
if (priv->components[0] == NULL)
priv->components[0] = (uint8_t *)malloc(priv->width * priv->height);
if (priv->components[1] == NULL)
priv->components[1] = (uint8_t *)malloc(priv->width * priv->height/4);
if (priv->components[2] == NULL)
priv->components[2] = (uint8_t *)malloc(priv->width * priv->height/4);
bytes_per_blocklines[0] = priv->width;
bytes_per_blocklines[1] = priv->width/4;
bytes_per_blocklines[2] = priv->width/4;
bytes_per_mcu[0] = 8;
bytes_per_mcu[1] = 4;
bytes_per_mcu[2] = 4;
break;
case TINYJPEG_FMT_RGB24:
colorspace_array_conv = convert_colorspace_rgb24;
if (priv->components[0] == NULL)
priv->components[0] = (uint8_t *)malloc(priv->width * priv->height * 3);
bytes_per_blocklines[0] = priv->width * 3;
bytes_per_mcu[0] = 3*8;
break;
case TINYJPEG_FMT_BGR24:
colorspace_array_conv = convert_colorspace_bgr24;
if (priv->components[0] == NULL)
priv->components[0] = (uint8_t *)malloc(priv->width * priv->height * 3);
bytes_per_blocklines[0] = priv->width * 3;
bytes_per_mcu[0] = 3*8;
break;
case TINYJPEG_FMT_GREY:
decode_mcu_table = decode_mcu_1comp_table;
colorspace_array_conv = convert_colorspace_grey;
if (priv->components[0] == NULL)
priv->components[0] = (uint8_t *)malloc(priv->width * priv->height);
bytes_per_blocklines[0] = priv->width;
bytes_per_mcu[0] = 8;
break;
default:
#if TRACE
fprintf(p_trace,"Bad pixel format\n");
fflush(p_trace);
#endif
return -1;
}
xstride_by_mcu = ystride_by_mcu = 8;
if ((priv->component_infos[cY].Hfactor | priv->component_infos[cY].Vfactor) == 1) {
decode_MCU = decode_mcu_table[0];
convert_to_pixfmt = colorspace_array_conv[0];
#if TRACE
fprintf(p_trace,"Use decode 1x1 sampling\n");
fflush(p_trace);
#endif
} else if (priv->component_infos[cY].Hfactor == 1) {
decode_MCU = decode_mcu_table[1];
convert_to_pixfmt = colorspace_array_conv[1];
ystride_by_mcu = 16;
#if TRACE
fprintf(p_trace,"Use decode 1x2 sampling (not supported)\n");
fflush(p_trace);
#endif
} else if (priv->component_infos[cY].Vfactor == 2) {
decode_MCU = decode_mcu_table[3];
convert_to_pixfmt = colorspace_array_conv[3];
xstride_by_mcu = 16;
ystride_by_mcu = 16;
#if TRACE
fprintf(p_trace,"Use decode 2x2 sampling\n");
fflush(p_trace);
#endif
} else {
decode_MCU = decode_mcu_table[2];
convert_to_pixfmt = colorspace_array_conv[2];
xstride_by_mcu = 16;
#if TRACE
fprintf(p_trace,"Use decode 2x1 sampling\n");
fflush(p_trace);
#endif
}
resync(priv);
/* Don't forget to that block can be either 8 or 16 lines */
bytes_per_blocklines[0] *= ystride_by_mcu;
bytes_per_blocklines[1] *= ystride_by_mcu;
bytes_per_blocklines[2] *= ystride_by_mcu;
bytes_per_mcu[0] *= xstride_by_mcu/8;
bytes_per_mcu[1] *= xstride_by_mcu/8;
bytes_per_mcu[2] *= xstride_by_mcu/8;
/* Just the decode the image by macroblock (size is 8x8, 8x16, or 16x16) */
for (y=0; y < priv->height/ystride_by_mcu; y++)
{
//trace("Decoding row %d\n", y);
priv->plane[0] = priv->components[0] + (y * bytes_per_blocklines[0]);
priv->plane[1] = priv->components[1] + (y * bytes_per_blocklines[1]);
priv->plane[2] = priv->components[2] + (y * bytes_per_blocklines[2]);
for (x=0; x < priv->width; x+=xstride_by_mcu)
{
decode_MCU(priv);
convert_to_pixfmt(priv);
priv->plane[0] += bytes_per_mcu[0];
priv->plane[1] += bytes_per_mcu[1];
priv->plane[2] += bytes_per_mcu[2];
if (priv->restarts_to_go>0)
{
priv->restarts_to_go--;
if (priv->restarts_to_go == 0)
{
priv->stream -= (priv->nbits_in_reservoir/8);
resync(priv);
if (find_next_rst_marker(priv) < 0)
return -1;
}
}
}
}
#if TRACE
fprintf(p_trace,"Input file size: %d\n", priv->stream_length+2);
fprintf(p_trace,"Input bytes actually read: %d\n", priv->stream - priv->stream_begin + 2);
fflush(p_trace);
#endif
return 0;
}
5, Decoder debugging
Decode the input JPG file:
1. Design of three structures
- struct huffman_table
Store Huffman code table
struct huffman_table
{
/* Fast look up table, using HUFFMAN_HASH_NBITS bits we can have directly the symbol,
* if the symbol is <0, then we need to look into the tree table */
short int lookup[HUFFMAN_HASH_SIZE];
/* code size: give the number of bits of a symbol is encoded */
unsigned char code_size[HUFFMAN_HASH_SIZE];
/* some place to store value that is not encoded in the lookup table
* FIXME: Calculate if 256 value is enough to store all values
*/
uint16_t slowtable[16-HUFFMAN_HASH_NBITS][256];
};
- struct component
Save current 8 × 8 information about decoding in image block
struct component
{
unsigned int Hfactor;
unsigned int Vfactor;
float *Q_table; /* Pointer to the quantisation table to use */
struct huffman_table *AC_table;
struct huffman_table *DC_table;
short int previous_DC; /* Previous DC coefficient */
short int DCT[64]; /* DCT coef */
#if SANITY_CHECK
unsigned int cid;
#endif
};
- struct jdec_private
The JPEG data stream structure is used to store the width and height of JPEG image, data stream pointer, Huffman code table and other contents, and contains struct huffman_table and struct component
struct jdec_private
{
/* Public variables */
uint8_t *components[COMPONENTS];
unsigned int width, height; /* Size of the image */
unsigned int flags;
/* Private variables */
const unsigned char *stream_begin, *stream_end;
unsigned int stream_length;
const unsigned char *stream; /* Pointer to the current stream */
unsigned int reservoir, nbits_in_reservoir;
struct component component_infos[COMPONENTS];
float Q_tables[COMPONENTS][64]; /* quantization tables */
struct huffman_table HTDC[HUFFMAN_TABLES]; /* DC huffman tables */
struct huffman_table HTAC[HUFFMAN_TABLES]; /* AC huffman tables */
int default_huffman_table_initialized;
int restart_interval;
int restarts_to_go; /* MCUs left in this restart interval */
int last_rst_marker_seen; /* Rst marker is incremented each time */
/* Temp space used after the IDCT to store each components */
uint8_t Y[64*4], Cr[64], Cb[64];
jmp_buf jump_state;
/* Internal Pointer use for colorspace conversion, do not modify it !!! */
uint8_t *plane[COMPONENTS];
};
2. Save the output file as YUV file
static void write_yuv(const char *filename, int width, int height, unsigned char **components)
{
FILE *F;
char temp[1024];
snprintf(temp, 1024, "%s.Y", filename);
F = fopen(temp, "wb");
fwrite(components[0], width, height, F);
fclose(F);
snprintf(temp, 1024, "%s.U", filename);
F = fopen(temp, "wb");
fwrite(components[1], width*height/4, 1, F);
fclose(F);
snprintf(temp, 1024, "%s.V", filename);
F = fopen(temp, "wb");
fwrite(components[2], width*height/4, 1, F);
//Output yuv file//
snprintf(temp, 1024, "%s.YUV", filename);
F = fopen(temp, "wb");
fwrite(components[0], width, height, F);
fwrite(components[1], width*height/4, 1, F);
fwrite(components[2], width*height/4, 1, F);
fclose(F);
}
Got output YUV: (not fully displayed)
3. Purpose and meaning of TRACE in video and audio codec debugging
- Turn TRACE on and off
- Modify TRACE
1 to open TRACE:
#define TRACE 1//add by nxn #define TRACEFILE "trace_jpeg.txt"//add by nxn
0 is to close TRACE:
#define TRACE 0//add by nxn #define TRACEFILE "trace_jpeg.txt"//add by nxn
trace_jpeg.txt file contains the parsing of Huffman table
4. Output all quantization matrices and all HUFFMAN code tables in txt file
Huffman code table is available, and the output of DCT matrix needs to be added,
After establishing the quantization table function, build_ quantization_ Add the output quantization table to the table:
static void build_quantization_table(float *qtable, const unsigned char *ref_table)
{
/* Taken from libjpeg. Copyright Independent JPEG Group's LLM idct.
* For float AA&N IDCT method, divisors are equal to quantization
* coefficients scaled by scalefactor[row]*scalefactor[col], where
* scalefactor[0] = 1
* scalefactor[k] = cos(k*PI/16) * sqrt(2) for k=1..7
* We apply a further scale factor of 8.
* What's actually stored is 1/divisor so that the inner loop can
* use a multiplication rather than a division.
*/
int i, j;
int temp;
static const double aanscalefactor[8] = {
1.0, 1.387039845, 1.306562965, 1.175875602,
1.0, 0.785694958, 0.541196100, 0.275899379
};
const unsigned char *zz = zigzag;
//Modify to make it output//
for (i=0; i<8; i++) {
for (j=0; j<8; j++) {
temp=ref_table[*zz++];
*qtable++ = temp * aanscalefactor[i] * aanscalefactor[j];
#if TRACE
fprintf(p_trace,"%-6d",temp);
#endif
}
fprintf(p_trace, "\n");
}
}
