Buffer of global variable

Buffer is generally called buffer buffer.

Buffer allows JavaScript to manipulate binaries.

JavaScript language initially served the browser platform, so the main data type of its internal operation is string.

The emergence of Nodejs makes it possible to program with JavaScript on the server side to complete IO operations, such as file reading and writing, data transmission in network services, etc. Buffer is used in this process.

binary data

Users use software to obtain information, and developers use language to process and display information. This "information" is actually data, such as characters, videos, pictures, audio, etc. seen on the client.

No matter what we see at present, for the computer, it deals with numbers, that is, binary.

The IO behavior operates on binary data.

Stream operation

Stream stream operation is not the original creation of Nodejs. It has existed for many years.

The * * "stream" * * can be understood as a data type similar to string and array, which is used to store data. The difference is that it can be segmented.

When large data is transmitted, stream operation can be used, which can avoid excessive data memory due to operation. And the memory is full.

With the cooperation of stream operation and pipeline technology, the data in the stream can be segmented and transmitted to the next end to realize the transmission of big data.

A typical application scenario is that we usually watch videos while downloading them.

Buffer

Binary data transmission will be carried out when the program runs, and this transmission is generally a process from from to, that is, the data producer and data consumer will use the stream to connect with the pipeline.

However, this model will also have problems in actual work. For example, the production speed of data cannot meet the consumption speed of data, or the consumption speed of data is much slower than the production speed.

In general, no matter what happens, the data will have a waiting process. Where are the extra data or data that is not enough for one consumption?

At this time, there is a Buffer, so it is also called Buffer buffer.

In Nodejs, Buffer is a piece of memory space, but this memory space is somewhat special.

The code in Nodejs is ultimately executed by the V8 engine, so theoretically all memory consumption should belong to the V8 heap memory, and Buffer is a space outside V8, and its size does not occupy the V8 heap memory.

In the memory consumption information obtained through process.memoryUsage(), there is an arrayBuffers, which refers to the memory requested by the Buffer.

The space application of Buffer is not completed by Node, but at the use level, its space allocation is controlled by JS code written by developers. Therefore, when space is recycled, it is still managed and recycled by V8 GC (garbage collection), and developers cannot participate in it.

Buffer summary

• Buffer is a global variable in Nodejs and does not need to be require d
• Buffer enables binary data to be operated on the Nodejs platform
• Buffer itself is a piece of memory space, but it does not occupy the heap memory size of V8. It is allocated directly at the C + + level
• The use of Buffer memory is controlled by the Node and recycled by the V8 GC. Human participation is not allowed
• Buffer is generally used in conjunction with Stream streams to act as a data buffer

Create Buffer instance

Buffer is a built-in class of Nodejs, and its related API is actually the static method of this class.

There are three ways to create a Buffer instance:

• alloc: creates a buffer of the specified byte size
• allocUnsafe: create a buffer with a specified byte size (unsafe creation method)
• From: receive data and create a buffer (the difference from the first two is to create a buffer with data by default)

Why not use new to create a Buffer?

Before the v6 version of Nodejs, the Buffer object can be instantiated directly through new.

However, the operation permissions provided to the Buffer instance object in this way are too large, so some processing has been carried out in subsequent versions.

Therefore, it is not recommended to create Buffer objects through class instantiation, but use the static methods of the class.

// Create a buffer with a space size of 10 bytes
// buffer stores each byte of data in hexadecimal format
const b1 = Buffer.alloc(10)
const b2 = Buffer.allocUnsafe(10)
console.log(b1) // <Buffer 00 00 00 00 00 00 00 00 00 00>
console.log(b2)

// The first parameter of from can receive three data types: string array buffer
const b3 = Buffer.from('in') // utf8 a Chinese character is represented by 3 bytes
console.log(b3) // <Buffer e4 b8 ad>
console.log(b3.toString()) // in

// If the first parameter of from is an array, the array elements should be in digital format (decimal, octal or hexadecimal, etc.), otherwise they will be ignored
const b4 = Buffer.from([1, 2, 'in']) // Chinese characters will be ignored
const b5 = Buffer.from([228/* 0xe4 Decimal system */, 0270/* 0xb8 Octal of */, 0xad]) // Replace Chinese characters with numbers
console.log(b4) // <Buffer 01 02 00>
console.log(b5) // <Buffer e4 b8 ad>

// If the first parameter of from is buffer, the created buffer is only a copy of the incoming buffer and will not share space
const b6 = Buffer.alloc(3)
const b7 = Buffer.from(b6)
b6[0] = 1
console.log(b6)
console.log(b7)

Buffer instance method

Common example methods:

• Fill: fill the buffer repeatedly until it is filled, and return the filled buffer
• write: writes data to the buffer and returns the number of bytes written
• toString: decodes the buffer into a string according to the specified character encoding
• Slice: intercept buffer, similar to slice of array
• indexOf: indexOf similar to array
• Copy: copy data from the buffer data source to the target buffer

// Create an empty buffer
const buf = Buffer.alloc(6)

// fill
buf.fill('123') // Fill repeatedly until full
console.log(buf) // <Buffer 31 32 33 31 32 33>
console.log(buf.toString()) // 123123
buf.fill('123456789')
console.log(buf.toString()) // 123456
buf.fill('0').fill('123', 1) // The second parameter represents the number of bytes to skip padding
console.log(buf.toString()) // 012312
buf.fill('0').fill('123', 1, 3) // The third parameter represents the index of the location where the population stops (the specified location is not populated)
console.log(buf.toString()) // 012000
buf.fill(123) // If the number is filled, it is converted to hexadecimal filling
console.log(buf) // <Buffer 7b 7b 7b 7b 7b 7b>
console.log(buf.toString()) // {{{{{{

// Create an empty buffer
const buf = Buffer.alloc(6)

// write
// Similar to fill, but written only once
buf.write('123')
console.log(buf) // <Buffer 31 32 33 00 00 00>
console.log(buf.toString()) // 123

buf.fill(0).write('123', 1)
console.log(buf) // <Buffer 00 31 32 33 00 00>

buf.fill(0).write('123', 1, 2) // The third parameter is the number of bytes written
console.log(buf) // <Buffer 00 31 32 00 00 00>

// Create an empty buffer
const buf = Buffer.from('abc Hello')

// toString
// The first parameter is the decoded character encoding
// The second parameter indicates the byte index position at which decoding starts
// The third parameter indicates the byte index position where decoding is stopped (not decoded)
console.log(buf) // <Buffer 61 62 63 e4 bd a0 e5 a5 bd>
console.log(buf.toString()) // Hello, abc
console.log(buf.toString('utf8', 1, 6)) // bc you

// slice
const b1 = buf.slice(1, 6)
console.log(b1) // <Buffer 62 63 e4 bd a0>
console.log(b1.toString()) // bc you

// indexOf
// Returns the byte position index of the specified character in Buffer
// The second parameter is where the lookup starts
const b2 = Buffer.from('a you b good a you b good')
console.log(b2.indexOf('you')) // 1
console.log(b2.indexOf('b')) // 4
console.log(b2.toString().indexOf('b')) // 2
console.log(b2.indexOf('b', 5)) // 12

// copy
// buf.copy(target[, targetStart[, sourceStart[, sourceEnd]]])
// buffer: copied data source buf fer
// Target: target buffer
// targetStart: write the starting byte position of the target buffer
// sourceStart: the starting byte position of the copy data source buffer
// sourceEnd: end byte position of the copy data source buffer (not included)
// The copied content ends when the specified sourceEnd or the target buffer is filled

const b1 = Buffer.from('Hello abcde')
const b2 = Buffer.alloc(6)

b1.copy(b2)
console.log(b1) // <Buffer e4 bd a0 e5 a5 bd 61 62 63 64 65>
console.log(b1.toString()) // Hello, abcde
console.log(b2) // <Buffer e4 bd a0 e5 a5 bd>
console.log(b2.toString()) // Hello

const b3 = Buffer.alloc(6)
b1.copy(b3, 2, 3, 7)
console.log(b3) // <Buffer 00 00 e5 a5 bd 61>
console.log(b3.toString()) // good

Buffer static method

Common static methods:

• concat: concatenate multiple buffers (arrays) into a new buffer to obtain data composed of multiple buffers
• isBuffer: determines whether the current data is of Buffer type

// concat
// The first parameter is an array of buffer s to be spliced
// The second parameter is the length of the splice result to be limited
const b1 = Buffer.from('Hello')
const b2 = Buffer.from('world')

const b3 = Buffer.concat([b1, b2])
console.log(b3.toString()) // Hello world

const b4 = Buffer.concat([b1, b2], 9)
console.log(b4.toString()) // Hello world

// isBuffer
console.log(Buffer.isBuffer([])) // false
console.log(Buffer.isBuffer('123')) // false
console.log(Buffer.isBuffer(b1)) // true

Custom Buffer split

Buffer is similar to array in many aspects, such as subscript operation, length attribute (buffer length is fixed and cannot be modified), etc.

However, there is no split operation for Buffer at present, and this operation is very common in business. Therefore, a split method is customized by using the native method provided by Buffer itself.

// Custom button split
Buffer.prototype.split = function (separator) {
  const len = Buffer.from(separator).length // Gets the number of delimiter bytes
  let res = [] // Final return result
  let start = 0 // Query start position
  let offset = 0 // Offset

  while ((offset = this.indexOf(separator, start)) !== -1) {
    res.push(this.slice(start, offset))
    start = offset + len
  }

  // Append tail
  res.push(this.slice(start))

  return res
}

const buf = Buffer.from('In a husband's war, you have courage, work hard, decline again, exhaust three times, do everything you can, Gu Kezhi,')

console.log(buf.split('，').map(v => v.toString()))