[JS elevation] Typed Array

0. Preface

​ About Typed Array, MDN There is a paragraph worth reading first to have a basic understanding.

0.1 what are typed arrays?

​ What is a stereotyped array? In one sentence, a stereotyped array is a kind of array object that has the ability to read and write native binary data in the memory buffer. (described later, it is actually an ArrayBuffer view)

JavaScript typed arrays are array-like objects that provide a mechanism for reading and writing raw binary data in memory buffers.

​ The length of the common array we used before can change dynamically, and its element type is not limited. JavaScript has a powerful engine, so these array operations can be processed quickly.
​ However, with the enhancement of modern web application ability, various features are constantly added to meet the needs. For example, audio and video, using WebSockets to access raw data, and so on. As well as the growing power of WebGL, JavaScript needs stronger data processing ability. The existing ordinary data is not convenient to deal with a large number of binary data, and there is a lot of performance loss in the middle.
​ Therefore, we need JavaScript code to operate the original binary data quickly and simply, which is the opportunity for the birth of typed arrays.
Each element in a stereotyped array is an original binary value, which can be from 8-bit integer to 64 bit floating-point number.

It is worth noting that ordinary arrays should not be confused with stereotyped arrays. Although we will introduce many of its methods later, ordinary arrays seem to be supported. But when learning, it's best to be completely independent. If you try to use the Array.isArray() static method provided by the Array constructor to judge a stereotyped Array, it will return false

1. Buffers and Views

​ In order to make the stereotyped array flexible and efficient, JavaScript stereotyped array divides its implementation into two parts: buffers and views
A buffer is an object (chunk of data) representing a data block, which is implemented by the ArrayBuffer object; It has no so-called format, nor does it provide a mechanism to access its content.
​ So you need to use a view to access the memory contained in a buffer. A view provides a context, that is, data type, starting offset, and the number of elements.

1.1 ArrayBuffer

ArrayBuffer is usually a data type used to represent general and fixed length binary data cache. The following figure shows the common ArrayBuffer:

Allocate a 16 byte ArrayBuffer in memory, which can be filled with elements of different bit lengths.

  • Uint8Array: uint8array type array represents an 8-bit unsigned integer array. (U is unsigned)
  • Uint16Array: 16 bit unsigned integer array;
  • Uint32Array: 32-bit unsigned integer array;
  • Float64Array: 64 bit floating point array;

Signed or not: the difference lies in the different representation ranges of values. For example, the value range of Int8Array is: - 128 ~ 127, but the value range of Uint8Array is: 0 ~ 255. The actual range size is the same, but the values are different.

Calculation of value range: for example, UInt16Array means that the element length is 16 bits, and the maximum value that can be represented is 16 bits. All bits are set to 1. The binary calculation result is decimal 65535, that is, 2 ^ 16 - 1, the minimum value is set to 0, and the decimal conversion is also 0. Therefore, the value range represented by unsigned 16bit is 0 ~ 65535. While Int16Array is signed, so the most bit is symbolic , indicates positive and negative, and the remaining 15 bits are used to represent the value, so the maximum value, i.e. 15 bits, is set to 1, i.e. 32767. As for the minimum value, it is set to negative in the high position, and the other bits are all 1, i.e. - 32727, but minus 1. Therefore, the value range represented by signed 16bit is - 32728 ~ 32727. As for why there is no exploration here, the same is true for other bits.

Another example is Uint32Array, which is 32 bit s. Because it is unsigned, the maximum binary value it can represent is 32 1, 4294967295, so the representation range is 0 ~ 4294967295. See here for more value ranges link.

1.1.1 create an ArrayBuffer instance

const buf = new ArrayBuffer(16); // Allocate 16 bytes in memory

1.1.2 read the byte length of ArrayBuffer

alert(buf.byteLength); // 16

⚠️ be careful:

  • Once the ArrayBuffer is created, it cannot be dynamically resized
  • To read or write to the ArrayBuffer, you must pass through the view. The views have different types, but they all refer to the binary data stored in the ArrayBuffer.

1.2 DataView

​ This is the first view type that allows you to read and write to ArrayBuffer - DataView.
​ This view is designed for file I/O and network I/O. its API supports a high degree of control over buffered data, but its performance is worse than other types of views. DataView has no preset for buffered content and cannot iterate.

​ A DataView instance can only be created when reading or writing to an existing ArrayBuffer. This instance can use all or part of the ArrayBuffer, and maintains a reference to the buffer instance and the starting position of the view in the buffer.

1.2.1 create DataView instance

​ The constructor of DataView receives three parameters: the target ArrayBuffer instance, the optional byte offset (byteOffset), and the optional byte length (byteLength)

const buf = new ArrayBuffer(16);
const firstHalfDataView = new DataView(buf, 0, 8);
alert(firstHalfDateView.byteLength); // 8

Accessing DataView instance properties

  • byteOffset: the buffer starting point for accessing the view. In the above example, firstHalfDataView.byteOffset returns 0;
  • byteLength: the byte length of the access view. In the above example, firstHaldDataView.byteLength returns 8;
  • Buffer: access the target buffer instance accessed by the view. In the above example, executing firsthalddataview. Buffer = = = buffer will return true

If only one optional parameter is provided, it will be treated as a byte offset, that is, the starting point of the buffer, and then all remaining arraybuffers will be used

const buf = new ArrayBuffer(16);
const secondHalfDataView = new DataView(buf, 8); // Will start from the 9th byte to all remaining bytes
alert(secondHalfDataView.byteLength); // 8

If no optional parameters are provided, the entire ArrayBuffer will be used by default

const buf = new ArrayBuffer(16);
const fullDataView = new DataView(buf);
alert(fullDataView.byteLength); //16

1.2.2 accessing the buffer through DataView

To read the buffer through DataView, several components are required:

  1. The first is the byte offset to be read or written. It can be regarded as some kind of "address" in DataView
  2. DataView should use ElementType to convert JavaScript Number type to binary format in buffer.
  3. The last is the byte order of the value in memory. The default is the big endian byte order.
1.2.2.1 ElementType

DataView has no preset data types stored in the buffer. Its exposed API forces developers to specify an ElementType when reading and writing. ES6 supports 8 different elementtypes

type Int8 Uint8 Int16 Uint16 Int32 Uint32 Float32 Float64
byte 1 1 2 2 4 4 4 8

DateView exposes get and set methods for each type in the above table. These methods use byteOffset (byte offset) to locate the location where the value is to be read or written. Types are interchangeable.

// Allocate two bytes in memory and declare a DataView
const buf = new ArrayBuffer(2);
const view = new DataView(buf);

// It indicates that all binary bits in the whole buffer are indeed 0
// Check the first and second characters
alert(view.getInt8(0));// 0
alert(view.getInt8(1));// 0
// Check the entire buffer
alert(view.getInt16(0)); // 0

// Set the entire buffer to 1
// The binary representation of 255 is 0xFF
view.setUint8(1,0xFF);

//Now, there are 1 in the buffer
// If you treat it as a signed integer with two complements, it should be - 1
alert(view.getInt16(0)); // -1
1.2.2.2 byte order

The large end byte order and small end byte order of 0x1234567 are written as follows.

01 is high (most significant bit) and 67 is low.

About byte order, you can see Ruan Yifeng's article Understanding byte order

The computer circuit first processes the low order bytes, which is more efficient, because the calculation starts from the low order. Therefore, the internal processing of the computer is small end byte order.

However, humans are still used to reading and writing large endian byte order. Therefore, in addition to the internal processing of the computer, almost all other occasions are large end byte order, such as network transmission and file storage.

All API methods of DataView take the large endian byte order as the default value, but receive an optional Boolean parameter. Set it to true to enable the small endian byte order. (omitted)

Basically, I don't need it. I'll look at it when I can use it. I don't need to study it deeply.

1.2.2.3 boundary conditions

DataView must have sufficient buffer before reading and writing, otherwise RangeError will be thrown:

const buf = new ArrayBuffer(6);
consr view = new DataView(buf);

// An attempt was made to read a value that was partially out of buffer
view.getInt32(4);
// An attempt was made to read a value outside the buffer range
view.getInt32(8);
// An attempt was made to write a value outside the buffer range
view.setInt32(4,123);

1.3 training array

A training array is another form of ArrayBuffer view. Conceptually, it is similar to DataView, but the difference of stereotyped array is that it is specific to an ElementType and follows the system's native byte order. Accordingly, the stereotyped array provides a broader API and higher performance.

​ The purpose of designing stereotyped array is to improve the efficiency of exchanging binary data with native libraries such as WebGL. Because the binary representation of stereotyped arrays is an easy-to-use format for the operating system, the JavaScript engine can heavily optimize arithmetic operations, bitwise operations and other common operations on stereotyped arrays, so it is very fast to use them.

​ The ways to create a training array include reading existing buffers, using free buffers, filling an iterative structure, and filling a training array based on any type. In addition, you can also create a training array through < ElementType >. From() and < ElementType. Of():

1.3.1 stereotype array creation

Create a 12 byte buffer

const buf = new ArrayBuffer(12);
// Create an Int32Array that references the buffer
const ints = new Int32Array(buf);

// This stereotyped array knows that each element needs 4 bytes / / 32bit / 8bit = 4byte
// Therefore, the length of 3 // new Int32Array() means that each element bit is 32bit, that is, 4 bytes. A 12 byte buffer occupies 3 element bits, that is, the length is 3
alert(ints.length); // 3

Create an Int32Array with a length of 6

const ints2 = new Int32Array(6);
// Each value uses 4 bytes, so ArrayBuffer is 24 bytes, Intl
alert(ints2.length); // 6

// Similar to DataView, the stereotyped array also has a reference to the key buffer
alert(ints2.buffer.byteLength); //24

Create an Int32Array containing [2,3,6,8]

const ints3 = new Int32Array([2,3,6,8]);
alert(ints.length);		//4
alert(ints3.buffer.byteLength);	//16 / / each element bit is 32bit, and four elements are 16 bytes
alert(ints3[2]);	//6

Create an Int16Array by copying the value of ints3

const ints4 = new Int16Array(ints3);

// This new type array allocates its own buffer
// Each value of the corresponding index is converted to the new format accordingly
alert(ints4.length);	//4
alert(ints4.buffer.byteLength);	//8 each element bit is 16bit, that is, 2 bytes, and four elements are 8 bytes
alert(ints5[2]);	//6

Create an Int16Array based on the ordinary array class

const ints5 = Int16Array.from([3, 5, 7, 9]);
alert(ints5.length);	// 4
alert(ints5.buffer.byteLength); //8
alert(ints5[2]);	//7

Create a Float32Array based on the passed in parameters

const floats = Float32Array.of(3.14, 2.718, 1.618);
alert(floats.length);		//3
alert(floats.buffer.byteLength); //12
alert(floats[2]);	//1.6180000305175781

1.3.2 BYTES_PER_ELEMENT property

Both the constructor and the instance of the stereotyped array have a BYTES_PER_ELEMENT property, which returns the size of each element in the type array:

alert(Int16Array.BYTES_PER_ELEMENT);//2
alert(Int32Array.BYTES_PER_ELEMENT);//4
const ints = new Int32Array(1);
const float = new Float64Array(2);
alert(ints.BYTES_PER_ELEMENT);//4
alert(floats.BYTES_PER_ELEMENT);//8

If the training array is not initialized with any value, its associated buffer is filled with 0:

const ints = new Int32Array(4);
alert(init[0]); //0
alert(init[1]); //0
alert(init[2]); //0
alert(init[3]); //0

1.3.3 stereotype array behavior

In many ways, a stereotyped array is similar to a normal array. The training array supports the following operators, methods and properties:

  • []
  • copyWithin()
  • entries()
  • every()
  • fill()
  • filter()
  • find()
  • findIndex()
  • forEach()
  • indexOf()
  • join()
  • keys()
  • lastIndexOf()
  • length
  • map()
  • reduce()
  • reduceRight()
  • reverse()
  • slice()
  • some()
  • sort()
  • toLocalString()
  • toString()
  • values()

⚠️ The method that returns a new array will also return a new stereotyped array containing the same element type:

const ints = new Int16Array([1,2,3]);
consr doubleints = ints.map(x=> 2*x);
alert(doubleints instanceof Int16Array); // true

1.3.4 iteration of training array

The training array has a Symbol.iterator symbol attribute, so it can be operated through the for...of loop and extension operator:

const ints = new Int16Array([1,2,3]);
for (const int of ints){
    alert(int);
}
// 1
// 2
// 3
alert(Math.max(...ints)) ; //3

1.3.5 the common array merging | copying | modifying method is not applicable to the stereotyped array

Stereotyped arrays also use array buffers to store data, which cannot be resized. Therefore, the following methods do not apply to training arrays:

  • concat()
  • pop()
  • push()
  • shift()
  • splice()
  • unshift()

However, the stereotyped array also provides two new methods to quickly copy data inward or outward:

1.3.6 training method

1.3.6.1 copy methods of training array set() and subarray()
  • set()
  • subarray()

set() copies the value from the provided array or training array to the index position specified in the current training array:

// Create int16 array with length 8
const container = new Int16Array(8);
// Assign the training array to the first four values
// The offset defaults to index 0 
container.set(Int8Array.of(1,2,3,4));
console.log(container);// [1,2,3,4,0,0,0,0]

// Assign a normal array to the last four values
// The offset of 4 indicates the insertion from index 4
container.set([5,6,7,8], 4);
console.log(container); // [1,2,3,4,5,6,7,8]

// Overflow will throw an error
container.set([5,6,7,8], 7);
// RangeError

subarray() does the opposite of set(), which returns a new training array based on the values copied from the original training array.

The start and end indexes when copying values are optional:

const source = Int16Array.of(2,4,6,8);

// Assign the entire array to a new array of the same type
const fullCopy = source.subarray();
console.log(fullCopy); //[2,4,6,8]

// Copy the array from index 2 (containing the value at index 2)
const halfCopy = souce.subarray(2);
console.log(halfCopy); // [6,8]

// Copy from index 1 to index 3 (including start index and excluding end index)
const partialCopy = source.subarray(1, 3);
console.log(partialCopy); // [4,6]
1.3.6.2 shaped array splicing capacity

The stereotyped array has no native splicing capability, but it can be built manually using many tools provided by the stereotyped array API:

// The first parameter is the array type that should be returned
// The remaining parameters are stereotyped arrays that should be spliced together

function typedArrayConcat(typedArraysConstructor, ...typedArrays) {
    // Calculates the total number of elements contained in all arrays
    const numElements = typedArrays.reduce((x,y) => (x.length || x) + y.length);
    
    // Create an array according to the type provided, leaving space for all elements
    const resultArray = new typedArrayConstructor(numElements);
    
    // Transfer array in turn
    let currentOffset = 0;
    typedAArrays.map(x => {
        resultArray.set(x, currentOffset);
        currentOffset += x.length;
    });
    
    return resultArray;
}

const concatArray = typedArrayConcat (IntArray,
                                      Int8Array.of(1,2,3),
                                      Int16Array.of(4,5,6),
                                      Float32Array.of(7,8,9));
console.log(concatArray); //[1,2,3,4,5,6,7,8,9]
console.log(concatArray instanceof Int32Array); // true

1.3.7 underflow and overflow

In a stereotyped array, the underflow or overflow of values will not affect other indexes, but we still need to consider what type the elements of the array should be.

The training array receives only one correlation bit for each index that can be stored, regardless of their impact on the actual value. The following code demonstrates how to handle underflow and overflow:

// Signed integer array of length 2
// Each index holds a signed integer in the form of two complements (the range is - 1 * 2 ^ 7 ~ (2 ^ 7 - 1)
const ints = new Int8Array(2);

// Unsigned integer array of length 2
// Each index holds an unsigned integer (0 ~ 255) / / (2 ^ 7 - 1)
const unsignedInts = new Uint8Array(2);

// Overflow bits do not affect adjacent indexes
// The index takes only 8 bits on the least significant bit
unsignedInts[1] = 256; //0x100
console.log(unsignedInts);//[0, 0]
unsignedInts[1] = 511; // 0x1FF
console.log(unsignedInts);// [0,255]

// The underflow bit is converted to its unsigned equivalent 
// 0xFF is - 1 in the form of two complements (intercepted to 8 bits), 
// But 255 is an unsigned integer 
unsignedInts[1] = -1        // 0xFF (truncated to 8 bits) 
console.log(unsignedInts);  // [0, 255] 
 
// Overflow automatically becomes a two complement form 
// 0x80 is 128 of unsigned integer and - 128 in the form of two complements 
ints[1] = 128;        // 0x80 
console.log(ints);    // [0, -128] 
 
// Underflow automatically becomes a two complement form 
// 0xFF is 255 of unsigned integer and - 1 in the form of two complements 
ints[1] = 255;        // 0xFF 
console.log(ints);    // [0, -1]

In addition to the eight element types, there is also a "splint" array type: uint8clapedarray, which does not allow overflow in any direction. Values that exceed the maximum value of 255 are rounded down to 255, while 0 less than the minimum value is rounded up to 0.

const clampedInts = new Uint8ClampedArray([-1, 0, 255, 256]);
console.log(clampedInts); // [0, 0, 255, 255]

Keywords: Javascript

Added by Joe689 on Tue, 07 Dec 2021 18:33:05 +0200