Catalog

1. Introduction to WebGL
   1.1. Basic principles of WebGL
   1.2. How WebGL works
   1.3. WebGL Colorizer and GLSL
2. image processing
   2.1. WebGL image processing
3. 2D Conversion, Rotation, Scaling, Matrix
   2.1. WebGL 2D image conversion
   2.2. WebGL 2D image rotation
   2.3. WebGL 2D Image Scaling
   2.4. WebGL 2D Matrix
4. 3D
   4.1. WebGL 3D Orthogonal
   4.1. WebGL 3D Perspective
   4.1. WebGL 3D Camera

Introduction to WebGL

WebGL is a 3D drawing standard that allows JavaScript to be combined with OpenGL ES 2.0. By adding a JavaScript binding to OpenGL ES 2.0, WebGL can provide hardware 3D accelerated rendering for HTML5 Canvas so that Web developers can display 3D scenes and models more smoothly in browsers using system graphics cards. It also creates complex navigation and data visualization.

WebGL Foundation

Basic principles of WebGL

The advent of WebGL makes it possible to display 3D images on browsers, which are essentially based on rasterized APIs rather than 3D-based APIs. WebGL only focuses on two aspects, the coordinates of the projection matrix and the color of the projection matrix. The task of using a WebGL program is to implement a WebGL object with projection matrix coordinates and colors. You can use the Shader to accomplish these tasks. The vertex shader provides the coordinates of the projection matrix, and the segment shader provides the color of the projection matrix.

The coordinates of the projection matrix always range from -1 to 1, regardless of the size of the graphic to be implemented. For example:

// Get A WebGL context 
var canvas = document.getElementById("canvas"); 
var gl = canvas.getContext("experimental-webgl");   
// setup a GLSL program 
var program = createProgramFromScripts(gl, ["2d-vertex-shader", "2d-fragment-shader"]); gl.useProgram(program);  
// look up where the vertex data needs to go. 
var positionLocation = gl.getAttribLocation(program, "a_position");  
// Create a buffer and put a single clipspace rectangle in 
// it (2 triangles) 
var buffer = gl.createBuffer(); 
gl.bindBuffer(gl.ARRAY_BUFFER, buffer); 
gl.bufferData( gl.ARRAY_BUFFER, new Float32Array([ -1.0, -1.0, 1.0, -1.0, -1.0, 1.0, -1.0, 1.0, 1.0, -1.0, 1.0, 1.0]), 
gl.STATIC_DRAW); 
gl.enableVertexAttribArray(positionLocation); 
gl.vertexAttribPointer(positionLocation, 2, gl.FLOAT, false, 0, 0);   
// draw 
gl.drawArrays(gl.TRIANGLES, 0, 6);

Below are two shaders.

<script id="2d-vertex-shader" type="x-shader/x-vertex">
attribute vec2 a_position; 
void main() { 
    gl_Position = vec4(a_position, 0, 1); 
} 
</script>
<script id="2d-fragment-shader" type="x-shader/x-fragment"> 
void main() { 
    gl_FragColor = vec4(0, 1, 0, 1); // green
} 
</script>

If you want a 3D effect, you can use a shader to convert 3D to a projection matrix because WebGL is a raster-based API. For 2D images, it may be possible to use pixels instead of projection matrices to represent dimensions, so here we change the shader so that the rectangle we implement can be measured in pixels. Below is the new vertex shader.

attribute vec2 a_position; 

uniform vec2 u_resolution; 

void main() { 
    // convert the rectangle from pixels to 0.0 to 1.0 
    vec2 zeroToOne = a_position / u_resolution; 
    // convert from 0->1 to 0->2 
    vec2 zeroToTwo = zeroToOne * 2.0; 
    // convert from 0->2 to -1->+1 (clipspace) 
    vec2 clipSpace = zeroToTwo - 1.0; 
    gl_Position = vec4(clipSpace, 0, 1); 
}

How WebGL works

How WebGL and GPU work. GPU has two basic tasks, the first is to process points as projection matrices. The second part describes the corresponding pixel points based on the first part. When called by the user

gl.drawArrays(gl.TRIANGLE, 0, 9);   

Here 9 means "nine vertices are processed", so there are nine vertices to be processed. [External chain picture transfer failed, source station may have anti-theft chain mechanism, it is recommended to save the picture and upload it directly (img-kEtRQ8hd-1631442055895)(en-resource://database/4815:1)]

On the left of the image above is the data provided by the user himself. The vertex shader is a function written by the user in GLSL. Each vertex is called once when it is processed. Users can store the values of the projection matrix in a specific variable gl_Position.

The GPU processes these values and stores them inside. Assuming the user wants to draw the triangle TRIANGLES, the first part above will produce three vertices each time it is drawn, and the GPU will use them to draw the triangle.

The GPU first draws the pixels corresponding to the three vertices, then rasterizes the triangles, or draws them using pixel points. For each pixel point, the GPU calls the user-defined fragment shader to determine what color the pixel point should be painted with. Of course, the user-defined fragment shader must be in gl_ Set the corresponding value in the FragColor variable.

The segment shader in our example does not store information for each pixel. We can store more information in it. We can define different meanings for each value from the vertex shader to the segment shader.

As a simple example, we will transfer the directly calculated projection matrix coordinates from the vertex shader to the segment shader. We'll draw a simple triangle. Let's make a change based on the previous example.

function setGeometry(gl) {
    gl.bufferData( 
        gl.ARRAY_BUFFER, new Float32Array([ 0, -100, 150, 125, -175, 100]),                                 gl.STATIC_DRAW
    ); 
}

Next, we draw three vertices.

// Draw the scene. 
function drawScene() { 
       ... // Draw the geometry. 
       gl.drawArrays(gl.TRIANGLES, 0, 3);
}

We can then define variables in the vertex shader to pass data to the fragment shader.

varying vec4 v_color; 
... void main() { 
    // Multiply the position by the matrix. 
    gl_Position = vec4((u_matrix * vec3(a_position, 1)).xy, 0, 1); 
    // Convert from clipspace to colorspace.
    // Clipspace goes -1.0 to +1.0 
    // Colorspace goes from 0.0 to 1.0 
    v_color = gl_Position * 0.5 + 0.5;
}

Then, we declare the same variables in the fragment shader.

precision mediump float; 
varying vec4 v_color; 
void main() { 
    gl_FragColor = v_color; 
}

WebGL will connect variables in the vertex shader to variables of the same name and type in the fragment shader. The following is an interactive version.

Move, scale, or rotate the triangle. Note that since the color is calculated from the projection matrix, the color will not always be the same as the triangle moves. They are set entirely according to the background color.

Now let's consider the following. We only calculate three vertices. Our vertex shader was called three times, so only three colors were calculated. And our triangles can have many colors, which is why they are called   varying.

WebGL uses the three values we compute for each vertex and rasterizes the triangles. For each pixel, the fragment shader is invoked with the modified value.

Based on the example above, we start with three vertices
[External chain picture transfer failed, source station may have anti-theft chain mechanism, it is recommended to save the picture and upload it directly (img-0dZv4qpK-1631442055897)(en-resource://database/4817:1)]

Our vertex shader references a matrix to convert, rotate, scale, and convert to a projection matrix. The default values for conversion, rotation, and scaling are to convert to 200,150, rotate to 0, and scale to 1,1, so in fact only conversion occurs. Our background cache is 400x300. Our vertex matrix applies a matrix and then computes the three vertices of the projection matrix below.
[External chain picture transfer failed, source station may have anti-theft chain mechanism, it is recommended to save the picture and upload it directly (img-Ao53q6pQ-1631442055898)(en-resource://database/4819:1)]

They will also be converted to color space and written to our declared variable v_color. [External chain picture transfer failed, source station may have anti-theft chain mechanism, it is recommended to save the picture and upload it directly (img-NewWzvrC-1631442055900)(en-resource://database/4821:1)]
These three values are written back to v_color, which is then passed to the segment shader for each pixel to be shaded. [External chain picture transfer failed, source station may have anti-theft chain mechanism, it is recommended to save the picture and upload it directly (img-VR0FGgng-1631442055902)(en-resource://database/4823:1)]

v_color is modified to one of three values, v0, v1, and v2. We can also store more data in the vertex shader to transfer to the segment shader.

So, for example, drawing a rectangle with two triangle colors in two colors. To implement this example, we need to attach more attributes to the vertex shader so that more data can be transferred, which will be directly transferred to the fragment shader.

attribute vec2 a_position; 
attribute vec4 a_color; 
... 
varying vec4 v_color;  
void main() { 
... 
// Copy the color from the attribute to the varying. 
    v_color = a_color; 
}

We now need to use the WebGL color-related functionality.

 var positionLocation = gl.getAttribLocation (program, "a_position"); 
 var colorLocation = gl.getAttribLocation(program, "a_color"); 
 ...
 // Create a buffer for the colors. 
 var buffer = gl.createBuffer(); 
 gl.bindBuffer(gl.ARRAY_BUFFER, buffer); 
 gl.enableVertexAttribArray(colorLocation); 
 gl.vertexAttribPointer(colorLocation, 4, gl.FLOAT, false, 0, 0); 
 // Set the colors. 
 setColors(gl); 
 // Fill the buffer with colors for the 2 triangles 
 // that make the rectangle. 
 function setColors(gl) { 
    // Pick 2 random colors. 
    var r1 = Math.random(); 
    var b1 = Math.random(); 
    var g1 = Math.random(); 
    var r2 = Math.random(); 
    var b2 = Math.random(); 
    var g2 = Math.random(); 
    gl.bufferData( 
        gl.ARRAY_BUFFER, 
        new Float32Array( [ r1, b1, g1, 1, r1, b1, g1, 1, r1, b1, g1, 1, r2, b2, g2, 1, r2, b2, g2, 1, r2, b2, g2, 1]), 
    gl.STATIC_DRAW); 
} 

Here are the results. Notice in the example above that there are two triangles with a bitter color. We will still store the values to be passed in a multivariable, so the variable will change within the associated triangle area. We just use the same color for the three vertices of each triangle. If we use different colors, we can see the entire rendering process.

// Fill the buffer with colors for the 2 triangles
// that make the rectangle. 
function setColors(gl) { 
    // Make every vertex a different color. 
    gl.bufferData( 
        gl.ARRAY_BUFFER, 
        new Float32Array( [ 
            Math.random(), 
            Math.random(), 
            Math.random(), 1, 
            Math.random(), 
            Math.random(), 
            Math.random(), 1, 
            Math.random(), 
            Math.random(), 
            Math.random(), 1, 
            Math.random(), 
            Math.random(), 
            Math.random(), 1, 
            Math.random(), 
            Math.random(), 
            Math.random(), 1, 
            Math.random(), 
            Math.random(), 
            Math.random(), 1]), gl.STATIC_DRAW); 
 }

More and richer data can be transferred from the vertex shader to the segment shader.

Caching and attribute directives

Caching is the method of getting vertex and vertex-related data into the GPU. gl.createBuffer is used to create the cache.
The gl.bindBuffer method is used to activate the cache in a ready state.
The gl.bufferData method copies data into the cache. Once the data is in the cache, you need to tell WebGL how to remove it from the cache and provide it to the vertex shader to assign values to the corresponding properties.
To do this, we first need to find out that WebGL provides a property storage location.

// look up where the vertex data needs to go. 
var positionLocation = gl.getAttribLocation(program, "a_position"); 
var colorLocation = gl.getAttribLocation(program, "a_color");

We can trigger two commands

gl.enableVertexAttribArray(location);

This directive tells WebGL that we want to assign the data in the cache to a variable.

gl.vertexAttribPointer( 
    location, 
    numComponents, 
    typeOfData, 
    normalizeFlag, 
    strideToNextPieceOfData, 
    offsetIntoBuffer,
);

This directive tells WebGL to fetch data from the cache, which is bound to gl.bindBuffer. Each vertex can have one to four parts, and the data type can be BYTE,FLOAT,INT,UNSIGNED_SHORT, etc. Skipping means how many bytes will be crossed from this piece of data to that piece of data. How far across is stored in the cache as an offset. The number of parts is usually 1 to 4. If only one cache is used for each data type, then the span and offset will be 0. A span of 0 means "use a span match type and size". An offset of 0 means that it is at the beginning of the cache. Assigning this value to a value other than O provides more flexibility. Although it has some performance advantages, it is not worth the complexity unless the programmer wants to use WebGL to its fullest.

Normalized flag normalizeFlag for vertexAttribPointer

Normalization flags are applied to non-floating point pointer types. If the value is set to false, it means that the value will be translated to type. The label range for BYTE is -128 to 127. UNSIGNED_BYTE ranges from 0 to 255 and SHORT from -32768 to 32767.

If the normalization flag is set to true, the label range for BYTE will change to -1.0 to + 1.0,UNSIGNED_BYTE will become 0.0 to + 1.0, and normalized SHORT will become -1.0 to + 1.0, which will be more accurate than BYTE. The most common place for standardized data is for color.

Most of the time, colors ranging from 0.0 to 1.0 Red, green, and blue require floating-point values to represent, and alpha requires 16 bytes to represent each color of the vertices. If you want more complex graphics, you can add more bytes.

Instead, the program can turn the color into UNSIGNED_BYTE type, which uses 0 for 0.0 and 255 for 1.0. Only four bytes are needed to represent each color of the vertex, which saves 75% of storage space. Let's change our code in the following way. When we tell WebGL how to get colors.

gl.vertexAttribPointer(colorLocation, 4, gl.UNSIGNED_BYTE, true, 0, 0);

Use the following code to fill our buffer

function setColors(gl) { 
    // Pick 2 random colors. 
    var r1 = Math.random() * 256; 
    // 0 to 255.99999 
    var b1 = Math.random() * 256; 
    // these values 
    var g1 = Math.random() * 256; 
    // will be truncated 
    var r2 = Math.random() * 256; 
    // when stored in the 
    var b2 = Math.random() * 256;
    // Uint8Array 
    var g2 = Math.random() * 256; 
    gl.bufferData( gl.ARRAY_BUFFER, new Uint8Array( 
    [ r1, b1, g1, 255, r1, b1, g1, 255, r1, b1, g1, 255, r2, b2, g2, 255, r2, b2, g2, 255, r2, b2, g2, 255]), gl.STATIC_DRAW); 
}

WebGL Colorizer and GLSL

Each time a WebGL draws, it requires two shaders, the vertex shader and the segment shader. Each shader is a function. Both vertex and fragment shaders are linked in the program. A typical WebGL program contains many of these shaders.

Vertex Shader

The task of the vertex shader is to generate the coordinates of the projection matrix. It takes the following form:

void main() { 
    gl_Position = doMathToMakeClipspaceCoordinates 
}

Each vertex calls your shader. Each caller needs to set a specific global variable gl_Position represents the coordinates of the projection matrix. The vertex shader requires data, which can be obtained in three ways.

• Properties (get data from buffer)
• Consistent variables (values that are consistent each time a painting is called)
• Texture (data from pixels)

attribute
The most common way is to use caches and attributes. Programs can create caches in the following ways.

var buf = gl.createBuffer();

Store data in these caches.

gl.bindBuffer(gl.ARRAY_BUFFER, buf); gl.bufferData(gl.ARRAY_BUFFER, someData, gl.STATIC_DRAW);

Thus, given a shader program, the program can find the location of attributes.

var positionLoc = gl.getAttribLocation(someShaderProgram, "a_position");

The following shows WebGL how to get data from the cache and store it in attributes.

// turn on getting data out of a buffer for this attribute 
gl.enableVertexAttribArray(positionLoc);
var numComponents = 3;  // (x, y, z) 
var type = gl.FLOAT; 
var normalize = false; // leave the values as they are 
var offset = 0; // start at the beginning of the buffer 
var stride = 0; // how many bytes to move to the next vertex // 0 = use the correct stride for type and numComponents 
gl.vertexAttribPointer(positionLoc, numComponents, type, false, stride, offset);

If we can put the projection matrix in our cache, it will start working. Attributes can be of type float, vec2, vec3, vec4, mat2, mat3, and mat4.

Consistency variable
For vertex shaders, the consistency variable is the value that remains constant in the shader for each call to the drawing. Below is an example of adding an offset shader to a vertex.

attribute vec4 a_position; 
uniform vec4 u_offset; 
void main() { 
    gl_Position = a_position + u_offset; 
}

Next, we need to offset each vertex by a certain amount. First, we need to find the location of the consistent variable.

var offsetLoc = gl.getUniformLocation(someProgram, "u_offset");

Then we need to set the consistency variable before drawing

gl.uniform4fv(offsetLoc, [1, 0, 0, 0]); // offset it to the right half the screen

There are many types of consistency variables. For each type, you can call the appropriate function to set it.

gl.uniform1f (floatUniformLoc, v); // for float 
gl.uniform1fv(floatUniformLoc, [v]); // for float or float array 
gl.uniform2f (vec2UniformLoc, v0, v1);// for vec2 
gl.uniform2fv(vec2UniformLoc, [v0, v1]); // for vec2 or vec2 array 
gl.uniform3f (vec3UniformLoc, v0, v1, v2);// for vec3 
gl.uniform3fv(vec3UniformLoc, [v0, v1, v2]); // for vec3 or vec3 array 
gl.uniform4f (vec4UniformLoc, v0, v1, v2, v4);// for vec4 
gl.uniform4fv(vec4UniformLoc, [v0, v1, v2, v4]); // for vec4 or vec4 array 
gl.uniformMatrix2fv(mat2UniformLoc, false, [ 4x element array ]) // for mat2 or mat2 array 
gl.uniformMatrix3fv(mat3UniformLoc, false, [ 9x element array ]) // for mat3 or mat3 array 
gl.uniformMatrix4fv(mat4UniformLoc, false, [ 17x element array ]) // for mat4 or mat4 array 
gl.uniform1i (intUniformLoc, v); // for int 
gl.uniform1iv(intUniformLoc, [v]); // for int or int array 
gl.uniform2i (ivec2UniformLoc, v0, v1);// for ivec2 
gl.uniform2iv(ivec2UniformLoc, [v0, v1]); // for ivec2 or ivec2 array 
gl.uniform3i (ivec3UniformLoc, v0, v1, v2);// for ivec3 
gl.uniform3iv(ivec3UniformLoc, [v0, v1, v2]); // for ivec3 or ivec3 array 
gl.uniform4i (ivec4UniformLoc, v0, v1, v2, v4);// for ivec4 
gl.uniform4iv(ivec4UniformLoc, [v0, v1, v2, v4]); // for ivec4 or ivec4 array 
gl.uniform1i (sampler2DUniformLoc, v); // for sampler2D (textures) 
gl.uniform1iv(sampler2DUniformLoc, [v]); // for sampler2D or sampler2D array 
gl.uniform1i (samplerCubeUniformLoc, v); // for samplerCube (textures) 
gl.uniform1iv(samplerCubeUniformLoc, [v]); // for samplerCube or samplerCube array

Common types have bool, bvec2, bvec3, and bvec4. Their corresponding call function is gl.uniform?f? Or gl.uniform?i?. You can set all the consistency variables in the array at once. For example:

uniform vec2 u_someVec2[3]; 
gl.getUniformLocation(someProgram, "u_someVec2");
gl.uniform2fv(someVec2Loc, [1, 2, 3, 4, 5, 6]);

However, if the program wants to set the members in the array individually, it must query each member's location individually.

var someVec2Element0Loc = gl.getUniformLocation(someProgram, "u_someVec2[0]"); var someVec2Element1Loc = gl.getUniformLocation(someProgram, "u_someVec2[1]"); var someVec2Element2Loc = gl.getUniformLocation(someProgram, "u_someVec2[2]"); 
// at render time 
gl.uniform2fv(someVec2Element0Loc, [1, 2]); // set element 0 
gl.uniform2fv(someVec2Element1Loc, [3, 4]); // set element 1 
gl.uniform2fv(someVec2Element2Loc, [5, 6]); // set element 2

Similarly, you can create a structure

struct SomeStruct { 
    bool active; vec2 someVec2; 
}; 
uniform SomeStruct u_someThing;

Programs can query each member individually.

var someThingActiveLoc = gl.getUniformLocation(someProgram, "u_someThing.active"); var someThingSomeVec2Loc = gl.getUniformLocation(someProgram, "u_someThing.someVec2");

Fragment Shader

The task of the segment shader is to provide color for the pixels that are currently rasterized. It is usually presented in the following way.

precision mediump float; 
void main() { 
    gl_FragColor = doMathToMakeAColor; 
 }

The segment shader is called once for each pixel. The global variable gl_is set for each call FragColor to set some colors. Fragment shaders need to store and retrieve data in three ways.

• Consistent variable (called each time a pixel point is drawn and remains consistent)
• Texture (getting data from pixels)
• Multivariate variable (value passed from a fixed-point shader and rasterized)

Texture in Segment Shader
We can get values from textures to create sampler2D consistent variables, and then use the GLSL function texture2D to get values from them.

precision mediump float; 
uniform sampler2D u_texture;
void main() { 
    vec2 texcoord = vec2(0.5, 0.5) 
    gl_FragColor = texture2D(u_texture, texcoord); 
}

The value extracted from the texture depends on many settings. Basically, we need to create and store values in the text. For example:

var tex = gl.createTexture(); 
gl.bindTexture(gl.TEXTURE_2D, tex); 
var level = 0; 
var width = 2; 
var height = 1; 
var data = new Uint8Array([255, 0, 0, 255, 0, 255, 0, 255]);
gl.texImage2D(gl.TEXTURE_2D, level, gl.RGBA, width, height, 0, gl.RGBA, gl.UNSIGNED_BYTE, data);

Then, query the shader program for the location of the consistent variable.

var someSamplerLoc = gl.getUniformLocation(someProgram, "u_texture");

WebGL needs to bind it to a texture cell.

var unit = 5;
gl.activeTexture(gl.TEXTURE0 + unit);
gl.bindTexture(gl.TEXTURE_2D, tex);

Then tell the shader which cell will be bound to the texture

gl.uniform1i(someSamplerLoc, unit);

Multivariable
Multivariate variables are the values passed from the vertex shader to the segment shader, which are covered in how WebGL works. In order to use multivariates, matching multivariates need to be set in both the vertex and segment shaders. We set the multivariable in the vertex shader. When a WebGL draws a pixel, it rasterizes the value and passes it to the corresponding fragment shader in the fragment shader.

Vertex Shader

attribute vec4 a_position; 
uniform vec4 u_offset; 
varying vec4 v_positionWithOffset; 
void main() { 
    gl_Position = a_position + u_offset; 
    v_positionWithOffset = a_position + u_offset; 
}

Fragment Shader

precision mediump float; 
varying vec4 v_positionWithOffset; 
void main() { 
    // convert from clipsapce (-1 <-> +1) to color space (0 -> 1). 
    vec4 color = v_positionWithOffset * 0.5 + 0.5 
    gl_FragColor = color; 
}

GLSL

GLSL is short for Image Library Shader Language. The language shader is written here. It has some unique features that do not exist in JavaScript. It is used to implement some logic for rendering images. For example, it can create values similar to vec2, where VEC 3 and vec4 represent 2, 3, and 4 values, respectively. Similarly, mat2, mat3, and mat4 represent matrices of 2x2,3x3,4x4. You can implement VEC to multiply by a scalar.

vec4 a = vec4(1, 2, 3, 4); vec4 b = a * 2.0;

Implements matrix multiplication and vector multiplication of matrices:

mat4 a = ???
mat4 b = ???
mat4 c = a * b;

vec4 v = ???
vec4 y = c * v;

You can also choose the part of vec, for example, vec4

vec4 v;

• v.x is equivalent to v.s, v.r, v[0]
• v.y is equivalent to v.t, v.g, v[1]
• v.z is equivalent to v.p, v.b, v[2]
• v.w is equivalent to v.q, v.a, v[3]
  The ability to adjust vec components means that components can be exchanged or duplicated.

v.yyyy

This is equivalent to

vec4(v.y, v.y, v.y, v.y)

Allied

v.bgra

Equivalent to

vec4(v.b, v.g, v.r, v.a)

When a vec or a mat is created, the program can operate on multiple parts at once:

vec4(v.rgb, 1)

This is equivalent to

vec4(v.r, v.g, v.b, 1)

You may realize that GLSL is a very strict type of language

float f = 1;

The correct way is as follows:

float f = 1.0; // use float 
float f = float(1) // cast the integer to a float

The vec4(v.rgb, 1) example above does not confuse 1 because vec4 is similar to float (1). GLSL is a branch of built-in functions. It can operate on multiple components at once. For example,

T sin(T angle)

This means that T can be float, vec2, vec3, or vec4. If the user passes data in vec4. That is v is vec4,

vec4 s = sin(v);

Equivalent to

vec4 s = vec4(sin(v.x), sin(v.y), sin(v.z), sin(v.w));

Sometimes one parameter is float and the other is T. This means that float will be applied to all parts. For example, if v1, v2 is vec4, f is flat, then

vec4 m = mix(v1, v2, f);

This is equivalent to

vec4 m = vec4( 
    mix(v1.x, v2.x, f), 
    mix(v1.y, v2.y, f), 
    mix(v1.z, v2.z, f), 
    mix(v1.w, v2.w, f),
);

Mix it up

WebGL is about creating various shaders, storing data on these shaders, and then calling gl.drawArrays or gl.drawElements to have WebGL process vertices invoke the current vertex shader for each vertex, then invoke the current fragment shader for each pixel.

In fact, the shader creation requires several lines of code. Because these codes are the same in most WebGL programs, and once written, you can almost ignore how they compile GLSL shaders and link them into a shader program.

WebGL Image Processing

WebGL manipulates the coordinates of the projection matrix, and WebGL acquires texture coordinates when reading textures. Texture coordinates range from 0.0 to 1.0.

Since we only need to draw a rectangle made up of two triangles, we need to tell WebGL which point the texture corresponds to in the matrix. We can use special so-called multivariates to transfer this information from the vertex shader to the segment shader.

WebGL inserts these values, which are invoked in the vertex shader when each pixel is drawn. We need to add more information to the texture coordinate transfer process and pass them to the fragment shader.

attribute vec2 a_texCoord;
... 
varying vec2 v_texCoord; 
void main() { 
    ... 
    // pass the texCoord to the fragment shader 
    // The GPU will interpolate this value between points
    v_texCoord = a_texCoord; 
}

Fragment Shader to Find Color Texture

precision mediump float; 
// our texture 
uniform sampler2D u_image; 
// the texCoords passed in from the vertex shader. 
varying vec2 v_texCoord; 
void main() { 
    // Look up a color from the texture. 
    gl_FragColor = texture2D(u_image, v_texCoord); 
}

Load a picture and create a problem to pass the picture into the texture

function main() { 
    var image = new Image();
    image.src = "http://someimage/on/our/server"; 
    image.onload = function() { 
        render(image); 
    } 
} 
function render(image) {
    ... 
    // all the code we had before.
    ... 
    // look up where the texture coordinates need to go. 
    var texCoordLocation = gl.getAttribLocation(program, "a_texCoord");
    // provide texture coordinates for the rectangle. 
    var texCoordBuffer = gl.createBuffer(); 
    gl.bindBuffer(gl.ARRAY_BUFFER, texCoordBuffer); 
    gl.bufferData(gl.ARRAY_BUFFER, new Float32Array([ 0.0, 0.0, 1.0, 0.0, 0.0, 1.0, 0.0, 1.0, 1.0, 0.0, 1.0, 1.0]), 
    gl.STATIC_DRAW); 
    gl.enableVertexAttribArray(texCoordLocation); 
    gl.vertexAttribPointer(texCoordLocation, 2, gl.FLOAT, false, 0, 0); 
    // Create a texture. 
    var texture = gl.createTexture(); 
    gl.bindTexture(gl.TEXTURE_2D, texture); 
    gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_S, 
    gl.CLAMP_TO_EDGE); gl.texParameteri(gl.TEXTURE_2D, 
    gl.TEXTURE_WRAP_T, gl.CLAMP_TO_EDGE); 
    gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MIN_FILTER, gl.NEAREST); 
    gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MAG_FILTER, gl.NEAREST); 
    // Upload the image into the texture. 
    gl.texImage2D(gl.TEXTURE_2D, 0, gl.RGBA, gl.RGBA, gl.UNSIGNED_BYTE, image); 
    ... 
}

Here is the image rendered by WebGL
[External chain picture transfer failed, source station may have anti-theft chain mechanism, it is recommended to save the picture and upload it directly (img-gJ8YRx1n-1631442055903)(en-resource://database/4827:1)]
Here's how to swap the red and blue colors in this picture

... 
gl_FragColor = texture2D(u_image, v_texCoord).bgra;
...

Now red and blue have been exchanged
[External chain picture transfer failed, source station may have anti-theft chain mechanism, it is recommended to save the picture and upload it directly (img-HR5D0XEE-1631442055904)(en-resource://database/4829:1)]

Since WebGL referenced texture texture coordinates from 0.0 to 1.0. Then we can calculate how many pixels are moved, onePixel = 1.0 / textureSize. Here is a segment shader to average the left and right pixels of each pixel in the texture.

precision mediump float;

// our texture
uniform sampler2D u_image;
uniform vec2 u_textureSize;

// the texCoords passed in from the vertex shader.
varying vec2 v_texCoord;

void main() {
   // compute 1 pixel in texture coordinates.
   vec2 onePixel = vec2(1.0, 1.0) / u_textureSize;

   // average the left, middle, and right pixels.
   gl_FragColor = (
   texture2D(u_image, v_texCoord) +
   texture2D(u_image, v_texCoord + vec2(onePixel.x, 0.0)) +
   texture2D(u_image, v_texCoord + vec2(-onePixel.x, 0.0))) / 3.0;
}

Texture size passed through JavaScript

... 
var textureSizeLocation = gl.getUniformLocation(program, "u_textureSize"); 
... 
// set the size of the image 
gl.uniform2f(textureSizeLocation, image.width, image.height); ...

Compare the two pictures above
[External chain picture transfer failed, source station may have anti-theft chain mechanism, it is recommended to save the picture and upload it directly (img-B10V21RS-1631442055905)(en-resource://database/4831:1)]

Use a 3x3 kernel. The convolution kernel is a 3x3 matrix in which each entry represents how many pixels are rendered. Then we divide this result by the weight of the kernel or 1.0.

To do this in the shader, here is a new segment shader

precision mediump float;

// our texture 
uniform sampler2D u_image; 
uniform vec2 u_textureSize; 
uniform float u_kernel[9]; 
uniform float u_kernelWeight; 
// the texCoords passed in from the vertex shader. 
varying vec2 v_texCoord; 
void main() { 
    vec2 onePixel = vec2(1.0, 1.0) / u_textureSize; 
    vec4 colorSum = 
    texture2D(u_image, v_texCoord + onePixel * vec2(-1, -1)) * u_kernel[0] + 
    texture2D(u_image, v_texCoord + onePixel * vec2( 0, -1)) * u_kernel[1] + 
    texture2D(u_image, v_texCoord + onePixel * vec2( 1, -1)) * u_kernel[2] + 
    texture2D(u_image, v_texCoord + onePixel * vec2(-1, 0)) * u_kernel[3] + 
    texture2D(u_image, v_texCoord + onePixel * vec2( 0, 0)) * u_kernel[4] + 
    texture2D(u_image, v_texCoord + onePixel * vec2( 1, 0)) * u_kernel[5] + 
    texture2D(u_image, v_texCoord + onePixel * vec2(-1, 1)) * u_kernel[6] + 
    texture2D(u_image, v_texCoord + onePixel * vec2( 0, 1)) * u_kernel[7] + 
    texture2D(u_image, v_texCoord + onePixel * vec2( 1, 1)) * u_kernel[8] ; 
    // Divide the sum by the weight but just use rgb 
    // we'll set alpha to 1.0 
    gl_FragColor = vec4((colorSum / u_kernelWeight).rgb, 1.0); }

Provides a convolution kernel and its weight:

function computeKernelWeight(kernel) { 
    var weight = kernel.reduce(function(prev, curr) { 
        return prev + curr; 
    }); 
    return weight <= 0 ? 1 : weight; 
} 
... 
var kernelLocation = gl.getUniformLocation(program, "u_kernel[0]"); 
var kernelWeightLocation = gl.getUniformLocation(program, "u_kernelWeight"); 
... 
var edgeDetectKernel = [ -1, -1, -1, -1, 8, -1, -1, -1, -1 ]; 
gl.uniform1fv(kernelLocation, edgeDetectKernel); 
gl.uniform1f(kernelWeightLocation, computeKernelWeight(edgeDetectKernel)); 
...

Real-time rendering

Use two or more textures and rendering effects to alternate rendering, apply one effect at a time, then apply it repeatedly.

Original Image -> [Blur]-> Texture 1 
Texture 1 -> [Sharpen] -> Texture 2 
Texture 2 -> [Edge Detect] -> Texture 1 
Texture 1 -> [Blur]-> Texture 2 
Texture 2 -> [Normal] -> Canvas

Create a frame cache. In WebGL and OpenGL, the frame cache is actually a very informal name. Frame caching in WebGL/OpenGL is actually just a collection of states, not a real cache. However, whenever a texture reaches the frame cache, we render it.

Write the old texture creation code as a function:

function createAndSetupTexture(gl) { 
    var texture = gl.createTexture(); 
    gl.bindTexture(gl.TEXTURE_2D, texture); 
    // Set up texture so we can render any size image and so we are 
    // working with pixels. 
    gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_S, gl.CLAMP_TO_EDGE); 
    gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_T, gl.CLAMP_TO_EDGE); 
    gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MIN_FILTER, gl.NEAREST); 
    gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MAG_FILTER, gl.NEAREST); 
    return texture; 
} 
// Create a texture and put the image in it. 
var originalImageTexture = createAndSetupTexture(gl); 
gl.texImage2D(gl.TEXTURE_2D, 0, gl.RGBA, gl.RGBA, gl.UNSIGNED_BYTE, image);

Use these two functions to generate two problems that are attached to two frame caches:

// create 2 textures and attach them to framebuffers. 
var textures = []; 
var framebuffers = []; 
for (var ii = 0; ii < 2; ++ii) { 
    var texture = createAndSetupTexture(gl); 
    textures.push(texture);
    // make the texture the same size as the image 
    gl.texImage2D( gl.TEXTURE_2D, 0, gl.RGBA, image.width, image.height, 0, gl.RGBA, gl.UNSIGNED_BYTE, null);
    // Create a framebuffer 
    var fbo = gl.createFramebuffer(); 
    framebuffers.push(fbo); 
    gl.bindFramebuffer(gl.FRAMEBUFFER, fbo); 
    // Attach a texture to it. 
    gl.framebufferTexture2D( gl.FRAMEBUFFER, gl.COLOR_ATTACHMENT0, gl.TEXTURE_2D, texture, 0);
 }

Generate a collection of cores and store them in a list to apply:

 var kernels = { 
    normal: [ 0, 0, 0, 0, 1, 0, 0, 0, 0 ], 
    gaussianBlur: [ 0.045, 0.122, 0.045, 0.122, 0.332, 0.122, 0.045, 0.122, 0.045 ], 
    unsharpen: [ -1, -1, -1, -1, 9, -1, -1, -1, -1 ], 
    emboss: [ -2, -1, 0, -1, 1, 1, 0, 1, 2 ]
 }; 
 // List of effects to apply. 
 var effectsToApply = [ "gaussianBlur", "emboss", "gaussianBlur", "unsharpen" ];

Finally, apply each, then alternate rendering:

// start with the original image 
gl.bindTexture(gl.TEXTURE_2D, originalImageTexture); 
// don't y flip images while drawing to the textures
gl.uniform1f(flipYLocation, 1); 
// loop through each effect we want to apply. 
for (var ii = 0; ii < effectsToApply.length; ++ii) {
    // Setup to draw into one of the framebuffers. 
    setFramebuffer(framebuffers[ii % 2], image.width, image.height); 
    drawWithKernel(effectsToApply[ii]); 
    // for the next draw, use the texture we just rendered to. 
    gl.bindTexture(gl.TEXTURE_2D, textures[ii % 2]); }
    // finally draw the result to the canvas. 
    gl.uniform1f(flipYLocation, -1); 
    // need to y flip for canvas 
    setFramebuffer(null, canvas.width, canvas.height); 
    drawWithKernel("normal"); 
    function setFramebuffer(fbo, width, height) { 
    // make this the framebuffer we are rendering to. 
    gl.bindFramebuffer(gl.FRAMEBUFFER, fbo); 
    // Tell the shader the resolution of the framebuffer. 
    gl.uniform2f(resolutionLocation, width, height); 
    // Tell webgl the viewport setting needed for framebuffer. 
    gl.viewport(0, 0, width, height); 
} 
function drawWithKernel(name) { 
    // set the kernel 
    gl.uniform1fv(kernelLocation, kernels[name]); 
    // Draw the rectangle. 
    gl.drawArrays(gl.TRIANGLES, 0, 6); 
}

Calling gl.bindFramebuffer with a null value tells the WebGL program that it wants to render textures on the drawing board instead of in the frame cache. WebGL has to convert the projection matrix to pixels. This is a gl.viewport-based setting. When we initialize WebGL, gl.viewport defaults to the size of the drawing board.

Because we render the frame cache to different sizes, the palette needs to set the appropriate view. Finally, in the original example, we flipped the Y coordinate when rendering was needed. This is because the WebGL displays the panel as 0. 0 means the bottom left coordinate, which is different from the top left coordinate of the 2D image. This is not needed when rendering as a frame cache.

This is because the frame cache is not displayed. It doesn't matter whether part of it is top or bottom. All that matters is pixel 0, which corresponds to 0 in the frame cache. To solve this problem, we can set whether or not to read alternately quickly by adding more input information to the shader.

... 
uniform float u_flipY; 
... 
void main() { 
... 
gl_Position = vec4(clipSpace * vec2(1, u_flipY), 0, 1);
... 
}

When we render, we can set it

var flipYLocation = gl.getUniformLocation(program, "u_flipY"); 
...
// don't flip 
gl.uniform1f(flipYLocation, 1); 
... 
// flip 
gl.uniform1f(flipYLocation, -1);

You may need many GLSL programs if you want to do a complete image processing. A program to adjust hue, saturation and brightness. Another implementation is brightness and contrast. One implements the opposite phase, and the other adjusts the level. You may need to change your code to update GLSL programs and update program-specific parameters.

WebGL 2D Conversion, Rotation, Scaling, Matrix

WebGL 2D Image Conversion

Basic example of image conversion based on Translation 2D:

 // First lets make some variables 
 // to hold the translation of the rectangle 
 var translation = [0, 0]; 
 // then let's make a function to 
 // re-draw everything. We can call this 
 // function after we update the translation.
 // Draw a the scene. 
 function drawScene() {
    // Clear the canvas. 
    gl.clear(gl.COLOR_BUFFER_BIT); 
    // Setup a rectangle 
    setRectangle(gl, translation[0], translation[1], width, height); 
    // Draw the rectangle. 
    gl.drawArrays(gl.TRIANGLES, 0, 6); 
}

The values of translation[0] and translation[1] can be modified by sliding buttons, and the drawScene function is called to update the interface when these two values are modified. Drag the slider bar to move the matrix.

Code used to change setRectangle value:

function setGeometry(gl, x, y) { 
    var width = 100; 
    var height = 150; 
    var thickness = 30; 
    gl.bufferData( gl.ARRAY_BUFFER, 
        new Float32Array([ 
        // left column 
        x, y, 
        x + thickness, y, 
        x, y + height, 
        x, y + height, 
        x + thickness, y, 
        x + thickness, y + height, 
        
        // top rung 
        x + thickness, y, 
        x + width, y, 
        x + thickness, y + thickness, 
        x + thickness, y + thickness, 
        x + width, y, 
        x + width, y + thickness, 
        
        // middle rung 
        x + thickness, y + thickness * 2, 
        x + width * 2 / 3, y + thickness * 2, 
        x + thickness, y + thickness * 3, 
        x + thickness, y + thickness * 3, 
        x + width * 2 / 3, y + thickness * 2, 
        x + width * 2 / 3, y + thickness * 3]), 
        gl.STATIC_DRAW);
}

Renderer section:

attribute vec2 a_position; 
uniform vec2 u_resolution; 
uniform vec2 u_translation; 
void main() { 
    // Add in the translation. 
    vec2 position = a_position + u_translation; 
    // convert the rectangle from pixels to 0.0 to 1.0 
    vec2 zeroToOne = position / u_resolution; ...

Set the geometry once for effect:

function setGeometry(gl) { 
    gl.bufferData( gl.ARRAY_BUFFER, 
    new Float32Array([ 
    // left column 
    0, 0, 30, 0, 0, 150, 0, 150, 30, 0, 30, 150, 
    
    // top rung 
    30, 0, 100, 0, 30, 30, 30, 30, 100, 0, 100, 30, 
    
    // middle rung 
    30, 60, 67, 60, 30, 90, 30, 90, 67, 60, 67, 90]), 
    gl.STATIC_DRAW); 
 }

Update Lower u_ The value of the translation variable:

... 
var translationLocation = gl.getUniformLocation( program, "u_translation"); 
... 
// Set Geometry. 
setGeometry(gl); 
.. 
// Draw scene. 
function drawScene() { 
    // Clear the canvas. 
    gl.clear(gl.COLOR_BUFFER_BIT); 
    // Set the translation. 
    gl.uniform2fv(translationLocation, translation); 
    // Draw the rectangle. 
    gl.drawArrays(gl.TRIANGLES, 0, 18);
}

WebGL 2D Image Rotation

At the highest point on the unit circle, Y is 1 and X is 0. X is 1 and Y is 0 at the rightmost position.

The X and Y values of any point are derived from the unit circle, and they are then multiplied by the geometry in the previous section. Update the renderer as follows:

attribute vec2 a_position; 
uniform vec2 u_resolution; 
uniform vec2 u_translation; 
uniform vec2 u_rotation; 
void main() { 
    // Rotate the position 
    vec2 rotatedPosition = vec2( a_position.x * u_rotation.y + a_position.y * u_rotation.x, a_position.y * u_rotation.y - a_position.x * u_rotation.x); 
    // Add in the translation. 
    vec2 position = rotatedPosition + u_translation;

Passwords:

 var rotationLocation = gl.getUniformLocation(program, "u_rotation"); 
 ... var rotation = [0, 1]; 
 .. 
 // Draw the scene. 
 function drawScene() { 
     // Clear the canvas. 
     gl.clear(gl.COLOR_BUFFER_BIT); 
     // Set the translation. 
     gl.uniform2fv(translationLocation, translation); 
     // Set the rotation. 
     gl.uniform2fv(rotationLocation, rotation); 
     // Draw the rectangle. 
     gl.drawArrays(gl.TRIANGLES, 0, 18); 
 }

First, let's look at the mathematical formula:

rotatedX = a_position.x * u_rotation.y + a_position.y * u_rotation.x; 
rotatedY = a_position.y * u_rotation.y - a_position.x * u_rotation.x;

Suppose you have a rectangle and you want to rotate it. Before you rotate it to the top right corner (3.0, 9.0). Let's first select a point in the unit circle that is clockwise offset by 30 degrees from 12 o'clock.
[External chain picture transfer failed, source station may have anti-theft chain mechanism, it is recommended to save the picture and upload it directly (img-tR7I8RWG-1631442055906)(en-resource://database/4833:1)]
The coordinates of the point at that position on the circle are 0.50 and 0.87:

3.0 * 0.87 + 9.0 * 0.50 = 7.1
9.0 * 0.87 - 3.0 * 0.50 = 6.3

That's exactly where we need it: [External chain picture transfer failed, source station may have anti-theft chain mechanism, we recommend saving the picture and uploading it directly (img-yz6uPW0n-1631442055907)(en-resource://database/4835:1)]
Rotate 60 degrees as above: [External chain picture transfer failed, source station may have anti-theft chain mechanism, it is recommended to save the picture and upload it directly (img-SB8E70Tm-1631442055908)(en-resource://database/4837:1)]
The coordinates of the position above the circle are 0.87 and 0.50:

3.0 * 0.50 + 9.0 * 0.87 = 9.3 
9.0 * 0.50 - 3.0 * 0.87 = 1.9

As we rotate that point clockwise to the right, the value of X becomes larger and the value of Y becomes smaller. If the rotation then exceeds 90 degrees, the value of X becomes smaller again and the value of Y becomes larger. This form can achieve the purpose of rotation.

Those points on the ring have another name. They are called sine and cosine. Therefore, for any given angle, we only need to query its corresponding sine and cosine values:

function printSineAndCosineForAnyAngle(angleInDegrees) { 
    var angleInRadians = angleInDegrees * Math.PI / 180; 
    var s = Math.sin(angleInRadians); 
    var c = Math.cos(angleInRadians); console.log("s = " + s + " c = " + c); 
}

If you copy and paste the above code into the JavaScript console, then type printSineAndCosineForAnyAngle(30), and you will see output s = 0.49 C = 0.87 (Note: This number is approximate.)

If you put the above code together, you can rotate your geometry at any angle you want. Simply pass the angle you need to rotate to sine and cosine.

var angleInRadians = angleInDegrees * Math.PI / 180; 
rotation[0] = Math.sin(angleInRadians); 
rotation[1] = Math.cos(angleInRadians);

WebGL 2D Image Scaling

Scale ratio control:

var scaleLocation = gl.getUniformLocation(program, "u_scale"); 
var scale = [1, 1]; 

// Draw the scene. 
function drawScene() {
    // Clear the canvas. 
    gl.clear(gl.COLOR_BUFFER_BIT); 
    // Set the translation. 
    gl.uniform2fv(translationLocation, translation); 
    // Set the rotation. 
    gl.uniform2fv(rotationLocation, rotation); 
    // Set the scale. 
    gl.uniform2fv(scaleLocation, scale); 
    // Draw the rectangle. 
    gl.drawArrays(gl.TRIANGLES, 0, 18); 
}

WebGL 2D Matrix

Scale transformation graphics. Translation, rotation, and scaling are all types of changes. Each change requires changing the renderer, and they depend on the order of operations. In the previous example, we scaled, rotated, and shifted.

If the order in which they perform the operation changes, they will get different results. For example, the XY scaling transformation is 2,1, rotates 30%, and then shifts 100,0. [External chain picture transfer failed, source station may have anti-theft chain mechanism, it is recommended to save the picture and upload it directly (img-xw90vdCv-1631442055909)(en-resource://database/4839:1)]

The following is a translation of 100, 0, rotation of 30%, followed by a scaling transformation of 2, 1. [External chain picture transfer failed, source station may have anti-theft chain mechanism, it is recommended to save the picture and upload it directly (img-i4UDf5hA-1631442055910)(en-resource://database/4841:1)]

The results are completely different. Worse yet, if we want the effect of the second example, we must write a different renderer that translates, rotates, and scales in the order we want it to be executed. However, some smarter people than I can solve this problem using matrices in mathematics. For 2d graphics, a 3X3 matrix is used. The 3X3 matrix resembles a 9-grid. [External chain picture transfer failed, source station may have anti-theft chain mechanism, it is recommended to save the picture and upload it directly (img-DUKvw2P-1631442055910) (en- resource://database/4843:1 )]

The operation in math is to multiply columns and add them together. A location has two values, expressed as x and y. But three values are needed to do this because we set the third value to 1. In the above example it becomes: [External chain picture transfer failed, the source station may have anti-theft chain mechanism, it is recommended to save the picture and upload it directly (img-TkiWfl1z-1631442055911)(en-resource://database/4845:1)]

For the above treatment, you might think "Where is the reason for this treatment?" Suppose you want to perform a shift transformation. The total number of translations we will want to perform is tx and ty. Construct the following matrix:
[External chain picture transfer failed, source station may have anti-theft chain mechanism, it is recommended to save the picture and upload it directly (img-C1Q1xLL4-1631442055912)(en-resource://database/4847:1)]

Next, calculate:
[External chain picture transfer failed, source station may have anti-theft chain mechanism, it is recommended to save the picture and upload it directly (img-Y7ZRFIqk-1631442055912)(en-resource://database/4849:1)]
If you remember algebra, you can have those places where the product results are zero. Multiply 1 equals nothing, so simplify the calculation to see what happens: [External chain picture transfer failed, source station may have anti-theft chain mechanism, it is recommended to save the picture and upload it directly (img-QhSgoSb7-1631442055913)(en-resource://database/4851:1)]
Or, more concisely:

newX = x + tx;
newY = y + ty;

We don't care about extra variables. This process is surprisingly similar to the code we wrote in the pan. Again, let's look at rotation. As mentioned in the Rotation article, when we want to rotate, we only need the sine and cosine values of the angles.

s = Math.sin(angleToRotateInRadians);
c = Math.cos(angleToRotateInRadians);

Construct the following matrix:
[External chain picture transfer failed, source station may have anti-theft chain mechanism, it is recommended to save the picture and upload it directly (img-4QHoUyW-1631442055914) (en- resource://database/4853:1 )]

Perform the above rectangular operation: [Outer chain picture transfer failed, source station may have anti-theft chain mechanism, it is recommended to save the picture and upload it directly (img-9WgqwoFQ-1631442055915)(en-resource://database/4855:1)]

The 0 and 1 results are shown in black blocks. [External chain picture transfer failed, source station may have anti-theft chain mechanism, it is recommended to save the picture and upload it directly (img-Qfjy0J6V-1631442055916)(en-resource://database/4857:1)]

Similarly, the calculation can be simplified:

newX = x * c + y * s;
newY = x * -s + y * c;

The result of the above processing is exactly the same as that of the rotation example. Finally, the scaling transformation. The two scaling transformation factors are called sx and sy. Construct the following matrix: [External chain picture transfer failed, source station may have anti-theft chain mechanism, it is recommended to save the picture and upload it directly (img-9BDdffxr-1631442055917)(en-resource://database/4859:1)]

Matrix operations will result in the following: [External chain picture transfer failed, the source station may have anti-theft chain mechanism, it is recommended to save the picture and upload it directly (img-wedyykTX-1631442055918)(en-resource://database/4861:1)]

Actual needs calculation:
[External chain picture transfer failed, source station may have anti-theft chain mechanism, it is recommended to save the picture and upload it directly (img-pySUkfn6-1631442055918)(en-resource://database/4863:1)]

Simplified to:

newX = x * sx;
newY = y * sy;

The same scaling example as we explained earlier. When you get here, I'm sure you're still thinking, after that? What does it mean? It looks like it just did the same thing we did before. Next is the magic place. It has been shown that we can multiply multiple matrices and perform all the transformations at once.

Assume there is a function   MatxMultiply, which takes two matrices as parameters, multiplies them and returns the product result. To make this clearer, write the following functions to construct a matrix for translation, rotation, and scaling:

function makeTranslation(tx, ty) { 
    return [ 1, 0, 0, 0, 1, 0, tx, ty, 1 ]; 
} 
function makeRotation(angleInRadians) {
    var c = Math.cos(angleInRadians); 
    var s = Math.sin(angleInRadians);
    return [ c,-s, 0, s, c, 0, 0, 0, 1 ]; 
} 
function makeScale(sx, sy) {
    return [ sx, 0, 0, 0, sy, 0, 0, 0, 1 ];
}

Modify the renderer:

attribute vec2 a_position; 
uniform vec2 u_resolution;
uniform mat3 u_matrix; void main() {
    // Multiply the position by the matrix. 
    vec2 position = (u_matrix * vec3(a_position, 1)).xy;

Using a renderer:

function drawScene() { 
    // Clear the canvas. 
    gl.clear(gl.COLOR_BUFFER_BIT);
    // Compute the matrices 
    var translationMatrix = makeTranslation(translation[0], translation[1]); 
    var rotationMatrix = makeRotation(angleInRadians); 
    var scaleMatrix = makeScale(scale[0], scale[1]); 
    // Multiply the matrices. 
    var matrix = matrixMultiply(scaleMatrix, rotationMatrix); 
    matrix = matrixMultiply(matrix, translationMatrix);
    // Set the matrix. 
    gl.uniformMatrix3fv(matrixLocation, false, matrix); 
    // Draw the rectangle. 
    gl.drawArrays(gl.TRIANGLES, 0, 18); 
}

Further simplify the renderer. The following is the complete vertex renderer.

attribute vec2 a_position; 
uniform mat3 u_matrix; void main() {
    // Multiply the position by the matrix. 
    gl_Position = vec4((u_matrix * vec3(a_position, 1)).xy, 0, 1); }
 
// Draw the scene. 
function drawScene() { 
    ...
    // Compute the matrices 
    var projectionMatrix = make2DProjection( 
    canvas.clientWidth, canvas.clientHeight);
    ... 
    // Multiply the matrices. 
    var matrix = matrixMultiply(scaleMatrix, rotationMatrix); 
    matrix = matrixMultiply(matrix, translationMatrix);
    matrix = matrixMultiply(matrix, projectionMatrix);
    ... 
}

WebGL 3D

WebGL Orthogonal 3D

Change vertex shading to handle 3-D:

attribute vec4 a_position; 
uniform mat4 u_matrix; void main() {
    // Multiply the position by the matrix. 
    gl_Position = u_matrix * a_position;
}

Rendering three-dimensional data:

function makeTranslation(tx, ty, tz) { 
    return [ 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1, 0, tx, ty, tz, 1 ];
} 

function makeXRotation(angleInRadians) { 
    var c = Math.cos(angleInRadians);
    var s = Math.sin(angleInRadians); 
    return [ 1, 0, 0, 0, 0, c, s, 0, 0, -s, c, 0, 0, 0, 0, 1 ]; 
}; 

function makeYRotation(angleInRadians) { 
    var c = Math.cos(angleInRadians); 
    var s = Math.sin(angleInRadians); 
    return [ c, 0, -s, 0, 0, 1, 0, 0, s, 0, c, 0, 0, 0, 0, 1 ]; 
}; 

function makeZRotation(angleInRadians) {
    var c =    Math.cos(angleInRadians); 
    var s =    Math.sin(angleInRadians); 
    return [ c, s, 0, 0, -s, c, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1, ];
} 

function makeScale(sx, sy, sz) {
    return [ sx, 0, 0, 0, 0, sy, 0, 0, 0, 0, sz, 0, 0, 0, 0, 1, ]; 
}

There are three rotation functions. Only one rotation function is needed in two dimensions, since only rotation around the Z-axis is required. Now, while doing 3D, you want to be able to rotate around the X and Y axes. You can see from this that they are very similar. If you let them work, you'll see that they're as simplified as they were before:
Z Rotate newX = x * c + y * s;

WebGL 3D Perspective

What is perspective? It is basically a feature of things that appear smaller as they get farther away.
[External chain picture transfer failed, source station may have anti-theft chain mechanism, it is recommended to save the picture and upload it directly (img-0cTsVZNf-1631442055919)(en-resource://database/4881:1)]

Looking at the example above, we see that the farther away things are drawn, the smaller they are. One simple way to make distant objects appear smaller given our current sample is to divide clipspace X and Y by Z. Think like this: If you have a segment from (10, 15) to (20, 15), it's 10 units long. In our current sample, it will draw 10 pixels long. But if we divide Z, for example, if Z is 1

10 / 1 = 10 
20 / 1 = 20 
abs(10-20) = 10

This will be 10 pixels, if Z is 2, then there will be

10 / 2 = 5 
20 / 2 = 10
abs(5 - 10) = 5

5 pixels long. If Z = 3, then there is

10 / 3 = 3.333
20 / 3 = 6.666 
abs(3.333 - 6.666) = 3.333

You can see that as Z increases and it gets farther and farther, we will eventually draw it smaller. If we divide in clipspace, we may get better results because Z will be a smaller number (-1 to+1). If we add a fudgeFactor multiplied by Z before dividing, we can adjust how small things are for a given distance. Let's try. First let's change the vertex shader by dividing Z after multiplying by our "fudgefactor".

... 
uniform float u_fudgeFactor;
...
void main() { 
    // Multiply the position by the matrix.
    vec4 position = u_matrix * a_position; 
    // Adjust the z to divide by 
    float zToDivideBy = 1.0 + position.z * u_fudgeFactor;
    // Divide x and y by z. 
    gl_Position = vec4(position.xy / zToDivideBy, position.zw);
}

Because in clipspace Z is from -1 to + 1, I add 1 to get zToDivideBy from 0 to + 2 * fudgeFactor We also need to update the code so let's set the fudgeFactor.

 ... 
 var fudgeLocation = gl.getUniformLocation(program, "u_fudgeFactor")
... 
var fudgeFactor = 1; 
... 
function drawScene() { 
    ...
    // Set the fudgeFactor 
    gl.uniform1f(fudgeLocation, fudgeFactor); 
    // Draw the geometry. 
    gl.drawArrays(gl.TRIANGLES, 0, 16 * 6);
}

Here are the results. If we don't explicitly change the "fudgefactor" from 1 to 0 to see what things look like before we divide by Z.
[External chain picture transfer failed, source station may have anti-theft chain mechanism, it is recommended to save the picture and upload it directly (img-F6y6H2S8-1631442055920)(en-resource://database/4883:1)]
WebGL assigns X, Y, Z, W values to our vertex shader   gl_Position   And automatically divided by W. We can prove that this is easy to achieve by changing the coloring rather than dividing by ourselves.   gl_Position.w   Central plus   zToDivideBy  .

... 
uniform float u_fudgeFactor; 
... void main() {
    // Multiply the position by the matrix. 
    vec4 position = u_matrix * a_position;
    // Adjust the z to divide by 
    float zToDivideBy = 1.0 + position.z * u_fudgeFactor; 
    // Divide x, y and z by zToDivideBy 
    gl_Position = vec4(position.xyz, zToDivideBy); 
}

Look at this exactly the same. Why is there the fact that WebGL is automatically divided by w? Because now, with more dimensional matrices, we can use another matrix to copy z to w. The matrix is as follows

1, 0, 0, 0, 
0, 1, 0, 0, 
0, 0, 1, 1, 
0, 0, 0, 0,

Copy z to w. You can see the following columns

x_out = x_in * 1 + 
           y_in * 0 + 
           z_in * 0 + 
           w_in * 0 ; 
y_out = x_in * 0 +
           y_in * 1 +
           z_in * 0 + 
           w_in * 0 ;
z_out = x_in * 0 +
           y_in * 0 +
           z_in * 1 + 
           w_in * 0 ; 
w_out = x_in * 0 +
            y_in * 0 +
            z_in * 1 +
            w_in * 0 ;

Simplified as follows

x_out = x_in;
y_out = y_in; 
z_out = z_in; 
w_out = z_in;

We can add 1 to the matrix we used before, because we know w_in is always 1.0.

1, 0, 0, 0,
0, 1, 0, 0, 
0, 0, 1, 1, 
0, 0, 0, 1,

This will change the W calculation as follows

w_out = x_in * 0 +
            y_in * 0 +
            z_in * 1 +
            w_in * 1 ;

Because we know w_in = 1.0 so there is

w_out = z_in + 1;

Finally, we can add the fudgeFactor to the matrix, which is as follows

1, 0, 0, 0,
0, 1, 0, 0, 
0, 0, 1, fudgeFactor,
0, 0, 0, 1,

This means

w_out = x_in * 0 +
            y_in * 0 +
            z_in * fudgeFactor
            + w_in * 1 ;

Simplified as follows

w_out = z_in * fudgeFactor + 1;

Let's modify the program again to use only matrices. *
First let's put back the vertex shader. It's simple

uniform mat4 u_matrix;

void main() {
  // Multiply the position by the matrix.
  gl_Position = u_matrix * a_position;
  ...
}

Next let's do a function to make the Z - > W matrix.

function makeZToWMatrix(fudgeFactor) {
    return [ 
        1, 0, 0, 0, 
        0, 1, 0, 0, 
        0, 0, 1, fudgeFactor,
        0, 0, 0, 1,
    ];
}

We'll change the code to use it

... 
// Compute the matrices 
var zToWMatrix = makeZToWMatrix(fudgeFactor); 
... 
// Multiply the matrices. 
var matrix = matrixMultiply(scaleMatrix, rotationZMatrix); 
matrix = matrixMultiply(matrix, rotationYMatrix); 
matrix = matrixMultiply(matrix, rotationXMatrix);
matrix = matrixMultiply(matrix, translationMatrix); 
matrix = matrixMultiply(matrix, projectionMatrix); 
matrix = matrixMultiply(matrix, zToWMatrix);
...

Notice that this time it's exactly the same. Basically, this shows you that dividing by Z gives us a perspective, and WebGL divides by Z conveniently. *
But there are still some problems. For example, if you set Z to around -100, you will see an animation similar to the one below. [External chain picture transfer failed, source station may have anti-theft chain mechanism, it is recommended to save the picture and upload it directly (img-BgFAtBPv-1631442055921)(en-resource://database/4885:1)]
What is going on? Why did F disappear so early?
Just like WebGL clips X and Y or + 1 to - 1 it also clips Z. This is where you see Z < 1. I can learn more about how to solve it, but you can get it in the same way that we do a two-dimensional projection.

We need to use Z, add some quantities and measure some quantities, we can do any mapping range from -1 to 1 we want. The really cool thing is that all these steps can be done in a matrix.

Even better, let's decide one   fieldOfView   Not one   fudgeFactor, and calculate the correct value to do this. Here is a function to generate the matrix.

function makePerspective(fieldOfViewInRadians, aspect, near, far) { 
    var f = Math.tan(Math.PI * 0.5 - 0.5 * fieldOfViewInRadians); 
    var rangeInv = 1.0 / (near - far);
    return [ 
        f / aspect, 0, 0, 0,
        0, f, 0, 0, 
        0, 0, (near + far) * rangeInv, -1, 
        0, 0, near * far * rangeInv * 2, 0
    ];
};

This matrix will do all the conversion for us. It will adjust the units, so they will do the math in clipspace, so we can choose the field of view by angle, it will let us choose our z-clipping space.

It assumes that there is an eye or camera at the origin (0, 0, 0) and that given a zNear and fieldOfView, it calculates what it needs, so objects in zNear end at z = - 1 and objects in zNear end at half of the fieldOfView above or below the center, at y = - 1 and y = 1, respectively.

X is calculated by multiplying the incoming aspect only. We usually set this to width / height in the display area. Finally, it calculates the size of the object in the Z region, so the object in the zFar ends at z = 1. Below is a diagram of the action matrix. The rotation of a cube shaped like a four-sided cone is called a truncated cone.

The matrix takes up space within the truncated cone and is converted to clipspace. ZNear defines the object clipped to the front, and zfar defines the object clipped to the back. Set zNear to 23 and you will see the front of the rotating cube clipped. Set zFar to 24 and you will see the clip behind the cube. There is only one question left. This matrix assumes an angle of view of 0, 0, 0 and that it is in the negative Z-axis and the positive Y-axis.

Our matrix has so far solved problems in different ways. In order for it to work, we need our object in the front view. We can do this by moving our F. We are drawing (45, 150, 0). Let's move it to (0, 150, - 360) Now, to use it, we just need to replace the old call to make2DProjection with a call to makePerspective

 var aspect = canvas.clientWidth / canvas.clientHeight; 
 var projectionMatrix = makePerspective(fieldOfViewRadians, aspect, 1, 2000);
 var translationMatrix = makeTranslation(translation[0], translation[1], translation[2]);
 var rotationXMatrix = makeXRotation(rotation[0]);
 var rotationYMatrix = makeYRotation(rotation[1]); 
 var rotationZMatrix = makeZRotation(rotation[2]);
 var scaleMatrix = makeScale(scale[0], scale[1], scale[2]);

So let's go back to a matrix multiplication, and we get a view of two domains where we can choose our z-space.

WebGL 3D Camera

We need to effectively move real objects in front of the camera. The easiest way to do this is to use the inverse matrix. Generally, the calculation of the inverse matrix is complex, but conceptually it is easy. The inverse is the value you use as the opposite of other values. For example, 123 is the inverse of -123. The inverse of the scale matrix with a scaling ratio of 5 is 1/5 or 0.2. The inverse of a matrix rotated by 30 degrees in the X domain is a matrix rotated by -30 degrees in the X domain. Until now we've used translation, rotation, and scaling to influence the position and direction of our'F'.

After multiplying all the matrices, we have a single matrix showing how to move "F" from the origin to the corresponding position in the size and direction we want. We can do the same thing with the camera. Once our matrix tells us how to move and rotate the camera from its origin to where we want it to be, we can calculate its inverse. It will give us a matrix telling us how to move and rotate the relative number of all other objects, which will effectively keep the camera at point (0, 0, 0), and we have moved everything in front of it. Let's make a three-dimensional scene with a circle of'F', just like the chart above. The following is the implementation code.

var numFs = 5; 
var radius = 200; 
var aspect = canvas.clientWidth / canvas.clientHeight; 
var projectionMatrix = makePerspective(fieldOfViewRadians, aspect, 1, 2000); // Draw 'F's in a circle 
for (var ii = 0; ii < numFs; ++ii) {
    var angle = ii * Math.PI * 2 / numFs; 
    var x = Math.cos(angle) * radius;
    var z = Math.sin(angle) * radius; 
    var translationMatrix = makeTranslation(x, 0, z); 
    var matrix = translationMatrix; 
    matrix = matrixMultiply(matrix, projectionMatrix);
    // Set the matrix. 
    gl.uniformMatrix4fv(matrixLocation, false, matrix); 
    // Draw the geometry. 
    gl.drawArrays(gl.TRIANGLES, 0, 16 * 6);
}

Just after we have calculated our projection matrix, we can calculate a camera that rotates around the'F'as shown in the chart above.

var cameraMatrix = makeTranslation(0, 0, radius * 1.5); 
cameraMatrix = matrixMultiply( cameraMatrix, makeYRotation(cameraAngleRadians));

Then, we calculate the View Matrix from the camera matrix. The View Matrix moves all objects to the opposite position of the camera, effectively making the camera look like it is at the origin (0,0,0) relative to all objects.

 var viewMatrix = makeInverse(cameraMatrix);

Finally, we need to apply the view matrix to calculate the matrix for each'F'

 var matrix = translationMatrix;
 matrix = matrixMultiply(matrix, viewMatrix);
 // <=-- added 
 matrix = matrixMultiply(matrix, projectionMatrix);

A camera can circle "F". Drag the cameraAngle slider to move the camera. It's all good, but it's not always easy to use rotation and panning to move a camera to where you want it to be and point it at where you want it to be.

For example, if we want the camera to always point to a specific''F', we have to do some very complex mathematical calculations to determine how to rotate the camera to point to that''F''when it rotates around it. Fortunately, there is an easier way. We can decide where we want the camera to be and what it is pointing to, then we can calculate the matrix that will put the camera there.

Matrix-based principles work very easily. First, we need to know where we want the camera to be. We'll call it CameraPosition. Then we need to know where we are looking at past or aimed objects. We'll call it target.

If we subtract the target from the CameraPosition, we will get a vector that points in the direction where we get the target from the camera. Let's call it zAxis. Because we know that the camera is pointing in the -Z direction, we can subtract the cameraPosition - target from the other direction. We normalize the results and copy them directly to the z-region matrix.

This part of the matrix represents the Z-axis. In this case, it is the Z-axis of the camera. The normalization of a vector means that it represents 1.0. If you go back to the article on two-dimensional rotation, where we talked about how to rotate with a unit circle and two-dimensional rotation, in three-dimensional we need a unit sphere and a normalized vector to represent a point on the unit sphere. [External chain picture transfer failed, source station may have anti-theft chain mechanism, it is recommended to save the picture and upload it directly (img-5g Rsyymh-1631442055922) (en- resource://database/4887:1 )]

Although there is not enough information. Just a single vector gives us a point within the unit range, but from this point to something east? We need to fill in the rest of the matrix. Special X- and Y-axis parts. We know that these three parts are perpendicular to each other. We also know that "normal" we don't point the camera at.

Because, if we know which direction is up, in this case (0,1,0), we can use a type called "cross-product and cross-product" "Calculate the X-axis and Y-axis matrices. I don't know what a cross-product means mathematically. All I know is that if you have two unit vectors and the cross-product you calculated, you get a vector that is perpendicular to those two vectors."

In other words, if you have a vector pointing southeast, and a vector pointing upward, and you calculate the cross product, you will get a vector pointing northwest or northeast from these two vectors, purpendicular to Southeast Asia and. Depending on the order in which you calculate the cross product, you will get the opposite answer. [External chain picture transfer failed, source station may have anti-theft chain mechanism, it is recommended to save the picture and upload it directly (img-XH18mwDm-1631442055923)(en-resource://database/4889:1)]

Now we have   xAxis, we can use   zAxis   and   xAxis   Get the camera's   yAxis [External chain picture transfer failed, source station may have anti-theft chain mechanism, it is recommended to save the picture and upload it directly (img-8BkM1BYp-1631442055924)(en-resource://database/4891:1)]

Now all we have to do is insert three axes into a matrix. This allows the matrix to point to the object from   cameraPosition   point   target. We just need to add   position
[External chain picture transfer failed, source station may have anti-theft chain mechanism, it is recommended to save the picture and upload it directly (img-ZcmRLpjD-1631442055925)(en-resource://database/4893:1)]

The following is the code used to calculate the cross product of two vectors.

function cross(a, b) { 
    return [a[1] * b[2] - a[2] * b[1], a[2] * b[0] - a[0] * b[2], a[0] * b[1] - a[1] * b[0]]; 
}

This is the code that subtracts two vectors.

function subtractVectors(a, b) {
    return [a[0] - b[0], a[1] - b[1], a[2] - b[2]];
}

Here is the code that normalizes a vector (making it a unit vector).

function normalize(v) {
    var length = Math.sqrt(v[0] * v[0] + v[1] * v[1] + v[2] * v[2]); 
    // make sure we don't divide by 0. 
    if (length > 0.00001) { 
        return [v[0] / length, v[1] / length, v[2] / length]; 
    } else { 
        return [0, 0, 0]; 
    } 
}

Here is the code for calculating a "lookAt" matrix.

function makeLookAt(cameraPosition, target, up) {
    var zAxis = normalize( subtractVectors(cameraPosition, target)); 
    var xAxis = cross(up, zAxis); 
    var yAxis = cross(zAxis, xAxis); 
    return [ xAxis[0], xAxis[1], xAxis[2], 0, yAxis[0], yAxis[1], yAxis[2], 0, zAxis[0], zAxis[1], zAxis[2], 0, cameraPosition[0], cameraPosition[1], cameraPosition[2], 1];
}

This is how we use it to point the camera at a specific'F'as we move it.

 ...
 // Compute the position of the first F 
 var fPosition = [radius, 0, 0]; 
 // Use matrix math to compute a position on the circle. 
 var cameraMatrix = makeTranslation(0, 50, radius * 1.5); 
 cameraMatrix = matrixMultiply( cameraMatrix, makeYRotation(cameraAngleRadians)); 
 // Get the camera's postion from the matrix we computed 
 cameraPosition = [ cameraMatrix[12], cameraMatrix[13], cameraMatrix[14]]; var up = [0, 1, 0];
 // Compute the camera's matrix using look at. 
 var cameraMatrix = makeLookAt(cameraPosition, fPosition, up); 
 // Make a view matrix from the camera matrix. 
 var viewMatrix = makeInverse(cameraMatrix);
 ...

Drag the slider and notice that the camera tracks an'F'. Note that you can use the "lookAt" function for more than just cameras. The common purpose is to have a person's head follow someone. Aim the tower at a target. Make the object follow a path. You calculate the path of the target. Then you calculate where your goal will be on the route in the next few minutes. Put these two values in your lookAt function, and you get a matrix that lets your object follow and follow the path.