web technology sharing | AudioContext to realize audio visualization

To realize audio visualization and achieve some cool effects, we need to use some methods AudioContext provided by Web Audio API.

  • The AudioContext interface represents the audio processing diagram constructed by linked audio modules, and each module is represented by an AudioNode. The audio context controls the creation of the nodes it contains and the execution of audio processing or decoding.

  • Before doing anything else, you need to create an AudioContext object because everything happens in the context. It is recommended to create an AudioContext object and reuse it instead of initializing a new AudioContext object at a time, and one AudioContext object can be used for multiple different audio sources and pipelines at the same time.

Prepare audio and canvas Tags

<audio class="audio" ref='audio' :src='tune' @ended='ended'></audio>
<canvas ref='canvas' width='300px' height='300px'></canvas>

Create and return a new AudioContext object.

const ctx = new AudioContext();

Create a new MediaElementAudioSourceNode object

  • The createMediaElementSource() method of the AudioContext interface is used to create a new MediaElementAudioSourceNode object. Enter an existing HTMLor 'element, and the corresponding audio can be played or modified
const audioSrc = ctx.createMediaElementSource(this.audioElement);

Create AnalyserNode object

  • The createanalyzer () method of AudioContext can create an AnalyserNode, which can be used to obtain audio time and frequency data and realize data visualization.
const analyser = ctx.createAnalyser();

Set the fftSize property

  • The value of the FFT size property of the AnalyserNode interface is an unsigned long integer that represents the window size of the (signal) sample. When fast Fourier transform (FFT) is performed, these (signal) samples are used to obtain frequency domain data.

  • The value of the fftSize property must be a non-zero power of 2 in the range from 32 to 32768; The default value is 2048.

analyser.fftSize = 512;

audioSrc and analyzer are linked

The connect() method of the AudioNode interface enables you to connect the output of a node to a specified target, which may be another AudioNode (so as to guide the audio data to the next specified node) or an AudioParam, so that the output data of the previous node can automatically change the next parameter value over time.


Link between analyzer and ctx.destination

  • The destination attribute of AudioContext returns an AudioDestinationNode, which represents the final target node of all audio (nodes) in the context, generally audio rendering devices, such as speakers.

Song playing in progress

  • The Uint8Array array type represents an 8-bit unsigned integer array. When it is created, the content is initialized to 0. After creation, the elements in the array can be referenced as objects or by using the array index.
  • The getByteFrequencyData() method of the AnalyserNode interface copies the current frequency data into the incoming Uint8Array (unsigned byte array).
  • If the length of the array is less than AnalyserNode.frequencyBinCount, the extra elements of the analyzer will be deleted. If it is greater than, the extra elements of the array will be ignored
visualization() {
    const arr = new Uint8Array(this.analyser.frequencyBinCount);

Drawing audio map with canvas

  • HTMLCanvasElement.getContext() method returns canvas If the context is not defined, the null .
  • 2D create a canvas renderingcontext2d 2d rendering context.
this.canvas = this.$refs.canvas.getContext('2d');draw(arr) {    canvas.clearRect(0, 0, document.body.clientWidth, this.canvasHeight);    const start = ((document.body.clientWidth / 2) - ((arr.length / 2) * 3));    arr.forEach((item, index) => {        this.canvas.beginPath();        this.canvas.strokeStyle = '#B2AFF4';        this.canvas.lineWidth = 3;        this.canvas.moveTo(start + (index * 4), this.canvasHeight);        this.canvas.lineTo(start + (index * 4), this.canvasHeight - item / 2);        this.canvas.stroke();    });}

Effect display

Keywords: Visualization webrtc

Added by livepjam on Fri, 03 Dec 2021 15:29:57 +0200