How to Use WebGL to Accelerate Web Barcode and QR Code Decoding Speed

While scanning barcodes and QR codes from a camera stream, it is necessary to first convert color images encoded in RGB or YUV to grayscale images, and then convert these grayscale images to binary images before decoding. Operating on image pixels consumes considerable CPU time. To speed up image processing, we can utilize the GPU. The Dynamsoft JavaScript barcode SDK features WebGL acceleration for barcode and QR code scanning. In this article, I will explain how to use WebGL to reduce the total CPU time when reading barcodes and QR codes from camera video streams in web apps.

This article is Part 2 in a 3-Part Series.

Online Demo

https://yushulx.me/javascript-barcode-qr-code-scanner/examples/9.x/webgl/

License Activation

Obtain a trial license to activate the Dynamsoft JavaScript Barcode SDK:

Dynamsoft.DBR.BarcodeReader.license = "LICENSE-KEY";

Installation

The Dynamsoft JavaScript Barcode SDK is available on npmjs.com:

https://www.npmjs.com/package/dynamsoft-javascript-barcode

Decoding Barcodes and QR Codes from HTML5 Canvas

The following code demonstrates how to render a video stream to an HTML canvas and invoke the decodeBuffer() method of the BarcodeReader class to decode barcodes or QR codes from the canvas. The pixel format of the canvas is RGBA.

var barcodereader = null;
(async()=>{
    barcodereader = await Dynamsoft.BarcodeReader.createInstance();
    await barcodereader.updateRuntimeSettings('speed');
    let settings = await barcodereader.getRuntimeSettings();
    settings.deblurLevel = 0;
    barcodereader.updateRuntimeSettings(settings);
})();

let canvas2d = document.createElement('canvas');
canvas2d.width = width;
canvas2d.height = height;
var ctx2d = canvas2d.getContext('2d');
ctx2d.drawImage(videoElement, 0, 0, width, height);
buffer = ctx2d.getImageData(0, 0, width, height).data;

if (barcodereader){
barcodereader
        .decodeBuffer(
            buffer,
            width,
            height,
            width * 4,
            Dynamsoft.EnumImagePixelFormat.IPF_ARGB_8888
        )
        .then((results) => {
            showResults(results);
        });
}

Converting a Color Image to a Grayscale Image with WebGL

Theoretically, processing grayscale images is faster than processing color images because a color image has four channels, whereas a grayscale image has only one. If the input data source is a grayscale image, the above code can be modified accordingly:

barcodereader
        .decodeBuffer(
            gray,
            width,
            height,
            width,
            Dynamsoft.EnumImagePixelFormat.IPF_GrayScaled
        )
        .then((results) => {
            showResults(results);
        });

The next question is: How can we use WebGL to convert a color image to a grayscale image? A website called WebGLFundamentals offers a crash course on WebGL.

By learning WebGL, we use the following steps for image pre-processing:

  1. Create a shader program for color conversion:

     <!-- https://gist.github.com/Volcanoscar/4a9500d240497d3c0228f663593d167a -->
   <script id="drawImage-fragment-shader" type="x-shader/x-fragment">
   precision mediump float;
      
   varying vec2 v_texcoord;
      
   uniform sampler2D u_texture;
   uniform float u_colorFactor;
      
   void main() {
     vec4 sample =  texture2D(u_texture, v_texcoord);
     float grey = 0.21 * sample.r + 0.71 * sample.g + 0.07 * sample.b;
     gl_FragColor = vec4(sample.r * u_colorFactor + grey * (1.0 - u_colorFactor), sample.g * u_colorFactor + grey * (1.0 - u_colorFactor), sample.b * u_colorFactor + grey * (1.0 - u_colorFactor), 1.0);
   }
   </script>

  2. Call the draw() function to bind the video element to a WebGL texture:

     var drawInfo = {
             x: 0,
             y: 0,
             dx: 1,
             dy: 1,
             textureInfo: loadImageAndCreateTextureInfo(videoElement)
           };
            
         draw(drawInfo);

  3. Read the image data into a Uint8Array:

     buffer = new Uint8Array(width * height * 4);
 gl.readPixels(
             0,
             0,
             gl.drawingBufferWidth,
             gl.drawingBufferHeight,
             gl.RGBA,
             gl.UNSIGNED_BYTE,
             buffer
         );

    An interesting point is that the output image is upside down. To flip the image data along its vertical axis, you can use gl.pixelStorei():

     function drawImage(tex, texWidth, texHeight, dstX, dstY) {
   gl.bindTexture(gl.TEXTURE_2D, tex);
   gl.pixelStorei(gl.UNPACK_FLIP_Y_WEBGL, true);

  4. Extract the grayscale data from the buffer:

     gray = new Uint8Array(width * height);
 let gray_index = 0;
 for (i = 0; i < width * height * 4; i += 4) {
     gray[gray_index++] = buffer[i];
 }

Here is the screenshot of the running complete code:

WebGL JavaScript Barcode

Barcode Decoding Performance Comparison: Grayscale Image vs Color Image

To measure the performance difference, use the formula:

total time = image data obtaining time + barcode decoding time

I conducted a simple test with a 640x480 video stream.

web barcode decoding: grayscale vs color

As you can see, the time cost of retrieving image data from WebGL (grayscale image) is approximately 5x slower than that from a canvas (color image) on my PC, as GPU data transfer takes time. However, the total CPU time cost of using WebGL for barcode decoding is less.

Source Code

https://github.com/yushulx/javascript-barcode-qr-code-scanner/tree/main/examples/9.x/webgl