How to Use WebGL to Accelerate Web Barcode and QR Code Decoding Speed

While scanning barcode and QR code from camera stream, we need to firstly convert color images encoded in RGB or YUV to grayscale images, then convert grayscale images to binary images before decoding. Operating image pixels takes much CPU time. To speed up image processing, we can utilize GPU. Dynamsoft JavaScript barcode SDK features WebGL acceleration for barcode and QR code scanning. In this article, I will demystify how to use WebGL to reduce the total CPU time when reading barcode and QR code from camera video stream in web apps.

This article is Part 1 in a 2-Part Series.

License Activation

Get a trial license to activate Dynamsoft JavaScript barcode SDK:

Dynamsoft.DBR.BarcodeReader.license = "DLS2eyJoYW5kc2hha2VDb2RlIjoiMjAwMDAxLTE2NDk4Mjk3OTI2MzUiLCJvcmdhbml6YXRpb25JRCI6IjIwMDAwMSIsInNlc3Npb25QYXNzd29yZCI6IndTcGR6Vm05WDJrcEQ5YUoifQ==";

Installation

Dynamsoft JavaScript Barcode SDK is available on npmjs.com:

https://www.npmjs.com/package/dynamsoft-javascript-barcode

Decoding Barcode and QR Code from HTML5 Canvas

The following code shows how to render video stream to a HTML canvas and invoke the decodeBuffer() method of BarcodeReader class to decode barcode or QR code from the canvas. The pixel format of the canvas is RGBA.

var barcodereader = null;
(async()=>{
    barcodereader = await Dynamsoft.BarcodeReader.createInstance();
    await barcodereader.updateRuntimeSettings('speed');
    let settings = await barcodereader.getRuntimeSettings();
    settings.deblurLevel = 0;
    barcodereader.updateRuntimeSettings(settings);
})();

let canvas2d = document.createElement('canvas');
canvas2d.width = width;
canvas2d.height = height;
var ctx2d = canvas2d.getContext('2d');
ctx2d.drawImage(videoElement, 0, 0, width, height);
buffer = ctx2d.getImageData(0, 0, width, height).data;

if (barcodereader){
barcodereader
        .decodeBuffer(
            buffer,
            width,
            height,
            width * 4,
            Dynamsoft.EnumImagePixelFormat.IPF_ARGB_8888
        )
        .then((results) => {
            showResults(results);
        });
}

Converting Color Image to Grayscale Image with WebGL

Theoretically, the speed of processing grayscale images is faster than that of processing color images, because a color image has 4 channels, whereas a grayscale image has only 1 channel. If the input data source is a grayscale image, the above code can be changed as follows:

barcodereader
        .decodeBuffer(
            gray,
            width,
            height,
            width,
            Dynamsoft.EnumImagePixelFormat.IPF_GrayScaled
        )
        .then((results) => {
            showResults(results);
        });

The next question is how we can use WebGL to convert a color image to a grayscale image? A website named WebGLFundamentals provides crash course for WebGL.

By learning WebGL, we use the following steps for image pre-processing:

  1. Create a shader program for color conversion:

     <!-- https://gist.github.com/Volcanoscar/4a9500d240497d3c0228f663593d167a -->
   <script id="drawImage-fragment-shader" type="x-shader/x-fragment">
   precision mediump float;
      
   varying vec2 v_texcoord;
      
   uniform sampler2D u_texture;
   uniform float u_colorFactor;
      
   void main() {
     vec4 sample =  texture2D(u_texture, v_texcoord);
     float grey = 0.21 * sample.r + 0.71 * sample.g + 0.07 * sample.b;
     gl_FragColor = vec4(sample.r * u_colorFactor + grey * (1.0 - u_colorFactor), sample.g * u_colorFactor + grey * (1.0 - u_colorFactor), sample.b * u_colorFactor + grey * (1.0 - u_colorFactor), 1.0);
   }
   </script>

  2. Call the draw() function to bind video element to a WebGL texture:

     var drawInfo = {
             x: 0,
             y: 0,
             dx: 1,
             dy: 1,
             textureInfo: loadImageAndCreateTextureInfo(videoElement)
           };
            
         draw(drawInfo);

  3. Read the image data to a Uint8Array:

     buffer = new Uint8Array(width * height * 4);
 gl.readPixels(
             0,
             0,
             gl.drawingBufferWidth,
             gl.drawingBufferHeight,
             gl.RGBA,
             gl.UNSIGNED_BYTE,
             buffer
         );

    An interesting thing is the output image is upside down. To flip the image data along its vertical axis, you can call gl.pixelStorei():

     function drawImage(tex, texWidth, texHeight, dstX, dstY) {
   gl.bindTexture(gl.TEXTURE_2D, tex);
   gl.pixelStorei(gl.UNPACK_FLIP_Y_WEBGL, true);

  4. Extract the grayscale data from the buffer:

     gray = new Uint8Array(width * height);
 let gray_index = 0;
 for (i = 0; i < width * height * 4; i += 4) {
     gray[gray_index++] = buffer[i];
 }

Here is the screenshot of running the complete code:

WebGL JavaScript Barcode

Barcode Decoding Performance Comparison: Grayscale Image vs Color Image

To measure the performance difference, we can use the formula:

total time = image data obtaining time + barcode decoding time

I made a simple test for 640x480 video stream.

web barcode decoding: grayscale vs color

As you can see, the time cost of getting image data from WebGL (grayscale image) is approximately 5x slower than that from canvas (color image) on my PC, for GPU data transfer takes time. However, the total CPU time cost of using WebGL for barcode decoding is less.

Source Code

https://github.com/yushulx/WebGL-JavaScript-Barcode-QrCode