QR Code Recognition - Inverted Color, Perspective Distortion, and Grayscale

When using a QR code reader or scanner SDK, we primarily focus on the detection speed and recognition accuracy. There are many factors that may affect the performance of the Barcode detection SDK. The factors can be approximately classified into two categories: image quality and algorithm performance. In this article, we take Dynamsoft Barcode Reader Python edition as the example to discuss how to optimize the performance of the Barcode detection SDK by preprocessing input QR code images and adjusting the algorithm parameters on desktop platforms.

Prerequisites

  • OpenCV Python

      pip install opencv-python
      pip install opencv-contrib-python
    
  • Dynamsoft Barcode Reader SDK

      pip install dbr
    
  • Download some QR code images by Google Image Search

QR Codes With Inverted Color

If we try to read the following QR code image printed with inverted color, it will fail to do the recognition.

inverted color QR code

After converting the image to binary image, we can see the black and white are inverted for QR code:

recognize inverted binary image

Using OpenCV to Invert the Color

To make the QR code recognizable, we use OpenCV function bitwise_not:

ret, thresh = cv2.threshold(grayscale_image, 150,255,cv2.THRESH_BINARY )
cv2.bitwise_not(thresh, thresh) 
detect('Inverted', thresh, dbr.EnumImagePixelFormat.IPF_GRAYSCALED)
Time taken: 0.0541 seconds
Barcode Format : 
QR_CODE
Barcode Text :
https://www.jalna.com.au/

Using Dynamsoft’s Built-in API to Invert the Color

With Dynamsoft’s QR code SDK, for the inverted QR code, the setting is as follows:

inverted QR code

The performance is even better when reading the inverted QR code in Python.

inverted QR code

Time taken: 0.0160 seconds
Barcode Format : 
QR_CODE
Barcode Text :
https://www.jalna.com.au/

QR Codes With Perspective Distortion

Let’s take a look at some complicated cases. Here is a perspective distorted QR code image.

perspective distorted QR code image

If we use the code above, there is no barcode recognized, no matter color image, grayscale image or binary image. However, it does not mean the barcode image cannot be recognized.

Adjust the Threshold Value with OpenCV

I managed to recognize the QR code by manually adjusting and setting the threshold value:

image = cv2.imread('perspective-distortion-qr-code.jpg')
grayscale_image = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
ret, thresh = cv2.threshold(grayscale_image, 125, 255,cv2.THRESH_BINARY) 
detect('Binary', thresh, dbr.EnumImagePixelFormat.IPF_GRAYSCALED)
Time taken: 0.0080 seconds
Barcode Format :
QR_CODE
Barcode Text :
http://goo.by/wb0Ric

recognition for perspective distorted QR

Change Settings in Dynamsoft’s SDK

Dynamsoft’s QR Code SDK is extensively customizable and offers a variety of scanning templates. Two of the mostly used templates are speed and coverage. There is a tradeoff between the barcode decoding speed and recognition accuracy. The more algorithms are enabled for barcode recognition, the more time it takes.

QR code parameter settings

Now, we upload the perspective distorted QR image to see the difference between the best speed and best coverage.

Best speed It takes 15 milliseconds but fails to recognize the QR code.

Best coverage It successfully recognizes the QR code but takes 828 milliseconds.

Best coverage vs. Best speed QR code parameter settings

Adjust the Threshold Value

Both of them are not ideal according to our test in Python code. To guarantee the best performance:

  1. Go to Advanced Settings > Binarization mode > Threshold.
  2. Change the threshold value to 125.

Now the QR code is recognized extremely fast.

Settings Result
Best speed Failed to recognize
Best coverage Success. 828 milliseconds
Threshold set to ‘125’ Success. 8 milliseconds

We can then export the template to a JSON file and load it in our Python code:

error = reader.init_runtime_settings_with_file('perspective-qr.json')
image = cv2.imread('perspective-distortion-qr-code.jpg')
detect('Color', image_copy, dbr.EnumImagePixelFormat.IPF_RGB_888)

QR Images: Color vs. Grayscale vs. Black/White

To evaluate the SDK performance, we use color images, grayscale images, and binary images as the input images.

Air-Force-Reserve-San-Diego

A quick summary:

Image Time Cost Image Thumbnail
Color Image 0.014 s QR Code Color Image
Grayscale Image 0.010 s QR Code grayscale Image
Binary Image 0.009 s QR Code Black and White Image

Decode A Color QR Code Image

Let’s get started with a color image of a QR code.

The following code loads and displays the image using OpenCV and reads barcode information using Dynamsoft Barcode Reader.

import cv2
import numpy as np
import dbr
import time

reader = dbr.BarcodeReader()
reader.init_license("DLS2eyJoYW5kc2hha2VDb2RlIjoiMjAwMDAxLTE2NDk4Mjk3OTI2MzUiLCJvcmdhbml6YXRpb25JRCI6IjIwMDAwMSIsInNlc3Npb25QYXNzd29yZCI6IndTcGR6Vm05WDJrcEQ5YUoifQ==") # https://www.dynamsoft.com/customer/license/trialLicense?product=dbr
image = cv2.imread('Air-Force-Reserve-San-Diego.jpg')

def detect(windowName, image, pixel_format):
    try:
        buffer = image.tobytes()
        height = image.shape[0]
        width = image.shape[1]
        stride = image.strides[0]
        start = time.time()
        results = reader.decode_buffer_manually(buffer, width, height, stride, pixel_format, "")
        end = time.time()
        print("Time taken: {:.3f}".format(end - start) + " seconds")
        cv2.putText(image, "Time taken: {:.3f}".format(end - start) + " seconds", (10, 30), cv2.FONT_HERSHEY_SIMPLEX, 0.5, (0, 0, 255), 2)

        if results != None:
            for result in results:
                print("Barcode Format : ")
                print(result.barcode_format_string)
                print("Barcode Text : ")
                print(result.barcode_text)

                points = result.localization_result.localization_points
                data = np.array([[points[0][0], points[0][1]], [points[1][0], points[1][1]], [points[2][0], points[2][1]], [points[3][0], points[3][1]]])
                cv2.drawContours(image=image, contours=[data], contourIdx=-1, color=(0, 255, 0), thickness=2, lineType=cv2.LINE_AA)

                x = min(points[0][0], points[1][0], points[2][0], points[3][0])
                y = min(points[0][1], points[1][1], points[2][1], points[3][1])
                cv2.putText(image, result.barcode_text, (x, y), cv2.FONT_HERSHEY_SIMPLEX, 0.5, (0, 0, 255), 2)

        cv2.imshow(windowName, image)
    except BarcodeReaderError as bre:
        print(bre)

detect('Color', image, dbr.EnumImagePixelFormat.IPF_RGB_888)
cv2.waitKey(0)

We initialize Dynamsoft Barcode Reader with the license key.

reader = dbr.BarcodeReader()
reader.init_license("DLS2eyJoYW5kc2hha2VDb2RlIjoiMjAwMDAxLTE2NDk4Mjk3OTI2MzUiLCJvcmdhbml6YXRpb25JRCI6IjIwMDAwMSIsInNlc3Npb25QYXNzd29yZCI6IndTcGR6Vm05WDJrcEQ5YUoifQ==")

By default, OpenCV loads images in BGR format. The corresponding enum value of pixel format provided by Dynamsoft Barcode Reader is dbr.EnumImagePixelFormat.IPF_RGB_888.

Then we call the decode_buffer_manually function to decode the image.

buffer = image.tobytes()
height = image.shape[0]
width = image.shape[1]
stride = image.strides[0]
results = reader.decode_buffer_manually(buffer, width, height, stride, dbr.EnumImagePixelFormat.IPF_RGB_888, "")

The function returns a list of BarcodeResult objects. Each BarcodeResult object contains the barcode information and the localization result. The drawContours function and putText function can be used to draw the barcode information and localization result on the image.

points = result.localization_result.localization_points
data = np.array([[points[0][0], points[0][1]], [points[1][0], points[1][1]], [points[2][0], points[2][1]], [points[3][0], points[3][1]]])
cv2.drawContours(image=image, contours=[data], contourIdx=-1, color=(0, 255, 0), thickness=2, lineType=cv2.LINE_AA)

x = min(points[0][0], points[1][0], points[2][0], points[3][0])
y = min(points[0][1], points[1][1], points[2][1], points[3][1])
cv2.putText(image, result.barcode_text, (x, y), cv2.FONT_HERSHEY_SIMPLEX, 0.5, (0, 0, 255), 2)

Decode A Grayscale QR Code Image

Grayscale image only carries intensity information. The barcode recognition API internally converts a color image to a grayscale image. So theoretically, if the input image is a grayscale image, the recognition speed is faster.

We run the following code to see the time cost:

grayscale_image = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
detect('Grayscale', grayscale_image, dbr.EnumImagePixelFormat.IPF_GRAYSCALED)

The pixel format is changed to dbr.EnumImagePixelFormat.IPF_GRAYSCALED accordingly. As expected, decoding the grayscale image converted from the original color image saves a little time.

Decode A QR Code Of Binary Image

Thresholding the grayscale image to binary image can make it easier to analyze. We can also try it out by using the OpenCV function threshold:

# https://docs.opencv.org/4.5.1/d7/d4d/tutorial_py_thresholding.html
blur = cv2.GaussianBlur(grayscale_image,(5,5),0)
ret, thresh = cv2.threshold(blur, 0,255,cv2.THRESH_BINARY + cv2.THRESH_OTSU)
detect('Binary', thresh, dbr.EnumImagePixelFormat.IPF_GRAYSCALED)

We use Gaussian blur to reduce noise and smooth the image, and then use the OpenCV function threshold to convert the grayscale image to binary image. The Otsu’s method is used to automatically determine the optimal global threshold value from the image histogram.

Image Preprocessing: Adjusting the QR Code Scanning Parameters

So far, we have tried some image preprocessing algorithms to read QR code. Actually, Dynamsoft’s barcode SDK contains more image processing and computer vision algorithms than the ones we have tried. To cover all cases, we need to know how to tune the parameters of the barcode SDK.

We use the following Python code to print the runtime parameters in console:

print(reader.get_runtime_settings().__dict__)
{'terminate_phase': 32, 'timeout': 10000, 'max_algorithm_thread_count': 4, 'expected_barcodes_count': 0, 'barcode_format_ids': -31457281, 'barcode_format_ids_2': 0, 'pdf_raster_dpi': 300, 'scale_down_threshold': 2300, 'binarization_modes': [2, 0, 0, 0, 0, 0, 0, 0], 'localization_modes': [2, 16, 4, 8, 0, 0, 0, 0], 'colour_clustering_modes': [0, 0, 0, 0, 0, 0, 0, 0], 
'colour_conversion_modes': [1, 0, 0, 0, 0, 0, 0, 0], 'grayscale_transformation_modes': [2, 0, 0, 0, 0, 0, 0, 0], 'region_predetection_modes': [2, 0, 0, 0, 0, 0, 0, 0], 'image_preprocessing_modes': [2, 0, 0, 0, 0, 0, 0, 0], 'texture_detection_modes': [2, 0, 0, 0, 0, 0, 0, 0], 'text_filter_modes': [2, 0, 0, 0, 0, 0, 0, 0], 'dpm_code_reading_modes': [0, 0, 0, 0, 0, 0, 
0, 0], 'deformation_resisting_modes': [0, 0, 0, 0, 0, 0, 0, 0], 'barcode_complement_modes': [0, 0, 0, 0, 0, 0, 0, 0], 'barcode_colour_modes': [1, 0, 0, 0, 0, 0, 0, 0], 'text_result_order_modes': [1, 2, 4, 0, 0, 0, 0, 0], 'text_assisted_correction_mode': 2, 'deblur_level': 9, 'intermediate_result_types': 0, 'intermediate_result_saving_mode': 1, 'result_coordinate_type': 1, 'return_barcode_zone_clarity': 0, 'region_top': 0, 'region_bottom': 0, 'region_left': 0, 'region_right': 0, 'region_measured_by_percentage': 0, 'min_barcode_text_length': 0, 'min_result_confidence': 0, 'scale_up_modes': [1, 0, 0, 0, 0, 0, 0, 0], 'accompanying_text_recognition_modes': [0, 0, 0, 0, 0, 0, 0, 0], 'pdf_reading_mode': 1, 'deblur_modes': [0, 0, 0, 0, 0, 0, 0, 0, 0, 0], 'barcode_zone_min_distance_to_image_borders': 0}

There are quite a lot of parameters to tune, but you cannot find a crash course on how to tune these parameters. Fortunately, we can leverage the parameter tuning tool of Dynamsoft Barcode Reader online demo to customize parameters for specific images.

QR code parameter settings

There are five modes available for selection. You can click Advanced Settings to expand the parameters of a mode. For better studying the parameter differences among different modes, we can scroll down the page and save all parameters to a json file.

In a nutshell, to get best performance of Dynamsoft Barcode Reader SDK, you should start with the simplest settings and then gradually increase and tune algorithm parameters.

Source Code

https://github.com/yushulx/python-qr-decode