1D Barcode Scanning Accuracy Benchmark and Comparison
Introduction
Accuracy is a key metric of barcode reading SDKs. Given an image, a barcode engine should correctly detect whether one or more barcodes exist. If barcodes exist, the percentage of correct values returned dictates the accuracy.
As a software developer, choosing a barcode SDK with the best combination of speed, accuracy, robustness, and support is a crucial step in building a reliable barcode scanner application. In this article, we evaluate 1D barcode scanning accuracy across four barcode engines using three publicly available datasets, and share techniques for pushing recognition accuracy even further.
This article is Part 2 in a 3-Part Series.
How to Evaluate Barcode Scanning Accuracy
There are two steps to read a barcode:
- Detection (AKA localization) - Upon initiating a scan, a barcode engine scans the entire width of an image trying to identify whether there are barcode candidates. One or more coordinates of quadrilaterals are returned.
- Decode (AKA recognition) - Once the regions of interest are identified, decoding is then initiated. In this step, the barcode scanner tries to unravel the encoded data.
Corresponding to the two steps, some datasets mark up the coordinates of the barcode candidates, while others mark up the barcode encoded values. Both approaches can be used to compare different barcode scanning engines.
This article uses the barcode values to measure barcode scanning accuracy. We will discuss the datasets in the following section.
Barcode Engines
In this comprehensive analysis, we compare Dynamsoft Barcode Reader against one other commercial barcode SDK (Scandit) and two widely used open-source solutions, ZBar and ZXing.
Datasets Used For Comparison
We have compiled a table of existing publicly available barcode datasets.
| Dataset | Artelab Medium Barcode 1D Collection | Muenster BarcodeDB | DEAL Lab’s Barcode Dataset |
|---|---|---|---|
| Symbologies | EAN13 | EAN13, UPC-A | EAN13 |
| Include images with multiple barcodes | No | No | No |
| Number of images | 430 | 1054 | 2000 |
| Image quality | Barcodes are relatively large compared to the image size | Barcodes are relatively large compared to the image size | Resolutions range from 400x217 to 3480x4640 |
| Web page | http://artelab.dista.uninsubria.it/downloads/datasets/barcode/medium_barcode_1d/medium_barcode_1d.html | https://www.uni-muenster.de/PRIA/en/forschung/index.shtml | https://www.resl.kaist.ac.kr/doc/datasets |
Metrics to Evaluate Accuracy
In this article, we use precision and reading rate to evaluate the accuracy.
The precision is the number of correct results divided by the number of all results. A precision of 100% means each of the results is correct, although some barcodes might be missed.
Precision = correct results / all returned results
The reading rate is the number of correct results divided by the number of all barcodes. A reading rate of 100% means that all barcodes were recognized successfully, although incorrect values may still be mixed into the results. For example, if a barcode is mistakenly recognized as two separate ones, this behavior would not lower the reading rate.
Reading rate = correct results / all barcodes
Performance Test Results
We ran performance tests on the three datasets described above. All annotations were manually reviewed and corrected to ensure ground-truth accuracy.
Key Findings
- Dynamsoft Barcode Reader achieved the highest reading rate on all three datasets.
- On in-focus images (Artelab Dataset 1), Dynamsoft reached a 100% reading rate with 0 misreads.
- On challenging out-of-focus images, Dynamsoft led with an 81.86% reading rate versus 79.07% for Scandit, 13.95% for ZBar, and 10.23% for ZXing.
- On the largest dataset (DEAL Lab, 2,000 images), Dynamsoft recognized 91.95% of barcodes correctly with zero misreads.
- Dynamsoft maintained 100% precision across all three datasets.
Note:
- If a barcode is recognized as UPC-A and the ground truth is in EAN13 format (the EAN13 format has one extra leading zero), it is considered as correct.
- Some images contain multiple barcodes while in the annotation, only one barcode is annotated. If the SDK reads the unannotated barcode, it is not considered as mis-read.
Performance Test Result on Artelab Medium Barcode 1D Collection
The Artelab Medium Barcode 1D Collection has two sets of images. Dataset 1 contains pictures of barcodes taken from devices with the autofocus (AF) function. Dataset 2 contains pictures taken by devices without AF. Each of the datasets has 215 images.
The test result on dataset 1:
| SDK | Dynamsoft Barcode Reader | Commercial SDK A (Scandit) | ZXing-CPP | pyZbar |
|---|---|---|---|---|
| Correctly recognized | 215 out of 215 | 197 out of 215 | 177 out of 215 | 193 out of 215 |
| Misread | 0 | 0 | 1 | 1 |
| Reading rate | 100% | 91.63% | 82.36% | 89.77% |
| Precision | 100% | 100% | 99.44% | 99.48% |
Sample image:
The test result on out-of-focus images dataset 2:
| SDK | Dynamsoft Barcode Reader | Commercial SDK A (Scandit) | ZXing-CPP | pyZbar |
|---|---|---|---|---|
| Correctly recognized | 176 out of 215 | 169 out of 215 | 22 out of 215 | 30 out of 215 |
| Misread | 0 | 1 | 2 | 8 |
| Reading rate | 81.86% | 79.07% | 10.23% | 13.95% |
| Precision | 100% | 100% | 91.67% | 78.95% |
Sample image:
.jpg)
Performance Test Result on Muenster BarcodeDB
The dataset contains 1054 valid barcode images in real-life. It covers typical distortions, such as inhomogeneous illumination, reflections, or blurriness due to camera movement. Here is the detailed performance test result:
| SDK | Dynamsoft Barcode Reader | Commercial SDK A (Scandit) | ZXing-CPP | pyZbar |
|---|---|---|---|---|
| Correctly recognized | 1022 out of 1054 | 984 out of 1054 | 792 out of 1054 | 744 out of 1054 |
| Misread | 0 | 0 | 1 | 34 |
| Reading rate | 96.96% | 93.26% | 75.14% | 70.59% |
| Precision | 100% | 100% | 99.87% | 95.63% |
Sample image:
Performance Test Result on DEAL Lab’s Barcode Dataset
Images in the DEAL Lab’s barcode dataset are EAN-13 single barcodes and have various sizes and formats. It has 2000 images for testing.
| SDK | Dynamsoft Barcode Reader | Commercial SDK A (Scandit) | ZXing-CPP | pyZbar |
|---|---|---|---|---|
| Correctly recognized | 1839 out of 2000 | 1221 out of 2000 | 1264 out of 2000 | 1452 out of 2000 |
| Misread | 0 | 0 | 26 | 9 |
| Reading rate | 91.95% | 61.05% | 63.20% | 72.60% |
| Precision | 100% | 100% | 97.98% | 99.38% |
Sample image:
Conclusion
We can see that the Dynamsoft Barcode Reader has the highest reading rate on the three datasets. The Dynamsoft Barcode Reader can read more blurred barcodes. You can check out this video to learn more:
Improve Accuracy Even Further
Confidence Level
Dynamsoft allows developers to refine barcode results by adjusting the confidence level, which determines whether a result should be kept or discarded. For general scenarios, the value of confidence level should be set to 30 of 100, which leads to a 99.873% barcode recognition accuracy according to the test report from Dynamsoft’s QA team. The confidence level may be varied based on different datasets of barcode images.
In addition to the confidence level, the following techniques are also applied to guarantee the barcode recognition accuracy:
- API for comparing adjacent frames
- Dynamsoft Camera Enhancer for filtering out blurry frames
- OCR of the accompanying text for verification
Verify the Results Between Adjacent Frames
Since version 8.6 for mobile, Dynamsoft Barcode Reader added a new method enableResultVerification to verify the barcode result between adjacent frames before returning it. The code for this method is as follows:
reader.enableResultVerification(true)
// To check the status of this mode:
boolean x = reader.getEnableResultVerificationStatus();
This will filter out the misread barcode results efficiently by comparing with an internal frame queue.
Filter Out Blurry Frames
Dynamsoft Camera Enhancer is a module that features camera control, image processing, and image enhancement. It mainly works as an extension of Dynamsoft Barcode Reader to filter out blurry images on Android and iOS (the JavaScript edition is coming soon). The higher quality of images you get, the more accurate the barcode recognition will be.
OCR of the Accompanying Text
Dynamsoft Label Recognition is an OCR SDK developed based on deep learning. It trained a Caffe model for getting a high text recognition rate. The barcode orientation returned by barcode localization is utilized by the OCR SDK to precisely recognize the accompanying text around a barcode. The barcode text result is then verified by comparing the OCR result.
Conclusion
Choosing the right barcode scanning SDK has a direct impact on go-to-market speed, development costs, and end-user satisfaction. High recognition accuracy reduces support burden and increases user trust in barcode-driven workflows.
Across all three public benchmark datasets, Dynamsoft Barcode Reader consistently outperformed Scandit, ZXing-CPP, and ZBar in reading rate while maintaining 100% precision with zero misreads. Its ability to decode blurred and low-quality barcode images makes it particularly well-suited for real-world deployment on mobile devices and fixed industrial cameras.
With advanced barcode detection algorithms, an integrated OCR verification layer, and camera enhancement features, Dynamsoft Barcode Reader is a dependable choice for developers and enterprises building high-accuracy barcode scanning applications.
Frequently Asked Questions
What is barcode scanning accuracy?
Barcode scanning accuracy measures how reliably a barcode engine reads barcodes from images. It is typically expressed as two metrics: reading rate (how many barcodes were decoded out of the total) and precision (how many returned results were correct).
How does Dynamsoft Barcode Reader compare to ZXing and ZBar?
Based on testing across three public datasets, Dynamsoft Barcode Reader significantly outperforms both ZXing-CPP and ZBar in reading rate. On the DEAL Lab dataset (2,000 images), Dynamsoft achieved a 91.95% reading rate versus 63.20% for ZXing and 72.60% for ZBar, with zero misreads in all cases.
How does Dynamsoft compare to Scandit for barcode accuracy?
Dynamsoft Barcode Reader outperformed Scandit on all three datasets tested. The gap is most notable on challenging image sets: on the DEAL Lab dataset, Dynamsoft read 91.95% of barcodes correctly while Scandit read 61.05%.
What barcode symbologies were tested in this benchmark?
The benchmark focused on 1D barcode symbologies, primarily EAN-13 and UPC-A, using three publicly available datasets.
How can I improve barcode recognition accuracy?
Dynamsoft provides three main techniques to improve accuracy beyond the baseline: (1) adjusting the confidence level threshold, (2) enabling adjacent-frame result verification via enableResultVerification, and (3) using Dynamsoft Camera Enhancer to filter blurry frames before decoding.