Intelligent Barcode Annotation Tool using Python and Qt
Image annotation is a key step in deep learning. It prepares the data for tasks like image classification, object detection, semantic segmentation, etc.
For image classification, we only need to annotate the label. For object detection, we also need the bounding box and for segmentation, we need to annotate objects in polygons.
Barcode annotation is a bit different since barcodes represent data. So in addition to their location, we can also annotate the data represented.
Annotated barcodes dataset can be used for machine learning and benchmark of the performance of barcode reading SDKs.
Here is a demo of using the finished barcode annotation tool.
Existing Image Annotation Tools
There are several open source image annotation tools:
- Labelimg. It is a popular graphical image annotation tool using Python + Qt. It can annotate objects in rectangular boxes.
- Labelme (Web). This is an online annotation tool which uses php as its backend. It can annotate polygons as well as rectangular boxes.
- Labelme. This one has similar functions to the web version. It is a desktop tool using Python + Qt and has a similar interface to Labelimg.
We are going to create a barcode annotation tool based on Labelme. It is easier to set up and can precisely annotate barcodes, even rotated ones.
Modify Labelme as an Intelligent Barcode Annotation Tool
Add a Content Property to Shape
The annotation is saved in a JSON file. It contains a shapes list which is the list of annotated objects. The shape object has properties like label, location points and shape type. But it does not have a property to store the barcode content.
So I added a content property to the shape object to store the barcode content.
JSON with content added:
{
"version": "4.5.10",
"flags": {},
"shapes": [
{
"label": "EAN_13",
"points": [
[
689.0,
333.0
],
[
1126.0,
641.0
],
[
947.0,
896.0
],
[
508.0,
584.0
]
],
"group_id": null,
"content": "4902030187590",
"shape_type": "polygon",
"flags": {}
}
],
"imagePath": "05102009081.jpg",
"imageData": "<base64 encoded content>",
"imageHeight": 1200,
"imageWidth": 1600
}
A text edit widget is added to the label dialog. Users can modify and check the barcode content.
You can check out the detailed changes here.
Use Dynamsoft Barcode Reader to Automatically Annotate Barcodes
The Dynamsoft Barcode Reader (DBR) can read barcodes in images and return their barcode format, barcode text and location points. We can use it to automatically annotate barcodes.
Create a Reader Class
A reader class is created. It can decode the barcodes in an image file and return a dict containing the decoded results. Please note that DBR is a commercial SDK. It requires a license to run. You can apply for a trial license here.
from dbr import *
class DynamsoftBarcodeReader():
def __init__(self):
self.dbr = BarcodeReader()
self.dbr.init_license("<license>")
def decode_file(self, img_path):
result_dict = {}
results = []
text_results = self.dbr.decode_file(img_path)
if text_results!=None:
for tr in text_results:
result = {}
result["barcodeFormat"] = tr.barcode_format_string
result["barcodeFormat_2"] = tr.barcode_format_string_2
result["barcodeText"] = tr.barcode_text
result["confidence"] = tr.extended_results[0].confidence
results.append(result)
points = tr.localization_result.localization_points
result["x1"] =points[0][0]
result["y1"] =points[0][1]
result["x2"] =points[1][0]
result["y2"] =points[1][1]
result["x3"] =points[2][0]
result["y3"] =points[2][1]
result["x4"] =points[3][0]
result["y4"] =points[3][1]
result_dict["results"] = results
return result_dict
if __name__ == '__main__':
import time
reader = DynamsoftBarcodeReader()
start_time = time.time()
results = reader.decode_file("image045.jpg")
end_time = time.time()
elapsedTime = int((end_time - start_time) * 1000)
Add Menu Actions to Detect Barcodes
A menu item named intelligence is added with two actions. One is to detect barcodes for the current image and the other is to detect barcodes for all images. A helper class named intelligenceHelper is created to handle relevant operations.
def __init__():
#intelligence actions
detect_barcodes = action(
self.tr("Detect Barcodes for the Current File"),
self.detectBarcodesOfOne,
None,
None,
self.tr("Detect Barcodes for the Current File")
)
detect_barcodes_all = action(
self.tr("Detect Barcodes for All Files"),
self.detectBarcodesOfAll,
None,
None,
self.tr("Detect Barcodes for All Files")
)
def detectBarcodesOfOne(self):
if os.path.exists(self.filename):
self.labelList.clearSelection()
s = self.intelligenceHelper.getBarcodeShapesOfOne(self.filename)
self.loadShapes(s)
self.actions.editMode.setEnabled(True)
self.actions.undoLastPoint.setEnabled(False)
self.actions.undo.setEnabled(True)
self.setDirty()
def detectBarcodesOfAll(self):
images=[]
for filename in self.imageList:
images.append(filename)
self.intelligenceHelper.detectBarcodesOfAll(images)
Here are the key parts of the intelligenceHelper class.
-
Decode an image and convert the results to shapes.
def getBarcodeShapesOfOne(self,filename): results = self.reader.decode_file(filename)["results"] s = [] for result in results: shape = Shape() shape.label = result["barcodeFormat"] shape.content = result["barcodeText"] shape.shape_type="polygon" shape.flags = {} shape.other_data = {} for i in range(1,5): x = result["x"+str(i)] y = result["y"+str(i)] shape.addPoint(QtCore.QPointF(x, y)) shape.close() s.append(shape) return s -
Detect barcodes in all files.
This process takes time. So we need to run it in a thread and show a progress dialog.
We use the QProgressDialog widget to display the progress. Users can cancel the process by clicking the cancel button.
def startOperationDialog(self): self.operationCanceled = False pd1 = QtWidgets.QProgressDialog('Progress','Cancel',0,100,self.parent) pd1.setLabelText('Progress') pd1.setCancelButtonText('Cancel') pd1.setRange(0, 100) pd1.setValue(0) pd1.setMinimumDuration(0) pd1.show() pd1.canceled.connect(self.onProgressDialogCanceledOrCompleted) return pd1QThread is used for decoding images and saving the results to JSON files. Since the UI should be updated in the GUI thread,
Signalis used to send messages from the decoding thread to the GUI thread to update the progress.The thread class:
class IntelligenceWorker(QThread): sinOut = pyqtSignal(int,int) def __init__(self, parent, images, source): super(IntelligenceWorker, self).__init__(parent) self.parent = parent # parent window self.source = source # the calling class self.images = images def run(self): index = 0 total = len(self.images) for filename in self.images: if self.parent.isVisible==False: return if self.source.operationCanceled==True: return index = index + 1 json_name = osp.splitext(filename)[0] + ".json" if os.path.exists(json_name)==False: try: print("Decoding "+filename) s = self.source.getBarcodeShapesOfOne(filename) self.source.saveLabelFile(filename, s) except Exception as e: print(e) self.sinOut.emit(index,total)Use the thread class:
def detectBarcodesOfAll(self,images): self.pd = self.startOperationDialog() self.thread = IntelligenceWorker(self.parent,images,self) self.thread.sinOut.connect(self.updateDialog) self.thread.start() def updateDialog(self, completed, total): progress = int(completed/total*100) self.pd.setLabelText(str(completed) +"/"+ str(total)) self.pd.setValue(progress) if completed==total: self.onProgressDialogCanceledOrCompleted()
With this tool, you can now take your barcode scanning and recognition projects to the next level, focus on other aspects of deep learning and let the tool handle the basic work.