Intelligent Barcode Annotation Tool using Python and Qt

Image annotation is a key step in deep learning. It prepares the data for tasks like image classification, object detection, semantic segmentation, etc.

For image classification, we only need to annotate the label. For object detection, we also need the bounding box and for segmentation, we need to annotate objects in polygons.

Screenshot QR code

Barcode annotation is a bit different since barcodes represent data. So in addition to their location, we can also annotate the data represented.

Annotated barcodes dataset can be used for machine learning and benchmark of the performance of barcode reading SDKs.

Existing Image Annotation Tools

There are several open source image annotation tools:

  1. Labelimg. It is a popular graphical image annotation tool using Python + Qt. It can annotate objects in rectangular boxes.
  2. Labelme (Web). This is an online annotation tool which uses php as its backend. It can annotate polygons as well as rectangular boxes.
  3. Labelme. This one has similar functions to the web version. It is a desktop tool using Python + Qt and has a similar interface to Labelimg.

We are going to create a barcode annotation tool based on Labelme. It is easier to set up and can precisely annotate barcodes, even rotated ones.

Modify Labelme as an Intelligent Barcode Annotation Tool

Add a Content Property to Shape

The annotation is saved in a JSON file. It contains a shapes list which is the list of annotated objects. The shape object has properties like label, location points and shape type. But it does not have a property to store the barcode content.

So I added a content property to the shape object to store the barcode content.

JSON with content added:

{
  "version": "4.5.10",
  "flags": {},
  "shapes": [
    {
      "label": "EAN_13",
      "points": [
        [
          689.0,
          333.0
        ],
        [
          1126.0,
          641.0
        ],
        [
          947.0,
          896.0
        ],
        [
          508.0,
          584.0
        ]
      ],
      "group_id": null,
      "content": "4902030187590",
      "shape_type": "polygon",
      "flags": {}
    }
  ],
  "imagePath": "05102009081.jpg",
  "imageData": "<base64 encoded content>",
  "imageHeight": 1200,
  "imageWidth": 1600
}

A text edit widget is added to the label dialog. Users can modify and check the barcode content.

Label Dialog

You can check out the detailed changes here.

Use Dynamsoft Barcode Reader to Automatically Annotate Barcodes

The Dynamsoft Barcode Reader (DBR) can read barcodes in images and return their barcode format, barcode text and location points. We can use it to automatically annotate barcodes.

Create a Reader Class

A reader class is created. It can decode the barcodes in an image file and return a dict containing the decoded results. Please note that DBR is a commercial SDK. It requires a license to run. You can apply for a trial license here.

from dbr import *

class DynamsoftBarcodeReader():
    def __init__(self):
        self.dbr = BarcodeReader()
        self.dbr.init_license("<license>")

    def decode_file(self, img_path):
        result_dict = {}
        results = []
        text_results = self.dbr.decode_file(img_path)
        
        if text_results!=None:
            for tr in text_results:
                result = {}
                result["barcodeFormat"] = tr.barcode_format_string
                result["barcodeFormat_2"] = tr.barcode_format_string_2
                result["barcodeText"] = tr.barcode_text
                result["confidence"] = tr.extended_results[0].confidence
                results.append(result)
                points = tr.localization_result.localization_points
                result["x1"] =points[0][0]
                result["y1"] =points[0][1]
                result["x2"] =points[1][0]
                result["y2"] =points[1][1]
                result["x3"] =points[2][0]
                result["y3"] =points[2][1]
                result["x4"] =points[3][0]
                result["y4"] =points[3][1]
        result_dict["results"] = results
        
        return result_dict
        
if __name__ == '__main__':
    import time
    reader = DynamsoftBarcodeReader()
    start_time = time.time()
    results = reader.decode_file("image045.jpg")
    end_time = time.time()
    elapsedTime = int((end_time - start_time) * 1000)

Add Menu Actions to Detect Barcodes

A menu item named intelligence is added with two actions. One is to detect barcodes for the current image and the other is to detect barcodes for all images. A helper class named intelligenceHelper is created to handle relevant operations.

def __init__():
    #intelligence actions
    detect_barcodes = action(
        self.tr("Detect Barcodes for the Current File"),
        self.detectBarcodesOfOne,
        None,
        None,
        self.tr("Detect Barcodes for the Current File")
    )

    detect_barcodes_all = action(
        self.tr("Detect Barcodes for All Files"),
        self.detectBarcodesOfAll,
        None,
        None,
        self.tr("Detect Barcodes for All Files")
    )
    
def detectBarcodesOfOne(self):
    if os.path.exists(self.filename):
        self.labelList.clearSelection()
        s = self.intelligenceHelper.getBarcodeShapesOfOne(self.filename)
        self.loadShapes(s)
        self.actions.editMode.setEnabled(True)
        self.actions.undoLastPoint.setEnabled(False)
        self.actions.undo.setEnabled(True)
        self.setDirty()

def detectBarcodesOfAll(self):
    images=[]
    for filename in self.imageList:
        images.append(filename)
    self.intelligenceHelper.detectBarcodesOfAll(images)

Here are the key parts of the intelligenceHelper class.

  • Decode an image and convert the results to shapes.

      def getBarcodeShapesOfOne(self,filename):
          results = self.reader.decode_file(filename)["results"]
          s = []
          for result in results:
              shape = Shape()
              shape.label = result["barcodeFormat"]
              shape.content = result["barcodeText"]
              shape.shape_type="polygon"
              shape.flags = {}
              shape.other_data = {}
              for i in range(1,5):
                  x = result["x"+str(i)]
                  y = result["y"+str(i)]
                  shape.addPoint(QtCore.QPointF(x, y))
              shape.close()
              s.append(shape)
          return s
    
  • Detect barcodes in all files.

    This process takes time. So we need to run it in a thread and show a progress dialog.

    We use the QProgressDialog widget to display the progress. Users can cancel the process by clicking the cancel button.

      def startOperationDialog(self):
          self.operationCanceled = False
          pd1 =  QtWidgets.QProgressDialog('Progress','Cancel',0,100,self.parent)
          pd1.setLabelText('Progress')
          pd1.setCancelButtonText('Cancel')
          pd1.setRange(0, 100)
          pd1.setValue(0)
          pd1.setMinimumDuration(0)
          pd1.show()
          pd1.canceled.connect(self.onProgressDialogCanceledOrCompleted)
          return pd1
    

    QThread is used for decoding images and saving the results to JSON files. Since the UI should be updated in the GUI thread, Signal is used to send messages from the decoding thread to the GUI thread to update the progress.

    The thread class:

      class IntelligenceWorker(QThread):
          sinOut = pyqtSignal(int,int)
          def __init__(self, parent, images, source):
              super(IntelligenceWorker, self).__init__(parent)
              self.parent = parent # parent window
              self.source = source # the calling class
              self.images = images
    
          def run(self):
              index = 0
              total = len(self.images)
              for filename in self.images:
                  if self.parent.isVisible==False:
                      return
                  if self.source.operationCanceled==True:
                      return
                  index = index + 1
                  json_name = osp.splitext(filename)[0] + ".json"
                  if os.path.exists(json_name)==False:
                      try:
                          print("Decoding "+filename)
                          s = self.source.getBarcodeShapesOfOne(filename)
                          self.source.saveLabelFile(filename, s)
                      except Exception as e:
                          print(e)
                  self.sinOut.emit(index,total)
    

    Use the thread class:

      def detectBarcodesOfAll(self,images):
          self.pd = self.startOperationDialog()
          self.thread = IntelligenceWorker(self.parent,images,self)
          self.thread.sinOut.connect(self.updateDialog)
          self.thread.start()
            
      def updateDialog(self, completed, total):
          progress = int(completed/total*100)
          self.pd.setLabelText(str(completed) +"/"+ str(total))
          self.pd.setValue(progress)
          if completed==total:
              self.onProgressDialogCanceledOrCompleted()
    

The Final Result

Here is a demo of using the finished barcode annotation tool.

Source Code

https://github.com/xulihang/labelme

Search Blog Posts