Object Detection and Barcode Reading

Object detection is a computer technology related to computer vision and image processing that deals with detecting instances of semantic objects of a certain class (such as humans, buildings, or cars) in digital images and videos.1 It can be used in a barcode reading task as barcodes are often printed on objects like labels, books, etc. We can detect the objects first and then read barcodes on these objects, which has the following potential benefits:

  • Improve the reading speed
  • Check whether there are unread barcodes
  • Save the image of the object with barcodes printed on it

In this article, we are going to explore some object detection methods and how to use them with barcode reading. The code is in Python with OpenCV for image processing and Dynamsoft Barcode Reader for barcode reading.

Sample image (5 bottles detected. 1 bottle without barcode read):

barcodes on bottles

Object Detection

There are many ways to detect objects. If the objects have vivid features, we can use some simple image processing methods to detect them. If the objects are complex, we can use deep learning to detect them.

Here, we are going to list some methods for object detection.

Object Detection using Contours Finding

For the labels placed on a blue board, we can detect them by finding the contours on the image’s binary version.

barcodes on labels

  1. Perform thresholding to get a binary image:

    original_image = cv2.imread("img.jpg")
gray = cv2.cvtColor(original_image, cv2.COLOR_BGR2GRAY)
ret,thresh = cv2.threshold(gray,200,255,cv2.THRESH_BINARY)

    barcodes on labels thresh

  2. Find contours, filter out small irregular ones and draw their bounding rectangles.

    contours, hierarchy = cv2.findContours(thresh, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
rects = []
for cnt in contours:
    rect = cv2.boundingRect(cnt)
    x, y, w, h = rect
    if w > 50 and h > 20 and w>h:
        rects.append(rect)

height, width, _ = original_image.shape
min_x, min_y = width, height
max_x = max_y = 0
for (x,y,w,h) in rects:
    color = (255,0,0)
    min_x, max_x = min(x, min_x), max(x+w, max_x)
    min_y, max_y = min(y, min_y), max(y+h, max_y)
    cv2.rectangle(original_image, (x,y), (x+w,y+h), color, 5)

    barcodes on labels detected

Object Detection using Color Segmentation

In the following image, DataMatrix codes are printed on cassettes with vivid colors. We can detect the cassettes based on their colors. To do this, we can convert the image into HSV color space and use OpenCV’s inrange method for segmentation.

cassette

  1. Convert the image into HSV color space.

    hsv = cv2.cvtColor(image, cv2.COLOR_BGR2HSV)

  2. Use inrange to separate the specific color.

    lower = np.array([26,120,150])
upper = np.array([34,255,255])
mask = cv2.inRange(hsv, lower, upper)

    Here is the mask for yellow.

    mask yellow

    Here is the dict of bounds for colors in this image.

    colors_hsv = {
  "red": {
    "lower": np.array([156,150,100]),
    "upper": np.array([180,255,255]) 
  },
  "yellow": {
    "lower": np.array([26,120,150]),
    "upper": np.array([34,255,255]) 
  },
  "pink": {
    "lower": np.array([156,80,100]),
    "upper": np.array([180,140,255]) 
  },
  "green": {
    "lower": np.array([88,50,100]),
    "upper": np.array([99,80,255]) 
  },
  "blue": {
    "lower": np.array([78,120,100]),
    "upper": np.array([99,255,255]) 
  }
}

    We can iterate the colors to detect all the cassettes.

  3. Find contours on the mask, filter out small and irregular ones and get the bounding rectangles.

    contours, hierarchy = cv2.findContours(img,cv2.RETR_EXTERNAL,cv2.CHAIN_APPROX_SIMPLE)
rects = []
height, width = img.shape
for cnt in contours:
    rect = cv2.boundingRect(cnt)
    x, y, w, h = rect
    if w > 80 and h > 80 and w < width/2 and w>4*h:
        rects.append(rect)
           
height, width, _ = original_image.shape
min_x, min_y = width, height
max_x = max_y = 0
for (x,y,w,h) in rects:
    min_x, max_x = min(x, min_x), max(x+w, max_x)
    min_y, max_y = min(y, min_y), max(y+h, max_y)
    cv2.rectangle(original_image, (x,y), (x+w,y+h), (255, 0, 0), 5)

    cassette detected

Object Detection using YOLO

In the following image, there are five bottles. It is not easy to detect them with simple image processing. We can use YOLOv8 to detect them which is a deep learning method.

bottles

  1. Install YOLOv8.

    pip install ultralytics

  2. Use the following code to detect the bottles.

    from ultralytics import YOLO
import cv2
model = YOLO("yolov8n.pt")
img = cv2.imread("image.jpg")
results = model.predict(source=img)

  3. Save the detections in a list.

    detections = []
for result in results:
    for box in result.boxes:
        xyxy = box.xyxy[0]
        detection = {}
        detection["x"] = int(xyxy[0])
        detection["y"] = int(xyxy[1])
        detection["w"] = int(xyxy[2]) - int(xyxy[0])
        detection["h"] = int(xyxy[3]) - int(xyxy[1])
        detection["confidence"] = float(box.conf)
        class_index = int(box.cls[0])
        detection["class"] = model.names[class_index]
        detections.append(detection)

  4. Draw the results.

    for box in boxes:
    x = int(box["x"])
    y = int(box["y"])
    w = int(box["w"])
    h = int(box["h"])
    color = (255,0,0)
    cv2.rectangle(resized, (x, y), (x+w, y+h), color=color, thickness=2)
    cv2.putText(resized, box["class"], (x, y-10), cv2.FONT_HERSHEY_SIMPLEX, 0.9, (36,255,12), 2)

    bottles detected

Barcode Reading

After detecting the objects, we can crop the images and read the barcodes on them. Here, we use Dynamsoft Barcode Reader to read them.

  1. Activate the license. You can apply for a license here.

    error = BarcodeReader.init_license("DLS2eyJoYW5kc2hha2VDb2RlIjoiMjAwMDAxLTE2NDk4Mjk3OTI2MzUiLCJvcmdhbml6YXRpb25JRCI6IjIwMDAwMSIsInNlc3Npb25QYXNzd29yZCI6IndTcGR6Vm05WDJrcEQ5YUoifQ==") #one-day trial
if error[0] != EnumErrorCode.DBR_OK:
   # Add your code for license error processing
   print("License error: "+ error[1])

  2. Create a new instance of a barcode reader.

    reader = BarcodeReader()

  3. Set the barcode format to DataMatrix which is printed on the sample images.

    settings = reader.get_runtime_settings()
settings.barcode_format_ids = EnumBarcodeFormat.BF_DATAMATRIX
reader.update_runtime_settings(settings)

  4. Crop the objects in the images, read the barcodes in them and draw the results.

    for box in boxes:
    has_barcodes = True
    box_image = img[box["y"]:box["y"]+box["h"],box["x"]:box["x"]+box["w"]]
    barcode_results = reader.decode_buffer(box_image)
    if barcode_results == None:
        has_barcodes = False
       
    color = (255,0,0)
    if has_barcodes == False:
        color = (0,0,255)
           
    cv2.rectangle(img, (x, y), (x+w, y+h), color=color, thickness=2)

    If there are no barcodes found on the objects, use red to draw the rectangles.

    barcodes on bottles

Source Code

You can find the complete code here: https://github.com/tony-xlh/object-detection-and-barcode-reading

References

  1. Dasiopoulou, Stamatia, et al. “Knowledge-assisted semantic video object detection.” IEEE Transactions on Circuits and Systems for Video Technology 15.10 (2005): 1210–1224. ↩