Panorama Stitching: New Technology for Barcode Industry

Dynamsoft recently added a new panorama module to barcode SDK, aiming to provide a brand-new solution for warehouse and inventory management. In this article, I will try my best to guide you through the usage of Dynamsoft Panorama version 1.0, though the first version is not stable and optimized yet.

Demystifying Dynamsoft Panorama

Dynamsoft Panorama combines the panorama stitching algorithm and the barcode algorithm. Unlike the panorama function that we usually used in a camera app, Dynamsoft panorama SDK was born for continuously scanning a whole barcode scene. Therefore, you can stitch as many as barcode images you can. The typical use case is warehouse management. The SDK stitches sequential image frames into panorama images, which help the warehouse manager to double-check packages marked with barcodes.

The Usage of Dynamsoft Panorama SDK on Windows 10

Let’s get started with a simple C++ barcode app to see what we can do with the panorama stitching SDK.

Required Software

Coding Steps

Instantiate the CPanoramaStitcher class:

CPanoramaStitcher* panoramaStitcher = new CPanoramaStitcher;

Set a valid license key, which can be obtained from the Windows installer or the online portal:

panoramaStitcher->InitLicense("DLS2eyJoYW5kc2hha2VDb2RlIjoiMjAwMDAxLTE2NDk4Mjk3OTI2MzUiLCJvcmdhbml6YXRpb25JRCI6IjIwMDAwMSIsInNlc3Npb25QYXNzd29yZCI6IndTcGR6Vm05WDJrcEQ5YUoifQ==");

Initialize the stitcher settings:

panoramaStitcher->InitRuntimeSettingsWithString("{\"InputSourceMode\":0, \"PreviewImgScaleRatio\": 0.5}", error, 512);
  • InputSourceMode: 0 means the input source is camera. 1 means the input source is video files. The reason we have to distinguish the input source is the adjacent frames from a video stream are much more similar than the ones from a webcam.
  • PreviewImgScaleRatio: the scale ratio of previewed images. The smaller, the faster.

We can use OpenCV to capture frames from a webcam or video files:

VideoCapture capture(0); 
VideoCapture capture("video.mp4");

Note: 0 means auto-detect. On Windows 10, it will invoke MSMF (Microsoft Media Foundation) to get the video input by default. It seems that the MSMF is not stable for grabbing video frames, because I got crash accidentally:

[ WARN:0] videoio(MSMF): can't grab frame. Error: -2147483638

After reading the OpenCV guide, I replaced the API with DirectShow which works better than Microsoft Media Foundation:

VideoCapture capture(0 + cv::CAP_DSHOW);

Next, set a callback function for receiving the panorama images and start the panorama stitching thread:

panoramaStitcher->SetResultCallback(&GetPanoramaResult, pUser);
panoramaStitcher->SetErrorCodeCallback(&GetErrorCode, pUser);
panoramaStitcher->StartPanoramaStitchingByBuffer(width, height, width * chnnel, DPS_IPF_RGB_888);

Loop the frame capture and append frames to the stitcher:

for (;;)
{
            capture >> frame; 
            int key = waitKey(1);
            if (key == 27/*ESC*/ || frame.empty()) {
                destroyWindow(windowName);
                break;
            }

            ret = panoramaStitcher->AppendNewFrameByBuffer(frame.data);
            if (ret)
            {
                cout << "Failed to append new frame" << endl;
                destroyWindow(windowName);
                break;
            }   
}

If there is no memory available, the AppendNewFrameByBuffer function will return DPSERR_NO_MEMORY.

Once we break the loop, we can save all panorama images to the local disk:

        panoramaStitcher->WaitForFinishStitching();
        panoramaStitcher->StopPanoramaStitching();
        PanoramaResultArray* finalResult = NULL;
        panoramaStitcher->GetFinalResults(&finalResult);

        if (finalResult->resultCount > 0)
        {
            printf("Total panorama results: %d\n", finalResult->resultCount);

            for (int j = 0; j < finalResult->resultCount; ++j)
            {
                int thickness = 2;
                Scalar color(0, 255, 0);

                String fileName = to_string(j) + ".bmp";

                if(SaveDMBitmapToLocal(finalResult->panoramaResult[j]->resultImg, fileName))
                {
                    printf("The panorama result is saved. Total stiched image number: %d\n", finalResult->panoramaResult[j]->stitchedImgsNumber);
                   
                }
            }
            waitKey(0);
        }

        CPanoramaStitcher::FreePanoramaResults(&finalResult);

You may wonder how to stop the stitching process when the first panorama image is done. So far, the only way is to check the value of indexOfFirstFrameInResultImg in the callback function:

void GetPanoramaResult(PanoramaResultArray** resultArray, void* pUser)
{
    if ((*resultArray)->panoramaResult[0]->indexOfFirstFrameInResultImg != 0)
     {
       // The first panorama image is done.
     }
    CPanoramaStitcher::FreePanoramaResults(resultArray);
}

The more images stored in the memory, the more time it will take to stop stitching.

The Panorama Stitching Effect

I used a 1080P webcam and a video file to test the SDK, and found some issues:

  • The effect of stitching the images from a webcam is not good. To stitch high-quality real-time images, you can reference the sample code of building a server for receiving and stitching camera images from your Android and iOS devices in the Dynamsoft Panorama installer. You also need to install Barcode Scanner X from the app store.
  • The stitching algorithm cannot work stably, because every time I run the app, the result is different.

Here are the different panorama stitching results I got.

panorama barcodes

barcode panorama stitching

Although the current stitching results are not as good as expected, a better panorama stitching algorithm is on the way. We will see the panorama stitching technology brings more efficiency to the barcode industry in the future.

Source Code

https://github.com/yushulx/barcode-panorama-stitch