Optimizing Android Barcode Scanning with NDK and JNI C++

Previously, I shared an article on how to use the Camera2 APIs and Dynamsoft Barcode Reader to build an Android barcode scanner app capable of detecting fast-moving barcodes. In that project, the barcode decoding was implemented in Java, which left some room for performance improvements. By accessing the native buffer of the camera frame, we can directly invoke native Barcode Reader APIs for better efficiency. This article will guide you through writing JNI code for Android barcode detection and using the Android NDK and CMake to build the C++ code, ultimately optimizing the barcode scanning process.

This article is Part 2 in a 7-Part Series.

Prerequisites

Getting Started with the Existing Android Barcode Scanner Project

Begin by cloning the source code from GitHub:

git clone https://github.com/yushulx/android-camera-barcode-mrz-document-scanner.git

Replacing Java Barcode Detection with JNI C++

Import the project into Android Studio to start optimizing the barcode detection process.

Define three native methods in Java:

private native ArrayList<SimpleResult> readBarcode(long hBarcode, ByteBuffer byteBuffer, int width, int height, int stride);
private native long createBarcodeReader(String license);
private native void destroyBarcodeReader(long ndkBarcodeReader);

The goal here is to instantiate the barcode reader object in C++ and store its memory address in Java.

The Android Camera2 APIs provide the ImageReader class to capture preview images from the camera. The returned data type is ByteBuffer instead of byte[]. A ByteBuffer is allocated from native code via JNI, so when using the buffer for barcode detection in Java, the data needs to be copied into a byte array:

byte[] bytes = new byte[buffer.remaining()];
buffer.get(bytes);

In contrast, when using C++, there’s no need for this extra memory copy step. The address of the buffer can be passed directly to the native method:

unsigned char * buffer = (unsigned char*)env->GetDirectBufferAddress(byteBuffer);

Dynamsoft Barcode Reader for Android is based on JNI as well. When calling the barcode decoding method in Java, the Java byte array is first copied into a native buffer to invoke the C++ methods. The optimization here is to eliminate this unnecessary image memory copy.

Create a src/main/cpp/android_main.cpp file:

#include <jni.h>
#include <cstring>
#include "DynamsoftBarcodeReader.h"
#include <android/log.h>

#define LOG_TAG "BarcodeReader"
#define LOGI(...) __android_log_print(ANDROID_LOG_INFO, LOG_TAG, __VA_ARGS__)
#define LOGE(...) __android_log_print(ANDROID_LOG_ERROR, LOG_TAG, __VA_ARGS__)
/**
 * BarcodeReader object:
 *
 */
void *pBarcodeReader = nullptr;

extern "C" JNIEXPORT jobject JNICALL
Java_com_example_android_camera2basic_Camera2BasicFragment_readBarcode(JNIEnv *env, jobject instance, jlong hBarcode,
                                                                       jobject byteBuffer, jint width, jint height, jint stride) {

    // ArrayList
    jclass classArray = env->FindClass("java/util/ArrayList");
    if (classArray == NULL) return NULL;
    jmethodID midArrayInit =  env->GetMethodID(classArray, "<init>", "()V");
    if (midArrayInit == NULL) return NULL;
    jobject objArr = env->NewObject(classArray, midArrayInit);
    if (objArr == NULL) return NULL;
    jmethodID midAdd = env->GetMethodID(classArray, "add", "(Ljava/lang/Object;)Z");
    if (midAdd == NULL) return NULL;

    // SimpleResult
    jclass cls = env->FindClass("com/example/android/camera2basic/Camera2BasicFragment$SimpleResult");
    if (NULL == cls) return NULL;
    jmethodID midInit = env->GetMethodID(cls, "<init>", "(Ljava/lang/String;Ljava/lang/String;)V");
    if (NULL == midInit) return NULL;


    unsigned char * buffer = (unsigned char*)env->GetDirectBufferAddress(byteBuffer);

    int ret = DBR_DecodeBuffer((void *)hBarcode, buffer, width, height, stride, IPF_NV21, "");

    if (ret) {
        LOGE("Detection error: %s", DBR_GetErrorString(ret));
//        return NULL;
    }

    TextResultArray *pResults = NULL;
    DBR_GetAllTextResults((void *)hBarcode, &pResults);
    if (pResults)
    {
        int count = pResults->resultsCount;

        for (int i = 0; i < count; i++)
        {
//            LOGI("Native format: %s, text: %s", pResults->ppResults[i]->pszBarcodeFormatString, pResults->ppResults[i]->pszBarcodeText);
            jobject newObj = env->NewObject(cls, midInit, env->NewStringUTF(pResults->results[i]->barcodeFormatString), env->NewStringUTF(pResults->results[i]->barcodeText));
            env->CallBooleanMethod(objArr, midAdd, newObj);
        }

        // release memory of barcode results
        DBR_FreeTextResults(&pResults);
    }
    return objArr;

}

extern "C" JNIEXPORT jlong JNICALL
Java_com_example_android_camera2basic_Camera2BasicFragment_createBarcodeReader(JNIEnv *env, jobject instance, jstring license) {
    if (!pBarcodeReader) {
        // Instantiate barcode reader object.
        pBarcodeReader = DBR_CreateInstance();

        // Initialize the license key.
        const char *nativeString = env->GetStringUTFChars(license, 0);
        char errorMsgBuffer[512];
        DBR_InitLicense(nativeString, errorMsgBuffer, 512);
        env->ReleaseStringUTFChars(license, nativeString);
    }

    return (jlong)(pBarcodeReader);

}

extern "C" JNIEXPORT void JNICALL
Java_com_example_android_camera2basic_Camera2BasicFragment_destroyBarcodeReader(JNIEnv *env, jobject instance, jlong hBarcode) {

    if (hBarcode) {
        DBR_DestroyInstance((void *)hBarcode);
    }

}

Explanation

  • DBR_DecodeBuffer: Decodes barcodes from the buffer.
  • DBR_GetAllTextResults: Retrieves all barcode results.
  • DBR_FreeTextResults: Frees the memory allocated for barcode results.
  • DBR_CreateInstance: Creates a new instance of the barcode reader.
  • DBR_InitLicense: Initializes the license key.
  • DBR_DestroyInstance: Destroys the barcode reader instance.

Comparison: Java vs. JNI Barcode Detection

  • Java Barcode Detection:

      ByteBuffer buffer = image.getPlanes()[0].getBuffer();
  int nRowStride = image.getPlanes()[0].getRowStride();
  int nPixelStride = image.getPlanes()[0].getPixelStride();
  byte[] bytes = new byte[buffer.remaining()];
  buffer.get(bytes);
  TextResult[] results = mBarcodeReader.decodeBuffer(bytes, mImageReader.getWidth(), mImageReader.getHeight(), nRowStride * nPixelStride, EnumImagePixelFormat.IPF_NV21, "");

  • JNI Barcode Detection:

      ByteBuffer buffer = image.getPlanes()[0].getBuffer();
  int nRowStride = image.getPlanes()[0].getRowStride();
  int nPixelStride = image.getPlanes()[0].getPixelStride();
  ArrayList<SimpleResult> results = readBarcode(hBarcode, buffer, mImageReader.getWidth(), mImageReader.getHeight(), nRowStride * nPixelStride);

After implementing the JNI-based method, here’s how your project structure should look:

camera project structure

Building C++ Code with Android NDK and CMake

Create a CMakeLists.txt file under src/main/cpp:

cmake_minimum_required(VERSION 3.4.1)

set(CMAKE_VERBOSE_MAKEFILE on)
set(CMAKE_ANDROID_ARCH_ABI arm64-v8a)

link_directories("${CMAKE_CURRENT_SOURCE_DIR}")

add_library(dynamsoft_barcode SHARED
    ${CMAKE_CURRENT_SOURCE_DIR}/android_main.cpp)

# add include path
target_include_directories(dynamsoft_barcode PRIVATE ${COMMON_SOURCE_DIR})

# add lib dependencies
target_link_libraries(dynamsoft_barcode dl android log m DynamsoftBarcodeReaderAndroid)

In the build.gradle file, add the following configuration:

defaultConfig {
    minSdkVersion 21
    targetSdkVersion 27
    ndk {
        abiFilters 'arm64-v8a'
    }
    externalNativeBuild {
        cmake {
            arguments '-DANDROID_STL=c++_static', '-DANDROID_ABI=arm64-v8a'
        }
    }
}

externalNativeBuild {
    cmake {
        version '3.10.2'
        path 'src/main/cpp/CMakeLists.txt'
    }
}

compileOptions {
    sourceCompatibility JavaVersion.VERSION_1_7
    targetCompatibility JavaVersion.VERSION_1_7
}

Since Google Play now only accepts 64-bit apps, we specify the arm64-v8a architecture.

Now you can successfully build and run the optimized barcode reader app:

android ndk barcode scanner

Source Code

https://github.com/yushulx/android-camera-barcode-mrz-document-scanner/tree/main/examples/9.x/ndk_jni