Build a React Native Vision Camera Frame Processor Plugin to Scan Barcodes for Android

Feb 09, 2022

The previous popular react-native-camera project is now deprecated in favor of react-native-vision-camera. Vision Camera offers new APIs, better performance, improved stability and more features. It has a feature called Frame processors, which makes it easy to integrate extra image processing abilities.

We are going to create a frame processor plugin to use Dynamsoft Barcode Reader (DBR) in a React Native app.

In this article, we will focus on creating the frame processor plugin for Android first.

Dynamsoft Capture Vision React Native

If your project doesn't use react-native-vision-camera, we recommend you use the Dynamsoft Capture Vision React Native package for barcode scanning instead.

Dynamsoft Capture Vision on NPM

This article is Part 1 in a 3-Part Series.

Other React Native Vision Camera Frame Processor Plugins

Getting started with Dynamsoft Barcode Reader

Building the React Native Vision Camera Frame Processor Plugin of Barcode Reader for Android

New Project

First, create a plugin project using bob.

npx create-react-native-library vision-camera-dynamsoft-barcode-reader

You can test the project using the following command:

cd example
npx react-native run-android

Implement the Plugin

Go to android\src\main\java\com\visioncameradynamsoftbarcodereader, remove VisionCameraDynamsoftBarcodeReaderModule.java and remove its reference in VisionCameraDynamsoftBarcodeReaderPackage.java

 public List<NativeModule> createNativeModules(@NonNull ReactApplicationContext reactContext) {
     List<NativeModule> modules = new ArrayList<>();
-    modules.add(new VisionCameraDynamsoftBarcodeReaderModule(reactContext));
     return modules;

 }

Create a new file named VisionCameraDBRPlugin.java with the following content:

import androidx.camera.core.ImageProxy;
import com.mrousavy.camera.frameprocessor.FrameProcessorPlugin;

public class VisionCameraDBRPlugin extends FrameProcessorPlugin {

  @Override
  public Object callback(ImageProxy image, Object[] params) {
    // code goes here
    return null;
  }

  VisionCameraDBRPlugin() {
    super("decode");
  }
}

 public List<NativeModule> createNativeModules(@NonNull ReactApplicationContext reactContext) {
     List<NativeModule> modules = new ArrayList<>();
+    FrameProcessorPlugin.register(new VisionCameraDBRPlugin());
     return modules;
 }

Add the following content to the android/build.gradle file to include Dynamsoft Barcode Reader and CameraX.

rootProject.allprojects {
    repositories {
        maven {
            url "https://download2.dynamsoft.com/maven/dbr/aar"
        }
    }
}
dependencies {
    // From node_modules
    implementation project(path: ':react-native-vision-camera')
    implementation "androidx.camera:camera-core:1.1.0-alpha12"
    implementation 'com.dynamsoft:dynamsoftbarcodereader:9.0.0@aar'
}

Create an instance of Dynamsoft Barcode Reader if the plugin is called

private BarcodeReader reader = null;
public Object callback(ImageProxy image, Object[] params) {
    if (reader == null){
        createDBRInstance(params);
    }
    return null
}

private void createDBRInstance(ReadableNativeMap config) {
    String license = "DLS2eyJoYW5kc2hha2VDb2RlIjoiMjAwMDAxLTE2NDk4Mjk3OTI2MzUiLCJvcmdhbml6YXRpb25JRCI6IjIwMDAwMSIsInNlc3Npb25QYXNzd29yZCI6IndTcGR6Vm05WDJrcEQ5YUoifQ==";
    if (config != null){
        if (config.hasKey("license")) {
            license = config.getString("license");
        }
    }

    BarcodeReader.initLicense(license, new DBRLicenseVerificationListener() {
        @Override
        public void DBRLicenseVerificationCallback(boolean isSuccessful, Exception e) {
            if (!isSuccessful) {
                e.printStackTrace();
            }
        }
    });
    try {
        reader = new BarcodeReader();
    } catch (BarcodeReaderException e) {
        e.printStackTrace();
    }
}

Decode the image using Dynamsoft Barcode Reader and return the result

public Object callback(ImageProxy image, Object[] params) {
    if (reader == null){
        createDBRInstance(params);
    }
    TextResult[] results = null;
    ByteBuffer buffer = image.getPlanes()[0].getBuffer();
    int nRowStride = image.getPlanes()[0].getRowStride();
    int nPixelStride = image.getPlanes()[0].getPixelStride();
    int length = buffer.remaining();
    byte[] bytes = new byte[length];
    buffer.get(bytes);

    try {
        results = reader.decodeBuffer(bytes, image.getWidth(), image.getHeight(), nRowStride*nPixelStride, EnumImagePixelFormat.IPF_NV21);
    } catch (BarcodeReaderException e) {
        e.printStackTrace();
    }
    WritableNativeArray array = new WritableNativeArray();
    if (results != null) {
        for (int i = 0; i < results.length; i++) {
            Log.d("DBR",results[i].barcodeText);
            array.pushMap(wrapResults(results[i]));
        }
    }

    return array;
}

private WritableNativeMap wrapResults(TextResult result) {
    WritableNativeMap map = new WritableNativeMap();
    map.putString("barcodeText",result.barcodeText);
    map.putString("barcodeFormat",result.barcodeFormatString);
    map.putInt("x1",result.localizationResult.resultPoints[0].x);
    map.putInt("x2",result.localizationResult.resultPoints[1].x);
    map.putInt("x3",result.localizationResult.resultPoints[2].x);
    map.putInt("x4",result.localizationResult.resultPoints[3].x);
    map.putInt("y1",result.localizationResult.resultPoints[0].y);
    map.putInt("y2",result.localizationResult.resultPoints[1].y);
    map.putInt("y3",result.localizationResult.resultPoints[2].y);
    map.putInt("y4",result.localizationResult.resultPoints[3].y);
    return map;
}

Open src/index.tsx and replace its content with the following to expose the native functions:

import type { Frame } from 'react-native-vision-camera'

export interface TextResult{
    barcodeText:string;
    barcodeFormat:string;
    x1:number;
    x2:number;
    x3:number;
    x4:number;
    y1:number;
    y2:number;
    y3:number;
    y4:number;
}

export interface DBRConfig{
  template?:string;
  license?:string;
}

export function decode(frame: Frame, config: DBRConfig): TextResult[] {
  'worklet'
  // @ts-ignore
  // eslint-disable-next-line no-undef
  return __decode(frame, config)
}

Use the Plugin in the Example Project

Next, we are going to use the plugin in the example project.

Update the Android Project

Vision Camera requires at least Android 5.0. You may need to update the example/android/build.gradle file.

ext {
    minSdkVersion = 21
    compileSdkVersion = 31
    targetSdkVersion = 31
}

To use cameras, add the following permission to the example/android/app/src/main/AndroidManifest.xml file:

<uses-permission android:name="android.permission.CAMERA" />

Add Vision Camera

Install react-native-vision-camera for the example project:

npm i react-native-vision-camera

Update the example/src/App.tsx file to use Vision Camera:

import * as React from 'react';
import { SafeAreaView, StyleSheet } from 'react-native';
import { Camera, useCameraDevices } from 'react-native-vision-camera';

export default function BarcodeScanner() {
  const [hasPermission, setHasPermission] = React.useState(false);
  const devices = useCameraDevices();
  const device = devices.back;

  React.useEffect(() => {
    (async () => {
      const status = await Camera.requestCameraPermission();
      setHasPermission(status === 'authorized');
    })();
  }, []);

  return (
      <SafeAreaView style={styles.container}>
        {device != null &&
        hasPermission && (
        <>
            <Camera
            style={StyleSheet.absoluteFill}
            device={device}
            isActive={true}
            />
        </>)}
      </SafeAreaView>
  );
}

const styles = StyleSheet.create({
  container: {
    flex: 1
  },
});

Enable the Barcode Reading Frame Processor

Install React Native Reanimated (REA) to enable the Frame Processor feature of Vision Camera.

We can follow its installation guide to install it.

Open example/babel.config.js. Add the following content:

 const path = require('path');
 const pak = require('../package.json');

 module.exports = {
   presets: ['module:metro-react-native-babel-preset'],
   plugins: [
     [
       'module-resolver',
       {
         extensions: ['.tsx', '.ts', '.js', '.json'],
         alias: {
           [pak.name]: path.join(__dirname, '..', pak.source),
         },
       },
     ],
+    [
+      'react-native-reanimated/plugin',
+      {
+        globals: ['__decode'],
+      },
+    ],
   ],
 };

+ import com.visioncameradynamsoftbarcodereader.VisionCameraDynamsoftBarcodeReaderPackage;  // <- add
 @Override
 protected List<ReactPackage> getPackages() {
   @SuppressWarnings("UnnecessaryLocalVariable")
   List<ReactPackage> packages = new PackageList(this).getPackages();
   //...
+  packages.add(new VisionCameraDynamsoftBarcodeReaderPackage()); // <- add
   return packages;
 }

Modify example/src/App.tsx to use the frame processor

Create a frame processor and add the frame processor props. The barcode results will be displayed using Text.

const [barcodeResults, setBarcodeResults] = React.useState([] as TextResult[]);
const frameProcessor = useFrameProcessor((frame) => {
    'worklet'
    const config:DBRConfig = {};        
    const results:TextResult[] = decode(frame,config)
    REA.runOnJS(setBarcodeResults)(results);
}, [])
return (
    <SafeAreaView style={styles.container}>
      {device != null &&
      hasPermission && (
      <>
          <Camera
          style={StyleSheet.absoluteFill}
          device={device}
          isActive={true}
          frameProcessor={frameProcessor}
          frameProcessorFps={5}
          />
          {barcodeResults.map((barcode, idx) => (
          <Text key={idx} style={styles.barcodeTextURL}>
              {barcode.barcodeFormat +": "+ barcode.barcodeText}
          </Text>
          ))}
      </>)}
    </SafeAreaView>
);

The complete file:

import * as React from 'react';
import { SafeAreaView, StyleSheet, Text } from 'react-native';
import { Camera, useCameraDevices, useFrameProcessor } from 'react-native-vision-camera';
import { DBRConfig, decode, TextResult } from 'vision-camera-dynamsoft-barcode-reader';
import * as REA from 'react-native-reanimated';

export default function BarcodeScanner() {
  const [hasPermission, setHasPermission] = React.useState(false);
  const [barcodeResults, setBarcodeResults] = React.useState([] as TextResult[]);
  const devices = useCameraDevices();
  const device = devices.back;
  const frameProcessor = useFrameProcessor((frame) => {
    'worklet'
    const config:DBRConfig = {};
    const results:TextResult[] = decode(frame,config)
    REA.runOnJS(setBarcodeResults)(results);
  }, [])

  React.useEffect(() => {
    (async () => {
      const status = await Camera.requestCameraPermission();
      setHasPermission(status === 'authorized');
    })();
  }, []);

  return (
      <SafeAreaView style={styles.container}>
        {device != null &&
        hasPermission && (
        <>
            <Camera
            style={StyleSheet.absoluteFill}
            device={device}
            isActive={true}
            frameProcessor={frameProcessor}
            frameProcessorFps={5}
            />
            {barcodeResults.map((barcode, idx) => (
            <Text key={idx} style={styles.barcodeTextURL}>
                {barcode.barcodeFormat +": "+ barcode.barcodeText}
            </Text>
            ))}
        </>)}
      </SafeAreaView>
  );
}

const styles = StyleSheet.create({
  container: {
    flex: 1
  },
  barcodeTextURL: {
    fontSize: 20,
    color: 'white',
    fontWeight: 'bold',
  },
});

Source Code

You can check out the source code for more details.

https://github.com/tony-xlh/vision-camera-dynamsoft-barcode-reader

LANGUAGES

PLATFORMS

FEATURED

Build a React Native Vision Camera Frame Processor Plugin to Scan Barcodes for Android

Other React Native Vision Camera Frame Processor Plugins

Getting started with Dynamsoft Barcode Reader

Building the React Native Vision Camera Frame Processor Plugin of Barcode Reader for Android

New Project

Implement the Plugin

Use the Plugin in the Example Project

Update the Android Project

Add Vision Camera

Enable the Barcode Reading Frame Processor

Source Code

Search Blog Posts

POPULAR TOPICS

LANGUAGES

PLATFORMS

FEATURED

Build a React Native Vision Camera Frame Processor Plugin to Scan Barcodes for Android

Other React Native Vision Camera Frame Processor Plugins

Getting started with Dynamsoft Barcode Reader

Building the React Native Vision Camera Frame Processor Plugin of Barcode Reader for Android

New Project

Implement the Plugin

Use the Plugin in the Example Project

Update the Android Project

Add Vision Camera

Enable the Barcode Reading Frame Processor

Source Code

Search Blog Posts

POPULAR TOPICS

Useful Links