How to Build a NativeScript Plugin for Camera Preview

Jan 09, 2023

NativeScript is an open-source native runtime for building native mobile apps with JavaScript. We can easily call native APIs using JavaScript.

A NativeScript plugin is any npm package that exposes a native API via JavaScript which is made convenient for projects to use.

In this article, we are going to create a NativeScript plugin for Dynamsoft Camera Enhancer which can be used for camera preview.

This article is Part 1 in a 2-Part Series.

Part 1 - How to Build a NativeScript Plugin for Camera Preview
Part 2 - How to Build a NativeScript Plugin for Barcode Reading

Build a Camera Preview NativeScript Plugin

Let’s do this in steps.

New Plugin Project

It is recommended to use the plugin workspace seed to create new plugins.

Hit the “Use this template” button in the seed’s page.
Follow GitHub instructions (set repo name, visibility, description) & clone your new repo. Here, we name it nativescript-dynamsoft-capture-vision.
Setup workspace: npm run setup.
Configure your npm scope: npm run config.

Then, we can add a new camera enhancer package:

npm run add

Follow the prompts to complete the setup. Here, we use nativescript-dynamsoft-camera-enhancer as the package’s name.

Write Definitions

Let’s write the definitions first.

Open index.d.ts to write function definitions.

The plugin will have the following functions to control the camera:

import { CameraEnhancerCommon } from './common';

export declare class CameraEnhancer extends CameraEnhancerCommon {
    captureFrame(): any;
    captureFrameAsBase64(): string;
    getAllCameras(): string[];
    getSelectedCamera(): string;
    getCameraEnhancer(): any;
    getResolution(): Resolution;
    /**
     * supported resolutions: 640x480, 1280x720, 1920x1080, 3840x2160 or pass empty for auto
     */
    setResolution(resolution:Resolution): void;
    setZoom(factor:number);
    getMaxZoomFactor(): number;
    open(): void;
    close(): void;
}
export * from "./common";

Open common.ts to define the interfaces, props and classes shared by both Android and iOS platforms.

Make the common class extend View as it provides a camera view.

export class CameraEnhancerCommon extends View {
        
}

Define a resolution interface.

export interface Resolution {
  width: number;
  height: number;
}

Define props to update the state of the camera preview.

export const activeProperty = new Property<CameraEnhancerCommon, boolean>({
    name: 'active'
})

export const torchProperty = new Property<CameraEnhancerCommon, boolean>({
    name: 'torch',
    defaultValue: false,
    valueConverter: booleanConverter,
})

export const cameraIDProperty = new Property<CameraEnhancerCommon, string>({
    name: 'cameraID'
})

//register the property with the view component class
activeProperty.register(CameraEnhancerCommon)
torchProperty.register(CameraEnhancerCommon)
cameraIDProperty.register(CameraEnhancerCommon)

Android Implementation

Add the Dependency

Create a include.gradle file under platforms\android to add the dependency of Dynamsoft Camera Enhancer.

allprojects {
    repositories {
        maven { url "https://download2.dynamsoft.com/maven/aar" }
    }
}

dependencies {
    implementation 'com.dynamsoft:dynamsoftcameraenhancer:2.3.10@aar'
}

Generate Typings

Download the aar file of Dynamsoft Camera Enhancer from here.
Use the Android d.ts Generator to generate TypeScript definitions for the aar file. Put the generated android.d.ts file under typings.

Implement the Camera Enhancer in TypeScript

Open index.android.ts to call the native APIs using TypeScript.

The complete file is like the following:

export class CameraEnhancer extends CameraEnhancerCommon {
  cameraView: com.dynamsoft.dce.DCECameraView;
  dce:com.dynamsoft.dce.CameraEnhancer;

  constructor(){
    super();
  }

  createNativeView() {
    let context = Utils.android.getApplicationContext();
    this.dce = new com.dynamsoft.dce.CameraEnhancer(Application.android.foregroundActivity);
    this.cameraView = new com.dynamsoft.dce.DCECameraView(context);
    this.dce.setCameraView(this.cameraView);
    return this.cameraView;
  }

  initNativeView() {
    
  }

  captureFrame():com.dynamsoft.dce.DCEFrame{
    return this.dce.getFrameFromBuffer(true);
  }

  captureFrameAsBase64():string{
    let frame = this.dce.getFrameFromBuffer(true);
    let bitmap = frame.toBitmap();
    return this.bitmap2Base64(bitmap);
  }

  getAllCameras():string[]{
    let array = [];
    let cameras = this.dce.getAllCameras();
    for (let index = 0; index < cameras.length; index++) {
        const camera = cameras[index];
        array.push(camera);
    }
    return array;
  }

  getSelectedCamera():string{
    return this.dce.getSelectedCamera();
  }

  getResolution():Resolution {
    let res:string = this.dce.getResolution().toString();
    let width = parseInt(res.split("x")[0]);
    let height = parseInt(res.split("x")[1]);
    return {width:width,height:height};
  }

  setResolution(res:Resolution) {
    let targetRes:com.dynamsoft.dce.EnumResolution;
    if (res.width === 640 && res.height === 480) {
      targetRes = com.dynamsoft.dce.EnumResolution.RESOLUTION_480P;
    }else if (res.width === 1280 && res.height === 720) {
      targetRes = com.dynamsoft.dce.EnumResolution.RESOLUTION_720P;
    }else if (res.width === 1920 && res.height === 1080) {
      targetRes = com.dynamsoft.dce.EnumResolution.RESOLUTION_1080P;
    }else if (res.width === 3840 && res.height === 2160) {
      targetRes = com.dynamsoft.dce.EnumResolution.RESOLUTION_4K;
    }else {
      targetRes = com.dynamsoft.dce.EnumResolution.RESOLUTION_AUTO;
    }
    this.dce.setResolution(targetRes);
  }

  setZoom(factor:number){
    this.dce.setZoom(factor);
  }

  getMaxZoomFactor():number{
    return this.dce.getMaxZoomFactor();
  }

  open(){
    this.dce.open();
  }

  close(){
    this.dce.close();
  }

  getCameraEnhancer(): com.dynamsoft.dce.CameraEnhancer {
    return this.dce;
  }

  bitmap2Base64(bitmap:android.graphics.Bitmap):string{
    let outputStream = new java.io.ByteArrayOutputStream();
    bitmap.compress(android.graphics.Bitmap.CompressFormat.JPEG, 100, outputStream);
    return android.util.Base64.encodeToString(outputStream.toByteArray(), android.util.Base64.DEFAULT);
  }

  [activeProperty.setNative](value: boolean) {
    if (value === true) {
      this.dce.open();
    }else{
      this.dce.close();
    }
  }

  [torchProperty.setNative](value: boolean) {
    if (value === true) {
      this.dce.turnOnTorch();
    }else{
      this.dce.turnOffTorch();
    }
  }

  [cameraIDProperty.setNative](value: string) {
    if (value) {
      this.dce.selectCamera(value);
    }
  }
}

iOS Implementation

Add the Dependency

Create a Podfile file under platforms\ios to add the Dynamsoft Camera Enhancer dependency.

platform :ios, '9.0'
pod 'DynamsoftCameraEnhancer','2.3.10'

Add Camera Permission

Create a Info.plist file under platforms\ios for camera permission:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
  <dict>
    <key>NSCameraUsageDescription</key>
    <string/>
  </dict>
</plist>

Generate Typings

Run the following to generate the TypeScript definitions for iOS. Then, copy the objc!DynamsoftCameraEnhancer.d.ts to packages\nativescript-dynamsoft-camera-enhancer\typings.

cd apps/demo
ns typings ios

Implement the Camera Enhancer in TypeScript

Open index.ios.ts to call the native APIs using TypeScript.

The complete file is like the following:

export class CameraEnhancer extends CameraEnhancerCommon {
  dce:DynamsoftCameraEnhancer
  cameraView:DCECameraView;

  createNativeView() {
    this.cameraView = DCECameraView.alloc().init();
    this.dce = DynamsoftCameraEnhancer.alloc().init();
    this.dce.dceCameraView = this.cameraView;
    return this.cameraView;
  }

  initNativeView() {
    
  }

  captureFrame() {
    return this.dce.getFrameFromBuffer(true);
  }

  captureFrameAsBase64() {
    let frame = this.dce.getFrameFromBuffer(true);
    let image = frame.toUIImage();
    return this.UIImage2Base64(image);
  }

  getAllCameras():string[] {
    let array = [];
    let cameras = this.dce.getAllCameras();
    for (let index = 0; index < cameras.count; index++) {
        const camera = cameras[index];
        array.push(camera);
    }
    return array;
  }

  getSelectedCamera():string {
    return this.dce.getSelectedCamera();
  }

  getCameraEnhancer(): DynamsoftCameraEnhancer {
    return this.dce;
  }

  UIImage2Base64(image:UIImage):string{
    let data = UIImageJPEGRepresentation(image,100);
    return data.base64Encoding();
  }

  open(){
    this.dce.open();
  }

  close(){
    this.dce.close();
  }

  getResolution():Resolution {
    let res = this.dce.getResolution();
    let width = parseInt(res.split("x")[0]);
    let height = parseInt(res.split("x")[1]);
    return {width:width,height:height};
  }

  setResolution(res:Resolution) {
    let targetRes:EnumResolution;
    if (res.width === 640 && res.height === 480) {
      targetRes = EnumResolution.ESOLUTION_480P;
    }else if (res.width === 1280 && res.height === 720) {
      targetRes = EnumResolution.ESOLUTION_720P;
    }else if (res.width === 1920 && res.height === 1080) {
      targetRes = EnumResolution.ESOLUTION_1080P;
    }else if (res.width === 3840 && res.height === 2160) {
      targetRes = EnumResolution.ESOLUTION_4K;
    }else {
      targetRes = EnumResolution.ESOLUTION_AUTO;
    }
    this.dce.setResolution(targetRes);
  }

  setZoom(factor:number){
    this.dce.setZoom(factor);
  }

  getMaxZoomFactor():number{
    return this.dce.getMaxZoomFactor();
  }

  [activeProperty.setNative](value: boolean) {
    if (value === true) {
      this.dce.open();
    }else{
      this.dce.close();
    }
  }

  [torchProperty.setNative](value: boolean) {
    if (value === true) {
      this.dce.turnOnTorch();
    }else{
      this.dce.turnOffTorch();
    }
  }

  [cameraIDProperty.setNative](value: string) {
    if (value) {
      this.dce.selectCameraError(value);
    }
  }
}

Update the Demo to Use the Plugin

Let’s update the plain TypeScript demo to use the plugin.

Open apps/demo/src/plugin-demos/nativescript-dynamsoft-camera-enhancer.xml to add the camera preview view. Here we use the GridLayout to make the view contain the whole space and add four buttons to test the functions.

<Page xmlns="http://schemas.nativescript.org/tns.xsd" 
    xmlns:dce="nativescript-dynamsoft-camera-enhancer"
    navigatingTo="navigatingTo" class="page">
    <GridLayout rows="*, auto, auto">
        <dce:CameraEnhancer 
        loaded="" 
        rowSpan="3" 
        active=""
        cameraID=""
        torch=""></dce:CameraEnhancer>
        <StackLayout row="1">
            <Button class="btn btn-primary" text="Switch Torch" tap=""></Button>
            <Button class="btn btn-primary" text="Switch Camera" tap=""></Button>
            <Button class="btn btn-primary" text="Toggle Zoom" tap=""></Button>
            <Button class="btn btn-primary" text="Capture Frame" tap=""></Button>
        </StackLayout>
    </GridLayout>
</Page>

Open apps/demo/src/plugin-demos/nativescript-dynamsoft-camera-enhancer.ts to add relevant functions.

Define props used by the camera preview view.

isActive: boolean = true;
desiredTorchStatus:boolean = false;
desiredCamera:string = "";

When the camera view is loaded, get the instance of the camera enhancer and set its resolution to 480P. Meanwhile, we have to handle the lifecycle events for Android.

dce:CameraEnhancer;
dceLoaded(args: EventData) {
  this.dce = <CameraEnhancer>args.object;
  this.registerLifeCycleEvents();
  let targetRes:Resolution = {width:640,height:480};
  this.dce.setResolution(targetRes);
}
      
registerLifeCycleEvents(){
  if (global.isAndroid) {
    let pThis = this;
    Application.android.on(AndroidApplication.activityPausedEvent, function (args: AndroidActivityBundleEventData) {
      console.log("paused");
      if (pThis.dce && pThis.isActive) {
        console.log("close camera");
        pThis.dce.close();
      }
    });
       
   Application.android.on(AndroidApplication.activityResumedEvent, function (args: AndroidActivityBundleEventData) {
     console.log("resumed");
     if (pThis.dce && pThis.isActive === true) {
       console.log("restart camera");
       pThis.dce.open();
     }
   });
  }
}

Add a function for switching the camera.

cameras:string[]|undefined;
onSwitchCamera(args: EventData) {
  if (this.dce) {
    if (!this.cameras) {
      this.cameras = this.dce.getAllCameras();
    }
    const selectedCamera = this.dce.getSelectedCamera();
    let nextIndex = this.cameras.indexOf(selectedCamera) + 1;
    if (nextIndex >= this.cameras.length) {
      nextIndex = 0;
    }
    const nextCamera = this.cameras[nextIndex];
    if (nextCamera != selectedCamera) {
      this.set("desiredCamera",nextCamera);
    }
  }
}

Add a function for switching the torch.

onSwitchTorch(args: EventData) {
  this.set("desiredTorchStatus",!this.desiredTorchStatus);
}

Add a function for capturing a frame and output its size.

onCaptureFrame(args: EventData) {
  if (this.dce) {
    let width,height;
    const frame = this.dce.captureFrame();
    if (global.isAndroid) {
      width = frame.getWidth();
      height = frame.getHeight();
    }else{
      width = frame.width;
      height = frame.height;
    }
    alert("Captured a "+width+"x"+height+" sized frame");
  }else{
    alert("dce undefined");
  }
}

Add a function to set zoom.

zoomed:boolean = false;
onToggleZoom(args: EventData) {
  if (this.dce) {
    if (this.zoomed) {
      this.dce.setZoom(1.0);
      this.zoomed = false;
    }else{
      this.dce.setZoom(2.0);
      this.zoomed = true;
    }
  }
}

All right, we’ve finished writing the plugin and the demo. We can run npm run start to run the app on Android or iOS devices for a test.

iOS Screenshot:

iOS camera

Source Code

https://github.com/tony-xlh/nativescript-dynamsoft-capture-vision

LANGUAGES

PLATFORMS

FEATURED

How to Build a NativeScript Plugin for Camera Preview

Build a Camera Preview NativeScript Plugin

New Plugin Project

Write Definitions

Android Implementation

Add the Dependency

Generate Typings

Implement the Camera Enhancer in TypeScript

iOS Implementation

Add the Dependency

Add Camera Permission

Generate Typings

Implement the Camera Enhancer in TypeScript

Update the Demo to Use the Plugin

Source Code