How to Scan MRZ in an Expo App

MRZ (machine-readable zone) is a section on ID documents. It is designed to be read by a machine to get the document owner’s info, like name, age, sex and nationality.

In this article, we are going to build an expo application to scan MRZ. Dynamsoft Label Recognizer is used to provide the OCR functionality.

PS: Expo is a platform which makes it easy to develop cross-platform mobile applications based on React Native.

Demo video:

New Project

  1. Create a new Expo project:

    npx create-expo-app MRZScanner --template blank

  2. Add camera permissions in app.json:

    "ios": {
  "infoPlist": {
    "NSCameraUsageDescription": "This app uses the camera to scan barcodes."
  }
},
"android": {
  "permissions": ["android.permission.CAMERA","android.permission.INTERNET"]
}

  3. Add dependencies:

    npx expo install react-native-webview expo-camera react-native-safe-area-context

Start the MRZ Scanner in Webview

Here, we are going to run the JavaScript version of Dynamsoft Label Recognizer in react-native-webview to scan MRZ. First, we need to write a web MRZ scanner and then, we need to create a component in React Native to use it.

Web MRZ Scanner

  1. Create a new HTML file with the following template:

    <!DOCTYPE html>
<html lang="en">
  <head>
    <meta charset="UTF-8">
    <meta
      name="viewport"
      content="viewport-fit=cover, width=device-width, initial-scale=1.0, minimum-scale=1.0, maximum-scale=1.0, user-scalable=no"
    />
    <title>MRZ Scanner in React-Native-WebView</title>
    <style>
    </style>
  </head>
  <body>
    <script>
    </script>
  </body>
</html>

  2. Include libraries. Use Dynamsoft Camera Enhancer to access the camera and Dynamsoft Label Recognizer to recognize MRZ.

    <script src="https://cdn.jsdelivr.net/npm/dynamsoft-camera-enhancer@3.3.4/dist/dce.js"></script>
<script src="https://cdn.jsdelivr.net/npm/dynamsoft-label-recognizer@2.2.31/dist/dlr.js"></script>

  3. Start the camera and bind the camera view in a container. An instance of camera enhancer is created to do this.

    HTML:

    <div class="scanner"></div>

    JavaScript:

    let enhancer;
window.onload = function(){
  init();
}
   
function startScan(){
  enhancer.open(true); //start the camera
}
   
async function init(){
  enhancer = await Dynamsoft.DCE.CameraEnhancer.createInstance();
  await enhancer.setUIElement(Dynamsoft.DCE.CameraEnhancer.defaultUIElementURL);
  let container = document.getElementsByClassName("scanner")[0];
  container.appendChild(enhancer.getUIElement());
}

  4. Initialize an instance of label recognizer and update its runtime settings to recognize MRZ. A license is needed. You can apply for one here.

    // use a one-day trial
Dynamsoft.DLR.LabelRecognizer.license = 'DLS2eyJoYW5kc2hha2VDb2RlIjoiMjAwMDAxLTE2NDk4Mjk3OTI2MzUiLCJvcmdhbml6YXRpb25JRCI6IjIwMDAwMSIsInNlc3Npb25QYXNzd29yZCI6IndTcGR6Vm05WDJrcEQ5YUoifQ==';
let recognizer = await Dynamsoft.DLR.LabelRecognizer.createInstance();
await recognizer.updateRuntimeSettingsFromString("mrz");

  5. Set up a scan region so that only a small part of the camera frame will be used for recognition to improve speed.

    enhancer.setScanRegion({
  regionLeft:0,
  regionTop:40,
  regionRight:100,
  regionBottom:60,
  regionMeasuredByPercentage: 1
})

  6. When the camera is opened, start an interval to recognize MRZ from camera frames.

    let interval;
let processing;
   
enhancer.on("played", (playCallbackInfo) => {
  //"camera started"
  startProcessingLoop();
});
   
function startProcessingLoop(){
  stopProcessingLoop();
  interval = setInterval(captureAndProcess,100);
}

function stopProcessingLoop(){
  if (interval) {
    clearInterval(interval);
    interval = undefined;
  }
  processing = false;
}

async function captureAndProcess() {
  if (!enhancer || !recognizer) {
    return
  }
  if (enhancer.isOpen() === false) {
    return;
  }
  if (processing == true) {
    return;
  }
  processing = true; // set processing to true so that the next frame will be skipped if the processing has not completed.
  let frame = enhancer.getFrame();
  if (frame) {  
    //get the OCR results
    let results = await recognizer.recognize(frame);
    processing = false;
  }
};

  7. Pass the MRZ string to React Native.

    if (results.length>0) {
  if (window.ReactNativeWebView) {
    window.ReactNativeWebView.postMessage(getMRZString(results[0]));
  }
}   
   
function getMRZString(result){
  let s = "";
  for (let index = 0; index < result.lineResults.length; index++) {
    const lineResult = result.lineResults[index];
    s = s + lineResult.text;
    if (index != result.lineResults.length - 1) {
      s = s + "\n";
    }
  }
  return s;
}

The scanner is done. We can deploy it on an online platform like GitHub Pages for further usage.

MRZ Scanner Component in React Native

Next, let’s use the web MRZ scanner in react native webview and wrap it as a component.

  1. Create a new MRZScanner.js file.

    import { StyleSheet, View } from 'react-native';
import React from 'react';

export default function MRZScanner(props) {
    return <View />;
  }
}

const styles = StyleSheet.create({
  container: {
    flex: 1,
    backgroundColor: '#fff',
    alignItems: 'center',
    justifyContent: 'center',
  },
});

  2. Ask for camera permission after the component is mounted.

    import { Camera } from 'expo-camera';
   
//...
   
const [hasPermission, setHasPermission] = useState(null);
useEffect(() => {
  (async () => {
    const { status } = await Camera.requestCameraPermissionsAsync();
    setHasPermission(status === "granted");
  })();
}, []);

  3. Use react-native-webview to load the web MRZ scanner if the camera permission is granted. The messages sent from the web app are captured in the onMessage props.

    if (hasPermission === null) {
  return <View />;
}
if (hasPermission === false) {
  return <Text>No access to camera</Text>;
}
if (hasPermission) {
  return (
    <WebView
      style={styles.container}
      allowsInlineMediaPlayback={true}
      mediaPlaybackRequiresUserAction={false}
      onMessage={(event) => {
        console.log(event);
        if (event.nativeEvent.data === "close") {
          if (props.onClosed) {
            props.onClosed();
          }
        }else{
          if (event.nativeEvent.data) {
            if (props.onScanned) {
              props.onScanned(event.nativeEvent.data);
            }
          }
        }
      }}
      source={{ uri: 'https://tony-xlh.github.io/Vanilla-JS-MRZ-Scanner-Demos/React-Native-Webview/' }}
    />
  );
}

Use the MRZ Scanner Component

In app.js, add a button to bring up the MRZ scanner component to scan MRZ.

import { Alert, Button, Text, View, StyleSheet } from 'react-native';
import React,{ useState } from 'react';
import { StatusBar } from 'expo-status-bar';
import MRZScanner from './MRZScanner';
import { SafeAreaView, SafeAreaProvider  } from 'react-native-safe-area-context';

export default function App() {
  const [scanning,setScanning] = useState(false);

  const showResults = (result) => {
    if (result) {
      Alert.alert("MRZ Scanner",result);
      setScanning(false);
    }
  }

  return (
    <SafeAreaProvider>
      <SafeAreaView style={styles.container}>
        {scanning &&
          <MRZScanner
            onScanned={(results)=>showResults(results)}
            onClosed={()=>setScanning(false)}
          ></MRZScanner>
        }
        {!scanning &&
          <View style={{alignItems:'center'}}>
            <Text style={styles.title}>
                Dynamsoft Label Recognizer Demo
              </Text>
            <Button title='Start MRZ Scanner' onPress={() => setScanning(true)}></Button>
          </View>
        }
        <StatusBar style="auto"/>
      </SafeAreaView>
    </SafeAreaProvider>
  );
}

const styles = StyleSheet.create({
  container: {
    flex: 1,
  },
  title: {
    textAlign: 'center',
    marginVertical: 8,
  },
});

Source Code

All right, we have covered the key parts of the expo MRZ scanner demo. Check out the source code to have a try:

https://github.com/tony-xlh/expo-mrz-scanner