How to Scan MRZ in a Next.js Application

MRZ stands for machine-readable zone, which is usually at the bottom of the identity page at the beginning of a passport or ID card.1 It can be read by a computing device with a camera to get information like document type, name, document number, nationality, date of birth, sex, and document expiration date.

In this article, we are going to build a web application in Next.js to scan MRZ via cameras. Dynamsoft Label Recognizer is used as the OCR engine.

Screenshot of the demo:

demo

You can find the online demo here.

New Project

Create a new Next.js project:

npx create-next-app@latest

Install Dependencies

Install Dynamsoft Label Recognizer and related libraries.

npm i dynamsoft-core@3.2.30 dynamsoft-camera-enhancer@4.0.2 dynamsoft-capture-vision-router@2.2.30 dynamsoft-code-parser@2.2.10 dynamsoft-label-recognizer@3.2.30 dynamsoft-license@3.2.21 dynamsoft-utility@1.2.20

Configure the SDK

Create a new file named configure.ts with the following content. A license is needed. You can apply for your license here.

import "dynamsoft-license";
import "dynamsoft-capture-vision-router";
import "dynamsoft-label-recognizer";
import { LicenseManager } from "dynamsoft-license";
import { CoreModule } from "dynamsoft-core";

if (CoreModule.isModuleLoaded("dlr") === false) {
  /** LICENSE ALERT - README
   * To use the library, you need to first specify a license key using the API "initLicense()" as shown below.
   */
  LicenseManager.initLicense("LICENSE-KEY");

  CoreModule.engineResourcePaths = {
    std: 'https://cdn.jsdelivr.net/npm/dynamsoft-capture-vision-std@1.2.10/dist/',
    dip: 'https://cdn.jsdelivr.net/npm/dynamsoft-image-processing@2.2.30/dist/',
    core: "https://cdn.jsdelivr.net/npm/dynamsoft-core@3.2.30/dist/",
    license: "https://cdn.jsdelivr.net/npm/dynamsoft-license@3.2.21/dist/",
    cvr: "https://cdn.jsdelivr.net/npm/dynamsoft-capture-vision-router@2.2.30/dist/",
    dlr: "https://cdn.jsdelivr.net/npm/dynamsoft-label-recognizer@3.2.30/dist/",
    dce: "https://cdn.jsdelivr.net/npm/dynamsoft-camera-enhancer@4.0.2/dist/",
    dnn: 'https://cdn.jsdelivr.net/npm/dynamsoft-capture-vision-dnn@1.0.20/dist/',
    dlrData: 'https://cdn.jsdelivr.net/npm/dynamsoft-label-recognizer-data@1.0.10/dist/',
    utility: 'https://cdn.jsdelivr.net/npm/dynamsoft-utility@1.2.20/dist/'
  };
}

In page.tsx, import the configure file to run the configuration:

import "../configure"

Create an MRZ Scanner Component

  1. Create a new component under src/components/MRZScanner.tsx with the following template.

    import { RecognizedTextLinesResult } from "dynamsoft-label-recognizer";
    import { MutableRefObject, useEffect, useRef } from "react";
    import React from "react";
    
    export interface MRZScannerProps{
      isScanning?:boolean;
      onInitialized?:()=>void;
      onScanned?:(results:RecognizedTextLinesResult)=>void;
    }
    
    const MRZScanner: React.FC<MRZScannerProps> = (props:MRZScannerProps) => {
      const container:MutableRefObject<HTMLDivElement|null>  = useRef(null);
      return (
        <div ref={container} style={{width:"100%",height:"100%"}}></div>
      )
    }
    
    export default MRZScanner;
    
  2. Add the following effect to initialize relevant libraries when the component is mounted.

    const initialized = useRef(false);
    useEffect(()=>{
      init();
    },[])
       
    const init = async () => {
      if (initialized.current == false) {
        initialized.current = true;
        await initCameraEnhancer();
        await initLabelRecognizer();
        if (props.onInitialized) {
          props.onInitialized();
        }
        if (props.isScanning === true) {
          startScanning();
        }
      }
    }
    

    The function to initialize the camera enhancer.

    const initCameraEnhancer = async () => {
      const cameraView = await CameraView.createInstance();
      cameraEnhancer.current = await CameraEnhancer.createInstance(cameraView);
      container.current!.append(cameraView.getUIElement());
    }
    

    The function to initialize the label recognizer.

    const initLabelRecognizer = async () => {
      // Preload "LabelRecogznier" module for recognizing text. It will save time on the initial recognizing by skipping the module loading.
      await CoreModule.loadWasm(["DLR"]);
      await LabelRecognizerModule.loadRecognitionData("MRZ");
      router.current = await CaptureVisionRouter.createInstance();
      router.current.initSettings("/template.json");
      // Define a callback for results.
      const resultReceiver = new CapturedResultReceiver();
      resultReceiver.onRecognizedTextLinesReceived = (result: RecognizedTextLinesResult) => {
        console.log(result);
        if (props.onScanned) {
          props.onScanned(result);
        }
      };
      router.current.addResultReceiver(resultReceiver);
      if (cameraEnhancer.current) {
        router.current.setInput(cameraEnhancer.current);
      }
    }
    

    We have to set the runtime settings for capture vision router to read MRZ with the following line of code. (PS: label recognizer is called using capture vision router)

    router.current.initSettings("/template.json");
    

    The template.json with the following content is put under public.

    {
      "CaptureVisionTemplates": [
        {
          "Name": "ReadMRZ",
          "OutputOriginalImage": 0,
          "ImageROIProcessingNameArray": [
            "roi-mrz"
          ],
          "Timeout": 2000
        }
      ],
      "TargetROIDefOptions": [
        {
          "Name": "roi-mrz",
          "TaskSettingNameArray": [
            "task-mrz"
          ]
        }
      ],
      "TextLineSpecificationOptions": [
        {
          "Name": "tls-mrz-passport",
          "BaseTextLineSpecificationName": "tls-base",
          "StringLengthRange": [ 44, 44 ],
          "OutputResults": 1,
          "ExpectedGroupsCount": 1,
          "ConcatResults": 1,
          "ConcatSeparator": "\n",
          "SubGroups": [
            {
              "StringRegExPattern": "(P[A-Z<][A-Z<]{3}[A-Z<]{39}){(44)}",
              "StringLengthRange": [ 44, 44 ],
              "BaseTextLineSpecificationName": "tls-base"
            },
            {
              "StringRegExPattern": "([A-Z0-9<]{9}[0-9][A-Z<]{3}[0-9]{2}[(01-12)][(01-31)][0-9][MF<][0-9]{2}[(01-12)][(01-31)][0-9][A-Z0-9<]{14}[0-9<][0-9]){(44)}",
              "StringLengthRange": [ 44, 44 ],
              "BaseTextLineSpecificationName": "tls-base"
            }
          ]
        },
        {
          "Name": "tls-mrz-visa-td3",
          "BaseTextLineSpecificationName": "tls-base",
          "StringLengthRange": [ 44, 44 ],
          "OutputResults": 1,
          "ExpectedGroupsCount": 1,
          "ConcatResults": 1,
          "ConcatSeparator": "\n",
          "SubGroups": [
            {
              "StringRegExPattern": "(V[A-Z<][A-Z<]{3}[A-Z<]{39}){(44)}",
              "StringLengthRange": [ 44, 44 ],
              "BaseTextLineSpecificationName": "tls-base"
            },
            {
              "StringRegExPattern": "([A-Z0-9<]{9}[0-9][A-Z<]{3}[0-9]{2}[(01-12)][(01-31)][0-9][MF<][0-9]{2}[(01-12)][(01-31)][0-9][A-Z0-9<]{14}[A-Z0-9<]{2}){(44)}",
              "StringLengthRange": [ 44, 44 ],
              "BaseTextLineSpecificationName": "tls-base"
            }
          ]
        },
        {
          "Name": "tls-mrz-visa-td2",
          "BaseTextLineSpecificationName": "tls-base",
          "StringLengthRange": [ 36, 36 ],
          "OutputResults": 1,
          "ExpectedGroupsCount": 1,
          "ConcatResults": 1,
          "ConcatSeparator": "\n",
          "SubGroups": [
            {
              "StringRegExPattern": "(V[A-Z<][A-Z<]{3}[A-Z<]{31}){(36)}",
              "StringLengthRange": [ 36, 36 ],
              "BaseTextLineSpecificationName": "tls-base"
            },
            {
              "StringRegExPattern": "([A-Z0-9<]{9}[0-9][A-Z<]{3}[0-9]{2}[(01-12)][(01-31)][0-9][MF<][0-9]{2}[(01-12)][(01-31)][0-9][A-Z0-9<]{8}){(36)}",
              "StringLengthRange": [ 36, 36 ],
              "BaseTextLineSpecificationName": "tls-base"
            }
          ]
        },
        {
          "Name": "tls-mrz-id-td2",
          "BaseTextLineSpecificationName": "tls-base",
          "StringLengthRange": [ 36, 36 ],
          "OutputResults": 1,
          "ExpectedGroupsCount": 1,
          "ConcatResults": 1,
          "ConcatSeparator": "\n",
          "SubGroups": [
            {
              "StringRegExPattern": "([ACI][A-Z<][A-Z<]{3}[A-Z<]{31}){(36)}",
              "StringLengthRange": [ 36, 36 ],
              "BaseTextLineSpecificationName": "tls-base"
            },
            {
              "StringRegExPattern": "([A-Z0-9<]{9}[0-9][A-Z<]{3}[0-9]{2}[(01-12)][(01-31)][0-9][MF<][0-9]{2}[(01-12)][(01-31)][0-9][A-Z0-9<]{8}){(36)}",
              "StringLengthRange": [ 36, 36 ],
              "BaseTextLineSpecificationName": "tls-base"
            }
          ]
        },
        {
          "Name": "tls-mrz-id-td1",
          "BaseTextLineSpecificationName": "tls-base",
          "StringLengthRange": [ 30, 30 ],
          "OutputResults": 1,
          "ExpectedGroupsCount": 1,
          "ConcatResults": 1,
          "ConcatSeparator": "\n",
          "SubGroups": [
            {
              "StringRegExPattern": "([ACI][A-Z<][A-Z<]{3}[A-Z0-9<]{9}[0-9][A-Z0-9<]{15}){(30)}",
              "StringLengthRange": [ 30, 30 ],
              "BaseTextLineSpecificationName": "tls-base"
            },
            {
              "StringRegExPattern": "([0-9]{2}[(01-12)][(01-31)][0-9][MF<][0-9]{2}[(01-12)][(01-31)][0-9][A-Z<]{3}[A-Z0-9<]{11}[0-9]){(30)}",
              "StringLengthRange": [ 30, 30 ],
              "BaseTextLineSpecificationName": "tls-base"
            },
            {
              "StringRegExPattern": "([A-Z<]{30}){(30)}",
              "StringLengthRange": [ 30, 30 ],
              "BaseTextLineSpecificationName": "tls-base"
            }
          ]
        },
        {
          "Name": "tls-base",
          "CharacterModelName": "MRZ",
          "CharHeightRange": [ 5, 1000, 1 ],
          "BinarizationModes": [
            {
              "BlockSizeX": 30,
              "BlockSizeY": 30,
              "Mode": "BM_LOCAL_BLOCK",
              "EnableFillBinaryVacancy": 0,
              "ThresholdCompensation": 15
            }
          ],
          "ConfusableCharactersCorrection": {
            "ConfusableCharacters": [
              [ "0", "O" ],
              [ "1", "I" ],
              [ "5", "S" ]
            ],
            "FontNameArray": [ "OCR_B" ]
          }
        }
      ],
      "LabelRecognizerTaskSettingOptions": [
        {
          "Name": "task-mrz",
          "ConfusableCharactersPath": "ConfusableChars.data",
          "TextLineSpecificationNameArray": [ "tls-mrz-passport", "tls-mrz-visa-td3", "tls-mrz-id-td1", "tls-mrz-id-td2", "tls-mrz-visa-td2" ],
          "SectionImageParameterArray": [
            {
              "Section": "ST_REGION_PREDETECTION",
              "ImageParameterName": "ip-mrz"
            },
            {
              "Section": "ST_TEXT_LINE_LOCALIZATION",
              "ImageParameterName": "ip-mrz"
            },
            {
              "Section": "ST_TEXT_LINE_RECOGNITION",
              "ImageParameterName": "ip-mrz"
            }
          ]
        }
      ],
      "CharacterModelOptions": [
        {
          "DirectoryPath": "",
          "Name": "MRZ"
        }
      ],
      "ImageParameterOptions": [
        {
          "Name": "ip-mrz",
          "TextureDetectionModes": [
            {
              "Mode": "TDM_GENERAL_WIDTH_CONCENTRATION",
              "Sensitivity": 8
            }
          ],
          "BinarizationModes": [
            {
              "EnableFillBinaryVacancy": 0,
              "ThresholdCompensation": 21,
              "Mode": "BM_LOCAL_BLOCK"
            }
          ],
          "TextDetectionMode": {
            "Mode": "TTDM_LINE",
            "CharHeightRange": [ 5, 1000, 1 ],
            "Direction": "HORIZONTAL",
            "Sensitivity": 7
          }
        }
      ]
    }
    
  3. Watch the changes of the isScanning prop.

    useEffect(()=>{
      if (props.isScanning === true) {
        startScanning();
      }else{
        stopScanning();
      }
    },[props.isScanning])
    

    The startScanning and stopScanning functions:

    const startScanning = async () => {
      stopScanning();
      if (cameraEnhancer.current && router.current) {
        cameraEnhancer.current.open();
        router.current.startCapturing("ReadMRZ")
      }
    }
    
    const stopScanning = () => {
      if (cameraEnhancer.current && router.current) {
        router.current.stopCapturing();
        cameraEnhancer.current.close();
      }
    }
    

Use the MRZ Scanner Component to Scan MRZ

  1. In page.tsx, use next/dynamic to import the MRZ scanner component to disable server-side rendering. If not, you may encounter an HTMLElement undefined error.

    import dynamic from "next/dynamic";
    const MRZScanner = dynamic(
      () => import("../components/MRZScanner"),
      {
        ssr: false,
      }
    );
    
  2. Add a button to toggle the scanning status of the MRZ scanner component and receive the MRZ results in the onScanned function.

    export default function Home() {
      const [isScanning,setIsScanning] = useState(false);
      const [initialized,setInitialized] = useState(false);
      const [MRZ,setMRZ] = useState("");
    
      const onScanned = (result:RecognizedTextLinesResult) => {
        setIsScanning(false);
        if (result.textLineResultItems.length>0) {
          let str = result.textLineResultItems[0].text
          setMRZ(str);
        }
      }
    
      const toggleScanning = () => {
        setMRZ("");
        setIsScanning(!isScanning)
      }
    
      return (
        <main className={styles.main}>
          <h2>MRZ Scanner</h2>
          {!initialized &&(
            <button disabled>Initializing...</button>  
          )}
          {initialized &&(
            <button onClick={()=>toggleScanning()} >{isScanning?"Stop Scanning":"Start Scanning"}</button>
          )}
          <div className={styles.scanner + ((initialized && isScanning) ? "" : " "+styles.hidden)}>
            <div className={styles.cameracontainer}>
              <MRZScanner 
                isScanning={isScanning}
                onScanned={(result:RecognizedTextLinesResult)=>{onScanned(result)}}
                onInitialized={()=>{setInitialized(true)}}
              ></MRZScanner>
            </div>
          </div>
        </main>
      );
    }
    

Parse the MRZ

  1. Create a MRZResultTable.tsx component under src/components with the following template. It displays the fields and values of the parsed result in a table.

    import { useEffect, useRef, useState } from "react";
    import "./MRZResultTable.css"
    
    export interface MRZResultTableProps {
      MRZ:string;
    }
    
    interface Field{
      name:string;
      value:string;
    }
    
    const MRZResultTable: React.FC<MRZResultTableProps> = (props:MRZResultTableProps) => {
      const [fields,setFields] = useState<Field[]|null>(null)
      return (
        <>
          {fields &&(
            <table className="resultTable">
              <thead>
                <tr>
                  <th>Field</th>
                  <th>Value</th>
                </tr>
              </thead>
              <tbody>
                {fields.map(field =>
                  <tr key={field.name}>
                    <td>{field.name}</td>
                    <td>{field.value}</td>
                  </tr>
                )}
              </tbody>
            </table>
          )}
        </>
      )
    }
    
    export default MRZResultTable;
    

    CSS:

    .resultTable {
      border-collapse: collapse;
      max-width: 100%;
      overflow: auto;
    }
    
    .resultTable, .resultTable td, .resultTable th {
      border: 1px solid;
    }
    
  2. Initialize Code Parser for parsing when the component is mounted.

    const initialized = useRef(false);
    const parser = useRef<CodeParser|null>(null);
    
    useEffect(()=>{
      const init = async () => {
        initialized.current = true;
        await initCodeParser();
        parse();
      }
      init();
    },[])
    
    const initCodeParser = async () => {
      CoreModule.engineResourcePaths.dcp = "https://cdn.jsdelivr.net/npm/dynamsoft-code-parser@2.2.10/dist/";
      await CodeParserModule.loadSpec("MRTD_TD1_ID");
      await CodeParserModule.loadSpec("MRTD_TD2_FRENCH_ID")
      await CodeParserModule.loadSpec("MRTD_TD2_ID")
      await CodeParserModule.loadSpec("MRTD_TD2_VISA")
      await CodeParserModule.loadSpec("MRTD_TD3_PASSPORT")  
      await CodeParserModule.loadSpec("MRTD_TD3_VISA")
      parser.current = await CodeParser.createInstance();
    }
    
  3. Parse the MRZ if the MRZ prop is changed.

    useEffect(()=>{
      parse();
    },[props.MRZ])
         
    const parse = async () => {
      if (parser.current && props.MRZ) {
        let result = await parser.current.parse(props.MRZ);
        let MRZFields = ["documentNumber","passportNumber","issuingState","name","sex","nationality","dateOfExpiry","dateOfBirth"];
        let parsedFields = [];
        for (let index = 0; index < MRZFields.length; index++) {
          const field = MRZFields[index];
          const value = result.getFieldValue(field);
          if (value){
            parsedFields.push({
              name:field,
              value:value
            })
          }
        }
        setFields(parsedFields);
      }else{
        setFields(null);
      }
    }
    
  4. In page.tsx, use the MRZResultTable component to display the parsed results.

    <MRZResultTable MRZ={MRZ}></MRZResultTable>
    

All right, we’ve now completed the Next.js demo of the MRZ scanner.

Source Code

Get the source code of the demo to have a try:

https://github.com/tony-xlh/NextJS-MRZ-Scanner

References

  1. https://en.wikipedia.org/wiki/Machine-readable_passport