How to Scan MRZ in a Next.js Application
MRZ stands for machine-readable zone, which is usually at the bottom of the identity page at the beginning of a passport or ID card.1 It can be read by a computing device with a camera to get information like document type, name, document number, nationality, date of birth, sex, and document expiration date.
In this article, we are going to build a web application in Next.js to scan MRZ via cameras. Dynamsoft Label Recognizer is used as the OCR engine.
Screenshot of the demo:
You can find the online demo here.
New Project
Create a new Next.js project:
npx create-next-app@latest
Install Dependencies
Install Dynamsoft Label Recognizer and related libraries.
npm i dynamsoft-core@3.2.30 dynamsoft-camera-enhancer@4.0.2 dynamsoft-capture-vision-router@2.2.30 dynamsoft-code-parser@2.2.10 dynamsoft-label-recognizer@3.2.30 dynamsoft-license@3.2.21 dynamsoft-utility@1.2.20
Configure the SDK
Create a new file named configure.ts
with the following content. A license is needed. You can apply for your license here.
import "dynamsoft-license";
import "dynamsoft-capture-vision-router";
import "dynamsoft-label-recognizer";
import { LicenseManager } from "dynamsoft-license";
import { CoreModule } from "dynamsoft-core";
if (CoreModule.isModuleLoaded("dlr") === false) {
/** LICENSE ALERT - README
* To use the library, you need to first specify a license key using the API "initLicense()" as shown below.
*/
LicenseManager.initLicense("LICENSE-KEY");
CoreModule.engineResourcePaths = {
std: 'https://cdn.jsdelivr.net/npm/dynamsoft-capture-vision-std@1.2.10/dist/',
dip: 'https://cdn.jsdelivr.net/npm/dynamsoft-image-processing@2.2.30/dist/',
core: "https://cdn.jsdelivr.net/npm/dynamsoft-core@3.2.30/dist/",
license: "https://cdn.jsdelivr.net/npm/dynamsoft-license@3.2.21/dist/",
cvr: "https://cdn.jsdelivr.net/npm/dynamsoft-capture-vision-router@2.2.30/dist/",
dlr: "https://cdn.jsdelivr.net/npm/dynamsoft-label-recognizer@3.2.30/dist/",
dce: "https://cdn.jsdelivr.net/npm/dynamsoft-camera-enhancer@4.0.2/dist/",
dnn: 'https://cdn.jsdelivr.net/npm/dynamsoft-capture-vision-dnn@1.0.20/dist/',
dlrData: 'https://cdn.jsdelivr.net/npm/dynamsoft-label-recognizer-data@1.0.10/dist/',
utility: 'https://cdn.jsdelivr.net/npm/dynamsoft-utility@1.2.20/dist/'
};
}
In page.tsx
, import the configure file to run the configuration:
import "../configure"
Create an MRZ Scanner Component
-
Create a new component under
src/components/MRZScanner.tsx
with the following template.import { RecognizedTextLinesResult } from "dynamsoft-label-recognizer"; import { MutableRefObject, useEffect, useRef } from "react"; import React from "react"; export interface MRZScannerProps{ isScanning?:boolean; onInitialized?:()=>void; onScanned?:(results:RecognizedTextLinesResult)=>void; } const MRZScanner: React.FC<MRZScannerProps> = (props:MRZScannerProps) => { const container:MutableRefObject<HTMLDivElement|null> = useRef(null); return ( <div ref={container} style={{width:"100%",height:"100%"}}></div> ) } export default MRZScanner;
-
Add the following effect to initialize relevant libraries when the component is mounted.
const initialized = useRef(false); useEffect(()=>{ init(); },[]) const init = async () => { if (initialized.current == false) { initialized.current = true; await initCameraEnhancer(); await initLabelRecognizer(); if (props.onInitialized) { props.onInitialized(); } if (props.isScanning === true) { startScanning(); } } }
The function to initialize the camera enhancer.
const initCameraEnhancer = async () => { const cameraView = await CameraView.createInstance(); cameraEnhancer.current = await CameraEnhancer.createInstance(cameraView); container.current!.append(cameraView.getUIElement()); }
The function to initialize the label recognizer.
const initLabelRecognizer = async () => { // Preload "LabelRecogznier" module for recognizing text. It will save time on the initial recognizing by skipping the module loading. await CoreModule.loadWasm(["DLR"]); await LabelRecognizerModule.loadRecognitionData("MRZ"); router.current = await CaptureVisionRouter.createInstance(); router.current.initSettings("/template.json"); // Define a callback for results. const resultReceiver = new CapturedResultReceiver(); resultReceiver.onRecognizedTextLinesReceived = (result: RecognizedTextLinesResult) => { console.log(result); if (props.onScanned) { props.onScanned(result); } }; router.current.addResultReceiver(resultReceiver); if (cameraEnhancer.current) { router.current.setInput(cameraEnhancer.current); } }
We have to set the runtime settings for capture vision router to read MRZ with the following line of code. (PS: label recognizer is called using capture vision router)
router.current.initSettings("/template.json");
The
template.json
with the following content is put underpublic
.{ "CaptureVisionTemplates": [ { "Name": "ReadMRZ", "OutputOriginalImage": 0, "ImageROIProcessingNameArray": [ "roi-mrz" ], "Timeout": 2000 } ], "TargetROIDefOptions": [ { "Name": "roi-mrz", "TaskSettingNameArray": [ "task-mrz" ] } ], "TextLineSpecificationOptions": [ { "Name": "tls-mrz-passport", "BaseTextLineSpecificationName": "tls-base", "StringLengthRange": [ 44, 44 ], "OutputResults": 1, "ExpectedGroupsCount": 1, "ConcatResults": 1, "ConcatSeparator": "\n", "SubGroups": [ { "StringRegExPattern": "(P[A-Z<][A-Z<]{3}[A-Z<]{39}){(44)}", "StringLengthRange": [ 44, 44 ], "BaseTextLineSpecificationName": "tls-base" }, { "StringRegExPattern": "([A-Z0-9<]{9}[0-9][A-Z<]{3}[0-9]{2}[(01-12)][(01-31)][0-9][MF<][0-9]{2}[(01-12)][(01-31)][0-9][A-Z0-9<]{14}[0-9<][0-9]){(44)}", "StringLengthRange": [ 44, 44 ], "BaseTextLineSpecificationName": "tls-base" } ] }, { "Name": "tls-mrz-visa-td3", "BaseTextLineSpecificationName": "tls-base", "StringLengthRange": [ 44, 44 ], "OutputResults": 1, "ExpectedGroupsCount": 1, "ConcatResults": 1, "ConcatSeparator": "\n", "SubGroups": [ { "StringRegExPattern": "(V[A-Z<][A-Z<]{3}[A-Z<]{39}){(44)}", "StringLengthRange": [ 44, 44 ], "BaseTextLineSpecificationName": "tls-base" }, { "StringRegExPattern": "([A-Z0-9<]{9}[0-9][A-Z<]{3}[0-9]{2}[(01-12)][(01-31)][0-9][MF<][0-9]{2}[(01-12)][(01-31)][0-9][A-Z0-9<]{14}[A-Z0-9<]{2}){(44)}", "StringLengthRange": [ 44, 44 ], "BaseTextLineSpecificationName": "tls-base" } ] }, { "Name": "tls-mrz-visa-td2", "BaseTextLineSpecificationName": "tls-base", "StringLengthRange": [ 36, 36 ], "OutputResults": 1, "ExpectedGroupsCount": 1, "ConcatResults": 1, "ConcatSeparator": "\n", "SubGroups": [ { "StringRegExPattern": "(V[A-Z<][A-Z<]{3}[A-Z<]{31}){(36)}", "StringLengthRange": [ 36, 36 ], "BaseTextLineSpecificationName": "tls-base" }, { "StringRegExPattern": "([A-Z0-9<]{9}[0-9][A-Z<]{3}[0-9]{2}[(01-12)][(01-31)][0-9][MF<][0-9]{2}[(01-12)][(01-31)][0-9][A-Z0-9<]{8}){(36)}", "StringLengthRange": [ 36, 36 ], "BaseTextLineSpecificationName": "tls-base" } ] }, { "Name": "tls-mrz-id-td2", "BaseTextLineSpecificationName": "tls-base", "StringLengthRange": [ 36, 36 ], "OutputResults": 1, "ExpectedGroupsCount": 1, "ConcatResults": 1, "ConcatSeparator": "\n", "SubGroups": [ { "StringRegExPattern": "([ACI][A-Z<][A-Z<]{3}[A-Z<]{31}){(36)}", "StringLengthRange": [ 36, 36 ], "BaseTextLineSpecificationName": "tls-base" }, { "StringRegExPattern": "([A-Z0-9<]{9}[0-9][A-Z<]{3}[0-9]{2}[(01-12)][(01-31)][0-9][MF<][0-9]{2}[(01-12)][(01-31)][0-9][A-Z0-9<]{8}){(36)}", "StringLengthRange": [ 36, 36 ], "BaseTextLineSpecificationName": "tls-base" } ] }, { "Name": "tls-mrz-id-td1", "BaseTextLineSpecificationName": "tls-base", "StringLengthRange": [ 30, 30 ], "OutputResults": 1, "ExpectedGroupsCount": 1, "ConcatResults": 1, "ConcatSeparator": "\n", "SubGroups": [ { "StringRegExPattern": "([ACI][A-Z<][A-Z<]{3}[A-Z0-9<]{9}[0-9][A-Z0-9<]{15}){(30)}", "StringLengthRange": [ 30, 30 ], "BaseTextLineSpecificationName": "tls-base" }, { "StringRegExPattern": "([0-9]{2}[(01-12)][(01-31)][0-9][MF<][0-9]{2}[(01-12)][(01-31)][0-9][A-Z<]{3}[A-Z0-9<]{11}[0-9]){(30)}", "StringLengthRange": [ 30, 30 ], "BaseTextLineSpecificationName": "tls-base" }, { "StringRegExPattern": "([A-Z<]{30}){(30)}", "StringLengthRange": [ 30, 30 ], "BaseTextLineSpecificationName": "tls-base" } ] }, { "Name": "tls-base", "CharacterModelName": "MRZ", "CharHeightRange": [ 5, 1000, 1 ], "BinarizationModes": [ { "BlockSizeX": 30, "BlockSizeY": 30, "Mode": "BM_LOCAL_BLOCK", "EnableFillBinaryVacancy": 0, "ThresholdCompensation": 15 } ], "ConfusableCharactersCorrection": { "ConfusableCharacters": [ [ "0", "O" ], [ "1", "I" ], [ "5", "S" ] ], "FontNameArray": [ "OCR_B" ] } } ], "LabelRecognizerTaskSettingOptions": [ { "Name": "task-mrz", "ConfusableCharactersPath": "ConfusableChars.data", "TextLineSpecificationNameArray": [ "tls-mrz-passport", "tls-mrz-visa-td3", "tls-mrz-id-td1", "tls-mrz-id-td2", "tls-mrz-visa-td2" ], "SectionImageParameterArray": [ { "Section": "ST_REGION_PREDETECTION", "ImageParameterName": "ip-mrz" }, { "Section": "ST_TEXT_LINE_LOCALIZATION", "ImageParameterName": "ip-mrz" }, { "Section": "ST_TEXT_LINE_RECOGNITION", "ImageParameterName": "ip-mrz" } ] } ], "CharacterModelOptions": [ { "DirectoryPath": "", "Name": "MRZ" } ], "ImageParameterOptions": [ { "Name": "ip-mrz", "TextureDetectionModes": [ { "Mode": "TDM_GENERAL_WIDTH_CONCENTRATION", "Sensitivity": 8 } ], "BinarizationModes": [ { "EnableFillBinaryVacancy": 0, "ThresholdCompensation": 21, "Mode": "BM_LOCAL_BLOCK" } ], "TextDetectionMode": { "Mode": "TTDM_LINE", "CharHeightRange": [ 5, 1000, 1 ], "Direction": "HORIZONTAL", "Sensitivity": 7 } } ] }
-
Watch the changes of the
isScanning
prop.useEffect(()=>{ if (props.isScanning === true) { startScanning(); }else{ stopScanning(); } },[props.isScanning])
The
startScanning
andstopScanning
functions:const startScanning = async () => { stopScanning(); if (cameraEnhancer.current && router.current) { cameraEnhancer.current.open(); router.current.startCapturing("ReadMRZ") } } const stopScanning = () => { if (cameraEnhancer.current && router.current) { router.current.stopCapturing(); cameraEnhancer.current.close(); } }
Use the MRZ Scanner Component to Scan MRZ
-
In
page.tsx
, usenext/dynamic
to import the MRZ scanner component to disable server-side rendering. If not, you may encounter anHTMLElement undefined
error.import dynamic from "next/dynamic"; const MRZScanner = dynamic( () => import("../components/MRZScanner"), { ssr: false, } );
-
Add a button to toggle the scanning status of the MRZ scanner component and receive the MRZ results in the
onScanned
function.export default function Home() { const [isScanning,setIsScanning] = useState(false); const [initialized,setInitialized] = useState(false); const [MRZ,setMRZ] = useState(""); const onScanned = (result:RecognizedTextLinesResult) => { setIsScanning(false); if (result.textLineResultItems.length>0) { let str = result.textLineResultItems[0].text setMRZ(str); } } const toggleScanning = () => { setMRZ(""); setIsScanning(!isScanning) } return ( <main className={styles.main}> <h2>MRZ Scanner</h2> {!initialized &&( <button disabled>Initializing...</button> )} {initialized &&( <button onClick={()=>toggleScanning()} >{isScanning?"Stop Scanning":"Start Scanning"}</button> )} <div className={styles.scanner + ((initialized && isScanning) ? "" : " "+styles.hidden)}> <div className={styles.cameracontainer}> <MRZScanner isScanning={isScanning} onScanned={(result:RecognizedTextLinesResult)=>{onScanned(result)}} onInitialized={()=>{setInitialized(true)}} ></MRZScanner> </div> </div> </main> ); }
Parse the MRZ
-
Create a
MRZResultTable.tsx
component undersrc/components
with the following template. It displays the fields and values of the parsed result in a table.import { useEffect, useRef, useState } from "react"; import "./MRZResultTable.css" export interface MRZResultTableProps { MRZ:string; } interface Field{ name:string; value:string; } const MRZResultTable: React.FC<MRZResultTableProps> = (props:MRZResultTableProps) => { const [fields,setFields] = useState<Field[]|null>(null) return ( <> {fields &&( <table className="resultTable"> <thead> <tr> <th>Field</th> <th>Value</th> </tr> </thead> <tbody> {fields.map(field => <tr key={field.name}> <td>{field.name}</td> <td>{field.value}</td> </tr> )} </tbody> </table> )} </> ) } export default MRZResultTable;
CSS:
.resultTable { border-collapse: collapse; max-width: 100%; overflow: auto; } .resultTable, .resultTable td, .resultTable th { border: 1px solid; }
-
Initialize Code Parser for parsing when the component is mounted.
const initialized = useRef(false); const parser = useRef<CodeParser|null>(null); useEffect(()=>{ const init = async () => { initialized.current = true; await initCodeParser(); parse(); } init(); },[]) const initCodeParser = async () => { CoreModule.engineResourcePaths.dcp = "https://cdn.jsdelivr.net/npm/dynamsoft-code-parser@2.2.10/dist/"; await CodeParserModule.loadSpec("MRTD_TD1_ID"); await CodeParserModule.loadSpec("MRTD_TD2_FRENCH_ID") await CodeParserModule.loadSpec("MRTD_TD2_ID") await CodeParserModule.loadSpec("MRTD_TD2_VISA") await CodeParserModule.loadSpec("MRTD_TD3_PASSPORT") await CodeParserModule.loadSpec("MRTD_TD3_VISA") parser.current = await CodeParser.createInstance(); }
-
Parse the MRZ if the MRZ prop is changed.
useEffect(()=>{ parse(); },[props.MRZ]) const parse = async () => { if (parser.current && props.MRZ) { let result = await parser.current.parse(props.MRZ); let MRZFields = ["documentNumber","passportNumber","issuingState","name","sex","nationality","dateOfExpiry","dateOfBirth"]; let parsedFields = []; for (let index = 0; index < MRZFields.length; index++) { const field = MRZFields[index]; const value = result.getFieldValue(field); if (value){ parsedFields.push({ name:field, value:value }) } } setFields(parsedFields); }else{ setFields(null); } }
-
In
page.tsx
, use theMRZResultTable
component to display the parsed results.<MRZResultTable MRZ={MRZ}></MRZResultTable>
All right, we’ve now completed the Next.js demo of the MRZ scanner.
Source Code
Get the source code of the demo to have a try:
https://github.com/tony-xlh/NextJS-MRZ-Scanner
References
-
https://en.wikipedia.org/wiki/Machine-readable_passport ↩