MRZ Recognition on Android Using Dynamsoft Label Recognizer SDK
Machine Readable Zone (MRZ) recognition is a critical feature in applications that require scanning and processing identity documents, such as passports and visas. On Android devices, implementing accurate MRZ recognition can significantly enhance the efficiency and user experience of such apps. In this article, we will explore how to use the Dynamsoft Label Recognizer SDK to integrate MRZ recognition into your Android application. The SDK provides powerful OCR capabilities specifically tailored for reading MRZ data, allowing you to capture and decode information quickly and reliably, making your app more robust and user-friendly.
This article is Part 7 in a 8-Part Series.
- Part 1 - Building Desktop Passport Scanner with Qt and USB Camera
- Part 2 - Making Web Passport MRZ Reader and Scanner in HTML5 and JavaScript
- Part 3 - How to Build Python MRZ Scanner SDK and Publish It to PyPI
- Part 4 - How to Recognize MRZ from Passport and ID Card with Node.js
- Part 5 - How to Make Java MRZ Detector with Dynamsoft Label Recognizer for Windows and Linux
- Part 6 - How to Create a Flutter plugin of Passport MRZ Recognition for Windows, Linux, Android, iOS and Web
- Part 7 - MRZ Recognition on Android Using Dynamsoft Label Recognizer SDK
- Part 8 - Developing a Desktop MRZ Scanner for Passports, IDs, and Visas with Dynamsoft C++ Capture Vision SDK
MRZ Recognition on Android
Prerequisites
Sample Passport Image with MRZ
Below is a sample MRZ image for testing:
Implementing MRZ Recognition on Android
Step 1: Configure Your Android Project with Dynamsoft Label Recognizer SDK
-
In your project’s root
build.gradle
file, add the Dynamsoft Maven repository:allprojects { repositories { google() mavenCentral() maven { url "https://download2.dynamsoft.com/maven/aar" } } }
-
In the module’s
build.gradle
file, include the necessary dependencies:implementation "com.dynamsoft:dynamsoftbarcodereaderbundle:10.2.1100" implementation "com.dynamsoft:dynamsoftcodeparser:2.2.11" implementation "com.dynamsoft:dynamsoftcodeparserdedicator:1.2.20"
Explanation
dynamsoftbarcodereaderbundle
: Provides camera controls and OCR recognition capabilities.dynamsoftcodeparser
anddynamsoftcodeparserdedicator
: These libraries are used to parse MRZ text and extract relevant data.
Step 2: Add MRZ Model Files to the Assets Folder
Ensure the MRZ recognition model files (MRZ.data, Confusable.data, and MRZScanner.json) are added to your project’s assets folder:
These files are used for recognizing MRZ text, with the JSON file specifying the algorithm settings.
Here is a snippet of the MRZScanner.json
configuration:
{
"CaptureVisionTemplates": [
{
"Name": "ReadMRZ",
"OutputOriginalImage": 0,
"ImageROIProcessingNameArray": [
"roi-mrz"
],
"SemanticProcessingNameArray": ["sp-mrz"],
"Timeout": 2000
}
],
"TargetROIDefOptions": [
{
"Name": "roi-mrz",
"TaskSettingNameArray": [
"task-mrz"
]
}
],
"TextLineSpecificationOptions": [
{
"Name": "tls-mrz-passport",
"BaseTextLineSpecificationName": "tls-base",
"StringLengthRange": [ 44, 44 ],
"OutputResults": 1,
"ExpectedGroupsCount": 1,
"ConcatResults": 1,
"ConcatSeparator": "",
"SubGroups": [
{
"StringRegExPattern": "(P[A-Z<][A-Z<]{3}[A-Z<]{39}){(44)}",
"StringLengthRange": [ 44, 44 ],
"BaseTextLineSpecificationName": "tls-base"
},
{
"StringRegExPattern": "([A-Z0-9<]{9}[0-9][A-Z<]{3}[0-9]{2}[(01-12)][(01-31)][0-9][MF<][0-9]{2}[(01-12)][(01-31)][0-9][A-Z0-9<]{14}[0-9<][0-9]){(44)}",
"StringLengthRange": [ 44, 44 ],
"BaseTextLineSpecificationName": "tls-base"
}
]
},
{
"Name": "tls-mrz-visa-td3",
"BaseTextLineSpecificationName": "tls-base",
"StringLengthRange": [ 44, 44 ],
"OutputResults": 1,
"ExpectedGroupsCount": 1,
"ConcatResults": 1,
"ConcatSeparator": "",
"SubGroups": [
{
"StringRegExPattern": "(V[A-Z<][A-Z<]{3}[A-Z<]{39}){(44)}",
"StringLengthRange": [ 44, 44 ],
"BaseTextLineSpecificationName": "tls-base"
},
{
"StringRegExPattern": "([A-Z0-9<]{9}[0-9][A-Z<]{3}[0-9]{2}[(01-12)][(01-31)][0-9][MF<][0-9]{2}[(01-12)][(01-31)][0-9][A-Z0-9<]{14}[A-Z0-9<]{2}){(44)}",
"StringLengthRange": [ 44, 44 ],
"BaseTextLineSpecificationName": "tls-base"
}
]
},
{
"Name": "tls-mrz-visa-td2",
"BaseTextLineSpecificationName": "tls-base",
"StringLengthRange": [ 36, 36 ],
"OutputResults": 1,
"ExpectedGroupsCount": 1,
"ConcatResults": 1,
"ConcatSeparator": "",
"SubGroups": [
{
"StringRegExPattern": "(V[A-Z<][A-Z<]{3}[A-Z<]{31}){(36)}",
"StringLengthRange": [ 36, 36 ],
"BaseTextLineSpecificationName": "tls-base"
},
{
"StringRegExPattern": "([A-Z0-9<]{9}[0-9][A-Z<]{3}[0-9]{2}[(01-12)][(01-31)][0-9][MF<][0-9]{2}[(01-12)][(01-31)][0-9][A-Z0-9<]{8}){(36)}",
"StringLengthRange": [ 36, 36 ],
"BaseTextLineSpecificationName": "tls-base"
}
]
},
{
"Name": "tls-mrz-id-td2",
"BaseTextLineSpecificationName": "tls-base",
"StringLengthRange": [ 36, 36 ],
"OutputResults": 1,
"ExpectedGroupsCount": 1,
"ConcatResults": 1,
"ConcatSeparator": "",
"SubGroups": [
{
"StringRegExPattern": "([ACI][A-Z<][A-Z<]{3}[A-Z<]{31}){(36)}",
"StringLengthRange": [ 36, 36 ],
"BaseTextLineSpecificationName": "tls-base"
},
{
"StringRegExPattern": "([A-Z0-9<]{9}[0-9][A-Z<]{3}[0-9]{2}[(01-12)][(01-31)][0-9][MF<][0-9]{2}[(01-12)][(01-31)][0-9][A-Z0-9<]{8}){(36)}",
"StringLengthRange": [ 36, 36 ],
"BaseTextLineSpecificationName": "tls-base"
}
]
},
{
"Name": "tls-mrz-id-td1",
"BaseTextLineSpecificationName": "tls-base",
"StringLengthRange": [ 30, 30 ],
"OutputResults": 1,
"ExpectedGroupsCount": 1,
"ConcatResults": 1,
"ConcatSeparator": "",
"SubGroups": [
{
"StringRegExPattern": "([ACI][A-Z<][A-Z<]{3}[A-Z0-9<]{9}[0-9][A-Z0-9<]{15}){(30)}",
"StringLengthRange": [ 30, 30 ],
"BaseTextLineSpecificationName": "tls-base"
},
{
"StringRegExPattern": "([0-9]{2}[(01-12)][(01-31)][0-9][MF<][0-9]{2}[(01-12)][(01-31)][0-9][A-Z<]{3}[A-Z0-9<]{11}[0-9]){(30)}",
"StringLengthRange": [ 30, 30 ],
"BaseTextLineSpecificationName": "tls-base"
},
{
"StringRegExPattern": "([A-Z<]{30}){(30)}",
"StringLengthRange": [ 30, 30 ],
"BaseTextLineSpecificationName": "tls-base"
}
]
},
{
"Name": "tls-base",
"CharacterModelName": "MRZ",
"CharHeightRange": [ 5, 1000, 1 ],
"BinarizationModes": [
{
"BlockSizeX": 30,
"BlockSizeY": 30,
"Mode": "BM_LOCAL_BLOCK",
"EnableFillBinaryVacancy": 0,
"ThresholdCompensation": 15
}
],
"ConfusableCharactersCorrection": {
"ConfusableCharacters": [
[ "0", "O" ],
[ "1", "I" ],
[ "5", "S" ]
],
"FontNameArray": [ "OCR_B" ]
}
}
],
"LabelRecognizerTaskSettingOptions": [
{
"Name": "task-mrz",
"ConfusableCharactersPath": "ConfusableChars.data",
"TextLineSpecificationNameArray": [ "tls-mrz-passport", "tls-mrz-visa-td3", "tls-mrz-id-td1", "tls-mrz-id-td2", "tls-mrz-visa-td2" ],
"SectionImageParameterArray": [
{
"Section": "ST_REGION_PREDETECTION",
"ImageParameterName": "ip-mrz"
},
{
"Section": "ST_TEXT_LINE_LOCALIZATION",
"ImageParameterName": "ip-mrz"
},
{
"Section": "ST_TEXT_LINE_RECOGNITION",
"ImageParameterName": "ip-mrz"
}
]
}
],
"CharacterModelOptions": [
{
"DirectoryPath": "",
"Name": "MRZ"
}
],
"ImageParameterOptions": [
{
"Name": "ip-mrz",
"TextureDetectionModes": [
{
"Mode": "TDM_GENERAL_WIDTH_CONCENTRATION",
"Sensitivity": 8
}
],
"BinarizationModes": [
{
"EnableFillBinaryVacancy": 0,
"ThresholdCompensation": 21,
"Mode": "BM_LOCAL_BLOCK"
}
],
"TextDetectionMode": {
"Mode": "TTDM_LINE",
"CharHeightRange": [ 5, 1000, 1 ],
"Direction": "HORIZONTAL",
"Sensitivity": 7
}
}
],
"SemanticProcessingOptions": [
{
"Name": "sp-mrz",
"ReferenceObjectFilter": {
"ReferenceTargetROIDefNameArray": [
"roi-mrz"
]
},
"TaskSettingNameArray": [
"dcp-mrz"
]
}
],
"CodeParserTaskSettingOptions": [
{
"Name": "dcp-mrz",
"CodeSpecifications": [ "MRTD_TD3_PASSPORT", "MRTD_TD2_VISA", "MRTD_TD3_VISA", "MRTD_TD1_ID", "MRTD_TD2_ID" ]
}
]
}
This setup supports various MRZ types, including MRTD_TD3_PASSPORT, MRTD_TD2_VISA, MRTD_TD3_VISA, MRTD_TD1_ID and MRTD_TD2_ID.
Step 3: Initialize Dynamsoft Label Recognizer
In your MainActivity.java
file, set up the license key for the SDK as shown below:
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
if (savedInstanceState == null) {
LicenseManager.initLicense("LICENSE-KEY", this, (isSuccessful, error) -> {
if (!isSuccessful) {
error.printStackTrace();
runOnUiThread(() -> ((TextView) findViewById(R.id.tv_license_error)).setText("License initialization failed: "+error.getMessage()));
}
});
}
...
}
This setup initializes the SDK and checks for any licensing errors, ensuring the app is ready for MRZ recognition.
Step 4: Create a Camera Preview for MRZ Recognition
-
Add a new layout file named
activity_scan.xml
in thelayout
folder:<?xml version="1.0" encoding="utf-8"?> <RelativeLayout xmlns:android="http://schemas.android.com/apk/res/android" android:layout_width="match_parent" android:layout_height="match_parent"> <com.dynamsoft.dce.CameraView android:id="@+id/dce_camera_view" android:layout_width="match_parent" android:layout_height="match_parent" /> <TextView android:layout_above="@+id/tv_result" android:id="@+id/tv_message" android:layout_marginHorizontal="24dp" android:layout_centerHorizontal="true" android:layout_marginBottom="80dp" android:textColor="@color/red00" android:textSize="16sp" android:layout_width="wrap_content" android:layout_height="wrap_content"/> <TextView android:layout_alignParentBottom="true" android:layout_centerHorizontal="true" android:layout_marginBottom="70dp" android:id="@+id/tv_result" android:layout_width="wrap_content" android:layout_height="wrap_content" /> </RelativeLayout>
Explanation
CameraView
: Displays the camera preview for capturing MRZ data.TextView
: Displays messages such as license information and detection errors.
-
Inflate the layout and configure the camera preview in the
MainActivity.java
file:@Override protected void onCreate(Bundle savedInstanceState) { super.onCreate(savedInstanceState); setContentView(R.layout.activity_scan); PermissionUtil.requestCameraPermission(this); LicenseManager.initLicense("LICENSE-KEY", this, (isSuccess, error) -> { if (!isSuccess) { runOnUiThread(() -> { ((TextView)findViewById(R.id.tv_message)) .setText("License initialization failed: "+error.getMessage()); }); error.printStackTrace(); } }); mCameraView = findViewById(R.id.dce_camera_view); mTextResult = findViewById(R.id.tv_result); mCamera = new CameraEnhancer(mCameraView, this); try { mCamera.enableEnhancedFeatures(EnumEnhancerFeatures.EF_FRAME_FILTER); } catch (CameraEnhancerException e) { throw new RuntimeException(e); } mRouter = new CaptureVisionRouter(this); MultiFrameResultCrossFilter filter = new MultiFrameResultCrossFilter(); filter.enableResultCrossVerification(EnumCapturedResultItemType.CRIT_TEXT_LINE, true); mRouter.addResultFilter(filter); try { mRouter.initSettingsFromFile("MRZScanner.json"); mRouter.setInput(mCamera); } catch (CaptureVisionRouterException e) { throw new RuntimeException(e); } } @Override protected void onResume() { super.onResume(); try { mCamera.open(); } catch (CameraEnhancerException e) { e.printStackTrace(); } mRouter.startCapturing("ReadMRZ", new CompletionListener() { @Override public void onSuccess() { } @Override public void onFailure(int errorCode, String errorString) { runOnUiThread(() -> showDialog("Error", String.format(Locale.getDefault(), "ErrorCode: %d %nErrorMessage: %s", errorCode, errorString))); } }); } @Override protected void onPause() { super.onPause(); succeed = false; try { mCamera.close(); } catch (CameraEnhancerException e) { e.printStackTrace(); } mRouter.stopCapturing(); } @Override protected void onStop() { mCameraView.getDrawingLayer(DrawingLayer.DLR_LAYER_ID).clearDrawingItems(); super.onStop(); }
-
Register the callbacks for handling recognized MRZ data:
mRouter.addResultReceiver(new CapturedResultReceiver() { @Override // Implement this method to receive RecognizedTextLinesResult. public void onRecognizedTextLinesReceived(@NonNull RecognizedTextLinesResult result) { onLabelTextReceived(result); } @Override public void onParsedResultsReceived(@NonNull ParsedResult result) { if (!succeed) { onParsedResultReceived(result); } } });
The
ParsedResult
object contains the MRZ data extracted from the captured image, formatted as key-value pairs. -
Bind the parsed MRZ data to an
Intent
and start a new activity to display the results:private void onParsedResultReceived(ParsedResult result) { if (result.getItems() == null) { return; } if (result.getItems().length == 0) { runOnUiThread(() -> { if (!mText.isEmpty()) { String errorMsg = "error: Failed to parse the content. The MRZ text is " + mText; mTextResult.setText(errorMsg); } }); } else { HashMap<String, String> labelMap = assembleMap(result.getItems()[0]); if (labelMap != null && !labelMap.isEmpty()) { succeed = true; Intent intent = new Intent(this, ResultActivity.class); intent.putExtra("labelMap", labelMap); startActivity(intent); runOnUiThread(() -> { mTextResult.setText(""); }); } else { runOnUiThread(() -> { if (!mText.isEmpty()) { String errorMsg = "error: Failed to parse the content. The MRZ text is " + mText; mTextResult.setText(errorMsg); } }); } } }
Step 5: Display the MRZ Data
-
Create a layout file named
activity_result.xml
in thelayout
folder:<?xml version="1.0" encoding="utf-8"?> <RelativeLayout xmlns:android="http://schemas.android.com/apk/res/android" xmlns:app="http://schemas.android.com/apk/res-auto" xmlns:tools="http://schemas.android.com/tools" android:layout_width="match_parent" android:layout_height="match_parent" android:background="@color/dy_black_3B" tools:context=".ResultActivity"> <RelativeLayout android:id="@+id/rl_title" android:layout_width="match_parent" android:layout_height="48dp" android:background="@color/dy_black_2B"> <ImageView android:id="@+id/iv_back" android:layout_width="wrap_content" android:layout_height="wrap_content" android:layout_centerVertical="true" android:layout_marginStart="16dp" android:background="@color/transparent" android:src="@drawable/ic_arrow_left" /> <TextView android:layout_width="wrap_content" android:layout_height="wrap_content" android:layout_centerVertical="true" android:layout_toEndOf="@id/iv_back" android:text="Back" android:textColor="@color/white" android:textSize="17sp" /> <TextView android:layout_width="wrap_content" android:layout_height="wrap_content" android:layout_centerInParent="true" android:layout_gravity="center_horizontal" android:text="Result" android:textColor="@color/white" android:textSize="20sp" android:textStyle="bold" /> </RelativeLayout> <ScrollView android:layout_below="@id/rl_title" android:layout_width="match_parent" android:layout_height="wrap_content"> <LinearLayout android:id="@+id/ll_content" android:layout_width="match_parent" android:layout_height="wrap_content" android:orientation="vertical" android:padding="20dp"> </LinearLayout> </ScrollView> </RelativeLayout>
-
Inflate the layout and display the MRZ data in the
ResultActivity.java
file:package com.dynamsoft.mrzscanner; import androidx.annotation.NonNull; import androidx.appcompat.app.AppCompatActivity; import androidx.core.content.ContextCompat; import android.graphics.Color; import android.os.Bundle; import android.view.View; import android.view.ViewGroup; import android.widget.LinearLayout; import android.widget.TextView; import java.util.HashMap; public class ResultActivity extends AppCompatActivity { private LinearLayout content; @Override protected void onCreate(Bundle savedInstanceState) { super.onCreate(savedInstanceState); setContentView(R.layout.activity_result); findViewById(R.id.iv_back).setOnClickListener((v) -> { finish(); }); content = findViewById(R.id.ll_content); HashMap<String, String> properties = (HashMap<String, String>) getIntent(). getSerializableExtra("labelMap"); if (properties != null) { fillViews(properties); } } @NonNull private View childView(String label, String labelText) { LinearLayout layout = new LinearLayout(this); LinearLayout.LayoutParams params = new LinearLayout.LayoutParams(ViewGroup.LayoutParams.MATCH_PARENT, ViewGroup.LayoutParams.WRAP_CONTENT); params.setMargins(0, 30, 0, 0); layout.setLayoutParams(params); layout.setOrientation(LinearLayout.VERTICAL); TextView labelView = new TextView(this); labelView.setPadding(0, 30, 0, 0); labelView.setTextColor(ContextCompat.getColor(this, R.color.dy_grey_AA)); labelView.setTextSize(16); labelView.setText(label); TextView textView = new TextView(this); textView.setTextSize(16); textView.setTextColor(Color.WHITE); textView.setText(labelText); layout.addView(labelView); layout.addView(textView); return layout; } private void fillViews(HashMap<String, String> properties) { content.addView(childView("Document Type:", properties.get("Document Type"))); content.addView(childView("Document Number:", properties.get("Document Number"))); content.addView(childView("Name:", properties.get("Name"))); content.addView(childView("Issuing State:", properties.get("Issuing State"))); content.addView(childView("Nationality:", properties.get("Nationality"))); content.addView(childView("Date of Birth(YYYY-MM-DD):", properties.get("Date of Birth(YYYY-MM-DD)"))); content.addView(childView("Sex:", Character.toUpperCase(properties.get("Sex").charAt(0)) + properties.get("Sex").substring(1))); content.addView(childView("Date of Expiry(YYYY-MM-DD):", properties.get("Date of Expiry(YYYY-MM-DD)"))); } }
Source Code
https://github.com/yushulx/android-camera-barcode-mrz-document-scanner/tree/main/examples/10.x/mrz