MRZ Text Recognition with Jetpack Compose and CameraX
Scanning the machine-readable zone (MRZ) of an ID card, visa, or passport in a Jetpack Compose app needs two pieces: CameraX for the live camera feed and Dynamsoft MRZ Scanner for recognition. This tutorial builds an MRZ text scanner in Jetpack Compose that feeds camera frames into the CaptureVisionRouter video pipeline, draws the detected MRZ lines as an overlay on the preview, and can also recognize MRZ from a picked image file.
Note: MRZ stands for machine-readable zone. We can find it on ID cards, Visa cards and Passports. It is a special zone designed for machines to get the info of its owner.
What you’ll build: a Compose MRZ scanner app with a full-screen CameraX preview, a guide frame and live MRZ line overlays, a fixed result panel showing the recognized MRZ lines, and a “Load Image” button that runs recognition on a picked static image through the same pipeline (com.dynamsoft:mrzscannerbundle:3.4.1300).
Here is a video of the final result:
Key Takeaways
- The MRZ templates are not loaded by default:
router.initSettingsFromFile("mrzscanner-mobile-templates.json")is required before the built-inReadPassportAndIdtemplate can be used. Without this call, every request returns zero results. - MRZ text results are produced by the video pipeline (
startCapturing), not by a singlecapture(bitmap)call. A single frame is verified against check digits, so frames that read imperfectly return nothing. Feeding frames continuously lets the multi-frame verification lock onto the read. - Camera frames and picked images share one pipeline: frames go into an
ImageSourceAdapterbuffer withaddImageToBuffer(ImageData.fromBitmap(bitmap)), and results come back through aCapturedResultReceiver. - The
RawTextLinesUnitintermediate result exposes the unverified lines of each frame, which is useful for live feedback, whileRecognizedTextLinesResultin the final result carries the verified lines. - Raise the
ImageAnalysistarget resolution (1080×1920 here): the default 640×480 stream makes MRZ characters too small to localize.
Common Developer Questions
How do I recognize MRZ text from a CameraX frame?
Convert the ImageProxy frame into a Bitmap (the BitmapUtils helper in this article does the YUV conversion and rotation), then push it into the router’s video pipeline with imageSourceAdapter.addImageToBuffer(ImageData.fromBitmap(bitmap)). The verified MRZ lines arrive in CapturedResultReceiver.onRecognizedTextLinesReceived and the unverified per-frame lines in the intermediate result receiver’s onRawTextLinesUnitReceived.
Why does capture(bitmap, "ReadPassportAndId") return no text lines?
The template verifies each MRZ read against check digits before emitting a final result. A single still-image capture() either fails this verification or only exposes the unverified lines through intermediate results. Run the video pipeline with startCapturing and feed the image as several frames, and the verified result plus parsed fields arrive through the result receiver.
Do I need to bundle MRZ model files in the app assets?
No. The mrzscannerbundle artifact includes the MRZ character models and the template definitions, so older steps such as copying .prototxt/.caffemodel files or appending character models are no longer required.
Why is the analyzer executed on a separate executor?
Every frame triggers a full recognition pass, which can take tens of milliseconds. Running it on the main thread would freeze the UI, so the sample uses a single-thread executor with CameraX’s STRATEGY_KEEP_ONLY_LATEST backpressure to drop queued frames.
What document types does the ReadPassportAndId template handle?
The template is designed for passports and ID cards, the two most common MRZ carriers. Point the camera at the MRZ zone and keep the document roughly flat and well lit for the best recognition rate.
Prerequisites
- Android Studio with the Android SDK (API level 24 or higher).
- An Android device or emulator with a camera.
- A Dynamsoft license key - the sample uses a time-limited trial key that requires a network connection on first use.
Get a 30-day free trial license
New Project
Open Android Studio and create a new project with an empty compose activity.
Add Dependencies
-
Open
settings.gradleto add Dynamsoft’s maven repository.dependencyResolutionManagement { repositoriesMode.set(RepositoriesMode.FAIL_ON_PROJECT_REPOS) repositories { google() mavenCentral() + maven { + url "https://download2.dynamsoft.com/maven/aar" + } } } -
Add CameraX and Dynamsoft MRZ Scanner to the module’s
build.gradle.// CameraX core library using the camera2 implementation def camerax_version = "1.3.4" implementation "androidx.camera:camera-core:${camerax_version}" implementation "androidx.camera:camera-camera2:${camerax_version}" implementation "androidx.camera:camera-lifecycle:${camerax_version}" implementation "androidx.camera:camera-view:${camerax_version}" implementation 'com.dynamsoft:mrzscannerbundle:3.4.1300'
Request Camera Permission
We have to request camera permission to use the camera.
-
Declare the camera permission in
AndroidManifest.xml.<uses-feature android:name="android.hardware.camera" android:required="false" /> <uses-permission android:name="android.permission.CAMERA" /> -
In
MainActivity.kt, add ahasCamPermissionstate.var hasCamPermission by remember { mutableStateOf( ContextCompat.checkSelfPermission( context, Manifest.permission.CAMERA ) == PackageManager.PERMISSION_GRANTED ) } -
Define a launcher to request the permission and call it when the app starts using
LaunchedEffect.val launcher = rememberLauncherForActivityResult( contract = ActivityResultContracts.RequestPermission(), onResult = { granted -> hasCamPermission = granted } ) LaunchedEffect(key1 = true) { if (!hasCamPermission) { launcher.launch(Manifest.permission.CAMERA) } }
Open the Camera and Bind the Analyzer
With the permission granted, we can display the preview and analyze frames. The analyzer resolution matters: the default 640×480 stream makes MRZ characters too small, so the sample targets 1080×1920.
AndroidView(
factory = { context ->
val previewView = PreviewView(context)
val preview = Preview.Builder().build()
val selector = CameraSelector.Builder()
.requireLensFacing(CameraSelector.LENS_FACING_BACK)
.build()
preview.setSurfaceProvider(previewView.surfaceProvider)
val imageAnalysis = ImageAnalysis.Builder()
.setTargetResolution(Size(1080, 1920))
.setBackpressureStrategy(STRATEGY_KEEP_ONLY_LATEST)
.build()
imageAnalysis.setAnalyzer(
analysisExecutor,
MRZAnalyzer(mrzRecognizer)
)
try {
cameraProviderFuture.get().bindToLifecycle(
lifecycleOwner,
selector,
preview,
imageAnalysis
)
} catch (e: Exception) {
e.printStackTrace()
}
previewView
},
modifier = Modifier.fillMaxSize()
)
Create the MRZ Recognition Engine
The recognition engine (MRZRecognizer) wraps one CaptureVisionRouter shared by the live camera and the still-image scanner.
-
Create the class with the license initialization.
public class MRZRecognizer { private final CaptureVisionRouter router; private final FrameSource frameSource; public MRZRecognizer(Context context) { initLicense(context); router = new CaptureVisionRouter(context); // ... } private void initLicense(Context context) { LicenseManager.initLicense("LICENSE-KEY", context, new LicenseVerificationListener() { @Override public void onLicenseVerified(boolean isSuccess, Exception error) { if (!isSuccess && error != null) { error.printStackTrace(); } } }); } } -
Load the MRZ templates bundled in the SDK. This step is mandatory: the
ReadPassportAndIdtemplate does not exist until the template file is loaded, and recognition silently returns zero results without it.try { router.initSettingsFromFile("mrzscanner-mobile-templates.json"); } catch (CaptureVisionRouterException e) { Log.e(TAG, "Failed to load MRZ templates: " + e.getMessage()); } -
Create a frame source and set it as the router’s input. The frame source is an
ImageSourceAdapterwhose buffer receives the frames from CameraX and the picked images. The overflow protection modeBOPM_UPDATEreplaces the oldest frame when the buffer is full, so the pipeline always processes the latest frame.private static class FrameSource extends ImageSourceAdapter { @Override public boolean hasNextImageToFetch() { return true; } } // in the constructor: frameSource = new FrameSource(); frameSource.setBufferOverflowProtectionMode(EnumBufferOverflowProtectionMode.BOPM_UPDATE); frameSource.setMaximumImageCount(3); try { router.setInput(frameSource); } catch (CaptureVisionRouterException e) { Log.e(TAG, "Failed to set input: " + e.getMessage()); } -
Register a result receiver for the verified MRZ lines and the parsed fields, and an intermediate result receiver for the unverified per-frame lines. Both also report the line locations, which the app draws as an overlay.
router.addResultReceiver(new CapturedResultReceiver() { @Override public void onRecognizedTextLinesReceived(RecognizedTextLinesResult result) { // Verified MRZ lines of the frame, if the check digits passed. if (result != null && result.getItems() != null && result.getItems().length > 0 && listener != null) { StringBuilder sb = new StringBuilder(); for (TextLineResultItem item : result.getItems()) { sb.append(item.getText()).append("\n"); } listener.onLines(sb.toString().trim(), true); } } @Override public void onParsedResultsReceived(ParsedResult result) { if (result != null && result.getItems() != null && result.getItems().length > 0 && listener != null) { listener.onParsed(result.getItems()[0].getCodeType(), result.getItems()[0].getParsedFields()); } } }); router.getIntermediateResultManager().addResultReceiver(new IntermediateResultReceiver() { @Override public void onRawTextLinesUnitReceived(RawTextLinesUnit u, IntermediateResultExtraInfo i) { // Unverified lines of the frame, used for live feedback. if (u.getRawTextLines() != null && u.getRawTextLines().length > 0) { // Build the lines string, merge with previous lines and // forward them plus the line locations to the listener. } } }); -
Add the control methods: start and stop the video pipeline, and feed frames into it.
public void start() { router.startCapturing("ReadPassportAndId", new CompletionListener() { @Override public void onSuccess() { } @Override public void onFailure(int errorCode, String errorString) { Log.e(TAG, "startCapturing failed: " + errorCode + " " + errorString); } }); } public void stop() { router.stopCapturing(); } public void feedBitmap(Bitmap bitmap) { if (fileScanActive) { return; } imageWidth = bitmap.getWidth(); imageHeight = bitmap.getHeight(); frameSource.addImageToBuffer(ImageData.fromBitmap(bitmap)); } -
For a static image, feed it through the same video pipeline several times so the multi-frame verification applies.
public void scanBitmapAsFrames(Bitmap bitmap, int frames, long intervalMs) { fileScanActive = true; frameSource.clearBuffer(); imageWidth = bitmap.getWidth(); imageHeight = bitmap.getHeight(); for (int i = 0; i < frames; i++) { frameSource.addImageToBuffer(ImageData.fromBitmap(bitmap)); try { Thread.sleep(intervalMs); } catch (InterruptedException e) { Thread.currentThread().interrupt(); break; } } } public void endFileScan() { fileScanActive = false; }
Create an Image Analyser for CameraX to Feed Camera Frames
CameraX needs an image analyzer instance to process camera frames. The analyzer converts each ImageProxy into a bitmap and pushes it into the pipeline.
class MRZAnalyzer(
private val mrzRecognizer: MRZRecognizer
): ImageAnalysis.Analyzer {
@OptIn(ExperimentalGetImage::class)
override fun analyze(image: ImageProxy) {
try {
val bitmap = BitmapUtils.getBitmap(image)
if (bitmap != null) {
mrzRecognizer.feedBitmap(bitmap)
}
} catch(e: Exception) {
e.printStackTrace()
} finally {
image.close()
}
}
}
We have to convert the camera frames in the ImageProxy format into a bitmap.
The BitmapUtils class used:
package com.tonyxlh.mrzscanner;
import android.annotation.TargetApi;
import android.graphics.Bitmap;
import android.graphics.BitmapFactory;
import android.graphics.ImageFormat;
import android.graphics.Matrix;
import android.graphics.Rect;
import android.graphics.YuvImage;
import android.media.Image;
import android.media.Image.Plane;
import android.os.Build.VERSION_CODES;
import androidx.annotation.Nullable;
import android.util.Log;
import androidx.annotation.RequiresApi;
import androidx.camera.core.ExperimentalGetImage;
import androidx.camera.core.ImageProxy;
import java.io.ByteArrayOutputStream;
import java.nio.ByteBuffer;
/** Utils functions for bitmap conversions. */
public class BitmapUtils {
private static final String TAG = "BitmapUtils";
/** Converts NV21 format byte buffer to bitmap. */
@Nullable
public static Bitmap getBitmap(ByteBuffer data, FrameMetadata metadata) {
data.rewind();
byte[] imageInBuffer = new byte[data.limit()];
data.get(imageInBuffer, 0, imageInBuffer.length);
try {
YuvImage image =
new YuvImage(
imageInBuffer, ImageFormat.NV21, metadata.getWidth(), metadata.getHeight(), null);
ByteArrayOutputStream stream = new ByteArrayOutputStream();
image.compressToJpeg(new Rect(0, 0, metadata.getWidth(), metadata.getHeight()), 80, stream);
Bitmap bmp = BitmapFactory.decodeByteArray(stream.toByteArray(), 0, stream.size());
stream.close();
return rotateBitmap(bmp, metadata.getRotation(), false, false);
} catch (Exception e) {
Log.e("VisionProcessorBase", "Error: " + e.getMessage());
}
return null;
}
/** Converts a YUV_420_888 image from CameraX API to a bitmap. */
@RequiresApi(VERSION_CODES.LOLLIPOP)
@Nullable
@ExperimentalGetImage
public static Bitmap getBitmap(ImageProxy image) {
FrameMetadata frameMetadata =
new FrameMetadata.Builder()
.setWidth(image.getWidth())
.setHeight(image.getHeight())
.setRotation(image.getImageInfo().getRotationDegrees())
.build();
ByteBuffer nv21Buffer =
yuv420ThreePlanesToNV21(image.getImage().getPlanes(), image.getWidth(), image.getHeight());
return getBitmap(nv21Buffer, frameMetadata);
}
/** Rotates a bitmap if it is converted from a bytebuffer. */
private static Bitmap rotateBitmap(
Bitmap bitmap, int rotationDegrees, boolean flipX, boolean flipY) {
Matrix matrix = new Matrix();
// Rotate the image back to straight.
matrix.postRotate(rotationDegrees);
// Mirror the image along the X or Y axis.
matrix.postScale(flipX ? -1.0f : 1.0f, flipY ? -1.0f : 1.0f);
Bitmap rotatedBitmap =
Bitmap.createBitmap(bitmap, 0, 0, bitmap.getWidth(), bitmap.getHeight(), matrix, true);
// Recycle the old bitmap if it has changed.
if (rotatedBitmap != bitmap) {
bitmap.recycle();
}
return rotatedBitmap;
}
/**
* Converts YUV_420_888 to NV21 bytebuffer.
*
* <p>The NV21 format consists of a single byte array containing the Y, U and V values. For an
* image of size S, the first S positions of the array contain all the Y values. The remaining
* positions contain interleaved V and U values. U and V are subsampled by a factor of 2 in both
* dimensions, so there are S/4 U values and S/4 V values. In summary, the NV21 array will contain
* S Y values followed by S/4 VU values: YYYYYYYYYYYYYY(...)YVUVUVUVU(...)VU
*
* <p>YUV_420_888 is a generic format that can describe any YUV image where U and V are subsampled
* by a factor of 2 in both dimensions. {@link Image#getPlanes} returns an array with the Y, U and
* V planes. The Y plane is guaranteed not to be interleaved, so we can just copy its values into
* the first part of the NV21 array. The U and V planes may already have the representation in the
* NV21 format. This happens if the planes share the same buffer, the V buffer is one position
* before the U buffer and the planes have a pixelStride of 2. If this is case, we can just copy
* them to the NV21 array.
*/
@RequiresApi(VERSION_CODES.KITKAT)
private static ByteBuffer yuv420ThreePlanesToNV21(
Plane[] yuv420888planes, int width, int height) {
int imageSize = width * height;
byte[] out = new byte[imageSize + 2 * (imageSize / 4)];
if (areUVPlanesNV21(yuv420888planes, width, height)) {
// Copy the Y values.
yuv420888planes[0].getBuffer().get(out, 0, imageSize);
ByteBuffer uBuffer = yuv420888planes[1].getBuffer();
ByteBuffer vBuffer = yuv420888planes[2].getBuffer();
// Get the first V value from the V buffer, since the U buffer does not contain it.
vBuffer.get(out, imageSize, 1);
// Copy the first U value and the remaining VU values from the U buffer.
uBuffer.get(out, imageSize + 1, 2 * imageSize / 4 - 1);
} else {
// Fallback to copying the UV values one by one, which is slower but also works.
// Unpack Y.
unpackPlane(yuv420888planes[0], width, height, out, 0, 1);
// Unpack U.
unpackPlane(yuv420888planes[1], width, height, out, imageSize + 1, 2);
// Unpack V.
unpackPlane(yuv420888planes[2], width, height, out, imageSize, 2);
}
return ByteBuffer.wrap(out);
}
/** Checks if the UV plane buffers of a YUV_420_888 image are in the NV21 format. */
@RequiresApi(VERSION_CODES.KITKAT)
private static boolean areUVPlanesNV21(Plane[] planes, int width, int height) {
int imageSize = width * height;
ByteBuffer uBuffer = planes[1].getBuffer();
ByteBuffer vBuffer = planes[2].getBuffer();
// Backup buffer properties.
int vBufferPosition = vBuffer.position();
int uBufferLimit = uBuffer.limit();
// Advance the V buffer by 1 byte, since the U buffer will not contain the first V value.
vBuffer.position(vBufferPosition + 1);
// Chop off the last byte of the U buffer, since the V buffer will not contain the last U value.
uBuffer.limit(uBufferLimit - 1);
// Check that the buffers are equal and have the expected number of elements.
boolean areNV21 =
(vBuffer.remaining() == (2 * imageSize / 4 - 2)) && (vBuffer.compareTo(uBuffer) == 0);
// Restore buffers to their initial state.
vBuffer.position(vBufferPosition);
uBuffer.limit(uBufferLimit);
return areNV21;
}
/**
* Unpack an image plane into a byte array.
*
* <p>The input plane data will be copied in 'out', starting at 'offset' and every pixel will be
* spaced by 'pixelStride'. Note that there is no row padding on the output.
*/
@TargetApi(VERSION_CODES.KITKAT)
private static void unpackPlane(
Plane plane, int width, int height, byte[] out, int offset, int pixelStride) {
ByteBuffer buffer = plane.getBuffer();
buffer.rewind();
// Compute the size of the current plane.
// We assume that it has the aspect ratio as the original image.
int numRow = (buffer.limit() + plane.getRowStride() - 1) / plane.getRowStride();
if (numRow == 0) {
return;
}
int scaleFactor = height / numRow;
int numCol = width / scaleFactor;
// Extract the data in the output buffer.
int outputPos = offset;
int rowStart = 0;
for (int row = 0; row < numRow; row++) {
int inputPos = rowStart;
for (int col = 0; col < numCol; col++) {
out[outputPos] = buffer.get(inputPos);
outputPos += pixelStride;
inputPos += plane.getPixelStride();
}
rowStart += plane.getRowStride();
}
}
}
Dependent FrameMetadata.java:
package com.tonyxlh.mrzscanner;
/** Describing a frame info. */
public class FrameMetadata {
private final int width;
private final int height;
private final int rotation;
public int getWidth() {
return width;
}
public int getHeight() {
return height;
}
public int getRotation() {
return rotation;
}
private FrameMetadata(int width, int height, int rotation) {
this.width = width;
this.height = height;
this.rotation = rotation;
}
/** Builder of {@link FrameMetadata}. */
public static class Builder {
private int width;
private int height;
private int rotation;
public Builder setWidth(int width) {
this.width = width;
return this;
}
public Builder setHeight(int height) {
this.height = height;
return this;
}
public Builder setRotation(int rotation) {
this.rotation = rotation;
return this;
}
public FrameMetadata build() {
return new FrameMetadata(width, height, rotation);
}
}
}
Draw the MRZ Lines as an Overlay on the Preview
The result listener of MRZRecognizer reports the polygon of every detected MRZ line in the analyzed image space. An MRZOverlayView stacked over the PreviewView draws them: verified lines in green, unverified lines in yellow, plus a dashed guide frame showing where to place the MRZ.
public void setTargets(List<float[]> quads, List<String> texts, List<Boolean> verified,
int imageWidth, int imageHeight) {
this.quads = quads;
this.texts = texts;
this.verified = verified;
this.imageWidth = imageWidth;
this.imageHeight = imageHeight;
postInvalidate();
}
The overlay maps the image-space coordinates to view coordinates with the FILL_CENTER scale mode of the PreviewView:
float scale = Math.max((float) viewWidth / imageWidth, (float) viewHeight / imageHeight);
float offsetX = (viewWidth - imageWidth * scale) / 2f;
float offsetY = (viewHeight - imageHeight * scale) / 2f;
// mapped point: (quad[i*2] * scale + offsetX, quad[i*2+1] * scale + offsetY)
In MainActivity, the two AndroidViews are stacked and the listener routes the polygons to the live overlay:
AndroidView(
factory = { context ->
overlayView = MRZOverlayView(context)
overlayView
},
modifier = Modifier.fillMaxSize()
)
The camera pipeline is started in onResume and stopped in onPause:
override fun onResume() {
super.onResume()
mrzRecognizer.start()
}
override fun onPause() {
super.onPause()
mrzRecognizer.stop()
}
The recognized lines are displayed in a fixed-height Text at the bottom of the screen.
Recognize MRZ from a Picked Image
Besides the live camera, the app can recognize the MRZ of a static image picked with the system photo picker.
-
Register a photo picker launcher.
val galleryLauncher = rememberLauncherForActivityResult( contract = ActivityResultContracts.PickVisualMedia() ) { uri: Uri? -> if (uri != null) { recognizeFromUri(uri) } } -
Decode the picked image and feed it through the same video pipeline. While the image is processed, the camera frames are skipped (
fileScanActive), the image is shown in a full-screen result view with its own overlay, and the app returns to the camera after a few seconds.private fun recognizeFromUri(uri: Uri) { runOnUiThread { codeText = "Recognizing..." } analysisExecutor.execute { val inputStream = contentResolver.openInputStream(uri) val bitmap = BitmapFactory.decodeStream(inputStream) inputStream?.close() if (bitmap != null) { runOnUiThread { fileImage = bitmap } mrzRecognizer.scanBitmapAsFrames(bitmap, 10, 150) Thread.sleep(3000) runOnUiThread { if (codeText == "Recognizing...") { codeText = "No MRZ recognized" } } mainHandler.postDelayed({ resumeLiveFromFileDialog() }, 8000) } } }
The verified result of the image is displayed like a live result. If the single-frame verification does not pass, the unverified raw lines of the image are shown instead.
Common Issues & Edge Cases
- Recognition silently returns zero results. The MRZ templates are not loaded by default. Call
router.initSettingsFromFile("mrzscanner-mobile-templates.json")before the firststartCapturing, otherwise no template exists. capture(bitmap, template)returns no text lines. A single frame must pass the check-digit verification on its own, which real photos often fail. Feed the image as several frames through the video pipeline (scanBitmapAsFrames) and read the verified result from the receiver, with the raw intermediate lines as a fallback.- Nothing is recognized in live mode. Check the analysis resolution first: the default 640×480 stream makes MRZ characters too small to localize. Target at least 1080×1920, and keep the MRZ inside the guide frame, well lit and in focus.
- The overlay jumps between positions. Raw and verified results use different coordinate spaces and may report slightly different polygons. Draw them in separate overlays (the sample routes live polygons to one view and file-scan polygons to another) and clear the overlay when a frame recognizes nothing.
- The app misses frames.
STRATEGY_KEEP_ONLY_LATESTkeeps the pipeline from backing up, but the MRZ should stay still for a moment to get a stable read.