Empowering .NET MAUI Android Apps with Document and MRZ Detection

Last week, we enhanced the .NET MAUI library Capture.Vision.Maui by adding document and MRZ (Machine-Readable Zone) detection capabilities for the Windows platform. We also enabled our .NET MAUI application to scan barcodes, documents, and MRZs in Windows. This week, we will further enhance the library by adding Android support. We will outline the entire process, from the Android AAR package to the .NET library package, and finally to the .NET MAUI library package. After upgrading the .NET MAUI library with integrated, platform-specific code for Android, your .NET MAUI application will be able to detect QR codes, documents, and MRZs from camera images without modifying any C# code.

Demo: Scanning QR Codes, Documents, and MRZs in .NET MAUI Android Applications

NuGet Package

https://www.nuget.org/packages/Capture.Vision.Maui

.NET MRZ SDK for Android

  1. Download https://github.com/yushulx/dotnet-mrz-sdk and create a new Android Java Library Binding project in the project folder, using the same name as the desktop project.
  2. Add DynamsoftCore.aar, DynamsoftLabelRecognizer.aar and MRZLib-release.aar to the project. Note: MRZLib-release.aar is generated by the MRZLib project.
  3. Copy MrzParser.cs, MrzResult.cs, and MrzScanner.cs from the desktop project to the Android Java Library Binding project.
  4. Refactor MrzScanner.cs to utilize the Dynamsoft Label Recognizer Android library.

     using Com.Dynamsoft.Core;
     using Com.Dynamsoft.Dlr;
        
     namespace Dynamsoft
     {
         public class MrzScanner
         {
             private MRZRecognizer? recognizer;
             ...
        
             public class LicenseVerificationListener : Java.Lang.Object, ILicenseVerificationListener
             {
                 public void LicenseVerificationCallback(bool isSuccess, CoreException ex)
                 {
                     if (!isSuccess)
                     {
                         throw new Exception(ex.ToString());
                     }
                 }
             }
        
             public static void InitLicense(string license, object? context = null)
             {
                 if (context == null) { return; }
        
                 LicenseManager.InitLicense(license, (Android.Content.Context)context, new LicenseVerificationListener());
             }
        
             private MrzScanner()
             {
                 recognizer = new MRZRecognizer();
             }
        
             public static MrzScanner Create()
             {
                 return new MrzScanner();
             }
        
             ~MrzScanner()
             {
                 Destroy();
             }
        
             public void Destroy()
             {
                 recognizer = null;
             }
        
             public static string? GetVersionInfo()
             {
                 return MRZRecognizer.Version;
             }
        
             public Result[]? DetectFile(string filename)
             {
                 if (recognizer == null) return null;
        
                 DLRResult[]? mrzResult = recognizer.RecognizeFile(filename);
                 return GetResults(mrzResult);
             }
        
             public Result[]? DetectBuffer(byte[] buffer, int width, int height, int stride, ImagePixelFormat format)
             {
                 if (recognizer == null) return null;
        
                 ImageData imageData = new ImageData()
                 {
                     Bytes = buffer,
                     Width = width,
                     Height = height,
                     Stride = stride,
                     Format = (int)format,
                 };
                 DLRResult[]? mrzResult = recognizer.RecognizeBuffer(imageData);
                 return GetResults(mrzResult);
             }
        
             private Result[]? GetResults(DLRResult[]? mrzResult)
             {
                 if (mrzResult != null && mrzResult[0].LineResults != null)
                 {
                     DLRLineResult[] lines = mrzResult[0].LineResults.ToArray();
                     Result[] result = new Result[lines.Length];
                     for (int i = 0; i < lines.Length; i++)
                     {
                         result[i] = new Result()
                         {
                             Confidence = lines[i].Confidence,
                             Text = lines[i].Text ?? "",
                             Points = new int[8]
                             {
                                 lines[i].Location.Points[0].X,
                                 lines[i].Location.Points[0].Y,
                                 lines[i].Location.Points[1].X,
                                 lines[i].Location.Points[1].Y,
                                 lines[i].Location.Points[2].X,
                                 lines[i].Location.Points[2].Y,
                                 lines[i].Location.Points[3].X,
                                 lines[i].Location.Points[3].Y
                             }
                         };
                     }
        
                     return result;
                 }
        
                 return null;
             }
         }
     }
    
  5. Generate a MrzScannerSDK.nuspec file at the root directory.

     <?xml version="1.0" encoding="utf-8"?>
     <package>
       <metadata>
         <id>MrzScannerSDK</id>
         <version>1.3.5</version>
         <title>MRZ Scanner SDK</title>
         <authors>yushulx</authors>
         <requireLicenseAcceptance>false</requireLicenseAcceptance>
         <license type="expression">MIT</license>
         <readme>README.md</readme>
         <!-- <icon>icon.png</icon> -->
         <projectUrl>https://github.com/yushulx/dotnet-mrz-sdk</projectUrl>
         <description>The MRZ Scanner SDK is a .NET wrapper for Dynamsoft Label Recognizer,
           supporting x64 Windows, x64 Linux and Android.</description>
         <releaseNotes>Fixed build condition for Windows.</releaseNotes>
         <copyright>$copyright$</copyright>
         <tags>MRZ;mrz-scan;machine-readable-zone;mrz-detection;passport;visa;id-card;travel-document</tags>
       </metadata>
       <files>
         <file src="README.md" target="" />
         <file src="LICENSE.txt" target="" />
        
         <!-- Desktop -->
         <file src="desktop\lib\win\**\*.*" target="runtimes\win-x64\native" />
         <file src="desktop\lib\linux\**\*.*" target="runtimes\linux-x64\native" />
         <file src="desktop\bin\Release\net7.0\MrzScannerSDK.dll" target="lib\net7.0" />
         <file src="desktop\MrzScannerSDK.targets" target="build" />
         <file src="desktop\model\**\*.*" target="model" />
        
         <!-- Android -->
         <file src="android\sdk\bin\Release\net7.0-android\**\*.*" target="lib\net7.0-android33.0" />
        
       </files>
     </package>
    
  6. Build the desktop and Android projects, then bundle them into a single NuGet package.

     cd desktop
     dotnet build --configuration Release
    
     cd android
     dotnet build --configuration Release
        
     nuget pack .\MrzScannerSDK.nuspec
    

    MRZ NuGet package

.NET Document SDK for Android

  1. Download https://github.com/yushulx/dotnet-document-scanner-sdk and create a new Android Java Library Binding project in the project folder, using the same name as the desktop project.
  2. Add DynamsoftCore.aar, DynamsoftDocumentNormalizer.aar, DynamsoftImageProcessing.aar and DynamsoftIntermediateResult.aar to the project.
  3. Copy DocumentScanner.cs from the desktop project to the Android Java Library Binding project.
  4. Refactor the DocumentScanner.cs to utilize the Dynamsoft Document Normalizer Android library.

     using Com.Dynamsoft.Core;
     using Com.Dynamsoft.Ddn;
        
     namespace Dynamsoft
     {
         public class DocumentScanner
         {
             private DocumentNormalizer normalizer;
        
             public class NormalizedImage
             {
                 public int Width;
                 public int Height;
                 public int Stride;
                 public ImagePixelFormat Format;
                 public byte[] Data = new byte[0];
             }
        
             public static string GetVersionInfo()
             {
                 return DocumentNormalizer.Version;
             }
        
             ...
        
             public class LicenseVerificationListener : Java.Lang.Object, ILicenseVerificationListener
             {
                 public void LicenseVerificationCallback(bool isSuccess, CoreException ex)
                 {
                     if (!isSuccess)
                     {
                         throw new Exception(ex.ToString());
                     }
                 }
             }
        
             public static void InitLicense(string license, object? context = null)
             {
                 if (context == null) { return; }
        
                 LicenseManager.InitLicense(license, (Android.Content.Context)context, new LicenseVerificationListener());
             }
        
             private DocumentScanner()
             {
                 normalizer = new DocumentNormalizer();
             }
        
             public static DocumentScanner Create()
             {
                 return new DocumentScanner();
             }
        
             public void SetParameters(string parameters)
             {
                 try
                 {
                     normalizer.InitRuntimeSettingsFromString(parameters);
                 }
                 catch (Exception ex)
                 {
                     Console.WriteLine(ex.ToString());
                 }
             }
        
             public Result[]? DetectFile(string filename)
             {
                 DetectedQuadResult[]? results = normalizer.DetectQuad(filename);
                 return GetResults(results);
             }
        
             public Result[]? DetectBuffer(byte[] buffer, int width, int height, int stride, ImagePixelFormat format)
             {
                 ImageData imageData = new ImageData()
                 {
                     Bytes = buffer,
                     Width = width,
                     Height = height,
                     Stride = stride,
                     Format = (int)format,
                 };
                 DetectedQuadResult[]? results = normalizer.DetectQuad(imageData);
                 return GetResults(results);
             }
        
             private Result[]? GetResults(DetectedQuadResult[]? results)
             {
                 if (results == null) return null;
        
                 var result = new Result[results.Length];
                 for (int i = 0; i < results.Length; i++)
                 {
                     DetectedQuadResult tmp = results[i];
                     Quadrilateral quad = tmp.Location;
                     result[i] = new Result()
                     {
                         Confidence = tmp.ConfidenceAsDocumentBoundary,
                         Points = new int[8]
                         {
                             quad.Points[0].X, quad.Points[0].Y,
                             quad.Points[1].X, quad.Points[1].Y,
                             quad.Points[2].X, quad.Points[2].Y,
                             quad.Points[3].X, quad.Points[3].Y,
                         }
                     };
                 }
        
                 return result;
             }
        
             public NormalizedImage NormalizeFile(string filename, int[] points)
             {
                 Quadrilateral quad = new Quadrilateral();
                 quad.Points = new Android.Graphics.Point[4];
                 quad.Points[0] = new Android.Graphics.Point(points[0], points[1]);
                 quad.Points[1] = new Android.Graphics.Point(points[2], points[3]);
                 quad.Points[2] = new Android.Graphics.Point(points[4], points[5]);
                 quad.Points[3] = new Android.Graphics.Point(points[6], points[7]);
                 NormalizedImageResult? result = normalizer.Normalize(filename, quad);
                 return GetNormalizedImage(result);
             }
        
             public NormalizedImage NormalizeBuffer(byte[] buffer, int width, int height, int stride, ImagePixelFormat format, int[] points)
             {
                 ImageData imageData = new ImageData()
                 {
                     Bytes = buffer,
                     Width = width,
                     Height = height,
                     Stride = stride,
                     Format = (int)format,
                 };
        
                 Quadrilateral quad = new Quadrilateral();
                 quad.Points = new Android.Graphics.Point[4];
                 quad.Points[0] = new Android.Graphics.Point(points[0], points[1]);
                 quad.Points[1] = new Android.Graphics.Point(points[2], points[3]);
                 quad.Points[2] = new Android.Graphics.Point(points[4], points[5]);
                 quad.Points[3] = new Android.Graphics.Point(points[6], points[7]);
                 NormalizedImageResult? result = normalizer.Normalize(imageData, quad);
                 return GetNormalizedImage(result);
             }
        
             private NormalizedImage GetNormalizedImage(NormalizedImageResult? result)
             {
                 NormalizedImage normalizedImage = new NormalizedImage();
                 if (result != null)
                 {
                     ImageData imageData = result.Image;
                     normalizedImage.Width = imageData.Width;
                     normalizedImage.Height = imageData.Height;
                     normalizedImage.Stride = imageData.Stride;
                     normalizedImage.Format = (ImagePixelFormat)imageData.Format;
                     normalizedImage.Data = imageData.Bytes.ToArray();
                 }
                 return normalizedImage;
             }
         }
     }
    
  5. Generate a MrzScannerSDK.nuspec file in the root directory.

     <?xml version="1.0" encoding="utf-8"?>
     <package>
         <metadata>
             <id>DocumentScannerSDK</id>
             <version>1.2.0</version>
             <title>Document Scanner SDK</title>
             <authors>yushulx</authors>
             <requireLicenseAcceptance>false</requireLicenseAcceptance>
             <license type="expression">MIT</license>
             <readme>README.md</readme>
             <!-- <icon>icon.png</icon> -->
             <projectUrl>https://github.com/yushulx/dotnet-document-scanner-sdk</projectUrl>
             <description>The Document Scanner SDK is a .NET wrapper for Dynamsoft Document Normalizer,
                 supporting x64 Windows, x64 Linux and Android.</description>
             <releaseNotes>Added support for Android.</releaseNotes>
             <copyright>$copyright$</copyright>
             <tags>document;document-scan;edge-detection;document-detection</tags>
         </metadata>
         <files>
             <file src="README.md" target="" />
             <file src="LICENSE.txt" target="" />
        
             <!-- Desktop -->
             <file src="desktop\lib\win\**\*.*" target="runtimes\win-x64\native" />
             <file src="desktop\lib\linux\**\*.*" target="runtimes\linux-x64\native" />
             <file src="desktop\bin\Release\net7.0\DocumentScannerSDK.dll" target="lib\net7.0" />
        
             <!-- Android -->
             <file src="android\bin\Release\net7.0-android\**\*.*" target="lib\net7.0-android33.0" />
        
         </files>
     </package>
    
  6. Build the desktop and Android projects, then bundle them into a single NuGet package.

     cd desktop
     dotnet build --configuration Release
    
     cd android
     dotnet build --configuration Release
        
     nuget pack .\DocumentScannerSDK.nuspec
    

    Document NuGet package

Troubleshooting Error XA4215 When Building a .NET MAUI Library Project

When building a .NET MAUI library project that includes dependencies on the .NET MRZ SDK and the .NET Document SDK, you may encounter the following error:

Severity	Code	Description	Project	File	Line	Suppression State	Details
Error	XA4215	  `mono.com.dynamsoft.core.LicenseVerificationListenerImplementor` generated by: Com.Dynamsoft.Core.ILicenseVerificationListenerImplementor, MrzScannerSDK, Version=0.1.4.0, Culture=neutral, PublicKeyToken=null	Capture.Vision.Maui.Example	C:\Program Files\dotnet\packs\Microsoft.Android.Sdk.Windows\33.0.95\tools\Xamarin.Android.Common.targets	1476		
Error	XA4215	  `mono.com.dynamsoft.core.LicenseVerificationListenerImplementor` generated by: Com.Dynamsoft.Core.ILicenseVerificationListenerImplementor, DocumentScannerSDK, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null	Capture.Vision.Maui.Example	C:\Program Files\dotnet\packs\Microsoft.Android.Sdk.Windows\33.0.95\tools\Xamarin.Android.Common.targets	1476		

The error XA4215 you’re encountering in your .NET MAUI project indicates a conflict between two assemblies trying to generate the same class, mono.com.dynamsoft.core.LicenseVerificationListenerImplementor. To resolve the issue, a feasible workaround is to merge the two packages into one.

Merging .NET MRZ SDK and .NET Document SDK into One .NET Library

  1. Create a Class Library project and an Android Java Library Binding project in the same folder, ensuring both projects share the same name, e.g., CaptureVision. The Class Library project will house .NET code and libraries for desktop platforms (Windows and Linux), whereas the Android Java Library Binding project will house .NET code and libraries for Android.
  2. Copy all Android .aar packages and C# files from MrzScannerSDK and DocumentScannerSDK android folder to the Android Java Library Binding project.

    .NET capture vision library

  3. Copy MRZ model files, *.so files, *.dll files, and C# files from MrzScannerSDK and DocumentScannerSDK desktop folder to the Class Library project, maintaining the original folder structure.

     ├── model
     ├── lib
         ├── linux
         ├── win
     ├── CaptureVision.csproj
     ├── CaptureVision.targets
     ├── DocumentScaner.cs
     ├── MrzParser.cs
     ├── MrzResult.cs
     ├── MrzScanner.cs
    
  4. Generate a CaptureVision.nuspec file in the root directory.

     <?xml version="1.0" encoding="utf-8"?>
     <package>
         <metadata>
             <id>CaptureVision</id>
             <version>1.0.1</version>
             <title>Capture Vision SDK</title>
             <authors>yushulx</authors>
             <requireLicenseAcceptance>false</requireLicenseAcceptance>
             <license type="expression">MIT</license>
             <readme>README.md</readme>
             <!-- <icon>icon.png</icon> -->
             <projectUrl>https://github.com/yushulx/Capture-Vision</projectUrl>
             <description>This is a package that is a compound of DocumentScannerSDK and MrzScannerSDK.</description>
             <releaseNotes>Merged DocumentScannerSDK and MrzScannerSDK into one package.</releaseNotes>
             <copyright>$copyright$</copyright>
             <tags>
                 MRZ;Android;passport;id-card;visa;machine-readable-zone;document;document-scan;edge-detection;document-detection</tags>
         </metadata>
         <files>
             <file src="README.md" target="" />
             <file src="LICENSE.txt" target="" />
        
             <!-- Desktop -->
             <file src="desktop\lib\win\**\*.*" target="runtimes\win-x64\native" />
             <file src="desktop\lib\linux\**\*.*" target="runtimes\linux-x64\native" />
             <file src="desktop\bin\Release\net7.0\CaptureVision.dll" target="lib\net7.0" />
             <file src="desktop\CaptureVision.targets" target="build" />
             <file src="desktop\model\**\*.*" target="model" />
        
        
             <!-- Android -->
             <file src="android\bin\Release\net7.0-android\**\*.*" target="lib\net7.0-android33.0" />
        
         </files>
     </package>
    
  5. Build the library projects respectively and pack them into a single NuGet package.

     cd desktop
     dotnet build --configuration Release
        
     cd android
     dotnet build --configuration Release
        
     nuget pack .\CaptureVision.nuspec
    

Adding Android Platform-Specific Code to .NET MAUI Library

After generating the NuGet package, incorporate it as a dependency into your .NET MAUI library project.

<ItemGroup Condition="'$(TargetFramework)' == 'net7.0-android'">
	<PackageReference Include="CaptureVision " Version="1.0.1" />
</ItemGroup>

<ItemGroup Condition="$([MSBuild]::IsOSPlatform('windows'))">
	<PackageReference Include="CaptureVision " Version="1.0.1" />
</ItemGroup>

Next, navigate to Platforms/Android/NativeCameraView.cs within your project, and insert the following code snippet:

private BarcodeQRCodeReader barcodeReader;
private MrzScanner mrzScanner;
private DocumentScanner documentScanner;

public NativeCameraView(Context context, CameraView cameraView) : base(context)
{
    ...

    barcodeReader = BarcodeQRCodeReader.Create();
    mrzScanner = MrzScanner.Create();
    documentScanner = DocumentScanner.Create();
}

private void StartPreview()
{
    ...
    backgroundHandler = new Handler(backgroundThread.Looper);
    frameListener = new ImageAvailableListener(cameraView, barcodeReader, mrzScanner, documentScanner);
    ...
}

class ImageAvailableListener : Java.Lang.Object, ImageReader.IOnImageAvailableListener
{
    private readonly CameraView cameraView;
    private BarcodeQRCodeReader barcodeReader;
    private MrzScanner mrzScanner;
    private DocumentScanner documentScanner;

    public ImageAvailableListener(CameraView camView, BarcodeQRCodeReader barcodeReader, MrzScanner mrzScanner, DocumentScanner documentScanner)
    {
        cameraView = camView;
        this.barcodeReader = barcodeReader;
        this.mrzScanner = mrzScanner;
        this.documentScanner = documentScanner;
    }

    public void OnImageAvailable(ImageReader reader)
    {
        try
        {
            var image = reader?.AcquireLatestImage();
            ...

            if (cameraView.EnableDocumentDetect)
            {
                DocumentScanner.Result[] results = documentScanner.DetectBuffer(bytes, width, height, nPixelStride * nRowStride, DocumentScanner.ImagePixelFormat.IPF_GRAYSCALED);
                DocumentResult documentResults = new DocumentResult();
                ...
                cameraView.NotifyResultReady(documentResults, width, height);
            }

            if (cameraView.EnableMrz)
            {
                MrzResult mrzResults = new MrzResult();
                try
                {
                    MrzScanner.Result[] results = mrzScanner.DetectBuffer(bytes, width, height, nPixelStride * nRowStride, MrzScanner.ImagePixelFormat.IPF_GRAYSCALED);
                    ...
                }

                catch (Exception ex)
                {
                    System.Diagnostics.Debug.WriteLine(ex.Message);
                }
                cameraView.NotifyResultReady(mrzResults, width, height);
            }
        }
        catch (Exception ex)
        {
            System.Diagnostics.Debug.WriteLine(ex.Message);
        }
    }
}

Upgrading Capture.Vision.Maui to Scan QR Code, Document, and MRZ in .NET MAUI Android Applications

In the .NET MAUI application project, upgrade the Capture.Vision.Maui library to its latest version. After upgrading, rebuild the project and execute it on an Android device.

QR Code Scan

.NET MAUI Android: scan QR code

Document Edge Detection

.NET MAUI Android: scan document

MRZ Recognition

.NET MAUI barcode: scan MRZ

Source Code

https://github.com/yushulx/Capture-Vision-Maui