Developing .NET MAUI iOS Apps to Scan Barcode, Document and MRZ
In the previous article, we demonstrated how to incorporate iOS MRZ detection frameworks into the MrzScannerSDK NuGet package. Having mastered the integration of iOS frameworks for a standalone NuGet package, we now turn our attention to integrating iOS SDKs for barcode, MRZ, and document detection into the comprehensive .NET MAUI library: Capture.Vision.Maui. Our ultimate goal is to enable the development of cross-platform .NET MAUI applications for Windows, Android and iOS using a single codebase.
This article is Part 4 in a 5-Part Series.
- Part 1 - How to Create a .NET MAUI Plugin for Camera Barcode Qr Code Scanning
- Part 2 - Integrating Document and MRZ Detection SDK into .NET MAUI for Windows
- Part 3 - Empowering .NET MAUI Android Apps with Document and MRZ Detection
- Part 4 - Developing .NET MAUI iOS Apps to Scan Barcode, Document and MRZ
- Part 5 - Building .NET MAUI Barcode Scanner with Visual Studio Code on macOS
Integrating iOS Frameworks into Document Detection NuGet Package
Based on our experience with integrating the MRZ detection SDK, we can apply similar steps to incorporate the iOS Document Detection SDK into the DocumentScannerSDK NuGet package.
Step 1: Add iOS Frameworks to the NuGet Package
The Dynamsoft Document Normalizer SDK offers a suite of frameworks for iOS:
DynamsoftCore
DynamsoftDocumentNormalizer
DynamsoftImageProcessing
DynamsoftIntermediateResult
First, we utilize Objective Sharpie to create the C# bindings for these frameworks. Following this, we add all iOS frameworks and generated files to an iOS binding project.
The csproj
file for the binding project is structured as follows:
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFramework>net7.0-ios</TargetFramework>
<Nullable>enable</Nullable>
<ImplicitUsings>true</ImplicitUsings>
<AssemblyName>DocumentScannerSDK</AssemblyName>
<IsBindingProject>true</IsBindingProject>
<GeneratePackageOnBuild>true</GeneratePackageOnBuild>
</PropertyGroup>
<ItemGroup>
<ObjcBindingApiDefinition Include="DynamsoftCore/ApiDefinitions.cs" />
<ObjcBindingCoreSource Include="DynamsoftCore/StructsAndEnums.cs" />
<NativeReference Include="DynamsoftCore.framework">
<Kind>Framework</Kind>
<Frameworks></Frameworks>
</NativeReference>
</ItemGroup>
<ItemGroup>
<ObjcBindingApiDefinition Include="DynamsoftDocumentNormalizer/ApiDefinitions.cs" />
<ObjcBindingCoreSource Include="DynamsoftDocumentNormalizer/StructsAndEnums.cs" />
<NativeReference Include="DynamsoftDocumentNormalizer.framework">
<Kind>Framework</Kind>
<Frameworks></Frameworks>
</NativeReference>
</ItemGroup>
<ItemGroup>
<ObjcBindingApiDefinition Include="DynamsoftImageProcessing/ApiDefinitions.cs" />
<NativeReference Include="DynamsoftImageProcessing.framework">
<Kind>Framework</Kind>
<Frameworks></Frameworks>
</NativeReference>
</ItemGroup>
<ItemGroup>
<ObjcBindingApiDefinition Include="DynamsoftIntermediateResult/ApiDefinitions.cs" />
<NativeReference Include="DynamsoftIntermediateResult.framework">
<Kind>Framework</Kind>
<Frameworks></Frameworks>
</NativeReference>
</ItemGroup>
</Project>
Step 2: Align the C# API with Android’s Implementation
Copy the DocumentScanner.cs
file from the Android binding project into the iOS binding project. Modify the code as necessary to ensure it functions correctly on iOS.
-
InitLicense
:public class LicenseVerification : LicenseVerificationListener { public override void LicenseVerificationCallback(bool isSuccess, NSError error) { if (!isSuccess) { System.Console.WriteLine(error.UserInfo); } } } public static void InitLicense(string license, object? context = null) { DynamsoftLicenseManager.InitLicense(license, new LicenseVerification()); }
-
SetParameters
:public void SetParameters(string parameters) { try { NSError error; normalizer.InitRuntimeSettingsFromString(parameters, out error); } catch (Exception ex) { Console.WriteLine(ex.ToString()); } }
-
DetectFile
andDetectBuffer
:public Result[]? DetectFile(string filename) { NSError error; iDetectedQuadResult[]? results = normalizer.DetectQuadFromFile(filename, out error); return GetResults(results); } private Result[]? GetResults(iDetectedQuadResult[]? results) { if (results == null) return null; var result = new Result[results.Length]; for (int i = 0; i < results.Length; i++) { iDetectedQuadResult tmp = results[i]; iQuadrilateral quad = tmp.Location; result[i] = new Result() { Confidence = (int)tmp.ConfidenceAsDocumentBoundary, Points = new int[8] { (int)((NSValue)quad.Points[0]).CGPointValue.X, (int)((NSValue)quad.Points[0]).CGPointValue.Y, (int)((NSValue)quad.Points[1]).CGPointValue.X, (int)((NSValue)quad.Points[1]).CGPointValue.Y, (int)((NSValue)quad.Points[2]).CGPointValue.X, (int)((NSValue)quad.Points[2]).CGPointValue.Y, (int)((NSValue)quad.Points[3]).CGPointValue.X, (int)((NSValue)quad.Points[3]).CGPointValue.Y, } }; } return result; } public Result[]? DetectBuffer(byte[] buffer, int width, int height, int stride, ImagePixelFormat format) { NSData converted = NSData.FromArray(buffer); iImageData imageData = new iImageData() { Bytes = converted, Width = width, Height = height, Stride = stride, Format = (EnumImagePixelFormat)format, }; NSError error; iDetectedQuadResult[]? results = normalizer.DetectQuadFromBuffer(imageData, out error); return GetResults(results); }
-
NormalizeFile
andNormalizeBuffer
:public NormalizedImage NormalizeFile(string filename, int[] points) { iQuadrilateral quad = new iQuadrilateral(); quad.Points = new NSObject[4]; quad.Points[0] = NSValue.FromCGPoint(new CGPoint(points[0], points[1])); quad.Points[1] = NSValue.FromCGPoint(new CGPoint(points[2], points[3])); quad.Points[2] = NSValue.FromCGPoint(new CGPoint(points[4], points[5])); quad.Points[3] = NSValue.FromCGPoint(new CGPoint(points[6], points[7])); NSError error; iNormalizedImageResult? result = normalizer.NormalizeFile(filename, quad, out error); return GetNormalizedImage(result); } public NormalizedImage NormalizeBuffer(byte[] buffer, int width, int height, int stride, ImagePixelFormat format, int[] points) { iImageData imageData = new iImageData() { Bytes = NSData.FromArray(buffer), Width = width, Height = height, Stride = stride, Format = (EnumImagePixelFormat)format, }; iQuadrilateral quad = new iQuadrilateral(); quad.Points = new NSObject[4]; quad.Points[0] = NSValue.FromCGPoint(new CGPoint(points[0], points[1])); quad.Points[1] = NSValue.FromCGPoint(new CGPoint(points[2], points[3])); quad.Points[2] = NSValue.FromCGPoint(new CGPoint(points[4], points[5])); quad.Points[3] = NSValue.FromCGPoint(new CGPoint(points[6], points[7])); NSError error; iNormalizedImageResult? result = normalizer.NormalizeBuffer(imageData, quad, out error); return GetNormalizedImage(result); } private NormalizedImage GetNormalizedImage(iNormalizedImageResult? result) { NormalizedImage normalizedImage = new NormalizedImage(); if (result != null) { iImageData imageData = result.Image; normalizedImage.Width = (int)imageData.Width; normalizedImage.Height = (int)imageData.Height; normalizedImage.Stride = (int)imageData.Stride; normalizedImage.Format = (ImagePixelFormat)imageData.Format; normalizedImage.Data = imageData.Bytes.ToArray(); } return normalizedImage; }
Step 3: Resolve Build Errors
The C# binding code generated may lead to build errors. These errors must be addressed and corrected manually.
Step 4: Compile the NuGet Package
# build dll for desktop
cd desktop
dotnet build --configuration Release
# build dll for android
cd android
dotnet build --configuration Release
# build dll for iOS
cd ios
dotnet build --configuration Release
# build nuget package
nuget pack .\DocumentScannerSDK.nuspec
Merging DocumentScannerSDK and MrzScannerSDK into a Single NuGet Package
The DynamsoftCore
framework serves as a crucial dependency for both the MrzScannerSDK
and DocumentScannerSDK
NuGet packages. Utilizing both packages within a .NET MAUI project leads to a build conflict due to this shared dependency. To overcome this challenge, we introduced a new NuGet package named CaptureVision, which consolidates all frameworks and C# files from both MrzScannerSDK
and DocumentScannerSDK
for compilation.
Resolving Build Conflicts between BarcodeQRCodeSDK and CaptureVision
The BarcodeQRCodeSDK NuGet package and the CaptureVision
NuGet package are compatible within the same .NET MAUI project for Windows and Android. However, a build conflict arises when attempting to compile our demo application for iOS. This conflict is due to certain data structures, such as iImageData
and iQuadrilateral
, being defined in both the DynamsoftCore
and BarcodeQRCodeSDK
. To address this issue, a viable solution involves removing the redundant data structures from the BarcodeQRCodeSDK
NuGet package.
Completing the Capture.Vision.Maui NuGet Package with iOS Support
Utilizing CaptureVision 2.0.1
and BarcodeQRCodeSDK 2.3.7
, we are now able to finalize the Capture.Vision.Maui
NuGet package by incorporating iOS-specific code within Platforms/iOS/NativeCameraView.cs
. The code snippet below illustrates the process of detecting documents and MRZ from camera images on iOS.
if (cameraView.EnableDocumentDetect)
{
DocumentScanner.Result[] results = documentScanner.DetectBuffer(bytearray,
(int)width,
(int)height,
(int)bpr,
DocumentScanner.ImagePixelFormat.IPF_ARGB_8888);
DocumentResult documentResults = new DocumentResult();
if (results != null && results.Length > 0)
{
documentResults = new DocumentResult
{
Confidence = results[0].Confidence,
Points = results[0].Points
};
if (cameraView.EnableDocumentRectify)
{
NormalizedImage normalizedImage = documentScanner.NormalizeBuffer(bytearray,
(int)width,
(int)height,
(int)bpr,
DocumentScanner.ImagePixelFormat.IPF_ARGB_8888, documentResults.Points);
documentResults.Width = normalizedImage.Width;
documentResults.Height = normalizedImage.Height;
documentResults.Stride = normalizedImage.Stride;
documentResults.Format = normalizedImage.Format;
documentResults.Data = normalizedImage.Data;
}
}
cameraView.NotifyResultReady(documentResults, (int)width, (int)height);
}
if (cameraView.EnableMrz)
{
MrzResult mrzResults = new MrzResult();
try
{
MrzScanner.Result[] results = mrzScanner.DetectBuffer(bytearray,
(int)width,
(int)height,
(int)bpr,
MrzScanner.ImagePixelFormat.IPF_ARGB_8888);
if (results != null && results.Length > 0)
{
Line[] rawData = new Line[results.Length];
string[] lines = new string[results.Length];
for (int i = 0; i < results.Length; i++)
{
rawData[i] = new Line()
{
Confidence = results[i].Confidence,
Text = results[i].Text,
Points = results[i].Points,
};
lines[i] = results[i].Text;
}
Dynamsoft.MrzResult info = MrzParser.Parse(lines);
mrzResults = new MrzResult()
{
RawData = rawData,
Type = info.Type,
Nationality = info.Nationality,
Surname = info.Surname,
GivenName = info.GivenName,
PassportNumber = info.PassportNumber,
IssuingCountry = info.IssuingCountry,
BirthDate = info.BirthDate,
Gender = info.Gender,
Expiration = info.Expiration,
Lines = info.Lines
};
}
}
catch (Exception ex)
{
System.Diagnostics.Debug.WriteLine(ex.Message);
}
cameraView.NotifyResultReady(mrzResults, (int)width, (int)height);
}
Developing a .NET MAUI iOS App for Barcode, Document, and MRZ Detection
To adapt the existing demo app for iOS, simply replicate the license initialization code from the Android section and incorporate it into Platforms/iOS/Program.cs
:
using ObjCRuntime;
using UIKit;
using Dynamsoft;
namespace Capture.Vision.Maui.Example
{
public class Program
{
static void Main(string[] args)
{
DocumentScanner.InitLicense("LICENSE-KEY");
BarcodeQRCodeReader.InitLicense("LICENSE-KEY");
MrzScanner.InitLicense("LICENSE-KEY");
UIApplication.Main(args, null, typeof(AppDelegate));
}
}
}