Implementing Flutter QR Code Scanner with Swift and AVFoundation for iOS
In the previous article, we implemented a Flutter barcode and QR code scanner for Android using Kotlin and CameraX. Since the Dart code is platform-independent, no changes are necessary. In this article, we will take steps to implement the native camera and barcode scanning logic for iOS using Swift, AVFoundation, and the Dynamsoft Barcode Reader SDK.
This article is Part 11 in a 11-Part Series.
- Part 1 - How to Turn Smartphone into a Wireless Keyboard and Barcode QR Scanner in Flutter
- Part 2 - How to Build Windows Desktop Barcode QR Scanner in Flutter
- Part 3 - Scan Documents Using Camera on Multiple Platforms: A Flutter Guide for Windows, Android, iOS, and Web
- Part 4 - How to Build a Prototype for a Last-Mile Delivery App with Flutter and Dynamsoft Vision SDKs
- Part 5 - How to Create a Cross-platform MRZ Scanner App Using Flutter and Dynamsoft Label Recognizer
- Part 6 - How to Build a Barcode Scanner App with Flutter Step by Step
- Part 7 - How to Build a Cross-platform Document Scanner App with Flutter
- Part 8 - Building an AR-Enhanced Pharma Lookup App with Flutter, Dynamsoft Barcode SDK and Database
- Part 9 - Implementing Flutter Barcode Scanner with Kotlin and CameraX for Android
- Part 10 - Building Multiple Barcode, QR Code and DataMatrix Scanner with Flutter for Inventory Management
- Part 11 - Implementing Flutter QR Code Scanner with Swift and AVFoundation for iOS
Prerequisites
Step 1: Installing Dynamsoft Barcode Reader for iOS
We use CocoaPods to install the Dynamsoft Barcode Reader SDK for iOS. If you haven’t installed CocoaPods yet, please follow the official instructions here to do so.
Once CocoaPods is ready, create a Podfile
in the iOS folder of your Flutter project:
cd ios
pod init
Next, edit the Podfile
to include the Dynamsoft Barcode Reader SDK:
target 'Runner' do
use_frameworks!
pod 'DynamsoftBarcodeReader','9.6.40'
end
Save the Podfile
and run pod install
. This command will install or update the CocoaPods dependencies, including the Flutter framework required for your iOS project.
Step 2: Adding Camera Permission to Info.plist
To enable camera access on iOS, open the Info.plist
file located in the ios/Runner
folder. Add the following keys:
<key>NSCameraUsageDescription</key>
<string>your usage description here</string>
<key>NSMicrophoneUsageDescription</key>
<string>your usage description here</string>
Step 3: Implementing Camera Preview with Flutter Texture in Swift
The Runner/AppDelegate.swift
file serves as the entry point of the Flutter application. By default, it contains the following boilerplate code:
override func application(
_ application: UIApplication,
didFinishLaunchingWithOptions launchOptions: [UIApplication.LaunchOptionsKey: Any]?
) -> Bool {
GeneratedPluginRegistrant.register(with: self)
return super.application(application, didFinishLaunchingWithOptions: launchOptions)
}
To integrate the camera functionality effectively, you’ll need to enhance the application method with the following steps:
- Establish a Flutter method channel. This is crucial for seamless communication between the Dart environment and Swift, allowing commands and data to be exchanged between the Flutter UI and native code.
- Implement a startCamera() method. This method should initiate the camera preview and continuously render this preview into a Flutter texture. This involves setting up the camera capture session, configuring input and output, and linking the camera output to a Flutter texture that can be displayed in the UI.
Flutter Method Channel in Swift
The Flutter method channel is a named channel that facilitates the sending of data between Dart and platform-specific code.
private var channel: FlutterMethodChannel?
private var width = 1920
private var height = 1080
override func application(
_ application: UIApplication,
didFinishLaunchingWithOptions launchOptions: [UIApplication.LaunchOptionsKey: Any]?
) -> Bool {
channel = FlutterMethodChannel(name: CHANNEL, binaryMessenger: flutterViewController.binaryMessenger)
channel?.setMethodCallHandler({
(call: FlutterMethodCall, result: @escaping FlutterResult) -> Void in
if call.method == "startCamera" {
self.startCamera(result: result)
} else if call.method == "getPreviewWidth" {
result(self.width)
} else if call.method == "getPreviewHeight" {
result(self.height)
}
else {
result(FlutterMethodNotImplemented)
}
})
GeneratedPluginRegistrant.register(with: self)
return super.application(application, didFinishLaunchingWithOptions: launchOptions)
}
- The
channel
variable is an instance ofFlutterMethodChannel
. It receives method calls fromDart
using thesetMethodCallHandler
method. Additionally, theinvokeMethod
method is used to send data fromSwift
toDart
. - The
width
andheight
variables, which store the camera preview size, are hardcoded to1920x1080
here. The methodsgetPreviewWidth
andgetPreviewHeight
retrieve and return these values to Dart, respectively.
Creating Flutter Texture and Camera Preview
Define the CustomCameraTexture
class that extends NSObject
and implements the FlutterTexture
protocol:
class CustomCameraTexture: NSObject, FlutterTexture {
private weak var textureRegistry: FlutterTextureRegistry?
var textureId: Int64?
private var cameraPreviewLayer: AVCaptureVideoPreviewLayer?
private let bufferQueue = DispatchQueue(label: "com.example.flutter/barcode_scan")
private var _lastSampleBuffer: CMSampleBuffer?
private var customCameraTexture: CustomCameraTexture?
private var lastSampleBuffer: CMSampleBuffer? {
get {
var result: CMSampleBuffer?
bufferQueue.sync {
result = _lastSampleBuffer
}
return result
}
set {
bufferQueue.sync {
_lastSampleBuffer = newValue
}
}
}
init(cameraPreviewLayer: AVCaptureVideoPreviewLayer, registry: FlutterTextureRegistry) {
self.cameraPreviewLayer = cameraPreviewLayer
self.textureRegistry = registry
super.init()
self.textureId = registry.register(self)
}
func copyPixelBuffer() -> Unmanaged<CVPixelBuffer>? {
guard let sampleBuffer = lastSampleBuffer, let pixelBuffer = CMSampleBufferGetImageBuffer(sampleBuffer) else {
return nil
}
return Unmanaged.passRetained(pixelBuffer)
}
func update(sampleBuffer: CMSampleBuffer) {
lastSampleBuffer = sampleBuffer
textureRegistry?.textureFrameAvailable(textureId!)
}
deinit {
if let textureId = textureId {
textureRegistry?.unregisterTexture(textureId)
}
}
}
- The
textureRegistry
variable is an instance ofFlutterTextureRegistry
. It is used to register and unregister the Flutter texture. - The
textureId
variable stores the Flutter texture ID, which will be used to render the camera preview in Flutter. - The
copyPixelBuffer()
method returns the latest pixel buffer for texture rendering. - The
update()
method appends a new camera frame and then notifies the Flutter texture that it needs to be updated. When thetextureFrameAvailable()
method is invoked, thecopyPixelBuffer()
method is triggered to fetch the latest pixel buffer.
Create a flutterTextureEntry
variable in the AppDelegate
class and obtain the Flutter texture registry within the application
method:
private var flutterTextureEntry: FlutterTextureRegistry?
override func application(
_ application: UIApplication,
didFinishLaunchingWithOptions launchOptions: [UIApplication.LaunchOptionsKey: Any]?
) -> Bool {
...
guard let flutterViewController = window?.rootViewController as? FlutterViewController else {
return super.application(application, didFinishLaunchingWithOptions: launchOptions)
}
flutterTextureEntry = flutterViewController.engine!.textureRegistry
GeneratedPluginRegistrant.register(with: self)
return super.application(application, didFinishLaunchingWithOptions: launchOptions)
}
In the startCamera()
initiate a camera session and add a video data output to capture the camera frames. When a new frame is captured in the captureOutput()
method, update the CustomCameraTexture
instance with the latest sample buffer:
private func startCamera(result: @escaping FlutterResult) {
if cameraSession != nil {
result(self.customCameraTexture?.textureId)
return
}
cameraSession = AVCaptureSession()
cameraSession?.sessionPreset = .hd1920x1080
guard let backCamera = AVCaptureDevice.default(for: .video), let input = try? AVCaptureDeviceInput(device: backCamera) else {
result(FlutterError(code: "no_camera", message: "No camera available", details: nil))
return
}
cameraSession?.addInput(input)
cameraPreviewLayer = AVCaptureVideoPreviewLayer(session: cameraSession!)
cameraPreviewLayer?.videoGravity = .resizeAspectFill
let cameraOutput = AVCaptureVideoDataOutput()
cameraOutput.videoSettings = [kCVPixelBufferPixelFormatTypeKey as String: kCVPixelFormatType_32BGRA]
cameraOutput.setSampleBufferDelegate(self, queue: DispatchQueue(label: "camera_frame_queue"))
cameraSession?.addOutput(cameraOutput)
self.customCameraTexture = CustomCameraTexture(cameraPreviewLayer: cameraPreviewLayer!, registry: flutterTextureEntry!)
cameraSession?.startRunning()
result(self.customCameraTexture?.textureId)
}
func captureOutput(_ output: AVCaptureOutput, didOutput sampleBuffer: CMSampleBuffer, from connection: AVCaptureConnection) {
if connection.isVideoOrientationSupported {
connection.videoOrientation = currentVideoOrientation()
}
self.customCameraTexture?.update(sampleBuffer: sampleBuffer)
}
At this point, the camera preview should function correctly on iOS/iPadOS. Next, we will integrate the Dynamsoft Barcode Reader SDK to decode barcodes and QR codes.
Step 4: Integrating Dynamsoft Barcode Reader SDK for iOS in Swfit
-
Import the Dynamsoft Barcode Reader SDK in the
AppDelegate.swift
file:import DynamsoftBarcodeReader
-
Create an instance of Dynamsoft Barcode Reader and activate it with a valid license key:
@UIApplicationMain @objc class AppDelegate: FlutterAppDelegate, AVCaptureVideoDataOutputSampleBufferDelegate, DBRLicenseVerificationListener { private let reader = DynamsoftBarcodeReader() override func application( _ application: UIApplication, didFinishLaunchingWithOptions launchOptions: [UIApplication.LaunchOptionsKey: Any]? ) -> Bool { DynamsoftBarcodeReader.initLicense("LICENSE-KEY", verificationDelegate: self) do { let settings = try? reader.getRuntimeSettings() settings!.expectedBarcodesCount = 999 try reader.updateRuntimeSettings(settings!) } catch { print("Error getting runtime settings") } ... GeneratedPluginRegistrant.register(with: self) return super.application(application, didFinishLaunchingWithOptions: launchOptions) } func dbrLicenseVerificationCallback(_ isSuccess: Bool, error: Error?) { if isSuccess { print("License verification passed") } else { print("License verification failed: \(error?.localizedDescription ?? "Unknown error")") } } }
-
Decode barcode and QR code from the camera frame in the
captureOutput()
method. Since the decoding API is CPU-intensive, to avoid blocking the camera preview rendering, we move the decoding logic to a separate thread:func captureOutput(_ output: AVCaptureOutput, didOutput sampleBuffer: CMSampleBuffer, from connection: AVCaptureConnection) { if connection.isVideoOrientationSupported { connection.videoOrientation = currentVideoOrientation() } self.customCameraTexture?.update(sampleBuffer: sampleBuffer) if !isProcessing { isProcessing = true DispatchQueue.global(qos: .background).async { self.processImage(sampleBuffer) self.isProcessing = false } } }
The
isProcessing
boolean variable ensures that only one frame is processed at a time, and new frames are ignored until the processing is complete. This approach helps mitigate the accumulation of asynchronous tasks and prevents the app from crashing due to memory exhaustion. -
Implement the
processImage()
method to decode barcodes fromCMSampleBuffer
and send the results to the Flutter UI via the method channel:func processImage(_ sampleBuffer: CMSampleBuffer) { let imageBuffer:CVImageBuffer = CMSampleBufferGetImageBuffer(sampleBuffer)! CVPixelBufferLockBaseAddress(imageBuffer, .readOnly) let baseAddress = CVPixelBufferGetBaseAddress(imageBuffer) let bufferSize = CVPixelBufferGetDataSize(imageBuffer) let width = CVPixelBufferGetWidth(imageBuffer) let height = CVPixelBufferGetHeight(imageBuffer) let bpr = CVPixelBufferGetBytesPerRow(imageBuffer) CVPixelBufferUnlockBaseAddress(imageBuffer, .readOnly) let buffer = Data(bytes: baseAddress!, count: bufferSize) let imageData = iImageData.init() imageData.bytes = buffer imageData.width = width imageData.height = height imageData.stride = bpr imageData.format = .ARGB_8888 imageData.orientation = 0 let results = try? reader.decodeBuffer(imageData) DispatchQueue.main.async { self.channel?.invokeMethod("onBarcodeDetected", arguments: self.wrapResults(results: results)) } } func wrapResults(results:[iTextResult]?) -> NSArray { let outResults = NSMutableArray(capacity: 8) if results == nil { return outResults } for item in results! { let subDic = NSMutableDictionary(capacity: 11) if item.barcodeFormat_2 != EnumBarcodeFormat2.Null { subDic.setObject(item.barcodeFormatString_2 ?? "", forKey: "format" as NSCopying) }else{ subDic.setObject(item.barcodeFormatString ?? "", forKey: "format" as NSCopying) } let points = item.localizationResult?.resultPoints as! [CGPoint] subDic.setObject(Int(points[0].x), forKey: "x1" as NSCopying) subDic.setObject(Int(points[0].y), forKey: "y1" as NSCopying) subDic.setObject(Int(points[1].x), forKey: "x2" as NSCopying) subDic.setObject(Int(points[1].y), forKey: "y2" as NSCopying) subDic.setObject(Int(points[2].x), forKey: "x3" as NSCopying) subDic.setObject(Int(points[2].y), forKey: "y3" as NSCopying) subDic.setObject(Int(points[3].x), forKey: "x4" as NSCopying) subDic.setObject(Int(points[3].y), forKey: "y4" as NSCopying) subDic.setObject(item.localizationResult?.angle ?? 0, forKey: "angle" as NSCopying) subDic.setObject(item.barcodeBytes ?? "", forKey: "barcodeBytes" as NSCopying) outResults.add(subDic) } return outResults }
Running the Flutter QR Code Scanner on iOS
flutter run
Source Code
https://github.com/yushulx/flutter-barcode-mrz-document-scanner/tree/main/examples/native_camera