Accelerating Android QR Code Detection with TensorFlow Lite
I conducted an experiment to evaluate QR code detection performance using YOLOv4 on Android. The results showed that running the YOLOv4 model on the CPU took over 100ms per inference, which fell short of my expectations. To improve inference speed, I decided to leverage the Android Neural Networks API (NNAPI) for hardware acceleration by training a TensorFlow Lite model.
This article is Part 6 in a 7-Part Series.
- Part 1 - High-Speed Barcode and QR Code Detection on Android Using Camera2 API
- Part 2 - Optimizing Android Barcode Scanning with NDK and JNI C++
- Part 3 - Choosing the Best Tool for High-Density QR Code Scanning on Android: Google ML Kit vs. Dynamsoft Barcode SDK
- Part 4 - The Quickest Way to Create an Android QR Code Scanner
- Part 5 - Real-time QR Code Recognition on Android with YOLO and Dynamsoft Barcode Reader
- Part 6 - Accelerating Android QR Code Detection with TensorFlow Lite
- Part 7 - Effortlessly Recognize US Driver's Licenses in Android Apps
Steps to Train a QR Code Detection Model with TensorFlow Lite Model Maker
-
Install TensorFlow Lite Model Maker:
pip install tflite-model-maker
-
Clone the GitHub repository containing the image dataset:
git clone https://github.com/yushulx/barcode-qrcode-images.git
-
Partition the image dataset into training and validation sets:
cd tensorflow-lite python partition_dataset.py -x -i ../images -r 0.1 -o ./
-
Follow the TensorFlow tutorial to create the following Python script for training the model:
from tflite_model_maker.config import QuantizationConfig from tflite_model_maker.config import ExportFormat from tflite_model_maker import model_spec from tflite_model_maker import object_detector import tensorflow as tf assert tf.__version__.startswith('2') tf.get_logger().setLevel('ERROR') from absl import logging logging.set_verbosity(logging.ERROR) spec = model_spec.get('efficientdet_lite0') train_data = object_detector.DataLoader.from_pascal_voc(images_dir="train", annotations_dir="train", label_map={1: "QR_CODE"} ) validation_data = object_detector.DataLoader.from_pascal_voc(images_dir="test", annotations_dir="test", label_map={1: "QR_CODE"} ) model = object_detector.create(train_data, model_spec=spec, batch_size=8, train_whole_model=True, validation_data=validation_data) model.export(export_dir='.')
EfficientDet-D0 offers comparable accuracy to YOLOv3 with less computational cost.
Once the model is trained and saved, you can download the TensorFlow Lite example and replace the default model file with your custom-trained model.
Integrating the Model into the Android Project
- Place the
model.tflite
file into the assets folder. -
Update the
TF_OD_API_MODEL_FILE
path inDetectorActivity.java
:private static final String TF_OD_API_MODEL_FILE = "model.tflite";
-
Clear the contents of
labelmap.txt
and add the following:QR_CODE
Your QR code detection model is now ready to be tested on an Android device.
Building a QR Code Scanner with TensorFlow Lite
Now, let’s integrate TensorFlow Lite into our Android QR Code Scanner:
- Copy
model.tflite
andlabelmap.txt
to the assets folder. -
Add the TensorFlow Lite dependency to
build.gradle
:// Import the Task Vision Library dependency (NNAPI is included) implementation 'org.tensorflow:tensorflow-lite-task-vision:0.3.0'
-
Copy
Detector.java
andTFLiteObjectDetectionAPIModel.java
from the TensorFlow Lite example project to your project. Use the NNAPI delegate for TensorFlow Lite:private TFLiteObjectDetectionAPIModel(Context context, String modelFilename) throws IOException { modelBuffer = FileUtil.loadMappedFile(context, modelFilename); optionsBuilder = ObjectDetectorOptions.builder(); optionsBuilder.setBaseOptions(BaseOptions.builder().useNnapi().build()); objectDetector = ObjectDetector.createFromBufferAndOptions(modelBuffer, optionsBuilder.build()); }
-
Initialize the model detector in
CameraXActivity.java
:// Configuration values for the prepackaged QR Code model. private static final int TF_OD_API_INPUT_SIZE = 416; private static final boolean TF_OD_API_IS_QUANTIZED = true; private static final String TF_OD_API_MODEL_FILE = "model.tflite"; private static final String TF_OD_API_LABELS_FILE = "labelmap.txt"; // Minimum detection confidence to track a detection. private static final float MINIMUM_CONFIDENCE_TF_OD_API = 0.5f; private Detector detector; int cropSize = TF_OD_API_INPUT_SIZE; private void initTFDetector() { try { detector = TFLiteObjectDetectionAPIModel.create( this, TF_OD_API_MODEL_FILE, TF_OD_API_LABELS_FILE, TF_OD_API_INPUT_SIZE, TF_OD_API_IS_QUANTIZED); cropSize = TF_OD_API_INPUT_SIZE; } catch (final IOException e) { e.printStackTrace(); Toast toast = Toast.makeText( getApplicationContext(), "Detector could not be initialized", Toast.LENGTH_SHORT); toast.show(); finish(); } }
-
In the camera frame callback function, convert
ImageProxy
toBitmap
:analysisUseCase.setAnalyzer(cameraExecutor, imageProxy -> { Bitmap bitmap = ImageUtils.getBitmap(imageProxy); imageProxy.close(); });
Note: The bitmap has already been rotated, so there’s no need to rotate the QR code bounding box coordinates for rendering overlays.
To understand the implementation of
getBitmap()
, please refer to BitmapUtils.java. -
Detect QR codes and draw bounding boxes:
final List<Detector.Recognition> tfResults = detector.recognizeImage(bitmap); if (tfResults.size() > 0) { for (final Detector.Recognition tfResult : tfResults) { final RectF location = tfResult.getLocation(); if (location != null && tfResult.getConfidence() >= MINIMUM_CONFIDENCE_TF_OD_API) { overlay.add(new BarcodeGraphic(overlay, location, null, isPortrait)); } } }
-
Decode QR codes using the Dynamsoft Barcode Reader API:
TextResult[] results = null; try { PublicRuntimeSettings settings = reader.getRuntimeSettings(); settings.barcodeFormatIds = EnumBarcodeFormat.BF_QR_CODE; settings.expectedBarcodesCount = tfResults.size(); reader.updateRuntimeSettings(settings); } catch (BarcodeReaderException e) { e.printStackTrace(); }
-
Finally, test the QR code detection model on your Android device.
Running the TensorFlow Lite model for QR code detection is faster than YOLOv4, but still slower compared to using Dynamsoft Barcode Reader alone.