How to Build Flutter Document Scanning Plugin for Windows and Linux
Last week, we finished a Flutter document scanning plugin for web, which is just a start. This week, we continue to empower the plugin for desktop development using Dynamsoft Document Normalizer C++ SDK. The plugin will support Windows and Linux. In this article, you will see the steps to build the Flutter desktop document scanning plugin and the differences of Flutter C++ code for Windows and Linux.
This article is Part 2 in a 3-Part Series.
Flutter Document Scan SDK
https://pub.dev/packages/flutter_document_scan_sdk
Development Environment
- Flutter 3.3.9
- Ubuntu 22.04
- Windows 10
Add Support for Windows and Linux in Flutter Plugin
To support Windows and Linux, run the following command in the project root directory:
flutter create --org com.dynamsoft --template=plugin --platforms=windows,linux .
After generating the platform-specific code, we update the pubspec.yaml
file:
plugin:
platforms:
linux:
pluginClass: FlutterDocumentScanSdkPlugin
windows:
pluginClass: FlutterDocumentScanSdkPluginCApi
web:
pluginClass: FlutterDocumentScanSdkWeb
fileName: flutter_document_scan_sdk_web.dart
You should have noticed that the Flutter plugin project contains flutter_document_scan_sdk_web.dart
and flutter_document_scan_sdk_method_channel.dart
files. The former is used for web. We had added implementations last week. The latter is used for mobile and desktop. It uses method channel to interact with the native platform:
class MethodChannelFlutterDocumentScanSdk
extends FlutterDocumentScanSdkPlatform {
@visibleForTesting
final methodChannel = const MethodChannel('flutter_document_scan_sdk');
@override
Future<String?> getPlatformVersion() async {
final version =
await methodChannel.invokeMethod<String>('getPlatformVersion');
return version;
}
@override
Future<int?> init(String path, String key) async {
return await methodChannel
.invokeMethod<int>('init', {'path': path, 'key': key});
}
@override
Future<int?> setParameters(String params) async {
return await methodChannel
.invokeMethod<int>('setParameters', {'params': params});
}
@override
Future<String?> getParameters() async {
return await methodChannel.invokeMethod<String>('getParameters');
}
@override
Future<List<DocumentResult>> detect(String file) async {
List? results = await methodChannel.invokeListMethod<dynamic>(
'detect',
{'file': file},
);
return _resultWrapper(results);
}
List<DocumentResult> _resultWrapper(List<dynamic>? results) {
List<DocumentResult> output = [];
if (results != null) {
for (var result in results) {
int confidence = result['confidence'];
List<Offset> offsets = [];
int x1 = result['x1'];
int y1 = result['y1'];
int x2 = result['x2'];
int y2 = result['y2'];
int x3 = result['x3'];
int y3 = result['y3'];
int x4 = result['x4'];
int y4 = result['y4'];
offsets.add(Offset(x1.toDouble(), y1.toDouble()));
offsets.add(Offset(x2.toDouble(), y2.toDouble()));
offsets.add(Offset(x3.toDouble(), y3.toDouble()));
offsets.add(Offset(x4.toDouble(), y4.toDouble()));
DocumentResult documentResult = DocumentResult(confidence, offsets, []);
output.add(documentResult);
}
}
return output;
}
@override
Future<NormalizedImage?> normalize(String file, dynamic points) async {
Offset offset = points[0];
int x1 = offset.dx.toInt();
int y1 = offset.dy.toInt();
offset = points[1];
int x2 = offset.dx.toInt();
int y2 = offset.dy.toInt();
offset = points[2];
int x3 = offset.dx.toInt();
int y3 = offset.dy.toInt();
offset = points[3];
int x4 = offset.dx.toInt();
int y4 = offset.dy.toInt();
Map? result = await methodChannel.invokeMapMethod<String, dynamic>(
'normalize',
{
'file': file,
'x1': x1,
'y1': y1,
'x2': x2,
'y2': y2,
'x3': x3,
'y3': y3,
'x4': x4,
'y4': y4
},
);
if (result != null) {
return NormalizedImage(
result['data'],
result['width'],
result['height'],
);
}
return null;
}
@override
Future<int?> save(String filename) async {
return await methodChannel
.invokeMethod<int>('save', {'filename': filename});
}
}
Linking Libraries and Writing C/C++ Code
Linking Shared Libraries in Flutter Plugin
Flutter uses CMake to build the Windows and Linux plugins. So the first step is to setting up the CMakeLists.txt
file. We need to configure link_directories
, target_link_libraries
, and flutter_document_scan_sdk_bundled_libraries
.
Windows
link_directories("${PROJECT_SOURCE_DIR}/lib/")
target_link_libraries(${PLUGIN_NAME} PRIVATE flutter flutter_wrapper_plugin "DynamsoftCorex64" "DynamsoftDocumentNormalizerx64")
set(flutter_document_scan_sdk_bundled_libraries
"${PROJECT_SOURCE_DIR}/bin/"
PARENT_SCOPE
)
Linux
link_directories("${PROJECT_SOURCE_DIR}/lib/")
target_link_libraries(${PLUGIN_NAME} PRIVATE flutter "DynamsoftCore" "DynamsoftDocumentNormalizer")
set(flutter_document_scan_sdk_bundled_libraries
PARENT_SCOPE
)
The linking library files of Dynamsoft Document Normalizer SDK are DynamsoftCore
and DynamsoftDocumentNormalizer
. On Windows, they are named DynamsoftCorex64.lib
and DynamsoftDocumentNormalizerx64.lib
, whereas on Linux, they are named libDynamsoftCore.so
and libDynamsoftDocumentNormalizer.so
. To make Windows plugin work, take one more step to bundle the corresponding dll files.
Implementing Flutter C/C++ Code Logic for Windows and Linux
The native entry points of the plugin are FlutterDocumentScanSdkPlugin::HandleMethodCall
and flutter_document_scan_sdk_plugin_handle_method_call
respectively for Windows and Linux. Because the Flutter header files and data types are different between the two platforms, you have to write code accordingly.
Windows
#include "flutter_document_scan_sdk_plugin.h"
#include <windows.h>
#include <VersionHelpers.h>
#include <flutter/method_channel.h>
#include <flutter/plugin_registrar_windows.h>
#include <flutter/standard_method_codec.h>
#include <memory>
#include <sstream>
void FlutterDocumentScanSdkPlugin::HandleMethodCall(
const flutter::MethodCall<flutter::EncodableValue> &method_call,
std::unique_ptr<flutter::MethodResult<flutter::EncodableValue>> result){}
Linux
#include "include/flutter_document_scan_sdk/flutter_document_scan_sdk_plugin.h"
#include <flutter_linux/flutter_linux.h>
#include <gtk/gtk.h>
#include <sys/utsname.h>
#include <cstring>
static void flutter_document_scan_sdk_plugin_handle_method_call(
FlutterDocumentScanSdkPlugin *self,
FlMethodCall *method_call){}
To handle the method call, we need to parse the method name and arguments firstly.
Windows
const auto *arguments = std::get_if<EncodableMap>(method_call.arguments());
if (method_call.method_name().compare("init") == 0)
{
std::string license;
int ret = 0;
if (arguments)
{
auto license_it = arguments->find(EncodableValue("key"));
if (license_it != arguments->end())
{
license = std::get<std::string>(license_it->second);
}
ret = DocumentManager::SetLicense(license.c_str());
}
result->Success(EncodableValue(ret));
}
...
Linux
if (strcmp(method, "init") == 0)
{
if (fl_value_get_type(args) != FL_VALUE_TYPE_MAP)
{
return;
}
FlValue *value = fl_value_lookup_string(args, "key");
if (value == nullptr)
{
return;
}
const char *license = fl_value_get_string(value);
int ret = DocumentManager::SetLicense(license);
g_autoptr(FlValue) result = fl_value_new_int(ret);
response = FL_METHOD_RESPONSE(fl_method_success_response_new(result));
}
...
As you can see, not only the APIs of parsing the method name are different, but also the data type of the arguments. For example, the license
argument is std::string
on Windows, whereas it is const char*
on Linux.
Now, we create a document_manager.h
file to implement the methods that calls Dynamsoft Document Normalizer API. Since Dynamsoft Document Normalizer allows you to write code in the same way on Windows and Linux, most of the code can be shared except for wrapping the return values as Flutter data type.
-
Import Dynamsoft Document Normalizer SDK header files:
#include "DynamsoftCore.h" #include "DynamsoftDocumentNormalizer.h" using namespace std; using namespace dynamsoft::ddn; using namespace dynamsoft::core;
-
Initialize and destroy Dynamsoft Document Normalizer:
class DocumentManager { public: ~DocumentManager() { if (normalizer != NULL) { DDN_DestroyInstance(normalizer); normalizer = NULL; } FreeImage(); }; const char *GetVersion() { return DDN_GetVersion(); } void Init() { normalizer = DDN_CreateInstance(); imageResult = NULL; } private: void *normalizer; NormalizedImageResult *imageResult; void FreeImage() { if (imageResult != NULL) { DDN_FreeNormalizedImageResult(&imageResult); imageResult = NULL; } } };
-
Set the license key:
static int SetLicense(const char *license) { char errorMsgBuffer[512]; // Click https://www.dynamsoft.com/customer/license/trialLicense/?product=dcv&package=cross-platform int ret = DC_InitLicense(license, errorMsgBuffer, 512); if (ret != DM_OK) { cout << errorMsgBuffer << endl; } return ret; }
-
Detect document edges by calling
DDN_DetectQuadFromFile()
. On Windows, the detected information is stored inEncodableMap
, whereas on Linux, it is stored inFlValue
created byfl_value_new_map()
.Windows
EncodableList Detect(const char *filename) { EncodableList out; if (normalizer == NULL) return out; DetectedQuadResultArray *pResults = NULL; int ret = DDN_DetectQuadFromFile(normalizer, filename, "", &pResults); if (ret) { printf("Detection error: %s\n", DC_GetErrorString(ret)); } if (pResults) { int count = pResults->resultsCount; for (int i = 0; i < count; i++) { EncodableMap map; DetectedQuadResult *quadResult = pResults->detectedQuadResults[i]; int confidence = quadResult->confidenceAsDocumentBoundary; DM_Point *points = quadResult->location->points; int x1 = points[0].coordinate[0]; int y1 = points[0].coordinate[1]; int x2 = points[1].coordinate[0]; int y2 = points[1].coordinate[1]; int x3 = points[2].coordinate[0]; int y3 = points[2].coordinate[1]; int x4 = points[3].coordinate[0]; int y4 = points[3].coordinate[1]; map[EncodableValue("confidence")] = EncodableValue(confidence); map[EncodableValue("x1")] = EncodableValue(x1); map[EncodableValue("y1")] = EncodableValue(y1); map[EncodableValue("x2")] = EncodableValue(x2); map[EncodableValue("y2")] = EncodableValue(y2); map[EncodableValue("x3")] = EncodableValue(x3); map[EncodableValue("y3")] = EncodableValue(y3); map[EncodableValue("x4")] = EncodableValue(x4); map[EncodableValue("y4")] = EncodableValue(y4); out.push_back(map); } } if (pResults != NULL) DDN_FreeDetectedQuadResultArray(&pResults); return out; }
Linux
FlValue* Detect(const char *filename) { FlValue* out = fl_value_new_list(); if (normalizer == NULL) return out; DetectedQuadResultArray *pResults = NULL; int ret = DDN_DetectQuadFromFile(normalizer, filename, "", &pResults); if (ret) { printf("Detection error: %s\n", DC_GetErrorString(ret)); } if (pResults) { int count = pResults->resultsCount; for (int i = 0; i < count; i++) { FlValue* result = fl_value_new_map (); DetectedQuadResult *quadResult = pResults->detectedQuadResults[i]; int confidence = quadResult->confidenceAsDocumentBoundary; DM_Point *points = quadResult->location->points; int x1 = points[0].coordinate[0]; int y1 = points[0].coordinate[1]; int x2 = points[1].coordinate[0]; int y2 = points[1].coordinate[1]; int x3 = points[2].coordinate[0]; int y3 = points[2].coordinate[1]; int x4 = points[3].coordinate[0]; int y4 = points[3].coordinate[1]; fl_value_set_string_take (result, "confidence", fl_value_new_int(confidence)); fl_value_set_string_take (result, "x1", fl_value_new_int(x1)); fl_value_set_string_take (result, "y1", fl_value_new_int(y1)); fl_value_set_string_take (result, "x2", fl_value_new_int(x2)); fl_value_set_string_take (result, "y2", fl_value_new_int(y2)); fl_value_set_string_take (result, "x3", fl_value_new_int(x3)); fl_value_set_string_take (result, "y3", fl_value_new_int(y3)); fl_value_set_string_take (result, "x4", fl_value_new_int(x4)); fl_value_set_string_take (result, "y4", fl_value_new_int(y4)); fl_value_append_take (out, result); } } if (pResults != NULL) DDN_FreeDetectedQuadResultArray(&pResults); return out; }
- Call
DDN_NormalizeFile()
to normalize the document based on detected corner coordinates:Quadrilateral quad; quad.points[0].coordinate[0] = x1; quad.points[0].coordinate[1] = y1; quad.points[1].coordinate[0] = x2; quad.points[1].coordinate[1] = y2; quad.points[2].coordinate[0] = x3; quad.points[2].coordinate[1] = y3; quad.points[3].coordinate[0] = x4; quad.points[3].coordinate[1] = y4; int errorCode = DDN_NormalizeFile(normalizer, filename, "", &quad, &imageResult); if (errorCode != DM_OK) printf("%s\r\n", DC_GetErrorString(errorCode)); if (imageResult) { ImageData *imageData = imageResult->image; int width = imageData->width; int height = imageData->height; int stride = imageData->stride; int format = (int)imageData->format; unsigned char* data = imageData->bytes; int orientation = imageData->orientation; int length = imageData->bytesLength; }
-
Create a Flutter C++ map object to pass the normalized image data to Flutter:
Windows:
EncodableMap map; map[EncodableValue("width")] = EncodableValue(width); map[EncodableValue("height")] = EncodableValue(height); map[EncodableValue("stride")] = EncodableValue(stride); map[EncodableValue("format")] = EncodableValue(format); map[EncodableValue("orientation")] = EncodableValue(orientation); map[EncodableValue("length")] = EncodableValue(length); std::vector<uint8_t> rawBytes(rgba, rgba + width * height * 4); map[EncodableValue("data")] = rawBytes;
Linux:
FlValue* result = fl_value_new_map (); fl_value_set_string_take (result, "width", fl_value_new_int(width)); fl_value_set_string_take (result, "height", fl_value_new_int(height)); fl_value_set_string_take (result, "stride", fl_value_new_int(stride)); fl_value_set_string_take (result, "format", fl_value_new_int(format)); fl_value_set_string_take (result, "orientation", fl_value_new_int(orientation)); fl_value_set_string_take (result, "length", fl_value_new_int(length)); fl_value_set_string_take (result, "data", fl_value_new_uint8_list(rgba, width * height * 4));
Note: According to the image color format, we need to do the following conversion before wrapping the data:
-
If the output image is black and white, each byte represents 8 pixels. Each bit of a byte needs to be converted to a byte as 0 or 255. The final length of the image bytes should be the same as the grayscale image.
unsigned char *grayscale = new unsigned char[width * height]; binary2grayscale(data, grayscale, width, height, stride, length); void binary2grayscale(unsigned char* data, unsigned char* output, int width, int height, int stride, int length) { int index = 0; int skip = stride * 8 - width; int shift = 0; int n = 1; for (int i = 0; i < length; ++i) { unsigned char b = data[i]; int byteCount = 7; while (byteCount >= 0) { int tmp = (b & (1 << byteCount)) >> byteCount; if (shift < stride * 8 * n - skip) { if (tmp == 1) output[index] = 255; else output[index] = 0; index += 1; } byteCount -= 1; shift += 1; } if (shift == stride * 8 * n) { n += 1; } } }
-
The format of pixel data given to the Flutter
decodeImageFromPixels()
method only supportsrgba8888
andbgra8888
. So, we need to construct the pixel data in the format ofrgba8888
orbgra8888
based on the output image color format.unsigned char *rgba = new unsigned char[width * height * 4]; memset(rgba, 0, width * height * 4); if (format == IPF_RGB_888) { int dataIndex = 0; for (int i = 0; i < height; i++) { for (int j = 0; j < width; j++) { int index = i * width + j; rgba[index * 4] = data[dataIndex + 2]; // red rgba[index * 4 + 1] = data[dataIndex + 1]; // green rgba[index * 4 + 2] = data[dataIndex]; // blue rgba[index * 4 + 3] = 255; // alpha dataIndex += 3; } } } else if (format == IPF_GRAYSCALED) { int dataIndex = 0; for (int i = 0; i < height; i++) { for (int j = 0; j < width; j++) { int index = i * width + j; rgba[index * 4] = data[dataIndex]; rgba[index * 4 + 1] = data[dataIndex]; rgba[index * 4 + 2] = data[dataIndex]; rgba[index * 4 + 3] = 255; dataIndex += 1; } } } else if (format == IPF_BINARY) { unsigned char *grayscale = new unsigned char[width * height]; binary2grayscale(data, grayscale, width, height, stride, length); int dataIndex = 0; for (int i = 0; i < height; i++) { for (int j = 0; j < width; j++) { int index = i * width + j; rgba[index * 4] = grayscale[dataIndex]; rgba[index * 4 + 1] = grayscale[dataIndex]; rgba[index * 4 + 2] = grayscale[dataIndex]; rgba[index * 4 + 3] = 255; dataIndex += 1; } } free(grayscale); }
-
So far, we have completed the implementation of the Flutter Document Scan SDK plugin for Windows and Linux. The next step is to test the plugin on desktop platforms.
Test Flutter Document Scan SDK Plugin on Windows and Linux
Since the example code we implemented for web is shared with Windows and Linux, there is no code change required. We can directly run the example code on Windows and Linux.
Windows
flutter run -d windows
Linux
flutter run -d linux