How to Build Flutter Document Scanning Plugin for Windows and Linux

Dec 06, 2022

Last week, we finished a Flutter document scanning plugin for web, which is just a start. This week, we continue to empower the plugin for desktop development using Dynamsoft Document Normalizer C++ SDK. The plugin will support Windows and Linux. In this article, you will see the steps to build the Flutter desktop document scanning plugin and the differences of Flutter C++ code for Windows and Linux.

This article is Part 2 in a 3-Part Series.

Flutter Document Scan SDK

https://pub.dev/packages/flutter_document_scan_sdk

Development Environment

Flutter 3.3.9
Ubuntu 22.04
Windows 10

Add Support for Windows and Linux in Flutter Plugin

To support Windows and Linux, run the following command in the project root directory:

flutter create --org com.dynamsoft --template=plugin --platforms=windows,linux .

After generating the platform-specific code, we update the pubspec.yaml file:

plugin:
    platforms:
      linux:
        pluginClass: FlutterDocumentScanSdkPlugin
      windows:
        pluginClass: FlutterDocumentScanSdkPluginCApi
      web:
        pluginClass: FlutterDocumentScanSdkWeb
        fileName: flutter_document_scan_sdk_web.dart

You should have noticed that the Flutter plugin project contains flutter_document_scan_sdk_web.dart and flutter_document_scan_sdk_method_channel.dart files. The former is used for web. We had added implementations last week. The latter is used for mobile and desktop. It uses method channel to interact with the native platform:

class MethodChannelFlutterDocumentScanSdk
    extends FlutterDocumentScanSdkPlatform {
  @visibleForTesting
  final methodChannel = const MethodChannel('flutter_document_scan_sdk');

  @override
  Future<String?> getPlatformVersion() async {
    final version =
        await methodChannel.invokeMethod<String>('getPlatformVersion');
    return version;
  }

  @override
  Future<int?> init(String path, String key) async {
    return await methodChannel
        .invokeMethod<int>('init', {'path': path, 'key': key});
  }

  @override
  Future<int?> setParameters(String params) async {
    return await methodChannel
        .invokeMethod<int>('setParameters', {'params': params});
  }

  @override
  Future<String?> getParameters() async {
    return await methodChannel.invokeMethod<String>('getParameters');
  }

  @override
  Future<List<DocumentResult>> detect(String file) async {
    List? results = await methodChannel.invokeListMethod<dynamic>(
      'detect',
      {'file': file},
    );

    return _resultWrapper(results);
  }

  List<DocumentResult> _resultWrapper(List<dynamic>? results) {
    List<DocumentResult> output = [];

    if (results != null) {
      for (var result in results) {
        int confidence = result['confidence'];
        List<Offset> offsets = [];
        int x1 = result['x1'];
        int y1 = result['y1'];
        int x2 = result['x2'];
        int y2 = result['y2'];
        int x3 = result['x3'];
        int y3 = result['y3'];
        int x4 = result['x4'];
        int y4 = result['y4'];
        offsets.add(Offset(x1.toDouble(), y1.toDouble()));
        offsets.add(Offset(x2.toDouble(), y2.toDouble()));
        offsets.add(Offset(x3.toDouble(), y3.toDouble()));
        offsets.add(Offset(x4.toDouble(), y4.toDouble()));
        DocumentResult documentResult = DocumentResult(confidence, offsets, []);
        output.add(documentResult);
      }
    }

    return output;
  }

  @override
  Future<NormalizedImage?> normalize(String file, dynamic points) async {
    Offset offset = points[0];
    int x1 = offset.dx.toInt();
    int y1 = offset.dy.toInt();

    offset = points[1];
    int x2 = offset.dx.toInt();
    int y2 = offset.dy.toInt();

    offset = points[2];
    int x3 = offset.dx.toInt();
    int y3 = offset.dy.toInt();

    offset = points[3];
    int x4 = offset.dx.toInt();
    int y4 = offset.dy.toInt();
    Map? result = await methodChannel.invokeMapMethod<String, dynamic>(
      'normalize',
      {
        'file': file,
        'x1': x1,
        'y1': y1,
        'x2': x2,
        'y2': y2,
        'x3': x3,
        'y3': y3,
        'x4': x4,
        'y4': y4
      },
    );

    if (result != null) {
      return NormalizedImage(
        result['data'],
        result['width'],
        result['height'],
      );
    }

    return null;
  }

  @override
  Future<int?> save(String filename) async {
    return await methodChannel
        .invokeMethod<int>('save', {'filename': filename});
  }
}

Linking Libraries and Writing C/C++ Code

Linking Shared Libraries in Flutter Plugin

Flutter uses CMake to build the Windows and Linux plugins. So the first step is to setting up the CMakeLists.txt file. We need to configure link_directories, target_link_libraries, and flutter_document_scan_sdk_bundled_libraries.

Windows

link_directories("${PROJECT_SOURCE_DIR}/lib/") 

target_link_libraries(${PLUGIN_NAME} PRIVATE flutter flutter_wrapper_plugin "DynamsoftCorex64" "DynamsoftDocumentNormalizerx64")

set(flutter_document_scan_sdk_bundled_libraries
  "${PROJECT_SOURCE_DIR}/bin/"
  PARENT_SCOPE
)

Linux

link_directories("${PROJECT_SOURCE_DIR}/lib/") 

target_link_libraries(${PLUGIN_NAME} PRIVATE flutter "DynamsoftCore" "DynamsoftDocumentNormalizer")

set(flutter_document_scan_sdk_bundled_libraries
  
  PARENT_SCOPE
)

The linking library files of Dynamsoft Document Normalizer SDK are DynamsoftCore and DynamsoftDocumentNormalizer. On Windows, they are named DynamsoftCorex64.lib and DynamsoftDocumentNormalizerx64.lib, whereas on Linux, they are named libDynamsoftCore.so and libDynamsoftDocumentNormalizer.so. To make Windows plugin work, take one more step to bundle the corresponding dll files.

Implementing Flutter C/C++ Code Logic for Windows and Linux

The native entry points of the plugin are FlutterDocumentScanSdkPlugin::HandleMethodCall and flutter_document_scan_sdk_plugin_handle_method_call respectively for Windows and Linux. Because the Flutter header files and data types are different between the two platforms, you have to write code accordingly.

Windows

#include "flutter_document_scan_sdk_plugin.h"

#include <windows.h>

#include <VersionHelpers.h>

#include <flutter/method_channel.h>
#include <flutter/plugin_registrar_windows.h>
#include <flutter/standard_method_codec.h>

#include <memory>
#include <sstream>

void FlutterDocumentScanSdkPlugin::HandleMethodCall(
      const flutter::MethodCall<flutter::EncodableValue> &method_call,
      std::unique_ptr<flutter::MethodResult<flutter::EncodableValue>> result){}

Linux

#include "include/flutter_document_scan_sdk/flutter_document_scan_sdk_plugin.h"

#include <flutter_linux/flutter_linux.h>
#include <gtk/gtk.h>
#include <sys/utsname.h>

#include <cstring>

static void flutter_document_scan_sdk_plugin_handle_method_call(
    FlutterDocumentScanSdkPlugin *self,
    FlMethodCall *method_call){}

To handle the method call, we need to parse the method name and arguments firstly.

Windows

const auto *arguments = std::get_if<EncodableMap>(method_call.arguments());

if (method_call.method_name().compare("init") == 0)
{
  std::string license;
  int ret = 0;

  if (arguments)
  {
    auto license_it = arguments->find(EncodableValue("key"));
    if (license_it != arguments->end())
    {
      license = std::get<std::string>(license_it->second);
    }
    ret = DocumentManager::SetLicense(license.c_str());
  }

  result->Success(EncodableValue(ret));
}
...

Linux

if (strcmp(method, "init") == 0)
{
  if (fl_value_get_type(args) != FL_VALUE_TYPE_MAP)
  {
    return;
  }

  FlValue *value = fl_value_lookup_string(args, "key");
  if (value == nullptr)
  {
    return;
  }
  const char *license = fl_value_get_string(value);

  int ret = DocumentManager::SetLicense(license);
  g_autoptr(FlValue) result = fl_value_new_int(ret);
  response = FL_METHOD_RESPONSE(fl_method_success_response_new(result));
}
...

As you can see, not only the APIs of parsing the method name are different, but also the data type of the arguments. For example, the license argument is std::string on Windows, whereas it is const char* on Linux.

Now, we create a document_manager.h file to implement the methods that calls Dynamsoft Document Normalizer API. Since Dynamsoft Document Normalizer allows you to write code in the same way on Windows and Linux, most of the code can be shared except for wrapping the return values as Flutter data type.

Import Dynamsoft Document Normalizer SDK header files:

#include "DynamsoftCore.h"
#include "DynamsoftDocumentNormalizer.h"

using namespace std;
using namespace dynamsoft::ddn;
using namespace dynamsoft::core;

Initialize and destroy Dynamsoft Document Normalizer:

class DocumentManager
{
public:
    ~DocumentManager()
    {
        if (normalizer != NULL)
        {
            DDN_DestroyInstance(normalizer);
            normalizer = NULL;
        }

        FreeImage();
    };

    const char *GetVersion()
    {
        return DDN_GetVersion();
  }

  void Init()
  {
      normalizer = DDN_CreateInstance();
      imageResult = NULL;
  }
  private:
    void *normalizer;
    NormalizedImageResult *imageResult;

    void FreeImage()
    {
        if (imageResult != NULL)
        {
            DDN_FreeNormalizedImageResult(&imageResult);
            imageResult = NULL;
        }
    }
};

Set the license key:

static int SetLicense(const char *license)
{
    char errorMsgBuffer[512];
    // Click https://www.dynamsoft.com/customer/license/trialLicense/?product=ddn to get a trial license.
    int ret = DC_InitLicense(license, errorMsgBuffer, 512);
    if (ret != DM_OK)
    {
        cout << errorMsgBuffer << endl;
    }
    return ret;
}

Detect document edges by calling DDN_DetectQuadFromFile(). On Windows, the detected information is stored in EncodableMap, whereas on Linux, it is stored in FlValue created by fl_value_new_map().

Windows

EncodableList Detect(const char *filename)
{
    EncodableList out;
    if (normalizer == NULL)
        return out;

    DetectedQuadResultArray *pResults = NULL;

    int ret = DDN_DetectQuadFromFile(normalizer, filename, "", &pResults);
    if (ret)
    {
        printf("Detection error: %s\n", DC_GetErrorString(ret));
    }

    if (pResults)
    {
        int count = pResults->resultsCount;

        for (int i = 0; i < count; i++)
        {
            EncodableMap map;

            DetectedQuadResult *quadResult = pResults->detectedQuadResults[i];
            int confidence = quadResult->confidenceAsDocumentBoundary;
            DM_Point *points = quadResult->location->points;
            int x1 = points[0].coordinate[0];
            int y1 = points[0].coordinate[1];
            int x2 = points[1].coordinate[0];
            int y2 = points[1].coordinate[1];
            int x3 = points[2].coordinate[0];
            int y3 = points[2].coordinate[1];
            int x4 = points[3].coordinate[0];
            int y4 = points[3].coordinate[1];

            map[EncodableValue("confidence")] = EncodableValue(confidence);
            map[EncodableValue("x1")] = EncodableValue(x1);
            map[EncodableValue("y1")] = EncodableValue(y1);
            map[EncodableValue("x2")] = EncodableValue(x2);
            map[EncodableValue("y2")] = EncodableValue(y2);
            map[EncodableValue("x3")] = EncodableValue(x3);
            map[EncodableValue("y3")] = EncodableValue(y3);
            map[EncodableValue("x4")] = EncodableValue(x4);
            map[EncodableValue("y4")] = EncodableValue(y4);
            out.push_back(map);
        }
    }

    if (pResults != NULL)
        DDN_FreeDetectedQuadResultArray(&pResults);

    return out;
}

Linux

FlValue* Detect(const char *filename)
{
    FlValue* out = fl_value_new_list();
    if (normalizer == NULL)
        return out;

    DetectedQuadResultArray *pResults = NULL;

    int ret = DDN_DetectQuadFromFile(normalizer, filename, "", &pResults);
    if (ret)
    {
        printf("Detection error: %s\n", DC_GetErrorString(ret));
    }

    if (pResults)
    {
        int count = pResults->resultsCount;

        for (int i = 0; i < count; i++)
        {
            FlValue* result = fl_value_new_map ();

            DetectedQuadResult *quadResult = pResults->detectedQuadResults[i];
            int confidence = quadResult->confidenceAsDocumentBoundary;
            DM_Point *points = quadResult->location->points;
            int x1 = points[0].coordinate[0];
            int y1 = points[0].coordinate[1];
            int x2 = points[1].coordinate[0];
            int y2 = points[1].coordinate[1];
            int x3 = points[2].coordinate[0];
            int y3 = points[2].coordinate[1];
            int x4 = points[3].coordinate[0];
            int y4 = points[3].coordinate[1];

            fl_value_set_string_take (result, "confidence", fl_value_new_int(confidence));
            fl_value_set_string_take (result, "x1", fl_value_new_int(x1));
            fl_value_set_string_take (result, "y1", fl_value_new_int(y1));
            fl_value_set_string_take (result, "x2", fl_value_new_int(x2));
            fl_value_set_string_take (result, "y2", fl_value_new_int(y2));
            fl_value_set_string_take (result, "x3", fl_value_new_int(x3));
            fl_value_set_string_take (result, "y3", fl_value_new_int(y3));
            fl_value_set_string_take (result, "x4", fl_value_new_int(x4));
            fl_value_set_string_take (result, "y4", fl_value_new_int(y4));

            fl_value_append_take (out, result);
        }
    }

    if (pResults != NULL)
        DDN_FreeDetectedQuadResultArray(&pResults);

    return out;
}

Call DDN_NormalizeFile() to normalize the document based on detected corner coordinates:

Quadrilateral quad;
quad.points[0].coordinate[0] = x1;
quad.points[0].coordinate[1] = y1;
quad.points[1].coordinate[0] = x2;
quad.points[1].coordinate[1] = y2;
quad.points[2].coordinate[0] = x3;
quad.points[2].coordinate[1] = y3;
quad.points[3].coordinate[0] = x4;
quad.points[3].coordinate[1] = y4;

int errorCode = DDN_NormalizeFile(normalizer, filename, "", &quad, &imageResult);
if (errorCode != DM_OK)
    printf("%s\r\n", DC_GetErrorString(errorCode));

if (imageResult)
{
    ImageData *imageData = imageResult->image;
    int width = imageData->width;
    int height = imageData->height;
    int stride = imageData->stride;
    int format = (int)imageData->format;
    unsigned char* data = imageData->bytes;
    int orientation = imageData->orientation;
    int length = imageData->bytesLength;
}

Create a Flutter C++ map object to pass the normalized image data to Flutter:

Windows:

EncodableMap map;
map[EncodableValue("width")] = EncodableValue(width);
map[EncodableValue("height")] = EncodableValue(height);
map[EncodableValue("stride")] = EncodableValue(stride);
map[EncodableValue("format")] = EncodableValue(format);
map[EncodableValue("orientation")] = EncodableValue(orientation);
map[EncodableValue("length")] = EncodableValue(length);
std::vector<uint8_t> rawBytes(rgba, rgba + width * height * 4);
map[EncodableValue("data")] = rawBytes;

Linux:

FlValue* result = fl_value_new_map ();
fl_value_set_string_take (result, "width", fl_value_new_int(width));
fl_value_set_string_take (result, "height", fl_value_new_int(height));
fl_value_set_string_take (result, "stride", fl_value_new_int(stride));
fl_value_set_string_take (result, "format", fl_value_new_int(format));
fl_value_set_string_take (result, "orientation", fl_value_new_int(orientation));
fl_value_set_string_take (result, "length", fl_value_new_int(length));
fl_value_set_string_take (result, "data", fl_value_new_uint8_list(rgba, width * height * 4));

Note: According to the image color format, we need to do the following conversion before wrapping the data:

If the output image is black and white, each byte represents 8 pixels. Each bit of a byte needs to be converted to a byte as 0 or 255. The final length of the image bytes should be the same as the grayscale image.

unsigned char *grayscale = new unsigned char[width * height];
binary2grayscale(data, grayscale, width, height, stride, length);

void binary2grayscale(unsigned char* data, unsigned char* output, int width, int height, int stride, int length) 
{
    int index = 0;

    int skip = stride * 8 - width;
    int shift = 0;
    int n = 1;

    for (int i = 0; i < length; ++i)
    {
        unsigned char b = data[i];
        int byteCount = 7;
        while (byteCount >= 0)
        {
            int tmp = (b & (1 << byteCount)) >> byteCount;

            if (shift < stride * 8 * n - skip) {
                if (tmp == 1)
                    output[index] = 255;
                else
                    output[index] = 0;
                index += 1;
            }

            byteCount -= 1;
            shift += 1;
        }

        if (shift == stride * 8 * n) {
            n += 1;
        }
    }
}

The format of pixel data given to the Flutter decodeImageFromPixels() method only supports rgba8888 and bgra8888. So, we need to construct the pixel data in the format of rgba8888 or bgra8888 based on the output image color format.

unsigned char *rgba = new unsigned char[width * height * 4];
memset(rgba, 0, width * height * 4);
if (format == IPF_RGB_888)
{
    int dataIndex = 0;
    for (int i = 0; i < height; i++)
    {
        for (int j = 0; j < width; j++)
        {
            int index = i * width + j;

            rgba[index * 4] = data[dataIndex + 2];     // red
            rgba[index * 4 + 1] = data[dataIndex + 1]; // green
            rgba[index * 4 + 2] = data[dataIndex];     // blue
            rgba[index * 4 + 3] = 255;                 // alpha
            dataIndex += 3;
        }
    }
}
else if (format == IPF_GRAYSCALED)
{
    int dataIndex = 0;
    for (int i = 0; i < height; i++)
    {
        for (int j = 0; j < width; j++)
        {
            int index = i * width + j;
            rgba[index * 4] = data[dataIndex];
            rgba[index * 4 + 1] = data[dataIndex];
            rgba[index * 4 + 2] = data[dataIndex];
            rgba[index * 4 + 3] = 255;
            dataIndex += 1;
        }
    }
}
else if (format == IPF_BINARY)
{
    unsigned char *grayscale = new unsigned char[width * height];
    binary2grayscale(data, grayscale, width, height, stride, length);

    int dataIndex = 0;
    for (int i = 0; i < height; i++)
    {
        for (int j = 0; j < width; j++)
        {
            int index = i * width + j;
            rgba[index * 4] = grayscale[dataIndex];
            rgba[index * 4 + 1] = grayscale[dataIndex];
            rgba[index * 4 + 2] = grayscale[dataIndex];
            rgba[index * 4 + 3] = 255;
            dataIndex += 1;
        }
    }

    free(grayscale);
}

So far, we have completed the implementation of the Flutter Document Scan SDK plugin for Windows and Linux. The next step is to test the plugin on desktop platforms.

Test Flutter Document Scan SDK Plugin on Windows and Linux

Since the example code we implemented for web is shared with Windows and Linux, there is no code change required. We can directly run the example code on Windows and Linux.

Windows

flutter run -d windows

Flutter windows document edge detection and normalization

Linux

flutter run -d linux

Flutter Linux document edge detection and normalization

Source Code

https://github.com/yushulx/flutter_document_scan_sdk

LANGUAGES

PLATFORMS

How to Build Flutter Document Scanning Plugin for Windows and Linux

Flutter Document Scan SDK

Development Environment

Add Support for Windows and Linux in Flutter Plugin

Linking Libraries and Writing C/C++ Code

Linking Shared Libraries in Flutter Plugin

Implementing Flutter C/C++ Code Logic for Windows and Linux

Test Flutter Document Scan SDK Plugin on Windows and Linux

Source Code

Search Blog Posts

POPULAR TOPICS

LANGUAGES

PLATFORMS

How to Build Flutter Document Scanning Plugin for Windows and Linux

Flutter Document Scan SDK

Development Environment

Add Support for Windows and Linux in Flutter Plugin

Linking Libraries and Writing C/C++ Code

Linking Shared Libraries in Flutter Plugin

Implementing Flutter C/C++ Code Logic for Windows and Linux

Test Flutter Document Scan SDK Plugin on Windows and Linux

Source Code

Search Blog Posts

POPULAR TOPICS

Useful Links