How to Build Web Apps to Scan Documents by Edge Detection Using JavaScript and Flutter

Dynamsoft Document Normalizer SDK provides a set of APIs to detect the edges of a document and normalize the document based on the found quadrilaterals. The JavaScript edition of the SDK has been published on npm. In this article, we will firstly show you how to use the JavaScript APIs in a web application, and then create a Flutter document scan plugin based on the JavaScript SDK. It will be convenient for developers to integrate the document edge detection and normalization features into their Flutter web applications.

This article is Part 1 in a 3-Part Series.

Flutter Document Scan SDK

https://pub.dev/packages/flutter_document_scan_sdk

Using JavaScript API for Document Edge Detection and Perspective Transformation

With a few lines of JavaScript code, you can quickly build a web document scanning application using Dynamsoft Document Normalizer SDK. Here are the steps:

  1. Include the JavaScript SDK in your HTML file:

     <script src="https://cdn.jsdelivr.net/npm/dynamsoft-document-normalizer@1.0.11/dist/ddn.js"></script>

  2. Apply for a trial license from Dynamsoft Customer Portal and set the license key in your JavaScript code:

     Dynamsoft.DDN.DocumentNormalizer.license =
         "DLS2eyJoYW5kc2hha2VDb2RlIjoiMjAwMDAxLTE2NDk4Mjk3OTI2MzUiLCJvcmdhbml6YXRpb25JRCI6IjIwMDAwMSIsInNlc3Npb25QYXNzd29yZCI6IndTcGR6Vm05WDJrcEQ5YUoifQ==";

  3. Initialize the document normalizer in an asynchronous function:

    
 (async () => {
     normalizer = await Dynamsoft.DDN.DocumentNormalizer.createInstance();
     let settings = await normalizer.getRuntimeSettings();
     settings.ImageParameterArray[0].BinarizationModes[0].ThresholdCompensation = 9;
     settings.ImageParameterArray[0].ScaleDownThreshold = 2300;
     settings.NormalizerParameterArray[0].ColourMode = "ICM_GRAYSCALE"; // ICM_BINARY, ICM_GRAYSCALE, ICM_COLOUR
     await normalizer.setRuntimeSettings(settings);
 })();

  4. Create an input button to load a document image and call the detectQuad() method to detect the edges of the document:

     <input type="file" id="file" accept="image/*" />

 document.getElementById("file").addEventListener("change", function () {
     let file = this.files[0];
     let fr = new FileReader();
     fr.onload = function () {
         let image = document.getElementById('image');
         image.src = fr.result;
         target["file"] = fr.result;

         const img = new Image();
         img.onload = () => {
             if (normalizer) {
                 (async () => {
                     let quads = await normalizer.detectQuad(target["file"]);
                     target["quads"] = quads;
                     points = quads[0].location.points;
                     drawQuad(points);
                 })();
             }
         }
         img.src = fr.result;
     }
     fr.readAsDataURL(file);
 });

  5. Call the normalize() method to normalize the document based on the detected points:

     function normalize(file, quad) {
     (async () => {
         if (normalizer) {
             var tmp = { quad: quad };
             normalizedImageResult = await normalizer.normalize(file, tmp);
         }

     })();
 }

    The data type of the normalized image buffer is Uint8Array. In order to render the image on a canvas, you need to convert it to ImageData.

     if (normalizedImageResult) {
     let image = normalizedImageResult.image;
     canvas.width = image.width;
     canvas.height = image.height;
     let data = new ImageData(new Uint8ClampedArray(image.data), image.width, image.height);
     ctx.putImageData(data, 0, 0);
 }

  6. Save the document to a PNG or JPEG file:

     function save() {
     (async () => {
         if (normalizedImageResult) {
             let data = await normalizedImageResult.saveToFile("document-normalization.png", false);
             console.log(data);
         }
     })();
 }

After finishing the above steps, you can test the single page HTML file by running python -m http.server in the terminal.

JavaScript web document edge detection and normalization

Encapsulating Dynamsoft Document Normalizer JavaScript SDK into a Flutter Plugin

We scaffolded a Flutter web-only plugin project using the following command:

flutter create --org com.dynamsoft --template=plugin --platforms=web flutter_document_scan_sdk

Then add the js package as the dependency in pubspec.yaml file. The Flutter JS plugin can make Dart and JavaScript code communicate with each other:

dependencies:
  ...
  js: ^0.6.3

Go to the lib folder and create a new file web_ddn_manager.dart, in which we define the classes and methods to interop with the JavaScript SDK.

@JS('Dynamsoft')
library dynamsoft;

import 'dart:convert';

import 'package:flutter/foundation.dart';
import 'package:flutter_document_scan_sdk/normalized_image.dart';
import 'package:flutter_document_scan_sdk/shims/dart_ui_real.dart';
import 'package:js/js.dart';
import 'document_result.dart';
import 'utils.dart';
import 'dart:html' as html;

/// DocumentNormalizer class.
@JS('DDN.DocumentNormalizer')
class DocumentNormalizer {
  external static set license(String license);
  external static set engineResourcePath(String resourcePath);
  external PromiseJsImpl<List<dynamic>> detectQuad(dynamic file);
  external PromiseJsImpl<dynamic> getRuntimeSettings();
  external static PromiseJsImpl<DocumentNormalizer> createInstance();
  external PromiseJsImpl<void> setRuntimeSettings(dynamic settings);
  external PromiseJsImpl<NormalizedDocument> normalize(
      dynamic file, dynamic params);
}

/// Image class
@JS('Image')
class Image {
  external dynamic get data;
  external int get width;
  external int get height;
}

/// NormalizedDocument class
@JS('NormalizedDocument')
class NormalizedDocument {
  external PromiseJsImpl<dynamic> saveToFile(String filename, bool download);
  external dynamic get image;
}

Afterwards, create a DDNManager class to add Flutter-specific methods:

  • init(): Initialize the document normalizer with the resource path and the license key. The resource path is where the js and wasm files are located.

      Future<int> init(String path, String key) async {
    DocumentNormalizer.engineResourcePath = path;
    DocumentNormalizer.license = key;

    _normalizer = await handleThenable(DocumentNormalizer.createInstance());

    return 0;
  }

  • getParameters(): Get the runtime parameters of the SDK.

      Future<String> getParameters() async {
    if (_normalizer != null) {
      dynamic settings =
          await handleThenable(_normalizer!.getRuntimeSettings());
      return stringify(settings);
    }

    return '';
  }
  • setParameters(): Set the parameters of the SDK. 
      Future<int> setParameters(String params) async {
    if (_normalizer != null) {
      await handleThenable(_normalizer!.setRuntimeSettings(params));
      return 0;
    }

    return -1;
  }
  • detect(): Detect the edges of the document. 
      Future<List<DocumentResult>> detect(String file) async {
    if (_normalizer != null) {
      List<dynamic> results =
          await handleThenable(_normalizer!.detectQuad(file));
      return _resultWrapper(results);
    }

    return [];
  }

    The result needs to be converted from a JavaScript object to a Dart object.

      List<DocumentResult> _resultWrapper(List<dynamic> results) {
    List<DocumentResult> output = [];

    for (dynamic result in results) {
      Map value = json.decode(stringify(result));
      int confidence = value['confidenceAsDocumentBoundary'];
      List<dynamic> points = value['location']['points'];
      List<Offset> offsets = [];
      for (dynamic point in points) {
        double x = point['x'];
        double y = point['y'];
        offsets.add(Offset(x, y));
      }
      DocumentResult documentResult =
          DocumentResult(confidence, offsets, value['location']);
      output.add(documentResult);
    }

    return output;
  }
  • normalize(): Transform the document image based on the four corners. 
      Future<NormalizedImage?> normalize(String file, dynamic points) async {
    NormalizedImage? image;
    if (_normalizer != null) {
      _normalizedDocument =
          await handleThenable(_normalizer!.normalize(file, points));

      if (_normalizedDocument != null) {
        Image result = _normalizedDocument!.image;
        dynamic data = result.data;
        Uint8List bytes = Uint8List.fromList(data);
        image = NormalizedImage(bytes, result.width, result.height);
        return image;
      }
    }

    return null;
  }
  • save(): Save the normalized document image as JPEG or PNG files to local disk. 
      Future<int> save(String filename) async {
    if (_normalizedDocument != null) {
      await handleThenable(_normalizedDocument!.saveToFile(filename, true));
    }
    return 0;
  }

The detected quadrilaterals and the normalized document image are stored respectively with DocumentResult and NormalizedImage classes:

class DocumentResult {
  final int confidence;

  final List<Offset> points;

  final dynamic quad;

  DocumentResult(this.confidence, this.points, this.quad);
}

class NormalizedImage {
  final Uint8List data;

  final int width;

  final int height;

  NormalizedImage(this.data, this.width, this.height);
}

In the flutter_document_scan_sdk_platform_interface.dart file, we define some interfaces that will be called by Flutter apps.

Future<int> init(String path, String key) {
  throw UnimplementedError('init() has not been implemented.');
}

Future<NormalizedImage?> normalize(String file, dynamic points) {
  throw UnimplementedError('normalize() has not been implemented.');
}

Future<List<DocumentResult>> detect(String file) {
  throw UnimplementedError('detect() has not been implemented.');
}

Future<int> save(String filename) {
  throw UnimplementedError('save() has not been implemented.');
}

Future<int> setParameters(String params) {
  throw UnimplementedError('setParameters() has not been implemented.');
}

Future<String> getParameters() {
  throw UnimplementedError('getParameters() has not been implemented.');
}

The corresponding methods for the web platform are implemented in the flutter_document_scan_sdk_web.dart file:

@override
Future<int> init(String path, String key) async {
  return _ddnManager.init(path, key);
}

@override
Future<NormalizedImage?> normalize(String file, dynamic points) async {
  return _ddnManager.normalize(file, points);
}

@override
Future<List<DocumentResult>> detect(String file) async {
  return _ddnManager.detect(file);
}

@override
Future<int> save(String filename) async {
  return _ddnManager.save(filename);
}

@override
Future<int> setParameters(String params) async {
  return _ddnManager.setParameters(params);
}

@override
Future<String> getParameters() async {
  return _ddnManager.getParameters();
}

So far, the Flutter document scan plugin is done. In the next section, we will create a Flutter app to test the plugin.

Steps to Build a Flutter Web Application to Scan and Save Documents

Before going through the following steps, you need to get a license key from here.

Step 1: Install Dynamsoft Document Normalizer JavaScript SDK and the Flutter Document Scan Plugin

Install the Flutter Document Scan Plugin.

flutter pub add flutter_document_scan_sdk

Since the Flutter plugin for web does not contain the JavaScript SDK, you must include the ddn.js file in the index.html file. The JavaScript SDK can be acquired via the npm i dynamsoft-document-normalizer command or use the CDN link https://cdn.jsdelivr.net/npm/dynamsoft-document-normalizer@latest/dist/ddn.js.

<script src="https://cdn.jsdelivr.net/npm/dynamsoft-document-normalizer@1.0.11/dist/ddn.js"></script>

Step 2: Initialize the Flutter Document Scan Plugin

When initializing the Flutter Document Scan Plugin, you need to specify the path of the ddn.js file and the license key.

int ret = await _flutterDocumentScanSdkPlugin.init(
        "https://cdn.jsdelivr.net/npm/dynamsoft-document-normalizer@1.0.11/dist/",
        "DLS2eyJoYW5kc2hha2VDb2RlIjoiMjAwMDAxLTE2NDk4Mjk3OTI2MzUiLCJvcmdhbml6YXRpb25JRCI6IjIwMDAwMSIsInNlc3Npb25QYXNzd29yZCI6IndTcGR6Vm05WDJrcEQ5YUoifQ==");

The built-in templates allow you to change the color mode of the normalized document image. The supported color modes are grayscale, binary and color.

ret = await _flutterDocumentScanSdkPlugin.setParameters(Template.grayscale);

Step 3: Load Image Files and Normalize Documents in Flutter

We are going to load a document image for edge detection and normalization. There are two existing plugins available for loading image files in Flutter web applications: image_picker and file_selector.

Here we use the file_selector plugin to load image files via a button click event.

MaterialButton(
  textColor: Colors.white,
  color: Colors.blue,
  onPressed: () async {
    const XTypeGroup typeGroup = XTypeGroup(
      label: 'images',
      extensions: <String>['jpg', 'png'],
    );
    final XFile? pickedFile = await openFile(
        acceptedTypeGroups: <XTypeGroup>[typeGroup]);
    if (pickedFile != null) {
      image = await loadImage(pickedFile);
      file = pickedFile.path;
    }
  },
  child: const Text('Load Document')),

As the image is loaded, we can call detect() and normalize() methods to detect the document edges and normalize the document image.

detectionResults =
    await _flutterDocumentScanSdkPlugin
        .detect(file);

if (detectionResults.isEmpty) {
  print("No document detected");
} else {
  print("Document detected");
  await normalizeFile(
      file, detectionResults[0].points);
}

Step 4: Draw Custom Shapes and Images with Flutter CustomPaint

To verify the detection and normalization results, we’d better show them by UI elements.

The type of the image we have opened is XFile. The XFile can be decoded into an Image object by the decodeImageFromList() method:

Future<ui.Image> loadImage(XFile file) async {
  final data = await file.readAsBytes();
  return await decodeImageFromList(data);
}

Although Image widget does not support drawing custom shapes, we can draw it with the CustomPaint widget. The CustomPaint widget allows us to draw custom shapes and images. The following code shows how to draw detected edges on the original image and how to draw the normalized document image.

class ImagePainter extends CustomPainter {
  ImagePainter(this.image, this.results);
  final ui.Image image;
  final List<DocumentResult> results;

  @override
  void paint(Canvas canvas, Size size) {
    final paint = Paint()
      ..color = Colors.red
      ..strokeWidth = 5
      ..style = PaintingStyle.stroke;

    canvas.drawImage(image, Offset.zero, paint);
    for (var result in results) {
      canvas.drawLine(result.points[0], result.points[1], paint);
      canvas.drawLine(result.points[1], result.points[2], paint);
      canvas.drawLine(result.points[2], result.points[3], paint);
      canvas.drawLine(result.points[3], result.points[0], paint);
    }
  }

Widget createCustomImage(ui.Image image, List<DocumentResult> results) {
    return SizedBox(
      width: image.width.toDouble(),
      height: image.height.toDouble(),
      child: CustomPaint(
        painter: ImagePainter(image, results),
      ),
    );
  }

Center(
  child: Row(
      mainAxisAlignment: MainAxisAlignment.spaceEvenly,
      children: [
    SingleChildScrollView(
      child: Column(
        children: [
          image == null
              ? Image.asset('images/default.png')
              : createCustomImage(image!, detectionResults),
        ],
      ),
    ),
    SingleChildScrollView(
      child: Column(
        children: [
          normalizedUiImage == null
              ? Image.asset('images/default.png')
              : createCustomImage(normalizedUiImage!, []),
        ],
      ),
    ),
  ])),

Step 5: Run the Flutter Web Document Scanning Application

flutter run -d chrome

Flutter web document edge detection and normalization

Source Code

https://github.com/yushulx/flutter_document_scan_sdk