Building a Web Document Rectification App with Blazor WebAssembly and Dynamsoft Document Normalizer SDK

Aug 23, 2023

Web-based document management systems are evolving rapidly, and a critical feature often in demand is document rectification. The Dynamsoft Document Normalizer SDK offers a suite of user-friendly JavaScript APIs for document detection and rectification in web browsers. This article will guide you through the process of building a web-based document rectification app.

This article is Part 3 in a 3-Part Series.

Try Online Demo

https://yushulx.me/dotnet-blazor-document-rectification/

Prerequisites

.NET SDK
Obtain the JavaScript edition of the Dynamsoft Document Normalizer SDK from NPM or load it directly from cdn.jsdelivr.net:
```
  <script src="https://cdn.jsdelivr.net/npm/dynamsoft-document-normalizer@1.0.12/dist/ddn.js"></script>
```
Request a trial license.

Step 1: Setting Up Your Blazor WebAssembly Project

Create a Blazor WebAssembly project with the following command:
```
 dotnet new blazorwasm -o BlazorDocRectifySample
```
Navigate to the new directory.
```
 cd BlazorDocRectifySample
```
Open wwwroot/index.html and add both the Dynamsoft Camera Enhancer SDK and the Dynamsoft Document Normalizer SDK:
```
 <script src="https://cdn.jsdelivr.net/npm/dynamsoft-camera-enhancer@3.3.5/dist/dce.js"></script>
 <script src="https://cdn.jsdelivr.net/npm/dynamsoft-document-normalizer@1.0.12/dist/ddn.js"></script>
```
- Dynamsoft Camera Enhancer is a tool used for accessing the webcam video stream. It is available free of charge.
- Dynamsoft Document Normalizer SDK is utilized for document rectification. To use it, a valid license key is required.
Create a jsInterop.js file for interoperation between C# and JavaScript, and then add it to the index.html file:
```
 <script src="jsInterop.js"></script>
```

Step 2: Building the UI

We create two Razor components: one for detecting documents from an image file and the other for detecting documents from a camera stream.

dotnet new razorcomponent -n ImageFile -o Pages
dotnet new razorcomponent -n CameraStream -o Pages

The `ImageFile` component

@page "/image-file"
@inject IJSRuntime JSRuntime

<div>
<button class="btn" @onclick="DocDetect">Load file</button>
<button class="btn" @onclick="Rectify">Rectify</button>
<button class="btn" @onclick="Save">Save</button>
</div>

<div>
    <input type="radio" name="format" value="grayscale" @onchange="HandleInputChange">Grayscale
    <input type="radio" name="format" value="color" checked @onchange="HandleInputChange">Color
    <input type="radio" name="format" value="binary" @onchange="HandleInputChange">Binary
</div>

<div class="container">
    <div>
        <div id="imageview">
            <img id="image" />
            <canvas id="overlay"></canvas>
        </div>
    </div>
    <div>
        <canvas id="canvas"></canvas>
    </div>
</div>

Blazor document rectification from image file

The DocDetect() method is invoked when the button is clicked. It invokes the selectFile() method in JavaScript and pass the C# object reference and canvas IDs to it. When a document is detected, one canvas is used to display the detected quadrilateral and the other to display the rectified document.
```
  public async Task DocDetect()
  {
      await JSRuntime.InvokeVoidAsync(
      "jsFunctions.selectFile", objRef, "image");
  }
```

The Rectify() method triggers document rectification manually.

  public async Task Rectify()
  {
      await JSRuntime.InvokeVoidAsync(
      "jsFunctions.rectify");
  }

The Save() method saves the rectified document to an image file.

  public async Task Save()
  {
      await JSRuntime.InvokeVoidAsync(
      "jsFunctions.save", objRef, "canvas");
  }

The HandleInputChange() method handles the radio button change event. It provides three options for the output color format: grayscale, color, and binary.

  public async Task HandleInputChange(ChangeEventArgs e)
  {
      await JSRuntime.InvokeVoidAsync(
      "jsFunctions.setOutputFormat", e.Value.ToString());
  }

The <img> element is used to display the selected image file.

The `CameraStream` component

@page "/camera-stream"
@inject IJSRuntime JSRuntime

<div class="select">
    <label for="videoSource">Video source: </label>
    <select id="videoSource"></select>
</div>

<div>
<button class="btn" @onclick="Rectify">Rectify</button>
<button class="btn" @onclick="Save">Save</button>
</div>

<div class="container">
    <div id="videoview">
        <div class="dce-video-container" id="videoContainer"></div>
        <canvas id="overlay"></canvas>
    </div>
    <div>
        <canvas id="canvas"></canvas>
    </div>
</div>

Blazor document rectification from camera stream

The <select> element lists all available cameras.
The buttons and canvases work the same as in the ImageFile component.
Camera stream is displayed in the <div> element with the dce-video-container class.

Step 3: Implementing the Document Rectification Logic

The primary logic for document rectification is implemented in JavaScript. The main interop methods are defined in the wwwroot/jsInterop.js file:

window.jsFunctions = {
    initSDK: async function (licenseKey) {
        
    },
    initImageFile: async function (dotnetRef, canvasId) {
        
    },
    initScanner: async function (dotnetRef, videoId, selectId, canvasOverlayId, canvasId) {
        
    },
    selectFile: async function (dotnetRef, canvasOverlayId, imageId) {
        
    },
    rectify: async function () {
        
    },
    updateSetting: async function (color) {
        
    },
    save: async function () {
        
    },
};

The initSDK() method initializes the Dynamsoft Document Normalizer SDK with a valid license key. It also sets the runtime settings for document rectification.

  async function init() {
      normalizer = await Dynamsoft.DDN.DocumentNormalizer.createInstance();
      let settings = await normalizer.getRuntimeSettings();
      settings.ImageParameterArray[0].BinarizationModes[0].ThresholdCompensation = 9;
      settings.NormalizerParameterArray[0].ColourMode = "ICM_COLOUR"; // ICM_BINARY, ICM_GRAYSCALE, ICM_COLOUR
    
      await normalizer.setRuntimeSettings(settings);
  }
    
  initSDK: async function (licenseKey) {
      let result = true;

      if (normalizer != null) {
          return result;
      }
        
      try {
          Dynamsoft.DDN.DocumentNormalizer.license = licenseKey;
      } catch (e) {
          console.log(e);
          result = false;
      }
        
      await init();

      return result;
  }

The initImageFile() method initializes the canvases for rendering the quadrilateral and the rectified document.

  initImageFile: async function (dotnetRef, canvasOverlayId, canvasRectifyId) {
      dotnetHelper = dotnetRef;
      initOverlay(document.getElementById(canvasOverlayId));
      canvasRectify = document.getElementById(canvasRectifyId);
      contextRectify = canvasRectify.getContext('2d');
      if (normalizer != null) {
          normalizer.stopScanning();
      }
      await init();

      return true;
  }

The initCameraStream() method initializes the camera stream and the canvases for rendering the quadrilateral and the rectified document.

  initCameraStream: async function (dotnetRef, videoId, selectId, canvasOverlayId, canvasId) {
      await init();
      canvasRectify = document.getElementById(canvasId);
      contextRectify = canvasRectify.getContext('2d');
      let canvas = document.getElementById(canvasOverlayId);
      data = {};
      initOverlay(canvas);
      videoContainer = document.getElementById(videoId);
      videoSelect = document.getElementById(selectId);
      videoSelect.onchange = openCamera;
      dotnetHelper = dotnetRef;

      try {
          enhancer = await Dynamsoft.DCE.CameraEnhancer.createInstance();
          await enhancer.setUIElement(document.getElementById(videoId));
          await normalizer.setImageSource(enhancer, { });
          await normalizer.startScanning(true);
          let cameras = await enhancer.getAllCameras();
          listCameras(cameras);
          await openCamera();

          normalizer.onQuadDetected = (quads, sourceImage) => {
              clearOverlay();
              if (quads.length == 0) {
                  return;
              }
              data["file"] = sourceImage;
              let location = quads[0].location;
              data["points"] = quads[0].location;
              drawQuad(location.points);
          };
          enhancer.on("played", playCallBackInfo => {
              updateResolution();
          });

      } catch (e) {
          console.log(e);
          result = false;
      }
      return true;
  }
    

The selectFile() method detects documents from an image file.

  selectFile: async function (dotnetRef, imageId) {
      data = {};
        
      if (normalizer) {
          let input = document.createElement("input");
          input.type = "file";
          input.onchange = async function () {
              try {
                  let file = input.files[0];
                  var fr = new FileReader();
                  fr.onload = function () {
                      let image = document.getElementById(imageId);
                      image.src = fr.result;
                      image.style.display = 'block';
                        
                      decodeImage(fr.result);
                  }
                  fr.readAsDataURL(file);

              } catch (ex) {
                  alert(ex.message);
                  throw ex;
              }
          };
          input.click();
      } else {
          alert("The SDK is still initializing.");
      }
  }

The rectify() method triggers document rectification based on the detected quadrilateral.

  async function normalize(file, location) {
      if (file == null || location == null) {
          return;
      }
      if (normalizer) {
          normalizedImageResult = await normalizer.normalize(file, {
              quad: location
          });
          if (normalizedImageResult) {
              let image = normalizedImageResult.image;
              canvasRectify.width = image.width;
              canvasRectify.height = image.height;
              let data = new ImageData(new Uint8ClampedArray(image.data), image.width, image.height);
              contextRectify.clearRect(0, 0, canvasRectify.width, canvasRectify.height);
              contextRectify.putImageData(data, 0, 0);
          }
      }
  }

  rectify: async function () {
      await normalize(data["file"], data["points"]);
  },

The updateSetting() method changes the color format of the rectified document.

  updateSetting: async function (color) {
      let colorMode = "ICM_GRAYSCALE";
      if (color === 'grayscale') {
          colorMode = "ICM_GRAYSCALE";
      } else if (color === 'color') {
          colorMode = "ICM_COLOUR";
      } else if (color === 'binary') {
          colorMode = "ICM_BINARY";
      }

      if (normalizer && data['file']) {
          let settings = await normalizer.getRuntimeSettings();
          settings.NormalizerParameterArray[0].ColourMode = colorMode;
          await normalizer.setRuntimeSettings(settings);
          normalize(data["file"], data["points"]);
      }
  }

The save() method saves the rectified document to an image file.

  save: async function () {
      if (normalizedImageResult) {
          await normalizedImageResult.saveToFile("document-normalization.png", true);
      }
  }

Step4: Editing Quadrilateral for Better Rectification

It is inevitable that the detected quadrilateral may not be accurate. To improve the rectification result, the quadrilateral editing feature is required.

We add mouse event handlers to the canvas to update the points of the quadrilateral and redraw it.

function initOverlay(ol) {
    canvasOverlay = ol;
    canvasOverlay.addEventListener("mousedown", updatePoint);
    canvasOverlay.addEventListener("touchstart", updatePoint);
    contextOverlay = canvasOverlay.getContext('2d');
}

function updatePoint(e) {
    let points = data["points"].points;
    let rect = canvasOverlay.getBoundingClientRect();

    let scaleX = canvasOverlay.clientWidth / canvasOverlay.width;
    let scaleY = canvasOverlay.clientHeight / canvasOverlay.height;
    let mouseX = (e.clientX - rect.left) / scaleX;
    let mouseY = (e.clientY - rect.top) / scaleY;

    let delta = 10;
    for (let i = 0; i < points.length; i++) {
        if (Math.abs(points[i].x - mouseX) < delta && Math.abs(points[i].y - mouseY) < delta) {
            canvasOverlay.addEventListener("mousemove", dragPoint);
            canvasOverlay.addEventListener("mouseup", releasePoint);
            canvasOverlay.addEventListener("touchmove", dragPoint);
            canvasOverlay.addEventListener("touchend", releasePoint);
            function dragPoint(e) {
                let rect = canvasOverlay.getBoundingClientRect();
                let mouseX = e.clientX || e.touches[0].clientX;
                let mouseY = e.clientY || e.touches[0].clientY;
                points[i].x = Math.round((mouseX - rect.left) / scaleX);
                points[i].y = Math.round((mouseY - rect.top) / scaleY);
                drawQuad(points);
            }
            function releasePoint() {
                canvasOverlay.removeEventListener("mousemove", dragPoint);
                canvasOverlay.removeEventListener("mouseup", releasePoint);
                canvasOverlay.removeEventListener("touchmove", dragPoint);
                canvasOverlay.removeEventListener("touchend", releasePoint);
            }
            break;
        }
    }
}

edge edit

Source Code

https://github.com/yushulx/dotnet-blazor-document-rectification

LANGUAGES

PLATFORMS

FEATURED

Building a Web Document Rectification App with Blazor WebAssembly and Dynamsoft Document Normalizer SDK

Try Online Demo

Prerequisites

Step 1: Setting Up Your Blazor WebAssembly Project

Step 2: Building the UI

The `ImageFile` component

The `CameraStream` component

Step 3: Implementing the Document Rectification Logic

Step4: Editing Quadrilateral for Better Rectification

Source Code

Search Blog Posts

POPULAR TOPICS

LANGUAGES

PLATFORMS

FEATURED

Building a Web Document Rectification App with Blazor WebAssembly and Dynamsoft Document Normalizer SDK

Try Online Demo

Prerequisites

Step 1: Setting Up Your Blazor WebAssembly Project

Step 2: Building the UI

The ImageFile component

The CameraStream component

Step 3: Implementing the Document Rectification Logic

Step4: Editing Quadrilateral for Better Rectification

Source Code

Search Blog Posts

POPULAR TOPICS

Useful Links

The `ImageFile` component

The `CameraStream` component