Ionic Document Scanner in React

Dynamic Web TWAIN has a new scanDocument API since v17.2, which makes it easy to capture documents using mobile cameras with its document border detection feature. Along with the RemoteScan feature that connects physical document scanners, we can now build a decent mobile document scanner with web technologies.

In this article, we are going to create a mobile document scanning app using Ionic React. Ionic is a cross-platform framework for developing mobile apps with web technologies. An Ionic app can not only run as a web app in browsers but also run as a native app on Android and iOS.

Screenshot of the final result:

Home

An online demo running on netlify: link.

What You Should Know About Dynamic Web TWAIN

Build an Ionic Document Scanner

Let’s build the app in steps.

New project

First, install Ionic according to its guide.

After installation, create a new project:

ionic start documentScanner tabs --type react

We can run ionic serve to start a test server.

Install Dynamic Web TWAIN

  1. Install the npm package mobile-web-capture:

     npm install mobile-web-capture

  2. Update the scripts in package.json to copy the resources of Dynamic Web TWAIN to the public folder.

    1. Install ncp as a devDependency.

      npm install ncp --save-dev

    2. Update package.json to use ncp to copy the resources.

       "scripts": {
-  "start": "react-scripts start",
-  "build": "react-scripts build",
+  "start": "ncp node_modules/mobile-web-capture/dist public/assets/dwt-resources && react-scripts start",
+  "build": "ncp node_modules/mobile-web-capture/dist public/assets/dwt-resources && react-scripts build",
 }

Create a Home Page and a Settings Page

We need two pages. A home page which displays scanned documents and provides all kinds of operations and a settings page where we can configure RemoteScan.

  1. Create Home.tsx and Settings.tsx under pages with the following template content:

    import Home from './pages/Home';
import Settings from './pages/Settings';
   
const Home: React.FC<RouteComponentProps> = (props:RouteComponentProps) => {
  return (
   <IonPage>
      <IonHeader>
        <IonToolbar>
          <IonTitle slot="start">Docs Scan</IonTitle>
        </IonToolbar>
      </IonHeader>
      <IonContent>
      </IonContent>
    </IonPage>
  );
}
export default Home;

  2. Manage the navigation in App.tsx:

    const App: React.FC = () => (
  <IonApp>
    <IonReactRouter>
    <IonRouterOutlet>
        <Route path="/" component={Home} exact={true} />
        <Route path="/settings" component={Settings} exact={true} />
      </IonRouterOutlet>
    </IonReactRouter>
  </IonApp>
);

Create a Scanner Component and Add it to the Home Page

We need to create a component to use Dynamic Web TWAIN in React.

Create Scanner.tsx under components with the following content:

import { useEffect, useRef } from "react";
import Dynamsoft from 'mobile-web-capture';
import { WebTwain } from "mobile-web-capture/dist/types/WebTwain";
import { ThumbnailViewer } from "mobile-web-capture/dist/types/WebTwain.Viewer";

interface props {
  license?:string;
  width?: string|number;
  height?: string|number;
  onWebTWAINReady?: (dwt:WebTwain) => void;
}

let DWObject:WebTwain | undefined;
let thumbnail:ThumbnailViewer | undefined;

const Scanner: React.FC<props> = (props: props) => {
  const containerID = "dwtcontrolContainer";
  
  const container = useRef<HTMLDivElement>(null);
  
  const OnWebTWAINReady = () => {
    DWObject = Dynamsoft.DWT.GetWebTwain(containerID);
    if (props.onWebTWAINReady) {
      props.onWebTWAINReady(DWObject);
    }
    if (container.current) {
      if (props.height) {
        DWObject.Viewer.height = props.height;
        container.current.style.height = props.height as string;
      }
      if (props.width) {
        DWObject.Viewer.width = props.width;
        container.current.style.width = props.width as string;
      }
    }

    let thumbnailViewerSettings = {
        location: 'left',
        size: '100%',
        columns: 2,
        rows: 3,
        scrollDirection: 'vertical', // 'horizontal'
        pageMargin: 10,
        background: "rgb(255, 255, 255)",
        border: '',
        allowKeyboardControl: true,
        allowPageDragging: true,
        allowResizing: false,
        showPageNumber: true,
        pageBackground: "transparent",
        pageBorder: "1px solid rgb(238, 238, 238)",
        hoverBackground: "rgb(239, 246, 253)",
        hoverPageBorder: "1px solid rgb(238, 238, 238)",
        placeholderBackground: "rgb(251, 236, 136)",
        selectedPageBorder: "1px solid rgb(125,162,206)",
        selectedPageBackground: "rgb(199, 222, 252)"
    };
    thumbnail = DWObject.Viewer.createThumbnailViewer(thumbnailViewerSettings);
    thumbnail.show();
  }

  useEffect(() => {
    console.log("on mount");
    Dynamsoft.DWT.RegisterEvent('OnWebTwainReady', () => {
      OnWebTWAINReady();
    });
    if (props.license) {
      Dynamsoft.DWT.ProductKey = props.license;
    }
    Dynamsoft.DWT.UseLocalService = false;
    Dynamsoft.DWT.Containers = [{
        WebTwainId: 'dwtObject',
        ContainerId: containerID,
        Width: '300px',
        Height: '400px'
    }];
    Dynamsoft.DWT.ResourcesPath = "assets/dwt-resources/";
    Dynamsoft.DWT.Load();
  }, []);

  return (
    <div ref={container} id={containerID}></div>
  );
}
  
export default Scanner;

In the above code, we load Dynamic Web TWAIN and display a viewer after it is ready to use. Here, ThumbnailViewer is used since it supports long tapping to reorder and multiple selection, which is suitable for mobile devices.

We can now use the component in the Home page. You may need to apply for a license to use Dynamic Web TWAIN.

let DWObject:WebTwain;
const Home: React.FC<RouteComponentProps> = (props:RouteComponentProps) => {
  return (
   <IonPage>
      <IonHeader>
        <IonToolbar>
          <IonTitle slot="start">Docs Scan</IonTitle>
        </IonToolbar>
      </IonHeader>
      <IonContent>
+       <Scanner
+          onWebTWAINReady={(dwt) =>{ DWObject = dwt; }}
+          width={"100%"} 
+          height={"100%"} 
+          license="your license key"
+        />
      </IonContent>
    </IonPage>
  );
}
export default Home;

We can get the object of Dynamic Web TWAIN in the onWebTWAINReady event for further use.

Scan Documents with Camera

Create a floating action button in Home page to start scanning with the camera.

const startCamera = () => {
  if (DWObject) {
    let container = document.createElement("div");
    container.className = "fullscreen";
    document.body.appendChild(container);

    const funcConfirmExit = (bExistImage:boolean):Promise<boolean> => {
      container.remove();
      return Promise.resolve(true);
    }

    const funcConfirmExitAfterSave = () => {
      container.remove();
    };

    let showVideoConfigs:DocumentConfiguration = {
      scannerViewer:{
        element: container,
        continuousScan: false,
        funcConfirmExit: funcConfirmExit,
      },
      documentEditorSettings:{
        element:container,
        funcConfirmExitAfterSave:funcConfirmExitAfterSave
      }
    };
    DWObject.Addon.Camera.scanDocument(showVideoConfigs);
  }
}
  

//...

return (
 <IonPage>
    <IonHeader>
      <IonToolbar>
        <IonTitle slot="start">Docs Scan</IonTitle>
      </IonToolbar>
    </IonHeader>
    <IonContent>
      <Scanner
        onWebTWAINReady={(dwt) =>{ DWObject = dwt; }}
        width={"100%"} 
        height={"100%"} 
        license="your license key"
      />
      <IonFab style={{display:"flex"}} vertical="bottom" horizontal="start" slot="fixed">
        <IonFabButton onClick={() => {
          startCamera();
        }} >
          <IonIcon icon={cameraOutline} />
        </IonFabButton>
      </IonFab>
    </IonContent>
  </IonPage>
);

A full-screen container is appended to body and the camera view will be bound to it. When the camera is closed by the users, the container will be removed.

Scanner.css under styles:

.fullscreen {
  position: absolute;
  left:0;
  top:0;
  width: 100%;
  height: 100%;
}

We can now test scanning documents with cameras. Please note that camera access needs localhost or https.

Camera

Scan Documents with RemoteScan

RemoteScan makes it possible to scan documents from physical document scanners.

We need to have a desktop device which runs Dynamsoft Service and make it accessible via an Intranet. Then, we can scan documents on other devices by communicating with that device through a proxy service. You can learn about how to configure it here.

Let’s add RemoteScan to the app.

  1. When Dynamic Web TWAIN is ready, load the list of all scanning devices found by the proxy service. The URL value of the proxy service is stored in the local storage.

    let scanners:Device[] = [];
let currentURL = "";
//...
const loadScanners = async (URL:string|null) => {
  if (URL) {
    if (currentURL != URL) {
      scanners = await Dynamsoft.DWT.FindDevicesAsync(URL);
      currentURL = URL;
    }
  }
}
return (
  <>
    <Scanner 
      width={"100%"} 
      height={"100%"} 
      license={license}
      onWebTWAINReady={(dwt) =>{ DWObject = dwt; loadScanners(localStorage.getItem("URL")); }}
    />
  </>
)

  2. In Home page, add a floating action button to start remote scan:

    <IonFabButton style={{marginRight:"10px"}} onClick={() => {
  remoteScan();
}} >
  <IonIcon icon={documentOutline} />
</IonFabButton>

    The remote scan function:

    import { Device, DeviceConfiguration } from "mobile-web-capture/dist/types/WebTwain.Acquire";
   
const [deviceConfiguration, setDeviceConfiguration] = useState<DeviceConfiguration|undefined>(undefined);
   
    
useEffect(() => {
  console.log("on mount");
  let deviceConfig:DeviceConfiguration = {
    SelectSourceByIndex: 0,
    ShowRemoteScanUI: false,
    IfShowUI: false,
    IfFeederEnabled: false,
    IfDuplexEnabled: false,
    PixelType: 0,
    Resolution: 300,
    RemoteScan: true
  }
  setDeviceConfiguration(deviceConfig); //default device configuration
}, []);

   
const remoteScan = () => {
  if (DWObject) {
    if (deviceConfiguration && deviceConfiguration.SelectSourceByIndex != undefined) {
      let scanner = scanners[deviceConfiguration.SelectSourceByIndex];
      if (scanner) {
        scanner.acquireImage(deviceConfiguration,DWObject);
      }else{
        alert("Scanner not available.");  
      }
    }else{
      alert("Not configured.");
    }
  }
}

Screenshot:

Remote Scan

Settings

In the settings page, we can add options related to remote scan.

Settings

  1. If a proxy service URL is entered, the user can load the scanners list and then select which scanner to use.

    const reloadScanners = async (selectedIndex?:number) => {
  console.log("find devices from URL: "+URL);
  let devices:Device[] = await Dynamsoft.DWT.FindDevicesAsync(URL);
  console.log(devices);
  let scannersList:string[] = [];
  for (let index = 0; index < devices.length; index++) {
    const device = devices[index];
    scannersList.push(device.displayName);
    if (selectedIndex) {
      if (selectedIndex === index) {
        setSelectedScanner(device.displayName);
      }
    }else{
      if (index === 0) {
        setSelectedScanner(device.displayName);
      }
    }
  }
  setScanners(scannersList);
}

  2. The user can also set up things like resolution, pixel type, whether to show a configuration UI, whether to enable duplex scan and auto document feeder.

    export interface ScanSettings{
  selectedIndex: number;
  showUI: boolean;
  autoFeeder: boolean;
  duplex: boolean;
  resolution: number;
  pixelType: number;
}

If the save button is clicked, save the settings and return to the Home page. The scan settings and URL are stored in local storage.

const save = () =>{
  const selectedIndex = Math.max(0, scanners.indexOf(selectedScanner));
  let scanSettings: ScanSettings = {
    selectedIndex: selectedIndex,
    showUI: showUI,
    autoFeeder: autoFeeder,
    duplex: duplex,
    resolution: resolution,
    pixelType: pixelType
  }
  localStorage.setItem("scanSettings",JSON.stringify(scanSettings));
  if (URL) {
    localStorage.setItem("URL",URL);
  }
  props.history.replace({state:{settingsSaved:true}});
  props.history.goBack();
};

In the Home page, load the settings if the settings are changed:

useEffect(() => {
  const state = props.location.state as { settingsSaved:boolean };
  if (state && state.settingsSaved == true) {
    loadSettings();
  }
}, [props.location.state]);

const loadSettings = () => {
  const settingsAsJSON = localStorage.getItem("scanSettings");
  if (settingsAsJSON) {
    let settings:ScanSettings = JSON.parse(settingsAsJSON);
    let deviceConfig:DeviceConfiguration = {
      SelectSourceByIndex: settings.selectedIndex,
      ShowRemoteScanUI: settings.showUI,
      IfShowUI: settings.showUI,
      IfFeederEnabled: settings.autoFeeder,
      IfDuplexEnabled: settings.duplex,
      PixelType: settings.pixelType,
      Resolution: settings.resolution,
      RemoteScan: true
    }
    setDeviceConfiguration(deviceConfig);
  }

  const URL = localStorage.getItem("URL");
  loadScanners(URL);
}

If the URL has already been set and the scanners are loaded on the Home page, pass the loaded scanners to the Settings page.

On Home page:

let scanners:Device[] = [];
const goToSettings = () => {
  props.history.push("settings",{scanners:getScannerNames()});
}

const getScannerNames = () => {
  let scannerNames:string[] = [];
  for (let index = 0; index < scanners.length; index++) {
    const scanner = scanners[index];
    scannerNames.push(scanner.displayName);
  }
  return scannerNames;
}

On Settings page:

useEffect(() => {
  const state = props.location.state as { scanners:[] };

  if (state && state.scanners) {
    setScanners(state.scanners);
    if (selectedIndex>=0 && selectedIndex<state.scanners.length) {
      setSelectedScanner(state.scanners[selectedIndex]);
    }
  }
  
}, []);

Edit Scanned Documents

Documents can be edited after scanning like cropping, rotation and deleting.

Action

  1. In Home page, add a floating action button to show an action sheet:

    <IonFab style={{display:"flex"}} vertical="bottom" horizontal="end" slot="fixed">
  <IonFabButton onClick={showImageActionSheet}>
    <IonIcon icon={ellipsisVerticalOutline} />
  </IonFabButton>
</IonFab>

  2. Users can select which action to do using the sheet:

    const showImageActionSheet = () => {
  const deleteSelected = () => {
    if (DWObject) {
      DWObject.RemoveAllSelectedImages();
    }
  }

  const editSelected = () => {
    if (DWObject) {
      let container = document.createElement("div");
      container.className = "fullscreen";
      document.body.appendChild(container);
      const funcConfirmExitAfterSave = () => {
        container.remove();
      };
      const funcConfirmExit = (bChanged: boolean, previousViewerName: string):Promise<number | DynamsoftEnumsDWT.EnumDWT_ConfirmExitType> =>  {
        container.remove();
        return Promise.resolve(Dynamsoft.DWT.EnumDWT_ConfirmExitType.Exit);
      };
      let config:DocumentConfiguration = {
        documentEditorSettings:{
          element:container,
          funcConfirmExit:funcConfirmExit,
          funcConfirmExitAfterSave:funcConfirmExitAfterSave
        }
      };
      let documentEditor = DWObject.Viewer.createDocumentEditor(config);
      documentEditor.show();
    }
  }

  present({
    buttons: [{ text: 'Edit selected', handler: editSelected }, 
              { text: 'Delete selected', handler: deleteSelected }, 
              { text: 'Cancel' } ],
    header: 'Select an action'
  })
}

The document editor:

DocumentEditor

Multiple Selection

We can let the thumbnail viewer show checkboxes so that users can select multiple images on mobile devices.

  1. Add a showCheckbox prop to Scanner.

    interface props {
+  showCheckbox?: boolean;
}

  2. Update the thumbnail viewer’s showCheckbox property when props.showCheckbox is changed.

    useEffect(() => {
  if (thumbnail && props.showCheckbox != undefined) {
    thumbnail.showCheckbox = props.showCheckbox;
  }
}, [props.showCheckbox]);

  3. Add a toggle selection item to the image editing action sheet.

    const [showCheckbox,setShowCheckbox] = useState(false);
//...
const toggleMultipleSelection = () => {
  setShowCheckbox(!showCheckbox);
}
   
present({
  buttons: [{ text: 'Toggle multiple selection', handler: toggleMultipleSelection }, 
            { text: 'Delete selected', handler: deleteSelected }, 
            { text: 'Edit selected', handler: editSelected }, 
            { text: 'Cancel' } ],
  header: 'Select an action'
})

Screenshot when checkboxes are shown:

Multiple selection

Export Scanned Documents

After documents are scanned and edited, we can export them.

We can save them as a PDF file and download it or share it.

  1. In Home page, add a share button on the toolbar.

    <IonButton onClick={showShareActionSheet} color="secondary">
  <IonIcon slot="icon-only"  icon={shareOutline} />
</IonButton>

  2. The showShareActionSheet function:

    const showShareActionSheet = () => {
  const save = () => {
    if (DWObject) {
      const OnSuccess = () => {
        console.log('successful');
      }
   
      const OnFailure = () => {
        console.log('error');
      }
      DWObject.SaveAllAsPDF("Scanned.pdf",OnSuccess,OnFailure);
    }
  }

  const share = () => {
    console.log("share");
    const success = async (result:Blob, indices:number[], type:number) => {
      let pdf:File = new File([result],"scanned.pdf");
      const data:ShareData = {files:[pdf]};
      await navigator.share(data);
    }
       
    const failure = (errorCode:number, errorString:string) => {
      console.log(errorString);
    }
    if (DWObject) {
      DWObject.ConvertToBlob(getImageIndices(),Dynamsoft.DWT.EnumDWT_ImageType.IT_PDF,success,failure)
    }
  }
     
  const getImageIndices = () => {
    var indices = [];
    if (DWObject) {
      for (var i=0;i<DWObject.HowManyImagesInBuffer;i++){
        indices.push(i)
      }
    }
    return indices;
  }
     
  present({
    buttons: [{ text: 'Save as PDF', handler:save }, { text: 'Export to PDF and share', handler:share }, { text: 'Cancel' } ],
    header: 'Select an action'
  })
}

Screenshots:

Export

Share

Load Existing Documents

We can also load previously scanned documents to the current viewer using the LoadImageEx API.

We can add a floating action button to call this API. It can load PDF files as well as images. We can use an action sheet for users to select the desired file format.

  1. Add the floating action button in JSX:

    <IonFabButton onClick={loadFile} >
  <IonIcon icon={imageOutline} />
</IonFabButton>

  2. The loadFile function:

    const loadFile = () => {
  if (DWObject) {
    present({
      buttons: 
      [{ text: 'PDF', handler: () => {
        DWObject.LoadImageEx("", Dynamsoft.DWT.EnumDWT_ImageType.IT_PDF);
      } }, 
      { text: 'Image', handler: () => {
        DWObject.LoadImageEx("", Dynamsoft.DWT.EnumDWT_ImageType.IT_ALL);
      }}, 
      { text: 'Cancel' } ],
      header: 'Select file type'
    })
  }
}

Screenshot:

Import

All right, we’ve finished building the ionic document scanner.

Turn the Document Scanner to Android/iOS Native Apps

It is also possible to make the scanner run as a native mobile app using Ionic.

Setup Android and iOS Platforms

  1. Add Android and iOS projects:

    ionic capacitor add android
ionic capacitor add ios

  2. Add permissions in project files:

    For Android, add the following to android\app\src\mainAndroidManifest.xml:

    <uses-permission android:name="android.permission.CAMERA" />
<uses-permission android:name="android.permission.READ_EXTERNAL_STORAGE"/>
<uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" />

    For iOS, add the following to ios\App\App\Info.plist:

    <key>NSCameraUsageDescription</key>
<string>For document scanning</string>

  3. Request camera permission for Android when the app is started:

    useEffect(() => {
  console.log("on mount");
  if (isPlatform("android")) {
    checkAndRequestCameraPermission();
  }
}, []);
   
const checkAndRequestCameraPermission = async () => {
  let result = await AndroidPermissions.checkPermission(AndroidPermissions.PERMISSION.CAMERA);
  if (result.hasPermission == false) {
    let response = await AndroidPermissions.requestPermission(AndroidPermissions.PERMISSION.CAMERA);
    console.log(response.hasPermission);
  }
}

  4. Update styles\Scanner.css to avoid blocking the status bar:

    .fullscreen {
  position: absolute;
  left:0;
  top:0;
+ top: env(safe-area-inset-top);
  width: 100%;
  height: 100%;
+ height: calc(100% - env(safe-area-inset-top));
}

  5. Run with Capacitor​:

    npm run build
ionic capacitor run android
ionic capacitor run ios

Use Native Plugins for Export

Saving and sharing are not handled correctly in WebView. We need to use native plugins to do these.

For saving as a PDF file, we convert the documents to a PDF file and save it to the external directory.

if (Capacitor.isNativePlatform()) {
  const OnSuccess = async (result:Base64Result, indices:number[], type:number) => {
    console.log('successful');
    let writingResult = await Filesystem.writeFile({
      path: getFormattedDate()+".pdf",
      data: result.getData(0,result.getLength()),
      directory: Directory.External
    })
    await Toast.show({
      text: "File is written to "+writingResult.uri,
      duration: "long"
    });
  }

  const OnFailure = () => {
    console.log('error');
  }
  DWObject.ConvertToBase64(getImageIndices(),Dynamsoft.DWT.EnumDWT_ImageType.IT_PDF,OnSuccess,OnFailure)
  
}

For sharing, we convert the documents to a PDF file, save it to the cache directory and then share it:

if (Capacitor.isNativePlatform()) {
  const success = async (result:Base64Result, indices:number[], type:number) => {
    let fileName = getFormattedDate()+".pdf";
    let writingResult = await Filesystem.writeFile({
      path: fileName,
      data: result.getData(0,result.getLength()),
      directory: Directory.Cache
    });
    Share.share({
      title: fileName,
      text: fileName,
      url: writingResult.uri,
    });
  }
  
  const failure = (errorCode:number, errorString:string) => {
    console.log(errorString);
  }

  if (DWObject) {
    DWObject.ConvertToBase64(getImageIndices(),Dynamsoft.DWT.EnumDWT_ImageType.IT_PDF,success,failure)
  }
}

Source Code

Get the source code of the demo to have a try:

https://github.com/xulihang/Ionic-React-Document-Scanner