How to Scan MRZ in an Oracle APEX Application

MRZ stands for machine-readable zone, which is usually at the bottom of the identity page at the beginning of a passport or ID card.1 It can be read by a computing device with a camera to get information like document type, name, document number, nationality, date of birth, sex, and document expiration date.

Oracle APEX is a low-code platform for building web applications. In this article, we are going to build an MRZ scanner plugin so that we can scan MRZ from cameras in an Oracle APEX application. Dynamsoft Label Recognizer is used as the OCR engine.

Screenshot of the demo:

demo

You can find the online demo here.

Build an Oracle APEX Application to Scan MRZ

Let’s do this in steps.

Build an APEX Plugin for Dynamsoft Label Recognizer

In order to use Dynamsoft Label Recognizer to scan MRZ, we need to create an APEX plugin first.

New Plugin

  1. On the app’s page, click shared components. Then click plug-ins in the other components section.
  2. Create a new plugin from scratch. Here, we select the region type as we need to display something and run actions on the page.
  3. Define the f_render function in the PL/SQL code.

    function f_render (
        p_region              in apex_plugin.t_region,
        p_plugin              in apex_plugin.t_plugin,
        p_is_printer_friendly in boolean
    ) return apex_plugin.t_region_render_result is
    

    Then set Render Procedure/Function Name to F_RENDER in the callbacks section.

Next, we are going to put the plugin aside, try to create an MRZ scanning web page first and then adapt it as a plugin.

Build an MRZ Scanning Web App

The web page loads the libraries of Dynamsoft Label Recognizer, Dynamsoft Camera Enhancer and other dependencies via CDN. It can use the camera enhancer library to open the camera and read MRZ from camera frames with the label recognizer library.

  1. Create a new HTML file with the following content.

    <!DOCTYPE html>
    <html>
    <head>
        <title>Dynamsoft MRZ Scanner Sample</title>
        <meta name="viewport" content="width=device-width,initial-scale=1.0,maximum-scale=1.0,user-scalable=0" />
        <script src="script.js"></script>
    </head>
    <body>
      <script>
        window.onload = function(){
        }
      </script>
    </body>
    </html>
    
  2. Create a new script.js file. Define a DLRExtension object in it.

    let DLRExtension = {}
    
  3. Add functions to load external JavaScript files.

    let DLRExtension = {
      loadLibrary: function (src,type,id,data){
        return new Promise(function (resolve, reject) {
          let scriptEle = document.createElement("script");
          scriptEle.setAttribute("type", type);
          scriptEle.setAttribute("src", src);
          if (id) {
            scriptEle.id = id;
          }
          if (data) {
            for (let key in data) {
              scriptEle.setAttribute(key, data[key]);
            }
          }
          document.body.appendChild(scriptEle);
          scriptEle.addEventListener("load", () => {
            console.log(src+" loaded")
            resolve(true);
          });
          scriptEle.addEventListener("error", (ev) => {
            console.log("Error on loading "+src, ev);
            reject(ev);
          });
        });
      }
    }
    
  4. Add a load function to load needed JavaScript files. It also sets the license for Dynamsoft Label Recognizer. You can apply for a license here.

    let DLRExtension = {
      try {
        window.Dynamsoft.CVR.CaptureVisionRouter;
      }catch{
        await this.loadLibrary("https://cdn.jsdelivr.net/npm/dynamsoft-core@3.0.33/dist/core.js","text/javascript");
        await this.loadLibrary("https://cdn.jsdelivr.net/npm/dynamsoft-license@3.0.40/dist/license.js","text/javascript");
        await this.loadLibrary("https://cdn.jsdelivr.net/npm/dynamsoft-label-recognizer@3.0.30/dist/dlr.js","text/javascript");
        await this.loadLibrary("https://cdn.jsdelivr.net/npm/dynamsoft-code-parser@2.0.20/dist/dcp.js","text/javascript");
        await this.loadLibrary("https://cdn.jsdelivr.net/npm/dynamsoft-capture-vision-router@2.0.32/dist/cvr.js","text/javascript");
      }
      try {
        window.Dynamsoft.DCE.CameraEnhancer;
      }catch{
        await this.loadLibrary("https://cdn.jsdelivr.net/npm/dynamsoft-camera-enhancer@4.0.2/dist/dce.js","text/javascript");
      }
      if (pConfig.license) {
        Dynamsoft.License.LicenseManager.initLicense(pConfig.license);
      }else{
        Dynamsoft.License.LicenseManager.initLicense("LICENSE-KEY"); 
      }
    }
    
  5. Add a function to initialize the libraries. A container is appended to the body to display the camera. We can also pass parameters like the styles for the container via the arguments.

    let DLRExtension = {
      router:undefined,
      enhancer:undefined,
      parser:undefined,
      regionID:undefined,
      init: async function(pConfig){
        Dynamsoft.Core.CoreModule.loadWasm(["DLR"]);
        this.router = await Dynamsoft.CVR.CaptureVisionRouter.createInstance();
        let cameraView = await Dynamsoft.DCE.CameraView.createInstance();
        this.enhancer = await Dynamsoft.DCE.CameraEnhancer.createInstance(cameraView);
        let container = document.createElement("div");
        container.id = "enhancerUIContainer";
        document.body.appendChild(container);
        if (pConfig.styles) {
          let styles = JSON.parse(pConfig.styles); //{width:"100%"} e.g.
          for (const key in styles) {
            container.style[key] = styles[key];
          }
        }
        container.append(cameraView.getUIElement());
        container.style.display = "none";
      }
    }
    
  6. Update the settings for scanning MRZ.

    1. Update the runtime settings with a JSON template for scanning MRZ. It will download MRZ models via CDN.

      await this.router.initSettings("{\"CaptureVisionTemplates\": [{\"Name\": \"mrz\",\"ImageROIProcessingNameArray\": [\"roi-mrz-passport\"]}],\"TargetROIDefOptions\": [{\"Name\": \"roi-mrz-passport\",\"TaskSettingNameArray\": [\"task-mrz-passport\"]}],\"TextLineSpecificationOptions\": [{\"Name\": \"tls-mrz-text\",\"CharacterModelName\": \"MRZ\",\"StringRegExPattern\": \"([ACI][A-Z<][A-Z<]{3}[A-Z0-9<]{9}[0-9][A-Z0-9<]{15}){(30)}|([0-9]{2}[(01-12)][(01-31)][0-9][MF<][0-9]{2}[(01-12)][(01-31)][0-9][A-Z<]{3}[A-Z0-9<]{11}[0-9]){(30)}|([A-Z<]{30}){(30)}|([ACIV][A-Z<][A-Z<]{3}[A-Z<]{31}){(36)}|([A-Z0-9<]{9}[0-9][A-Z<]{3}[0-9]{2}[(01-12)][(01-31)][0-9][MF<][0-9]{2}[(01-12)][(01-31)][0-9][A-Z0-9<]{8}){(36)}|([PV][A-Z<][A-Z<]{3}[A-Z<]{39}){(44)}|([A-Z0-9<]{9}[0-9][A-Z<]{3}[0-9]{2}[(01-12)][(01-31)][0-9][MF<][0-9]{2}[(01-12)][(01-31)][0-9][A-Z0-9<]{14}[A-Z0-9<]{2}){(44)}\",\"StringLengthRange\": [30,44],\"CharHeightRange\": [5,1000,1],\"BinarizationModes\": [{\"BlockSizeX\": 30,\"BlockSizeY\": 30,\"Mode\": \"BM_LOCAL_BLOCK\",\"MorphOperation\": \"Close\"}]},{\"Name\": \"tls-mrz-passport\",\"StringRegExPattern\": \"(P[A-Z<][A-Z<]{3}[A-Z<]{39}){(44)}|([A-Z0-9<]{9}[0-9][A-Z<]{3}[0-9]{2}[(01-12)][(01-31)][0-9][MF<][0-9]{2}[(01-12)][(01-31)][0-9][A-Z0-9<]{14}[0-9<][0-9]){(44)}\",\"StringLengthRange\": [44,44],\"BaseTextLineSpecificationName\": \"tls-mrz-text\"}],\"LabelRecognizerTaskSettingOptions\": [{\"Name\": \"mrz-text-task\",\"TextLineSpecificationNameArray\": [\"tls-mrz-text\"],\"SectionImageParameterArray\": [{\"Section\": \"ST_REGION_PREDETECTION\",\"ImageParameterName\": \"ip-mrz-text\"},{\"Section\": \"ST_TEXT_LINE_LOCALIZATION\",\"ImageParameterName\": \"ip-mrz-text\"},{\"Section\": \"ST_TEXT_LINE_RECOGNITION\",\"ImageParameterName\": \"ip-mrz-text\"}]},{\"Name\": \"task-mrz-passport\",\"TextLineSpecificationNameArray\": [\"tls-mrz-text\"],\"BaseLabelRecognizerTaskSettingName\": \"mrz-text-task\"}],\"CharacterModelOptions\": [{\"Name\": \"MRZ\"}],\"ImageParameterOptions\": [{\"Name\": \"ip-mrz-text\",\"TextureDetectionModes\": [{\"Mode\": \"TDM_GENERAL_WIDTH_CONCENTRATION\",\"Sensitivity\": 8}],\"TextDetectionMode\": {\"Mode\": \"TTDM_LINE\",\"CharHeightRange\": [20,1000,1],\"Sensitivity\": 7}}]}");
      
    2. Set a scan region so that only a part of the frame will be processed.

      this.enhancer.setScanRegion({x:0,y:30,width:100,height:30,isMeasuredInPercentage:true})
      
  7. Add a function to open the camera.

    let DLRExtension = {
      open: async function(){
        document.getElementById("enhancerUIContainer").style.display = "";
        await this.enhancer.open(true);
      }
    }
    
  8. Add a function to close the camera.

    let DLRExtension = {
      close: function(){
        this.enhancer.close(true);
        document.getElementById("enhancerUIContainer").style.display = "none";
      }
    }
    
  9. Add functions to start scanning from the camera frames and stop scanning.

    let DLRExtension = {
      interval:undefined,
      processing:undefined,
      textResults:undefined,
      callback:undefined,
      startScanning: function(){
        this.stopScanning();
        let pThis = this;
        const captureAndProcess = async function() {
          if (!pThis.enhancer || !pThis.router) {
            return;
          }
          if (pThis.enhancer.isOpen() === false) {
            return;
          }
          if (pThis.processing === true) {
            return;
          }
          pThis.processing = true; // set processing to true so that the next frame will be skipped if the processing has not completed.
          let frame = pThis.enhancer.fetchImage();
          if (frame) {
            let result = await pThis.router.capture(frame,"mrz");
            if (result.items && result.items.length > 0) {
              pThis.textResults = result.items;
              if (pThis.callback) {
                pThis.callback(result.items);
              }
            }
            pThis.processing = false;
          }
        }
        this.interval = setInterval(captureAndProcess,100); // set an interval to read MRZ
      },
      stopScanning: function(){
        if (this.interval) {
          clearInterval(this.interval);
          this.interval = undefined;
        }
        this.processing = false;
      }
    }
    
  10. Register the played event for the camera enhancer so that when the resolution or the camera is changed, restart scanning.

    this.enhancer.on("played", (playCallbackInfo) => {
      if (this.interval) {
        this.startScanning();
      }
    });
    
  11. Add functions using Dynamsoft Code Parser to extract info from the MRZ string.

    init():function(){
      await Dynamsoft.DCP.CodeParserModule.loadSpec("MRTD_TD1_ID");
      await Dynamsoft.DCP.CodeParserModule.loadSpec("MRTD_TD2_FRENCH_ID");
      await Dynamsoft.DCP.CodeParserModule.loadSpec("MRTD_TD2_ID");
      await Dynamsoft.DCP.CodeParserModule.loadSpec("MRTD_TD2_VISA");
      await Dynamsoft.DCP.CodeParserModule.loadSpec("MRTD_TD3_PASSPORT");
      await Dynamsoft.DCP.CodeParserModule.loadSpec("MRTD_TD3_VISA");
      this.parser = await Dynamsoft.DCP.CodeParser.createInstance();
    },
    getParsedString: async function(mrzString){
      let str = "";
      let parsedResultItem = await this.parser.parse(mrzString);
      console.log(parsedResultItem);
      let MRZFields = ["documentNumber","passportNumber","issuingState","name","sex","nationality","dateOfExpiry","dateOfBirth"];
      for (let index = 0; index < MRZFields.length; index++) {
        const field = MRZFields[index];
        const value = parsedResultItem.getFieldValue(field);
        if (value){
          str = str + field + ": " + value + "\n";
        }
      }
      return str;
    },
    getMRZString: function(items){
      let str = "";
      for (let index = 0; index < items.length; index++) {
        const item = items[index];
        str = str + item.text;
        if (index != items.length - 1) {
          str = str + "\n";
        }
      }
      return str;
    },
    
  12. Add the following scripts in the HTML file to start the MRZ scanner.

    window.onload = async function(){
      let styles = {width:"100%",height:"100%",left:0,top:0,position:"absolute"}
      await DLRExtension.load({});
      await DLRExtension.init({styles:JSON.stringify(styles)});
      await DLRExtension.open();
      await DLRExtension.setCallback(async function(items){
        DLRExtension.close();
        let mrzString = await DLRExtension.getMRZString(items);
        let parsedString = await DLRExtension.getParsedString(mrzString);
        alert(parsedString);
      });
      DLRExtension.startScanning();
    }
    

All right, we have finished writing the web page. Next, we are going to adapt it for the APEX plugin.

Adapt the Web Page for APEX

  1. If it is running in an APEX app, append the container to the region in the init function.

    if ('apex' in window) {
      this.regionID = pConfig.regionID;
      const region = document.getElementById(this.regionID);
      region.appendChild(container);
    }else{
      document.body.appendChild(container);
    }
    
  2. Get the page item from the pConfig argument and set its content to the text of the parsed MRZ string.

    let DLRExtension = {
      item: undefined,
      init: function(pConfig){
        //...
        if ('apex' in window) {
          this.regionID = pConfig.regionID;
          this.item = pConfig.item;
          const region = document.getElementById(this.regionID);
          region.appendChild(container);
        }
        //...
      },
      startScanning: function(){
        //...
        if (result.items && result.items.length > 0) {
          if ('apex' in window) {
            if (pThis.item) {
              let parsedString = await pThis.getParsedString(pThis.getMRZString(result.items));
              apex.item(pThis.item).setValue(parsedString);
            }
          }
        }
        //...
      },
    }
    
  3. On the setup page of the plugin, add several custom attributes.

    Attributes

    • styles: JSON string to set the styles.
    • MRZ result container: page item for the parsed MRZ result.
    • license: license for Dynamsoft Label Recognizer. You can apply for a license here.
  4. Upload the script.js file and add it to the list of file URLs to load.

    Script files

  5. In the PL/SQL code, run the functions for initialization.

    begin
      apex_javascript.add_onload_code (
          p_code => '(async () => { await DLRExtension.load({' ||
              apex_javascript.add_attribute(p_name => 'license', p_value => p_region.attribute_02, p_add_comma => false ) ||
            '});
            DLRExtension.init({' ||
              apex_javascript.add_attribute(p_name => 'styles', p_value => p_region.attribute_01, p_add_comma => true ) ||
              apex_javascript.add_attribute(p_name => 'item', p_value => p_region.attribute_03, p_add_comma => true ) ||
              apex_javascript.add_attribute(p_name => 'regionID', p_value => p_region.static_id, p_add_comma => false ) ||
            '}); })();',
          p_key  => null );
      return null;
    end;
    

Use the Plugin to Scan MRZ

  1. Create a new MRZ scanner page and in its page designer, drag the MRZ scanner region into the app, add a button and a text field to display the MRZ result. In addition, set the attributes for the region.

    Page designer

  2. Add a dynamic action for the StartScanButton button. When it is clicked, execute the following JavaScript code:

    (async () => {
      if (DLRExtension.router) {
        await DLRExtension.open();
        DLRExtension.startScanning();
      }else{
        alert("The MRZ scanner is still initializing.");
      }
    })();
    

All right, we’ve now added the MRZ scanning function in an Oracle APEX application.

Source Code

Get the source code of the plugin to have a try:

https://github.com/tony-xlh/APEX-MRZ-Scanner

References

  1. https://en.wikipedia.org/wiki/Machine-readable_passport 

Disclaimer:

The wrappers and sample code on Dynamsoft Codepool are community editions, shared as-is and not fully tested. Dynamsoft is happy to provide technical support for users exploring these solutions but makes no guarantees.