MobbScan Web: Face Matching

Introduction

MobbScan can scan a user's face and check if it matches the photo of a scanned document. It is a step that should be performed when a document has been successfully scanned.

Installation

To enable the face matching step, an additional script should be added in the head HTML element, replacing {version} with your current MobbScan version, and specifying the correct path to your mobbscan-api folder:

<script type="text/javascript" src="./mobbscan-api/mobbscan-api-face-{version}.min.js"></script>

After starting the scan, a listener should be added, which will be called when a face frame has been analyzed, similar to how it has been done with MobbScanAPI.setDetectionListener during the scan of a document:

MobbScanAPI.setFaceDetectionListener(onFaceDetectionResult);

After that call, MobbScanAPI.start will render an additional hidden HTML element that will start a face scan when clicked.

Adding face scanning step

After receiving an OK result in the MobbScanAPI.scan listener, the face matching step can be started. First of all, the HTML elements related to the face scan should be visible (and optionally, the document HTML elements could be hidden):

MobbScanAPI.hideDocumentsAndShowFace();

When the face UI is clicked, the scan of the face will start and onFaceDetectionResult will be triggered for each captured frame.

MobbScanAPI.setFaceDetectionListener

As specified above, a listener that will be called every time a frame is analyzed should be defined, and it can be done using this method. The listener that is set will receive three arguments:

  • result: String which indicates the result of the last detection:

    • STARTED: Nothing has been detected.

    • CAPTURING: A face has been detected, but the server should check the image.

    • NOT_DETECTED: The face could not be detected.

    • DETECTED: The detection has been confirmed by MobbScan server.

    • TIMEOUT : A timeout error has occurred during the face detection process. If you want to know more about this, you can go to Error Handling section.

    • ERROR: An error has ocurred during the process.

  • data: An object which contains data about the current detection process. Relevant attributes:

    • source: The source of the processed image. It can take the following values:

      • WEBCAM
      • FILE
      • MOBILE
      • PDF

    • image: This attribute will be available when the result is equal to DETECTED. It contains a Blob which contains the image of the detected face. This image should be used with the method MobbScanAPI.validate() to complete the scanning of a face.

  • error: In case of an error, this parameter will contain further details about it. If you want to know more about the information received in this field, you can go to Error Handling section.

MobbScanAPI.validate

This method should be called after a face has been captured with the listener set with MobbScanAPI.setFaceDetectionListener. This method has three arguments:

  • validationType: This argument should be "FACE".

  • faceImage: Blob which contains the image of the detected face. It can be retrieved in the listener defined in MobbScanAPI.setFaceDetectionListener.

  • listener: A function that will be called when the scan of the face finishes. It receives the following parameters:

    • result: It indicates the scan result. It can take the following values:

      • OK
      • ERROR

    • resultData: This contains relevant data about the face validation performed in the MobbScan server in its data attribute. The most relevant fields are:

      • state: It contains the VALID value if the face matches, INVALID otherwise.

      • score: It is a score over 100 that represents the similarity between the face that appears in the document and the face detected by MobbScan. It can be used to establish your own threshold instead of relying on the state field.

    • error: In case of an error, this parameter will contain further details about it. If you want to know more about the information received in this field, you can go to Error Handling section.

Example of a face detection listener

NOTE: Error handling omitted for educational purposes.

const onFaceDetectionResult = (result, data, error) => {
    if (result === "DETECTED") {
        MobbScanAPI.validate(
            "FACE",
            data.image,
            (result, resultData, error) => {
                if (result === "OK") {
                    console.log("Face matched with score", resultData.data.score)
                }
            }
        )
    }
}

Perform face matching with new instance of the component

If you want to scan the document and perform the face matching in two separate files you should destroy the instance of MobbScanAPI at face matching step. To recreate the interface on face matching step and finish the process you have to follow these steps:

  • Initialize the api as usual, if the api is initialized you can omit this step.
  • Set face detection listener with setFaceDetectionListener method
  • Call detectFace with the scanId of the process to match face and the placeholder if you didn't call setPlaceHolder before.
  • On face detection listener call validate when result is MobbScanFaceDetectionResult.DETECTED

Example

    // Sets license
    MobbScanAPI.initAPI('YOUR LICENSE HERE');
    // Sets base url
    MobbScanAPI.setBaseURL("https://yourmobbscaninstance.com");
    // This listener is called when a face frame has been analyzed
    MobbScanAPI.setFaceDetectionListener(onFaceDetectionResult);
    // If you want to use the native mode on mobile you have to call this method
    MobbScanAPI.enableVideoMobileCapture();
    // Sets the timeoutListener
    MobbScanAPI.setTimeOutListener(console.error);
    // Sets the timeoutListener on mobile
    MobbScanAPI.setTimeOutListenerVideoMobile(console.error);
    // Sets the error accessing camera
    MobbScanAPI.setErrorAccessingCamera(console.error);
    // Starts face detection
    MobbScanAPI.detectFace("YOUR SCAN ID", "mobbscan-placeholder")

    function onFaceDetectionResult(result, data, error) {
        if (error) {
            throw error;
        }
        // If is a DETECTED result...
        if ((result === MobbScanFaceDetectionResult.DETECTED)) {
            // After a DETECTED image, it should be sent to server so it can be checked with the data extracted
            // from the document
            MobbScanAPI.validate(
                MobbScanValidationType.FACE,
                data.image,
                (result, resultData, error) => {
                // Callback called when `validate` request finishes
                if (result === MobbScanScanResult.OK) {
                }
            });
        }
    }