Perform face matching with MobbScan · Mobbeel for developers

MobbScan allows you to perform a face biometric matching between the user using the app and the picture on the ID document.

Once the document scan is completed you just need to call a new method validateFace that will guide the user to capture their face biometrics to be able to match it against the document picture.

As on the scan process, we have two different processes here, a first process where the selfie is taken, and a second process where the validation is performed (check the selfie with the document face).

There are six different operation modes to perform face matching:

Default face matching.
Default face matching with MobbScan's UI
Default face matching with a custom UI.
Face matching with liveness check.
Provide the SDK with a selfie image captured by other means.
Video identity verification.

These are the different modes to perform the face detection:

Default: this will look for a face in front of the camera and will take a picture when the face is detected.
Liveness by head movement via video: this will record a video asking the user to move their head and evaluate the liveness check in the server.

There are specific value in the MobbScanFaceValidationMode for each of them. So, to invoke each of the methods, you only have to change the appropiate value on the method call.

Kotlin

Java

MobbScanFaceValidationMode.DEFAULT
MobbScanFaceValidationMode.LIVENESS_VIDEO_HEAD_MOVEMENT

MobbScanFaceValidationMode.DEFAULT
MobbScanFaceValidationMode.LIVENESS_VIDEO_HEAD_MOVEMENT

Default face matching

This mode provides a simple biometric validator using a selfie captured with the tools already included in the SDK. With no extra configuration required.

Kotlin

Java

var validationMode = MobbScanFaceValidatorMode.DEFAULT
MobbScanAPI.getInstance().validateFace("YOUR_SCAN_ID_HERE", validationMode, { faces -> 
    // Get the selfie(s)... This will be invoked as soon as the SDK captures the user's selfie
}, { result, resultData, error -> 
    // Your validation listener code here...
})

MobbScanFaceValidationMode validationMode = MobbScanFaceValidationMode.DEFAULT;
MobbScanAPI.getInstance().validateFace ("YOUR_SCAN_ID_HERE", validationMode, new FaceAcquisitorListener() {
    @Override
    public void onFaceAcquired(ArrayList<Bitmap> faces) {
        // Get the selfie(s)... This will be invoked as soon as the SDK captures the user's selfie
    }
}, new ValidationCheckListener() {
    @Override
    public void onValidationChecked(MobbScanValidationState result, MobbScanValidationResultData resultData, MobbScanAPIError error) {
        // Your validation listener code here...
    }
});

Note: The FaceAcquisitorListener can be omitted if you don't want to get the selfie image as soon as it is captured.

As you can see, the FaceAcquisitorListener returns the selfie image captured, but the ValidationCheckListener returns 3 objects.

In order to retrieve the information from the ValidationCheckListener, you can check the Retrieving the results from the callback section of this document.

Only perform face detection

You can use this method if you just want MobbScan to get a crop of the user face without carry on the validation. The returned image can be used to perform a validation later on as shown here.

Kotlin

Java

MobbScanAPI.getInstance().detectFace(MobbScanFaceValidationMode.DEFAULT) { result, data, error -> 
    // Your face acquisitor listener code here...            
}

MobbScanAPI.getInstance().detectFace(MobbScanFaceValidationMode.DEFAULT, new FaceAcquisitorListener() {
    @Override
    public void onFaceAcquired(MobbScanFaceAcquisitorResult result, MobbScanFaceAcquisitorData data, MobbScanAPIError error) {
        // Your face acquisitor listener code here...
    }
});

Default face matching with custom UI

It may be of your interest to display your own UI for the face detection. In this case, you only need to add a FrameLayout in your view, and set it before calling the detection method.

Also, you need to define a FaceDetectionFeedbackListener in order to receive all the feedback from the detection process.

Like the previous process, this one only returns a selfie image. No validation is performed, but you can use the returned image to perform the validation as shown here.

Layout

Kotlin

Java

<YourLayout>

    ...

    <FrameLayout
        android:id="@+id/cameraViewContainer"
        android:layout_width="match_parent"
        android:layout_height="wrap_content" />

    ...

</YourLayout>

val detectionTrack = MobbScanDetectionTrack(cameraViewContainer)
MobbScanAPI.getInstance().setFaceDetectionTrack(detectionTrack)
MobbScanAPI.getInstance().setFaceDetectionFeedbackListener { result, feedbackData ->
        // Your face detection feedback listener code here...
}
MobbScanAPI.getInstance().detectFace(MobbScanFaceValidationMode.DEFAULT) { result, resultData, error ->
        // Your face acquisitor listener code here...
}

ViewGroup cameraViewContainer = findViewById(R.id.cameraViewContainer);
MobbScanDetectionTrack detectionTrack = new MobbScanDetectionTrack(cameraViewContainer);
MobbScanAPI.getInstance().setFaceDetectionTrack(detectionTrack);
MobbScanAPI.getInstance().setFaceDetectionFeedbackListener(new FaceDetectionFeedbackListener() {
    @Override
    public void onFaceDetection(MobbScanDetectionFeedbackResult result, MobbScanFaceDetectionFeedbackData faceFeedback) {
        // Your face detection feedback listener code here...
    }
});
MobbScanAPI.getInstance().detectFace(MobbScanFaceValidationMode.DEFAULT, new FaceAcquisitorListener() {
    @Override
    public void onFaceAcquired(MobbScanFaceAcquisitorResult result, MobbScanFaceAcquisitorData data, MobbScanAPIError error) {
        // Your face acquisitor listener code here...
    }
});

The FaceDetectionFeedbackListener returns 2 objects:

MobbScanDetectionFeedbackResult: represents the result of the detection performed.
MobbScanFaceDetectionFeedbackData: object that returns the position of the face on the FrameLayout, the progress of the process and, if obtained, a bitmap with the face.

Provide the SDK with a selfie image.

Once you have an already scanned document and a selfie from the user, MobbScan allows you tu use it in order to perform the face validation.

Kotlin

Java

val bitmapArray = ArrayList<Bitmap>()
bitmapArray.add(YOUR_SELFIE_BITMAP_HERE)
MobbScanAPI.getInstance().validateFace(scanId, bitmapArray) { result, resultData, error ->
    // Your validation listener code here...
}

ArrayList<Bitmap> bitmapArray = new ArrayList<>();
bitmapArray.add(YOUR_SELFIE_BITMAP_HERE);
MobbScanAPI.getInstance().validateFace(scanId, bitmapArray, new ValidationCheckListener() {
    @Override
    public void onValidationChecked(MobbScanValidationState result, MobbScanValidationResultData resultData, MobbScanAPIError error) {
        // Your validation listener code here...
    }
});

In order to retrieve the information from the ValidationCheckListener, you can check the Retrieving the results from the callback section of this document.

Retrieving the results from the callback.

Once you have performed one of the previous calls to MobbScanAPI, you will have to wait for the response on the ValidationCheckListener you provided on the method call. As seen previously, this callback is called returning 3 objects:

MobbScanValidationState: this object represents the possible results of the face validation process. This object can take the following values:

value	description
`VALID`	Operation finished sucessfully and the validation has been checked as correct
`NOT_VALID`	Operation finished sucesfully and the validation has been checked as not correct
`NOT_CHECKED`	The validation process has not been checked or it is not applicable to the document

MobbScanValidationResultData: object that contains the validation type performed, the score obtained from the process, and the bitmap with the selfie image. The information that this object contains is the following:

Attribute description

mobbscanValidationType The type of validation performed

score The score value of the validation process

faceBitmap The image with the face used in the validation process

All this attributes are accesible throught their own getter method.
MobbScanAPIError: object that specifies the error ocurred during the process. Contains the error code and extra information about the error.

Attribute	description
`mobbscanValidationType`	The type of validation performed
`score`	The score value of the validation process
`faceBitmap`	The image with the face used in the validation process

Video identity verification.

Since MobbScan Android SDK 2.16, it is possible to verify the identity of the user carrying out the onboarding process by means of a video recording in which they will be ask to perform a small gesture as a head movement. For further explanation about how our system works please consult here.

NOTE: From version 2.16.3 onwards, adding the androidx.constraintlayout:constraintlayout:2.0.0-beta4 dependency is required for this process.

In order to use this feature, there is a new MobbScanFaceValidationMode that corresponds to this new method. The code would end up being something like this:

Kotlin

Java

MobbScanAPI.getInstance().validateFace("YOUR_SCAN_ID", MobbScanFaceValidatoinMode.LIVENESS_VIDEO_HEAD_MOVEMENT, { faces ->
    // Get the selfie(s)... This will be invoked as soon as the SDK captures the user's selfie
}, { resut, resultData, error ->
    // Your validation listener code here...
})

MobbScanAPI.getInstance().validateFace("YOUR_SCAN_ID", MobbScanFaceValidationMode.LIVENESS_VIDEO_HEAD_MOVEMENT, new FaceAcquisitorListener() {
    @Override
    public void onFaceAcquired(ArrayList<Bitmap> faces) {
        // Get the selfie(s)... This will be invoked as soon as the SDK captures the user's selfie
    }
}, new ValidationCheckListener() {
    @Override
    public void onValidationChecked(MobbScanValidationState mobbScanValidationState, MobbScanValidationResultData mobbScanValidationResultData, MobbScanAPIError mobbScanAPIError) {
            // Your validation code here...
    }
});

Steps of the process

This process is divided into three steps:

The first step of the process consists in waiting for a face to appear in the detection area. At this point the FaceAcquisitorListener will be called.
Once a face has been detected, MobbScan will start recording a video of five seconds. While this video is being recorded, the user will be prompted with a message to move the head from side to side.
Having the video recorded, it is sent to MobbScan Server to verify user identity and detect possible spoofing attacks. The result will be obtained in the MobbScanValidationResultData of the ValidationCheckListener passed as a parameter.

Retrieving the results from the callback.

As we have previously showed, the result of the video evaluation will be returned in the ValidationCheckListener. This listener will return a MobbScanValidationResultData with some basic information about the validation.

Should the information given by this object is not enough for your application, all you have todo is to cast from MobbScanValidationResultData to MobbScanValidationVideoResultData. This class contains the following information:

Attribute	description
`livenessScore`	Score that indicates the confidence of the liveness detection.
`faceMatchingScore`	Score that indicates the confidence of the facial matching between the ID card headshot and the face image obtained from the video
`imageFaceNow`	Face of the user obtained from the video
`imageFaceDocument`	Image of the picture that is present on the document
`identityVerificationScore`	Overall indentity verification process confidence
`livenessValidation`	`MobbScanValidationState` that indicates if the liveness score is considered valid or not according to MobbScan standards
`faceMatchingValidation`	`MobbScanValidationState` that indicates if the face matching score is considered valid or not according to MobbScan standards
`identityVerificationValidation`	`MobbScanValidationState` that indicates if the overall confidence of the process is valid or not according to MobbScan standards

How to record a video

MobbScan allows you to obtain the video of the user performing the actions required. In order to do this, a new method has been included in MobbScanAPI.

Kotlin

Java

MobbScanAPI.getInstance().detectFaceVideoLiveness(MobbScanFaceValidationMode.LIVENESS_VIDEO_HEAD_MOVEMENT, { result, data, error -> 
    // Get the selfie(s)... This will be invoked as soon as the SDK captures the user's selfie
}, { result, data, error -> 
    // Your video acquisitor code here...      
})

MobbScanAPI.getInstance().detectFaceVideoLiveness(MobbScanFaceValidationMode.LIVENESS_VIDEO_HEAD_MOVEMENT, (result, data, error) -> {
    // Get the selfie(s)... This will be invoked as soon as the SDK captures the user's selfie
}, (result, data, error) -> {
    // Your video acquisitor code heer...      
});

The video will be returned in the MobbScanVideoAcquisitorData, this class has an attribute videoStream that contains the video recorded.

Passing an image to the verification

The previous methods require for a document to be scanned in online mode before performing the face validation process. If you don't need to perform an online scan or you don't need to perform a scan at all, but still want to perform the validation process, the next method is the correct one.

This method allows you to pass an image of the user you want to verify, and the MobbScan SDK will automatically open the interface to capture the video of the user and return the result in the provided ValidationCheckListener.

Kotlin

Java

MobbScanAPI.getInstance().setApiMode(MobbScanAPIMode.ONLINE);
MobbScanAPI.getInstance().validateFace(null, YOUR_IMAGE_OF_THE_USER, MobbScanFaceValidationMode.LIVENESS_VIDEO_HEAD_MOVEMENT,
    { result, data, error -> 
        // Your face acquisitor code here...
    }, { result, resultData, error -> 
        // Your validation check listener code here...
    })

MobbScanAPI.getInstance().setApiMode(MobbScanAPIMode.ONLINE);
MobbScanAPI.getInstance().validateFace(null, YOUR_IMAGE_OF_THE_USER, MobbScanFaceValidationMode.LIVENESS_VIDEO_HEAD_MOVEMENT,
    (result, data, error) -> {
        // Your face acquisitor code here...
    }, (result, resultData, error) -> {
        // Your validation check listener code here...
    });