Face detection, recognition, and tracking.

Required Header

#include <mv_face.h>

Related Features

This API is related with the following features:

http://tizen.org/feature/vision.face_recognition

It is recommended to design feature related codes in your application for reliability.
You can check if a device supports the related features for this API by using System Information, thereby controlling the procedure of your application.
To ensure your application is only running on the device with specific features, please define the features in your manifest file using the manifest editor in the SDK.
More details on featuring your application can be found from Feature Element.

Overview

Media Vision Face contains mv_face_detect() function to detect faces on mv_source_h, and mv_face_detected_cb() callback to process detected faces. Also it contains mv_face_recognize() function which performs face recognition on mv_source_h for recognition model mv_face_recognition_model_h handle. Results of the recognition will be passed to the mv_face_recognized_cb(). Results of the recognition consist of unique face label and confidence of the recognition model that face was recognized correctly. Unique face label is the integer identifier should be introduced to the model before starting recognition, when learning examples are added (see mv_face_recognition_model_add() function documentation for details about unique face labels).
Recognition model should be created with mv_face_recognition_model_create() and destroyed with mv_face_recognition_model_destroy(). Model can be cloned with mv_face_recognition_model_clone(), saved to the file with mv_face_recognition_model_save(), loaded with mv_face_recognition_model_load(). Model learning can be provided with mv_face_recognition_model_add(), and mv_face_recognition_model_learn() functions. These two methods has to be called in the direct order: first, labeled face examples should be added to the model using mv_face_recognition_model_add(). It is expected that images of the same face will be added specifying the same face label. When examples were added, model has to be learned based on the collected set of labeled face images. mv_face_recognition_model_learn() function will perform learning. If it is required to get the list of unique face labels learned by the model, mv_face_recognition_model_query_labels() function can be used.
Module contains function mv_face_track() which performs tracking on mv_source_h for mv_face_tracking_model_h and mv_face_tracked_cb() which process tracked face. Tracking model should be created with mv_face_tracking_model_create() and destroyed with mv_face_tracking_model_destroy(). Tracking model should be prepared with mv_face_tracking_model_prepare() before each session of tracking. Model can be cloned with mv_face_tracking_model_clone(), saved to the file with mv_face_tracking_model_save(), loaded with mv_face_tracking_model_load().
Module provides function for detecting eye-blink status - mv_face_eye_condition_recognize(), which provides detection on mv_source_h and face location - mv_rectangle_s. Callback mv_face_eye_condition_recognized_cb() handles result.
Also module contains mv_face_facial_expression_recognize() function which detects face expression on mv_source_h, face location is determined by mv_rectangle_s. Result is handled with mv_face_facial_expression_recognized_cb().

Functions
int	mv_face_detect (mv_source_h source, mv_engine_config_h engine_cfg, mv_face_detected_cb detected_cb, void *user_data)
	Performs face detection on the source for the engine_conf.
int	mv_face_recognize (mv_source_h source, mv_face_recognition_model_h recognition_model, mv_engine_config_h engine_cfg, mv_rectangle_s face_location, mv_face_recognized_cb recognized_cb, void user_data)
	Performs face recognition on the source image.
int	mv_face_track (mv_source_h source, mv_face_tracking_model_h tracking_model, mv_engine_config_h engine_cfg, mv_face_tracked_cb tracked_cb, bool do_learn, void *user_data)
	Performs face tracking on the source for the tracking_model.
int	mv_face_eye_condition_recognize (mv_source_h source, mv_engine_config_h engine_cfg, mv_rectangle_s face_location, mv_face_eye_condition_recognized_cb eye_condition_recognized_cb, void *user_data)
	Determines eye-blink condition for face_location on media source.
int	mv_face_facial_expression_recognize (mv_source_h source, mv_engine_config_h engine_cfg, mv_rectangle_s face_location, mv_face_facial_expression_recognized_cb expression_recognized_cb, void *user_data)
	Determines facial expression for face_location on media source.
int	mv_face_recognition_model_create (mv_face_recognition_model_h *recognition_model)
	Creates a face recognition model handle.
int	mv_face_recognition_model_destroy (mv_face_recognition_model_h recognition_model)
	Destroys the face recognition model handle and releases all its resources.
int	mv_face_recognition_model_clone (mv_face_recognition_model_h src, mv_face_recognition_model_h *dst)
	Creates a copy of existed recognition model handle and clones all its resources.
int	mv_face_recognition_model_save (const char *file_name, mv_face_recognition_model_h recognition_model)
	Saves recognition model to the file.
int	mv_face_recognition_model_load (const char file_name, mv_face_recognition_model_h recognition_model)
	Loads recognition model from file.
int	mv_face_recognition_model_add (const mv_source_h source, mv_face_recognition_model_h recognition_model, const mv_rectangle_s *example_location, int face_label)
	Adds face image example to be used for face recognition model learning with mv_face_recognition_model_learn().
int	mv_face_recognition_model_reset (mv_face_recognition_model_h recognition_model, int *face_label)
	Removes from recognition_model all collected with mv_face_recognition_model_add() function face examples labeled with face_label.
int	mv_face_recognition_model_learn (mv_engine_config_h engine_cfg, mv_face_recognition_model_h recognition_model)
	Learns face recognition model.
int	mv_face_recognition_model_query_labels (mv_face_recognition_model_h recognition_model, int *labels, unsigned int number_of_labels)
	Queries labels list and number of labels had been learned by the model.
int	mv_face_tracking_model_create (mv_face_tracking_model_h *tracking_model)
	Calls this function to create a face tracking model handle.
int	mv_face_tracking_model_destroy (mv_face_tracking_model_h tracking_model)
	Calls this function to destroy the face tracking model handle and release all its resources.
int	mv_face_tracking_model_prepare (mv_face_tracking_model_h tracking_model, mv_engine_config_h engine_cfg, mv_source_h source, mv_quadrangle_s *location)
	Calls this function to initialize tracking model by the location of the face to be tracked.
int	mv_face_tracking_model_clone (mv_face_tracking_model_h src, mv_face_tracking_model_h *dst)
	Calls this function to make a copy of existed tracking model handle and clone all its resources to the copy.
int	mv_face_tracking_model_save (const char *file_name, mv_face_tracking_model_h tracking_model)
	Calls this method to save tracking model to the file.
int	mv_face_tracking_model_load (const char file_name, mv_face_tracking_model_h tracking_model)
	Calls this method to load a tracking model from file.
Typedefs
typedef void(*	mv_face_detected_cb )(mv_source_h source, mv_engine_config_h engine_cfg, mv_rectangle_s faces_locations, int number_of_faces, void user_data)
	Called when faces are detected for the source.
typedef void(*	mv_face_recognized_cb )(mv_source_h source, mv_face_recognition_model_h recognition_model, mv_engine_config_h engine_cfg, mv_rectangle_s face_location, const int face_label, double confidence, void *user_data)
	Called each time when face is recognized by mv_face_recognize() function.
typedef void(*	mv_face_tracked_cb )(mv_source_h source, mv_face_tracking_model_h tracking_model, mv_engine_config_h engine_cfg, mv_quadrangle_s location, double confidence, void user_data)
	Called when face determined by tracking_model is tracked.
typedef void(*	mv_face_eye_condition_recognized_cb )(mv_source_h source, mv_engine_config_h engine_cfg, mv_rectangle_s face_location, mv_face_eye_condition_e eye_condition, void *user_data)
	Called when eye blink condition is recognized.
typedef void(*	mv_face_facial_expression_recognized_cb )(mv_source_h source, mv_engine_config_h engine_cfg, mv_rectangle_s face_location, mv_face_facial_expression_e facial_expression, void *user_data)
	Called when facial expression is recognized.
typedef void *	mv_face_recognition_model_h
	The handle to the model aggregating recognition face features.
typedef void *	mv_face_tracking_model_h
	The handle to the model aggregating tracking face features.
Defines
#define	MV_FACE_DETECTION_MODEL_FILE_PATH "MV_FACE_DETECTION_MODEL_FILE_PATH"
	Defines MV_FACE_DETECTION_MODEL_FILE_PATH to set face detection haarcascade xml file attribute of the engine configuration.
#define	MV_FACE_RECOGNITION_MODEL_TYPE "MV_FACE_RECOGNITION_MODEL_TYPE"
	Defines MV_FACE_RECOGNITION_MODEL_TYPE to set the method used for face recognition model learning attribute of the engine configuration.
#define	MV_FACE_DETECTION_ROI_X "MV_FACE_DETECTION_ROI_X"
	Define MV_FACE_DETECTION_ROI_X to set X coordinate of face detection roi (Region of Interest) as attribute of the engine configuration.
#define	MV_FACE_DETECTION_ROI_Y "MV_FACE_DETECTION_ROI_Y"
	Define MV_FACE_DETECTION_ROI_Y to set Y coordinate of face detection roi (Region of Interest) as attribute of the engine configuration.
#define	MV_FACE_DETECTION_ROI_WIDTH "MV_FACE_DETECTION_ROI_WIDTH"
	Define MV_FACE_DETECTION_ROI_WIDTH to set width of face detection roi (Region of Interest) as attribute of the engine configuration.
#define	MV_FACE_DETECTION_ROI_HEIGHT "MV_FACE_DETECTION_ROI_HEIGHT"
	Define MV_FACE_DETECTION_ROI_HEIGHT to set height of face detection roi (Region of Interest) as attribute of the engine configuration.
#define	MV_FACE_DETECTION_MIN_SIZE_WIDTH "MV_FACE_DETECTION_MIN_SIZE_WIDTH"
	Define MV_FACE_DETECTION_MIN_SIZE_WIDTH to set minimum width of face which will be detected as attribute of the engine configuration.
#define	MV_FACE_DETECTION_MIN_SIZE_HEIGHT "MV_FACE_DETECTION_MIN_SIZE_HEIGHT"
	Define MV_FACE_DETECTION_MIN_SIZE_HEIGHT to set minimum height of face which will be detected as attribute of the engine configuration.

Define Documentation

#define MV_FACE_DETECTION_MIN_SIZE_HEIGHT "MV_FACE_DETECTION_MIN_SIZE_HEIGHT"

Define MV_FACE_DETECTION_MIN_SIZE_HEIGHT to set minimum height of face which will be detected as attribute of the engine configuration.

Default value is -1 (all detected faces will be applied) can be changed to specify the minimum face height

Since :: 3.0

See also:: mv_engine_config_set_int_attribute(); mv_engine_config_get_int_attribute()

#define MV_FACE_DETECTION_MIN_SIZE_WIDTH "MV_FACE_DETECTION_MIN_SIZE_WIDTH"

Define MV_FACE_DETECTION_MIN_SIZE_WIDTH to set minimum width of face which will be detected as attribute of the engine configuration.

Default value is -1 (all detected faces will be applied) can be changed to specify the minimum face width

Since :: 3.0

See also:: mv_engine_config_set_int_attribute(); mv_engine_config_get_int_attribute()

#define MV_FACE_DETECTION_MODEL_FILE_PATH "MV_FACE_DETECTION_MODEL_FILE_PATH"

Defines MV_FACE_DETECTION_MODEL_FILE_PATH to set face detection haarcascade xml file attribute of the engine configuration.

Face detection haarcascade model can be changed to specify the path to the file

Since :: 3.0

See also:: mv_engine_config_set_string_attribute(); mv_engine_config_get_string_attribute()

#define MV_FACE_DETECTION_ROI_HEIGHT "MV_FACE_DETECTION_ROI_HEIGHT"

Define MV_FACE_DETECTION_ROI_HEIGHT to set height of face detection roi (Region of Interest) as attribute of the engine configuration.

Default value is -1 (the roi will be a full image) can be changed to specify the roi for face detection

Since :: 3.0

See also:: mv_engine_config_set_int_attribute(); mv_engine_config_get_int_attribute()

#define MV_FACE_DETECTION_ROI_WIDTH "MV_FACE_DETECTION_ROI_WIDTH"

Define MV_FACE_DETECTION_ROI_WIDTH to set width of face detection roi (Region of Interest) as attribute of the engine configuration.

Default value is -1 (the roi will be a full image) can be changed to specify the roi for face detection

Since :: 3.0

See also:: mv_engine_config_set_int_attribute(); mv_engine_config_get_int_attribute()

#define MV_FACE_DETECTION_ROI_X "MV_FACE_DETECTION_ROI_X"

Define MV_FACE_DETECTION_ROI_X to set X coordinate of face detection roi (Region of Interest) as attribute of the engine configuration.

Default value is -1 (the roi will be a full image) can be changed to specify the roi for face detection

Since :: 3.0

See also:: mv_engine_config_set_int_attribute(); mv_engine_config_get_int_attribute()

#define MV_FACE_DETECTION_ROI_Y "MV_FACE_DETECTION_ROI_Y"

Define MV_FACE_DETECTION_ROI_Y to set Y coordinate of face detection roi (Region of Interest) as attribute of the engine configuration.

Default value is -1 (the roi will be a full image) can be changed to specify the roi for face detection

Since :: 3.0

See also:: mv_engine_config_set_int_attribute(); mv_engine_config_get_int_attribute()

#define MV_FACE_RECOGNITION_MODEL_TYPE "MV_FACE_RECOGNITION_MODEL_TYPE"

Defines MV_FACE_RECOGNITION_MODEL_TYPE to set the method used for face recognition model learning attribute of the engine configuration.

Switches between three types of methods used for face recognition model learning. Possible values of the attribute are: 1 - Eigenfaces, 2 - Fisherfaces, 3 - Local Binary Patterns Histograms (LBPH) Default type is LBPH

Since :: 3.0

See also:: mv_engine_config_set_int_attribute(); mv_engine_config_get_int_attribute()

Typedef Documentation

typedef void(* mv_face_detected_cb)(mv_source_h source, mv_engine_config_h engine_cfg, mv_rectangle_s *faces_locations, int number_of_faces, void *user_data)

Called when faces are detected for the source.

This type callback can be invoked each time when mv_face_detect() is called to process the results of face detecting.

Since :: 3.0

Remarks:: If no face is detected then the callback will be invoked, but faces_locations array will be NULL, and number_of_faces will be equal to 0.

Parameters:

[in]	source	The handle to the source of the media where faces were detected
[in]	engine_cfg	The handle to the configuration of engine was used for face detecting, or NULL if default settings were applied
[in]	faces_locations	Rectangular locations of detected faces
[in]	number_of_faces	Number of detected faces
[in]	user_data	The user data passed from callback invoking code

Precondition:: Call mv_face_detect() function to perform detection of the face for the face image and invoke this callback as a result

See also:: mv_face_detect()

typedef void(* mv_face_eye_condition_recognized_cb)(mv_source_h source, mv_engine_config_h engine_cfg, mv_rectangle_s face_location, mv_face_eye_condition_e eye_condition, void *user_data)

Called when eye blink condition is recognized.

This type callback can be invoked each time when mv_face_eye_condition_recognize() is called for face_location to recognize eye-blink condition for the face at the source.
Usage example for this callback can be found in mv_face_eye_condition_recognize() documentation.

Since :: 3.0

Parameters:

[in]	source	The handle to the source of the media for which eye-blink condition was recognized
[in]	engine_cfg	The handle to the configuration of engine was used for eye-blink condition recognition, or NULL if default settings were applied.
[in]	face_location	The location bounding the face at the source
[in]	eye_condition	The type of eye-blink condition recognized for face bounded by face_location
[in]	user_data	The user data passed from callback invoking code

Precondition:: Call mv_face_eye_condition_recognize() function to perform eye-blink condition recognition and invoke this callback as a result

See also:: mv_face_eye_condition_recognize()

typedef void(* mv_face_facial_expression_recognized_cb)(mv_source_h source, mv_engine_config_h engine_cfg, mv_rectangle_s face_location, mv_face_facial_expression_e facial_expression, void *user_data)

Called when facial expression is recognized.

This type callback can be invoked each time when mv_face_facial_expression_recognize() is called for face_location to recognize facial expression for the face at the source.
Usage example for this callback can be found in mv_face_facial_expression_recognize() documentation.

Since :: 3.0

Parameters:

[in]	source	The handle to the source of the media for which facial expression was recognized
[in]	engine_cfg	The handle to the configuration of engine was used for expression recognition
[in]	face_location	The location bounding the face at the source
[in]	facial_expression	The type of facial expression recognized for face bounded by face_location
[in]	user_data	The user data passed from callback invoking code

Precondition:: Create a source handle by calling mv_create_source(); Create a face engine configuration handle by calling mv_create_engine_config()

See also:: mv_face_facial_expression_recognize()

typedef void* mv_face_recognition_model_h

The handle to the model aggregating recognition face features.

This handle can be used for faces recognizing with mv_face_recognize() function. Handle has to be created by mv_face_recognition_model_create() and destroyed by mv_face_recognition_model_destroy() functions. To use recognition models effectively learning process has to be performed before recognition. In other words, appropriate set of the face image examples has to be collected with mv_face_recognition_model_add() function before mv_face_recognition_model_learn() function call, then recognition can be performed with mv_face_recognize().

Since :: 3.0

See also:: mv_face_recognition_model_create(); mv_face_recognition_model_destroy(); mv_face_recognition_model_learn()

typedef void(* mv_face_recognized_cb)(mv_source_h source, mv_face_recognition_model_h recognition_model, mv_engine_config_h engine_cfg, mv_rectangle_s *face_location, const int *face_label, double confidence, void *user_data)

Called each time when face is recognized by mv_face_recognize() function.

Since :: 3.0

Parameters:

[in]	source	The handle to the image source for which face has been recognized/not recognized
[in]	recognition_model	The handle to the recognition model has been used for recognition
[in]	engine_cfg	The handle to the configuration of engine was used for recognition, or NULL if default settings were applied
[in]	face_location	The pointer to the location of the face recognized on source. If face wasn't recognized, then pointer is NULL
[in]	face_label	The label that identifies face which was recognized in the source. NULL if recognition was performed, but no faces were recognized in the source
[in]	confidence	The confidence of the recognition_model that face has been recognized correctly (value from 0.0 to 1.0). No faces were recognized if confidence was 0.0. When model has been learned on large amount of examples, threshold for this value can be high (0.85-0.95). If model was learned for small amount of examples, then threshold can be reduced (0.5-0.85)
[in]	user_data	The user data passed from callback invoking code

Precondition:: Call mv_face_recognize() function to perform recognition of the face for the face image and invoke this callback as a result

See also:: mv_face_recognize()

typedef void(* mv_face_tracked_cb)(mv_source_h source, mv_face_tracking_model_h tracking_model, mv_engine_config_h engine_cfg, mv_quadrangle_s *location, double confidence, void *user_data)

Called when face determined by tracking_model is tracked.

This type callback can be invoked each time when mv_face_track() is called to process the results of face tracking.

Since :: 3.0

Parameters:

[in]	source	The handle to the video frame or image from sequence for which face was tracked
[in]	tracking_model	The handle to the model that was used for tracking
[in]	engine_cfg	The handle to the configuration of engine was used for tracking, or NULL if default settings were applied.
[in]	location	The pointer to the quadrangle-shaped location which determines new position of the tracked face on the source. If NULL, then face was lost by tracking algorithm during last iteration
[in]	confidence	The confidence of the tracking_model that new location of the face was determined correctly (value from 0.0 to 1.0). If no location was determined during last track iteration, then value is 0.0
[in]	user_data	The user data passed from callback invoking code

Precondition:: Call mv_face_track() function to perform track iteration for the video frame or the image from sequence and invoke this callback as a result

See also:: mv_face_track()

typedef void* mv_face_tracking_model_h

The handle to the model aggregating tracking face features.

This model can be used for face tracking with mv_face_track() function. Handle has to be created by mv_face_tracking_model_create() and destroyed by mv_face_tracking_model_destroy() function. Tracking model can be improved during tracking task and allows to track face more accurately. So, you can create several tracking models independently by default, then apply tracking task for each of them. After some tracking these models will be different and each of them will be more efficient for tracking of face for which was created.

Since :: 3.0

Remarks:: Create each tracking model for single face.

See also:: mv_face_tracking_model_create(); mv_face_tracking_model_destroy()

Enumeration Type Documentation

enum mv_face_eye_condition_e

Enumeration for eyes state type.

Since :: 3.0

See also:: mv_face_eye_condition_recognize()

Enumerator:

MV_FACE_EYES_OPEN	Eyes are open
MV_FACE_EYES_CLOSED	Eyes are closed
MV_FACE_EYES_NOT_FOUND	The eyes condition wasn't determined

enum mv_face_facial_expression_e

Enumeration for expression types can be determined for faces.

Since :: 3.0

See also:: mv_face_facial_expression_recognize()

Enumerator:

MV_FACE_UNKNOWN	Unknown face expression
MV_FACE_NEUTRAL	Face expression is neutral
MV_FACE_SMILE	Face expression is smiling
MV_FACE_SADNESS	Face expression is sadness
MV_FACE_SURPRISE	Face expression is surprise
MV_FACE_ANGER	Face expression is anger
MV_FACE_FEAR	Face expression is fear
MV_FACE_DISGUST	Face expression is disgust

Function Documentation

int mv_face_detect	(	mv_source_h	source,
		mv_engine_config_h	engine_cfg,
		mv_face_detected_cb	detected_cb,
		void *	user_data
	)

Performs face detection on the source for the engine_conf.

Use this function to launch face detection algorithm configured by engine_conf configuration. Each time when mv_face_detect() is called, detected_cb will receive a set of the detected faces at the media source.

Since :: 3.0

Parameters:

[in]	source	The handle to the source of the media where faces will be detected
[in]	engine_cfg	The handle to the configuration of engine will be used for detecting. If NULL, then default settings will be used.
[in]	detected_cb	The callback which will be called for all face locations detected on media source. This callback will receive detecting results
[in]	user_data	The user data passed from the code where mv_face_detect() is invoked. This data will be accessible from detected_cb callback.

Returns:: 0 on success, otherwise a negative error value

Return values:

MEDIA_VISION_ERROR_NONE	Successful
MEDIA_VISION_ERROR_INVALID_PARAMETER	Invalid parameter
MEDIA_VISION_ERROR_INVALID_OPERATION	Invalid operation
MEDIA_VISION_ERROR_NOT_SUPPORTED_FORMAT	Source colorspace isn't supported
MEDIA_VISION_ERROR_NOT_SUPPORTED	Not supported

Precondition:: Create a source handle by calling mv_create_source()

Postcondition:: detected_cb will be called to process detection results

See also:: mv_face_detected_cb()

int mv_face_eye_condition_recognize	(	mv_source_h	source,
		mv_engine_config_h	engine_cfg,
		mv_rectangle_s	face_location,
		mv_face_eye_condition_recognized_cb	eye_condition_recognized_cb,
		void *	user_data
	)

Determines eye-blink condition for face_location on media source.

Use this function to recognize eye-blink condition for the face bounded by face_location at source.

Since :: 3.0

Parameters:

[in]	source	The handle to the source of the media to recognize eye-blink condition for
[in]	engine_cfg	The handle to the configuration of engine will be used for eye-blink condition recognition. If NULL, the default configuration will be used.
[in]	face_location	The location bounding the face at the source
[in]	eye_condition_recognized_cb	The callback for processing result of eye-blink condition recognition
[in]	user_data	The user data passed from the code where mv_face_eye_condition_recognize() is invoked. This data will be accessible from eye_condition_recognized_cb callback.

Returns:: 0 on success, otherwise a negative error value

Return values:

MEDIA_VISION_ERROR_NONE	Successful
MEDIA_VISION_ERROR_INVALID_PARAMETER	Invalid parameter
MEDIA_VISION_ERROR_NOT_SUPPORTED_FORMAT	Source colorspace isn't supported
MEDIA_VISION_ERROR_NOT_SUPPORTED	Not supported

Precondition:: Create a source handle by calling mv_create_source()

See also:: mv_face_eye_condition_recognized_cb

int mv_face_facial_expression_recognize	(	mv_source_h	source,
		mv_engine_config_h	engine_cfg,
		mv_rectangle_s	face_location,
		mv_face_facial_expression_recognized_cb	expression_recognized_cb,
		void *	user_data
	)

Determines facial expression for face_location on media source.

Use this function to determine facial expression for the face bounded by face_location at source.

Since :: 3.0

Parameters:

[in]	source	The handle to the source of the media to recognize facial expression for
[in]	engine_cfg	The handle to the configuration of engine will be used for expression recognition
[in]	face_location	The location bounding the face at the source
[in]	expression_recognized_cb	The callback for processing result of facial expression determining
[in]	user_data	The user data passed from the code where mv_face_facial_expression_recognize() is invoked. This data will be accessible from expression_recognized_cb callback.

Returns:: 0 on success, otherwise a negative error value

Return values:

MEDIA_VISION_ERROR_NONE	Successful
MEDIA_VISION_ERROR_INVALID_PARAMETER	Invalid parameter
MEDIA_VISION_ERROR_NOT_SUPPORTED_FORMAT	Source colorspace isn't supported
MEDIA_VISION_ERROR_NOT_SUPPORTED	Not supported

Precondition:: Create a source handle by calling mv_create_source(); Create a face engine configuration handle by calling mv_create_engine_config()

See also:: mv_face_facial_expression_recognized_cb()

int mv_face_recognition_model_add	(	const mv_source_h	source,
		mv_face_recognition_model_h	recognition_model,
		const mv_rectangle_s *	example_location,
		int	face_label
	)

Adds face image example to be used for face recognition model learning with mv_face_recognition_model_learn().

Since :: 3.0

Remarks:: It is possible to destroy source after calling this method. Source isn't used for learning directly.; Face image example_location location can be determined using mv_face_detect() function.

Parameters:

[in]	source	The handle to source that contains face image
[in]	recognition_model	The handle to the recognition model which could be learned based on example
[in]	example_location	The pointer to the rectangular location of the face image at the source image. If NULL, then full image will be analyzed as the face image
[in]	face_label	The label that identifies face for which example is adding. Specify the same labels for the face images of a single person when calling this method. Has to be unique for each face

Returns:: 0 on success, otherwise a negative error value

Return values:

MEDIA_VISION_ERROR_NONE	Successful
MEDIA_VISION_ERROR_INVALID_PARAMETER	Invalid parameter
MEDIA_VISION_ERROR_NOT_SUPPORTED_FORMAT	Source colorspace isn't supported
MEDIA_VISION_ERROR_NOT_SUPPORTED	Not supported

Precondition:: Create a face recognition handle by calling mv_face_recognition_model_create() function

Postcondition:: When appropriate amount of face image examples is added to the recognition_model, this model has to be learned by mv_face_recognition_model_learn() function call. Only after learning of the model it can be used for face recognition with mv_face_recognize() function

See also:: mv_face_recognition_model_reset(); mv_face_recognition_model_learn()

int mv_face_recognition_model_clone	(	mv_face_recognition_model_h	src,
		mv_face_recognition_model_h *	dst
	)

Creates a copy of existed recognition model handle and clones all its resources.

Since :: 3.0

Remarks:: Cloning perform not only handle copy, but also copies all internal resources of the model. dst must be released using mv_face_recognition_model_destroy().

Parameters:

[in]	src	The handle to the recognition model to be copied
[out]	dst	The handle to the copy of existed recognition model specified as src

Returns:: 0 on success, otherwise a negative error value

Return values:

MEDIA_VISION_ERROR_NONE	Successful
MEDIA_VISION_ERROR_INVALID_PARAMETER	Invalid parameter
MEDIA_VISION_ERROR_OUT_OF_MEMORY	Out of memory
MEDIA_VISION_ERROR_NOT_SUPPORTED	Not supported

Precondition:: Create face recognition handles by calling mv_face_recognition_model_create()

See also:: mv_face_recognition_model_create()

int mv_face_recognition_model_create ( mv_face_recognition_model_h * recognition_model )

Creates a face recognition model handle.

Use this function to create default face recognition model. Creating process is defined by concrete face engine library. After creation recognition model has to be learned with mv_face_recognition_model_learn() function to provide appropriate results of face recognition functionality. Or learned earlier model can be loaded by mv_face_recognition_model_load() function.

Since :: 3.0

Remarks:: It can cause incompatibility issues when saved models (see mv_face_recognition_model_save(), mv_face_recognition_model_load() functions documentation) are used in applications for different platforms which use different computer vision libraries underlying this API.; You must release recognition_model by using mv_face_recognition_model_destroy() function.

Parameters:

[out] recognition_model The handle to the recognition model to be created

Returns:: 0 on success, otherwise a negative error value

Return values:

MEDIA_VISION_ERROR_NONE	Successful
MEDIA_VISION_ERROR_INVALID_PARAMETER	Invalid parameter
MEDIA_VISION_ERROR_OUT_OF_MEMORY	Out of memory
MEDIA_VISION_ERROR_NOT_SUPPORTED	Not supported

Postcondition:: Model can be loaded from the file after creation. Use mv_face_recognition_model_load() function to load it from file; Release recognition_model by using mv_face_recognition_model_destroy() function when it is not needed anymore

See also:: mv_face_recognition_model_destroy()

int mv_face_recognition_model_destroy ( mv_face_recognition_model_h recognition_model )

Destroys the face recognition model handle and releases all its resources.

Since :: 3.0

Parameters:

[in] recognition_model The handle to the face recognition model to be destroyed

Returns:: 0 on success, otherwise a negative error value

Return values:

MEDIA_VISION_ERROR_NONE	Successful
MEDIA_VISION_ERROR_INVALID_PARAMETER	Invalid parameter
MEDIA_VISION_ERROR_NOT_SUPPORTED	Not supported

Precondition:: Create recognition model by using mv_face_recognition_model_create()

See also:: mv_face_recognition_model_create()

int mv_face_recognition_model_learn	(	mv_engine_config_h	engine_cfg,
		mv_face_recognition_model_h	recognition_model
	)

Learns face recognition model.

Before you start learning process, face recognition models has to be filled with training data - face image examples. These examples has to be provided by mv_face_recognition_model_add() function. Usually, recognition accuracy is increased when number of not identical examples is large. But it depends on the used learning algorithm.

Since :: 3.0

Remarks:: Common flow is to collect face examples as much as possible, add them to the recognition model with mv_face_recognition_model_add(), then call mv_face_recognition_model_learn() for this recognition model to learn it (or update the model if updating is supported by the used algorithm).; Selection of the learning algorithm can be performed by setting corresponding attributes for the engine_cfg. You can check supported by engine_cfg attributes using mv_engine_config_foreach_supported_attribute() function call. By default, Local Binary Patterns Histograms (LBPH) based recognition algorithm will be used.

Parameters:

[in]	engine_cfg	The handle to the configuration of engine will be used for learning of the recognition models. If NULL, then default settings will be used
[in,out]	recognition_model	The model which will be learned. After learning process these model may be changed, so mv_face_recognize() results may differ before and after method call respectively to the face examples collected for the recognition_model

Returns:: 0 on success, otherwise a negative error value

Return values:

MEDIA_VISION_ERROR_NONE	Successful
MEDIA_VISION_ERROR_INVALID_PARAMETER	Invalid parameter
MEDIA_VISION_ERROR_NO_DATA	No data
MEDIA_VISION_ERROR_NOT_SUPPORTED	Not supported

Precondition:: Create a face engine configuration handle by calling mv_create_engine_config() and set supported parameters if needed. Or just set engine_cfg as NULL to learn with default settings; Create a face recognition model handles by calling mv_face_recognition_model_create() function; Add face image examples to the recognition_model by calling mv_face_recognition_model_add() function

Postcondition:: If it is not planned to learn the model again, clear memory by mv_face_recognition_model_reset() function; When model has been learned, it can be used for face recognition with mv_face_recognize() function

See also:: mv_face_recognition_model_add(); mv_face_recognition_model_reset(); mv_face_recognize()

int mv_face_recognition_model_load	(	const char *	file_name,
		mv_face_recognition_model_h *	recognition_model
	)

Loads recognition model from file.

Since :: 3.0

Remarks:: This function doesn't modify the set of face image examples added with mv_face_recognition_model_add() function. Model will be loaded from file without loss of collected examples. If you want to free memory from examples, use mv_face_recognition_model_reset() function. It is recommended to clear the memory if learning algorithm doesn't support reinforcement learning.; recognition_model is loaded from the absolute path directory. Use app_get_data_path for the private app storage path. recognition_model must be destroyed using mv_face_recognition_model_destroy().

Parameters:

[in]	file_name	Name of path/file to load the model
[out]	recognition_model	The handle to the recognition model to be loaded from the file

Returns:: 0 on success, otherwise a negative error value

Return values:

MEDIA_VISION_ERROR_NONE	Successful
MEDIA_VISION_ERROR_INVALID_PARAMETER	Invalid parameter
MEDIA_VISION_ERROR_INVALID_PATH	Invalid path
MEDIA_VISION_ERROR_PERMISSION_DENIED	Not permitted
MEDIA_VISION_ERROR_OUT_OF_MEMORY	Out of memory
MEDIA_VISION_ERROR_NOT_SUPPORTED_FORMAT	Not supported model format
MEDIA_VISION_ERROR_NOT_SUPPORTED	Not supported

Precondition:: Recognition model can be preliminary saved with mv_face_recognition_model_save() function

See also:: mv_face_recognition_model_save(); mv_face_recognition_model_destroy(); app_get_data_path()

int mv_face_recognition_model_query_labels	(	mv_face_recognition_model_h	recognition_model,
		int **	labels,
		unsigned int *	number_of_labels
	)

Queries labels list and number of labels had been learned by the model.

Since :: 3.0

Remarks:: labels array has to be released using free().

Parameters:

[in]	recognition_model	The handle to the recognition model for which set of the learned labels will be queried
[out]	labels	The array which will be filled with labels had been learned by the model
[out]	number_of_labels	The number of labels in labels array

Returns:: 0 on success, otherwise a negative error value

Return values:

MEDIA_VISION_ERROR_NONE	Successful
MEDIA_VISION_ERROR_INVALID_PARAMETER	Invalid parameter
MEDIA_VISION_ERROR_NOT_SUPPORTED	Not supported

Precondition:: Add face image examples with labels to the recognition_model by calling the mv_face_recognition_model_add() function; Learn the recognition_model by labeled examples using mv_face_recognition_model_learn() function

Postcondition:: labels array has to be freed in the function invoking code

See also:: mv_face_recognition_model_add(); mv_face_recognition_model_reset(); mv_face_recognition_model_learn()

int mv_face_recognition_model_reset	(	mv_face_recognition_model_h	recognition_model,
		int *	face_label
	)

Removes from recognition_model all collected with mv_face_recognition_model_add() function face examples labeled with face_label.

Since :: 3.0

Remarks:: Be aware that if this function is called before mv_face_recognition_model_learn() function call, all or part of the required for learning data will be lost. It means that face image examples determined by the face_label label will be removed from the model and not taken into account when mv_face_recognition_model_learn() will be called next time.; Call of this function will free all the memory has been allocated during previous mv_face_recognition_model_add() calls for the corresponding face_label label.

Parameters:

[in]	recognition_model	The handle to the recognition model for which face image examples will be reset.
[in]	face_label	The label that identifies face for which examples will be removed from the recognition_model. If NULL, then all known by recognition_model face image examples will be removed

Returns:: 0 on success, otherwise a negative error value

Return values:

MEDIA_VISION_ERROR_NONE	Successful
MEDIA_VISION_ERROR_INVALID_PARAMETER	Invalid parameter
MEDIA_VISION_ERROR_KEY_NOT_AVAILABLE	Key not available
MEDIA_VISION_ERROR_NOT_SUPPORTED	Not supported

See also:: mv_face_recognition_model_add(); mv_face_recognition_model_learn()

int mv_face_recognition_model_save	(	const char *	file_name,
		mv_face_recognition_model_h	recognition_model
	)

Saves recognition model to the file.

Since :: 3.0

Remarks:: This function doesn't save face image examples (image itself) added by mv_face_recognition_model_add() function. This examples can be removed by mv_face_recognition_model_reset() function if it is needed to clear the memory.; recognition_model is saved to the absolute path directory. Use app_get_data_path for the private app storage path. After model is saved to the file, it can be loaded from this file by mv_face_recognition_model_load() function.

Parameters:

[in]	file_name	Name of the path/file to save the model
[in]	recognition_model	The handle to the recognition model to be saved to the file

Returns:: 0 on success, otherwise a negative error value

Return values:

MEDIA_VISION_ERROR_NONE	Successful
MEDIA_VISION_ERROR_INVALID_PARAMETER	Invalid parameter
MEDIA_VISION_ERROR_INVALID_OPERATION	Invalid operation
MEDIA_VISION_ERROR_INVALID_PATH	Invalid path
MEDIA_VISION_ERROR_PERMISSION_DENIED	Not permitted
MEDIA_VISION_ERROR_NOT_SUPPORTED_FORMAT	Not supported model format
MEDIA_VISION_ERROR_NOT_SUPPORTED	Not supported

Precondition:: Create a face recognition handle by calling mv_face_recognition_model_create() function

Postcondition:: Saved model can be loaded later by calling mv_face_recognition_model_load() function

See also:: mv_face_recognition_model_load(); mv_face_recognition_model_create(); app_get_data_path()

int mv_face_recognize	(	mv_source_h	source,
		mv_face_recognition_model_h	recognition_model,
		mv_engine_config_h	engine_cfg,
		mv_rectangle_s *	face_location,
		mv_face_recognized_cb	recognized_cb,
		void *	user_data
	)

Performs face recognition on the source image.

Use this function to launch face recognition algorithm configured by engine_conf configuration using recognition_model recognition model. Each time when mv_face_recognize() is called, recognized_cb will receive recognition results:

Location in the source of the face has been recognized;
Label of the face has been recognized;
Confidence of the recognition_model that face has been recognized correctly (value from 0.0 to 1.0).

Since :: 3.0

Remarks:: Using of untrained or weakly trained recognition models will cause not accurate results even if resulting confidence will be high. Use mv_face_recognition_model_learn() function before mv_face_recognize() call. Best results can be achieved when big set of face image examples were added by mv_face_recognition_model_add() before mv_face_recognition_model_learn() call.

Parameters:

[in]	source	The handle to the source of the media to recognize face(s) for
[in]	recognition_model	The handle to the model will be used for recognition
[in]	engine_cfg	The handle to the configuration of engine will be used for recognition. If NULL, then default settings will be used
[in]	face_location	Rectangular box bounding face image on the source. If NULL, then full source will be analyzed
[in]	recognized_cb	The callback which will be called for the face recognition results on the source.
[in]	user_data	The user data passed from the code where mv_face_recognize() is invoked. This data will be accessible from recognized_cb callback

Returns:: 0 on success, otherwise a negative error value

Return values:

MEDIA_VISION_ERROR_NONE	Successful
MEDIA_VISION_ERROR_INVALID_PARAMETER	Invalid parameter
MEDIA_VISION_ERROR_INVALID_OPERATION	Invalid operation
MEDIA_VISION_ERROR_NOT_SUPPORTED_FORMAT	Source colorspace isn't supported
MEDIA_VISION_ERROR_NOT_SUPPORTED	Not supported

Precondition:: Create a source handle by calling mv_create_source(); Create a face recognition model handle by calling mv_face_recognition_model_create()

Postcondition:: recognized_cb will be called to process recognition results

See also:: mv_face_recognized_cb()

int mv_face_track	(	mv_source_h	source,
		mv_face_tracking_model_h	tracking_model,
		mv_engine_config_h	engine_cfg,
		mv_face_tracked_cb	tracked_cb,
		bool	do_learn,
		void *	user_data
	)

Performs face tracking on the source for the tracking_model.

Use this function to launch face tracking algorithm configured by engine_conf configuration using tracking_model tracking model. Each time when this function is called, tracked_cb will receive updated tracking_model, new location determined for the tracked face and model confidence that location is determined correctly.

Since :: 3.0

Remarks:: To allow correct tracking tracking_model has to be already used in previous tracking process(es) or prepared with mv_face_tracking_model_prepare(). Preparation requires specifying the face location for the source on which tracking was started. I.e. mv_face_tracking_model_prepare() function has to be called at least once before this method call.

Parameters:

[in]	source	The handle to the source of the media to recognize face for
[in]	tracking_model	The handle to the model will be used for tracking
[in]	engine_cfg	The handle to the configuration of engine will be used for tracking. If NULL, the default configuration will be used.
[in]	tracked_cb	The callback which will be called for tracking event on the source where face would be tracked. This callback will receive tracking results
[in]	do_learn	The model learning flag. If it is set `true` then model will try to learn (if it supports learning feature), otherwise model will be not learned during the invoking tracking iteration. Learning process improves tracking correctness, but can decrease tracking performance
[in]	user_data	The user data passed from the code where mv_face_track() is invoked. This data will be accessible from tracked_cb callback

Returns:: 0 on success, otherwise a negative error value

Return values:

MEDIA_VISION_ERROR_NONE	Successful
MEDIA_VISION_ERROR_INVALID_PARAMETER	Invalid parameter
MEDIA_VISION_ERROR_INVALID_OPERATION	Invalid operation
MEDIA_VISION_ERROR_NOT_SUPPORTED_FORMAT	Source colorspace isn't supported
MEDIA_VISION_ERROR_NOT_SUPPORTED	Not supported

Precondition:: Create a source handle by calling mv_create_source(); Create a face tracking model handle by calling mv_face_tracking_model_create()

Postcondition:: tracked_cb will be called to process tracking results

See also:: mv_face_tracked_cb()

int mv_face_tracking_model_clone	(	mv_face_tracking_model_h	src,
		mv_face_tracking_model_h *	dst
	)

Calls this function to make a copy of existed tracking model handle and clone all its resources to the copy.

Since :: 3.0

Remarks:: Cloning performs not only handle copy, but also copies all internal resources of the model. dst must be released using mv_face_tracking_model_destroy().

Parameters:

[in]	src	The handle to the tracking model to be copied
[out]	dst	The handle to the copy of existed tracking model specified as src

Returns:: 0 on success, otherwise a negative error value

Return values:

MEDIA_VISION_ERROR_NONE	Successful
MEDIA_VISION_ERROR_INVALID_PARAMETER	Invalid parameter
MEDIA_VISION_ERROR_OUT_OF_MEMORY	Out of memory
MEDIA_VISION_ERROR_NOT_SUPPORTED	Not supported

Precondition:: Create face tracking src handle by calling mv_face_tracking_model_create()

See also:: mv_face_tracking_model_create()

int mv_face_tracking_model_create ( mv_face_tracking_model_h * tracking_model )

Calls this function to create a face tracking model handle.

Use this function to create default face tracking model handle. After creation this handle has to be initialized with mv_face_tracking_model_prepare() function to provide appropriate results of face tracking functionality. When handle is prepared, it is possible to use it for tracking on continuous sequence of the sources. Call mv_face_tracking_model_prepare() function each time before starting tracking on the new sequence. The exception is situation when the new sequence is continuation of the previous sequence for which model has been tracked.

Since :: 3.0

Parameters:

[out] tracking_model The pointer to the handle to the tracking model that will be created

Returns:: 0 on success, otherwise a negative error value

Return values:

MEDIA_VISION_ERROR_NONE	Successful
MEDIA_VISION_ERROR_INVALID_PARAMETER	Invalid parameter
MEDIA_VISION_ERROR_OUT_OF_MEMORY	Out of memory
MEDIA_VISION_ERROR_NOT_SUPPORTED	Not supported

Postcondition:: Model can be loaded from the file after creation. Use mv_face_tracking_model_load() function to load it from file; Use mv_face_tracking_model_prepare() function before tracking on the new video or continuous images sequence; Release tracking_model by using mv_face_tracking_model_destroy() function when it is not needed anymore

See also:: mv_face_tracking_model_destroy(); mv_face_tracking_model_prepare(); mv_face_tracking_model_load()

int mv_face_tracking_model_destroy ( mv_face_tracking_model_h tracking_model )

Calls this function to destroy the face tracking model handle and release all its resources.

Since :: 3.0

Parameters:

[in] tracking_model The handle to the face tracking model that will be destroyed

Returns:: 0 on success, otherwise a negative error value

Return values:

MEDIA_VISION_ERROR_NONE	Successful
MEDIA_VISION_ERROR_INVALID_PARAMETER	Invalid parameter
MEDIA_VISION_ERROR_NOT_SUPPORTED	Not supported

Precondition:: Create tracking model by using mv_face_tracking_model_create()

See also:: mv_face_tracking_model_create()

int mv_face_tracking_model_load	(	const char *	file_name,
		mv_face_tracking_model_h *	tracking_model
	)

Calls this method to load a tracking model from file.

Since :: 3.0

Remarks:: tracking_model is loaded from the absolute path directory. Use app_get_data_path for the private app storage path. tracking_model must be destroyed using mv_face_tracking_model_destroy().

Parameters:

[in]	file_name	Name of path/file to load the model
[out]	tracking_model	The handle to the tracking model to be loaded from file

Returns:: 0 on success, otherwise a negative error value

Return values:

MEDIA_VISION_ERROR_NONE	Successful
MEDIA_VISION_ERROR_INVALID_PARAMETER	Invalid parameter
MEDIA_VISION_ERROR_INVALID_OPERATION	Invalid operation
MEDIA_VISION_ERROR_INVALID_PATH	Invalid path
MEDIA_VISION_ERROR_PERMISSION_DENIED	Not permitted
MEDIA_VISION_ERROR_OUT_OF_MEMORY	Out of memory
MEDIA_VISION_ERROR_NOT_SUPPORTED_FORMAT	Not supported model format
MEDIA_VISION_ERROR_NOT_SUPPORTED	Not supported

Precondition:: Models has been saved by mv_face_tracking_model_save() function can be loaded with this function

Postcondition:: After model has been loaded and if further tracking will be performed on the video which is not continuation of the last tracking performed for the model, it is recommended to call mv_face_tracking_model_prepare() function

See also:: mv_face_tracking_model_save(); mv_face_tracking_model_destroy(); app_get_data_path()

int mv_face_tracking_model_prepare	(	mv_face_tracking_model_h	tracking_model,
		mv_engine_config_h	engine_cfg,
		mv_source_h	source,
		mv_quadrangle_s *	location
	)

Calls this function to initialize tracking model by the location of the face to be tracked.

This function is usually called once after tracking model is created and each time before tracking is started for the new sequence of sources which is not the direct continuation of the sequence for which tracking has been performed before. But it is allowed to call it between tracking sessions to allow Media Vision start to track more accurately.

Since :: 3.0

Parameters:

[in]	tracking_model	The handle to the tracking model that will be prepared for tracking on new video or image sequence
[in]	engine_cfg	The handle to the configuration of engine will be used for model preparing. If NULL, then default settings will be used.
[in]	source	The handle to the source where face location is specified. Usually it is the first frame of the video or the first image in the continuous image sequence planned to be used for tracking
[in]	location	The quadrangle-shaped location (actually, rectangle can be used) determining position of the face to be tracked on the source. If `NULL`, then tracking model will try to find previously tracked face by itself. Don't set NULL when called first time for the tracking model.

Returns:: 0 on success, otherwise a negative error value

Return values:

MEDIA_VISION_ERROR_NONE	Successful
MEDIA_VISION_ERROR_INVALID_PARAMETER	Invalid parameter
MEDIA_VISION_ERROR_NOT_SUPPORTED	Not supported

Precondition:: Create a face tracking model handle by calling mv_face_tracking_model_create() function; Create a source handle by calling mv_create_source() function

Postcondition:: When model is prepared, mv_face_track() function can be used to track on the video or continuous image sequence

See also:: mv_face_tracking_model_create(); mv_face_track()

int mv_face_tracking_model_save	(	const char *	file_name,
		mv_face_tracking_model_h	tracking_model
	)

Calls this method to save tracking model to the file.

Since :: 3.0

Remarks:: tracking_model is saved to the absolute path directory. Use app_get_data_path for the private app storage path. After model is saved to the file, it can be loaded from this file with mv_face_tracking_model_load() function.

Parameters:

[in]	file_name	Name of the path/file to save the model
[in]	tracking_model	The handle to the tracking model to be saved to the file

Returns:: 0 on success, otherwise a negative error value

Return values:

MEDIA_VISION_ERROR_NONE	Successful
MEDIA_VISION_ERROR_INVALID_PARAMETER	Invalid parameter
MEDIA_VISION_ERROR_INVALID_OPERATION	Invalid operation
MEDIA_VISION_ERROR_INVALID_PATH	Invalid path
MEDIA_VISION_ERROR_PERMISSION_DENIED	Not permitted
MEDIA_VISION_ERROR_NOT_SUPPORTED_FORMAT	Not supported model format
MEDIA_VISION_ERROR_NOT_SUPPORTED	Not supported

Precondition:: Create a face tracking handle by calling mv_face_tracking_model_create()

Postcondition:: Saved model can be loaded from file using mv_face_tracking_model_load() function

See also:: mv_face_tracking_model_load(); mv_face_tracking_model_create(); app_get_data_path()

Required Header

Related Features

Overview

Functions

Typedefs

Defines

Define Documentation

Typedef Documentation

Enumeration Type Documentation

Function Documentation