Pose landmark detection.

Required Header

#include <mv_pose_landmark.h>

Overview

Media Vision Pose Landmark Detection contains mv_pose_landmark_h handle to perform pose landmark detection. Pose landmark handle should be created with mv_pose_landmark_create() and destroyed with mv_pose_landmark_destroy(). mv_pose_landmark_h should be configured by calling mv_pose_landmark_configure(). After configuration, mv_pose_landmark_h should be prepared by calling mv_pose_landmark_prepare() which loads models and set required parameters. After preparation, mv_pose_landmark_inference() or mv_pose_landmark_inference_async() can be called to detect pose landmarks in images on mv_source_h. Use mv_pose_landmark_get_result_count() to get the number of results and mv_pose_landmark_get_position() to get the position of each landmark.

Related Features

This API is related with the following features:

http://tizen.org/feature/vision.inference.image
http://tizen.org/feature/vision.inference.face

It is recommended to design feature related codes in your application for reliability.
You can check if a device supports the related features for this API by using System Information, thereby controlling the procedure of your application.
To ensure your application is only running on the device with specific features, please define the features in your manifest file using the manifest editor in the SDK.
More details on featuring your application can be found from Feature Element.

Functions
int	mv_pose_landmark_create (mv_pose_landmark_h *handle)
	Creates pose landmark object handle.
int	mv_pose_landmark_destroy (mv_pose_landmark_h handle)
	Destroys pose landmark handle and releases all its resources.
int	mv_pose_landmark_configure (mv_pose_landmark_h handle)
	Configures the backend to the inference handle.
int	mv_pose_landmark_prepare (mv_pose_landmark_h handle)
	Prepares inference.
int	mv_pose_landmark_inference (mv_pose_landmark_h handle, mv_source_h source)
	Inferences with a given facial on the source.
int	mv_pose_landmark_inference_async (mv_pose_landmark_h handle, mv_source_h source)
	Performs asynchronously the pose landmark inference on the source.
int	mv_pose_landmark_get_result_count (mv_pose_landmark_h handle, unsigned long frame_number, unsigned int result_cnt)
	Gets the result count to objects.
int	mv_pose_landmark_get_position (mv_pose_landmark_h handle, unsigned int index, unsigned int pos_x, unsigned int pos_y)
	Gets the pose landmark position values to a given index.
Typedefs
typedef void *	mv_pose_landmark_h
	The pose landmark object handle.

Typedef Documentation

typedef void* mv_pose_landmark_h

The pose landmark object handle.

Since :: 9.0

Function Documentation

int mv_pose_landmark_configure ( mv_pose_landmark_h handle )

Configures the backend to the inference handle.

Since :: 9.0

Parameters:

[in] handle The handle to the inference

Returns:: 0 on success, otherwise a negative error value

Return values:

MEDIA_VISION_ERROR_NONE	Successful
MEDIA_VISION_ERROR_NOT_SUPPORTED	Not supported
MEDIA_VISION_ERROR_INVALID_PARAMETER	Invalid parameter
MEDIA_VISION_ERROR_OUT_OF_MEMORY	Out of memory

int mv_pose_landmark_create ( mv_pose_landmark_h * handle )

Creates pose landmark object handle.

Use this function to create an pose landmark object handle. After creation the handle has to be prepared with mv_pose_landmark_prepare() function to prepare a pose landmark object.

Since :: 9.0

Parameters:

[out] handle The handle to the pose landmark object to be created

Returns:: 0 on success, otherwise a negative error value

Return values:

MEDIA_VISION_ERROR_NONE	Successful
MEDIA_VISION_ERROR_INVALID_PARAMETER	Invalid parameter
MEDIA_VISION_ERROR_INTERNAL	Internal Error

Postcondition:: Release handle by using mv_pose_landmark_destroy() function when it is not needed anymore

 #include <mv_pose_landmark.h>
 ...
 mv_pose_landmark_h handle = NULL;
 mv_pose_landmark_create(&handle);
 ...
 mv_pose_landmark_destroy(handle);

See also:: mv_pose_landmark_destroy()

int mv_pose_landmark_destroy ( mv_pose_landmark_h handle )

Destroys pose landmark handle and releases all its resources.

Since :: 9.0

Parameters:

[in] handle The handle to the pose landmark object to be destroyed.

Returns:: 0 on success, otherwise a negative error value

Return values:

MEDIA_VISION_ERROR_NONE	Successful
MEDIA_VISION_ERROR_INVALID_PARAMETER	Invalid parameter

Precondition:: Create an pose landmark handle by using mv_pose_landmark_create()

See also:: mv_pose_landmark_create()

int mv_pose_landmark_get_position	(	mv_pose_landmark_h	handle,
		unsigned int	index,
		unsigned int *	pos_x,
		unsigned int *	pos_y
	)

Gets the pose landmark position values to a given index.

Since :: 9.0

Remarks:: pos_x and pos_y arrays are allocated internally by the framework and will remain valid until the handle is released. Please do not deallocate them directly, and if you want to use them after the handle is released, please copy them to user memory and use the copy.

This function operates differently depending on the inference request method.

After mv_facial_landmark_inference() calls, this function returns facial landmark positions immediately.
After mv_facial_landmark_inference_async() calls, this function can be blocked until the asynchronous inference request is completed or the timeout occurs if no result within 3 seconds.

Additionally, after calling the mv_facial_landmark_inference_async function, the function operates in asynchronous mode until the handle is released.

Parameters:

[in]	handle	The handle to the inference
[in]	index	A result index.
[out]	pos_x	An array containing x-coordinate values.
[out]	pos_y	An array containing y-coordinate values.

Returns:: 0 on success, otherwise a negative error value

Return values:

MEDIA_VISION_ERROR_NONE	Successful
MEDIA_VISION_ERROR_NOT_SUPPORTED	Not supported
MEDIA_VISION_ERROR_INVALID_PARAMETER	Invalid parameter

Precondition:: Create a source handle by calling mv_create_source(); Create an inference handle by calling mv_pose_landmark_create(); Prepare an inference by calling mv_pose_landmark_configure(); Prepare an inference by calling mv_pose_landmark_prepare(); Prepare an inference by calling mv_pose_landmark_inference(); Get result count by calling mv_pose_landmark_get_result_count()

int mv_pose_landmark_get_result_count	(	mv_pose_landmark_h	handle,
		unsigned long *	frame_number,
		unsigned int *	result_cnt
	)

Gets the result count to objects.

Since :: 9.0

Parameters:

[in]	handle	The handle to the inference
[out]	frame_number	A frame number inferenced.
[out]	result_cnt	A number of results.

Returns:: 0 on success, otherwise a negative error value

Return values:

MEDIA_VISION_ERROR_NONE	Successful
MEDIA_VISION_ERROR_NOT_SUPPORTED	Not supported
MEDIA_VISION_ERROR_INVALID_PARAMETER	Invalid parameter

Precondition:: Create a source handle by calling mv_create_source(); Create an inference handle by calling mv_pose_landmark_create(); Prepare an inference by calling mv_pose_landmark_configure(); Prepare an inference by calling mv_pose_landmark_prepare(); Request an inference by calling mv_pose_landmark_inference()

int mv_pose_landmark_inference	(	mv_pose_landmark_h	handle,
		mv_source_h	source
	)

Inferences with a given facial on the source.

Use this function to inference with a given source.

Since :: 9.0

Parameters:

[in]	handle	The handle to the pose landmark object.
[in]	source	The handle to the source of the media.

Returns:: 0 on success, otherwise a negative error value

Return values:

MEDIA_VISION_ERROR_NONE	Successful
MEDIA_VISION_ERROR_INVALID_PARAMETER	Invalid parameter
MEDIA_VISION_ERROR_NOT_SUPPORTED_FORMAT	Source colorspace isn't supported
MEDIA_VISION_ERROR_OUT_OF_MEMORY	Out of memory

Precondition:: Create a source handle by calling mv_create_source(); Create an pose landmark handle by calling mv_pose_landmark_create(); Prepare an inference by calling mv_pose_landmark_configure(); Prepare an pose landmark by calling mv_pose_landmark_prepare()

Inference Example

int main()
{
    int error_code = 0;
    mv_pose_landmark_h handle;
    mv_source_h mv_source = NULL;

    error_code = mv_create_source(&mv_source);
    if (error_code != MEDIA_VISION_ERROR_NONE)
        dlog_print(DLOG_ERROR, LOG_TAG, "error code = %d", error_code);

    load_image_to_source("/path/to/image", mv_source);

    error_code = mv_pose_landmark_create(&handle);
    if (error_code != MEDIA_VISION_ERROR_NONE)
        dlog_print(DLOG_ERROR, LOG_TAG, "error code = %d", error_code);

    error_code = mv_pose_landmark_configure(handle);
    if (error_code != MEDIA_VISION_ERROR_NONE)
        dlog_print(DLOG_ERROR, LOG_TAG, "error code = %d", error_code);

    error_code = mv_pose_landmark_prepare(handle);
    if (error_code != MEDIA_VISION_ERROR_NONE)
        dlog_print(DLOG_ERROR, LOG_TAG, "error code = %d", error_code);

    error_code = mv_pose_landmark_inference(handle, mv_source);
    if (error_code != MEDIA_VISION_ERROR_NONE)
        dlog_print(DLOG_ERROR, LOG_TAG, "error code = %d", error_code);

    unsigned long frame_number;
    unsigned int cnt;

    int error_code = mv_pose_landmark_get_result_count(handle, &frame_number, &cnt);
    if (error_code == MEDIA_VISION_ERROR_INVALID_OPERATION)
        break;

    for (unsigned int idx = 0; idx < cnt; ++idx) {
        unsigned int pos_x, pos_y;

        error_code = mv_pose_landmark_get_position(handle, idx, &pos_x, &pos_y);
        if (error_code != MEDIA_VISION_ERROR_NONE)
            dlog_print(DLOG_ERROR, LOG_TAG, "error code = %d", error_code);

        dlog_print(DLOG_INFO, LOG_TAG, "frame = %lu, idx = %u, x = %u, y = %u", frame_number, idx, pos_x, pos_y);
    }

    error_code = mv_destroy_source(mv_source);
    if (error_code != MEDIA_VISION_ERROR_NONE)
        dlog_print(DLOG_ERROR, LOG_TAG, "error code = %d", error_code);

    error_code = mv_pose_landmark_destroy(handle);
    if (error_code != MEDIA_VISION_ERROR_NONE)
        dlog_print(DLOG_ERROR, LOG_TAG, "error code = %d", error_code);
}

int mv_pose_landmark_inference_async	(	mv_pose_landmark_h	handle,
		mv_source_h	source
	)

Performs asynchronously the pose landmark inference on the source.

Since :: 9.0

Remarks:: This function operates asynchronously, so it returns immediately upon invocation. The inference results are inserted into the outgoing queue within the framework in the order of processing, and the results can be obtained through mv_pose_landmark_get_result_count() and mv_pose_landmark_get_position().

Parameters:

[in]	handle	The handle to the inference
[in]	source	The handle to the source of the media

Returns:: 0 on success, otherwise a negative error value

Return values:

MEDIA_VISION_ERROR_NONE	Successful
MEDIA_VISION_ERROR_NOT_SUPPORTED	Not supported
MEDIA_VISION_ERROR_INVALID_PARAMETER	Invalid parameter
MEDIA_VISION_ERROR_NOT_SUPPORTED_FORMAT	Source colorspace isn't supported

Precondition:: Create a source handle by calling mv_create_source(); Create an inference handle by calling mv_pose_landmark_create(); Prepare an inference by calling mv_pose_landmark_configure(); Prepare an inference by calling mv_pose_landmark_prepare()

Async Inference Example

void pose_landmark_callback(void *user_data)
{
    mv_pose_landmark_h handle = (mv_pose_landmark_h) user_data;
    bool is_loop_exit = false;

    while (!is_loop_exit) {
        unsigned long frame_number;
        unsigned int cnt;

        int error_code = mv_pose_landmark_get_result_count(handle, &frame_number, &cnt);
        if (error_code == MEDIA_VISION_ERROR_INVALID_OPERATION)
            break;

        for (unsigned int idx = 0; idx < cnt; ++idx) {
            unsigned int pos_x, pos_y;

            error_code = mv_pose_landmark_get_position(handle, idx, &pos_x, &pos_y);
            if (error_code != MEDIA_VISION_ERROR_NONE)
                dlog_print(DLOG_ERROR, LOG_TAG, "error code = %d", error_code);

            dlog_print(DLOG_INFO, LOG_TAG, "frame = %lu, idx = %u, x = %u, y = %u", frame_number, idx, pos_x, pos_y);

            if (frame_number > MAX_INFERENCE_ITERATION - 10)
                is_loop_exit = true;
        }
    }
}

int main()
{
    int error_code = 0;
    mv_pose_landmark_h handle;

    error_code = mv_pose_landmark_create(&handle);
    if (error_code != MEDIA_VISION_ERROR_NONE)
        dlog_print(DLOG_ERROR, LOG_TAG, "error code = %d", error_code);

    error_code = mv_pose_landmark_configure(handle);
    if (error_code != MEDIA_VISION_ERROR_NONE)
        dlog_print(DLOG_ERROR, LOG_TAG, "error code = %d", error_code);

    error_code = mv_pose_landmark_prepare(handle);
    if (error_code != MEDIA_VISION_ERROR_NONE)
        dlog_print(DLOG_ERROR, LOG_TAG, "error code = %d", error_code);

    pthread_t thread_id;

    for (unsigned int iter = 0; iter < MAX_INFERENCE_ITERATION; ++iter) {
        mv_source_h mv_source = NULL;

        error_code = mv_create_source(&mv_source);
        if (error_code != MEDIA_VISION_ERROR_NONE)
            dlog_print(DLOG_ERROR, LOG_TAG, "error code = %d", error_code);

        load_image_to_source("/path/to/image", mv_source);

        error_code = mv_pose_landmark_inference_async(handle, mv_source);
        if (error_code != MEDIA_VISION_ERROR_NONE)
            dlog_print(DLOG_ERROR, LOG_TAG, "error code = %d", error_code);

        if (iter == 0)
            pthread_create(&thread_id, NULL, pose_landmark_callback, handle);

        error_code = mv_destroy_source(mv_source);
        if (error_code != MEDIA_VISION_ERROR_NONE)
            dlog_print(DLOG_ERROR, LOG_TAG, "error code = %d", error_code);
    }

    /* This thread will be exited after inference is performed MAX_INFERENCE_ITERATION times. */
    pthread_join(thread_id, NULL);

    error_code = mv_pose_landmark_destroy(handle);
    if (error_code != MEDIA_VISION_ERROR_NONE)
        dlog_print(DLOG_ERROR, LOG_TAG, "error code = %d", error_code);
}

int mv_pose_landmark_prepare ( mv_pose_landmark_h handle )

Prepares inference.

Use this function to prepare inference based on the configured network.

Since :: 9.0

Parameters:

[in] handle The handle to the inference

Returns:: 0 on success, otherwise a negative error value

Return values:

MEDIA_VISION_ERROR_NONE	Successful
MEDIA_VISION_ERROR_INVALID_PARAMETER	Invalid parameter
MEDIA_VISION_ERROR_INVALID_DATA	Invalid model data
MEDIA_VISION_ERROR_OUT_OF_MEMORY	Out of memory

Required Header

Overview

Related Features

Functions

Typedefs

Typedef Documentation

Function Documentation