Tizen(Headed) Native API  6.5

The NNStreamer Single API provides interfaces to invoke a neural network model with a single instance of input data.

Required Header

#include <nnstreamer/nnstreamer-single.h>

Overview

The NNStreamer Single API provides interfaces to invoke a neural network model with a single instance of input data. This function is a syntactic sugar of NNStreamer Pipeline API with simplified features; thus, users are supposed to use NNStreamer Pipeline API directly if they want more advanced features. The user is expected to preprocess the input data for the given neural network model.

This function allows the following operations with NNStreamer:

  • Open a machine learning model with various mechanisms.
  • Close the model.
  • Interfaces to enter a single instance of input data to the opened model.
  • Utility functions to get the information of opened model.

Note that this function set is supposed to be thread-safe.

Related Features

This function is related with the following features:

  • http://tizen.org/feature/machine_learning
  • http://tizen.org/feature/machine_learning.inference

It is recommended to probe features in your application for reliability.
You can check if a device supports the related features for this function by using System Information, thereby controlling the procedure of your application.
To ensure your application is only running on the device with specific features, please define the features in your manifest file using the manifest editor in the SDK.
For example, your application accesses to the camera device, then you have to add 'http://tizen.org/privilege/camera' into the manifest of your application.
More details on featuring your application can be found from Feature Element.

Functions

int ml_single_open (ml_single_h *single, const char *model, const ml_tensors_info_h input_info, const ml_tensors_info_h output_info, ml_nnfw_type_e nnfw, ml_nnfw_hw_e hw)
 Opens an ML model and returns the instance as a handle.
int ml_single_open_full (ml_single_h *single, const char *model, const ml_tensors_info_h input_info, const ml_tensors_info_h output_info, ml_nnfw_type_e nnfw, ml_nnfw_hw_e hw, const char *custom_option)
 Opens an ML model and returns the instance as a handle with custom options.
int ml_single_close (ml_single_h single)
 Closes the opened model handle.
int ml_single_invoke (ml_single_h single, const ml_tensors_data_h input, ml_tensors_data_h *output)
 Invokes the model with the given input data.
int ml_single_invoke_fast (ml_single_h single, const ml_tensors_data_h input, ml_tensors_data_h output)
 Invokes the model with the given input data and fills the output data handle. This is generally faster than ml_single_invoke().
int ml_single_invoke_dynamic (ml_single_h single, const ml_tensors_data_h input, const ml_tensors_info_h in_info, ml_tensors_data_h *output, ml_tensors_info_h *out_info)
 Invokes the model with the given input data with the given tensors information.
int ml_single_get_input_info (ml_single_h single, ml_tensors_info_h *info)
 Gets the information (tensor dimension, type, name and so on) of required input data for the given model.
int ml_single_get_output_info (ml_single_h single, ml_tensors_info_h *info)
 Gets the information (tensor dimension, type, name and so on) of output data for the given model.
int ml_single_set_input_info (ml_single_h single, const ml_tensors_info_h info)
 Sets the information (tensor dimension, type, name and so on) of required input data for the given model.
int ml_single_set_timeout (ml_single_h single, unsigned int timeout)
 Sets the maximum amount of time to wait for an output, in milliseconds.
int ml_single_set_property (ml_single_h single, const char *name, const char *value)
 Sets the property value for the given model.
int ml_single_get_property (ml_single_h single, const char *name, char **value)
 Gets the property value for the given model.

Typedefs

typedef void * ml_single_h
 A handle of a single-shot instance.

Typedef Documentation

typedef void* ml_single_h

A handle of a single-shot instance.

Since :
5.5

Function Documentation

int ml_single_close ( ml_single_h  single)

Closes the opened model handle.

Note that this should be called before destroying the inference data by ml_tensors_data_destroy(). If not, the inference engine might try to access the data that is already freed. And it causes the segmentation fault.

Since :
5.5
Parameters:
[in]singleThe model handle to be closed.
Returns:
0 on success. Otherwise a negative error value.
Return values:
ML_ERROR_NONESuccessful.
ML_ERROR_NOT_SUPPORTEDNot supported.
ML_ERROR_INVALID_PARAMETERFail. The parameter is invalid (Pipeline is not negotiated yet.)

Gets the information (tensor dimension, type, name and so on) of required input data for the given model.

Note that a model may not have such information if its input type is flexible. The names of tensors are sometimes unavailable (optional), while its dimensions and types are always available.

Since :
5.5
Parameters:
[in]singleThe model handle.
[out]infoThe handle of input tensors information. The caller is responsible for freeing the information with ml_tensors_info_destroy().
Returns:
0 on success. Otherwise a negative error value.
Return values:
ML_ERROR_NONESuccessful.
ML_ERROR_NOT_SUPPORTEDNot supported.
ML_ERROR_INVALID_PARAMETERFail. The parameter is invalid.

Gets the information (tensor dimension, type, name and so on) of output data for the given model.

Note that a model may not have such information if its output type is flexible and output type is not determined statically. The names of tensors are sometimes unavailable (optional), while its dimensions and types are always available.

Since :
5.5
Parameters:
[in]singleThe model handle.
[out]infoThe handle of output tensors information. The caller is responsible for freeing the information with ml_tensors_info_destroy().
Returns:
0 on success. Otherwise a negative error value.
Return values:
ML_ERROR_NONESuccessful.
ML_ERROR_NOT_SUPPORTEDNot supported.
ML_ERROR_INVALID_PARAMETERFail. The parameter is invalid.
int ml_single_get_property ( ml_single_h  single,
const char *  name,
char **  value 
)

Gets the property value for the given model.

Since :
6.0
Parameters:
[in]singleThe model handle.
[in]nameThe property name.
[out]valueThe property value. The caller is responsible for freeing the value using g_free().
Returns:
0 on success. Otherwise a negative error value.
Return values:
ML_ERROR_NONESuccessful.
ML_ERROR_NOT_SUPPORTEDNot supported.
ML_ERROR_INVALID_PARAMETERFail. The parameter is invalid.
int ml_single_invoke ( ml_single_h  single,
const ml_tensors_data_h  input,
ml_tensors_data_h output 
)

Invokes the model with the given input data.

Even if the model has flexible input data dimensions, input data frames of an instance of a model should share the same dimension. Note that this will wait for the result until the invoke process is done. If an application wants to change the time to wait for an output, set the timeout using ml_single_set_timeout().

Since :
5.5
Parameters:
[in]singleThe model handle to be inferred.
[in]inputThe input data to be inferred.
[out]outputThe allocated output buffer. The caller is responsible for freeing the output buffer with ml_tensors_data_destroy().
Returns:
0 on success. Otherwise a negative error value.
Note:
If the data for the output buffer is allocated by the neural network framework (ML_NNFW_TYPE_CUSTOM_FILTER supports this), then this buffer will be freed when closing the single automatically by the neural network framework, and will not available for use later. It is recommended to copy the output buffer from output if it is required to use it after the single handle is closed.
Return values:
ML_ERROR_NONESuccessful.
ML_ERROR_NOT_SUPPORTEDNot supported.
ML_ERROR_INVALID_PARAMETERFail. The parameter is invalid.
ML_ERROR_STREAMS_PIPEFailed to push a buffer into source element.
ML_ERROR_TIMED_OUTFailed to get the result from sink element.
ML_ERROR_OUT_OF_MEMORYFailed to allocate required memory.
int ml_single_invoke_dynamic ( ml_single_h  single,
const ml_tensors_data_h  input,
const ml_tensors_info_h  in_info,
ml_tensors_data_h output,
ml_tensors_info_h out_info 
)

Invokes the model with the given input data with the given tensors information.

This function changes the input tensors information for the model, and returns the corresponding output data. A model/framework may not support changing the information. Note that this will wait for the result until the invoke process is done. If an application wants to change the time to wait for an output, set the timeout using ml_single_set_timeout().

Since :
6.0
Parameters:
[in]singleThe model handle to be inferred.
[in]inputThe input data to be inferred.
[in]in_infoThe handle of input tensors information.
[out]outputThe allocated output buffer. The caller is responsible for freeing the output buffer with ml_tensors_data_destroy().
[out]out_infoThe handle of output tensors information. The caller is responsible for freeing the information with ml_tensors_info_destroy().
Returns:
0 on success. Otherwise a negative error value.
Return values:
ML_ERROR_NONESuccessful.
ML_ERROR_NOT_SUPPORTEDNot supported.
ML_ERROR_INVALID_PARAMETERFail. The parameter is invalid.
ML_ERROR_STREAMS_PIPEFailed to push a buffer into source element.
ML_ERROR_TIMED_OUTFailed to get the result from sink element.
ML_ERROR_OUT_OF_MEMORYFailed to allocate required memory.
int ml_single_invoke_fast ( ml_single_h  single,
const ml_tensors_data_h  input,
ml_tensors_data_h  output 
)

Invokes the model with the given input data and fills the output data handle. This is generally faster than ml_single_invoke().

The caller should preallocate memory buffers of the given output handle before calling the API. Note that ml_single_invoke() allocates memory buffers of the output handle in the API, which may incur memcpy.

Since :
6.5
Parameters:
[in]singleThe model handle to be inferred.
[in]inputThe input data to be inferred.
[in,out]outputThe output data to be filled by the API. Output should be preallocated before calling the API.
Returns:
0 on success. Otherwise a negative error value.
Return values:
ML_ERROR_NONESuccessful.
ML_ERROR_NOT_SUPPORTEDNot supported.
ML_ERROR_INVALID_PARAMETERFail. The parameter is invalid.
ML_ERROR_STREAMS_PIPEFailed to push a buffer into source element.
ML_ERROR_TIMED_OUTFailed to get the result from sink element.
ML_ERROR_OUT_OF_MEMORYFailed to allocate required memory.
int ml_single_open ( ml_single_h single,
const char *  model,
const ml_tensors_info_h  input_info,
const ml_tensors_info_h  output_info,
ml_nnfw_type_e  nnfw,
ml_nnfw_hw_e  hw 
)

Opens an ML model and returns the instance as a handle.

Even if the model has flexible input data dimensions, input data frames of an instance of a model should share the same dimension.

Since :
5.5
Remarks:
http://tizen.org/privilege/mediastorage is needed if model is relevant to media storage.
http://tizen.org/privilege/externalstorage is needed if model is relevant to external storage.
Parameters:
[out]singleThis is the model handle opened. Users are required to close the given instance with ml_single_close().
[in]modelThis is the path to the neural network model file.
[in]input_infoThis is required if the given model has flexible input dimension, where the input dimension MUST be given before executing the model. It is required by some custom filters of NNStreamer. You may set NULL if it's not required.
[in]output_infoThis is required if the given model has flexible output dimension.
[in]nnfwThe neural network framework used to open the given model. Set ML_NNFW_TYPE_ANY to let it auto-detect.
[in]hwTell the corresponding nnfw to use a specific hardware. Set ML_NNFW_HW_ANY if it does not matter.
Returns:
0 on success. Otherwise a negative error value.
Return values:
ML_ERROR_NONESuccessful.
ML_ERROR_NOT_SUPPORTEDNot supported.
ML_ERROR_PERMISSION_DENIEDThe application does not have the privilege to access to the media storage or external storage.
ML_ERROR_INVALID_PARAMETERFail. The parameter is invalid.
ML_ERROR_STREAMS_PIPEFailed to start the pipeline.
ML_ERROR_OUT_OF_MEMORYFailed to allocate required memory.
int ml_single_open_full ( ml_single_h single,
const char *  model,
const ml_tensors_info_h  input_info,
const ml_tensors_info_h  output_info,
ml_nnfw_type_e  nnfw,
ml_nnfw_hw_e  hw,
const char *  custom_option 
)

Opens an ML model and returns the instance as a handle with custom options.

Even if the model has flexible input data dimensions, input data frames of an instance of a model should share the same dimension.

Since :
6.5
Remarks:
http://tizen.org/privilege/mediastorage is needed if model is relevant to media storage.
http://tizen.org/privilege/externalstorage is needed if model is relevant to external storage.
Parameters:
[out]singleThis is the model handle opened. Users are required to close the given instance with ml_single_close().
[in]modelThis is the path to the neural network model file.
[in]input_infoThis is required if the given model has flexible input dimension, where the input dimension MUST be given before executing the model. It is required by some custom filters of NNStreamer. You may set NULL if it's not required.
[in]output_infoThis is required if the given model has flexible output dimension.
[in]nnfwThe neural network framework used to open the given model. Set ML_NNFW_TYPE_ANY to let it auto-detect.
[in]hwTell the corresponding nnfw to use a specific hardware. Set ML_NNFW_HW_ANY if it does not matter.
[in]custom_optionComma separated list of options. Use this parameter to fine-tune and optimize specific neural network framework. (e.g. NumThreads:N to set the number of threads in TensorFlow-Lite) You may set NULL if it's not required. See NNStreamer documentation (https://nnstreamer.github.io/gst/nnstreamer/tensor_filter/README.html) for the details.
Returns:
0 on success. Otherwise a negative error value.
Return values:
ML_ERROR_NONESuccessful.
ML_ERROR_NOT_SUPPORTEDNot supported.
ML_ERROR_PERMISSION_DENIEDThe application does not have the privilege to access to the media storage or external storage.
ML_ERROR_INVALID_PARAMETERFail. The parameter is invalid.
ML_ERROR_STREAMS_PIPEFailed to start the pipeline.
ML_ERROR_OUT_OF_MEMORYFailed to allocate required memory.
int ml_single_set_input_info ( ml_single_h  single,
const ml_tensors_info_h  info 
)

Sets the information (tensor dimension, type, name and so on) of required input data for the given model.

Note that a model/framework may not support changing the information. Use ml_single_get_input_info() and ml_single_get_output_info() instead for this framework.

Since :
6.0
Parameters:
[in]singleThe model handle.
[in]infoThe handle of input tensors information.
Returns:
0 on success. Otherwise a negative error value.
Return values:
ML_ERROR_NONESuccessful.
ML_ERROR_NOT_SUPPORTEDNot supported.
ML_ERROR_INVALID_PARAMETERFail. The parameter is invalid.
int ml_single_set_property ( ml_single_h  single,
const char *  name,
const char *  value 
)

Sets the property value for the given model.

Note that a model/framework may not support changing the property after opening the model.

Since :
6.0
Parameters:
[in]singleThe model handle.
[in]nameThe property name.
[in]valueThe property value.
Returns:
0 on success. Otherwise a negative error value.
Return values:
ML_ERROR_NONESuccessful.
ML_ERROR_NOT_SUPPORTEDNot supported.
ML_ERROR_INVALID_PARAMETERFail. The parameter is invalid.
int ml_single_set_timeout ( ml_single_h  single,
unsigned int  timeout 
)

Sets the maximum amount of time to wait for an output, in milliseconds.

Since :
5.5
Parameters:
[in]singleThe model handle.
[in]timeoutThe time to wait for an output.
Returns:
0 on success. Otherwise a negative error value.
Return values:
ML_ERROR_NONESuccessful.
ML_ERROR_NOT_SUPPORTEDNot supported.
ML_ERROR_INVALID_PARAMETERFail. The parameter is invalid.