Pipeline API

The Pipeline API defines interfaces, methods and data structures to manage NNStreamer machine learning inference pipelines.

The following functionalities are provided:

  • Creation of NNStreamer pipelines.
  • Starting and stopping processing in NNStreamer pipeline.
  • Manipulation of pipeline node properties.
  • Feeding the data into pipeline.
  • Getting the output from pipeline.
  • Development of custom pipeline filters.
  • Switching direction of the data flow inside pipeline.
  • Starting and stopping data flow in pipeline branches.

NNStreamer is a set of plugins extending the capabilities of the GStreamer framework. See the GStreamer documentation to learn more about the multimedia pipelines. To get descriptions of pipeline nodes you can also use gst-inspect-1.0 command line tool.

Remark: In order to access files, a proper privilege has to be defined additionally:

Remark: In order to access device's camera and recorder, a proper privilege has to be defined additionally:

For more information on the Pipeline API features, see Pipeline Guide.

Since: 6.5

Table of Contents

Summary of Interfaces and Methods

Interface Method
MachineLearningPipeline
Pipeline createPipeline (DOMString description, optional PipelineStateChangeListener? listener)
void registerCustomFilter (DOMString filterName, CustomFilter filter, TensorsInfo inputInfo, TensorsInfo outputInfo, optional ErrorCallback? errorCallback)
void unregisterCustomFilter (DOMString filterName)
Pipeline
void start ()
void stop ()
void dispose ()
NodeInfo getNodeInfo (DOMString name)
Source getSource (DOMString name)
Switch getSwitch (DOMString name)
Valve getValve (DOMString name)
void registerSinkListener (DOMString sinkName, SinkListener sinkListener)
void unregisterSinkListener (DOMString sinkName)
NodeInfo
Property getProperty (DOMString name, PropertyType type)
void setProperty (DOMString name, PropertyType type, Property value)
Source
Switch
DOMString[] getPadList ()
void select (DOMString padName)
Valve
void setOpen (boolean open)
PipelineStateChangeListener
SinkListener
void ondata (DOMString sinkName, TensorsData data)
CustomFilter
long filter (TensorsData input, TensorsData output)

1. Type Definitions

1.1. PipelineState

Pipeline state. 
  enum PipelineState { "NULL", "PAUSED", "PLAYING", "READY", "UNKNOWN" };

Since: 6.5

  • NULL - the state of the pipeline after calling Pipeline::dispose().
  • READY - a transient state of the pipeline.
  • PAUSED - a state of the pipeline ready to make an inference.
  • PLAYING - a state of the pipeline that currently makes inference.
  • UNKNOWN - unknown state.

Remark: The pipeline can remain in blue, steady states for longer, whereas white states are transient. For more information, see the GStreamer documentation.

1.2. Property

Pipeline node property type. 
  typedef (boolean or DOMString or long or long long or double or unsigned long or unsigned long long) Property;

Since: 6.5

1.3. PropertyType

Type of a pipeline node property to be set. 
  enum PropertyType { "BOOLEAN", "DOUBLE", "ENUM", "INT32", "INT64", "UINT32", "UINT64", "STRING" };

Since: 6.5

  • BOOLEAN - a boolean.
  • DOUBLE - a double-precision floating point number.
  • ENUM - an enum.
  • INT32 - a 32-bit integer.
  • INT64 - a 64-bit integer.
  • UINT32 - a 32-bit unsigned integer.
  • UINT64 - a 64-bit unsigned integer.
  • STRING - a string.

1.4. SwitchType

Type of a switch node. 
  enum SwitchType { "INPUT_SELECTOR", "OUTPUT_SELECTOR" };

Since: 6.5

  • INPUT_SELECTOR - directs one of input streams to the output pad.
  • OUTPUT_SELECTOR - directs input stream to one of output pads.

2. Interfaces

2.1. MachineLearningPipeline

The MachineLearningPipeline interface allows creation of machine learning pipelines. 
  [NoInterfaceObject] interface MachineLearningPipeline {
    Pipeline createPipeline(DOMString description, optional PipelineStateChangeListener? listener) raises(WebAPIException);
    void registerCustomFilter(DOMString filterName, CustomFilter filter, TensorsInfo inputInfo, TensorsInfo outputInfo,
                              optional ErrorCallback? errorCallback) raises(WebAPIException);
    void unregisterCustomFilter(DOMString filterName) raises(WebAPIException);
  };

Since: 6.5

Methods

createPipeline
Creates a machine learning pipeline. 
Pipeline createPipeline(DOMString description, optional PipelineStateChangeListener? listener);

Since: 6.5

A pipeline is created from a description, which is a string containing all the nodes and links. For more information about pipeline description, see the Pipeline Description section of gst-launch-1.0 documentation.

Remark: Pipeline creation is an asynchronous process, during which the pipeline state transitions through READY to PAUSED. The pipeline is ready to use when it is in PAUSED state.

Parameters:

  • description: A description of the pipeline.
  • listener [optional] [nullable]: Listener function that will be called when pipeline state changes.

Return value:

    Pipeline: A newly created Pipeline object.

Exceptions:

  • WebAPIException

    • with error type NotSupportedError, if the feature is not supported.

    • with error type SecurityError, if the application does not have the privilege to call this method.

    • with error type TypeMismatchError, if any input attribute is not compatible with the expected type for this attribute.

    • with error type AbortError, if the description is invalid or any other error occurs.

Code example:

var pipelineDescription = "videotestsrc ! tizenwlsink";
function pipelineStateChangeListener(newState)
{
  console.log("New pipeline state: " + newState);
}

var pipeline = tizen.ml.pipeline.createPipeline(pipelineDescription, pipelineStateChangeListener);
/* after use */
pipeline.dispose();

Output example:

New pipeline state: READY
New pipeline state: PAUSED
registerCustomFilter
Registers a CustomFilter, which implements a custom transform to the data coming through the pipeline. 
void registerCustomFilter(DOMString filterName, CustomFilter filter, TensorsInfo inputInfo, TensorsInfo outputInfo,
                          optional ErrorCallback? errorCallback);

Since: 6.5

errorCallback is used to report errors that occur during processing of the data by the filter. When an error is caused by CustomFilter::filter or an internal error occurs during processing of the data, the effect is the same as if CustomFilter::filter returned -1 and the errorCallback method is called with these error types:

Remark: Performance of a computationally intensive custom filter implemented in JavaScript may be unsatisfactory due to the overhead of transferring the data between the native pipeline implementation and a web application.

Remark: errorCallback is not triggered when CustomFilter returns a negative status.

Parameters:

  • filterName: Name of the filter.
  • filter: The function used as the data filter.
  • inputInfo: Information about tensors taken as input.
  • outputInfo: Information about tensors returned from the filter.
  • errorCallback [optional] [nullable]: The method to call when an error occurs during processing of the data by the filter.

Exceptions:

  • WebAPIException

    • with error type InvalidValuesError, if filterName is invalid or a custom filter with this name is already registered.

    • with error type NotSupportedError, if the feature is not supported.

    • with error type TypeMismatchError, if any input attribute is not compatible with the expected type for this attribute.

    • with error type AbortError, if any other error occurs.

Code example:

var inputInfo = new tizen.ml.TensorsInfo();
inputInfo.addTensorInfo("3D", "UINT8", [4, 20, 15, 1]);
var outputInfo = new tizen.ml.TensorsInfo();
outputInfo.addTensorInfo("flat", "UINT8", [1200]);

var flattenFilter = function(inputData, outputData)
{
  console.log("Custom filter called");
  var rawInputData = inputData.getTensorRawData(0);
  outputData.setTensorRawData(0, rawInputData.data);
  return 0;
};

function errorCallback(error)
{
  console.error("flattenFilter error: " + error.name + ". Reason: " + error.message);
}

tizen.ml.pipeline.registerCustomFilter("flattenFilter", flattenFilter, inputInfo, outputInfo);

var pipelineDescription = "videotestsrc num-buffers=3 " +
                          "! video/x-raw,width=20,height=15,format=BGRA " +
                          "! tensor_converter " +
                          "! tensor_filter framework=custom-easy model=flattenFilter " +
                          "! fakesink";

function pipelineStateChangeListener(newState)
{
  console.log("New pipeline state: " + newState);
}

var pipeline = tizen.ml.pipeline.createPipeline(pipelineDescription, pipelineStateChangeListener);
pipeline.start();

Output example:

New pipeline state: READY
Custom filter called
New pipeline state: PAUSED
New pipeline state: PLAYING
Custom filter called
Custom filter called
unregisterCustomFilter
Unregisters a pipeline's CustomFilter. 
void unregisterCustomFilter(DOMString filterName);

Since: 6.5

Remark: CustomFilter can only be unregistered when it is currently not processing data. Stop pipelines using the filter to ensure that condition. Otherwise an InvalidStateError may be thrown.

Parameters:

  • filterName: The name of the filter.

Exceptions:

  • WebAPIException

    • with error type InvalidStateError, if the filter is currently processing data.

    • with error type InvalidValuesError, if filterName is invalid.

    • with error type NotSupportedError, if the feature is not supported.

    • with error type TypeMismatchError, if any input attribute is not compatible with the expected type for this attribute.

    • with error type AbortError, if any other error occurs.

Code example:

/* Suppose that a CustomFilter named "flattenFilter" has been registered and is */
/* no longer needed */
tizen.ml.pipeline.unregisterCustomFilter("flattenFilter");

2.2. Pipeline

The Pipeline interface provides access to control machine learning pipeline. 
  [NoInterfaceObject] interface Pipeline {
    readonly attribute PipelineState state raises(WebAPIException);
    void start() raises(WebAPIException);
    void stop() raises(WebAPIException);
    void dispose() raises(WebAPIException);
    NodeInfo getNodeInfo(DOMString name) raises(WebAPIException);
    Source getSource(DOMString name) raises(WebAPIException);
    Switch getSwitch(DOMString name) raises(WebAPIException);
    Valve getValve(DOMString name) raises(WebAPIException);
    void registerSinkListener(DOMString sinkName, SinkListener sinkListener) raises(WebAPIException);
    void unregisterSinkListener(DOMString sinkName) raises(WebAPIException);
  };

Since: 6.5

Attributes

  • readonly PipelineState state
    The current state of the pipeline.

    Since: 6.5

    Exceptions:

    • WebAPIException

      • with error type NotSupportedError, if the feature is not supported.

      • with error type AbortError, if any other error occurs.

Methods

start
Starts the pipeline. 
void start();

Since: 6.5

The pipeline starts asynchronously and its state may not be set to PLAYING right after start() returns. Register a PipelineStateChangeListener with MachineLearningPipeline::createPipeline() to watch state changes.

Remark: Before calling start(), the Pipeline::state should be PAUSED.

Remark: Pipeline::state after successful start() will be set to PLAYING.

Exceptions:

  • WebAPIException

    • with error type NotFoundError, if the pipeline was disposed.

    • with error type NotSupportedError, if the feature is not supported.

    • with error type AbortError, if any other error occurs.

Code example:

var pipelineDescription = "videotestsrc ! tizenwlsink";
function pipelineStateChangeListener(newState)
{
  console.log("New pipeline state: " + newState);
}

var pipeline = tizen.ml.pipeline.createPipeline(pipelineDescription, pipelineStateChangeListener);
pipeline.start();

Output example:

New pipeline state: READY
New pipeline state: PAUSED
New pipeline state: PLAYING
stop
Stops the pipeline. 
void stop();

Since: 6.5

The pipeline stops asynchronously and its state may not be set to PAUSED right after stop() returns. Register a PipelineStateChangeListener with MachineLearningPipeline::createPipeline() to watch state changes.

Remark: Before calling stop(), state should be PLAYING.

Remark: The pipeline state after successful stop() will be set to PAUSED.

Exceptions:

  • WebAPIException

    • with error type NotFoundError, if the pipeline was disposed.

    • with error type NotSupportedError, if the feature is not supported.

    • with error type AbortError, if any other error occurs.

Code example:

var pipelineDescription = "videotestsrc ! tizenwlsink";
function pipelineStateChangeListener(newState)
{
  console.log("New pipeline state: " + newState);
}

var pipeline = tizen.ml.pipeline.createPipeline(pipelineDescription, pipelineStateChangeListener);
pipeline.start();

/* Application needs to have background-support enabled */
/* A few moments later pipeline is stopped and disposed */
setTimeout(function()
{
  pipeline.stop();
  pipeline.dispose();
}, 5000);

Output example:

New pipeline state: READY
New pipeline state: PAUSED
New pipeline state: PLAYING
New pipeline state: PAUSED
dispose
Releases the resources allocated by the pipeline. 
void dispose();

Since: 6.5

A pipeline may be expensive in terms of allocated resources such as memory, and dispose() releases them.

Remark: Before calling dispose(), Pipeline::state should be PLAYING or PAUSED.

Remark: Pipeline::state after successful dispose() will be set to NULL.

Remark: Pipeline can only be disposed when no CustomFilter is currently processing data. Dispose pipelines when no custom filter is processing data to ensure that condition. Otherwise an InvalidStateError will be thrown.

Exceptions:

  • WebAPIException

    • with error type InvalidStateError, if any CustomFilter is currently processing data.

    • with error type NotFoundError, if the pipeline has already been disposed.

    • with error type NotSupportedError, if the feature is not supported.

    • with error type AbortError, if any other error occurs.

Code example:

var pipelineDescription = "videotestsrc ! tizenwlsink";
function pipelineStateChangeListener(newState)
{
  console.log("New pipeline state: " + newState);
}

var pipeline = tizen.ml.pipeline.createPipeline(pipelineDescription, pipelineStateChangeListener);
pipeline.start();

/* Application needs to have background-support enabled */
/* A few moments later pipeline is stopped and disposed */
setTimeout(function()
{
  pipeline.stop();
  pipeline.dispose();
}, 5000);

Output example:

New pipeline state: READY
New pipeline state: PAUSED
New pipeline state: PLAYING
New pipeline state: PAUSED
getNodeInfo
Gets a NodeInfo object allowing to get and set pipeline node's properties. 
NodeInfo getNodeInfo(DOMString name);

Since: 6.5

Parameters:

  • name: Name of the pipeline node.

Return value:

    NodeInfo: The retrieved NodeInfo object.

Exceptions:

  • WebAPIException

    • with error type InvalidValuesError, if name is invalid.

    • with error type NotFoundError, if the pipeline was disposed.

    • with error type NotSupportedError, if the feature is not supported.

    • with error type TypeMismatchError, if any input attribute is not compatible with the expected type for this attribute.

    • with error type AbortError, if any other error occurs.

Code example:

var pipelineDescription = "videotestsrc name=vsrc is-live=true " +
                          "! videoconvert " +
                          "! videoscale name=vscale " +
                          "! video/x-raw,format=RGBx,width=224,height=224,framerate=60/1 " +
                          "! tensor_converter " +
                          "! valve name=valvex " +
                          "! input-selector name=is01 " +
                          "! tensor_sink name=sinkx";
var pipeline = tizen.ml.pipeline.createPipeline(pipelineDescription);

var inputSelectorInfo = pipeline.getNodeInfo("is01");
console.log("Retrieved NodeInfo: " + inputSelectorInfo.name);

Output example:

Retrieved NodeInfo: is01
getSource
Gets a Source object that allows input to the pipeline. 
Source getSource(DOMString name);

Since: 6.5

Remark: This function can only retrieve sources meant to be used as application's data feeds, in particular appsrc. Use gst-inspect-1.0 tool to check, if a given source can be used to input data from an application.

Parameters:

  • name: Name of the source.

Return value:

    Source: The retrieved Source object.

Exceptions:

  • WebAPIException

    • with error type InvalidStateError, if the pipeline is not ready yet.

    • with error type InvalidValuesError, if name is invalid.

    • with error type NotFoundError, if the pipeline was disposed.

    • with error type NotSupportedError, if the feature is not supported.

    • with error type TypeMismatchError, if any input attribute is not compatible with the expected type for this attribute.

    • with error type AbortError, if any other error occurs.

Code example:

var pipelineDescription =
    "appsrc name=srcx " +
    "! other/tensor,dimension=(string)1:1:1:1,type=(string)int8,framerate=(fraction)0/1 " +
    "! fakesink";

var pipeline = tizen.ml.pipeline.createPipeline(pipelineDescription);

var source = pipeline.getSource("srcx");
console.log("Retrieved source: " + source.name);

Output example:

Retrieved source: srcx
getSwitch
Gets a Switch object that allows to select a pipeline branch to be used as a source or sink. 
Switch getSwitch(DOMString name);

Since: 6.5

Parameters:

  • name: Name of the switch.

Return value:

    Switch: The retrieved Switch object.

Exceptions:

  • WebAPIException

    • with error type InvalidStateError, if the pipeline is not ready yet.

    • with error type InvalidValuesError, if name is invalid.

    • with error type NotFoundError, if the pipeline was disposed.

    • with error type NotSupportedError, if the feature is not supported.

    • with error type TypeMismatchError, if any input attribute is not compatible with the expected type for this attribute.

    • with error type AbortError, if any other error occurs.

Code example:

var pipelineDescription = "videotestsrc is-live=true " +
                          "! videoconvert " +
                          "! tensor_converter " +
                          "! output-selector name=outs outs.src_0 " +
                          "! tensor_sink name=sink0 async=false outs.src_1 " +
                          "! tensor_sink name=sink1 async=false ";

var pipeline = tizen.ml.pipeline.createPipeline(pipelineDescription);

var outsSwitch = pipeline.getSwitch("outs");
console.log("Retrieved switch: " + outsSwitch.name + " (type: " + outsSwitch.type + ")");

Output example:

Retrieved switch: outs (type: OUTPUT_SELECTOR)
getValve
Gets a Valve object that allows to start and stop streaming data to a branch of a pipeline. 
Valve getValve(DOMString name);

Since: 6.5

Parameters:

  • name: Name of the valve.

Return value:

    Valve: The retrieved Valve object.

Exceptions:

  • WebAPIException

    • with error type InvalidValuesError, if name is invalid.

    • with error type NotFoundError, if the pipeline was disposed.

    • with error type NotSupportedError, if the feature is not supported.

    • with error type TypeMismatchError, if any input attribute is not compatible with the expected type for this attribute.

    • with error type AbortError, if any other error occurs.

Code example:

var pipelineDescription = "videotestsrc is-live=true " +
                          "! videoconvert " +
                          "! videoscale " +
                          "! video/x-raw,format=RGBx,width=16,height=16,framerate=10/1 " +
                          "! tensor_converter " +
                          "! valve name=valve1 " +
                          "! fakesink";

var pipeline = tizen.ml.pipeline.createPipeline(pipelineDescription);
var valve = pipeline.getValve("valve1");
console.log(
    "Retrieved valve: " + valve.name + " (state: " + (valve.isOpen ? "open" : "closed") + ")");

Output example:

Retrieved valve: valve1 (state: open)
registerSinkListener
Registers a SinkListener for a given sink. The listener is used to get output data from a pipeline. 
void registerSinkListener(DOMString sinkName, SinkListener sinkListener);

Since: 6.5

Remark: This function can only register listeners for sinks meant to provide a direct output to an application, in particular appsink. Use gst-inspect-1.0 tool to check, if a given sink can be used as an application output.

Remark: Before calling registerSinkListener(), the pipeline state should be PAUSED.

Parameters:

  • sinkName: Name of the sink.
  • sinkListener: The listener to be registered and used as pipeline's output.

Exceptions:

  • WebAPIException

    • with error type InvalidValuesError, if sinkName is invalid.

    • with error type NotFoundError, if the pipeline was disposed.

    • with error type NotSupportedError, if the feature is not supported.

    • with error type TypeMismatchError, if any input attribute is not compatible with the expected type for this attribute.

    • with error type AbortError, if any other error occurs.

Code example:

var pipelineDescription = "videotestsrc num-buffers=3 " +
                          "! videoconvert " +
                          "! tensor_converter " +
                          "! tensor_sink name=sinkx";

var pipeline = tizen.ml.pipeline.createPipeline(pipelineDescription);

pipeline.registerSinkListener("sinkx", function(sinkName, data)
{
  console.log("SinkListener for " + sinkName + " sink called. Data dimensions: " +
              data.tensorsInfo.getDimensions(0));
});

pipeline.start();

Output example:

SinkListener for sinkx sink called. Data dimensions: 4,320,240,1
SinkListener for sinkx sink called. Data dimensions: 4,320,240,1
SinkListener for sinkx sink called. Data dimensions: 4,320,240,1
unregisterSinkListener
Unregisters a sink's SinkListener. 
void unregisterSinkListener(DOMString sinkName);

Since: 6.5

Remark: Before calling unregisterSinkListener(), the Pipeline::state should be PAUSED.

Parameters:

  • sinkName: Name of the sink.

Exceptions:

  • WebAPIException

    • with error type InvalidValuesError, if sinkName is invalid.

    • with error type NotFoundError, if the pipeline was disposed.

    • with error type NotSupportedError, if the feature is not supported.

    • with error type TypeMismatchError, if any input attribute is not compatible with the expected type for this attribute.

    • with error type AbortError, if any other error occurs.

Code example:

var pipelineDescription = "videotestsrc num-buffers=3 " +
                          "! videoconvert " +
                          "! tensor_converter " +
                          "! tensor_sink name=sinkx";
var unregisterSinkListener = false;
function pipelineStateChangeListener(newState)
{
  /* pipelineStateChangeListener will be called with newState === "PAUSED" */
  /* just after pipeline creation, but we don't want the SinkListener to be */
  /* unregistered right then. */
  /* Thus we need the unregisterSinkListener flag to signal when we want to */
  /* unregister the SinkListener. */
  if (newState === "PAUSED" && unregisterSinkListener)
  {
    pipeline.unregisterSinkListener("sinkx");
    console.log("sinkx unregistered");
  }
}

var pipeline = tizen.ml.pipeline.createPipeline(pipelineDescription, pipelineStateChangeListener);

pipeline.registerSinkListener("sinkx", function(sinkName, data)
{
  console.log("SinkListener for " + sinkName + " sink called. Data dimensions: " +
              data.tensorsInfo.getDimensions(0));
});

pipeline.start();

/* Application needs to have background-support enabled */
/* A few moments later pipeline is stopped and disposed */
setTimeout(function()
{
  unregisterSinkListener = true;
  pipeline.stop();
  pipeline.dispose();
}, 5000);

Output example:

SinkListener for sinkx sink called. Data dimensions: 4,320,240,1
SinkListener for sinkx sink called. Data dimensions: 4,320,240,1
SinkListener for sinkx sink called. Data dimensions: 4,320,240,1
sinkx unregistered

2.3. NodeInfo

The NodeInfo interface provides access to pipeline elements' properties. 
  [NoInterfaceObject] interface NodeInfo {
    readonly attribute DOMString name;
    Property getProperty(DOMString name, PropertyType type) raises(WebAPIException);
    void setProperty(DOMString name, PropertyType type, Property value) raises(WebAPIException);
  };

Since: 6.5

Attributes

  • readonly DOMString name
    Name of the node.

    Since: 6.5

Methods

getProperty
Retrieves the value of node's property. 
Property getProperty(DOMString name, PropertyType type);

Since: 6.5

Parameters:

  • name: Name of the node.
  • type: Data type of the property.

Return value:

    Property: Value of the retrieved property.

Exceptions:

  • WebAPIException

    • with error type InvalidValuesError, if name is invalid.

    • with error type NotFoundError, if the node belongs to a disposed pipeline.

    • with error type NotSupportedError, if the feature is not supported.

    • with error type TypeMismatchError, if any input attribute is not compatible with the expected type for this attribute.

    • with error type AbortError, if any other error occurs.

Code example:

var pipelineDescription = "videotestsrc name=vsrc is-live=true " +
                          "! videoconvert " +
                          "! videoscale name=vscale " +
                          "! video/x-raw,format=RGBx,width=224,height=224,framerate=60/1 " +
                          "! tensor_converter " +
                          "! valve name=valvex " +
                          "! input-selector name=is01 " +
                          "! tensor_sink name=sinkx";
var pipeline = tizen.ml.pipeline.createPipeline(pipelineDescription);

var inputSelectorInfo = pipeline.getNodeInfo("is01");
inputSelectorInfo.setProperty("sync-streams", "BOOLEAN", true);
var syncStreams = inputSelectorInfo.getProperty("sync-streams", "BOOLEAN");
console.log("Retrieved sync-streams: " + syncStreams);

Output example:

Retrieved sync-streams: true
setProperty
Sets the value of node's property. 
void setProperty(DOMString name, PropertyType type, Property value);

Since: 6.5

Parameters:

  • name: Name of the property.
  • type: The type of the property.
  • value: The new value of the property.

Exceptions:

  • WebAPIException

    • with error type InvalidValuesError, if any of the input parameters contain an invalid value.

    • with error type NotFoundError, if the node belongs to a disposed pipeline.

    • with error type NotSupportedError, if the feature is not supported.

    • with error type TypeMismatchError, if any input attribute is not compatible with the expected type for this attribute.

    • with error type AbortError, if any other error occurs.

Code example:

var pipelineDescription = "videotestsrc name=vsrc is-live=true " +
                          "! videoconvert " +
                          "! videoscale name=vscale " +
                          "! video/x-raw,format=RGBx,width=224,height=224,framerate=60/1 " +
                          "! tensor_converter " +
                          "! valve name=valvex " +
                          "! input-selector name=is01 " +
                          "! tensor_sink name=sinkx";
var pipeline = tizen.ml.pipeline.createPipeline(pipelineDescription);

var inputSelectorInfo = pipeline.getNodeInfo("is01");
inputSelectorInfo.setProperty("sync-streams", "BOOLEAN", true);
var syncStreams = inputSelectorInfo.getProperty("sync-streams", "BOOLEAN");
console.log("Retrieved sync-streams: " + syncStreams);

Output example:

Retrieved sync-streams: true

2.4. Source

The Source interface provides access to those pipeline sources that allow feeding the pipeline with input. 
  [NoInterfaceObject] interface Source {
    readonly attribute TensorsInfo inputTensorsInfo raises(WebAPIException);
    readonly attribute DOMString name;
    void inputData(TensorsData data) raises(WebAPIException);
  };

Since: 6.5

Attributes

  • readonly TensorsInfo inputTensorsInfo
    The information about the format of tensor input expected by the source.

    Since: 6.5

    Exceptions:

    • WebAPIException

      • with error type NotFoundError, if the source belongs to a disposed pipeline.

      • with error type AbortError, if any other error occurs.

  • readonly DOMString name
    Name of the source.

    Since: 6.5

Methods

inputData
Feeds the source with input data. 
void inputData(TensorsData data);

Since: 6.5

Parameters:

  • data: The input data for the source.

Exceptions:

  • WebAPIException

    • with error type InvalidStateError, if the pipeline is not ready yet.

    • with error type NotFoundError, if the source belongs to a disposed pipeline.

    • with error type NotSupportedError, if the feature is not supported.

    • with error type TypeMismatchError, if any input attribute is not compatible with the expected type for this attribute.

    • with error type AbortError, if any other error occurs.

Code example:

var pipelineDescription =
    "appsrc name=srcx " +
    "! other/tensor,dimension=(string)1:1:1:1,type=(string)int8,framerate=(fraction)0/1 " +
    "! fakesink";

var pipeline = tizen.ml.pipeline.createPipeline(pipelineDescription);
pipeline.start();

var source = pipeline.getSource("srcx");
var inputInfo = new tizen.ml.TensorsInfo();
inputInfo.addTensorInfo("tensor1", "UINT8", [1, 1]);
var inputData = inputInfo.getTensorsData();

for (var i = 0; i < 5; i++)
{
  var data = [i];
  console.log("Input: " + JSON.stringify(data));
  inputData.setTensorRawData(0, data);
  source.inputData(inputData);
}

pipeline.stop();
pipeline.dispose();

Output example:

Input: [0]
Input: [1]
Input: [2]
Input: [3]
Input: [4]

2.5. Switch

The Switch interface provides access to pipeline switches. They are elements with multiple source or sink pads and allow choosing between these pads. 
  [NoInterfaceObject] interface Switch {
    readonly attribute SwitchType type;
    readonly attribute DOMString name;
    DOMString[] getPadList() raises(WebAPIException);
    void select(DOMString padName) raises(WebAPIException);
  };

Since: 6.5

Attributes

  • readonly SwitchType type
    Determines the switch type.

    Since: 6.5

  • readonly DOMString name
    Name of the switch.

    Since: 6.5

Methods

getPadList
Retrieves the list of pad names of the switch. 
DOMString[] getPadList();

Since: 6.5

Return value:

    DOMString[]: The list of switch's pads.

Exceptions:

  • WebAPIException

    • with error type NotSupportedError, if the feature is not supported.

    • with error type NotFoundError, if the switch belongs to a disposed pipeline.

    • with error type InvalidStateError, if the pipeline is not ready yet.

    • with error type AbortError, if any other error occurs.

Code example:

var pipelineDescription = "videotestsrc is-live=true " +
                          "! videoconvert " +
                          "! tensor_converter " +
                          "! output-selector name=outs outs.src_0 " +
                          "! tensor_sink name=sink0 async=false outs.src_1 " +
                          "! tensor_sink name=sink1 async=false";

var pipeline = tizen.ml.pipeline.createPipeline(pipelineDescription);

var outsSwitch = pipeline.getSwitch("outs");
var pads = outsSwitch.getPadList();

console.log("Pads:");
for (var i = 0; i < pads.length; i++)
{
  console.log("  " + pads[i]);
}

Output example:

Pads:
  src_0
  src_1
select
Selects a pad to be used as a source or sink of the switch node. 
void select(DOMString padName);

Since: 6.5

The switch selects a sink if its Switch::type is OUTPUT_SELECTOR or it selects a source if its Switch::type is INPUT_SELECTOR.

Parameters:

  • padName: Name of the pad to be used as a sink or source.

Exceptions:

  • WebAPIException

    • with error type InvalidValuesError, if padName is invalid.

    • with error type NotFoundError, if the switch belongs to a disposed pipeline.

    • with error type NotSupportedError, if the feature is not supported.

    • with error type TypeMismatchError, if any input attribute is not compatible with the expected type for this attribute.

    • with error type AbortError, if any other error occurs.

Code example:

var pipelineDescription = "videotestsrc is-live=true " +
                          "! videoconvert " +
                          "! tensor_converter " +
                          "! output-selector name=outs outs.src_0 " +
                          "! tensor_sink name=sink0 async=false outs.src_1 " +
                          "! tensor_sink name=sink1 async=false";

var pipeline = tizen.ml.pipeline.createPipeline(pipelineDescription);

var outsSwitch = pipeline.getSwitch("outs");
outsSwitch.select("src_1");

2.6. Valve

The Valve interface provides access to pipeline's valves. Opening and closing these elements enables and disables data flow in consecutive pipeline branches. 
  [NoInterfaceObject] interface Valve {
    readonly attribute DOMString name;
    readonly attribute boolean isOpen raises(WebAPIException);
    void setOpen(boolean open) raises(WebAPIException);
  };

Since: 6.5

Valves are open by default.

See the the gstreamer documentation for more details.

Attributes

  • readonly DOMString name
    Name of the valve.

    Since: 6.5

  • readonly boolean isOpen
    State of the valve.

    Since: 6.5

    Remark: isOpen is an alias for negated value of the Valve element's drop property in the GStreamer framework.

    Exceptions:

    • WebAPIException

      • with error type NotSupportedError, if the feature is not supported.

      • with error type NotFoundError, if the valve belongs to a disposed pipeline.

      • with error type AbortError, if any other error occurs.

Methods

setOpen
Enables or disables the flow of the data through the valve by setting it to open or closed, respectively. 
void setOpen(boolean open);

Since: 6.5

Parameters:

  • open: When it is true, the valve will be open and closed otherwise.

Exceptions:

  • WebAPIException

    • with error type TypeMismatchError, if open argument is missing.

    • with error type NotSupportedError, if the feature is not supported.

    • with error type NotFoundError, if the valve belongs to a disposed pipeline.

    • with error type AbortError, if any other error occurs.

Code example:

var pipelineDescription = "videotestsrc is-live=true " +
                          "! videoconvert " +
                          "! videoscale " +
                          "! video/x-raw,format=RGBx,width=16,height=16,framerate=10/1 " +
                          "! tensor_converter " +
                          "! valve name=valve1 " +
                          "! fakesink";

var pipeline = tizen.ml.pipeline.createPipeline(pipelineDescription);
var valve = pipeline.getValve("valve1");
console.log(
    "Retrieved valve: " + valve.name + " (state: " + (valve.isOpen ? "open" : "closed") + ")");

valve.setOpen(false);
console.log("State: " + (valve.isOpen ? "open" : "closed"));

Output example:

Retrieved valve: valve1 (state: open)
State: closed

2.7. PipelineStateChangeListener

The PipelineStateChangeListener interface defines the listener registered with MachineLearningPipeline::createPipeline(). 
  [Callback=FunctionOnly, NoInterfaceObject] interface PipelineStateChangeListener {
    void onstatechange(PipelineState newState);
  };

Since: 6.5

Methods

onstatechange
Called when pipeline state changes. 
void onstatechange(PipelineState newState);

Since: 6.5

Remark: Usage example can be found in MachineLearningPipeline::createPipeline()'s code example.

Parameters:

  • newState: The new state of the pipeline.

2.8. SinkListener

The SinkListener interface defines the listener registered with Pipeline::registerSinkListener(). 
  [Callback=FunctionOnly, NoInterfaceObject] interface SinkListener {
    void ondata(DOMString sinkName, TensorsData data);
  };

Since: 6.5

Methods

ondata
Called when new data arrives to the sink. 
void ondata(DOMString sinkName, TensorsData data);

Since: 6.5

Parameters:

  • sinkName: Name of the sink.
  • data: The data that arrived to the sink.

2.9. CustomFilter

The CustomFilter interface defines the callback registered with MachineLearningPipeline::registerCustomFilter(). 
  [Callback=FunctionOnly, NoInterfaceObject] interface CustomFilter {
    long filter(TensorsData input, TensorsData output);
  };

Since: 6.5

Methods

filter
Called when data to be processed arrives to the filter. 
long filter(TensorsData input, TensorsData output);

Since: 6.5

This user-defined callback transforms the input and writes the transformation result to output. output contents will be passed to the further stages of the pipeline.

Remark: The input and output are only valid within the callback. They are automatically disposed after callback's return. To use their contents after callback return, copy them to objects valid outside the callback.

Remark: The input and output cannot be disposed. Calling the TensorsData::dispose()has no effect.

Remark: The input cannot be modified. Calling TensorsData::setTensorRawData() on input has no effect.

Remark: The output passed to the callback contains random values. Use TensorsData::setTensorRawData() to modify it.

Remark: Returning a negative status value stops the pipeline and changes its state to UNKNOWN.

Remark: The only legal positive return value is 1. On an attempt of returning another positive status value, the errorCallback registered with MachineLearningPipeline::registerCustomFilter() will report an error and the pipeline will stop as if the callback would return -1.

Remark: Usage example can be found in MachineLearningPipeline::registerCustomFilter()'s code example.

Parameters:

  • input: The data passed to filter to be processed.
  • output: Filter's output.

Return value:

    long: The status of data processing. This should be 0 when processing is successful, 1 when current data should be ignored or a negative value to report an error.

3. Related Feature

Method tizen.systeminfo.getCapability() can be used in application runtime to check whether this API is supported.

To guarantee the running of the application on a device with Machine Learning feature, declare the following feature requirement in the config file:

  • http://tizen.org/feature/machine_learning

    • To guarantee the running of the application on a device with Machine Learning Inference feature, declare the following feature requirement in the config file:

  • http://tizen.org/feature/machine_learning.inference

    • For more information, see Application Filtering.

    4. Full WebIDL

    module Pipeline {
  typedef (boolean or DOMString or long or long long or double or unsigned long or unsigned long long) Property;
  enum PipelineState { "NULL", "PAUSED", "PLAYING", "READY", "UNKNOWN" };
  enum PropertyType { "BOOLEAN", "DOUBLE", "ENUM", "INT32", "INT64", "UINT32", "UINT64", "STRING" };
  enum SwitchType { "INPUT_SELECTOR", "OUTPUT_SELECTOR" };
  [NoInterfaceObject] interface MachineLearningPipeline {
    Pipeline createPipeline(DOMString description, optional PipelineStateChangeListener? listener) raises(WebAPIException);
    void registerCustomFilter(DOMString filterName, CustomFilter filter, TensorsInfo inputInfo, TensorsInfo outputInfo,
                              optional ErrorCallback? errorCallback) raises(WebAPIException);
    void unregisterCustomFilter(DOMString filterName) raises(WebAPIException);
  };
  [NoInterfaceObject] interface Pipeline {
    readonly attribute PipelineState state raises(WebAPIException);
    void start() raises(WebAPIException);
    void stop() raises(WebAPIException);
    void dispose() raises(WebAPIException);
    NodeInfo getNodeInfo(DOMString name) raises(WebAPIException);
    Source getSource(DOMString name) raises(WebAPIException);
    Switch getSwitch(DOMString name) raises(WebAPIException);
    Valve getValve(DOMString name) raises(WebAPIException);
    void registerSinkListener(DOMString sinkName, SinkListener sinkListener) raises(WebAPIException);
    void unregisterSinkListener(DOMString sinkName) raises(WebAPIException);
  };
  [NoInterfaceObject] interface NodeInfo {
    readonly attribute DOMString name;
    Property getProperty(DOMString name, PropertyType type) raises(WebAPIException);
    void setProperty(DOMString name, PropertyType type, Property value) raises(WebAPIException);
  };
  [NoInterfaceObject] interface Source {
    readonly attribute TensorsInfo inputTensorsInfo raises(WebAPIException);
    readonly attribute DOMString name;
    void inputData(TensorsData data) raises(WebAPIException);
  };
  [NoInterfaceObject] interface Switch {
    readonly attribute SwitchType type;
    readonly attribute DOMString name;
    DOMString[] getPadList() raises(WebAPIException);
    void select(DOMString padName) raises(WebAPIException);
  };
  [NoInterfaceObject] interface Valve {
    readonly attribute DOMString name;
    readonly attribute boolean isOpen raises(WebAPIException);
    void setOpen(boolean open) raises(WebAPIException);
  };
  [Callback=FunctionOnly, NoInterfaceObject] interface PipelineStateChangeListener {
    void onstatechange(PipelineState newState);
  };
  [Callback=FunctionOnly, NoInterfaceObject] interface SinkListener {
    void ondata(DOMString sinkName, TensorsData data);
  };
  [Callback=FunctionOnly, NoInterfaceObject] interface CustomFilter {
    long filter(TensorsData input, TensorsData output);
  };
};
    Except as noted, this content - excluding the Code Examples - is licensed under Creative Commons Attribution 3.0 and all of the Code Examples contained herein are licensed under BSD-3-Clause.
    For details, see the Content License.