Tizen Native API  8.0
Media Render

The Media Render API provides functions to render audio or video stream received from the remote peer.

Required Header

#include <webrtc.h>

Overview

The WebRTC Media Render API allows you to:

  • set a sound policy including routing and volume to the audio stream received from the remote peer
  • set a display type and a surface to render the video stream received from the remote peer
  • choose not to render the audio or video stream itself but forward it via the callback function with media packet
  • enable loopback rendering of the media sources

Related Features

This API is related with the following features:

  • http://tizen.org/feature/display (Since 8.0)

It is recommended to design feature related codes in your application for reliability.
You can check if a device supports the related features for this API by using System Information, thereby controlling the procedure of your application.
To ensure your application is only running on the device with specific features, please define the features in your manifest file using the manifest editor in the SDK.
More details on featuring your application can be found from Feature Element.

Functions

int webrtc_set_sound_stream_info (webrtc_h webrtc, unsigned int track_id, sound_stream_info_h stream_info)
 Sets a sound manager stream information to the audio track to be rendered.
int webrtc_set_audio_mute (webrtc_h webrtc, unsigned int track_id, bool mute)
 Sets mute to the audio track.
int webrtc_get_audio_mute (webrtc_h webrtc, unsigned int track_id, bool *muted)
 Gets the mute state of the audio track.
int webrtc_set_display (webrtc_h webrtc, unsigned int track_id, webrtc_display_type_e type, webrtc_display_h display)
 Sets a display to the video track to be rendered.
int webrtc_set_display_mode (webrtc_h webrtc, unsigned int track_id, webrtc_display_mode_e mode)
 Sets the display mode of the video track.
int webrtc_get_display_mode (webrtc_h webrtc, unsigned int track_id, webrtc_display_mode_e *mode)
 Gets the display mode of the video track.
int webrtc_set_display_visible (webrtc_h webrtc, unsigned int track_id, bool visible)
 Sets the display visibleness of the video track.
int webrtc_get_display_visible (webrtc_h webrtc, unsigned int track_id, bool *visible)
 Gets the display visibleness of the video track.
int webrtc_set_encoded_audio_frame_cb (webrtc_h webrtc, webrtc_encoded_frame_cb callback, void *user_data)
 Sets an encoded audio frame callback function to be invoked when each audio frame is ready to be rendered.
int webrtc_unset_encoded_audio_frame_cb (webrtc_h webrtc)
 Unsets the encoded audio frame callback function.
int webrtc_set_encoded_video_frame_cb (webrtc_h webrtc, webrtc_encoded_frame_cb callback, void *user_data)
 Sets an encoded video frame callback function to be invoked when each video frame is ready to be rendered.
int webrtc_unset_encoded_video_frame_cb (webrtc_h webrtc)
 Unsets the encoded video frame callback function.
int webrtc_media_source_set_audio_loopback (webrtc_h webrtc, unsigned int source_id, sound_stream_info_h stream_info, unsigned int *track_id)
 Sets an audio loopback to render the audio frames of the media source.
int webrtc_media_source_unset_audio_loopback (webrtc_h webrtc, unsigned int source_id)
 Unsets the audio loopback.
int webrtc_media_source_set_video_loopback (webrtc_h webrtc, unsigned int source_id, webrtc_display_type_e type, webrtc_display_h display, unsigned int *track_id)
 Sets a video loopback to render the video frames of the media source.
int webrtc_media_source_unset_video_loopback (webrtc_h webrtc, unsigned int source_id)
 Unsets the video loopback.

Typedefs

typedef void * webrtc_display_h
 WebRTC display handle type.
typedef void(* webrtc_encoded_frame_cb )(webrtc_h webrtc, webrtc_media_type_e type, unsigned int track_id, media_packet_h packet, void *user_data)
 Called when each audio or video frame is ready to be rendered via the WebRTC pipeline after the negotiation.

Typedef Documentation

typedef void* webrtc_display_h

WebRTC display handle type.

Since :
6.5
Remarks:
The pointer of this handle can be obtained from EFL UI API.
For example, in case of WEBRTC_DISPLAY_TYPE_OVERLAY, elm_win_add() can be used to get the window handle.
In case of WEBRTC_DISPLAY_TYPE_EVAS, evas_object_image_add() can be used to get the renderable image handle.
For more details, please refer to https://docs.tizen.org/application/native/guides/ui/efl/.
typedef void(* webrtc_encoded_frame_cb)(webrtc_h webrtc, webrtc_media_type_e type, unsigned int track_id, media_packet_h packet, void *user_data)

Called when each audio or video frame is ready to be rendered via the WebRTC pipeline after the negotiation.

Since :
6.5
Remarks:
The webrtc is the same object for which the callback was set.
The webrtc should not be released.
Use media_packet_get_buffer_data_ptr() with packet to get the Gstreamer buffer pointer.
The packet should be released using media_packet_destroy().
Parameters:
[in]webrtcWebRTC handle
[in]typeThe media type
[in]track_idThe track id
[in]packetThe media packet which has a frame data
[in]user_dataThe user data passed from the callback registration function
See also:
webrtc_set_encoded_audio_frame_cb()
webrtc_unset_encoded_audio_frame_cb()
webrtc_set_encoded_video_frame_cb()
webrtc_unset_encoded_video_frame_cb()
media_packet_get_buffer_data_ptr()

Enumeration Type Documentation

Enumeration for WebRTC display mode.

Since :
6.5
Enumerator:
WEBRTC_DISPLAY_MODE_LETTER_BOX 

Letter box

WEBRTC_DISPLAY_MODE_ORIGIN_SIZE 

Origin size

WEBRTC_DISPLAY_MODE_FULL 

Full screen

Enumeration for WebRTC display type.

Since :
6.5
Enumerator:
WEBRTC_DISPLAY_TYPE_OVERLAY 

Overlay

WEBRTC_DISPLAY_TYPE_EVAS 

Evas image object


Function Documentation

int webrtc_get_audio_mute ( webrtc_h  webrtc,
unsigned int  track_id,
bool *  muted 
)

Gets the mute state of the audio track.

Since :
8.0
Remarks:
The default value is false.
Parameters:
[in]webrtcWebRTC handle
[in]track_idThe track id
[out]mutedMuted or not (true = muted, false = not muted)
Returns:
0 on success, otherwise a negative error value
Return values:
WEBRTC_ERROR_NONESuccessful
WEBRTC_ERROR_INVALID_PARAMETERInvalid parameter
WEBRTC_ERROR_INVALID_OPERATIONInvalid operation
Precondition:
webrtc_track_added_cb() must be set by calling webrtc_set_track_added_cb().
Call webrtc_set_sound_stream_info() before calling this function.
See also:
webrtc_set_audio_mute()
int webrtc_get_display_mode ( webrtc_h  webrtc,
unsigned int  track_id,
webrtc_display_mode_e mode 
)

Gets the display mode of the video track.

Since :
6.5
Remarks:
The default value is WEBRTC_DISPLAY_MODE_LETTER_BOX.
The display feature(http://tizen.org/feature/display) is required. (Since 8.0)
Parameters:
[in]webrtcWebRTC handle
[in]track_idThe track id
[out]modeThe display mode
Returns:
0 on success, otherwise a negative error value
Return values:
WEBRTC_ERROR_NONESuccessful
WEBRTC_ERROR_NOT_SUPPORTEDNot supported
WEBRTC_ERROR_INVALID_PARAMETERInvalid parameter
WEBRTC_ERROR_INVALID_OPERATIONInvalid operation
Precondition:
For remote video track, webrtc_set_display() must be called with track_id from webrtc_track_added_cb().
For loopback video track, webrtc_media_source_set_video_loopback() must be called to get track_id.
See also:
webrtc_set_display_mode()
int webrtc_get_display_visible ( webrtc_h  webrtc,
unsigned int  track_id,
bool *  visible 
)

Gets the display visibleness of the video track.

Since :
6.5
Remarks:
The default value is true.
The display feature(http://tizen.org/feature/display) is required. (Since 8.0)
Parameters:
[in]webrtcWebRTC handle
[in]track_idThe track id
[out]visibleThe display visibleness
Returns:
0 on success, otherwise a negative error value
Return values:
WEBRTC_ERROR_NONESuccessful
WEBRTC_ERROR_NOT_SUPPORTEDNot supported
WEBRTC_ERROR_INVALID_PARAMETERInvalid parameter
WEBRTC_ERROR_INVALID_OPERATIONInvalid operation
Precondition:
For remote video track, webrtc_set_display() must be called with track_id from webrtc_track_added_cb().
For loopback video track, webrtc_media_source_set_video_loopback() must be called to get track_id.
See also:
webrtc_set_display_visible()
int webrtc_media_source_set_audio_loopback ( webrtc_h  webrtc,
unsigned int  source_id,
sound_stream_info_h  stream_info,
unsigned int *  track_id 
)

Sets an audio loopback to render the audio frames of the media source.

The following media source types are available for this function:
WEBRTC_MEDIA_SOURCE_TYPE_AUDIOTEST
WEBRTC_MEDIA_SOURCE_TYPE_MIC
WEBRTC_MEDIA_SOURCE_TYPE_FILE

Since :
6.5
Remarks:
The following sound stream types can be used to create the stream_info :
SOUND_STREAM_TYPE_MEDIA
SOUND_STREAM_TYPE_VOIP
SOUND_STREAM_TYPE_MEDIA_EXTERNAL_ONLY
For more details, please refer to Sound Manager.
Parameters:
[in]webrtcWebRTC handle
[in]source_idThe audio source id
[in]stream_infoThe sound stream information
[out]track_idThe track id (optional, this can be NULL)
Returns:
0 on success, otherwise a negative error value
Return values:
WEBRTC_ERROR_NONESuccessful
WEBRTC_ERROR_INVALID_PARAMETERInvalid parameter
WEBRTC_ERROR_INVALID_OPERATIONInvalid operation
Precondition:
Add media source to webrtc to get source_id by calling webrtc_add_media_source().
See also:
webrtc_media_source_set_video_loopback()
int webrtc_media_source_set_video_loopback ( webrtc_h  webrtc,
unsigned int  source_id,
webrtc_display_type_e  type,
webrtc_display_h  display,
unsigned int *  track_id 
)

Sets a video loopback to render the video frames of the media source.

The following media source types are available for this function:
WEBRTC_MEDIA_SOURCE_TYPE_VIDEOTEST
WEBRTC_MEDIA_SOURCE_TYPE_CAMERA
WEBRTC_MEDIA_SOURCE_TYPE_SCREEN
WEBRTC_MEDIA_SOURCE_TYPE_FILE

Since :
6.5
Remarks:
The display feature(http://tizen.org/feature/display) is required. (Since 8.0)
Parameters:
[in]webrtcWebRTC handle
[in]source_idThe video source id
[in]typeThe display type
[in]displayThe display handle
[out]track_idThe track id (optional, this can be NULL)
Returns:
0 on success, otherwise a negative error value
Return values:
WEBRTC_ERROR_NONESuccessful
WEBRTC_ERROR_NOT_SUPPORTEDNot supported
WEBRTC_ERROR_INVALID_PARAMETERInvalid parameter
WEBRTC_ERROR_INVALID_OPERATIONInvalid operation
Precondition:
Add media source to webrtc to get source_id by calling webrtc_add_media_source().
See also:
webrtc_media_source_set_audio_loopback()
webrtc_set_display_mode()
webrtc_get_display_mode()
webrtc_set_display_visible()
webrtc_get_display_visible()
int webrtc_media_source_unset_audio_loopback ( webrtc_h  webrtc,
unsigned int  source_id 
)

Unsets the audio loopback.

Since :
7.0
Parameters:
[in]webrtcWebRTC handle
[in]source_idThe audio source id
Returns:
0 on success, otherwise a negative error value
Return values:
WEBRTC_ERROR_NONESuccessful
WEBRTC_ERROR_INVALID_PARAMETERInvalid parameter
WEBRTC_ERROR_INVALID_OPERATIONInvalid operation
Precondition:
Add media source to webrtc to get source_id by calling webrtc_add_media_source().
See also:
webrtc_media_source_set_audio_loopback()
int webrtc_media_source_unset_video_loopback ( webrtc_h  webrtc,
unsigned int  source_id 
)

Unsets the video loopback.

Since :
7.0
Remarks:
The display feature(http://tizen.org/feature/display) is required. (Since 8.0)
Parameters:
[in]webrtcWebRTC handle
[in]source_idThe video source id
Returns:
0 on success, otherwise a negative error value
Return values:
WEBRTC_ERROR_NONESuccessful
WEBRTC_ERROR_NOT_SUPPORTEDNot supported
WEBRTC_ERROR_INVALID_PARAMETERInvalid parameter
WEBRTC_ERROR_INVALID_OPERATIONInvalid operation
Precondition:
Add media source to webrtc to get source_id by calling webrtc_add_media_source().
See also:
webrtc_media_source_set_video_loopback()
int webrtc_set_audio_mute ( webrtc_h  webrtc,
unsigned int  track_id,
bool  mute 
)

Sets mute to the audio track.

Since :
8.0
Remarks:
If mute is set to true, playback of audio track received from a remote peer will be muted.
Parameters:
[in]webrtcWebRTC handle
[in]track_idThe track id
[in]muteMute or not (true = mute, false = not mute)
Returns:
0 on success, otherwise a negative error value
Return values:
WEBRTC_ERROR_NONESuccessful
WEBRTC_ERROR_INVALID_PARAMETERInvalid parameter
WEBRTC_ERROR_INVALID_OPERATIONInvalid operation
Precondition:
webrtc_track_added_cb() must be set by calling webrtc_set_track_added_cb().
Call webrtc_set_sound_stream_info() before calling this function.
See also:
webrtc_get_audio_mute()
int webrtc_set_display ( webrtc_h  webrtc,
unsigned int  track_id,
webrtc_display_type_e  type,
webrtc_display_h  display 
)

Sets a display to the video track to be rendered.

Since :
6.5
Remarks:
Call this function within webrtc_track_added_cb(), otherwise WEBRTC_ERROR_INVALID_OPERATION will be returned.
If webrtc_set_encoded_video_frame_cb() has been called, it will return WEBRTC_ERROR_INVALID_OPERATION.
The display feature(http://tizen.org/feature/display) is required. (Since 8.0)
Parameters:
[in]webrtcWebRTC handle
[in]track_idThe track id
[in]typeThe display type
[in]displayThe display handle
Returns:
0 on success, otherwise a negative error value
Return values:
WEBRTC_ERROR_NONESuccessful
WEBRTC_ERROR_NOT_SUPPORTEDNot supported
WEBRTC_ERROR_INVALID_PARAMETERInvalid parameter
WEBRTC_ERROR_INVALID_OPERATIONInvalid operation
Precondition:
webrtc_track_added_cb() must be set by calling webrtc_set_track_added_cb().
See also:
webrtc_set_track_added_cb()
webrtc_unset_track_added_cb()
webrtc_set_display_mode()
webrtc_get_display_mode()
webrtc_set_display_visible()
webrtc_get_display_visible()
int webrtc_set_display_mode ( webrtc_h  webrtc,
unsigned int  track_id,
webrtc_display_mode_e  mode 
)

Sets the display mode of the video track.

Since :
6.5
Remarks:
The display feature(http://tizen.org/feature/display) is required. (Since 8.0)
Parameters:
[in]webrtcWebRTC handle
[in]track_idThe track id
[in]modeThe display mode
Returns:
0 on success, otherwise a negative error value
Return values:
WEBRTC_ERROR_NONESuccessful
WEBRTC_ERROR_NOT_SUPPORTEDNot supported
WEBRTC_ERROR_INVALID_PARAMETERInvalid parameter
WEBRTC_ERROR_INVALID_OPERATIONInvalid operation
Precondition:
For remote video track, webrtc_set_display() must be called with track_id from webrtc_track_added_cb().
For loopback video track, webrtc_media_source_set_video_loopback() must be called to get track_id.
See also:
webrtc_get_display_mode()
int webrtc_set_display_visible ( webrtc_h  webrtc,
unsigned int  track_id,
bool  visible 
)

Sets the display visibleness of the video track.

Since :
6.5
Remarks:
The display feature(http://tizen.org/feature/display) is required. (Since 8.0)
Parameters:
[in]webrtcWebRTC handle
[in]track_idThe track id
[in]visibleThe display visibleness
Returns:
0 on success, otherwise a negative error value
Return values:
WEBRTC_ERROR_NONESuccessful
WEBRTC_ERROR_NOT_SUPPORTEDNot supported
WEBRTC_ERROR_INVALID_PARAMETERInvalid parameter
WEBRTC_ERROR_INVALID_OPERATIONInvalid operation
Precondition:
For remote video track, webrtc_set_display() must be called with track_id from webrtc_track_added_cb().
For loopback video track, webrtc_media_source_set_video_loopback() must be called to get track_id.
See also:
webrtc_get_display_visible()
int webrtc_set_encoded_audio_frame_cb ( webrtc_h  webrtc,
webrtc_encoded_frame_cb  callback,
void *  user_data 
)

Sets an encoded audio frame callback function to be invoked when each audio frame is ready to be rendered.

Since :
6.5
Remarks:
If callback is set, audio data from the remote peer will be forwarded to callback without being rendered by itself.
The registered callback will be invoked in an internal thread of the webrtc.
Parameters:
[in]webrtcWebRTC handle
[in]callbackCallback function pointer
[in]user_dataThe user data to be passed to the callback function
Returns:
0 on success, otherwise a negative error value
Return values:
WEBRTC_ERROR_NONESuccessful
WEBRTC_ERROR_INVALID_PARAMETERInvalid parameter
WEBRTC_ERROR_INVALID_STATEInvalid state
Precondition:
webrtc state must be set to WEBRTC_STATE_IDLE or WEBRTC_STATE_NEGOTIATING.
Postcondition:
webrtc_encoded_frame_cb() will be invoked.
See also:
webrtc_unset_encoded_audio_frame_cb()
webrtc_encoded_frame_cb()
int webrtc_set_encoded_video_frame_cb ( webrtc_h  webrtc,
webrtc_encoded_frame_cb  callback,
void *  user_data 
)

Sets an encoded video frame callback function to be invoked when each video frame is ready to be rendered.

Since :
6.5
Remarks:
If callback is set, video data from the remote peer will be forwarded to callback without being rendered by itself.
The registered callback will be invoked in an internal thread of the webrtc.
Parameters:
[in]webrtcWebRTC handle
[in]callbackCallback function pointer
[in]user_dataThe user data to be passed to the callback function
Returns:
0 on success, otherwise a negative error value
Return values:
WEBRTC_ERROR_NONESuccessful
WEBRTC_ERROR_INVALID_PARAMETERInvalid parameter
WEBRTC_ERROR_INVALID_STATEInvalid state
Precondition:
webrtc state must be set to WEBRTC_STATE_IDLE or WEBRTC_STATE_NEGOTIATING.
Postcondition:
webrtc_encoded_frame_cb() will be invoked.
See also:
webrtc_unset_encoded_video_frame_cb()
webrtc_encoded_frame_cb()
int webrtc_set_sound_stream_info ( webrtc_h  webrtc,
unsigned int  track_id,
sound_stream_info_h  stream_info 
)

Sets a sound manager stream information to the audio track to be rendered.

Since :
6.5
Remarks:
Call this function within webrtc_track_added_cb(), otherwise WEBRTC_ERROR_INVALID_OPERATION will be returned.
If webrtc_set_encoded_audio_frame_cb() has been called, it will return WEBRTC_ERROR_INVALID_OPERATION.
The following sound stream types can be used to create the stream_info :
SOUND_STREAM_TYPE_MEDIA
SOUND_STREAM_TYPE_VOIP
SOUND_STREAM_TYPE_MEDIA_EXTERNAL_ONLY
For more details, please refer to Sound Manager.
Parameters:
[in]webrtcWebRTC handle
[in]track_idThe track id
[in]stream_infoThe sound stream information
Returns:
0 on success, otherwise a negative error value
Return values:
WEBRTC_ERROR_NONESuccessful
WEBRTC_ERROR_INVALID_PARAMETERInvalid parameter
WEBRTC_ERROR_INVALID_OPERATIONInvalid operation
Precondition:
webrtc_track_added_cb() must be set by calling webrtc_set_track_added_cb().
See also:
webrtc_set_track_added_cb()
webrtc_unset_track_added_cb()

Unsets the encoded audio frame callback function.

Since :
6.5
Parameters:
[in]webrtcWebRTC handle
Returns:
0 on success, otherwise a negative error value
Return values:
WEBRTC_ERROR_NONESuccessful
WEBRTC_ERROR_INVALID_PARAMETERInvalid parameter
WEBRTC_ERROR_INVALID_OPERATIONInvalid operation
WEBRTC_ERROR_INVALID_STATEInvalid state
Precondition:
webrtc state must be set to WEBRTC_STATE_IDLE or WEBRTC_STATE_NEGOTIATING.
See also:
webrtc_set_encoded_audio_frame_cb()

Unsets the encoded video frame callback function.

Since :
6.5
Parameters:
[in]webrtcWebRTC handle
Returns:
0 on success, otherwise a negative error value
Return values:
WEBRTC_ERROR_NONESuccessful
WEBRTC_ERROR_INVALID_PARAMETERInvalid parameter
WEBRTC_ERROR_INVALID_OPERATIONInvalid operation
WEBRTC_ERROR_INVALID_STATEInvalid state
Precondition:
webrtc state must be set to WEBRTC_STATE_IDLE or WEBRTC_STATE_NEGOTIATING.
See also:
webrtc_set_encoded_video_frame_cb()