WebRTC WebSocket Messaging Reference

This documentation is for developers who needs to implement signalling between Ant Media Server and clients for publishing & playing streams. Let's make it step by step

Publishing WebRTC Stream

To connect to Ant Media Server, client can use WebSocket with a URL in the format:

wss://SERVER_NAME:5443/WebRTCAppEE/websocket

wss indicates that it's a secure WebSocket connection using SSL/TLS. /WebRTCAppEE/websocket specifies the endpoint for the WebSocket connection in Ant Media Server.

To publish a stream, clients first sends the publish command to the server to start the stream.

{
  command : "publish",
  streamId : "stream1",
  streamName : "streamName",
  token : "token",
  mainTrack : "mainTrack",
  metaData : "metaData",
  subscriberCode : "subscriberCode",
  subscriberId : "subscriberId",
  video : "true",
  audio : "true",
}

token: The token field is required if any stream security (token control) is enabled. If the user has enabled stream-security, they need to fill in the token field with the correct token.
subscriberId and subscriberCode: These are the values for the Time-based One-time Password (TOTP). If the user is using the TOTP mechanism, they need to pass the subscriberId and subscriberCode.
streamName: Zombie streams are streams that aren't in the database. Therefore, users can give these "on the fly" streams a streamName.
mainTrack: mainTrack is related to multitrack streaming and is required if the user wants to start the stream as a subtrack for this mainTrack. For multitrack conferences, mainTrack is set as the room ID.
metaData: The metaData is free text information for the stream to server.
enableVideo and enableAudio: These parameters define whether to enable video and audio for the stream. If enableVideo is true, then video will be sent to the server. If enableAudio is true, then audio will be sent to the server. If enableVideo is false and enbleAudio is true, then it means it's an audio-only stream.
Only command and streamId are mandatory. Audio and video is enabled default. When the server receives the publish commands, it checks whether the license is suspended, whether the server has enough resources, and retrieves the token if necessary.

If Server accepts the stream, it replies back with start command

{
  command : "start",
  streamId : "stream1",
  subscriberId : "",
}

Client inits peer connections, creates offer sdp and send the sdp configuration to the server with takeConfiguration command

{
  command : "takeConfiguration",
  streamId : "stream1",
  type : "offer",  
  sdp : "${SDP_PARAMETER}"
}

Server creates answer sdp and send the sdp configuration to the client with takeConfiguration command

{
  command : "takeConfiguration",
  streamId : "stream1",
  type : "answer",  
  sdp : "${SDP_PARAMETER}"
}

Client and Server get ice candidates several times and sends to each other with takeCandidate command

{
  command : "takeCandidate",
  streamId : "stream1",
  label : "${CANDIDATE.SDP_MLINE_INDEX}",
  id : "${CANDIDATE.SDP_MID}",
  candidate : "${CANDIDATE.CANDIDATE}"
}

After a stream has started, the server sends a publish_started command

{
  command : "notification",
  definition : "publish_started",
  streamId : "stream1",
}

Clients sends stop JSON command to stop publishing

{
  command : "stop",
  streamId: "stream1",
}

Server responds with publish_finished message to indicate that the stream has stopped

{
  command : "notification",
  definition : "publish_finished",
  streamId : "stream1",
  subscriberId : "subscriberId",
}

Playing WebRTC Stream

To connect to Ant Media Server, client can use WebSocket with a URL in the format:

wss://SERVER_NAME:5443/WebRTCAppEE/websocket

Client sends play command to the server with streamId parameter.

{
  command : "play",
  streamId : "stream1",
  token : "token",
  subscriberCode : "subscriberCode",
  subscriberId : "subscriberId",
  trackList : [enabledtracksarray],
  viewerInfo : "viwerInfo",
}

Only command and streamId is mandatory, rest are situational, such as subscriber id, code, token and enabled tracks.
If stream has subtracks, trackList is enabled by default. If there are 2 tracks on the stream, user can specify the both and both track will be played. To get all tracks in a stream you can take a look in getTrackList command that is in the miscellaneous part.
viewerInfo is a kind of metaData used to collect informations.

If Server accepts the stream, it replies back with offer command

{
  command : "takeConfiguration",
  streamId : "stream1",
  type : "offer",  
  sdp : "${SDP_PARAMETER}"
}

Client creates answer sdp and send the sdp configuration to the server with takeConfiguration command

{
  command : "takeConfiguration",
  streamId : "stream1",
  type : "answer",  
  sdp : "${SDP_PARAMETER}"
}

Client and Server get ice candidates several times and sends to each other with takeCandidate command

{
  command : "takeCandidate",
  streamId : "stream1",
  label : "${CANDIDATE.SDP_MLINE_INDEX}",
  id : "${CANDIDATE.SDP_MID}",
  candidate : "${CANDIDATE.CANDIDATE}"
}

Server notifies with play_started once the stream starts to play

{
  command : "notification",
  definition : "play_started",
  streamId : "stream1",
  subscriberId : "subscriberId",
}

Client sends toggle video to stop/start incoming video packets from a video track ( streamId = trackId for single track use ) Enabled is true as default. trackId and streamId is mandatory.

{
  command : "toggleVideo",
  streamId: "stream1",
  trackId: "track1",
  enabled: boolean
}

Client sends toggle audio to stop/start incoming audio packets from an audio track ( streamId = trackId for single track use ) Enabled is true as default. trackId and streamId is mandatory.

{
  command : "toggleAudio",
  streamId: "stream1",
  trackId: "track1",
  enabled: boolean
}

If there are mutiple subtracks, user can enabled/disable any track using toggle so that the server does not send audio/video packets for that track.

Clients sends stop JSON command to stop playing

{
  command : "stop",
  streamId: "stream1",
}

Server notifies with play_finished to notify that play has stopped

{
  command : "notification",
  definition : "play_finished",
  streamId : "stream1",
  subscriberId : "subscriberId",
}

Conference WebRTC Stream

Peers connects to Ant Media Server through WebSocket.

wss://SERVER_NAME:5443/WebRTCAppEE/websocket

Client sends joinRoom JSON command to the server with room name parameter.

{
  command : "joinRoom",
  room : "room1",
  streamId : "stream_id_you_want_to_use",
  mode : "multitrack"
}

roomName is the roomId and it acts as mainTrack
streamId represents the desired streamId that the client wishes to use to publish his stream to the room and it is an optional field. If it is not sent, server returns with a random streamId in the next message.
mode should be multitrack

Server notifies the client with available streams in the room

{
  command : "notification",
  definition : "joinedTheRoom",
  room : "room1",
  streamId : "unique_stream_id_returned_by_the_server"
  streams: [ stream_id_1, stream_id_2, ...]
}

streamId returned by the server is the stream id client uses to publish stream to the room.
streams is the json array which client can play via WebRTC. Client can play each stream by play method above. This streams array can be empty if there is no stream in the room.

Client publish the stream with publish command as we discussed in publish section above and these will be same.

{
  command : "publish",
  streamId : "streamId",
  streamName : "streamName",
  mainTrack : "room1",
  metaData : "metaData",
  subscriberCode : "subscriberCode",
  subscriberId : "subscriberId",
  video : "true",
  audio : "true",
}

mainTrack and roomId are being used interchangeably for multitrack conference. In near future, we'll change these to broadcasts and subTracks for better understanding and clarity.

Client sends the play command to the server with streamId as the roomId

{
  command : "play",
  room : "room1",
  streamId : "room1",
  token : "token",
  subscriberCode : "subscriberCode",
  subscriberId : "subscriberId",
  trackList : [enabledtracksarray],
  viewerInfo : "viwerInfo",
}

We only play the roomId as this mainTrack has all the subTracks in the room and therefore it is not required to play each streamId separately.

Web app should pull the server periodically for the room info as follows

{
  command : "getRoomInfo",
  room : "room1",
  streamId: "unique_stream_id_returned_by_the_server",
}

Server returns the active streams in the room as follows. Application should synchronize the players in their side.

{
  command: "roomInformation",
  room: "room1",
  streams: [
    "stream1_in_the_room",
    "stream2_in_the_room",
    ...
  ]
}

When a client wants to leave from the room they send leaveFromRoom command to the server.

{
  command : "leaveFromRoom",
  room: "room1",
}

Server responds back with leavedFromRoom message.

{
  command : "notification",
  definition : "leavedFromRoom",
}

The JavaScript SDK/SDKs handle the background processing of multitrack streaming for conferences on their own. If you are implementing your own code using WebSocket references, it is likely that you will need to listen for onTrackEvents within the initPeerConnections function. When a new track, stream, or subTrack is dynamically added to the room during runtime, the onTrack(event, streamId) function is triggered. This function notifies the application that a new track is available, allowing the application to handle and play the newly added track as needed.
When a new streamId is added to or removed from the room, the server and client initiate a renegotiation process. During this process, the server sends a new Session Description Protocol (SDP) to the client, suggesting a change in the configuration. This change prompts the addition or removal of a new track to or from the room in real-time.

Peer to Peer WebRTC Stream

Peers connects to Ant Media Server through WebSocket.

wss://SERVER_NAME:5443/WebRTCAppEE/websocket

Client sends join JSON command to the server with streamId parameter. If only want to play, mode can be set to play, if user wants to publish and play at the same time, both can be set. As default, mode is set to both. Only command and streamId are mandatory.

{
  command : "join",
  streamId : "stream1",
  mode: "play or both"
}

Server notifies with joined.

{
  command : "notification",
  definition : "joined"
  streamId : "stream1",
}

If there is only one peer in the stream1, server waits for the other peer to join the room.
When second peer joins the stream, server sends start JSON command to the first peer.

{
  command : "start",
  streamId : "stream1",
}

First peer create offer sdp and send to the server with takeConfiguration command.

{
  command : "takeConfiguration",
  streamId : "stream1",
  type : "offer",  
  sdp : "${SDP_PARAMETER}"
}

Server relays the offer sdp to the second peer

Second peer creates answer sdp and sends to the server with takeConfiguration command.

{
  command : "takeConfiguration",
  streamId : "stream1",
  type : "answer",  
  sdp : "${SDP_PARAMETER}"
}

Server relays the answer sdp to the first peer

Each peers get ice candidates several times and sends to each other with takeCandidate command through server.

{
  command : "takeCandidate",
  streamId : "stream1",
  label : "${CANDIDATE.SDP_MLINE_INDEX}",
  id : "${CANDIDATE.SDP_MID}",
  candidate : "${CANDIDATE.CANDIDATE}"
}

Clients sends leave JSON command to leave the room.

{
  command : "leave",
  streamId: "stream1"
}

Server notifies with leaved

{
  command : "notification",
  definition : "leaved",
}

When second peer stops the stream or stream is ended, server sends stop JSON command to the first peer.

{
  command : "stop",
  streamId : "stream1",
}

WebSocket Error Callbacks

noStreamNameSpecified: it is sent when stream id is not specified in the message.

{
  command : "error",
  definition : "noStreamNameSpecified",
}

not_allowed_unregistered_streams: The not_allowed_unregistered_streams error message is returned to the user when they try to send a stream with an unregistered streamId, and server is configured not to accept undefined streams. For conference, if it is not set then the room is created automatically and is removed once it finishes.

{
  command : "error",
  definition : "not_allowed_unregistered_streams",
  streamId : "stream1",
}

no_room_specified: This is sent back to the user when there is no room specified in joining the video conference.

{
  command : "error",
  definition : "no_room_specified",
}

unauthorized_access: When stream security is enabled but the token is either incorrect or not validated, or the subscriberId and/or subscriberCode are incorrect.

{
  command : "error",
  definition : "unauthorized_access",
  streamId: "stream1",
}

no_encoder_settings: This is sent back to the user when there are no encoder settings available in publishing the stream.

{
  command : "error",
  definition : "no_encoder_settings",
}

no_peer_associated_before: This is peer to peer connection error definition.It is sent back to the user when there is no peer associated with the stream.

{
  command : "error",
  definition : "no_peer_associated_before",
}

notSetRemoteDescription: This is sent if there is a mismatch between the encoder and the decoders between the Ant Media Server and the client.

{
  command : "error",
  definition : "notSetRemoteDescriptio",
}

notSetLocalDescription: It is sent when local description is not set successfully.

{
  command : "error",
  definition : "notSetLocalDescription",
}

highResourceUsage: It is sent when server is overloaded. Server overload means over CPU usage or over RAM usage. Over CPU usage means CPU load is more than the server.cpu_limit value in conf/red5.properties. Its default value is %75. Over RAM usage means available memory in the system is less than server.min_free_ram value in conf/red5.properties. Its unit is MB and default value is 10.

{
  command : "error",
  definition : "highResourceUsage",
}

streamIdInUse: This error message is returned by the server when a user tries to publish a stream with a streamId that is already in use by an active stream in either the preparing or publishing state. This error can occur if a user attempts to re-publish a stream with the same streamId without first closing the previous WebRTC connection.

{
  command : "error",
  definition : "streamIdInUse",
  streamId : "stream1",
}

publishTimeoutError: The server sends the publishTimeoutError message when WebRTC publishing fails to start within a specified time period.

This may occur due to network issues, such as the lack of an established ICE connection or the failure to send video and audio streams to the server. The timeout value can be customized using the settings.webrtc.client.start.timeoutMs property in the App-Configuration, with a default value of 5000 milliseconds.

Using a TURN-server helps mitigate these network-related issues.

{
  command : "error",
  definition : "publishTimeoutError",
  streamId : "stream1",
}

invalidStreamName: it is send when stream name contains special characters.

{
  command : "error",
  definition : "invalidStreamName",
}

data_store_not_available: It's sent when data store is not available. It's not available if it's not initialized or closed.

{
  command : "error",
  definition : "data_store_not_available"
}

license_suspended_please_renew_license: It's send when license is suspended

{
  command : "error",
  definition : "license_suspended_please_renew_license"
}

already_playing: This is sent back to the user when a new play message received while it is playing or it is about to play.

{
  command : "error",
  definition : "already_playing",
  streamId: "stream1"
}

already_publishing: This message is sent when a new publish message is received while the server is either publishing or about to publish. The message indicates that publishing is already in progress.

{
  command : "error",
  definition : "already_publishing",
  streamId: "stream1",
}

encoderNotOpened: If the encoder fails to open, the server sends this error message to the client indicating that the encoder could not be opened.

{
  command : "error",
  definition : "encoderNotOpened",
  streamId : "stream1", 
}

encoderBlocked: The server sends this error message if the encoder is blocked for some reason or not performing efficiently.

{
  command : "error",
  definition : "encoderBlocked",
  streamId : "stream1",
}

no_codec_enabled_in_the_server: This is sent back to the user when there is no codec enabled in the server and someone try to make a publish

{
  command : "error",
  definition : "no_codec_enabled_in_the_server",
  streamId: "stream1"
}

stream_not_active_or_expired: The message stream_not_active_or_expired is returned to the user when the plannedStartDate and plannedEndDate of a stream are either not in the specified interval or have expired.

{
  command : "error",
  definition : "stream_not_active_or_expired",
  streamId : "stream1",
}

viewerLimitReached: This is send when viewer limit reached. when user is trying to watch a broadcast that already hit the limit, this error is send to client.

{
  command : "error",
  definition : "viewerLimitReached",
}

no_room_specified: if roomId is not specified for a conference, server returns with "no_room_speficified" error message.

{
  command : "error",
  definition : "no_room_specified",
}

Miscellaneous WebSocket Methods

ping & pong Some load balancers may start to close connections after a certain amount of time to prevent idle connections from consuming resources.

To prevent this from happening, the client sends ping messages to the server, and server returns with a pong response. This keeps the connection active and prevents it from being closed by the load balancer.

{
command : "ping",
}

Pong Response from Server

{
command : "pong",
}

getStreamInfo: Get Stream Information from Server. You may use this method to learn more about stream status and bitrates. Client should send the following message.

{
 command: "getStreamInfo",
 streamId: "stream_id_that_you_want_to_get_info",
} 

Server returns in two ways. It may return stream information as follows

{
 command: "streamInformation",
 streamId: "stream_id_of_the_stream_information",
 streamInfo: [{
     streamWidth: "resolution_width",
     streamHeight: "resolution_height",
     videoBitrate: "video_bitrate",
     audioBitrate: "audio_bitrate",
     videoCodec: "codec_of_the_video", 
     },
   ...
   ]
}

If stream is not active, it will return no_stream_exist

{
command : "error",
definition : "no_stream_exist",
streamId: "id_of_the_stream",
}

getRoomInfo: The function getRoomInfo is called when a new participant or track is added to the room in order to retrieve information about the current state of the room, including the active streams. Client should send the following message to get the response from the server.

{
 command: "getRoomInfo",
 room: "room_id_that_you_want_to_get_info",
 streamId: "server_returns_while_you_join_the_room",
} 

Server responds in following format with the list of streams available in the room

{
 command: "roomInformation",
 room: "room_id_that_this_information_belongs_to",
 streams: [ stream_id_1, stream_id_2, ...]
}

bitrateMeasurement: Server periodically sends this information to the WebRTC viewers. It lets develop show a message to the user if it's internet bandwidth is not good enough. If the targetBitrate is bigger than the sum of videoBitrate and audioBitrate, it means internet bandwidth is good enough to play the video. If the targetBitrate is less than the sum of videoBitrate and audioBitrate, it means some playback issues(pixelating, packet drop, etc.) may happen and it disturbs the user experience.

{
  command : "notification",
  definition : "bitrateMeasurement",
  streamId: "unique_stream_id_returned_by_the_server"
  targetBitrate: "measured_bandwidth_of_the_client",
  videoBitrate: "video_bitrate_of_the_current_playing_video",
  audioBitrate: "audio_bitrate_of_the_current_playing_audio",
}

forceStreamQuality: If there are adaptive-bitrates(multi-bitrate) for that stream, you can get bitrates with getStreamInfo method and then you can make the Ant Media Server force to send you a specific resolution. If you want to switch back to auto stream quality, you can use give 0 for streamHeight and send the message below.

{
command: "forceStreamQuality",
streamId: "write_the_stream_id",
streamHeight: "write_the_height_of_the_resolution_you_want_to_force",
}

server_will_stop: When the server is about to stop and it receives a command from a service or the command line to initiate the shutdown process, it notifies the user with server_will_stop

{ 
command : "notification",
definition : "server_will_stop",
}

leavedFromRoom: It's sent after stop command received or if client sents leaveFromRoom command.

{        
command : "notification",
definition : "leavedFromRoom",
ATTR_ROOM_NAME: "roomName",
}

getTrackList: Sends a request to server to get track list in specified stream. Token is not mandatory. If stream has token, token needs to be used, otherwise not needed.

{ 
command : "getTrackList",
streamId: "stream1",
token: "token",
}

trackList: Server returns track list of the specified stream after receiving getTrackList command.

{ 
command : "trackList",
streamId: "stream1",
trackList: "tracks",
}

enableTrack: Player can enable or disable the tracks in the broadcast.

{
command : "enableTrack",
streamId: "stream1",
trackId: "id of track",
enabled: "boolean",
}

streaming_Session_Restored: when a WebRTC publish stream is interrupted due to network issues but is restored within the timeout duration, the server sends this message to indicate that the streaming session has been restored. The viewers connection remains active and are not dropped during the timeout duration.The timeout duration, webRTCClientStartTimeoutis configurable and has a default value of 10 seconds.

{
  command : "notification",
  description: "streaming_session_Restored",
}

Publishing WebRTC Stream​

Playing WebRTC Stream​

Conference WebRTC Stream​

Peer to Peer WebRTC Stream​

WebSocket Error Callbacks​

Miscellaneous WebSocket Methods​