ID3 is a metadata container for both media files and media streams. When I worked on it for the first time, I was a bit confused about that. In this blogpost and in Ant Media Server, we are using ID3 as timed metadata for the media streams.

We will explore how to add ID3 timed metadata to HLS streams and discuss its applications. In general, timed metadata helps us to create some interactivity on HLS streams, even if there is an 8–10 second latency between publisher and players.

Actually, it is easy to create interactivity with real-time streaming protocols like WebRTC. Because there is very low video/audio latency (less than 500ms in Ant Media Server), it also provides a data channel to send/receive text or binary data. However, HLS introduces a delay of 8–10 seconds, which limits interactivity. But it is still possible to create some interactivity in HLS streams. Timed metadata is a solution to overcome the latency challenge.

Ant Media Server (AMS) which is a WebRTC streaming server, also supports HLS and ID3 timed metadata. Lets start and see how and where we can use ID3 time metadata in AMS.

What is Timed Metadata?

Timed metadata provides information to viewers to receive at the correct time. This information may include overlaying text or images for comments, emojis, ads, markers, etc. This information is presented as ID3 tags in HLS. It carries text-based data. So you can add JSON data for enhanced messaging and effect capabilities on the viewer’s.

You can use different applications by using timed-metada in the HLS streams. HQ trivia, like online quiz applications, is a good use case for using timed metadata. In such applications, the speaker asks the question, and at the end of the question, multiple choices appear on the screen. So these choices have to appear on the screen at the point where the speaker finishes the question. So you may add metadata to show the choices at that point. This ensures that all viewers, despite playback time differences, see the choices at the same point in the video.

Another use case is adding effects or messages at specific points in the stream. By incorporating timed metadata, everyone sees the effect at the designated moment. For example, you may add a smiling face emoji 😆at a funny moment of the stream. Similarly, with an extended JSON text, you may also define the position where the action will be displayed. For example, you can add a pointer to an object on the screen for the on-going stream.

How to Add ID3 Timed Metadata Step by Step?

Step 1: Enabling ID3 Tags

To use ID3 tags, it’s necessary to enable it for the streaming web application used. This can be done in the Advanced settings on the Ant Media Server Web Panel by setting “id3TagEnabled” to true under application settings.

Step 2: Create a Stream on AMS

Lets create a WebRTC stream on AMS by using the default WebRTC publish page. To do this, open the following page in your browser and click publish.

https://{YOUR_SERVER_DOMAIN}:5443/LiveApp

Step 3: Adding ID3 timed medatada

You can insert ID3-timed metadata into an ongoing stream by calling the REST method and providing the text-based metadata in the call. Here is a sample cURL command is provided for reference.

export MY_ANT_MEDIA_SERVER=localhost
export APP_NAME=LiveApp
export STREAM_ID=stream1
export TEXT=hello

curl -i -X POST -H "Accept: Application/json" -H "Content-Type: application/json" "http://${MY_ANT_MEDIA_SERVER}:5080/${APP_NAME}/rest/v2/broadcasts/${STREAM_ID}/id3" -d "${TEXT}"

Step 4: Playing HLS with ID3 timed metadata

Note: Currently ID3 Tags does not work with Ant Media Server default player (play.html).

You can use the provided Codepen sample for testing purposes. You need to provide an HLS stream URL, as in the example code and start playing. You will see the metadata text you provided on the player. 

Conclusion

It is easy to create interactive applications if you are using real-time streaming protocols like WebRTC. Furthermore, by leveraging ID3 tags, you can add timed metadata to HLS streams, enabling interactive applications even in delayed broadcasts. This feature enhances user engagement and opens up new possibilities for creative content delivery.

Ant Media Server both supports the real-time streaming protocol WebRTC and also HLS with the traditional delay of 8–10 seconds. You are free to choose the appropriate protocol to create your application. If you want to learn more about these streaming protocols, please take a look at this blogpost.