Revolutionize Your WebRTC Streaming Experience with Ant Media’s Deep AR Integration

Published by Usama Tahseen on

Try Live Demo

  DeepAR Integration

Augmented Reality (AR) technology has the potential to revolutionize the way we interact with the world around us. By overlaying digital content onto the physical environment, AR allows us to see and interact with information in a whole new way and can enhance our experiences using technology such as WebRTC video streaming.

The DeepAR JavaScript SDK is a powerful tool that allows developers to easily add AR capabilities to their projects. With just a few lines of code, developers can create immersive and engaging AR experiences for their users.

We have integrated DeepAR into Ant Media Server so that we can provide more realistic and engaging experiences for users.  By leveraging DeepAR’s SDK, we have implemented advanced facial AR filters and enabled interactive experiences.

How it Works ?

The DeepAR libraries make use of web assembly technology, a low-level programming language that is designed to be fast and efficient, to perform various media processing tasks, such as applying different video filters and effects to video streams in real-time.

Additionally, the use of web assembly technology enables the libraries to perform these tasks quickly and efficiently, without consuming large amounts of system resources.

Getting an API key:

1.  Signup to DeepAR by navigating to here.

2 Click on create a  project

Screenshot from 2023 04 01 23 25 21

3. Enter a project name (anything)

Screenshot from 2023 04 01 23 26 09

4. Select Web App

Screenshot from 2023 04 01 23 26 32

4. Enter the domain name where your Ant Media Server is hosted (Important)

Screenshot from 2023 04 01 23 26 46

5.Click on copy SDK key

Screenshot from 2023 04 01 23 26 59

Now we have an API key for the DeepAR SDK, we can use this key in our Ant Media Server.

Using DeepAR API Key in Antmedia Server

Publish video to AMS with DeepAR Effect

Navigate to the following Ant Media Server URL and add the copied API key as the value for the APIkey parameter:

https://domain:5443/LiveApp/publish_webrtc_deep_ar_effects.html?Apikey=REPLACE_YOUR_API_KEY_HERE

With DeepAR now enabled, a drop down menu is available so you can select different kinds of effects. Select an effect and click on the publish button.

How to Use DeepAR with JS SDK

The full code example is available here

  1. Get the API key for DeepAR SDK as described in the above steps under the getting API key section
  2. If required, you can fetch all the available effects using
  3. VideoEffect.deepARModelList
  4. After receiving all the available effects, you can apply an effect by calling the enableEffect method on your webrtcAdapter and passing the API key that you generated from the previous section.
  5. The full method call should look like this:
    webRTCAdaptor.enableEffect(VideoEffect.DEEPAR, "APIKey", VideoEffect.deepARModelList[0])
  6. It expects an API key and the name of the effect. In the example, the first effect from the effect list is being used
    VideoEffect.deepARModelList[0].
  7. Set the path of the directory which contains the .deepar files
  8.  VideoEffect.DEEP_AR_EFFECTS_URL=“/WebRTCAppEE/js/external/deepar-effects/”;

It is also possible to list all the available effects by using the JavaScript console.log method:

console.log(VideoEffect.deepARModelList[0])

 Code Example

<!DOCTYPE html>
<html lang=“en”>

<head>
<meta charset=“UTF-8”>
<meta http-equiv=“X-UA-Compatible” content=“IE=edge”>
<meta name=“viewport” content=“width=device-width, initial-scale=1.0”>
<title>Document</title>
<script type=“text/javascript” src=“https://cdn.jsdelivr.net/npm/deepar@4.0.3/js/deepar.js”></script>

</head>
<video id=“myvideo” autoplay muted></video>

<body>
Select Effects :
<select id=“select_effect”>
<option>No Effect</option>
</select>
<script type=“module”>
import { WebRTCAdaptor } from ‘./js/webrtc_adaptor.js’;
import { VideoEffect } from ‘./js/video-effect.js’;
VideoEffect.DEEP_AR_EFFECTS_URL = “/WebRTCAppEE/js/external/deepar-effects/”;
const webRTCAdaptor = new WebRTCAdaptor({
websocket_url: “ws://localhost:5080/WebRTCAppEE/websocket”,

mediaConstraints: {
video: true,
audio: true,
},
peerconnection_config: {
‘iceServers’: [{ ‘urls’: ‘stun:stun1.l.google.com:19302’ }]
},
sdp_constraints: {
OfferToReceiveAudio: false,
OfferToReceiveVideo: false,
},
localVideoId: “myvideo”, // <video id=”id-of-video-element” autoplay muted></video>
bandwidth: 25000, // default is 900 kbps, string can be ‘unlimited’
dataChannelEnabled: true | false, // enable or disable data channel
callback: (info, obj) => {
if (info == “initialized”) {
webRTCAdaptor.publish(“stream1”, null, null, null, “abcd”);
}
}, // check info callbacks bellow
callbackError: function (error, message) {
}, // check error callbacks bellow
});

let select_effect = document.getElementById(“select_effect”);

let APIKey = “Enter Your API KEY HERE”;
window.a = APIKey;

var effectName = VideoEffect.NO_EFFECT;

for (let i = 0; i < VideoEffect.deepARModelList.length; i++) {
let option = document.createElement(“option”);
option.value = VideoEffect.deepARModelList[i];
option.text = VideoEffect.deepARModelList[i];
select_effect.appendChild(option);
}
select_effect.onchange = () => {
let selected_option = select_effect.options[select_effect.selectedIndex].text;
if (selected_option === “No Effect”) {
effectName = VideoEffect.NO_EFFECT;
} else {
effectName = VideoEffect.DEEPAR;
}
console.log(“effect changed”, selected_option);
webRTCAdaptor.enableEffect(effectName, APIKey, selected_option).then(() => {
console.log(“Effect: “ + effectName + ” is enabled”);
}).catch(err => {
console.error(“Effect: “ + effectName + ” is not enabled. Error is “ + err);
alert(err.name);
});
}
</script>
</body>
</html>

Using your own DeepAR files.

If you have your own files that you want to in Integrate, please follow these steps.you can get all the free effects from here.

1. Copy the files to your Ant Media Server at the following location:

/usr/local/antmedia/webapp/{APP_NAME}/js/external/deepar-effects

2. Append the name of the effect file in the array or you can add the file name in video-effect.js file located at /usr/local/antmedia/webapps/WebRTCAppEE/js/video-effect.js 

VideoEffect.deepARModelList.push("filename")

3. Now you can use the new Effect

Result 

By integrating DeepAR with Ant Media’s WebRTC JS SDK, you can easily add augmented reality effects to your WebRTC streaming application. With just a few lines of code, you can fetch all available effects and apply them to your video stream in real-time. This integration opens up new possibilities for interactive and engaging streaming.

Overall, integrating DeepAR with Ant Media’s WebRTC JS SDK provides a simple and powerful solution for adding augmented reality effects to your WebRTC streaming application 

Categories: Tutorial
Tags:

Usama Tahseen

Usama is a Software Engineer at Ant Media. His technical stack includes WebRTC, Gstreamer , FFmpeg , Python, Javascript, Node.js

Related Posts

restream circle with media push plugin
Tutorial

Re-streaming Conference with Media Push Plugin

In today’s interconnected digital landscape, extending your conference’s reach to a global audience is more important than ever. Re-streaming your conference across several social media channels can greatly increase your message’s reach, engagement, and impact. Read more…

Web Player
Tutorial

How to Easily Integrate Ant Media Web Player into Your Project for Excellent Playback

The Web Player, developed by Ant Media, is a powerful live stream player specially designed to simplify the playback of server-hosted live streams. It supports various streaming protocols such as WebRTC, HLS, CMAF Dash, and Read more…

Ant Media Server Cluster using GCP Deployment Manager
Tutorial

Ant Media Server Cluster Deployment Simplified with Jinja Templates on GCP

In this guide, we’ll explore how to leverage Jinja Templates and GCP Deployment Manager to simplify the deployment of Ant Media Server clusters on GCP. We’ll delve into the fundamentals of Jinja Templates and GCP Read more…

chatsimple
Warning: Undefined array key "promocode" in /home/mirror.antmedia.io/public_html/wp-content/themes/hestia-child/footer.php on line 70