The Internet of Things is burgeoning—and so is the amount of camera-based connected devices that produce real-time media data and can potentially send it to a PC or smartphone. But there’s a catch.
Most users don’t need a standalone app to play audio or video clips: they have browsers for that. And here’s where WebRTC comes into play by integrating an IP camera with a browser. This relatively new technology might not ensure maximum device coverage at the moment but easily beats its counterparts (such as HLS and MPEG Dash) in terms of performance.
This article will give a brief overview of WebRTC, describe its advantages, highlight its use cases in IoT, and outline what a WebRTC embedded device is. We’ll also explain how to integrate this technology with connected devices through the GStreamer WebRTC framework.
Web Real-time Communication (WebRTC) is an open-source technology, developed by Google and Ericsson. It allows developers to integrate real-time video and audio communication capabilities into web and mobile applications.
Simply put, WebRTC establishes peer-to-peer communication between web or mobile browsers without any additional plugins. It gains access to a gadget’s camera and microphone, and is capable of streaming media files with only a half-second lag. In fact, it’s regarded as the primary real-time media file transfer solution.
The rise of remote work has triggered a new stage of developing real-time applications with WebRTC. The technology’s use cases include:
Our Linphone-based VoIP mobile app is a vivid example of how WebRTC solutions allowed our clients to make their corporate communication more convenient and cost-effective.
Since WebRTC uses web and mobile browsers to establish a peer-to-peer network, it cannot use the same communication protocols that browsers use for accessing websites. The challenge here is that the users’ laptops or smartphones are protected by network access translation (NAT) devices or firewalls. Unlike HTTPS websites, whose location is known to the entire Internet, laptops and smartphones do not have static web addresses. Thus, in order to begin a communication session between two users, browsers should locate each other and obtain permission to exchange media data in real time.
To set up a phone or video call with a user outside a home network, WebRTC utilizes Session Traversal Utilities for NAT (STUN) and Traversal Using Relays around NAT (TURN) servers, along with signaling/communication protocols. The latter include:
WebRTC’s flexibility allows any company to improve their business communication tools using fast and secure web applications.
Before WebRTC, real-time media communications relied on C/C++. As a result, the development of custom conferencing and collaboration tools meant longer project timeframes and higher costs. Although WebRTC still uses the technology stack under the hood, developers no longer have to dig into those layers, as it has a JavaScript API on top, which is enough for building applications that interact with browsers.
Thanks to restful APIs, developers can create WebRTC embedded devices and applications with relative ease. This allows for using sensor data produced by connected devices to trigger alert notifications, audio calls, and video sessions. WebRTC introduces a new security layer to IoT solutions, and can be used as a safe data transfer channel. Finally, there’s much more to the technology than audio and video streaming: a WebRTC-based Smart Home security system, for instance, can both produce a live video feed and tell which door is open using the data from a connected lock.
The Internet of Things concept revolves around real-time data exchange — and that’s what WebRTC is particularly good at.
For instance, the technology could be a welcome addition to an RFID-based retail store security system. In addition to label tags and security antennae, a WebRTC-based solution can incorporate an IP camera, which monitors store traffic and streams video data in real time to a web application running on a security manager’s computer. When an item that hasn't been paid for sets the detectors off, the manager can correlate the alert signal with camera footage, and identify the shoplifter.
Although you can compile the WebRTC native code package to create a peer-to-peer connection without an intermediary media server, WebRTC is not directly supported by most IoT devices and embedded systems.
Further on the subject of smart cameras, you usually can't access the data produced by smart security systems outside a proprietary network, which prevents users from monitoring industrial facilities and connected homes remotely.
As a result, no business or consumer-facing IoT solution currently provides WebRTC out-of-the-box.
The official WebRTC Native APIs lack flexibility and can be difficult to work with. A better solution may be GStreamer—an open-source pipeline-based framework for creating multimedia streaming applications for desktops, connected devices, and servers. It has added a native WebRTC API to its feature set.
For better understanding, the GStreamer framework can be compared to a plumbing system with water instead of media data and GStreamer pipelines serving as the actual pipes. These pipes are capable of changing the quality and quantity of the water on the way from the public water supply (device one) to a home plumbing system (device two).
Suppose the source device can read video files. We can build a pipe bend (GStreamer demuxer) to separate outgoing traffic into audio and video data streams. The data is decoded via h264 (video) and Opus (audio) down the pipeline and delivered to the target device—namely, its video and audio output components, or to the cloud, where it can be analyzed using machine learning algorithms.
In GStreamer, those functional pipe bends are called elements. They can be divided into source elements, which produce data, and sink elements, which accept it. The elements, in their turn, have pads—the interfaces with the outside world, which allow developers to connect elements based on their capabilities. Additionally, GStreamer features built-in synchronization mechanisms which ensure audio and video samples are played in the correct sequence and at the right time.
With WebRTC and GStreamer, we can build smarter home automation and enterprise security systems and more. Below you will find a list of embedded software development projects which can potentially use GStreamer:
As more embedded software engineers discover GStreamer, we will soon see more WebRTC embedded products, including industrial solutions for remote equipment maintenance, Smart Home products, telemedicine applications, intelligent motor cars collecting real-time telemetry data, and wearables.
Dmitry Zhadinets is an embedded software development specialist at Softeq. With a PhD degree in Speech Analysis and Synthesis and over ten years of experience in embedded system development, Dmitry is involved in multiple enterprise-grade CCTV projects, and is particularly interested in media file transfer technologies. He has been in the IT industry since 2002.