{"id":2120,"date":"2017-03-31T15:20:39","date_gmt":"2017-03-31T13:20:39","guid":{"rendered":"https:\/\/blog.mi.hdm-stuttgart.de\/?p=2120"},"modified":"2023-08-06T21:50:52","modified_gmt":"2023-08-06T19:50:52","slug":"livestreaming-with-libav_-tutorial-part-1","status":"publish","type":"post","link":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/2017\/03\/31\/livestreaming-with-libav_-tutorial-part-1\/","title":{"rendered":"Livestreaming with libav* – Tutorial (Part 1)"},"content":{"rendered":"

\"Green<\/p>\n

Lifestreaming is the real deal of video today, however there aren’t that many content creation tools to choose from. YouTube, Facebook and Twitter are pushing hard to enable their users to stream vlogging-style content live from their phones with proprietary Apps, and OBS<\/a> is used for Let’s Plays and Twitch streams. But when you want to stream events or lectures you are pretty much on your own.<\/p>\n

In this series of posts I want to share the experiences I gained over the past couple of weeks while writing an application that captures video and audio and creates a simple livestream. This application is designed to be the basis of a simple streaming desktop application. This series is also supposed to help people to better understand the ffmpeg libraries for creating videos. While there are some great tutorials on how to build video players, almost no one troubles himself on writing tutorials for creating\/encoding videos.
\nThe posts are intended for people who already have a little experience with video creation. The first part of the series will be some background stuff, the second post will mainly focus on building an applications with the ffmpeg-libraries.<\/p>\n

Assuming that you want to embed your livestream in a web page and play it with common browsers one of the first things to do as a content creator is to think about which codecs and container formats to use. This greatly influences the experience of the users.<\/p>\n

Not all operating systems and Browsers support the same set of techniques. Therefore I want to give a very quick overview on the most common codecs and containers. This overview is mainly based on playback support for the codec rather than quality issues because this will be your first concern when starting with lifestreaming. Ronald S. Bultje<\/a> provided an awesome quality and performance oriented comparison of modern video codecs<\/a>.<\/p>\n

Video and audio formats<\/h2>\n

First of all you have to look at your users: Which operating systems and devices do they use?<\/p>\n

While on PC all codecs and containers you can think of are probably supported in some library, it is a very different story on mobile devices.<\/p>\n

Because video playback (decoding) uses a lot of computing power and therefore drains your battery, mobile operating systems try to use hardware acceleration as much as possible. But this also means that not all codecs and containers are supported for livestreaming. Here is a support list of the most common operating systems and browsers.<\/p>\n

                           vp8\/9+webm  h264+mp4\/DASH   h264+mkv\n                            \nChrome (+Opera, Vivaldi...)     x               x           -\nFirefox                         x               x           -\nSafari                          -               x           -\nEdge                            vp8             x           -\niOS (all browsers)              -               x           -\nAndroid (Chrome)                x               x           -<\/pre>\n
Sources: caniuse.com<\/a>, MDN<\/a>, Android developer guide<\/a>, iOS developer guide<\/a>, Microsoft developer guide<\/a><\/p>\n
Even on PC support for hardware accelerated video playback is important to think about, since it relieves the CPU of a lot of work and lets the PC run quieter.<\/p>\n
When you want to livestream to mobile devices and the most common browsers on PC, there are basically three options to go with:<\/p>\n
h264 + rtmp<\/h3>\n
With h264<\/a> being an older codec it is now well supported on almost all devices, most of the time even with hardware acceleration. On lower bitrates you get the characteristic block artefacts, but all in all this codec offers goodish performance and efficiency. If you want to archive or postprocess your videos, the h264\/5 is a good basis. Keep in mind that the use of both h264 and h265 may be associated with license costs if you plan on using the codecs in a commercial way.<\/p>\n
The rtmp<\/a> container is probably the most broadly used container with Twitch being one of its biggest users.
\nBut rtmp is a proprietary software by Adobe. The company’s support (or rather lack of) for some of their products should make you think about whether to bet on such a technology.<\/p>\n
h264\/5 + mp4<\/h3>\n
h265<\/a> (HEVC) is merely an iteration of h264. Because it is relatively new, not all hardware supports this codec (only Intel > Skylake, AMD -> Carizzo, AMD > Radeon X400, Nvidia > GeForce 900).<\/p>\n
Unfortunately the mp4 container is not usable for live-streaming because of its necessity to know exactly how long the video should be (which is kind of hard to know in advance). MPEG-DASH<\/a> can be used as a workaround for the problem. It basically splits the video of unknown size into many small chunks with known size. This also allows you to change the resolution of the stream while playback (like YouTube does).<\/p>\n
h264\/5 + mkv<\/h3>\n
Because of browser limitations this combination cannot be used to stream directly to the user. But if you are using a server to distribute the stream to all the users (what you will most certainly do), you can use this combination to stream against the server as an intermediate step in your lifestreaming pipeline.
\nThe open-source container matroska<\/a> is probably the best choice when it comes to streaming h264\/5, because unlike mp4 it doesn’t need to know how long the video is. Since it is supposed to be streamed via HTTP you also won’t run into any issues with restrictive firewalls.<\/p>\n
Because webm is a subset of matroska, you can very simply convert the container formats and transcode the video to vp8\/9 on a server and stream it to your users from there.<\/p>\n
vp8\/9 + webm<\/h3>\n
Wanting to avoid license costs for h264\/5, Google implemented the open and royalty free VP8\/9<\/a> codecs as well as the webm<\/a> container. This combination shines especially in combination with the HTML5 video-tag<\/a>. Being an HTML standard most common browsers (Chrome, Firefox, Edge) implement the codec and container. Building a player is as simple as inserting the video-tag into your page. VP9 offers far better efficiency with better image quality (especially when using low bitrates) than h264 while being resource-friendly on the decoding side. A very good tutorial on setting up VP9 can be found in the webm-wiki<\/a>.<\/p>\n
A real issue is Apple’s refusal to support this combination in Safari and on iOS. While on macOS you can simply switch to another browser, on iOS video playback isn’t supported at all.<\/p>\n
Audio<\/h3>\n
So far we haven’t talked about audio a lot. But fortunately this is a very easy choice:<\/p>\n