More Efficient Mobile Encodes for Netflix Downloads
Last January, Netflix launched globally, reaching many new members in 130 countries around the world. In many of these countries, people access the internet primarily using cellular networks or still-developing broadband infrastructure. Although we have made strides in delivering the same or better video quality with less bits (for example, with per-title encode optimization), further innovation is required to improve video quality over low-bandwidth unreliable networks. In this blog post, we summarize our recent work on generating more efficient video encodes, especially targeted towards low-bandwidth Internet connections. We refer to these new bitstreams as our mobile encodes.
Our first use case for these streams is the recently launched downloads feature on Android and iOS.
What’s new about our mobile encodes
We are introducing two new types of mobile encodes — AVCHi-Mobile and VP9-Mobile. The enhancements in the new bitstreams fall into three categories: (1) new video compression formats, (2) more optimal encoder settings, and (3) per-chunk bitrate optimization. All the changes combined result in better video quality for the same bitrate compared to our current streams (AVCMain).
New compression formats
Many Netflix-ready devices receive streams which are encoded using the H.264/AVC Main profile (AVCMain). This is a widely-used video compression format, with ubiquitous decoder support on web browsers, TVs, mobile devices, and other consumer devices. However, newer formats are available that offer more sophisticated video coding tools. For our mobile bitstreams we adopt two compression formats: H.264/AVC High profile and VP9 (profile 0). Similar to Main profile, the High profile of H.264/AVC enjoys broad decoder support. VP9, a royalty-free format developed by Google, is supported on the majority of Android devices, Chrome, and a growing number of consumer devices.
High profile of H.264/AVC shares the general architecture of H.264/AVC Main profile and among other features, offers other tools that increase compression efficiency. The tools from High profile that are relevant to our use case are:
- 8x8 transforms and Intra 8x8 prediction
- Quantization scaling matrices
- Separate Cb and Cr control
VP9 has a number of tools which bring improvements in compression efficiency over H.264/AVC, including:
- Motion-predicted blocks of sizes up to 64×64
- ⅛th pel motion vectors
- Three switchable 8-tap subpixel interpolation filters
- Better coding of motion vectors
- Larger discrete cosine transforms (DCT, 16×16, and 32×32)
- Asymmetric discrete sine transform (ADST)
- Loop filtering adapted to new block sizes
- Segmentation maps
More optimal encoder settings
Apart from using new coding formats, optimizing encoder settings allows us to further improve compression efficiency. Examples of improved encoder settings are as follows:
- Increased random access picture period: This parameter trades off encoding efficiency with granularity of random access points.
- More consecutive B-frames or longer Alt-ref distance: Allowing the encoder to flexibly choose more B-frames in H.264/AVC or longer distance between Alt-ref frames in VP9 can be beneficial, especially for slowly changing scenes.
- Larger motion search range: Results in better motion prediction and fewer intra-coded blocks.
- More exhaustive mode evaluation: Allows an encoder to evaluate more encoding options at the expense of compute time.
Per-chunk encode optimization
In our parallel encoding pipeline, the video source is split up into a number of chunks, each of which is processed and encoded independently. For our AVCMain encodes, we analyze the video source complexity to select bitrates and resolutions optimized for that title. Whereas our AVCMain encodes use the same average bitrate for each chunk in a title, the mobile encodes optimize the bitrate for each individual chunk based on its complexity (in terms of motion, detail, film grain, texture, etc). This reduces quality fluctuations between the chunks and avoids over-allocating bits to chunks with less complex content.
Video compression results
In this section, we evaluate the compression performance of our new mobile encodes. The following configurations are compared:
- AVCMain: Our existing H.264/AVC Main profile encodes, using per-title optimization, serve as anchor for the comparison.
- AVCHi-Mobile: H.264/AVC High profile encodes using more optimal encoder settings and per-chunk encoding.
- VP9-Mobile: VP9 encodes using more optimal encoder settings and per-chunk encoding.
The results were obtained on a sample of 600 full-length popular movies or TV episodes with 1080p source resolution (which adds up to about 85 million frames). We encode multiple quality points (with different resolutions), to account for different bandwidth conditions of our members.
In our tests, we calculate PSNR and VMAF to measure video quality. The metrics are computed after scaling the decoded videos to the original 1080p source resolution. To compare the average compression efficiency improvement, we use Bjontegaard-delta rate (BD-rate), a measure widely used in video compression. BD-rate indicates the average change in bitrate that is needed for a tested configuration to achieve the same quality as the anchor. The metric is calculated over a range of bitrate-quality points and interpolates between them to get an estimate of the relative performance of two configurations.
The graph below illustrates the results of the comparison. The bars represent BD-rate gains, and higher percentages indicate larger bitrate savings.The AVCHi-Mobile streams can deliver the same video quality at 15% lower bitrate according to PSNR and at 19% lower bitrate according to VMAF. The VP9-Mobile streams show more gains and can deliver an average of 36% bitrate savings according to PSNR and VMAF. This demonstrates that using the new mobile encodes requires significantly less bitrate for the same quality.
Viewing it another way, members can now receive better quality streams for the same bitrate. This is especially relevant for members with slow or expensive internet connectivity. The graph below illustrates the average quality (in terms of VMAF) at different available bit budgets for the video bitstream. For example, at 1 Mbps, our AVCHi-Mobile and VP9-Mobile streams show an average VMAF increase of 7 and 10, respectively, over AVC-Main. These gains represent noticeably better visual quality for the mobile streams.
How can I watch with the new mobile encodes?
Last month, we started re-encoding our catalog to generate the new mobile bitstreams and the effort is ongoing. The mobile encodes are being used in the brand new downloads feature. In the near future, we will also use these new bitstreams for mobile streaming to broaden the benefit for Netflix members, no matter how they’re watching.
— by Andrey Norkin, Jan De Cock, Aditya Mavlankar, and Anne Aaron
See Also:
Originally published at techblog.netflix.com on December 1, 2016.
