Hardware-accelerated AV1 Video Encoding | Intel Chip Chat ep. 717
(upbeat music) - Hi, welcome to Intel Chip Chat. We have a very special episode for you today, where we talk about the future of video encoding, decoding, and all the technology behind that. Joining us today is Matt Frost, the Chairman of the Alliance for Open Media. Matt, thanks for joining me and would you mind giving us a little intro of who you are and what you do? - Sure, happy to. I'm Matt Frost, I'm a Director at Google and I'm Chairman of the Board of the Alliance for Open Media or AOM. AOM is a consortium of global tech innovators and leaders in streaming media in video on demand, in live video, in video conferencing and leaders in the manufacturer of the devices and chips.
The power, those online media experiences. - You mentioned that you're at Google as well as Alliance of Open Media. What's the connection between these two companies and why do you work at both? - Yeah, so I work at Google.
Google is a member company in AOM the Alliance. The Alliance for Open Media is a consortium that Google and other leading tech companies, Intel® being one of them. Microsoft, Netflix, Amazon formed back in 2015 with the goal of developing the next generation of cutting edge media technologies. And in particular, with a focus on developing, royalty free open source technologies.
And the common element for all of these companies is that we are all leaders in online media services, whether as you know, Google, a company that has YouTube is one of our product lines that has video conferencing applications like Meet and Duo. Obviously we work to support a Chrome ecosystem and an Android ecosystem. And for all of these experiences and devices and platforms, having a great media experience is vital. The Alliance.
So Google is very focused on working with our partners all of whom recognize that we really have to work together as a community to ensure that we are developing common cutting edge technologies, that meet the needs of all of these emerging experiences on the web. I think we are all coming. We hope out of two and a half years of COVID where many of us were were, probably most of us were working at home. Many of us were communicating with video conferencing applications or watching our kids engaging in online learning, whether it was streaming education, educational videos, or video conferencing with their classmates. We were entertaining ourselves with online streaming services like YouTube and Netflix and Amazon Prime.
And we were communicating with our family and with our friends using video conferencing applications, many of which came from member companies in AOM, like Cisco with Webex or Microsoft with Skype or Google with Duo and Meet and obviously Facebook and Apple and all of these other online communication experiences that we use. And the common element for all of these experiences is a video codec, a video compression technology. And so we got together in the AOM community to develop these technologies. And then once we finalized the standard, the AV1 standard which we released in AV1.
Then came the second phase of hard work, which is rolling it out in the ecosystem and making sure that it's supported on all the devices that we rely on now to capture content, to create content and to play back that content. - So clearly video is a huge priority these days for all of us, whether we're doing remotely or we're back at work, or we're seeing, sharing experiences with our friends, but to start at the very basics, you talked about AV1, a new video codec, can you start at the most basic level and explain AV1 to me like I'm five. - AV1 is the next generation video codec or video compression technology. It allows users to capture higher resolution video using the same amount of bandwidth than storage that they used with earlier generation technology or to capture the same resolution, using leaving less storage.
So when you compare it with previous technologies like VP9 or HEVC. AV1 offers 20 to 30% gains. And when you compare it with the very common technologies, like H. 264, AV1 offers well over 50% gains over those earlier technologies. - Now, obviously there have been other codecs that proceed AV1.
So let's start also with kind of explaining I'm five, first can you tell me what is a codec? And then you can go a little bit deeper in telling me like how is AV1 different from previous codecs, like a AVC or HEVC and VP9. - A codec is a compression technology. The word codec stands for compressor and decompressor. And so with each generation of technology, the engineers are coming up with a new way to, a new set of tools that more efficiently compress. And then once they come up with these tools, we're developing a bitstream, it's essentially sort of a code that we use to on the encoder side, on the content creation side to capture the compressing that we're doing to express it in a bitstream. So essentially the language of AV1.
And then on the playback side, you have a decoder something that understands the code that you've written on the compression side, on the encoder side, and blows it up into what you and I see on our video screen. In terms of how it's different from previous generations. What we're looking for is compression gaining. So as I said to more efficiently store video than you could with the previous generation. And so every codec is largely just an improvement over the previous generation, you're getting 25 or 35% gains over previous generation. And you asked about AVC or H. 264.
AV1 is well over 50% more efficient than those tech, than H. 264. So that means if I have a full HD file in AV1, I can compress it using 50% the space, or I can send it over the internet using 50% the bandwidth that I did with H. 264. And if you compare AV1 with VP9 or HEVC, which were the last generation technologies there the gains are 20 to 30% over those previous technologies. So really a lot of what we're doing is just the same as previous generations.
We're just more efficient in the way that we are compressing the technology. And what that means is for a service provider, it means that I can send a full HD video using 20% fewer. 20% less bandwidth than I did using an older generation technology, or I can send you a higher resolution or a higher frame rate video using exactly the same amount of bandwidth.
There are some tools in AV1 which didn't exist in previous generation. We have film grain synthesis tools, which allow for cinematic content, where you're really interested in capturing that sort of film grain. There's a tool in AV1 that doesn't exist in previous generations. There's some tools that allow for faster resolution switching or sort of on the fly changing and quality that you couldn't do with earlier generation technologies, but largely it's just really about higher quality, more efficiency than earlier technologies. - Wow, so codecs are about putting a lot of information and trying to pack into a smaller space, which is actually kind of a feat that Intel®'s very familiar with. I like it to be how Intel®'s manufacturing technologies are all about packing the most amount of transistors and logic into the smallest space possible.
And these are challenging feats. So I imagine it's got to be somewhat similar when you're going from one codec to the other. So can you tell me developing AV1 over the previous codecs? What were some of the biggest challenges in making AV1 significantly better than previous generations? - Well, I think your analogy of packing more circuits on a chip is sort of similar, which is we're already doing a pretty good job with previous generation technologies. They're pretty well evolved and so every additional gain requires a lot of work, a lot of experimentation. And in particular, we always know that we can get the gains. We can get additional quality with each generation, but it comes at a processing cost.
We have do more searching. We have to make more decisions on not only on each frame. So if you think typically a video is 30 frames per second or 60 frames per second, on each of those frames, we divide the frames up into blocks and segments. And so within each video, for every frame of the video, we're making many, many, many decisions at the encoding level about how to most efficiently capture that. And that requires that we do more processing for each generation.
If you've got a late model computer, you can do that in software, on your CPU, but it becomes more difficult to encode really quickly at the very highest resolution at high frame rate and in particular in real time, and to do so in a power efficient way. And so that's why part of the process of rolling out a new codec and the reason that we can't, just day one after we've released a codec, just flip a switch and upgrade everyone. Is that there's work that has to be done by companies like Intel® to build really power efficient circuits within their broader graphics cards or CPU or motherboards to ensure that we can do efficient encoding and efficient decoding. And so, again, the challenge is just we are developing these tools.
We're asking the algorithms to make more and more decisions very quickly. And we've got such a well evolved technology that it's just very difficult to get the gains from generation to generation. And that's why it's so important that we bring together. All of these companies at AOM really pull our knowledge and pull our resources to work at developing next generation media technologies. - Compute-intensive is one thing that I do want to focus on.
For, I mean, years, last few years, I've seen the word AV1 and decoding on your mobile devices, your setup boxes, people who are consuming that content. But as you just said, to make those encoding decisions on AV1 is very computationally intensive. And of course, I want to talk about the hardware encoding technology that we're bringing out here.
Could you tell me more about your thoughts on the challenges of hardware encoding and what the new Intel® hardware, brings to enable that for people who are into AV1 in streaming? - It's really difficult to build a good hardware encoder. First silicon lead times, hardware lead times are much longer than, take much longer than it takes to just develop a software library that you can run on your CPU. So there's really a lot of engineering work that goes into developing a hardware encoder. And then it's a challenge. Not just to develop any old hardware AV1 encoder or new format encoder put to do a really good job with it.
You and produce AV1 encoder that is in hardware, but is not that much more efficient than previous technologies. It really takes work and engineering skill to make sure that you're doing a really good job of it. If I'm somebody who is, he is an avid gamer who wants to stream my games to my viewers on Twitch or on YouTube. If I'm a content creator that wants to share my content on YouTube or an influencer that wants to create really good looking content.
The things that I'm thinking about are how do I capture it in the highest resolution? How do I capture it in the highest frame rate? How do I store it efficiently on my disc drive so that I'm not using up all my disc space? How do I upload it to whatever the service is that I'm using YouTube or Facebook or Twitch? And all of these are problems that AV1 helps solve. And all of these are problems that a really efficient, really good hardware encoder solve. And again one of the powerful things about the Alliance for Open Media is if you look at its membership, if you are a game streamer, you care about YouTube. You care about Twitch and Amazon and Google are both members of AOM and either already supported AV1 or talking publicly about their plans to support AV1. Obviously if you're interested in sharing through a sharing site like YouTube or Facebook or Snap, again, those are all controlled by AOM members.
- I do want to make a bit of a distinction here because AV1 is definitely a growing technology that it's being democratized to the masses as you're talking about. But Intel® has been working with companies like Netflix and other premium streaming providers to encode AV1 video. And this is using a lot of our data center technology, dating back a couple years from now.
But what we're talking about today is the hardware encoder that is Intel® ARC™ hardware, and you mentioned game streamer. So are we moving to a model from where AV1 was just for the large streaming providers. Now it's down to someone who just wants to stream their games and they can do this on real time, just using a hardware encoder on Intel® ARC™ graphics hardware? - Yeah, so I wouldn't say that it was only available to the big companies. I think one of the things about some of the companies in AOM, teams behind YouTube, teams behind Facebook is they have a lot of experience, rolling out new technologies, and they have a lot of processing power behind them in their data centers that allow them to be really early adopters of new technologies like AV1, but it's an open source technology. It's royalty free, it's available to anyone.
And I think people are out there thinking, how can I improve my video service? Can look at these big companies that care a lot about video quality. Care a lot about encoding costs, care a lot about storage costs and streaming costs and figure that those companies, probably know something about video technology and about the value of new technology. So in terms of the example that you gave, which is game streamers. Yeah, I think big video companies, YouTube it's already been publicly announced, has already built AV1 hardware encode for their cloud services.
And so the tools that you, that Intel® and other hardware companies are giving in terms of hardware encode capability on graphics cards, or when you buy your next laptop computer that has a built-in Intel® graphics card with AV1 support. You're absolutely getting access to the same really power efficient, high quality video encoders that the big companies have been using for a couple of years now. But it's not that as you said, the open source model is all about democratization of media and the emergence of these hardware encoding solutions is just giving another really powerful tool to creators who care about capturing the very highest resolution video, and frankly who at the same time, want to make sure that their video conferences are really good when they're talking to their clients or talking to their friends.
- So there's another thing I do want to touch on when you're talking about creators and we're talking a lot about quality, AV1 does bring a lot of quality. And you mentioned a little bit earlier that, yeah, the other benefit is that you can achieve a very high quality similar of previous generations, but at a smaller file size. So these days, I mean, I think that we have the luxury of more bandwidth than we had before, but sometimes networks get congested. Can you tell me a bit about the technology that allows smaller file sizes while retaining that higher quality? - There is, we're moving to 5G. People are getting more fiber at the home, but I think our experience over the last couple of years is that we're also just using more video in the home. So even if you have a really strong fiber connection to your home.
Your video conferencing, your kids are watching educational videos and video chatting with their friends. Somebody else in the family is watching a Netflix video or YouTube videos. And all of this is just adding to greater and greater demands on our home network. And certainly during COVID, we saw that the global networks were being overcome by the amount of video related data that was going over the network. So I think another thing that's bringing all of these companies at AOM together is just a desire to be good corporate citizens and good netizens. And to ensure that we're using the most efficient technologies to allow more video use in the home and more efficient use over global networks.
- You just mentioned AOM and its collection of companies. And these are all industry leaders in their own segment. And some of them also are competitors in the same segment. So what is the incentive for these companies to work together on an open standard? - Well, I think there are several, I mean, one is there is a belief among all member companies that we have an obligation as large companies to work together to ensure that data moves efficiently over global networks. But I think there's also.
And this goes to the challenge of getting a new technology like AV1 on a chip from Intel®, a graphics card from Intel® or from other another hardware manufacturer. It's a lot of work to get each new technology in hardware and to get it supported across all of these devices that we now rely on to create content and to consume content. And we can't all go out and develop our own technologies and vie with one another to get them supported on devices. We need a common set of technologies that work across platforms that we can all work on.
And then all of us will compete in building the best products and the best services for those technologies. And I think as we've seen, people don't just want one video streaming service, they want a choice. They don't just want one video conferencing application, they want a choice.
So this is really about a bunch of companies that recognize that we can only build good products if they're support throughout the devices that our users use and the way to make sure that every new generation technology is supported as quickly as possible is to collaborate in building the best technologies and in getting them supported as broadly as possible. - Yeah, that actually sounds really great. And I wonder if it's probably no accident that I think some of my favorite technologies have come from open standards and Intel®'s been involved in a lot of them, such as USBC. I love the fact that now we can have a flippable connector and it works with almost everything.
You can charge a lot of things with the same connector. And AV1, I mean, AV1 is great. I definitely appreciate, the 4k streaming I have with HDR, but I actually, I think my favorite part about AV1 is you get, it's so much more efficient. I love efficiency and having the great video quality, same video quality at smaller file sizes or smaller demands in network is great. So I want to ask you someone who's very, very close to AV1.
What is your favorite aspect of AV1? And if you don't have one, then maybe something that maybe not is as appreciated or as unsung to common knowledge. - Yeah, so I guess I would say, it's difficult to say my favorite tool in AV1. I think I agree that the efficiency that you get from AV1, the ability to have a video conference with somebody, even in bandwidth conditions that aren't as good as they should be. And to continue to maintain that high resolution video without stuttering, without dropping frames. That's really great. As I said, there are new tool like film grain synthesis is a tool that's unique to AV1 that lets particular for content creators, for people who are interested in capturing that sort of realistic film experience, that's a new tool that allows creators to really explore new fields, a new feel in their content.
But I think again, it's really just comes down to the greater efficiency, the higher resolution, the higher frame rate, the more efficient storage on your device, so that you're not quickly blowing up the memory on your mobile phone or on your computer and trying to figure out what video file you're going to have to throw out. What video file that you may have spent time capturing, you have to get rid of to free up space. - Great, so AV1 is definitely a cutting edge, video codec technology, but light technology, everything kind of moves at a pace and someday something better will come along because people are working harder on improving what we have.
But right now, AV1 sounds cutting edge. It's coming, a hardware encoders is now hitting, people can have it on their laptops with Intel® Arc graphics. Where are we in the life cycle right now with AV1? Is this something that's still going to be with us for quite a while? - It, oh, it's certainly going to be with us for quite a while. Because as we've been discussing, it takes quite a while to, it takes two to three years to take a new technology, like AV1, turn hardware engineers loose on it to design the circuits that then get built into an Intel® chip or into a chip built by another manufacturer. So there's just always going to be some lag between release of a new codec, like AV1 and support in all the devices. Support at the highest resolution and the most power efficient implementation across all of these devices that we care about.
So we are seeing now AV1 playback, AV1 decode in hardware, appearing on mobile devices. We already had AV1 playback and software on devices, but now we're seeing the first generation of mobile devices that are coming out with AV1 playback and hardware. Obviously the announcement we're talking about today is about AV1 encoders, which are coming for graphics chips. Some CPUs are already supporting it. So all of those chips have to make their way in devices, those devices have to make their way onto store shell into consumers hands. So we have years of taking increasing advantage of AV1 and content providers, YouTube is already using a lot of AV1.
Netflix is using AV1, Facebook and other AOM members are talking about 2023 rollout plans for AV1. So AV1's already being heavily used. It's going to be more heavily used as years go by, but AV1 members are working on next generation technologies as well and we have eyes on the success to AV1 down the road. - Well, Matt, that's a great overview.
And can't wait to see more of what AV1 brings and just better video. You know I'm a big fan of videos. So anything that makes the experience better for anyone consuming it or anyone actually creating it sounds great.
So, hey, thanks so much. I want to give you the chance to have like, is there any one thought or parting thought that you want people to know about AV1 before we wrap up? - So one of the things that we're very excited about is the rapid adoption we're seeing in the ecosystem and the work that chip manufacturers and device manufacturers are doing to improve on AV1 encode and decode by adding support for hardware encoders and hardware decoders. And that's why we're particularly excited to be able to participate in Intel®'s announcement and to talk about the promise of new lines of hardware, super efficient AV1 encoders that put the power of the highest resolution, highest frame rate video creation in the hands of consumers. - Matt, that sounds great.
Thank you very much for your time and thanks for joining me. - Yeah, thank Marcus. (upbeat music)
2022-06-03 17:02
