Steve Heffernan - It's time to rebuild your player with web components
Hi, my name is Steve Heffernan, thanks to Demuxed for having me talk this year. This is my Covid copilot Reid, and it is time to rebuild your player with web components. So let's do this. I was one of the co-founders of Zencoder back in 2010 and one of the co-founders and head of product at Mux. I helped start the San Francisco Video Technology Meetup with Matt as well as Demuxed - although deserve almost no credit for that except for registering the domain and designing this hideous logo for the first year's conference. What most people know me for is creating VideoJS - the open-source player.
I've had tons of projects on top of that that have just totally failed. But hopefully that's enough to let me say something like browser video is a mess, and current popular JavaScript framework is not the answer. What do I mean by that? First of all, embedding players, customizing them, styling them is complicated today. Each player has its own unique approach and ecosystem that you have to learn in order to customize it. And then, on top of that, what the heck is a player? Video and audio elements - are those players? hls.js and dash.js - which use the video and audio elements -
are those players? Is Video.js and JW Player, which might use hls.js and dash.js - are those players? Is the Azure Player and the Brightcove player - which are built on top of Video.js - are those players?
If you customize those players for your site player, is that a player? They're all players. We need more words. And of course, then there's the issue of just this exploding number of JavaScript frameworks, which I think we all probably understand at this point, if you're working in JavaScript. I started building Video.js back in 2010, built it with vanilla JavaScript, but still had to make sure that it worked well with jQuery at the time.
And then there was jQuery UI and then there was SproutCore, which became Ember, and then there's Angular and Angular 2, which is pretty different from Angular 1. There's React and there's Vue. Now there's Svelte. With Video.js, we want to work with all of these frameworks and it takes work to do that.
And you end up with a couple of different types of players. You end up with general players which are trying to work across all the different frameworks and doing the work to do that. And then you have framework-specific players that are building themselves in the language of the framework itself.
So they don't work anywhere else, but maybe they work best for that framework. But you end up with this exploding matrix of compatibility and asking the questions of, OK, if I'm using this framework, what's the best player for that - is it a general player or the one specific to the framework? Or if I'm using players, the framework that works best around it. And all that's adding complexity to the browser ecosystem while, meanwhile, over in the mobile landscape, in iOS, you have AVPlayer. Android, you have ExoPlayer. And I'm not a mobile developer - I know it's more complicated than that - but if you're a mobile developer, this is what your life looks like to me. And on top of all of that, browser video itself is evolving in a way - we have this new concept of interactive live video where it really is expanding what it means to be a video player, because it's bringing in these interactive components of people, setting up hearts and thumbs up as interesting things in the video are happening or polls and chat happening alongside the video. So building and using a video player is only going to get more complex as this concept gets more traction.
OK, so let's imagine a world, then, where... ..this is what an embed code looks like - just a simple HTML tag - and all of the players and the frameworks just kind of all work together. So what's going to get us there? Well, it's not the current popular JavaScript framework. First of all, popular JavaScript framework speaks its own language. If you build something there it's not going to work with any other framework.
And then, secondly, it's not going to be around for that many more years. You're probably already thinking about what's...the framework you're going to try on the next project that you're starting from scratch? And it's probably not the one you're using today. And then there's the detail of most JavaScript frameworks are meant more for bigger web applications. So web applications where you have lots of state that you're managing, lots of data, multiple views and pages that you need to organize. And that's a different problem than a video player, which is more on the spectrum of a widget.
A widget that you're sharing between multiple views and maybe between pages and even between websites. Right? And that's a different problem than your JavaScript framework is trying to solve. So what's the answer? I think, obviously, you can see where this is going - web components. What is a Web component? Web component is a new feature in browsers that allows you to create your own custom elements or your own custom HTML tags, and name them whatever you want. You do need a little dash in there, but you can name them anything.
And so that gets us away from a current state where we're building applications that have just divs inside of divs inside of divs, and you look at the HTML and you just have no idea what's going on. With custom elements in HTML I can put just my simple HTML tags to embed my component into the page. I can use JavaScript APIs - the native JavaScript APIs - to create other versions of my component.
By just using the tag name, I can use querySelector to select my element without any IDs or classes. And then again, in CSS, I don't need classes or IDs - I can just style the element itself, which then keeps me from creating styles that might be affecting other elements on the page of the same name. And then, speaking of this idea of protecting the styles, we also get this concept of encapsulation. So I have a screenshot here of the video element in the Chrome browser. So this isn't a web component - this is the actual video element itself. And I've inspected it in the JavaScript tools here.
And you can see that it's got a concept called shadow-root, also known as the shadow DOM. And inside of that, you'll find all the controls that we're familiar with that show up on the video element when you turn on controls in a Chrome browser. As JavaScript developers, we have limited access to mess with that HTML, to change the styles of it.
Those things are protected from the outside page because they're inside of this idea of a shadow DOM With web components, we get to create our own shadow DOMs, too, for our elements and so we can protect our styles and our HTML from everything else in the page. With Video.js, we have to create CSS that's very specific to make sure it doesn't break other people's pages and that other people's CSS doesn't break the UI of Video.js, and that takes a lot of work and a lot of knowledge of CSS. And with the shadow DOM, we don't have to do that anymore.
We can protect our CSS in this idea of encapsulation. So where are we with using web components? Well, in 2014 at San Francisco Video Tech Meetup, I said we should probably using web components for video players. A little bit early at the time. The Web component API has changed dramatically since then. In 2015, at Demuxed, I said, hey, you know we should probably use web components for video players.
I said, I'm pretty sure that this is how we'll be doing things in the future. And then we started Mux, and I had no more time to convince you guys need to use Web components and some of you are now like, OK, stop trying to make fetch happen. But then just a year ago, Apple shipped Web components to production inside of their media player, and if I'm reading this title right, this is the first time Apple's put Web components in production and it's in using their media player, and I don't think that's a coincidence. I think that speaks back to the idea before that video players are a little bit different concept. They're more of a widget that you're trying to share multiple places and therefore they're just a good use case for web components.
So why else haven't people used Web components up to this point? Well, there's been a compatibility issue, as you normally have with browsers, but that's not an issue today. Web components and the latest spec, the latest API of Web components is working across all the evergreen browsers. If you need support in IE 11, there's a there's a polyfill to support that. But you don't have to be scared about browser compatibility at this point.
Another reason that people haven't really bought into web components yet is a lot of people are just happy with their current JavaScript framework, like React. It's a good experience, building components inside of React. But again, the framework that you're using today is not going to survive forever.
And so with this video player that you're putting a lot of work into, if you want that to survive beyond the usage of your current framework, then you need to think differently than using your current framework. I equate it a little bit to how I thought about Node and NPM back in, I guess 2013, 2012, when they were first coming out and everybody was writing these small little JavaScript functions to do something simple, like convert a date from one format to another, uploading that to GitHub, publishing that to NPM, and now you don't ever have to write that JavaScript function again. It's not dependent on a framework, it doesn't have a ton of dependencies. And in the future, as I need to do that same function again, I can just pull down that JavaScript, do it again. I don't have to write it again.
So I feel the same about Web components in the sense that as I'm creating a Web component, I can think about, what is the this unique small bit of functionality that I can build into a UI component, publish that and then I can use that in all future projects no matter what framework that I'm using in the future and I think that's pretty cool. OK, so let's get a little bit more specific on what this world we're talking about can look like and dig to embed codes. Back in 2010, this is how you embedded video using HTML5. And this was awesome because back then, most browsers didn't support HTML5 video.
So it gave us a mechanism for using HTML5 video, falling back to Flash when Flash wasn't supported, falling back to text and a download link when Flash wasn't supported. And it allowed us to start using HTML5 video, which was great. But also, look at that embed code. We tried to make that a little bit easier. You just use the video tag itself and then JavaScript comes in, replaces the video element in the HTML with the div, with the video element inside of it, so that we can then overlay controls and other features on top of the player and simplifies the embed code a little bit.
Still not as clean as maybe I'd like it to be. JW Player does something similar. We have a div in the page and JavaScript is used to then replace that div in the page with the video player.
By the way, your current JavaScript framework probably hates that approach of swapping HTML in the page. In fact, React is like, don't touch the DOM, stop touching the DOM. So all of these different widgets that use an element in the page is a placeholder, which is a pretty popular way of embedding a widget in the page, had to rethink how they're embedding this widget because of frameworks like REACT thinking about the ecosystem a little bit differently. So here's the HLS embed code, still a lot of JavaScript there that you have to understand and embed in the page to get video working, dash.js maybe
a little bit cleaner, a couple of lines of JavaScript along with the video element, but still not this, like, just a simple HTML tag in the page. You point to a video and includes all the functionality that you want out of video player in it and you understand what's going on there. That's pretty clean, and there's a lot more people that can write HTML today, that can write JavaScript. It's a simpler language, it's a markup language, it's not a programing language. So this might include your designers, your marketing people who can now get in and maybe play around and do something different with it that, maybe you don't want that to happen. But it's a much more accessible way of embedding video for people who are not JavaScript experts.
OK, let's look at some examples of this. Custom video element examples. Jumping over some code. We talked about, we talked about HLS.js. Right? Why not build a custom element for HLS.js
to just playback HLS video? So here I have a simple HTML tag, HLS video. Open that up in the browser. I have a video playing HLS. With a simple HTML tag. It uses the same API
as the video element. So you can build on top of this in the same way that you're building on top of the video element. And it just works, that simple code. And then, of course, we've got to include dash in the party, so dash video is a simple tag.
You give it an MPD URL, that's it. You don't need any extra JavaScript on top of that. Play. It's in the browser. And here we have a version of Dash. Playing the browser
with a simple custom element, very simple embed code. Will and the dash.js team thought this was cool enough to pull into the Dahs industry forums, so you can find it there as an alternative way to embed dash.js on your page.
Here's another one that you should check out. So this is a project built by Paul Irish, who is kind of performance guru, browser, performance guru on the Chrome team. He's built an alternative embed for the YouTube player that he says loads 224 times faster than the default YouTube embed.
That's pretty awesome. He's got some stats here to back that up. If you're a player developer, you should definitely check out what he's doing in this project because he's using some advanced browser APIs and doing some very cool things that you should check out and consider using on your own player. But of course, he's embedding this and created the embed code to be just a simple Web component, with Light YouTube as the name of it, to make it really easy to just use this player. Then, one of the Vimeo engineers did the same with the Vimeo embed, using a lot of the same techniques, also wrapping it in a custom element to make it easy to embed. I think these are super cool projects.
One other example worth calling out. Matt McClure has a hack project where he's created a simple element that's just called Video Player that you can pass it, the player can embed, whether it be video.js, hls.js, YouTube, another player, and what it does is behind the scenes, loads an iframe that has the player embedded in it. It passes the source to the iframe to the player, player plays the video at full screen within the iframe and then you can just resize this custom element however you want, and it makes it really easy to switch between players. It is going to limit the amount of customization you can do with that player. But if you have a case where you need to switch between players or test them out in a lot different ways, this is a pretty cool way to do that, I think.
Let's go back to this idea of framework compatibility and this potential world where frameworks and players just kind of work together. back in 2014. Glen Madder at JS Conf EU was demoing a new component that he created called X-Gif and talked about how he used a custom element to build it and how everything talks HTML. And by that he meant all of the different frameworks that are out there.
At the end of the day, they speak HTML because that's the core language of the browser. So if we can build our elements, if we can build our player in a way that just looks like HTML, then it can just be naturally compatible with all the different frameworks out there. And this was kind of an epiphany moment for me because at the time we're building all these different plug ins for video.js to support these different frameworks. But if we can make video.js look
like just HTML, then maybe we didn't have to do all that work. I thought that was pretty great. So let's extend that analogy maybe a little bit further and say, what if everything talked video HTML? And by everything I mean, what if features talked video HTML, players talked video HTML in a way that they talk the same API that the video element exposes today. If that were the case, then I could build a feature for a player and then potentially easily swap it over to another player because they're speaking the same language, the same API. Which brings me to a quick PSA.
Stop redesigning the video API. I see some new players out there that are creating different APIs from the video element. I don't really understand why, say, oh, it doesn't have a stop function.
It's called pause. Current time to go zero. It's not that bad. And you're not adding anything to the ecosystem by coming up with your own names for things like seeking and other functions. What you are doing, however, is preventing something like, well, what if Apple, who's created their own player on top of the video element and customized controls, what if they want to switch to a new player, a new player that supports ads or has better adaptability and they're looking at your player? Wouldn't it be great if they could just swap in your player for the video element, which is what they're using today, and be done? And of course, the other thing we can do if everybody is speaking the same API is build features that that work across players. Down to the core controls that drive video playback, So here's an example of just what it could look like to build a play button that points to your player or my player and just works because we're all speaking the same API. Of course, I built a demo of this.
So, here I have an example of just some basic controls. We've got a play button progress range, mute button, volume, full screen, all built as individual custom elements with a simple amount of functionality that can be moved around because their HTML styled with CSS and pretty simple APIs. If I go over and look at the result of this, you can see.
Basic controls. Because it's HTML,, I can go back and, let's say I want to do something like move the play button to the end of the controls, I can do that. I'm just moving HTML around, reload the page and now play button's at the end. Over here.
It seems pretty powerful to me to be able to move HTML and style with CSS, these simple concepts, without needing to get into JavaScript and learning unique ecosystems to customize your player. And that includes adding new functionality to player, like just dropping in new components into the player that all speak HTML and work together because it's all speaking the same language. If you're interested in trying out those individual controls, you can find them on GitHub under the account, under Media Chrome. Another thing we can do with this is create a playlist component, right. And drop different video elements, different video players inside of this playlist and just have it play through. I know it's an interesting approach.
This is the first demo that I showed back in 2014 when I was first talking about web components and this works and you can find the demo there at that URL. What else? Accessibility. We could build accessibility features that work across players like something that shows the descriptions on the side and speaks them out using the text to speech API in the browser.
That'd be pretty cool. I haven't built this. Somebody go build this. Or what about this, on the analytics? Everybody loves Stats For Nerds on YouTube. What if we add a component that you could point
at any player, any media element and see the same stats come up for the player? That'd be pretty great. Haven't built this either, somebody go build this for us. OK, here's something I made for no good reason. A video sound pad where I can pass it any video source, tell it where to start and end in the video. And any time I click the pad, it plays that part of the video. It's kind of like all those sound boards that some of us used to use to make crank calls back in the day.
It's kind of like that. Here's a demo. Here I have Charlie's brother saying "ow". >>Ow. Ow. Ow. Ow. Ow.
>>Kind of boring. And then let's load up a bunch of other memes. >>Bark. Bark. >>No. No. >>Cheers. Cheers. Cheers.
Kind of cool, right? Then, what else do you do in a pandemic? We have nothing better to do except then hook this up to the browser midi API so you can use it with your electric drum set. >>Cheers. Cheers. >Cheers. Cheers. Cheers. Ch-ch-cheers. Ch-cheers. Cheers. Cheers, Cheers. >>No. No. No.
>>Cheers. Cheers. Cheers. >>OK, then there's this, a picture of my car in high school, my Honda Civic. I loved that car and the cool thing to do in my high school was to put a subwoofer in your trunk and then drive to Del Taco and show off your bass to your friends by playing Ginuwine's Pony.
All it really did was shake your license plate a lot. But if you were here for my 2015 talk, you learned about the vibration API. I hope you see where I'm going with this.
The vibration API only works on Android devices, but if you have an Android device, pull it out, follow along, go to woofer.media. OK. If you're at woofer.media, you should be able to do this. (PONY BY GINUWINE PLAYS) I think that's the best thing I've ever created.
That is it for me. I hope you feel encouraged to use Web components, I hope you feel inspired to try something interesting with them. Thanks to Demuxed for having me again this year. Thanks to my copilot, Reid.
Hope you guys have a great rest of the show. And until next year.
2020-12-25 17:22
