Um I am John deutscher I'm the principal program manager for Azure, media services and, and, I'm working all same. Team same team been working together on media for over five years now yeah you've probably seen Martin do a lot of presentations, over the past few years on media services services, and VI so. I'm gonna cover today Azure. Media services and, what's, new with our v3, API and, then, Martin's gonna go through video indexer yeah absolutely, okay so let's. Let's start let's, get going so let's click this forward so if, you're. Familiar with the media industry you know we've all been going through this transition, from a mass media, broadcast, world to a online distributed. Over-the-top, delivery, of media now media. Has become kind of the primary, data type on the Internet as if. You have teenagers. If you have young. Kids that, are on mobile devices you already met it you automatic automatically, know that this is kind of the fact everybody's. Spending most of their time watching. Video, on their phones my, kids they. They're now this generation where to them popular, culture is a vine, and, from. Me growing, up it was always TV, shows Brady. Bunch or something like that that was my popular, culture but this generation, is now used to everything, happening, online and video being. Mobile, being persistent on your devices at all times so, what, we have in Azure, media services is, a, set of capabilities that lets you build apps, to. Reach that audience as, well as to scale out and provide. A set of distributed. Services for delivering, media around, the world as a media services targets, multiple, use cases so. Typical. Use cases would be around storage, and archiving of. Major. Motion. Picture videos as. Well as building out Netflix. In the cloud such. As, customers. That we've worked with like Talk Talk in in. UK, as well, as building, out live events, for. The Olympics, as well as the Super Bowl which we've done many times in the past. Corporate. Video galleries, we are the backend engine for. Microsoft. Streams so if you're familiar with the office 365 video platform, Microsoft. Stream, sits, on top of Azure, media services and, we provide the scalable, encoding, as well, as the delivery platform for them we. Also provide, capabilities, such as marketing. And advertising, so, the ability to do in-band. Signaling of. Advertisement. And then scaling. That out to do custom. Ad signaling. As well as customized, ad delivery out on, the network. Media, Services has three main sections that I'd like to talk to you about our. Core process, section, which is all about encoding, as well, as analysis of video so we have two, new processors, that we're introducing here at the show about, video, analysis, and audio analysis, that I'm going to go over as well as we support the, scalability. Of encoding, such as you can spin. Up as many encoders, as you need in one of our multiple, data centers around the world we're at 30 30 data centers and plus I think now and. You, can do this all with. The security, and the. Compliance. Of the azure platform so we support these. Several, compliance. Certifications. Such as Motion, Picture Association of America ISO. Sock compliance, and. We've guarantee, that your content is going to be secure throughout the entire encoding, workflow. Distribution. Side we have our own packager. Service so we're able to do dynamic on-the-fly. Delivery, of content in multiple formats and what, that allows you to do is deliver content to. IPhones. Android. Devices PCs, X boxes we, handle all that complexity, you, just upload. Your file we encode it we put it into multiple bitrate, mp4 files and then, on the fly we can, translate.
That To whatever file format you need to get to those devices. In. Addition, we provide the player framework, for you and I'm going to show you that in a little while we, provide as your media player which is html5. Player, framework, which, lets you play back all of these different file formats, on those devices, we. Can also scale, your delivery out through the azure CDN, so the azure CDN, supports. The broad, distribution and scalability that you need for getting out to millions of customers with, either on-demand video or live, streaming, of video as well on the. Protection side and the content protection call, them we, provide the. Dynamic. Encryption, capability, to encrypt, content, either for, standard, AES encryption if, you're just trying to protect the content on delivery, or if, you want full drm support, we, also support. Play ready fair play and wide, line which are the three major DRM providers out there that are native, to the platforms, that you're trying to deliver to usually. When you're delivering to a Windows Device or an Xbox device you're, gonna be delivering, play ready and there's multiple TV, manufacturers, out there that also support play ready if, you're going to an iOS device you'll want to use fair play and if, you're going to go to an Android device or something in the Google Google, ecosystem you, will use wide line so, our service, handles, the encryption, of all of those formats for you automatically, and then, we also handle, the key delivery of that so and scale that out as well so as you reach more and more users will handle the scale you need to deliver the keys to those customers. So. To, give you a quick hello world demo I think I'm gonna pan it off to Martin to show you what it looks like within Azure media services yeah, and you're, gonna show like an upload of a file right yeah so this is like the most common scenario John that our customers, and developers, ask so, we figure you know while we're here let's show you something really easy, so. That we. Demystify. Media services right so, we created this tool and you can download this tool off, of, our. We. Have a github. Site so you can download this is called the Azure media services Explorer. Tool or I. Put. The link right back up here so you can see it one more time so it's academic. AMS, II and this, tool was created to allow you to experiment, and try media, services without first committing, to a set of code and learning our api's and using, the SDKs, so, we thought this was a great example to show to dive. Right in what, I'm gonna do is I've got the Explorer tool it's connected to my Azure account I've got an azure blob. Storage account and a media services account that it's pointing, at and this, tool is taking, the place of the azure portal so many of you have used the portal in the past what's nice about this tool is it, only focuses on the media services, capability so it lets me do something really quick so, for example one, of the most. Tasks is i want to create a website that, has a video embedded on the website right everybody. Does this something and I can watch on a mobile phone, so this is the audience participation part, of the show you all can take out your phones we're gonna do this right now we're gonna take a file that's on my laptop move, it to the Azure cloud encode, it publish, it to a common, URL and then give it to you guys to playback and stream on your device so that no matter what device you have whatever, you guys brought in the room it will play with only one encode that I'm doing right you think the dynamic packaging feature that John just did so get your phone ready we're gonna do this together the first thing I'm going to do is upload a video file into my media store at my storage account so I'll just select the file that's local to my surface here I'll.
Take Microsoft. Cloud. Ad because, it's pretty short and it's, really fast so let's go ahead and just move, that in the first thing gonna do is what storage account do I want to use if you have multiple accounts so you can choose which one you want now, what is this tool doing well it's calling me API for you right it's just calling the API on your behalf so you didn't have to program it through, Visual Studio so the first thing I'm gonna do is just pull the, transfer. Up to my. My, storage account okay that was fast that's done so now I have what's called an asset that sits up there there's the mp4 file, it's called Microsoft, cloud insect's, mp4. But, I can't do anything with this yet this is just sitting as a video file in storage, it's not yet playable on your devices, but, we want to make it playable on your devices and we wanted to be published on a website so. I can use the same tool to invoke. Our next. Product, of the process that John spoke of which is the encoding process so. To encode it we, have as John said two different encoders that you can choose from one more standard, over-the-top, web formats, and one, if you're a premium, maybe, movie studio you're, using a lot of high quality maybe, multiple audio tracks you, can use our premium encoder but I'll just use the standard encoder for now and it's gonna come up and give me a whole lot of choices now obviously you know most. People just want it to play on everything, but you can select the preset you can select the quality you can control all these, different aspects, of the video encoding process, but, I'm gonna let it default, to just adaptive, multi bitrate mp4. So that it can plays on everything, that you guys and, just, go ahead and launch this encoding job now again all I'm doing is using the, tool to invoke the API that, are part of the media services, API and SDK and that's it's gonna adapt to the file that you uploaded and give you a bunch of different bit rates so, that it streams smoothly, across the network that's right and this is important because you, know as we all know we all connect to the Internet in different places in different ways some people aren't high bitrate networks, I've. Got a fast one at home yep I can't. See the offices all that fast this certainly. This conference hall has not been all that fast for for, a networking connection, and you guys all know this so, what we're what what I'm doing is I'm asking you to create a multi bitrate file, so that it'll adapt based, on and and then when you put that in the player the player then makes that decision exactly, exactly so I'm kind of like this is the cooking show, part of a part, of the show where I'm just waiting, for it to pick itself up off the cloud and and, part of what I'm I'm, running, this on is my own media, services account which is part of the scalability, of media service it looks like it finished it finished so that was pretty fast and that's really good so now let's look back at the asset so now I have an encoded, version of my mp4, that you can see it's a multi bitrate mp4, so the very last step now is to publish that and then for you all to play it so get out your your phone's what, we're gonna do is I'm gonna go ahead and, publish. This by, creating a URL so this is a well-known URL, that, I'm, gonna set I can set the date I can, decide you know when I wanted to expire either ten years from now or maybe just till you, know tomorrow, or Sunday or in the year 2028, or 100 years from now 300 years but yeah hundred, years well, I'll be here I'm sure create, a locator that, allows me to make this content, public, so that's running, and then now that's done and the, last part is to play it so, what, I'm going to do is I'm, going to go, in here and in the tool we've already embedded the player that John was speaking about it's called the azure Media Player and what.
I'm Gonna do is first, I'm gonna take it and play it back to make sure it looks right to me it's gonna ask me where I want to stream it from and I have a streaming endpoint, account and my media services and those are all the different protocols I was referring that's right and you can identically, change between these protocols for you and I'm gonna let it do automatic. That's. I want and go. Ahead and set okay and what it's gonna do is it's going to embed, that video on a player on a webpage on the demo site and I have it now well this is all impressive, because here is my video playing from the cloud it's, not as impressive if I take this URL that I just created and go, stick it over here so that you all can do it so get out your phones, I'm gonna, take my, URL that I have here and replace, the. URL with the new one that I just created, apply. It and. Now. As. Soon as that saves, there. We go here is the URL that I have pre created for you guys so, go to this website and what. You will see is the, video that I have created. Now. Streaming to your device so, this is the final step which is the playback, so you see I've done enough and it works and it's working I hear it different, points, right so, the, idea here is that we've tried to make managing. Video and. Using, video streaming. Video and as you can see from this website, example, embedding, video into, this both. A desktop I'm running a surface with edge I don't know what you had been but. Looks. Like an iPhone or, Android, if, anyone you've got an Xbox in their room or. Wow. Oh my god glass an ad. Fantastic. There. We go there, we go so, there we go you've seen the step of how easy, it is just to get to the first part which is bringing, video, to you could also progressively, download, that video as well I certainly to take it offline and you could side load it onto your Zune yeah. Because I want to watch this video on the airplane when I'm going, home exactly. Alrighty John back to you all right back to me so next, slide I can.
See It down there. Alright, so what we're announcing so. For. Azure media services we, have some major announcements, at Build this year we've. Been working really hard the team's been really. Just pushing, hard to get this out the door in time for build we. Have a new simplified, programming. Model and what we've done is we've, merged our existing, API back, into, Azure resource, management so, that we now have full, support, from arm for. Doing things such as arm, templates, role-based. Access control on, all the objects, that we have within our API we. Are actually now able to auto generate several different SDKs that, you guys can use that are a lot simpler, than in the past, previous, SDK, was tied fairly, tightly to the storage SDK we've separated that, out so. That you can now version. The storage SDK, separate. From our actual Azure. Media services management. API, as well we've, made it a lot easier, to build workflows, and I'm gonna show you some of that in a second we've, added as I said previously at the start of the event that we've added these two media analysis. Capabilities, which, is video analyzer and audio analyzer these are now built-in. Presets for, Azure media services and, they give you a lot of the same exact. Same features, that you had within video indexer, and we'll, talk a little bit about what the differences are between the azure, media services analyzers, and the video indexer side we. Also have the, new azure CLI, support, which I'm going to demo next, and then, on the video indexer side we we have a button now within VI dot the, video, indexer dot eight put a I I think's right that's right named Xai, video, today I and. At the video indexer site you can now log in and press, the connect button and it'll connect, your, video, indexer site directly to your Azure subscription so. You can now scale, infinitely. Within, your ad your subscription, using video indexer, you. Can also process in three three new regions we've expanded, out our global, footprint. For video indexer and we've. Also announced the introductory, preview pricing which Martin's, going to go over shortly, there. Is also a new REST API on the video indexer, side we've version, to the API there and we've added a lot new, capabilities. On the video indexer side and. Next. I'm gonna go through some of the new API features, on the video on, the media services side. So. What. I'm gonna actually do is I'm gonna start demoing some of these just, a lot easier to take a look at this so I'm gonna punch over to 8. Thank. You. So. What is new, in Azure media services we. Do have a bunch of new documentation. Up on our Doc's Microsoft. Comm so if you're, not following, along on all these these topics, you can read is offline later we, also have, a bunch of QuickStart samples, as well, so, new, QuickStart, for streaming videos and a tutorial for uploading and, analyzing, your video within Azure media services v3. There's. A new button down here that I didn't notice before also this downloads a full PDF so if you want to take the, documentation. Offline with you on the airplane click. That download PDF, file and you'll get all of the new v3, Doc's down. There for you we're still working on a lot of these so more Doc's will come over the next few months we're in preview, we, hope to get to GA quite quickly so we'll be filling in a lot of the details here waiting, for feedback from all of you so, definitely, send feedback on the new api's to our MSDN forum or directly, to me definitely. Would love it if you guys contact it contacted, me and sent me your feedback for. Build. I created, a repo, and. This one the URL is github. Comm. /. Jo. Hn d, EU /. Build, 2018. So. This is in my my repository. And up. Here I've put samples, around all of the v3, API s so I'll walk through here what's what's up here, we. Put in a set of arm templates, so now you can take a look how easy it is to use an arm template, to deploy a media, services account create. Assets, within media services, do. Simple, encoding. And pass through channels for live as well. As create a simple, transform, and we'll, keep adding to these so that there's more and more samples up here that you guys can deploy. In. The. CLI, folder, I have. Two, folders I have a bash shell folder, so, if you want to run bash shell scripts, and here I'll have a set.
Of Bash shell scripts, for creating. Accounts, creating assets how. To register with event grids Jobs, transforms, and assets and then. On this side I do have a set of vyas code, extensions. So, there is a extension, for Visual Studio code that lets you execute CLI, commands, and if, you just download that and you open these it's a lot easier to run these. CLI commands directly in vs code and get results back. And. Then. The. One that everybody's, been asking me for is dotnet. Core, so. We finally have a dotnet core library for Media Services this has been a bit of big ask for many customers, and, I've put up here several, samples, of using. Analyzing. Of a local file encoding. A custom transform, shows you how to set up your own custom encoding, settings. Encoding. Just, a standard preset as well, as encoding. And delivering, a published, a EES encrypted, video to the add remediate player and finally. An example, of doing live within dotnet, core so any of these will run on Linux Mac, PC. Just basically download them and hit, f5 and, then. Finally a postman. Collection for you guys so this, will let you see what's going on in the API and, if I go over to postman really quick give. You a quick tour what that looks like so. The. API itself, is a set of swagger files that we've published. At. The Asha REST API specs. Github. And up, here you'll see that there's a preview folder, for all of our Swagger's. This. Is what we've built the API off of so you can easily go out here and download these pulleys into a tool that you need to generate your own clients, from or just use, these as a reference, point and. When. You pull these over to swagger I'm, sorry you pull them into postman, pulley Swagger's in you'll. See that I have all of these broken. Out by their, actual use case scenarios so. I haven't gonna, exhume in a little bit on that so, you guys can see better there we go in. Encoding an analysis, section, assets. Content, key policies, account. Operations, streaming, policies, and streaming and live and it's, real easy to get started in here. Essentially. What you need to do first is get an ad auth token, so, when you're working with Azure media services we've, moved now to use a ad service. Principles for authentication, if. You're still using ACS please move over to a ad because, we're gonna be deprecating, acs shortly, and that's been announced quite a while ago so if you if you haven't moved a ad yet please contact me after the event and I'll talk to you, for. Getting an aad. Key here, all I need to send in is. My. Tenant ID and. Then. The, client ID and client secret. For. My Ashur application. And as. Soon as I send that in I'll get back a token, and then that token, lets me do anything, within. The API here in postman so I'll, let you download this and play. With this offline and. Won't go too much further into that right now and, then, once. You've got. The. CLI, installed. Go. Over here so. In the azure CLI, module, if you download the latest CLI, or if you log into shell. Dodger. Comm, which is the azure, shell, if anybody hasn't seen this site that's really, cool. Anybody. Else use this already, this. Is this is pretty awesome so. If you log in to shell Dodger comm, I get. A shell. Wait. Till it connects. It's. Busy. It's tired let's, let it go oh there, it is okay. Then. I gonna type in. It's. Not. Responding, yet so well wait till that comes back but I can do the same thing over here essentially. When you're in Visual, Studio code, you, can see that we now have these, AMS. Modules. For, Azure media service. My, zoom in on that so. You can see here it's very simple to create an account create a service principle, I can reset, my credentials. On and, what happens here and if I show you this this is actually really handy if you actually create, a service principle, and an, account and then i, zoom back out so you can see what comes back. It. Gives you all, the information you need I'll, reset, these keys later but. Essentially it gives me all the information that, I need to copy and paste into my app config, so. I can literally just copy and paste this XML into, a, visual, studio project or into one of my sample projects, so.
If I downloaded, one of those projects, I showed you and I can go into my app config, I can just copy and paste that directly, in here hit f5 and I'm off and running so we make it real simple for you to get started with. Your AAB credentials. And, then. Over, here we have some new concepts, in media services, we. Created a concept of a transform. Transform. Is. Very similar to what we had in the past for job templates, so when you did an encoding, job in the past when you set up a custom encoding, job you could have saved that as a job, template, but, now we've made this kind of first-class so, that you can have this within arm and you, can deploy them as templates, and you, can also put, our back control on these as well so, if you've come up with your own custom, transform. With special encoding, sections, analysis. Thumbnailing. You may have multiple steps within, a job that, output, multiple. Things, to storage. You. Can define that as a template, and then you can easily deploy that and we, support, having, multiple, processes. Write content, into a single, container into. A single asset or, into, multiple, containers and multiple assets if you want, another. Cool thing about transforms. Is that you, MIT Jobs now to transform. Directly, instead, of in the past we had a single queue of jobs, so, that you can control, access to these and allow, your users to, submit content directly, to a specific transform. And. I'll show you what that looks like here so if I've got this particular encoding, transform, I can create this really fast and. Then. I'm gonna create a video analyzer and audio, analyzer transform. You'll. See at the bottom they're coming back quite quickly so. Now. I can go, over to a job. Creation. And. In. Here I'm just gonna start a audio analyzer, job and what this is gonna do it's gonna take this. File, from, a URL so, Ignite short mp4, you'll. Notice that this is a SAS, URL, which. Is another new feature so now in jobs we can ingest from any URL. So. Typical, typically, it'll be a tokenized, URL be a. SAS. SAS URL, or, it could be a s3, could be a Google, URL as well as long as it's a tokenized, URL and. I'm just gonna kick. Off that job, you'll. See there on the command line that I've started a job named my first audio analyzer job oops but I didn't leave that asset. Hold. On a second. First. Need to create that you need to create an output asset, first and, now. I'm gonna submit that job with that output asset. So. What it's going to do is gonna ingest, that file and it's. Gonna run, the audio analyzer process, on that and then it's gonna output, the JSON, results, into that output asset container and, we'll. Come back to that. Now. I'm. Gonna show you what. Happens in. Event. Grid. So. While that is running I. Have. Already registered for event. Grid notifications. This is another new feature for Azure media services v3. Where, we can now a register for job, state, changes, job completions, as well as any events on a live channel or a live event and, we. Can use those to kick off workflows. So we use Azure functions. Or we can use logic, apps or anything, that is able to register for an event through event grid to, listen to these two these events. That are coming through. See. If this actually loads. Hello. Functions apt. I'm. Gonna go back refresh. The portal.
Okay, Let's go over to logic app see if logic app loads faster. See. Who wins, yay logic apps. I'm. Running out of credit. Okay. Here we see the. New event, grid listener, in. Logic. Apps. So. You pick, the subscription, that you're on you can now register for, media services events directly within, logic, apps you. Pick the resource in this case build, 18 is my account name and then. You can choose a particular event, grid, event and in. This case I'm listening for job state changes, I can, also listen for things like live event encoder connected. Disconnected. Things. Such as the data chunks are being dropped, issues. On that live stream so we're gonna be keep adding more and more of these events so, that you can have more, workflow, capabilities on, top of media services as we expand, this. So. This particular logic app is listening. For that job state change and then, it's looking for the data state to be finished and then it's just sending me an email at, the end and here. I'll look at the overview of this and you'll. See the last time it ran just because I ran it a few seconds ago I, should. See 456. And. Open this up. And, in here it was false. But actually I'm gonna show you the message that came through. So. It shows me that it scheduled that particulars, oh, that's when I just kicked off in the CLI that particular, job and when that finishes, I'm gonna get a final result there and that, I can pull down the JSON file within, that logic app analyze. That see what's in there that's useful for me and then kick that off to another process, as well. And. Then. Go back over to functions so functions works the same way. Let's, see if that loads yep. Give. It a second if not I'm gonna move on. Okay, so in functions you. Have your choice you can write a generic web web web, hook and I have a sample of this in my build. Repository. As well so this is how you take an event, and process this within as your functions or you can go back over and use just an event grid trigger directly, and, it's, a little simpler a little bit less code that, you have to write and then. In here you can monitor, for those events as well. And you'll. See these events coming through from that job I kicked off. So. Here's another. Refresh. That. Was five hours ago that was. Not listening. Go. Back to this guy here, few. Seconds ago. So. In here you can see the log this, process, it's finished and then, at the very end I can see that, I have a JSON, file, insights. Dot JSON that, was output and. If, I take the results of this. I'm. Gonna pull this up in postman. Just. Go get it get that file out of storage and there, you see. The. Analysis, that was done on my video file with. The text speech-to-text. I'll. Collapse this down so you can see the transcript, we have the keywords that were in that, file we have the sentiment, and we. Have the speaker detection, as well so you. Have all this information from, that video file which you can then analyze and, use in other processes, in your workflow, so. Now I'm gonna hand it back over to Martin. To go a little bit deeper on this. Video analysis, and these video index your capabilities, and show, you some more use cases around video AI awesome. Cool that's great Johnson what great, work on the api's and all the additions, so, if, you, were wondering that, last bit what was John doing with that analysis part, well what he was using is a new area that we've really been, really. Building in our portfolio, for the last 12 months and that's what we're referring to now as video artificial, intelligence, or video AI and.
The. The, thing about video AI is it. Really. Aligns with all the other things that Microsoft, has, really going. To be focusing, on the future with our AI cognitive, capabilities, right and so whether you're talking about doing, image recognition on, photos, or on, text, files on PDF, if you, have video content well that's where the Services team has really been trying to build those capabilities, into these into. This part of the portfolio so this is where we, hope you'll, find the, section really, exciting, because what. If you a lot. Of tasks. Using. These new video AI capabilities. That maybe your customers. Or you have been doing yourselves, manually, for forever. Right, and some, of these are the most commonly, done manuals, they. Are the transcription, of video content right I mean somebody's typing what I'm saying as I'm. Saying it to you and it's appearing on the screen but wouldn't it be great if that was being done automatically. Right or what, about if you operate, any user-generated. Content, websites, right now maybe you have somebody who Flags for adult content or, profanity or something, in that content that you have to scan, for again, that would be a great service to allow the cloud to do for you automatically, for. Years we've all complained, about the lack of true video search across. The internet you know how do I find a video that contains, this, word this, person, this. Object sitting, at this place right that's. A capability. That we've all wanted to have on the Internet but really isn't available unless somebody, goes and manually. Types all that metadata, in through. A tagging process well it would be great if that could be done automatically. And so, a lot of these imagine. If you could scenarios. Are precisely, what we have heard from you our developer, community as well, as the greater media, content, community around the world whether, you are doing, news content, whether, you have corporate content, whether your Education Center, like a university, or a, high, school or elementary school, or whatever your field is as John correctly. Said we are all using an increasing, amount of video but. We are not really, analyzing. The video very deeply, where we're able to now play it on all these devices but, let's go even deeper so that's what video AI is really, intended to do so, what we are talking to you today about is there are two ways now that you can access this capability, in medias, services the first is what John just described, to you what, will refer to as a sure media analytics, the ability for you to call upon media, on video or audio based analysis.
Of Video content that you're uploading into Azure and you, can summon these capabilities, and they will return for you these, values. As part of a JSON return file so faces, visual. Objects, the audio transcript, the keywords the, sentiment, that we see or notice. In in in the, audio track right and all. Of those are available to you on an individual, basis and you can call them from, our audio analytic, capabilities, just like John showed you but, we a year. Ago at this very conference we introduced, you to a brand new concept, which is the second way and that's what we refer to as the video indexer. The video indexer, is a bundled. Approach so it's a slightly different way to, approach the analysis, of your video content by, uploading, your content, into Azure and then performing, all of these tasks, in combination. In parallel. And with context. Awareness so, if I see a face well who is the face use, celebrity, recognition, and if the face is speaking, what are they saying and where, are they, speaker. Dyer is ation and on and on and so in fact when, we introduced, you to the video index or a year ago we, said to you we intend to bundle, these cognitive AI features, together well that's exactly what we've done, and today there, are all of these features are available to you to be using and I'd, love you, know has anybody used a video indexer, in this room over the last 12 months it's been available great a lot of you have well, there have been a tremendous use. Of usage of this we have literally. Indexed, millions, and millions of minutes sort of like McDonald's, hamburgers, now what's. So fascinating about, this is that what our customers, are doing with all this metadata is very, very different some, of you are using it for subtitling. Some are using it for translation, services some are using it for shot and object detection so I'm you're using it for all kinds of reasons and so, what. I want to make sure that we that, I show you today is if you haven't had an opportunity to play with the video indexer, well let's make sure that you you so, we created, for you a year ago, this portal.
This. Portal is a visualization. Tool so you can see what. The metadata, that we create looks like now. I should, say that, if I just showed you the metadata, it. Would look like this, and I'm. Sure everyone would clap. And applaud how beautiful, our metadata is and how powerful, it is now, this is lovely. To look at but if you don't speak JSON, or. You. Know you know or you don't have an editor like this to break down exactly. What's in the, metadata well we created this, visual. Gallery. Tool for you so for those of you in the room who didn't raise your hand like, I've never heard of the video, index or Martyn let, me give you a very brief demo as to what the video indexer is capable, of so, I have uploaded into, this, gallery, lots. Of different videos I just uploaded, the keynotes from Scotland, from sure that they gave this week I've had news. Programming, that we're working with customers around the world sports. Programming, you can use any video or audio content, that you like sometimes. This demo works really well with local, news if. You watch a lot of local news you know there's a lot of exciting. Stuff. That, happens in local news so let's use this example from, a station, right here in Seattle which is como news and I'm gonna go ahead and launch this we're, gonna bring the audio up just a tad so you can hear that there is audio, in. The demo. Play. That okay, so the first thing you'll see in this in this visualization tool is the source video that I used is. On the left and it's using the embedded azure Media Player on the right side of the screen, let's. Bring the audio up just. A bit and. I'm. Going to turn on that. Others. Taken to the hospital. The. Project, did, not hire a pre. The. Captioning, that you're seeing was generated, not by a human but rather by our speech to text, and. The. Words that are being spoken by, these folks on a news broadcast, I, can even show you the after the full transcript, by just flipping over to this tab here, and you, can see it's not only showing me the words that are spoken but, it's also doing an identification that there are different speakers so, as the speaker changes its recognizing, that there's people one there's a six. Seven. And that kind of thing that's happening with now. Many of our customers the, communis, tool just, to do captioning, just, to do the transcript, right because they have a video they, don't have closed captioning, that's available to them and they're using this tool to generate the transcript, file download. It visit. Video inductor, tool and apply it to the video this. Event, like. That, we're doing here at build it, does this quite a lot they have a lot of content they use this tool they auto transcript, it they pre publish it and now it's available with closed captioning, that can be used and looks like my network connection just died so I'm gonna relaunch. The. Video indexer, and we'll continue down on the demo, so. Transcription. Is a very, common, thing that's asked for what else does the video indexer tool give you the ability to do well, if you noticed in the transcript, because I have all the speech-to-text created. I can also invoke. The. Trent and Microsoft translation, services we've already built in arts right now turn. That audio down a little bit more. So. That it doesn't it, doesn't interrupt. Me so we are already connected to the Microsoft, translator service. So for example what if I just want to change the entire, of. The. Entire. What's. Happening in my network connection there we go to, Spanish, any Spanish speakers in the audience, let's. Take the entire metadata, set that we've created and translate, it into Spanish now, it's available to me in Spanish. And. Let's, bring up. The. Transcript. Again and you'll see that it's already translated, for me into Spanish and of course the Microsoft translator service, supports well over 50 some-odd languages, so I could have done this to any language this is the second use case it's very common using, these capabilities to, translate, into other languages, for publishing, in my videos that, I don't have to hire translators. To do so. That's probably, another, very, common use case what. Else does the video indexer provide for you well you can see here biting face recognition. And, an identification there, are 98, different faces, in this video. Here. They all are with thumbnails and you can see down here below the, black line indicates a video. Those, that vid that face appears, Somali. Shinji. Appears for. Fourteen. Point nine five percent, of the video and. All the places where Molly's faces, appears to the screen is here with, links, to be able to jump from point to, point to, point everywhere.
Molly's, Face occurs, on camera with, the relationship, between them now how am I able to do that well of course just. Like I showed you the faces are all in here in the JSON here. Are the faces, here, are the instances, of the faces, here's Molly space and these are the 27, appearances that you, see. The, tool is giving a visualization. Of JSON, the, JSON is the power right, JSON is the value of this tool let's. Go back to my, demo. Where. Keeps dropping in and out on I gotta love that. Now. Other things that, John mentioned are things like keywords, words. That occur but. Maybe. We'll bring it back to english so that we can all we can all see it together let's. Bring it back to english so we can see it other, things that are in here are keywords, so. The, story that they're doing on the bridge collapsed, the story that they have a double murder, gotta love local news story. About binge, drinking, and just, to get, to those stories, labeled. Very. Common. This. Is object. Detection right. So where. Is there an aircraft, in my video where, does the color yellow appear where is their mountain, these. Are all ways to make your video searchable, right, so to, the extent that you find all of this stuff to be valuable that's, where we're providing it and i'm. Gonna go back to my little chart here my. Slides. We. Continue, to add more and more metadata. All the time. These. Are things that are our roadmap in the, months to come, the ability to detect emotions. The ability to detect logos. And even, the ability to take a live, video, stream in, and analyze, it on the fly a very. Popular, request but you can imagine how, amazing, that would be to, be able to do this so that for example as you are watching something you could see it in real-time in another language that the captions, coming at you in real-time and I, can share with you that we already have customers, testing. This already, so, it's that close to being available the ability to do this we have customers already working, with us to make this possible, so, yeah, and that cool I like. Applause so, as you. Can see from, this demonstration while. I have been doing a lot of clicking, all, these clicking, things that I have been doing in this demo, obviously. You. Are going to access, via our api's and John mentioned a brand-new set of API, is that are now available for video indexer. And that. You can use so there the manual, part of me just clicking around are you clicking around while, if lovely in a demo that's not how we expect, you to work with the system right. So. What, are we announcing about, video index are at the show for those of you who have, said hey I've been using this for a year so what's new well what's new is as John said, earlier is, now, a paid, service, we have gotten, to the point where it's not just a free demo it's not just something that we're experimenting with, this is a real service, and we are super excited to, bring the video index or into the Azure fold, and make it a formal, compliant. Azure, service, so that's, awesome, yay, for that so how do you take advantage of using that if you, have using, video, index or for the past year you know that you have been quota limited. We've only allow you to either use maybe, a couple. Dozen, hours to maybe up to 400 hours we, have removed the quota so, the way you do this is when you're in video indexer. You just connect, your video index or account to. Your Azure subscription and, by. Doing that you, are now able to index. To your heart's content, you will use your own, Azure storage account you will use your own Azure, media services account. And all the videos that you upload and, index. Will, be left, there and you'll be running it there and there. Are now new billing, meters that go with it so we are announcing at the build, conference, two new billing meters do this service four. Dollars an hour for video, and audio processing, and that's all of the, processing, all twenty. Of those different boxes that we showed you this is why we refer to this as a bundled. AI service. So, it does everything, all at once for one fixed.
Price Per hour or, you, can just do the audio portion, if you're just indexing. Audio content, only, as. John. Said we have previously, only had this available in one data region in the world we've just expanded, it by yet three, more just, for this week and we expect to expanded, by even more so, that it will be present in all data centers possible, as soon as we can roll that out so right now it's available in the US it's available in Europe it's available in Asia with, all the other data centers we intend to roll it out too. So. Now. I kind of want to give you a clue as to once. You've got you have your video indexer, account it's linked to your Azure you can begin to use it what, can you do as a developer to take advantage, of it the first thing is and. The workflow right, so just like I showed you when I was running the demo using the UI the. Way it works is you upload, a video into, your storage account via asher right. Or as john said you can use the new REST API to, get things from AWS. S3, storage, or Google or Dropbox, or wherever you keep your content you can point those URLs into the video indexer, and ingest. That content in we, will then create for you and extract, the metadata and we now create a JSON, but, we don't stop there because part, of the power of AI is the ability for you to customize and curate so, we are now taking advantage, of custom. Language, custom. Face and soon, custom, object, model recognition. So you can help make the data that we extract, from your videos, much, more accurate much, more tailored to your, particular. Content. Right and that's very important, many of you have content where you use a lot of Street words, or slang, or, three-letter, acronyms, many of use localized. Celebrities, that aren't well known perhaps, globally. Many, of you have different. Names or different landmarks for places maybe, you want, to customize things so we are going to this, is going to continue to follow the path that all the rest of Microsoft's, cognitive, many our capabilities, are by giving you the ability to curate and customize, now, you have a choice you can take. The content the data metadata you've created and download that off the cloud you. Can integrate that into a third-party tool or you, can leave all of it up in your Azure, account link. It to add your search and do. What I just showed you with, this demo which is the ability to search right, so what does that look like well, if I have, a lot of videos as I as I do in this is, this in this instance, and I want to be able to search for things like I, want to search for anywhere. Where the word cloud appears, across my video I can do a word search or maybe, I want to do an object, search like where is there, a reference to coffee either, the object coffee, or the word coffee or, the name coffee and I, can get these links I can do complex searches right like oh maybe I want to find coffee, but. Ones that are held by the actor, rami malek who, many, of you may recognize as, one, of the actors in the series mr. robot right so in this case this, is identified. For me not just the fact that there's coffee but. In fact that there's being coffee being held in a video frame by the, actor, rami malek present.
So Complex, cool, video, searches, become, possible. When you combine either, search and metadata. Extracted. From this kind of tool. Back. To my slides. The. Building blocks within video index or should be pretty obvious to. You by now they, are the portal that I've shown you a. Platform. Set of services, for indexing, search, and editing, and encoding and, streaming. And, those. Widgets so, those widgets that are on the screen are embeddable widgets, them cell if you like them the Player Widget and the insights widget you can just embed them as is into your website but. We, anticipate. That the most common use case will actually, be the use of our api's and, via. Those api's, that's, how you'll interact. With the system and extract. The JSON so, it's at this point that we expect, that you are going to be using the the, system in that way you, call the api's and, then extract, the JSON that you need and pull out whatever you want and whether, you're looking for faces or you're looking for words you're looking for objects, or whatever you're using and since. The most common, way that we work with all of you is, through. The development, of our eyes you, know we expect, that our partners. Are the ones who are become, most versatile, with these ap eyes and the ones who are going to embed these capabilities, into their products so I am super, super excited to introduce Ben, cus from box who, is one of our earliest, and most aggressive. Partners. Who have taken the api's, that we've done with video indexer, and with, our video AI suite and said you know this, is capabilities, that we want to expose to our customers, by merging, your capabilities, into our platform so I'd like to introduce Ben to talk to us about how box has done precisely but, then, thank. You. So. I'm then, chris, vp. Of product at box and if, you don't know what box is we do what, we call cloud content management so. I, think big. Companies enterprises. I think, I'm a slide I was, listed as a corporate, content we, do all of that we let people upload. All of their different, files we let them Bua we you had that we let people share. Their their information, that be, part of work, business.
Process Workflows and so on we, work with a lot of different companies of all sizes across. All industries, and. One. Of the interesting things is if you look at kind of what, these kinds of companies are doing these days in terms of what are they putting in box an awful. Lot of it is unstructured. Content or. Media. Files audio. Video, and images, in fact video. Is our number 1. File. Type by size and it's being growing quite rapidly, one. Of the challenges, that these companies start to have is that that, you get a lot of video people are uploading. For. Their themselves. Or for the different, business, processes that they're doing and they're, unable to actually. They're. Able to view what they're able to see to go look at it and they can tag it so they can find it later but they have a really hard time when you get enough of it to find. It and to organize it in general and so, we've, never had a solution for them we. Always just kind, of we treated videos just like a big blob of data and. Then. But. Then we saw some of the capabilities, that, from. Video index theorem from our friends at Microsoft and, so, what we did is we built a new new. Feature around it we built and something, we call box skills so, you can think of box skills as the ability to enhance your content, using, techniques like the ones that we just saw from from. Microsoft. And. So we have taken these ml api's maybe put them in, automatically. Applied in the files box and. We can we call these skills. Like image intelligence audio and video intelligence, and, I. Will show you a demo of these so, you can see kind of how it works. So. So. Similar. To what. You just saw, almost. All of the, features. That you. Guys were just looking at are interesting, for our customers, so. In this particular case this is a video. From. Mad, Money and it got sent around our our company and you can imagine that every company has this kind of thing or they had like a video or like hey check this out this is a bizarre, CEO he was on Mad Money talking about box and. This. Happens all the time like. We deal with tens. Of millions of hours of video per, month and. So. These. These files get sent around and people say ok cool but it's a it's a it's a seven minute video I'm kind, of busy like I want to see what this video is all about and so by taking these features that we just saw we can do a few things we can see a like. Who's in the video and I can see that our CEO Erin's in here and actually he was a celebrity so that's that's nice and, then I can jump to the part of the video he and then I can also extract a topic so Samia is a corporate user if. I'm looking at like a internal. Training video or some other video that people are sending around we, can look and say I. Want, to see the part where they talk about. Salesforce. And then I can click and then it shows me the pieces just like you saw in the video in Bexar. And. So then, you. Could then jump to that as you guys saw in the in the video they're so. And. It's similarly all of the data all the information around the transcript, is in, here I can go click to that I can read it I can do it and, so. On and, so one of the nice things for us is that for the first time in in, cloud content management, or the very first time for a box, is the idea of searching. Your videos and being, able to not only find. What you need for. The all, of your bit throughout all your video libraries but also find what you need within the video when you're viewing it and. We see this not just for video but for audio a lot, of our customers have. A lot, of audio files as well so, things like earning calls meetings. And, so on you. Can apply these same techniques so that we can go in and see like what is this earnings calls about it's a 45 minutes call I don't really wanna listen to it I just wanna jump to the part we talk about artificial intelligence and, I and I jump across or I can read the transcript, these.
Things Are from. Our view are. Changing. The way that corporate, about, our corporate customers are dealing with their their files and, and, just by uploading them to box they get these kind of functions because. We integrate with with. Microsoft we, have a lot of customers. Who are, in. Universities. Or schools and, they, are constantly now recording, each of the different sessions that they have lectures and one, thing they want to do is they want to send it out to all of their all. Their students but. They have to wait for the subtitling. Because they have to. Make. Sure that it's available for, people who can't hear well and, I usually takes them a little while as they go through it so by adding it to a box were able to let, them just simply upload, the data to box which they've been doing for a while and then send it out immediately with the bomb both with, all the features that you saw here including the topics in the transcript and so on we. Have a customer, who's. The. City. Council's and, they wanted to take who's, a city council of southern california and they wanted to take all of their city council. Weekly. Meetings and publish. Them online but, then make it easier for people to go view which they do through. These. Similar types of features, here and. So these are some examples and. What. We're finding with our corporate customers is that there is. For, all of the many different reasons that they use these types of videos and audio files just. Through, the day-to-day work by. Applying this type of machine learning from. From Microsoft. Were able are able to just do everything better faster. And become. More productive through the way that they work. And. So. I'll. Tell you a little bit about how it works and. How we integrated, with it it, was actually pretty straightforward, so. The. Data from, our point of view is stored in a box and. So the, way what we did was whenever somebody uploaded a file the box behind the scenes what we did is we created a, basically. A web hook that, would send data immediately to or would send a job. Immediately to as your functions. Server. This architectures work really well for us in this case because in some cases we're dealing with, a. Peak. Load of maybe thousands, of files a second and so, we wanted something that scaled very well and and. Then the video are. Those your, function which we call the video intelligence skill then goes and gets the file it could be various it, could be fairly. Big in some cases and then and it sends it to our, media. Analytics. API or the, the. Video index or service and then it then turns around and writes that metadata that you saw which then shows up in box so, from our users point of view from from a corporate customers point of view they, just upload a file and then it gets that that, all of those great features that you saw and, it. Becomes, immediately searchable and has all. Of the benefits, so. We've. Been very happy to be. A part of the trial and we are going to be launching this in GA in just a few months and. And. Where, all, of those new features that you saw from the Microsoft. Video. Indexer, we're. Also going, to be incorporating those and so, it's a box so that the customers can not only just have this work very well. Just. For their standard content but also customize, it so that their corporate. Employees. And users can see things, customized. To their needs internally, and. So. What. Ben just alluded to was, sort of what will will sort of end with today which is how can you take advantage of the same kind of capabilities, to integrate all of these things together in, other words tie it all up and use as your logic apps at your functions media, services the new event grid that we have available new. Three. Api's and, the video index are all that together and and who's doing that and how are they doing it so as, I think Ben just orchestrated. This but it's very specific to this scenario, that box is talking about this, is precisely you know the, most common architecture, for, taking advantage of all this all wrapped, up, we. Have, the media services capabilities our customers, are building, and partner and developers are building Azure functions that call whatever they need they're tying it to other parts, of the Azure ecosystem. Whether. It's notification, hub or cosmos DB for storing their video or using power bi to visualize, their data. Whatever. Your. Particular, scenario. Happens, to be and then they're tying it all together with the orchestration, known as logic apps and of. Course our, media customers, do this all the time so you, know I've got a customers. That, build very, simple, to very complex, logic, apps this is an example I know it's hard to read but this is one that basically says every time I upload a video into, my blob storage account I want you to encode, it I want you to index, it I want you to translate it into three different languages I want you to republish it until the two videos with all these subtitles and once you send me a text I want you to send me an email I want you to tell a million, people who registered.
That There's a new piece of video about that and do all this stuff right so you, know in that would be a quite, a daunting task to construct. That because I'd have to learn the SDKs, I have to connect all these things logic. Apps sort of washes. All the a and, now allows, you to do these things in order and then deploy it at scale so. You know the ability for me just to set it and forget it depending, on how much usage my system needs to generate that's precisely the simplicity that, we're doing and we, have already build for, you a whole, lot of Azure functions, that you can just take from, github at. This iCal, link, which is AMS functions, so if you'd like to use these functions you, can john showed you the repos that he's got for some of the other capabilities, as the api so we've got a lot of code that's already been written for you to, automate, the deployment of these media scenarios, so, what are some of the most common, media scenarios, well I've kind of alluded to them been covered, I think some, of the most common ones but, I'll just share a couple with you that I'm working with with customers, one, customer is in the news business and says you know I'd love to create a, personal. News, service, you, know the idea that I am, too busy to watch the news so only send me notifications or. Links to watch videos of stories, that I'm pre registered by topic, well, unless you had an army of people that was going through every video that said well this video had this in it in this video had this person in it and this video was about this you, know that would be very hard to do well not any longer because, what they've done is they've constructed a, logic. App that basically says I upload, a video into, a system it goes and sends it to a j''r it indexes, it and puts, it up and then it scans, through the topics the keywords or, the objects, or the faces and user. A who registered, to be notified when there's a story about volleyball. User, B wanted business user C wanted swimming, or whatever they. Can create, a true, personalized. Service, for their, consumers, who want to be watch. Content that's specific, to a particular topic, or a person. Or an actor or whatever that happens to be so these are the way they're doing and of course they're integrating all the capabilities, that we have within a juror and media services similarly. I have customers, that as I mentioned earlier are running, these websites, where they're having their users upload a lot of user generated content they're, looking for a way to automate, the policy. Of whether, that content should be published, or it should be flagged for review so using, the content moderator services, instead. What they're doing is now that content gets scanned by our content moderator services, it checks it for certain, metrics, maybe it's certain words may, certain images maybe it's whatever their, business, rules are and, that's obviously different in every country around the world so everybody defines, what their particular. Standard. Is and then if it passes muster then it gets published or it, gets sent into somebody's, inbox, for publish if it, doesn't if it gets flagged then somebody else looks at it but the act of doing the the, analysis. Can be automated, and. Save quite a lot of time I have, another customer that does a lot of this with syndication they, create, versions, of all of their video and audio files depending on, but. Maybe I'm sending. It to websites maybe I'm sending it to people. To distribute, maybe I'm creating a Spanish language for, this region in the world an Arabic language for this region in the world I need, to turn, my one-person. Production, house into, the capability, of a hundred people can. I use this process to really add that kind of scalability, precisely, what we can do with this stuff just by combining the events the, translation, service is the transcription, services the re-encoding, process and the publishing process all together so, that you can move content, around the world as you needed one. Of the most commonly used things as I said is working, with customers that just say hey, I need just transcripts. I need subtitles to be created can you help me with that and, so you know every time I upload a video file create, a subtitle, for it and possibly.
Translate, It into 50 languages while you're doing that as well very, very common, thing we do with these events all the time if you're hosting events you, probably have to put your content up on a website you don't have the kind of manpower to. Transcribe, and translate all. Of your content this is one way of doing that in an automated way many. Of you have used consumer, video services, where it's like if you like this video you'd like these videos, kind of recommendation, services, why, not apply that in better. Ways it's not just enough to be, I've, watched you know a lot of. Sci-fi. Or. Romantic. Comedies, you, know the way we do John, Rose maybe I'm interested in happy, movies maybe I'm interested in movies that have the word Terminator, in it maybe I'm looking for you, know movies, that have a preponderance of the color red however. You're, sort, of you know is, how. Can you combine these and, analysis. Video analysis, capabilities, with, a the, cloud capabilities, that we offer you build a better smarter. Deeper. And what. I'll call is objectively. Subjective. Recommendation. Engines, that really take advantage of the best in AI that, we have and so I've got customers who are doing this you, know to build things like this you know better, user. Experiences. That, are maybe based on a key metric like an actor or a word but, they're trying, to have the system build, it for them as opposed to a human, building. A Matt. Damon website and a Julia Roberts website. And a CEO of my company website. Right have the how the system, build those, things automatically. Based, on the auto categorization. Of, this, based on the data that we create so. These are probably. The most common, sort, of scenarios, and all, of these things are just simply logic. Apps calling. Functions, building. A set of logic and, data and then, performing, tasks that the cloud can do on your behalf and and and and the power of course of logic apps is you don't have to learn a lot of different SDKs because, they're all sort of integrated via the swagger code, so.
With, That I'm gonna invite
2018-05-11