AWS re:Invent 2024 - NVIDIA AI startups: Innovations in action (AIM121)
Ho Hello. Move us over here. Hi, everyone. Thank you. Thanks for coming. Welcome. Let me move this on over. Ok. Thanks for coming. Good evening. Welcome,
Jen Hoskins Global Head of Cloud Partnerships for inception. That is Nvidia's start up program. Thank you all for being here. I don't know if anyone else had an adventure on the way here, but it's Vegas. There's always something I had a flat tire this morning. That was a lot of fun.
So thank you for coming. Really excited. You're all here. I do a lot of these panels moderating. I'm not a speaker, not good at that, but this is the most exciting panel. I have to say we have six of the most fascinating A I start ups really big names. I am sure you've been reading about them without a doubt. So I'm very, very excited to welcome
them up. We're gonna split this into two sessions because we have six speakers, which is a lot. So we'll have three on stage at a time, splitting it into two with our co-founders to start. This is the whole group. We'll have a co-founder CEO first section and then we'll move into the Growth Strategy section for the second half. All right. So
without further ado, I wanna welcome up our first three speakers, whatever you want. All right, first. Thank you for being here. Really exciting to have you introduce yourselves. Please
name company. All right, I run Dawla CEO at contextual A I also an adjunct professor at Stanford. Um So what we do with contextual A I is we, we started from this observation that everybody was very excited about gen A I. Uh But everybody
soon after they discovered gen A, I became very frustrated because it didn't actually work for their enterprise use cases. So we knew that uh part of the solution to these problems would be R A retrieval augmented generation. And we knew that because uh I am one of the pioneers of Rag.
Uh I was uh the project lead of Rag when I was at Facebook a research. And so uh what we have built a contextual A I is R A 2.0 doing Rag the right way. And we have a platform for uh solving these problems. I'm Manjal Shah, uh co-founder and Ceo of Hippocratic A I.
Um I'm a serial entrepreneur. It's actually my fourth company. So I can't get enough punishment a start up. But the um uh what we do is we built a general A I model, a vertical agent uh specifically focused on healthcare. Um
We did not try to build a doctor. We actually think that the right place is what we call in the scope of nursing because nurses don't diagnose, they don't prescribe, but the world is short. Um millions and millions of nurses frankly, especially after the pandemic. And so we're focused
on building that and deploying it for health systems and for payers as well as for Pharma. Uh Sorry, Will Falcon. Thank you for having me as well. I'm the founder of Lightning A I creator of Pytorch Lightning. Um Anyone who has heard of Pytorch? Raise your hands? Yes, great. Who uses lighting as well? A few of you? Ok. Well, I'm gonna convince you today to do that
um as a platform. So lighting A I um created Python lighting. That's one. But second, we're an A I development platform. So we basically give you a set of tools that allow you to train, deploy models, monitor them like literally everything under the sun from running batch jobs to cleaning data, to deploying them to training, etcetera.
Um Yeah. And um I don't know, I'll get into it more, more as well. I will, I will plug out that way because um you know, he was working on rag uh when I was at, at fair as well. And um at the
time I didn't think it was like super interesting, I guess. But, you know, many years later, me neither. Honestly, many years later it shows up and I'm like, oh, this is what you guys are working on. So, uh
yeah, definitely, definitely great to have him as well. That's awesome. Thank you all for being here. I really appreciate it. Ok, let's talk inspiration. I will start with you. What inspired you to
co found contextual A I and how are you creating more human sense, conversational A I systems? So, when we started the company, as I said, we saw this frustration and we knew that we had a good solution for moving past the demo. So what we could see was that a lot of companies, they were coming up with all these interesting use cases, some of them may be a little bit too ambitious and then they were not able to actually solve for these use cases. So they were getting stuck at the demo. It just wasn't good enough. And so we started the company because we wanted to make it possible for people to actually get to what we call the production bar. So make it, make it good enough for actual deployment in the real world.
And so yeah, the, the journey of the company has really been about building a platform to enable that. So that's our R A 2.0 technology. It is a platform that has all the enterprise, great features that you would expect and you can solve these problems in a matter of minutes now. Ok. Let's talk, fundraising. Let's stick with you for a minute now. Um So have you, what have you found effective
or fundraising as an A I start up in a competitive landscape? I think you're all experiencing the fundraising, you're deep in it, you're in the thick of it. So, so I, I guess if there are start up in the audience, I would say, just make sure that you're very different from everybody else. It sounds very easy, right? But you, you need to be differentiated and I don't necessarily mean on the technology levels, there is a lot of disdain around companies that are like rapper companies like you're only calling somebody else's api or using somebody else's language model, that's totally fine. What you should care
about is how you are differentiated. So if you're in a specific vertical, great example here or if you're in a specific area, you need to make sure that you have a very good story about why you are better than everybody else. And if you have that story, then fundraising is very easy differentiation. I like that same question. Fundraising a make sure you put the words Ja I in every slide.
Um It is, it's a tale of two cities out there. If you're trying to raise money for anything, that's not gen A I right now. Good luck. And if you're raising money for jai, I mean, literally our very first round um was on a Sunday, five of the major VC firms came to my house and pitched me and the founders on why we should take their money. We never even pitched. We didn't even have a deck actually. Technically.
Um, and my, my wife and my son who's in college were sitting there and my son after that was like, I'm gonna do this start up thing. It seems really easy, like it's not always this easy. But, um, it is, it is something that I would say the ja I focus is one I think what he said is actually correct, make sure you differentiate well, um and also reframe the conversation.
I think that one of the things people have not are starting to get their heads around now is this realization. And I'll, I'll give you an example in in healthcare. So healthcare is $4.4 trillion in total spend in the US software and health care is only 100 billion.
So the whole software market software and it is only 100 billion nurse wages is 750 billion nurse openings that can't be filled is another 150 billion. That means the wage market for nursing is 900 billion. You know, what else is 900 billion, all of Sass. And so for the first time, we have technology and A I that's smart enough to really go after frankly, you know, some of these elements that we've not been able to expand into and, and, you know, even just solving the unfilled roles that we can't fill is a market that's bigger than the software market. And so I think part of it is recasting what's happening with gen A I into not just the technology wave or an A I wave, but really a, a set of tools that are going to transform how we work.
Um And the total available market is so much bigger that honestly, they can pay more valuation wise for these companies because the exits will be bigger. We'll get to you on fundraising in just a minute. You're deep in the thick of it as well. We let's continue with you. Super staffing, explain what it means and its potential to transform healthcare, please.
So this is probably another thing I would say if you're fundraising to talk about this as well. But um when we focused on building um Hippocratic, we said first we were going to do autopilot, not copilots and everybody else was doing copilots. It was 18 months ago. Remember copilot this copilot that and actually copilots have struggled a bit if you look at it. Um and we said, no, we're gonna do autopilot. Why? Because making somebody 10% more efficient, you almost never see that.
I would say the coffee machine takes your 10%. The kids soccer game takes your 10%. Like you need to get a bigger jump for the person buying your product to actually materialize the the benefit. So we went with autopilot then we realized there's when you drop the cost of anything 10 X. So we charge $9 an hour for our agents. Uh And, and normally those same clinicians are about $90 an hour, especially $90 per hour that they're actually on the phone, right? Because they have time taking notes and they have sick days and vacation and benefits and taxes. And
so it's really this big, um delta in that. And if you look at that element, um, when a cost drops that much, you don't just do the same things you did before cheaper, you do novel things you never did. And so super staffing refers to this novel idea that says, you know, I'll give you a concrete example. We have a, a big pair that has a Medicare Advantage book of about 5 million patients. And in Medicare Advantage they get
paid a fixed amount per year per patient. So if the patient goes to the, er, too many times they lose money. Um, and if they get too sick and need lots of, er, or hospital time, they lose money. So they said, you know what in the next heatwave? Can you call these 200,000 out of our 5 million and do a heatstroke assessment and then send them an Uber and get them to a cooling center. And can you do that at the hottest? Two hours of the day, every single day of the heatwave? I'm like, yeah, perfect use of A I. So
the idea of supers staffing is this idea that with this technology, we not only can do the things we do and fill gaps, we can't staff or do them cheaper, we can do novel things that will change health outcomes, make the country healthier in ways we've never been able to do before. And the math never penciled. It's amazing. I don't know, health care, health tech A I always inspires me quite a bit. It's the most fun and interesting. Uh William, you're deep in the thick of it. Like I said, your
fundraising, you just announced a around. Congratulations. Tell us about it. Yeah. Um So our latest round was 50 million. Um So we've raised over 100
and eight so far in total and one of the big investors was NVIDIA. So excited to have NVIDIA on board. Um They're also joining our, our board as well, which is really cool uh among other investors, uh including like Cisco and JP Morgan Chase and so on. So I, I, you know, the journey for us from a fundraising perspective has been a bit different for sure. So, uh you know, the company started uh like January 2020 fundraise like November 2019 before all the JA I stuff I was up fair at the time. And um you know, I had, I had a radio open source piper sliding and it was kind of taken off in the grand scheme of things. The attraction was very small compared to where it is
today. I think at that moment I had, we had about 60,000 downloads today. We have over 100 and 60 million downloads, about 10 million per month. Right? But at the time 60,000 felt like a lot.
And so I got emails from all these V CS being like, hey, what are you working on there? And I was, you know, deep in my phd and, and they were like, oh, well, you know, you should leave and to start a company. And at the time, um Emma Labs wasn't really a thing yet. People were like coming to me uh saying, hey, I work at this company. Can you help me train models at scale and so on? And I was like, yeah, this seems like a bigger problem. So I, I left Facebook and then started the company and we raised like 3 million that fund raise, took like a week, right? Um And then the, the follow ones, the following one ones were pretty good.
I would say that the number one thing that I have learned, you know, I'm not from Silicon Valley, I'm actually from Latin America and I'm not like an insider. I think a lot of founders are right. If there, there, I think the the world splits into two categories. If you're coming from like a known place, like you have, I guess, fair kind of counts for that now. But
back then I don't think it was on people's radars that much if you're coming from like, uh, a Stanford or something like that. And, you know, people in the BC world the bar is different for you. It means you can raise money just by knowing people if you're not coming from that. Like I was, you already have to have significant attraction to get that funding.
You have to understand that, right? Because I don't think people understand that. So, you know, I needed to have over 60,000 people using the thing ton of traction, et cetera, et cetera. Before we even consider a seed round today, you can go and start as long as you're coming from that network, you can go and pitch and get like three or 4 million because you're from that network, right? So you have to understand that the bar is different depending on your background. It is a bit unfair, but it is what it is. You have
to prove a record. Like, yes, if I went and start a new company tomorrow, it would be different for me now because I have a track record, right? So keep that in mind when you're, when you're fundraising and understand that some of these big rounds that are getting done are people who are already kind of in the system as well, right? Um But ultimately, I would say instead of spending a year fundraising spend a year building your product, get traction and then spend a week fundraising. It's a lot different that way. That's great. Um Let's
talk more about that founder, journey actually, Dal, if you wouldn't mind talking about your experience as a founder and then when we talk about you as a multi time founder, my experience as a founder. Um I, I, you mean how I got into it? OK. Yeah. So, uh I, I guess I was always entrepreneurial uh in spirit. I started my first company when I was 18 in the Netherlands. Uh it was called
Brainstorm. It, I was that cousin who could make websites. Uh And uh so that actually grew into a thing that uh was pretty successful. So we had to sweep stakes for soccer
tournaments. Uh But then I, I went to school and decided to stop doing this and get a phd. So I went to Cambridge, uh but I always wanted to start a company. Um But I never actually got the chance because after my phd Fair was just starting. Um And
uh uh I happened to have interned with one of the founders of Fair and, and he was like, you should join us in New York Young starting this amazing thing. I couldn't really say no to that. So I did that and then it was amazing.
So I couldn't leave and then finally was ready to start a company. But then I was talking to hugging face where they were like, why don't you become our head of research? So I also couldn't start a company then because that was too good of an opportunity. So I think that the timing for conceptual was really just right. I think if we had done it a year earlier, nobody would have cared about what R A even is. And uh yeah, the timing was just right where everybody was frustrated. We had the solution and we knew we could do better than that solution.
Sounds like a lot of A I right now, the timing is just right, quite honestly, this is the timing when you're multi time success founder, tell us about your journey. Uh So yes, I did my graduate work at Stanford and A I um you know, this was back and we call them back propagation neural networks. And um and I literally wrote the code that paralyzed it on a 64 node cray supercomputer myself because there was no such code by the way. Um But the um but then I did a I did two companies and one, we sold to Google, the other one, we sold Alibaba. Um And the second one to Google was a machine learning computer vision company. And then I did a healthcare company that actually was not as successful as my first two. And then I've done now Hippocratic.
And so, but I've also helped to start another five companies. Um I will tell you guys one thing, um which is this in this kind of wave analysis that I've seen over the years and, and that is that when you're looking to build a company, a lot of times you're looking for a space in the forest where some big trees fallen down. And so there's a gap in the market and the sunlight can now hit the ground and you can grow a new tree.
So in this analogy, that's what you're looking for. But then you grow that tree and you kind of hit other people all around, right? You kind of see that you're like, oh, I got to differentiate once every 5 to 7 years, it happened with the internet, it happened with Mobile, it happened with SAS and now it's happening with generative A I there's just a forest fire and it burns every last big tree and almost anything you grow will work. And now in watching this, I've also invested in 42 start ups and 23 venture funds. And when I look at all my returns and all that, they come, I've now come to realize whenever you see a dam, forest fire, you run as fast as you can and you start a company. And actually, and then somebody asked me in another panel, I was on, I gave us and I was like, should you not start a company in between the forest fires? And I'm actually like, yes, you should not, but we're in the middle of a forest fire. In fact, I still think we're early enough in this forest fire that there's some real opportunities uh to be created.
Um You know, the, the thing I worry about the most with Hippocratic because we're, we're out and ahead in this category and we seem to be right place right time. But I'm worried, I, I don't want to become Yahoo or excite and lose to Google. Like I do worry that the, you know, so I think there's still, there's always an initial uh growth and then there'll be a second rebound and sometimes the rebounds end up much better. And that's where the ending winning companies come out of. So I'm probably being very paranoid about that, but I would just say that there's a forest fire.
It just happened, it burned every last big company. People don't even want to just buy big companies products that pivoted to gen A I, they want gen A I native companies, investors want to invest in gen A I native companies. They do not want to invest in. I took some code and I pivoted it and I made this product because they just believe long term, those companies won't have the I pe advantage um to really sustain themselves if they're just taking other people's stuff and wrapping it.
Absolutely. Thank you. All right, let's continue on with you. Um M so, uh why focus on latency and LL MS and how has NVIDIA technology helped address this challenge? Personal plug. Yeah. So before I talk about that, there's one thing I do want to say, um this is the other thing to try to do is get Jensen to fall in love with your product because it happened to us by accident. We got the demo to him. I think
the inception program is the one that set up those demos with the V CS and, and some of their companies. So we ended up getting a demo to him, one on one with about 10 other companies and he just loved it. And because of that, he put it in his keynote at GTC last year. And then we just blew up like the guy literally made us and um and we got a ton of interest and then the NVIDIA team, um he put it in the Executive Briefing Center. So now every Fortune 500 CEO rolling through there got exposed to it. And then every time they
want an intro, we got an intro. So people are like, wow, how'd you get so much traction? How'd you get all your leads? Like what marketing program are you running? I'm like, my chief marketing officer is named Jensen Wang. I like that's kind of, it just happened by luck. And um and, and you know, he just really enjoyed the product.
And so I think that there was um it's, he's a very accessible person. I mean, he literally answers his own email. It's normally on Sunday nights, tip, pro tip. Um, but, um, I think some of, you know, that already.
Yeah, he's not gonna appreciate that. Now, I just thought about that. Actually. It's Saturday night. You Saturday. So, uh, the, um, but I think that, uh, they're able to just drive a ton of interest because everybody, every Fortune 500 company thinking of ja I ends up making a stop with them at some point in the process, at the senior most levels. And so the access that they can share with their companies is like, I've not seen from any other partner of ours, but coming back to your other question, um we're solving a different technical problem than most people. Most people are looking at throughput and optimizing their systems for throughput and optimizing cost per token.
Um We're doing voice based conversations and we're offsetting a very expensive clinical resource. So we don't actually need it to be very cheap. We need it to be very safe and we need it to be very fast. And you can't feel empathy if I, if it's too slow, you know, I always joke with people. I'm like, you know, when your girlfriend said I love you and you were like, hm. Oh yeah, I love you too. Not the same
as I love you. I love you too. Like, not the same. And so the latency actually takes away from true emotional transmission and everybody else worked on kind of speed, you know, cost per token. And because in tech search an extra two seconds is not the end of the world really, right. Like, you know, you're getting such a better answer
and you don't have to click on five links and you don't have to research them like you're, you're happy to wait. But in a voice conversation, latency matters and we found the whole stack has not been optimized for latency. It's really been optimized more for throughput.
Even the inference engines have not been optimized. We finally had to take an open source thing and build our own because we found so many optimizations in there that would make it faster and faster. Um The TT SS are not optimized for latency as much as we would have wanted. Um
And so we've ended up realizing that there's a whole area of exploration and optimization that's more latency focussed. Um And then Nvidia's, you know, really their whole team is actually working with us on multiple layers of the stack to try to optimize latency because we've realized that and I think we called it um uh uh kind of the empathy based latency. And that's really what we're after is is working together to speed things up. And this was one of the things Jensen did like because he said, you know, when I built uh video cards for games, we realized that the 10 milliseconds extra of the laser on the mouse hitting, it would really interrupt your ability to win in that first person shooter. And he realized
he had to optimize a bunch of different things there. And so he, he said, oh, with Hippocratic, you will help us figure out how in gen A I in inference engines to optimize it for latency because everybody else is teaching us how to optimize it more for throughput. Yeah, I think, um, I won't speak to Jensen specifically but I, I think partnering with, uh, large corporations or the large size corporations is really difficult for some start ups and something that I personally experience, uh, I have a lot of start-ups contacting me and it feels like they're not always sure what question to ask or how to get help. Um, William, if you could talk about navigating companies like, uh, NVIDIA and, uh, large corporations. Aws. How do you navigate these companies? How do you start? Yeah. I mean, if there's like a lot of politics you have to learn, I guess, which is, um, I, I guess if you're coming from those companies it's a, it's, it's not news to you but it was for me, I think probably the, the first real important thing is like that a site contacts and all the meetings that you're having because you're, you're gonna have people joining on there that may not be, they, they have no clue what you're talking about or, or why, why they're there and you've got different business units. So you can always have to cite
context uh from the very beginning. But um it always helps to have like a very specific use case or like a tangible goal that you're going after and have executive sponsors, like try to latch on to that. I think without that things are gonna fail. So especially if you're selling to enterprises,
right? Um From a partnership perspective, I think those are harder for sure. I think NVIDIA is one of our most successful partnerships because they're also some of the earliest partners, right? So um has anyone heard of Nemo, for example, all the uh Nemo tools. So that's built on our stack from, from like 2020 super early on, right? So we've been partnering with them for a very long time um on doing things like this optimizing servers, like at the kernel level, at the compiler level, at the server level, et cetera. So um you, you have to know who the people are that you're trying to get out, get at and have something concrete. So probably the Hippocratic example here is pretty successful, I would imagine because you have a very tangible case just here's what I want to do and it drives everyone to like align with that if you can distill it down to that. That's great.
I think most of the time it's actually really hard to do that at a big company because people have different, different goals that they're trying to shoot for. So say no to most things, pick the one thing you want to do and then focus on that. That would be my, my suggestion. That's a good suggestion. Dial, same question you're on stage here with NVIDIA at Aws re invent. How did you get here? What are you doing with what's working? Yeah. So, uh
if you do it wrong, it's going to be a giant time sync and you get almost nothing out of it. So, so I think we'll basically answer the question. You have to be really focused, you need to be very opinionated about what you want to get out of the partnership. Uh You need to choose your partnerships carefully. NVIDIA has been a great partner. Uh So the inception program has been fantastic, is really about collaboration rather than, you know, trying to maybe compete in different ways. It's
really every partnership is different and you can really feel that very early on and sometimes you should just walk away. But that's very hard to do when you're an early stage start up. It's hard to make that determination and to walk away. Absolutely. All right, William
uh back to you. Um many start ups have and enterprises begin their A I journey by prioritizing the development of A I infrastructure, often integrating numerous point solutions. What are the key advantages and challenges of this approach? And how can companies uh accelerate by focusing on products instead? Yeah, it's actually interesting. So we help a lot of start ups today and big companies and, and kind of the first pattern that I'm seeing uh that we've seen for many years now is people kind of build their own A I in for first. So as a new
start up will get funding and then they'll sit here and build all this like optimized stuff instead of shipping the product, right? So the way that it starts is very simple. So let me ask, let me get a like raise your hands to answer this question. How many of you have built your own? Ml Infra? OK. I'm gonna ask it again. I have to explain what that means, right? So you went out to some cloud provider and you rented a bunch of uh H one hundreds, right? So you got the machines, you got some quotas, whatever. Then you started layering in a few services here and there. You started like putting ac two, you started putting monitoring tools, you started buying a bunch of point solutions, you started stringing them together.
How many of you have done that? Yeah, that's an MO platform. You don't know it yet, but you built an MO platform and in about a year or two, you're gonna run into issues. When you look at your head count, you're gonna have five or 10 engineers who are managing that thing specifically, then you're looking at your burn and you're like, well, what happened? Where's my product? They're like, oh, but we have to build this better Inference engine or this better kernel or this better way of doing distributed or cold starts or whatever. Well,
I think I would ask you this question if you believe that you're doing that and spending time into that particular thing is going to truly separate you from your competitors. Like if you can look at your competitors and say, oh my Ml Infra is the thing, the reason why we're winning. If that's true, then continue doing that for most of you. I don't think that's true for most of you. It's because your data, right? You have unique data, you have better models, you have like a domain expertise that no one else has. That's really where you, where, where you have your differentiator. So anything
that's not invested into that, you're gonna be wasting time and money, right? And people don't know it until they're spending way too much time on it. Um There are people in this room today, I know already that have talked to us because they can't came, they, they said, oh, we're gonna do this on our own then a year later they came back and they're like, oh my God, I'm spending way too much money on this. We need help, right? It would be like building your own slack today. Like how many of, you would build your own slack. Probably no
one. Right. Well, that, that's kind of what we solved, right. It's give you the platform out of the box so you can focus on the product, not necessarily the thing. Um It, it gets really expensive. It's a management. Uh What, what happens when those people quit? There's like a whole there, it's like, sorry, there are a lot of things that go into that, that you won't necessarily see upfront.
Uh Yeah, thank you. Um Mal you're saying crazy growth at Hippocratic. Uh you're citing health systems every single week. What is driving that? How do you see growth? I mean, I think it's just a product market fit. I mean, look, we entered an era
where everybody is short clinical staff and we're offering effectively infinite supply of clinical staff. And so combine that with the excitement around gen A I um you know, we are just seeing people be like I want to try it, you know, nobody's like, oh, I don't know, they're like, yes, you know what I have such a problem. I want to try this. I'll try it right now. I think the other part is um we realized that our business was um a dual mandate, meaning it wasn't just enough to make them more efficient. We had to ensure it would be safe because even a conversation with a nurse can kill a patient. If you say, can I take Ibuprofen and the nurse says, sure, you can just don't take too much.
It turns out if you have chronic kidney disease stage three or four, you can't take ibuprofen, it'll kill you. And so we had to build in all of these safety checks and we did that. You can read on our website. We have this constellation architecture. We call it where it's a multi agent architecture where there's a primary agent and 19 supervisor agents and the supervisors are looking for overdose statements.
They're looking for uh these conditions specific disallowed over the counter medications. They're looking for drug stoppage instructions like all these things that are deeply clinical, but that actually can hurt patients. And so we just came to them. I mean, look at even how we presented the company, we call it Hippocratic after the Hippocratic Oath, we made the tagline do no harm. We decided that there was one other variable other than efficiency gain that we were going to focus the company on and then just put everything against it. We spent $10 million on having real us nurses test the product even before we put it in front of the first patient and talk to it and mark all the errors. And then we
fix those errors and then we put out another version, they mark those errors and we did that for 18 months. Now, I think we're at 260,000 conversations before it even saw one patient. We've now talked about 100 and 60,000 patients. So now it's, it's kind of out in the wild scaling.
But I think that, um, you know, you, you just, this is what we've seen, is it, the product succeeds because the demand is there. But there's usually one other criteria you have to have. And in our case, it was, it was really safe to you. Absolutely. Ok. Last couple of minutes before we switch to
our, our next panel member. So we'll do a couple of rapid fires, short sentences, few words. What is your one sentence? Pitch to an investor, William. Um One sentence. Pitch to an ambassador. Wow. OK. Um So lighting and A I development platform um kind of brings all the tools together into a single place so that you can build A I quickly not worry about infrastructure, not worry about um anything else that's other than the data, the models and your team itself.
Um I guess I don't know, pretty good A I agent that focuses on the scope of nursing. Very succinct. I like it. Winner, production grade enterprise A I uh from the pioneers of rag powered by R A 2.0. Hm. Not bad. All right. I'd fund all of you. It's pretty good. All right. What is one A I myth you'd like to debunk that we're plateauing. Just,
there's this, there's this narrative out there now that, that A I is plateauing. I think that's, that's wrong, but this is rapid fire. So I can't explain why, uh, that, um, what you train that if you know, if you know what I've trained on my model is safe or, you know about something about the safety of the model's total crap. This thing is PUBMED GP T trained only on pub med still make stuff up just because you have, you know, garbage in might be garbage out, but good stuff in might still be garbage out. Like the only way to know if it's safe is to test output but output testing is way bigger than input testing. Sorry, that was more than a sentence.
Um I would say that it's, it's, it's a lot easier to productive than you think. Um It's just, you don't have the right tools, right? It's like having five G without a smartphone. Like you can't really do anything with it. OK. Last question, it was the best advice you've ever received as a founder. Um Higher slow fire, fast eat the appetizers when they're being passed around.
Not when you're hungry, be comfortable with being uncomfortable. Always good. All right. Last thing, what is your call to action? What do you want people here at re Invent to do William. You go to lighting dot A I right now. Sign up, it's free, free tier, get up there.
Um I'm not gonna name all the tools that this will replace, but it replace like 18 different point solutions for you. And it's a lot cheaper as well, take his stuff and go start a company if you even have any interest in it. Because we just had a forest fire. If you need any software to build applications to address the forest fire, then come talk to me. That was perfect.
All right, let's thank our panel members our first round. Thank you. Thank you. Thank you. Ok. And I wanna welcome,
welcome up our second panel. If our three panel members could please join me on stage. Let's welcome them up. All right. Thank you for joining.
I might say for this one, we have a nice transition of people coming in too. All right. If you could start by introducing yourselves, your role and your company, please.
Hi. Hey, everyone. I'm so young. Uh I'm one of the co-founders at 12 labs at where I head up go to market and 12 labs is a company that builds foundation models for multimodal video understanding. Uh So these are large pre trained A I models that can contextually comprehend everything that is inside a video, put all of those contexts together and enable uh intelligent downstream tasks such as semantically searching across your videos. Um automatically summarizing, prompting to retrieve responses of video rag um highlight chapter and so on. Um OK, thanks. I love what you're doing. You saw you at ABC?
Um Hi, I'm Verra, I'm Chief Strategic Partnerships at III I is a platform for generative media and for builders. So basically, we're not an application, you can generate images as a consumer. But if you are building applications, uh product solutions that require generative media being visuals, video, music, audio, this is where you need to go. Hi, I'm Kev. I'm uh the strategy guy at Ryder. Um A uh the leading generative A I company for enterprises.
Uh We serve 40% of the Fortune 500 I'm a serial operator having done about five different uh technology innovation start ups uh that included mobile ads, uh cloud storage, uh visual collaboration and, and outright are doing the alternative A I. That's great. Thank you. Let's continue with you, Kevin. What are you doing with NVIDIA? How are you using our tech? How are you partnering with us? How are we helping everything we, we try to take everything from NVIDIA that they'll give us? So we've been part of the inception program from, from day one. You know, what's different about writer is that we're a full stack solution. So we have everything from building our own models to the go to market, to the rag to, to everything around the sun. And we've been using uh Nvidia's team, they're like in our Slack channels, right? So like we're constantly with the team, they're helping us train our models, we used, you know, AWS hyper pod using, you know, Nvidia's H one hundreds to train do inference do all that stuff. So we work with you guys on the
technology side but as well as the go to market side. So everything across this line, we we are not shy about asking for help from NVIDIA at any point in time, which makes sense from the strategy. That's great. Yeah, so similar. So
we are a developer platform but for enterprises mainly because we solved all the challenges for enterprises in adopting gen technology mainly being trust and safety. The way we did this was that the first technology we ever developed was attribution technology. So for every synthetic asset that coming out of any of our media models, we can trace it back to the original assets within our training catalog that impacted the most specific synthetic assets. And we also share the revenue with their ancestors. So this attribution technology is what actually enabled us to collaborate and partner with some of the world's greatest media companies including Getty images and vato deposit photos Association Media Group and many many more. And we only train our models on licensed content. So
we never engage in scraping of the internet, working with enterprises. Obviously being vetted by NVIDIA carries a lot of weight and media has very broad overview of the market. Um and and opening the door to sea level um within this organization has been um really crucial to our go to market. A lot of our customers folks.
Um Warner Brothers are actually customers that were introduced to us by video and video. Recognize that we saw a gap in the market, especially if you are an enterprise where content is your product, you're very sensitive to all issues of copyright infringement. And this is where the Prius solution is actually something that you would absolutely require in order to engage with this technology. Yeah, uh 12 Bs, we train our models on NVIDIA GP US, we serve our models on NVIDIA GP US. So in France,
um and uh while we focus on building uh state of the art models, the most powerful and useful models for video understanding, we also want to make sure our models are accessible and running at the highest efficiency possible. Um So a model being able to control model size and also be able to serve our models really efficiently for customers who have massive scale of uh data that they're kind of running our models against uh is also very crucial. So the other thing that we are also working very closely with NVIDIA and also inception on is uh go to market and partnerships. And uh soon we will our models will also be available as N microservices.
That's great. We'll get back to that one. What challenges did you face in developing A I for video search search specifically? So, so we started uh the company in like four years ago back when uh not many people, too many people knew or understood, especially enterprise uh like foundation models or especially multimodal models. So what, what that meant for us because we had started out trying to uh a general purpose model that can contextually understand any video, whether it be a sports um game film um animation, maybe it's your car dash cam footage and be able to create a generalized understanding of it, to be able to power your intelligent tasks um that can go in so many different directions. So for 12 labs, I think and, and video understanding because video is so diverse, it's, you know, it's everything around us is basically video. Um It's how do you focus? That was the biggest problem like challenge that I would say we we would all any start up, but especially like we've had the phased and for 12 labs, I think it was just being laser focused on. Let's make sure that we are serving
the most um useful but also the most powerful models specifically for multimodal and video. Um And making sure that's also uh like accessible so it's being able to run at scale. Um being able to, I think latency was something that was brought up. So being able to open up new use cases by reducing latency. Um And now we're working on um things that like streaming in real time as well. Um
But making sure we're really deeply understanding all of the specific use cases that cost of different verticals industries have around video and um we're building the model of the product and also the api uh experience to be able to um power their internal workflows as well. Laws of products. That's great. OK, let's look at uh Brea continuing the creative journey. Um How is brea revolutionizing visual content creation with A I? And how has inception supported your journey? Yeah. So I think um the challenge with Gene A I when it comes to media industry, media industry is that the quality and the fidelity that professional creatives expect is very different from what consumer markets can accept. And this requires actually a dialogue between engineering and creative people and it's not as smooth as you could probably expect.
So for these teams, for creative teams to be able to effectively communicate their requirements to engineering teams, we actually created a team that is composed of creative people in our company that can help the creative teams from our customers organization to communicate with customers engineering teams actually, because they are going to deploy our models and control nets and adapters in order to facilitate pipelines that are going to cater to their creative teams eventually. So that was um one of the things that was um um um very important to actually reach success uh for these specific audience. And the other thing was that um it's always um and I think um previous partner also touched this with Jenny I, yeah. Um 18 months ago, people especially in enterprise space. Were thinking, OK, it's going to uh maybe um be closely linked to automation and gaining more efficiencies and production pipelines and just productivity. But I think the more advancement especially but also more and more generally are now realizing that this is a this technology is actually going to dramatically change a lot of the industries, not just bringing more, introducing more efficiencies, but actually changing fundamental things about how people work in these industries, what they are even selling, what is my product. So content,
for example, um is unlikely to be something that we will all consume in a very centralized way like we consume today would probably be very hyper personalized and people talk about this for years. But I think now finally, the technology is there to actually achieve this. Um I can give you an example from one of our customers. It's the second largest marketing agency in Japan, catering mainly to the automotive industry in Japan, which is obviously a very large one.
Um and they were previously in, in the old world, they were selling, they, they were actually doing conducting this really amazing production, they would select uh locations and uh photoshoot these cars in all different scenarios. But at the end of the day, you would still get this SUV in specifics in which might not be relevant to a lot of their customers globally, right? So, but but it was very expensive and the business model was that they were selling these images, one image, it could be $10,000 but it, but still this is, this was the product now with Gen A I, what we're doing to them is to completely automate this pipeline. And, and actually these cars now can be depicted in many different scenarios.
Um talking better and more ideally to their customers localizing this for them. I can tell you we don't get much snow in Tel Aviv. So putting this SUV in the Swiss Alps would probably make less sense. But if you can put it in the sand in the desert, yeah, that resonates with our local audience, for example. So now you can do all of this. And now what are you selling? You're
not selling this image because this image it's incremental, it's nothing to generate this, right? So now you're selling Toyota a platform that is going to generate unlimited volume of images for marketing purposes that are personalized for their very highly segmented audiences, right? So this is change management. This is why this organization now need to understand what is their business, what they're selling now. And this requires a lot of adaptation. And this is um uh working with NVIDIA is being um um really pivotal to helping them go through this change.
Thank you. I continue on working with partners working with NVIDIA. I think a lot of, again, a lot of start ups come to us and ask us questions. Like, how do you work with big companies? How do you work with NVIDIA Kevin? Let's talk about writer. How are you on stage today again?
How do you partner with us? How do you partner with large companies? Yeah, I think there's probably two misconceptions and we have a pretty humorous but also very painful story around. The first one, it's not about n but the first one is around just like, you know, typically we'll ask for whatever we want under the sun and we'll get maybe it answered half the time, but that's pretty good. So, you know, whatever the best thing to do, just do that. It's also what's really interesting is sometimes you expect that larger companies move slower but they actually move faster because they have like 100 times the number of people, maybe thousands of times.
So, you know, we typically will try to move quickly, but it's actually pretty interesting that NVIDIA moves really fast. So we're able to get a lot of things done. So that's the first one. The second piece that was kind of an interesting story for us was a tale of basically two years. So we were actually stuck in the wrong account with Aws in a different region. So basically, every time
we reached out to Aws, they're like, oh, you're not in the North America region. So therefore we don't have anybody to talk to you. So I was basically filling out forms like the help forms being like, hey, can somebody at Aws come talk to us? And their response was always like, no, nobody can talk to you. You don't have an account manager
in the wrong region. So the the big thing that really changed it for us was just like, you know, if you connect with the right people, they can help you get to the right place. So within a year or like probably within like two months, actually, when we got in touch with the right folks at Aws, our, our tides completely turned, we got invited as a special guest to last year's reinvention and we're able to do a bunch of things. And now this year we're doing a bunch of stuff with both NVIDIA and Aws. So I think it's more about just asking all the time and actually expecting that they're going to move a lot faster than you can. And the second is just also just leveraging the people that you know, and also online forms don't work. That's three
pieces of uh OK, let's continue on with this. What, what's the secret sauce? How are you? Is it the people asking don't be coy obviously. But at the same time, I think um large organizations tend to be structured around verticals and industries. And I know that, you know, if we are a platform for developers and we cater to be honest to many different industries, so we cater to marketing, creative platforms. CPG retail,
we discussed media and entertainment gaming. So these are different verticals and larger organizations like NVIDIA like a WS will have different actually looking after these different markets. So um even if you are going to uh um directly when you're selling directly to these markets or when you're going after these customers yourself, you will, you know, um be less focused. Perhaps you take a let's focus approach and perhaps when you're going through partners, make sure that you start with one industry uh first and expand from there and the currency that is most important. Uh working with larger uh partners like NVIDIA NWS is case studies. So actually,
uh making uh their customers happy and satisfied and um um delighted this is what they want to see. So succeed once amplify this, make sure that everyone uh learn about this. Um And, and, and see how this can be replicated with other customers. This will actually um y a lot of uh inbound from other accounts. Yeah, same, same question. Um So I think for 12 laws, II, I largely agree on the case studies. I would say
if you can get customers to advocate for what you do and why it's valuable for their organization with partners. That's the most valuable um push that you can get um to be able to solidify whether it's a product partnership, Cosell or, you know, go to market partnership um to be able to get these things done. Uh Like for us, for instance, we recently released a case study with the Maple leaf sports entertainment. Um They're one of the largest Canadian sports uh kind of groups where they have a hockey team, they have a basketball team, um soccer team. And the case study was around, there's so much valuable content and footage that lives in the archives for any sports league or sports team.
Um In this case, of course, there's some such valuable content around every player. Um and every kind of uh you know, every team that exists. So how do you actually uh understand that vast amount of content and you're able to create content at scale efficiently, uh understand what's inside, first of all, create content or highlights. And if you can do that at scale, uh and without having to manually have somebody painfully tag and log and you know, go like take notes on what's actually happening while they're like scrubbing through footage. And then third, you're actually creating content at scale, then you can actually personalize what content is pushed in front of every audience group.
Um And that's an example of like it's a really good representation of what some of our other customer, you know, our other customers would see in some of the product level partnerships or solution comb comment of solutions um that we will put together with partners. And I think it's in things like that um that have really kind of helped us help 12 apps a lot. Um Both with NVIDIA and Aws and what both companies though large have in common is that they're incredibly customer focused. Um And especially for NVIDIA, I, I feel like uh it's really the next gen like most strategic use cases for any enterprise, large enterprise organization that NVIDIA will really advocate for and make happen.
Um So if you can get, have a customer say, hey, this is a technology or this is a product uh that is going to be meaningful for our business in the next years to come. Um That's, that's how you are able to kind of work together really easily. Yeah, work backward from the customer. It's great advice. Uh forms don't work also good advice and focus on a vertical. I completely agree. I think a lot of start ups go really broad and it makes for a difficult story. I think I have
another um advice for the audience here. If you are in a space that is really innovative, that is pushing the boundaries of uh technology, um then you're probably going to collaborate with this video, for example, on their product that are also very innovative, right? For example, um our um generative media capabilities, whether it's image generation or editing or what have you, all of this can be consumed whether through A P. But also you can also take the models complete with source code and weight and everything could continue your own development. So this actually interacts with two different platforms of NVIDIA and these companies. So um when it comes to the API we're running uh in our uh models or our platform on NBC F which is formerly Picassa, if you're familiar with and when it comes to our models, they are available um on the name, OK. Uh in video. So what's important to understand is that these giants also have their products and these products are babies of someone OK, in these organizations.
And they want actually you especially if you are um um running a company that is doing something that is unique that not many out there, not many group out there doing is your experience working with their product and their environment that actually is actually uh and your feedback to them is actually going uh to drive forward these products. So um um be very open and share with them, your feedback and help them uh uh put in the market but and better products. So this is what we're actively doing with them.
Not many groups train this large models in individual space in the media space out there. So working with them on their environment, making sure that they constantly improve their products that relate to this space is something that has been really appreciated by them. Yeah, I would second that that's a little known and uh not often exploited way in is not exploited, but used way into companies like NVIDIA is uh focusing on product and feedback is really important and we pay attention. I'd say that's also true of um any large corporations. So it's a, it's a good way to get in. All right, let's talk about um, strategy. How do you
identify and prioritize strategic opportunities in the fast moving industry? Strategy guy Kevin. I mean, so the, the first thing is like companies, particularly how fast things are going there. There'll be companies that would be like, hey, what's your one year plan? I don't think we can write light, light plans that are longer than, you know, a month in many cases. So really, it's just how do you quickly be able to identify what you want to do and go do it? And so, you know, there are places that will say help me write a business plan. In this cases,
we don't even write them, right. It's more like if you have a hunt, you have an idea and you feel you have strong conviction, go ahead and try it, try it for a month and see what the results are. And that's usually sort of the fastest way. We really try to figure how we learn because you can create a doc. That's like here are my 10 priorities for the year most likely in a month, those 10 priorities will be, will be different.
So we typically like to just kind of pick the one or two things you want to go after and go do them and figure out in a month, look back and say that worked or didn't work and that's been working out pretty well for us. Only because as I mentioned, things are changing so dynamic, the whole industry is so dynamic things because things change so quickly, you can't really write this stuff up. By the time you write it, it's probably going to be different anyways. Yeah, absolutely. Anything. So for us, it was really going against the motion of the or the trend in the market. So um everyone were doing um Ja I obviously when we started five years ago or, you know, the, the pioneers of JA I were already there. Um So for example, stability um open A I with Dali uh and, and, and Adobe started training a firefly.
Uh So what we did was um actually look at what's there out there in the market and try to solve problems that um were were not even on the horizon. So understanding that um um there will be enterprises that safety and trust uh will be something that is going, this is going to be a roadblock. So we need to solve this for them. So five years ago, when responsible A I was not in fashion, the term J I was not even going back then, right? So responsible A I was not something that anyone talked about, there was no regulation on the horizon. So try to think how the market would look like 5, 10 years ahead. So this is what we did
and then solve this gap in the market basically for those enterprises, for these clients, for us, it's been incredibly helpful to have a shared. So to, to today to labs is about 75 people as a co
2024-12-09 04:14
