(intro beeping) (upbeat music) - Hello everyone. Welcome to Tech Innovator Spotlight. Today I am thrilled to cast a spotlight on Relativity. I am joined by two amazing folks. One of them is gonna be my co-host partner in crime. Valerie, why don't you introduce yourself? - Hi. - As well as
your role with Relativity. - Hi, I'm Valerie Bergman, and I am the data and AI specialist who is working with Relativity here at Microsoft. - Awesome. And now I have the privilege to welcome our guest Chris Brown.
Chris, please share a little bit about your role at Relativity and what Relativity does - Hey, Vrushali, Valerie, it's great to be here. Thanks for having me on this podcast. So excited to be talking with you today. So my role as chief product officer at Relativity, I've been at Relativity, I'm in my seventh year right now, and I look after our product group, our user experience team, our documentation team, sometimes some of the new ventures we start in the business or acquire. And I work across our whole R&D portfolio to set the strategy for where we're gonna take the products in market around the globe.
We manage a portfolio that's a little over $150 million a year. And so we try to put all those dollars to good use. Relativity itself is a company that's been around about 20 years, headquartered in Chicago, serves about three or 50,000 users worldwide in about 40 different countries, thousands of different organizations. And our customers tend to be in four segments, either end corporations, law firms, legal service providers, and the public sector.
And there's a mix of them throughout the ecosystem. And they also worked, you know, together with each other quite a bit in our software. And it's great to be here today. - That's amazing. So I know Relativity aiR has created some buzz, and I would love to understand from you what that product is, what kind of offer it provides to our end customers. And if you can take a tangent to which industry it serves, it would be awesome.
- Awesome. So at the core mission of Relativity that we have internally is organize data, discover the truth and act on it. And that's been the mission for the company since its founding.
And it's really built in the principle of organizing all this potential evidence in the world, finding what is relevant to a legal matter, an investigation or some reactive type of case or workflow. And then providing the tools by which we can act on it with those users being experts in the legal field or in a different field, and be able to take those to the next step in the various outcomes that they have. And this is a problem that's been really around since the dawn of common law, which is when there's a dispute or an investigation or something like that, we gather evidence, we talk to witnesses, we look at the scene, we look at information. The last 50 years, a lot of that information has started to become electronic.
And so with Relativity the tools and capabilities we bring to bear, it's all about essentially that holy grail of finding that piece of truth in sort of a mountain of information. When you look at all the data that's out there in the world when it comes to a legal matter, an investigation, or any of these different use cases, a certain piece of data all of a sudden becomes sensitive or useful or necessary. And then there's a bunch of data that is not, right? And so our platform really equips practitioners, you know, alongside with our AI and everything else, to really pick out those pieces of data that are the most important and helping you take action on them. Now, aiR sort of accelerates that to levels that prior were kind of unheard of. And so I like to think of aiR number one, you know, why people are excited about it is, it's a new generative AI solution, but it is targeted at the core problem that's been around since the dawn of law, which is what is the relevant information to this matter that's precise? What is the hottest piece of evidence? You know, what is that piece of material that I need to know to one, bring into the next level of the workflow, but two, be defensible, three, build into a case narrative or take next steps with.
So aiR basically accelerates that process because it can kind of take the lawyers or the expert practitioner's wisdom in the form of an input and then immediately sift through all the different data and possible material on the other side and bring the most relevant stuff directly to that lawyer with citations, with rationale, with consideration so that they can make a choice about how important it's to a case. So in some respects, it's solving a very old problem, but it's doing it in a way that's far, far superior to current methods, which includes either humans doing it themselves or even with other levels of, you know, sort of, you know, predecessor AI technologies can't quite achieve the same level of recall without the same level of results and information and context for the end user. - That's very cool.
You kind of touched upon this briefly, right? You serve so many industries, but you picked this and you sort of said that this is sort of an old problem, right? And a harder one to solve. I'm sure that's why like, you know, everybody's not kind of ganging up on revolutionizing and disrupting legal industry. So tell us a little bit about the thought process of sort of saying, "Okay, we'll take this." You know, the big bet as our problem.
Like how did you come to that conclusion? - I think there's two things in that question and document review itself, sometimes it's the process I'm describing is sometimes considered e-discovery or document review. That's in a wider sort of umbrella of all the things that you do in a legal field or a legal matter. And the actual review of sort of electronic evidence amongst everything else, you know, 50 years ago probably wasn't that important relative to all the other fancy things that lawyers and folks might do when they're delivering really important strategy or counsel to folks. But over the years, that small thing that was sort of just a technical thing or one of the check boxes along the way has become more and more where all the most vital information is and where more and more of those insights and turning points really lie. And so the field of document review itself has gone from being, you know, something that's interesting than at the, you know, the dawn of scanning and OCR and all this electronic data, now it's become vital. And as you all know, like expanding rapidly with all the different types of data growth and mobility, expanded it to smartphones and chat solutions and teams and video.
So it's all expanding rapidly. It's become much more vital. And so when you think about document review being as vital as it is and continue to expand aiR itself, you know, focused on this problem set.
We had a choice with generative AI, like other large technology disruptions before it, you know, you can kind of think about where do you wanna apply those disrupt, where do you wanna apply those technologies? And you know, if you look about two years ago with chatGPT and other things being launched, a lot of folks were launching products in sort of the chat or conversational space. They're trying, trying to look at generative AI and say, how can I introduce conversations with these tools to kind of help different workflows, which I think is highly valuable. But we took a different point of view at the onset of where we could apply generative AI in our platform. And why we did that is we felt that if we could bring sort of the power of generative AI, but acutely into a workflow like document review or privilege review, which we can talk about in a second, we can, one, get even higher results out of the generative AI technology, but we can kind of bound it to, or ground it to reality, two citations, to rationales. And we we could put it into existing defensible workflows that the industry could adopt and so much more than they could kind of adopt a chatbot just as it were. And so that's proven really successful in terms of getting this technology to thousands of practitioners.
And over time the chat solutions and other things will also add a ton of value and it'll probably change the use cases over time. But that was one of the reasons we focused on legal. One of the reasons we focused on core use cases, sort of chat, both cases are kind of counterintuitive. There are different approaches than others would've taken, but I think they've met the mark so far. - I just like to say that one thing I think Relativity has done that's really differentiated them from how I've seen some other companies try to leverage generative AI, is Relativity is extremely thoughtful about where do we have this giant rock that we're trying to push up a hill that's heavy and requires a lot of man hours, but is something that could potentially be automated. And with the review process, Chris, I know that you have some good stats and I also know that you have some good customer quotes, but that process is like, could be up to hundreds of hours of review for a lawyer who's an expensive resource and Relativity thought if they could take the generative AI capabilities and apply it to this use case, is there the possibility to shorten that review process and not introduce additional risk? So, Chris and his team have been really thoughtful about managing risk, managing potential objections, fear in the marketplace, and they did a lot of testing with real-world data, with customers getting engaged and involved to qualify that the results that they're seeing are what they expected, so that when they were ready to roll out to markets, they have a huge mountain of evidence that says that this use case is a good one and it's gonna produce quality results and it's gonna save a lot of money for Relativity customers.
- Yeah, well said Valerie, I think you both alluded to that, right? It's like there are several chatbots out there. They are all great, but you wanted to find that niche and solve that hard problem and kind of take it head on because AI where it is today, like everybody's gonna follow, right? Like everybody is gonna follow the suit. So it's not gonna be a case of if it's just a case of when they will follow the suit and when they will kind of. So it's really interesting and very successful endeavor, I would say, from Relativity'a part to kind of have this niche, understand the problem and solve it in a way that AI could be much more beneficial beyond just, you know, giving that generative AI feature of like chat bot and kind of interacting with it for fun, right? So that's amazing. I know you both have worked closely together on this project and I know Valerie has some questions in terms of the partnership between the two of you. So why don't you, Valerie kind of go for it.
- Sure, Chris, in your opinion, what are the different differentiators that Microsoft provides to you as an ISV and why do you choose to partner with Microsoft in a transformative project like this surround AI and innovation? - Yeah, so great question, Valerie. I think, you know, in our industry and what we serve, the one differentiator, the one thing that was very apparent to me the founder and other folks at relatively way back when was that, you know, in our world we're examining potential digital evidence to see if it's actionable or relevant to a downstream matter. And so first and foremost, the reason to partner with Microsoft and why Microsoft is a great partner is because it is the world leading creator of digital evidence with all the folks that you serve. And then secondarily, your Azure platform really allows us to bring in all this technology, AI capabilities in a way very close to where the data is at rest in a secure environment.
Most of your, you know, many of your customers are also in Azure right now. So it allows us to bring our technology and our platform closer to those end customers and buyers and build a, like a common workflow and experience that really makes sense for them both on the value it creates, but also how it complies with their needs and how it's, how it re remains secure. Now, one of the things that we got really excited about with Azure is whenever you have something of value, you wanna bring it to market worldwide, you also have all this investment you've done in all these different regions and geographies to bring not just Azure worldwide, but all the services inside of Azure that are innovating all the time, including the latest generative AI things.
And now that we're tapping into generative AI not only are you providing all the services and expertise, I mean, you're kind of co building it. You're helping to almost co build some of these solutions with us. And I know Valerie very personally is focusing on all the ways Microsoft can help us and equally we're sharing with you pretty much every week, you and your teams, what we're learning from customers and how that is impacting what we need to do, where we need to do it, what features we kind of need in the platform next.
And that's a great, just an amazing partnership and especially when you're going to, you know, transform an industry, you wanna be with a partner that can do it well with you, but also has proven to be a great steward of all the different responsibilities that Microsoft has. And we also lean on that as well with our own focus on security and AI principles and things like that. - And Relativity's willingness to partner with Microsoft up to an executive level has been really powerful in making sure that we can move any blockers on Relativity's behalf and have foresight as far as what Relativity needs coming up so that we can make sure that we meet them where they are. - That's amazing. You kind of touched on this a little bit, Chris, security, governance, compliance, and then also you mentioned the responsible AI piece.
Responsible AI is important to all of our customers, all of our ISVs, but the degree is varying. In your case, because the industry that you serve, it probably takes a much higher precedence than some of the other ones. Tell us a little bit about how Microsoft's portfolio kind of influenced decision making. What kind of groundedness were you looking for in terms of, okay, if this kind of checklist is checked, then we are good from responsible AI front and we can go forward with this product? - Yeah, so great question. So I think first and foremost, you're right, the data that we are managing, the use cases that we're helping our partners and customers manage is really highly sensitive information. And you know, I think back to our flagship product, which is RelativityOne running on Azure around the world or azure.gov here as well.
The number one feature of RelativityOne is the security. You know, again, when you think about the industry prior 10 years ago, this data were in on-prem locations, security was up and down, variable depending on who was running it. And so of course we have all these features and capabilities that help you get to the truth in all the ways I described. But you don't have the ability to do that if you cannot manage the data and secure it.
And we all know security isn't like a one-time thing. It's an always on increasing investment. There's new actors all the time. And so, you know, first and foremost, Azure has been about that notion about security. Now we're also in an industry that because of the ever-changing world, there are a lot of needs and demands for innovation in the legal space because there's a new challenge every week rolling up to a lawyer or to a practitioner, to a forensic specialist, to an investigator.
And so they're in this position where they need a highly secure platform and a highly available platform, but they also need a ton of innovation in that platform. And they also need to access the very latest generative AI models and other things. And so those two, those kind of like barbell needs are a great fit for Azure, and they're a great fit for Relativity in what we do. And then in between that if you lean on the AI piece of it, you know, we also know that the technology around AI, and particularly generative AI, is powerful. And so we did really think deeply about our AI principles in ensuring that the products we're building our fit for purpose. They're transparent, they're secure, we're keeping your data private, and we kind of publish those principles out to our market and our communities so that they can live by them, they can enhance them, they can challenge us with them, and together we can kind of build thoughtful solutions together.
And in practice, they've really helped us make decisions internally in our R&D roadmap, make decisions about how we work with our advanced access customers. And also, you know, we've had discussions with Microsoft about how those things can work together as well. And all that I think is actually helping us just build a better product. So thank you. - Yeah, and so the same for Microsoft too.
I'm sure Valerie can attest to that, like all of the information and the questions and even challenges sometimes, right? Helps us build better products as the platform for you all to build on top of it. So that's amazing. - In a highly regulated industry. And I work with Relativity in the legal industry, I've worked with customers in financial services, I have a lot of peers who work with healthcare customers, and I think that it's easy to look at a technology like generative AI and kind of dismiss it as being not ready for prime time or something that's too scary for a highly regulated industry. And Relativity has really differentiated themselves by trying to figure out how they can leverage this technology in a way to disrupt the industry instead of just trying to make little tiny incremental changes. And Relativity hasn't shied away from the fact that there's risk.
They've leaned heavy into it and established their responsible AI practices to ensure that their models are accurate, fair, compliant, secure for their customers. And they've spent a lot of time gathering voice of customer information in a way to take this technology and do something really cool and new. And whether you're a customer in the legal space or in a different highly regulated industry, I would just really encourage all people to follow the pattern that Relativity has set forth in saying, let's really find this use case. Let's find this high value add use case, and then let's document the heck out of the risk and let's figure out how we can mitigate it so that we can do something really exciting in the industry. And I think Relativity has just knocked that out of the ballpark. But Relativity has also published case studies with Microsoft for people who would like to learn more.
And I believe that we can share a link to that. And Chris, I was trying to figure out when I was gonna ask this question of you, but Relativity has a conference that they do annually in the United States, they do when also in Australia and London called Rel Fest, And I had the opportunity to go to Rel Fest in October in Chicago, and as Chris was up on stage doing his keynote, he was talking about generative AI being applied to a robot, could they create a robot? So I'm curious, Chris, for those of us who are familiar with your keynote that you did, am I talking to legacy Chris Brown or am I talking to Chris Brown, B2 generative AI Chris Brown? (Chris and Valerie laughing) - Yeah, I guess you're talking to vintage Chris Brown here. - (laughing) Vintage. - I love that. - I enrolled very secretly, very secretly, maybe eight people in the company knew, my identical twin brother to play the part of AI Chris during the keynote, and I think he had such a good time. He may show up again at another Relativity Fest at some point in the future.
- So that keynote that Chris did is also available online. You can download it and watch it. And in addition to what Chris has shared today on aiR and the future of aiR as well, there's a whole bunch that he covers during the keynote, for those who'd like to get some more information, some more meat. - That's amazing and much more entertaining when we have like two Chrises.
I mean. - It was really well done. It was so well done. (indistinct) - And that is super fun. Yeah, are you both like identical identical, like no difference whatsoever, or there are like nuances that people close to you would know? - I think there's nuances that folks close to us would know, but I will tell. I will say it was a lot of fun, you know, you have a job to do during a keynote obviously. And I really, you know, we had a story to tell about aiR and all the different things we were doing, but you also want to try to connect with the audience in a surprising way.
And so as as I said, I was kind inspired by something that Reid Hoffman had done when he had created a digital twin of himself and had a conversation online, if folks saw that. And so I kind of used that as the hook to how could I have more of me at the Relativity Conference so we could be in more places at once, right, selling aiR. So it was a lot of fun and I think for a while it was great watching the audience kind of be like, "What is that? Who is that?" - What is happening? Yeah. - What is happening here?
And that's always a good place to be. So that's funny. - That's amazing. Do you have your kids kind of, do you prank your kids sometimes with this or no? - No, I don't prank them, but you can, you know, if you are with an identical twin, people can get confused pretty easily. So we try, we do it in small doses, especially. - Not to overwhelm them. - We don't have as much differentiating as I did when I was a kid.
Different haircut. - Chris, do you have any statistics that you could share about t how Relativity in practice has been working for your customers? How, sorry, Relativity aiR for review specifically has been affecting your customers? - I think the, yeah, so with Relativity aiR, I'm trying to think of the stats to really zone in on there. There's a lot, but I'll say a couple things. Number one is, you know, aiR for Review are like, so AIR is a suite of AI products that we have now. The first product being aiR for Review, which focuses on finding relevant information. There's aiR for Privilege, which is helping to find out which of that subset of information is conferring legal privilege and making sure you can protect that.
And we've also got some investments coming in a product called aiR for Case Strategy, which is about building your case narrative, understanding the facts of the case, tying them into that narrative. And so they all actually have different purposes and needs. On aiR for Review side, the first and foremost thing is how good are you at finding relevant information? And so the stats that we tend to think about in the industry in that zone are what is your percent recall, and what is your precision and what is the speed by which you can do that, and how expensive is it? And so in the case of recall is probably one of the main metrics we will talk about.
Because what recall really means is of all the possible relevant information, what percentage have you found, right? And then from inside of that, you know, inside of that view, you can then kind of drill into the next aspects. And so what we found with aiR, with well-written review protocols, which is bringing the attorneys directly into the product, you know, we're consistently above 90% recall and often much higher than that. Now, why that matters is every other method that people use all around the justice systems today are less than that. So if you're doing manual review, you know, it's challenging to be above 60%, 60, 70%. Again, it varies, but it's not 90%.
And then even in some, you know, precessing technologies, technology assisted review, 70, 80%, you know, you might be in that zone. And so not only is aiR just, you know, at the end of the day you're trying to find all the relevant material, it just does it better. Now, it also does it much faster.
Does it, you know, 70% faster, and probably more than that. It does it more cost effectively overall. And it doesn't just tell you that it's relevant, it tells you why. And it cites, it says, "Here's why it's relevant.
Here's the citations that back that up. Here's some considerations why it might not be relevant." Right? So it, it fully pivots around that tool.
And so that when you're getting that end answer, as an attorney or somebody in an investigation, it's far superior than a bit click somewhere that says, "Yep, it's relevant." 'Cause you don't know why you have to go back and re-review it. - And the why showing the work of the model, showing the logical path that the model took in arriving at a decision that something is relevant or not relevant, that is one of the risk mitigation approaches too, for ensuring that the model doesn't have the issue where it it confidently tells you the wrong answer.
- Right. - Or, yeah, I'm forgetting that word for that. - Groundedness. Or, you know, we focus on some of the elements around groundedness and grounding the platform. And, you know, and again, there are all kinds of stats around customer use and customer adoption and millions of documents run and a bunch of anecdotes that you'll be able to see if you see some of the materials we share and then you hop on the different websites. But at the end of the day, I would say the solution is delivering better results faster, more defensively, cheaper, and it's telling you why. And so that's the answer.
- That's fantastic. So, tell us, Chris, if our Microsoft sellers wanna co-sell Relativity aiR, what are some of the things they should be listening from their customers where they would feel, "Okay, this is a clear case where we need to bring in Relativity." What would that be? - Yeah, so even aiR for Review right now, it handles kind of different types of use cases, just getting a sense of your data. So we would call like early case insights, you know, its powers not just litigation, but internal investigations.
Like is there anything in this data set that's interesting to this concern I have that I'm investigating a fraud concern, you know, an internal harassment concern, whatever the case might be. It's clearly obviously helps with some of the litigation things that are more downstream, like a review for production and then a bunch of other different use cases. But stepping back, you know, for your sellers, they're out working with all kinds of customers that have sort of big data challenges. What they're really looking for are data challenges that have some in, you know, reactive components.
So the way I think about it is, you know, when I say big data, I don't just mean sort of like Azure big data, I mean, inside of all your data you have all these bits running around, but every once in a while there's some triggering event that makes a subset of them highly sensitive or valuable for a period of time. And so if a client, you know, if one of your sellers has many of those things where for a period of time, you know, the data set has some of these triggering events. How do they handle those? What kind of products and tools do they use to manage that? That's where Relativity shines. And there's all kinds of use cases.
So it's not just the ones I mentioned. We help folks with data breach responses in the public sector, freedom of information requests, data subject access requests in Europe, dispute, resolvement in Europe. So there's all of these things, but they all kind of start with a kernel of has the data been triggered by some event? In the case of FOIA, it's just simply about a citizen asking for information. But that all of a sudden makes that information more interesting, more valuable, and we wanna make it more accessible with the software.
- That's amazing. And Relativity's platform is, you know, obviously if you have a customer who is in the legal industry, you can be like, "Oh, this would be a great fit for you." But absolutely it's broader than that because if you have a customer who has a legal department, their legal department could use Relativity's product and minimize the amount of outsourcing that they would have to do in order to meet review of large cases that are brought to them. So Microsoft HR, for example, we do use Relativity's product.
- That's amazing, 'cause that's exactly what I was thinking. I was like, "Oh, we have this huge portfolio." It's fantastic to hear that we have that joint collaboration there as well. This has been amazing. Chris, I think final question from my side would be, you touched upon this earlier in our conversation, in terms of how, you know, there are people who are apprehensive about AI, and then there are people who are excited about AI, your peers kind of sit across the spectrum. What guidance you have for, you know, companies like yourselves out there to onboard onto AI.
How should they think about it and how should they grapple with this? - Yeah, I think there's always a spectrum of adopters, you know, conservative to aggressive, and things like that. I would say with any technology change, you want their choice of where they're sitting on that adoption curve to be informed. And the best way to be informed is for them to actually try the software, right? And so one of the things we even did at the user conference, Valerie had mentioned earlier is we put so much effort into getting as many hands on keyboard as we could at that conference and then all the follow up events because we'll you will get objections, adopting any technology, right? You'll get objections, but you want it to be informed through trial, not through, you know, anecdote and fear, uncertainty and doubt, right? And so you really want to do that.
And I'd say, and for any software company, you know, that's where you wanna kind of bridge the digital world to the analog world and get the people actually using the software. I'd say there's obviously a ton of trust building that you have to do. It's kind of neatly, you know, categorized under thought leadership. But what you're really trying to do is build trust amongst a lot of different folks in an industry.
Our industry relies on defensibility. It's a really important factor. So it's not even just the players within an organization that have to build trust or the end users, but it's also the judiciary, it's also the other folks in the industry. And so they all kind of lean on each other with various different groups and things like that to build understandings of how this will work out. And you have to invest and be in all those places and listen, and that's kind of. - That's fantastic.
- Yeah. - Start small, but start, essentially is the message start somewhere. - Well, and giving your customers an opportunity to get their hands on the solution prior to purchase in a proctored setting was really smart of Relativity to do at their conference. And they started out with, I think it was two, or two or three labs that they had on the schedule. And the lab had 20 computers for people to work at.
And I was trying to get in one of the labs and it was like waiting room outside. There was so much interest in it. So, I don't know, Chris, if you even remember how many you added on, I think you guys ended up going, you like, went from three sessions to like five sessions and the room size, they doubled it from 20 computers to I think 50, so more than double. - Yeah, it was our own little, you know, popular concert moment where we had to add some, add more sessions, add virtual sessions. And it was exciting, you know, obviously it was exciting to keep expanding it, but it was an interesting thing when you saw the line kind of around the halls and into the hallways to go to the hands-on sessions.
But people were just really, genuinely curious. and excited. And they hear their peers talking about it and they want be a part of it.
I'll probably lead with that, is we're not just like delivering the software to somebody who's like, "Alright, I'll take it." They wanna be a part of it. They wanna be part of building it you know, and so a big part of, and they should be because we can't really build a great solution without their hands on it.
And so we've invested a lot as Valerie mentioned earlier, in so many advanced access customers with us from really the kernel of the idea all the way through to the various versions of the software and many industries. Ours probably especially, it's so essential because there's so many different viewpoints, different segments to serve and it's highly advanced work. And so we really need the practitioner's opinions into the software itself.
- That's amazing, and it's a great problem to have, right? You can see how much interest is out there to get their hands on your product and kind of try it out, which is always good. And kudos to the flexibility of kind of expanding the labs overnight. That's amazing.
So thank you so much. Thank you, Valerie, for being the co-host. It has been fantastic and thank you, Chris, for sharing all these nuggets with us. We will provide all the assets when we release this podcast and so people will be able to go explore, learn more, and then hopefully we will continue to build together amazing products.
So thank you everybody, and thank you for our audience for listening to Tech Innovator Spotlight. See you, until next time. - Yeah, thank you. - Thank you, thank you.
- Can't wait for Relativity to keep shaking things up. - (giggling) Thank you. Thank you, team. - Thanks for listening to the Tech Innovators Spotlight podcast. If you enjoyed today's episode, be sure to subscribe, rate and review us on your favorite podcast platform.
Wanna learn more about the AI innovations discussed? Visit techinnovatorspotlight.com for more resources and insights from Microsoft and our technology partners. Stay tuned for more episodes where we continue to uncover the stories that are shaping the future of AI.
2025-01-28 09:44