Is there a future for business intelligence Understand the key trends technologies and techniques

Show video

So in the beginning, back least the beginning for me was the early 90s there   was business intelligence as we just talked  about, and I would say within a decade it   became synonymous with reporting and  more specifically with dashboarding. That's what people kind of think of it now when  they use that term, correct me if I'm wrong.   And I wrote a book on dashboards in 2005 and a second  edition in 2010, so I was kind of in the middle of   that whole phenomenon of business intelligence when I worked at the Data Warehousing Institute,   but more recently BI has come under assault I  would say a little bit. Last year ThoughtSpot started to market this campaign that dashboards were dead, provoked a lot of interesting commentary   and I of course didn't quite agree with that but I might be biased.  

But over the years, people have been quick to  point out business intelligence limitations if   we think of BI not as the the global umbrella  for things that use data to make your business   more intelligent, but more as the reporting side of  that whole equation, people have pointed   out that BI is too reactive, it's only  looking at past results, it's too generalised   you're only getting the top level summary of  information and the devils and the details   it's too manual to put together as well  as to interpret and it's too descriptive   in other words it's not analytical  enough it's not predictive enough,   it's too inflexible you know someone has to  define upfront what the metrics are that you're   going to measure, what are the thresholds that  that colour code performance, too hard to use   especially for what I would call casual users  or regular business users, people who aren't   hired to crunch data on a day-to-day basis,  and probably that the biggest complaint that   I've heard is that it doesn't turn insights  into action. So, people have been saying that   BI has a last mile problem, and as a  result they're questioning what is the value that   we're really getting out of these reporting  tools, we're spending a lot of money on them   still today but are we getting a lot of value  are we getting not just insights but are we   getting actions based on those insights, and can  we measure those actions, and are they moving   the needle for our organisation. So I think that's  the big question and I think a lot of BI vendors   out there, wrestle with this if they're not power  BI or Tableau, they face this situation a lot.   What value do we bring to our customers how can we  help them justify the expenditures on these tools,  so that they continue to do business with  us, or make room for us in their BI portfolio   and as a result we're seeing this is jumping  ahead a little bit and just from a product   perspective that vendors who are not Tableau or  Power BI are kind of taking one of two approaches.  

They're either going very broad and focusing  about something called this decision intelligence,   which is kind of putting everything in the kitchen sink, into their platform   from reporting, to dashboarding, to everything up to predictive analytics, and also some AI to help close that gap between data and actions, or they're   drilling down into analyst specific tools to really move the needle on self-service, and making our power users, especially our data analysts much   more productive and collaborative, so we'll talk about those two trends. So, Eckerson group TDWI or I used to  work in BARC which is a big partner of ours   in Germany, we've all done these surveys to try to track the adoption of BI in Enterprises and we always come up with the  same results, that on average the BI tools are   used by about 20 to 25% of employees  and that figure is not budged in the last   10, 15 years really. And people have always  scratched their heads, BI vendors are   scratching their heads, even BI managers  sometimes you know, they bring tools in and   expect immediate adoption especially as the tools  get easier to use and they're all self-service,   but still you know there's still a lot of  shelf wear, so what does this mean?   And I kind of ask a contrary question which  is, is this really a problem? Because to me the   BI market is not just one market, it's actually  four markets and I've had this framework for who   are the users of BI tools or analytics tools, for  many many years that's why it kind of looks a little bit   dated here, the graphics, but I kind of see that  there's two major categories of users within   those categories there's two subsets and these  really define the market for BI and why I   think some vendors have difficulty addressing it. As you can see here, that data consumers and  

those are folks are just essentially viewing  dashboards and reports those are executives   frontline workers frankly as well as  customers and suppliers to some extent, and   they're the vast majority of the consumers of BI and a lot of times those folks aren't counted   in surveys that we do and in fact a lot of  times those folks don't even know they're   using BI because they're looking at an embedded  visualisation inside of another application a   portal or ERP or a CRM application and they  don't to them that really doesn't count as BI. They're also data explorers and these are data  consumers who actually want to do more with the   data they want to dig into it a little bit more  they want to open up a dashboard and modify it,   maybe even create their own dashboard or add their  own Excel data to it, so they want to enrich   reports, that's about 30 percent of your employees  and that's actually the fastest growing group   as people graduating from college get more data literate, and tool literate, analytics   literate, I think we'll see the explorers poach more and more of the consumers. Now these folks, don't do cell service in my opinion they're doing silver service which   means they really want data served up to them on  a silver platter, in the form that they want   and can use immediately they really don't want  to do a lot of hard work because frankly they're   not getting paid to be analysts right, so they're  not getting paid and not being hired to be power   users like the data analysts and data scientists  below, those are the folks who in my estimation   really need self-service tools and by that  I mean the ability to create stuff from scratch.   you know go out and find data go out and pull it  together, prepare it, modify clean it, visualise it   analyse it, share it and publish it and data  scientists two percent of your employees they   obviously are much more academically trained,  they're the ones who are using ML and AI to   create predictive models, and discover patterns in  large volumes of data. So when I look at the market   we're seeing a lot of tools and functionality that  are geared to these different types of users or   should be geared to these different types of users,  but vendors kind of have a blunt instrument   when they market, and they tend to provide tools  for everybody because they don't want to short   shift their revenue stream right, but in reality  a lot of times they're pitching power user tools   to business users which is what causes a lot  of the shelf wear that I talked about earlier.   So to me, what do data consumers really need? Well they need those interactive   dashboards that contain just the metrics that  they're responsible for, and the ability to drill   down and drill across and see performance of themselves or their unit, especially if they're   frontline workers or even customers and suppliers  they want embedded analytics, and that market from   a vendor perspective this exploded, vendors see  that as a whole new outlet for their products and   tools, and a number of vendors have converted  from selling you know retail BI, to embedded   BI, Logi analytics was one, and more recently PsySense has focused 100% now on embedded analytics.  

And now of course with the advent of Chat gbt,  you have to ask is that going to replace all the   tools that we already have for data consumers, and maybe even everyone else as well it's a,   as you probably all looked at, it's an incredibly  powerful tool, it's conversational it remembers   the questions you asked prior, it's fast, it gets  it right most of the time and people   have told me that version 4 is going to be miles  above what we're seeing today and I can't imagine   what the impact is going to be so we'll talk a little bit more about Chat gbt in a second.   So for these data explorers, they need to do some ad hoc stuff right so,   this notion of natural language queries where you can type words into a search bar and   generate sequel behind the scenes, which returns  an answer and displays it as a chart or table   and you can keep iterating, you know as long as  someone has built a nice model behind the scenes   that can work pretty well, so ThoughtSpot they came out of the the gate doing this right, made   a big name for themselves being the Google of BI, we're going to simplify BI so everyone can use it   well it's never as easy as anyone makes out. The number two here is something that's been   around for a long time that's the ability if you  see a dashboard, there should be a big button up   in the top right or somewhere, that if you have the right permissions you can click that   button and you can open up the model behind  the dashboard and begin to to modify it right. So you can add dimensions that weren't added into  the dashboard but we're in the model, that power's it.   You can perhaps combine columns, you know you can you can do your own calculations and things   like that, maybe even add your own spreadsheet data  to. So that's something that these explorers   want to do. So those are like managers, who are  very data driven and want to get a little  

bit more detail than what's you know available  in the canned reporter dashboard. And then there's   this third thing that you've probably seen, I  call it assisted analytics, that's where you get   a dashboard and if you put your cursor on a number  or a chart behind the scenes there's an algorithm   that runs against the data powering that dashboard  and it will work on calculate correlations things that are hidden in the data below the level of the dashboard,   that you may not have considered or thought about  looking at, and those products I don't know how   well used they are frankly from the conversations  that I've had, not that many people use them but   for a data explorer it's something that would  help them, instead of them having to do an edit   right of the dashboard or report, they could play  around with that assisted analytics button and see   what the tool automatically generates for insight  and correlations that might be worth looking at. But I think where the real market is today and a  lot of the innovation is, is tools for these data   analysts and to me when we do consulting I always  try to find out you know who are the data analysts   in the organisation because they're kind of spread across all the departments and functional areas, and they're kind of the linchpin  for an analytics data and analytics strategy,   because they've got one foot in the business and one foot in data and technology, they're the   quintessential purple people as we used to call  them at TDWI, not totally in the business,   not totally in IT they're a blend of both and they can really make or break your whole   dating analytics strategy so it really behoves us to pay attention to them, one find out who the   heck they are because a lot of companies don't  even know and they have no idea how much they're   spending on them which is usually millions  more than they they would ever guess because   these are the folks who are actually trying  to make data work in the trenches right, trying   to help the department heads use data to answer  questions they have usually, it's ad hot questions,   but they're overwhelmed because to generate a decent insight oftentimes means   lots of operational work. Finding data, cleaning it, blending it you know just way too much   work, they spend 80% of their time fixing data and 20% analysing it so there's a huge opportunity it's been there for decades to  really empower these folks and we're starting   to see some tools that really make progress, so  one is business monitoring and I'll drill into   tha.t It's kind of a turbocharged version of the  assisted analytics that I talked about earlier.   There's an analytics workbench which try to pull us in all the capabilities that a data   analyst would need, to support their end-to-end  workflow and collaborative intelligence is an   extension of that which essentially says analyst you're all isolated, you're all   reinventing the wheel, let's create one platform  that you can all use together, follow each other,   reuse components that you've built and workflows  you've built and allow administrators who are   super super users to define things once standardised where they are placed and have   everyone use them right, and that alone would  increase the productivity of this group of   folks tremendously so I'm pretty excited about  some of the products we're seeing down there. And  

then finally data scientists, they're in a whole category by themselves, they also have   workbenches from Data IQ and companies like that  kind of coordinate their activities across the   Enterprise. These big cloud data platforms are increasingly catering to data scientists and   including functions and stored procedures that can kick off machine learning functions   providing zones or discovery zones inside their  platform, so they don't have to download all this   data to their desktop they can actually do  their work and train their models on a very   robust environment, processing environment, where  all the data already exists right. And then finally   the hardest part, we've verified this in our own consulting is actually deploying these models, creating pipelines that will distribute the data to the models in real time embedded inside applications and that whole process of productionising models and maintaining the quality and the accuracy of the output, is tremendously difficult and there's this whole I guess discipline, I'm just calling ML Ops that's involved in doing that. So Neil, if there are any questions that you think are relevant just stop me and you   know insert them here because I really can't do  that. There's a few coming down so "how do we categorise employee usage?" was a question I  think you may have addressed that already in here   or maybe not. Yeah I think for the surveys we tried  to, I know the last round we did with BARC we tried to include the embedded usage but it still came out to 20% but yeah it's across   all employees that's the percentage that we're  tracking, how many are using BI in an active way.

That's it, I don't think there's many others I'm  not sure that was from Nathan, he was   talking about people who don't view reports or dashboards so I guess there are other types of   consumers nowadays because how warehouses have shifted from   being just a reporting centre to a data feed as well for downstream consumers as well. Right, so not much else in there at the moment, anyone else please add things, oh   "will Power BI and Tableau model become  passé" was a question as well, good question.   Some people are saying that the whole markets become commoditised, you look at all the   layoffs from salesforce, a lot of them were Tableau  folks so it doesn't quite bode well.  I think BI has to, it has to move beyond and add more value, as I said earlier it has to   it has to close that last mile or really improve  the productivity of your data analysts and data   scientists to be worth the investment and from a pricing perspective   you know Power BI comes almost for free so it's very easy to pick up and use so it's hard to   compete with that, just from a market perspective. I think certainly agree   with that, and there's another one from Chris, which I think is a good one as well which is   "is it a warehouse analytic or is it integrating operational data feeds, or is it both?" Are we converging to where we  have a sort of dual function of a warehouse, it's no longer just reporting, it's also part of the operational data feeds.

It's always been a data hub of sorts, I mean obviously  it's designed to support analytics right, but going back decades, I always remember every company I went into you know   people were taking exports of the data warehouse  and even real-time feeds to populate operational   applications so, it's always been a hub  of sorts on its own, although I just got off   the phone with Roche Pharmaceuticals and  some of you guys probably know that company and   they've made a very public move to the data  mesh and we had a one-on-one with the head of   data warehousing or I'm not sure what his title  is but you know part of the data team and you   know the whole mesh thing is very interesting and they're implementing the mesh using Snowflake,   which is a centralised data platform but it works to support the   mesh in a decentralised way um but the  last question we asked him was "all right so what's happening to your data warehouse  and your data lake are you keeping that? And he gave me kind of a mixed answer, he said well we're migrating our Oracle on-prem   data warehouse to the cloud, but yes the goal is  to have all the data domains building out our mesh   to ingest their own operational data and make that  available as data products, so that we don't need   a data warehouse anymore, now that actually would be an interesting conversation for this   group and maybe we have another webinar on that topic alone because   it's very interesting. There are a lot of problems with the data mesh and   it's probably going to crash and burn, that's always been my prediction and   my conversation with this really nice fella there, confirmed that yet at the same time they're getting some real benefits out of it and they've got IT out of the way of what users need to do and want to do with their data so, all right so I don't even know how I   got on that topic. Absolutely fine I think it's a natural consequence of what the question was   I think I fully agree with you, we did actually have Paul Rankin from a couple of sessions ago to present, but if he's moved on a bit we need to drag him back in again   to talk more. Well he's learning a lot that's and that's exactly what I just got off the phone   with him. Oh wonderful wonderful, it's interesting  I agree mesh is an interesting thing, and I hope that the industry learns from mesh takes the best bits out and moves forward because   it's got a lot of good intentions in there. It certainly has implications for  

the warehouse right and what you guys all do with your Data Vault model so, interesting. Okay so back to you again there's  we're out of questions, others please enter more   questions if you have any and we'll carry on. So, we talk about business intelligence and   it's not going away, it's actually the foundation for everything that we do, as you can see from   this diagram here. There are three areas of intelligence at least in my estimation and

the business intelligence was the IT driven era that started in 1990 and went roughly to 2005,  and then the self-service intelligence era began and that's gone for 15 years, and now we're into the model driven era where we're using ML and AI extensively,   but none of these go away, none of them supplant the other, they all actually build on each other   and as you move up this food chain here, the benefits start increasing as you can   see from down below, we go you know the focus moves from past, to present, to future,   reports are moving from IT generated, to business generated, to machine generated, analysis goes from intuition driven, hypothesis  driven, to model driven, actions from reactive,   to proactive, to automated and the impact is neutral to positive to absolutely game changing. There are things you need to master at each of  these levels, now every organisation goes through   this orbit in different ways, in different paces but from my perspective these are   the keys that you need to master in each of these errors or stages of maturity, in your data and   analytics journey. So in the business intelligence we really are trying to figure out how to   consolidate our data, integrate it, ensure it's clean and trustworthy, we've got standard reference   data and master data and most importantly from a  business perspective we've got standard KPI's and   dashboards that display them. Every company that  we consult with, still has problems with those.   There's still a lot of work to do there and the self-service era, people think that's a   slam dunk, easy, everyone wants to do their own data, but I think we've come to realise that unless you   govern your self-service deployments, they become  absolute chaotic messes and you actually can   make things worse than before you started with  self-service so you need strong data and report   governance, you need data literacy, you need this  workbench that's basically self-service tools, for your power users and you need a data refinery,  that's basically a warehouse or a lake or a lake house or whatever centralised repository  you use to store your data or repositories   where the zones are places where you give users access based on their   role and what they're trying to do. So you have you know your basic landing zone that   no one accesses really, it's just the raw data, it's permanently stored, your refined   zone, your trusted zone, your formatted zone, your  discovery zone, your integration zone, those are   just some of the zones that come on top of my  mind that we use in a recent client engagement,   and they were all important. And then, in this last era it's all about  

machine learning and AI, creating those robust pipelines and deployment vehicles, which I said   was it was pretty hard. The interesting thing about AI is that we're not only using it to do analysis and prediction for the company, AI is being built into all of our data and   analytics tools itself, making those tools much more productive and automated than they were   5, 10 years ago and that's pretty impressive and that's what's fuelling a lot of these new   tools, for data analysts and data scientists and even for casual users as well. Kind of a slightly different take on this, as you said the evolution of intelligence we've   gone from Human Intelligence where actually we're  pretty good computers ourselves, we collect data using our brains, we detect patterns and make decisions based on those   people call that intuition right?   When we make decisions based on our gut feel, but there's actually a lot of data   and pattern detection going on there and sometimes our intuition is extremely accurate,   but other times it can be completely way off the mark and we just have to be aware of our   internal biases that might affect the quality of  our decisions. So with BI now, we've introduced  

machines and they're doing some of the heavy  lifting for us, they're collecting all this data   and organising it and then summarising it based on what we think we need, and we look at   those summarised data sets or dashboards and make  judgments and business decisions so that's great! Augmented Intelligence goes to the next step and  uses machines and AI to not only collect the   data, but also rummage through it with these very powerful algorithms, to understand the   patterns, the correlations, the trends, the anomalies,  and then suggest possible actions. Then humans look at those suggestions and either adopt them or override them, and make a decision. In my opinion this is the most powerful way that our organisations can make use of AI, is by using the algorithms to augment what we do, and if they're good and in a lot of cases they are much better, more accurate, more consistent than humans, we begin to trust them quite a bit, that doesn't   mean you should just let them necessarily go on unattended, because bad things can happen like we   saw the financial crisis of 2008, where people start trusting these models implicitly and   didn't see the Black Swan right in front of them,  they didn't know the limitations of those models.   So obviously the end goal, and for some people is completely automated actions and that   that can happen in our space, we're certainly trying to do that with autonomous vehicles   we're seeing that with chat bots and Chat gbt is an exemplary of that for sure.

So again, you can't trust these things implicitly I've always said that I would   never be the first person, I didn't know if I would ever   take a an autonomous vehicle and let it drive me  around Boston because it I just don't think it   would be capable of handling all the different  variations. We have really bad traffic   patterns here in Boston, and then if you add snow, sleet, hail, I think even it's too much for a machine to handle, but there's going to be instances where it works perfectly fine. All right so that brings us to ChatGBT-3, I'm sure all of you guys have looked at this right? And gals and it begs these questions, will it become the interface   for analytics? Will it replace analytical tools? Will it replace data analysts? I mean, what I've   seen so far is quite impressive, but you have to remember it's a language model right now   that deals with text right, so there are some  considerations there. I actually   just did a podcast with the CEO of Sisense and they're building to ChatGPT's API,  so, in this extension, allows you to convert queries or questions into ChatGPT   questions and converts the answers into tabular format which it displays in Sisense, I'm not sure how useful that is at the moment, but you have to believe that, every analytical   tool, every data tool, is going to figure out how to use and benefit from ChatGPT but there are   limitations, I'd love to hear what you guys think too. So as I said, it only works with  

text right now, not taggler, data doesn't have DBMS connectors, so you can't really point it at   your own data and and let it rummage, although I've heard someone said that they could do that,  so wouldn't that be interesting if you could  let ChatGPT loose on your own databases,   and then enable people to ask questions, so  maybe they're just more tactical questions.   They don't require the joining of data, but then again ChatGPT can pull data from multiple sources   and come up with a coherent response so, we actually don't know the full extent of the   power of this tool for our spatia, I think we're  still early on and obviously it doesn't always get   things right, because there's a lot of crud out in the internet and there's a lot of crud   in our own databases too. It doesn't visually display anything right now and I read   somewhere that requires a lot of data to achieve or approximate a reasonable degree of accuracy   let me stop there, I'd love to see if anyone  has some comments on this ChatGPT.

One question about visual prompts for ChatGPT is   that "are there any visual prompts yet?" don't remember seeing anything there. I haven't does anybody else, has anyone seen that I don't think so. I'm worried about the integrity of source information if ChatGPT   generates stuff which it then feeds off itself for learning from,  there'll be no one creating original content and it'll end up being self-referential.  That was my initial reaction to the whole thing, that it would just been breathing its own exhaust,  I was told recently that actually that makes it smarter, because the first pass it gives a reasonable response to your question right? And then if it gets another try, it will take what it already did and make it much better I'm like huh that's interesting, it's like   I make my research analysts do multiple drafts of reports right and each   time it gets better and maybe the same is true with ChatGPT, I mean this thing is starting to feel like it's got human blood, so.

We have a lot of questions coming in, not all relevant necessary to the point we're talking about here but the challenges of getting governance right, because that's probably the hardest part of BI, the technology is the easy bit to solve it's the governance and political   aspects that kill the projects. I couldn't agree more. "What's the alternative to Power BI and client-side  ETL in the future as you see it?" Could be this.   Client-side ETL, well there's a lot of these data prep tools,   and there's a big movement to create citizen data integrators right, so you don't need IT   people to build your pipelines anymore you can  have your regular run-of-the-mill data analysts.   There's a few people are wondering "are we building Hal from   2001?". We're building what? Hal, the computer that took over the spaceship. Yeah,  ChatGPT is yeah. You heard it first here, Wayne said we're building Hal. Okay I think we've got a few more questions towards the you've got some more   slides, do you want to cover yeah. So how much time  do we have here? We've got a 10 minutes or so and  

then we can take questions at the end and if necessary we can take things offline at that point.   All right um. But we can't overrun this, there's no real hard limit so, if you want to spend an extra few minutes there's no trouble.  Yeah so, one of those tools for power use, I call business monitoring that, some people call   it augmented analytics, there's a lot of different names for it, but it's like these   IT monitoring or application monitoring tools we use to manage our infrastructure but it's   actually for business metrics. It's just releasing these very sophisticated  

algorithms on your business data and allowing them to rummage through them on a   regular basis, and then surface only the things that are really relevant for you as an individual, and I think this holds a lot of promise to improve the productivity of data analysts. So, a dashboard I always said, if you had a thousand metrics that was a good   size dashboard right? 10 metrics, 10 Dimensions, 10  hierarchies and a business monitoring system   it might be evaluating a million or more metrics yeah, 100 metrics   100 Dimensions, 10 hierarchies at the minimum right but maybe even much much more and   because it's a machine it doesn't ever get tired  and you just let it loose on your business data   and it is going to spot and surface the relevant trends correlations and anomalies   and basically it's a modern AI driven alerting system or surveillance system, that is going to be   very useful for data analysts to point them in the right direction it's not going to necessarily do   the work for them, although it might in some cases spot things that they should have spotted,   but in many cases it's going to help them figure out where to do their analysis and really   save them a bunch of time. And the way these things work is, you just give it your data and it starts   processing it and it learns, it creates a baseline for every metric and combination of metrics right   so you may have a million combinations of metrics and dimensions, and time and it's going to create   a baseline for each of those metrics right, by a day, by a season and gradually it will refine that baseline, so that if there's an anomaly, it will detect it very quickly.

So let's see here one, learn metric behaviour, two, detective significant   deviations, personalise the alerts so here's the thing, if every deviation you were alerted   to it you would just ignore it right and that that's what happened to the BI tools back in   the early 2000s that started to offer alerts everyone just ignored them, because they were   dumb right. Now we've got smart alerts, and the next step for these things is to correlate deviations   to actually suggest root causes and ideally in the future remediation. So this is going to help us   close that castle right, the last mile BI between insights and action, these tools will help do that.

So here's an example from Outlier, which I really like this product and this tool, unfortunately it's   a funny story, they had a deal to be acquired by Twitter and the deal is going to close   unfortunately the day before Elon Musk decided to buy Twitter and then the whole thing fell   apart, the company ran out of money and I don't even know where the IP is, I asked   because I thought this was a really good tool, you can see here how they the spot, detect anomalies but they do a lot more than just anomalies, they do correlations and what causes, which are essentially very, there are correlations that there's a high degree   of confidence in, that's really what they determine as a root cause. So you can   see here at the bottom here they've got three potential root causes.   So, the cool thing about this tool, is that it uses algorithms to personalise the alerts so that you   weren't overwhelmed and it gave you each day a daily briefing book with three interesting   pieces of data that you probably need to know  based on you and things you were interested in.

So it's like hiring an army of analysts because it's doing a lot of the heavy lifting. Makes them 10 times, 100 times more productive once in the right direction, separates the signal   from the noise which is what an analyst does, call it a data surveillance engine. Well I guess we got a couple more minutes, so another thing we're seeing is tools   that kind of pull together all the different capabilities that a data analyst might need   to do their job well and there's a lot of different capabilities as I'm   building out this slide here, and we're seeing some tools pull all these together   in a coherent way and one tool I really like is Promethium it's kind of a   data catalogue that was its heritage, and it uses  AI to do natural language certs and underneath the   covers there's a data virtualisation capability that federates data across the Enterprise. And then once you find your data, you can profile and evaluate it, then it lets you   prepare the data through joining the data, building pipelines and visualising the data.

So, it's a complete workbench, very powerful and it also has got some interesting workflows   built in which I'll talk about in the next slide, which is the collaborative intelligence   capability. So you see the workflow in the middle here, and if you put this platform and   make it available to all of your data analysts throughout the Enterprise, you create a fabric that   stitches them all together right so now they can start seeing what everyone else is doing   and start to perhaps reuse work instead of reinventing the wheel so, the first thing they need   is this catalogue and ideally it's got metrics and queries in there that they can reuse. It allows   them to follow each other and rate workflows, share of workflows that they build, it allows   administrators or authorised super data analysts to build guard rails into the platform so that   maybe they're all joining data the same way or handling slowly changing dimensions the same way. And that the cool thing is ideally they've got workflow that enables business users   who have hit the limits of their dashboard or maybe they're even you know junior data analysts   right, and they need data to do their analysis and how's that happen today? Well they find a   friendly data analyst or a data engineer and they send an email or create a ticket right, but it's not a very iterative process it's not a very transparent process,   so some of the new tools like promethium does a really good   job of building a workflow between business  users and data analysts and data engineers   so that requests one, if they've already been asked they're answered immediately with existing data sets. If they're not asked people can then start to have a

chat and collaborate more closely with the data analyst to refine the query or   the question, until they get a satisfactory answer. So I think that's very heartening. All right we kind of talked about this. If you're not Microsoft or Power BI what do you do you? You either go broad and try to be all things to all people or you   go deep and try and build one of these analyst tools we're just talking about. This is the go broad version, it's got everything in the kitchen sink in it.

For any type of user, they're not really differentiating between power users and casual   users. Some vendors are marketing or messaging this  decision intelligence Pyramid Domo Tellius and Diwo,   but most BI tools are heading this way. They're adding   all the capabilities, even data science built into a BI tool, data prep built into a BI tool, simulation built into a BI tool. You got to wonder if a bi tool is really capable of   handling all of that, and whether people would  really want to go to a BI tool to do all that!

2023-03-02

Show video