In our insightful conversation with Alex Singla, Senior Partner at McKinsey & Company and Global Leader of QuantumBlack, AI by McKinsey, we tackle the most commonly asked questions about generative AI and how to implement it effectively.
In our insightful conversation with Alex Singla, Senior Partner at McKinsey & Company and Global Leader of QuantumBlack, AI by McKinsey, we tackle the most commonly asked questions about generative AI and how to implement it effectively.
Listen in as Alex sheds light on identifying company-specific opportunities, organizing and governing AI tools, balancing risk and value creation, and navigating the future of talent and tech stacks. We also take a look at the journey of getting started and learning quickly to harness the power of generative AI. We explore the art of creating cost-efficient, scalable solutions that drive adoption and navigating the learning process to maximize speed, cost structure, and reusable code.
Our conversation focuses on the delicate balance between moving too fast and too slow in the competitive AI space. We also weigh in on the role of large language models in achieving industry-specific solutions and how to optimize them for efficiency. Further into our discussion, we address the complexities of building and leveraging large language models, focusing on the costs, pros and cons of in-house building, and the importance of data privacy and IP protection. We then examine the skills needed to run these models, the learning curve, and the economic value derived from this process.
The discussion concludes with a look at how generative AI can be used to improve customer experience and how to implement safeguards to avoid unethical behavior. Join us for this informative and thought-provoking episode with Alex Singla!
Key Quotes:
Time stamps:
Links:
Alex: [00:00:00] You don't want to be the last person in the race when your industry is taking off. Now, at the same time, you know, I have, in my conversations with clients, it's actually funny. I get framed two things with CEOs often, which is, I don't want to be the fastest person out there who blows up my organization because I've made a bunch of mistakes.
And, I don't want to be the last guy who was waiting so long that everyone overtook me and I became irrelevant. And so, neither of those ends you want to be.
Vahe: I would like to welcome Alex Singler. he is the global leader of QuantumBlack AI by McKinsey. Advises clients on strategy, digital, and analytics, operations, enterprise transformation, and value creation across industries with a focus on financial services.
and insurance. Alex is a senior partner at the firm's Chicago office. with over 20 years at McKinsey, he specializes in aiding companies with digital transformations by implementing data and analytic [00:01:00] solutions. Alex, welcome. What are the questions clients ask you most?
Alex: I hate great questions. listen, I think it is, it's evolved tremendously over the last six or seven months.
You know, I would have said six or seven months ago, and there's still some companies that ask, like, pretty basic questions around, like, what is generative AI? What is it? What is it not? you know, is this hype a reality? Will it create value? And amazingly, what I've seen is, usually, to get to the level of conversations we're having today, usually it will take clients six, nine, twelve, twenty four months to get to that level of sophistication.
This happened in about four to six weeks. It was, like, mind boggling. We went from, like, super basic questions to, like, you know, I would bucket them in an oversimplified way in, like, six questions. And they're all around, how do you scale Gen AI or AI? You know, it's just keep that in mind, right? It's not about a proof of concept, but it's about scaling.
And the six questions basically fall into like, what are my company specific opportunities? So, if I'm a bank, if I'm an insurance company, if I'm a telecom, I understand all the cool things I've heard other people do. You know, I don't want to create, you know, a birthday card poem for my nephew over the weekend.
What are the opportunities for my business, first and foremost? Second question, I will put second and third question kind of together, which is How do we organize ourselves and govern around generative AI? You know, historically, you know, AI was kind of, was more structured within the IT, data science, data engineering world.
So it was easier to organize ourselves. Now that generative AI is basically ubiquitous across our institution, how do we organize and govern the tools and the things we're doing internally? At the same time, you know, the third question is always around, This notion that, you know, the ecosystem for generative AI is pretty complicated.
You know, there's multiple LLM players, whether that be OpenAI, or Cohere, or Anthropic, or you name it, there's a whole bunch of them, right? And every third party provider is providing JNI tools, whether that be Microsoft, or Adobe, or Salesforce, you know, they're all doing it. Of course they should, right?
That's all the right stuff. And at the same time, you have all the cloud providers who are also major players in this, in this game. And so, as you think about not just organizing your own self, but how do you navigate the partnerships that you're going to have to have to deliver this at scale? It's actually a pretty complicated, you know, puzzle that you got to figure out.
And it's not a one size fits all either. So, that's kind of question three. Question four, and this is particularly important for banking, for everybody, but banking in particular, is how do we balance risk and value creation, right? Risk and value creation. Of course we all see value, but risk is not something we should just sweep under the rug.
It is something that, you know, I would argue should be thought of from day one. Not as a, you know, we've created something, now how do we make sure we manage risk? But how do we think about risk from day one? And in particular... In the business, in the banking industry, as you can imagine, right, you have regulate, you have, you know, it's a regulated industry, it differs dramatically, whether we're talking about like internal employees versus like high value customers, like those are two very different elements of solutions and risks you're talking about.
And so how do you manage that is a particularly important for all institutions, but banking and those regulated ones, you know, it just raises the bar to be thoughtful about that. the fifth question is around, you know, talent and tech stack implications. this will have obvious implications on talent, people I have today and people I have tomorrow, like what do I have to hire for tomorrow, not just from a technical perspective.
But how do I hire people and train people to think about, like, asking the tools the right questions the right way to get to better answers? And obviously the tools will help guide people through that more and more, but they'll have real implications. And, you know, of course there's always an efficiency conversation that, you know, exists in the market as well around [00:05:00] what does this mean for our future employee base.
tech stack implications, real implications. If you want to do things at scale, you need to have generative AI at some level baked into your tech stack, right? It's not like these are just external tools you can just leverage and you get the value. Of course there are some solutions like that, don't get me wrong, but some of the ones that will have more transformative impact will have tech stack implications, whether that be You know, who is our cloud provider?
How are we structuring our data? How are we making sure that, you know, our results come out in a good fashion? It's all these things that will matter. And then lastly, which I think is the really important question is, how do we get going? How do we learn fast? And how do we ensure we're creating value along the way?
Because, like, for any industry, any company, you know, anyone can come up with, like, here's the 50 use cases you could think about, right? Like, there's, there's use case roadmaps all over the place. But the real question... How do you start? How do you start creating value? How do you learn? Because, to me, this is a bit [00:06:00] of a learning journey, you know, I have this notion of learning quotient, so how quick, you know, like how you have IQ, you know, intellectual quotient, emotional quotient, relationship quotient, I think there's an LQ too, which is learning quotient, which is how fast am I learning and, and how do I How fast do I learn and how do I bake those learnings into my continuous improvement of generative AI?
So sorry, but those six questions are, you know, there's a thousand others, but they usually fall into those six buckets at some level.
Vahe: Very interesting. You talked at the last point was really about learning at speed a lot. Why do you think that speed matters that much, speed of learning? Does it matter because AI centralizes a lot.
Is that the problem with
Alex: AI? I have two frames. One is, I'd say, in this world, and in many worlds, speed itself is a strategy. If you and I were doing exactly the same thing, but I'm just faster, every time I'm faster. I [00:07:00] will continuously beat you, right? And so, speed itself is a strategy. Now, what I don't mean by that is, like, reckless speed.
I'm not saying that. I'm saying, in general, speed is a strategy. And so, why that's important is because those who, and I can speak for even what, you know, a little of our own experiences, McKinsey and Quantum Black, what we've done to ourselves, and how that has helped us learn so much because we were so fast out of the gates.
Deploying tools, putting in solutions. We learned so much along the way. And some of that, I'll just give you an example. Some of what you learn is not just what you build, like what solution you build, but how you build it matters a lot. So, and why the how matters is because If you build things in a way that are, that get to the better answer faster, you ping off the cloud less often, and therefore you have less consumption.
If you provide better results on your first query versus your fifth query, your employees or [00:08:00] your customers get a better response faster. And so all these things make, have impact on scalability and cost. And these are really two important factors, right? Lots of organizations, some organizations I interact with, They celebrate they created a proof of concept and then you'd say, okay, how do we break down the cost structure of that proof of concept and the scalability of it and the adoption of it?
And you quickly realize the cost structure, first and foremost, is completely not scalable because they didn't, the how they built it was not cost efficient. And so you need to build these tools in a way that they're cost efficient and there you learn how to make it more efficient. Like even in our own experience.
The first time we ran some queries against our knowledge database was much, much more costly and never could have been scalable at that cost structure. Now it's less than two cents. It's like, it's like a cent. It's completely a scalable solution because the cost structure allows us to do that. But we have to learn how to get there.
Now that learning occurred in weeks, not months or years, [00:09:00] but that learning was important. And so I'd say that's one element. The second element of the learning is You know, you're going to put things out there and you're going to try to figure out like, why are, why is only 30 percent of my employees adopting this?
Why are only 10 percent of my employees or customers adopting this? And 10 30 percent is not what you want. For these types of solutions, you want tremendous amounts of, of consistent usage of these tools or of these programs or of these insights in a consistent manner. That's hard. And so the learning around how do you scale is really important.
The speed of learning matters a lot because it will help improve your cost structure. And then it will also just teach you how to do things faster in a way in which one of the key learnings we've had is how do you have reusable code? So as I build these solutions and I have modular components of code, how can I leverage those for the second and third models that I'm going to go build?
Which then, of course, will be cheaper and faster because I'm leveraging reusable code. [00:10:00] But that's a learning journey. That's not something that happens overnight. and then the last one is, as I kind of commented and joked on the very beginning of, you know, if I was faster than you is... You know, you don't want to be the last person in the race when your industry is taking off.
Now, at the same time, you know, I have, in my conversations with clients, it's actually funny. They frame, I get framed two things with CEOs often, which is, I don't want to be the fastest person out there who blows up my organization because I made a bunch of mistakes. And, I don't want to be the last guy who was waiting so long that everyone overtook me and I became irrelevant.
And so, neither of those ends you want to be. But I would argue you want to be closer to the first, the right hand side of that curve, than the back end side of the curve, but not in reckless abandonment, right? So, like, it's an important balancing act, but I truly, truly believe speed matters. We talk
Vahe: a lot about cost and cost structure.
I think we, GPT and other very large language models, [00:11:00] so we call them now frontier models, have achieved tremendous things. Like GPT has passed Antwerp University Medical School of Medicine clinical reasoning exams with a score of 72%, which is I think even above average. And it has passed law exams and other things.
It does it at the cost of, supposedly it's not open, but supposedly GPT 4 has 1. 7 trillion parameters. So it actually costs a lot to do even, to use it even in inference. Do you see that trend reversing? Do you see that language models getting smaller but targeted for industries? And would you advise your clients to build their own language model or use frontier
Alex: models?
Wow, great question. The interesting thing is my response today might be different in six months. So all of the, you know, the realities, I don't know for sure. but what I, but what I would say, I do think I have some truths at least sitting here today. Which is, I do think the large language models are converging.
So what I mean by [00:12:00] that is, as we've sandboxed all of them since, you know, for the last long period of time now, what you saw was at the very beginning, the pros and cons of the different models were pretty clear. Like, some were really good in text, some were better in audio, some were better in, you know, different types of modems, mediums.
And what you see is now that you're, that they've had more time to refine, you see things more converging than diverging on models. And so I think that's one important concept that, you know, will there be 20 large language models out there? I don't think so. I think there'll be a handful, but, you know, time will tell.
I think there'll be a handful. I do think though, to your question around specific sectors and industries. the large language models do a great job on general stuff, but as you get to more, at least today, the more specific solutions in an insurance company, a bank, a retail customer in the UK with seven competitors.
The large language models don't quite get to that level of specificity [00:13:00] today. And so can I imagine a world in which there are sector based or function based large language models that might sit on top of a large language model that are sector based or more specific? I can see that happening, right? And so you can envision a world in which that happens.
If you just take traditional AI, that's exactly what's happened, right? When I stepped in and did my, my role, you could just have, you could have great data scientists who are creating great models, and that was good enough, because regardless of sector or industry, because that was moving them quite a lot.
Years back, a couple years ago, We said, we can't, that's not good enough. We need to have models by sector. And so, like, all of our technical folks, over time, are sector based technical folks, or functional based technical folks. So, they're creating solutions that are fit for purpose for that technical, for that sector.
Can I imagine a world where Jenny and I will have to, the same things? I [00:14:00] can. I can see that being, that, that's going to be the differentiation at some level, right? Because if everyone just leverages the same large language model, it's In theory, you've competed away the, the, you've competed away the differentiation.
And so, those who customize, add some things on top of that, and make it more specific, and leverage their own proprietary data and insights, I think that's how you win. I think that's how you create differentiation and strategic distance. So, I can see that happening. Now, I do want to address one thing, which is, you know, should companies build their own large language models?
The models might not be exactly right, but everything in, at least in the conversations we've had, it sounds like it takes, you know, somewhere north of a hundred million dollars to build a large language model. And then, you know, tens of millions of dollars just to maintain, keep it up to date and so forth.
Most organizations aren't willing to invest that type of money. In, in, in that large language model, but yet leverage the world's large language models, bring them in house, and then [00:15:00] customize and put things on top of the value add. I think that's what the vast majority of organizations will do. will there be companies that actually would warrant building their own?
Yeah, I think there are. I do think there are. I think there's, you know, Bloomberg has come out and said they're going to build one. I don't know where that stands today, but they've said that in the past. Intuitively, that would make sense to me, given the amount of data and information they have and money they have.
That could make sense to me. But I think you have to think of it like at that scale and size, where it would make financial sense and the differentiation of your model would be that much greater than taking a large lot of LLM, customizing it, and getting a solution versus building from scratch. Yeah, but that brings
Vahe: another point, right?
So, you have to be able to take a publicly available large language model and then customize it to fine tuning and other methods, right? and there is, even there are two trends, right? You have, some language models that are now commercially available, done by [00:16:00] Facebook, Falcon, and others. And you have, of course, OpenAI, Anthropic, and others who have an API, basically.
So, like, I'm asking myself, if you're using such an API, Um, aren't you then helping these companies to get better and at some point then compete with you?
Alex: I think that, you know, when we were talking about the how question earlier, I think this is the how you build matters a lot, right? So. You know, there isn't a company that I don't talk to that worries about IP, data privacy, you know, every, rightfully, right, it's, it's actually quite inspiring to hear every time, every CEO, every top team thinking about IP and data privacy and protecting their customer's data and, you know, basically not making sure their information doesn't just make the LLM better, which then makes everybody better.
And so the how, I think, is a structured way in terms of, do you bring those models in house within your four walls and then only run your [00:17:00] proprietary information on those, on those models? You know, I think that's what most companies are doing. That's what I see most companies are doing, right? They're leveraging the large language, the external large language models for things that are completely not competitive.
And so, you know, for instance, whether you're SAP or Workday or whoever, they're, they're gonna bake in some basic HR tools into their, into their platform, right? That's obvious. Like they, they will, they will. And they are, you should leverage those models. They'll be pretty darn good. and unless you think that's your point of competition and differentiation.
And we just leverage that model that's automatically handed to you. And are you improving their tools? Sure, but if it's not a point of differentiation, I'm not sure you care. One could argue one way or the other, that would be my ingoing. But then if it's like the core of my business, what I do day in and day out, my proprietary data, my proprietary insights, I'm definitely bringing that, you know, I'm bringing those models in my four walls, fine tuning them above and beyond, making them better, but then [00:18:00] I'm not improving the, the large language model of the provider.
I, I
Vahe: couldn't agree actually more with this, but it also requires lots of skills, right? Because you need some data science skills, but also lots of engineering skills, right? Because to run these models, quantize them, make them smaller, that they actually run on affordable hardware and so on is actually a big engineering practice, right?
So, so the learning curves.
Alex: Back to the learning curve point here, I think everything you were saying is spot on and how do you do it cost effectively? It's a really, really important point that, you know, I think the game will still be played out on how do the economics work out and who captures the economic value from this, right?
And that's, you know, will it be the large language providers, will it be the cloud providers, will it be the companies, will it be the consumers? That's still a, you know, like that game still needs to be played. I know the cloud providers will by default make money because they can compute, right? So [00:19:00] consumption, that will happen, understandably, by the way.
But the rest of it is still, I think, yet to be figured out. Yeah, I think,
Vahe: that's, that's very much true. Even if you look at this large language model providers, right? I mean. The OpenAI is putting in millions in research and training costs and then Stanford Alpaco paper shows with like 700, you're replicating it almost at like 90 percent accuracy of that level, right?
It's 700. So it gets, of course, to be a problem, like what is your business model going to be as OpenAI? Like, how are you going to capitalize that? That's interesting. And I think in banking, Nikinzi has pointed out a couple of areas where they see a good, good, use for generative AI. And, and one is, of course, chatbots that help, you know, the customer experience, right?
How, or how, what do you think about that? And, and specifically, how would you guardrail these chatbots that they don't give wrong advices or don't do, you know, unethical [00:20:00] things
Alex: and so on? Listen, I think in the customer experience world, there, there's some great solutions That are, you know, that exist out there today and they're always getting better.
Right. And so if you just think of, you know, chatbot on steroids, it's just so much more efficient. It's, it's very, it's very accurate and helpful, in terms of your question around, like, how do you just make sure it's providing, you know, good information in an unbiased way that is customer friendly, right?
At the end of the day, you're trying to combine all those things. So it's also, you know, empathetic to the customer coming in with a question, right? And it's, it's framed right. And all those things are completely modifiable, right? So like, Even as we build our own, and I'd advise any of our clients that we work with, is this is part of the learning journey too, right?
It's like, let's ask it all the questions, look at the responses. Make sure the responses are, fits all those solutions. And then you, you basically create the model to self police itself to some degree to ensure that it doesn't have any of those things that you're worried about. But at [00:21:00] some point you're just feeding the model a bunch of, you know, inputs of what the questions are and an accurate, accurate output.
And the model over time will, will get that accurate. You know, I think the, the one thing that we, we of course do before we like deploy some of our solutions internally and obviously with our clients is You're checking, you're making sure there's not errors in the system. Right? And so when you have confidence, you know, that's when you start to deploy to customers, right?
I usually talk about the fact of, let's do some things inside my own organization with my own employees, get confidence it's working, confidence it's not going to blow something up, before I start, you know, the first thing I wouldn't deploy is to like, my wealth management clients, like my high net worth clients.
I wouldn't just like, throw something out there to see what happens. I would do that in a more thoughtful way. So there's a little bit of a test, again, test and learn atmosphere. The good news is you can use, you know, tools and systems to check things. You don't need humans to check everything. And that
Vahe: brings up an interesting point.
What are the roles of humans then? Do [00:22:00] you think that AI will take off, take, take, take out entire roles or will it always be an augmentation with a human or always more like in the foreseeable future, obviously?
Alex: somebody just talked about this. Someone in the bank. I think it was Jamie Dimon, but I could be wrong.
But, listen, I think we're talking about activities. Like, if you look at our reports around productivity of, of generative AI, it's about activities, not necessarily jobs, but activities people do. and so it's a, it's a really important distinction. Right, so if you look at the, the research we did, we looked at activities in job, in job classifications, or activities within a job.
And then said, could that be, technically, could that be automated through any tool, but generative AI as well? And so, can swaths of activities be automated? Absolutely. But, I think a lot of things today still have what I would describe as a human in the loop solution. And so, even if you [00:23:00] have, like, let's just imagine, back to your customer service or contact center example, you could absolutely still have a human on the phone.
talking to a customer, but sitting on his shoulder is a generative AI machine, providing them information, listening to the conversation real time, feeding your rep what to say, how to, you know, what to say, and how to guide them, but that human is still in the loop at the end of the day making a call.
You could see, you know, for certain questions, for certain questions like, you know, help me change my address, I don't need a human in the loop. For other questions, I need a human in the loop, right? More complicating things. And so, again, it comes down to activities, not necessarily whole scale jobs, I think is what will happen.
The interesting thing with generative AI, you know, it hits much more knowledge workers, right? Which is traditionally, you know, technology was going not against the knowledge worker. Generative AI is actually knowledge worker activities. And so, there's a shift there in terms of, you know, what are the skills [00:24:00] that you'll have to learn, and what are the capabilities you'll have to have within your organization to, To have your employees still, you know, keep adding value because there are some things that they're just going to get faster and better at because of technology.
And
Vahe: of course, this combination between human and AI also needs to be designed well, right? So because if you don't design well, it's not going to be cost effective, for example. Yeah.
Alex: Can I sit on one thing there? Because I love the way you just said that. I think the other reason things need to be designed well, you know, I, I always say like the reason chat GBT did so darn well, there's many reasons, but one of the reasons is whether you're eight years old or 80 years old.
It's just so darn easy to use, right? The interface, the input and the output are just so easy, right? And that user, that UI UX interface just makes such a critical difference for people to adopt and use and understand how to take that output. So, that's an important dimension. But you're hitting on something I don't want to lose sight of, [00:25:00] which is the vast majority of times, at least in my experience, technological advances don't get fully the value isn't because the technology wasn't good.
It's usually because at the end of the day, and I would argue the vast majority of big opportunity, gen A opportunities, at least that exists today. Some human being will have to do something different, whether that be your employee, your customer, your sales agent, whoever it is, will have to do something different.
And whenever you make a human do something different, all of us, that actually takes real work. That's hard. That's not easy. Right, so, like, 50, 60, 70 percent of the work is the human change that needs to occur and the education that needs to occur. And so, as you build these tools, as people build these tools, it's so critical they build them into, like, the day to day workflow of how someone acts, right, and someone behaves.
So if it's, you know, I'll use, I'm probably aging myself, but if you remember the days where you used to have [00:26:00] other tools and you have to, like, alt tab to another screen, go into a new login, type in your name. Copy and paste or write it down and then come back over to your new one and like that never scaled.
I know it's not as bad as it is, that is, it was, you know, 20 years ago when we were doing that stuff, but you get my point. If it's not in the consistent workflow of how an employee works or how a customer interacts, that takes real change to occur. And then secondly, it doesn't take real change. If all of a sudden I've, you know, freed up 30 percent of my employees capacity.
Then the question is, how do I capture that value? Do I get them, you know, there's, do I get them to invest in more, more customer interactions, more innovation, you know, more the backlog of things I need them to do? Do I take that to the bottom line? Those are all real questions, but at the end of the day, somehow you have to capture that value in the PNF, whether that be, I got, I, I, I grew without adding headcount.
I took headcount down, [00:27:00] I, I shifted those people to work on projects that, you know, that capacity that I wasn't working on before and therefore I'll get lift from there. But somehow you have to capture that value, and it's a really, really important point because otherwise customer clients, our clients will incur cost but actually see P& L value from it.
They'll be more efficient, they'll be more effective. But if I help my sales rep do better, then I need to, at some point, think about, do I need to raise the targets? you know, so that's the way I capture the value. And so it's an important dimension around, value capture at the same time. Wonderful.
Vahe: Alex, it was such a pleasure. Thank you very much for all your insights and, and all your, knowledge that you provided to our listeners.
Alex: Thank you very much. No problem. Thanks for having me. Thanks for listening to this edition of Hybrid Minds. This podcast is brought to you by Cognize, the first of its kind intelligent document processing company, which automates [00:28:00] unstructured data with hybrid intelligence and a data centric AI platform.
To learn more, visit cognize. com. And be sure to catch the next episode of Hybrid Minds wherever you get your podcasts.