How Moody’s is leveraging adaptive AI

Jonathan, DEPT: Hello everyone. Nick, first question: who are Moody’s and what do you do? Nick, Moody’s: First of all, thank you for having me here. It’s a pleasure to be here. Moody’s is a global integrated risk assessment firm. Our clients come to us with a number of different challenges, but what they try and do is understand, assess, monitor, and report on the risk exposures that they have in portfolios, in the partners that they deal with over a number of different workflows. And we help them by leveraging an unrivaled set of data analytics and domain expertise to help them really understand where their exposures are, how to value those exposures, and how to make great decisions based on that. Jonathan, DEPT: Perfect. What I’m particularly interested in is that big tech have spent billions of dollars and over the past 20 years, building out AI and machine learning capabilities and infrastructure to be able to support real time decision making and insights, but with the recent advancements and breakthroughs in AI such as large language models, those capabilities and that infrastructure is now available to any business. We help a lot of our clients at the moment at DEPT look at how to get started, the first things to do, and we’ve heard a few times this afternoon that the big challenge is the data, but Moody’s seem to be in a very enviable position where you have access to a huge amount of data. You already have a very mature data practice and data focus in the way that you think, so I’m just wondering how are you combining these brand new breakthroughs throughs in technology and your capabilities that you already have in the organization to build out new products and services? And how are you harnessing AI in building these new services and products? Nick, Moody’s: Sure. Great question. I think we approach it in the same way that you’ve heard many others speak about it today, which is, we’re getting started just as everyone else is getting started. We do have the advantage that we have in-house, a large set of data, whether that’s credit ratings, credit data, credit research, proprietary, public and non-public company data, climate and ESG. So we have a lot of data in house that we’re leveraging today to provide insights back to our customers and clients. And what we think about is how do we leverage that data in a world where consumption of that data and experiences is changing very rapidly with customers? And these are some of the challenges that we’re thinking through today. The benefit is that we do have that set of data. We have a set of skills because we’ve been leveraging AI and ML technologies for some years, and we do have resources internally that have been working on these technologies. Now we just need to reorient them to what does the future mean for us? And what that means is that we need to focus on how do we drive value for our customers? How do we make their decisions better? How do we get the information and data that we have access to into their hands faster? How do we make sense of the noise? How do we provide additional insights? And then how can we help create connections and relationships in the data that were not apparent before, but now with the models that we have access to and the technology that we have access to, how do we leverage that to provide greater insights that we weren’t able to do before? Jonathan, DEPT: A big element for this, particularly when working with your clients, is trust. How are you thinking about trust when providing insights, providing potential, not guidance or recommendations, but certainly insights on data that you have using AI and new kind of AI enabled models. Is it something you’re going to specifically indicate and say, “Hey, you know, an AI model was used to produce this insight?” Or is it something that you just think will be integrated into everything that everyone does? So it’s not something that you need to specifically worry about from a trust perspective? Nick, Moody’s: Well, I mean, I think it’s very important for us because we believe that we have a voice of authority in the marketplace, and people rely on the insights and the data and the opinions that we have. And so we want to make sure that as we go through and we start to work with technologies such as these, that we protect that authoritative voice, we protect the trust that we have in the marketplace. And I think some of the things that we have to consider along the way are what are the outcomes of some of these models? And we’ve heard that a lot today, which is garbage in, garbage out, right? So it starts with the data. We want to make sure that we have access to all of the data that we have, we make that available to the model, and then we have to validate what those outcomes are. That’s a process that we’re still sorting through at the moment, but we understand it to be a very important one, right? We just don’t want to release this data out into the wild, assume that the models are going to interpret it the way that that they will. We want to make sure that there’s some sort of control that’s over that. Yeah. Jonathan, DEPT: So, the the margin for error that you have is very, very low. How are you thinking about managing that risk? You are a risk-based organization helping your clients manage risk. How are you doing it yourself? And also the data is incredibly confidential in some areas, I imagine. So, how would you think about data security? Is it different in an AI world or is it just the same as you’ve always managed data, proprietary data that you have? Nick, Moody’s: Well, I think it’s the principles that we’ve applied in the past and applying those to the future state here. I think as, as everyone thinks about these kind of solutions that you create in this world, and the investments that you can leverage that have already been made, we think about whether or not that’s built on open source technologies, but a closed data set, or whether or not you’re building something that’s completely proprietary and built for purpose. Those are different levels of investment. Those are different time horizons to deliver those products. One of the things that we have to consider is where do we participate along the way? The value that we bring to our customers is in that data and in those insights, and so there’s going to be a strong interest on our part to make sure that we protect that intellectual property, to protect those insights. So, to the extent that we participate in these models, in these large language models, at what point do we draw the line between where our data exists and gets combined with others? Or how do we help refine or maybe fine tune some of these models? And these are some of the decisions and considerations that we have as well as others. Jonathan, DEPT: I guess also using open source or open source foundational models could be quite risky from a a security perspective. How does this require new skills or new capabilities within your organization to assess how good the models are? How are you thinking about that at the moment? Nick, Moody’s: Well, we again, have been in a great position that we’ve invested in AI and ML technologies for some time, and so we have a very strong stable of data scientists, of data engineers. We have a very strong foundation in building cloud-based applications, and so from an engineering perspective, I think we feel we have a pretty good foundation in how we move forward. I think there’s some new roles that we need to think about as we go into these technologies; I don’t know that we’ve had to hire for prompt engineers in the past. So, you know, getting tech skill sets that are specialized at fine tuning. The outcomes that we get from these models are not something that we’ve had to deal with in the past, just as everyone else is going through these same sort of considerations. So we look at the engineering stack and we think that we’re pretty well covered, but we need to add some capabilities here and there. And I think beyond that, then it’s just from a product perspective; we have a lot of product folks that are in place that are thinking about how we leverage adaptive AI in our products and in our capabilities. It’s now orienting those individuals towards the ideation process and what’s gonna deliver the best value to our customers in the short term. Jonathan, DEPT: How do you, because you have the capabilities, you have the data, you have a desire to integrate AI into, into your products and services, how do you go about deciding where your investments are gonna be made, what the things you’re going to tackle first? Is it about new services increasing potentially revenue of your organization, or is it about efficiencies and making things more efficient operationally? How are you going about that sort of thinking? Nick, Moody’s: Well, I think we’re lucky enough to have at the leadership level an enthusiasm and an advocacy for investing in these technologies, and so it’s a mandate that’s being given to each and every one of us within the organization to come up with our path to the right ideas that we should consider. And so that democratization of that innovation within the organization is really key because what it does is it makes the responsibility of every individual to come up with where we think we can focus. The people that are closest to our clients and closest to our clients’ challenges are the ones that are coming back and saying, “here’s how we think we can solve for that”. Now, it’s not just a bunch of folks going out there and, and coming up with ideas and throwing them all in the pot and picking one out and saying “this is where we go”. We do want to have some sort of centralization of the evaluation of those efforts, and also a centralization of how we start to develop, because we wanna make sure that we’re coordinating those efforts. We don’t want to build something in two places and find out that we’ve come up with two different solutions; we wanna make sure that we’re all heading in the same general direction. And so it’s the process of ideation that gets all of those ideas out there. Now, where do you then focus your attention? It’s the same process in any investments that we would be making: what’s the value that we bring back to our customers? That value is both in ‘how do we make their lives better’ and how can we help them make their decisions better and faster? But how do we also monetize those efforts as well? And then what are the costs to doing so. Are we investing for a short-term gain or are we investing for a long-term play? That’s challenging in this particular environment because it changes so often. Right? And so where do you make those longer term investments. And then I think you have to think about, again, the risk in terms of what is it gonna take for us to be able to deliver this. What are the data implications for us doing so? And then it’s also around the time to market, how quickly can we get something that’s actually driving value and driving our business objectives as well. Jonathan, DEPT: One of the things you mentioned before is about the quality of the insights that the models are creating or putting out in the world. How are you thinking about that in terms of validating the outputs from the model? I know that speaking to other clients, that’s something they’re spending a huge amount of time working on. How are you guys thinking about that? Nick, Moody’s: Well, in the same ways in that what we want is quality outputs that gives the right insights to our customers as quickly as possible. We are exploring a number of different options in what we’re doing today. Those are some of the considerations that we have part of it. Jonathan, DEPT: What skills have you found you’ve been missing as you’ve been building this out? Are there any particular things that you’ve been caught off guard and thought, you know what, we really need this particular skill set, or we’re missing this. We really need to, to train or upskill or, or hire these people in there. Anything been a surprise in building this? Nick, Moody’s: You know, I think the biggest challenge for us is scale. So, how do you do this at scale? So again, we’ve, you know, identified a number of resources internally that we can point towards these efforts, but how do you do that at scale to be able to deliver something quickly? Right? There’s gonna be a lot of initiatives, a lot of ideas that come up in the short term. How do you make sure that you’ve got the resources pointed in the right direction, and then how do you do that at scale to be able to impact the whole breadth of the business that we have. Jonathan, DEPT: Just coming up to the end, so probably one or two last questions. We’ve been talking about, as you said, doing it at scale where you do have the capabilities, where you do have access to the data. For businesses who don’t have access to these things, what would you say are the biggest considerations that they need to consider or they need to look at if they were looking to do or start this AI journey themselves? Nick, Moody’s: I think first and foremost, you need to, as an organization, get aligned behind the fact that we’re doing this, we’re doing this today, and it’s okay to move forward without having all of the answers. I think, you know, we might have a lot of questions on where we can participate and what value we can bring, but I think without having a clear sense of every step along the way it’s okay to take that first step as long as we agree on the direction of travel. We’ve had the benefit of having advocates at the senior level that are saying, “this is where we need to focus our efforts because it’s very powerful”. I think it’s just taking that first step and then I think it’s also around allocating the resources to do it. I think we can talk about what these opportunities are, but until you start to allocate those resources, until you start to allocate your investment behind that, it’s not gonna move forward unless you get real sort of traction behind that as well. Jonathan, DEPT: And do you see that as being able to get traction is one big kind of a lighthouse project where you go, okay, this is something we’ve made better with AI and it’s kind of a big thing that the whole organization rallies behind? Or do you think it’s small incremental things to each product or each element that makes everything better over time? Nick, Moody’s: Well, I think it’s probably more incremental. I think we’re going to probably explore a lot of ideas and we’re probably gonna toss a lot of ideas out. We’re probably gonna find a couple of gems that are going to be things that we’re going to want to really double down on, invest in and drive forward. And that’s the process of innovation, right? So, there’s going to be two out of 10 ideas that are going to make it to the next level, and then we are gonna want to evolve those over time. And so we see this as sort of a starting point, right? So what we get out there, when we get that out there, it’s going to be tested by markets and tested by customers and we’re gonna continually evolve as the technology evolves as well. Jonathan, DEPT: Perfect. Well, that’s our 15 minutes up. Thank you very much, Nick. Nick, Moody’s: Thanks, Everyone.