Unlocking ROI: How to Measure Impact in Learning & Development

ROI methodology helps organizations ensure their training investments drive measurable results. In this episode of Powered by Learning, Dr. Patti Phillips, CEO of ROI Institute explains how to embed ROI into program design, maximize organizational value, and unlock the power of data-driven decision-making.

Show Notes:

Patti Phillips, CEO at ROI Institute shares best practices for measuring ROI in training. Her takeaways include the following: 

Start with ROI in Mind: Designing a program with clear ROI objectives from the outset increases the likelihood of achieving positive results and simplifies the evaluation process.

Involve All Stakeholders: Engaging managers and stakeholders in the ROI process ensures support for implementation and helps learners apply their new skills effectively.

Methodology Matters: Surveys, interviews, and other data collection tools are essential but should be carefully designed to gather meaningful, objective data.

Use ROI for Strategic Decisions: ROI methodology allows organizations to make informed choices about resource allocation by evaluating the financial impact of their training investments.

ROI Institute offers free resources like the ROI Estimator, along with certification programs and tools, to support organizations in building their ROI capability.

Learn more about the ROI Institute

Check out the ROI Estimator

Connect with the ROI Institute on LinkedIn

Connect with Patti on LinkedIn

www.linkedin.com/in/pattipphillips

Powered by Learning earned Awards of Distinction in the Podcast/Audio and Business Podcast categories from The Communicator Awards and a Gold and Silver Davey Award. The podcast is also named to Feedspot's Top 40 L&D podcasts and Training Industry’s Ultimate L&D Podcast Guide.

Learn more about d'Vinci at www.dvinci.com.
Follow us on LinkedIn
Like us on Facebook

Transcript

Susan Cort: [00:00:00] L& D teams want training to have an impact, but how and when you measure ROI can be challenging without a plan. 

Patti Phillips: How do you increase the chances of getting a positive ROI? Well, it's all in how you design it. It's beginning with ROI in mind up front. Too much of the time we roll out a program and then someone says, Hey, show me the ROI, and now you're having to figure out what the measures are.

Patti Phillips: But if we begin with that business impact and that ROI in mind up front, Develop those objectives, not just the learning objectives, but the application objectives, the impact objectives, and the ROI objectives, and then use the objectives as the architectural blueprint and build evaluation into the program.

Patti Phillips: You get better results, and you make the evaluation much easier. 

Susan Cort: That's Patti Phillips, CEO of ROI. ROI institute. Patty will share how their ROI methodology helps organizations ensure training investments drive measurable results. Next on [00:01:00] Powered By Learning. 

Announcer: Powered by Learning is brought to you by d’Vinci Interactive d’Vinci's approach to learning is grounded in 30 years of innovation and expertise.

Announcer: We use proven strategies and leading technology to develop solutions that empower learners to improve quality and boost performance. Learn more at dvinci.com. 

Susan Cort: I'm joined today by d'Vinci Senior Instructional Designer, Project Manager, Beth Buchanan, and our guest, Patti Phillips, CEO of ROI Institute. For more than 30 years, ROI Institute has helped organizations evaluate the success of projects and programs, including measuring the financial return on investment by implementing their ROI methodology into organizations.

Susan Cort: Welcome, Patti. 

Patti Phillips: Thank you so much. It's a pleasure to be here today. Great to talk with you, Patti. Thank you, Beth. 

Susan Cort: Patti, let's start out by sharing a little bit about you and ROI Institute and how you help organizations measure [00:02:00] learning. 

Patti Phillips: Yes, thank you so much. So, ROI Institute is founded by myself and my husband, Jack Phillips, who developed the methodology around which we have built our business.

Patti Phillips: He developed the ROI methodology back in the 1970s when he was a young engineer at Lockheed, and he was asked to show the ROI on a cooperative education program. He began with a framework, bringing in cost benefit analysis, and then operationalized it. Putting in place a process and standards and over the years through research and vetting through academics and chief financial officers and executives, we've developed the process that has now become pretty much the gold standard process to demonstrating impact in ROI.

Patti Phillips: So we built the business about 30 years ago. We each have corporate executive history, so we've been in corporate, built the business, and we work with organizations worldwide, so for profit, not for profit, non-governmental organizations, and we work in a variety of fields. Training and development, the human resources space is our [00:03:00] primary, that's where we built the business, but we also have expanded the methodology to Cut across the enterprise, which makes it a really robust process.

Patti Phillips: And we work with organizations multiple ways. One is, you know, we'll do the work for them. So, we have a consulting team that goes in and conducts studies. So, we conduct impact studies, ROI studies on major projects for clients. We have the consulting arm, but we also have the coaching and the other support.

Patti Phillips: And another piece of the business and a real big piece of the business is we like to teach people how to do it. We know that through learning and development education, that's where sustainability comes in. So we have a certification process where if an individual for a company has a program that they want to evaluate and demonstrate the impact and ROI for it and build capability.

Patti Phillips: Our certification process is one way to do it very efficiently, because when we look at the cost of bringing us in to do it versus the cost of them learning how to do it, and they still get the coaching and the support that comes with [00:04:00] doing it, as well as the educational portion, is a real efficient way to get it done while building capability.

Beth Buchanan: Sounds like you approach it from, from all angles. Um, you know, everybody is involved from the top down, coaches, certifications, consulting. Can you just take a step back and tell us a little bit about the importance of ROI in training? What steps can you take to sort of improve your chances for a positive ROI?

Patti Phillips: So, you know, it's important, one, to understand what you're doing, how you're doing it, why you're doing it that way, and that's where education comes in, regardless of whether it's ROI or anything, right? We need to understand what it's all about, and when it comes to ROI, there's a lot of misinformation, and in some cases, disinformation around it.

Patti Phillips: People are misinformed. You know, often afraid of it. They think we shouldn't be doing it. There's a lot of reasons that they have for some of the misinformation out there. So, that again is where education comes in. Understanding what it is, why we use it, how we use it, the context in which [00:05:00] we use ROI, and why it's important not to just focus on ROI.

Patti Phillips: That's the key. The ultimate measure because that's the measure where we can compare benefits of our programs to the cost in one metric, but it is only one. If you look at any annual report of any organization, they're working off multiple measures where we do the same. So, education clarifies the what, the how, and the why.

Patti Phillips: It gives you methodology. It helps show you how you can use the existing processes that you have in place and how it all fits under the umbrella of the ROI methodology. And the, the other thing that it will do is help you see how Integrate assessment, measurement, and evaluation into the design of the solution.

Patti Phillips: And that gets that second part of the question, Beth, is how do you increase the chances of getting a positive ROI? Well it's all in how you design it. It's beginning with ROI in mind up front. Too much of the time we roll out a program [00:06:00] and then someone says, hey Beth, show me the ROI, and now you're having to figure out what the measures are.

Patti Phillips: But if we begin with that business impact and that ROI in mind up front. Develop those objectives, not just the learning objectives, but the application objectives, the impact objectives, and the ROI objectives, and then use the objectives as the architectural blueprint and build evaluation into the program.

Patti Phillips: You get better results and you make the evaluation much easier. 

Beth Buchanan: How is that different than, you know, evaluation in terms of like, kind of like a survey or, you know, a lot of people feel like they're evaluating with a survey or something like that at the end. How does this go a little bit deeper?

Patti Phillips: Well, it's methodology. It's process, right? I mean, you can collect data with a survey and so we use surveys for sure. Surveys are useful tools, but it's all in what are the questions you're asking? How are you asking them? Why are you asking them? How are you going to use them? And what insights do the answers to those questions actually [00:07:00] deliver?

Patti Phillips: So it's not just Send a survey, collect data, call it good evaluation. There's really a process to it. And as you know, I think it was Debbie Edwards Deming, father of total quality management, said if we can't explain what we do as a process, we don't know what we're doing. And so we're giving process and then good research methodology.

Patti Phillips: And so We have standards that support the process. We also comply with standards that are just fundamental to good program evaluation. 

Beth Buchanan: I learned that lesson with teaching, you know, having to stand up and explain, you know, I taught literature and you can read the book, but Teaching that to someone else is a little bit different.

Beth Buchanan: Um, so I love what you said there about asking those right questions and sort of building it into the design process. Um, you guys have tools for that, don't you? Can you tell us a little bit about, um, you mentioned something in previous conversations called the ROI estimator. What is that? 

Patti Phillips: So, the ROI Estimator is a tool we have on our website, and it's free for anyone to use.

Patti Phillips: [00:08:00] I'm happy to share with you guys the link if you want to push it out to your audience. We created the ROI Estimator in partnership with Warwick University many years ago, and we allowed them to use it for a few years before we opened it up to the entire community. And what the Estimator is, it's not It's not forecasting the ROI by any means.

Patti Phillips: It is estimating the ROI based on the design of your programs and projects. We took, we have hundreds and hundreds of studies, client studies as well as the studies that we have published in the case studies, which are also client studies, but they've given us permission to share. We've taken all these studies and we looked at the studies and said, what are the elements?

Patti Phillips: from a design standpoint that really differentiate those programs that are delivering really high ROIs for those who are not working at all. So we took those design elements and we provided behind each design element a rating scale. [00:09:00] And so you can go use the Estimator to rate yourself along these design elements to just get an idea of what the potential ROI could be.

Patti Phillips: Again, it's not a true forecast because it's not. necessarily quantitative in nature. You're not really using business metrics and doing your analysis to do the forecast, but you are considering, given these design elements and the weighting behind the elements, here's the potential ROI we have if we can make it work, if we can build it so that it works.

Patti Phillips: But that also includes, Beth, and this is something that a lot of people forget, it is about that transfer of learning. Because it's not just building a program and delivering the program, we have to look at who is supporting the application of the program, and that's a big part of design. Who needs to be involved?

Patti Phillips: What do they need to know? What do they need to do with what they know to make this program work? 

Beth Buchanan: Yes, and there's a lot to unpack there, so I, I would love it if you could tell us a [00:10:00] little bit about, I know this process, you have 12 steps, you have your 12-step process, can you tell us a little bit about How that unfolds.

Beth Buchanan: What, what is that kind of the shorthand? 

Patti Phillips: Yeah, the shorthand, so it's 12 steps. Someone once told us 12 steps is too many. It's like, okay, call it four phases. Okay, so go with 12 step, four phases. So, we break it down into the four phases and it begins with planning that evaluation up front. So, the very first three steps is all about the alignment.

Patti Phillips: So, starting with why. What are the business measures you're trying to improve and what's the value of improving them? And is this. So, we're going to talk about what is a problem or opportunity worth pursuing, and then doing the work to ensure you have the right solution in place in developing those objectives.

Patti Phillips: So, the first three steps are all about alignment, aligning those programs to the needs of the organizations, and then setting them up through those objectives so that when you build it and roll it out, you're doing so with the objectives in the end game in [00:11:00] mind. And then there's data collection. So, we collect data throughout the process.

Patti Phillips: So we're going to collect those data during the program. We collect the data after the program. Of course, you have your preprogrammed data that you collect, and we use a lot of different techniques depending on context. You mentioned surveys. Surveys, number one, most frequently used data collection method, and many people say they do it because it's easy.

Patti Phillips: Surveys and questionnaires are easy to administer. But if you want good data, they are hard to design because we've got to make sure we're asking the right questions the right way to the right people in such a way, they will provide the most objective data possible, given the nature of the data collection.

Patti Phillips: So we spend a lot of time on survey design. We actually wrote a book, Survey Basics with ATD a few years ago. I taught survey design in a doctoral program. And so all of the for about 10 years. And so all of that. We combined in the very brief book on survey design [00:12:00] because we use them so frequently. But we use other techniques too, action planning, focus groups, interviews.

Patti Phillips: So a lot of different ways we collect data. That phase can be the most expensive part of our process if we're not careful. It should not be. But we will often see people use a survey, use interviews, use focus groups, and there's something to be said for triangulation, but there's also something to be said for the benefits and cost of the data collection.

Patti Phillips: So we always have to go back to what is it we're trying to do? What are we trying to accomplish? And then there's the analysis piece, which is the most fun part, because that's where results come in. The first step in our analysis process is that step to isolate the effects of the program. And that is one of the areas that's been debated for centuries.

Patti Phillips: Should you isolate the impact of training? A lot of times that debate occurs because there's a lack of understanding what we're trying to do. When you go into a So, we're going to talk a little bit about how to solve a problem. To solve a problem with any kind of [00:13:00] intervention, the question is how much of the problem is solved as a result of that intervention.

Patti Phillips: And if you want any credibility in making that claim, you need a technique to help you get there. And that's what we're doing with Isolating the Effects of the Program. So, we'll use control groups sometimes, trendline analysis, mathematical modeling. Sometimes we'll use estimates because sometimes you just go ask.

Patti Phillips: If you can't use the other techniques, that gives us credibility and that gives us good impact story. And then to get to ROI, we convert measures to money, bring in the cost, calculate the ROI, and we always report the intangibles. And in the last phase is data optimization, and that's your storytelling and then using the data to improve the process.

Beth Buchanan: One thing that I, that I was struck with through all of that, um, was this idea of, you know, can you isolate impact or not? I've, I've heard that discussion before and, you know, one person said, you know, it's really, you're a contributor to impact, right? [00:14:00] You're, you're part of the story because when you work with, um, You know, there's so many facets to it. What do you say to that, you know, point of view? 

Patti Phillips: Well, you know, this is an age-old debate. We've been, it's been done for decades, right? It's nothing new. And it's one of those issues that comes around and it goes away and it comes back around. And that's where I, you know, gets to not really understand what we're doing.

Patti Phillips: We all contribute. The question is, how much of the improvement in a measure is due to a specific investment? So if you're of the mentality that training or learning and development are activities, If that's your mentality, it's an activity, then we're going to do our activity, do a descriptive analysis, and say somewhere along the way we may have contributed.

Patti Phillips: When you look at yourself or your programs as a real investment, an intervention for change, and change occurs, we need a way to make the claim, [00:15:00] or we need a way to give us some credibility in our claim that this is how much our program contributed. So it's really about what are you doing? I hear these things.

Patti Phillips: And people say, oh, we don't need to waste the resources on isolating the effects of the program. It's not even, it's not resource intensive. 

Beth Buchanan: Yeah. 

Patti Phillips: So, that debate, don't even understand it, because it's not resource intensive. I look at the thing around the issue around, well, we shouldn't use control groups.

Patti Phillips: Well, you know what, if you're piloting a solution to see if it works, you've kind of done that anyway, right? And we're not, we're not, We're experimenting on people. We're experimenting to see, does this innovative solution deliver the value we need it to so we can justify spending the money on it? So, again, it's, what are you really trying to do here?

Patti Phillips: The other thing is, You know, when you're in the C suite, and we've sat in it, we've been there, [00:16:00] done that on both sides as consultants and being participants at that table, asking the question, how do you know, how do you know your program contributed, and what is the basis for the claim you're making that sales went up or productivity went up as a result of the program?

Patti Phillips: So it is an age old debate. Again, a lot of times it's debated because people don't understand exactly what we're doing with it. It's not necessarily taking anything away from the fact that training is part of everything, because surely it is, and surely we need to develop our people. But on these big investments, That we're making are those investments delivering the impact that they need to, to justify the use of those resources for that investment.

Patti Phillips: That's where we're going for. It's not something you want to do for everything, but we're just really looking at those big investments. Are they delivering the impact? 

Beth Buchanan: Yeah, and that that's, um, [00:17:00] sort of like having you had mentioned before, having a problem worth solving, right? For these, these types of things.

Beth Buchanan: Um, is that what you mean? There is. This type of thing, the big impact. Yeah. 

Patti Phillips: Yeah. Is the problem worth solving and then looking for the solution or combination of solutions to solve the problem and then being able to say, as a result of this investment. Here's how much impact, here's how much improvement we've seen in these measures.

Patti Phillips: And then given the value of that improvement compared to the cost, we actually use our resources efficiently. Because a lot of people will say, well, you know, training's not about ROI. Well, certainly it is, right? Training's not about making money. Well, yeah, it kind of is. I mean, you can't, how do you, Tell the sales people going through training, it's not about making money, but it's not necessarily about sales.

Patti Phillips: It could be about more efficiency. It could be about reducing errors, [00:18:00] which is costly. It could be about reducing turnover, which is really costly. So when we invest in our people, we are helping the organization make money in some respects. Some cases, save money, avoid cost, and do greater good while making money, saving money, and avoiding cost.

Beth Buchanan: Yeah, I like what you said there. It made me think, you know, of just the idea of value, right? I mean, we think of training as solving a job problem, but we also have to tie it to, um, increasing value or finding where value is being lost, right? Where's the waste that we can, you know, save on or, you know, so making that leap.

Beth Buchanan: And that's your bridge to your business solution or your business goals, right? Is having, going from what does the job need to How do we bring value to the organization? 

Patti Phillips: That's right. And we bring value to the organization by bringing value to our people. And that's where the [00:19:00] other measures come in, right?

Patti Phillips: Are they building capability that they are using on the job and that they can use in future jobs? That's value to our people. And are we, you know, is the system supporting our people as they use these skills that we say are so important to them? That is value to our people. By bringing value to our people and helping them develop their capability, you're bringing value to the organization.

Patti Phillips: And we all know, we've all, you know, we've all been around, right? If organizations don't do well, they cannot do good by their people. And that's just fundamental economics. So you're bringing value, we call it a value chain or a chain of impact, but there's value throughout the implementation of a program.

Patti Phillips: But it begins with getting clear on, One, we're implementing the program in the first place and ensuring that the people involved are bought into whatever that program or process is, because now you're increasing the chances they'll actually use them. 

Susan Cort: So, Patty, when you think [00:20:00] about maybe some of the barriers for organizations implementing or looking into the ROI of their training, what are they and how do you dispel those so that more people Embrace the whole concept of looking for the ROI.

Patti Phillips: Yeah, and I love it. Um, so a lot of the barriers are perceived more than real. Um, people will say it's too expensive. Well, I mean, not necessarily. We make it expensive because we wait until it's all over to do it. And now you're having to bring the consultant group in here and pay, you know, 40, to do a study, whatever it is.

Patti Phillips: So we make it a little bit expensive because we don't plan it. Plan for it. We make it a little bit difficult because we don't plan for it. We make it hard on ourselves, hard on our participants. Um, so we hear cost, difficulty, it takes too much time, not if you build it in. You know, the heavy lift should be on the front end, the planning part.

Patti Phillips: And then, of course, the design of the solution is [00:21:00] usually the investment, the big investment. ROI study. One study should cost you no more than 5 to 10 percent of your program cost. If, and if you set it up right, that's what you'll see. It doesn't cost that much. So, we cause ourselves pain, time, expense. A big one that we hear, and this may be number one, is because no one's asking.

Patti Phillips: So, we often wait. We see people wait until they ask. And that, By the time they're asking for it, they've made a decision about that program. So, rather than wait until you're asked, go ahead and start building it in. And maybe, you know, some people in your audience, maybe they're brand new to evaluation, or maybe they're just getting good at level two and they want to move the needle up to that follow up application and behavior change.

Patti Phillips: Do it. You don't have to jump right into it. But if, you know, if we wait, we have to remember they probably are asking for good reason. And [00:22:00] that reason is, they're looking very closely at whatever that program is, and when we wait, sometimes that bridge just breaks out from under us, and that's when we panic and we cost money.

Patti Phillips: So 

Susan Cort: yeah, I would think if, if leadership is asking you at the tail end, what's the ROI They're asking 'cause they're questioning more than seeing the ROI that you've been planning for at the beginning of the project. 

Patti Phillips: That's a great way to put it. Yeah, it's a great way to put it. Yeah. They're not asking, they're questioning.

Patti Phillips: Right. And a lot of times the questions rhetorical. Right, because they've already worked it out. 

Beth Buchanan: So, so what would you say, to you just mentioned people who may be a little bit new to this or who are, who have fallen victim to that, you know, getting to the end of a learning solution or a program and going, oh, wait a minute, forgot about the whole evaluation piece.

Beth Buchanan: And how would you say, One could take baby steps in kind of building the muscle memory to work this into their design, work it into the beginning, be cognizant of this [00:23:00] from the start. 

Patti Phillips: Yeah. Start asking questions up front. Um, don't worry about the evaluation right now. Let's work up front and say, okay, let's get clear on why we're doing this.

Patti Phillips: And there's different ways you can approach it, right? The ideal way is there's a, Problem or opportunity we need to take advantage of. So we look at the business measures. We partner with our clients and talk to them. Say, what's your problem? Well, too much time in meetings. Okay, let's investigate. Why is there too much time in meetings and look at the solutions and then think through what do people need to know to change what they're doing to reduce that time?

Patti Phillips: And how can we roll it out so they buy into it? So, they learn what they need to know to do what we want them to do to reduce time in meetings and get value from there. So, that's ideal, but something simple that people who are just getting started can do. That is, look at that program that you're working with and ask, okay, what exactly are they going to learn?

Patti Phillips: What exactly do they need to [00:24:00] know? And then, if they learn it and they know it, what exactly are they going to do with it? How are they going to use it? What behaviors are going to change? How exactly is this going to work? And if all of that takes place, what's the consequence of it? If they do exactly what we want them to do, the way we want them to do it, what's the consequence?

Patti Phillips: How's it going to impact output, quality, cost, time, customer satisfaction, job satisfaction, work habits, and innovation? How's it going to impact those measures? And how much might those measures and Improve as a result of this. And then if they improve, what's the value? How much money is that gonna help us save?

Patti Phillips: Just thinking through that logic, that is a very easy thing to do. Mm. And it gets them in the mindset of critically thinking through a. What they're doing, how they're doing it, and why they're doing it that way. We have a tool, and if you guys want to share it with them, happy to do it. It's really simple.

Patti Phillips: We call it the Alignment Conversation. It's one of [00:25:00] probably our, the favorite tools that we offer through the certification process, but it demonstrates that logic because it's what we do up front that leads to what we get in the end. So, if they'll just start doing that, and then look at a real easy thing they can do is take a look at those, and of course, Program evaluations they're running.

Patti Phillips: Look at the questions, you know, are they meaningful questions? Are they giving you any real insight around what people might do or should do when they go back to the job? 

Beth Buchanan: Well, that's great advice. I love that of just taking the time in the beginning and kind of walking through that framework of just asking those questions because We, we so often jump to solutioning, right?

Beth Buchanan: We jump to the bandaid and think we have the answer, but we haven't stopped to question and critically think like just, just that framework that you offered, I think is, is great. Before we go, I did want to still talk a little bit about implementation. You know, implementation is a hot topic in L& D [00:26:00], and why is that so important?

Beth Buchanan: You're someone who talks and lives ROI. Why is implementation such a big deal? 

Patti Phillips: Well, I mean, there's two paths, right? There's implementation of the measurement strategy, implementation of the program, and implementation of the program has to occur if you want. To get to impact in ROI. So let's look at implementation of the program.

Patti Phillips: You can run a program, people can learn, but to get to impact in ROI, they have to do. And that's where that implementation and application of knowledge and skills comes in. Because if they don't do it, you're not going to get impact in ROI. It is mathematically impossible because they have to apply it.

Patti Phillips: And so we look at learning. You know, for the long haul, right? Holly Burkett, I'll give Holly a shout out. She has done a lot of research around program implementation, measurement implementation, and what it takes to ensure that, that processes are sustained within the organization. [00:27:00] A big part of learning and development, the programming, and I mentioned earlier, is programming.

Patti Phillips: Really focusing on that transfer of learning, and Mary Broad did some early research years ago on transfer of learning and put it on the map, but if you look at her work and that of many others, there are three roles in the organization that have the most influence on the implementation of a program or process, and there's three time frames in which each of these roles can take action to do it.

Patti Phillips: The number one influence. on successful sustainability and implementation is the manager of that learner. And a lot of times we don't set the manager up for success. We may do prework for the participants. We may give them the objectives. We may have them help us identify the impact measures, but a lot of times we just forgo the conversation with the manager of the learner.

Patti Phillips: And if we don't set that manager up in the beginning [00:28:00] to help the learner. Be successful with the use of the knowledge, skills, information, and insights they're gaining. Then we are hurting ourselves in terms of that implementation, because you can't get from here's what I know to here's the impact that this program has had without the implementation.

Patti Phillips: And we've got to get that manager involved. And that, so it is critical for people to use what they learn. Your, your folks, they're developing dashboards, right? Dashboards are a big thing. All right, we develop a dashboard. Well, if people aren't using the data from the dashboard, you're getting nothing for it, so stop doing it.

Patti Phillips: The same with evaluation. If people aren't using the insights coming from your evaluations, then stop doing them. And that's even with your Level 1. If you've got a bunch of level one questions you're asking and nobody's paying attention to them, then stop. You don't need to do that, right? So, implementation is key throughout.

Patti Phillips: People have to use what they've learned. You know, we've heard [00:29:00] that old adage, knowledge is power. Knowledge is only powerful if you put it to use. And so, whether it's the learning program, whether it's engaging the manager, what they need to know, or whether it's the evaluation, if we're not leveraging the insights we're gaining.

Patti Phillips: From these investments, then we can't get to impact in ROI. 

Susan Cort: Patty, such great information that you're sharing. And I know you're inspiring our listeners to take further action. You mentioned some of the resources that you have, uh, just kind of give an overview of what people will find when they come to your website.

Patti Phillips: Um, yes, we have a lot of resources. We've been doing this for many, many years. And so we've. She's got tons of resources. One is the ROI estimator that we talked about earlier. Love for people to try that. It's wide open on the website. Um, we've got boot camps. We have a portal of free tools that include case studies and job aids and worksheets.

Patti Phillips: And then we also have the Center for Talent Measurement and Reporting, if anyone is interested in that. Um, that [00:30:00] includes a library of metrics that Dave Vance and the team at CTR, Center for Talent Reporting, created a few years ago. We purchased the assets. We are happy to have your audience, give your audience access to that too.

Patti Phillips: And I'm happy to provide all that information to you so you can share with the audience. 

Susan Cort: And that just scratches the surface of the resources that you have to offer. There are many books and many things that people can engage with and ongoing content. So we want to thank you, Patti, for taking the time to talk with us today and our Powered by Learning listeners.

Patti Phillips: Thank you for the opportunity. Thank you, Patti. 

Susan Cort: Beth, Patti had some great points about the importance of ROI in training, including that you should think about ROI before you design your course. 

Beth Buchanan: Yeah, exactly. One of my biggest takeaways here from this podcast was that ROI doesn't have to be costly or complicated as long as you build it into that planning phase.

Beth Buchanan: You know, if you think about it from the beginning, it makes me think of our learning strategy work that we do all the time for [00:31:00] clients where we're developing objectives and aligning with business needs so we address real problems and find real opportunities. 

Susan Cort: Absolutely. Great reminders for our listeners.

Susan Cort: Thanks, Beth. Thanks, Susan. And thanks to our guest, Patti Phillips of ROI Institute. If you have an idea for a topic or guest, please reach out to us at poweredbylearning@dvinci.com. And don't forget that you can subscribe to Powered by Learning wherever you listen to your podcasts.