HR Trailblazers Season 2: Jay Caldwell, Chief Talent Officer at ADP
ADP Chief Talent Officer Jay Caldwell joins Chris Rainey of HR Leaders for season two of Trailblazers. Jay explains what gave ADP the confidence to scale AI coaching across its leaders: a clear tie to the leadership development strategy, privacy and security at the center, and strong feedback from the leaders using Nadia. He also covers how his team measures impact, how an AI coach helps leaders apply what they learn in training, and how the manager role and HR’s own role will change.
Video Transcript
Key Points
Transcript
Chris Rainey (voiceover) [00:00]: Welcome back to season two of Trailblazers. In season one, we met the pioneers taking AI from theory to practice and proving what's possible. Now the conversation has moved on. AI has shifted from experimentation to implementation and transformation. So in season two, we're sitting down with senior HR executives, talent and learning leaders from the world's leading organizations to ask a bigger question: how do you turn AI into meaningful transformation across your organization? We'll explore what happens when AI moves from pilots to scale and starts changing how people work, learn, and lead.
Trailblazers is brought to you in partnership by our friends at Valence, and before we get into the conversation, I'd like to introduce you to Nadia, Valence's AI coach, which brings personalized coaching into the flow of work at scale. Let's take a closer look.
Nadia spot (narrator) [00:54]: An organization is made up of hundreds of teams with millions of connections between them. Right now, work is changing faster than most of those teams and connections can keep up. But AI gives us a new way to support and develop every employee. Meet Nadia, the enterprise AI coach custom-built to amplify the impact of talent programs by delivering expert coaching in the flow of work. By supporting your workforce in the small moments, Nadia increases the impact of the big moments. With Nadia as a front door to connect leadership, learning, performance, and transformation, talent initiatives become more impactful, and leaders have deeper insight into what's happening in their organization.
From pilot to rollout
Chris Rainey [01:46]: Hey, Jay. Welcome to Trailblazers, my friend. How you doing?
Jay Caldwell [01:50]: Hey. I'm well, Chris. How are you?
Chris Rainey [01:51]: I'm good. Listen, before we jump in, tell everyone [?] a little bit about your background and the journey to where we are now, and then we'll jump into some of the topics.
Jay Caldwell [01:59]: Sure. I've been in this field of HR in some form or another for 20-plus years now. I started my career on the consulting side, focused on leadership development, talent management, culture, org development, these types of things, and then I made a transition to internal roles. Thirteen years ago, I found myself here at ADP in our talent organization, and I've stuck since. So they've stuck with me since. I've had various roles in our talent COE as well as an HR business partner leadership role here, and now I've been the chief talent officer for about the last three years.
Chris Rainey [02:32]: Nice. So one of the reasons I was excited to talk to you: a lot of people last year were experimenting with AI, and I know ADP has moved from experimenting with AI coaching to now scaling it across a much broader manager population. [?] So I suppose one question is, firstly, what gave you the confidence to move from pilot to enterprise rollout? Because many of our audience are in that phase right now, where they're just in pilot mode, still trying to figure stuff out. So I'd be interested to hear from you to start there.
Jay Caldwell [03:06]: Yeah. First, where we started is really connecting the whole concept of AI coaching to our leadership development strategy and what we were looking to offer to the organization. So it wasn't really just AI for AI's sake. We were looking at modernizing and evolving how we think about development and growth here at ADP.
And second was safety and security, which is really, really important, which you might not think of initially when it's just coaching, right? We're not talking about core sensitive data, until you think about it as relationship data and how important the dynamics of you and your team are. So making sure that we can offer something like this with security and privacy at the center.
And then third and most important was how fast we saw the potential impact and really the experience. So we spent a lot of time talking to our initial pilot users, which was 1,000 people. It wasn't small. And just consistently, you can see how meaningful something like AI coaching can be and how much of a better fit it can be at really an individualized level, because you can offer learning and development for that moment of need or that topic of need. You might be the only one at this moment with that need, but if you can solve that need in that moment, it's far more meaningful for the individual. So the feedback was incredibly strong, both on experience and the impact it was having for those leaders.
Chris Rainey [04:35]: When you say the pilot, was that manager population? Over what period, roughly?
Jay Caldwell [04:41]: It was a mix. Initially, we had a group of, call it, about 500 leaders that had just recently actually helped us redefine our leadership model. So they were already very much engaging with our...
Chris Rainey [04:55]: Across the entire...
Jay Caldwell [04:56]: Very much. Yeah, across the entire organization. So we had representation from all levels, from all countries, all of our business units. It just gave us a really diverse perspective on the pilot. And it took us about six months for them to use it and us to gather enough data and feedback. We're a data-driven organization, so we wanted to see, beyond just the initial feedback, will people sustain the usage as well? As cool as it sounds, having an AI coach, will people continue to use that? And that sustained adoption is really important given the investment.
Chris Rainey [05:28]: So beyond those metrics, did you have specific outcomes that were tied to the pilot that you were measuring? What did that look like?
Jay Caldwell [05:39]: Absolutely. And listen, it is tough, especially in the leadership development space. How do you measure the outcome of investments in leadership development?
Chris Rainey [05:45]: Sure. B2B sales would be a bit easier, right?
Jay Caldwell [05:48]: Totally. But I look at it like levels of evaluation. So initially, it was really just the feedback. Yes, it's great that they adopt, but we actually did ask for views from those pilot participants about how did the AI coaching help with their confidence as a leader, as an example. How did it help their own self-perceived view of their performance and impact as a leader? And then how much time did they feel it was saving in terms of conversation quality or helping them prepare faster for conversations, things like that. So that was the initial.
And then we're now starting to look at the outcomes as well. So do we see an impact on things like engagement, or feedback we gather through 360 surveys, [?] things like that. It'll take longer to see the impact on things like productivity, turnover. We certainly have a strategy to measure all these things, but we're confident enough with these leading indicators that those will come as well.
One other thing I'd mention, Chris, is what we also saw as we integrated AI coaching into our core leadership development programs is more completion of those programs. So it was actually a more engaging experience as well. So beyond just purely the leader's effectiveness, we also saw even more potential in terms of the effectiveness of the things that we, call it, traditionally offer as it relates to leadership development.
Chris Rainey [07:15]: Was there an intentionality of connecting those? Or did you just find that because people were doing more coaching with the AI, they were just having better outcomes across the...
Jay Caldwell [07:27]: No, it's actually back to our strategy. That was one of the initial things that excited me the most, especially since I've spent 15 years [?] in this function supporting leadership development. So I was excited about the potential impact on, again, the traditional ways of doing things. For example, I'm sure you've talked to many leaders who get frustrated that they offer training, and the learning just kind of goes by the wayside, or the binders sit in drawers and things like that. Things leave your mind pretty quickly. So that was one of our first hypotheses: wow, how much more effective would it be if we actually connected an AI coach to the learning, and the coach could help reinforce what you learned in the classroom and actually apply that to the job?
Chris Rainey [08:11]: What does that look like in practice?
Jay Caldwell [08:14]: So let me give you a simple example. Let's say you join a webinar with 300 other people, right? That's about leadership or giving feedback, whatever concept that might be. Let's call it leading change in the age of AI. You go to a webinar like that. Usually, they're very inspirational. You listen to a great speaker, something like that. You might have a couple good takeaways. But in that case, what if you end that webinar by saying, "Hey, Chris, think about one thing you can take from the webinar here, and go talk to the AI coach about it, and think about how you could put that into action."
Chris Rainey [08:46]: Exactly. Okay.
Jay Caldwell [08:47]: So it's really that learning transfer, from "I took something away that was interesting here" to "now how do I actually apply that?" That's really hard to scale for 300 people that were in that webinar, but dare I say it's very easy with tools like that.
Chris Rainey [09:03]: That's when you're kind of moving from consumption to capability, right? That's the difference.
Jay Caldwell [09:08]: Totally agree. And leadership is all about behaviors, and changing behaviors requires repetition and habits. And now imagine your AI coach follows up with you four weeks after that webinar and says, "Hey, Chris, you said you were going to do this. How's it going? What did you put into action? What do you need to remember to do this next week?" So it's almost like you have all these leadership development professionals supporting your leaders all at once.
Chris Rainey [09:33]: And I think what's exciting now, with the kind of integrations that are coming out right now: if it's connected to Copilot, it has access to your calendar, it has access to the transcripts of your meetings, and now it starts to become embedded in the work itself, not something you have to go to. So it's like, "Oh, you've got a meeting coming up with Chris tomorrow. Do you want to practice?"
Jay Caldwell [09:56]: Totally agreed. We're excited about integrating it into just what we call moments that matter, similar to those. I'll give you one other interesting example: if you work internally, at some point, someone on your team is going to apply for a job elsewhere in the organization. They don't always get that job. And that's a hard moment, right? A direct report on your team didn't get the role. What if in that moment we can reach out to you with the coach and say, "Hey, Chris had his heart set on this role, didn't get it. Let's spend a minute preparing for that next conversation with him to make sure he feels just as motivated as he did before he went through that process."
Chris Rainey [10:34]: Yeah. I think this is a huge opportunity for the entire employee lifecycle, right? All the way from pre-boarding, not even onboarding, to have a coach, an agent that you can talk to all the way through, that can onboard you, that you can ask questions to. You maybe don't want to ask your manager in the first week a question that may seem a bit silly, you know? So having that show up and create a personalized experience, because no manager's going to remember or know all of those things. It's impossible, right? Whereas the agent can know when it's Chris's birthday or his anniversary or that thing that's coming up, and then have that there. So there's so many opportunities.
Jay Caldwell [11:14]: Absolutely. The fascinating thing you said, you might have a silly question to ask your manager in the first week: we actually hear that a lot, even from people that have been working with their manager for a long time. They say, "I'd kind of rather ask the AI first, so then I can bring a formed opinion."
Chris Rainey [11:33]: I think we all do. I ask my agent questions all the time before having conversations with my co-founder. I feel like it's almost like the ultimate psychological safety in that sense, right? To be able to do it.
Jay Caldwell [11:47]: Yeah, totally. It's having a superpower and that teammate at your side that can help you be more effective, for sure.
Talent model and skills
Chris Rainey [11:52]: How has this made you rethink your talent model overall? As these AI models evolve and become more capable, we saw again last year and even this year, companies make the mistake of trying just to throw AI on top of existing processes and ways of working, and now they're unwinding, they're going backwards again to undo all of that work. How are you looking at this now?
Jay Caldwell [12:19]: Yeah. In the area of, let's call it, skill development, I think we get to be really bold in a space like this because there's less, call it, process to integrate with, and it's more, as we've been talking about, getting into these moments. A lot of this is net new. It's blue sky, right? So offering our services in ways that we never have before.
So it's forcing us to think a lot about how do we scale in a different way? That's not an easy thing to do necessarily: how do you go from serving, I don't know, call it a few hundred, or let's call it 10% of your leaders in one year, to 100% at all times? So thinking through, as a talent or development organization, does that change how you might serve those leaders, communicate with those leaders, update them on what's available, or even motivate them to consume these types of things? Because not everyone is motivated to learn. It's great when you have those leaders, and those typically are your best leaders, but how do you motivate those that might be a little bit more passive about their own development? So it's really helping us in our organization think through how do we deliver in new and different ways.
Chris Rainey [13:32]: Something I spoke about years ago: we were talking about the intersection of HR and marketing. Back then I was talking about that, and people were like, "Eh." But I feel like it's more true now than ever.
Jay Caldwell [13:46]: I fully agree. Let's talk about benefits for a moment, right? There's so many great benefits your organization probably offers that people may not know about. It's not because of lack of motivation, it's because we in HR and every other function in the organization are competing for attention of your workforce. So how do you actually break through to make sure they're consuming what actually matters and what's going to help?
Chris Rainey [14:12]: And that's the point. That is one of the exciting things about AI, though. We've always talked about a personalized experience, right? Now we have also, with AI, personalized communication. Even cultural nuance is exciting. Some of these coaches I'm seeing, it knows that you're in Asia, and the way you give feedback and receive feedback and do a one-to-one is very different culturally, right? So the fact that the tool knows that now and can adjust, I've seen some incredible examples of this stuff.
Jay Caldwell [14:43]: I agree. Or it starts to learn that, well, me and my teammates, we work in the US versus the UK. [?] I'm from the Northeast. I have somebody else from California. There's cultural nuances just within that team that the AI can really help you with.
Chris Rainey [14:58]: Yeah. And also how people like to give feedback. Or receive, sorry. I've made the mistake earlier in my career thinking everyone likes to be recognized in the same way, and everyone likes to receive feedback in the same way. And boy, did I get that wrong early on in my career. So that's something that I'm seeing also as a great opportunity.
Jay Caldwell [15:19]: Yeah. So could you imagine, Chris, everyone in your company shares how they like to receive feedback. They offer that to the AI, so then the AI can actually pick that up and feed it to your colleagues and managers. It can really help promote a really effective culture.
Chris Rainey [15:35]: I know this wasn't a question we had planned, but how has this made you rethink the skills that you need within your own team?
Jay Caldwell [15:44]: I don't know that it's extremely different from other functions. I think, call it, product and technical acumen is becoming more and more important. I spent a few years supporting our product and technology organization here at ADP as an HR business partner, and that really shaped a lot of the ways I think now, and I try to bring that to my team. Again, back to that point that we were talking about in terms of marketing, but if we think about product management: do we really understand our users at the end of the day? Are we spending enough time with our users? Do we know what problems we're really trying to solve? When we make an offering, are people using it? And is it having the actual outcome that we aimed for to begin with?
And being able to work with our technical partners to navigate through that, the engineers, because as I mentioned, things like privacy, security, data integration, or even data science that goes into some of these solutions, it is helpful to have some level of understanding of how those things work or how those teams work, because we're collaborating more with these different functions than we ever have before. So those skills are becoming more and more important, maybe even in balance with some of the expertise around things like I/O psychology and analytics. You have to understand these other functions just as much to be able to operate with them.
Chris Rainey [17:06]: Have you found, though, interestingly enough, that with all the talk of technology, really the most important thing now has actually become the power skills?
Jay Caldwell [17:18]: To me, it's always been important.
Chris Rainey [17:20]: I know, but do you know what I mean? It's kind of gone full 360, though.
Jay Caldwell [17:23]: 100%. Now there's no avoiding it. It's always been important, but people could get by with stronger technical skills, right? But now I believe there's no avoiding it anymore. These skills, like being able to collaborate, work with others, your judgment, your creativity, these are going to be the ones that differentiate talent in the future. And I think most companies are going to be looking more and more for those skills or, like I said, putting them at a higher level of priority than the technical skills of the past.
Chris Rainey [17:52]: Yeah. I feel like with AI now, things like critical thinking take on a whole new different meaning, when you think about AI and how people approach using this tool.
Jay Caldwell [18:05]: 100%. And this is part of what excites me about AI in the world of coaching and learning: it's really good at helping develop those skills. Self-awareness, critical thinking, decision-making. So we're already looking at how do we bring a tool like this beyond the leadership population, because they matter for everyone that works with people, which is our entire workforce. So we want to make that coaching available to everyone as a benefit that can help their career.
Chris Rainey [18:35]: When you launched this, what was the message that really landed with people? When you shared this with your leaders and managers, what was something that they really connected to?
Jay Caldwell [18:44]: Individualized, confidential support.
Chris Rainey [18:48]: Okay. The framing.
Jay Caldwell [18:50]: Yeah. And that it's really all about you and helping you get better and developing those human skills. So this is an investment that we're making in our leaders and all of our associates here at ADP, and that's what's most important. Our people are the most important thing, and we want to make sure they have the skills for the future, and tools like this will help.
The manager role and what comes next
Chris Rainey [19:09]: If AI is giving managers more capacity and better support, what would the role look like? I don't want to get too deep into unlocking these hours and what you do with those, but what should the manager role look like in the future, then?
Jay Caldwell [19:31]: In this function, we've talked for so long about how we want our managers to be leaders. And I think tools like this offer the ability to make that transition for more people, faster. [?] So traditional management things like control or preparing reports, tracking performance, scheduling work, routine questions, teaching people the basics, AI is going to help a lot with those types of things and take away some of the more, call it, transactional work that managers do. So now it's going to free up the time for them to really lean in and lead their teams. Really lead their teams. So thinking about how do they actually accelerate the performance of their team, not just manage the performance of their team. And lean into skills like coaching, skill development, building trust, fostering innovation. So all these things we've really been aiming to teach leaders. But I'll tell you, the biggest constraint I always hear is time. "I just don't have the time to think about those things. I'm managing my team."
Chris Rainey [20:31]: Yeah. It's like I talk to Shane, [?] my co-founder, and we're like, "We're so busy in the business, we can't focus on the business," right? And it's like a vicious cycle. We have this challenge because we don't have enough time, so now we're spending more time. And he's like, "Where's the time [?] to step away and fix that issue? Because I've got to deliver on this," right? That's kind of what we're putting our managers through. "Hey, you need to do all these things." They're like, "Yeah, cool, but I've got to deliver on my day-to-day, and now you're asking me to do all this other stuff." And you're like, "Well, no, that's part of your role." But we all know that feeling.
Jay Caldwell [21:08]: And the job's only getting harder. Things aren't slowing down. So if we can help managers in that moment where they have a need, just elevate a little above the work, and think higher order about how they're leading their team and the skills required there, that's super exciting.
Chris Rainey [21:25]: Looking ahead, what are talent leaders still, in your opinion, underestimating about AI's impact on work and the workforce itself?
Jay Caldwell [21:36]: I'd say two things. One, we might be underestimating the impact on our own work and our own role. HR often has the shoemaker's children [?] syndrome: we're always thinking about everybody else before we think about ourselves. So I think really the extent our organizations, our work, and as I said, our own job will evolve is very significant. And examples like thinking about how do we scale talent management work more than we have in the past, because historically it was all about being the expert. When the expert's in everyone's pocket, then what's the value add? It's the influence and the relationships and understanding the environment and context. So our roles will change very significantly.
And I think the other, apropos of this conversation, [?] is that certainly work's going to change for all employees. Work's going to evolve. But we might be underestimating the impact of how much that leadership job is going to change, and how the expectations are going to rise even further, and the requirements of what that role is really going to require. Both to help us execute in the world of AI, [?] but then also how to operate in that world. So, yes, we need to rethink redesigning work and how we help evolve the tasks that people really perform day in and day out. But we need to up-level our leaders to help guide that change. Because I think at the end of the day, your leaders will drive success of AI or not, more than anything else.
Chris Rainey [23:17]: Oh, 100%. I think you hit the nail on the head right there. [?] We've seen it time and time again, right? Before I let you go, as you look ahead, what are you most excited about?
Jay Caldwell [23:30]: I am most excited about the impact that HR is going to have in this new world. It's such a critical function, thinking about how we're evolving our organizations, how we're moving the organizations forward. And we actually didn't talk about all the change management that comes along, and the leadership [?] that comes along with all these things.
Chris Rainey [23:47]: We'll have a series on this. Yeah, let's not skirt over that part. For everyone listening, the technology is probably 5%. As we know, it's the culture shift, the mindset shift. We should probably mention that part.
Jay Caldwell [24:01]: That's right. Totally. It's so interesting to me, after decades of hearing, "How do we get HR at the table? How do we get a seat at the table?" The seat's there. So that's really exciting: the opportunity for all of our HR colleagues out there to step into that role and really lead within their organization.
Chris Rainey [24:20]: Yeah. Exciting times ahead. Jay, before I let you go, where can people connect with you, reach out, say hi? Where can they find you?
Jay Caldwell [24:28]: LinkedIn is probably the best place. That or skywriting, depending on where we're traveling at any given time. But totally LinkedIn. I pay attention to that. Of all media platforms, that's the one I pay attention to most.
Chris Rainey [24:40]: Amazing. Well, listen, congrats to you and the team on the journey so far. It is a journey, right? Not a destination. But I appreciate you taking the time out to join us, and wish you all the best until next week. [?]
Jay Caldwell [24:53]: Happy to do so. Thank you, Chris.
Chris Rainey [24:55]: Thanks.
Chris Rainey (voiceover) [24:56]: AI is moving fast, and for HR leaders, the challenge now is turning that momentum into meaningful change across the organization. To learn how the world's leading organizations like Kraft Heinz, Coca-Cola, Schneider Electric, HP, Delta Air Lines, and General Mills are reinventing talent strategy with Nadia and putting AI-powered transformation into practice, click the link in the description below. See you next episode.

.png)