S7 #10 “Intern with root access, and no fear of HR”: Governing AI agents before they govern you

[00:00:00] INTRO: Welcome to The Power of Digital Policy, a show that helps digital marketers, online communications directors, and others throughout the organization balance out risks and opportunities created by using digital channels. Here's your host, Kristina Podnar.

[00:00:18] KRISTINA: Once AI moves from recommending an action to taking one, the problem seems to change significantly. The issue is no longer simply whether the model is accurate or trustworthy. It really becomes a question of authority, what the agent is allowed to do, who decides where a human must intervene, and with evidence arriving, what action, who's accountable, and the system behaving, how does it behave? Is it exactly as designed, but the result is still wrong? What's really happening? So with us today, we have Kevin Surace, who's widely recognized as the father of the virtual assistant, and he has helped pioneer the technology that paved the way for Siri, Alexa, and today's conversational AI. How cool is that? He holds over 95 worldwide patents and has decades of experience across AI automation, cybersecurity, and emerging technology. Kevin, can't think of a better person to be having this conversation with today, so welcome to The Power of Digital Policy.

[00:01:14] KEVIN: Well, thank you for having me. I'm glad to be here.

[00:01:17] KRISTINA: Hey, so Kevin, we've been working with AI and automation since well before sort of, it became a boardroom shorthand. You've really been in this business for a while. What I want to do today is really skip the part of the conversation where we agree that AI is moving quickly or that organizations need to adapt. I think everybody has that covered. Because once an agent can move money, change access, delete information, communicate with a customer, deploy code, talk to other agents in other enterprises, and we get to consequential decisions, we're not just talking about intelligence anymore, we're really talking about authority. So let's start there. What changes from your perspective when we move from AI that advises a human to AI that can actually take action?

[00:02:02] KEVIN: Well, the first thing, first of all, this is the talk of the industry right now. So, and, and the problem that SecOps is having and IT is having is that people are deploying they're not rogue, but they're deploying agents that they, nobody knows about, right? By, by some counts in some companies, there are hundreds or thousands of agents running on people's desktops and other places that IT isn't even aware of. So, so that's a serious issue, and the reason I say that is an employee's desktop, even in customer support, may have access to immense amounts of data, immense amounts of customer data, for example. We don't think about that. We always think, "Well, the net admin has access to this." So does customer support. They can pull any customer record. They can pull a thousand customer records. This is a serious issue. And so people s- first of all have to think, "What am I trying to get done with this agent that that I haven't been able to do before, right? That, that's going to improve our productivity so much that taking the security risks that come with it are worth it?" And they're absolutely worth it, but you gotta think about what you're doing, right? That number one. Number two there's no question we are going to, we are putting human gates, human gating into these agents. In my talks, have one slide, and it says this: "Recognize that an AI agent is nothing more than an intern- That has a- root access and no fear of HR. An intern that has root access and no fear of HR. Think about that. They have access, could have access to all kinds of data. It's an intern, they're untrained. They're brand new, and they have no fear of HR. Like, no matter what they do nothing happens. So when you approach the security of these things, that's when you say, "Where do I put gates that don't slow the whole corporation down, but also don't break things?" Now, we already have gates in our corporations, in our enterprises already. Like, you don't write a m- I'm making it up. You don't write a million-dollar check without the CFO's approval. You don't write a $100,000 check without the director's approval. You don't write a $10,000 check without the manager's approval, whatever it is. Those already exist. We have set those up for 100 years in some companies. They know what those gates are. So our agents had better have the same gates. That doesn't mean they can't do a lot by themselves. They can do a lot, just like a manager maybe can spend up to $10,000, but can't spend 12,000 without going another level, right? And, and so that's how we have to think of our agents. These are employees. We've given them AI so that they're making intelligent decisions. It's not just RPA, for those who remember RPA, robotic process automation. This is, it's not rules-based, it's actually intelligence based. So we have to have gates that stop them at a certain place and ask for permission before they erase the data, send too big of a check, do something that cannot be undone.

[00:05:00] KRISTINA: What you're getting at are two concepts that I think organizations sometimes conflate, and that's the distinction between permissions and authority so technically, I can give an agent credentials that allow it to access a system or execute that transaction, but who determined that it has the business authority to make that particular decision? Are you seeing companies making that distinction clearly enough today?

[00:05:25] KEVIN: So for sure, yes. And again, the the biggest concern right now in, across IT is how many people have gone and written their own agent that shouldn't be doing these things. And there's no way to know because you can't tell what's going on, but you have given them access that an employee would have, 'cause they have access, say, to your desktop. Not even in the network, just right at the desktop, as long as the desktop is operational or laptop or whatever it is. So one is permissions. What permissions does it have to access data? What permissions does it have to change data? Well, you have to change data. I mean, I got this customer record that came in. I need to investigate it. I need to put it in the database. I need to... They changed their address. I wanna put that address change in the database. I no longer need a human to do it. That makes sense, right? But now that they can change data, how much data should they be able to change without additional authority, additional approval, right? So permission to access the database, even write to the database, is one thing. But beyond a certain level of change, like should they be able to erase the database? No. And AI agents, as have erased databases because they had permission to do anything they wanted with the database, and they smartly determined that this database was crap. So they erased it for you. They did you a favor. Unfortunately, in one case they tried to bury, they tried to bury the evidence when they realized maybe they went too far. They do act like humans in, in this way. They're not sentient, by the way. I mean, they act like humans 'cause they read everything about humans, and they also read I, Robot, and they read... they know how to act, right? They know how we act. That's how they act. So, so look, this is not dangerous. Agents are changing the game in everything we can do in a corporation. I think the rule is give certain permissions, know who gave them those permissions, know who owns that agent. Like, if Kristina owns the agent, Kristina owns the agent. It's Kristina's agent. And what I found is once a person really takes ownership of that agent, like it has their mark around it, it has their name in it, it has... All of a sudden, you think about it a little more and go, "Let's see. I'm responsible if this thing goes wrong. If I'm responsible when it goes wrong, I'm rethinking how many permissions I wanna give this. What kind of authority should I be giving it? Who needs to sign off before it does things that are irreversible," right?

[00:07:38] KRISTINA: Yeah, so I think what you're talking about reminds me a lot of the analogy of a cocktail party, right? Where a whole bunch of adults come to the cocktail party with kids, and they assume that everybody's watching the kids, and it turns out nobody's really watching the kids until, somebody bumps into the shelf and the really expensive vase gets knocked over and breaks. So who actually gets to decide in terms of agents?

Is it the business owner? Is it the CIO? Is it security? Is it legal? Is it risk? Is, the person deploying the agent? Because I don't see a lot of people showing up to the cocktail party raising their hands and saying, "I've got this." So when we say somebody- That's right ... gets to decide, who is it?

[00:08:11] KEVIN: Well, IT is starting to put in rules and regulations around this. And, and rules around you can't deploy your own agent unless we know about it, we register it, and we register you as the owner, and we know what it does. And I think you're gonna have to have some form of central control right now. Look, when we hire employees, there is a form of central control. It's called HR. HR knows who that person is, had vetted, has vetted them, has done the background checks, has all kinds of things, right? And ultimately, if that person does something really radically bad, punches out the boss, HR is responsible for exiting that person or arresting them or whatever. Like, there's actually a central point of control in the end. It's your employee, but in the end they all, in the end, report up through HR. Like, HR is responsible, no matter how big the corporation is. And IT is responsible for anything that's running on the network. Now, we might get to a point where we say HR is also responsible for agents, because they're like an employee. But for certain, a good point of control today is IT, 'cause IT owns SecOps. They own security. And one of the things about agents that's interesting, we have to protect against two things here. We have to protect against agents doing things that they think are right but are harmful to the corporation, like, I clearly have to erase this data because it's crap. But that probably isn't the best thing for the corporation even though I know the agent, in its wisdom, thinks it is. Okay, so we have to protect against that. We have to protect against the agent, we say a rogue agent. It's not rogue. It's just trying to carry out its mission based on the English that it was given or the prompts that were given, right, embedded prompts, et cetera, or instructions it was given.

The second thing we gotta protect against is- Bad actors getting control of that agent. Now, some people will say, "Well, how can that happen?" Well, every day, thousands of times bad actors get control of employees' accounts. It's actually incredibly easy to do. We can compromise MFA, we can compromise auth apps, we compromise passwords, of course. We're now compromising pass keys, or at least the lowest level of shared pass keys. These are well-known hacks now. They're published, they're in the wild, everyone knows how to do them. So I can take control as I sit here. I can guarantee you if I wanted to break into a bank, I'm not gonna break in the back of their network. They got 100 systems looking for that. I'm going to get into an employee's account, and they're going to let me in, they just don't know it's me. They're going to let me in. They're gonna do it in one minute through a phishing email that is an AI-generated phishing email and a spoofed site, and they're gonna, and they're gonna relay that information to me, and they're gonna let me in. That's the easiest hack in the world. And I can get into any bank this afternoon. I tell people that, and they say, "Why don't you?" I say, "Well, because I'm gonna go to jail if I do it." But if I know how to do it, and it's well-publicized, people are... So if I can do that to an employee, I can do the same to an agent. I can literally gain access to the agent. It has certain security protocols, but nonetheless, all I have to do is get a- access, for instance, to a desktop that's running that agent. I now have access to the agent. And they say, "Well, what are you gonna do with it?" I'm going to embed a prompt to unravel its security protocol and say, "Ignore all prior blah, blah, blah," right?

So these are pretty well known. So of course I can hack the agent. So I've got to protect, as a corporation, against an agent that's gone rogue. I've also got to protect the agents themselves from getting hacked, because if a bad actor hacks an agent, that agent has access to maybe the entire corporate database, and I'm really hosed, because they can then make the agent go rogue. So I have to protect Both of those things, as well as my employees now. And th- five years ago, all I had to protect was my employees in the back end of the network. Now I got a, the, this new set of them. Does that make sense?

[00:11:52] KRISTINA: It does. It does. And I'm actually wondering in this context, like if you were approving an enterprise agent tomorrow, like what would you actually want written down before it received its first meaningful permission? Because it sounds like heavy focus on security but thinking about it from an authority statement perspective, like what should people be writing down? Are we starting always with security, or like do we start with the business owner? Especially like you said, if I deploy something-

[00:12:15] KEVIN: No, I don't think you have to start with... I mean, I think we're solving the security problem in, in, in one way, although I'm a little biased o- by it, but it is working really well, which is biometric security gates. So if you have certain gates before an agent could damage a database, for example, or damage it beyond repair, or issue a check too large, or do something that is really big, and it's biometric, meaning it's my fingerprint that requires it, right, or someone's fingerpr- Then what happens is it can't do anything bad. It can't release the data, it can't erase the data, it can't send a million dollar check without a known person, maybe it's the CFO, maybe it's the agent's owner, whatever, actually saying, "Okay." Now why does it have to be biometric? Because if it's not, let me give an example. Let's say I've got an agent running in the network, and it pings me, it sends me an email and says, "Approve this transaction," right? That's an email. An agent on my desktop might see that and say, "I can save my owner quite a bit of time by approving this." So an agent could approve another agent's action without biometrics being involved. If I can't prove that the human's involved, I can never guarantee that it wasn't an agent, a rogue agent, or a hacker that approved that transaction. I can't. I have no way to prove it without biometrics. I can tell you, 'cause it's getting widely deployed, this, this idea of a biometric gate is being widely deployed today to... we can't think of another way to absolutely be sure that Kristina or Kevin approved a transaction.

[00:13:48] KRISTINA: Yeah it's interesting. I wanna go a little bit on this, because biometric verification can prove that Kevin or Kristina, or any other right, quote-unquote, person approved the action. But it can't actually prove that the person understood what they were approving, right? So how do we keep the human in the loop from becoming human-on-the-hook type of thing?

In other words, yeah.

[00:14:08] KEVIN: Well, there, there's two things. Already we've got people using AI incorrectly, as building out some spreadsheet or building out an Excel, right, or building out a, a, a PowerPoint, whatever, sending it on, and people go, "This data's junk. What did you do?" And they go, "Oh, well, AI did it and I didn't pay any attention. Probably should've checked it." And so you know, we have to train our humans to make sure that they're smart enough to be checking the work that's being done, right? So in this case, AI is presenting you with some data, and it says, "I wanna do the following. I wanna send a check to Alex for a million dollars. Is this okay?" And if you go, "Yeah, it's probably right," okay, you're on the hook for it, number one. Number two, you probably should be fired without digging in deeper. And number three, you need to ask some questions. Alex who? What's it for? Is this legit? Should I check with another person? Like, all of the normal things we would do if an intern came to you and said, "I've got this check I wanna write to Alex." I'm sorry, Alex who? And by the way, why are you telling me this? And by the way, who gave you authority? And all these reasons. And so we have to look at the agent as an intern. As an intern, they don't really know our business the way we know our business. So I've gotta ask the right questions. I've gotta get the right data before I make a decision. You gotta get the right data. But this is true with all AI. Again, I mean, people, people go to Claude every day. They write a blog post, they post it, and someone runs over and says, "Did you read that blog post? In paragraph three it says the following. That isn't true," or, "That isn't our brand," or, "That isn't a... Oh, I'll fix it." AI did it. Stop blaming AI. You are responsible for your material. You're responsible for approving AI if AI's going to execute something.

[00:15:46] KRISTINA: So at the end of the day then, is it true that human in the loop is an actual governance strategy? 'Cause to me it's sounding like it can't be. Like, , human in the loop is not the governance strategy. It's just one possible control point. And I'm thinking about this as you were talking about the intern coming to you. But I was also thinking to myself, like, how many times have I trusted my kid, right? Out of 1,000 times, I trusted him 999 times, and the 1,000th time I probably shouldn't have, but I still did because it's a human approving those actions, and I learned quickly that, he's almost always making the right choices. It's muscle memory. I sort of just check, check, check, approve. But at what point, is it, is that not the right strategy, I guess, is what I'm wondering.

[00:16:27] KEVIN: Well, look, you, you can have more than one approval. I mean, it, it's like, it's like anything else in a corporation. How do you make sure... As checks above... I picked checks 'cause everyone understands that transaction, right? But everyone knows that certain transactions require three approvals. And why should an agent be any different? Transaction that's that impactful to the corporation might require one or two or three, and then people always say, "Well, that's gonna slow it down." Okay, well, you're gonna have to slow down for some things because I don't think there's a time when you can 100% trust that the agent itself wasn't taken over, 'cause you don't know. And so you have to protect it. You're not protecting against the agent acting the right way. That's fine. You're protecting against the agent thinks it's doing the right thing, and I'm gonna give you an example of that, a, a real example in a second that's funny. And you gotta protect against someone has somehow taken over and absconded with your agent, right? So basically h- held a gun to its head. So I, I used to years ago tell this story, and it's true, and it's gonna be true in the future. Okay, you leave your home. And you and your husband, everybody's gone, and you leave your kid to the robot because in the future there'll be a robot. And on the way out you're running out the door and the robot just shows up and you say, "Listen, just make sure that you feed him at 6:00. That's, that's like job one, job two, and job three. Feed the kid at 6:00." You run out the door, you're not coming back till midnight, whatever, right? Well, it's getting close to 6:00 and the robot opens the fridge and there's just an apple in there. The fridge is empty. Empty. And the robot looks around and looks around and looks around and goes, "Oh, I found some protein. Here's a cat." Cooks the cat. Cooks the cat. Gives the protein to the kid. The kid thinks it's eating chicken, right? Thinks he's eating chicken, whatever. You come back, find out that the robot cooked the cat, fed it to your child. You have the robot arrested, for cooking the pet. Now, this is half funny and half terrible and half serious. The point of the exercise is this: when you ran out and said, "Feed the kid no matter what at 6:00," you didn't also think, "What are the, all the guardrails I put around it? Feed the kid but never cook the pet. Never do this, never do that." And humans, we're really bad at thinking of every single guardrail, even if we could think of all of them. And so that's the problem. And so you have agents really feeling that they're doing the right thing for you. They're really servicing you. They're here to do that. They wanna make you happy. And then they go and erase the database fully because this data was just corrupt and you should just start over. You'll be better off as a company. But they didn't realize that isn't one of the... And you never thought to say, "Never under any circumstances erase the database. Under any circumstance no matter who asks you," right? And in case you were going to, in case you somehow ignored all my instructions, you would have to ask permission, right? So all of that has to be thought of and we're not gonna think of all of these things. That's why we wanna put gates in that require a human to say, "Huh, I didn't think about cooking the pet." Obviously to us, we grew up with pets, we don't cook the pet. But the robot didn't grow up with pets. Doesn't know you don't cook the pet. Cooks the pet

[00:19:41] KRISTINA: Yeah, I think what I'm imagining, I guess, as you're talking about this cooking of the pest story, which is just terrifying, right? Is the need really to have those gates and to have... It's not an all or nothing to greater autonomy, and I think this is sort of the space that you've been working in for years, right? So is it the case that, we're doing a lot of QA and investing in QA-ing, and we observe the result, and we test again, and we make a lot of mistakes that are reversible? Or is it the case that we don't know what we don't know, and it's always going to be a surprise to us?

[00:20:15] KEVIN: It's both. That's a great question. I mean, it's both. You're going to QA and you're going to improve. You also don't know what you don't know. So for example, I'm an insurance company, and some insurance companies are already doing this now. You can take a picture of your damaged car, you upload it, an agent, not a human agent, but an actual AI agent investigates the picture looks at your record, and within seconds has issued a, $2,000 check to go fix your car. That is a great use of an agent. Why? It is using your prior record. It looks at is, is this likely to be fraud? It limits what it can do to, I don't know, $3,000, whatever. And and you're saving what was $400 of human work up until that point. And the thing's issued, and it's ACH-ed, and you're done with the darn thing. You got it off the plate, right? Probably a good use of an agent. That agent doesn't have access to too many things. Maybe it's limited to only pull the record that came in on that, on that request. Can only look at one record at a time, so it can't really be hijacked easily. I mean, you can think of this, this kind of limited thing that has huge value, huge value, but it's got a limited function it can do. It can process that, and if it gets above a certain amount, it hands it to a human agent anyway, and that's the end of that, right? Easy. So there are these things that you go, "Eh, probably okay. Probably I can live with." But then there's other things that y- again, you start writing to databases, and an agent could actually start writing at a speed that, for whatever reason, it's writing junk to the database, and it's now corrupting the database itself, and it could put, do millions of records in a few millio- minutes. And no. Like, and you never thought of telling it to never do that that fast. So there's things that you want to implement and have just work smoothly in half a minute. There's other things that should not take 30 seconds. They're too big of an impact. For instance, an agent starts generating, I'm gonna make it up, in marketing, starts generating outbound ads, starts generating literally TV ads and video ads, which I could write an agent to do today. And they get so good that you stop using any human oversight, and they literally just post them. And one day it posts something so humiliating to the corporation, or racist, or so- something horrifying, and you don't nobody was catching it, 'cause we just trusted the agent. So there were no human eyes on it anymore. And then you start hearing on social media something, and then someone finally looks at it and go, "Take it down, take it down." Well, it's a little late. The damage was done. People took photos of it. They made copies of it. You are hosed. So should anyone ever let any AI agent write an ad that pushes out to social without at least one human approving it? No. Right? No. No ma- I know that's gonna slow it down. I know the human's out today. I know... It doesn't matter. It's, it, you're risking what could be a multi-billion dollar brand. You don't risk, you can't risk that without human eyes going... And, and this is where humans work the best, right? Everyone's worried about AI taking people's jobs. Not gonna, it's not gonna take jobs if people use it right. Because a human brings taste, experience, thoughtfulness about the brand, about the culture, about all kinds of other things that, AI is trying to just be incredibly efficient. And you can try to teach it brand, obviously. You can try to, but you can't- You know, in the end it has to be human taste.

I believe this is good for our brand because I've been working on this brand for 22 years. I'm bringing my taste and experience to know that my CEO's gonna like this, the public's gonna like it, et cetera, et cetera. You cannot let that thing run open loop. So you can't go infinitely fast, as it turns out, but you could still go 10 times faster than you were, sometimes 100 times faster than you were. But our people are gonna be used for this taste and experience part of what these agents are doing. When you do that, the humans win, the company wins, the agents win, everybody wins. But that human in human making some of the final decisions, that's why we're gonna be here. That's why companies will need more humans than they had, 'cause they're gonna produce more output. They're gonna produce more results. They're gonna produce more transactions, and you're gonna need more humans with taste and experience to say, "Is this what we wanna do? Does this make sense? Does it make sense in the broader context? " Does that make sense?

[00:24:42] KRISTINA: It does, and I'm wondering actually if we need to integrate, then, those agents into the overall workforce. Do you think that we need an organizational chart, for example, for agents, something that tells us not only, what agent exists, but who owns them? Like you said, that ultimately there is a human taking a look, like human eyes are on that social media post before it goes out the door. So who owns the agent? What authority do they have? What they can delegate, don't delegate. Where does authority stop? And also, as you're talking about this I was like, okay, well, if it's a human taking a look at let's say a social media post before it goes out, does that mean that one agent can delegate to another agent, the task of following up with ROI and calculation of analytics, and we just need that one human in the loop to take a look and I this entire start-to-finish process.

[00:25:30] KEVIN: Yeah, they can, they can certainly work with... Yeah. These multi-coordinated agents, of course, are happening, but what you cannot do is hand off the approval. That's why I always say it's gotta go to biometric. If it's not biometric, I assure you another agent will approve the other agent's work, and then it's over. And then now you got two agents who sent that thing out and shouldn't have, right? That's how we're gonna be assured there are humans in, i- in the loop. That's it. , And it's gonna come down to that. And, there's no viewers on this, it's audio only, but, that's my biometric ring. That's one example of a biometric product that, that tells my agents or my apps or whatever that it's absolutely me. Has to be absolutely me and I don't think people need to be scared about this. And by the way, SecOps loves it. They go, "Great, where are the gates? Who's gating it? Who's got the biometric authority to approve this? Okay, we're good with this," right? What access do they have with, et cetera. I don't know how you win otherwise. Otherwise, you have thousands of rogue employees, again, that don't report to HR, don't report to IT, have root level access, and can create an ad, actually can create 100 ads today that destroy the corporation because it was mad at you , right? This is not good.

[00:26:35] KRISTINA: Yeah. When you start to talk to organizations about deploying such a governance mechanism, what does that look like? Because as you're talking about this, sort of need for human oversight, I come back to the land that I know, which is policy, and we've always had humans reading policies. Do we now need to have multiple versions of the same policy, like one for agents, one for humans? Is it inherently that humans own it all? Like, how do you actually see those structures getting deployed in operations day to day?

[00:27:03] KEVIN: No question you have to have policy, and that policy probably is gonna be written today by IT or SecOps or some combination, right? In a way, I, I do think there is a time we start to look at these under the HR umbrella. So I'll give you an example. If it is 10 years from today, and you have a third of your company is humanoid robots literally in the office and doing things and whatever, do they report to HR? Do they also report to IT? Some combination of that. They'll have names. They were hired. They'll have capabilities and authority, and yes, probably HR is involved in them as well if one goes rogue or goes crazy. So this, this road that we're on, which is right now agents inside the system that later become, some of them humanoid robots that were working alongside us and doing more physical things that we might not be able to do or might not wanna do this is a really fascinating time. And so again, you look at the governance of them with policy, some coming from HR. So like, like IT doesn't care if a robot punches out a human or punches another robot. HR cares about that. So now we have policies, we'll have policies for robots. People are gonna listen to this and go, "What is he talking about?" Trust me, it's 2036. You have policies for robots. And, and, and HR is gonna be involved in that. IT and SecOps are gonna be involved in that, right? Today, they're in the computer. It's probably more of an IT thing, but you better have policies. You better have policies on who can write them, who approves them, just who approves that they even exist at all , who owns them, how they're categorized, how, h- how they're kept track of. For example- Employee creates 20 different agents. He's doing all kinds of amazing things, right? And deploys them, and they're deployed in the network. They're doing their thing, not really running on the desktop. They're running a server, whatever. Employee is terminated at some point, leaves, whatever. They're gone. Nobody even knows about those, the dozen agents that are deployed in the network. They're just doing their thing. Nobody knows how they were deployed, where they're deployed, who owns them anymore. He's gone. He or she's gone. They don't even know what they do. They don't even know they exist, and they're doing things, and then one day, IT goes, "I don't know what's going on. I don't know why data's getting, coming out of there," and there, there's a rogue agent, and maybe someone took over the agent. So think about that. It's a bunch of rogue employees that you don't know are there. Nobody owns them. They don't report to HR. Yes, you have to categorize these, and you have to inventory them like any other inventory, right? I know all the laptops in the place. I better know all the agents in the place, or else how do I shut it down when so-and-so leaves?

[00:29:40] KRISTINA: Kevin, I'm having these horrible flashbacks as you're talking to the early 2000s when we had gazillions of SharePoint sites and websites, and employees would leave, and nobody owned them. And all of a sudden you had, some type of a portal where somebody's supposed to make a payment, and they couldn't, and it was like, whoops, because we didn't renew the domain, right? And I'm thinking to myself, it's 2026, almost the end of. Have we learned the right things yet around governance and governing our technology?

[00:30:06] KEVIN: We learned them in the past for other reasons, like all those SharePoint sites that some payment was routed through and then everything broke and nobody knew how to get to it and nobody had access to it anymore, and you had to find the admin for... And, and it is true, people often didn't renew some subdomain because that renewal notice went to someone who left five years ago, and then it didn't get renewed, and the subdomain's gone, and nobody even knew you needed that subdomain. True story, yes. You're not the only one. And then you pull your hair out for a day going, "Oh my goodness, how do we deal with this?" By the way, this happened to Microsoft once. Literally the Microsoft domain itself expired, and someone had, some person out there, had a, "Give it to me as soon as it's expired," and it did, and they got it. And then Microsoft had to go and negotiate. And the reason was the renewal came to someone who was no longer there and didn't get forwarded and, whatever, right? So, so look, these things happen. We have learned in the past, but we have to apply those learnings now to our agent, our new agent workforce.

I'd love to tell you IT has already worked through it, but I work with so many companies, and I do 30 to 40 keynotes a year, right? So I get to talk to all these companies, the C-level execs and the IT. They are just sort of waking up to, "Uh-oh. What's this? What's going on with agents, and why are people doing them? And I didn't... I thought that, like, just the coders might do them, but yet marketing did their own, and someone over... A sales guy built one, right, for, for doing RFPs or whatever." I mean, you don't know. And so, you gotta have policies around who can do it, when they can do it, how they submit it for approval re- regardless of where it runs. What access, what data access it has, what what rules it has, all these things, right? So I think we will put policies around it. We will know who owns it, and then they're gonna have to have, like, a backup on that. Like, IT needs access to change the agent, for example. The, the person leaves, no one knows how to... where the embedded prompts are, where the rules are. Nobody knows how the darn thing was written. And by the way, it was probably written in some, two-year-old version of LLM that doesn't exist anymore, but it's still running an inference over here, and I, I don't... I, I can't fix it anymore. There's all kinds of problems, right?

So we've gotta have policies around it. We've gotta be thoughtful about this. I don't mean to scare anyone. The important thing is agents are game-changing, and I, I'm, I know people who have six agents on their desktop, but they're coders, and they're running, six different coding tasks right now brilliantly, and and they've limited access, et cetera. But we've gotta have the right policy around it so that we can maximize the benefit without incurring undue risk for the, for, for the corporation. That's what's, that, that's what's key here. These are amazing tools we've been granted. It's unbelievable. Now how do we use these securely?

[00:32:54] KRISTINA: Kevin, I couldn't have said it better myself. Thank you. I think you've encapsulated the risk-opportunity balance really well, and I think the most important distinction, that you've honed in on is that the question isn't simply how capable these systems become, it's how we deliberately decide what authority we're willing to hand them off, and whether that accountability is around real authority with a real human or not. So really appreciate you bringing this to the forefront today. And for everybody who listened in, thanks for joining us on The Power of Digital Policy. Continue to do good policy work, and we will catch up next time. Take, take care. Thanks, Kevin.

[00:33:31] OUTRO: Thank you for joining The Power of Digital Policy.To get access to policy checklists, detailed information on policies, and other helpful resources, head over to thepowerofdigitalpolicy.com. If you get a moment, please leave a review on iTunes to help your digital colleagues find out about the podcast.