How to Choose Tools that Survive the Long Haul

Summary

Agentic AI is a powerful security tool, but it won’t replace SOC analysts any time soon. It needs a human in the loop, ongoing maintenance, and proof that it works. In this Containment Island session, Chris Steffen, VP of Research at EMA, explains how to choose security tools that last: match tools to your organization’s maturity, demand explainable systems and quantifiable results instead of roadmap promises, plan for constant care and feeding rather than set-and-forget, and review your security architecture more than once a year so you keep pace with AI innovation while staying aligned with business goals.

Key takeaways

• Agentic AI is a tool, not a replacement for people. Keep a human in the loop until you’ve built confidence in what the tool does and doesn’t do.
• Choose tools that fit your organization’s size and security maturity, and invest in training staff on the tools you already own. Steffen’s research found about 5.63 tools for every solution area in a typical enterprise, many of them underused.
• Don’t buy on aspirational roadmaps. Ask vendors to show you, with verifiable demonstrations and quantifiable metrics, such as cutting false positives from 35% to 5%.
• Every autonomous tool should be explainable. If a vendor can’t explain how it’s trained, how it makes decisions, and how it handles data, pick a different solution.
• Agentic tools aren’t fire-and-forget. They need ongoing training, maintenance, monitoring, and securing.
• Review your security architecture at least twice a year, and tie every new tool to a business goal, such as compliance, supply chain risk, or zero trust.

This webinar was recorded in 2025.

Evin Safdia: Thanks for joining us, and welcome to Containment Island. My name is Evin Safdia, and with me is Chris Steffen. Chris, can you tell us a little, or a lot, about yourself? We’re looking forward to this discussion.

Chris Steffen, VP of Research, EMA: Evin, thanks for having me. I’m Chris Steffen, VP of Research with EMA. I cover information security, compliance, and whatever else you can imagine under the sun. I’m really excited to be here. I’ve worked with Zero Networks a ton, and I’m happy to be talking about agentic AI and automation in security. It’s a topic I’ve been doing quite a bit of research on, and I think we’re going to have a great conversation today.

What do people misunderstand about agentic AI in security?

Evin Safdia: Let’s dive right in. You’ve been doing a lot of research on this, and it’s a very popular topic lately, one I think we’ll see more of as time goes on. With that comes some misunderstanding. What are the biggest misunderstandings people have about agentic AI and its role in security?

Chris Steffen: The number one misunderstanding, the one you hear at every conference, is that people believe agentic solutions are going to replace the SOC analyst. Nothing could be further from the truth. You need a human in the loop, which I think is the current term du jour, to confirm what the tool is doing and gain a certain amount of confidence in its ability to do its job. We had some of the same conversations about virtualization and containerization, when people worried they would virtualize themselves out of a job. It seems that whenever the next trend comes along, one of the concerns is that we won’t have jobs anymore, and every time, that has proven not to be true.

Chris Steffen: So when you think about agentic AI, it’s really important to understand its limitations. It is a tool. It is not a replacement. It comes with risk, especially without some kind of human-in-the-loop intervention. I don’t want people to freak out that the Terminators are coming to take over. That’s unwarranted and not realistic. At the same time, it’s a very, very powerful tool. I’m not saying you shouldn’t be looking at it. What I am trying to get across is: understand the limits of agentic tools.

Chris Steffen: I say that about today. Things are evolving, and tomorrow everything could be automated and agentic, and so much the better. I hope so. But I don’t think we’re close. I don’t think it’s next year. In fact, I don’t think it’s in the next two, three, or four years. There’s quite a bit of time before we reach the level of comfort where agentic solutions can truly act independently and autonomously to augment human staffing. We can use it as a tool, and that tool will continue to improve, but we’re nowhere near the point where I buy an agentic solution, it replaces my tier-one SOC analyst, and I live happily ever after. That’s pretty unrealistic.

How can security teams do more with less?

Evin Safdia: That’s interesting, because we hear a lot of talk about AI lately, and some companies use AI as an excuse to downsize their workforce. So it’s interesting to hear the perspective that it’s meant to enhance our defenders’ capabilities rather than replace them altogether. That’s really about doing more with less. Beyond the misunderstandings about replacing people and relying on these tools to make autonomous decisions, what are people getting right when it comes to the do-more-with-less approach in security?

Chris Steffen: People need to look at their tools and have a realistic view of what those tools will accomplish, given their environment and their organization’s maturity. Let’s throw some numbers out there. Depending on who you believe, there are somewhere between three and four million unfilled cybersecurity jobs. I think there is undoubtedly a talent shortage. If everybody had all the cybersecurity talent they wanted, that would be a glorious thing, but I don’t think there are nearly enough qualified people. Plenty of other people will say there’s no such thing as a talent shortage. Whatever the case, there’s a lot of opportunity out there for security-minded people to take advantage of the market as it is.

Chris Steffen: But organizations need to be very clear about their goals. Hiring a highly credentialed person for an entire IT team of five people may not make sense. Maybe you don’t have that maturity level. Maybe you’re at cybersecurity maturity level one with aspirations to reach level two, and the person you’re trying to hire is firmly at level four or higher. The culture isn’t going to match. The budget isn’t going to match. The infrastructure isn’t going to match.

Chris Steffen: That doesn’t mean you shouldn’t be aspirational, or that you shouldn’t use the tools you’ve already invested in, understand how automation will affect those tools, and do more with less. You want to understand the infrastructure you have, how best to manage it, and how you, as a leader or a practitioner, can help your organization move to the next level. It’s not always easy. It means focusing on the tools that do the most for your organization, investing in understanding how they work and how they affect your infrastructure, and investing in training your staff to take full advantage of them.

Chris Steffen: There’s a stat I found a few years ago in my research: there are about 5.63 tools for every possible solution area in an enterprise environment. That means you’re probably not really using four of them, or you’re not using any of the five completely effectively. Those are the kinds of conversations you need to be having with your executive team. And whenever possible, automating your simple processes is definitely the way to go. Every organization will gain maturity, and, quite bluntly, ROI, by automating some of those things. If you don’t automate some of your security apparatus, it will be much harder to mature and much harder to save money.

What skills do security professionals need as agentic AI grows?

Evin Safdia: That makes perfect sense. I want to ask the same question from an individual’s point of view. We mentioned the talent shortage and the state of hiring in cybersecurity. For someone looking to break into or grow in cybersecurity, how important is it to develop familiarity with these new agentic solutions?

Chris Steffen: I think it’s perfect. One of the great things about cybersecurity professionals in general, and certified people specifically, is that they’re experts, and a cybersecurity expert is not an expert at one thing. They’re almost universally experts at a lot of things. I’ll pick on the CISSP for a moment. ISC2 has a great curriculum there. There are something like 12 different areas you have to understand, and you can’t really be deficient in any of them. It includes networking, basic administration, legal, physical security, and more.

Chris Steffen: I don’t need you to be able to do Diffie-Hellman on the back of a napkin. But I need you to be aware enough to know whether Diffie-Hellman is something my organization needs, and that I need something that will do Diffie-Hellman for me. Maybe it’s understanding the nuances of microsegmentation and networking. I know you and I have talked about what Zero Networks does, and I think that’s wonderful, but not everybody is an expert at microsegmentation. This is where “phone a friend” really comes into play.

Chris Steffen: So become an expert at one, two, three, or five things, but understand that your team will depend on you being knowledgeable about all of them. You’ll find that those of us with gray beards like mine, who have been around the block forever, have a lot of letters behind our names. I don’t want to exaggerate, but I’m at over a dozen technical certifications, including some that don’t even exist anymore, because, again, I’m old. So keep that in mind. Be well-rounded. Be an expert at all things IT, but understand that people are depending on you to apply that knowledge to be the best security professional you can be.

Evin Safdia: That’s amazing advice, and it reminds me of my own career in IT. I have a lot of certifications that have probably expired and probably aren’t relevant anymore. There’s never been a situation where I could specialize in one thing and only do that. There were always related areas, like Active Directory and other basic Windows work.

Chris Steffen: I never have either. I have all the Microsoft certs and all the Cisco certs, a Check Point cert or two, and the great big Novell cert from the 3.1x days, if you really want to date me. Most of those have expired. My MCSE is from the NT 3.51 days. How relevant is that today?

Evin Safdia: You’re super dating yourself. That predates me a bit.

Chris Steffen: That’s what I’m saying. I’m not a spring chicken. But I’m proud of those certifications. They helped me understand what I wanted to do, and they helped me get my CISSP. To this day, I maintain three or four of them. One is the CISSP. From a compliance perspective, I maintain my CISA. And I helped the Cloud Security Alliance create the CCZT, the Certificate of Competence in Zero Trust, which I highly recommend, along with a couple of others. I really appreciate certifications. They’re a great way to get an understanding of the environment. But in the security realm, understand that you’re expected to be an expert at a lot of things, not just one thing.

Evin Safdia: I love that. Some of those certifications are definitely on my list, if I ever find an extra hour in the day.

Chris Steffen: Got it. I’m happy to help.

How do you evaluate which agentic AI vendors will last?

Evin Safdia: I appreciate that. That leads into the idea that things don’t necessarily last. With AI, there’s probably a new vendor or more every day, and a portion of them are in agentic security. How do you evaluate which ones might last and which ones might not be worth investing in right now? Real-world examples would be great too.

Chris Steffen: The problem is that it’s such an emerging technology space that there are a lot of what we’ll call aspirational features. When you look at a vendor’s roadmap, they say, “We want to be able to do this,” and it’s very hard to quantify that or demonstrate that the product has that ability. When I’m looking at these tools, I want to see that there’s no black-box problem: that you don’t have to trust the man behind the curtain to make those decisions, and that the vendor’s claims aren’t just pie in the sky. So ask them. Make them prove it to you. Make certain they’re doing things that are verifiable. It’s great to be aspirational, but if they can’t articulate what they’re doing today, maybe you come back to them at the next conference. If they can’t do it today, there’s no proof they’ll be able to do it tomorrow, and that’s really important.

How do you balance the value of a new tool against the effort to run it?

Evin Safdia: I love that. What I’d add, and what’s important to me, is usability. It could be the most amazing tool in the world, but if it’s impossible for me to use, given my skill set, the people on hand, and my ability to learn, it may not be a good fit. So as we roll out new agentic solutions, how do we balance the work they actually save against the added effort of deploying and maintaining another solution?

Chris Steffen: There are two aspects to that. The first, which you mentioned, is maturity. Not every tool is right for every organization. You could be a very small, greenfield organization that needs only basic security infrastructure. You may not need a huge, dynamic SIEM, or an agentic solution, because you just don’t have those needs. If you’re a huge organization, it might be exactly the opposite: you need something massively scalable that can handle multiple types of workloads at once. It depends on your organization’s maturity, and specifically its security maturity. Do you need those kinds of tools? Will you actually be able to implement and properly use them?

Chris Steffen: The second aspect is that people believe a lot of tools are fire-and-forget, and with agentic solutions, nothing could be further from the case. You have maintenance. You have training. You have constant intervention. You have to understand how these tools will be implemented, maintained, monitored, and secured, and the second you finish all that, you start over and do it again. It’s not like Microsoft Word, where you update it once a quarter and you’re good to go. It requires constant maintenance and constant evaluation. Just as you wouldn’t fire-and-forget an employee, think of it the same way. I’m not saying these tools will replace an employee, but you wouldn’t tell a SOC analyst, “Go do your thing, and we’ll catch up with you at the end of the year.” It doesn’t work that way. You have to keep feeding it, educating it, training it, and administering it to get the most out of the tool.

How do you build trust and transparency in autonomous security tools?

Evin Safdia: I love what you said about not every tool being a fit for every company. Your Fortune 100 companies and small SMBs have different competencies and different budgets, so picking a tool that aligns with your capabilities goes a long way, and it’s great to say that out loud. That brings up another question. As we build more trust in the autonomy of these solutions, how do we maintain transparency, so we understand what’s happening, while still leveraging them to take work off our plates?

Chris Steffen: That’s a great question. From the very start, keep in mind that every agentic solution, every autonomous system, is nothing more than a tool, and every tool should be explainable. If you or the vendor can’t explain in detail how something works, how it’s trained, and how it makes decisions, it’s probably not the tool for you, because you don’t want to trust the man behind the curtain. You want a verification framework: “I want to be able to do this, but I want to be able to verify that you do it.”

Chris Steffen: That may not just be for you. You may have a supply chain partner or a regulatory requirement that says, “You can use whatever you want, but we need to see behind the curtain. We need to understand what you’re doing with that data and how that solution is handling it.” If the solution can’t do that, then, quite bluntly, pick a different solution. And lastly, like we talked about before, make certain it supports a human in the loop, so you can reach that level of comfort and trust in the solution.

Chris Steffen: Over time, you will. Take a very common example: firewall rules. I was one of those original firewall monkeys. I mentioned I had a Check Point certification a thousand years ago, back when you’d go through zillions of firewall rules, looking at allows and denies and trying to figure out what made sense. We don’t do that anymore, and trust me, I’m thankful. It was horrible, and everybody thought it was horrible. Today, automated systems catch 99.99999% of what’s classified as weird, weird being a technical term for an anomaly. And they’re way better at it than I would ever admit, because after the first thousand rules, you almost [lose your mind]. So trust me, that is the better way to do it. Over time, you gain confidence that the system does that job better than a human can.

Chris Steffen: Agentic models are the same way. Until you’re confident they do the things you need them to do, and don’t do the things you don’t, it’s going to take some time. That’s where the human in the loop really makes a difference.

What should buyers ask before purchasing an AI security product?

Evin Safdia: So the human element is important, and, as we’ve mentioned a few times, so is selecting the right solution. We’re in an era of, not to say AI washing, but almost AI washing, where everything is AI or has some AI aspect. At Zero Networks, we took a stance that AI is useful to some extent, but for microsegmentation, we want deterministic automation, not something random on the fly where who knows what will happen. So I like that distinction. With everything being AI nowadays, what advice would you give every security buyer before they buy their next AI product?

Chris Steffen: The simple one is: show me. I’m going to give you a set of problems, and I want you to show me how you respond. I want you to show me how it will evolve over time. I want quantifiable metrics on how you’re going to improve my security and infrastructure. Vendors should be able to do that, and if they can’t, potentially move on.

Chris Steffen: Let’s take a simple one. I want you to reduce my alerts from the 50 bazillion I get today to something extremely manageable. That’s a problem every SOC in the world has, and in theory, an agentic solution should help reduce false positives. So give me a number: “We’re going to reduce your false positives from 35% to 5%,” or something like that. That’s quantifiable. If they give you vague answers, that’s fine; I understand they may not know. Then go find a solution that does. If they can only talk about the algorithms, the model, and the training, and a lot of them are doing this, you’ll see it on the show floor at RSAC Conference, where they’re all going to talk about the exact same things. I get it, but that means they have a marketecture, not something that’s actually going to improve my infrastructure.

How do you keep security current as AI innovation accelerates?

Evin Safdia: I think you said that very well. We’re wrapping up, and this has been a really awesome conversation. In my experience, security is always trying to keep up with innovation and with bad actors. The firewall came after the hacker, the antivirus came after the virus, and so on. How can we keep our ability to secure things despite all this rapid innovation, especially on the AI front?

Chris Steffen: It sounds a little crazy, but you do need to adopt the latest tools. You need to look at how automation and the latest advances will improve your infrastructure. That doesn’t mean one tool is one-size-fits-all; I’m not making that claim. But you need to understand how the latest innovations will help you move the needle and improve your overall security maturity, and you need to do that constantly.

Chris Steffen: That comes with a mandatory security review. You should be looking at your security architecture regularly. The old standard was once a year, but the pace of innovation is now such that once a year may be too little. I think twice a year is probably more relevant, and I know a lot of companies do it quarterly. Regardless, continually look at that architecture and make certain you’re taking advantage of the latest advances, or you will be left behind. Look at the innovation in agentic AI models over the last 12 to 18 months. It’s been insanely massive. If you only review once a year, you might be falling behind.

Chris Steffen: Lastly, and I can’t stress this enough: figure out how to be an asset to your business, and look out for the best interests and priorities of your business. That will always keep your security innovations in line with the business. If you don’t, you’re doing security for fun, and it’s not fun. Find the real reason to implement something. Maybe it’s a new set of compliance concerns. Maybe it’s supply chain considerations. Maybe it’s to improve your overall zero trust infrastructure. Those are all laudable goals, but make certain they align with business goals. When you do that consistently, your security team will be happier, your business leadership will be happier with your security and compliance teams, and everybody will be in a better place.

Evin Safdia: It sounds like we should ramp up the frequency of security reviews, and approach them not from the old perception of security as the department of no, but as a way to enable the business securely and keep things running. That’s all very important. That’s all I had. Thank you again, Chris, for joining us, and thank you, everyone, for joining us on Containment Island. Chris, any parting words for the audience?

Chris Steffen: Again, thanks for having me, Evin. I love doing this. If you have any questions, you can always reach me on LinkedIn. Evin and I, and the folks at Zero Networks, talk quite regularly, so I’m always happy to help. Thanks again for having me.

Evin Safdia: That’s amazing. Thank you, everyone.