What engineering leaders are learning at the AI frontier

At this year's Engineering Leadership Forum, we sat down with three engineering leaders from Toast, Zscaler, and Motive, and it was clear how far the conversation has moved in just a year.

Nobody on stage was debating whether engineers should use AI. The conversation centered on how much work to hand over, and what engineers should focus on once AI gives their teams more capacity.

That's a big shift from our inaugural forum last year, when much of the discussion was about getting engineering organizations ready to adopt AI. At the time, many senior engineers still saw AI mainly as a way to write more code faster. Few were thinking about what it could do for how software runs in production.

Three takeaways from this year's panel:

1. Trust is paramount
2. The best AI tools remove high-toil work from engineering teams
3. Advancements in AI are increasing the value of engineers

Trust is paramount

Trust was a recurring theme this year, and there is still an extreme range in how much trust engineering leaders are willing to place in AI agents. Prashant Ramarao, Head of Platform Engineering at Motive, is taking a more deliberate approach, keeping humans in the loop until agents prove they can be trusted over the long term. Steve Brewer, SVP of Engineering at Toast, and his team are taking a more ROI-driven approach. As AI continues to improve at tasks like code review, Steve sees a point where it will surpass humans and make more sense, both economically and in terms of accuracy, to hand certain types of work over. He expects that same shift to happen across different parts of the software engineering job as AI continues to improve.

There is also the question of trusting the vendors providing these agents. Steve emphasized that companies need to be careful with the vendors they bet on and make sure vendor incentives align with their internal business outcomes. This becomes increasingly important as those vendors take on more work across the engineering organization.

The best AI tools remove work from engineering teams

Each leader had a different red herring metric they told the audience to avoid focusing on, but the one thing they all agreed on is that the best AI tools are the ones that take time-intensive work from an engineer’s hands and allow them to focus on more customer-impacting work that aligns to business objectives.

We’re already seeing what that looks like in practice. AI is taking on work like refactors, runbooks, incident investigation, and other tasks that have historically taken up engineers’ time. Jim Tsiamis, VP and Lead of SRE at Zscaler, wants fewer of his engineers on-call for low-severity incidents, and more of them focused on bigger-ticket items like rearchitecting their infrastructure and making their systems more reliable. At Toast, AI has enabled engineers to solve issues on-demand instead of just finding and declaring them and adding them to the backlog. Steve has also seen renewed interest across his organization: engineers who once felt like task-doers now feel like innovators because they can go out, find, and solve problems on their own.

The same principle is shaping how these leaders think about what to build internally versus what to buy. Jim summed up Zscaler's approach as "buy the commodity, build the differentiator." The commodity here is the work itself. Investigating a low-severity alert is necessary, but it doesn't make Zscaler's product any better than a competitor's, no matter how hard the problem is to solve. If there is an established solution that can take that work off your engineering team's plate, buy it and let your engineers spend their time on the problems that are actually specific to your business. As Steve put it, Toast doesn't sell developer hours.

"Any vendor that can come in and help us move faster, innovate better, and improve our margins all while helping us deliver a better product for our customers, that's a good partner." - Steve Brewer, SVP Engineering, Toast

Buying more doesn't mean building less, though. At Motive, AI has pushed Prashant's team to build more internally. He pitched it to his CTO with a simple analogy: if you want a Ferrari, you need a good road for it to drive on. For Motive, that means a major emphasis on its platform and infrastructure teams and making sure they're building the right foundation for AI agents to operate effectively. An agent is only as effective as the systems it runs on, and only Motive's engineers can build that road.

With the rate at which new AI tools and capabilities are coming out, staying focused on those problems is increasingly important. Steve cautioned against seeing what another company is doing with AI and assuming it should be applied to your organization too. Just because there’s an interesting blog demonstrating some incredible AI capability doesn’t mean it’s the right area for your team to focus on. Start with your organization's most important problems, and figure out where AI can help.

Advancements in AI are increasing the value of engineers

Despite the sentiment over the last few years that AI might replace the software engineering job, our three guest leaders have each experienced the opposite outcome. As AI tooling has strengthened and evolved from assisting engineers to completing work on its own, our guests noted that they are now hiring more engineers than before.

When posed with the question, "Are you hiring more agents or more engineers?" Jim and Steve answered with “both,” and Prashant with “engineers.” All three pointed back to the work they now want their engineers spending more time on. At Zscaler, Jim is offloading tasks to agents while hiring more engineers to take on more complex, value-adding problems, like making Zscaler's systems more reliable and resilient. Prashant wants his SREs on higher-order issues: "We don't want our SREs sitting in incident rooms trying to debug incidents. We want them looking at the next architecture we can implement to increase our reliability. That's way more important to us." For Steve, agents make his engineers more valuable by letting them solve more important customer problems instead of spending time on manual work and maintenance, so Toast is bringing on more of both.

As AI takes on more of the work that has historically consumed engineers' time, these leaders see more opportunities for their engineers to solve higher-order problems, and they are hiring accordingly.

What will the next year bring us?

Last year, we were asking how engineering organizations should prepare for AI and what its impact on software engineering might look like. A year later, many of those questions are already being answered in practice. AI is taking real work off engineers' plates and leaders are getting more comfortable handing it over. At our customers like Toast, Zscaler, and Motive, that shift has given engineers more time for the work their leaders care about most, and all three are hiring more engineers than before. If the progress between our first two forums is any indication, next September's panel will have even more to share, and we're already looking forward to it.