AI and the future of Software Engineering

Key takeaways

• Agents vs. co-pilots: True transformation comes from autonomous agents, not just "glorified autocomplete" co-pilot tools.
• Infrastructure challenges are real: Building agents requires rethinking company organization, system architecture, and user interaction patterns.
• Top-down adoption needed: Unlike other technologies, AI adoption often requires leadership push due to engineer reluctance, especially among senior developers.
• More programmers, different skills: The future will have orders of magnitude more "agentic programmers" who manage AI systems rather than write code directly.
• Reliability is crucial: Current agents need better testing and autonomy capabilities to move beyond the babysitting phase.
• Mobile-first development: The future of programming may be increasingly mobile, with "ambient building" happening throughout the day.
• Agent-to-agent collaboration: The next frontier involves agents spawning and managing other specialized agents for different business functions.

What sparked my early interest in programming and starting Replit?

I'd been programming since I was six or seven, and I remember using those code generation wizards - you know, the Microsoft Visual Basic ones that would generate tons of code.

I've been at it for a long time now through multiple companies and startups, but the vision has always been the same: if you make programming a little easier and more accessible, a lot more people would want to do it. We started talking about potentially a billion people wanting to make software at some point, maybe in 2019-2020. It was kind of laughable even then that it would get to that point.

The problem I kept running into was that setting up development environments was such an annoying and boring thing to do over and over again. That's what started the idea for Replit.

"I just felt like you know programming is probably going to get automated pretty soon like there was like a bit of an AI hype even back then."

When did I realize AI agents were finally possible?

When we saw GPT-2 back in 2020, it was obvious that we were just going to be on this trajectory where software was going to radically transform. We started adding different AI features, but it wasn't until last year where it was a combination of us being a lot bolder and feeling that AI was at a point where agents were possible.

We always felt like the co-pilot style was glorified autocomplete. The thing that was going to truly transform software was agents. So in late 2023, early 2024, we basically did a reset - everyone in the company was working on Replit Agent. We released it in September 2024, and I think it was the first in its category.

"We always felt like the co-pilot style is like you know glorified autocompletes and the thing that was going to truly transform software is agents."

What were the biggest challenges in building AI agents?

There are a lot of different difficult things about building agents. One thing that might be surprising is how you structure the company, because now you go from your user being human to your actual user being an AI, and the human is managing the AI. It really changes the paradigm.

Here's an interesting example: we had an Asia cluster so that when people were programming in Replit, they would have a short distance to get their keystrokes in - you needed to be as fast as possible. But suddenly when we released agents, our Japan and India users were reporting it to be very slow.

What was happening was the AI models were in the US, so your message would go from the Asia cluster back to the US, and then all the coding happening between the compute environment and AI needed to be in this fast loop, but instead it was traveling across the world. We ended up deleting the Asia cluster because as a human, you're just texting and all the coding is happening in the AI.

"Now you go from your user being human to your actual user being an AI and the human is managing the AI. It really changes the paradigm."

How do we organize companies around building agents?

For a while, we really didn't know how to organize the company. We have some idea now - we have the core agent group, and then we think of infrastructure, dev tools, cloud services, all these different teams as kind of pouring into this core agent group. So the core agent group is working on the prompts and the core functionality, and everyone else is sort of servicing them.

“We're also sort of in the jQuery era of building agents. I remember early on in the web, even before jQuery, no one knew how to build web applications. With jQuery it got a little better, but before React, making apps was always very painful. React was the first thing that made us think "this is how we're supposed to make web applications." With agents, no one knows how to make them right now.”

One breakthrough we had recently is leaning really heavily on Temporal, so durable execution has really transformed how we do agents. The way we conceptualize it is as a state machine, and then every one of those state nodes is sort of like a Temporal function.

We're making progress on what the architecture should be, and we're hoping to open source and talk more about it.

How do I see the role of software engineers evolving?

It sounds lame when you say it like this, but I think it will become sort of this management position. You're managing a lot of different things, and a lot of us are already doing that. We have this mobile app where people like to go between their computer and mobile - they'll go into a meeting, get a notification from their agent, and respond to it or take a look at it.

This idea of ambient building - you're sort of like managing a company, sort of like being a CEO. I'm almost never at my desk doing one specific thing. I'm doing a lot of different things, I have access to all these different dashboards, I'm managing all these different people. Software engineers will be a CEO of sorts.

"Software engineers will be a CEO of sorts um and uh and I think um like every aspect of uh software engineering is going to be taken over by agents."

Do I think there will be fewer people working on software?

More, actually. I think we're going through a similar phase change where we're going from primarily industrial society to primarily information society, where most people are working in the world of bits. Even the world of atoms will turn into bits as robotics comes on the scene.

The need for software will just be ever increasing. That's not to say anything about VR and AR. People will spend more and more of their time in digital spaces, so the need for software is nearly infinite.

That said, the kind of skill set needed will vary. In the same way that early on, a lot of software engineers were low-level C and C++ programmers, and now there's an order of magnitude more JavaScript programmers than C programmers, there's going to be an order of magnitude, perhaps two orders of magnitude, more agentic programmers than JavaScript or Python programmers.

"There's going to be an order of magnitude, perhaps two orders of magnitude of agent agentic programmers than there are your typical JavaScript python programmers."

What advice do I have for teams trying to adopt AI?

One thing that's been really interesting - and not surprising to me but surprising to a lot of people - is how reluctant some engineers have been to adopt AI. You hear that a lot from engineering leaders and founders: they just can't get their engineers to use AI.

There was a story on Hacker News where someone was saying if you're not using AI for programming, you're kind of insane. He talked about how there are a lot of senior programmers out there that are still skeptical - they think of AI as basically another hype cycle, another NFT that's going to go nowhere.

What I've found is that AI adoption needs to be somewhat top-down. You need constant reminders and you need someone on the team that's extremely excited about AI. I would designate either a team or a person to continuously explore the latest and greatest and then train people on it. Even track how much they're using it.

"AI adoption needs to be somewhat top down. You need constant reminders and need someone in the team that's extremely excited about AI."

What should teams look at when exploring AI?

I don't think you can just give you a formula for what to look at - you need to kind of experience all of it. More than many other trends, like you didn't have to force engineers to use Git - Git was adopted kind of bottoms up. But AI is one of those things where some of it needs to be top down.

The paradigm shifts constantly, and the models continue to beat each other at various benchmarks. Which is why I would say maybe dedicate someone in the company to be up to date on all this stuff.

What's my vision for Replit's future?

When I look at the use cases that people use Replit for, there are a few categories: building a business, building internal tools, or building personal apps. If you zoom out a little bit, a lot of these things are about problem solving - they're not necessarily just building software for the sake of building software.

So you go from Replit Agent to software to user to problem. What if you go from software agent to solving a problem directly? I'm not talking about AGI, but there are a lot of software problems out there in your business and daily lives that could be immediately solved by a very competent set of software engineering agents.

"What if you go from software agent to solving a problem, it sounds a little like AGI but that's not the kind of thing that I'm talking about. There are a lot of software problems out there in your business in your daily lives that could be immediately solved by a very competent set of software engineering agents."

What excites me most about the near future?

Another thing I'm really excited about is being able to spawn other agents from Replit. If we've figured out the best way to build Replit Agent, I think we have an opportunity to create a platform so other people can build agents on top of it. You can imagine Replit Agent being your agent manager - your command and control for creating a sales agent, marketing agent, events agent, all sorts of different agents that you can manage through Replit.

I'm also really excited about our mobile app. I talk to people every day using the Replit mobile app for eight or nine hours a day, which is just wild to me. Some of them don't even have a computer - they use it on their phone and they're building businesses and apps. This idea of ambient building where you're constantly building - maybe that's stressful to some people, but to me that's really fun.

What's the most important thing for Replit right now?

The most important thing over the next few months is reliability. Right now, as users probably can relate, Replit can sometimes make two steps forward and one step backward. It can break features, and it also requires a lot of babysitting - it keeps asking the same questions, making you a QA person.

We want to solve these problems, so a lot of it's testing actually. We're going to introduce a way for the agent to be able to test its own code and applications, so have a computer use aspect but make it fast enough that we can run these tests on every change and be able to roll back for you.

Once we have testing in place, that unlocks full autonomy. You can tell Replit to go work for an hour and it can build the entire app on its own, or you can create branches and have parallel agents.

What AI tools do we use at Replit?

We use all kinds of tools. People at Replit use Cursor, we try Devon, obviously we use Replit a ton, but we really try competitors as well to make sure we're on the cutting edge. Our platform engineers really like Cloud Code - that's been really popular recently.

The way they use it is very interesting - it's more like letting it go to work on routine tasks. Personally, I end up building a ton of tools with Replit to manage my life and work. I have a ton of Slack bots just to manage the information flow in my company. I'm always building something on my phone that makes my life a little easier and better.

What gets me excited about this moment in tech?

We're at the birth of a new type of entity. We've had LLMs now for five years, so we know what LLMs look like, but agents are a new type of entity. Obviously it's LLMs at heart, but it's all the tools that we're building around LLMs, and they're going to start being trained differently.

I've been grateful to be part of a few seminal moments in programming - I was part of the team at Facebook that built React. I feel the same way about agents today. No one knows how to build them. It's a great time to be building startups, it's a great time if you really want a hard software engineering challenge.