Poteto (creator of pstack) on shipping thousands of PRs a month at SpaceX

Hello, folks. I've got another treat for you today. Last time on this kind of podcasty thing, I suppose, we had Uncle Bob, and we talked about software quality, we talked about agents, we talked about lots of cool stuff. Now we have an incredible guest, someone who I'm delighted to welcome on, who's been exploding on Twitter recently about software factories, about increasing the quality of your work, about increasing your velocity and climbing the trust ladder with agents so that you can ship more and more and more. And it is poteto. Welcome. Thank you so much for joining.

Thanks for having me. Yeah, very excited to be here. Big fan of yours.

And a huge fan of yours. I think people have been talking about this as the meeting of the skill minds. The skill Mount Olympus or something. Because both of us have very popular skill libraries. As I was saying before we started, I've not used a ton of yours, and I want to get all of the juice out of your brain so that I can go and use it properly and use it better.

Climbing the trust ladder

I think where I want to start with this is: you gave a talk pretty recently, about 10 days ago, and posted it on X, which went absolutely nuts, about how you shipped 2,500 PRs last month to production. Got about 3 million views or something on X, and I watched it and I loved it and I recommended it. And I kind of want to run this almost as a Q&A of that talk, because I just had tons of questions about it and I wanted to dive into it.

And I think where I want to start is: you talk about a trust ladder with agents, where as you trust agents more, you can get them to do better and better things, or scale them up to use more and more agents. So what is your story of how you climbed the trust ladder? And how did that work when you got to SpaceX and started running more and more?

So I think the journey sort of began even before I joined Cursor, which is now SpaceX AI. The story is, after Meta (I used to work at Meta on the React team) I took a month off because I was feeling kind of burnt out. And of course, what do you do when you're burnt out? You go and start a new side project. So I started a side project. I was, of course, using AI to write code. But then I started to realize I was spending so many hours just micromanaging one agent.

At the time, this was back in February, maybe early February or January, people were really obsessed with this idea of orchestration. This was before things like the Cursor agents window had become popular. So people were still in TUI land in their terminal, and they were all talking about, okay, here, I built a custom orchestrator. And so of course I was a bit nerd-sniped by that. As I was building my toy project, I got nerd-sniped by: how do I make my AI coding setup more efficient?

So I kind of started the journey there, where I just took a step back and realized I was spending all this time micromanaging a single agent. I was creating skills, and I was finding it quite difficult to measure the output or the results, the impact of the skill. So I was kind of flying blind, but I was iterating really fast. And that project eventually became the basis of pstack, even though I didn't know it at the time, along with a lot of the tricks I had learned building that early set of skills. Actually, it's still open source, if anybody wants to take a look. It's on my GitHub, poteto/noodle, N-O-O-D-L-E. In there you will see some skills and a brain directory.

I was really interested in this idea of how do I extract my own ability, if that makes sense, and give it to the agent? Because I realized that all I was trying to do was teach the agent to write code more like me, to do workflows more like me. So the skills were an entry point to doing that.

And then after I joined Cursor, I was starting to work on the agents window, and it had a lot of performance issues. It was pretty laggy. Since I had experience working on React, I was asked, hey, do you want to come and help out with the agents window? And so the beginning of the Cursor journey was very manual. I was deep in looking at flame graphs and heap snapshots and trying to see why exactly the app was so slow. But then I came back to the same realization: I was the bottleneck. I was doing everything manually. I was sort of the meat proxy, in a way. I was the meat proxy between my agent and Chrome DevTools. And I was really annoyed by that.

And what month of the year is that, let's say? Where are we in the timeline?

So I joined Cursor in March. So this was early April, probably. And I didn't have any skills. I had sort of abandoned my personal skills because I didn't think they'd be relevant anymore. But then working on the agents window, and now working on Grokbot, I realized that a lot of the lessons I had learned building that initial set of skills were very relevant, especially around things like verification, being very rigorous in your work. Because agents, from my experience, even the frontier ones, tend to take shortcuts. They tend to do the easy thing. So a lot of the skills that I've built have been around: how do I make the easy thing the right thing? How do I make that the best thing?

Domain expertise and the power of words

The idea of distilling your expertise and turning what you do every day into processes, that's something that feels super familiar to me. That's exactly what I've been doing with the skills. And I suppose there's something in that, which is that a lot of people think domain expertise is getting less useful now, as people start to rely more on AI. What do you think about that, just as a sort of vibe check before we start to talk about that?

I actually feel like domain expertise is more important than ever. I think I wrote this on my X at some point, but I sometimes think of AI, especially as the models get smarter and smarter and more capable, and the frontier models are just getting so good (I love Opus 5.5, by the way), that as the models get really, really, really good, the bottleneck is no longer the agent. It becomes your ability to express your intent and your goals in a clear way that the agent can understand and actually carry out.

And that's why I think people with a lot of domain expertise have a huge advantage, in my opinion, especially if you're a little bit techno-curious. I think of people like a doctor or a lawyer, someone who has deep expertise in a particular non-engineering domain. If they're actually just a little bit tech-savvy and they can figure out how to use agents, they can build really, really great products, if they have a clear enough vision in their head and they can articulate it in a way that the agent can build it. I think that is really the bottleneck these days: the transfer of your intent and your vision to the agent.

Yeah, I've been obsessed with language basically since agents dropped. I'm just obsessed with thinking constantly about the composition of words, how I can make things sharper, what might be hidden in the phrases that I'm using. And what I love is when you find a word that the agent then hooks onto and goes, okay, I'm going to reinforce that word, I'm going to reuse that in my thinking traces. I found that TDD was an early example of that. There's a lot of chat about TDD recently: people say, should you use TDD with agents? It doesn't matter. What you're doing is getting the agent to think about TDD, getting it to write tests, getting it to prioritize things in a different way than it did before. And that's why I think grilling works effectively, because grilling...

Yeah, it draws those words out of you. Or at least it helps the agent understand your thinking so that it can propose those words to you, and you can pick them up and say, yes, exactly that.

I've actually copied some of the tips that you've shared as well. One of my favorite ones that you've shared, maybe in the past couple of weeks, is about reducing or eliminating tautological tests. One of my pet peeves with agents is all of the useless tests that they write. And that was one thing where the word "tautology" is, I guess, not something many people necessarily know, especially if English isn't your first language. But there's a lot of meaning in that word. It's almost compressed: you compress a lot of intent and meaning into words.

So I totally agree with you. I've always been interested in language, actually: programming languages, natural human languages and how they came to be. And it's so interesting that now with agents, it's this meeting of natural language with programming language, but it's all language under the hood. It's all communication.

Totally. I did a drama degree, right? So I've been thinking about language and Shakespeare and stuff for a long time. This all feels very familiar.

The Michelin kitchen

So, okay. I think the thing I want from you is the software factory stuff. Software factory is the big buzzword. Software factory is the thing that I'm thinking about too; I'm releasing a course in that direction. And it's this scaling yourself up to unrealistic numbers of PRs, basically, or PR numbers that sound ridiculous to people who don't understand how this works. So where I want to get to is from people who are running one to five agents today up to hundreds of agents running at once, and how that functions. And so I'd love to hear about your metaphor of the Michelin kitchen instead of the software factory, because I think that says a bit about the way you think about this stuff.

Yes. I've never really liked the term software factory, not because it's not accurate, but because I think a lot of people, when they think factory, don't necessarily associate that with quality or craft, things which are very important to me and to a lot of technologists who build products. We care about the user experience; we care about the things we're building. So while I do think software factory is an apt term, it maybe conjures up negative connotations sometimes.

Michelin kitchen is the thing I've landed on. I feel like it's much more aspirational, and I like the metaphor a lot because I like food (I'm called poteto, of course) and I like cooking, and I see a lot of parallels. When you're cooking a meal for yourself, for example, it's utilitarian: you're just trying to feed yourself and survive. But it can actually be transformed into art. That's what a Michelin-starred chef, or even just a chef or a cook, can do with food: take something very ordinary and turn it into a delicious meal that takes you back to your childhood days or something like that.

And it almost mirrors that trust ladder I talk about, where you can imagine your own journey as a home cook. As a home cook, you are doing all of the cooking yourself. You cut all the vegetables, you do all the prep work, you do all the cleanup. You are the one-man or one-woman show, really. And it's an interesting thought experiment: okay, you're cooking a meal and then you add people. Your partner joins, then your brother, your sister, and now suddenly you have your whole family in the kitchen. I think most people would get very stressed by that. The thought of, oh, so many people are just mucking around in my kitchen. They have no idea where all the utensils are. It's my kitchen.

I have a max capacity of one person in the kitchen.

Yeah, exactly. Absolutely. So I feel like that's really apt, because when you ask yourself the question of how do I go from being a solo cook to having an army, or not even an army, but a few sous-chefs helping me in the kitchen, how do I think about dividing the work in a way that makes sense? I'm not dividing work just for the sake of it, but in a way that actually makes the whole better than the sum of its parts.

So the Michelin kitchen metaphor works really well for me in that regard, because if you become a chef, you're in a position where you're not necessarily cooking all the food yourself anymore. You're almost like the tech lead for the kitchen. Chefs have to think about not just cooking; they basically have to organize the whole kitchen. They're like the CEO of the kitchen. They have to think about when do you order ingredients, how do you store them, how do you prepare them, when do they have to be prepared. It's a whole job that's not just cooking.

And again, it mirrors so much of how engineers code today, where you are not writing the code yourself anymore. You have agents. But you, as a human, are still responsible for the final outcome. Your name is still associated with the work that you do, your reputation. And how you set up your kitchen, how you set up your skills, your environment, your codebase, are ultimately the new ingredients that go into building product.

Verification: giving the agent hands and eyes

Yeah, I think what I love about your approach is the amount of focus that you put into the environment that the agent operates in. Because I think a lot of people think: right, the agent is good, I'm probably not going to be able to make it better. Let's just trust what these magic model people have put into the harness and the model combination. There's nothing I can really do; I can't mess about with Claude Code's internals or whatever you're using.

But what I love about your approach, and it's something I advocate for too, is that you can change the environment the agent operates in. You can make changes in the codebase, and also give it tools for verification and allow it to verify its own work. So the thing I loved about watching that talk is the amount of focus you put on verification, and that is the lever with which you can start to generate trust. Can you talk about that and what that concretely looks like? Let's start looking at practical ways that people can improve their own processes, their own kitchens.

Yeah, I've said this a lot, actually: even if you don't use pstack or your skills, I think the single most important skill that should be in your toolkit is verification. For those watching who don't know what that means, it's this idea that you can give your agent hands and eyes. That's the analogy I like, where the agent is able to run the code and actually interact with it like a normal human user would, and also do things like debug it, take traces and snapshots.

Funnily enough, that was actually the first skill I built when I joined Cursor. That was the thing that started to let me ascend the trust ladder, in a way that some of the other skills I had looked at or built had not. Because no matter how good some of the other skills were, like the how skill, the why skill, the unslop skill, I was still the proxy between my agent and the output. If the agent can't actually see the result of its work, there's no way it can iterate.

And this is where people start to talk about loops, this idea of a loop. The term loop seems almost abstract: what is a loop? What is an agent loop? But really, to me, the most important part of the loop, the part that allows it to be a loop, is the verification part, because the agent is able to verify its own work, and that takes you out of the equation.

One of the very first use cases I had for verification was the performance work I was doing on Cursor's agents window. I wanted to get to a point where I could do something called hill climbing, which is a term I think the labs talk about a lot: this idea that you have some kind of rubric, a way to judge or score something, and now, because you have a loop, you can have an agent continually try to make improvements to that. Andrej Karpathy also famously released something called autoresearch that has a lot of these ideas.

But yeah, verification I would say is probably the most important skill in pstack, and in many other tool sets, and I think it's the most important thing to focus on. I've spent a lot of time tuning the create-verification skill. And internally we have so many verification skills now: every app that Cursor, or SpaceX AI, has has a verification skill that is auto-maintained as well. It's become critical infrastructure for our team, because everybody uses it.

Deterministic scripts inside skills

And you went pretty far with that too. In your talk I saw that you actually built a custom CLI for that. So what does that CLI do? How does it execute things, and why did you build it? I mean, that's proof of how deep you're going, of how much you're pushing that.

Yeah, so this is actually a tip I learned early on. I guess back in January, or late last year, the thing people were very concerned about was the context window. That was the big topic at the time: how do I manage the context window? Because compaction and summarization weren't really that good yet. There was almost a meme in the community that once your agent summarized or compacted once, it would become stupid for the rest of your session. So there was a lot of thinking around being very efficient with your context usage. And that was actually the inspiration for some of the CLI work inside the verification skills.

I guess now it's less about context, because harnesses have gotten a lot better at summarization. I still think there are some benefits to having a clean context window. But the CLI is really more a way for me to take the deterministic parts of what the skill does and encode them into a script or a CLI, to take away the judgment that would otherwise be used unnecessarily.

I think of agents and skills as a gradient. You have some parts of the work that are entirely judgment-based: something that requires thought, putting together multiple pieces of context, thinking. And then you have the more deterministic parts. If you wanted to refactor some code from one pattern to another, I don't know, that's very mechanical. You don't need an agent to think about it and come up with it in a novel way each time. So that was really the inspiration for the CLI, and you'll see this in a lot of the other skills I've built. I try to extract the deterministic parts, turn them into code, and leave only the parts that actually require judgment to the agent.

So in a way, I think of the CLI as a wrapper. It's a wrapper with some light instructions around how to use these custom tools that are inside the skill. I don't think the CLI is really that interesting on its own. It's not a novel piece of software. It's just something that interacts with Playwright and the Chrome DevTools protocol and calls a bunch of APIs. It's just a bunch of...

No, it's fascinating, because it's a way of hiding information from the skill. It's a way of conserving the skill, keeping the skill quite small, I imagine. And then you're able to delegate more of the complicated deterministic stuff into a script within the skill. So you're compressing information and making the agent do more things more consistently. That's fascinating.

Yeah, and it also helps if you care about the context window, because now the agent doesn't need to reinvent things. One thing I had noticed early on, when we didn't have a CLI, was that while the agent would try to verify its work, it would basically rebuild the world each time, and every agent did it differently, and I started to notice that's very inefficient. It wasn't just about context usage but also speed. Because now my agent had to actually go off and write the scripts or the CLI and test it, and it doesn't work. And the last agent did it and it worked, but it discarded it. So it was just very obvious at that point: I should turn this into a CLI and put it inside the skill, so that every agent that uses it benefits from that same piece.

But I also think it's a good push for people to think about how much of your skills and rules could actually be deterministic. That's one of my core principles: how do I make very efficient use of determinism and non-determinism, and let agents shine at the non-deterministic parts, because that's what they're trained to do? The other parts, which are much more mechanical or straightforward, can just be pure determinism. You'll see this as well with things like doing migrations, which is another big thing I've talked about: going from one technology to another, especially one that is better for our agents. And a lot of how you can do that migration is through things like scripts and CLIs, the deterministic parts, like codemods, crawling the AST, the abstract syntax tree, and transforming code literally mechanically. A script does it for you instead of the agent.

Totally makes sense. I think there's another thing there, which is that you're taking stuff away from the agent and you're putting it in the environment too, a little bit. Let's say you have a thing you notice the agent always gets wrong: you want to make that just impossible within the environment.

And that comes down to code quality as well. I talk a lot about what a good codebase means. What is a good codebase? There's a definition I like, which is that a good codebase is a codebase that's easy to make changes in. Easy to change stuff without things screwing up. And that means you have a lot of guardrails: the agent or the human is constrained to very narrow paths. That's again something you talk about in your talk, and you talk about it not only on the automated-checks side of things, so linting and type checking, blah, blah, blah, but also in the way you design abstractions. And I think you guys even built a framework for your agents to work in. What I'd love to hear is: you obviously think of that as very important. How important is that compared to other things you could be doing, like building features or shipping work?

The new job is the environment

Yeah, I almost feel like the new job of the engineer is really to spend time on the environment. I almost wrote a tweet about this yesterday, but I didn't. I think that if you haven't spent time building trust in your agents, building skills and tools, you can get stuck in this mode where you're very low on that trust ladder. You don't have a lot of trust in your agent's work. And so the only way to cope when you're in that situation is to lock in and micromanage your agents. And that's very time-consuming. When you're stuck in that mode, you don't really have the luxury to think about higher-level things, like making yourself more productive.

I guess an analogy would be: you're a developer writing code yourself, and you've never taken the time to learn your tools. You've never heard of VS Code, you've never heard of Vim, you only know about Notepad, and you haven't even heard of Git. You haven't spent the time sharpening your own knives. And of course, if you have a dull knife, everything takes a long time. Especially if deadlines are looming, you're stuck in this rut where you don't have sharp knives, you don't have good tools, but you're under all this pressure to ship, and all you can do is focus on that.

But I do think that if you can find the time to actually think about your setup... Going back to the cooking analogy: if cutting the garlic is super slow, there are garlic mashers you can buy. You put it in the thing and you squeeze it out, and it's super fast. Machines and tools are invented for a reason. If you're operating a Michelin kitchen and your cooks had no tools, then of course everything is going to be extremely inefficient. Every cook is going to make something up of their own. So the tools and the determinism, to me, are taking that part away. And just like you said, the constraints are part of that as well.

Constraining the space: types, lint rules and Dune

Actually, a slight tangent on that: I think we should talk about TypeScript, because we both share a background in TypeScript. You've obviously done a lot of work with TypeScript and Total TypeScript, and you're a leader in that space. And I had adopted TypeScript pretty early, and I gave a talk or two at TSConf many years ago. One of the talks I did was about type systems and constraining types.

One of my favorite things about TypeScript is type narrowing, this idea that you go from a very broad type that could be anything, and then through type guards and type narrowing and runtime checks you can narrow the space and say: oh, this isn't just a string, this is a very special type of string. It's a constant. But I determined that through the type system. And there are a lot of parallels to that with constraints in your codebase, where you're constraining the space. If you think about category theory as well, you're constraining the number of possible types that can exist, and you're saying there's only one type.

For us, that's the framework I'm building, called Dune. It's not an open-source framework. The way I describe it to people, it's kind of like our internal Next.js for our Electron apps. But it comes with a lot of really, really restrictive lint rules, and the codebase is designed so that there's really only one way to do something. We make use of a lot of conventional patterns. Every feature has its own directory, as an example, and there's a thing that discovers features through a registry, crawling the codebase and stuff like that. But this conventional pattern and the lint rules make for an environment where it's actually very hard to write bad code.

And that both frees up your own mental capacity and means the agent doesn't have to think about that anymore. It's just: if I want to add a new feature, it goes into the new feature directory, all the code goes in there, and I'm not going to append to a god file. That was really the inspiration for those feature directories: the very first couple of versions of Grokbot were composed of eight god files, which were at least 10,000 lines long, if not longer. So I had to break it up into smaller pieces.

But it was just observing. Actually, that's another important part: observing how agents fail. Every time you see a mistake, every time you see something that could be done better, you step back and think: how do I turn this into a lint rule? How do I make it so that the codebase makes this impossible? And it comes back to my background learning TypeScript and type systems: how do I constrain the space so that I know precisely what I'm working with? I think there are a lot of parallels there.

Totally makes sense. It's funny that you mention TypeScript and god files in the same sentence, because TypeScript famously has a 25,000-line file.

Yeah, I realize the irony of that.

I don't know if they've rewritten that in Go. They probably have, haven't they? Okay, so the environment is important. You should watch your agent like a hawk to make sure that any mistakes it makes, you turn into things in the environment. And the benefit of the environment is that you're not overloading your agent in terms of rules, in terms of things it has to remember. It's just in the environment. So it stumbles into the rules and bounces off them and hits them at the right moment.

Where 2,500 PRs come from: chains of restaurants

So, okay, we still haven't talked about the 2,500 PRs. Where do those come from? You've built your trust ladder, you've worked on your environment, and you decide, okay, I now want to scale up. So what are the mechanics of that scaling? Are you initiating 2,500 chats per month? That can't be right. So are there any automated triggers that trigger stuff in your repo? How do you get the software factory triggering work by itself?

Right. I'll definitely say that the prerequisite to something like a very high volume of pull requests is the environment, the stuff we just talked about. I definitely would not have been able to do this if I had not spent the time thinking about the kitchen and the knives and the tools for my agents. So in a way, I think of this as: I've spent the time building one kitchen and one restaurant, and now I'm in a position where I don't actually have to be there anymore, because the environment... The same analogy works really well.

You've opened a chain of restaurants.

Exactly. It's like you're Gordon Ramsay and you've taught your executive chef all the tricks of coming up with a great menu, and the kitchen is set up really well, everything is just perfect, and you're now in a position where you can open your second, your third restaurant. I see each project I work on, each big chat, as a restaurant, and I have multiple of them operating at the same time. And I'm helicoptering between them, some more than others, depending on how in the loop I am.

The inner loop and the outer loop

But definitely, I think there are external triggers and context that those projects don't have, and for a long time I was the proxy for that. The best example I have: you have a project that's working on a feature, or you're trying to fix a bug, and you're getting bug reports. But the bug reports are going to things like Slack or Linear or X, and these are external systems that aren't connected to your inner loop.

So I like to talk about the outer loop and the inner loop. I don't know if I'm using the definition correctly, but to me, my inner loop is basically my agent engineers working on the code and building towards an intent, or a snapshot of my intent. The thing about that is that the snapshot can go stale. New information comes to light that I then have to be the proxy for, and transfer that context to my agent. So if you don't have these triggers pulling information back into the inner loop, you have to play that role where you're off in Slack or X or whatever, gathering context. You're getting context about bug reports, about feature requests, about something someone said about our backend infrastructure having some limitation. All that information, you have to ferry across to your agent.

So that's where I think tools like Grokbot are really good, because they help you automate the outer loop as well. And when you connect those two loops, it's very, very powerful, because all of a sudden your agents have the ability to get context for themselves. Just as a simple example, maybe you have the Slack MCP, or you have your own harness that you've built a Slack subscription into for a particular Slack channel. Now all of a sudden you can tell your agents: okay, subscribe to this Slack channel. Every time there's a bug report about something, go off and triage that thing. Go reproduce the issue using the verification skills that we've already spent time building, and all of the other skills we've set up so that I have a lot of trust. I have a lot of trust that these agents can actually go off and understand the bug, verify that the bug actually still exists on main, and that it wasn't something about the user's setup or their data, or maybe, I don't know, they didn't install a dependency or something like that.

Basically, I think creating those two loops and connecting them is a very important part of the job these days, especially if you're thinking about how to scale yourself. A big theme here is always thinking about: where am I the bottleneck in this process? Why do my agents need me to answer this question? I always like to think about that. And so I try to think about how do I actually get the agent to answer its own question, but not by hallucinating, not by guessing, with actual real data.

A lot of people talk about this idea of a company brain or a context graph. I feel like those terms are unnecessarily complex, or even abstract. To me, it's just about taking information that I would otherwise have to go and get and pass along myself, and teaching my agent how to do it. That removes me from the equation. And so how I arrive at 2,000 or however many PRs is the fact that I have all these loops set up. It allows me to open chain restaurants. I can really parallelize myself.

So no, I'm not sitting there creating 2,500 chats, of course. Cursor actually has a new feature called Projects, which are these coordinator agents. The coordinator agents are really good at delegating and not doing work of their own; they manage and supervise a list of tasks, almost, and they spawn subagents to go and do them. And so I'm just constantly feeding context, or teaching the agents how to get their own context, and then they're going off and doing the work for me.

And really, the last thing I'll say on this is that the big unlock for me getting to 2,000 PRs was starting from the question and working backwards: how do I get to the point where my agent can merge its own code? Because the obvious thing people ask me when I tell them I shipped 2,500 pull requests last month is: how did you review that? That's just a lot of PRs to review. Your team must hate you.

Do you mind if we go there in a second? Because I've got a good question about that.

Yes, absolutely.

Grokbot and Cursor Projects: the chief of staff

I want to deepen this analogy a bit, because it's great. Before, if you're manually initiating all those chats, it's like you're bringing the orders to your chefs manually. Whereas if you've got an agent doing the expo, sort of, then you're able to let it run itself. What does that concretely look like? You've got these Grokbots that are subscribing to channels, pulling in Slack messages, and it sounds like you have a couple of coordinator agents or chief-of-staff agents that monitor that or something. When you look at your computer to manage your agents, what does it look like?

Yeah, so this is, I guess, somewhat confusing, but we're working on simplifying and unifying. There's Grokbot (or you can use other tools as well, of course), but I largely think of these tools as your outer loop. These are tools like Grokbot that have connectors. I guess a lot of people call them personal agents. But they have connectors to things like your email, your calendar, Slack, all these different services, and they are a great source of pulling context into your work.

It's the same as with humans. If I was a manager leading a team of engineers... When I used to work at Netflix, one of the biggest things managers would talk about was this idea of context, not control, which funnily enough has so much carryover to the agents world. You can of course drive to the outcome you want through control, by micromanaging. But what you want is to provide context instead: teach the agent, teach your engineers, how to be self-sufficient, and then you don't have to micromanage them. So I see a lot of parallels there.

But yeah, Grokbot, concretely: I have some Grokbots that look at my Slack channels, look at my X or my emails or Linear. They have routines that subscribe, so they're constantly watching. And I'll tell them things like: watch for issues with bugs in the Grokbot desktop app, as an example, and whenever you find one, send it to my Cursor project.

One of the really cool things about Grokbot is that it connects to Cursor. Cursor has, like I just mentioned, the new feature called Projects. With a project, again, you get a coordinator agent that's in the cloud. It has its own computer. And all it really does is manage agents. It's like your executive chef, your chief of staff. It doesn't do the work itself. It delegates and orchestrates and manages the work of other subagents that report to your chief of staff. And it's basically responsible for driving the work forward, managing things and passing context to them.

So if you get a sudden burst of issues, let's say you get 30 issues at once in one payload or something, very quickly, the coordinator agent can figure it out and delegate.

Yeah, exactly. It gets the 30 or so payloads and spawns subagents, or a single coordinator agent. It can actually do a bunch of different topologies of agents. And it will figure out the best way to efficiently distribute the tasks to your team of agents.

So I use Cursor Projects a lot, and I also use Grokbot a lot. Cursor Projects are my inner loop and Grokbot is my outer loop. Grokbot takes all the external context and gives it to the projects, because it can actually just send messages to those projects. You don't even have to open Cursor. You can just tell your Grokbot: okay, create a project for this series of tasks, because they're all related.

As an example, maybe you've had a big burst of issues that are all about performance. Your app is slow, and they're all connected. Maybe some of them even have a similar fix. And you can certainly go off and spawn one agent per task. But then you've lost the thread between them, and you may duplicate work, or you may not really think about the higher-level problem. Sometimes when you solve bugs, it helps to have multiple bug reports that are slightly different, because it helps you zoom out and see: actually, when I looked at this one report, I thought the bug was here, but when I see the multitude of other bugs, it's actually up here.

Yeah, got you. So that's why you have so many agents in that loop, then. Because if a bug report comes in and you spawn a single agent to look at that bug report, that single agent will be duplicating work with other agents, if there are multiple bug reports coming in about the same thing.

Yeah.

Fascinating. That's really fascinating. Okay, so this endless series of triggers, coming from real users reporting real bugs, builds up and accelerates the factory, adds more orders in. Other than bug reports, are there any other sources that you use for pushing these PRs?

Gardening, routines and the buffer

Well, funnily enough, some of it comes from reading the code too. To clarify, the 2,500 PRs are obviously not 2,500 features. A lot of the work is actually spent on gardening, another term that I really love. I guess this is more important when you have a big team of human engineers that you work with. And this goes back a little to what we were talking about with the environment: setting up a really good environment doesn't just help you and your agents, but everybody on your team. Think of a new hire, who doesn't have a lot of context on all of your engineering practices, joining your team. If you have a really good environment, they can be productive from day one. They don't have to open a bunch of low-quality PRs. They can start turning out really good code. And, yeah, I think I lost my train of thought.

I've got a follow-up, which is: what are the mechanics of triggering? When do you trigger an agent to go and look at the code? Because some people might say, well, let's just do that every hour or something, or on a cron job.

Oh, yeah. I saw some of your recent tweets about that as well, which are great, about routines. I have some routines like that as well. Just another simple example: React has a lot of footguns, as I'm sure you're aware. And so I have an agent that's just constantly looking for bad patterns. The interesting thing about that one is that I don't actually tell it to fix the issue first. I tell it to append it to a document. And then every couple of days I look at it and I say: actually, these are all the same thing.

So you almost want a buffer, a queue. Sometimes that's actually more effective than just spawning off a lot of subagents to fix every single thing. Because when you're in pure execution mode, just trying to execute on the orders that are coming in very fast, you sometimes miss the big picture. Having a buffer forces you to think about the big picture, because you have these artifacts that you can look at as a human and use your own human judgment on (or I guess you can use an agent to do that as well). You give the agent and yourself a way to identify patterns that you might otherwise miss if you're only solving one bug at a time. And that's also really the benefit of having something like a chief-of-staff agent: it can see the forest in addition to actually doing the execution.

Fascinating. My brain is exploding a bit there with the chief of staff in the software factory. I might have to change some of the course I'm filming next week.

Review: tasting the dishes

Let's talk about review. This is the reply you get. Did you taste all 2,500 of those dishes as they swept past you? I assume the answer is a version of no.

Yeah. I think you don't want to be in a position where you're never tasting your food again. But for scale, you cannot be tasting every single dish that comes out of your kitchen, especially if you have multiple restaurants. So it becomes more about sampling and thinking about the processes. Maybe this is where the factory analogy is a bit more apt: as a quality supervisor in a factory, you can't look at every single item. You sample. You go in there every day and you look at the quality of the pull requests. You look at the code the agents are writing and you scrutinize it very rigorously, and you think about all the inefficiencies, the bad patterns the agents are falling into, and then you think about how to course-correct the environment, not that single agent.

Because if it was a one-off incident, it's fine; maybe there's nothing to fix there. But if you notice that multiple agents are having the same issue, they're taking the same shortcut, they're propagating the same workaround everywhere, that's a sign that you should go off and think about how to amend your kitchen or your factory: your skills, your constraints, your lints, your type systems, adjusting them so that the problem doesn't happen again. And when you do that enough times, you get to a place where the environment is so constrained, and it guides you so well, that you can just step away. That's the dream.

And I'll definitely say it's very hard to get to this point. I don't want to sell this as something you can just do easily by using pstack. It takes a lot of time and effort to think about your code and where you see your agents failing, and to think very thoughtfully and intentionally and set up guardrails and constraints so that they do the right thing by default. And to go back to the software factory analogy, this isn't a dark factory. The lights are on.

It kind of is, actually.

Is it? Yeah, well, it's dark in the sense that... Well, I think if my agents are merging their own pull requests, it has sort of become dark. I go to sleep; my agents now work 24/7. I have the equivalent of more than 10 chiefs of staff, each working on a different area. For example, I have one that's working on performance of the Grokbot desktop app. I have one that's working on fixing bugs that users report. I have one that's exploring rewriting it in a different language just for fun: reimagining what it would be if it was a native app. It's just a toy.

But the idea is that when you spend the time setting up your environment, you get to a point where... I review the pull request after it's landed. Full autopilot is something you can tell your agents to do in pstack, and that triggers this very intense, rigorous verification loop where it spawns a bunch of verifier agents for every pull request, and it will fuzz. Fuzzing meaning it will actually run the application, click around and try to use it like a real human, look for regressions, look for bugs in your implementation, try to find issues with the thing, and then it will fix them itself. It will do that again and eventually get the PR to a state where it can land.

So it is quite token-intensive. You can tune this, of course: instead of 10 verifier agents you might do one, or just tell the agent to verify its own work. But the key thing is that the verification part is really the key piece. That gives me a lot of confidence: verification plus the environment. The combination of those two things is what allows me to step away and say: agents, go off and merge your thing. I'll review it in the morning by looking at my commit history.

If I see problems, I go in and course-correct. I'll go in and revert or modify, add new lint rules and whatever. So it does take time to get to that point. But once you get there, oh, it feels so magical. I tell people: I'm sleeping so much better now. Because the very first day I turned on that sort of dark factory was very scary. I was like, ooh, what if I cause a sev? What if I break something overnight? And it took a lot of bravery, I think, to do that. But somehow I did it, and now I'm in a place where my agents are merging their own code while I sleep. So I think it's dark in that sense.

Yes, it's dark sometimes.

That's true, that's true. Because I think of a dark factory as almost like if you take the original definition of Karpathy's vibe coding, which is that the code almost doesn't exist, forget that the code might be a thing. I think your approach is totally different from that, which is that the code and the environment are essential. And if the code and the environment are bad, then you will get bad outputs: garbage in, garbage out. So I think this is a different thing. Maybe there's a dimmer switch or something. Some parts of the factory are dark, some parts aren't. This is why maybe the kitchen, the restaurant, is a better analogy. Because even as a restaurateur you still might go to your restaurants every now and then, take a peek in, taste the food.

Yeah, I think that's the idea of sampling instead of blocking. I think that's really important.

One-way doors and verifiability

What would you say to people who are... I guess you're obviously in a pretty security-conscious environment where you're working, very security-conscious. Maybe there are folks working in medical applications or law or finance or something. I think of that as: some PRs are kind of like two-way doors, which is you can merge it and then revert it. It's change you can walk back through. But there are some PRs that are one-way doors, that will cause data loss of some kind, that will do something that can't be easily walked back. How do you deal with situations where most of your PRs, let's say, are one-way doors? Is this something you just wouldn't recommend, or what do you think?

Yeah, I think that's a really good question. I think it all comes back to the quality of the verification that you're able to get out of your agents. And I think for domains where the work is verifiable, this is easier. And the one-way doors become two-way doors, in a sense. But if you're working on something that is extremely hard to verify programmatically, then I think you're definitely in a position where it's very hard to get to that point. So I do think verifiability of the domain is an important aspect of being able to do this. And software engineering is just one of those things where it's quite verifiable in a lot of cases, maybe not totally.

Other domains like mathematics are another example. Not all of it, of course, but some aspects of mathematics can be verifiable if you write a proof, for example. And so, yeah, I think it's a great question that I don't really have the answer to. And I think this is something the industry and us as engineers will have to figure out.

My hope and prediction for the future is that we'll see more and more interesting new agent-oriented programming languages. And one of the most fascinating ones that I've seen so far is this one called Bend, B-E-N-D. And that language is one where it kind of marries programming with proofs. There used to be a time where you actually had to write your proofs in a different language. And proofs, by the way, for those who aren't familiar, is this idea that you can formally verify that some code is correct mathematically, especially if you've written your code in a very functional-programming way. But for the longest time, you had to do that in a separate language, like Lean or TLA+, or I'm blanking on some of the other examples. Languages like that, where you would construct the mathematical proof and then use a solver, essentially, to check that you've covered all the cases, you don't have a race condition, or whatever.

So trying to sum up the question: if you are in a position where you can figure out how your agents can truly verify the work in a way that gives you confidence, you can actually have the PRs merged. Because if it compiles, if the proofs show you that it's correct, then why wouldn't you just merge it? But of course, not all domains are verifiable.

Yeah, it's a tough one.

Combining skill libraries

Okay, I think we've got to think about wrapping up, because we are nearly on the hour. Have you got something after this? I mean, I've got something after this. I'm going to give my son dinner, but...

I can go a bit longer after you.

Okay, let's go five minutes longer then. I think I just want to have one more question, which is: I want to ask how you see pstack and how you see skills in general. We talked about this before we went on air: people think of it as my skills versus your skills. How do you combine frameworks together? How do you use pstack with my stuff? What should you take from each one? Because I think I see skills as just derived from process, basically. They're just processes turned into words. And I would love to know how you recommend people take pstack and take my stuff as well and turn it into their own processes.

I think you shared a tip, actually, that I thought was very relevant, which is this idea that you go off and look at your previous transcripts, and you sort of mine for information of your own. Look through your own prompts to the agents where you correct them, where you have to constantly intervene, and take that higher-level learning and turn that into a reusable skill, so that agents stop repeating that mistake.

I think that pstack and your skills are very complementary. I totally agree with you that a skill is really just process. At the end of the day, a skill is just English or language. It's just markdown. And I think you can definitely weave them, combine them in a way that makes sense to you. But I do think that everyone should have their own set of knives. I keep going back to the cooking analogy, but it's so apt. Because every chef, when they go to a different job, when they go to a different restaurant, they bring their knives with them. The tools go with them. And so trust, to me, is really about trust in your own tools. And when you spend the time sharpening them and understanding them really, really well, you can do great things.

And everybody's skills and tool set is going to look different. Someone might find a lot of success combining your grill-me-with-docs or wayfinder skill with some of the execution skills in pstack, as an example. Some people might use more of your skills. Some people might use more of my skills. I think at the end of the day, it really just comes back to how much do you trust me and Matt. If you trust us both, of course, use our skills. But I also encourage you to look at your own transcripts. Tell the agent to look through all of the patterns that you've used, the times you've had to intervene, suggest turning them into lint rules or new skills.

The past chats, I often say, are like a treasure trove of context, because it's like the process materialized. It's the real process, not an abstract idea in your head. It's the actual thing. And you can actually see how it happened in practice and extract so much information from that. And there's so much signal that I actually turned half of that into a skill, a piece called Recall, which is exactly that. This was a pattern where I was working on a similar problem. Specifically, I was working on virtualization for the Cursor application, and there were a lot of bugs. And so every time I started a new chat, I was like, oh, there's so much good context from the last one, though. I want to bring it over to the new chat. How do I do that? And that's where the transcript came about. And then Recall was just a way for me to collapse and compress that workflow into a skill so that I didn't have to write a long essay every time: go look at all these chats, and then blah, blah, blah.

So I largely think of skills, especially as agents get more capable, as really encoding workflows. Skills from last year were really more about almost like implementation details: here are the exact script commands you should use. I think with the latest models, you can just delete those parts and really focus on the workflow. The skill becomes more like a series of steps, a series of your process. And I think over time, we'll see that skills get smaller and smaller, more compact. And yeah, they're very, very compatible. Or, if you want, why not read our skills and come and combine them into your own? Combine Wayfinder with poteto-mode and make your own custom mode. The thing I love about skills is that they're so malleable. You can do anything you want with this language.

Absolutely. There's nothing magical in them. They're just words.

Yes. And if there is any magic in them, it's just the words chosen and the phrases used and the thinking that's been done to take abstract process and turn it into language. And once that thinking has been done, then it's just there. It's available. It's on the surface. You just nick it.

Lauren, thank you so much. It's been glorious.

Yeah, this has been super fun. I really enjoyed talking to you.

Really, really enjoyed talking to you. And we can do it again. I'd love to do it again.

Yeah, I'd love to do it again. Absolutely. Yeah, we'll check in in, well, I don't know, in three... yeah, in one day. Let's do it. Let's do it. Part two.

Well, thanks so much. I'm going to close the stream here. Lauren and I will chat a little bit and stay here. But thank you guys so much for watching.

Of course.