Here's the setup. You've got a few coding agents running in iTerm. Each one gets a terminal window or a tmux pane. Every tool call dumps output into the scrollback, and the messages that actually tell you what an agent is doing (yours, and the agent's replies) get buried under all of it. So you scroll. Then you scroll in the next pane. Then you forget which pane was waiting on you.
Patrick wanted one place where he could watch several agents, jump in when one needed him, and get back to his own work without digging through scrollback. That want turned into pi-deck, then Task Factory, then Neon Pilot. Today he uses bb.
It's easiest to read these as one experiment with four versions. Each one changed what the interface was built around. First it was conversations. Then it was tasks. Then it was features the agent could add to the app itself. The last change was to stop building the base and pick one that already worked well.
Pi-deck: get Pi out of the terminal
Pi-deck started as a web app for your phone. Patrick ran it on a machine near his projects and reached it over Tailscale. The README describes his Mac mini setup. A browser was the fastest way out of iTerm that didn't mean building and maintaining a separate client app.
The mobile interface, rebuilt from pi-deck commit ab0e8e1 with made-up sample messages. The UI code is real. The sessions aren't Patrick's.
It wasn't just terminal text piped into a webpage. The server created Pi SDK sessions, and one workspace could hold several independent sessions. That meant the UI got structured events instead of a stream of characters, and it could do useful things with them. A tool call could show its command or file path next to the result. File reads got line numbers. Edits got diffs. The transcript renderer did a lot of work so you could follow what the agent was actually doing.
There were controls too. You could steer a running session or queue a follow-up. You could attach an image by pasting or dropping it into the composer. You could pick a model and thinking level per session. Workspace and draft state lived on the server, with SQLite storage and WebSocket updates, so the phone could pick up a session that was already running.
The browser fixed the "how do I reach it" problem. But Patrick found a browser tab awkward for long sessions. He moved pi-deck toward the desktop and started splitting transcripts into panes. The split tree could put one conversation next to another, each with its own controls.
The pane phase, rebuilt from commit 082822e with sample data. The split control is real. The conversations are invented.
If you squint, this is a problem. The project set out to replace a pile of terminal panes, and here it was, arranging long agent transcripts into panes. Nicer panes, sure. Still panes. The public history shows he later removed the pane system in favor of tabs. So the panes don't appear in the last public version, but they're a big part of the story, because they showed where this approach was heading.
Meanwhile, a different idea had already shown up inside pi-deck: Jobs. A job was a Markdown record that moved through backlog, planning, ready, executing, review, and complete. That's a different question. Panes ask "how many conversations fit on my screen?" Jobs ask "what work should move next?"
Task Factory: the human is the slow part
Task Factory took the jobs idea and built a whole app around it. Patrick was thinking about the Toyota Production System: one production line, with work moving down it as cards. The repo states the premise plainly: the human is the bottleneck. Agents can produce code fast. Patrick still has to read the plan and check the result. Ten agents writing code at once doesn't give him ten times the attention. It just gives him a longer review queue.
A public Task Factory screenshot. Foreman is on the left, ideas are on the right, and the production line runs along the bottom.
Here's how a task moved. It started in the backlog while an agent wrote a plan. Patrick reviewed the plan, then moved the task to Ready. A server-side queue pulled the oldest ready task whenever there was room. By default, "room" meant one task at a time. That's the whole idea in one setting: let the backlog fill up as much as you like, but keep the line that's actually running narrow enough for one person to review.
A task was more than a title on a board. The task record could hold a plan, acceptance criteria, model choices, attachments, skills, and usage history. The agent had to send an explicit completion signal to finish. That sounds small, but it matters: an agent stopping to ask Patrick a question is not the same as an agent being done. After a task ran, skills could do a code review or checkpoint the work. Then the app wrote a summary against the acceptance criteria for Patrick to check. That summary helped with review, but it was still the agent grading its own homework.
The task view from the public repo: the original request, agent transcript, post-execution skills, and generated criteria summary in one place.
This worked well for producing code. It fit less well when Patrick wanted to research something, talk through a half-formed idea with an agent, or do anything that wasn't a code task. Task Factory did grow a Foreman agent that could brainstorm, research, draft task cards, and make HTML artifacts. But everything still led to the production line. When your tool is a factory, every idea starts to look like a work order. Patrick wanted an agent workspace that could handle more kinds of work, so he moved on.
Neon Pilot: let the agent change the app
Neon Pilot opened things up again. Patrick liked the multi-thread desktop UI in Codex and the small, flexible harness under Pi, and he combined the two in a desktop app. It had conversations that persist, background work, a knowledge base, and a workbench for files and artifacts. The project README covers those pieces. The part he cared about most was the extension system.
The Extensions image published with Neon Pilot's site. It's a staged product illustration, not a capture of Patrick's setup. It shows how extensions can add pages, tools, commands, and settings.
Patrick has written about self-extensible software before. His recipe has four parts: a coding agent inside the app, a runtime, an extension API, and docs the agent can read. The docs matter as much as the API. If the agent can read the contract, it can build a new tool instead of guessing how the app works. Neon Pilot's extension guide spells out the loop: ask the agent for a feature, it writes an extension, and the app loads it.
That changes what "adding a feature" means. An Excalidraw input, an artifact view, a new command, a tool for one specific project: none of these have to become permanent core code. The extension SDK exposes places in the app that an agent can build against. Task Factory's answer was a queue. Neon Pilot's answer was "the agent can reshape the workspace for whatever you're doing next." That's a much bigger answer.
Then the scope grew. T3 Code helped make one interface over several coding harnesses feel like a normal thing to want. Patrick wanted Codex and Claude Code in the same workspace next to Pi. That's a reasonable thing to want, and it's also a lot more engineering. The frontend had to represent different runtimes cleanly. Patrick ended up doing a lot of manual QA: fixing styles, sorting out basic UI behavior, and deciding which features belonged in the core. On top of that, he had little interest in the work of packaging and distributing a polished app. All of this was eating time he wanted to spend on experiments.
bb: pick a base and build on top
Patrick found bb while working through that problem. bb already organized work into threads you can steer, across different agent providers, with a desktop interface and a plugin system. In his view, its UI was better done and the app more stable than what he'd built, so he switched to it as his harness. That's his judgment from using both, not a benchmark.
A public screenshot from the bb repository. bb is an existing project. Patrick adopted it and contributes to it.
The self-extending idea came along with him. bb's plugin architecture was already there when he arrived, and he started building on it. His bb plugin collection includes tools like Excalidraw and agent analytics. Excalidraw was also a feature in Neon Pilot. So the same kind of tool moved to a different app, and Patrick didn't have to maintain the whole app to use it. He's also made upstream contributions to bb.
So who owns the harness?
In his Personal Agent post, Patrick wrote, "You need to own your own harness." He still thinks that. What changed is what "own" means.
Owning it means he can change it. He can add a tool, give an agent a new UI, try another runtime, or send a missing feature upstream. It doesn't mean he has to write every line of the core himself. A good general-purpose app also needs frontend styling, a settled core experience, packaged releases, and support for other people's setups. Patrick found he just doesn't care much about those jobs. They are real work, just not the work he wants to be doing.
Across the four projects, the interface was built around conversations, then tasks, then extensions, and finally someone else's well-built base. The thing Patrick wants to spend his time on now is the layer on top: extensions, and the experiments they make possible. The question he's chasing is how far agents and LLMs can go when the tool around them is open to change. He gets more time for that question when he isn't also polishing buttons.