Patrick wanted to play around with remodel ideas for the house. Pretty quickly you run into a problem: before you can move a wall on a plan, you need the plan.
Normally that means walking around with a tape measure, writing numbers on a notepad, and then drawing everything in some CAD tool you have to learn first. That's a whole weekend (or three) before you get to the fun part. Patrick wanted to skip it.
So the idea was simple. Scan the house with a phone, hand the scan to an agent, and ask for a floor plan.
Scanning
Patrick used Polycam on his phone. The phone got hot. The app crashed once partway through, so the scan had to be redone. Still, it worked well overall, and the result was good enough to hand over to Codex.
In the first post about it, Patrick was pretty clearly surprised this worked at all. Codex took the 3D file into Blender, pulled out the existing floor plan, and started generating remodel ideas. Something that normally needs a tape measure, pen and paper, and some CAD lessons had turned into a conversation.
That lasted right up until the agent started laying out rooms.
Why the scan was useful
The first Polycam export was an interior scan. This turned out to matter a lot.
The file was a GLB model, and it wasn't just a blob of geometry with a photo texture on it. It had separate, named objects: walls, floors, doors, windows, stairs, fixtures. So the agent didn't have to look at a picture and guess where a wall started. It could read the objects directly, group them by floor, and pull positions and bounds out of each one.
That's the whole trick, really. Extracting a floor plan from a point cloud or a textured mesh is a hard problem. Extracting one from a file that already says "this is a wall, it goes from here to here" is much easier.
Codex imported the model into Blender and kept an untouched copy of the original scan as a baseline. Blender makes sense here because it's scriptable. For this first pass the agent was the one doing the edits, not a person dragging stuff around in a desktop app.
On top of that baseline, Codex built a small workflow:
- One script audited the model, recording each object's type, floor, position, and dimensions.
- Proposed changes were written as a list of JSON operations, like remove this wall, move that object, add a wall between two points.
- Another script applied those operations to a fresh copy of the baseline and rendered the result.
The original scan never got touched. Want a different option? Change the list of operations and regenerate. That made the options repeatable and easy to compare against the existing house, which is a nice property when you're about to generate a lot of bad ideas.
Zones
The first round of ideas was colored blocks. Kitchen roughly here, living space over there, stairs somewhere in the middle.
Patrick posted two of the plans and said they worked for general zones, but the floor plans were "super wonky."
Editorial diagram of the workflow. It contains no floor plan, measurements, or details of Patrick's house. The plans in his public post are not reproduced here.
At the zone level, this was actually kind of useful. Colored blocks are a quick way to ask "what if the kitchen moved" without committing to anything.
The problem is that blocks don't tell you how anyone moves through a house. Where are the doors? Does the hallway go anywhere? Can you get to the bathroom without walking through a bedroom? So Patrick asked for real plans: walls, doors, bathrooms, stairs, furniture.
Codex drew those too. And they looked a lot more like floor plans. Unfortunately, looking more like a floor plan also made it much easier to see everything wrong with them.
Rooms
Some highlights from the private session:
- A powder room with three doors.
- A sink and a toilet occupying the same space.
- Stairs that dumped you straight into a bedroom.
- Rooms you could only reach by walking through other rooms.
- A bedroom with no windows.
- A toilet facing a wall.
The toilet facing a wall is funny. The three-door powder room is also funny. After a few rounds of this it gets less funny.
Patrick caught all of these just by looking at the drawings and imagining actually living there. The agent would fix one issue and then something new and weird would show up somewhere else. Whack-a-mole, but the moles are bathrooms.
It wasn't for lack of instructions either. Patrick gave the agent a list of rooms and explained how the levels connected. He corrected its assumptions about the entry, yard access, bathrooms, and stairs. When the broad options got too confusing, he narrowed things down to one scenario at a time.
More constraints did make the plans look more like plans. The errors were still obvious to a human reading them.
The underlying issue is that a plan can be dimensionally tidy and still make no sense as a place to live. A rectangle fits inside the footprint. Great. That says nothing about whether the door is somewhere useful, whether the stairs connect the right spaces, or whether you can stand in front of the sink without sitting on the toilet.
Codex could render a plausible-looking drawing much faster than Patrick could check all of those things.
There's another wrinkle worth calling out. In the session, the agent sometimes described a plan as checked or validated, and then Patrick found a basic flaw in it anyway. The agent had produced a drawing. Patrick was checking whether that drawing worked as a house. Those are very different tests.
Patrick's impression is that current models just aren't very good at room layout, maybe because they haven't been trained much on floor plans. That's a hunch from one experiment, not something that was tested here.
A blank shell
After several rounds, Patrick changed the ask. No more speculative layouts. Just give me the measured shell of the house, with nothing invented inside.
He also wanted to drag walls around himself, which Blender scripts don't exactly make pleasant. Codex suggested Sweet Home 3D, installed it, and created editable project files, including measured empty shells of the existing house.
This was a real shift in the workflow. Instead of the agent generating options for Patrick to reject, Patrick could open a 2D editor, grab a wall, move it, and see the dimensions while he worked.
The agent's job got smaller and more useful. It could still:
- Extract dimensions from the scan.
- Build the blank shell.
- Answer narrow questions about a specific wall or doorway.
The later part of the session shows Patrick actually working in these files: opening them, figuring out the controls, asking how to add doors, sending screenshots. When he asked the agent for layout suggestions again, he kept catching bad choices. So the division of labor stuck: the agent handles measurements, the human handles the part where people have to walk through doors.
The exterior scan
The day after the first scan, Patrick scanned the outside of the house too. The goal was to check the building footprint and poke at the idea of an addition.
Codex compared the exterior scan against the interior model and refined its estimate of the space available. That was a helpful second check.
It was also a lot messier than the interior scan. There were incomplete surfaces and stuff in the way. The private project notes are pretty clear that the scan and anything derived from it are good for concept work, not a survey or a construction document. Good enough to play with an idea. Not good enough to decide what can actually be built.
The verdict
Patrick's next-day verdict was blunt. GPT was bad at generating floor plans, but good at extracting measurements from the 3D files. The layout work was going to have to be done by hand.
So the part Patrick hoped to skip, actually designing the rooms, didn't go away. What did mostly go away was the tedious part before that: measuring the existing house and drawing it from scratch.
This was not a finished renovation plan. It was a better starting point: an editable shell measured from the scan, without walking around with a tape measure first. The session does not show how much Patrick used Sweet Home 3D afterward.