Why does AI get the size and position wrong when I ask it to add something to my garden plan?
Ask a consumer AI tool for a 4m x 3m pergola against the garage wall and you will get a picture of a pergola, not a 4m x 3m pergola. More precise prompting makes it worse.
What it cost us
Roughly forty hours of design work invested in geometry that could not be built from, and a budget calculated from quantities that were fiction.
What we were trying to do
We had a measured plan of the garden with dimensions written on it, and a tool that produced better looking images than anything we could draw by hand. We wanted a four metre by three metre pergola against the garage wall, so we could see whether it worked before committing to it.
That is the whole ask. It is the first thing almost everybody tries.
What we tried
We asked for it. Then we asked more precisely, on the assumption that the problem was our wording. Then we uploaded the measured plan so the tool could see the real dimensions. Then we tried editing the existing image rather than regenerating it, on the theory that small corrections would converge on the right answer. Then we tried a dedicated AI garden design app instead of a general one.
Over and over. We never once got a four metre by three metre pergola against the garage wall.
Each remedy failed differently, and each failure is worth naming because you will reach for all of them in the same order we did. A more precise prompt failed because dimensions in a prompt are text conditioning, not measurements. Iterating on the image failed because each edit re-synthesises the region rather than preserving it, so drift compounds across turns and the fifth attempt is further from the truth than the first. Uploading the measured plan failed because the model reads a drawing as an image to imitate stylistically, not as a coordinate space to work within. The dedicated app failed because it wraps the same generative models underneath.
What went wrong
The pictures were good. That was the problem.
What we did not have, and did not realise we did not have, was any mechanism by which the dimensions on our plan could reach the picture. There was no point at which the number four metres was ever applied to anything. We spent weeks designing against geometry that had been invented, and we only found out when somebody asked us what the actual measurements were and we could not answer.
The failure is invisible at the moment it happens, which is what makes it expensive. A render that is wrong by two metres looks exactly as convincing as one that is right. Nothing in the output flags which parts were measured and which were guessed, so the error survives every review you do of it and only surfaces when the plan meets a tape measure or a quote.
“The measurements are not to scale - like with the planting plan they are a drawing of measurement rather than architecturally drawn plans with a clear scale”
Urban Plot, UK garden design practice — read the source
Interest to disclose: Urban Plot sell the garden design service this problem prices at GBP 2,000 to GBP 20,000, so they benefit commercially if AI output is judged unfit. Quoted verbatim and confirmed against the source page, but they are not an independent assessor.
Why it happens
- 1Why is the pergola the wrong size and in the wrong place?Because the model generated a plausible image of a garden containing a pergola, rather than placing a defined object at defined coordinates.
- 2Why did it generate a plausible image instead of placing an object?Because a diffusion model produces pixels conditioned on text and reference images. It has no object model, no coordinate system and no persistent representation of the scene.
- 3Why does stating 4m x 3m not constrain it?Because there is no unit system in the model. The string 4m x 3m correlates loosely with how things captioned that way looked in training data. It is a style hint, not a measurement.
- 4Why is there no unit system in the model?Because it only ever learned to reproduce the statistical appearance of captioned pictures, and captions almost never carry reliable measurements. Nothing in the training objective rewards getting a metre right, so no internal representation of a metre is ever formed to apply the number to.
- 5Why has nobody fixed that?Because supplying real units means adding a second, non-generative channel that the model is conditioned on. That is an architectural change rather than a tuning change, and consumer image products are built around the single text-to-pixel path.
What actually works
Stop asking the image model for geometry at all.
Build the layout to true dimensions in a free modelling tool. Export a plain grey view from a fixed camera position. Then use AI only to restyle that view, without letting it move anything. We call it the greybox-first method, and it takes an afternoon to learn.
The AI is still doing the job you wanted it for. It makes the picture look real, which is the thing it is genuinely excellent at. It is simply no longer the thing deciding where the pergola goes or how big it is. That decision moves to a tool that understands what a metre is.
The check that makes it trustworthy is boring and takes two minutes: pick three dimensions you specified, measure them off the finished image against a known reference in the same frame, and record whether each one held or drifted. We call that the tape check. Do it on every render, because whether a given image model preserves the grey view's geometry when it restyles is a property of that model, not of your method, and it changes without warning.
- It does not make AI output survey accurate. It makes it dimensionally honest.
- It assumes you have taken, or will take, real measurements of your own site.
- It does not produce construction drawings or replace a structural, drainage or arboricultural professional.
- It applies to consumer AI tools, not professional CAD workflows.
- It is proven on one garden so far. Treat it as a method to test on your own site, not a guarantee.
Words used above
- To-scale
- A drawing where every distance is a fixed proportion of the real distance, so measurements taken from it are true.
- Greybox
- A plain untextured 3D model showing only shapes and sizes, used to fix geometry before any visual styling is applied.
- Diffusion model
- The type of AI that generates images by refining noise into a picture. It works in pixels and has no concept of metres.
- Setting out
- Transferring positions from a plan onto the actual ground with tapes, pegs and string before building.
The explanation is free. The instrument is paid.
You now know why it fails. Here is what fixes it.
- CourseGet AI to Draw Your Garden to Scale from Your Own Measurements — Ninety minutes, ending with a photorealistic view of your garden built from your own measurements, plus a tape check that tells you which dimensions held and which drifted.
- BriefingCheck in Ten Seconds Whether an AI Garden Image Is To Scale — Know before you spend whether the image contains a single real measurement, and have it written down for the partner and the landscaper.
- BlueprintBuild a To-Scale Garden Layout Maker — The full build spec, screens, data model and acceptance tests, if you would rather build the tool than learn the modelling step.
Free trial, then paid. Everything above is inside The AI Garden Project.
Published 25 July 2026