The most interesting number in AI and product teams this week was not a model benchmark. It was a count of humans in a growth experiment. One.
Figma has an agent that designs and runs growth experiments with a single human checkpoint. On its first experiment, it reportedly found a big win.
What Figma built
The details come from an episode of How I AI on Lenny's Newsletter, where Claire Vo talked with Ezra Mechaber, a growth product manager at Figma. His stated goal is to be "lazy in the right places", which is the best description of good automation I have heard this year.
When Mechaber joined Figma, he found almost nobody outside engineering using AI for real work. He and a couple of other PMs turned his personal Claude Code setup into PM OS, a shared repository of 31 skills. More than 60 PMs at the company now use it.
The second piece is AutoXP, an automated growth PM. It runs growth experiments with one human checkpoint, and it found a significant win on its very first run. He also uses Codex for bug bashes and has a real-time writing assistant he built himself.
The shared repo is the real story
AutoXP gets the headline, but PM OS is the part most companies should copy first. One person's private productivity hack became shared infrastructure, with skills, hooks and private sub-repositories for sensitive work.
That matters because AI adoption inside companies usually dies in the gap between individuals. One person builds a brilliant setup. Nobody else can see it, nobody maintains it, and it leaves the company when they do.
A shared skills repository turns tacit knowledge into something versioned and reviewable. When a PM improves the experiment brief template, everyone gets the improvement on the next pull. It is the same principle as a design system, applied to how work gets done.
At Madison AI, we went through the same transition. The skills that mattered were not the clever ones. They were the boring, repeatable ones, written down well enough that someone else could run them on a Tuesday morning.
Where the one human should sit
A growth experiment has roughly five steps. Pick a hypothesis, design the variant, ship it, read the result, decide what to do next. With one human checkpoint, the design question is which of those five gets the human.
Most teams would instinctively put the person before shipping, as an approval gate. That protects the brand and the product from embarrassing variants, and it feels safe.
I would argue the more dangerous step is reading the result. An agent optimising for a metric will find ways to move it. Some of those ways are real improvements. Others are noise, novelty effects, or changes that move the measured number while hurting something nobody measured.
I wrote about this failure mode in your agent will cheat to hit the number. A human approving the variant does not catch it. A human interrogating the result does.
More experiments means more false wins
There is a statistical trap here that automation makes worse. If you run one test at a 95% confidence threshold, you accept a small chance of a false positive. Run twenty tests, and you should expect about one "win" that is pure chance.
An agent can run far more than twenty experiments. That is the point of automating it. So the number of false wins rises with velocity, unless the system corrects for it with stricter thresholds, holdout groups or replication of every positive result.
The first-experiment win at Figma may well be real. I would just want it replicated before anyone builds a roadmap on it. The faster the experiment machine runs, the more discipline the reading step needs.
What I would put in the brief
If I were setting up a growth agent this quarter, I would write three things down before it runs a single test. First, the primary metric and two guardrail metrics it is not allowed to hurt, such as retention and support tickets. An agent with no guardrails will trade them away.
Second, the rules for declaring a result. Minimum sample size, minimum run time, and an automatic replication of every win on a fresh cohort. Write it as policy, not as a suggestion in a prompt.
Third, the human checkpoint, placed at the decision to scale. The agent can generate, ship and measure within agreed limits. A person decides whether a result changes the product for everyone.
That brief is the real product spec. I made the case that evals are the new product spec, and experiment rules are the growth version of the same idea.
Being lazy in the right places is a skill. The hard part is deciding which places those are, and it is the one decision you cannot hand to the agent.