← Library

Experimentation

I’ve been building a Claude skill to help people across LawnStarter turn their ideas into experiments.

One of my priorities is to bring more of the great ideas from teams across the company into our experimentation program, improving results across the whole customer journey.

But the program is only a few months old, and teams are still building familiarity with some of the principles and practices that make A/B tests as reliable as possible.

Nevertheless, I really wanted people to start testing great ideas ASAP, without having to learn the whole process first.

And I definitely didn't want to be a bottleneck.

So now they can call this Claude skill and describe an idea in normal human language:

“I’d like to make our money-back guarantee more visible on our Google Ads landing page to see if it improves lead conversion.”

Then the skill helps with the parts that usually require a CRO expert, like:

CHECK THE EVIDENCE

It looks through quantitative and qualitative sources, including our data warehouse, customer feedback and discovery docs, for signals that support or challenge the idea.

More supporting evidence helps with prioritization. Contradictory evidence starts a conversation about refining or dropping the idea.

CHECK WHETHER WE CAN LEARN ANYTHING

Given the available traffic and planned duration, how big would the effect need to be for us to detect it?

If we can only detect a 50% lift, a tiny copy change is probably not a great use of that traffic. The skill flags that before anyone spends time building it.

HELP PEOPLE GET BETTER AT CRO

Every hypothesis needs an If / Then / Because. The “Because” needs at least one real signal specific to our customers or site. No “Because it’s a best practice.”

There’s also a library of 18 behavioral principles (Hick’s Law, loss aversion, social proof, Fitts’s Law…) to help people articulate why the change might work.

HANDLE THE DOCUMENTATION

This part makes me particularly happy.

The skill uses our definitions of metrics and funnel stages, so terms like “lead” means the same thing across tests, including how we query it.

It also captures the required details: more than 20 data points and five documents per hypothesis.

Every experiment enters the program through this skill, so documentation is part of the process from the start.

That consistency would be hard to maintain manually.

HELP SHAPE THE ACTUAL CHANGE

Other skills handle later stages of the test lifecycle, but this one already helps people develop initial treatment ideas.

It checks past tests for what has and hasn’t worked, and connects to Mobbin’s MCP to show the user how other products have approached similar problems.

There’s much more built in the skill and more coming.

But LinkedIn won’t let me write more. So I hope some of these examples can already inspire you. 🙂