December 24, 2025
Experiments vs Puzzles
Lemme start with some informal definitions, off the top of my head:
- EXPERIMENT—An experiment is an attempt to support or refute a hypothesis. Proving or disproving the hypothesis are both equally successful outcomes.
- PUZZLE—A puzzle is a finite game in which the player attempts to transform a current state into a pre-defined future state. Success or failure depends on whether the player produces the future state.
Experiments and puzzles are VERY MUCH NOT the same thing.
BUT!
People seem to conflate them.
When I talk about running experiments in a business, it seems like most people think I’m talking about solving puzzles.
It’s like they want to start off with a predetermined future state in mind and then puzzle and puzzle until their puzzlers are sore, trying to figure out how to produce that future state.
An experiment, on the other hand, would start by presenting a loosely held, falsifiable hypothesis and then find a credible way to prove or disprove it as quickly as possible.
Signing up to solve a puzzle in a business context is based on an implied desirability of the future state. This creates all kinds of sunk costs, which is bad, but more importantly, it presumes that the future state is actually valuable.
For example, “Can I figure out how to get more engagement on LinkedIn?” is not an experiment. It’s a puzzle. And it presumes that more engagement on LinkedIn will create business value.
Will it? Maybe, maybe not.
An experiment, on the other hand, would start with a hypothesis in which you have little emotional involvement. Which is to say, you would be equally delighted to prove or disprove the hypothesis. And the faster you disprove it, the better.
Why? Because...
It’s an opportunity to move on to a better hypothesis.
Stop doing puzzles and start running experiments.
Yours,
—J