My Idea Machine Told Me No
Last month I built a machine to find my next product. It crawled the top 10,000 JetBrains plugins, pulled 18 months of reviews for the ones that made the first cut, and had a model score 40 finalists on demand, pain, whether people in the category already pay, and whether one person could credibly build a replacement in two or three months. The output is a ranked digest of product opportunities.
Its first real answer: nothing here. Two candidates surfaced, both failed the gates. An empty list. And I consider that a good thing.
Some backstory. I recently read Mark Pincus’s book Life at the Speed of Play (great read btw), and it got me thinking about ways to find latent demand, or opportunities that come pre-validated. His “failure machines” frame stuck with me: most new things fail, so build a process that makes failing cheap and fast. It pushed me to formalize some thinking I’d already been doing around this.
Validation is still the long pole on every new product idea. I have to build some small MVP, then find an audience for it, then convince them to come check it out, and the whole thing is only as good as the weakest of those steps. I used to think a month was fast, but this got me to push myself even further. What if I could find where users are already telling me their pain, in their own words, and what they want built to fix it? So I started writing little scripts and tools to go find them.
JetBrains plugins were one of my first attempts. I build a handful of dev tools for myself, so I figured the Marketplace might surface developer problems, and more importantly, problems that developers already pay to solve.
Just about everything I found in that directory that people were paying for was a subset of a larger service offering like Copilot, Windsurf, or Claude Code. The rest failed the same gates.
The good news is that my framework for saying no is working. I figured out in a few hours and 46 cents that JetBrains isn’t likely to be a good target audience for me.
So the new standard I’m holding myself to is two weeks or less from idea to verdict. A fake door landing page, five real conversations, and get to an answer: build or kill.
I’ve been applying the same data approach to a few other sources and one looks interesting. I’m building out a landing page and going to start talking to some potential customers to see how real it is. More on that one in a future post.
What would your failure machine check? Where are your users already telling you what they want, in their own words?
