The Problem Was Never Lack of Information
For a long time, my product research workflow looked the same as everyone else's.
Open dozens of tabs.
Browse product pages manually.
Compare prices.
Read reviews.
Take scattered notes.
Eventually, I’d end up with a vague sense of the market — which products felt overpriced, which features kept appearing, which brands seemed more premium — but very little I could actually validate.
The more listings I looked at, the worse the problem became.
At a certain scale, the issue isn’t access to information anymore. It’s information overload.
Humans are surprisingly bad at comparing hundreds of products side by side. After a while, everything starts blending together: titles, features, ratings, positioning, reviews.
That was the point where I started rethinking the entire product research workflow.
Not the AI part.
The workflow itself.
I wanted to see what would happen if AI wasn’t treated as a “product recommendation machine,” but as part of a larger research system.
So I picked a niche that seemed perfect for testing this idea:
ergonomic desk accessories.
Monitor arms, laptop stands, keyboard trays, cable organizers, wrist rests — the kind of category that already has thousands of listings, strong competition, and a huge volume of customer reviews.
Mature enough to contain patterns.
Messy enough to still hide opportunities.
The Real Bottleneck Was Data Collection
At first, I assumed the hard part would be prompting the AI correctly.
It wasn’t.
The hardest part was getting usable data out of the web in the first place.
Traditional scraping workflows introduce a surprising amount of friction: selectors, schemas, pagination rules, broken extractions, inconsistent formats.
None of those problems are individually difficult.
But together, they constantly interrupt the research process.
And product research is fundamentally a momentum-driven activity.
Every time you stop to debug a scraper or fix extraction logic, you lose context.
Eventually, the workflow stops feeling like research and starts feeling like maintaining infrastructure.
So I simplified the approach entirely.
Instead of building a complicated scraping setup, I focused on one thing: getting structured data into a CSV as quickly as possible.
I used Newise to extract listing data directly from marketplace pages, including:
- product titles
- pricing
- ratings
- review counts
- feature bullets
- partial review text
The workflow itself was almost boringly simple.
Open the category page.
Select the product grid.
Let pagination run.
Export the dataset.
No custom selectors.
No schema configuration.
No code.
What mattered wasn’t automation by itself.
What mattered was preserving continuity.
The workflow never broke my concentration.
That ended up being more important than I expected.
AI Was Most Useful When It Stopped Trying to Give Answers
Once I had the dataset exported, I uploaded the CSV into Claude.
Interestingly, I didn’t start by asking: “What product should I sell?”
Instead, I asked the AI to process information at scale.
Things like:
Analyze this ecommerce dataset and identify:
1. recurring customer complaints
2. common positioning patterns
3. pricing clusters
4. repeated feature claims
5. differences between high-rated and low-rated products
Focus on patterns that appear repeatedly across listings and reviews.
The results were much better than I expected.
Not because the AI generated some magical business insight.
But because it became extremely good at identifying repetition.
That changed how I thought about AI-assisted research.
Most people expect AI to replace judgment.
In practice, AI works far better as a system for compressing large amounts of information into recognizable patterns.
It excels at:
- clustering
- summarization
- comparison
- repetition detection
- anomaly spotting
Tasks that would normally require hours of manual reading suddenly become manageable in minutes.
That was the moment I started mentally splitting the workflow into separate responsibilities.
Humans ask questions.
AI processes scale.
Reliable tools handle structured data collection.
That division of labor became the most important insight from the entire experiment.
The Most Valuable Insights Came From Repeated Signals
One of the first things I noticed was how repetitive the market positioning had become.
Almost every listing used the same language:
- ergonomic
- adjustable
- premium
- comfortable
- space-saving
At first glance, the market looked differentiated.
But structurally, most brands were saying almost identical things.
The interesting part came from the reviews.
The phrases customers repeated most often were completely different:
- easy to assemble
- feels stable
- doesn’t wobble
- cable management
- doesn’t scratch the desk
That gap turned out to be surprisingly revealing.
The market was optimizing for marketing language.
Customers were optimizing for friction reduction.
Those are not the same thing.
And this kind of disconnect is difficult to spot through casual browsing because no single product page makes the pattern obvious.
The pattern only emerges once the information becomes structured.
Another thing that stood out was how repetitive negative reviews were.
The same complaints appeared over and over again across different products: poor stability, frustrating assembly, weak materials, inaccurate dimensions.
None of these issues were isolated.
They were systemic.
And systemic complaints are often far more useful than isolated compliments because they expose where customer expectations consistently break down.
From a product strategy perspective, that matters enormously.
It directly influences:
- positioning
- feature prioritization
- differentiation
- landing page messaging
- pricing psychology
I Stopped Thinking About AI as a Tool — And Started Thinking About It as a Workflow Layer
By the end of the experiment, my biggest takeaway wasn’t that AI had become incredibly powerful.
It was that the bottleneck had shifted.
A few years ago, the difficult part of research was analysis.
Now, the difficult part is moving information efficiently between systems.
Modern product research no longer feels like browsing the internet.
It feels more like building and analyzing datasets.
That changes the structure of the workflow itself.
The most effective setup isn’t handing everything over to AI.
It’s assigning different types of work to the systems best suited for them.
Humans are still best at:
- asking strategic questions
- understanding context
- evaluating tradeoffs
- making final decisions
AI is best at:
- processing scale
- identifying repetition
- summarizing large volumes of text
- comparing patterns across datasets
Reliable data tools are best at:
- collecting structured information consistently
- reducing workflow friction
- maintaining research momentum
Once I started thinking about product research this way, AI became much more useful.
Not as a replacement for thinking.
But as part of a collaborative system.
Final Thoughts
Going into this experiment, I assumed the most important part of AI product research would be prompting.
Now I think prompting is probably the least interesting part.
The real leverage comes from transforming messy web information into structured, AI-readable data.
Because once webpages become datasets, entirely new kinds of analysis become possible.
And that changes product research from a browsing activity into a systems workflow.
