Your reviews usually already contain the next round of product fixes, the next support FAQ and the next ad angle. But dumping a few thousand of them into a model and asking it to "summarise" produces exactly the sentences you can't act on: quality is good, shipping needs improvement, customers want more colours.
The better route is to turn reviews into structured data first, let AI do theme labelling and evidence collection, and let the team rank the results by frequency, impact and how much you actually control.
Decide what question you're answering
The same batch of reviews can support very different decisions. Pick one before you start:
- why low-star reviews are increasing;
- which problems cause returns;
- the reasons customers say they bought;
- which parts of the product description are unclear;
- which feature requests deserve a place in the next version;
- which customer phrases could become product page or ad language.
Ask a model to do product, support, advertising and market research at once and the themes become too broad to verify.
Step 1: build the minimum data table
Export from the platform and keep only the fields the analysis needs:
| Field | Example |
|---|---|
| review_id | Platform review identifier |
| sku | Product or variant |
| rating | 1–5 |
| review_date | Date posted |
| review_text | Body |
| country / language | Market and language |
| verified_purchase | Platform's purchase flag |
| return_reason | If your systems can join it |
Strip names, emails, order numbers, addresses and phone numbers. Before putting anything into an external AI tool, confirm your company's data policy and the service's data settings; if you can't confirm them, work on a redacted sample or in an approved environment.
Step 2: clean before you analyse
Review exports routinely contain duplicates, empty text, templated incentivised reviews, mixed languages, and ratings that contradict the text.
Do this first:
- de-duplicate on
review_idand on body text; - count empty and emoji-only reviews separately and exclude them from theme analysis;
- keep the original text; put translations in a separate column;
- group by SKU, market and month so different products don't merge into one conclusion;
- flag "five stars but complaining" for human review instead of trusting the star rating as sentiment.
Step 3: ask for fixed labels, not prose
Take a sample of 100–200 reviews and have a human define the first label set, for example:
- size doesn't match;
- material not as expected;
- difficult to assemble;
- damaged in transit;
- missing parts;
- product page unclear;
- positive: easy to use, portable, appearance, value.
Then run a fixed-format pass:
You are a customer review analyst. Label strictly from the review text; do not infer causes that aren't written.
For each review output: review_id, primary theme, secondary theme, sentiment, product part involved,
whether human review is needed, and the phrase supporting your judgement.
Use the supplied labels first; if nothing fits, mark new_theme. Do not merge labels on your own.
The supporting phrase matters. Without an evidence snippet, nobody can spot-check whether the model mislabelled. Keep only the short quotes you need for internal verification; don't republish large amounts of customer text.
Step 4: rank on three dimensions
The most frequent theme isn't automatically the most worth fixing. Score each one:
- Frequency: share of the relevant SKU's reviews;
- Impact: does it link to one-star reviews, returns, refunds or safety;
- Controllability: can the team improve it via product, packaging, page or support.
A problem appearing in 5% of reviews that makes the product unusable outranks a 20% colour preference. A simple formula is enough:
priority = frequency score x impact score x controllability score
The score orders the list; it doesn't replace a decision-maker. Safety and compliance issues escalate immediately, regardless of rank.
Step 5: convert themes into actions
Every high-priority theme gets an owner and a verification metric:
| Review theme | Action | Owner | Verification metric |
|---|---|---|---|
| Size misunderstood | Add a size comparison graphic and measuring guide | Product page | Related enquiries and return rate |
| Hard to assemble | Reshoot a 60-second assembly video | Content | Share of assembly-related negatives |
| Missing parts | Add packing scan and contents card | Warehouse | Missing-item ticket rate |
| Delivery time misread | Rewrite shipping copy and checkout notice | Ops | Shipping enquiry volume |
That's what stops the analysis ending as a nice report nobody uses.
Step 6: close the loop with support and the product page
Push high-frequency questions into the FAQ, then run them through the knowledge-base-plus-AI-draft-plus-human-approval pattern in how small Shopify teams automate customer support with AI. Product page problems go back into the fact templates in turning one product photo into a full Amazon and Etsy listing.
Re-run the same labels monthly and watch whether the share of a theme falls after a change. Don't delete old reviews to make the trend look better: trend analysis needs a stable baseline.
Quality checklist
- personal information and unnecessary order data removed;
- SKUs, markets and languages not incorrectly merged;
- label definitions and examples confirmed by a human;
- every theme retains a checkable evidence phrase;
- random spot-checks on high-impact themes and on
new_theme; - frequency uses the relevant review count as the denominator;
- every conclusion has an owner, an action and a verification metric;
- AI output does not stand in for product safety or compliance judgement.
Worked example: a laundry hamper with rising one-star reviews
Take a fictional collapsible laundry hamper sold in two sizes as an example. The question is narrow: why have low-star reviews on the large size gone up over the last three months?
After de-duplicating and dropping empty reviews, the team labels a sample by hand and settles on five labels: frame bends, smaller than expected, handle tears, colour differs from photo, damaged in transit. The fixed-format pass then returns rows like this:
review_id: R-1042 | primary: frame bends | secondary: none | sentiment: negative
part: wire frame | human review: no | evidence: "collapsed when half full"
review_id: R-1077 | primary: new_theme | secondary: smaller than expected | sentiment: negative
part: lid | human review: yes | evidence: "lid won't stay on"What a person changes before ranking:
- Spot-check the frame label. Some "bends" reviews actually describe the bag sagging, not the frame. Splitting that into its own label changes which team owns the fix.
- Promote the new theme. Several
new_themerows mention the lid, so "lid won't stay closed" becomes a sixth label and the batch is re-run. - Check the denominator. The frame complaints come almost entirely from the large size, so frequency is calculated against large-size reviews only.
The result is a short list: frame strength goes to the supplier with the evidence quotes attached, and the lid gets a product page note and a support FAQ while the supplier responds.
Prompt for turning one theme into a fix brief
Once a theme makes the priority list, a second short pass helps whoever owns it:
Below are the labelled reviews for one theme: [THEME NAME] on [SKU].
Summarise in under 120 words: what customers describe, when it happens (on arrival, after use,
during assembly), and which product part is involved. Quote at most three evidence phrases.
Do not suggest causes that aren't written in the reviews.
Reviews: [PASTE LABELLED ROWS]FAQ
How many reviews do I need before this is worth doing? The method works at any size, but with only a handful of reviews per SKU, read them yourself. The labelling pass pays off once reading every review by hand stops being realistic.
Can I run competitors' reviews through the same process? The labelling method works the same way, but keep it to internal analysis, stay within the platform's terms, and don't copy their customers' wording into your own listings.
Should I trust the sentiment column? Treat it as a sorting aid. The evidence phrase and your spot-checks are what make a label reliable, which is why "five stars but complaining" goes to a human.
AI is good at turning a lot of text into a checkable structure. The original value comes from your own SKU data, evidence and the results of the changes you make. Recording "what we changed and whether the metric moved" beats generating one more review summary.
Method compiled 2026-07.
