"Reaction to this campaign: 82% positive." The number is clean and the meeting ends quickly. But if the repurchase rate dropped the following month, what was that 82% actually telling you?
The bottom line
Sentiment analysis (also called emotion analysis) is a method for automatically sorting what customers say into positive and negative. It's a useful tool, but a positive ratio alone can't decide anything. For it to lead to a decision, you need two more things: what the positive or negative sentiment is about (the aspect), and how it has changed since last time (the trend).
Sentence-level vs aspect-level
Let's read the same review two ways.
"The scent is really nice, but at this price for this amount, I'm not sure I'd buy again."
- Sentence-level analysis: One review gets one score. "Really nice" registers strongly, so it becomes one positive, and the hesitation about repurchasing disappears from the numbers.
- Aspect-level analysis: The review is split into units of meaning: scent (+), price (-), amount (-), repurchase (-). Now a decision comes into view: "keep the scent, revisit the size and pricing."
The latter is called 'aspect-based sentiment analysis.' LG Electronics also broke its product reviews down this way to analyze reactions by feature (Syncly blog). If you're handed sentiment analysis results, the first thing to ask is at what level they were split.
Five traps that distort the numbers
1. Sarcasm and slang. "Wow, amazing, shipping took 2 weeks" reads as positive if you only look at the words. There are also plenty of expressions whose direction can't be determined without context, like "legit so good" or "is this for real." Just randomly opening 20 posts classified as positive gives you a sense of how much is misclassified.
2. Sponsored posts mixed in. Product-tester and sponsored content is almost always positive. If those posts aren't filtered out, the positive ratio comes out higher than reality. Check whether posts with sponsorship disclosures were separated out.
3. What's the denominator? "82% positive" means something different depending on whether it's out of all mentions or only out of posts where sentiment could be determined. If half are neutral or undeterminable, 82% is just the ratio within the other half.
4. Words dropped at the collection stage. If you collect only via hashtags and captions, reviews spoken aloud inside videos are left out from the start. In short-form video, honest complaints often come out in speech rather than text, so the collection method can change the sentiment ratio (see The limits of hashtag analysis).
5. Looking at the level but not the change. 82% positive is bad news if it was 90% last month, and good news if it was 70%. Sentiment analysis says far more through a trend measured on the same basis than through a single figure.
How to read sentiment analysis so it leads to decisions
| If you report like this | Try changing it to this |
|---|---|
| 82% positive, 18% negative | 60% of the 18% negative is concentrated on size and price |
| Overall reaction is good | Praise for the scent holds steady; complaints about size up from last quarter |
| Negative keywords: expensive, too little | Attach 2~3 lines of original text like "at this price for this amount…" |
| This month's sentiment score | The change versus last quarter, measured on the same basis |
To sum up, the order is: split by aspect → find the aspect where negatives cluster → compare with last time → attach the original text. A one-page format for this flow is laid out in VOC analysis report.
What to ask when choosing a tool
- Does it split a single review into aspects for sentiment, or is it one score per sentence?
- Can the aspect categories be tailored to our product category?
- Does it filter out sponsored and advertising posts?
- Does speech inside short-form videos count toward sentiment analysis?
- Can we reopen the original text to verify?
Syncly's approach is to analyze e-commerce reviews and speech in TikTok, Reels, and Shorts videos at the aspect level, and to let you go back to the original text for every result. If you only need a light read on mention volume and positive/negative trends across Korean text channels, a self-serve subscription tool is enough (see Sometrend vs Syncly).
Alpaca's takeaway
The question for sentiment analysis isn't "do customers like it?" but "what do they like, and what makes them hesitate?" In your next report, next to the positive ratio, be sure to attach the one aspect where negatives cluster most and two lines of its original text.
🦙 That was Researcher Alpaca's research note. If anything is wrong or you'd like to know more, let me know via 'Contact Josh' below. I'll fold it into the next round of research.
Related reading
- VOC analysis report: how to write one that leads to decisions, with a template
- Competitor analysis: what you miss when you only look at sales and ads
Sources
- LG Electronics case (Syncly blog) — aspect-based sentiment analysis
- Syncly website — analysis of customer reactions in reviews and videos
