I love you (0.7): why sentiment analysis sometimes isn’t enough

Ever heard anyone say “I (0.7) love you” or “I (0.1) really, really hate the idea of going to work today ? How about “I (0.2) don’t like spinach”. We’re willing to bet you have never heard statements of sentiment qualified by a number. That’s because we just don’t use numbers to describe emotional states. Yet sentiment analysis, ascribing a positive or negative number to a piece of text, is an established way to “measure” emotion, for example in social media mining or survey open ends.

It’s true that it is useful to have some numeric measure to gauge the direction of emotionality in text, but a single sentiment number is often a gross oversimplification of what may be expressed. It can’t capture the nuances expressed in a longer piece of text.

Take hotel reviews. You loved one aspect of the experience, but hated another – so what is the average of love and hate? Neutral? 3/5? And that is indeed what happens when you use sentiment analysis on longer pieces of text.

For example, this review is clearly very negative (and gets a sentiment score close to 0.0):

The rooms are far too small in this hotel for a family of four. All beds were jammed tightly together. We’ve never experienced this issue in any other of their hotels. We ended up having to pay for an additional room which was 3 times the price of the original room. terrible value at the end of the day….very disappointed.

And this one is very positive, scoring at the other end of the scale:

Great location, great facilities, very polite and professional staff who were also very helpful. lovely food, and the rooms and restaurants were very clean and tidy. Overall we had a great stay and would definitely stay again.

A typical (genuinely) neutral review might read:

Stay was OK, nothing bad, nothing special.

But what about this one, which also scores ‘neutral’:

We had a thoroughly enjoyable stay. As ever, the staff are brilliant and you always get what you’re promised room wise. On the downside, we didn’t expect quite a lot of steps into the hotel which can be quite difficult, indeed scary to negotiate with a heavy suitcase, and disappointingly the air conditioning didn’t work properly.

Really, it’s not neutral at all. The sentiment analysis is being pulled in two different directions – both positive and negative.

We conclude that while useful as a directional indicator, and especially when comments are only either positive or negative, sentiment analysis has its place as a general measurement – but it can mask or even ignore the real emotional content of a piece of text or an image.

To get to the deeper content of text or images we have developed two types of text and image analysis that allows you to really understand the deeper emotional content. The first is “emotionality analysis” (EA). EA analyses the emotional content of text so you can tell what is being expressed within it. Below is a typical analysis of emotionality of text:

This shows the EA of text items that scores 3.0 on our sentiment analysis, which is the middle of the scale (and therefore typically labelled ‘neutral’. It shows that there is a fair amount of fear, sadness and so on present, but overall people are lower than average in their EA (the middle number of 19%). EA emotionality is typically between 22-25% for “neutral text”.

The second type of analysis tells us the relative density of of positive versus negative sentiment within the text (the blue positive and red negative blobs to the right). We can see from this that positive emotions are nearly double the level of negative emotions, but crucially both are demonstrably present. Therefore, it tends to the middle of the overall sentiment scale – but that doesn’t capture the essence of what’s going on.

The fundamental point is that this analysis tells us not just the ‘overall’ score, but the mix of positive and negative emotions that are present in more complex comments such as our hotels example. Returning to the comment we saw earlier:

We had a thoroughly enjoyable stay. As ever, the staff are brilliant and you always get what you’re promised room wise. On the downside, we didn’t expect quite a lot of steps into the hotel which can be quite difficult, indeed scary to negotiate with a heavy suitcase, and disappointingly the air conditioning didn’t work properly.

Embedded in this simple bit of feedback are emotions of trust, joy, surprise (not in a good way), fear, and sadness. Positive and negative emotions are both present, tending to the positive overall, but getting a ‘false’ neutral score on overall sentiment which in this instance would be better labeled ‘both positive and negative’.

As you can see EA gives a much more refined analysis of text or images. It can also pick up contradicting emotionality in a piece of text than can be “averaged out” by sentiment analysis. There are strong emotional themes in the shown above but the sentiment score of 3.0 (on a 6 point scale, in this instance) does not indicate that. Sentiment analysis will not show that there is a significant level of anger in a piece of text nor that despite this, trust is the dominant emotion. Sentiment analysis is useful as a general high level tool but it simply can’t encapsulate the nuances of emotion within a piece of text or an image.

All this is crucial in understanding the all important multifaceted aspects of brand feedback.

Analysing and classifying the emotional content of large scale text via EA – call it sentiment 2.0 – and pulling out the themes that drive these emotions is bread and butter for our automated analytics platform.

This is exactly what Signoi’s clients are doing across a range of categories, from finance, technology, FMCG, through to telecoms and leisure.

If you want to try our advanced emotional analytics approaches on text or imaregy, please contact us at andy.dexter@signoi.com