This article by Bill William Comcowich made some interesting points about automatic sentiment analysis.
Although auto sentiment analysis systems have been getting better, they need to be trained (if that is an option). They also need monitoring.
Bill suggests a combined approach. When you need quick results auto sentiment analysis is helpful. But be pragmatic about the results. Bill asserts that human analysis is more accurate. If volumes are large this level of human-checking can be time-consuming and cost prohibitive.
Many of media monitoring tools enable users to double check the auto sentiment analysis. I have worked with the Brandwatch tool for some years. I have cut the time spent validating sentiment by sampling a selection of the clips. This is a technique I would like to share.
The tool enables users to select a random sample of clips to check. We have tried various permutations and this is what we found. If you analyse a sample of 400 clips or 25% (which ever is greater) you will be close to the results of checking them all.
Analysing 400 clips is equal to a day’s work for an analyst. It does not work well with the production of results on-the-fly. It works well when producing weekly monthly quarterly or sentiment reports.
This is how it’s done
The process of using Brandwatch to sample a selection of clips is straightforward. Once you have established your search query and created a dashboard go to the Mentions tab.
Select all the items by ticking the box on the top left of the listing and Selecting All at the top of the listing.
Then click More Actions, Add Tags then tick the Tag Random Sample box.
You can then select your percentage or number of mentions to add your random tag to. Once you have added the random tag open the Filters drop-down menu on the right side of the screen. Select Tags and filter by your random tag name. This will bring up a listing of those items which have your random tag applied.
You can now use the Workflow filters to go through those items selected. Once undertaken you can then generate proportionate results based only on the random sample.
This is a far more accurate than letting a computer decide sentiment. It is also many times quicker than manually checking all the clips.
Leave a Reply