Social Media Will Become Dominant Force in Forecasting

This is according to the leading British Economist Noreena Hertz. Over the last few years she’s been quite obsessed on how technology is changing the world, last year she conducted a research to show the power of social media as a means to better understand human behavior.

Hertz told CNN at Names not Numbers, an idea-sharing and networking conference in the UK:

I was really interested to see whether we could make predictions or forecasts by listening in on what people were saying on social media. Because, if we could improve upon the political forecasts; if we could improve upon sales forecasts by listening in on people’s chatter on Twitter, Facebook, those sorts of media, that would be a really exciting prospect.

A team of computer scientists, sociologists, and economists developed a sophisticated algorithm that proved to be more accurately that ‘bookies’ in relation to predicting the winner of the X Factor.

Language and action

What I find intriguing in her elaborations are things such as:

“Language is too complex for a computer to understand. It’s not going to be able to make sense of what people are saying en masse. We need a new type of discipline that puts together computer scientists and social scientists, who can add context to the situation.”

And

Under her school of her thought, in order for social media marketing to have any real value we need to understand what people’s actions really mean. “If somebody tweets ‘I like Coca-Cola’, does that mean that they’re actually going to buy Coca-Cola? One can? Two cans? Three cans? If they retweet someone else’s Tweet, does that mean they’re going to buy it?”

It’s a big challenge to distill action from an expliciet piece of text. I’ve done enough social media analytics and monitoring to back this up. It’s doing a lot of expectation management towards organizations because being mentioned is great, but doesn’t have to mean a lot.

HP’s research

Another research on forecasting the future with social media is conducted by HP Labs.

The focussed on forecasting box-office revenues for movies. They built a simple model from
the rate at which tweets are created about particular topics can outperform market-based redictors. They demonstrate that sentiments extracted from Twitter can be further utilized to improve the forecasting power of social media.

This is another challenge, sentiment and the real effect on action and the (incorrect) conclusions one might take from it. If a person is not happy about X, but still uses it because there’s not a viable alternative, it means that sentiment is one thing, but that action is still not influenced due to the context.

Their conclusions are:

  • Social media feeds can be effective indicatorsof real-world performance.
  • The rate at which movie tweets are generated can be used to build a powerful model for predicting movie box-office revenue. The predictions are consistently better than those produced by an information market such as the Hollywood Stock Exchange, the gold standard in the industry.
  • The analysis of the sentiment content in the tweets shows that they can improve box-office revenue predictions based on tweet rates only after the movies are released.

Additional layer of intel

Social media intelligence is powerful, but it is one of the many layers and datasets out there, from what I experienced it is the combination of datasets and insight extraction will produce great results.