Can Big Data help us predict – and possibly change – the future?
This was an interesting thread that wove through several presentations during the Big Data Innovation Summit that took place in Toronto last month (June 20 -21, 2013 – http://theinnovationenterprise.com/summits/big-data-innovation-toronto)
The summit presenters provided examples of how to apply Big Data and predictive analytics algorithms in different sectors, ranging from healthcare, weather prediction, traffic navigation optimization, to social media. Such a broad spectrum of examples indicates that just about any organization in either the private or public sector can find significant benefits by using Big Data and predictive analytics technology.
Predicting Social Media Virality
Mark Zohar, CEO & Co-founder of Trendspottr offered examples of how algorithms can help us predict trends in social media sharing. From a mathematical point of view, there is a certain – and fascinating – similarity between the spreading of virus-related infections and how viral social media is distributed through social networks. Trendsspottr developed the engine that is capable of analyzing real-time data streams such as Twitter and Facebook and spotting emerging trends at their earliest acceleration point – hours or days before they have become “popular” and reached mainstream awareness. One of the examples Mark shared at the summit was TrendSpottr’s ability to predict the virality of a YouTube video such as PSY’s Gangnam Style.
Improving Your Email Marketing Subscriber List Quality
How can predictive analytics help email marketers with subscriber list hygiene, and ultimately avoid ISP blocking and a bad reputation? For starters, if we are able to predict bad addresses in the subscriber list before sending emails, then we would be able to avoid deliverability problems with the rest of the list. We could also predict the best time to send emails to each individual subscriber, as well as the best subject line and message content to significantly improve open and conversion rates.John Foreman, Chief Data Scientist for MailChimp talked about how the company is using Big Data and data science to improve the quality of their email services to their clients. MailChimp.com sends 5 billion emails each month, and from those emails, they track the billions of opens, clicks, abuse reports, and unsubscribes for their 3 million users. The richness of data allows MailChimp to develop predictive models to predict abuse, automate segmentation, and optimize content.
Cross-Channel Marketing and Big Data
Digital marketing is undeniably one of the biggest catalysts of Big Data advancements in the last 20 years.
Andrew Covato, Analytical Lead with Google, spoke about the challenges with measuring and optimizing digital marketing during the purchase cycle. The typical purchase cycle of consumers includes 4 stages:
- Dormant
- Aware
- Considering
- Converted
What adds new complexity to the modeling is that the outcome also depends on the sequence of consumer interaction with different media during the purchase cycle. For example, we might make a mistake when we attribute the conversion to the last interaction. Legacy aggregate methodologies such as Marketing Mix Modeling are simply insufficient to address the added complexities of marketing in the digital realm. Addressable interactions can be hyper-granular, and user behavior is dependent on many variables, which can now be measured.
Mistakes in attribution models lead to wrong decisions about optimal investment in the marketing advertising portfolio. We need more sophisticated attribution and marketing optimization models to measure, plan and automate digital, and all addressable marketing.
Many marketers will know about the famous quote made by John Wanamaker (1838 – 1922), considered by some to be the father of modern advertising and a pioneer in marketing. He said: “I know half the money I spend on advertising is wasted, but I can never find out which half.”
The question is: Do we know how to manage our marketing investments better now, versus in John Wanamaker’s times?
I say yes. But while we may be little bit better now, we are facing new challenges in cross-channel marketing. Andrew Covato compared the state of digital or cross-channel marketing models with the state of financial models in 1980s. Data scientists are investing a lot of efforts to improve the models and tools to help marketers plan and execute cross-channel advertising.
As an astrophysicist by education, I personally see similarities between the complexities of models to predict an individual consumer response to advertising, to the Quantum Mechanic uncertainty principle and the wave-particle duality. According to quantum physics, the concept of causality cannot be applied to what is observed. In the case of the electrons of an atom, the closest we can get to describe the electron’s position is by giving a number for the probability of it being at a particular place. Moreover, particles have other “disturbing” properties: They have a tendency to decay into other particles or into energy, and sometimes – under special circumstances – they merge to form new particles. They do so after indeterminate time spans. Although we can make statistical assertions about a particle’s lifetime, it is impossible to predict the fate of an individual particle.
Similarly in marketing, we cannot with certainty predict when a particular individual consumer may move to the next stage in the purchase cycle. We may have only statistical probability values.
Data Science helps to Personalize Digital Experiences
We all know about how Amazon and Netflix use their recommendation engines based on collaborative filtering to provide consumers with personalized targeted recommendations. The same technology actually may help to predict what should be on our shopping list while we are in a grocery store with our smart phones.
Diego Maniloff, Data Scientist with Unata shared the company’s recommendation platform that mines loyalty program data and builds a recommendation list based on frequency and recency of past purchases. Such recommendation models may predict when I need to buy sugar before I find the empty sugar container, thus providing value for both consumer and retailer marketers. This factor makes recommendation models far more effective than pushing advertising or in-store flyers.
Conclusion
For many years, I have been personally involved in the areas of data science, Big Data and digital marketing. I follow specialized research papers, news articles and blogs related to Big Data and data science.
Yet after listening to 20+ presentations during the Big Data Innovation Summit in Toronto, I was still impressed at the broad spectrum of application that utilizes Big Data technologies. Today, practically all fields of business and public sectors – including even the government – have new opportunities to innovate with Big Data, in an effort to improve customer and constituent experiences.
