7.2 Language AI, Vision AI, and Predictive AI

Language AI gives smart phones, smart TVs, smart homes, and many other smart devices the ability to understand and converse in written and spoken language. Hands-free voice assistants such as, Siri, Alexa, and Google Assistant can be set in motion and controlled by voice commands. They can send texts or voice messages, set alarms and reminders, make phone calls, and answer questions.

Each assistant has the ability to integrate with lights, thermostats, and home security systems. Each can stream music, books, videos, and other media to your favorite mobile or desktop device from various services. Each extend their capabilities by supporting third-party skills, actions, and shortcuts. And through the AI power of machine learning, each adapts, matures, and improves.

The ability to understand (NLP) and converse (NLG) in written and spoken language has many applications when making marketing smarter. eBay’s Associate Creative Director, Molly Prosser, was looking for a way to personalize each message sent out to their customers. But it was impossible for human copywriters to generate unique messages for millions of customers and avoid cookie-cutter emails. Prosser hired AI company Phrasee to help.1 In their collaborative email campaign with Phrasee, Prosser focused on increasing open rates, clickthrough rates, and customer engagement. Phrasee started their work by digesting eBay’s historical email performance data to understand what had worked and what hadn’t in previous campaigns for multiple segments of eBay’s customer base. In only minutes, Phrasee then produced thousands of email subject lines selected to engage eBay customers without straying from eBay’s human-like brand voice. The subject lines were A/B tested on small segments of eBay’s email list, and then adjusted and improved before launching wider campaigns. Through each campaign, and spanning multiple years, Phrasee continued to change and adapt email subject lines used in past campaigns to improve the performance of new campaigns. In the US, eBay improved it’s average open rate by 15.8% and average clickthrough rate by 31.2% by using AI generated email subject lines. Across campaigns, Phrasee delivered over 700,000 incremental opens and 56,000 incremental clicks per campaign. 2

Vision AI has the ability to understand and interpret still images and video. In many ways, Vision AI can do what humans do. It can recognize human emotions, identify faces and images, and detect movement. In some ways, however, Vision AI may be able to do some things better than humans. Realeyes is a world-class Vision AI company founded at Oxford University. Realeyes AI recognizes as many as 150 different emotions by tracking subtle movements in the eyes and mouth. How many humans can do that? Mars (the candy company) partners with Realeyes to track eye and mouth movements of consumers watching advertising for M&M’s and Skittles. Sorin Patilinet, the global insights director at Mars, strongly prefers using Vision AI tools to fine tune product advertising, rather than using more traditional research approaches. He observed, “In 99% of cases, the traditional consumer watches an ad and then is asked some form of question. We don’t believe in that. We have data to prove that the correlation between those kinds of studies, which are declarative studies, and the in-market performance of that ad is not that good.” Going forward, the company plans to analyze thousands of video clips to measure facial expressions of consumers reacting to the message. The Vision AI data will then be correlated with traditional online metrics, such as skip rates, clickthrough rates and view-through rates. Feedback from Realeyes analysis indicates whether a messaging campaign is working, needs to be edited or should be dropped altogether. “In general, our criteria of success are an increase in emotion,” Patilinet shares. “We want to see the positive emotions going up during the ad.”

Patilinet is confident their messaging can elicit positive emotions in a 30 second ad, but the company has found that today’s consumers often skip 30 second ads for shorter formats. To address the challenge, Mars is experimenting with 6-second video ads. Patilinet admits that it is difficult to communicate a persuasive emotional hook in six seconds. He is looking to Vision AI to help provide a solution and quick feedback. When making decisions, says Patilinet, “Our gold standard is using sales data. If you run any test using sales data, you’re talking about weeks if not months until you have the results. Here [with Vision AI], we’re talking days.” 3

Vision AI is not without its downsides. It is now possible to generate fake videos of people that appear and sound real with off-the-shelf software and a few dollars.4 One can only hope that as Vision AI algorithms develop, some of the algorithms will be directed at detecting deepfake videos.

Predictive AI uses machine learning algorithms to improve predictions by updating old data with new data. While predicting future sales, consumer attitudes, and consumer behaviors have long been part of the marketing research toolkit, predictive AI advances the marketing research toolkit because today’s predictions can quickly integrate current data. And as we’ve previously discussed regarding the emergence of Big Data, up-to-date—and even real-time data—are more available now than anytime in our history. Prediction may be the most relevant area of AI for marketers at this time. Better predictions about future sales, underlying patterns in consumer data, how to personalize brand experiences, and which marketing recommendations will have the greatest impact all can lead to better business outcomes.5