The algorithm may have fast become global shorthand for the brutal simplicity to be found in data-driven marketing, but the growing focus on emotional analytics is proving that the algorithm can in fact have a heart. It is a shift that will be of great importance for brands and, inevitably, has spawned a new rash of marketing jargon.
"The best marketing may be based on clever analysis, but you mustn’t be able to see the spinning underneath"
Alex Ward-Booth, Danone
Speaking on the phone to Lydia Daly, senior vice-president of social media and branded content strategy at full-service marketing agency Viacom Velocity, it is hard not to imagine that she’s rolling her eyes when she says: "Then we take all that data we’ve gathered and we run it through a tool which analyses it to pull out the ‘emotionality’. Yes, it does kill me a little inside that this is now a word and I hate myself a little bit that it’s one I’m using more and more in business meetings."
Daly is describing how she reports back to brands advertising on the Viacom Velocity network, such as Pepsi, Cover Girl and Target, on the extent to which their campaigns have engaged audiences on an emotional level.
The somewhat resigned tone contrasts starkly with the hyperbole coming from tech companies claiming emotional insight is the missing piece of the data jigsaw for marketers. Nonetheless, despite Daly's reservations about the current limitations of the technology, she’s in no doubt that – to paraphrase a recent headline in The Daily Telegraph – "the emotional machines are coming".
"It is not something we can ignore," she says. "It’s the evolution of measuring sentiment. At the moment, I’d say take it with a grain of salt, because it should be seen as something that is evolving and marketers should be exploring."
Marketing machines such as Procter & Gamble, Unilever, Coca-Cola, Diageo and Mondelez may be reluctant to talk about it, but they are all using emotional analytics (EA). The fact that they are doing so suggests that it could be the "next frontier" of big data. Most are hiring companies that offer emotion-recognition software, such as Affectiva, Sticky, Realeyes and Amido, to test and optimise content and media strategy using techniques including facial coding and analysing mood on social media through language and tone.
Like Daly, Mark Beard, The Economist’s senior vice-president, global digital media and content strategy, has a measured response. He is starting to experiment by looking at the emotions customers experience on their path to purchase, through "real-time marketing" agency Brilliant Noise. However, while he acknowledges that machines and data have "transformed" marketing at his brand, he’s not letting the hype overtake him – or, indeed, the machines take over marketing.
"As marketers, we have to decide where to draw the line between how much we use machines and data and how much we rely on human intuition and input," Beard says. "That line is changing as artificial intelligence gets better and machines get cleverer. Marketers must stay at the cutting edge of tech so they know where that line lies."
As it stands at The Economist, that line is still controlled by man, not machine. When seeking to create a "hyper-relevant, real-time" ad campaign around last year's US presidential election debates, for instance, The Economist hired UK copywriters to create content through the night that "played on the themes that had arisen". These were then fed into "machine-led marketing" and ready for the US audience when it awoke. While Beard concedes that one day a machine could do it all, that time is not now.
The big fear of relying on an algorithm to create your marketing is that you lose the human touch – something defeated US presidential candidate Hillary Clinton was criticised for, with more than 23,000 tweets last October describing her as "robotic", according to social software tool Spredfast. Some commentators have blamed this on an over-reliance on the "invisible guiding hand" (Politico Magazine) of her analytics expert, Elan Kriegel.
Creative agencies, however, are far more effusive about the possibilities that lie in using emotional analytics to improve marketing. "Tools like facial recognition, reaction-time testing or neuro tools are sort of the ‘Holy Grail’ for a creative agency," says Rob Ward, strategy partner at 18 Feet & Rising. "They add a level of scientific rigour to your argument that most people can’t deny."
His agency works with sister neuroscience shop Walnut Unlimited for clients such as the National Trust (see case study, p28) and House of Fraser, and is bringing in neuro-research techniques much earlier in the creative process. In 2015, the partnership between the two agencies led to an overhaul of House of Fraser’s Christmas campaign.
"The danger with data is that it appears to do all the work for you – it doesn’t. data is only as good as its interpretation"
Tash Walker, The Mix
"If you look at most of the content being produced by fashion retailers, it’s the same," says Ward. "Just fairly pretty models vacuously dancing around. We were able to demonstrate through Walnut that this approach spiked a short-term ‘that’s nice’ response, but it didn’t in any way emotionally connect. It showed we needed to create something else that was more emotionally engaging and narrative-based."
There is a growing recognition in the industry that it needs to get past the rational "system-2" thinking and delve into the hidden "system-1" responses to really understand consumer behaviour. Emotional analytics can offer a window into this unconscious world. Some experts, such as Realeyes’ scientific adviser, Maja Pantic, argue that machines are able to pick up expressions that humans can’t see.
"EA is going to be amazingly important, particularly as we move into the area of really starting to transform customer engagement"
Food company Danone’s strategy and insights manager, Alex Ward-Booth, is strongly drawn to this possibility. "I find it endlessly fascinating that people can say they ration-ally believe in something and then behave in a way completely at odds with that," he says.
Danone invests in shopper research to establish consumers’ genuine feelings about fixtures and categories, including analysing micro-expressions. It, too, works with Walnut. Ward-Booth singles out mothers as one consumer group for which this method has proved particularly powerful. "If you ask a mum in a focus group or traditional interview about baby foods, they will say everything is made from scratch by hand and put in Tupperware, because they’re terrified of being judged as a bad mum," he says. "We have to unpick what is really going on by looking at behavioural data, but this data has to be used with empathy. The best marketing may be based on clever analysis, but you mustn’t be able to see the spinning underneath, or how it is trying to push you in a certain direction, or it becomes too transactional."
As a result of this approach, Danone identified the need for a healthy, convenient toddler snack and launched Super Yummies at the beginning of last year.
Ward-Booth’s hunch that the "exciting, emerging" area of emotional analytics will grow is backed up by research from Crone Consulting which forecasts that the market will soar from $20m in 2015 to $10bn globally by 2020.
The game-changer is not the ability to measure emotion per se (already possible in the lab), but doing it on a big, automated scale. "Wearables" are a key innovation that will take this to the next level; described by Mindshare in its "Huddle Trends 2016" report as an "interesting hybrid". The report stated: "[Wearables] can measure emotion [through data such as pulse] and even communicate haptically, but their real strength is the whole host of other data they collect which could be applied to the overall picture. Elements like location and browsing history could tell a great deal more about where people are, what they are doing in the context of how they are feeling, and responses tailored accordingly."
Growth will be fuelled, Mindshare predicts, by interpreting this data to convey emotions more successfully. In other words: "AI with a heart" (see box, right). Given that the latest TVs now integrate eye-tracking and facial-recognition technology, the opportunities based on knowing viewers’ emotions and how they respond to the ads they see are clear.
The implications for connected cars are similarly promising. "EA is going to be amazingly important, particularly as we move into the area of really starting to transform customer engagement," says Jaguar Land Rover’s head of communications, Robert Herd. "Inside every vehicle you have sensors like driver awareness, position, how fast they’re going. You can imagine what insight EA could give us, which could inform our communications and interaction."
There are, however, huge caveats. Herd is concerned about how brands use emotional analytics without appearing "creepy" or inappropriate. "What right does a brand owner have to react to emotion? I’d struggle with that," he says.
The overriding concern is that brands will take huge swathes of data claiming to show "emotionality" and make assumptions on the back of it, without digging deeper into what it truly means. Some agencies already report that clients are increasingly unwilling to invest in crucial qualitative research because of this new type of "big" data.
Tash Walker, founder of research consultancy The Mix, a millennial on a mission to disrupt the research market, is typically feisty in her damnation of this trend.
"The danger with data is that it appears to do all the work for you, which it doesn’t. Data is only as good as its interpretation. More data has only led to more confused marketers, few of whom know what the hell to do with the data they have," she says.
"Emotional analytics may be the next frontier, that brings us a human version of big data, but there is a strong argument that we should sort out the frontier we’ve got first. More is most definitely no more human."
♠ Emotive Billboards
Billboards are being tested with hidden Microsoft Kinect cameras that read viewers’ emotions and react according to a person’s facial expression.
♠ Huggies Haptic Waistband
A waistband for the partners of expectant mothers, enabling them to feel the movements of their baby in real time.
♠ Olive Stress Wristband
A bracelet that tracks the wearer’s heartbeat, skin temperature and other external factors to monitor stress levels. Users are prompted by LED lights or haptic feedback to carry out stress-management exercises.
An app used to send sensory messages based on physical sensations, such as a heartbeat. These are delivered to a wearable device – a TACTpuck.
Software that can measure facial "micro expressions" to construct a detailed representation of a person’s emotions. Currently being used to test audience reactions to ads and TV shows.
♠ Uniqlo’s Umood
Suggests products by reading brainwaves that determine the customer’s mood.
A program with built-in camera that captures students’ facial expressions in response to elearning content, and adapts further content accordingly.
♠ Haptic Gaming Gloves
A glove that can be hooked up to a video game so that when you grab an object in the game, you actually feel it.
♠ Kitkat Massage Billboard
Billboards that give free massages via tiny motors that transfer vibrations to the back of the user as they lean against the ad.
♠ Microsoft’s Project Oxford
An artificially intelligent system with the ability to detect mood in photos using updated facial-recognition capabilities.
♠ Watson Tone Analyser
Cognitive computing platform that can now tell how tone comes across in written communication.
♠ Hug Shirt
A piece of wearable clothing that allows people to send hugs over distance. Embedded sensors feel and recreate the strength, duration, and location of the touch, the skin warmth and the heartbeat.
Source: Mindshare "Huddle Trends 2016" report
Meet the disrupters
Jaguar Land Rover
Emotional analytics was at the centre of Jaguar Land Rover and Mindshare’s 2015 "Feel" campaign around the Wimbledon tennis championships. Selected spectators were given wearables, which tracked indicators of their emotional state, such as body heat and heart rate. The system then also measured crowd movement, group energy levels and audio levels. Through these biometrics and sociometrics, the brand was able to generate 45m data points every day, providing real-time mood analysis, which it tapped into to create relevant communi-cations, outdoor and online, around the event.
For instance, commuters arriving at London's Waterloo station were greeted by scenes from Wimbledon on billboards that captured the current mood at the tournament and shared it. The activity sought to draw a parallel with driving a Jaguar, on the basis that the emotional experience of being at Wimbledon or in the driver’s seat is similarly highly charged; this was a way of sharing those emotions with people that weren't experiencing it first-hand.
To date, the brand has used emotional analytics only as a marketing channel, but Jaguar Land Rover's head of communications, Robert Herd, says he’s looking at it in relation to how it could inform the direction of the company's own marketing.
"It comes down to appropriateness and relevance. I keep coming back to the privacy versus value exchange," he adds.
Understanding how its audience emotionally connects with its brand is vital to achieving its purpose, which is why the National Trust has recently revamped its visitor survey.
Previously this focused on experience ( "On a scale of one to five how would you rate your visit?") and functional metrics ("How would you rate the cleanliness of the toilets?").
However, over the past year it has started to include questions about how audiences feel. "Only by understanding [this] can we make ourselves more relevant," says Richard Clayton, insight lead, data science and analytics, at the National Trust.
The organisation has also been working with neuroscience consultancy Walnut to track and analyse how people feel about it, through implicit and explicit testing and in its communications research.
According to 18 Feet & Rising’s strategy partner, Rob Ward, customers typically have a warm response to the brand and are "too nice" to criticise creative, so facial coding and reaction-time testing has been a powerful aid to picking executions that genuinely connect.
"Emotional analytics adds richness to our behavioural analysis," says Christina Finlay, head of audience insight at the National Trust. "It has been an important part of seeing how differently people can experience the same thing. This allows us to engage different segments in ways that can ensure their experience feels more personalised."
However, both Finlay and Clayton stress that, while emotional analytics is a significant driver for change, it must be combined with other types of insight.
Viacom is taking data gathered on social media via its brands’ 800m followers and analysing it to gauge emotional response to advertiser content on its networks (which include MTV, VH1 and Comedy Central). It’s doing this through software firm Spredfast, which uses the data to create an "echo social graph" providing real-time insights into campaign performance.
"It’s like an online dashboard measuring success through social listening," says Lydia Daly, senior vice-president of social media and branded content strategy at Viacom Velocity. "When we’re working with Pepsi at the VMAs (Video Music Awards), for example, we’ve got social influencers walking the red carpet and creating content for us; we capture all the social conversation and organic content that the audience is creating around it."
The content is scanned by analytics tool Canvs for language, particularly millennial and social-media slang, which betrays emotion. Canvs categorises the content into 56 identified wide-ranging emotions, including words and phrases such as "badass" and "guilty pleasure".
This type of analysis is particularly useful in "teasing out" the nuances of how emotions change during a campaign, but Daly is realistic about the drawbacks. She says that, currently, the volumes of social data are too low to really have an impact. In addition, the data is restricted to what social-media networks are willing to provide.