A recent Harvard Business school study reveals that a primary reason around two-thirds of analytics projects fail is C-suite resistance to data-driven decisions. It seems that despite bold claims of being “data-driven,” the reality is that 62% of executives still prefer to trust their gut, with 61% claiming “real-world insight tops hard analytics when making decisions”. What is this?

It appears that subjective factors such as company culture, values and reputation were at the forefront of decision-making – not data. The more complex decisions become, the more top executives are looking at emotional factors as their key catalysts. And why not, as the opening statement of Gyro’s “Only Human” report says…”business is personal”

The question behind this statement is – how important is the power of emotion in business today?

So let’s explore this from several perspectives: corporate culture, the limitation of machines, the overwhelm of data, and the fallacy behind the high value placed on judgement based on past experience.

Corporate Culture of Engagement

Well, if we consider Fortune’s most admired companies and best companies to work for around the world, they all share a culture of engagement. Now whilst I admit that this term ‘engagement’ has been somewhat overused to apply to everything from supplying cafes and crèches to surprise bonuses, the bigger picture is that these companies value their employees, and treat them as people, not just revenue earning devices.

However the main focus of the report is to “quantify the extent to which emotional, situational and cultural factors influence business decisions”, and to determine “to what extent can new technological tools and advanced analytics substitute for human knowledge and experience”.

Machines Lack Human Relevance

Overall, it appears that whilst technology certainly helps in collating and filtering the constant flow of data in ways no human team could emulate, it is still human hearts and brains that hold the greater influence in decision making. Personal feelings overrule data and information. The thing that machines lack in processing decisions is human relevance. Only humans can provide human context to rational outputs of rational models. Only humans can provide the ‘why’.

‘Why’ is the vision behind every company – it is what inspires and drives the hierarchy missions, strategies, tactics, actions and thoughts. Lose the why, you lose the power of the people.

People need to have an emotional connection with what they do. And emotional connection equals engagement. Yep! I did it. I just hurled myself up onto that ‘engagement’ bandwagon. Because it has substance.

But let’s leave the human factor for a moment and look at some hard facts revealed by the study:

• Nearly 65% of executives believe subjective factors that can’t be quantified (including company culture and corporate values) increasingly make a difference when evaluating competing proposals.
• 62% of executives say it is often necessary to rely on gut feelings and soft factors.
• 70% cite company reputation as the most influential factor; with 53% also citing culture, as deciding factors in choosing a company to do business with,
• 61% agree that when making decisions, human insights must precede hard analytics.
• 52% believe that ambition, admiration and potential rewards outweigh fear of failure and being blamed for making a bad call, and
• 71% claim short-term financial sacrifices are worth it to maintain long-term relationships with organisations they trust.

The outcome clearly supports the overall feeling that despite data analytics support in decision making, emotions  still precede rational when it comes to business decisions.

(62%) freely say it is necessary to rely on gut feelings. They state that soft factors should be given the same consideration as hard data.

Emotion-first findings. When a trigger goes off, being conscious of it and then exploring what it means is extremely important. And the more complex a decision, the more emotional factors play a role in decision making. However, digging underneath this sentiment, it appears that the difficulty in accessing actionable insight from data plays a pivotal role. The combination of increasing variety of information types, lack of analytical capacity, and weak transfer of data to insight carries significant weight in executives preferring to stick with their gut. Even neuroscientists support this behaviour, with Antonio Damasio stating:

“IT’S IMPORTANT TO MOVE AWAY FROM THE VIEW OF LOGIC AND REASON AS THE CENTER OF GRAVITY OF ONE’S ACTIVITY. THINGS ARE FAR MORE COMPLEX.”

Diminishing Returns of Data

As in economics, there is a point at which having more information has diminishing returns. In fact, it can overwhelm decision makers to a point where the value of all data is reduced. This challenge is leading executives back from business insight to personal insight, in an attempt to trust a more innate approach to decision-making. The greater challenge in doing so is to understand ones own decision environment – ones emotions are driving by ones personal values and beliefs. Understanding how these impact our emotions, and in turn our decisions and behaviour is a primary objective of emotional intelligence, and why EQ is becoming a hot topic in the executive suite.

Neuroscientists estimate some 90% of our responses are biologically driven – whether we are aware of that or not. Studies have proven that without emotions we cannot make decisions – even seemingly totally logic-based ones. Rationality is fluid; with cognitive thought only entering the decision process after emotions have largely determined our biases. We are also typically unaware of our biases, as these are also tightly interwoven in the values, beliefs and culture that make up the fabric of who we are.

The critical point is to recognise, that both cognitive and emotional functions are involved in decision making, and to develop more formal decision processes that better ensure that we engage the right mix of emotion and analytics.

The Judgement Fallacy

Judgement from ‘experience’ helps us appreciate the dynamics of the ecosystems in which businesses operate. What is often not appreciated, is that the assumptions underlying those experience-based judgements often do not apply to the current scenario. It was once reasonably expected that something that happened 3-5 years ago was sufficiently recent to still apply today. However, with the immense complexity of todays device and media-driven societies, the cultural cycles in any layer of society are changing every few years. This means that unless something occurred months, rather than years, ago, assumptions need to be validated for currency.

Human experience tends to direct the data analytics in the search of the most effective result. We need to ensure that we are not adding errors of judgement, such as confirmation bias, out of date assumptions, subconscious bias, and personal subjectivity into this ‘direction’. This is where a more formal decision process can help.

For more understanding on how errors creep silently into decision making, and how to avoid their negative impact – download ‘The Missing Element’.

Author: Gail La Grouw. Insight Mastery Program Director, and managing Strategic Performance Consultant for Coded Vision Ltd.