Big Data - is it all that?

In his book "The Hitchhikers Guide to the Galaxy", Douglas Adams posed the “Ultimate Question of Life, The Universe, and Everything”. As many of you know, a huge super computer famously came up with the answer of 42. What less people might know is that this super computer also went on to say that the answer was meaningless. This popped into my head earlier this week as I was reading an article about Brentford Football Club. It was not a football story but one about ‘Big Data’ and a question as to whether or not it is the answer to all our prayers that some think it is. 

Brentford FC are unique in British football because they are the only club that use highly sophisticated and complex data analysis as the basis of all their football decisions. Huge amounts of different data about players, movement, positions on the pitch, physical attributes, etc. is collected, crunched and analysed continuously. Where Brentford differ from other clubs is that for every game, the team is picked not by the manager but by the outputs of this data analysis. Players are signed, not signed or let go based on the same data analysis approach. Other major employment decisions are clearly also based on these outputs as mid-way through last season, it was announced that their manager Mark Warburton - who had guided this unfashionable London club to 5th in the Championship at the time - would be leaving at the end of the season. The data analysis had suggested that they should only be 11th. So he had obviously got lucky!

Evidently Mark Warburton didn’t quite buy into the data driven philosophy as much as the club’s hierarchy wanted so he was to leave by “mutual consent” - whatever that means! The manager however carried on with the team until the end of the season where they finished a highly creditable 5th in the table.

This season, they bought in a new manager - one who brought into the whole data analysis based project and philosophy. This week, with the team languishing in 19th place in the table after 8 games, this new manager was sacked! So much for the data!

I have also seen similar differences between data and real performance play out in business.

We were working with a global technology organisation who had invested huge amounts in a very large sales skills development programme. At huge expense and effort, 12 discrete sales competencies were identified that encompassed all aspects of the sales role. There were slight variations depending on the role of the salesperson - for example: if they were in new business, account management or pre-sales etc. Very precise definitions of each of these competencies were developed at 4 different levels ranging from Needs Development (level 1) through to Outstanding Performance (level 4).

A complex benchmarking process was put in place that required managers to assess each of their salespeople’s current position on the 1-4 scale for each of the 12 competencies. They then had to coach their salespeople at regular intervals to progress them towards the Outstanding Performance level on each of the 12 competencies. To help these managers, a set of management coaching competencies were also developed with a similar 1-4 ranking scale and these managers were given coaching on their coaching skills. Detailed data was gathered each month on every salesperson and every manager. This included competency scores, improvements, lack of improvements, frequency of coaching, speed of improvement etc. and detailed charts and reports were produced by a dedicated team. Decisions were made as to certain financial rewards and other benefits on the basis of this data.

Wow, what an effort and it all sounds very positive until you start looking at some of the results.

For example, there was the case of one salesperson improving their Creating Customer Needs competency from level 1 - Needs Development - right through to Outstanding Performance. Fantastic until you learn that this same salesperson had little or no sales pipeline. Clearly any salesperson with no pipeline is NOT very good at Creating Customer Needs. There was another salesperson who scored strongly on the Commercial Acumen competency but didn’t know one end of a customer business case from the other when challenged. There was a manager who was progressing nicely against the Giving Feedback competency but who continually ignored and made excuses for his salespeople’s lack of performance rather than telling them what’s what. There were other examples as well. 

Clearly competency scores were not always indicative of real life performance.  

Now in no way am I saying Big Data is a waste of time. Of course it isn’t and I am a major league advocate of the power of data and information having worked for many years in the data warehousing industry. It has huge potential and a massive role to play.

However what I am saying is that Big Data is not the one and only panacea to all our ills. It’s not ‘THE’ answer as some would have us think. In his 2015 book “Humans Are Underrated”, Geoff Colvin, Editor-in-Chief of Fortune Magazine suggested that the human dynamic still plays and will continue to play a uniquely critical role in the workplace despite the accelerating advances in processing power and robotics.

Big Data is NOT a replacement for critical human capabilities such as judgement, intuition, emotion, team dynamics etc. The challenge is in proactively developing these human capabilities to a level so that they can go hand-in-glove with advances in technology and thereby positively impact human and organisational performance in the most effective way.