Too much love can kill you, sang Freddie Mercury. So can bad statistics…

Freddie Mercury sang “too much love will kill you”. That might be true, but as an accountant, I’ve got to say that love isn’t usually on anybody’s mind when they come to see me.

So I’m not qualified in the love department, but I do have stronger views on how businesses can end up doing crazy things just because people don’t understand basic statistics.

Now, I know what you’re thinking… “it’s easy for you, you play around with numbers every day so you’re used to them”. But I always struggled with statistics while doing my accounting exams, until, mercifully, something “clicked” shortly before my statistics exam and I managed to pass.

The problem I had with statistics was that, like most people, I was taught statistics as a series of techniques – how to do such-and-such a calculation – without being taught the broader concepts and underlying principles which made a particular technique work.

It wasn’t until I realised that the trick to making statistics easy was to do the thinking bit first, and only after sorting that out, getting your calculator out to crunch the numbers.

Don’t get me wrong, there are still some complications in getting the correct answer, but the way statistics is usually taught, emphasising techniques over thinking, accounts for many people struggling to understand what it’s all about.

In just the last couple of weeks I’ve come across these three examples which hopefully illustrate the point.

“Consistently high quality”

A recent article on suggesting that leadership teams in further education colleges were consistently better than leadership teams in schools caught my eye at the weekend.

I’ve worked extensively with schools, colleges and universities and encountered lots of leadership teams across the education sector. And while it;s almost certainly is true that a good leadership team in a college is better than an average leadership team in a school, this isn’t comparing like with like.

I’ve encountered appalling leadership teams in colleges and brilliant leadership teams in schools, and indeed vice versa. But on the average a good leadership team pretty much anywhere, inside or outside education, looks pretty much like every other good leadership team in the world.

There’s nothing “consistently” good about leadership in colleges, although Further Education does have some exceptional leaders, including some I’ve been privileged to work with.

However the sector also has its share of leaders who have led their colleges to disaster.

The TES article is only one person’s perspective, admittedly, and the writer makes that clear. I’ve no reason to suppose it isn’t an accurate reflection of his time working in both schools and further education colleges.

But one person’s perspective, however valuable it might be for them, isn’t necessarily reflective of an entire sector with hundreds of schools and colleges and thousands of people in leadership teams across the country.

So don’t make the mistake of scaling up one person’s experience and imagine it’s reflective of some bigger sector or market. It’s very unlikely to be anything of the sort.

Using outliers to prove a point

An “outlier” is a out-of-the-ordinary result that’s so far away from the “average” that relying on it might not be wise.

You see this in marketing a lot – “Apple do this and it works, so we should do it too”. Apple…or indeed any other business darling…does a lot of things that nobody else makes work. Just because Apple makes a premium price offering work like gangbusters doesn’t mean you can 10x your prices and still expect to win any business.

HR people like this line of argument too – “Google organise their teams this way, so we should too.”

Some people even wrote a book about how Indian managers are unusually good, due to, according to the authors, a combination of the Indian education system and the competitive upbringing for children there. They even cite half-a-dozen or so global CEOs who do, in fact, happen to have an Indian background.

But it’s also true that most global CEOs do not have an Indian upbringing, and it’s also true that there are many non-Indian CEOs who achieve the same or better results than the Indian superstar managers used as case-studies in the book.

India has approximately 17% of the world’s population, but considerably less than 17% of superstar global CEOs. India does admittedly have some business leaders who successfully steered global companies to considerable success. But, statistically speaking, you can’t infer that Indian managers are exclusively excellent, just because a handful of them clearly are.

Here’s the simple truth. Hard-working, smart, competitive people in just about every country of the world are more likely to end up as global CEOs than their countryfolk who prefer an easier life. This is true whether you’re American British, Bulgarian, German, French or any other nationality.

I’ve been to India several times and worked with Indian managers both there and outside India. My experience is that there are undoubtedly some exceptional managers amongst them, but average managers there are about the same as average managers in every other country, and a similar proportion to any other country in the world are truly appalling.

The percentages in each category are not materially different from any other nationality of ethnic grouping between India and any other place on the planet. Intelligence, a capacity for hard work, a competitive nature and many of the other personal qualities the book cites as evidence for Indian managers being so good are approximately normally distributed across every single human being on the planet as far as I can tell.

Nobody would write a book about smart, hardworking kids who went to Eton and Oxford ended up in a top job somewhere. A book about a Harvard graduate running a global business probably wouldn’t be a best-seller either. In both cases, that’s pretty much what we’d expect to happen – both are non-stories.

So it should be no surprise that, statistically, at least some of the population of India are world-beating international business executives. I’d be a lot more surprised if there weren’t any Indian executives leading global multinationals, given that 17% of all the people on the planet, 1.4 billion of them, live in India.

So don’t use outliers…management practices from Apple, Google, India or anywhere else for that matter…to evidence your arguments.

Outliers exist in every field, and I’ve no problem celebrating them where they exist. But be clear that the results of exceptional businesses or individuals are unlikely to be repeated if you try the same thing elsewhere.

By all means use outliers to inspire you to make improvements, but don’t imagine for a moment that the results outliers deliver are likely to be reproduced somewhere else by entirely different people, because you’ll almost certainly be disappointed.

Small groups

“Research shows…” is one of the most-overused expressions in management.

It’s often the magic key that someone hopes will unlock a budget or a strategic decision of some sort, but I always want to know exactly what sort of research has been carried out, and how rigorous it really was.

The tweet above is a great example. Even if you do research which is decent enough in itself, the results from any small group are, at best, no more than mildly comforting in terms of how an entire target market might view your business.

Sometimes people feel the need to draw statistical conclusions where none exist. Everybody likes to see some numbers and. as an accountant, I get to see more than most, to be fair.

However I also see enough numbers to get a feel for when I’m being fed a line.

If you’ve called together a focus group of 20 people and 58% of them say they like (or don’t like) something, in my book that means almost nothing.

58% of 20 people means 12 people in the whole universe like what you’re doing. All it takes is a couple fewer saying “yes” and it’s a 50:50 deadlock.

Even if those 12 are theoretically representative of your entire customer base, which has about the same odds as looking out your window and seeing unicorns dancing on top of rainbows, the realities of everyday life means just about no meaningful conclusions can be drawn from those 12 people’s responses.

Yet I’ve seen many a business case constructed on a similar premise. You might be surprised how few people seem able to explain their “research” in statistical terms.

Never take statistics at face value. Always find out how they did the research and if the answer isn’t “we ran a statistically-valid survey across a representative sample of couple of thousand people”, the level of reliance you can place on the numbers meaning anything sensible is probably close to zero.

Even professional polling businesses know their 2000-people surveys have an error rate of a few percentage points one way or the other. Your 20 person focus group will have an error rate many times that so you’d be unwise to make significant business decisions based on conclusions that could be out by 20-30% or more.

That doesn’t mean small group research hasn’t got any value. It most definitely does.

Individuals and small groups are ideal ways to get qualitative feedback about your business and its operations, or how people feel and experience your services.

Good qualitative research is under-appreciated in businesses because people are mostly looking to use numbers to support the case they’re making.

But most business cases which come my way have not been constructed using research methods rigorous enough to support the conclusion they propose.

Business cases tend to draw conclusions using quantitative research methods – that is, just using the numbers drawn from their research as if they were statistically valid – even though there was no statistical validity to the data at all.

That doesn’t meant heavy-duty quantitative research is necessary for every single business case, though.

If there isn’t a statistical basis for the conclusions, just say so, give me the verbatim feedback from a focus group to read through and let me understand what you’re proposing based on that.

Don’t present statistically-meaningless tables of numbers, graphs and charts to support your case when the underlying research isn’t rigorous enough to justify any conclusion, much less the one you’re putting forward.


The world would probably be a better place if we stopped allowing politicians to quote highly selective “statistics” (which is what they call their numbers…they’re nearly always a completely partisan interpretation of numeric information which was not been collected using valid statistical methods, just because it happens to suit their case).

But much as we rail against politicians’ misuse of numbers and statistics, something very similar goes on in every business in the country on a more or less daily basis.

And a large proportion of those end up on my desk “because we have to show the Finance Director some numbers to get his buy-in”.

I’d rather they didn’t bother. Either do the job properly and show me something that’s statistically-valid or stop pretending that some random numbers which were chosen because they happen to support your preferred outcome have any statistical meaning.

No Finance Director or CFO should be close-minded enough to rule out every initiative which doesn’t come accompanied by a bevy of numbers, charts and data tables.

We just want to know that you know your stats, because if you don’t, we’re unlikely to believe that anything else in your business case is going to be a solid foundation for decision-making either.

Next time someone presents you with a plethora of numbers to support a business case, try a quick “sense check” against the experiences above. You might be surprised by the insights you get from questioning people about how they’ve approached their research, and hopefully you’ll make better business decisions as a result.

(Photo by Stephen Dawson on Unsplash )

How not to screw things up like a politician

Grouch Marx once said “Politics is the art of looking for trouble, finding it everywhere, diagnosing it incorrectly, and applying the wrong remedies”.

At the moment, there’s no finer example of this phenomenon than the UK’s Transport Secretary, Chris Grayling. .

On top of a less-than-impressive few months in this role, yesterday he blamed the wage demands by rail unions for this year’s annual price rise in train tickets (BBC News report here).

Yet what the UK rail industry calls “regulated fares”, such as the season tickets which were at the heart of many of yesterday’s protests, are set according to a government formula agreed back in 2003 which has no connection whatsoever to wage rises in the railway industry (more details on the rail fare formula here).

This is important because once you’ve diagnosed the problem, rail unions in this case, that tends to be the problem you’ll set out to “fix”.

That’s not to say rail unions are blameless, or that they are tireless champions of anything other than their own members’ interests…it’s just to recognise that the Department of Transport’s ticket price formula can be in no way related to the wage rates of people working in the railway industry, as it’s not even part of the publicly-available formula for calculating ticket prices.

It’s hard to imagine that the interests of the UK’s long-suffering rail travellers would be improved by the Transport Secretary picking a fight with rail unions due to his lack of understanding of the pricing mechanism put in place by his own department.

Unless Chris Grayling was just playing politics to enhance his own standing within the Conservative Party, of course, which is probably several orders of magnitude worse than just being ignorant about pricing mechanisms for which he is the responsible cabinet minister.

By now, even if you weren’t sure before, I think we can probably agree that if you want something screwing up far more comprehensively than any mere mortal could hope for, the thing to do is to get a politician involved.

The question is how can business people like you and me you avoid screwing things up quite as badly as the average politician?

Groucho Marx was right – the diagnosis of a problem is key. Once you’ve decided what the problem is, that’s probably what you’ll do your best to solve. Get that wrong and the likelihood is that not only won’t you solve your original problem, you’ll probably have created another problem you didn’t have before in the process.

Albert Einstein said that if he was given an hour to save the plant, he’d spend 59 minutes defining the problem and one minute resolving it.

That may be a little extreme, but you can’t hope to find a good solution unless you’re solving the right problem. That’s where most politicians get it wrong – they’re taken in by their own political philosophies and see everything through those distorting lenses.

And, I’m sorry to say, business leaders, including myself, have distorting lenses of our own. Having a preferred view of the world distort our own perception of reality is a natural human characteristic. Everybody has it to at least some extent.

But there are things you can do to find better solutions. And that starts by how well you define the problem, as Albert Einstein said.

That depends in turn on how good you are at asking questions, and that’s a skill you can work on and improve, however good or bad you are at it now.

Research published in the Harvard Business Review set out a four-stage process for learning to ask better questions. In summary, this involves:

  • Establish the problem in as simple terms as possible – “We are looking for X in order to achieve Z as measured by W” is the format the article uses.
  • Secondly, you need to be clear how solving the problem contributes towards achieving a strategic organisational objective. You might think this was self-evident, but I’ve sat in plenty of meeting rooms where people thrashed out how to “solve” problems that, even if they were solved, would be unlikely to have any major impact on the organisation one way or the other.
  • Next, take a look at what your organisation has tried in the past and, if relevant, how other people in your industry have attempted to solve the same, or a similar, problem.
  • Finally, write a problem statement which defines the problem, its proposed solution and the requirements you’re trying to achieve, taking account of everything you’ve learned through the earlier stages in the process.

For the UK rail industry, it seems we haven’t even got past the first stage of defining the problem terribly well yet, but you can do better within your own business quite easily.

The HBR article has some great questions to ask during each stage of the process, but the key to solving any problem is always to cast your net as widely as possible when looking for answers.

If all important decisions are only made by senior executives in a conference room, I can guarantee a substantial proportion of those decisions will turn out to be wrong, or at least sub-optimal.

That’s not the executives’ fault – generally they come up with the best solutions that make sense to them, based on their qualifications and experience.

But usually they don’t have current experience of being a front-line sales person trying to sell whatever the company’s new business model is to a potential customer, or someone in the factory trying to produce against a revised product spec with tools that haven’t been properly designed for the purpose, or a call centre operator who handles the irate customers that call in when the revised product or service doesn’t work properly.

Of course, if it’s a strategic decision involving a large financial investment, senior executives have to make the final decision. To do otherwise would be to abdicate their responsibilities to the business.

But during the problem solving process, and especially the question-asking parts of it, involving as many people as possible, especially from the front-line can make a huge difference to the quality of the eventual solution and the cost-effectiveness with which it’s implemented.

If you want to screw things up like a politician, by all means sit in your conference room and make all the decisions amongst the senior executive team.

But I can guarantee that if your business learns to ask better questions, and involves a wider cross-section of staff than just your senior management team in solving them, even quite big issues for your business can be solved quickly and cost-effectively.

Let’s just hope that one day this might be the way we run our railways too.