You Are Not The Mean

April 11, 2015

There are generally three different ways of acquiring knowledge:

  1. Logic
  2. Experience
  3. Science

Logic allows us to derive axiomatic truths that are true independent of empirical validation, such as 2 + 2 = 4. Personal experience is the kind of knowledge each of us accumulates by processing sense data and introspection. Lastly, science is the process of acquiring knowledge by testing hypothesis using objective measurements and statistics.

Speaking from my own personal experience, scientific knowledge is often regarded to be more valuable than the the other two types, especially than knowledge won from personal experience. While there is truth to that in certain contexts, I think the value of scientific knowledge is misunderstood when it comes to decisions in your own life.

The method of empirical science — testing hypotheses using objective and repeatable measurements — works great in the realm of inanimate things. It has brought tremendous scientific and technical progress, and improved the quality of life of many. However, for increasingly complex subjects of investigation, the results become more fraught with uncertainty and all kinds of caveats, and their interpretation gets increasingly more difficult as well.1

At the core of empirical research is the collection of data and its analysis using statistics. Statistics can be descriptive, such as the mean value or the standard deviation of a data set, or inferential, i.e. you draw conclusions from data that’s subject to some random variation and generalize those conclusions beyond the sample you investigated.

I want to shine a light specifically on how conclusions drawn from inferential statistics might or might not be helpful to make decisions in your personal life.

Let’s take some fictional nutritional advice as an example — an area where there’s a seemingly endless stream of supposedly scientifically based advice thrown at us, and where we have to make decisions every day.

Imagine you read about a study that showed that you’re 50% more likely to become overweight when eating a carbohydrate rich vs. a low carbohydrate diet. Leaving all questions about such a study’s methodology as well as the existence of potentially conflicting evidence aside, you could easily conclude that you’re faced with the following dilemma in light of these new findings:

I really do like my mediterranean, carbohydrate rich diet, but is my short-term pleasure worth the 50% increased risk of becoming overweight in the long run?

The fundamental problem with this conclusion is this: the fact that a study has found a 50% increased risk by examining a sample of people does not mean that you as an individual are necessarily at a 50% greater risk as well.

As mentioned above, inferential statistics is used to draw conclusions from data that’s subject to some random variation. Since the human body is extremely complex and there’s a lot of intra-individual variation in terms of physiology and environmental influences, it’s no surprise that the connection between two simple variables — carbohydrate intake and body weight — is subject to a large amount of “random” variation.

If you’d look more closely at the group who showed the increased risk, you’d probably find quite a lot of variation in how the diet affected their weight. You might even find subjects who lost weight. Those variations are not random though, but they have causes that we don’t yet understand or choose to ignore for the purpose of such a study.

While you can make a statistical statement about the increased risk for a certain population, you cannot pin a number on yourself as an individual, because you don’t know what caused the variation in the results and where you would fall on this spectrum yourself.

This is where personal experience and self knowledge are often a much better basis for decisions than empirical findings. If we care to pay attention to it, we have access to a huge amount of information about ourselves. On this basis we can make much more informed decisions than by looking at empirical studies.

Personal experience and scientific knowledge are not at odds, but complement each other. Science is a great tool to advance our understanding of the world and the quality of life for many people. However, when it comes down to specific decisions in your personal life, I mostly consider personal experience to be a better guide.

  1. For an in-depth discussion of the issues surrounding empirical research in e.g. economics and the social sciences check out the book Uncontrolled by Jim Manzi, or listen to this episode of EconTalk to get a taste of it.