Eventually, as human beings, we search for the data that supports our hypotheses. We’ll slice the data in a certain way, only looking at specific segments or specific periods and so on to prove our points.
The problem is that if you keep slicing and dicing your data, you will eventually find specific data points that support your arguments. 🤫 🌟
Great analytics teams are aware of this bias, and develop ways to avoid it:
👉 Sticking to one source of definitions of KPIs, segments, etc. or defining clear success criteria of experiments BEFORE launching.
👉 They also develop a culture that supports “critical thinking”, that questions the assumptions of the insights people present.
If you’re not aware of this bias, or you don't proactively take actions to avoid it, it usually means that your “insights” are biased.
