Stay open minded

Even though we have set up our metrics in the best way we could foresee, we might not track the right data. Therefore it is important to stay open minded (sceptical) about your data. It might be that we didn't target the correct user group as the changes in working software have a greater impact on another group we didn't expect. Open minded refers to the fact that you don't just blindly trust your metrics but check the metrics once more against your data.

Be aware of biases

Think about idea bias: we perceive our own ideas as better than those of others. Statistics show that up to 80% of all ideas are not awesome for users. Sometimes what we consider as great ideas are even making things worse for our users. This means that 8 out of 10 ideas will statistically not bring the expected value for users and might even result in sunk cost.

There are many types of biases such as

• Statistical bias
• Bias of self reporting data,
• Cognitive bias,
• (…)

I recommend the following medium article for a deep dive into different types of biases: “Cognitive bias cheat sheet”