Welcome, guys, to another ML article.

In this one, I want to talk about a problem I ran into while working on my Clash of Clans ML lab.

While preparing the dataset for the ML problem, I found something strange. A group of trophy-related features seemed to contain a huge amount of predictive information about the target. At first, that sounds great, because a highly predictive feature is usually exactly what we want in a machine learning model. But something about these features made me want to look a little closer.

When a feature is too predictive, there is always a question worth asking: why?

Maybe the feature contains information the model shouldn't have. Maybe it's indirectly encoding the target. Or maybe it's simply a very strong signal that the model is completely justified in using.

This is where data leakage comes into the picture.…