Data Science Notes 4: Feature scaling, normalisation, standardisation

All the statistics in the world can not measure the brilliance of compassion.

In this post I will try to explain scaling the features in a dataset, and using normalisation and standardisation to do it. To be honest this topic shows its effect mostly in distance based machine learning algorithms such as KNN (K-Nearest Neighbors), which we will see later. It is worth learning now and letting it settle in, because it is part of preprocessing. If you have not read my earlier posts, you can read them here:

• Data Science Notes 1, Introduction to machine learning
• Data Science Notes 2, Machine learning and Python
• Data Science Notes 3, Data preprocessing

What is feature scaling for? When is it used?

Feature scaling is a method that lets us analyse the values of two different features (variables) in a dataset in the most accurate way. So when is it used, and how is it done? Let us look for the answers together.

To understand feature scaling, let us set ourselves a problem. There are two people. One of them earns 5000 Turkish lira a month, the other earns 200000 Turkish lira a month. Both of them set aside part of their salary for charity every month. Say the person earning 5000 Turkish lira sets aside 1500 lira, while the person earning 200000 Turkish lira sets aside 10000 lira.

At first glance we see that the person giving 10000 lira gives more. But once we look at the ratio, it turns out the person earning 5000 lira gives far more in proportion. With feature scaling we bring the numeric values of these features close to each other and compare ratios instead of raw values. We then see that the person earning 5000 lira has the higher ratio. Now that we have an example of what feature scaling is for and what it means, we can move on to the methods.

Normalisation

Normalisation is the method of mapping variables to values between 0 and 1.

x ′ = ( x − x m i n ) / ( x m a x − x m i n )

We can add the normalisation code after the code we wrote during preprocessing.

from sklearn.preprocessing import MinMaxScaler
mms = MinMaxScaler()
X_train_normed = mms.fit_transform(X_train) //applying normalisation to the training set
X_test_normed= mms.transform(X_test) //applying normalisation to the test set

Standardisation

This builds a standard normal distribution in which the variable (feature) columns have a mean of 0 and a standard deviation of 1.

μ: mean, σ: standard deviation, x: value of the variable

As shown above we can find the z scores. From then on the columns hold z scores instead of the raw variables, which makes it easier to run analyses in terms of probabilities.

from sklearn.preprocessing import StandardScaler
stdsc = StandardScaler()
X_train_std = stdsc.fit_transform(X_train)//standardising the training set
X_test_std = stdsc.transform(X_test)//standardising the test set

Conclusion

If you have any doubt about whether your algorithm needs standardisation or normalisation, do it. In areas like KNN and neural networks it will make your algorithm train far more accurately. Doing both makes no sense. One of the two is enough.

In my next post I will start on regression, and we will properly begin with machine learning. See you then.

Good luck with it.