Feature Scaling: Min-Max Normalization vs Standardization
Feature scaling puts numeric attributes on a similar range. Min-max scaling squeezes values into 0 to 1; standardization gives zero mean and unit variance.
With few exceptions, ML algorithms do not perform well when the input numerical attributes have very different scales (for example total rooms from 6 to 39,320 while median income runs from 0 to 15). Scaling target values is generally not required.
Min-max scaling (normalization): subtract the minimum and divide by max minus min, so values end up between 0 and 1. Scikit-Learn provides MinMaxScaler, with a feature_range hyperparameter if you want a range other than 0 to 1. Standardization: subtract the mean (so the standardized values have a zero mean) and divide by the standard deviation (so the distribution has unit variance). It does not bound values to a specific range, which can be a problem for some algorithms such as neural networks that expect inputs from 0 to 1, but it is much less affected by outliers. Scikit-Learn provides StandardScaler.
Fit the scalers on the training data only, then transform the training and test sets with the learned numbers, to avoid leaking test information. Pipelines chain such transformers in order.
Key points
- Min-max: (x - min) / (max - min), range 0 to 1, MinMaxScaler.
- Standardization: (x - mean) / std, StandardScaler.
- Standardization does not bound values but is much less affected by outliers.
- Fit scalers on the training set only.
- Most ML algorithms prefer numbers on similar scales.
Worked example
Values 10, 20, 40 with min 10 and max 40: min-max gives 0, 0.33, 1. For standardization the mean is 23.3 and std about 12.5, so the values become -1.07, -0.27, 1.33.
Common questions
Which is less affected by outliers?
Standardization. One extreme value can squeeze all other min-max scaled values into a tiny range.
Does standardization bound values between 0 and 1?
No, it does not bound values to a specific range.
Practice questions on this topic
Try these, then open the app to check your answers with explanations.
- If your training data is full of errors, outliers, and noise is.......
- A. Poor Quality Data
- B. Irrelevant Features
- C. Nonrepresentative Training Data
- D. All of them are true
- In Min-max Scaling, by default, values are shifted and rescaled so that they end up ranging from -1 to 1
- A. true
- B. false
- Standardization bound values to a specific range
- A. true
- B. false
- Scikit-Learn provides a transformer called StandardScaler for standardization.
- A. true
- B. false
- Scikit-Learn provides a transformer called MinMaxScaler for normalization
- A. true
- B. false
- Standardization is much less affected by outliers
- A. true
- B. false
Related topics
- RMSE vs MAE: Regression Error Metrics and Norms
- What is Machine Learning?
- Supervised vs Unsupervised vs Semi-supervised vs Reinforcement Learning
- Overfitting and Underfitting in Machine Learning
- Batch Learning vs Online Learning
- Confusion Matrix, Precision, Recall and F1 Score
- All topics: Machine Learning
BASMAGLY is an independent student project, not an official Delta University site. Check lectures and your instructor for the final word.