When should feature scaling be used in machine learning?
Conditions
- Model uses distance metrics or iterative optimization sensitive to feature scale.
- Model is not tree-based or an ensemble of trees.
Reasoning, step by step
- Identify the algorithm type.
- If it uses Euclidean distance (e.g., KNN, K-means) or gradient descent (e.g., ANN, CNN, linear/logistic regression), apply scaling.
- If it is tree-based (e.g., decision trees, random forests, XGBoost), scaling is usually unnecessary because splits depend on feature conditions, not absolute magnitude.
Example
The speaker lists KNN, K-means, linear regression, logistic regression, ANN, and CNN as needing scaling, while stating decision trees, random forests, and XGBoost do not.
Common misconceptions
- Applying scaling to tree-based models unnecessarily.
- Skipping scaling for distance-based or gradient-based models, leading to poor convergence or biased distance calculations.
Watch the explanation
Connected concepts
Explore next
Related questions
Decision-tree models do not require feature scaling because they split data based on feature conditions (thresholds) rather than absolute magnitudes. Monotonic rescaling of feature values does not change the branching structure or the order of splits.
Conditions: Model is a decision tree or an ensemble of trees (random forest, XGBoost, etc.).; Splitting logic is based on comparing feature values to thresholds.
Feature scaling is needed because raw features often differ in units and magnitudes, making them incomparable. Scaling brings them onto a common scale, which is necessary for many machine learning algorithms to function properly.
Conditions: Dataset contains multiple features with different units or ranges.; The model is sensitive to feature magnitudes (e.g., distance-based or gradient-based).
Import `MinMaxScaler` from `sklearn.preprocessing`, instantiate it, and call `fit_transform` on the selected DataFrame columns. This fits the scaler to the data (learning min/max) and transforms the values to the [0, 1] range.
Conditions: Input data is numerical.; Scikit-learn library is installed.
Standardization transforms a feature value into , targeting a mean of 0 and a standard deviation of 1 for the transformed feature.
Conditions: Applied per numerical feature.; Uses the feature mean and standard deviation .; Assumes .
Import `StandardScaler` from `sklearn.preprocessing`, instantiate it, and call `fit_transform` on the selected DataFrame columns. This transforms the data to have mean 0 and standard deviation 1.
Conditions: Input data is numerical.; Scikit-learn library is installed.
Answers are generated from source material and independently checked. Consult the original video or creator if something is unclear.