Content
Summary
This lesson covers the classification of numerical data types and essential preprocessing techniques like min-max scaling and standardization to ensure features are on comparable scales. It also highlights the importance of using the median over the mean when dealing with outliers to maintain accurate data representation.
Intuition
Machine learning algorithms often struggle when features have vastly different ranges, such as age in years versus income in dollars. To fix this, we rescale data so all features contribute equally to the model. Min-max scaling squeezes data into a fixed box, usually 0 to 1, while standardization centers the data around zero with a consistent spread. Additionally, when data is skewed by extreme values, the median provides a more honest picture of the 'typical' value than the mean, which gets pulled by those extremes.
Key equations
Min-max scaling formula where x_i is the original value, min x is the minimum value in the feature, max x is the maximum value, and x_i' is the rescaled value.
Standardization formula where is the original value, is the mean of the feature, is the standard deviation, and is the standardized value.
Derivation
- Identify the type of numerical data (continuous, discrete, or ordinal) to determine appropriate preprocessing.
- For min-max scaling, locate the minimum and maximum values of the feature vector.
- Subtract the minimum value from each data point to shift the range to start at zero.
- Divide by the range (max minus min) to scale the data to the interval [0, 1].
- For standardization, calculate the mean () and standard deviation () of the feature.
- Subtract the mean from each data point to center the distribution at zero.
- Divide by the standard deviation to ensure the resulting distribution has a standard deviation of one.
Worked example
Given a feature vector [2, 4, 6, 8, 10], calculate the min-max scaled value for the element 6.
- Identify min x = 2 and max x = 10.
- Apply the formula: x_i' = (6 - 2) / (10 - 2).
- Calculate numerator: 6 - 2 = 4.
- Calculate denominator: 10 - 2 = 8.
- Divide: 4 / 8 = 0.5.
Answer 0.5
Common mistakes
- Treating ordinal data (like health ratings) as purely numerical without considering its categorical nature may lead to incorrect modeling.
- Relying solely on the mean in the presence of outliers can be misleading, as outliers skew the average significantly.
- Assuming that fit_transform and separate fit/transform operations are mathematically different, when they are not, though separate operations allow applying the same transformation to new data.