Content
Summary
Unsupervised learning lacks labels, making evaluation difficult and often requiring manual inspection to verify if clusters are semantically meaningful. Preprocessing techniques like scaling are essential unsupervised steps that prepare data for supervised algorithms by handling feature magnitudes and outliers.
Intuition
Think of unsupervised learning as exploring a dark room without a map; you can group objects, but you don't know if your groups make sense without looking closely. Because there are no correct answers to guide the algorithm, you must manually check if the results, like grouping faces by angle instead of identity, are actually useful. Before feeding this data into supervised models like SVMs or neural networks, which are sensitive to scale, you must preprocess the data. Scaling ensures that features with larger magnitudes don't dominate the model, using methods like StandardScaler for statistical balance or MinMaxScaler for fixed ranges.
Derivation
- Unsupervised learning organizes data without prior knowledge of correct outputs, such as clustering faces without knowing identities.
- The absence of labels means there is no objective way to determine if the model 'did well,' requiring manual inspection.
- Clustering algorithms may produce valid but undesired divisions, such as grouping by orientation rather than person.
- Preprocessing techniques like scaling are unsupervised because they do not use label information.
- Algorithms like neural networks and SVMs are sensitive to data scaling, necessitating preprocessing.
- StandardScaler rescales features to have a mean of 0 and variance of 1.
- RobustScaler uses median and quartiles to ignore outliers.
- MinMaxScaler shifts data to fall exactly between 0 and 1.
- Normalizer scales vectors to have a Euclidean length of 1, preserving direction.
- Scaling is applied after splitting data into training and test sets to prevent information leakage.
Worked example
You have a dataset with 569 samples and 30 measurements. You need to prepare it for a kernel SVM. How should you split the data and apply scaling to ensure valid evaluation?
- Split the dataset into training and test sets to maintain evaluation integrity.
- In this specific case, split into 426 training samples and 143 test samples.
- Instantiate a scaler (e.g., MinMaxScaler) on the training data only.
- Apply the scaler to both training and test sets to ensure inputs are properly scaled.
- This prevents information from the test set from leaking into the training process.
Answer Split 569 samples into 426 training and 143 test samples, then apply the scaler to both sets after fitting on the training data.
Common mistakes
- Assuming clustering results are correct just because groups form; they may not match semantic meaning.
- Confusing lack of labels with lack of evaluation; manual inspection is still required.
- Thinking StandardScaler ensures data falls within a specific range like [0, 1]; it only standardizes mean and variance.
- Believing RobustScaler is sensitive to outliers; it is designed to ignore them using median and quartiles.