AI, machine learning and deep learning · Intermediate

Example – Diagnosing breast cancer with the k-NN algorithm

Written by SignalsLab

Early edition: these lessons are being reviewed.

Content

Summary

The k-NN algorithm is applied to the Wisconsin Breast Cancer Diagnostic dataset, which contains 569 examples with 30 numeric features, to automate the classification of cell nuclei as benign or malignant. This instance-based, non-parametric approach relies on measuring similarity between new instances and existing labeled examples to predict diagnoses. A critical preprocessing step is the exclusion of the patient identification number to prevent overfitting and ensure the model generalizes effectively.

Intuition

Think of k-NN as a 'lazy' learner that doesn't build a complex theoretical model of cancer biology. Instead, it memorizes the 569 known biopsy cases from the dataset. When a new patient's data arrives, the algorithm simply looks at the 'nearest neighbors'—the most similar existing cases based on physical characteristics like radius and texture. If most of those neighbors are malignant, the new case is classified as malignant. This method avoids preconceived functional forms, allowing it to find natural patterns in the data, but it requires careful data preparation, specifically removing non-predictive identifiers that could trick the model.

Derivation

  1. The goal is to automate the diagnosis of breast cancer masses as benign or malignant.
  2. The dataset used is the Wisconsin Breast Cancer Diagnostic dataset, containing 569 examples.
  3. Each example has 32 features: an ID, a diagnosis label, and 30 numeric features.
  4. The 30 numeric features represent the mean, standard error, and worst value for 10 nuclear characteristics (e.g., radius, texture).
  5. The algorithm is instance-based and non-parametric, meaning it does not learn parameters or generate theories about data structures.
  6. The identification number is excluded from the model because it uniquely predicts each example, leading to overfitting.
  7. The remaining 30 numeric features are used to calculate similarity between new instances and existing labeled examples.
  8. The classification is determined by the majority vote of the nearest neighbors in the feature space.

Worked example

Problem

A researcher loads the Wisconsin Breast Cancer dataset into R and attempts to train a k-NN model using all 32 columns, including the patient ID. What is the likely outcome and why?

  1. The dataset contains 569 unique patient IDs.
  2. Including the ID variable allows the model to memorize the specific ID for each training example.
  3. Since each ID is unique, the model can perfectly predict the training data by matching IDs.
  4. This leads to overfitting, where the model fails to generalize to new, unseen data.
  5. The correct approach is to drop the first column (ID) before training.

Answer The model will suffer from overfitting and fail to generalize because the ID variable uniquely predicts each example, providing no useful information for predicting malignancy based on physical characteristics.

Common mistakes

  • Including ID variables in the model leads to overfitting because the ID can uniquely predict each example.
  • Non-parametric methods limit the ability to understand how the classifier uses data because they do not generate theories about underlying data.

Animation

Animation coming soon

The animated version of this lesson is being prepared for the web. In the Signals AI app, lessons are drawn step by step.

Code

This is a text lesson: it has no animation code yet.

ccBYNC© 2026 SignalsLab. Lesson text shared as is under CC BY-NC 4.0.