AI, machine learning and deep learning · Intermediate

Estimating Class Probabilities

Written by SignalsLab

Early edition: these lessons are being reviewed.

Content

Summary

Decision trees estimate class probabilities by calculating the ratio of training instances for each class within the leaf node reached by an input instance. This method produces interpretable, white-box predictions where the final class is simply the one with the highest estimated probability. However, these probabilities remain constant for all instances falling into the same leaf node, regardless of minor input variations.

Intuition

Think of a decision tree as a set of rigid buckets. When you drop an instance (like a flower) into the tree, it falls into a specific leaf node, which acts as a bucket containing a fixed collection of training examples. The probability of the instance belonging to a certain class is determined solely by the composition of that bucket. If the bucket contains 54 flowers, and 49 are of a specific type, the probability is fixed at that ratio. Even if you change the input slightly but it still lands in the same bucket, the probability does not change because the underlying data count in that node has not changed. This makes the model highly interpretable but also means the probability estimates are discrete rather than continuous.

Derivation

  1. The model traverses the tree structure based on the input instance's features to identify the specific leaf node.
  2. The system counts the total number of training instances present in that identified leaf node.
  3. For each class k, the system counts the number of training instances in that leaf node belonging to class k.
  4. The estimated probability for class k is calculated as the ratio of class k instances to the total instances in the leaf.
  5. The predicted class is selected as the one with the maximum estimated probability.

Worked example

Problem

Estimate class probabilities and predict the class for a flower with petals 5 cm long and 1.5 cm wide, given that it falls into a leaf node with 54 training instances: 0 Iris setosa, 49 Iris versicolor, and 5 Iris virginica.

  1. Identify the leaf node containing 54 total training instances.
  2. Calculate probability for Iris setosa: 0/54 = 0%.
  3. Calculate probability for Iris versicolor: 49/54 ≈ 90.7%.
  4. Calculate probability for Iris virginica: 5/54 ≈ 9.3%.
  5. Compare probabilities: 90.7% is the highest.
  6. Select Iris versicolor as the predicted class.

Answer Probabilities: [0.0, 0.907, 0.093]; Predicted Class: Iris versicolor (Class 1)

Common mistakes

  • Assuming that probability estimates change continuously or intuitively with small changes in input features within the same leaf node; the estimates are fixed based on the training data count in that node.

Animation

Animation coming soon

The animated version of this lesson is being prepared for the web. In the Signals AI app, lessons are drawn step by step.

Code

This is a text lesson: it has no animation code yet.

ccBYNC© 2026 SignalsLab. Lesson text shared as is under CC BY-NC 4.0.