AI, machine learning and deep learning · Intermediate

Candidate Generation

Written by SignalsLab

Early edition: these lessons are being reviewed.

Content

Summary

Candidate generation is the initial phase in recommendation systems that narrows a vast item pool into a manageable subset for ranking, primarily using content-based or collaborative filtering. Content-based filtering relies on feature matrices and similarity metrics like the dot product to match user history, while collaborative filtering leverages collective user-item interaction data to suggest items based on similar users.

Intuition

Think of a recommendation system as a two-stage filter. First, you have millions of items, which is too many to process deeply. Candidate generation acts as the coarse sieve, quickly selecting a small, relevant subset of items to pass on. It does this either by looking at what an individual user has liked before (content-based) or by seeing what similar users have liked (collaborative). This step is crucial because it prevents the system from wasting computational resources on irrelevant items before the final ranking occurs.

Derivation

  1. Candidate generation is defined as the preliminary step occurring before the final ranking step.
  2. Its primary function is to narrow down a vast pool of items to a manageable subset.
  3. Two methodologies are employed: content-based filtering and collaborative filtering.
  4. Content-based filtering uses a feature matrix where rows are items and columns are features.
  5. A similarity metric, such as the dot product, scores items against the user's profile.
  6. Collaborative filtering analyzes similarities between users and items using interaction data.
  7. Content-based filtering scales well but cannot expand interests beyond existing history.
  8. Collaborative filtering mitigates content-based limitations by incorporating cross-user data.

Worked example

Problem

A user has previously liked two apps: App A (features: [1, 0, 1]) and App B (features: [1, 1, 0]). A new candidate, App C, has features [1, 0, 0]. Using a simplified binary content-based approach, determine if App C is a strong candidate based on the dot product similarity with the user's profile, assuming the user profile is the sum of liked items.

  1. Construct the user profile vector by summing the feature vectors of liked items: [1, 0, 1] + [1, 1, 0] =.
  2. Identify the feature vector for the candidate App C: [1, 0, 0].
  3. Calculate the dot product between the user profile and App C [1, 0, 0].
  4. Dot product = (2*1) + (1*0) + (1*0) = 2.
  5. A non-zero score indicates similarity; thus, App C is a candidate.

Answer App C is a candidate with a similarity score of 2.

Common mistakes

  • Assuming content-based filtering can discover new interests outside a user's historical data.
  • Confusing the candidate generation step with the final ranking step.
  • Overlooking that content-based filtering does not leverage other users' features.

Animation

Animation coming soon

The animated version of this lesson is being prepared for the web. In the Signals AI app, lessons are drawn step by step.

Code

This is a text lesson: it has no animation code yet.

ccBYNC© 2026 SignalsLab. Lesson text shared as is under CC BY-NC 4.0.