Biostatistics · Intermediate

Generalized Linear Models

Written by SignalsLab

Early edition: these lessons are being reviewed.

Content

Summary

Generalized Linear Models (GLMs) extend linear regression to handle non-normal responses by combining an exponential family distribution, a linear predictor, and a link function. This framework allows for modeling discrete, count, or bounded data while maintaining interpretability and respecting natural constraints. GLMs address limitations of linear models, such as predicting impossible values or violating homoscedasticity assumptions.

Intuition

Linear regression assumes a constant variance and normal errors, which fails for data like counts or probabilities. For instance, predicting negative bike usage is nonsensical, and variance often increases with the mean. GLMs solve this by specifying the correct distribution (like Poisson for counts) and using a link function to map the mean to the linear predictor. This ensures predictions stay within valid ranges, such as probabilities between 0 and 1 or positive counts, without needing ad-hoc transformations that may be undefined for zeros.

Key equations

ηi=∑k=1pXikβk
(1)

The linear predictor, a linear combination of predictors and coefficients.

g(μi)=ηi
(2)

The link function connecting the mean of the response to the linear predictor.

μi=exp⁡(ηi)1+exp⁡(ηi)
(3)

The inverse logit (expit) function for logistic regression, mapping the linear predictor back to a probability.

μi=exp⁡(ηi)
(4)

The inverse log link for Poisson regression, ensuring the mean is positive.

0=∑i=1n(Yi−μi)Var(Yi)Wi
(5)

The normal equations solved to obtain GLM estimates, where Wi are derivative terms.

Derivation

  1. GLMs are defined by three components: an exponential family distribution, a linear predictor, and a link function.
  2. The linear predictor is calculated as a weighted sum of predictors: ηi=∑k=1pXikβk.
  3. The link function g connects the mean μi to the linear predictor: g(μi)=ηi.
  4. For binary data, the logit link maps probabilities to the real line, using the inverse expit function to recover probabilities.
  5. For count data, the log link ensures positive means, using the inverse exp function.
  6. Parameters are estimated by maximizing likelihood, solving the normal equations involving variance-dependent weights.

Worked example

Problem

A researcher models bike-sharing usage (count data) using a GLM. The linear predictor for a specific day is calculated as ηi=2.5. Using the Poisson GLM framework, calculate the expected number of bikes used (μi).

  1. Identify the appropriate link function for count data: the log link, g(μ)=log⁡(μ).
  2. The inverse link function is μi=exp⁡(ηi).
  3. Substitute the given linear predictor value: μi=exp⁡(2.5).
  4. Calculate the exponential: exp⁡(2.5)≈12.18.

Answer The expected number of bikes used is approximately 12.18.

Common mistakes

  • Using transformations like log(Y) can be problematic if Y contains zeros or negative values, whereas GLMs handle these through the link function.
  • Linear models assume constant variance (homoscedasticity), but many responses like counts have variance that depends on the mean (heteroscedasticity).
  • GLMs model the mean/probability via the link function, not the raw outcome variable Y, which remains discrete or bounded.
  • Standard errors in GLMs are obtained numerically and rely on large sample theory, which may not hold for small samples or boundary data.

Animation

Animation coming soon

The animated version of this lesson is being prepared for the web. In the Signals AI app, lessons are drawn step by step.

Code

This is a text lesson: it has no animation code yet.

ccBYNC© 2026 SignalsLab. Lesson text shared as is under CC BY-NC 4.0.