Generative Classifier vs. Discriminative Classifier

Generative (e.g., naïve bayes, GMM)

• Assume some functional form P(X|Y), P(Y)
• This is the ‘generative’ model
• Estimate parameters of P(X|Y),P(Y) directly from training data
• Use bayes rule to calculate P(Y|x=xi)

Discriminative  (e.g., SVM, Linear Regression, LDA)

• Assume some functional form for P(Y|X)
• This is the ‘discriminative’ model
• Estimate parameters of P(Y|X) directly from data

Examples:


Generative Model:

Gaussian mixture model and other types of mixture model
Hidden Markov model
Probabilistic context-free grammar
Naive Bayes
Averaged one-dependence estimators
Latent Dirichlet allocation
Restricted Boltzmann machine
Probabilistic Linear discriminant analysis (PLDA)

Discriminative Model:

Logistic regression
Support vector machines
Boosting (meta-algorithm)
Conditional random fields
Linear regression
Neural networks
Linear discriminant analysis (LDA)