Generative (e.g., naïve bayes, GMM)
- Assume some functional form P(X|Y), P(Y)
- This is the ‘generative’ model
- Estimate parameters of P(X|Y),P(Y) directly from training data
- Use bayes rule to calculate P(Y|x=xi)
Discriminative (e.g., SVM, Linear Regression, LDA)
- Assume some functional form for P(Y|X)
- This is the ‘discriminative’ model
- Estimate parameters of P(Y|X) directly from data
Examples:
Generative Model:
Gaussian mixture model and other types of mixture model
Hidden Markov model
Probabilistic context-free grammar
Naive Bayes
Averaged one-dependence estimators
Latent Dirichlet allocation
Restricted Boltzmann machine
Probabilistic Linear discriminant analysis (PLDA)
Discriminative Model:
Logistic regression
Support vector machines
Boosting (meta-algorithm)
Conditional random fields
Linear regression
Neural networks
Linear discriminant analysis (LDA)
