Abstract

In this RCC , I will cover the topic of combining generative and discriminative learning approaches for classification. I will continue where we left off two weeks ago on the topic of combining generative and discriminative learning. We will look at the Bishop’s paper and I will briefly present the Bouchard’s paper, contrasting the frequentist philosophy with the Bayesian one. For the Bouchard’s paper, only look at the first two sections and maybe quickly browse over the experimental section – the theoretical section is full of typos and most probably not correct.

Below is the summary from two weeks ago for your reference:

—————-

I will start by covering a classic:

• On Discriminative vs. Generative Classifiers: A comparison of logistic regression and Naive Bayes, Andrew Y. Ng and Michael Jordan. In NIPS 14 , 2002. pdf

This paper is unfortunately not an easy read, but it is one of the most cited on the topic of comparing generative and discriminative learning. The analysis is somewhat specific, but still provides interesting insight.

I will then compare the two hybrid generative / discriminative approaches proposed in the following two papers (the first one being more Bayesian; the second one being more Frequentist):

• Generative or Discriminative? getting the best of both worlds , Bishop, C. M. and Lasserre, J. (2007) In Bayesian Statistics 8, Bernardo, J. M. et al. (Eds), Oxford University Press. 3–23. With discussion. pdf
• Bias-variance tradeoff in hybrid generative-discriminative models, G. Bouchard. In proc. of the Sixth International conference on Machine Learning and Applications (ICMLA 07), Cincinnati, Ohio, USA , 13-15 Dec.2007., 2007. pdf

If you have time to read only one paper, read the Bishop’s one (I will focus more on it) and skim through the other two. A slightly shorter conference version is available here .

If you are quite interested on the topic and you want more details, have a look at the PhD thesis of Julia Lassere and chapter 5 of the PhD thesis of Guillaume Bouchard (chapter 5 is in English even though the several other chapters are in French).