7 Linear approaches

The previous chapters distinguished between two types of supervised learning tasks according to the type of output

  • Regression : when we predict quantitative outputs, e.g. real or integer numbers. Predicting the weight of an animal on the basis of its age and height is an example of regression problem.

  • Classification (or pattern recognition) : where we predict qualitative or categorical outputs which assume values in a finite set of classes (e.g. black, white and red) where there is no explicit ordering. Qualitative variables are also referred to as factors. Predicting the class of an email on the basis of English words frequency is an example of classification task.

This chapter will consider learning approaches to classification and regression where the hypothesis functions are linear combinations of the input variables.