Logistic Regression

“Those who can imagine anything, can create the impossible…”
– Alan Turing

We have already seen that the classification task falls under supervised machine learning, where we assume that there is a hidden rule f\colon\mathcal{X}\to\mathcal{Y} mapping inputs in \mathcal{X} to a set \mathcal{Y} containing K many unordered, discrete class labels, i.e., \mathcal{Y}=\{\tt 1, \tt 2, \ldots, \tt K\}.

Our objective is to learn the rule from a given labeled or classified dataset \mathcal{D}=\{(\bold{x}_i, y_i)\}_{i=1}^N, where \bold{x}_i\in\mathcal{X} is an observed feature vector and y_i\in\mathcal{Y} is the corresponding label. Once we have learned an approximate mapping \widehat{f}\colon\mathcal{X}\to\mathcal{Y}, it can be used to predict the class of an unseen observation. Examples include classifying Iris flowers, handwritten digits, etc.

In this chapter, we use Logistic Regression to handle two types of classification problem: binary and multiclass. There is another kind, known as multilabel classification, which you will learn next semester.

TipChapter Objectives

In this chapter, we visit logistic regression

  1. Logistic Regression: binary and multiclass

  2. Evaluation metrics: confusion matrix, F1-score

  3. Introduction to scikit-learn

  4. Applications to Breast Cancer and MNIST datasets