Bayes and Naive Bayes are very important techniques in machine learning. I am going to cover the Naive Bayes Classifier which is widley used in machine learning, but before that I will explain in this post the Bayes’ theorm by examples

## A brief introduction for Bayes’ theorem:

Let ** A** and

**denote two events. In Bayes’ theorem,**

*B***the probability that**

*P(A|B)***occurs given**

*A***already occured can be computed by:**

*B*Where ** P(B|A)** is the probability of observing

**given**

*B***occurs, and**

*A***,**

*P(A)***the probability of**

*P(B)***and**

*A***occurs repectively.**

*B*## Examples:

I am copying these examples from many books and tutorial sites, and I am listing these references here:

### Example 1: Coin flip

Given two coins, one is unfair with 90% of flips getting a head and 10% getting a tail, another one is fair. Randomly pick one coin and flip it. What is the probability that this coin is the unfair one, if we get a head?

Let’s denote ** U**, the event of picking the unfair coin and

**the event of getting a head. so the probability of unfair coin given a head is already observed is ***P(U|H) can be calculated as:**

*H*** P(H|U)** = 90%

** P(U)** = 50% as we randomly pick a coin out of two.

** P(H)** can be observed by two paths, the fair coin is picked

**and the unfair coin is picked**

*P(F)***.**

*P(U)***= P(H|U)P(U) + P(H|F)P(F)**

*P(H)*### Example 2: Cancer screening

Suppose a physician reported the following cancer screening test scenario:

Cancer | No Cancer | Total | |
---|---|---|---|

Text Positive | 80 | 900 | 980 |

Text Negative | 20 | 9000 | 9020 |

Total | 100 | 9900 | 10000 |

The problem: if the result of this screening test on a person is positive, what is the probability that they actually have cancer?

Let’s assign the event of having a cancer as ** C** and positive testing as

**. let’s calculate**

*Pos***:**

*P(C|Pos)*### Example 3: Email Spam detector:

Let us start with this training data

ID | Terms in email | Is spam | |
---|---|---|---|

Training Data | 1 | Click win prize | Yes |

2 | Click meeting setup meeting | No | |

3 | Prize free prize | Yes | |

4 | Click prize free | Yes | |

Testing Case | 5 | Free setup meeting free | ? |

First we define two events: spam and not spam $S$ and $NS$ respectively.

From the training set:

$P(S) = \frac{3}{4}$

$P(NS) = \frac{1}{4}$

Or we can **impose** an assumption of prior to be $P(s) = 1\%$.

To calculate $P(S|x)$ where $x = (free, setup, meeting, click)$, we start by calculate $P(x_i|S)$.

$P(free|S) = \frac{2+1}{9+6} = \frac{3}{15}$

$P(free|NS) = \frac{0+1}{4+6} = \frac{1}{10}$

$P(setup|S) = \frac{0+1}{9+6} = \frac{1}{15}$

$P(setup|NS) = \frac{1+1}{4+6} = \frac{2}{10}$

$P(meeting|S) = \frac{0+1}{9+6} = \frac{1}{15}$

$P(meeting|NS) = \frac{2+1}{4+6} = \frac{3}{10}$

So, the solution:

## List of posts

This post is part of a series of posts

- Preperation and introduction
- Naive Bayes by example (this post)
- Scrubbing natural language text.
- Naive Bayes’ Classifire.
- Writing Naive Bayes from scratch
- Using Scikit-learn library