{"id":3565,"date":"2024-09-02T20:55:10","date_gmt":"2024-09-02T20:55:10","guid":{"rendered":"https:\/\/workhouse.sweetdishy.com\/?p=3565"},"modified":"2024-09-02T20:55:11","modified_gmt":"2024-09-02T20:55:11","slug":"naive-bayes-classification","status":"publish","type":"post","link":"https:\/\/workhouse.sweetdishy.com\/index.php\/2024\/09\/02\/naive-bayes-classification\/","title":{"rendered":"\u00a0Na\u00efve Bayes Classification"},"content":{"rendered":"\n

Data science has progressed from simple linear regression models to complex techniques but practitioners still prefer the models that are simple and easy to interpret. In this widely used category of algorithms Na\u00efve Bayes algorithm is one of the prominent names as it is not only simple but so powerful that it outperforms complex algorithms for very large datasets.<\/p>\n\n\n\n

Naive Bayes is a probabilistic machine learning algorithm based on the Bayes Theorem. This algorithm is extensively used in a wide range of classification tasks varying from<\/em> filtering spam, classifying documents to predicting sentiments.<\/p>\n\n\n\n

 <\/p>\n\n\n\n

Programming Tip<\/strong>: The tern na\u00efve means that features that are used in the model are independent of each other. So when one feature change, the other is not affected.<\/p>\n\n\n\n

<\/a>2.6.1 Understanding Conditional Probability<\/strong><\/h4>\n\n\n\n

To understand conditional probability, consider an example of flipping a coin. There are equal chances of getting either heads or tails. So, the probability of getting heads is 50%.<\/p>\n\n\n\n

Now, in case we are supposed to find the probability of getting a queen spade from a deck of 52 cards then the denominator is 13 (the eligible population) and not 52. Since there is only one queen in 13 spades, the probability that we get a queen spade is 1\/13.<\/p>\n\n\n\n

\"images\"<\/p>\n\n\n\n

<\/a>FIGURE 2.27<\/strong> Sample Dataset<\/p>\n\n\n\n

Therefore, we can say that the conditional probability of A given B, denotes the probability of occurring A given that B has already occurred. Mathematically, it can be expressed as, P(A|B) = P(A AND B) \/ P(B).<\/p>\n\n\n\n

Let\u2019s apply this concept on the data set given in\u00a0Fig. 2.27\u00a0and calculate the conditional probability that the teacher is a male.<\/p>\n\n\n\n

So, the required conditional probability P(Teacher | Male) = 12 \/ 60 = 0.2.<\/p>\n\n\n\n

 <\/p>\n\n\n\n

Programming Tip:<\/strong> If Y has two categories, then calculate the probability of each class of Y and select the one with higher value.<\/p>\n\n\n\n

\"images\"\/<\/figure>\n\n\n\n

Likewise, the conditional probability of B given A can be computed as,<\/p>\n\n\n\n

\"images\"\/<\/figure>\n\n\n\n
\"images\"\/<\/figure>\n\n\n\n

<\/a>2.6.2 The Bayes Rule<\/strong><\/h4>\n\n\n\n

The Bayes Rule uses P(X|Y), known from the training dataset, to find P(Y|X). For observations in test dataset, X would be known while Y is unknown. And for each row of the test dataset, we have to find the probability of Y given that X has already happened (refer\u00a0Fig. 2.28).<\/p>\n\n\n\n

\"images\"<\/p>\n\n\n\n

<\/a>FIGURE 2.28<\/strong> Bayes Rule for Conditional Probability<\/p>\n\n\n\n

<\/a>2.6.3 Types of Events<\/strong><\/h4>\n\n\n\n

If two probabilities, P(B) and P(B|A) are the same, then it means that the occurrence of event A had no effect on event B. Therefore, event A and event B are independent events<\/em>.<\/p>\n\n\n\n

If the conditional probability is zero, then it means that the occurrence of event A implies that event B cannot occur. If the reverse is also true, then the events are said to be mutually exclusive events. In such a case, only one event can take place at a time.<\/p>\n\n\n\n

All other cases are classified as dependent events where the conditional probability can be either lower or higher than the original.<\/p>\n\n\n\n

For example, if we toss a coin twice and we want to calculate the probability of getting a head both times, then we must consider that the second event is independent of the first. The desired probability is P(Heads in first throw)\"images\"P(Heads in second throw)= \u00bd X \u00bd = \u00bc<\/p>\n\n\n\n

Mathematically, we can say that if the events are not independent, we multiply the probability of one event with the probability of the second event.<\/p>\n\n\n\n

For expressing dependent events mathematically, we multiply the probability of any one event with the probability of the second event after the first has happened.<\/p>\n\n\n\n

P(A and B)=P(A)\"images\"P(B|A) or P(A and B)=P(B)\"images\"P(A|B)<\/p>\n\n\n\n

<\/a>For example, while drawing two cards (King and Queen) without replacement, the probability of the first event is dependent of 52 cards whereas the probability of the second event is dependent on 51 cards.<\/p>\n\n\n\n

This means that P(King and Queen)=P(Queen)P(King|Queen)<\/strong><\/p>\n\n\n\n

So if P(Queen) is 4\/52 then once the King is drawn, P(King|Queen) is 4\/51 as now only 51 cards are left. Therefore, the desired probability = 4\/52 X 4\/51<\/p>\n\n\n\n

The probability of two non-mutually exclusive events is given as, P(A OR B)=P(A) + P(B) – P(A AND B)<\/strong>.<\/p>\n\n\n\n

For example, in a throw of dice, the probability of getting a number that is multiple of 2 or 3 is a scenario of events which are not mutually exclusive since 6 is both a multiple of 2 and 3 and is counted twice. Therefore, P(multiple of 2 or 3) = P(Multiple of 2) + P(Multiple of 3) – P(Multiple of 2 AND 3) = P(2, 4, 6) + P(3, 6) – P(6) = 3\/6 + 2\/6 -1\/6 = 4\/6 = 2\/3<\/strong><\/p>\n\n\n\n

<\/a>2.6.4 Naive Bayes Algorithm<\/strong><\/h4>\n\n\n\n

In real-world problems, there are multiple X variables. So, when the features are independent, the Bayes Rule can be extended to what is called Naive Bayes. It is called \u2018Naive\u2019 because of the naive assumption that the Xs are independent of each other.<\/p>\n\n\n\n

Naive Bayes classification algorithm can be used with continuous features but performs best for categorical variables. When dealing with numeric features, the algorithm assumes that numerical values are normally distributed.<\/p>\n\n\n\n

<\/a>2.6.5 Laplace Correction<\/strong><\/h4>\n\n\n\n

We can better understand the importance of Laplace correction with an example. If we were to classify fruits as mango, orange or banana, then the value of P(Long | Orange) will be zero as there are no \u2018Long\u2019 oranges.<\/p>\n\n\n\n

When we use this probability in the model having multiple features, then the entire probability will become zero as anything multiplied by zero is zero. To avoid this, we need to increase the count of the variable with zero to a small value (usually 1) in the numerator, so that the overall probability does not become zero.<\/p>\n\n\n\n

This correction is called \u2018Laplace Correction and the function for building the Naive Bayes model will accept this correction as a parameter.<\/p>\n\n\n\n

<\/a>2.6.6 Pros and Cons of Naive Bayes Algorithm<\/strong><\/h4>\n\n\n\n

Pros<\/strong><\/h4>\n\n\n\n
    \n
  1. It is a simple algorithm that is easy to build and understand.<\/li>\n\n\n\n
  2. It predicts classes faster than many other classification algorithms.<\/li>\n\n\n\n
  3. It can be easily trained using a small data set.<\/li>\n\n\n\n
  4. It can be used to predict values for even large data sets.<\/li>\n<\/ol>\n\n\n\n

    <\/a>Cons<\/strong><\/h4>\n\n\n\n