Naive Bayes Classifier Algorithm

In the world of Machine Learning, simplicity often brings powerful results. One such simple yet remarkably effective algorithm is the Naïve Bayes Classifier. Based on Bayes’ Theorem, this supervised learning algorithm is a go-to choice for solving classification problems—especially when working with large, high-dimensional datasets like text data.

Complete Python Course with Advance topics:-Click Here
SQL Tutorial :-Click Here

🔍 What is Naïve Bayes?

Using Bayes’ Theorem, the probabilistic classifier Naïve Bayes predicts a given data point’s class based on the likelihood of its attributes. It’s widely used in applications like:

📧 Spam filtering
💬 Sentiment analysis
📰 News classification

Why is it called “Naïve” Bayes?

The name comes from two concepts:

Naïve: It makes the assumption that each attribute exists independently of the others. When determining if a fruit is an apple, for example, Naïve Bayes takes into account each of the following factors separately: the fruit’s colour, shape, and sweetness.
Bayes: Its foundation is Bayes’ Theorem, which determines the likelihood of a hypothesis based on known facts.

📘 Bayes’ Theorem: The Heart of It All

Bayes’ Theorem gives us a mathematical way to update our beliefs based on new evidence: P(A∣B)=P(B∣A)⋅P(A)P(B)P(A|B) = \frac{P(B|A) \cdot P(A)}{P(B)}

Where:

P(A|B): Probability of hypothesis A given data B is known as posterior probability.
P(B|A): Probability of data B in the event that hypothesis A is correct is known as likelihood.
P(A): Prior probability (Initial probability of hypothesis A)
P(B): Marginal likelihood (Total probability of data B)

Download New Real Time Projects :-Click here

⚙️ How Naïve Bayes Works: A Practical Example

Let’s say you have a dataset of weather conditions and whether a player plays or not.

Outlook	Play
Rainy	Yes
Sunny	Yes
Overcast	Yes
…	…

If the forecast is sunny, let’s figure out if the player will play.

Step 1: Frequency Table

Weather	Yes	No
Overcast	5	0
Rainy	2	2
Sunny	3	2

Step 2: Probabilities

P(Sunny|Yes) = 3/10 = 0.3
P(Sunny|No) = 2/4 = 0.5
P(Yes) = 10/14 = 0.71
P(No) = 4/14 = 0.29
P(Sunny) = 5/14 = 0.35

Step 3: Applying Bayes’ Theorem

P(Yes|Sunny) = (0.3 * 0.71) / 0.35 = 0.60
P(No|Sunny) = (0.5 * 0.29) / 0.35 = 0.41

✅ Conclusion: Since P(Yes|Sunny) > P(No|Sunny), the player should play on a sunny day.

✅ Advantages of Naïve Bayes

Fast and simple to implement
Works well with large datasets
Efficient in multi-class prediction
Performs well with text classification

⚠️ Disadvantages

Presupposes feature independence, which may not always be the case.
Can’t learn interactions between features

🛠 Applications of Naïve Bayes

Email spam filtering
Medical diagnosis
Credit scoring
Sentiment analysis
Real-time prediction systems

🧠 Types of Naïve Bayes Models

Gaussian Naïve Bayes: Assumes a normal distribution for features. ideal for continuous data.
Multinomial Naïve Bayes: Perfect for classifying documents according to word frequencies.
Bernoulli Naïve Bayes: Using binary/boolean characteristics, it is comparable to multinomial.

💻 Naïve Bayes Classifier in Python – Step-by-Step

Let’s walk through a simple implementation using the user_data.csv dataset.

🔹 Step 1: Data Pre-processing

import numpy as np  
import matplotlib.pyplot as plt  
import pandas as pd  

# Load dataset
dataset = pd.read_csv('user_data.csv')  
x = dataset.iloc[:, [2, 3]].values  
y = dataset.iloc[:, 4].values  

# Split data
from sklearn.model_selection import train_test_split  
x_train, x_test, y_train, y_test = train_test_split(x, y, test_size=0.25, random_state=0)  

# Feature Scaling
from sklearn.preprocessing import StandardScaler  
sc = StandardScaler()  
x_train = sc.fit_transform(x_train)  
x_test = sc.transform(x_test)

🔹 Step 2: Fit Naïve Bayes Model

from sklearn.naive_bayes import GaussianNB  
classifier = GaussianNB()  
classifier.fit(x_train, y_train)

🔹 Step 3: Predict Test Set

y_pred = classifier.predict(x_test)

🔹 Step 4: Confusion Matrix (Accuracy Check)

from sklearn.metrics import confusion_matrix  
cm = confusion_matrix(y_test, y_pred)  
print(cm)

🔹 Step 5 & 6: Visualization of Training and Test Set

from matplotlib.colors import ListedColormap

def visualize_set(x_set, y_set, title):
    X1, X2 = np.meshgrid(
        np.arange(start = x_set[:, 0].min() - 1, stop = x_set[:, 0].max() + 1, step = 0.01),
        np.arange(start = x_set[:, 1].min() - 1, stop = x_set[:, 1].max() + 1, step = 0.01)
    )
    plt.contourf(X1, X2, classifier.predict(np.array([X1.ravel(), X2.ravel()]).T).reshape(X1.shape),
                 alpha = 0.75, cmap = ListedColormap(('purple', 'green')))
    plt.xlim(X1.min(), X1.max())
    plt.ylim(X2.min(), X2.max())

    for i, j in enumerate(np.unique(y_set)):
        plt.scatter(x_set[y_set == j, 0], x_set[y_set == j, 1],
                    c = ListedColormap(('purple', 'green'))(i), label = j)
    plt.title(title)
    plt.xlabel('Age')
    plt.ylabel('Estimated Salary')
    plt.legend()
    plt.show()

# Visualize Training set
visualize_set(x_train, y_train, 'Naive Bayes (Training set)')

# Visualize Test set
visualize_set(x_test, y_test, 'Naive Bayes (Test set)')

🎯 Final Thoughts

The Naïve Bayes Classifier might appear “naïve,” but in practice, it delivers speed, simplicity, and surprisingly high accuracy, especially in text-based applications. Whether you’re classifying emails or diagnosing medical conditions, Naïve Bayes offers a foundational, beginner-friendly path into the world of Machine Learning classification models.

naive bayes classifier algorithm
naive bayes classifier algorithm numerical example
naive bayes classifier algorithm in machine learning
naive bayes classifier algorithm example
naive bayes classifier algorithm in data mining with example
Naive Bayes Classifier Algorithm python code
naive bayesian classification in data mining
naive bayes’ theorem
Naive Bayes Classifier Algorithm python
naive bayes classifier algorithm in machine learning
naive bayes classifier algorithm geeksforgeeks

Post Views: 582