📊 Logistic Regression in Machine Learning – A Complete Guide

Logistic Regression in Machine Learning

Machine Learning is a powerful force behind the technological revolution we’re experiencing today, and at the heart of many classification problems lies one elegant yet powerful algorithm: Logistic Regression.

Often misunderstood as a regression algorithm (because of its name), Logistic Regression is, in fact, a classification technique that helps predict categorical outcomes such as yes/no, spam/not spam, cancerous/not cancerous, and so on.

Let’s take a deep dive into this fundamental yet immensely useful algorithm.

Complete Python Course with Advance topics:-Click Here
SQL Tutorial :-Click Here

🤔 What is Logistic Regression?

Logistic Regression is a Supervised Learning algorithm used for binary classification problems. While Linear Regression predicts continuous values, Logistic Regression predicts a categorical dependent variable — typically a class label such as 0 or 1.

However, rather than directly assigning these binary values, Logistic Regression predicts probabilities between 0 and 1 using a function called the Sigmoid Function (or Logistic Function).

This probability is then mapped to a class using a threshold value — usually 0.5:

• If probability > 0.5 → class = 1
• If probability < 0.5 → class = 0

🧠 Why Logistic Regression?

• It provides a probabilistic interpretation of classification.
• Can work well with both discrete and continuous features.
• Offers clear insights into feature importance.
• It’s efficient, easy to implement, and widely used in industry.

📉 The Sigmoid (Logistic) Function

At the core of logistic regression lies the sigmoid function, defined as: S(z)=11+e−zS(z) = \frac{1}{1 + e^{-z}}

Where z is the linear combination of input variables (like in Linear Regression). This function outputs values between 0 and 1, making it perfect for modeling probabilities.

Image Source: Wikipedia – The classic S-shaped curve of the logistic function

📌 Logistic Regression Equation

Starting from linear regression: z=b0+b1x1+b2x2+…+bnxnz = b_0 + b_1x_1 + b_2x_2 + … + b_nx_n

We pass this through the sigmoid function: p=11+e−zp = \frac{1}{1 + e^{-z}}

To make this usable for classification, we apply the log-odds (logit) function: log⁡(p1−p)=b0+b1x1+b2x2+…+bnxn\log\left(\frac{p}{1 – p}\right) = b_0 + b_1x_1 + b_2x_2 + … + b_nx_n

📚 Types of Logistic Regression

1. Binomial Logistic Regression
   → Two possible outcomes: Yes/No, 0/1, Spam/Not Spam
2. Multinomial Logistic Regression
   → Three or more unordered outcomes: Dog/Cat/Rabbit
3. Ordinal Logistic Regression
   → Three or more ordered outcomes: Low/Medium/High

🛠 Python Implementation: Predicting SUV Purchase

Let’s walk through a practical implementation of Logistic Regression in Python using a real-life example:

📁 Step 1: Data Preprocessing

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt

# Load dataset
data_set = pd.read_csv('user_data.csv')

# Extract features (Age, Salary) and target (Purchased)
x = data_set.iloc[:, [2, 3]].values
y = data_set.iloc[:, 4].values

✂️ Step 2: Splitting the Dataset

from sklearn.model_selection import train_test_split

x_train, x_test, y_train, y_test = train_test_split(
    x, y, test_size=0.25, random_state=0)

📏 Step 3: Feature Scaling

from sklearn.preprocessing import StandardScaler

sc = StandardScaler()
x_train = sc.fit_transform(x_train)
x_test = sc.transform(x_test)

🔍 Step 4: Fitting Logistic Regression

from sklearn.linear_model import LogisticRegression

classifier = LogisticRegression(random_state=0)
classifier.fit(x_train, y_train)

📈 Step 5: Predicting Test Set Results

y_pred = classifier.predict(x_test)

✅ Step 6: Confusion Matrix

from sklearn.metrics import confusion_matrix

cm = confusion_matrix(y_test, y_pred)
print("Confusion Matrix:\n", cm)

Confusion Matrix will show True Positives, False Positives, etc., helping evaluate model accuracy.

🎨 Step 7: Visualizing the Results

from matplotlib.colors import ListedColormap

x_set, y_set = x_train, y_train
x1, x2 = np.meshgrid(np.arange(start=x_set[:, 0].min()-1, stop=x_set[:, 0].max()+1, step=0.01),
                     np.arange(start=x_set[:, 1].min()-1, stop=x_set[:, 1].max()+1, step=0.01))

plt.contourf(x1, x2, classifier.predict(np.array([x1.ravel(), x2.ravel()]).T).reshape(x1.shape),
             alpha=0.75, cmap=ListedColormap(('red', 'green')))

plt.xlim(x1.min(), x1.max())
plt.ylim(x2.min(), x2.max())

for i, j in enumerate(np.unique(y_set)):
    plt.scatter(x_set[y_set == j, 0], x_set[y_set == j, 1],
                c=ListedColormap(('red', 'green'))(i), label=j)

plt.title('Logistic Regression (Training set)')
plt.xlabel('Age')
plt.ylabel('Estimated Salary')
plt.legend()
plt.show()

This plot visually explains the model’s decision boundaries.

🧾 Assumptions of Logistic Regression

• The dependent variable must be categorical.
• There should be no multicollinearity among independent variables.
• The log odds of the outcome should be linearly related to the independent variables.

Download New Real Time Projects :-Click here
Complete Advance AI topics:- CLICK HERE

🎯 Conclusion

Logistic Regression is a fundamental yet powerful tool in the machine learning arsenal. Whether you’re classifying emails, diagnosing diseases, or predicting customer behavior, it lays the foundation for more advanced techniques.

Its interpretability, simplicity, and effectiveness make it a go-to algorithm for many real-world problems — especially when probability estimation is as important as the prediction itself.

💡 Pro Tip:

Before diving into complex models like Random Forests or Neural Networks, master Logistic Regression — because a well-tuned simple model often outperforms a poorly tuned complex one.

logistic regression in machine learning with example
logistic regression in machine learning python
logistic regression formula machine learning
linear regression in machine learning
logistic regression in machine learning geeksforgeeks
logistic regression explained with example
logistic regression algorithm
logistic regression algorithm steps