Data Preprocessing in ML (Machine Learning)

Rishabh saini April 3, 2025 4 min read

Data Preprocessing in ML

Introduction

Data preprocessing is an important part of the machine learning pipeline. It involves cleaning and transforming raw data into a format that can be properly used for modeling. Real-world datasets often contain missing values, noise, and inconsistencies, which can negatively affect the accuracy and performance of machine learning models.

By using appropriate preprocessing techniques, we can make sure that the data is clean, complete, and ready for analysis.

Data Preprocessing in ML (Machine Learning)

Complete Advance AI Topics: Click Here
SQL Tutorial: Click Here

Why Do We Need Data Preprocessing?

Raw datasets commonly contain inconsistencies, noise, duplicate records, and missing values. If this data is directly provided to a machine learning model, it can result in inaccurate predictions and reduced model performance.

Data preprocessing helps to:

Improve model accuracy
Increase efficiency
Reduce bias in predictions
Maintain a structured and usable dataset

Steps in Data Preprocessing

1. Getting the Dataset

The first step is to collect and prepare the dataset. Machine learning models generally work with structured data, which can be stored in CSV files, Excel sheets, or databases.

Some common sources for datasets include:

2. Importing Libraries

For data preprocessing in Python, we use some essential libraries:

import numpy as np  # For numerical operations
import pandas as pd  # For data handling
import matplotlib.pyplot as plt  # For visualization

3. Importing the Dataset

After obtaining the dataset, we can load it into Python using Pandas:

dataset = pd.read_csv('data.csv')
print(dataset.head())  # View first few rows

4. Handling Missing Data

There are two common ways to handle missing data:

Remove rows or columns containing missing values. However, this approach is not recommended for large datasets.
Replace missing values with the mean, median, or mode.

Using Scikit-learn:

from sklearn.impute import SimpleImputer

imputer = SimpleImputer(
    missing_values=np.nan,
    strategy='mean'
)

dataset.iloc[:, 1:3] = imputer.fit_transform(
    dataset.iloc[:, 1:3]
)

5. Encoding Categorical Data

Machine learning models generally work with numerical data, so categorical variables need to be converted into a suitable numerical format.

Encoding Labels (e.g., Yes/No 1/0)

from sklearn.preprocessing import LabelEncoder

label_encoder = LabelEncoder()

dataset['Purchased'] = label_encoder.fit_transform(
    dataset['Purchased']
)

Encoding Categories into Dummy Variables

from sklearn.preprocessing import OneHotEncoder
from sklearn.compose import ColumnTransformer

ct = ColumnTransformer(
    [("encoder", OneHotEncoder(), [0])],
    remainder='passthrough'
)

dataset = np.array(ct.fit_transform(dataset))

6. Splitting Dataset into Training and Test Sets

To properly evaluate a machine learning model, the dataset is divided into training and test sets:

from sklearn.model_selection import train_test_split

X_train, X_test, y_train, y_test = train_test_split(
    dataset[:, :-1],
    dataset[:, -1],
    test_size=0.2,
    random_state=42
)

7. Feature Scaling

Feature scaling brings variables onto a similar scale and helps prevent differences in feature magnitude from creating bias in the model.

from sklearn.preprocessing import StandardScaler

scaler = StandardScaler()

X_train = scaler.fit_transform(X_train)
X_test = scaler.transform(X_test)

Download New Real Time Projects :- Click here

Conclusion

Data preparation is a crucial stage in the machine learning process. It helps transform raw data into a structured, clean, and optimized format, making it more suitable for building accurate and efficient models.

By following these preprocessing steps, such as handling missing data, encoding categorical variables, splitting the dataset, and scaling features, we can prepare the data for developing robust machine learning models.

data preprocessing in ml machine learning python
data preprocessing in ml machine learning research paper
data preprocessing in ml machine learning with example
data preprocessing in ml machine learning geeksforgeeks
data preprocessing in ml machine learning ppt
data preprocessing in python
data preprocessing in machine learning pdf
data preprocessing techniques
data preprocessing in machine learning with example
data preprocessing in python
data preprocessing steps
data preprocessing techniques
data preprocessing in deep learning
data preprocessing in ml machine learning geeksforgeeks
data preprocessing techniques in machine learning python
data preprocessing in ml machine learning pdf
data preprocessing in ml python
data preprocessing in ml geeksforgeeks

Data Preprocessing in ML (Machine Learning)