Introduction In the age of multicore processors, leveraging the power of multiprocessing is essential for building efficient and fast applications. Python offers the built-in multiprocessing module, which provides tools for parallelizing tasks. This article explores multiprocessing in Python, its importance, and advanced features like queues, locks, and pools.
Multiprocessing refers to running multiple processes simultaneously. It uses multiple CPU cores to enhance performance and efficiently manage tasks.
Key Features:
Tasks run independently and simultaneously.
Shared access to memory and peripherals.
Allocation of tasks to different CPUs by the operating system.
Example Analogy
Imagine a single chef handling all kitchen tasks. Without help, they switch between tasks like chopping, cooking, and cleaning. But if there are multiple chefs, each can work on specific tasks, ensuring faster and smoother operations.
Why Use Multiprocessing?
Without multiprocessing, systems use a single CPU core, leading to slower task execution due to frequent interruptions. Multiprocessing:
Increases efficiency.
Reduces execution time.
Simplifies tracking and managing tasks.
Multiprocessing in Python
The multiprocessing module in Python offers a user-friendly API for making use of several cores. It makes it simple for developers to create parallel programs.
Basic Multiprocessing Example
from multiprocessing import Process
def greet():
print('Hello! Welcome to Python Multiprocessing!')
if __name__ == '__main__':
p = Process(target=greet)
p.start()
p.join()
Output:
Hello! Welcome to Python Multiprocessing!
Explanation:
Process: Creates a new process.
start(): Starts the process.
join(): Waits for the process to complete.
Multiprocessing with Arguments
import multiprocessing
def calculate_square(n):
print(f"The square of {n} is {n * n}")
def calculate_cube(n):
print(f"The cube of {n} is {n * n * n}")
if __name__ == "__main__":
process1 = multiprocessing.Process(target=calculate_square, args=(4,))
process2 = multiprocessing.Process(target=calculate_cube, args=(4,))
process1.start()
process2.start()
process1.join()
process2.join()
print("Both processes are completed.")
Output:
The square of 4 is 16
The cube of 4 is 64
Both processes are completed.
Key Points:
args: Passes arguments to the target function.
Processes run concurrently, reducing overall execution time.
Advanced Multiprocessing Features
1. Queue Class
Used to share data between processes.
from multiprocessing import Queue
queue = Queue()
# Adding items to the queue
for item in ['Apple', 'Orange', 'Banana']:
queue.put(item)
# Removing items from the queue
while not queue.empty():
print(queue.get())
Output:
Apple
Orange
Banana
2. Lock Class
Ensures only one process accesses critical sections of code at a time.
from multiprocessing import Lock, Process
def print_message(lock, message):
with lock:
print(message)
if __name__ == "__main__":
lock = Lock()
messages = ["Task 1", "Task 2", "Task 3"]
processes = [Process(target=print_message, args=(lock, msg)) for msg in messages]
for p in processes:
p.start()
for p in processes:
p.join()
Output:
Task 1
Task 2
Task 3
3. Pool Class
Used for parallel execution of functions.
Example 1: Distributing tasks
from multiprocessing import Pool
def square(n):
return n * n
if __name__ == "__main__":
with Pool(4) as pool:
results = pool.map(square, [1, 2, 3, 4])
print(results)
Output:
[1, 4, 9, 16]
Example 2: Simulating tasks
from multiprocessing import Pool
import time
def task(name):
print(f"Task {name} started.")
time.sleep(2)
print(f"Task {name} completed.")
if __name__ == "__main__":
with Pool(2) as pool:
pool.map(task, ['A', 'B', 'C', 'D'])
Output:
Task A started.
Task B started.
Task A completed.
Task B completed.
Task C started.
Task D started.
Task C completed.
Task D completed.
Common Multiprocessing Functions
Function
Description
cpu_count()
Returns the number of CPUs available.
start()
Starts a process.
join()
Waits for a process to complete.
Queue()
Creates a queue for data sharing.
Lock()
Prevents concurrent access to shared resources.
Pool()
Manages multiple processes for parallel execution.
Post Comment