Python Generators: A Simplified Guide

Python Generators

Python Generators provide an efficient way to create iterators and process values one at a time. Instead of building and storing an entire list in memory, a generator produces each value only when it is needed.

This makes generators especially useful for large datasets and sequences that may contain a very large or unlimited number of values. They also provide a simpler alternative to manually creating iterators with __iter__() and __next__().

What is a Python Generator?

A generator is a special type of function that produces an iterable sequence using the yield statement. Unlike a normal function that generally returns a result and finishes, a generator can pause its execution and continue from the same point when another value is requested.

When a generator has no more values to produce, Python automatically raises StopIteration.

Generators are useful when you want to traverse values sequentially without creating the complete collection in memory at once.

How to Create a Generator Function in Python?

A generator function is created using the def keyword and contains one or more yield statements.

def simple():
    for i in range(10):
        if i % 2 == 0:
            yield i

# Calling the generator using a for loop
for value in simple():
    print(value)

Output:

0
2
4
6
8

Each time the generator produces a value, its current execution state is preserved so that it can continue from that point later.

Yield vs. Return

The yield and return statements behave differently in Python functions.

• Yield: Pauses the generator and preserves its current state. When another value is requested, execution continues from where it stopped. A generator function can contain multiple yield statements.
• Return: Immediately terminates the function and returns a value to the caller.

Example with Multiple yield Statements

def multiple_yield():
    yield "First String"
    yield "Second String"
    yield "Third String"

obj = multiple_yield()

print(next(obj))
print(next(obj))
print(next(obj))

Output:

First String
Second String
Third String

The next() function requests the next available value from the generator.

Generator Function vs. Normal Function

Feature	Generator Function	Normal Function
Statements	Can contain multiple yield statements	Uses return to return results
Execution	Pauses and resumes execution	Terminates when return is executed
State Retention	Retains its state between values	Does not retain generator-style execution state
Termination	Raises StopIteration when exhausted	Normally completes by returning a result

Generator Expressions

A generator expression provides a concise way to create a generator. Its syntax is similar to a list comprehension, but parentheses are used instead of square brackets.

A list comprehension creates all its values immediately, whereas a generator expression produces values one at a time.

nums = [1, 2, 3, 4, 5]

# List comprehension
list_comp = [x**3 for x in nums]

# Generator expression
gen_exp = (x**3 for x in nums)

print(gen_exp)
print(list_comp)

Output:

<generator object <genexpr> at 0x01BA3CD8>
[1, 8, 27, 64, 125]

You can retrieve values from the generator expression using next().

gen_exp = (x**3 for x in nums)

print(next(gen_exp))
print(next(gen_exp))

Output:

1
8

Example: Multiplication Table Using Generator

Generators can also be used to produce values for a multiplication table one at a time.

def table(n):
    for i in range(1, 11):
        yield n * i

for value in table(15):
    print(value)

Output:

15
30
45
60
75
90
105
120
135
150

Advantages of Generators

1. Ease of Implementation

Generators simplify iterator creation. Instead of manually implementing __iter__() and __next__(), you can use a function containing yield.

2. Memory Efficiency

A generator does not need to store all generated values simultaneously. It calculates each value when required.

import sys

# List comprehension
nums_list = [i * 2 for i in range(1000)]
print("Memory in Bytes:", sys.getsizeof(nums_list))

# Generator expression
nums_gen = (i * 2 for i in range(1000))
print("Memory in Bytes:", sys.getsizeof(nums_gen))

Output:

Memory in Bytes: 4508
Memory in Bytes: 56

3. Data Pipelining

Generators can be connected together to process data step by step. This approach is useful when working with large files or datasets without loading everything into memory at once.

with open('sales.log') as file:
    burger_col = (line.split()[3] for line in file)
    per_hour = (int(x) for x in burger_col if x.isdigit())
    print("Total burgers sold =", sum(per_hour))

4. Infinite Sequence Generation

Generators can produce an infinite sequence because values are generated only when requested.

Complete Advance AI Topics: Click Here
SQL Tutorial: Click Here
YT:- DecodeIT

def infinite_sequence():
    num = 0

    while True:
        yield num
        num += 1

for value in infinite_sequence():
    print(value)

The sequence continues producing values until the consuming code stops requesting them.

Frequently Asked Questions

1. What is a generator in Python?

A generator is a function that produces values one at a time using the yield statement.

2. What is the purpose of yield in Python?

yield pauses a generator’s execution and preserves its state so that execution can continue when the next value is requested.

3. Are generators memory efficient?

Yes. Generators generally produce values on demand instead of storing the complete sequence in memory.

4. What is a generator expression?

A generator expression is a compact way of creating a generator using syntax similar to a list comprehension but with parentheses.

5. How can I get the next value from a generator?

You can use the next() function to request the next available value from a generator.

6. What happens when a generator has no more values?

When a generator is exhausted, Python raises the StopIteration exception.

Conclusion

Python Generators provide a simple and efficient approach to creating iterators and processing values one at a time. The yield statement allows a function to pause and resume execution while retaining its state.

Because generators calculate values on demand, they can reduce memory usage and are particularly useful for large datasets, data pipelines, and infinite sequences. Generator functions and generator expressions therefore provide an important Python technique for efficient iteration.

Keywords: Python Generators, what is generator in Python with example, Python generator expression, Python generators list, Python generator class, Python generator tutorial, Python generator example, Python yield, Python yield vs return, Python output generator, Python generators W3Schools, generator Python typing