Generator Function Vs Normal Function
Functions are the building blocks of Python but not all functions are created equal. In the realm of Python programming, normal functions and generator functions serve distinct purposes, each with its unique strengths. While normal functions execute and return results immediately, generator functions take a more efficient approach, yielding values lazily, one at a time as needed. Understanding the differences between these two types of functions is key to writing optimized and flexible code. In this article, we will delve into the comparison between generator functions and normal functions, highlighting their use cases and benefits to elevate your Python skills.
To understand what generator functions are, we will compare them with the
normal functions that we already know. Both of them are defined by using
the def statement. Python considers a function as a generator function if
one or more yield statement appears inside the function. The yield
statement consists of the yield keyword followed by a value. So, if you
see a yield statement inside a function, then it is a generator function.
Normal Function
def fn():
Generator Function
def gen_fn():
yield value
Now let us see how both of them behave when they are called. We know
that when we call a normal function, the code inside the function is
executed and the value that is there in the return statement is given out and if there is no return statement, None is given out.
When you call a generator function, the code written inside the function is
not executed, instead a generator object is given out which can be assigned
to a variable.
Calling a generator function does not execute the function’s
code, it just creates and returns a generator object.
We have seen that this
generator object is actually an iterator and it can be iterated over.
x = fn() # Calling a normal function executes its code, returns a value
g = gen_fn() # Calling a generator object gives a generator object
The code of a generator function is not executed when the generator
function is called, so when and how is this code executed?
This code is
executed when you iterate over the generator object either automatically or
manually.
You can iterate automatically by using any iteration tool like a
for loop or a comprehension and you can iterate manually by using the
next function.
So, when a normal function is called, its code is executed and when a
generator function is called it just returns a generator object. When this
generator object is iterated over, then the code of the generator function gets
executed.
The code of a normal function is executed each time it is called
and code of a generator is executed each time it is iterated over.
A normal function executes and returns a single value while a generator
function produces a sequence of values.
These values are produced by
iterating over the generator object. It is similar to what we have seen in
iterators.
The values that are produced and given out are created in the
yield statement.
The normal function gives out its value using the return statement, while
the generator function gives out its values using the yield statement.
Each time you call a normal function, the code inside it is executed from the
beginning. When a return statement is encountered, the function
execution stops, all local variables are destroyed and the value in the
return statement is given out. A normal function does not remember
anything about the previous calls, it always starts with the same initial state.
A generator function is different from a normal function in that it retains the
state when it was last called.
During the execution of a generator function,
the function execution is stopped when a yield statement is encountered and value in the yield statement is given out.
When the function
execution stops due to the yield statement, the local variables including
the parameters are not destroyed, function remembers values of all the local
variables and also the place where the function execution stopped so that in
the next execution the function resumes from there.
So, when next time the
generator is invoked by iterating over it, the code does not execute from the
beginning but it continues where the previous execution had stopped.
So, the code of a normal function always starts executing from the
beginning of the function, the first line, while a generator when
executed starts from the place where the previous call had left.
The
difference between a return statement and a yield statement is that the
return statement when executed throws away the local state of the
function while the yield statement retains the local state of the function.
Let us understand all this with the help of examples.
We have the generator function which is not of any use but it will
help us understand how generator function works.
>>> def gen_fn():
n = 0
print('ABC', n)
n += 2
yield 10
print('GHI', n)
print('XYZ')
yield 20
print('JKL', n)
n *= 5
yield 30
print('MNO', n)
>>> g = gen_fn()
We called the generator function and got the generator object in variable g.
Now we will iterate over this generator object manually using the next
function.
>>> v = next(g)
Output
ABC 0
First three statements of the function were executed and then the yield
statement was encountered so the execution stopped and 10 was returned which is assigned to variable v. We can see that the value of v is 10.
>>> v
Output
10
Again, we call the next function on this generator object.
>>> v = next(g)
Output
GHI 2
XYZ
In the previous call, the execution had stopped at yield 10, so now the
execution starts from the statement which is just after it.
Next two
statements are executed and again a yield statement is encountered so the
execution stops and this time 20 is returned.
>>> v
Output
20
Note that the value of n was remembered from the previous call. Again, we
call the next function.
>>> v = next(g)
Output
JKL 2
>>> v
Output
30
>>> v = next(g)
Output
MNO 10
StopIteration
This time the whole function code was finished and there was nothing to
yield so now in this case the function execution stops and the
StopIteration error is raised to indicate the exhaustion of the
generator object.
This error is raised to indicate that it has generated all the
values and there are no more values left to provide.
If you try to reiterate over this generator object and you cannot, it is because
it has been exhausted. Any attempt to iterate over this generator will raise
the StopIteration error.
>>> v = next(g)
Output
StopIteration
>>> for i in g:
print(i)
If we use it in a for loop, nothing happens because this exhausted
generator object raises the StopIteration error which is caught by the
loop and immediately terminates.
It is not possible to restart or reiterate an exhausted generator object. If we
want to iterate again, then we have to create another generator object by
calling this function.
>>> g = gen_fn()
Now we have this fresh generator object. When we write this loop, it works.
>>> for i in g:
print(i)
Output
ABC 0
10
GHI 2
XYZ
20
JKL 2
30
MNO 10
Generally, the yield statement in a generator function appears inside a
loop but here we have used it multiple times to make the working clear.
A generator function is like a generator factory, you can call it many times
to get generator objects, each one will have their own state information independent of each other.
Now, let us see the cubes generator that we have seen before.
def cubes(start, stop):
for n in range(start, stop+1):
yield n * n * n
In this generator function, we have used the yield statement inside a loop.
We will get a generator object by calling this function with arguments 2 and
5.
y = cubes(2, 5)
Now, we will call the next function for this generator object.
>>> next(y)
Output
8
When this next function is called, the execution starts from the for loop and the value of n is 2. The yield statement is executed, so 8(2*2*2) is
returned. The function execution has stopped but the loop has not finished
so when next time we iterate over this generator object the loop will
continue from where it had left. So now let us call next again.
>>> next(y)
Output
27
The loop continues, n becomes 3 and then the yield statement is
executed. 27 is returned and execution stops but the loop is still not fully
finished. Again, we call next.
>>> next(y)
Output
64
>>> next(y)
Output
125
Now the loop has finished, so it will terminate. There is nothing to execute
and return, so the next time when we call next, the StopIteration
error is raised.
>>> next(y)
Output
StopIteration
Now this generator object is exhausted. This is how a generator function
works and produces values.
Since the yield statement can be inside a loop, you can write generators
that give long sequences or even infinite sequences.
Let us change the
generator function so that it gives the cubes infinitely.
def cubes(start):
n = start
while True:
yield n * n * n
n = n + 1
y = cubes(2)
Now, we do not have the parameter stop in our generator function. The
variable n is initialized to start and we have written the yield statement
inside an infinite while loop. So now we have an infinite generator object
which will give cubes infinitely.
def cubes(start):
n = start
while True:
yield n * n * n
n = n + 1
y = cubes(2)
Now, we do not have the parameter stop in our generator function. The
variable n is initialized to start and we have written the yield statement
inside an infinite while loop. So now we have an infinite generator object
which will give cubes infinitely.
class Fibonacci:
def __init__(self, max):
self.max = max
self.a = 0
self.b = 1
def __iter__(self):
return self
def __next__(self):
f = self.a
if f > self.max:
raise StopIteration
self.a, self.b = self.b, self.a + self.b
return f
x = Fibonacci(100)
for i in x:
print(i, end = ' ')
print(50 in x, 55 in x)
Now, let us write a generator function to do the same job. The generator
function will produce an iterator automatically for us.
def fibo_gen(max):
a = 0
b = 1
while a < max:
yield a
a, b = b, a + b
fib = fibo_gen(100)
for i in fib:
print(i, end=' ')
Output
0 1 1 2 3 5 8 13 21 34 55 89
This generator function generates Fibonacci numbers. If you want an
infinite generation of numbers, then in place of the condition a < max,
you can write True.
we saw that we need two classes to support multiple scans.
class Fibonacci:
def __init__(self, max):
self.max = max
def __iter__(self):
return FiboIterator(self)
class FiboIterator:
def __init__(self, source):
self.source = source
self.a = 0
self.b = 1
def __next__(self):
f = self.a
if f > self.source.max:
raise StopIteration
self.a, self.b = self.b, self.a + self.b
return f
x = Fibonacci(100)
for i in x:
print(i, end = ' ')
print(55 in x, 50 in x)
The __iter__ method of the Fibonacci class should return an iterator,
so we have created an instance of the FiboIterator class and returned
it. Generators provide an easy way to get an iterator, so we can use a
generator function here. Instead of writing the whole FiboIterator
class for instantiating an iterator object, we can simply make the
__iter__ method a generator. So, then it will return a generator object
which is an iterator.
class Fibonacci:
def __init__(self, max):
self.max = max
def __iter__(self):
a = 0
b = 1
while a < self.max:
yield a
a, b = b, a+b
x = Fibonacci(100)
for i in x:
print(i, end = ' ')
for i in x:
print(i, end = ' ')
Now this construct supports multiple active iterators. So, you can define
your iterable class by implementing its __iter__ method as a generator.
Code bundle here:
https://github.com/codeaihub1998/Ultimate_Python_1-to-21
If you read this article till the end, please consider the following:
- Follow the author to get updates of upcoming articles 🔔
- If you like this article, please consider a clap 👏🏻
- Highlight that inspired you
- If you have any questions regarding this article, please comment your precious thoughts 💭
- Share this article by showing your love and support ❤️
