Jose: Hey guys welcome back
and in this video we're talking about generators in Python.
I'm really excited for this section
because we're going to talk about some more advanced stuff
and generators are one of these topics in Python that
a lot of courses don't even mention
but they are essential for performance
and also because the e-sync development in Python,
the e-synchronous development
that we're going to be learning later on
is built upon these generators.
When you understand generators
a lot of things in Python start becoming a lot clearer.
So what is a generator?
A generator in Python is a function.
It is a special function.
See a function that remembers the state it's in
in between executions,
so you can run the function multiple times
and it will remember what it did the
last time that you ran it.
So that sounds pretty weird, let me explain with an example.
Imagine you wanted to build a list of 100 numbers,
very straightforward,
you want to get a new list with numbers from zero to 99,
not including 100.
So you could make a function like this one.
Obviously not using range and so forth, you know range.
Let's not use range instead let's build a list from scratch.
How would you do it.
A good way to do it would be start with an empty list.
Let's say call it nums and make it equal to an empty list.
And also start with an index equal to zero.
Then while i is less than 100,
say nums dot append i.
So you're gonna append zero at the start.
Then you're going to increment i by one,
which is gonna make it into one.
You're gonna append one, repeat, append two,
then three and so forth, until you get to 99.
And at the end of this function return nums,
which is your list of numbers.
If we do print hundred numbers and we run that.
We can see that it contains here
our list of a hundred numbers.
Of course we could use list comprehension for this.
We could use the range function,
but for now let's assume this is a cool way of doing it,
because what I'm really explaining is
essentially how the range function works on the inside.
So we construct a list,
fill it with the first hundred numbers and then return.
Now we have a hundred numbers in a list.
This entire list is in your computers memory stored as
this return value of the function.
Now hundred numbers is not very big
it's quite a small amount of space,
but let's say we wanted to store 10 million,
million is like a hundred numbers,
but it would be something like a million numbers.
You would generate a list of a million numbers
and store it in this variable
that goes into your computers memory
and it's presumably a bigger amount of memory
that it's taking, because each one of these numbers
is taking up a small amount of your computers memory.
If you wanted 10 million numbers then it is bigger and so.
Now having 10 million numbers not a great example,
but just imagine you have a very long list of something.
For example, these are often used in
calculating prime numbers and in big data
and things like that you maybe sometimes
want big lists of things.
So a generator is used to circumvent this problem
because the assumption is you don't need 100 numbers
all at once.
What you need is zero so that you can do something with it.
And then one so that you can do something with it.
And then two so that you can do something with it.
For example with this hundred numbers we could then go
and multiply them all by two and store them in another list.
How would you do that?
You'd do something like less comprehension,
x times two for x in hundred numbers.
What you're doing here is you're using this list
to isolate over it, and multiply each number by two.
Let's think that this multiplication by two
is an operation that we want to do in these numbers
in order to calculate something.
Another example would be, if you have a lot of users
and you wanted to calculate the engagement around them,
or how engaged they are with your system,
or something like that.
Essentially you want to go over each element
and perform an operation.
The thing here, is you don't need all the elements at once.
You need them one by one.
So this is what generators are used for.
Instead of having the entire list of elements,
what you do, is the generator gives you
the first element of the list,
without storing all of the list in memory.
The next time you call the generator,
it remembers the element it gave you last,
and it knows to give you the second element.
Then you run it again and it gives you the third element,
and so forth.
It never stores the list in memory,
it always only remembers the last number that it gave you,
so it can then give you the next one.
You have to fun the function every time you want
a new number, and that's why it's called a generator,
because it generates numbers, or indeed strings,
or users or whatever it is that you want to generate.
Here's how you would do it, instead of "nums.append(i)",
you say "yield i",
for i is the number we want to return,
we no longer need this nums equal square bracket,
and we no longer need the return segment either.
So now this function here is called a generator function.
Yield is very much like return, but what happens is
it gives you i when you call it,
and then it remembers that it's here, it's stopped
right before line five. Right after line four
there in the middle.
Next time you call the function, it will continue,
so what it'll do is it will increment i by one,
it'll rerun the loop and yield one,
give you one and then stop.
Next time you call it, it will continue
increment i buy one and repeat the loop
and then give you two, and then it will stop, and so forth.
Eventually when you run out of numbers,
it will yield none.
And then you know that it's finished.
So, how are we gonna do this?
Let's, first of all, print out hundred numbers.
Notice how it no longer gives us a list,
now it gives us a generator object,
which is called hundred numbers,
thus the name of our function.
It also gives us the memory address of our object.
Notice how we defined a function,
but the print is telling us this is an object,
because Python in the background realises
that you've used yield, you're making a generator,
and it turns those into a python object.
I'll tell you how to make these objects yourself
without a function later on.
So, let's store our generator in a variable.
We're gonna call it "g", that's a fairly popular
name for a generator in a small programme.
If you have many generators of course you'll have to
go for a more descriptive name.
Then we're going to print the next value that
the generator would give us.
Notice that when you initialise a generator, like so,
it doesn't run the function, it starts at the very top
without running at all.
When you ask it for the next value, it will
run up until the yield and give you
whatever value you've yielded.
The way you ask it for the next value, is you don't
run the function again, but you say next of "g".
So next is a built in function that affects generators,
and it tells it to go up to the yield.
Okay, so if we run this, you can see now we get zero,
because that's the very first number yielded.
Now our generator, "g", has stopped here
at the line 4.5
between 4 and 5, so if we ask it for another number,
notice that of course, at the end of a programme
the generator is destroyed, so if we run this programme
again this would be zero again and this would be one now.
It doesn't remember it across programme executions,
that's just not how Python works.
So now we can ask it for the first number,
and then for the next number.
After the first number it'll be stopped
between four and five.
When we say next it continues up until the next yield,
which means repeating this loop once and gives us one.
Okay.
Very important, the function remembers where it stopped.
So at any point, you can stop the function from running,
and then you can call next in order to continue it.
So, it's not useful only for generating numbers or whatever,
it's also useful when you want to stop a function
from running temporarily, and then you can
continue running it whenever you want,
by calling next on it.
So that's maybe not something that makes too much sense
just now, but just remember that for when we will need it
when we do a synchronised programming.
So what about the range function?
I said earlier that this was explaining essentially
how the range function works on the inside, and that's true,
but the range function doesn't
give you a generator directly.
Here, let's do something like
"my range object",
it's gonna be range 10.
This behaves very much like a generator,
but if you call next of "my range object"
you get an error, and look at that.
That's because the range object isn't quite a generator
even though it behaves in this way,
it doesn't generate a list and then allows you
to iterate over it, it behaves in this generation-ish manner
to be more efficient.
"My range object", though,
you can remember that we
printed it out much earlier on in the course
and we received some weird range zero to ten thing.
That's just the "repr" function being called,
and as you can see if we print out the "repr" outcome,
we also see range of zero to ten.
So "my range object" is something that behaves
very much like this, we're gonna learn in a couple
of videos time exactly how we can iterate over an object
in order to make it behave like a range object.
So, we've learned about these generators, and one more
thing I wanted to mention actually, about the generators
is the list function can turn a generator into a list.
So we can print list of "g".
Right here what we've called "next" then "next" again,
and then "list".
So now we have zero and one printed out for those are
the two things in our "next" functions,
and then we've generated a list of the generator,
but that list of course continues from where we left off.
The generator itself remembers what it was,
so then the list starts from two, not from zero.
So, it's really important to remember in generators,
they remember where they were.
So, when you create one, that's it from essentially
the moment you start using it, the moment you start
calling "next" on it, it remembers where it is,
and you can't go backwards. Okay.
Now, in the next few videos, we're going to learn
more about generators.
We're going to learn about generator classes,
how you can create a class that behaves like a generator,
and we're also going to learn about iterators and iterables
in the Python, and the range objectives
is one such example.
Then we're going to move onto some
functional programming examples,
like filter, map, any and all, and enumerate.
So, pretty excited for this section,
I'm confident that generators are going to be something
that you're going to need in your Python development,
particularly as we move on further in the course.
So with that said, I'll see you on the next video.