Python reserved word Yield

Question

What is the difference between the following codes ?

def bottom():
    yield 42

def bottom():
    return (yield 42)

Answer 1

Briefly, this is the mechanism used for co-routines to return values in Python - details of how it works follow below:

First: if the body of a function has the keyword yield, yield from or is declared as async def, the return doesn’t work like a normal Return. What is returned when calling a function like this is a normal function is a co-routine or a Generator:

>>> def a():
...     return (yield 42)
...  

>>> a()
<generator object a at 0x7fc904b167d8>

The result value - what is in return - of a Generator or co-routine appears as the attribute .value of Exception StopIteration caused by the iterator.

That is, this code:

def a():
    return (yield 42)
try:
    b.send("final value")
except StopIteration as end:
    end2 = end

print(end2.value)

It will print "initial value". Note that this mechanism will rarely be used directly by the programmer, and is used internally by asynchronous code support.

The method send was also introduced in Python 2, a version after the creation of the keyword yield - I believe in Python 2.5 - and allows sending values back into Generator (as the end value of the expression yield). From this mechanism, plus the method .throw which allows an exception to be made where the yield, allowed generators to function as co-routines.

A more direct use happens when Enerator is used from within another Enerator, with the form yield from. In this case, the value returned as a result of yield from is the value of return of the most internal generator. That is: the code below prints "None" (which is the result of the yield in the previous function when it is used only with next:

def c():
    value = yield from a()
    print(value)
    return None

[_ for _ in c()]

If you are seeing code that does not use asynchronous programming, and has values of return within bodies of functions with yield probably these generators are being used that way.

So, this mechanism of separating the return value within the exception that indicates the end of the Generator was the way found of these co-routines generate a final value, while the yield within them comes to serve not to generate a significant value, as in the case of common generators, but to be a point of pause of those functions. Co-routines are always called, unlike generators, by a more complex code responsible for coordinating their call, and calling other co-routines - this is the mechanism used in asyncio - the official library for asynchronous code execution in Python. This coordinating code is the "loop Event" - event loop - of the program that works asynchronously.

In this case, the code that uses Generator is not called directly - it is called from within other co-routines with the keyword await, or is scheduled on Event loop, so that it returns a result when finished. (The Event loop does all the coordination to catch the .value of Stopiteration, or throw an exception in the co-routine correctly, and call other co-routines when the code is "paused" by a "Yield from" or "await")

In Python 3.4, the use of the Decorator asyncio.coroutine marked a Generator using yield from to function as co-routine. So if you have asynchronous code made to work with Python 3.4 (or emulate this with Python 2.7), it will make use of yield from to call other co-routines in your body, and use the return to return the final result . From Python 3.5 a specialized syntax has been created - as described above. Co-routines are now declared as async def instead of needing the developer coroutine, and the keyword await is used in place of the yield from. (In short, in other co-routines called, the value in "Return" is returned "normally" to a function called in the form valor = await funcao().

I recently wrote another extensive response where I talk about a few more aspects of asynchronous programming - take a look there: Python asynchronous generators

Answer 2

The first case returns a generator, which is an object containing an enumerator, has a state from where that enumerator is. Whoever calls the function can take the result of the enumerator. This is transparent and will give 42.

In the second case the yield returns the enumerator of its expression right there and then the result of this enumerator is returned from the function, it is already 42.

In practice nothing changes in this case, but the internal mechanism is quite different.

You can see more in: What is Yield for?.