How does Python handle common functions and internal Python functions?

Asked

Viewed 704 times

2

I’m looking at Python’s dual functions compared to the common functions that use def to create them.

See an example of a common function that converts a number to binary:

def decimalParaBinario(valor, zerosEsquerda=0): 
    return format(valor, 'b').zfill(zerosEsquerda)

Her type:

print(type(decimalParaBinario))

would be:

<class 'function'>

Where the function decimalParaBinario is a class object function. Now notice the same behavior with a lambda expression:

b = lambda x, n=0 : format(x, 'b').zfill(n)

The lambda guy would be the same class 'function' for the lambda with the name b. And unlike common lambda functions can be anonymous, for example:

print(lambda x=1, n=8 : format(x, 'b').zfill(n))

who would return:

<function <lambda> at 0x7fae8de04158>

A reference to lambda in memory, in which there is no name explicit for lambda function. Also note that the behavior is different from a common function, when expressing a lambda function as a parameter of another function that would not expect a lambda function.

Despite this, I still can’t visualize in my mind how Python treats internally a common function def in relation to expressions lambda.


Question

  • How Python handles common functions and internal functions?
  • No STO in English has this question and I agree with the accepted answer, I agree with Luciano Ramalho who says in, Fluent python: "The lambda syntax is only a syntactic sugar: a lambda expression creates a function object as well as def", and I agree with what it says: "... The Anonimas functions rarely have python utility. Syntactic constraints tend to leave non-trivial, illegitimate, or impractical Lambdas" .

  • Very interesting this method for refactoring the use of Leds 1. Write a comment explaining what the heck that lambda does; 2. Study the comment and think of a name that captures the essence of it; 3. Convert the lambda to a def, using that name; 4. Remove the comment. :-)

1 answer

5


Both types of function are identical and treated in the same way:

After created in memory of the Python application, that is, since the code that creates the function - tando one with def as one with lambda is executed, there is no any difference between a lambda function and a non-lambda function.

The two have the same type, behavior, and can be passed as parameters anywhere that accepts a "chargeable object" (callable) as parameter.

So much so that there is no difference that the "formal and well educated" way of verifying if an object is a function: import the module types and call isinstance(meu_objeto, types.FunctionType) use that one FunctionType which is defined in the file types.py from the standard Python library exactly as FunctionType = type(lambda: None) - that is, functions declared with def are, in well-behaved code that can be used in production, etc... compared to the type of functions lambda.

The only difference (after created) really is that when using def, there is an implicit and mandatory name for the function, which is stored in the attribute __name__ of the same. In office lambda, the __name__ always contains the string '<lambda>'. (Note that this attribute can be written at will, however).

Why the difference shown in the question then?

Realize that in your print in the question, you printed two different things: when printing data on the "function with def", you printed only the type function. When printing data on the "lambda function", you printed the representation of the function itself (the instance of the class 'function').

Check the interactive terminal if we print the same things about each function:

In [30]: def a(): pass                                                                                                         

In [31]: b = lambda: None                                                                                                      

In [32]: print(a, type(a), a.__name__)                                                                                         
<function a at 0x7f5c8ff466a8> <class 'function'> a

In [33]: print(b, type(b), b.__name__)                                                                                         
<function <lambda> at 0x7f5c8ff06f28> <class 'function'> <lambda>

Your great concern that "the lambda function is anonymous" makes no difference to any Python code where you go to make use of a function - and that means you probably didn’t understand some other point of the language. Then there’s a legal argument. Although most texts, and even the official documentation always speaks in "variables", formally in Python we have no "variables": we have names for objects. An object can have multiple names, or none - what really counts is how many references we have to an object. The command def creates a "name" for the function that is created with it, in the same way as a = creates a "name" for the object that is the result of the expression to the right of the sign. (See above, both "a" and "b" can be used in the same way - each name is created in a way). Other commands that create names for objects are, for example, the import, the for, with and class. Each of these creates one or more names for objects, and during the execution of the program, after the names have been created, there’s no difference on how the name was created.

For example, I can create a function "a" with "def", associate it to another name with the sign of =, and delete the original name - it keeps working:

In [34]: def a(): 
    ...:     print("função a") 
    ...:                                                                                                                       

In [35]: b = a                                                                                                                 

In [36]: del a                                                                                                                 

In [37]: a()                                                                                                                   
---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
<ipython-input-37-8d7b4527e81d> in <module>
----> 1 a()

NameError: name 'a' is not defined

In [38]: b()                                                                                                                   
função a

In fact, we can go beyond - a function does not need to have a name (as well as any Python object) - I can insert the function, like any other object in a list. delete the name b, and it keeps working:

In [39]: c = [b,]                                                                                                              

In [40]: del b                                                                                                                 

In [41]: c                                                                                                                     
Out[41]: [<function __main__.a()>]

In [42]: c[0]()                                                                                                                
função a

(A list or any other data structure - dictionary, attribute in an instance, etc...). The only thing, which turns out to be just one detail, is that the function is an object that has the attribute __name__, that when she’s raised with def contains the name given in the command def and when it is created with lambda always contains ''. But once the function is created, it’s an almost normal attribute, the only constraint is that it should be set to a string. (This is: if you try to do a.__name__ = None, for example, Python gives a TypeError)

Differences between functions before to be created:

In Python language syntax yes, there is difference between functions with def and functions lambda: the former may contain an arbitrary code block, with lines containing commands (such as for, if, raise), and, to return a value, must contain the command return. Already functions lambda must necessarily be written in a single expression: that is, the body of a lambda function can call functions, do operations, use the if in ternary mode, contain Generator Expressions, etc... but may not contain any commando. The outworking of this expression is automatically the return value of the lambda function. The Python compiler, which can be manually called for any string with the function compile, has the distinction of 3 modes: the mode that executes blocks, the mode that executes expressions and the "command line" mode, made to compile expressions typed in the interactive environment. The first mode is used by exec and the second by eval - which are generally used instead of compile, which is lower level - but you can check in the documentation: https://docs.python.org/3/library/functions.html#Compile

follows advanced information - feel free to skip to the next session if it is too complicated: The interesting thing is that this difference only exists at the moment when a file is compiled - that is, at the moment when Python reads the file ". py" as a text file and processes it, it makes that distinction - soon after is created the bytecode file (". pyc", which in recent versions is inside the folders __pycache__.) In these files, the body of functions is already compiled into an object called code object (types.CodeType), and the lambda functions are already indistinguishable of functions with "def", (remembering: except for the attribute __name__). Detail that the functions even exist in the files . pyc - Function objects are only created in memory when the execution of the program reaches the lines with the block def or with the keyword lambda, but the sequence of bytecode operations to create them is identical.

You can see this using the "disassembler" of the standard library in a function that defines, internally, a function of each:

In [57]: def exemplo(): 
    ...:     def a(): 
    ...:         pass 
    ...:     b = lambda: None 
    ...:                                                                                                                       

In [58]: import dis                                                                                                            

In [59]: dis.dis(exemplo)                                                                                                      
  2           0 LOAD_CONST               1 (<code object a at 0x7f5c9c738b70, file "<ipython-input-57-4845f3e66234>", line 2>)
              2 LOAD_CONST               2 ('exemplo.<locals>.a')
              4 MAKE_FUNCTION            0
              6 STORE_FAST               0 (a)

  4           8 LOAD_CONST               3 (<code object <lambda> at 0x7f5c8fe44270, file "<ipython-input-57-4845f3e66234>", line 4>)
             10 LOAD_CONST               4 ('exemplo.<locals>.<lambda>')
             12 MAKE_FUNCTION            0
             14 STORE_FAST               1 (b)
             16 LOAD_CONST               0 (None)
             18 RETURN_VALUE

Disassembly of <code object a at 0x7f5c9c738b70, file "<ipython-input-57-4845f3e66234>", line 2>:
  3           0 LOAD_CONST               0 (None)
              2 RETURN_VALUE

Disassembly of <code object <lambda> at 0x7f5c8fe44270, file "<ipython-input-57-4845f3e66234>", line 4>:
  4           0 LOAD_CONST               0 (None)
              2 RETURN_VALUE

Note that the sequence is: put the "code Object" on the VM stack, put the name on the stack - and the lambda name already appears there - and call the opcode MAKE_FUNCTION.

But then, there are "fundamentally different functions"?

The answer is yes. A few days ago you even posted a question about how the yield modifies a function, and the answer I drew up for this question demonstrates this difference. The keyword yield,and her, as well as the async def, change a thing an information in the object function when it is created that makes these functions completely different from normal functions when called. Unlike defined functions with def and with lambda, which are working identities. Worth reading the answer there:

How Python handles the "Yield" command internally?

It is interesting to note that the markup that the language makes to distinguish these functions is in the attribute .__code__.co_flags of a function - and, again, for functions def and lambda the value of these flags is the same, reflecting once again that they are the same thing:

In [60]: def a(): pass                                                                                                         

In [61]: b = lambda : None                                                                                                     

In [62]: a.__code__.co_flags                                                                                                   
Out[62]: 67

In [63]: b.__code__.co_flags                                                                                                   
Out[63]: 67

In [64]: def c(): yield                                                                                                        

In [65]: c.__code__.co_flags                                                                                                   
Out[65]: 99
  • I really appreciate your answers, and I’m flattered by your help. This is pointing the way for one day I understand at least the basics of how the Python interpreter works and how certain internal mechanisms of the language behave +50

  • I wrote another answer that adds on the inner workings of the language today: https://answall.com/questions/368884/m%C3%A9all-of-a-class-s%C3%A3o-recreated-for-every-Inst%C3%A2ncia-no-python/368905#368905

Browser other questions tagged

You are not signed in. Login or sign up in order to post.