Relation __dir__ method and __getattribute__method

Question

Guys, I wanted to ask you a question:

class Frob(object):
...     def __init__(self, bamf):
...             self.bamf=bamf
...     def __getattribute__(self,atr):
...             return object.__getattribute__(self,'teste')

When I implement the method getattribute and I try to do the following:

frob = Frob('random')
dir(frob)

I end up getting an empty list, someone can explain to me why this behavior, and what is the relationship between these two methods?

Answer 1

This example of yours has fallen into some difficult ways to understand and explain. And the reason why is that you replace the __getattribute__ - this is a method very delicate, because it really involves all the search for attributes of the object - including those made internally by the method __dir__.

However, contrary to what the other reply comments, the problem does not is just the attribute __class__ - most of these __class__ the Python does internally, in native code, and does not go through __getattribute__. Within the __dir__ however, yes, it uses the mechanism of __getattribute__ to arrive the class to object. To see the difference between the two, in an instance of its class Frob original make print (type(Frob(0))), and then print(Frob(0).__class__ - the type get the __class__ directly, without using the mechanism of __getattribute__.

Understanding exactly what is happening is complicated. Ideally you create the method __getattr__ ,and not __getattribute__ unless you know very well what you’re doing. The __getattr__ is only called when Python has not found an attribute in any other path. (Including the call itself to __getattr__ is made within the __getattribute__ original.

If it replaces the __getattribute__, the best is to intervene as little as possible - exchange the attributes you want, in an "if" only seeing the desired attributes, and delegate the rest of the attributes to the __getattribute__ original - and not the other way around.

If you put a print within the __getattribute__ will see that he is called by __dir__ to obtain __class__ and __dict__ - why this is the normal implementation of dir: it shows the class attributes, and the instance attributes of an object - and the normal instance attributes are stored inside the __dict__.

Then, note how, when returning __dict__ and __class__ of __getattribute__ you have listed the attributes set in the instance:

class Frob(object):
    def __init__(self, bamf):
            self.bamf=bamf
            self.teste = 5
    def __getattribute__(self,atr):
           print(atr)
           if atr in ("__class__", "__dict__"):
               return super().__getattribute__(atr)
           return object.__getattribute__(self,'teste')

In the terminal:

In [54]: print(dir(Frob(1)))                                                                                                         
__dict__
__class__
['__class__', '__delattr__', '__dict__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__le__', '__lt__', '__module__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__', 'bamf', 'teste']

Summary of the mechanism of __getattribute__

If it’s about-writing __getattribute__ have to do carefully - and remember that in this code is done the search for normal Python attributes, which is reasonably complex:

1. it first checks if the attribute exists in the object class (and does so using "type", not obj.__class__, that would be recursive:
   • If the attribute exists in class it checks if this attribute is an object that has the methods __get__ and __set__ (a "data Descriptor"):
     • If yes (but not only the __get__ and does not have the __set__) - he calls the method __get__ of the attribute, passing the instance (self) and the class (type(self)) and returns that value.
2. following, the __getattribute__ checks if the object has a __dict__ (no recursiveness, using internal mechanisms), if yes, checks if the attribute exists in this __dict__ and returns this value. (the __dict__ of an object has to be a real Python dictionary, and if you try to put a subclass that changes the __getitem__, this special method will be ignored)
3. The __getattribute__ checks if the object class has __slots__ and return the slot value corresponding to the name
4. The __getattribute__ checks if the value exists in the object class.
   • if it exists and has the method __get__, this method is called, with the instance and the class (the normal methods of objects are created at that point), and returns the value obtained.
   • else only returns the attribute found in the class. (the search for class values in that item (4) and item (1) nay calls the method __getattribute__ of the object’s class (type(self)), it simply looks at the __dict__ of the class, and proceeds, looking at the __dict__ of the super-classes (the linear order of the super-classes, which is always in the attribute __mro__ class).
     1. If attribute not yet found, if method exists __getattr__ he is called. If the __getattr__ return a value, it is returned. If the __getattr__ raise an exception of AttributeError, the __getattribute__ with the search for us __getattr__ of the super-classes of the object.
     2. The __getattribute__ raises the exception of AttributeError.

So this is it - this complexity is what makes Python so simple and easy to use in the end- why does he do "what we expect him to do", even if we don’t stop to think that all this happens behind the scenes.

And also, because of this complexity, they invented the sister method __getattr__, that can be customized much more simply.

Answer 2

When you run the dir() he needs the __class__ to be able to return the information of the object’s scope. How you reimpled the __getattribute__ forever to return teste, the function does not find __class__.

You can reimplementate the __dir__, but it is also possible to modify a little the method to return teste except where the requested attribute is __class__.

def __getattribute__(self, atr):
    if atr == "__class__":
        return object.__getattribute__(self, atr)
    return object.__getattribute__(self, "teste")

However you will find another problem, an error will be returned because there is no attribute teste in its class, so for everything to work it must be initialized.

    def __init__(self, bamf):
        self.bamf = bamf
        self.teste = "TESTE"

Follows full code:

class Frob(object):
    def __init__(self, bamf):
        self.bamf = bamf
        self.teste = "TESTE"

    def __getattribute__(self, atr):
        if atr == "__class__":
            return object.__getattribute__(self, atr)
        return object.__getattribute__(self, "teste")


frob = Frob('random')
print(dir(frob))
print(frob.bamf)

Console output:

['__class__', '__delattr__', '__dict__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__le__', '__lt__', '__module__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__']
TESTE