What is the difference between __init__ and __new__?

Question

What is the difference between the special methods __init__ and __new__ in Python?

Answer 1

The __new__ is called by Python before of the object being effectively built: that is, having memory allocated to it, initialized pointers, and possivelment a few more things - all these things are done in native code, in C, within the method __new__ elementary class object.

When you override a __new__, at some point you will have to (in normal code) call the __new__ superclass - and this repeats until the highest class in the hierarchy calls the __new__ of object - that __new__ returns the instance of the object - that is, the object that is passed as self for all instance methods. Your method __new__ have to return this self.

It is possible to have a method __new__ which returns something else - but in this case, it is impossible to have an instance proper of the class that has this method __new__: the Python class mechanism is used for something else. For example, the interfaces of Zope (package zope.interface) does not create objects themselves, and only uses class syntax when Voce inherits from zope.interface.Interface.

After calling the __new__, the method __init__ is called. It is technically called "initializer", and already receives the object instance in the first parameter - "self", in general - with Tod the memory allocation part made - the __init__then you can do all the initialization of attributes and resources done at the Python level.

In almost every case we just need to write the method __init__ - it is very rare to need to touch the __new__. And anyway, if there was no __init__, everything that is done in it could be done within the __new__ thus:

class SemInit(object):
     def __new__(cls, param1, param2, **kwargs):
          self = super().__new__(cls)
          # código que iria no __init__ vai aqui:
          ...
          return self

So, in short, the step by step of the instantiation of an object is as follows::

when doing a = MinhaClasse(), the method __call__ of the object MinhaClasse- in itself (that, in Python classes are themselves objects and via rule are instances of the special class type (it is called "metaclass"). so, this method __call__ class type is called - it is written in C, but the pseudo code for it, if it were in Python would be:

class type:
    def __call__(cls, *args, **kwargs):
         self = cls.__new__(*args, **kwargs)
         result = cls.__init__(self, *args, **kwargs)
         if result is not None:
              raise TypeError("TypeError: __init__() should return None, not '{}'".format(result))
         return self

And yes, in Python it is valid the "metaclasses" feature that consists in making a type subclass and modifying the mechanism of creating classes and instances above - then writing a method __call__ as above, provided he calls the object.__new__ passing the class as a parameter at some point, you can create classes that do not use the __init__, or has more than one type of __init__, etc....

I said everyone has to call the __new__ of object from Python : in, with pure Python code is the only way to create an object.

It is possible, however, to create code using a C extension, or even by allocating data structures "at hand" with the "ctypes" module that creates an object differently.

updating: And when should we encode the __new__?

When creating a subclass of a "immutable" native Python data type, such as int or another number, str or tuple. In these cases, if we want to change the content value, it is necessary to act on __new__ before calling the __new__ original. After that, the value is already set and cannot be changed.

For example, a string sub-class that converts all content to uppercase already in creation could be:

class StrUpper(str):
    def __new__(cls, value):
         value = value.upper()
         return super().__new__(cls, value)

Note another feature when implementing __new__: when calling the method __new__ of the superclass, with super(), it is necessary to put the parameter cls explicitly in the arguments. For normal class methods and methods, the super() already does that part. The __new__ is a special method, because of how the language works - although it looks like a class method, internally it is a static method, and the cls is always added explicitly, and not by the language.

In other words: we use the __new__ only to intercept and change parameters in classes that consume these parameters already in creation. In addition to the immutable types, it is common to write the __new__ when you create a metaclass - it will change some behavior in creating classes themselves. Other than these cases, even inheriting from native types like dict and list, just use the __init__ even.