Why do formatting options not work with lists, dictionaries, and other objects?

Question

When I want to print a number or string, I can use f-strings (in Python >= 3.6) or str.format, and I can only pass the variable between keys, or use the formatting options. Ex:

numero, texto = 10, 'abc'

# passando somente as variáveis
print(f'{numero} {texto}')
# ou
#print('{} {}'.format(numero, texto))

# usando as opções de formatação
# número alinhado à esquerda, ocupando 6 posições, texto alinhado à direita ocupando 10 posições
print(f'{numero:<6} {texto:>10}')
# ou
#print('{:<6} {:>12}'.format(numero, texto))

Exit:

10 abc
10            abc

But if I do the same with lists or dictionaries, only the first option works:

lista = [1, 2]
dic = {'a': 1}
print(f'{lista} {dic}')
# ou
#print('{} {}'.format(lista, dic))

print(f'{lista:<10} {dic:>15}')
# ou
#print('{:<10} {:>15}'.format(lista, dic))

The first print prints:

[1, 2] {'a': 1}

But the second print (with the formatting options <10 and >15) makes a mistake:

TypeError: unsupported format string passed to list.__format__

---

If I try the same thing with an instance of some class I created, the same thing happens. Ex:

class Teste:
    def __init__(self, valor):
        self.valor = valor

    def __str__(self):
        return f'Teste({self.valor})'

t = Teste(42)
print(f't={t}')
print(f'{t:>10}')

The first print prints:

t=Teste(42)

Already the second print makes a mistake:

TypeError: unsupported format string passed to Teste.__format__

---

My question nay it’s about how to fix (just turn the list/dictionary/object into string - for example, using str - or iterate through its elements/attributes and mount the string manually, etc).

What I want to know is why it happens. Why is it not possible to use formatting options with lists, dictionaries and instances of any classes, while passing them without any option works normally? There is some detail in the inner workings of these types that differs them from numbers and strings, when these are formatted?

Answer 1

When you pass only the variable without any formatting option (print(f'{variavel}')), internally is being called the method __str__ of the same. That is, in the case of the class Teste, as she already had this method implemented:

class Teste:
    def __init__(self, valor):
        self.valor = valor

    def __str__(self):
        print('chamando __str__') # incluindo um print só para mostrar que realmente passa por aqui
        return f'Teste({self.valor})'

t = Teste(42)
print(f'{t}')

The exit will be:

chamando __str__
Teste(42)

But in fact what is being called in fact is the method __format__, that according to the documentation, is called when evaluating the object when it is in a f-string, or by passing it to str.format or when passing it to the built-in format.

But as the class Teste did not define the method __format__, then she uses what was inherited from object. And if we check the implementation of the same (no source code of Cpython - version consulted on 04/28/2020, with comments removed):

static PyObject *
object___format___impl(PyObject *self, PyObject *format_spec)
{
    if (PyUnicode_GET_LENGTH(format_spec) > 0) {
        PyErr_Format(PyExc_TypeError,
                     "unsupported format string passed to %.200s.__format__",
                     Py_TYPE(self)->tp_name);
        return NULL;
    }
    return PyObject_Str(self);
}

That is, when no formatting option is passed (only {variavel}), he doesn’t get into the if and returns PyObject_Str(self) (and according to the documentation, PyObject_Str(algo) is equivalent to calling str(algo) - which in turn, flame algo.__str__()).

Now if I pass some formatting option (like for example print(f'{variavel:>10}')), he enters the if and shows the error message ("Unsupported format string etc").

According to the documentation, this behavior of launching the TypeError if the format string is not empty has been since Python 3.4, and the call to str(self) has been since Python 3.7 (before it was called format(str(self), ''), as stated in this commit).

I mean, for this to work in my class, I would have to implement the method __format__:

class Teste:
    def __init__(self, valor):
        self.valor = valor

    def __str__(self):
        print('chamando __str__')
        return f'Teste({self.valor})'

    def __format__(self, format_spec):
        print(f'chamando __format__ com formato: "{format_spec}"')
        return f'{self.valor:{format_spec}}'

t = Teste(42)
print(f'{t}')
print(f'{t:>10}')

Now yes both print's work (and I implemented the method __format__ so as not to delegate __str__, but could have done it if you wanted to). The output is:

chamando __format__ com formato: ""
42
chamando __format__ com formato: ">10"
        42

And the same output would be obtained if I did:

print('{}'.format(t))
print('{:>10}'.format(t))

---

Anyway, that’s why formatting options don’t work with lists and dictionaries, because the respective classes (list and dict) do not implement the method __format__ and use the legacy implementation of object (which it delegates to str when no options are passed, and gives error when options are used - so it works when I do only print(f'{lista}')).

Unfortunately we can not implement __format__ in lists and dictionaries, since cannot add new methods to native classes, then the way is to turn them into strings (whether using str, iterating through its elements and assembling the string in the desired format).

Numbers and strings implement the method __format__ and therefore work with formatting options.

And as already said, this is not limited to f-strings. The same behavior occurs with str.format and the built-in format:

t = Teste(42) # usando a última versão acima, com a classe Teste implementando __format__
# ambas as linhas abaixo chamam Teste.__format__
print('{:>10}'.format(t))
print(format(t, '>10'))

lista = [1, 2]
# ambas as linhas abaixo lançam TypeError: unsupported format string passed to list.__format__
print('{:>10}'.format(lista))
print(format(lista, '>10'))

---

In fact, this is why it is also possible to format the classes of module datetime in this way - for example f'{datetime.now():%d/%m/%Y}' - because those classes override the method __format__, delegating the call to strftime. So I could also do something similar with the class Teste, and define the formats I want:

class Teste:
    # ... construtor, etc

    def __format__(self, format_spec):
        formatos = { # formatos customizados
          'formato_x' : f'formato x -> {self.valor}',
          'outro formato' : f'outro: {self.valor}'
        }
        if format_spec in formatos:
            return formatos[format_spec]

        return f'{self.valor:{format_spec}}'

t = Teste(42)
print(f'{t:formato_x}') # formato x -> 42
print(f'{t:outro formato}') # outro: 42
print(f'{t:>10}')

# ou
#print('{:formato_x}'.format(t))
#print('{:outro formato}'.format(t))
#print('{:>10}'.format(t))

Exit:

formato x -> 42
outro: 42
        42