Organizing code within __init__.py

Question

I’m having trouble organizing my code. Ex: I would like to put generic functions, as pro example those that deal with reading a configuration file in only one place (read that the __init__.py is perfect for this). However, I cannot.

This is the current organization of my code:

├── nome_projeto
│   ├── nome_projeto.py
│   ├── config.ini
│   ├── database.py
│   └── __init__.py
├── README.md
├── requirements.txt
└── setup.py

This is the contents of my __init__.py, which includes the log settings, the configuration file and also a function that returns a parameter from the configuration file.

"""Funções básicas usadas pelo programa"""


import os
import logging
import pathlib
import datetime
import configparser

#   Arquivo de configuração
CONFIG_FILE = 'config.ini'
CONFIG_PATH = ''.join([str(pathlib.Path(__file__).parent), '/', CONFIG_FILE])

if os.path.exists(CONFIG_PATH):
    config = configparser.ConfigParser()
    config.read(CONFIG_PATH)
else:
    logging.error(f'Conf file not found: {CONFIG_FILE}')


#   Cria diretório temporário se não existir
try:
    if not os.path.exists(config['DEFAULT']['TempDir']):
        #   Temp dir
        os.makedirs(config['DEFAULT']['TempDir'])

    if not os.path.exists(config['DEFAULT']['Log']):
        #   Verifica se existe, se não, cria
        log_file = open(config['DEFAULT']['Log'])
        log_file.write(f"Created at {str(datetime.datetime.today())}")
        log_file.close()
    elif not os.access(config['DEFAULT']['Log'], os.W_OK):
        #   Verifica se tem permissão de escrita
        logging.error(f"No permision to write: {config['DEFAULT']['Log']}")

except Exception as e:
    logging.error(f'Fail during creation: {e}')


#   Log
log_format = '%(asctime)s [%(filename)s:%(lineno)d] %(message)s'
logging.basicConfig(filename=config['DEFAULT']['Log'],
                    format=log_format, level=logging.DEBUG)


def get_conf(param):
    """Retorna parametro"""
    return config['CONF'][param]

And here is one of the files of my project that would use the function get_config

#!/usr/bin/env python3
# -*- coding: utf-8 -*-

"""
Database access.
"""

import pymysql
import logging
from . import get_conf
from pymysql import Error


def connect():
    """Abre coneção"""
    try:
        connection = pymysql.connect(host=get_conf('Address'),
                                     user=get_conf('DatabaseUser'),
                                     password=get_conf('DatabasePassword'),
                                     db=get_conf('Database'),
                                     charset='utf8mb4')
    except Error as e:
        logging.error(e)
    cursor = connection.cursor()
    return connection, cursor


def close(connection):
    """Commita e fecha coneção"""
    connection.commit()
    connection.close()


def lookup(option, tell):
    """ Recebe uma opção e o telefone.
    Retorna ou a senha, ou a contrasenha baseado na opção.
    """
    try:
        connection, cursor = connect()
        if option == 'pass':
            col = 'password_col_name'
        else:
            col = 'counter_password_col_name'
        cursor.execute(f"""
            SELECT {col} FROM table_name WHERE col_phone_number = {tell};
        """)
        return cursor.fetchone()
    except Error as e:
        logging.error(e)


if __name__ == "__main__":
    """Caso seja executado sozinho"""
    print(f"""
    Database: \t\t{get_conf('Database')}
    Address: \t\t{get_conf('Address')}
    Database User: \t{get_conf('DatabaseUser')}
    Database Password: \t{get_conf('DatabasePassword')}
    """)

First I’d like to understand if what I’m trying to do is right. I should store this kind of thing inside the __init__.py? And I would also like to try to understand why I’m not being able to use the function, since I care about the . and only get an error message Import: attempted relative import with no known Parent package?

Answer 1

Come on -

in fact, what goes inside the __init__ might be a little wild.

If the project is a library - with the idea of being imported and used by other projects, the best thing to do is to leave the classes and public functions - or at least those that will be used most of the time by those who will use your project, exposed in __init__.py. A project that I invest a lot in is in my library finished.

Doesn’t mean we need to declare those features there - and in general we shouldn’t - just import them there on filing cabinet __init__.py at the right point, make them visible.

For projects that are the final application, as a service (web or other) or a desktop app - then you can leave what is convenient in the __init__.py or break as you please.

You have to remember one thing: whether the __init__ imports the others modules of its application, and commands of import are at the beginning of file, when other modules are loaded, functions and variables defined in __init__ have not yet been created. That is, can fall into a problem of circular imports, and the project will not work.

Some configuration things like in the example you gave, can stay there - but no need, it’s not nice to leave "loose" code in the body of the module - better organize in a function, and call it.

(And then, if that’s the case, you can put the call inside a if __name__ == "__main__": which makes your project, without any other code, can be a stand-alone application or a library):

"""Funções básicas usadas pelo programa"""

import os
import logging
import datetime
import configparser
from pathlib import Path

#   Arquivo de configuração
CONFIG_FILE = 'config.ini'
CONFIG_PATH = Path(__file__) / CONFIG_FILE

if CONFIG_PATH.exists():
    config = configparser.ConfigParser()
    config.read(CONFIG_PATH)
else:
    logging.error(f'Conf file not found: {CONFIG_FILE}')


#   Log
log_format = '%(asctime)s [%(filename)s:%(lineno)d] %(message)s'
logging.basicConfig(filename=config['DEFAULT']['Log'],
                    format=log_format, level=logging.DEBUG)

def prepare_files():
    #   Cria diretório temporário se não existir
    try:
        if not (tmpdir:= Path(config['DEFAULT']['TempDir'])).exists():
            # (acima o operador da "Morsa" (Walrus) - cria a variável
            # dentro da expressão de um if, economizando uma linha.
            # Requer Python 3.8
            # )
            #   Temp dir
            tmpdir.mkdir(parents=True)

        if not (log_path:=Path(config['DEFAULT']['Log'])).exists():
            log_path.write_text(f"Created at {str(datetime.datetime.today())}\n")
            # write_text já abre o arquivo, escreve a string, e fecha.
        elif not os.access(log_path, os.W_OK):
            #   Verifica se tem permissão de escrita
            logging.error(f"No permision to write: {log_path}")

    except Exception as e:
        logging.error(f'Fail during creation: {e}')

prepare_files()

def get_conf(param):
    """Retorna parametro"""
    return config['CONF'][param]

As for your mistake:

Import: attempted relative import with no known Parent package

So - this happens because you must be within from the same directory where __init__.py and trying to run the file database.py straight from there.

If you switch to the directory where the setup.py and type python nome_projeto/database.py that same import will work.

It turns out that imports relative using "." or "." as a module name prefix only work if the file that makes the "import" is in the same package (package) that Arq it imports. And for Python to "know" that it is in the same package, it has rather than call the __init__.py from the same directory, and has to be called "outside" from the "project name_folder".

Only "call" is no use in writing python nome_projeto or python -m nome_projeto - neither of these two ways first executes the __init__.py. For this to work, you must also create a file __main__.py - (which may be empty) - then you can run your entire project as a module, using the -m option (python -m nome_projeto) - Python then loads the __init__.py, runs it all, and then executes the file __main__.py. If you want to rotate the database.py or another file, it needs to be imported from one of these two files - and then, the database.pywill be recognized as "I’m inside a package" - and will be able to have the relative import as you wrote.

Without the file __main__.py your project can already be imported by other packages, and the __init__ will work - you can run an interactive Python in the folder where setup.py is, and use import nome_projeto, for example. But to be called straight from the command line, you need the file __main__.py.

So that’s it - just be careful to avoid circular imports on __init__ as I wrote above - can put a import for each of the other files at the end of the __init__ (thus, when they are called, the function get_conf will already be set:

__init__.py:

import datetime
...
def get_conf(conf):
   ...

from . import database, nome_projeto

database py.:

...
from . import get_conf
...