Pool and Transactions with Pymongo

Question

I am making use of Mongodb + Pymongo for the first time, I am looking for information on how to work with Connections Pool and Transactions with pymongo. To date with the documentation found the basics:

#Acessando
>>> from pymongo import MongoClient
>>> client = MongoClient('localhost', 27017)
>>> db = client['test-database']

#Inserindo
>>> import datetime
>>> post = {"author": "Mike",
...         "text": "My first blog post!",
...         "tags": ["mongodb", "python", "pymongo"],
...         "date": datetime.datetime.utcnow()}

>>> posts = db.posts
>>> post_id = posts.insert_one(post).inserted_id
>>> post_id
ObjectId('...')

But how to work with Pool of connections and transactions with Pymongo? Mongodb 4 already enables multiple transactions, how do you do these things in pymongo? Where do you "enter" the commit and rollback as in relational banks?

Answer 1

Transactions

Transaction support is obtained by 3 object methods client_session:

1. start_transaction(read_concern=None, write_concern=None, read_preference=None)

2. commit_transaction()

3. abort_transaction(), who plays the role of rollback

Reference: https://api.mongodb.com/python/current/api/pymongo/client_session.html

A super simple example would be:

posts = client.db.posts
with client.start_session() as session:
    with session.start_transaction():
        post_id = posts.insert_one(post, session=session).inserted_id

The documentation says that this way, you don’t have to worry about calling commit_transaction() or abort_transaction():

Upon normal Completion of with Session.start_transaction() block, the transaction Automatically calls Clientsession.commit_transaction(). If the block exits with an Exception, the transaction Automatically calls ClientSession.abort_transaction().

More information about transactions, you can find at https://docs.mongodb.com/manual/core/transactions/

In this case, that would be the tricky example:

def run_transaction_with_retry(txn_func, session):
    while True:
        try:
            txn_func(session)  # performs transaction
            break
        except (ConnectionFailure, OperationFailure) as exc:
            # If transient error, retry the whole transaction
            if exc.has_error_label("TransientTransactionError"):
                print("TransientTransactionError, retrying "
                      "transaction ...")
                continue
            else:
                raise

def commit_with_retry(session):
    while True:
        try:
            # Commit uses write concern set at transaction start.
            session.commit_transaction()
            print("Transaction committed.")
            break
        except (ConnectionFailure, OperationFailure) as exc:
            # Can retry commit
            if exc.has_error_label("UnknownTransactionCommitResult"):
                print("UnknownTransactionCommitResult, retrying "
                      "commit operation ...")
                continue
            else:
                print("Error during commit ...")
                raise

# Updates two collections in a transactions

def meu_metodo_que_adiciona_o_post(session):        
    post = {"author": "Mike",
        "text": "My first blog post!",
        "tags": ["mongodb", "python", "pymongo"],
        "date": datetime.datetime.utcnow()}
    posts = session.client.db.posts

    with session.start_transaction(
            read_concern=ReadConcern("snapshot"),
            write_concern=WriteConcern(w="majority")):
        post_id = posts.insert_one(post, session=session).inserted_id

        commit_with_retry(session)

# Start a session.
with client.start_session() as session:
    try:
        run_transaction_with_retry(meu_metodo_que_adiciona_o_post, session)
    except Exception as exc:
        # Do something with error.
        raise

In summary, in this way, the code handles several exceptions that may occur, and attempts to effect the transaction again, if possible.

Sessions

Previously, the object client_session has been quoted, that in Mongodb represents the concept of Session (session in Portuguese).

Currently, a Session has the function of offering causal consistency(causal Consistency) and transactions.

Causal consistency is the property of ensuring that orders of transactions take place in the manner specified, and that if an operation A takes place before operation B, operation B will receive all effects of transaction A.

Reference on causal-Consistency: https://docs.mongodb.com/manual/core/read-isolation-consistency-recency/#causal-Consistency

With causally consistent Sessions, Mongodb executes causal Operations in an order that respect their causal relationships, and clients note Results that are consistent with the causal relationships.

That being said, causal consistency does not mean that it is the same as transactions, nor does it guarantee isolation. That is, other operations of other connections may affect the content of what is seen during operations within the session:

Operations Within a causally consistent Session are not Isolated from Operations Outside the Session. If a Concurrent write Operation interleaves between the Session’s write and read Operations, the Session’s read Operation may Return Results that reflect a write Operation that occurred after the Session’s write Operation.

So, the indicated use of causally consistent, is to prevent the database from attempting to perform all operations at the same time by running them in sequence.

By default, all Session already is causally consistent, but you can disable it if you want.

WARNING: In relation to sessions and transactions, always use ReadConcern('majority') and WriteConcern('majority').

If you do not follow this warning, the behavior of the database is far more complicated. It is logical that if you understand what different Read Write Concerns operations, and how they affect Replica Sets/Shards, this warning shall be ignored.