Problems with Memcached + Django + Celery(Rabbitmq)

Asked

Viewed 43 times

1

I’m having trouble using the memcached + Celery(Django). My program makes a query in a database, through a cursor, creates chuncks with the return of the bank and sends these Chunks to be processed by Celery (I use Rabbitmq as worker). I need the result of this processing to give sequence, for this I’m trying to save it via Memcached.

py views.:

class Sound(APIView):

    def get(self, request):

        #Aqui são feitos alguns pré-processamentos necessarios para a sequencia do código.

        query = ('MINHA CONSULTA SQL AQUI')

        cache.clear()
        cursor = connection.cursor()
        cursor_size = cursor.execute(query)
        print(cursor_size)
        idcelery = 0

        while True:
            row = cursor.fetchmany(int(cursor_size/20))
            if not row:
                break;

            control = teste.s(idcelery, variaveis).apply_async()
            idcelery += 1

        while control.ready() == False:
            pass

        resultado = 0

        if cache.get('20'):
            resultado = 20
        elif cache.get('40'):
            resultado = 40
        elif cache.get('80'):
            resultado = 80
        elif cache.get('90'):
            resultado = 90

        print(cache.get('20'))
        print(cache.get('40'))
        print(cache.get('80'))
        print(cache.get('90'))

        print(resultado)

        return HttpResponse(json.dumps(resultado), content_type="application/json; charset=utf-8")

tasks py.

@shared_task
def teste(idcelery, variaveis):

    for campos in row:
        #Aqui é feito todo o processamento da minha query, no final ela resulta em um valor para a minha registerpercent.

        if registerpercent == 20:
            print(registerpercent)
            cache.set('20', True)

        elif registerpercent == 40:
            cache.set('40', True)

        elif registerpercent == 80:
            cache.set('80', True)

        elif registerpercent == 90:
            cache.set('90', True)

In this case, based on the result of my task, a value is saved in cache memory, being 20, 40, 80 or 90. These values have different weights, being the order of priority: 20>40>80>90 (that is, if 20 occurs, the same is who must be returned by the program, idependente of the other values that may occur, and so on until 90).

The problem is that in some tests I did, even though the worker found the value of more weight, it did not get to be saved in the cache, thus returning a value of less weight (90). Can anyone tell me why this is happening? I also noticed that not always in the worker’s terminal, the registerpercent print with the value 20 appears, but the value is returned correctly.

No answers

Browser other questions tagged

You are not signed in. Login or sign up in order to post.