1
Following what I had asked here, I keep trying to run my code asynchronously. I’m trying to use Celery to perform this procedure, but I’m having some problems.
I had the following:
def get(self, request):
queryset = Model.objects.raw() # Faço um MATCH com parametros digitados pelo usuario e valores do banco.
for res in queryset:
processamento_dos_dados
return resultado_do_processamento
What I did was this:
With Celery, in my tasks.py
put my function:
@shared_task
def teste(queryset, variavel1, lista1, variavel2, lista2):
for res in queryset:
processamento_de_dados # Aqui são alguns procedimentos de comparação necessarios para minha aplicação
return lista2
In my views.py
:
from .tasks import teste
class CLASSE(APIView):
def get(self, request):
# Processamentos para gerar variavel1, lista1, variavel2
queryset = Model.objects.raw() # Faço um MATCH com parametros digitados pelo usuario e valores do banco.
lista2 = []
lista2 = teste.delay(queryset, variavel1, lista1, variavel2, lista2)
This is where the problems begin. If I try to pass the way it is there teste.delay(variaveis)
, returns an error saying that queryset is not a serializable JSON object. If I remove . delay, it performs synchronously, as if not using Celery.
Researching I found this possible solution:
test = serializers.serializer('json', queryset)
lista2 = teste.delay(test, variavel1, lista1, variavel2, lista2)
However, the serialization time for JSON is out of the question (the code is stuck there a long time, I could not in any of my tests make it pass the serialization).
I have also tried to exchange . delay() for .apply_async(), however I do not know how to pass all the necessary variables within the args=
.
Another solution I found was this one, but again, I do not know how to pass all the parameters necessary for the execution of the function.
As indicated in my first question, I tried to use generators to optimize my code, but I didn’t feel any difference in the execution. I followed this here, doing as follows:
queryset = Model.objects.raw() #MATCH de valores
for res in queryset.iterator():
Does this error only happen when you add parameters? I recently tested background worker and gave the same error when using http requests. I’ll share my implementation with Celery, use parameters with . delay().
– Ernesto Casanova
So, I tested only with parameters, because I couldn’t find a way to use my function without them... I even managed to use . apply_async(), but I get the same error as Queryset. Apparently there is no way to pass queryset as parameter to Celery
– Willian Freitas
You do the queries in the task, that should be in the task, it will take time. Do not pass queryset.
– Ernesto Casanova
Meanwhile I found a post with your difficulty, and with my suggestion and with another, take a look. https://stackoverflow.com/questions/34765276/celery-raise-error-while-passing-my-queryset-obj-as-parameter
– Ernesto Casanova