Update values from a chart using Django

Question

I’m playing in a solo project to make a website to show the current coffee quotation along with a line chart of the daily variation of this price as a function of time.

I get the price value of coffee making "webscraping" (yes, I know it’s not cool but while I do not find an API that does this for me I will use webscraping, even if the site is far from staying on the air, I’m only doing tests for now).

What I want to happen is: Given an update period (10, 15 I know how many seconds), the chart should be updated with the last value (coming from webscraping) collected and the current time.

The current code is as follows::

class CafeView(TemplateView):
    def get_context_data(self, **kwargs):
        class AppURLopener(urllib.request.FancyURLopener):  # Usado para simualar um browser e conseguir obter o html da pagina
            version = "Mozilla/5.0"
        opener = AppURLopener()
        response = opener.open('https://www.investing.com/commodities/us-coffee-c-contracts').read()
        soup = BeautifulSoup(response, 'html.parser')
        price = float(soup.find("td", {"class": "pid-8832-last"}).get_text())  # Extrai a cotação atual.
        price_var_points = float(soup.find("td", {"class": "pid-8832-pc"}).get_text())  # Extrai a variação referente à abertura da bolsa.
        context = super(CafeView, self).get_context_data(**kwargs)
        context['preco'] = price  # Passa a cotação para o template
        # Abaixo estão os parâmetros necessários para plotar o gráfico, x e y são duas listas que tem que ser preenchidas com os valores de preço/hora.
        trace1 = go.Scatter(x=x, y=y, marker={'color': 'red', 'symbol': 104, 'size': "10"},
                        mode="lines",  name='1st Trace')
        data=go.Data([trace1])
        layout=go.Layout(title="Coffee Price", xaxis={'title':'x1'}, yaxis={'title':'x2'})
        figure=go.Figure(data=data,layout=layout)
        div = opy.plot(figure, auto_open=False, output_type='div')
        context['graph'] = div
        return context
    template_name = 'cafe.html'

What I tried to do and it didn’t work was to create 2 lists within my Class based view CafeView one to store the schedules and one to store the quotations, so every time a refresh is given on the page it was pro grafico to be created with values of the incremented lists, but the lists are reset each time the template is executed and there is no way to store the previous values. One solution I thought was to save the values in the database and each time the user updates the screen the view uses the values stored in the database to build the list.

I haven’t tried this approach yet, it probably works, I would like the opinion of someone who has more knowledge to tell me which is the best way to solve this problem.

NOTE: The graph is generated using the plotly library, it generates the html in the variable div and I send this html to the template in the context variable graph.

Answer 1

Yes, the idea of storing the data in the bank is good, but if Oce does not need to persist them, since they are temporary, this strategy would consume the bank’s resources, unnecessarily. I would choose to create a bank in memory that in addition to avoiding unnecessary access to the bank, would probably have better performance.

To define a database in memory, see this link.

Since Sqlite is "built-in" in python, it’s easy to create a database in memory, with "python Pure":

sqlite3.connect(database[, timeout, detect_types, isolation_level, check_same_thread, factory, cached_statements])¶

Opens a Connection to the Sqlite database file database. You can use ":memory:" to open a database Connection to a database that resides in RAM Instead of on disk.

Specifically for Django, you can do:

DATABASES = {
  'default': {
    'ENGINE': 'django.db.backends.sqlite3',
    'NAME': ':memory:',
  }
}