Background routine for data compilation is a good idea?

Asked

Viewed 35 times

0

The last company I worked for had some very peculiar rules, these not being as common (as far as I know) in other IT companies. Basically we compiled all the data that would be used in the system charts. In this compilation the data were grouped by day and by hour, and saved in two tables, one being compiled_diaand the other being compiled_hora. The idea of doing this was as follows, I have to suppose 1 million sales records in the sales table, instead of loading all 1 million sales table records, only the compiled_daily table data were loaded, thus there was a substantial drop in the number of records that were carried. As for the customer, it is only important how he will view the data no matter how the monthly chart data was assembled.

Giving a clearer example, let’s assume that we have 100,000 records in the sales chart this month. If I take the raw table data, I will have to load 100,000 records, if I take the compiled data table, it will be only 30 records, one for each day, then in the compilation I have the sum of all records.

What do you think is a good idea? Or do you have some technical problem with it?

  • 2

    No problem if the solution is appropriate to the rules of the business. Currently several DBMS already have materialized views (Materialized Views) that meet this requirement, as long as the refresh is not automatic.

  • 2

    You can delve into the subject by studying BI, Data Warehouse and Cubes... everything will make more sense and maybe even have how to improve this strategy

No answers

Browser other questions tagged

You are not signed in. Login or sign up in order to post.