Bonus for better performance in massively updated records search using a secondary table with Laravel

Asked

Viewed 23 times

0

I am using the Laravel framework in a project and in it there is the model Product.

On this system, I also have an API where one of the endpoints gets a date and should return the changed or removed products from that date onwards.

On the table products got the spine updated_at index-free.

My first approach to the endpoint mentioned above would be to list all products where the column updated_at is greater than or equal to the date passed by parameter to the API.

However I see a performance problem ai (updated_at has no index) which led to think about other approaches.

The Laravel offers the Observer and with this feature I decided to create a secondary table that would always receive information from id of the product that has been registered, modified or removed.

For that I would have the model ProductSync to insert the product ids and the product_sync table would have an index in the column created_at

class ProductObserver
{
    /**
     * Handle the Product "created" event.
     *
     * @param  \App\Product  $product
     * @return void
     */
    public function created(Product $product)
    {
        ProductSync::create(['product_id' => $product->id]);
    }

    /**
     * Handle the Product "updated" event.
     *
     * @param  \App\Product  $product
     * @return void
     */
    public function updated(Product $product)
    {
        ProductSync::create(['product_id' => $product->id]);
    }

    /**
     * Handle the Product "deleted" event.
     *
     * @param  \App\Product  $product
     * @return void
     */
    public function deleted(Product $product)
    {
        ProductSync::create(['product_id' => $product->id]);
    }
}

Then I’d make a relationship hasOne with the model Product.

But I found out with this approach I have 2 problems:

1) Laravel does not trigger events for massive changes.

2) Duplicate records in the table product_sync (could solve with a select distinct)

Another approach I thought was to create a Rigger straight into the database to popular the table product_sync, but the performance problem can be big for mass upgrades.

Another approach when there are mass changes would be to play the routine of creating the records in the table product_sync in a row and she would do the job slowly.

Taking into account that I can not put index in the column updated_at table products, how can I solve this problem for massive changes?

For example, imagine that I have on the table products 5 million records and a mass change were to change 75,000 records?

Product::whereIn('category_id', [421, 923, 1321])->update([
    'status' => 'paused'
]);
  • Why not add the index ?

  • The idea is to use another table.

  • You could put up the index and test it, but is this volume of data real? if it is the problem is how could improve these upgrades, is that even the amount of records?

  • Does Productsync only accept product id? As you are sending data to create by what I see you pass the Model Products and id to it, but you should not pass a Request to receive from a view?

No answers

Browser other questions tagged

You are not signed in. Login or sign up in order to post.