Delete thousands of records from a table

Asked

Viewed 6,983 times

31

I have a table, for example, pessoa, and would like to delete all people over 40, with the script below do it:

DELETE FROM pessoa WHERE pessoa.idade > 40;

The problem there is more than 2 million record in the table, with people over 40, and as the bank is in production, while making this will lock all users.

There is a more efficient way to solve this problem?

  • 9

    You can limit the query DELETE FROM pessoa WHERE pessoa.idade > 40 LIMIT 5000 and make a script to run this query in x seconds until no lines exist

  • Following more or less the same line of reasoning that lazyFox said, this topic here also says, to create batch files and run little by little: http://stackoverflow.com/questions/24785439/deleting-1-millions-rows-in-sql-server

  • The table has fk s that depend on it ? How many records would be in the table ? There are solutions like exporting what will be in an Insert script , truncate in the table and import , open the exclusive bd , turn off the Audit , delete , turn on the Audit and reopen the bd

  • 1

    Copy Data that matters to Novatable and delete or rename the Old Table and rename the New Table to Old Table :)

  • @Motta, I mistook me for the chart, she’s 4 fk, and six Eagles, you’d be left with 82 million.

  • maybe it is better something like : saves the records that should be in scripts of Insert , disables the constraints , gives truncate in the tables , performs the script of (re)Insert , habitita as constrainsts , in any case search in sites like asktom because it has enough material about it.

  • The problem of entering this data again, with the scripts of Insert, is that there are many Inserts to be made, it will take a lot of time. !

  • I am far from being a deep connoisseur of Oracle and SQL. However, I think there are two relevant questions: 1) is this removal something that occurs frequently during the operation? 2) Does this removal need to be immediate? I ask this because maybe you can simply mark these records as irrelevant (pending for deletion) and run a trial overnight (or at a low system usage time) to do the removal itself.

  • 1

    The operation does not occur frequently, but occurs occasionally. Yes the operation needs to be done immediately. Thanks Luiz.

  • @lazyFox, the hard one I need to remove all at once, and do x and x seconds, splitting 2kk/5k would be 6 minutes, wouldn’t be so bad, but wanted something in the second house.. thank you

  • How the operation occurs occasionally: first eliminates those that already exist. And from there every time you enter new records also deletes those on the condition you wish, in this case > 40.

  • There are some techniques described in Delete row set in huge tables -> https://portosql.wordpress.com/2019/10/16/delete-vlt/

Show 7 more comments

4 answers

19


You should proceed with parallel operations, if I had a good processor can run as follows:

ALTER SESSION ENABLE PARALLEL DML;
DELETE /*+ parallel(pessoa, 20) */
  FROM  pessoa
  WHERE idade > 40;

Follow the link from Oracle website where it talks about process parallelization, another good reference, and another reference to how to delete large amounts of information.


In my opinion, I work with very large banks with over 100 million records. A delete of these that you want to do should not take more than 2 or 3 minutes if your server is good.

  • Thanks, @Jaser for the reply, I will read the documentation and test.

  • I ran a test in another table with 800k and it took 19s, very interesting, I will wait some more answer, otherwise I will run your suggestion in the main table. Thank you

  • thanks just posted it here because that’s how I work with my banks. Remembering that it creates a parallel thread so it of the processing priority but it switches with other processes , is not a real parallel process.

  • managed to do what he wanted?? ^^

7

Instead of you running delete to delete all records, you can create a PL and gradually delete and commit every x records, this will prevent you from "locking" other users.

declare
  cursor c is select p.rowid r from pessoa p where p.idade > 40;
  c_commit number:= 0;
  l_commit number:= 10000;
  begin
    for t in c 
    loop
      c_commit := c_commit + 1;
      delete from pessoa p where p.rowid = t.r;

      if mod( c_commit, l_commit ) = 0 then
        commit;
      end if;
    end loop;
    commit;
end;
/

7

I’m not an oracle expert, but you can also create a table with the records you want keep up.

Something like:

create table pessoa_new as select * from pessoa where idade <= 40;

After that, run a truncate in the table original:

truncate table pessoa

Note that truncate does not create log in oracle, that is, you will not can do rollback or commit. Once done, it cannot be undone.

After that, copy the data back to the original table:

alter session set rollback_segment = 'HUGE_RBS';   

insert into pessoa as select * from pessoa_new

Or if you prefer, change the name of the new and old table:

alter table pessoa rename to pessoa_old;
alter table pessoa_new rename to pessoa;

In either case, remember rebuild If you choose to rename tables, consider recreating constraints, triggers, etc.

1

The Aron Linhares solution works well, it would only add the use of BULK COLLECT to improve performance.

DECLARE
  CURSOR C_CURSOR IS SELECT P.ROWID AS CHAVE FROM PESSOA P WHERE P.IDADE > 40;
  TYPE TYPE_CURSOR IS TABLE OF C_CURSOR%ROWTYPE INDEX BY BINARY_INTEGER;
  R_CURSOR    TYPE_CURSOR;       
  BEGIN
    OPEN C_CURSOR;
      LOOP
      --preeche a memória de 1000 em 1000 registros
        FETCH C_CURSOR BULK COLLECT INTO R_CURSOR LIMIT 1000;
        EXIT WHEN R_CURSOR.COUNT = 0;
          FOR I IN 1 .. R_CURSOR.COUNT
          LOOP 
              DELETE FROM PESSOA P WHERE P.ROWID = R_CURSOR(I).CHAVE;
          END LOOP;
          --VAI APLICAR COMMIT DE 1000 EM 1000;
          COMMIT;
     END LOOP C_CURSOR;
    CLOSE C_CURSOR;
END;
/

In the documentation of Oracle, you will find more details on how to work with large data mass in PL/SQL.

http://www.oracle.com/technetwork/issue-archive/2012/12-sep/o52plsql-1709862.html

Browser other questions tagged

You are not signed in. Login or sign up in order to post.