Board index » delphi » fastest way to delete a lot of record in a Paradox table

fastest way to delete a lot of record in a Paradox table

My application records in a table (events.db) some events.
One field of the table is EventDate (type: date/time) and the primary index
is based on this field.

To limit the size of the table, I have to run periodically a procedure to
delete the oldest events.

Here is how this procedure looks like:

   MyTable := TTable.Create(nil);
   Try
      MyTable.DatabaseName := ChemStat;
      MyTable.TableName := 'Events.db';
      MyTable.Open;
      Ok := False;
      NbRec := MyTable.RecordCount;
      if NbRec >= 1697792 then
      begin
         MessSys('Pack statistics, please wait...',1);
         MyTable.First;
         MyTable.MoveBy(NbRec-1400000);
         Ok := True;
         OlderDate := MyTable.FieldByName(' EventDate').AsDateTime;
      end;
      MyTable.Close;
   finally
      MyTable.Free;
   end;
   if Ok then
   begin
      MyQuery := TQuery.Create(nil);
      Try
         MyQuery.DatabaseName := ChemStat;
         MyQuery.SQL.Clear;
         MyQuery.SQL.Add('DELETE FROM "Events.db" WHERE EventDate < ');
         MyQuery.SQL.Add(''''+FormatDateTime('mm/dd/yyyy hh:nn:ss',
OlderDate)+'''');
         Try
            MyQuery.ExecSQL;
         Except
            on e:Exception do
            Begin
               MessageDlg('Events: can not pack Events.db' + #13 +
                          'Error: ' + e.message, mtError, [mbOK], 0);
            end;
         End;
         MyQuery.Close;
      finally
         MyQuery.Free;
      end;
   end;

The problem is that this procedure is very slow.
Is there a way to optimize the procedure or at least have something to
display the progression of the operation.

 

Re:fastest way to delete a lot of record in a Paradox table


use delete key
:)

Quote
"MEE" <mee-e...@magic.fr> wrote in message

news:3da521ef@newsgroups.borland.com...
Quote
> My application records in a table (events.db) some events.
> One field of the table is EventDate (type: date/time) and the primary
index
> is based on this field.

> To limit the size of the table, I have to run periodically a procedure to
> delete the oldest events.

> Here is how this procedure looks like:

>    MyTable := TTable.Create(nil);
>    Try
>       MyTable.DatabaseName := ChemStat;
>       MyTable.TableName := 'Events.db';
>       MyTable.Open;
>       Ok := False;
>       NbRec := MyTable.RecordCount;
>       if NbRec >= 1697792 then
>       begin
>          MessSys('Pack statistics, please wait...',1);
>          MyTable.First;
>          MyTable.MoveBy(NbRec-1400000);
>          Ok := True;
>          OlderDate := MyTable.FieldByName(' EventDate').AsDateTime;
>       end;
>       MyTable.Close;
>    finally
>       MyTable.Free;
>    end;
>    if Ok then
>    begin
>       MyQuery := TQuery.Create(nil);
>       Try
>          MyQuery.DatabaseName := ChemStat;
>          MyQuery.SQL.Clear;
>          MyQuery.SQL.Add('DELETE FROM "Events.db" WHERE EventDate < ');
>          MyQuery.SQL.Add(''''+FormatDateTime('mm/dd/yyyy hh:nn:ss',
> OlderDate)+'''');
>          Try
>             MyQuery.ExecSQL;
>          Except
>             on e:Exception do
>             Begin
>                MessageDlg('Events: can not pack Events.db' + #13 +
>                           'Error: ' + e.message, mtError, [mbOK], 0);
>             end;
>          End;
>          MyQuery.Close;
>       finally
>          MyQuery.Free;
>       end;
>    end;

> The problem is that this procedure is very slow.
> Is there a way to optimize the procedure or at least have something to
> display the progression of the operation.

Re:fastest way to delete a lot of record in a Paradox table


Your source code talks about packing the table, but I don't see
anywhere where you actually do pack the table; you're just
deleting records.  Deleting records doesn't reduce the size of
the table, or reorder the table.  Packing the table may improve
performance, and it never hurts.

Try packing the table interactively with Database Desktop
(Table-Restructure, and check the "Pack table" box). If that
improves performance, consider making a BDE API call to pack
the table; see info and examples by following the links from
http://info.borland.com/devsupport/bde/

Rick Carter
Rick.Car...@cincww.rcc.org
Chair, Paradox/Delphi SIG, Cincinnati PC Users Group

Quote
"MEE" <mee-e...@magic.fr> wrote:
>My application records in a table (events.db) some events.
>One field of the table is EventDate (type: date/time) and the primary index
>is based on this field.

>To limit the size of the table, I have to run periodically a procedure to
>delete the oldest events.

>Here is how this procedure looks like:

>   MyTable := TTable.Create(nil);
>   Try
>      MyTable.DatabaseName := ChemStat;
>      MyTable.TableName := 'Events.db';
>      MyTable.Open;
>      Ok := False;
>      NbRec := MyTable.RecordCount;
>      if NbRec >= 1697792 then
>      begin
>         MessSys('Pack statistics, please wait...',1);
>         MyTable.First;
>         MyTable.MoveBy(NbRec-1400000);
>         Ok := True;
>         OlderDate := MyTable.FieldByName(' EventDate').AsDateTime;
>      end;
>      MyTable.Close;
>   finally
>      MyTable.Free;
>   end;
>   if Ok then
>   begin
>      MyQuery := TQuery.Create(nil);
>      Try
>         MyQuery.DatabaseName := ChemStat;
>         MyQuery.SQL.Clear;
>         MyQuery.SQL.Add('DELETE FROM "Events.db" WHERE EventDate < ');
>         MyQuery.SQL.Add(''''+FormatDateTime('mm/dd/yyyy hh:nn:ss',
>OlderDate)+'''');
>         Try
>            MyQuery.ExecSQL;
>         Except
>            on e:Exception do
>            Begin
>               MessageDlg('Events: can not pack Events.db' + #13 +
>                          'Error: ' + e.message, mtError, [mbOK], 0);
>            end;
>         End;
>         MyQuery.Close;
>      finally
>         MyQuery.Free;
>      end;
>   end;

>The problem is that this procedure is very slow.
>Is there a way to optimize the procedure or at least have something to
>display the progression of the operation.

Re:fastest way to delete a lot of record in a Paradox table


I would use TTable to filter the table so the only records being
processed are the records you want to delete.  Then,

    while Table1.EOF = false do
      begin
        Table1.Delete;
      end;

                     Maurie

Quote
MEE wrote:

> My application records in a table (events.db) some events.
> One field of the table is EventDate (type: date/time) and the primary index
> is based on this field.

> To limit the size of the table, I have to run periodically a procedure to
> delete the oldest events.

> Here is how this procedure looks like:

>    MyTable := TTable.Create(nil);
>    Try
>       MyTable.DatabaseName := ChemStat;
>       MyTable.TableName := 'Events.db';
>       MyTable.Open;
>       Ok := False;
>       NbRec := MyTable.RecordCount;
>       if NbRec >= 1697792 then
>       begin
>          MessSys('Pack statistics, please wait...',1);
>          MyTable.First;
>          MyTable.MoveBy(NbRec-1400000);
>          Ok := True;
>          OlderDate := MyTable.FieldByName(' EventDate').AsDateTime;
>       end;
>       MyTable.Close;
>    finally
>       MyTable.Free;
>    end;
>    if Ok then
>    begin
>       MyQuery := TQuery.Create(nil);
>       Try
>          MyQuery.DatabaseName := ChemStat;
>          MyQuery.SQL.Clear;
>          MyQuery.SQL.Add('DELETE FROM "Events.db" WHERE EventDate < ');
>          MyQuery.SQL.Add(''''+FormatDateTime('mm/dd/yyyy hh:nn:ss',
> OlderDate)+'''');
>          Try
>             MyQuery.ExecSQL;
>          Except
>             on e:Exception do
>             Begin
>                MessageDlg('Events: can not pack Events.db' + #13 +
>                           'Error: ' + e.message, mtError, [mbOK], 0);
>             end;
>          End;
>          MyQuery.Close;
>       finally
>          MyQuery.Free;
>       end;
>    end;

> The problem is that this procedure is very slow.
> Is there a way to optimize the procedure or at least have something to
> display the progression of the operation.

Re:fastest way to delete a lot of record in a Paradox table


I'd use the SQL statement to delete (faster then table.delete. If you use
Table.Delete do a DisableControls first, then re-activate them > faster).
Then i'd pack the table using the BDE API PackTable.

--
Regards,
Pedro MG
www.tquadrado.com

Re:fastest way to delete a lot of record in a Paradox table


Time/Resources consuming features:

MyTable.Open;
NbRec := MyTable.RecordCount;
MyTable.MoveBy(NbRec-1400000);

So problebly you dont have contros connected to the table, since you created
it there and don't have DataSources attached...

--
Regards,
Pedro MG
www.tquadrado.com

Quote
"MEE" <mee-e...@magic.fr> wrote in message

news:3da521ef@newsgroups.borland.com...
Quote
> My application records in a table (events.db) some events.
> One field of the table is EventDate (type: date/time) and the primary
index
> is based on this field.

> To limit the size of the table, I have to run periodically a procedure to
> delete the oldest events.

> Here is how this procedure looks like:

>    MyTable := TTable.Create(nil);
>    Try
>       MyTable.DatabaseName := ChemStat;
>       MyTable.TableName := 'Events.db';
>       MyTable.Open;
>       Ok := False;
>       NbRec := MyTable.RecordCount;
>       if NbRec >= 1697792 then
>       begin
>          MessSys('Pack statistics, please wait...',1);
>          MyTable.First;
>          MyTable.MoveBy(NbRec-1400000);
>          Ok := True;
>          OlderDate := MyTable.FieldByName(' EventDate').AsDateTime;
>       end;
>       MyTable.Close;
>    finally
>       MyTable.Free;
>    end;
>    if Ok then
>    begin
>       MyQuery := TQuery.Create(nil);
>       Try
>          MyQuery.DatabaseName := ChemStat;
>          MyQuery.SQL.Clear;
>          MyQuery.SQL.Add('DELETE FROM "Events.db" WHERE EventDate < ');
>          MyQuery.SQL.Add(''''+FormatDateTime('mm/dd/yyyy hh:nn:ss',
> OlderDate)+'''');
>          Try
>             MyQuery.ExecSQL;
>          Except
>             on e:Exception do
>             Begin
>                MessageDlg('Events: can not pack Events.db' + #13 +
>                           'Error: ' + e.message, mtError, [mbOK], 0);
>             end;
>          End;
>          MyQuery.Close;
>       finally
>          MyQuery.Free;
>       end;
>    end;

> The problem is that this procedure is very slow.
> Is there a way to optimize the procedure or at least have something to
> display the progression of the operation.

Other Threads