Board index » delphi » Multiple file open/close anomaly?

Multiple file open/close anomaly?

Take this snippet:

const
  sfile: integer = -1;

procedure open(_file: integer);
begin
{$ifdef fast}
  if _file <> sfile then
    begin;
      sfile:= _file;
{$endif}
      close(datatout);
      assign(datatout, files[_file]);
      settextbuf(datatout, databuf^, 8192);

      if __first_event then
        begin
          rewrite(datatout);
          __first_event:= false;
        end
      else
        append(datatout);
{$ifdef fast}
    end;
{$endif}
end; {open}

It's used a program in which many procedures can write their results to one
of two files (a per event file and a summary file). Going through the
program, I suddenly realized that I had never tracked calls to "open", and
much to my surprise I discovered that on several occasions it's called with
the same value for _file on several consecutive invovations. So I added the
conditionals, and timed again, clean DOS, SMARTDRV with write caching, and
running both the old and the new versions several times in alternating
succession. Much to my surprise the old version, which performs the code
dozens of times more than the the new version, is actually faster! (NB: The
file is initially opened for summary data in the main program)

Does anyone have an explanation?

Thanks,

Robert
--
Robert AH Prins
prin...@williscorroon.com

-----------== Posted via Deja News, The Discussion Network ==----------
http://www.dejanews.com/       Search, Read, Discuss, or Start Your Own    

 

Re:Multiple file open/close anomaly?


Quote
Robert AH Prins wrote:

> Take this snippet:

> const
>   sfile: integer = -1;

> procedure open(_file: integer);
> begin
> {$ifdef fast}
>   if _file <> sfile then
>     begin;
>       sfile:= _file;
> {$endif}
>       close(datatout);
>       assign(datatout, files[_file]);
>       settextbuf(datatout, databuf^, 8192);

Are you SURE that this SetTextBuf is done also in your
first rewrite (which is not shown here obviously)?
Cannot find any other reason for your experience :-)
If not, how often do you skip the code here...

Quote

>       if __first_event then
>         begin
>           rewrite(datatout);
>           __first_event:= false;
>         end
>       else
>         append(datatout);
> {$ifdef fast}
>     end;
> {$endif}
> end; {open}

-----> snip

Franz Glaser

Re:Multiple file open/close anomaly?


In article <36685A89.70BC2...@eunet.at>,
  "Ing. Franz Glaser" <meg-gla...@eunet.at> wrote:

Quote
> Robert AH Prins wrote:

> > Take this snippet:

> > const
> >   sfile: integer = -1;

> > procedure open(_file: integer);
> > begin
> > {$ifdef fast}
> >   if _file <> sfile then
> >     begin;
> >       sfile:= _file;
> > {$endif}
> >       close(datatout);
> >       assign(datatout, files[_file]);
> >       settextbuf(datatout, databuf^, 8192);

> Are you SURE that this SetTextBuf is done also in your
> first rewrite (which is not shown here obviously)?
> Cannot find any other reason for your experience :-)
> If not, how often do you skip the code here...

Yes, it is!

As for the calling of this procedure, the program where I discovered the
anomaly calls open 64 times, s/39e/s/e/s/e/s/18e/s (s = summary, e =
individual event), in other words, with "fast" defined it skips the code 55
times! BTW, first group of 39 'e's range in size from about 10k to 18k, the
second from about 12k to 24k.

Quote
> >       if __first_event then
> >         begin
> >           rewrite(datatout);
> >           __first_event:= false;
> >         end
> >       else
> >         append(datatout);
> > {$ifdef fast}
> >     end;
> > {$endif}
> > end; {open}

> -----> snip

Robert
--
Robert AH Prins
prin...@williscorroon.com

-----------== Posted via Deja News, The Discussion Network ==----------
http://www.dejanews.com/       Search, Read, Discuss, or Start Your Own    

Re:Multiple file open/close anomaly?


Quote
Robert AH Prins wrote:

> Yes, it is!

....
This was a doubtful idea only. Let me make a more professional
suggestion:

Open/create as many files as you need, and send the appropriate
file to your procedures as a Var parameter or even berrer as a
pointer to the file. This makes it unnecessary to open and
close them so frequently.

Var ProtFiles : Array[1..xxx] of Text;
    PT     : ^Text;

Procedure AnyProc;
Begin
  Writeln(PT^,...);
End;

  PT^ := @ProtFiles[3];
  AnyProc;

In case the file handles exhausted, look for the article in the
TP-links site, FAQ-page, about >15 handles.

Regards, Franz Glaser

Re:Multiple file open/close anomaly?


In article <36699C17.8EE47...@eunet.at>,
  "Ing. Franz Glaser" <meg-gla...@eunet.at> wrote:

Quote
> Robert AH Prins wrote:

> > Yes, it is!
> ....
> This was a doubtful idea only. Let me make a more professional
> suggestion:

> Open/create as many files as you need, and send the appropriate
> file to your procedures as a Var parameter or even berrer as a
> pointer to the file. This makes it unnecessary to open and
> close them so frequently.

> Var ProtFiles : Array[1..xxx] of Text;
>     PT     : ^Text;

> Procedure AnyProc;
> Begin
>   Writeln(PT^,...);
> End;

>   PT^ := @ProtFiles[3];
>   AnyProc;

> In case the file handles exhausted, look for the article in the
> TP-links site, FAQ-page, about >15 handles.

Thanks Franz,

As useful as this may be, it is unuseful in his particular case, as I don't
have enough heap left over to allocate buffers for both files at the same
time, unless I give them only 4K.

Robert
--
Robert AH Prins
prin...@williscorroon.com

-----------== Posted via Deja News, The Discussion Network ==----------
http://www.dejanews.com/       Search, Read, Discuss, or Start Your Own    

Re:Multiple file open/close anomaly?


What do you want to do? I could not see what you wanted to achieve...
(Sorry. Maybe this is a silly question but perhaps I can help you if I know
the problem more detailled :-)) ).

Bye, Tobi

Re:Multiple file open/close anomaly?


Robert AH Prins [mailto:prin...@williscorroon.com] decided it was an
auspicious moment for:
Quote
>Take this snippet:

>const
>  sfile: integer = -1;

>procedure open(_file: integer);
>begin
>{$ifdef fast}
>  if _file <> sfile then
>    begin;
>      sfile:= _file;
>{$endif}
>      close(datatout);
>      assign(datatout, files[_file]);
>      settextbuf(datatout, databuf^, 8192);

>      if __first_event then
>        begin
>          rewrite(datatout);
>          __first_event:= false;
>        end
>      else
>        append(datatout);
>{$ifdef fast}
>    end;
>{$endif}
>end; {open}

>It's used a program in which many procedures can write their results to one
>of two files (a per event file and a summary file). Going through the
>program, I suddenly realized that I had never tracked calls to "open", and
>much to my surprise I discovered that on several occasions it's called with
>the same value for _file on several consecutive invovations. So I added the
>conditionals, and timed again, clean DOS, SMARTDRV with write caching, and
>running both the old and the new versions several times in alternating
>succession. Much to my surprise the old version, which performs the code
>dozens of times more than the the new version, is actually faster! (NB: The
>file is initially opened for summary data in the main program)

>Does anyone have an explanation?

I initially assumed that you were opening the file if actually needed.
However, re-reading your code with the {$ifdef} more carefully, I see
that the extra code is not actually doing anything that affects the
process so the code is probably a bit slower just due to the extra bits
you've added.

I am assuming the fast option is to prevent reopening the same file and
I have assumed that one of the files will already be open when this
procedure is called. If there is no open file then perhaps you need a
procedure to do the first open that can be separate from this one.

If you change the code to:

const
  sfile: integer = -1;

procedure open(_file: integer);
begin
  if _file <> sfile then
    begin
      sfile:= _file;            

{only close then assign and rewrite/append the file if different to old
one - otherwise you do not actually need to do anything so the if..then
is checking that you have a different file}

      close(datatout);
      assign(datatout, files[_file]);
      settextbuf(datatout, databuf^, 8192);

      if __first_event then
        begin
          rewrite(datatout);
          __first_event:= false;
        end
      else
        append(datatout);
    end;

end; {open}

--
Pedt

In Idle village (near Bradford), do they have an Idle Working Mens Club ?

Re:Multiple file open/close anomaly?


In article <iIck5MAYzqa2Y...@pedt.demon.co.uk>,
  Pedt Scragg <newsmas...@pedt.demon.co.uk> wrote:

Quote
> Robert AH Prins [mailto:prin...@williscorroon.com] decided it was an
> auspicious moment for:
> >Take this snippet:

> >const
> >  sfile: integer = -1;

> >procedure open(_file: integer);
> >begin
> >{$ifdef fast}
> >  if _file <> sfile then
> >    begin;
> >      sfile:= _file;
> >{$endif}
> >      close(datatout);
> >      assign(datatout, files[_file]);
> >      settextbuf(datatout, databuf^, 8192);

> >      if __first_event then
> >        begin
> >          rewrite(datatout);
> >          __first_event:= false;
> >        end
> >      else
> >        append(datatout);
> >{$ifdef fast}
> >    end;
> >{$endif}
> >end; {open}

> >It's used a program in which many procedures can write their results to one
> >of two files (a per event file and a summary file). Going through the
> >program, I suddenly realized that I had never tracked calls to "open", and
> >much to my surprise I discovered that on several occasions it's called with
> >the same value for _file on several consecutive invovations. So I added the
> >conditionals, and timed again, clean DOS, SMARTDRV with write caching, and
> >running both the old and the new versions several times in alternating
> >succession. Much to my surprise the old version, which performs the code
> >dozens of times more than the the new version, is actually faster! (NB: The
> >file is initially opened for summary data in the main program)

> >Does anyone have an explanation?

> I initially assumed that you were opening the file if actually needed.
> However, re-reading your code with the {$ifdef} more carefully, I see
> that the extra code is not actually doing anything that affects the
> process so the code is probably a bit slower just due to the extra bits
> you've added.

Yes, but don't you think (that's at least what I did!) that a simple if test
should be quite a lot faster than closing a file, and opening another one?

Quote
> I am assuming the fast option is to prevent reopening the same file and
> I have assumed that one of the files will already be open when this
> procedure is called.

Correct & correct

- Show quoted text -

Quote
> If there is no open file then perhaps you need a
> procedure to do the first open that can be separate from this one.

> If you change the code to:

> const
>   sfile: integer = -1;

> procedure open(_file: integer);
> begin
>   if _file <> sfile then
>     begin
>       sfile:= _file;

> {only close then assign and rewrite/append the file if different to old
> one - otherwise you do not actually need to do anything so the if..then
> is checking that you have a different file}

>       close(datatout);
>       assign(datatout, files[_file]);
>       settextbuf(datatout, databuf^, 8192);

>       if __first_event then
>         begin
>           rewrite(datatout);
>           __first_event:= false;
>         end
>       else
>         append(datatout);
>     end;
> end; {open}

I only used the conditional to allow easy switching between the two different
modes, the close-and-open-regardless and the
only-close-and-open-when-changed. Conditionals have the advantage that you
don't have to change your source every time - the code is in an include file,
and not a unit.

Robert
--
Robert AH Prins
prin...@williscorroon.com

-----------== Posted via Deja News, The Discussion Network ==----------
http://www.dejanews.com/       Search, Read, Discuss, or Start Your Own    

Re:Multiple file open/close anomaly?


Robert AH Prins [mailto:prin...@williscorroon.com] decided it was an
auspicious moment for:

<opening and closing file if needed>

Quote

>> I initially assumed that you were opening the file if actually needed.
>> However, re-reading your code with the {$ifdef} more carefully, I see
>> that the extra code is not actually doing anything that affects the
>> process so the code is probably a bit slower just due to the extra bits
>> you've added.

>Yes, but don't you think (that's at least what I did!) that a simple if test
>should be quite a lot faster than closing a file, and opening another one?

Should definitely be faster. The file opening/closing overhead is
significant if needed to be called a lot of times in the program.

Quote
>{$ifdef fast}
>  if _file <> sfile then
>    begin;
>      sfile:= _file;
>{$endif}
>      close(datatout);
>      assign(datatout, files[_file]);

The problem with your {$ifdef fast}/{$endif} pairs is that they are
adding the conditional test to the file number but you then run the
close/assign/open part as well.

Leaving the conditionals in but rearranging them and the code slightly
to use the opposite conditional check (I deleted the conditionals from
the previous post which in hindsight perhaps I should not have done)

const
  sfile: integer = -1;

procedure open(_file: integer);
begin

{$ifdef fast}
  if _file = sfile then exit else
    begin
{$endif}
      sfile:= _file;            
      close(datatout);
      assign(datatout, files[_file]);
      settextbuf(datatout, databuf^, 8192);

      if __first_event then
        begin
          rewrite(datatout);
          __first_event:= false;
        end
      else
        append(datatout);
{$ifdef fast}
    end;
{$endif}
end; {open}

Quote
> Conditionals have the advantage that you
>don't have to change your source every time - the code is in an include file,
>and not a unit.

Yes, and it is a great advantage. I keep source handy for V5, V5.5, V6
and V7 and find it invaluable not only for version changes but for
switching on and off things like stack and range checks. IMHO, the usage
of conditionals is very underrated.

--
Pedt

In principle,   is there uncertainty that
Heisenberg was working his best in chaos?

Re:Multiple file open/close anomaly?


In article <J5r+FNAGY7a2Y...@pedt.demon.co.uk>,
  Pedt Scragg <newsmas...@pedt.demon.co.uk> wrote:

Quote
> Robert AH Prins [mailto:prin...@williscorroon.com] decided it was an
> auspicious moment for:

> <opening and closing file if needed>

> >> I initially assumed that you were opening the file if actually needed.
> >> However, re-reading your code with the {$ifdef} more carefully, I see
> >> that the extra code is not actually doing anything that affects the
> >> process so the code is probably a bit slower just due to the extra bits
> >> you've added.

> >Yes, but don't you think (that's at least what I did!) that a simple if test
> >should be quite a lot faster than closing a file, and opening another one?

> Should definitely be faster. The file opening/closing overhead is
> significant if needed to be called a lot of times in the program.

> >{$ifdef fast}
> >  if _file <> sfile then
> >    begin;
> >      sfile:= _file;
> >{$endif}
> >      close(datatout);
> >      assign(datatout, files[_file]);

> The problem with your {$ifdef fast}/{$endif} pairs is that they are
> adding the conditional test to the file number but you then run the
> close/assign/open part as well.

No, I don't, as can easily be demonstrated by putting a
"writeln('Close/Open');" in the code between begin & end.

- Show quoted text -

Quote
> Leaving the conditionals in but rearranging them and the code slightly
> to use the opposite conditional check (I deleted the conditionals from
> the previous post which in hindsight perhaps I should not have done)

> const
>   sfile: integer = -1;

> procedure open(_file: integer);
> begin

> {$ifdef fast}
>   if _file = sfile then exit else
>     begin
> {$endif}
>       sfile:= _file;
>       close(datatout);
>       assign(datatout, files[_file]);
>       settextbuf(datatout, databuf^, 8192);

>       if __first_event then
>         begin
>           rewrite(datatout);
>           __first_event:= false;
>         end
>       else
>         append(datatout);
> {$ifdef fast}
>     end;
> {$endif}
> end; {open}

Your code is doing exactly what my code is doing, be it that we use opposite
test to skip the close/open section...

But if you look at the code generated for the if-statement in the two cases,
you'll see that my code is a trifle shorter:

My code:

cs:05AE 55             push   bp
cs:05AF 89E5           mov    bp,sp
cs:05B1 8B4604         mov    ax,[bp+04]
cs:05B4 3B060A2A       cmp    ax,[2A0A]
cs:05B8 7503           jne    05BD
cs:05BA E98000         jmp    063D
cs:05BD 8B4604         mov    ax,[bp+04]
cs:05C0 A30A2A         mov    [2A0A],ax

Your code:

cs:05AE 55             push   bp
cs:05AF 89E5           mov    bp,sp
cs:05B1 8B4604         mov    ax,[bp+04]
cs:05B4 3B060A2A       cmp    ax,[2A0A]
cs:05B8 7506           jne    05C0
cs:05BA E98300         jmp    0640 -+---> Bit silly...
cs:05BD E98000         jmp    0640 -+
cs:05C0 8B4604         mov    ax,[bp+04]
cs:05C3 A30A2A         mov    [2A0A],ax

Quote
> > Conditionals have the advantage that you
> >don't have to change your source every time - the code is in an include file,
> >and not a unit.

> Yes, and it is a great advantage. I keep source handy for V5, V5.5, V6
> and V7 and find it invaluable not only for version changes but for
> switching on and off things like stack and range checks. IMHO, the usage
> of conditionals is very underrated.

Robert
--
Robert AH Prins
prin...@williscorroon.com

-----------== Posted via Deja News, The Discussion Network ==----------
http://www.dejanews.com/       Search, Read, Discuss, or Start Your Own    

Other Threads