Board index » delphi » tStrings Memory Management

tStrings Memory Management

I need to display messages to the user, in such a way that the user
can scroll up and see old messages. I am using a class derived from
tCustomMemo; the lines are stored in a tStrings instance. In some
newsgroup, probably here, I read that in Win9x these can only display
a limited amount of data, because they have access to a limited amount
of memory.

What is the limitation? Is it the number of strings that is limited,
or the total of all the characters, or some combination?

The help for tStrings.Add doesn't say what will happen if there is
insufficient memory, but by experiment I have found that no exception
is raised, the string is simply not added. So when this happens, I
delete the oldest message, which must be mmoStatus.Lines[0].

  Repeat
    nBefore := mmoStatus.Lines.Count;
    mmoStatus.Lines.Add(s);
    nAfter := mmoStatus.Lines.Count;
    If nAfter <= nBefore then
    Begin
      mmoStatus.Lines.Delete(0);
    end;
  until (nAfter>nBefore) or (mmoStatus.Lines.Count=0);

I wanted to be sure that this was working properly, so I added a
couple of counter variables:

  Repeat
    nBefore := mmoStatus.Lines.Count;
    mmoStatus.Lines.Add(s);
    nAfter := mmoStatus.Lines.Count;
    If nAfter <= nBefore then
    Begin
      Inc(nDeleted, length(mmoStatus.Lines[0]));
      mmoStatus.Lines.Delete(0);
    end;
    If nDeleted>length(s) then
      inc(i);
  until (nAfter>nBefore) or (i>10) or (mmoStatus.Lines.Count=0);

and fired a lot of random strings at it.

I put a breakpoint on the "inc(i)", because I thought that should
never get hit - more chars have been deleted than I want to add, and
at least one string has been deleted.

Why would the program reach the inc(i) line?

A colleague suggested adding Application.ProcessMessages after the
.Add(s) line. Would memory released by tStrings.Delete(0) only get
freed when windows messages are processed? That seems strange to me.

I feel I'm missing something obvious. And is there a better way to add
strings?

FP

 

Re:tStrings Memory Management


1. Windows 9x has a limit of 32Kb data in TEdit/TMemo. Delphi has a
limitation on the number of lines (134217727), but you will probably never
reach this. If you want to display more lines you should use a TRichEdit,
set PlainText to True and set MaxLength to MaxInt - 2 (= 2147483645).
2. Assume the first line is 20 characters long and you add a string of 10
characters. In that case 1 line of 20 characters is deleted, but 1 line of
only 10 characters is added, so nDeleted is larger than Length(s) !
3. I use the following code to add text (not only full lines) to a memo:

procedure MemoAddText(const Memo: TMemo; const S: String);
const
  MaxTextLimit = 32767;
var
  Index, i, j, k: Integer;
begin
  Memo.HandleNeeded;
  Index := Memo.Perform(WM_GETTEXTLENGTH, 0, 0);
  i := Memo.Perform(EM_GETLIMITTEXT, 0, 0);
  if (i <= 0) or (i > MaxTextLimit) then
    i := MaxTextLimit;
  i := i - Memo.Perform(WM_GETTEXTLENGTH, 0, 0);
  if i >= Length(S) then
  begin
    Memo.Perform(EM_SETSEL, Index, Index);
    Memo.Perform(EM_REPLACESEL, WPARAM(False), LPARAM(PChar(S)));
  end
  else
  begin
    j := i + Index;
    if j > Length(S) then
    begin
      j := Memo.Perform(EM_LINEFROMCHAR, Length(S) - i, 0);
      k := Memo.Perform(EM_LINEINDEX, j + 1, 0);
      if k < 0 then
      begin
        k := Memo.Perform(EM_LINEINDEX, j, 0);
        if k < 0 then
          k := Length(S) - i
        else
          k := k + Memo.Perform(EM_LINELENGTH, j, 0);
      end;
      Memo.Perform(WM_SETREDRAW, WPARAM(False), 0);
      Memo.Perform(EM_SETSEL, 0, k);
      Memo.Perform(EM_REPLACESEL, WPARAM(False), LPARAM(PChar('')));
      Memo.Perform(WM_SETREDRAW, WPARAM(True), 0);
      Memo.Perform(EM_SETSEL, Index - k, Index - k);
      Memo.Perform(EM_REPLACESEL, WPARAM(False), LPARAM(PChar(S)));
    end
    else
    begin
      k := Length(S) - j;
      while (k < Length(S)) and (S[k] <> #13) do
        Inc(k);
      if k < Length(S) then
        Inc(k);
      if (k < Length(S)) and (S[k] = #10) then
        Inc(k);
      Memo.Perform(EM_SETSEL, 0, Index);
      Memo.Perform(EM_REPLACESEL, WPARAM(False), LPARAM(PChar(S)[k]));
    end;
  end;
end;

Quote
"Frank Peelo" <fpe...@eircom.net> wrote in message

news:9n2dgd$6ll$1@kermit.esat.net...
Quote
> I need to display messages to the user, in such a way that the user
> can scroll up and see old messages. I am using a class derived from
> tCustomMemo; the lines are stored in a tStrings instance. In some
> newsgroup, probably here, I read that in Win9x these can only display
> a limited amount of data, because they have access to a limited amount
> of memory.

> What is the limitation? Is it the number of strings that is limited,
> or the total of all the characters, or some combination?

> The help for tStrings.Add doesn't say what will happen if there is
> insufficient memory, but by experiment I have found that no exception
> is raised, the string is simply not added. So when this happens, I
> delete the oldest message, which must be mmoStatus.Lines[0].

>   Repeat
>     nBefore := mmoStatus.Lines.Count;
>     mmoStatus.Lines.Add(s);
>     nAfter := mmoStatus.Lines.Count;
>     If nAfter <= nBefore then
>     Begin
>       mmoStatus.Lines.Delete(0);
>     end;
>   until (nAfter>nBefore) or (mmoStatus.Lines.Count=0);

> I wanted to be sure that this was working properly, so I added a
> couple of counter variables:

>   Repeat
>     nBefore := mmoStatus.Lines.Count;
>     mmoStatus.Lines.Add(s);
>     nAfter := mmoStatus.Lines.Count;
>     If nAfter <= nBefore then
>     Begin
>       Inc(nDeleted, length(mmoStatus.Lines[0]));
>       mmoStatus.Lines.Delete(0);
>     end;
>     If nDeleted>length(s) then
>       inc(i);
>   until (nAfter>nBefore) or (i>10) or (mmoStatus.Lines.Count=0);

> and fired a lot of random strings at it.

> I put a breakpoint on the "inc(i)", because I thought that should
> never get hit - more chars have been deleted than I want to add, and
> at least one string has been deleted.

> Why would the program reach the inc(i) line?

> A colleague suggested adding Application.ProcessMessages after the
> .Add(s) line. Would memory released by tStrings.Delete(0) only get
> freed when windows messages are processed? That seems strange to me.

> I feel I'm missing something obvious. And is there a better way to add
> strings?

> FP

Re:tStrings Memory Management


Quote
Frank Peelo <fpe...@eircom.net> wrote in message

news:9n2dgd$6ll$1@kermit.esat.net...

Quote
> I need to display messages to the user, in such a way that the user
> can scroll up and see old messages. I am using a class derived from
> tCustomMemo; the lines are stored in a tStrings instance. In some
> newsgroup, probably here, I read that in Win9x these can only
display
> a limited amount of data, because they have access to a limited
amount
> of memory.

> What is the limitation? Is it the number of strings that is limited,
> or the total of all the characters, or some combination?

Hi Frank
Strange and wonderful things happen if you try to stuff too much into
a tMemo <gr>

I had a play around with your code today (using Delphi 1) and managed
to add over 60 00 btytes to a tMemo but odd and upredictable things
happened.

Quote
> Why would the program reach the inc(i) line?

   I don't know but do know that various things including integer
overflow can happen inside a tMemo without raising exceptions. Today I
found that during most runs of your code, I raised exceptions but very
occasionally I did drop into the "inc(i) line". I could not reproduce
the behaviour reliably and so did not track down the problem. I
suspect some sort of memory overwrite.

My advice when using tMemo is either to
1) make *sure* that it never exceeds 32Kb (actually $7FFF) or
2) avoid most of the tMemo methods and properties and use API calls.
(See my reply to M.H. Avegaart).

For your purposes, I think that 32Kb should be plenty.

Try this:
const MemoLimit = $7EFE;  {$7EFE + max length of ShortString + 2 chars
for CRLF = $7FFF}

procedure StuffMemo;
var S: shortstring;
begin
  S := 'some load of rubbish';
  Repeat
      Memo1.Lines.Add(S);
  until Memo1.GetTextLen >= MemoLimit;
end;

procedure TForm1.AddALine (const S: shortstring);
var Len: word;
begin
  Memo1.Lines.BeginUpdate;
  While (Memo1.GetTextLen + Length(S)  > MemoLimit) do
     Memo1.Lines.Delete(0);
  Memo1.Lines.Add(S);
  Memo1.Lines.EndUpdate;
  Len:= Memo1.GetTextLen;
  Memo1.Perform (EM_SetSel, 0, MakeLong (Len, Len));
end;

--

Henry Bartlett

HABit utilities

http://www.hotkey.net.au/~hambar/habit/

email: ham...@Spamlock.microtech.com.au
Please remove the "Spamlock." from my address when replying.

Re:tStrings Memory Management


Quote
M.H. Avegaart <avega...@NOSPAMmccomm.nl> wrote in message

news:9n2ehe$15sp$1@scavenger.euro.net...

Quote
> 1. Windows 9x has a limit of 32Kb data in TEdit/TMemo.

Hi

I am sorry, but as far as I can tell, this is simply not correct.

I have several apps with tMemos with a maximum of 50Kb. They were
compiled in Delphi 1 but run quite well in Wn98.

As WINDOWS 98 Notebook (and also WINDOWS 95 NOTEBOOK - not the same)
have maximum capacities of around 52Kb, I suspect that the maximum
capacity of tMemo in Win 9X may be even greater than the 50Kb I am
getting.

Quote
> procedure MemoAddText(const Memo: TMemo; const S: String);
> const
>   MaxTextLimit = 32767;

Try
MaxTextLimit = $CA6C;

I have a feeling that the 32K limit has something to do with the VCL
code as I find that when working with memos above 32K, I cannot use
tMemo properties or methods but have to resort to API calls instead..

See also my reply to Frank Peelo.

If you doubt that the 32K "limit" can be exceeded, check out HABEdit,
HABPad or HABExam from my site (see sig)

If anyone would like a copy of the code library I use (not a
component), they can contact me direct (but watch out for the
spam-blocker).

It is a bit too large to post on a NG.

--

Henry Bartlett

HABit utilities

http://www.hotkey.net.au/~hambar/habit/

email: ham...@Spamlock.microtech.com.au

Please remove the "Spamlock." from my address when replying.

Quote
M.H. Avegaart <avega...@NOSPAMmccomm.nl> wrote in message

news:9n2ehe$15sp$1@scavenger.euro.net...

Re:tStrings Memory Management


Well, according to MS documentation the limit is 64Kb, but either some
EM_XXX messages or the VCL restrict correct usage to 32Kb.

Quote
"Henry Bartlett" <ham...@SpamLock.microtech.com.au> wrote in message

news:3b9643c4@news.iprimus.com.au...
Quote
> M.H. Avegaart <avega...@NOSPAMmccomm.nl> wrote in message
> news:9n2ehe$15sp$1@scavenger.euro.net...
> > 1. Windows 9x has a limit of 32Kb data in TEdit/TMemo.

> Hi

> I am sorry, but as far as I can tell, this is simply not correct.

> I have several apps with tMemos with a maximum of 50Kb. They were
> compiled in Delphi 1 but run quite well in Wn98.

> As WINDOWS 98 Notebook (and also WINDOWS 95 NOTEBOOK - not the same)
> have maximum capacities of around 52Kb, I suspect that the maximum
> capacity of tMemo in Win 9X may be even greater than the 50Kb I am
> getting.

[...]

Re:tStrings Memory Management


Quote
"M.H. Avegaart" <avega...@NOSPAMmccomm.nl> wrote in message

news:9n2ehe$15sp$1@scavenger.euro.net...

Quote
> 1. Windows 9x has a limit of 32Kb data in TEdit/TMemo. Delphi has a
> limitation on the number of lines (134217727), but you will probably
never
> reach this. If you want to display more lines you should use a
TRichEdit,
> set PlainText to True and set MaxLength to MaxInt - 2 (=
2147483645).
> 2. Assume the first line is 20 characters long and you add a string
of 10
> characters. In that case 1 line of 20 characters is deleted, but 1
line of
> only 10 characters is added, so nDeleted is larger than Length(s) !

Akk. Right. Sorry. I'm seeing what I thought I wanted to write,
instead of what's actually there - the "If nDeleted..." clause should
only have happened after one or more lines have been deleted, more
chars have been deleted than are to be added, and then the memo has
refused the string.

I moved it so the code reads

  Repeat
    nBefore := mmoStatus.Lines.Count;
    mmoStatus.Lines.Add(s);
    application.ProcessMessages;
    nAfter := mmoStatus.Lines.Count;
    If nAfter <= nBefore then
    Begin
      If nDeleted>length(s) then
        inc(i);

      Inc(nDeleted, length(mmoStatus.Lines[0]));
      mmoStatus.Lines.Delete(0);
    end;

  until (nAfter>nBefore) or (i>10) or (mmoStatus.Lines.Count=0);

(The application.processmessages is just snake-oil, I think; someone
suggested that maybe it might help. It doesn't.)

Then I put a breakpoint on the inc(i), and ran again. The breakpoint
is hit when trying to add a 91-char string, 120 chars have been
deleted, mmoStatus.Lines.Count is 528 and mmoStatus.GetTextLen is
26815.

It's not a big problem, because after another deletion or two the
string gets accepted; I'd just like to know whether it means that I'm
using the tMemo wrong or that tMemo is broken as Henry suggested.

Quote
> 3. I use the following code to add text (not only full lines) to a

memo:
...

Thanks, I have stored a copy of that.

But you are asking the tMemo to handle windows messages, - does this
mean that the defined methods for adding and deleting lines in a tMemo
are simply broken?

Thanks to Henry & M.H. for the help.

FP

Re:tStrings Memory Management


No, TMemo uses a Windows EDIT subclass. There is no difference in using the
VCL functions or Windows messages.

Quote
"Frank Peelo" <fpe...@eircom.net> wrote in message

news:9n7hmm$6ad$1@kermit.esat.net...
Quote
> "M.H. Avegaart" <avega...@NOSPAMmccomm.nl> wrote in message
> news:9n2ehe$15sp$1@scavenger.euro.net...
[...]
> But you are asking the tMemo to handle windows messages, - does this
> mean that the defined methods for adding and deleting lines in a tMemo
> are simply broken?

> Thanks to Henry & M.H. for the help.

> FP

Re:tStrings Memory Management


Quote
----- Original Message -----
From: M.H. Avegaart <avega...@NOSPAMmccomm.nl>
Newsgroups: comp.lang.pascal.delphi.misc

Hi M.H.

> Well, according to MS documentation the limit is 64Kb, but either
some
> EM_XXX messages or the VCL restrict correct usage to 32Kb.

I believe that it is the VCL that's at fault since I can use
tMemo.Perform to work around all the problems (up to 50Kb).

Henry Bartlett

HABit utilities

http://www.hotkey.net.au/~hambar/habit/

email: ha...@bigmailbox.net

Re:tStrings Memory Management


Quote
M.H. Avegaart <avega...@NOSPAMmccomm.nl> wrote in message

news:9n7j36$2ncg$1@scavenger.euro.net...

Hi M.H.

Quote
> > But you are asking the tMemo to handle windows messages, - does
this
> > mean that the defined methods for adding and deleting lines in a
tMemo
> > are simply broken?
> No, TMemo uses a Windows EDIT subclass. There is no difference in
using the
> VCL functions or Windows messages.

Sorry but there I must disagree with you.

IMHO the VCL methods and properties for tMemo are, if not broken, at
least badly bent <gr>

Please note that my experience is limited to Delphi version 1 but
everything I read leads me to understand that the only change to tMemo
in later versions is the tStrings change from ShortString to
longstring and the increase in the maximum number of lines due to the
increase in the size of an integer.

If the tMemo 32Kb limit for properties and methods is still there in
later versions then there must be something in the VCL that
artificially limits the maximum size.

By using tMemo.Perform, you **can** overcome this artificial 32Kb
limit. Most of these techniques are well known but perhaps not so well
documented in the Help file.

I have proven that you can exceed the 32Kb limit by doing so; my
code is available to anyone who wants it and utilities (HABPad,
HABEdit and HABExam) using these techniques are available free from my
web site.

(My Memos unit is a bit too large to post on a NG so email me if
you want a copy - but beware the spam blocker)

Henry Bartlett

HABit utilities

http://www.hotkey.net.au/~hambar/habit/

email: ham...@Spamlock.microtech.com.au

Please remove the "Spamlock." from my address when replying.

Other Threads