Board index » delphi » memory management for long strings

memory management for long strings

I have an app that needs to build one long string from a set of short
ones,i.e:

var
  strLong: String;
  i:       Integer;
begin
  strLong := '';
  for i := 0 to aObject.Items.Count-1
  do strLong := strLong+aObject.Items[i];
end;

When the string gets around 6K bytes the concatenations start to slow
down on my machine, around 12K, it really crawls, i.e. one concatenation
per second with 100% processor usage.

It seems like the memory manager is severely thrashing trying to expand
storage for the long string.

Incidently, I get the same bottleneck if I use a PChar and GetMem for
the full size before concatenating!Any help would be much appreciated!
Thanks
Peter

-----== Posted via Deja News, The Leader in Internet Discussion ==-----
http://www.dejanews.com/   Now offering spam-free web-based newsreading

 

Re:memory management for long strings


Hi Peter -

Quote
> When the string gets around 6K bytes the concatenations start to slow
> down on my machine, around 12K, it really crawls

Firstly, unless there's something you didn't tell us, or just something I'm
not aware of, I don't understand how this can possibly work with a normal
string var to begin with, since these are limited to 255 bytes in length.

Next, following the PChar route, instead of concatenating (using StrCat I
assume), you should just keep a pointer to the end of the big string, copy
the new string data at that point, then advance the end-of-string pointer
by that length again. If you need an example of this, I can work one up.

You could also emulate this behavior yourself using a big array of char,
and simply Move() in each substring to the next available point. Top it off
with a Null byte or a CR/LF pair, add a dash of nutmeg, stir and enjoy!

Cheers, Todd

Re:memory management for long strings


JRS:  In article <6ivdfd$d1...@nnrp1.dejanews.com> of Fri, 8 May 1998
16:56:13 in comp.lang.pascal.borland, pko...@pfksystems.com wrote:

Quote
>I have an app that needs to build one long string from a set of short
>ones,i.e:

>var
>  strLong: String;
>  i:       Integer;
>begin
>  strLong := '';
>  for i := 0 to aObject.Items.Count-1
>  do strLong := strLong+aObject.Items[i];
>end;

>When the string gets around 6K bytes the concatenations start to slow
>down on my machine, around 12K, it really crawls, i.e. one concatenation
>per second with 100% processor usage.

>It seems like the memory manager is severely thrashing trying to expand
>storage for the long string.

>Incidently, I get the same bottleneck if I use a PChar and GetMem for
>the full size before concatenating!Any help would be much appreciated!

Judging from experience with
        http://www.merlyn.demon.co.uk/programs/del_timr.pas ,
Quote
>  do strLong := strLong+aObject.Items[i];

will create a temporary string and copy it into strLong again.

You need to keep track of the end of StrLong (if it is PChar), copy
character-by-character the new chars to the tail of the old string, and
update the length (if it is String).  You can copy quickest (I hope)
with Move.  Del_timr does not do Concat (</g>), but that's a subset of
Inject.  Del_Time uses strings, not Pchars, but the principle applies.

How you get a string of 6 kBytes in comp.lang.pascal.borland, ... !!!

HTH.

--
John Stockton, Surrey, UK.    j...@merlyn.demon.co.uk    Turnpike v1.12    MIME.
  Web <URL: http://www.merlyn.demon.co.uk/> - FAQqish topics, acronyms & links.
  Correct 4-line sig separator is as above, a line comprising "-- " (SoRFC1036)
  Do not Mail News to me.    Before a reply, quote with ">" or "> " (SoRFC1036)

Other Threads