Board index » delphi » Maximum size of word list ??

Maximum size of word list ??

I have written a program to sort through all the words in a text file, and
on certain conditions write words from that list to another text file.

I used Turbo Pascal to do this, but have also pasted the code into a Delphi
project.

The programs runs properly in both dos and windows with my orig{*word*7}word
list of 14390 words. however I have now got a wordlist
of 42696 words, and the program can't handle it at all.

Could someone please tell me what the maximum size of the text file is,
for this program to work. Or even better, how can I make the program work
with the larger text file. The size in bytes is 374739.

Thanks

Loren

wbr...@xtra.co.nz

 

Re:Maximum size of word list ??


Quote
Loren wrote:

> I have written a program to sort through all the words in a text file, and
> on certain conditions write words from that list to another text file.
> Could someone please tell me what the maximum size of the text file is,
> for this program to work. Or even better, how can I make the program work
> with the larger text file. The size in bytes is 374739.

     There is (in principle) no maximum size of a text file -- you are
only limited by your disk (or tape) size.  There IS, however, a maximum
amount of stuff you can store in memory at one time.  If your program
tries to read the entire file into memory, it will eventually (probably)
{*word*88}.

     There are several ways to get around this.  One is to just keep the
word list in memory.  Read the text file in smaller units (for example
a line or a word at a time) and add to the word list as you read.
Again, you will eventually fill up memory (depending on how "wordy" you
are), but you are now limited by the number of unique words, not by the
total number of words.  If you still run out of room, you can create
smaller word lists, such as "All words starting with A", and process
your text list multiple times.

Bob Schor
Pascal Enthusiast

Re:Maximum size of word list ??


On 26 May 1997 06:48:38 GMT, "Loren" <wbr...@xtra.co.nz> wrote:

Quote
>Could someone please tell me what the maximum size of the text file is,
>for this program to work. Or even better, how can I make the program work
>with the larger text file. The size in bytes is 374739.

You have reached some limit in one of the data structures in your
program.  You will have to redesign the data structure and/or
algorithm you use to handle the increased amount of data.

Other Threads