Most efficient sort text file routine, sort algorithms

Com...@lottery.powernet.co.uk (Matthew)

Delphi Developer

Mon, 01 Nov 1999 03:00:00 GMT

Most efficient sort text file routine, sort algorithms

I have a text file. Each line begins with a date like "19970515".
There might be up to a few hundred dates in this text file, possibly
in random order, but I cannot be sure
someone might not put thousands of lines in there.

I am avoiding a database with an index, because
that way users cannot edit the text file with Notepad.exe or other
text editor.

File example:

19970514 This is a record
19970521 This is more info
19961212 This is some data
19980202 This is another record
19950608 This is yet more info

I am looking for a clever way to sort this file in the most efficient
manner. Each line is less than 255 characters.

So I am thinking:

TStringList.loadFromFile;
TStringList.sort;
TStringList.saveToFile;

Using Delphi 16-bit, I wonder, what are the limits to the number of
strings? If the limit is 64K, and 200 bytes per
line, then 5 records per K, 60*5 +4*5 = 300+20= 320 records
max. That would not be enough.

I seem to remember some old-fashioned sort algorithm. Two files would
run against each other with some sort of buffer where records are
compared, but don't recall how this is done without scanning and
re-reading the entire
file countless numbers of times.

Email me. I reply to every email.
Matthew

Sundial Servic

Delphi Developer

Tue, 02 Nov 1999 03:00:00 GMT

Re:Most efficient sort text file routine, sort algorithms

Quote

In article <337b21a...@news.power.net.uk> Com...@lottery.powernet.co.uk (Matthew) writes:
>I have a text file. Each line begins with a date like "19970515".
>There might be up to a few hundred dates in this text file, possibly
>in random order, but I cannot be sure
>someone might not put thousands of lines in there.
>I am avoiding a database with an index, because
>that way users cannot edit the text file with Notepad.exe or other
>text editor.
>File example:
>19970514 This is a record
>19970521 This is more info
>19961212 This is some data
>19980202 This is another record
>19950608 This is yet more info
>I am looking for a clever way to sort this file in the most efficient
>manner. Each line is less than 255 characters.
>So I am thinking:
> TStringList.loadFromFile;
> TStringList.sort;
> TStringList.saveToFile;
>Using Delphi 16-bit, I wonder, what are the limits to the number of
>strings? If the limit is 64K, and 200 bytes per
>line, then 5 records per K, 60*5 +4*5 = 300+20= 320 records
>max. That would not be enough.
>I seem to remember some old-fashioned sort algorithm. Two files would
>run against each other with some sort of buffer where records are
>compared, but don't recall how this is done without scanning and
>re-reading the entire
>file countless numbers of times.
>Email me. I reply to every email.
>Matthew

Umm... TurboPower Software sells a great sort engine as part of its SysTools
package. Generally speaking the best way to get a large-scale sort done is to
buy a package to do it. Seriously. I mean that very seriously!

Robert Kapla

Delphi Developer

Wed, 03 Nov 1999 03:00:00 GMT

Re:Most efficient sort text file routine, sort algorithms

Quote

Sundial Services wrote:

> In article <337b21a...@news.power.net.uk> Com...@lottery.powernet.co.uk (Matthew) writes:

> >I have a text file. Each line begins with a date like "19970515".
> >There might be up to a few hundred dates in this text file, possibly
> >in random order, but I cannot be sure
> >someone might not put thousands of lines in there.

> >I am avoiding a database with an index, because
> >that way users cannot edit the text file with Notepad.exe or other
> >text editor.

> >File example:

> >19970514 This is a record
> >19970521 This is more info
> >19961212 This is some data
> >19980202 This is another record
> >19950608 This is yet more info

> >I am looking for a clever way to sort this file in the most efficient
> >manner. Each line is less than 255 characters.

> >So I am thinking:

> > TStringList.loadFromFile;
> > TStringList.sort;
> > TStringList.saveToFile;

> >Using Delphi 16-bit, I wonder, what are the limits to the number of
> >strings? If the limit is 64K, and 200 bytes per
> >line, then 5 records per K, 60*5 +4*5 = 300+20= 320 records
> >max. That would not be enough.

> >I seem to remember some old-fashioned sort algorithm. Two files would
> >run against each other with some sort of buffer where records are
> >compared, but don't recall how this is done without scanning and
> >re-reading the entire
> >file countless numbers of times.

> >Email me. I reply to every email.
> >Matthew

> Umm... TurboPower Software sells a great sort engine as part of its SysTools
> package. Generally speaking the best way to get a large-scale sort done is to
> buy a package to do it. Seriously. I mean that very seriously!

Turbo Power used to give away their Pascal sort (MSortP), works great.
Yo should be able to port it to Delphi.
--

Robert

Return address changed to deter malevolent spammers.

Other Threads

1. Most efficient sort text file routine, sort algorithms

2. Efficient sorting algorithm?

3. Shell sort vs. Comb sort (was: sorting arrays)

4. Program to sort real values in a text file

5. Help: Sorting a text file

6. SORTING RECORDS-NON TEXT FILES

7. Sorting Text Files

8. Text File Sort function

9. Counting Comparisons in a Sorting Algorithm

10. Help! sorting algorithm