Board index » delphi » Most efficient sort text file routine, sort algorithms

Most efficient sort text file routine, sort algorithms

Quote
Matthew wrote:

> I have a text file. Each line begins with a date like "19970515".
> There might be up to a few hundred dates in this text file, possibly
> in random order, but I cannot be sure
> someone might not put thousands of lines in there.

> I am avoiding a database with an index, because
> that way users cannot edit the text file with Notepad.exe or other
> text editor.

> File example:

> 19970514 This is a record
> 19970521 This is more info
> 19961212 This is some data
> 19980202 This is another record
> 19950608 This is yet more info

> I am looking for a clever way to sort this file in the most efficient
> manner. Each line is less than 255 characters.

> So I am thinking:

>  TStringList.loadFromFile;
>  TStringList.sort;
>  TStringList.saveToFile;

> Using Delphi 16-bit, I wonder, what are the limits to the number of
> strings?  If the limit is 64K, and 200 bytes per
> line, then 5 records per K, 60*5 +4*5 = 300+20= 320 records
> max. That would not be enough.

        I _think_ the limit is 16,000 strings but I could well be wrong,
D1 seems like a long time ago. Why not try it?

Quote
> I seem to remember some old-fashioned sort algorithm. Two files would
> run against each other with some sort of buffer where records are
> compared, but don't recall how this is done without scanning and
> re-reading the entire
> file countless numbers of times.

        There are lots of old-fashioned sort algorithms around -
see Sedgwick "Algorithms" (or lots of other books).

--
David Ullrich

?his ?s ?avid ?llrich's ?ig ?ile
(Someone undeleted it for me...)

 

Re:Most efficient sort text file routine, sort algorithms


Quote
> I have a text file. Each line begins with a date like "19970515".
> There might be up to a few hundred dates in this text file, possibly
> in random order, but I cannot be sure
> someone might not put thousands of lines in there.
>  TStringList.loadFromFile;
>  TStringList.sort;
>  TStringList.saveToFile;

That will work fine, but ONLY if the length of each line is the same.  If
not, E-Mail me, and I'll show you how to do it.

Quote
> Using Delphi 16-bit, I wonder, what are the limits to the number of
> strings?  If the limit is 64K, and 200 bytes per
> line, then 5 records per K, 60*5 +4*5 = 300+20= 320 records
> max. That would not be enough.

The limit is 64000 strings of up to 255 characters.  Thus, it should be
plenty.

--
--------------------------
Eric Lawrence
Delta Programming Group
Delta...@juno.com

Re:Most efficient sort text file routine, sort algorithms


On 16 May 1997 22:20:35 GMT, "Eric Lawrence" <delta...@keynetcorp.net>
wrote:

Quote
>> Using Delphi 16-bit, I wonder, what are the limits to the number of
>> strings?  If the limit is 64K, and 200 bytes per
>> line, then 5 records per K, 60*5 +4*5 = 300+20= 320 records
>> max. That would not be enough.

>The limit is 64000 strings of up to 255 characters.  Thus, it should be
>plenty.

No. In Delphi 1, a TStringList can contain up to 16380 strings. The
limit is that the array of pointers must fit into a single segment. At
4 bytes per segment, that leaves 16K items, minus a few for Windows
overhead in each segment.

http://www.tempest-sw.com/freeware has a huge string list, which can
hold any number of strings. You can try using that to sort your
strings. It should work fine in most cases.
--
Ray Lischner            
Author of Secrets of Delphi 2 (http://www.tempest-sw.com/secrets/)

Re:Most efficient sort text file routine, sort algorithms


Quote
Com...@lottery.powernet.co.uk  (Matthew) wrote:
>Using Delphi 16-bit, I wonder, what are the limits to the number of
>strings?  If the limit is 64K, and 200 bytes per
>line, then 5 records per K, 60*5 +4*5 = 300+20= 320 records
>max. That would not be enough.

TStrings has a limit of 64K for the list itself.  However, it is a
list of pointers to strings, not a list of the actual strings.  Thus,
it has a limit of approximately 16,000 strings.  (It's not a full
16,368 because the list's properties and pointers to methods also have
to fit into the 64K segment.)  Each string can be up to 255
characters, though you can also have an object associated with each
string and each of these objects could be/contain a TStrings...

Other Threads