Board index » delphi » Large Text File

Large Text File

I have a large text file to import every month from my harddrive to the
Interbase Database on our Winnt 3.51 Server the file is about 6,000
records. The file is not delmitted it is fixed but each record takes up
three lines. I have been using a readln to a temp var. Then checking the
forst char to see which fields are on that line the using
Copy(String,StartPos,NumberToCopy) to get in my fields. This takes
forever. Anyone know a better way.
                        Thanks in Advance
                                Jon

 

Re:Large Text File


Quote
J...@MAIL.STATE.KY.US wrote:

> I have a large text file to import every month from my harddrive to the
> Interbase Database on our Winnt 3.51 Server the file is about 6,000
> records. The file is not delmitted it is fixed but each record takes up
> three lines. I have been using a readln to a temp var. Then checking the
> forst char to see which fields are on that line the using
> Copy(String,StartPos,NumberToCopy) to get in my fields. This takes
> forever. Anyone know a better way.
>                         Thanks in Advance
>                                 Jon

I have two suggestions you might want to consider.  The examples are
from Turbo Pascal but they should still work in Delphi (with some
changes like TextFile instead of Text).

(1) Add an 8K buffer (or empirically determine the optimal size for
your application).  Dynamically allocate buffer space from heap if
stack space is tight.

Example:

  ...
  VAR
    FromBuffer  :  ARRAY[1..8192] OF CHAR;
    FromFile    :  TEXT;
    FromName    :  STRING;
    ..
    ToBuffer    :  ARRAY[1..8192] OF CHAR;

    ASSIGN (FromFile,FromName);
    SetTextBuf (FromFile,FromBuffer);
    {$I-} RESET (FromFile); {$I+}
    error := IOResult

(2) Use BlockRead instead of READLN and read relatively large
blocks.  You 'll have to find the "logical records" within the block,
but this can often result in much faster I/O

Hope this helps.  
--
Earl F. Glynn          EarlGl...@WorldNet.att.net
EFG Software              913/859-9557  Voice/Fax
   Scientific/Engineering/Medical Applications
             Overland Park, KS  USA

Re:Large Text File


Quote
J...@MAIL.STATE.KY.US wrote:
>I have a large text file to import every month from my harddrive to the
>Interbase Database on our Winnt 3.51 Server the file is about 6,000
>records. The file is not delmitted it is fixed but each record takes up
>three lines. I have been using a readln to a temp var. Then checking the
>forst char to see which fields are on that line the using
>Copy(String,StartPos,NumberToCopy) to get in my fields. This takes
>forever. Anyone know a better way.

Check out

Reset(f, 1)

and the

BlockRead/Blockwrite procedures.

Just read in the fixed length record including the CR/LF's in the
size.

Now, if these records are small, you might want to download my
filebuff.zip from compuserve.  It handles some better buffering for
quick reads of small records.  Source is included.  I think it is at
the Delphi SuperPage also.

If you want to try the file buffering yourself, you might want to take
a look at memory-mapped files in 95/NT and/or the CreateFile() API
call.

Jay Cole
============
NitroSort, Mainframe file sort performance at PC prices, see
www.nitrosort.com.

Other Threads