Hi all,

I have several large raw binary files (>20MB).  I need to do some
subsampling from those files, however, it appears that Delphi is running
right by the EOF long before it ever should.

I know that any given scan line in these datafiles are less than 10000,
so I have created a variable:

bytebuffer : array[0..10000] of Byte;

and I open the file.  When I do a FileSize, it returns only 175000, not
21,000,000!  My first trial is to subsample at a rate of 12.  Thus I read
in the first data line with:

BlockRead(fin, bytebuffer, linelength, result);
where linelength for the test file is 7200.  Result returns 31? :-(.

Then the program merrily skips through the array pulling every 12th
element -- it does this first line correctly.

Then I do a Seek with my new file position being my current position +
11*linelength, which should drop me at the proper place, which it almost
does (79220 bytes in, not 79189 bytes...) before it reads in the next
line of data.

It does this a couple times until the file position is greater than the
FileSize.  Sadly, the data it pulls within the first 175000 bytes is
valid, but afterwards, it is like it did go beyond the EOF. Can anyone
give some suggestions to dealing with this?

I know my algo works as it is a straight port from some solid C code.

--
Cheers,
------------------------------------------------------------------------------
Christopher M. Keane                                    ke...@jei.umd.edu
Research Associate                                      ke...@earthsun.umd.edu
Joint Education Initiative
3433 A. V. Williams
University of Maryland at College Park
/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
| Flames > /dev/nul                                                          |
\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/

In article <44s3jr$...@cville-srv.wam.umd.edu>,
   ke...@glue.umd.edu (Christopher Keane) wrote:
]-
]-I have several large raw binary files (>20MB).  I need to do some
]-subsampling from those files, however, it appears that Delphi is running
]-right by the EOF long before it ever should.
]-
]-I know that any given scan line in these datafiles are less than 10000,
]-so I have created a variable:
]-
]-bytebuffer : array[0..10000] of Byte;
]-
]-and I open the file.  When I do a FileSize, it returns only 175000, not
]-21,000,000!  My first trial is to subsample at a rate of 12.  Thus I read
]-in the first data line with:
]-
]-BlockRead(fin, bytebuffer, linelength, result);
]-where linelength for the test file is 7200.  Result returns 31? :-(.
]-

sounds like a problem with "reset".  reset takes an optional 2nd
parameter that specifies the record size of an untyped file.  try
                        reset(fin, 1);
that should clear up your problems.

Mark Vaughan

Hi Christopher,

<<snip>>

1) You haven't said how you open the file. Opening a binary file with reset creates a
record buffer, by default 128 bytes. You need to use Reset(fin,nnn) where nnn is your
recordsize.

2)Filesize will return the size of the file in records.

3)Using Seek, the file starts at record 0 so use this in any loops.

If you need any further help, send me some code.

HTH

Ray Cramer
 R...@Pol.ac.uk

...... prolonged squeals of frustrations deleted..........

My guess is,  you opened the file without setting the optional record length
to 1.  Below see excerpt from the Help: note the distinction between bytes and
records.
  The default record length is 128 for some forgotten historical reason, and
has remained so for compatibility. In practice, always set it to 1.
( If you have fixed length records you should not use an untyped file anyway)

From the Help....
The BlockRead procedure reads one or more records from an open file into
a variable.
F is an untyped file variable, Buf is any variable, Count is an expression of
type Word, and Result is an optional variable of type Word.
BlockRead reads Count or fewer records from the file F into memory,
starting at the first byte occupied by Buf. The actual number of complete
records read (less than or equal to Count) is returned in Result.
The entire transferred block occupies at most Count * RecSize bytes.
RecSize is the record size specified when the file was opened
 (or 128 if the
         ^^^                                                                  
                   record size was not specified). An error occurs if Count *
RecSize is greater than 65,535 (64K). You can handle this error using
exceptions.If the entire block was transferred, Result is equal to Count.