Board index » cppbuilder » Out of memory for TMemoryStream

Out of memory for TMemoryStream


2005-05-25 01:19:39 PM
cppbuilder21
Dear All,
I want to use TMemoryStream to read a file over 300MB
but when TMemoryStream->LoadFromFile, it claim "Out of Memory"
How can I solve it?
Emmanuel
 
 

Re:Out of memory for TMemoryStream

Emmanuel wrote:
Quote
I want to use TMemoryStream to read a file over 300MB
but when TMemoryStream->LoadFromFile, it claim "Out of Memory"
How can I solve it?
Untested: Try a TFileStream and then read pieces of the file which you
place in a TMemoryStream if you want.
Or buy some extra memory.
Hans.
 

Re:Out of memory for TMemoryStream

"Emmanuel" < XXXX@XXXXX.COM >wrote in message
Quote
I want to use TMemoryStream to read a file over 300MB
but when TMemoryStream->LoadFromFile, it claim "Out
of Memory" How can I solve it?
Install more memory chips on your machine. Chances are, you don't have
300MB of free RAM space available.
You should not be reading that much data into memory at one time to begin
with. Read and work with the file in much smaller chunks.
Gambit
 

{smallsort}

Re:Out of memory for TMemoryStream

"Remy Lebeau (TeamB)" < XXXX@XXXXX.COM >дʼ
Quote

"Emmanuel" < XXXX@XXXXX.COM >wrote in message
news: XXXX@XXXXX.COM ...

>I want to use TMemoryStream to read a file over 300MB
>but when TMemoryStream->LoadFromFile, it claim "Out
>of Memory" How can I solve it?

Install more memory chips on your machine. Chances are, you don't have
300MB of free RAM space available.

You should not be reading that much data into memory at one time to begin
with. Read and work with the file in much smaller chunks.


Gambit


Would you tell me how to read part of the data into the MemoryStream??
 

Re:Out of memory for TMemoryStream

"Emmanuel" < XXXX@XXXXX.COM >wrote in message
Quote
Would you tell me how to read part of the data into the
MemoryStream??
Use a TFileStream to read chunks of the data into the TMemoryStream.
Eg:
// Untested.
TFileStream *FileStream;
TMemoryStream *MemoryStream;
FileStream = new TFileStream( "file.txt", fmOpenRead |
fmShareDenyWrite );
MemoryStream = new TMemoryStream;
MemoryStream->CopyFrom( FileStream, BytesToRead );
// Now use the contents of the memory stream and keep
// doing CopyFrom to get new chunks until it returns 0,
// for no bytes left.
delete MemoryStream;
delete FileStream;
HTH
Jonathan
 

Re:Out of memory for TMemoryStream

"Jonathan Benedicto" < XXXX@XXXXX.COM >wrote in message
Quote
Use a TFileStream to read chunks of the data into the TMemoryStream.
Your example does the exact same thing that TMemoryStream::LoadFromFile()
does internally, which is what I'm advising Emmanuel to NOT use in the first
place. See my other reply.
Gambit
 

Re:Out of memory for TMemoryStream

"Emmanuel" < XXXX@XXXXX.COM >wrote in message
Quote
Would you tell me how to read part of the data into the MemoryStream??
I would not advise using a TMemoryStream at all. The whole point I am
trying to make (and Jonathan's example does the same thing that you are
trying to do), is that you should *NOT* be loading the entire 300MB file
into memory at one time at all. Process it in smaller pieces instead, ie:
BYTE buffer[1024];
TFileStream *fs = new TFileStream("the file here", fmOpenRead |
fmShareDenyWrite);
try
{
int size = fs->Size;
while( fs->Position < size )
{
int read = fs->Read(buffer, sizeof(buffer));
// process buffer up to read number of bytes...
}
}
__finally {
delete fs;
}
What EXACTLY are you trying to accomplish by loading a 300MB fle into memory
to begin with?
Gambit
 

Re:Out of memory for TMemoryStream

"Remy Lebeau (TeamB)" < XXXX@XXXXX.COM >wrote in message
Quote

Your example does the exact same thing that
TMemoryStream::LoadFromFile()
does internally, which is what I'm advising Emmanuel to NOT use in
the first
place. See my other reply.
I didn't mean the example to load the entire file. I forgot to have
the MemoryStream->Seek(0, soFromBeginning) before the CopyFrom, so
that the stream didn't expand.
Jonathan