Board index » cppbuilder » Huge size Buffers

Huge size Buffers


2005-05-31 05:52:03 PM
cppbuilder87
Hi, I'm using Borland Buolder C++ Version 4.0 on Win-XP computer,
i need to use some Huge buffers of float and int types, don't ask
me what size exactly, becouse i don't know, i only know that i
want this buffers to be as large as possible, my program need to
work on lot of data and as the buffers i use are larger it's
better. So my question is if there is any limit to the size of
the buffers that i use in my application? if my PC have lot of
RAM mamory how can i be sure that i use all of it, as much as
it's possible?
Thanks very much!
Ramy.
( by the way please tell me if there is a
better group for this question, thanks )
 
 

Re:Huge size Buffers

For example for this line -
double HugeBuffer[0x5FFFFF];
i'm getting the following errors -
[Linker Error] Fatal: Error detected (LME1398).
[Linker Error] Fatal: Error detected (LME2024).
[Linker Error] Fatal: Error detected (LME1613).
[Linker Error] Fatal: Error detected (LME1548).
[Linker Warning] Unable to perform incremental link - performing full link....
[Linker Error] Fatal: Error detected (LME1398).
[Linker Error] Fatal: Error detected (LME2024).
[Linker Error] Fatal: Error detected (LME1613).
[Linker Error] Fatal: Error detected (LME1548).
 

Re:Huge size Buffers

"Ramy" < XXXX@XXXXX.COM >writes:
Quote
For example for this line -

double HugeBuffer[0x5FFFFF];

i'm getting the following errors -
Don't declare the array on the stack like that. Your stack space is
usually far less than your heap space. Instead, use dynamic memory:
const int BUFSIZE = 0x5FFFFF;
double * Doubles = new double[SIZE];
...
delete [] HugeBuffer;
Or better:
std::vector<double>Doubles(SIZE);
--
Chris (TeamB);
 

{smallsort}

Re:Huge size Buffers

That was a great tip, thanks very much!
When i use a dynamic memory allocation as you showed me, will
the only limit be the size of the memory in my PC ? does the
Builder 4.0 C++ gives me full access to the whole PC memory
space, even if it's HUGE ? isn't any other limit?
Thanks,
Ramy
Quote
Don't declare the array on the stack like that. Your stack space is
usually far less than your heap space. Instead, use dynamic memory:

const int BUFSIZE = 0x5FFFFF;

double * Doubles = new double[SIZE];
...
delete [] HugeBuffer;

Or better:

std::vector<double>Doubles(SIZE);

--
Chris (TeamB);
 

Re:Huge size Buffers

"Ramy" < XXXX@XXXXX.COM >writes:
Quote
That was a great tip, thanks very much!

When i use a dynamic memory allocation as you showed me, will
the only limit be the size of the memory in my PC ? does the
Builder 4.0 C++ gives me full access to the whole PC memory
space, even if it's HUGE ? isn't any other limit?
There shouldn't be any hard-coded limit, though there may be reasons
that the whole address space isn't available. For example, on
platforms where BCB runs, a pointer is only 32-bits and can therefore
only hold about 4 billion values (let's say 4GB.)
The limitations should be closer to the amount of RAM you have
available, minus some for fragmentation in your memory manager, etc.
--
Chris (TeamB);
 

Re:Huge size Buffers

OK, Thanks :-)
Quote
There shouldn't be any hard-coded limit, though there may be reasons
that the whole address space isn't available. For example, on
platforms where BCB runs, a pointer is only 32-bits and can therefore
only hold about 4 billion values (let's say 4GB.)

The limitations should be closer to the amount of RAM you have
available, minus some for fragmentation in your memory manager, etc.

Chris (TeamB);
 

Re:Huge size Buffers

When it exceed your available memory, it starts swapping to the pagefile.
 

Re:Huge size Buffers

Thanks :-)
"tinyabs" < XXXX@XXXXX.COM >wrote:
Quote
When it exceed your available memory, it starts swapping to the pagefile.
 

Re:Huge size Buffers

"Chris Uzdavinis (TeamB)" < XXXX@XXXXX.COM >wrote in message
Quote
"Ramy" < XXXX@XXXXX.COM >writes:

>That was a great tip, thanks very much!
>
>When i use a dynamic memory allocation as you showed me, will
>the only limit be the size of the memory in my PC ? does the
>Builder 4.0 C++ gives me full access to the whole PC memory
>space, even if it's HUGE ? isn't any other limit?

There shouldn't be any hard-coded limit, though there may be reasons
that the whole address space isn't available. For example, on
platforms where BCB runs, a pointer is only 32-bits and can therefore
only hold about 4 billion values (let's say 4GB.)
My understanding has always been that a process in Windows has access to a
virtual address space of 2GB, with the upper 2GB reserved for Windows
itself. The amount of RAM will not affect allocations in an app, but will
have a big bearing on speed because of swapping to the hard drive.
Fragmentation will affect the amount you can allocate in a continuous block.
Also I believe there is a trick to allow 3GB of the address space to be
available.
Simon.
 

Re:Huge size Buffers

"Simon D" <lightsOnNo1@home>writes:
which should also be considered.
In any case, 4GB is an upper bound, though 2GB is a smaller (better)
upper bound. If they can't hold 2GB, they certainly can't hold four!
Quote
The amount of RAM will not affect allocations in an app, but will
have a big bearing on speed because of swapping to the hard drive.
With today's larger ram sizes, the swap size should be about equal or
less to actual ram. Certainly, the sum of ram+swap is an upper
bound on total ram available. If we're swapping, of course it's
slower; the hard drive runs as mechanical speeds, while ram runs at
electrical speeds.
Quote
Fragmentation will affect the amount you can allocate in a continuous block.
Yes, but it also can waste memory and thereby deny the application
from allocating large blocks. This is especially problematic with the
use-case of allocating and destroying increasingly larger buffers
fairly often.
Quote
Also I believe there is a trick to allow 3GB of the address space to be
available.
2GB, 3GB, 4GB, it's still the same order of magnitude, and IMHO
doesn't really change the answer all that much. We're in the single
gigabyte range, until we move to a 64-bit machine which can hold and
address a lot more memory.
--
Chris (TeamB);
 

Re:Huge size Buffers

"Chris Uzdavinis (TeamB)" < XXXX@XXXXX.COM >wrote in message
Quote
>The amount of RAM will not affect allocations in an app, but will
>have a big bearing on speed because of swapping to the hard drive.

With today's larger ram sizes, the swap size should be about equal or
less to actual ram. Certainly, the sum of ram+swap is an upper
bound on total ram available. If we're swapping, of course it's
slower; the hard drive runs as mechanical speeds, while ram runs at
electrical speeds.
Another thing to think about (I believe) is the number of processes that are
running on the system. They will have their own memory demands which could
cause swapping in and out of memory earlier than expected.
Simon.