Board index » cppbuilder » memory question

memory question


2008-04-08 02:37:01 AM
cppbuilder112
Hi,
I have an array of unsigned ints data[500]
These bytes are mapped into a sample structure
struct _SAMP_BLK {
INT8U channelID; // Channel (band) ID for this blocks I/Q data
INT16S sampleI; // In Phase Sample
INT16S sampleQ; // Quadrature Phase Sample
};
to get the data "into" a struc object I do this now
_SAMP_BLK sample; //Problem? copying memory
//First 8 bits is the channel ID (frequency band)
sample.channelID = aChannelSample->data[i];
//Next 16 bits is the I channel
sample.sampleI = aChannelSample->data[i+1]<<8;
sample.sampleI |= aChannelSample->data[i+2];
//Next 16 bits is the Q channel
sample.sampleQ = aChannelSample->data[i+3] << 8;
sample.sampleQ |= aChannelSample->data[i+4];
I would like to "map" the data directly to a _SAMPL_BLK structure pointer. Is that possible?
I'm thinking about something like
_SAMP_BLK *sample = (_SAMP_BLK *) aChannelSample->data[i] | aChannelSample->data[i+1] | ...;
but I don't know the syntax for such a thing..
then there would be no copying of memory, I guess?
My problem is that I have billions of these and try not to ever copy them..
-totte
 
 

Re:memory question

"Totte Karlsson" < XXXX@XXXXX.COM >wrote in message
Quote
to get the data "into" a struc object I do this now
Use one of the following instead (assuming the data is at least
sizeof(_SAMP_BLK) in size):
// this makes a copy of the data
_SAMP_BLK sample;
memcpy(&sample, aChannelSample->data, sizeof(_SAMP_BLK));
Or:
// this makes a copy of the data
_SAMP_BLK sample =
*(reinterpret_cast<_SAMP_BLK*>(aChannelSample->data));
Or:
// this points to the original data directly
_SAMP_BLK *sample = reinterpret_cast<_SAMP_BLK*>(aChannelSample->data);
Gambit
 

Re:memory question

Perfect, I tried it and it works nice
Quote

// this points to the original data directly
_SAMP_BLK *sample = reinterpret_cast<_SAMP_BLK*>(aChannelSample->data);
I modified it to traverse the data array (that contains many _SAMPL_BLK's in succesion)
for(int i = 0; i < 500; i+=5)
_SAMP_BLK *sample = reinterpret_cast<_SAMP_BLK*>(aChannelSample->data + i);
that will speed up my program a lot!
Thanks,
-totte
 

{smallsort}

Re:memory question

Actually, I did not get it right.
The channelID field is the only one correctly "decoded"
typedef struct _SAMP_BLK {
INT8U channelID; // Channel (band) ID for this blocks I/Q data
INT16S sampleI; // In Phase Sample
INT16S sampleQ; // Quadrature Phase Sample
}SAMP_BLK, *PSAMP_BLK;
the sampleI and sampleQ seems to have a "byteswap". I'll explain below.
Here is how I read in the bytes in a object (total of 40 bits),
//First 8 bits is the channel ID (frequency band)
sample.channelID = aChannelSample->data[i];
//Next 16 bits is the I channel
sample.sampleI = aChannelSample->data[i+1]<<8; //high byte ???
sample.sampleI |= aChannelSample->data[i+2]; //low byte ????
//Next 16 bits is the Q channel
sample.sampleQ = aChannelSample->data[i+3] << 8;
sample.sampleQ |= aChannelSample->data[i+4];
Am I not reading the bytes "in order" ??
When I'm inspecting the reinterpreted cast pointer and a sample as obtained above
I get for example in hex 0x0200 for a sampleQ record from the casted pointer and a 0x0002 from the
sample above. That is "swapped" right?
I guess when I fill out the sample object above, I do the |= or the (<<) wrongly somehow? Do I make sense?
thanks
-totte
Remy Lebeau (TeamB) wrote:
Quote
"Totte Karlsson" < XXXX@XXXXX.COM >wrote in message
news:47fa69e2$ XXXX@XXXXX.COM ...

>to get the data "into" a struc object I do this now

Use one of the following instead (assuming the data is at least
sizeof(_SAMP_BLK) in size):

// this makes a copy of the data
_SAMP_BLK sample;
memcpy(&sample, aChannelSample->data, sizeof(_SAMP_BLK));

Or:

// this makes a copy of the data
_SAMP_BLK sample =
*(reinterpret_cast<_SAMP_BLK*>(aChannelSample->data));

Or:

// this points to the original data directly
_SAMP_BLK *sample = reinterpret_cast<_SAMP_BLK*>(aChannelSample->data);


Gambit


 

Re:memory question

"Totte Karlsson" < XXXX@XXXXX.COM >wrote in message
Quote
the sampleI and sampleQ seems to have a "byteswap".
The data is likely being stored in a different endian than your machine
uses. The channelID field does not need any swapping because it is only 1
byte in size. Because your other fields are 2 bytes in size each, you have
to make sure you interpret them in the correct byte order when accessing
them. For example:
{
//...
PSAMP_BLK sample = reinterpret_cast<PSAMP_BLK>(aChannelSample->data
+ 1);
INT16S sampleI = ntohs(sample->sampleI);
INT16S sampleQ = ntohs(sample->sampleQ);
//...
}
Quote
Here is how I read in the bytes in a object (total of 40 bits),
You already know how to use casted pointers now to avoid such copying.
Quote
Am I not reading the bytes "in order" ??
No.
Gambit
 

Re:memory question

thanks again,
This is how I store the bytes before I'm writing them.
I guess it is messed up here
INT16S sampleI = 1234567;
data[i + 1] = (INT8U) sampleI>>8; //Store upper 8 bits of sampleI in data[i+1]
data[i + 2] = (INT8U) sampleI; //Store lower 8 bits of sampleI in data[i+1]
I don't see anything wrong with that (but it must be)?
Should I not be able to do
INT16S sampleI = data[i + 1] << 8; //Put the upper 8 bits in high byte of sampleI
sampleI |= data[i + 2];
to get the correct number back?
Something with the>>or << that I'm not doing right?
For the moment, I only read data that I'm producing. I will read data later which will need to be swapped.
Quote
You already know how to use casted pointers now to avoid such copying.
I'll use the casted pointer, but I want to get the same result using both methods..
-totte
 

Re:memory question

"Totte Karlsson" < XXXX@XXXXX.COM >wrote in message
Quote
This is how I store the bytes before I'm writing them.
I guess it is messed up here
Why are you storing the values to bytes like that? You can use the same
memcpy() trick I showed you earlier, just in reverse:
memcpy(data, &some_sample, sizeof(_SAMP_BLK));
Alternatively, using the same pointer casting trick I showed you, you can
assign values directly to the bytes instead:
PSAMP_BLK data_sample = reinterpret_cast<PSAMP_BLK>(data);
data_sample->sampleI = some_sample_sampleI;
Quote
Something with the>>or << that I'm not doing right?
I suggest you stop trying to use manual bit-shifting at all, and let the
actual SAMP_BLK structure do all of the work for you.
Gambit
 

Re:memory question

OK,
so instead of (which is not doing what i expect it to)
recSamplesIV.data[i] = pSampleBlk->channelID;
recSamplesIV.data[i + 1] = pSampleBlk->sampleI>>8;
recSamplesIV.data[i + 2] = (INT8S) pSampleBlk->sampleI;
recSamplesIV.data[i + 3] = pSampleBlk->sampleQ>>8;
recSamplesIV.data[i + 4] = (INT8S) pSampleBlk->sampleQ;
I could do
memcpy(data[i], *pSampleBlk, sizeof(_SAMP_BLK)
that's more elegant. I'll try that
Thanks
-totte
 

Re:memory question

Quote
memcpy(data[i], *pSampleBlk, sizeof(_SAMP_BLK)
should be
memcpy(&(data[i]), pSampleBlk, sizeof(_SAMP_BLK))
 

Re:memory question

Totte Karlsson < XXXX@XXXXX.COM >writes:
Quote
Perfect, I tried it and it works nice

>// this points to the original data directly
>_SAMP_BLK *sample = reinterpret_cast<_SAMP_BLK*>(aChannelSample->data);

I modified it to traverse the data array (that contains many
_SAMPL_BLK's in succesion)
Your code has two problems:
Quote
for(int i = 0; i < 500; i+=5)
1. Why 5?
Quote
_SAMP_BLK *sample = reinterpret_cast<_SAMP_BLK*>(aChannelSample->data + i);
2. The expression aChannelSample->data+i may evaluate to a pointer
that isn't suitably aligned for holding a _SAMP_BLK structure. The
result may be undefined behavior.
Quote
that will speed up my program a lot!
Particularly if the result of the undefined behavior is a quick
crash. :-)
NB: Names starting with _[A-Z] are reserved for for the C/C++
implementation; user code must not declare such names.
 

Re:memory question

Quote

Your code has two problems:


>for(int i = 0; i < 500; i+=5)

1. Why 5?

because the size of my structure is 40 bits and each element in the data array is 8 bits. 8*5 = 40;
Quote

>_SAMP_BLK *sample = reinterpret_cast<_SAMP_BLK*>(aChannelSample->data + i);

2. The expression aChannelSample->data+i may evaluate to a pointer
that isn't suitably aligned for holding a _SAMP_BLK structure. The
result may be undefined behavior.
Is there another way to create a SAMP_BLK pointer to the 40 bits memory in the data array that is safer.
I can't copy the memory since its to slow, but I can't live with undefined behavior either..
Quote
>that will speed up my program a lot!

Particularly if the result of the undefined behavior is a quick
crash. :-)
thats true!
Quote
NB: Names starting with _[A-Z] are reserved for for the C/C++
implementation; user code must not declare such names.
I did not know. I will tell the people naming the structures like that to stop it...
 

Re:memory question

"Thomas Maeder [TeamB]" < XXXX@XXXXX.COM >wrote in message
Quote
>for(int i = 0; i < 500; i+=5)

1. Why 5?
sizeof(_SAMP_BLK) is 5.
Quote
2. The expression aChannelSample->data+i may evaluate to a
pointer that isn't suitably aligned for holding a _SAMP_BLK structure.
It will be if 'data' is a byte array, and the contents of the structure are
byte-aligned.
Gambit
 

Re:memory question

"Totte Karlsson" < XXXX@XXXXX.COM >wrote in message
Quote
I could do

memcpy(data[i], *pSampleBlk, sizeof(_SAMP_BLK)
Almost. It should be this instead:
memcpy(&data[i], pSampleBlk, sizeof(SAMP_BLK));
Gambit
 

Re:memory question

Totte Karlsson wrote:
Quote
>>_SAMP_BLK *sample = reinterpret_cast<_SAMP_BLK*>(aChannelSample->data + i);
>
>2. The expression aChannelSample->data+i may evaluate to a pointer
>that isn't suitably aligned for holding a _SAMP_BLK structure. The
>result may be undefined behavior.

Is there another way to create a SAMP_BLK pointer to the 40 bits memory in the data array that is safer.
I can't copy the memory since its to slow, but I can't live with undefined behavior either..
He said it Might Be, not Definately.
Reason for it being undefined is because you didn't
define aChannelSample->data in the message.
Nor did your code specify the structure packing.
In your actual code, it is hopefully properly defined,
so your usage is defined.
Another option:
SAMP_BLK *sample = reinterpret_cast<SAMP_BLK*>(aChannelSample->data);
for( int i=0; i<100; ++i )
Process( sample[i] );
 

Re:memory question

"Remy Lebeau \(TeamB\)" < XXXX@XXXXX.COM >writes:
Quote
"Thomas Maeder [TeamB]" < XXXX@XXXXX.COM >wrote in message
news: XXXX@XXXXX.COM ...

>>for(int i = 0; i < 500; i+=5)
>
>1. Why 5?

sizeof(_SAMP_BLK) is 5.
Is the code correct if the contents of _SAMP_BLK change?
If not, then one could argue that 5 is the wrong value to use today,
and it is only coincidence that its value matches the correct
expression: sizeof(_SAMP_BLK).
There is no benefit to "inlining" the evaluation of a sizeof
expression, other than to add fragility and possible error to the
code.
--
Chris (TeamB);