Board index » cppbuilder » Serializing struct for TCP sending

Serializing struct for TCP sending


2007-05-16 02:10:20 AM
cppbuilder73
Hi all!:)
In short I would like to send whole structure instances by using
TCP-IP.. for example through Indy TCPClient.
The first and secure way is to serialize the members of a structure in a
buffer and then send them with
procedure WriteBuffer(const ABuffer; AByteCount: Longint; const
AWriteNow: boolean);
Anyway for simple structures for example:
struct
{
char name[255];
int intnum;
float fnum;
} my_struct;
it seems to work even something like that:
WriteBuffer (&myStruct,sizeof(my_struct));
that means that in memory they are simply placed one right the other..
so on the other side I can receive the buffer by knowing that structure
size, and cast the buffer to that struct pointer!
My question is: does that work always on every compiler? May the
procedure of sending that buffer to a program, running on a different
OS, produce a different object on destination even by using the same
struct type on the second program?
Maybe my problem is silly,and the answer obvious, but I am not sure of
what a compiler does exactly allocating data when I create structures or
classes, and what happen to data when sent by Indy or TCP-IP.
Thank U very much in advance,
Stefano B.
 
 

Re:Serializing struct for TCP sending

Stefano Bonifazi wrote:
Quote
My question is: does that work always on every compiler?
Yes, but the results may not be what you want them to be.
Quote
May the
procedure of sending that buffer to a program, running on a different
OS, produce a different object on destination even by using the same
struct type on the second program?
Yes.
See Endian issues.
See also alignment issues.
Quote
Maybe my problem is silly,and the answer obvious, but I am not sure of
what a compiler does exactly allocating data when I create structures
Depends on the alignment setting.
The default is probably for your char name[255] to be 256 bytes of
storage
Most compilers align the internal items for fast processor access.
So, a compiler for 8086 might default to byte-alignment while one for
pentium would be 4 byte align, and perhaps one for mainframe would be
64 or even 128 bit aligned.
This alignment is overridable by the programmer.
Quote
and what happen to data when sent by Indy or TCP-IP.
It is sent as-is.
Bottom line:
If you are sending exclusively to your own apps, you can do whatever
you want (send raw structures).
If you are sending to someone else's app, or your app that someone
compiles for their harware, you might be better off converting
integers and such into network-endian format and transmit the
individual pieces.
You might Google this group for endian
tinyurl.com/28s62e
 

Re:Serializing struct for TCP sending

"Bob Gonder" < XXXX@XXXXX.COM >wrote in message
Quote
If you are sending to someone else's app, or your app that
someone compiles for their harware, you might be better off
converting integers and such into network-endian format
You should also use 1-byte alignment for your structures so that there
is no unnecessary padding added by the compiler.
Gambit
 

{smallsort}

Re:Serializing struct for TCP sending

Hi thank U very much to both!
Quote
>If you are sending to someone else's app, or your app that
>someone compiles for their harware, you might be better off
>converting integers and such into network-endian format
sorry how? I always used Indy SendBuffer and on the other side even from
Linux they receive fine!
Quote

You should also use 1-byte alignment for your structures so that there
is no unnecessary padding added by the compiler.
U mean in "advanced compiler" options, "data alignment" right?
Thank U again!
 

Re:Serializing struct for TCP sending

"Stefano Bonifazi" < XXXX@XXXXX.COM >wrote in message
Quote
I always used Indy SendBuffer and on the other side even
from Linux they receive fine!
Windows and Linux run on the same hardware architecture, so the endian
is the same. But if you were to exchange data with to a Mac or other
non-PC machine, the endian would likely not match. Read/WriteBuffer()
do not perform any endian conversions at all.
Read/WriteInteger/Cardinal/SmallInt(), on the other hand, do perform
conversions from/to network byte ordering by default. All systems
understand network byte ordering, and the socket API provides
functions (which Indy wraps internally) to convert numbers between
network byte ordering (which is universal across networks) and host
byte ordering (which is specific to the machine being run on). So,
what you could do is send the individual members of your struct
separately, ie:
my_struct s;
TCPClient->WriteBuffer(s.name, 255);
TCPClient->WriteInteger(s.intnum);
// and so on...
TCPClient->ReadBuffer(s.name, 255);
s.intnum = TCPClient->ReadInteger();
// and so on...
Also, you might want to stay away from floating point numbers, if you
can. The byte size and binary representation may work differently on
different hardware architectures as well, regardless of the endian
used.
Quote
U mean in "advanced compiler" options, "data alignment" right?
No. That is a global setting that effects the entire application as a
whole. That would be very dangerous and error-prone. What you should
do instead is wrap your individual structs with #pragma pack
statements instead, ie:
#pragma pack(push, 1)
struct my_struct
{
//...
};
#pragma pack(pop)
Or:
#include <pshpack1.h>
struct my_struct
{
//...
};
#include <poppack.h>
Gambit
 

Re:Serializing struct for TCP sending

Stefano Bonifazi wrote:
Quote
>>converting integers and such into network-endian format

sorry how?
See the link in my first message
 

Re:Serializing struct for TCP sending

Hi Thank U very much!
Remy Lebeau (TeamB) ha scritto:
Quote
my_struct s;

TCPClient->WriteBuffer(s.name, 255);
TCPClient->WriteInteger(s.intnum);
// and so on...

what I exactly did always :)
Quote
TCPClient->ReadBuffer(s.name, 255);
s.intnum = TCPClient->ReadInteger();
// and so on...
Wish they used Indy on Linux..they uses simple Liux socket API.. hope
they perform conversion from Network endian to Host machine endian as
well as!!
Quote

Also, you might want to stay away from floating point numbers, if you
can. The byte size and binary representation may work differently on
different hardware architectures as well, regardless of the endian
used.
Engineering numbers are very seldom integers! .. So what to do?
Quote

>U mean in "advanced compiler" options, "data alignment" right?

No. That is a global setting that effects the entire application as a
whole. That would be very dangerous and error-prone.
Oh really?? I wonder why they put the option if that is dangerous!
Thank U!!
What you should
Quote
do instead is wrap your individual structs with #pragma pack
statements instead, ie:

#pragma pack(push, 1)
struct my_struct
{
//...
};
#pragma pack(pop)

Or:

#include <pshpack1.h>
struct my_struct
{
//...
};
#include <poppack.h>
Thank U.. never heard about those before!
My best regards!
Stefano B.
 

Re:Serializing struct for TCP sending

Bob Gonder wrote:
Quote
See the link in my first message



Thank U again!
 

Re:Serializing struct for TCP sending

"Stefano Bonifazi" < XXXX@XXXXX.COM >wrote in message
Quote
Wish they used Indy on Linux..they uses simple Liux socket API..
Indy works on Linux, and is built on top of the same API.
Quote
hope they perform conversion from Network endian to Host
machine endian as well as!!
Everybody does. It is the only way to ensure proper data integrity
over a network across different platforms.
Gambit
 

Re:Serializing struct for TCP sending

Hi Mr. Lebeau! Thank U for Ur answer!
Quote
>Wish they used Indy on Linux..they uses simple Liux socket API..

Indy works on Linux, and is built on top of the same API.
Sure, sorry my english:( I meant I wished they(the programmers on Linux
I am working with) would use Indy, but they don't, even though after
my recommendation! :(
Quote

>hope they perform conversion from Network endian to Host
>machine endian as well as!!

Everybody does. It is the only way to ensure proper data integrity
over a network across different platforms.
I c.. that's why that works even though I never did anything special to
take care of it before :)
Please have U suggestiond about sending floating point numbers? I can't
do without! Should I convert them to string representation and send the
strings instead to be sure nothing odd happens?
Thank U again!
Stefano B.