Board index » cppbuilder » Re: TWinSocketStream Read in BYTE buffer

Re: TWinSocketStream Read in BYTE buffer


2006-01-06 02:50:09 AM
cppbuilder43
"Maurice Anderson" < XXXX@XXXXX.COM >wrote in message
Quote
Also found an issue in ReadBuf which causes TFileStream *FS
to write fewer bytes to the file than what is in the stream.
In order to ensure that all data gets written to the file, you have to look
at the return value of TFileStream::Write(), just like you have to with
TWinSocketStream::Write(), so that you know how many bytes were actually
written to the stream. That way, you can call Write() again if needed to
save any remaining unwritten bytes.
The easiest way to do that is to change ReadBuf() and WriteBuf() to take a
TStream* parameter instead of a TWinSocketStream* parameter, since the inner
logic already handles incomplete reads/writes. That way, you can pass both
TWinSocketStream and TFileStream to the same processing code:
bool __fastcall ReadBuf(TStream *Stream, void *Buffer, int BufSize)
{
BYTE *pBuf = (BYTE*) Buffer;
int NumRead;
try
{
while( BufSize>0 )
{
NumRead = Stream->Read(pBuf, BufSize);
if( NumRead < 1 )
return false;
pBuf += NumRead;
BufSize -= NumRead;
}
}
catch(const Exception &)
{
return false;
}
return true;
}
bool __fastcall WriteBuf(TStream *Stream, void *Buffer, int BufSize)
{
BYTE *pBuf = (BYTE*) Buffer;
int NumSent;
try
{
while( BufSize>0 )
{
NumSent = Stream->Write(pBuf, BufSize);
if( NumSent < 1 )
return false;
pBuf += NumSent;
BufSize -= NumSent;
}
}
catch(const Exception &)
{
return false;
}
return true;
}
Then you can update ReadFile() and SendFile() accordingly:
bool __fastcall SendFile(TWinSocketStream *Stream, const AnsiString
&FileName)
{
bool error = false;
try
{
TFileStream *FS = new TFileStream(FileName, fmOpenRead |
fmShareDenyWrite);
try
{
char type = 0x02;
if( WriteBuf(Stream, &type, sizeof(char)) )
{
if( WriteString(Stream, ExtractFileName(FileName)) )
{
int size = FS->Size;
if( WriteBuf(Stream, &size, sizeof(int)) )
{
BYTE rbuf[256];
int bufsize;
while( size>0 )
{
bufsize = size;
if( bufsize>sizeof(rbuf) )
bufsize = sizeof(rbuf);
if( !ReadBuf(FS, rbuf, bufsize) ) // <--
here
{
error = true;
break;
}
if( !WriteBuf(Stream, rbuf, bufsize) )
{
error = true;
break;
}
size -= bufsize;
}
}
}
}
}
__finally
{
delete FS;
}
}
catch(const Exception &)
{
error = true;
}
return !error;
}
bool __fastcall ReadFile(TWinSocketStream *Stream, AnsiString &FileName)
{
AnsiString fn;
if( ReadString(Stream, fn) )
{
fn = ("C:\\Some Folder\\" + ExtractFileName(fn));
int size = 0;
if( ReadBuf(Stream, &size, sizeof(int)) )
{
bool error = false;
try
{
TFileStream *FS = new TFileStream(fn, fmCreate);
try
{
FS->Size = size;
FS->Position = 0;
BYTE rbuf[256];
int bufsize;
while( size>0 )
{
bufsize = size;
if( bufsize>sizeof(rbuf) )
bufsize = sizeof(rbuf);
if( !ReadBuf(Stream, rbuf, bufsize) )
{
error = true;
break;
}
if( !WriteBuf(FS, rbuf, bufsize) ) // <-- here
{
error = true;
break;
}
size -= bufsize;
}
}
__finally
{
delete FS;
}
}
catch(const Exception &)
{
error = true;
DeleteFile(fn);
}
if( !error )
{
FileName = fn;
return true;
}
}
}
return false;
}
Quote
(5) So ReadBuf is called 249 times to read 63,744 Bytes.
And each time ReadBuf returns, FS faithfully writes the contents
to itself the same 249 times. This leaves the last 76 Bytes to go.
That is not a correct assessment.
The size value that ReadFile() receives will indicate the full size of the
file. In this case, the size should be coming through as 63820, not 63744.
ReadFile() will keep reading until all 63,820 bytes have been received.
That is because of this line:
while( size>0 )
The size of the buffer being used does not matter. In this case, since the
buffer size is 256, after 249 iterations of the loop, the size variable will
be 76, and ReadFile() will then read the last 76 bytes instead of 256 bytes
and then exit the loop since the size variable falls to 0.
Quote
(7) You see, once the Stream reads those 76 Bytes into pBuf, it cycles
back once more. On this last cycle in encounters 0 bytes to read
so it kicks back a false as the return variable:
That is not a correct assessment, either.
On input, the BufSize parameter is set to 76. ReadBuf() loops only as long
as there is data expected to be received:
while( BufSize>0 )
Every time pStream->Read() returns without error, the actual number of bytes
received is substracted from the BufSize parameter:
BufSize -= NumRead;
After 76 bytes have eventually been received, the BufSize parameter has
fallen to 0 and the loop exits without calling pStream->Read() again. At
which time, a 'true' value is returned, not 'false'.
Quote
if( !ReadBuf(rbuf, bufsize) )
{
error = true;
break; < - exists loop with 76
bytes
left in rBuf
}
No, it does not. ReadBuf() return true, not false, on the last 76 bytes.
Quote
if( FS->Write(rbuf, bufsize) < 1 ) <- Never
sees
the last 76 bytes
{
Yes, it will.
Quote
(10) To resolve this I change around ReadBuf to return the number of
bytes read. If the bytes read from ReadBuf are < 1, then I kick
out of the while loop.
You should not have done that. There is nothing wrong with the ReadBuf()
code I have given you.
Not only that, but you have introduced bugs into ReadBuf() now. If
successful, you are returning a hard-coded 256 rather than the number of
bytes actually read. If failed, you are returning the number of bytes
received in the last read, not the total number of bytes received before the
error occured.
Gambit
 
 

Re:Re: TWinSocketStream Read in BYTE buffer

Quote
Not only that, but you have introduced bugs into ReadBuf() now. ..
Ok, I see your points and undid my changes.
Couple things that are not clear with me though:
(1) What chunk sizes does TClientSocket and
TServerSocket use to send/receive bytes over
the network? 4K, 8K, 16K, etc?
(2) What is the transfer speed on say a 100 /mbits LAN?
(3) What is the limit size in terms of Byes that that the Stream
can hold? And once this limit is reached, what happens if
I don't do a ClientSocket1->Socket->SendBuf(..);
(4) Using your ReadBuf/writeBuf methods at what point is the
data considered transfered? For example on the Client end
we have: ClientSocket1->Socket->SendBuf(..) and on the
server we have: NumRead = Stream->Read(pBuf, BufSize);
Once Socket->SendBuf returns, is the data considered 'sent'
over or is it considered sent when we read from the Stream in:
Stream->Read(pBuf...)?
(5) Does the whole 25 MB file already exists in the stream ready
to be written to pBuf (and by extension fully copied from the client)
or is the ClientSocket holding it's horses ready to transmit another
chunk as soon as NumRead = Stream->Read(pBuf, BufSize) is completed?
(6) Pertaining to the sending/receiving of bytes in #4, how can I count
said bytes transferred I basically want to add some sort of data
transfer BYTE counter to let the user know file/data copy progress.
(7) What is the most efficient BYTE buffer size to allocate for the reading
and
writing functions for small, medium and large files? Or does it
matter?
Currently, I have BYTE rbuf[256];, what happens if I set it to say:
BYTE rbuf[1024], etc?
(8) How can I get the Client to open its own thread for sending data?
I use TServerClientThread to receive connections, but how can I
use it to open a connection to a server and use it to transmit?
(9) How can I track each individual connection received on the server
to say get the source IP address?
(10) How an I convert a BYTE to binary 1s and 0s?
Thanks
 

Re:Re: TWinSocketStream Read in BYTE buffer

"Maurice Anderson" < XXXX@XXXXX.COM >wrote in message
Quote
What chunk sizes does TClientSocket and TServerSocket use to
send/receive bytes over the network? 4K, 8K, 16K, etc?
That is managed by the socket internally, and by the network hardware. The
VCL does not control that.
Quote
What is the transfer speed on say a 100 /mbits LAN?
That is entirely dependant on the speed of your code when sending/reading
data, as well as the amount of other traffic on the network at the time.
Speed will vary.
Quote
What is the limit size in terms of Byes that that the Stream can hold?
TWinSocketStream does not hold any data. It is just a passthrough to the
socket. The socket itself has separate internal buffers for incoming and
outgoing data. The VCL does not control those sizes, but you can use the
Win32 API getsockopt() function to retreive the sizes, and setsockopt() to
change the sizes.
Quote
And once this limit is reached, what happens if I don't do a
ClientSocket1->Socket->SendBuf(..);
The data remains buffered inside the socket. Eventually, the inbound buffer
will fill up and the socket will not be able to accept any further data
until buffer space is cleared. While the buffer is full, the sender will
likely receive an error, such as WSAENOBUFS (10055) or
WSAEWOULDBLOCK(10035).
Quote
Using your ReadBuf/writeBuf methods at what point is the data
considered transfered?
Inbound data is "transferred" as soon as it appears in your target buffer.
As soon as decide not to read any more data, you have finished transferring
everything.
Outbound data is "transferred" as soon as the socket accepts it. The socket
will buffer the data internally and then transmit it over the network at its
leisure. TWinSocketStream::Write() will return how many bytes the socket
accepted into its buffer. There is no way to know when the data is actually
transmitted over the network. The only thing you can do is assume that it
will be transmitted *at some point in time* if the socket accepted it, and
move on.
Quote
For example on the Client end we have: ClientSocket1->Socket->SendBuf(..)
and on the server we have: NumRead = Stream->Read(pBuf, BufSize);
Once Socket->SendBuf returns, is the data considered 'sent' over or is it
considered sent when we read from the Stream in: Stream->Read(pBuf...)?
On the client side, it is considered "sent" when SendBuf() returns,
indicating that the socket accepted the data into its internal buffer.
On the server side, it is considered "received" when the data appears in
pBuf.
Quote
Does the whole 25 MB file already exists in the stream ready to be
written to pBuf (and by extension fully copied from the client)
The buffers inside a socket are not large enough to hold that much data at
one time.
Quote
is the ClientSocket holding it's horses ready to transmit another
chunk as soon as NumRead = Stream->Read(pBuf, BufSize) is completed?
Yes.
Quote
Pertaining to the sending/receiving of bytes in #4, how can I
count said bytes transferred
That is what the return value of the stream's Read/Write() methods is for,
as are the return value of the socket's SendBuf/ReceiveBuf() methods. The
return value is the number of bytes actually received/accepted at that
particular moment.
Quote
What is the most efficient BYTE buffer size to allocate for the
reading and writing functions for small, medium and large files?
That is entirely up to you to choose. Like I told you earlier, the code I
gave you does not care what sizes are used. What you can do, however, is
use get/setsockopt() to have your buffer sizes match the socket's internal
buffer sizes.
Quote
Currently, I have BYTE rbuf[256];, what happens if I set it to
say: BYTE rbuf[1024], etc?
All that does is lets more data be buffered in memory at a time. Of course,
the larger the buffer, the more space that is available when receiving data
from a socket, which will likely allow the socket to clear its incoming
buffer more quickly. But the size of the buffer for outgoing data is still
limited to the size of the socket's internal outgoing buffer.
Quote
How can I get the Client to open its own thread for sending data?
TClientSocket is not a threaded component at all. You will have to manage
the thread yourself manually. Derive a class from TThread and override its
Execute() method. Pass the TClientSocket's Socket property to the thread,
such as a parameter to the thread's constructor:
class TClientThread : public TThread
{
private:
TCustomWinSocket *FSocket;
TWinSocketStream *FStream;
protected;
virtual void __fastcall AfterConstruction();
virtual void __fastcall Execute();
public:
__fastcall TClientThread(TCustomWinSocket *ASocket);
__fastcall ~TClientThread();
};
__fastcall TClientThread::TClientThread(TCustomWinSocket *ASocket)
: TThread(true)
{
FSocket = ASocket;
FStream = new TWinSocketStream(FSocket, 5000);
}
void __fastcall TClientThread::AfterConstruction()
{
Resume();
}
__fastcall TClientThread::~TClientThread()
{
delete FStream;
}
void __fastcall TClientThread::Execute()
{
while( (!Terminated) && (FSocket->Connected) )
{
// use FStream as needed ...
}
}
Then you can start the thread after you have connected the TClientSocket to
the server:
TClientThread *FThread = NULL;
void __fastcall TForm1::ClientSocket1Connect(TObject *Sender,
TCustomWinSocket *Socket)
{
FThread = new TClientThread(Socket);
}
void __fastcall TForm1::ClientSocket1Disconnect(TObject *Sender,
TCustomWinSocket *Socket)
{
if( FThread )
{
FThread->Terminate();
FThread->WaitFor();
delete FThread;
FThread = NULL;
}
}
Quote
I use TServerClientThread to receive connections, but how can
I use it to open a connection to a server and use it to transmit?
You cannot use TServerClientThread on the client side. It only works on the
server side. Use TThread instead on the client side, as shown above.
Quote
How can I track each individual connection received on the server
to say get the source IP address?
TServerSocket has ActiveConnections and Connections[] properties available.
Or else you can maintain your own list manually, such as by grabbing the IP
address at the beginning of the TServerClientThread's ClientExecute() method
and then add it to your own thread-safe list somewhere, and then remove the
address at the end of ClientExecute().
Quote
How an I convert a BYTE to binary 1s and 0s?
It is already stored in a binary format. Everything in memory is binary.
Gambit
 

{smallsort}