Board index » delphi » tidTCPClient/Server - problem transfering data.

2007-11-21 01:16:35 AM
delphi174
Hi,
I am using Delphi 2006 and Indy 10.
We are using TidTCPServer and TidTCPClient. We can't rewrite this to
TidCmdTCPServer and TidCmdTCPClient!!!
On the client side we the code looks like this:
result := IdTCPClient1.SendCmd('100 Test', [201, 202]);
if result = 201 then
begin
while not IdTCPClient1.IOHandler.InputBufferIsEmpty do
begin
lline := IdTCPClient1.IOHandler.ReadLn;
"processing of lline".
end;
end
else
...
On the server side we have a code like this:
procedure TForm5.IdTCPServer1Execute(AContext: TIdContext);
var
lline: String;
begin
lline:= AContext.Connection.IOHandler.ReadLn;
if trim(lline) = '' then
AContext.Connection.IOHandler.WriteLn('')
else
begin
AContext.Connection.IOHandler.Write(Memo1.lines);
end;
end;
This works just fin 99% of the time - but now and then the client does not
receive all data from the server.
For testing I code a "end of message string", that I put at the end on the
TStringList (Memo1.lines) written in the server, and modified the loop on
the client side to wait for the "end of message string". This works find -
but now and then it still does not get all data. I coded a 5 second delay to
prevent a neverending loop. This times out.
For both cases the data is there next time I try to write something from the
client. That means that the sendcmd will fail since it gets som rubish data
left from last session instead the expected command result.
Anyone seen this problem??? The server/client can handle thousands of
commands, but then suddenly it fails. The messages that are not transfered
can actually be smaller than messages transferred... I can't find any
special about the messages that fail...
Should I use some kind of flush command to ensure that everything is
written??? that is the only thing I can think of...
 

"Arvid Haugen" <XXXX@XXXXX.COM>writes
anyway, as they are not designed to work together. Use TIdTCPClient with
TIdCmdTCPServer, and TIdCmdTCPClient with TIdTCPServer.
potentially miss data. You are misusing the IOHandler's InputBuffer, which
you really shouldn't be relying on directly to begin with. It is meant for
Indy's internal use, and its semantics do not follow the logic you are
trying to use with it.
manual synchronization method, to safely access the TMemo. For example:
type
TMemoSync = class(TIdSync)
protected
fMemo: TMemo;
fLines: TStrings;
procedure DoSynchronize; override;
public
constructor Create(AMemo: TMemo; ALines: TStrings);
class procedure GetLines(AMemo: TMemo; ALines: TStrings);
end;
constructor TMemoSync.Create(AMemo: TMemo; ALines: TStrings);
begin
inherited Create;
fMemo := AMemo;
fLines := ALines;
end;
procedure TMemoSync.DoSynchronize;
begin
fLines.Assign(fMemo1.Lines);
end;
class procedure TMemoSync.GetLines(AMemo: TMemo; ALines: TStrings);
begin
with Create(AMemo, ALines) do
try
Synchronize;
finally
Free;
end;
end;
procedure TForm5.IdTCPServer1Execute(AContext: TIdContext);
var
LLines: TStrings;
begin
//...
LLines := TStringList.Create;
try
TMemoSync.GetLines(Memo1, LLines);
// send LLines as needed...
finally
LLines.Free;
end;
//...
end;
one. You need to do so, ie:
procedure TForm5.IdTCPServer1Execute(AContext: TIdContext);
var
LLine: String;
LLines: TStrings;
begin
LLine := Trim(AContext.Connection.IOHandler.ReadLn);
if Fetch(LLine) = '100' then
begin
LLines := TStringList.Create;
try
TMemoSync.GetLines(Memo1, LLines);
// send LLines as needed...
finally
LLines.Free;
end;
end
else
AContext.Connection.IOHandler.WriteLn('500 Unknown command');
//...
end;
As for the lines themselves, I strongly suggest that you either:
1) set the AWriteLinesCount parameter of Write(TStrings) to True, and then
use ReadStrings() on the client:
--- server ---
procedure TForm5.IdTCPServer1Execute(AContext: TIdContext);
//...
begin
// ...
AContext.Connection.IOHandler.Write('201 Data follows');
AContext.Connection.IOHandler.Write(LLines, True);
// ...
end;
--- client ---
var
LLines: TStrings;
begin
Result := IdTCPClient1.SendCmd('100 Test', [201, 202]);
if Result = 201 then
begin
LLines := TStringList.Create;
try
IdTCPClient1.IOHandler.ReadStrings(LLines);
// loop through LLines as needed...
finally
LLines.Free;
end;
end;
else
//...
end;
2) write an explicit "end of data" line after sending the rest of your
lines, and then your client can look for that. Stop looking at the
InputBuffer at all, just keep calling ReadLn() unconditionally until that
ending line is received:
--- server ---
procedure TForm5.IdTCPServer1Execute(AContext: TIdContext);
//...
begin
// ...
AContext.Connection.IOHandler.Write('201 Data follows');
AContext.Connection.IOHandler.Write(LLines);
AContext.Connection.IOHandler.Write('<endofdata>');
// ...
end;
--- client ---
var
LLine: String;
begin
Result := IdTCPClient1.SendCmd('100 Test', [201, 202]);
if Result = 201 then
begin
LLine := IdTCPClient1.IOHandler.ReadLn;
while LLine <>'<endofdata>' do
begin
// process LLine as needed...
end;
end;
else
//...
end;
You can take that a step further by using standard RFC formatting, which
uses '.' as the ending line, and escapes any leading '.' on the lines. This
way, you can use WriteRFCStrings() and Capture() instead:
--- server ---
procedure TForm5.IdTCPServer1Execute(AContext: TIdContext);
//...
begin
// ...
AContext.Connection.IOHandler.Write('201 Data follows');
AContext.Connection.IOHandler.WriteRFCStrings(LLines);
// ...
end;
--- client ---
var
LLines: TStrings;
begin
Result := IdTCPClient1.SendCmd('100 Test', [201, 202]);
if Result = 201 then
begin
LLines := TStringList.Create;
try
IdTCPClient1.IOHandler.Capture(LLines);
// loop through LLines as needed...
finally
LLines.Free;
end;
end;
else
//...
end;
3) since your client is using SendCmd() anyway, write the lines in a format
that SendCmd() can handle for you directly. The client's LastCmdResult will
then contain the lines after SendCmd() exits:
--- server ---
procedure TForm5.IdTCPServer1Execute(AContext: TIdContext);
var
LLine: String;
LLines: TStrings;
I: Integer;
begin
LLine := Trim(AContext.Connection.IOHandler.ReadLn);
if Fetch(LLine) = '100' then
begin
LLines := TStringList.Create;
try
TMemoSync.GetLines(Memo1, LLines);
if LLines.Count>0 then
begin
AContext.Connection.IOHandler.WriteLn('201-Data
follows');
if LLines.Count>1 then
begin
for I := 0 to LLines.Count-2 do
AContext.Connection.IOHandler.WriteLn('201-' +
LLines[I]);
end;
AContext.Connection.IOHandler.WriteLn('201 ' +
LLines[LLines.Count-1]);
end else
AContext.Connection.IOHandler.WriteLn('202 No data');
finally
LLines.Free;
end;
end
else
AContext.Connection.IOHandler.WriteLn('500 Unknown command');
//...
end;
--- client ---
begin
Result := IdTCPClient1.SendCmd('100 Test', [201, 202]);
if Result = 201 then
begin
// loop through IdTCPClient1.LastCmdResult.Text needed...
end;
else
//...
end;
#3 is how TIdCmdTCPServer works internally. SendCmd() is designed to be
used with the types of protocols that TIdCmdTCPServer is designed to handle
automatically for you. I know you said you don't want to use
TIdCmdTCPServer, but you really should reconsider that. #3 above can be
translated to the following simplified OnCommand event handler:
procedure TForm5.100Command(ASender: TIdCommand);
begin
TMemoSync.GetLines(Memo1, ASender.Response);
if ASender.Response.Count>0 then
ASender.Reply.SetReply(201, 'Data follows')
else
ASender.Reply.SetReply(202, 'No data');
end;
It is exiting its loop prematurely, leaving the rest of the data unread.
Gambit
 

Thank you for a very good answer!!!
Actually the code that I implemented with an "End of message string" - looks
pretty much like your solution #2.
HOWEVER - I still check the InputBufferIsEmpty - so I assume this is the
problem after all.
My code looks like this (cut from the context - but I think you understand).
Do you think simply removing the check for
ElWebServiceClient.IdTCPClient[indeks].IOHandler.InputBufferIsEmpty will
solve my problem? It is a problem occuring once every 40000 messages or so -
so it is not easy to reproduce...
while not lEndBitOK do
begin
/// Check that buffer is not empty.
if not
ElWebServiceClient.IdTCPClient[indeks].IOHandler.InputBufferIsEmpty then
begin
lEbNo := 0;
/// Read data.
lData := ElWebServiceClient.IdTCPClient[indeks].IOHandler.ReadLn;
if trim(lData) = cElwinMessageEndBit then
lEndBitOk := true
else
begin
lParamListe.Add(lData)
end;
end
else /// Else if input buffer empty and it has been the last 1000
reads (10 seconds) - quit...
if lEbNo>1000 then
begin
logg('Can not find ENDBIT ('+cElwinMessageEndBit+') in message,
giving up...', clogError);
break;
end
else
begin
logg('Endbit not found, waiting for data to be written.',
clogError);
lEbNo := lEbNo+1;
Sleep(10);
Application.ProcessMessages;
end;
Note: It is not stated in the documentation that this is an internal
function. Below is the content of the help documentation for
InputBufferIsEmpty for TidIOHandler:
"Indicates the input buffer for the IOHandler does not contain any data.
Pascal
function InputBufferIsEmpty: Boolean;
Returns
Boolean - True when the input buffer is empty.
Description
InputBufferIsEmpty is a Boolean function used to indicate if the InputBuffer
for the IOHandler contains any unhandled data.
InputBufferIsEmpty uses InputBuffer to determine if the buffer has been
assigned, and to get the existing length of the buffer.
Copyright ?1993-2006, Chad Z. Hower (aka Kudzu) and the Indy Pit Crew. All
rights reserved."
But I belive you when you say so - an will stop using that function :-)
I tried to implement solution #1 that you outlined, but this makes my client
hang - if I understand it correctly I will have to send the number of lines
sendt from the server so that the client know how many lines to read - or is
there some other way of doing it.
I changed that code to something like this to prevent the client from
hanging. Is this the correct way of doing it?
--- server ---
Thanks!
/Arvid.
 

"Arvid Haugen" <XXXX@XXXXX.COM>writes
filled it in. If you are using InputBufferIsEmpty to decide when to read,
then you are getting yourself into a catch-22 situation. 99.999999% of the
time, you do not need to rely on the contents of the InputBuffer at any
given time. Like I said earlier, it is for Indy's internal use. Just read
normally, and Indy will handle the rest for you.
directly at all. Your logic is counter to how Indy is designed to work.
Once you finish reading all of the data from the InputBuffer, you are not
performing any further reads that will re-fill the InputBuffer. Just call
ReadLn() in the loop and forget the rest of the validation logic, it is not
necessary. For example:
while not lEndBitOK do
begin
/// Read data.
lData := ElWebServiceClient.IdTCPClient[indeks].IOHandler.ReadLn;
if Trim(lData) = cElwinMessageEndBit then
lEndBitOk := true
else
lParamListe.Add(lData);
//...
If you really want to use InputBufferIsEmpty(), then you need to use
CheckForDataOnSource() as well, ie:
while not lEndBitOK do
begin
/// Check that buffer is not empty.
if
ElWebServiceClient.IdTCPClient[indeks].IOHandler.InputBufferIsEmpty then
ElWebServiceClient.IdTCPClient[indeks].IOHandler.CheckForDataOnSource(10);
if not
ElWebServiceClient.IdTCPClient[indeks].IOHandler.InputBufferIsEmpty then
begin
lEbNo := 0;
/// Read data.
lData :=
ElWebServiceClient.IdTCPClient[indeks].IOHandler.ReadLn;
if Trim(lData) = cElwinMessageEndBit then
lEndBitOk := true
else
lParamListe.Add(lData);
end
else /// Else if input buffer empty and it has been the last 1000
reads (10 seconds) - quit...
if lEbNo>1000 then
begin
logg('Can not find ENDBIT ('+cElwinMessageEndBit+') in message,
giving up...', clogError);
break;
end
else
begin
logg('Endbit not found, waiting for data to be written.',
clogError);
lEbNo := lEbNo+1;
Sleep(10);
Application.ProcessMessages;
end;
//...
for you automatically. By default, ReadStrings() expects that count to be
present in the socket data (if you specify a Count>=0 as a parameter to
ReadStrings(), it will not be read from the socket).
sending your own manual line count as well (you are also subtracting 1 from
the count, which was wrong to do). In your client code, you are passing
your manual line count (which is 1 off) to ReadString(), which tells it to
not read the line count that Write(TStrings) sent. So if nothing else, your
first string is going to be corrupted because it has extra bytes in it.
If you want to send your own line count (which is perfectly valid to do if
you want to keep your protocol strictly textual - many protocols do), then
you need to set the AWriteLineCount parameter of Write(TSrings) back to
False, which you were originally using, ie:
--- server ---
procedure TForm5.IdTCPServer1Execute(AContext: TIdContext);
//...
begin
// ...
AContext.Connection.IOHandler.Write('201 ' +
IntToStr(LLines.Count));
AContext.Connection.IOHandler.Write(LLines{, False});
// ...
end;
--- client ---
var
LLines: TStrings;
LNumLines: Integer;
begin
Result := IdTCPClient1.SendCmd('100 Test', [201, 202]);
if Result = 201 then
begin
LNumLines := StrToInt(Trim(IdTCPClient1.LastCmdResult.Text[0]));
LLines := TStringList.Create;
try
IdTCPClient1.IOHandler.ReadStrings(LLines, LNumLines);
// loop through LLines as needed...
finally
LLines.Free;
end;
end;
else
//...
end;
Gambit
 

only the code you supplied but the client keeps{*word*154}.
However that is not a big issue. I implemented the RFC formatted code you
outlined. This works fine both in my test app and the real code - and was a
simple rewrite - so I will go with that.
Thank you very much for your help!
 

A little followup question.
I want to make a code that in case I do not get a correct response to my
send command I will clear whatever is in the input buffer.
In case the response from the server is som non-numeric data the "SendCmd"
will throw an exception. I then want to clear whatever is in the buffer. I
know that the code below work - but is this the correct way to do this - or
is the inputbuffer also only for Indy internal use?
try
outResponse := IdTCPClient1.SendCmd('100 Test', [101]);
except
IdTCPClient1.IOHandler.InputBuffer.Clear;
end;
Is there any documentation explaining what functions can be used - and which
ones should not be used?
 

"Arvid Haugen" <XXXX@XXXXX.COM>writes
work just fine when used properly.
Gambit
 

"Arvid Haugen" <XXXX@XXXXX.COM>writes
work. You can clear the InputBuffer itself easy enough (it has a Clear()
method), but you can not erase any unread data that may still be pending on
the socket. You would have to continuously read from the socket until there
is no more data to read. And even then, if the connection is slow, it can
be tricky to determine when the data has ended versus when a timeout has
occured instead. The best thing to do is to simply not send anything at all
when you send an error response back. And format your responses in such a
way that the client always knows exactly how much to read for each
responses, even if it is going to discard the content afterwards.
designed for RFC-compliant responses, which have very rigid designs. If
your responses are formatted correctly, then SendCmd() will read the full
responses data before raising an exception on a reply code that does not
match the specified acceptable code(s).
InputBuffer, and how much remains on the socket.
Gambit