Board index » cppbuilder » Problem with HTTP download locking up.

Problem with HTTP download locking up.


2006-01-24 08:18:31 AM
cppbuilder90
I'm having a problem with a customers computer locking up as soon as a
request
to download a file via http is made. To my knowledge this is the only
customer
with this problem. He's using XP SP2, I'm still waiting for a response to
make
sure a Proxy server isn't needed, but even then I would think it would
simply
throw an exception.
I've tried both TNMHttp and TIdHTTP and it seems to happen at the same
point with both controls..
Because TNMHttp controsl are supposed to be buggy what is below is using
the TId component. I'm using Indy 9.0.50.
The first call I make is to a web page and return the result as a string.
Then I download a small file (22 bytes) this seems to work fine. The next
file that gets downloaded is 24MB and as soon as the Get is called the
program
freezes solid. I do have the AntiFreeze component on the form, and on my
machine
that doesn't lock up it functions just fine with being able to move the
window, click
buttons etc without it appearing to be frozen. On his, it's locked up solid,
can't
move the windows, click the cancel button, nothing.
I have the OnWork event setup to calculate the current progress of the
transfer
using the Http1->Response->ContentStream->Position and the ContentLength
variables. With a Application->ProcessMessages() thrown in to update the
display.
Initially I had it setup in a thread for the downloading with synchronize
calls for updating the display, but rather than lockup, the thread was just
dying without
any errors being thrown.
I'm including the code below for downloading:
<excerpt from function of the code that is not locking up>
if(!Download("www.companyname.com/SmallFile.nfo", InstallDir +
"\\SmallFile.nfo"))
{
Msg = "Unable to download Information File.";
Write_Debug(Msg);
UpdateStatus(); // Adds a line to a TMemo and
Application->ProcessMessages();
DownloadFailed = true;
AbortUpdate = true;
return;
}
<end excerpt>
<excerpt from function of the code that IS locking up>
Msg.sprintf("Downloading Update number %d.", iCtr);
Write_Debug(Msg);
UpdateStatus(Msg);
if(!FileExists(FileName5) && !Download("www.companyname.com/" +
FileName, InstallDir + "\\" + FileName))
{
if(!AbortUpdate)
{
Msg.sprintf("Unable to download update number %d.", iCtr);
Write_Debug(Msg);
UpdateStatus();
}
DownloadFailed = true;
AbortUpdate = true;
return;
}
<end excerpt>
<Code from the Download Function>
bool Download(AnsiString const FileName, AnsiString const FullPath)
{
Retry = 10;
IsDownloading = true;
DLResult = false;
while(IsDownloading && !DLResult && Retry != 0)
{
frmUpdate->StartTime = Now();
try
{
TMemoryStream *ms = new TMemoryStream;
frmUpdate->Http1->Get(FileName, ms);
if(!AbortUpdate)
{
if(FileExists(FullPath))
DeleteFile(FullPath);
ms->SaveToFile(FullPath);
DLResult = true;
}
delete ms;
IsDownloading = false;
}
catch(const EIdSocketError &e)
{
AnsiString Message = "EIdSocketError: " + AnsiString(e.LastError) +
" - " + e.Message;
Write_Debug(Message);
--Retry;
}
catch(const EIdHTTPProtocolException &e)
{
AnsiString Message = "EIdHTTPProtocolException: " +
AnsiString(e.ReplyErrorCode) + " - " + e.ErrorMessage;
Write_Debug(Message);
--Retry;
}
catch(const EIdException &e)
{
AnsiString Message = "EIdException: " + e.Message;
Write_Debug(Message);
--Retry;
}
catch(const Exception &e)
{
AnsiString Message = "Exception: " + e.ClassName() + " - " +
e.Message;
Write_Debug(Message);
--Retry;
}
}
return DLResult;
}
<Code from OnWork Event>
void __fastcall TfrmUpdate::Http1Work(TObject *Sender,
TWorkMode AWorkMode, const int AWorkCount)
{
if(IsDownloading)
{
if(AbortUpdate)
{
Http1->Disconnect();
DLResult = false;
Retry = 0;
Msg = "Download was aborted by the user.";
UpdateStatus();
Write_Debug(Msg);
return;
}
if(Http1->Response->ContentLength>= 1048576)
{
lblFileSize->Caption = lblFileSize->Caption.FormatFloat("0.00",
float(Http1->Response->ContentLength) / 1048576);
Label5->Caption = "MB";
}
else if(Http1->Response->ContentLength>= 1024)
{
lblFileSize->Caption = lblFileSize->Caption.FormatFloat("0.00",
float(Http1->Response->ContentLength) / 1024);
Label5->Caption = "KB";
}
else
{
lblFileSize->Caption = Http1->Response->ContentLength;
Label5->Caption = "Bytes";
}
if(Http1->Response->ContentStream->Position>= 1048576)
{
lblBytesReceived->Caption =
lblBytesReceived->Caption.FormatFloat("0.00",
float(Http1->Response->ContentStream->Position) / 1048576);
Label6->Caption = "MB";
}
else if(Http1->Response->ContentStream->Position>= 1024)
{
lblBytesReceived->Caption =
lblBytesReceived->Caption.FormatFloat("0.00",
float(Http1->Response->ContentStream->Position) / 1024);
Label6->Caption = "KB";
}
else
{
lblBytesReceived->Caption = Http1->Response->ContentStream->Position;
Label6->Caption = "Bytes";
}
TDateTime CurrentTime = Now();
double NumSecs = SecondSpan(CurrentTime, frmUpdate->StartTime);
double BytesLeft = Http1->Response->ContentLength -
Http1->Response->ContentStream->Position;
float TimeLeft;
if(Http1->Response->ContentStream->Position != 0)
TimeLeft = BytesLeft / (Http1->Response->ContentStream->Position /
NumSecs);
else
TimeLeft = BytesLeft / (1 / NumSecs);
if(TimeLeft>= 3600)
{
int Hours = (int)(TimeLeft / 3600);
int Minutes = (int)((TimeLeft - (Hours * 3600)) / 60);
int Seconds = (int)((TimeLeft - (Hours * 3600) - (Minutes * 60)));
lblTimeRemaining->Caption.sprintf("%d hrs %d mins %d secs", Hours,
Minutes, Seconds);
}
else if(TimeLeft>= 60)
{
int Minutes = (int)(TimeLeft/ 60);
int Seconds = (int)((TimeLeft - (Minutes * 60)));
lblTimeRemaining->Caption.sprintf("%d mins %d secs", Minutes, Seconds);
}
else
frmUpdate->lblTimeRemaining->Caption.sprintf("%d secs", (int)TimeLeft);
Application->ProcessMessages();
}
}
Here is the contents of the Log file that Write Debug writes to.
2006-01-23 7:54:27 AM : Downloading Update Infomation File
2006-01-23 7:54:27 AM : Information file downloaded sucessfully.
2006-01-23 7:54:27 AM : Old Information file opened. Old AM Version: 77. Old
Updater Version: 6. Last Update Number: 5.
2006-01-23 7:54:27 AM : New Information file opened. New AM Version: 80. New
Updater Version: 7. New Update Number: 8.
2006-01-23 7:54:27 AM : New version of Updater.exe found, downloading it!
2006-01-23 7:54:30 AM : Downloading Update number 6.
If anymore information needs to be provided please let me know.
Thanks,
Tom
 
 

Re:Problem with HTTP download locking up.

"Tom" < XXXX@XXXXX.COM >wrote in message
Quote
The next file that gets downloaded is 24MB and as soon as the
Get is called the program freezes solid.
Are you calling Get() in the context of the main thread? If so, then the
freeze is to be expected.
Quote
I do have the AntiFreeze component on the form, and on my machine
that doesn't lock up it functions just fine with being able to move the
window, click buttons etc without it appearing to be frozen.
TIdAntiFreeze is not perfect. A better solution would be to move the
downloading into a separate worker thread instead.
Quote
On his, it's locked up solid, can't move the windows, click the cancel
button, nothing.
Then TIdHTTP (or something else in your code) has been completely blocked
Quote
to the point where TIdHTTP cannot even call into TIdAntiFreeze at all.
Again, you should move the download into a separate thread. At least that
way, the program will not be frozen even if such a blockage occurs again.
Quote
I have the OnWork event setup to calculate the current progress
of the transfer using the Http1->Response->ContentStream->Position
and the ContentLength variables.
The ContentStream->Position is always at the end of the stream You should
be using the AWorkCount value that is provided by the OnWork event instead.
That is the total number of bytes that have been transferred since the
beginning of the transfer.
Quote
With a Application->ProcessMessages() thrown in to update
the display.
TIdAntiFreeze calls ProcessMessages() internally. Or, if you move the code
into its own thread, then ProcessMessages() is not needed at all.
Quote
Initially I had it setup in a thread for the downloading with
synchronize calls for updating the display, but rather than lockup,
the thread was just dying without any errors being thrown.
Then your thread code was likely wrong to begin with. Please show what you
were doing. Also keep in mind that TThread swallows uncaught exceptions, so
if you want an exception to be reported, you have to catch the exceptions
yourself so that you can respond to them accordingly.
Quote
bool Download(AnsiString const FileName, AnsiString const FullPath)
That code has a memory leak if either TIdHTTP::Get() or
TMemoryStream::SaveToFile() fail. You are also doing a lot of processing
inside the OnWork event handler. You might consider streamlining the code
better to reduce the numbr of placesd where errors can occur.
Gambit
 

Re:Problem with HTTP download locking up.

I got some more information from the user after adding
some more logging in the Connect,Disconnect and WorkBegin
events.
It's calling all of the above before locking up.
Here's the latest text from the log file:
2006-01-23 8:11:24 PM : Downloading Update Infomation File
2006-01-23 8:11:24 PM : Connect to server.
2006-01-23 8:11:24 PM : Beginning Download
2006-01-23 8:11:24 PM : Download Completed
2006-01-23 8:11:24 PM : Disconnected from server.
2006-01-23 8:11:24 PM : Information file downloaded sucessfully.
2006-01-23 8:11:24 PM : Old Information file opened. Old AM Version: 77. Old
Updater Version: 6. Last Update Number: 5.
2006-01-23 8:11:24 PM : New Information file opened. New AM Version: 80. New
Updater Version: 7. New Update Number: 8.
2006-01-23 8:11:24 PM : New version of Updater.exe found, downloading it!
2006-01-23 8:11:24 PM : Connect to server.
2006-01-23 8:11:24 PM : Beginning Download
2006-01-23 8:11:39 PM : Download Completed
2006-01-23 8:11:39 PM : Disconnected from server.
2006-01-23 8:11:39 PM : Downloading Update number 6.
2006-01-23 8:11:39 PM : Connect to server.
2006-01-23 8:11:40 PM : Beginning Download
-Tom
 

{smallsort}

Re:Problem with HTTP download locking up.

Quote
>Are you calling Get() in the context of the main thread? If so, then the
>freeze is to be expected.
It is if I wasn't calling ProcessMessages() in the OnWork event.
I plan on moving all of that work out of the event, it was just a quick
conversion from TNMHttp over to TIdHTTP to see if that could correct
the problem with it freezing.
Quote
>The ContentStream->Position is always at the end of the stream
Not true, it updates as the data is received. I'm using this
method because of another post by you that says AWorkCount
is not accurate if the data is sent in different chunks. From my
testing the ContentStream->Size (which is what you said
to use in that other post) is set to the same size as the ContentLength.
when the transfer is started. The Position updates as that data is received.
Quote
>Then your thread code was likely wrong to begin with.
It was the same code except it was inside a Tthread. The UpdateStatus
function was synchronized for access to the main forms visual controls.
DLResult, IsDownloading and Retry all were public variables of the thread.
Quote
>That code has a memory leak if either TIdHTTP::Get() or
>TMemoryStream::SaveToFile() fail.
Would an auto_ptr take care of that? I'm still looking for a good place
to learn about them.
Thanks,
Tom
"Remy Lebeau (TeamB)" < XXXX@XXXXX.COM >wrote in message
Quote

"Tom" < XXXX@XXXXX.COM >wrote in message
news:43d571a6$ XXXX@XXXXX.COM ...

>The next file that gets downloaded is 24MB and as soon as the
>Get is called the program freezes solid.

Are you calling Get() in the context of the main thread? If so, then the
freeze is to be expected.

>I do have the AntiFreeze component on the form, and on my machine
>that doesn't lock up it functions just fine with being able to move the
>window, click buttons etc without it appearing to be frozen.

TIdAntiFreeze is not perfect. A better solution would be to move the
downloading into a separate worker thread instead.

>On his, it's locked up solid, can't move the windows, click the cancel
button, nothing.

Then TIdHTTP (or something else in your code) has been completely blocked
>to the point where TIdHTTP cannot even call into TIdAntiFreeze at all.
Again, you should move the download into a separate thread. At least that
way, the program will not be frozen even if such a blockage occurs again.

>I have the OnWork event setup to calculate the current progress
>of the transfer using the Http1->Response->ContentStream->Position
>and the ContentLength variables.

The ContentStream->Position is always at the end of the stream You should
be using the AWorkCount value that is provided by the OnWork event
instead.
That is the total number of bytes that have been transferred since the
beginning of the transfer.

>With a Application->ProcessMessages() thrown in to update
>the display.

TIdAntiFreeze calls ProcessMessages() internally. Or, if you move the
code
into its own thread, then ProcessMessages() is not needed at all.

>Initially I had it setup in a thread for the downloading with
>synchronize calls for updating the display, but rather than lockup,
>the thread was just dying without any errors being thrown.

Then your thread code was likely wrong to begin with. Please show what
you
were doing. Also keep in mind that TThread swallows uncaught exceptions,
so
if you want an exception to be reported, you have to catch the exceptions
yourself so that you can respond to them accordingly.

>bool Download(AnsiString const FileName, AnsiString const FullPath)

That code has a memory leak if either TIdHTTP::Get() or
TMemoryStream::SaveToFile() fail. You are also doing a lot of processing
inside the OnWork event handler. You might consider streamlining the code
better to reduce the numbr of placesd where errors can occur.


Gambit


 

Re:Problem with HTTP download locking up.

Sorry, I meant to cut out the extra text at the bottom.
-Tom
 

Re:Problem with HTTP download locking up.

To be more specific I guess. What would cause
it to lock up like that for 1 person and not the
rest including me?
-Tom
 

Re:Problem with HTTP download locking up.

"Tom" < XXXX@XXXXX.COM >wrote in message
Quote
Here's the latest text from the log file:
Logs are useless without seeing the actual code that is producing them.
Even more useful would be a log of the raw socket activity, not the
application's higher-level behaviors. If you attach one of the TIdLog...
components to TIdHTTP, or alternatively use an external packet sniffer such
as Ethereal (www.ethereal.com), then youcan view the actual HTTP
commands and responses that are being transmitted over the socket.
Gambit
 

Re:Problem with HTTP download locking up.

I'll try the TIdLog and see if I can get the customer
to give it another go.
-Tom
"Remy Lebeau (TeamB)" < XXXX@XXXXX.COM >wrote in message
Quote

"Tom" < XXXX@XXXXX.COM >wrote in message
news: XXXX@XXXXX.COM ...

>Here's the latest text from the log file:

Logs are useless without seeing the actual code that is producing them.
Even more useful would be a log of the raw socket activity, not the
application's higher-level behaviors. If you attach one of the TIdLog...
components to TIdHTTP, or alternatively use an external packet sniffer
such
as Ethereal (www.ethereal.com), then youcan view the actual HTTP
commands and responses that are being transmitted over the socket.


Gambit



 

Re:Problem with HTTP download locking up.

Tom wrote:
Quote
I'll try the TIdLog and see if I can get the customer
to give it another go.
Any updates on how this worked out?
--
Mark Jacobs
www.dkcomputing.co.uk
 

Re:Problem with HTTP download locking up.

I ended up giving the customer the alternative of being
able to manually download the files or continue to
work with me on the problem.
He chose to manually download the patch files rather
than to continue to try and figure out the problem.
It's one customer out of 1000+ so unless another
customer has the same problem and is willing to work
with me it will remain one of those unknowns...
-Tom
"Mark Jacobs" < XXXX@XXXXX.COM >wrote in message
Quote
Tom wrote:
>I'll try the TIdLog and see if I can get the customer
>to give it another go.

Any updates on how this worked out?
--
Mark Jacobs
www.dkcomputing.co.uk