Is Wininet the only choice for FTP?

[Bogdan]

Hi,

My app needs to download files from an ftp server. I've been using wininet
apis with no problems for a year or so. Recently I've run into a problem on
one of the machines. The app is unable to download large files. It always
times out on the last block. For example, if the file is ~300MB then the
wininet apis will timeout on the last 3-4K.
I tried FtpGetFile() and FtpOpenFile()/InternetReadFile() and got the same
results.

The command line ftp works fine with the same file/server so the problem
must be in wininet.

I'm afraid that this problem might come up on other new installs.

Is wininet the only option for C/C++? I noticed that most of the posts
about wininet on the Net are 5 or more years old. The
win32.programmer.networks group is almost dead too. What would be the
recommended api for ftp nowadays?

Any active newsgroups/blogs that deal with the subject?

I'd appreciate _any_ suggestions.
Thanks,
Bogdan

[Tom Handal]

Hi,
My app needs to download files from an ftp server.  I've been using wininet
apis with no problems for a year or so.  Recently I've run into a problem on
one of the machines.  The app is unable to download large files.  It always
times out on the last block.  For example, if the file is ~300MB then the
wininet apis will timeout on the last 3-4K.
I tried FtpGetFile() and FtpOpenFile()/InternetReadFile() and got the same
results.
The command line ftp works fine with the same file/server so the problem
must be in wininet.
I'm afraid that this problem might come up on other new installs.
Is wininet the only option for C/C++?  I noticed that most of the posts
about wininet on the Net are 5 or more years old.  The
win32.programmer.networks group is almost dead too.  What would be the
recommended api for ftp nowadays?
Any active newsgroups/blogs that deal with the subject?
I'd appreciate _any_ suggestions.
Thanks,
Bogdan

You could always try using a third-party FTP library. Something like
http://sourceforge.net/projects/libftp/

Tom

[Pavel A.]

Post by Bogdan
Hi,
My app needs to download files from an ftp server. I've been using
wininet apis with no problems for a year or so. Recently I've run into a
problem on one of the machines. The app is unable to download large
files. It always times out on the last block. For example, if the file
is ~300MB then the wininet apis will timeout on the last 3-4K.
I tried FtpGetFile() and FtpOpenFile()/InternetReadFile() and got the same
results.
The command line ftp works fine with the same file/server so the problem
must be in wininet.
I'm afraid that this problem might come up on other new installs.
Is wininet the only option for C/C++? I noticed that most of the posts
about wininet on the Net are 5 or more years old. The
win32.programmer.networks group is almost dead too. What would be the
recommended api for ftp nowadays?
Any active newsgroups/blogs that deal with the subject?
I'd appreciate _any_ suggestions.
Thanks,
Bogdan

Yep, no one here. Everybody got sucked into the cloud...
Have you tried to debug the ftp failure with a sniffer, if is so easily
reproes?

--pa

[Darko Miletic]

I suggest you try out libcURL

http://curl.haxx.se/

wininet is probably not the best choice to begin with.

[Brian Muth]

We have used Wininet to FTP very large files, and have never had any
trouble. I'm not convinced that there is a hitherto unknown bug in your
program. I think it is worth troubleshooting with a debugger and a sniffer
before giving up on Wininet.

Brian

[Bogdan]

Post by Brian Muth
We have used Wininet to FTP very large files, and have never had any
trouble. I'm not convinced that there is a hitherto unknown bug in your
program. I think it is worth troubleshooting with a debugger and a sniffer
before giving up on Wininet.
Brian

Thanks Brian. This gives me some comfort and hope.

I'm quite certain that the code is OK. In its simplest form it is just one
line of code which is a call to FtpGetFile(). I can see that after the call
is issued the file on the local disk keeps growing until it almost reaches
its size. For example, if I try to transfer 150 MB file then the file on
the local disk gets pretty close to 150MB (minus 1-2K) and then FtpGetFile()
returns false.

I finally managed to put WireShark (as suggested by Pavel) on the
problematic machine. I discovered that at the end of file transfer both
command line ftp and my app run into identical problems. The difference is
that command line ftp is able to recover somehow but FtpGetFile() is not.

This is what happens after about 180K packets:
1) servers sends "TCP Previous segment lost"
2) client responds with ACK
3) server sends "TCP Out-of-order"
4) client responds with ACK
5) client sends FIN,ACK

The above is identical for ftp.exe and FtpGetFile(), including values in Ack
fields. The next exchange is what differentiates ftp from FtpGetFile().

ftp.exe:
6) server sends "226 Transfer complete"
7) server sends ACK
8) client sends ACK

FtpGetFile():
6) server sends ACK (identical to 7 for ftp.exe above)
7) client sends FIN,ACK
... nothing happens for 30 seconds
9) client sends FIN,ACK
10) server responds with reset (RST)


It looks like FtpGetFile() expected "226 Transfer complete" in step 6) and
got ACK instead. This caused it to response with FIN/ACK.

Would you know what could be causing the above? Or, do you know of any
newsgroups/blogs that I can post this problem to?

Thanks,
Bogdan

[Pavel A.]

I have a similar symptom with "Live Mail" on XP SP3 - a misterious timeout
when sending mail
with attachment. But it does not always reproduce.
When I got around to install a sniffer, it now works fine...

The appropriate NGs for this are microsoft.public.win32.programmer.networks,
microsoft.public.windowsxp.network_web, maybe
microsoft.public.windows.vista.networking_sharing.

Of course, not this one.

Regards,
--pa

Post by Bogdan

Post by Brian Muth
We have used Wininet to FTP very large files, and have never had any
trouble. I'm not convinced that there is a hitherto unknown bug in your
program. I think it is worth troubleshooting with a debugger and a
sniffer before giving up on Wininet.
Brian

Thanks Brian. This gives me some comfort and hope.
I'm quite certain that the code is OK. In its simplest form it is just one
line of code which is a call to FtpGetFile(). I can see that after the
call is issued the file on the local disk keeps growing until it almost
reaches its size. For example, if I try to transfer 150 MB file then the
file on the local disk gets pretty close to 150MB (minus 1-2K) and then
FtpGetFile() returns false.
I finally managed to put WireShark (as suggested by Pavel) on the
problematic machine. I discovered that at the end of file transfer both
command line ftp and my app run into identical problems. The difference
is that command line ftp is able to recover somehow but FtpGetFile() is
not.
1) servers sends "TCP Previous segment lost"
2) client responds with ACK
3) server sends "TCP Out-of-order"
4) client responds with ACK
5) client sends FIN,ACK
The above is identical for ftp.exe and FtpGetFile(), including values in
Ack fields. The next exchange is what differentiates ftp from
FtpGetFile().
6) server sends "226 Transfer complete"
7) server sends ACK
8) client sends ACK
6) server sends ACK (identical to 7 for ftp.exe above)
7) client sends FIN,ACK
... nothing happens for 30 seconds
9) client sends FIN,ACK
10) server responds with reset (RST)
It looks like FtpGetFile() expected "226 Transfer complete" in step 6) and
got ACK instead. This caused it to response with FIN/ACK.
Would you know what could be causing the above? Or, do you know of any
newsgroups/blogs that I can post this problem to?
Thanks,
Bogdan

[Bogdan]

Thanks again Pavel for your reply.

I got it working on the problematic machine by replacing an older version of
Linksys router with a new Trendnet (nothing wrong with Linksys, I think,
just its age).

I'm still puzzled by that whole experience because I do not understand why
one app can work fine with a 'faulty' router and another cannot. This would
indicate that wininet API has some weaknesses. But, it is free and very
easy to use so I think I can live with it.

When it comes to NGs... The microsoft.public.win32.programmer.networks is
almost dead. Other that you have suggested - your 'maybe' is a bit
discouraging :).

This newsgroup seems to be most active so posting an issue about
wininet/FtpGetFile() seemed like a best bet. And it worked for me!

Many thanks to everyone who responded by my post.
Bogdan

Post by Pavel A.
I have a similar symptom with "Live Mail" on XP SP3 - a misterious timeout
when sending mail
with attachment. But it does not always reproduce.
When I got around to install a sniffer, it now works fine...
The appropriate NGs for this are
microsoft.public.win32.programmer.networks,
microsoft.public.windowsxp.network_web, maybe
microsoft.public.windows.vista.networking_sharing.
Of course, not this one.
Regards,
--pa

Post by Bogdan

Post by Brian Muth
We have used Wininet to FTP very large files, and have never had any
trouble. I'm not convinced that there is a hitherto unknown bug in your
program. I think it is worth troubleshooting with a debugger and a
sniffer before giving up on Wininet.
Brian

Thanks Brian. This gives me some comfort and hope.
I'm quite certain that the code is OK. In its simplest form it is just
one line of code which is a call to FtpGetFile(). I can see that after
the call is issued the file on the local disk keeps growing until it
almost reaches its size. For example, if I try to transfer 150 MB file
then the file on the local disk gets pretty close to 150MB (minus 1-2K)
and then FtpGetFile() returns false.
I finally managed to put WireShark (as suggested by Pavel) on the
problematic machine. I discovered that at the end of file transfer both
command line ftp and my app run into identical problems. The difference
is that command line ftp is able to recover somehow but FtpGetFile() is
not.
1) servers sends "TCP Previous segment lost"
2) client responds with ACK
3) server sends "TCP Out-of-order"
4) client responds with ACK
5) client sends FIN,ACK
The above is identical for ftp.exe and FtpGetFile(), including values in
Ack fields. The next exchange is what differentiates ftp from
FtpGetFile().
6) server sends "226 Transfer complete"
7) server sends ACK
8) client sends ACK
6) server sends ACK (identical to 7 for ftp.exe above)
7) client sends FIN,ACK
... nothing happens for 30 seconds
9) client sends FIN,ACK
10) server responds with reset (RST)
It looks like FtpGetFile() expected "226 Transfer complete" in step 6)
and got ACK instead. This caused it to response with FIN/ACK.
Would you know what could be causing the above? Or, do you know of any
newsgroups/blogs that I can post this problem to?
Thanks,
Bogdan

[barbara]

Bogdan wrote on 12/11/2009 12:46 ET

"Brian Muth" wrote in messag
news

Post by Brian Muth
We have used Wininet to FTP very large files, and have never had an
trouble. I'm not convinced that there is a hitherto unknown bug in you
program. I think it is worth troubleshooting with a debugger and a sniffe
before giving up on Wininet
Bria

Thanks Brian. This gives me some comfort and hope
I'm quite certain that the code is OK. In its simplest form it is just on
line of code which is a call to FtpGetFile(). I can see that after the cal
is issued the file on the local disk keeps growing until it almost reache
its size. For example, if I try to transfer 150 MB file then the file o
the local disk gets pretty close to 150MB (minus 1-2K) and then FtpGetFile(
returns false
I finally managed to put WireShark (as suggested by Pavel) on th
problematic machine. I discovered that at the end of file transfer bot
command line ftp and my app run into identical problems. The difference i
that command line ftp is able to recover somehow but FtpGetFile() is not
This is what happens after about 180K packets
1) servers sends "TCP Previous segment lost
2) client responds with AC
3) server sends "TCP Out-of-order
4) client responds with AC
5) client sends FIN,AC
The above is identical for ftp.exe and FtpGetFile(), including values in Ac
fields. The next exchange is what differentiates ftp from FtpGetFile()
ftp.exe
6) server sends "226 Transfer complete
7) server sends AC
8) client sends AC
FtpGetFile()
6) server sends ACK (identical to 7 for ftp.exe above
7) client sends FIN,AC
... nothing happens for 30 second
9) client sends FIN,AC
10) server responds with reset (RST
It looks like FtpGetFile() expected "226 Transfer complete" in ste
6) an
got ACK instead. This caused it to response with FIN/ACK
Would you know what could be causing the above? Or, do you know of an
newsgroups/blogs that I can post this problem to
Thanks
Bogda

Hello Bogdan, I see you had this problem back in 2009, I am having a
identica
problem now, did you ever resolve it? Any help would be greatly apprecitated
I've been struggling with this for a couple weeks now!