fclose and CreateFile are very slow on some machines

Post by Rahul
Hi All,
My application writes data in a file using fopen/fwrite/fclose and
immediately after fclose I try to open the same file using. the
windows CreateFile() function.
In one of my machine all are very fast and the total process finishes
in < 4 seconds.
But on one machine fclose itself takes around 15 seconds, I tried
using setbuf(fp, NULL) and calling fflush(fp) after every write, but
then also fclose took 15 seconds. I also tried using the low level
functions _open, _close etc. but there also the _close took 15 second
of time.
What is more strange to me is that there is one more machine on which
fopen/fwrite/fclose works quite fast but the last CreateFile() call
takes > 15 seconds to open the same file.
Is thee any way of debugging such machine specific issues. All the
machine have same OS (win XP SP3 and exactly the same windows
updates). The only difference is in the hardware i.e. CPU, hard disk,
RAM configuration all are different. But I could not find any clue
about where to look at.
Could somebody please tell me about how do I go debugging such
problems, Is the delay caused by the OS or the Hard disk device driver
or something else?
Thanks in advance
Rahul

Are you using local disks in all computers ? Then you could check
disk throughput with hdtune .

br
ismo

Rahul

2010-02-18 13:31:27 UTC

Post by Ismo Salonen

Are you using local disks in all computers ? Then you could check
disk throughput with hdtune .
br
ismo

Hi Ismo,

I checked dis throughput using DskSpeed, there was not much difference
in that

Francois PIETTE

2010-02-18 12:51:15 UTC

Post by Rahul
My application writes data in a file using fopen/fwrite/fclose and
immediately after fclose I try to open the same file using. the
windows CreateFile() function.
In one of my machine all are very fast and the total process finishes
in < 4 seconds.
But on one machine fclose itself takes around 15 seconds, I tried
using setbuf(fp, NULL) and calling fflush(fp) after every write, but
then also fclose took 15 seconds. I also tried using the low level
functions _open, _close etc. but there also the _close took 15 second
of time.
What is more strange to me is that there is one more machine on which
fopen/fwrite/fclose works quite fast but the last CreateFile() call
takes > 15 seconds to open the same file.

--
***@overbyte.be
The author of the freeware multi-tier middleware MidWare
The author of the freeware Internet Component Suite (ICS)
http://www.overbyte.be

Rahul

2010-02-18 13:34:23 UTC

Look for so called "security product". For example an antivirus program may
be buggy and tak a lot of time processing the data.
Also have a look at disk fragmentation and bad sectors.
On the computer experiencing slow response, does other software suffer from
the same problem or is this specific to one application ?
--
The author of the freeware multi-tier middleware MidWare
The author of the freeware Internet Component Suite (ICS)http://www.overbyte.be

Hi Francois,

I do not know about other softwares, but the disk is highly
fragmented. however for small files the fclose/ CreateFile works
pretty fast. the problem happens whiel trying to write > 5 MB files.
and I always disable the antivirus before doing any performance test
on the machines.

Thanks & Regards

2010-02-18 23:33:09 UTC

IMHO, many anti-virus programs are worse than the viruses they are meant to
defend against. Usually, the 'disabled' mode is just an auto-allow all
mode, and the performance penalties still apply. Some products don't even
uninstall cleanly and the only way to remove them is to reinstall the OS.

But besides the possibility of a bad AV program, here are a few other things
to consider:
- check the disk type (ie SAS, SATA, IDE, SCSI etc.) since some of the
in-box drivers (esp IDE) are known to cause performance issues
- check the controller and bus topology of the machines. Many machines will
have SATA disks connected using the AHCI controller built into the Intel
supplied south bridge, but if there are several bridges in play, or other
non-standard HW, then there may be driver or HW effects that come into play
- does defragmenting the disk, or writing to another clean disk on this
machine affect the result? If the available space is low, and the FS
heavily fragmented, then performance will suffer
- Are the disks are formatted with NTFS, or is there FAT32? While XP can
work with FAT, the performance will differ significantly in many cases
- are the versions of the CRT in use on all machines identical (as they
would be if it were statically linked)?

Also, in the interval while it appears to hang, do the process IO counters
increase? Do the system process IO counters increase? This info will help
you to determine if the delay that you are seeing is caused by cache
flushing or something else.

Hi Francois,
I do not know about other softwares, but the disk is highly
fragmented. however for small files the fclose/ CreateFile works
pretty fast. the problem happens whiel trying to write > 5 MB files.
and I always disable the antivirus before doing any performance test
on the machines.
Thanks & Regards

Tom Serface

2010-02-19 05:25:44 UTC

If you are doing the same test with fopen vs. CreateFile on the same machine
under the same conditions I would expect CreateFile to be faster if anything
at all. It might be in the way you are opening CreateFile. Maybe you could
list the code where you do the call to open the file...

Tom

Rahul

2010-02-19 05:51:48 UTC

Post by Tom Serface
If you are doing the same test with fopen vs. CreateFile on the same machine
under the same conditions I would expect CreateFile to be faster if anything
at all. It might be in the way you are opening CreateFile. Maybe you could
list the code where you do the call to open the file...
Tom

Hi Francois,
I do not know about other softwares, but the disk is highly
fragmented. however for small files the fclose/ CreateFile works
pretty fast. the problem happens whiel trying to write > 5 MB files.
and I always disable the antivirus before doing any performance test
on the machines.
Thanks & Regards

The code is as follows

FILE * fp = fopen(fileName, "wb");
// a for loop for writing data into fp;
fclose(fp);
......
// in another function immediately after the above one
CreateFile (fileName,
GENERIC_READ,
FILE_SHARE_READ,
NULL,
createDist,
FILE_ATTRIBUTE_NORMAL | FILE_FLAG_SEQUENTIAL_SCAN ,
NULL );

On some machine the fclose is slow and on some the CreateFile. I guess
the calls are slow because the process is waiting for the IO to
complet.
I put a log message just after fclose() and as soon as the message
appeared in the log I tried kiling my application from task manager.
But it did not get killed immediately though I kept trying again an
again. It took around 15 seconds before it got killed and disappeared
from Taskmanager.
I guess the OS was flushing the buffers to the disk and hence was not
allowing the task to get killed, as soon as the bufferes were flushed
the task got killed.

But why does the flushing behavior differs significantly across
systems when the CRT is statically linked.

I will check those harddrive driver and other configurations also.

Thanks & Regards

Cholo Lennon

2010-02-19 13:08:37 UTC

Hi Francois,
I do not know about other softwares, but the disk is highly
fragmented. however for small files the fclose/ CreateFile works
pretty fast. the problem happens whiel trying to write > 5 MB files.
and I always disable the antivirus before doing any performance test
on the machines.
Thanks & Regards

The code is as follows
FILE * fp = fopen(fileName, "wb");
// a for loop for writing data into fp;
fclose(fp);
......
// in another function immediately after the above one
CreateFile (fileName,
GENERIC_READ,
FILE_SHARE_READ,
NULL,
createDist,
FILE_ATTRIBUTE_NORMAL | FILE_FLAG_SEQUENTIAL_SCAN ,
NULL );
On some machine the fclose is slow and on some the CreateFile. I guess
the calls are slow because the process is waiting for the IO to
complet.
I put a log message just after fclose() and as soon as the message
appeared in the log I tried kiling my application from task manager.
But it did not get killed immediately though I kept trying again an
again. It took around 15 seconds before it got killed and disappeared
from Taskmanager.
I guess the OS was flushing the buffers to the disk and hence was not
allowing the task to get killed, as soon as the bufferes were flushed
the task got killed.
But why does the flushing behavior differs significantly across
systems when the CRT is statically linked.
I will check those harddrive driver and other configurations also.

As M said, check hardware configuration, especially if you have an IDE
drive. It's possible that you have a drive with DMA disabled (go to
Control Panel/System/Device Manager/IDE controllers)

Post by Rahul
Thanks & Regards

Regards

--
Cholo Lennon
Bs.As.
ARG

Francois PIETTE

2010-02-20 09:22:30 UTC

Post by Francois PIETTE
Look for so called "security product". For example an antivirus program may
be buggy and tak a lot of time processing the data.
Also have a look at disk fragmentation and bad sectors.

I do not know about other softwares, but the disk is highly
fragmented.

Defragment the disk is then mandatory.
Having a too much fragmented disk not only result in poor performance but
may result in a disk which can't no more be defragmented with defrag tool
and require a full backup/formet/rerstore cycle.

Post by Rahul
On the computer experiencing slow response, does other software suffer from
the same problem or is this specific to one application ?

You have not answered this one.

--
***@overbyte.be
The author of the freeware multi-tier middleware MidWare
The author of the freeware Internet Component Suite (ICS)
http://www.overbyte.be

Hector Santos

2010-02-20 14:01:28 UTC

It is also quite possible these are early signs the drive is going
bad. I once lived with a drive that only OE seem to have a problem
with. It got to a point where trying to read mail slowed down the
disk. Over time, other it was slow to I/O and only under certain
apps. Eventually the GPF began to occur with OE only. Running Steve
Gibson's SpinRite found problems, marked out the clusters, and it
would last longer. But eventually it kaputs and storing anything else
need the same spots was no go.