WaitForMultipleObjects returning ERROR_INVALID_HANDLE

[Felix]

I have a problem with WaitForMultipleObjects sometimes (not allways)
returning ERROR_INVALID_HANDLE.
Printing the array of handles in question did not show any unusual thing,
i.e. all handles retain their values, no matter if the error occures or
not.
Is there any funktion I can call (like IsHandleValid) that would tell me if
a handle is valid or invalid? Or is there any value I can compare my
handles to, that would tell me if the handle is valid or not?


many thanks, Felix

[Alex Blekhman]

Post by Felix
I have a problem with WaitForMultipleObjects sometimes
(not allways)
returning ERROR_INVALID_HANDLE.
Printing the array of handles in question did not show any
unusual thing,
i.e. all handles retain their values, no matter if the
error occures or
not.

What did you expect? The value itself cannot be definitive
indicatiuon of handle's validity. Value of a handle is jsut
a pointer to some data in kernel space. By the same analogy
with dangling pointer a handle can contain plausible value,
while being invalid.

Post by Felix
Is there any funktion I can call (like IsHandleValid) that
would tell me if
a handle is valid or invalid? Or is there any value I can
compare my
handles to, that would tell me if the handle is valid or
not?

First of all you can compare with `INVALID_HANDLE_VALUE'.
Then with NULL. Then you can call `GetHandleInformation'.
However, the final answer is in the documentation of a
function, which returned the handle.

Alex

[Felix]

Post by Alex Blekhman

Post by Felix
I have a problem with WaitForMultipleObjects sometimes
(not allways)
returning ERROR_INVALID_HANDLE.
Printing the array of handles in question did not show any
unusual thing,
i.e. all handles retain their values, no matter if the
error occures or
not.

What did you expect? The value itself cannot be definitive
indicatiuon of handle's validity. Value of a handle is jsut
a pointer to some data in kernel space. By the same analogy
with dangling pointer a handle can contain plausible value,
while being invalid.

Post by Felix
Is there any funktion I can call (like IsHandleValid) that
would tell me if
a handle is valid or invalid? Or is there any value I can
compare my
handles to, that would tell me if the handle is valid or
not?

First of all you can compare with `INVALID_HANDLE_VALUE'.
Then with NULL. Then you can call `GetHandleInformation'.
However, the final answer is in the documentation of a
function, which returned the handle.
Alex

I did not expect anything from just looking at the handle value, that's
exactly the point. As I already stated: the handle values do not 'look
bad', i.e. have values of either NULL or even INVALID_HANDLE_VALUE (this
was of course the first thing I checked).

The last point however is quite strange: calling 'GetHandleInformation'
for each handle in the array passed to 'WaitForMultipleObjects' (which
failed) does work fine and returns with success. Why does
'GetHandleInformation' not fail for at least one handle passed to
'WaitForMultipleObjects'?

Looking at these facts I am not sure anymore if 'WaitForMultipleObjects'
tells me the truth: does the error code 'ERROR_INVALID_HANDLE' really
mean that a handle from the array I passed to 'WaitForMultipleObjects'
was invalid or might this error refer to some other (which?) handle?

Felix

[Alex Blekhman]

Post by Felix
The last point however is quite strange: calling
'GetHandleInformation'
for each handle in the array passed to
'WaitForMultipleObjects' (which
failed) does work fine and returns with success. Why does
'GetHandleInformation' not fail for at least one handle
passed to
'WaitForMultipleObjects'?

I can think of two problems:

1. Some handle is not supposed to be waited on. See waitable
object list in documentation for `WaitForMultipleObjects'.

2. There is a race condition and some handle is closed
during a call to `WaitForMultipleObjects'.

As an experiment try to call `WaitForMultipleObjects' (or
`WaitForSingleObject') in a loop with one handle at a time
to figure out which handle is invalid.

Alex

[Felix]

Post by Alex Blekhman
1. Some handle is not supposed to be waited on. See waitable
object list in documentation for `WaitForMultipleObjects'.

I am passing just handles to events, so that should be fine.

Post by Alex Blekhman
2. There is a race condition and some handle is closed
during a call to `WaitForMultipleObjects'.
As an experiment try to call `WaitForMultipleObjects' (or
`WaitForSingleObject') in a loop with one handle at a time
to figure out which handle is invalid.

This one sounds very promising...
If I add a second, identical call to 'WaitForMultipleObjects' just after
the first one has failed, everything runs fine. To make it very clear: the
first call to 'WaitForMultipleObjects' sometimes fails; if it does, I just
issue _one_ other call with _identical_ parameters. This second call
_always_ succeeds.

'Unfortunately' the second call always succeds so I am not able to find the
offending handle since there is no more error.

I totally agree that there must be a race condition, I am just a little bit
out of ideas about how to track it down, as 'WaitForMultipleObjects' does
not precisely tell me which handle is 'temporarily' invalid and the second
call to 'WaitForMultipleObjects' does succeed. Any more good ideas about
how to catch this bug without rewriting the entire code? Is there some tool
I could run that would help me find the race condition without eliminating
it just from running?

Felix

[Alex Blekhman]

Post by Felix
I totally agree that there must be a race condition, I am
just a little bit
out of ideas about how to track it down, as
'WaitForMultipleObjects' does
not precisely tell me which handle is 'temporarily'
invalid and the second
call to 'WaitForMultipleObjects' does succeed. Any more
good ideas about
how to catch this bug without rewriting the entire code?
Is there some tool
I could run that would help me find the race condition
without eliminating
it just from running?

I'm not sure whether there is any tool that will detect race
condition. Unfortunately, the most feasible solution is to
look at the copde and try to cut down pieces until the
problem disappears.

Alex

[Felix Brack]

I'm not sure whether there is any tool that will detect race condition.
Unfortunately, the most feasible solution is to look at the copde and
try to cut down pieces until the problem disappears.
Alex

Many thanks for your help Alex. I will now try to find the bug by
placing some traps in the code. If there is any result from debugging
that relates to 'WaitForMultipleObjects' I will post another article here.

Felix

[Mihajlo Cvetanovic]

Post by Felix Brack
Many thanks for your help Alex. I will now try to find the bug by
placing some traps in the code. If there is any result from debugging
that relates to 'WaitForMultipleObjects' I will post another article here.

To track this rogue handle you could WFSO for each handle, instead of
WFMO for all of them. And trace handle values before and after each call
to WFSO.

[Tom Widmer [VC++ MVP]]

Post by Felix

Post by Alex Blekhman
2. There is a race condition and some handle is closed
during a call to `WaitForMultipleObjects'.
As an experiment try to call `WaitForMultipleObjects' (or
`WaitForSingleObject') in a loop with one handle at a time
to figure out which handle is invalid.

This one sounds very promising...
If I add a second, identical call to 'WaitForMultipleObjects' just after
the first one has failed, everything runs fine. To make it very clear: the
first call to 'WaitForMultipleObjects' sometimes fails; if it does, I just
issue _one_ other call with _identical_ parameters. This second call
_always_ succeeds.

A handle could be temporarily invalid if it were closed during (or
before) the WaitForMultipleObjects call, and then a new handle was
created with the exact same value (synonymous with having a hanging
pointer that becomes valid due to the allocator reallocating that
memory). I don't know enough about how handles values are allocated to
know whether this is likely though.

Any more good ideas about

Post by Felix
how to catch this bug without rewriting the entire code? Is there some tool
I could run that would help me find the race condition without eliminating
it just from running?

Take a look at all code that closes any one of these handles. Note also
that some handles close implicitly, e.g. a thread handle created with
_beginthread.

Tom

[Felix Brack]

Post by Tom Widmer [VC++ MVP]
A handle could be temporarily invalid if it were closed during (or
before) the WaitForMultipleObjects call, and then a new handle was
created with the exact same value (synonymous with having a hanging
pointer that becomes valid due to the allocator reallocating that
memory). I don't know enough about how handles values are allocated to
know whether this is likely though.

The handles in question only get closed when the destructor of the class
the handles belong to is called. Since the destructor is only called
when the application is closed I am quite sure that none of my handles
get closed before.

Post by Tom Widmer [VC++ MVP]
Take a look at all code that closes any one of these handles. Note also
that some handles close implicitly, e.g. a thread handle created with
_beginthread.

Although I do not think that this is the problem here the following came
across my mind: if I have 3 open handles, say h1, h2 and h3 and pass two
of them to 'WaitForMultipleObjects', is it possible that this call to
'WaitForMultipleObjects' will fail if handle h3 (which was _not_ passed
to 'WaitForMultipleObjects') gets closed?

Felix

[Mihajlo Cvetanovic]

Post by Felix Brack
Although I do not think that this is the problem here the following came
across my mind: if I have 3 open handles, say h1, h2 and h3 and pass two
of them to 'WaitForMultipleObjects', is it possible that this call to
'WaitForMultipleObjects' will fail if handle h3 (which was _not_ passed
to 'WaitForMultipleObjects') gets closed?

Unless h1==h3 || h2==h3 this should not be the problem. But you could
set a breakpoint in CloseHandle SDK function. Stop execution immediately
before WFMO, add breakpoint to CloseHandle (in my case, Win2000 SP4,
address is 0x7C573D43, but you can find it when on any CloseHandle call
you enter into Disassembly window, and step into first "call" assembler
instruction), and continue execution. Every time the execution breaks
here compare the value in EAX register (Ctrl+Alt+G in VC2003) with the
values of handle array used for WFMO.

Furthermore, you could use Process Explorer,

http://www.microsoft.com/technet/sysinternals/utilities/processexplorer.mspx


to verify valid handles.

[Scott Seligman]

Post by Mihajlo Cvetanovic

Post by Felix Brack
Although I do not think that this is the problem here the following came
across my mind: if I have 3 open handles, say h1, h2 and h3 and pass two
of them to 'WaitForMultipleObjects', is it possible that this call to
'WaitForMultipleObjects' will fail if handle h3 (which was _not_ passed
to 'WaitForMultipleObjects') gets closed?

Unless h1==h3 || h2==h3 this should not be the problem. But you could
set a breakpoint in CloseHandle SDK function. Stop execution immediately
before WFMO, add breakpoint to CloseHandle (in my case, Win2000 SP4,
address is 0x7C573D43, but you can find it when on any CloseHandle call
you enter into Disassembly window, and step into first "call" assembler
instruction), and continue execution. Every time the execution breaks
here compare the value in EAX register (Ctrl+Alt+G in VC2003) with the
values of handle array used for WFMO.

I'm jumping into this thread late, so this might be duplicate advice,
but I'd recommend that you verify that the handle you're passing to
WaiteForMultipleObjects is indeed the same as whatever is returned
from your Create event. There's a chance you have a buffer overflow
somewhere that's clobbering your handle and causing this problem.

[Doug Harrison [MVP]]

Post by Felix
I have a problem with WaitForMultipleObjects sometimes (not allways)
returning ERROR_INVALID_HANDLE.
Printing the array of handles in question did not show any unusual thing,
i.e. all handles retain their values, no matter if the error occures or
not.
Is there any funktion I can call (like IsHandleValid) that would tell me if
a handle is valid or invalid? Or is there any value I can compare my
handles to, that would tell me if the handle is valid or not?

Where are these handles coming from? Some handles are closed automatically,
such as those returned from _beginthread and belonging to CWinThread. This
is a problem because the behavior is undefined if a handle is closed while
someone is waiting on it, and of course, there is the potential for the
handle to be closed before someone tries to wait on it. The solution is to
defeat the auto-deletion behavior and assume ownership yourself; I talk
about doing this for CWinThread here:

http://members.cox.net/doug_web/threads.htm#Q1

[Alexander Grigoriev]

Does it return ERROR_INVALID_HANDLE, or it returns WAIT_FAILED and
GetLastError() returns ERROR_INVALID_HANDLE?
ERROR_INVALID_HANDLE is 6, the same as WAIT_OBJECT_6.

Post by Felix
I have a problem with WaitForMultipleObjects sometimes (not allways)
returning ERROR_INVALID_HANDLE.
Printing the array of handles in question did not show any unusual thing,
i.e. all handles retain their values, no matter if the error occures or
not.
Is there any funktion I can call (like IsHandleValid) that would tell me if
a handle is valid or invalid? Or is there any value I can compare my
handles to, that would tell me if the handle is valid or not?
many thanks, Felix

[Felix Brack]

Post by Alexander Grigoriev
Does it return ERROR_INVALID_HANDLE, or it returns WAIT_FAILED and
GetLastError() returns ERROR_INVALID_HANDLE?
ERROR_INVALID_HANDLE is 6, the same as WAIT_OBJECT_6.

According to the documentation 'WaitForMultipleObjects' never returns
'ERROR_INVALID_HANDLE'. 'WaitForMultipleObjects' returns 'WAIT_FAILED'
and the subsequent call to 'GetLastError' returns 'ERROR_INVALID_HANDLE'.

Felix

[Alexander Grigoriev]

That was my point.
In the starting post, you said your WFMO returns ERROR_INVALID_HANDLE.

Post by Felix Brack

Post by Alexander Grigoriev
Does it return ERROR_INVALID_HANDLE, or it returns WAIT_FAILED and
GetLastError() returns ERROR_INVALID_HANDLE?
ERROR_INVALID_HANDLE is 6, the same as WAIT_OBJECT_6.

According to the documentation 'WaitForMultipleObjects' never returns
'ERROR_INVALID_HANDLE'. 'WaitForMultipleObjects' returns 'WAIT_FAILED' and
the subsequent call to 'GetLastError' returns 'ERROR_INVALID_HANDLE'.
Felix

[Felix Brack]

Post by Alexander Grigoriev
That was my point.
In the starting post, you said your WFMO returns .

You are right, sorry: it is 'GetLastError' returning 'ERROR_INVALID_HANDLE'.

Felix

[Felix Brack]

This post might be inappropriate. Click to display it.

[Alexander Grigoriev]

The rules are:

1. Create an event per OVERLAPPED.
2. Events used for notification should be manual reset. Such as used for
overlapped IO. Pattern of use: reset the event before you start an operation
that may signal it. The event gets set only once per use cycle.
3. Events used for synchronization should be auto reset. Pattern of use: the
event can be set at any moment by a signaller and be waited multiple times
by waiter.

First of all, many thanks to everyone of you for your tips and ideas. If I
did not respond to some posts, that does not mean the information therein
did not help me; I just did not know what to respond.
In fact my problem had to do with an event that was not correctly used,
i.e. I used that event in more then one place; on a multicore machine
'more then one place' refers to time (when is the code running) and space
(which core(s) is (are) running the code).
The event is part of an overlapped I/O operation used for serial
communication that signals the end of transmission ('WriteFile') operation.
The bad code did not take into account that this overlapped operation
could easily be run more then once at the same time on different cores and
that made my application crash. Even with one core the application did not
work on a fast machine since the event was used for more then one
overlapped I/O operation running at a time and the code was not designed
to handle this.
If this does not make any sense to you I am sorry, but it is quite
complicated to describe and posting the code would not help at all.
Anyway, here are my conclusions, maybe it help one or the other following
1. Overlapped operations take time until they finish.
I know it is some kind of platitude but I sometimes tend to forget this
fact. When using overlapped IO one should write code that respects the
current status of the overlapped IO operation at any time as long as the
operation lasts. I dare to say that code using overlapped operations, but
has no explicit checks for the status of the overlapped operation (this
might be a call to 'GetOverlappedResult' for example), is bad code and it
will fail sooner or later.
2. Use auto reset events sparingly
I think it is better to use manual events then auto events that are able
to reset automatically. The reason for this is quite simple: I stay in
better control about the status of the event and I have to take care
myself that the event gets reset at the correct location in space and
time. I really prefer my software not to work at all because of an event
that only fires once (wrong reset) then running every now and then,
depending on the machine it is running on (the code with the bug described
here did run for more then 8 years on many different machines without ever
failing!).
Not really much you might say and you are right, but believe it or not, it
took me about 4 weeks to find it (and just a few minutes to fix it). In
the end it's the result that counts and that was worth the time: the code
runs like charm.
Again many thanks to all of you, Felix

[Ben Voigt [C++ MVP]]

Post by Alexander Grigoriev
1. Create an event per OVERLAPPED.
2. Events used for notification should be manual reset. Such as used for
overlapped IO. Pattern of use: reset the event before you start an
operation that may signal it. The event gets set only once per use cycle.

Although, if you listen from the same thread, set the event before starting
an operation, the functions like ReadFile, WriteFile will reset the event if
the operation goes overlapped.

You do not need to reset the event.

"ReadFile resets the event specified by the hEvent member of the OVERLAPPED
structure to a nonsignaled state when it begins the I/O operation.
Therefore, the caller does not need to do that."

Post by Alexander Grigoriev
the event can be set at any moment by a signaller and be waited multiple
times by waiter.

Post by Felix Brack
First of all, many thanks to everyone of you for your tips and ideas. If
I did not respond to some posts, that does not mean the information
therein did not help me; I just did not know what to respond.
In fact my problem had to do with an event that was not correctly used,
i.e. I used that event in more then one place; on a multicore machine
'more then one place' refers to time (when is the code running) and space
(which core(s) is (are) running the code).
The event is part of an overlapped I/O operation used for serial
communication that signals the end of transmission ('WriteFile') operation.
The bad code did not take into account that this overlapped operation
could easily be run more then once at the same time on different cores
and that made my application crash. Even with one core the application
did not work on a fast machine since the event was used for more then one
overlapped I/O operation running at a time and the code was not designed
to handle this.
If this does not make any sense to you I am sorry, but it is quite
complicated to describe and posting the code would not help at all.
Anyway, here are my conclusions, maybe it help one or the other following
1. Overlapped operations take time until they finish.
I know it is some kind of platitude but I sometimes tend to forget this
fact. When using overlapped IO one should write code that respects the
current status of the overlapped IO operation at any time as long as the
operation lasts. I dare to say that code using overlapped operations, but
has no explicit checks for the status of the overlapped operation (this
might be a call to 'GetOverlappedResult' for example), is bad code and it
will fail sooner or later.
2. Use auto reset events sparingly
I think it is better to use manual events then auto events that are able
to reset automatically. The reason for this is quite simple: I stay in
better control about the status of the event and I have to take care
myself that the event gets reset at the correct location in space and
time. I really prefer my software not to work at all because of an event
that only fires once (wrong reset) then running every now and then,
depending on the machine it is running on (the code with the bug
described here did run for more then 8 years on many different machines
without ever failing!).
Not really much you might say and you are right, but believe it or not,
it took me about 4 weeks to find it (and just a few minutes to fix it).
In the end it's the result that counts and that was worth the time: the
code runs like charm.
Again many thanks to all of you, Felix

[Alexander Grigoriev]

Post by Ben Voigt [C++ MVP]

Post by Alexander Grigoriev
1. Create an event per OVERLAPPED.
2. Events used for notification should be manual reset. Such as used for
overlapped IO. Pattern of use: reset the event before you start an
operation that may signal it. The event gets set only once per use cycle.

Although, if you listen from the same thread, set the event before
starting an operation, the functions like ReadFile, WriteFile will reset
the event if the operation goes overlapped.
You do not need to reset the event.
"ReadFile resets the event specified by the hEvent member of the
OVERLAPPED structure to a nonsignaled state when it begins the I/O
operation. Therefore, the caller does not need to do that."

I was telling about general pattern, not Read/Write file. In Read/Write
case, "reset event" part is done for you. But it's done.

[f***@gmail.com]

I ran into the same issue and realized I was passing first parameter (nCount) wrong. Hope you are not doing the same mistake. I was passing sizeof(handle_array) here and that is wrong since it returns size in bytes of the array and not the size of array.

Post by Felix
I have a problem with WaitForMultipleObjects sometimes (not allways)
returning ERROR_INVALID_HANDLE.
Printing the array of handles in question did not show any unusual thing,
i.e. all handles retain their values, no matter if the error occures or
not.
Is there any funktion I can call (like IsHandleValid) that would tell me if
a handle is valid or invalid? Or is there any value I can compare my
handles to, that would tell me if the handle is valid or not?
many thanks, Felix

[Cholo Lennon]

Post by f***@gmail.com
I ran into the same issue and realized I was passing first parameter (nCount)
wrong. Hope you are not doing the same mistake. I was passing
sizeof(handle_array) > here and that is wrong since it returns size

in bytes of the array and

Post by f***@gmail.com
not the size of array.

Post by Felix
I have a problem with WaitForMultipleObjects sometimes (not allways)
returning ERROR_INVALID_HANDLE.
Printing the array of handles in question did not show any unusual thing,
i.e. all handles retain their values, no matter if the error occures or
not.
Is there any funktion I can call (like IsHandleValid) that would tell me if
a handle is valid or invalid? Or is there any value I can compare my
handles to, that would tell me if the handle is valid or not?
many thanks, Felix

Funny, the original post is 7 1/2 years old! (not a record in usenet for
a response by the way)