IO Completion Ports

[g***@hotmail.com]

Hi, i have recently been working with C# async IO. I have now moved
back to C++ to write some server software.

I quickly realised the C#/.NET IO mechanism is extremely efficient
compared to my old ways in C++ where i was creating a seperate thread
for each client.

I would like to create a similair architecture with my new software,
and have been playing about with IO completion ports. They seem to be
working, but i would appreciate clarification on a few points.

After creating my worker threads and completion port, I loop on
GetQueuedCompletionStatus() in my worker threads (i have been using
example 6-4 from "Multithreading Applications in Win32").

I have noticed if i put a blocking sleep inside the worker thread loop
(for testing only of course!), only one thread is released to service a
client at any one time... now this is fine for me as i dont have to
deal with async issues per client when buffering the read data for
instance.

However in the book "Multithreading Applications in Win32" it mentions
the "completion port will notice that Thread 1 blocked on disk I/O and
will release Thread 2 to bring the count of currently running threads
back up to the requested number".

Perhaps i am getting confused, but from my tests it seems threads are
only 'released' to service IO completion one at a time for individual
handles, yet IO completions will be serviced in parallel (waking
threads as necessary) for seperate handles?

Sorry for the somewhat confusing explanation, please let me know if
parts dont make sense so i can try to clarify. Thanks in advance,

Chris

[William DePalo [MVP VC++]]

Post by g***@hotmail.com
I have noticed if i put a blocking sleep inside the worker thread loop
(for testing only of course!), only one thread is released to service a
client at any one time... now this is fine for me as i dont have to
deal with async issues per client when buffering the read data for
instance.
However in the book "Multithreading Applications in Win32" it mentions
the "completion port will notice that Thread 1 blocked on disk I/O and
will release Thread 2 to bring the count of currently running threads
back up to the requested number".

...

So you created a pool of threads and an I/O completion port. You specified
the nominal number of threads to execute concurrently as the last parameter
of CreateIOCompletionPort(), yes?

Now, threads for which GetQueuedCompletionStatus() returns cause an
increment to the count of running threads. If one of those threads blocks
then the count is decremented. At that point, the o/s compares your nominal
thread count to the real count. If the nominal count is larger and if there
is another thread waiting on GetQueuedCompletionStatus() then it is allowed
to run. This is what the book is trying to explain.

Note that it is possible that more threads than the nominal count run
concurrently. This happens, as _one_ example, when a thread blocks
decrementing the count which "lets another thread in". If that blocked
thread ends its wait before the one that just became active calls
GetQueuedCompletionStatus() then you will have one more thread running than
you planned. The next thread to block and decrement the count _will_ not
release another thread.

So the questions for you to think about the behavior you see

1) What is your nominal count of active threads?
2) How many threads are active?
3) How many completed I/O operations are pending?

It is only when fewer threads than the nominal count are active AND when an
I/O operation is completed (i.e. a completion packet has been queued) that
another thread should be released.

Regards,
Will

[g***@hotmail.com]

Cheers for the reply, yes makes sense now, for some reason i forgot i
was only issuing another read request after i had slept... as there
were no IO requests outstanding no other threads would wake up.

Very powerful and i think a nicer way to write software. I am currently
rewriting some other bits of software as to be honest, i have used
threads here there and everywhere, when really all they are doing is
sleeping a lot whilst waiting for events to occur. I have read up on
win2k thread pools - hopefully will help to neaten up my code so i can
fix some of the hard to find asynchronous bugs i know i have.

Cheers,

Chris

[William DePalo [MVP VC++]]

Post by g***@hotmail.com
Cheers for the reply, yes makes sense now, for some reason i forgot i
was only issuing another read request after i had slept... as there
were no IO requests outstanding no other threads would wake up.

That would have been my first guess. :-)

Post by g***@hotmail.com
Very powerful and i think a nicer way to write software.

Absolutely correct. IOCPs with thread-pools provide _the_ most scalable
approach to building servers on Win32.

It wins hands-down over the tired old "poll and select" and "one new thread
per client" approaches so popular on those ancient operating systems which
never really grokked the concept of asynchronous events even when they were
shiny and new. <GD&R>

Regards,
Will