Board index » delphi » Re: (Re: Ritchie Annand) Deadlock detection & more.

Re: (Re: Ritchie Annand) Deadlock detection & more.


2005-01-06 07:02:05 PM
delphi210
Martin Harvey (Demon Account) writes:
Quote
Ahh... a real device. If things go wrong - well, you can connect a
kernel de{*word*81} via serial port to another machine, and if that is not
available at the time, then the de{*word*81} can likewise analyse the
crash dump - the BSOD also provides some useful introductory pointers.
Seems almost ironic to get useful information from a BSOD, but I have managed
to track down system issues by writing down what I saw, so I guess it is at
least a bit more useful than the run-of-the-mill access violation dialog...
if a little harder to recover from. How bare-bones do you need to run the
machine you develop on?
Quote
>Joy :) Fair enough, mind you - Windows is a lot bigger a project than I'd
>ever want to think of maintaining - something at the edges would slip
>through. Ran into the 95/98/ME-series internally leaking when rendering
>metafiles with a ClipRect on the DC. *shrug* Go figure :)
The NT / 2K / XP kernel is far more solid - thank god!
How much of the XP kernel comes from the NT/2K line, I often puzzle. I
puzzle mostly because running Delphi 8 can still run me out of Windows
resources (!), which it doesn't seem to under 2K :)
Quote
>Shutdown behavior still vexes me, though, especially when doing
>asynchronous coding, because there's nothing *actually* waiting on the
>condition in question. Then you have to get into arbitrary standards such
>as "I told this rather important task to stop as soon as it can - what's a
>reasonable time limit"?
Unfortunately, those sorts of issues are fairly fundamental and
insurmountable. The best one can do is to define acceptable behaviour,
and then regard execution outside that as erroneous.
I've found one of the best ways to stabilize my code has been simply to be
utterly draconian about conditions. No, I won't accept this blank filename,
no, you can not edit this object after it has been 'committed'. Unit testing
is nice for making sure things don't break the rules... actually, sometimes
it's a good place to think of the rules, because you end up trying all
combinations and start wondering "should I just return blank? should I
throw a hissyfit?" and then codify it.
At least I managed to codify *some* exit conditions. My major async base
class "TBasicTask" participates in a global running task list. At the very
least in my UI apps, I have it bring up a task shutdown dialog within 5
seconds of shutdown if there's still a task 'alive'.
Apart from that, I am left to negotiate things on a case-by-case basis, e.g.
in a remoting session, it is the responsibility of the client to ask for
termination only after it knows it has done all its tasks, and it is the
server's to terminate only upon request but to delay the reply until its
own tasks are complete.
Quote
A good reason to avoid synchronize. FYI, the stuff I am doing seems to
cope with modal dialogs just fine, and I am mainly using PostMessage
for communication to the main thread.
I wrote a Synchronize() that is independent from the thread object (only
makes sense, since I don't descend very much from TThread :). The
underlying pieces would probably work quite well for a launch-and-forget
system, although I would need to give them a Holder to poke when done.
Actually, we've got a bit of a naming conundrum, where the pieces are named
for their historical/technical purpose, not necessarily their function.
We call the asynchronous parameter-or-result object an ICOREValuePackage,
and the object that expects the result, but is usually involved in starting
up execution or passing on notification an ICOREValuePackageHolder (you set
its ValuePackage property when an async item completes).
Was trying to come up with a better name today, because we pass
ICOREValuePackageHolders all over the place. I was thinking ICOREMessage /
ICOREMessageFuture to use the naming convention in POSA, but I am not sure
it's intuitive enough :) (Rendezvous? Completion?)
Quote
LOL! I haven't worked it out that far yet - all I have is pictures in
my head.
The inventor of the What's-In-My-Head Projector will be rich! :)
Quote
It's in NT, and I *suspect* also inside earlier kernels, but not
explicitly exposed.
We'll see :) It sounds like it is definitely of good value in driver code.
Hmmm, on a quick search, I came across a .NET Threading error whose string
was:
System.Threading.WaitHandle.SignalAndWait(System.Threading.WaitHandle,System.Threading.WaitHandle)
System.PlatformNotSupportedException(R"Windows 98 and Millennium do not
support this operation.")
So I guess that is out for code you want to be universal (not too outlandish;
I have 98FE at home :)
Quote
It watches the calls at runtime & a lot more besides: dynamic memory
allocation occur from a special heap which (due to the joys of being
able to manipulate page tables) results in a bugcheck if one over or
underruns memory. It also checks that one adheres to some basic
etiquette regarding passing data & control up and down the driver
stack, and a whole load more besides.
That's sweet :) I wish there was something more robust than BoundsChecker
like that for user mode apps :)
Mind you, I will bet it is a piece of code they took from that, put it in SP2
that made my laptop unbootable with an unrecoverable "Unsafe Driver" BSOD
that required an OS reinstallation. Grrr :)
Quote
Apparently they hired me cos they thought I could rise to the
challenge. I like to think I haven't completely let them down.
It sure sounds like you rose to the challenge :) On multiple fronts too, no
doubt. Attacking a multithreaded kernel-mode driver isn't something the
garden variety CS grad is going to be a good fit for ;)
Quote
It's great most of the time - but one has to be an experienced driver
writer indeed before one ditches the assistance of the microsoft
sample code - I think just about no-one writes drivers entirely from
scratch - it is just too easy to foul up. Most people at least look at
samples, & I have a couple of books that I refer to constantly when
considering the "do's & don'ts".
That makes sense. There's a whole lot of prefix and postfix code just to
start playing, and a whole lot of skeleton code to fill in, no doubt :)
Are the samples still straight-C? Or is there an inkling of OOP set up
already - if not in the performance-sensitive pieces, then at least in the
performance-insensitive pieces?
Quote
Err right. Off the top of my head. Here goes.
I'm scared :)
Quote
Drivers basically deal with packets, which detail read / write or IO
control operations. On the whole, drivers are asynchronous - you'll
get called with a new chunk of data in the context of any thread at
any time, and there tend to do 3 main things when dealing with a
request:
So it looks somewhat like overlapped I/O in files or sockets?
Quote
1. Complete it now.
2. Pass it down the stack now.
Pass it down the stack? Ah, just had to Google on that - that is a fairly
overloaded word, that. (here I was thinking "uh, down the stack? isn't that
just a function call?" :) So this is the 'device stack' - would it be fair
to make a rough analogy to mouse clicks? If it is irrelevant for a child
object, then it keeps getting passed on until someone is interested?
Quote
3. Pend it
a) Temporarily because you're processing another request in another
thread, and need to serialise request processing as it pertains to
shared resources.
b) Less temporarily because you're waiting for the device to
respond.
Are there any major differences in how you pend it in those two
circumstances?
Quote
Anyways, to do Step 3 a) you basically want to have a handler that is
(bastardized translation into pascal:)
function HandleNewIrp(var NewIrp: IRP): NTSTATUS;
IRP = I/O Request Packet, gotcha.
So this would be a callback set by DriverObject.MajorFunction[IRP_MJ_READ]
:= HandleNewIrp; (approximately?)
Quote
and you want the basic logic:
AddNewIRP to queue.
If queue is not owned by anyone else, then
claim ownership of the queue
process as many IRP's as possible.
release ownership of the queue.
Is this a queue of your own devising, or is there a standard IRP queue you
can use?
Process as many as possible... within a given timeframe? Or is the check a
little rougher than that?
Quote
Now, the "ownership" deals with *taking* requests out of the queue.
Anyone can add new requests, but because the handling of requests is
likely to deal with shared resources (like the device!) you only allow
one thread to take requests out at once ... so one not only has a lock
to proctect the shared data in the queue, but one also has an idea of
whether one is allowed to process outstanding requests.
So ownership = allowed to process?
Quote
Now, because the model is asynchronous, you're not allowed to block
threads - hence why there's a queue. As you can see, there are some
potentially subtle race conditions: you may well need to do an atomic:
"add to queue, claim ownership and take one out" - where if you can't
claim ownership, then you skip the last two steps, and return some
pending status.
Isn't the process of adding an item to a queue a separate concern from
taking ownership and processing?
Just wondering how that would make for a subtle race condition :) I will trust
you on this point, though!
Quote
However, you need to make sure that there are no holes. Even in this
simple case, you need to be sure that whenever you release ownership
of the queue, at that moment when you can atomically view queue state,
you need to ensure that if there are any items left in the queue, that
some other thread will in fact process those items at some point -
this may be an issue if, for example, you've claimed ownership, the
handler has been called by another thread, putting another packet in,
and because you have ownership of the queue, the other thread gets
told to "go away", and you then have another item to deal with - you
want to make sure you don't leave it in limbo.
Now *that's* a situation I have encountered. Was a good 1 1/2 hours of
whiteboarding and "could we break *this*?" with a coworker. I should dig up
my research notes. I think we ended up with two flags and a lot of jumping
in and out of a critical section.
Quote
All this is, of course, because of the delightful fact that one is
getting called in an abritrary thread context - you need a mechanism
like this to ensure that you serialise execution,
I've had subtle errors manage to skip items (at about a 0.25% rate) I put on
a queue with a single line-of-Pascal bad exposure. I have also had cases
where I try to make parallel remoting calls to help saturate the bandwidth
(much remoting is otherwise terribly "ping-pongy"), but the handlers happen
in different threads... Needless to say, serializing properly simplifies
life so very much.
Quote
Got that simple case? Okay. Here's where it gets real interesting.
There are tow other situations one then needs to deal with:
"backpedalling" (my term!) and cancellation.
Always the dang exceptional conditions :)
Quote
The situation may be where you take an IRP off the queue for
processing, start to deal with it, and then realise you can not handle
it. This may either be because of the state of the device, or, you may
realise that you need more subsequent IRP's to deal with it - e.g. a
message that is "continued" from one IRP to the next. Luckily, the
concept of queue ownership comes to the rescue here. Anyone can add to
the "tail" end of the queue, but to either take *or put* IRP's from
the head of the queue, you need to have ownership. This has the lucky
side-effect that once you have the queue locked, you can take as many
off the queue as you want, and put them back as well, provided (of
course!) that you maintain the original order.
That's a sensible model for backpedalling. Seems a lot like parsing or
run-of-the-mill packet detection in that way... except for the concurrency
part. Pop, pop, pop... end of queue... dang, I need something more - better
wait for the next IRP... stuff, stuff, stuff.
Quote
The real {*word*193} bit comes with cancellation. User mode applications may
crash or quit, and user (or kernel) code may close the device handle.
In such situations, as well as getting a device close IRP, all
outstanding IRP's for that device will be *cancelled*. Cancellation is
simply a notification to a driver, that it should ditch this IRP.
It would be an easier problem if the cancellation were simply a queued-up
IRP itself. I have thought about a similar solution to some of my troubles,
but in many cases, you just can not trust it to get that far; sometimes the
cause of the cancellation is something that makes the task wait, or fails
to spur the task on.
Quote
Now - you are allowed to complete processing on IRP's that are in
progress: that is, if you're currently processing the IRP in thread A,
and you get a cancellation request in thread B, but thread A is just
about to make the call to complete the request, then that is fine. What
you are NOT allowed to do is to hold on to cancelled IRP's for any
length of time:
Let's say you're holding on to an IRP pending further requests or
commands. If the IRP is cancelled, you can pretty much bet that those
further requests or commands won't come.
How doe the cancellation 'notification' arrive? Is it a held-onto state
somewhere?
Quote
Of course, just to add amu{*word*224}t, IRP cancellation occurs in a totally
separate thread context, so if you send an IRP down a stack of
drivers, and you also send a cancellation request, the two will race
each other down the driver stack. You don't know whether the IRP will
complete sucessfully - but you do know that by the time the two
processes have got to the bottom of the driver stack, the IRP should
either have sucessfully been completed, or alternatively, it should
have been cancelled. What should NOT have happened is that any driver
has got the IRP squirreled away in some queue.
So, just before you release ownership of the queue, you decide that you need
more information, so you do your backpedal, and then drop out. Would the
driver be effectively "suspended" at this point if a cancellation
interceded?
Quote
Due to the way the I/O manager deals with calls down the driver stack
(which is WAAAY too complicated for me to discuss in detail here), you
know that by the time the function call giving the IRP to the driver
at the top of the stack has returned, and the function call requesting
the cancel has returned, the IRP should have been freed or completed.
Got a link to any diagrams? :)
Quote
So. How do we deal with cancellation. Realistically it means that an
IRP can be in one of four states:
a) it is sitting in the queue waiting from sonething to deal with it.
b) it is been taken out of the queue and is being processed, and will
shortly be completed (or passed on to another driver).
c) it is been taken out of the queue, and is being processed, but will
shortly be backpedalled into the same queue.
d) it is been taken out of the queue, is being processed, and will
shortly be put into a different queue.
I haven't seen what processing causes (d) to happen, but that is fair :)
Quote
When processing cancel requests you deal with them as follows:
a) This is simple. Pluck the IRP out of the queue, and complete it
with the appropriate cancellation status.
Simple as pie :)
Quote
b) This is also simple: just let the processing complete. Maybe the
cancellation request will overtake the IRP in another driver, maybe it
won't - not your problem.
The infamous Hitchhiker's Guide's SEP field :)
Quote
c) Ahhh. This is where it gets very interesting. In order to deal with
this, we need to actually make each queue be *two* queues: a "waiting
processing" queue, and a "currently processing" queue. If you do this,
you can then deal with cancellation requests which occur whilst
something is being processed - e.g. set a flag for that item. If you
then subsequently backpedal the item, you can take note of its
cancelled status, and bin it instead.
Do the items actually get cancelled *while* you're processing them, or is
this check entirely up to you? As long as an IRP is marked "cancelled"
properly, you're okay? Is there any lock you can hold onto to make sure the
cancel status can not be updated while you're looking at it?
Quote
d) Also very interesting. The I/O manager deals with passing IRP's
from one driver to the next, and hence from your output for the queued
input of another driver, via magic that we need not discuss here -
however, when you're taking from one queue, and putting into another,
you need to be very careful to avoid a "hole" - what if the cancel
occurs at the exact point when this IRP passes from one queue to
another? The DDK handles this with guidelines for the locking strategy
allowed in queues, as well as the global cancel lock. I will skip the
details because they require some *major* headscratching. Suffice to
say that a quick bit of hand waving at this point that you need to
think carefully about the ordering and manner in which you remove from
one, add to another, and check whether the IRP can be cancelled.
Ah, so there IS a global cancel lock :)
Hmmm, I might want to read those details - I have headscratching of my own -
maybe I can find an analogue of my async waits and queues in there :)
Quote
Those who feel the need for a bit of MSDN {*word*36} can consult:
tinyurl.com/6dbxj
Not enough {*word*36} ;)
Quote
Okay, and what use is this to user mode programmers?
Well, the point is that, using the concept of "owned queues", you can
easily implement queueing and serialisation without faffing around
with locks by hand. Additionally, the knowledge of the cancellation
stategies mentioned above means that:
The closest analogue I have is my thread pool, although as it stands, it
isn't technically "cancellable" :) Basically, it is the case where you have
a single queue and many things able to process on it. It would be overkill
(but a subset of the techniques would be valuable) for the
only-processed-by-a-single-object queue. Might work well for a notification
system.
I've got an odd-job other case called a "shared queue". This is the case
where multiple processing objects all need to process the same queue items,
locking constraints prevent them from simply using multiple queues, but you
want it to otherwise *behave* like multiple queues. I wish I could remember
more of the details - it was an interesting piece of work, and cleaned up
the queue's "tail" as the consumers consumed :)
Quote
- With a little bit of thought (and using asynchronous or overlapped
I/O) you can see that the problem of waiting for threads to quit could
be bypassed. Provided you can queue I/O instead of blocking waiting
for it to complete, can you reasonably implement your own schemes in
your app where you can just cancel large amounts of stuff, and be sure
you can shutdown once you've issued all the cancellations.
Doing non-blocking async, I find I can cut a number of things short simply
by getting rid of the objects, because they aren't "actively waiting" (the
callback that would start them back up simply goes in the trash). The very
last item that is actually being processed occasionally holds up the game,
but I tend to ensure that those items *can't* wait forever (though what am
I going to do if network file access goes off into la-la land?)
Quote
- Additionally, it is a nice (relatively!) way of writing thread-safe
library code, where you can allow just about anyone to call a bunch of
entry points you might export, and be sure that you will serialise the
calls internally in a sensible manner. Additionally, if you export
something resembling a "handle" or "connection" or "device", you could
also allow them to close that handle from within any thread, and deal
with it in a sensible manner.
Mind you, I think you can avoid some of the tribulations if you can
completely encapsulate a "processing unit" inside a single item in a queue
- this would likely cut down the number of cases to something more easily
manageable :)
Hmmm... I know we're nowhere close to a Guiness World Record of longest
post, but we're trying :) Feel free to ignore some stuff on the next round
- these are taking a long time to reply to *grin*
Quote
MH.
Cheers, Martin :)
- Ritchie Annand
Senior Software Architect
Malibu Software & Engineering Ltd.
Bus. www.malibugroup.com
Per. nimblebrain.net
 
 

Re: (Re: Ritchie Annand) Deadlock detection & more.

You may wish to read the VERY LAST paragraph first!
On Thu, 06 Jan 2005 04:02:05 -0700, Ritchie Annand
<XXXX@XXXXX.COM>writes:
Quote
How bare-bones do you need to run the
machine you develop on?
Well, the development machine is of course, not the target machine. As
far as the target machine is concerned...
As little or as much as you want. Realistically though, you don't
normally want to have loads of stuff running when you're testing,
unless you're stress or load testing. I simply have a machine that
dual boots 2K and XP SP2, with the Platform SDK, DirectX DSK, and a
copy of WinDbg installed.
Debugging stuff between user & kernel mode gets tricky though: you end
up running the user mode process in WinDbg on the target machine, just
like an ordinary de{*word*81}, and then you kernel debug the box remotely.
However, you have to remember that if you've stopped the machine in
the kernel de{*word*81}, trying to play with the user mode de{*word*81} is not
going to be entirely successful!
Quote
How much of the XP kernel comes from the NT/2K line, I often puzzle.
All of it. The 2K and XP kernel are surprisingly similar - the latter
is basically the former with an extra few years of development thrown
in.
As far as driver development goes, the NT kernel is a slightly
different kettle of fish, because it doesn't support the newest driver
models.
Quote
I
puzzle mostly because running Delphi 8 can still run me out of Windows
resources (!), which it doesn't seem to under 2K :)
Oh really? That *does* surprise me. I would suspect a bug somewhere.
Quote
Apart from that, I am left to negotiate things on a case-by-case basis, e.g.
in a remoting session, it is the responsibility of the client to ask for
termination only after it knows it has done all its tasks, and it is the
server's to terminate only upon request but to delay the reply until its
own tasks are complete.
Experience in designing comms protocols is astually a great benefit in
this regard, because all the same techniques apply. In fact, you can
view messages in threads the same way as messages between machines -
in terms of synchronisation design, there's no real difference, except
that communication is more likely to be guaranteed.
Quote
Actually, we've got a bit of a naming conundrum, where the pieces are named
for their historical/technical purpose, not necessarily their function.
It always happens. Every successful project undergoes at least 3
changes of direction, and 4 different sets of engineering or marketing
names.
Quote
The inventor of the What's-In-My-Head Projector will be rich! :)
It's technically possible now - it is been done with a cat. The thing
is, humans don't take kindly to have the back of their skull levered
off, and thousand of electroned implanted in their V1 visual cortex.
Quote
That's sweet :) I wish there was something more robust than BoundsChecker
like that for user mode apps :)
Well I think BC isn't too bad - it is just not as comprehensive as it
could be. There's nothing to stop it hooking the process / thread
Win32API calls.
Quote
Mind you, I will bet it is a piece of code they took from that, put it in SP2
that made my laptop unbootable with an unrecoverable "Unsafe Driver" BSOD
that required an OS reinstallation. Grrr :)
Shouldn't require a reinstall - but having said that, you'd have to
know how to how to poke around with the registry after mounting the
drive on another machine - second thoughts ... perhaps reinstalling is
better!
Quote
It sure sounds like you rose to the challenge :) On multiple fronts too, no
doubt. Attacking a multithreaded kernel-mode driver isn't something the
garden variety CS grad is going to be a good fit for ;)
I suspect it is probably only really well suited to people with
Asperger's syndrome - the rest of us are just too fallible!
Quote
That makes sense. There's a whole lot of prefix and postfix code just to
start playing, and a whole lot of skeleton code to fill in, no doubt :)
For a device that handles PnP and Power management properly, about
2,000 lines of boilerplate code.
Quote
Are the samples still straight-C? Or is there an inkling of OOP set up
already - if not in the performance-sensitive pieces, then at least in the
performance-insensitive pieces?
There are plenty of OOP inklings, and at lot of the code is object
based. C++ is used in the kernel, and some of the more experienced
driver writers write the drivers in C++.
However, whilst C++ is not forbidden, it is not officially supported
either. There are many areas in the language where you have to know
how the compiler implements a feature before you can use it safely in
the kernel. Huge swathes of the standard library and STL are not
suitable for use in the kernel. C++ Exception handling mechanisms
likewise are not suitable. There are issues with the compiler using a
lot more stack in C++ - and you only have 16 of kernel stack to play
with.
Quote
I'm scared :)
I'm scared you're asking questions ;-)
Quote
So it looks somewhat like overlapped I/O in files or sockets?
Very much so. Take a look at this bit of code (first random example I
could find on the web):
www-ivs.cs.uni-magdeburg.de/~trikalio/opencbm/PortAccess_8c-source.html
And particularly the function: "ParPortIoctlInOut". This function
builds an IOCTL, sends it down the stack, and if it doesn't complete
immediately, it waits on an event. Looks like overlapped I/O yeah?
This is of course, exactly what the synchronous versions of ReadFile /
WriteFile and DeviceIoControl do internally at the top levels of the
kernel to turn asynchronous I/O into synchronous I/O.
Quote
>1. Complete it now.
>2. Pass it down the stack now.

Pass it down the stack? Ah, just had to Google on that - that is a fairly
overloaded word, that. (here I was thinking "uh, down the stack? isn't that
just a function call?" :) So this is the 'device stack' - would it be fair
to make a rough analogy to mouse clicks? If it is irrelevant for a child
object, then it keeps getting passed on until someone is interested?
Yes and no. In principle, that is what happens. In practice, (alas)
it's more complicated. The driver stack does actually bear a close
resemblance to the call stack on the way down, but not typically on
the way back up.
Let's imagine that we have 3 drivers in a stack. The top one (A)
passes down to the middle (B), and the middle passes down to the
bottom(C). Control works basically like this:
The IO Manager calls IOCallDriver for driver A
Driver A entry point starts.
Driver A calls IoSkipCurrentIrpStackLocation and returns
Driver A calls IoCallDriver for B
IoCallDriver in the IO manager calls entry point for B
Driver B calls IoSkipCurrentIrpStackLocation and returns
Driver B calls IoCallDriver for C
Driver C calls IoCompleteRequest
Driver C returns value from IoCompleteRequest
Driver B returns value from IoCallDriver
Driver A returns value from IoCallDriver
IoManager handles the return code.
Seems simple, huh? Well it is, until you notice that drivers may want
to process the IRP on its way back up the stack when it is returning
results. In order to this you need to register a completion routine.
The completion routines gets stored in the IRP stack allocated. Now,
depending on exactly what the drivers want to do, and whether they
have to wait, the completion routines could get called synchronously,
or asynchronously.
When I/O CompleteRequest is called, it calls the various completion
routines for the drivers in the stack, from bottom to top. In our
example above, we'd have something like:
...
Driver B calls IoCallDriver for C
Driver C calls IoCompleteRequest
IoManager calls Driver C's completion routine.
C's routine returns
IoManager calls Driver B's completion routine.
B's routine returns
IoManager calls Driver A's completion routine.
A's routine returns
IoCompleteRequest returns
Driver C returns value from IoCompleteRequest
Driver B returns value from IoCallDriver
...
Now of course, the IRP may not be completed synchronously. It's
possible for the call stack to unwind because the IRP has been set to
STATUS_PENDING before IoCompleteRequest has been called. In that case,
the IRP will be stored somewhere, and IoCompleteRequest for that IRP
may be called by the bottom driver (C) at some later point (and in a
different thread context!).
Of course, The event setting done by the IoManager in response to
IoBuildDeviceIoControlRequest is going to look remarkably like what's
done by the IoCompleteRequest if all the completion routines have
indicated that they've finished ;-)
Quote
Are there any major differences in how you pend it in those two
circumstances?
There may be, and I would have to look into the books to find out. It
depends what IRQL you're called at. In most cases, you don't want to
block, or pass the IRP down, so you simply stash the IRP away in a
queue, call IoMarkIRPPending and return STATUS_PENDING (or something
like that!).
Quote
IRP = I/O Request Packet, gotcha.
Yep - you got it.
Uh oh - I have given you another post with lots of new function names
that you can google for ;-)
Quote
So this would be a callback set by DriverObject.MajorFunction[IRP_MJ_READ]
:= HandleNewIrp; (approximately?)
Exactly that. You set that pointer in your driver object, and then
pass your driver object into IoCreateDevice, and that hooks it up. Of
course, if the driver verifier is running, it also inserts an extra
bit of hooking in to do some checking.
Quote
Is this a queue of your own devising, or is there a standard IRP queue you
can use?
It could be either. XP now has Cancel safe queues, so you'd use those,
but many existing drivers implement their own queues. When I say
"their own", they'll use interlocked list functions like
"ExInterlockedInsertHeadList" which allow you to maintain a
thread-safe doubly linked list, and then they'll later their own IRP
handling on top of that.
Quote

Process as many as possible... within a given timeframe? Or is the check a
little rougher than that?
Well, given the model I showed you above, you normally deal with one
IRP at a time. There are two cases:
Pending an IRP in an indeterminate thread context (which is most of
the time): one does so in order to send another IRP off eg: "To handle
this request to enable my device, I as the function driver, need to
send a whole load of IRP's to the PCI bus driver", and once the last
completion routine for the IRP you sent off has completed, you'll
complete the IRP you were given. Typically, you complete as fast as
possible, because you're not sure which thread context, and hence,
whose time you're stealing - especially in a completion routine.
The other situation is when you're in a known thread context. This is
common for me, when, for example, I can complete read requests on my
device when a new video frame comes in. In such situations, I queue
the read requests up. I then have a worker thread that waits on an
event. An interrupt comes from the device saying I have a new video
frame, and in that, I can set the event, and kick the worker thread
off. My worker thread can then fill out the data in the read requests,
and call IoCompleteRequest for those IRP's at its leisure.
It's actually more complicated than that (surprise surprise) because
you can not play with kernel dispatcher objects (events) at interrupt
time, so you have to schedule a DPC (deferred procedure call), and
theye are yet more syncornisation mechanisms (like interrupt spin
locks) that you might also have to deal with.
Easy, aint it? ;-)
Quote
So ownership = allowed to process?
Yes - ownership means that you're allowed to manipulate the "head" end
of the queue. This is pretty easy: just implement a queue with a lock
round it, and then implement ownership by checking the ThreadID.
Quote
Isn't the process of adding an item to a queue a separate concern from
taking ownership and processing?
Adding to the tail is a separate concern.
Quote
Just wondering how that would make for a subtle race condition :) I will trust
you on this point, though!
Well, you can protect all the calls with a single lock, just like any
shared datastructure - you just need inside that to Check ThreadID's,
and implement a suitable "ClaimOwnership" and "ReleaseOwnership"
function.
Of course, if you wanted a "WaitForOwnership" function, then you're
back to allocating another event for "CurrentlyOwned".
Quote
Now *that's* a situation I have encountered. Was a good 1 1/2 hours of
whiteboarding and "could we break *this*?" with a coworker. I should dig up
my research notes. I think we ended up with two flags and a lot of jumping
in and out of a critical section.
Yep - makes sense. Depending on how you wanted to do things, you might
even have ended up with a "try again" flag.
Quote
I've had subtle errors manage to skip items (at about a 0.25% rate) I put on
a queue with a single line-of-Pascal bad exposure.
In my current Delphi app, I actually have the opposite race condition:
A queue has Get and Put functions, and because of other constraints, I
couldn't do strict counting with semaphores, and wanted instead to
have a "BlockWhileEmpty" function. As it turns out, you can do:
BlockWhileEmpty
NewItem := GetItem;
However, these aren't atomic (you've left & re-entered the lock), so
you have to be prepared for someone else to have got there first:
loop
BlockWhileEmpty;
NewItem := GetItem();
if NewItem ...
endloop
And you just need to be aware that if a lot of threads are trying to
take stuff out at once, they may zip round the loop a small number of
times before actually getting an item. Actually - the block is in the
conditional, cos it is more efficient, but that is another story.
Quote
That's a sensible model for backpedalling. Seems a lot like parsing or
run-of-the-mill packet detection in that way... except for the concurrency
part. Pop, pop, pop... end of queue... dang, I need something more - better
wait for the next IRP... stuff, stuff, stuff.
Yes - that is it, and of course you need ownership to make sure no-one
nicks anything else off the queue before you decide to backpedal -
which would result in requests in the wrong order.
Quote
It would be an easier problem if the cancellation were simply a queued-up
IRP itself. I have thought about a similar solution to some of my troubles,
but in many cases, you just can not trust it to get that far; sometimes the
cause of the cancellation is something that makes the task wait, or fails
to spur the task on.
Would be nice - but the problem is that you can forward IRP's onto
other stacks, duplicate them, hide them away, and do all sorts of
stuff, and the problem is that once you're actually doing stuff in a
tree of devices, you then start having to duplicate and forward
cancellation requests to multiple places. Whilst it *could* work fine,
in that you'd then have a set of rules that dictated how to handle
farming the cancellation IRP's out (and get them back...) it is then
another huge set of cancellation completion routines etc etc ... and
the hair on your head starts falling out - and in that respect, a
central registration system seems to work better.
In my current Delphi app that does a lot of queueing, cancellation is
fairly immediate, and via pointer. Instead of flushing through the
whole way, cancelled packets typically get disposed of next time they
get to a queue of some description.
Quote
How doe the cancellation 'notification' arrive? Is it a held-onto state
somewhere?
There's both some cancellation state, and a routine that gets invoked
at cancellation time - as you've probably discovered.
Quote
So, just before you release ownership of the queue, you decide that you need
more information, so you do your backpedal, and then drop out. Would the
driver be effectively "suspended" at this point if a cancellation
interceded?
At this point, you need to synchronise the cancel handling and the
lock on the queue, hence cancel locks. If a cancellation intervenes,
then you need to set a cancelled flag (whilst not actually putting it
into or taking it out of the queue) such that if you were going to
backpedal it so that you put it in the queue, instead of doing that,
you free it.
Quote
>Due to the way the I/O manager deals with calls down the driver stack
>(which is WAAAY too complicated for me to discuss in detail here),
Oh well, got there anyway.
Quote
Got a link to any diagrams? :)
I *wish*. This is a good page:
msdn.microsoft.com/library/default.asp
But unfortunately, is not that hot on useful diagrams.
By the way, this is, IMHO the best book I have seen on the subject, when
it comes to real understanding:
www.amazon.com/exec/obidos/tg/detail/-/0735618038/qid=1105056486/sr=8-1/ref=sr_8_xs_ap_i1_xgl14/103-2293265-6327068?v=glance&s=books&n=507846
It does some to have received a couple of bad reviews, but it looks to
me like the people that wrote those reviews were looking for a "bullet
point explanation", instead of long prose ... unfortunately, it seems
that you need to ponder and wade through the prose in order to
understand enough to really "get it". For more concise info, MSDN
seems to do fine.
Even better - the sample code contains a GREAT library module which
actually implements cancel-safe queues for you.
Unfortunately, the drivers I am working on got written before the book
came out ;-)
Quote
>c) Ahhh. This is where it gets very interesting. In order to deal with
>this, we need to actually make each queue be *two* queues: a "waiting
>processing" queue, and a "currently processing" queue. If you do this,
>you can then deal with cancellation requests which occur whilst
>something is being processed - e.g. set a flag for that item. If you
>then subsequently backpedal the item, you can take note of its
>cancelled status, and bin it instead.

Do the items actually get cancelled *while* you're processing them, or is
this check entirely up to you? As long as an IRP is marked "cancelled"
properly, you're okay? Is there any lock you can hold onto to make sure the
cancel status can not be updated while you're looking at it?
The cancellation is actually done by setting a flag, and your own
cancel handler for the IRP being called. I need to look more carefully
at cancel locks, because as you can see, the issue of synchronisation
boils down to synchronisation between three things.
- The cancelled flag / status of the IRP.
- The execution of the cancellation handler.
- The synchronisation of cancellation with queuing operations.
It's kinda complicated, but this section deals with it - as you can
see, it spans several pages.
msdn.microsoft.com/library/default.asp
This is, of course, the kind of stuff where once you have your
cancel-safe queueing library, you leave it at that!
Quote
Ah, so there IS a global cancel lock :)
Yes, but it is use is deprecated. The recommended method nowadays is
for drivers to maintain their own cancel locks: the system cancel lock
turned out to be a fairly serious bottleneck.
Quote
>Those who feel the need for a bit of MSDN {*word*36} can consult:
>tinyurl.com/6dbxj

Not enough {*word*36} ;)
Well that is okay, I have included plenty more for you :-P
Quote
I've got an odd-job other case called a "shared queue". This is the case
where multiple processing objects all need to process the same queue items,
locking constraints prevent them from simply using multiple queues, but you
want it to otherwise *behave* like multiple queues. I wish I could remember
more of the details - it was an interesting piece of work, and cleaned up
the queue's "tail" as the consumers consumed :)
Sounds rather similar to what we've been discussing here.
Quote
Doing non-blocking async, I find I can cut a number of things short simply
by getting rid of the objects, because they aren't "actively waiting" (the
callback that would start them back up simply goes in the trash). The very
last item that is actually being processed occasionally holds up the game,
but I tend to ensure that those items *can't* wait forever (though what am
I going to do if network file access goes off into la-la land?)
Yep - my queueing systems seem similar to this: ditching dtuff in
queues is relatively easy - with the exception that you might have to
inform other components that are waiting for them to "pop out"
somewhere else having completed, but the tricky bit is cancelling
things that aren't in queues and are actually in progress.
Quote
Mind you, I think you can avoid some of the tribulations if you can
completely encapsulate a "processing unit" inside a single item in a queue
- this would likely cut down the number of cases to something more easily
manageable :)
There are areas in the video capture drivers I am working on which do
this: there's only one queue, and no passing from one queue to the
next, and it then becomes relatively simple (i.e. "only" a CS final's
exam question of difficulty).
Quote
Hmmm... I know we're nowhere close to a Guiness World Record of longest
post, but we're trying :) Feel free to ignore some stuff on the next round
- these are taking a long time to reply to *grin*
No problem. I would meant to dash off a quick reply, and then found myself
getting rather engrossed - and wanting to check/recheck the details on
how it worked. I suspect this post may be even longer, but I won't
know the line count until I have sent it!
No need to trawl thru all my points if you don't want to - because
it's at that point where the discussion of minutiae could get scary.
What I think gives you a nice flavour of the issues however is this
document (which is readable narrative, not techie) which explains why
there's _yet_another_ driver model coming out soon, WDF, which
apparently will enable us to get rid of many of these woes - and he
nicely describes some of the challenges faced:
download.microsoft.com/download/e/b/a/eba1050f-a31d-436b-9281-92cdfeae4b45/WDFpreview_BMc.doc
MH.
 

Re: (Re: Ritchie Annand) Deadlock detection & more.

In article <XXXX@XXXXX.COM>,
XXXX@XXXXX.COM says...
Quote
You may wish to read the VERY LAST paragraph first!
Thanks for that ;)
Quote
Debugging stuff between user & kernel mode gets tricky though: you end
up running the user mode process in WinDbg on the target machine, just
like an ordinary de{*word*81}, and then you kernel debug the box remotely.
However, you have to remember that if you've stopped the machine in
the kernel de{*word*81}, trying to play with the user mode de{*word*81} is not
going to be entirely successful!
I take it that the kernel puts itself on hold for the duration for *any*
calls other than from the de{*word*81}? :)
Quote
>I
>puzzle mostly because running Delphi 8 can still run me out of Windows
>resources (!), which it doesn't seem to under 2K :)
Oh really? That *does* surprise me. I would suspect a bug somewhere.
Either that or there's some GDI(16) pieces still{*word*154} around in Win
XP. I have managed to run out of resources without Delphi 8 running as well.
It's rarer, but it happens.
Quote
Experience in designing comms protocols is astually a great benefit in
this regard, because all the same techniques apply. In fact, you can
view messages in threads the same way as messages between machines -
in terms of synchronisation design, there's no real difference, except
that communication is more likely to be guaranteed.
*laugh* I have had some of that experience :) The message and message-
future model IS akin to comms in some respects. There are probably a
couple of lessons from the analogy I have still yet to implement. One
that crossed my mind was making a message future that timed itself out.
Quote
It always happens. Every successful project undergoes at least 3
changes of direction, and 4 different sets of engineering or marketing
names.
The components themselves, though. Gads, I always agonize over the
names. it is for good reason, really. I can tell where I didn't agonize
enough (TInterposedMethodValuePackageHolderListener, anyone? :).
Computers can understand code just fine, they don't care if "Aimfiz"
isn't a good class name :)
Quote
It's technically possible now - it is been done with a cat. The thing
is, humans don't take kindly to have the back of their skull levered
off, and thousand of electroned implanted in their V1 visual cortex.
Doesn't that just cover off direct visual stimuli? Still pretty neat; I
hadn't heard of that research. I love the layered/timing approach of the
sensory cortices - V1 is the "raw" stuff (which then gets into lines,
movement, etc. up the layers - the movement 'detection' is cool). I
guess we'll have to wait for the breakthrough that can read the field
from further away. Maybe RFID research will help? :)
Quote
>Mind you, I will bet it is a piece of code they took from that, put it in SP2
>that made my laptop unbootable with an unrecoverable "Unsafe Driver" BSOD
>that required an OS reinstallation. Grrr :)
Shouldn't require a reinstall - but having said that, you'd have to
know how to how to poke around with the registry after mounting the
drive on another machine - second thoughts ... perhaps reinstalling is
better!
They managed to come up with a fix for it just this past week (and it
worked!). Turned out to be fixable with registry entries... which is
what makes the nonrecoverability so annoying :)
Quote
I suspect it is probably only really well suited to people with
Asperger's syndrome - the rest of us are just too fallible!
Having heard the stories from my fiancee's teaching at a learning-
disabled school, that would be right up their alley :)
Quote
There are plenty of OOP inklings, and at lot of the code is object
based. C++ is used in the kernel, and some of the more experienced
driver writers write the drivers in C++.
I'm glad to hear there's some OOP in there :)
Quote
However, whilst C++ is not forbidden, it is not officially supported
either. There are many areas in the language where you have to know
how the compiler implements a feature before you can use it safely in
the kernel. Huge swathes of the standard library and STL are not
suitable for use in the kernel. C++ Exception handling mechanisms
likewise are not suitable. There are issues with the compiler using a
lot more stack in C++ - and you only have 16 of kernel stack to play
with.
*laugh* The exception handling would sorely be missed, but I can sure
understand why you'd really want to avoid the standard library and STL
:)
Quote
I'm scared you're asking questions ;-)
*grin*
8< snip 8<
I took a look at MSDN's "Passing IRPs down the driver stack" page. :)
It's bizarre having that sort of control over the 'call stack' - being
able to pretend you don't exist as a caller to return to when you call
the next driver.
Mind you, the async techniques I have work approximately this way. Many
functions and constructors take message futures (equivalent to the
completion routines I'd say). Often, you just call/start something
else, passing on the message future you got (sort of like the
IoSkipCurrentIrpStackLocation scenario) to something else to handle.
Sometimes, you need to intercept what comes back and do something with
it. (e.g. in one case I have, a remoting call needs to open a file on
the other end, but the file access is also asynchronous), akin to the
normal IoCallDriver. Sometimes you just need a "results processor" (e.g.
I have a "call ID" which is only required during remoting), akin to the
IoCopyCurrentIrpStackLocationToNext / IoSetCompletionRoutine /
IoCallDriver scenario (if I understand it correctly)
Quote
There may be, and I would have to look into the books to find out. It
depends what IRQL you're called at. In most cases, you don't want to
block, or pass the IRP down, so you simply stash the IRP away in a
queue, call IoMarkIRPPending and return STATUS_PENDING (or something
like that!).
Are you supposed to avoid using STATUS_MORE_PROCESSING_REQUIRED? :)
Quote
It could be either. XP now has Cancel safe queues, so you'd use those,
but many existing drivers implement their own queues. When I say
"their own", they'll use interlocked list functions like
"ExInterlockedInsertHeadList" which allow you to maintain a
thread-safe doubly linked list, and then they'll later their own IRP
handling on top of that.
Gotcha - don't really want to do the multiprocessor safety bits on one's
own :)
Actually, there's a whole lot of things in that Executive Support
Routines. How long have these functions been in the kernel?
Quote
It's actually more complicated than that (surprise surprise) because
you can not play with kernel dispatcher objects (events) at interrupt
time, so you have to schedule a DPC (deferred procedure call), and
theye are yet more syncornisation mechanisms (like interrupt spin
locks) that you might also have to deal with.
DPCs don't look *too* terrible to use, but it still must be annoying the
restrictions you get. When are you inside an interrupt, and when not?
Quote
Easy, aint it? ;-)
Oh, certainly (*nods head overly vigorously* :)
Quote
BlockWhileEmpty
NewItem := GetItem;
However, these aren't atomic (you've left & re-entered the lock), so
you have to be prepared for someone else to have got there first:
There have been a couple of spots where I have had to put together weird-
looking calls just to get the atomicity (i.e. the parameters don't go
together, but the operations on them DO :)
Quote
loop
BlockWhileEmpty;
NewItem := GetItem();
if NewItem ...
endloop
That scenario sounds familiar; the once-in-a-blue-moon plain ol' access
violations if you forget that next check :)
Quote
And you just need to be aware that if a lot of threads are trying to
take stuff out at once, they may zip round the loop a small number of
times before actually getting an item. Actually - the block is in the
conditional, cos it is more efficient, but that is another story.
My thread pool does the occasional zipping around the loop a couple of
times, but the collisions are pretty rare, mostly due to the things that
are queued on the thread pool take a much longer time to execute than
the loop itself does, so the chances of it taking more than one trip
around the loop are vanishingly small :)
Conditional block? Do tell :)
Quote
Yes - that is it, and of course you need ownership to make sure no-one
nicks anything else off the queue before you decide to backpedal -
which would result in requests in the wrong order.
I had a communications protocol where things could arrive in the wrong
order, and it wasn't cleaned up by the time it got to the code that
processed it. It was a nightmare!
Quote
Would be nice - but the problem is that you can forward IRP's onto
other stacks, duplicate them, hide them away, and do all sorts of
stuff, and the problem is that once you're actually doing stuff in a
tree of devices, you then start having to duplicate and forward
cancellation requests to multiple places. Whilst it *could* work fine,
in that you'd then have a set of rules that dictated how to handle
farming the cancellation IRP's out (and get them back...) it is then
another huge set of cancellation completion routines etc etc ... and
the hair on your head starts falling out - and in that respect, a
central registration system seems to work better.
Understandable - cancelling in a situation like drivers has got to be
{*word*193}. I suppose I should be a little more forgiving when my machine
goes AWOL due to video drivers... but I am not :)
I find people usually end up with blunt trauma foreheads more than
missing hair, but to each their own ;)
Quote
In my current Delphi app that does a lot of queueing, cancellation is
fairly immediate, and via pointer. Instead of flushing through the
whole way, cancelled packets typically get disposed of next time they
get to a queue of some description.
Similarly, I find disconnecting the rendezvous works for my purposes.
Just keep the items in their state, and instead of doing a lot special
if they're cancelled, the message future simply is disconnected, causing
no activity when it is responded to.
Quote
There's both some cancellation state, and a routine that gets invoked
at cancellation time - as you've probably discovered.
That's nice - that lets you handle both suspended (I find I often have
to poke an async signal to ensure my tasks exit) and currently-running
processes with aplomb.
Quote
I *wish*. This is a good page:
msdn.microsoft.com/library/default.asp
Actually, that is a pretty nice page - thanks! The "IRP as Thread-
Independent Call Stack" just gelled what I was trying to get my mind
around saying before, when I was following up on your references to the
driver stack.
Quote
But unfortunately, is not that hot on useful diagrams.
And MS bought out Visio - and for what!? :)
Quote
By the way, this is, IMHO the best book I have seen on the subject, when
it comes to real understanding:
www.amazon.com/exec/obidos/tg/detail/-/0735618038/qid=1105056486/sr=8-1/ref=sr_8_xs_ap_i1_xgl14/103-2293265-6327068?v=glance&s=books&n=507846
I've never seen that book locally. I hope it shows up in the discount
pile-o'-computer-books that {*word*110}tores and Safeway often seem to have :)
Quote
It does some to have received a couple of bad reviews, but it looks to
me like the people that wrote those reviews were looking for a "bullet
point explanation", instead of long prose ... unfortunately, it seems
that you need to ponder and wade through the prose in order to
understand enough to really "get it". For more concise info, MSDN
seems to do fine.
That's the kind of criticism that is just fine - it is just like never
needing to wonder why a Dawkins book got less than 5 stars :)
Quote
Even better - the sample code contains a GREAT library module which
actually implements cancel-safe queues for you.
Wow, a sample CD you can actually USE? :)
Quote
Unfortunately, the drivers I am working on got written before the book
came out ;-)
...and it shows? :)
Quote
>Ah, so there IS a global cancel lock :)
Yes, but it is use is deprecated. The recommended method nowadays is
for drivers to maintain their own cancel locks: the system cancel lock
turned out to be a fairly serious bottleneck.
I can imagine that - one bad player could start shorting out your system
performance. I guess it happens to all systems when they start to scale
- you need an entirely new way to do what used to be efficient.
Quote
>Not enough {*word*36} ;)
Well that is okay, I have included plenty more for you :-P
It has made for some interesting reading :)
Quote
No need to trawl thru all my points if you don't want to - because
it's at that point where the discussion of minutiae could get scary.
What I think gives you a nice flavour of the issues however is this
document (which is readable narrative, not techie) which explains why
there's _yet_another_ driver model coming out soon, WDF, which
apparently will enable us to get rid of many of these woes - and he
nicely describes some of the challenges faced:
It does give me a good taste of the issues, and hopefully a little bit
of inspiration, because there are certainly a number of driver issues
that mirror the troubles I encounter in the async application-side world
(cancellation, completions, passing between threads, queue safety) -
hopefully I will be able to extract some lessons from this :)
Thanks for the discussion, Martin - really keeps the neurons from
clogging :)
-- Ritchie Annand
Senior Software Architect
www.malibugroup.com
nimble.nimblebrain.net
wiki.nimblebrain.net
 

Re: (Re: Ritchie Annand) Deadlock detection & more.

On Thu, 13 Jan 2005 14:42:14 -0700, Ritchie Annand
<XXXX@XXXXX.COM>writes:
Quote
I take it that the kernel puts itself on hold for the duration for *any*
calls other than from the de{*word*81}? :)
Yep - locked up solid, mouse cursor won't move, scroll lock & caps
lock don't work. It truly is stopped, apart from running a tiny
routine hidden away somewhere in the kernel to handle de{*word*81}
commands.
Quote
Either that or there's some GDI(16) pieces still{*word*154} around in Win
XP. I have managed to run out of resources without Delphi 8 running as well.
It's rarer, but it happens.
Strange things happen - I got an External Exception today calling
CloseHandle() on a file handle twice - I was expecting the handle to
be sufficiently opaque that I would get an error code back, but it seems
not to be the case.
There's just so much code buried away down there....
Quote
The components themselves, though. Gads, I always agonize over the
names. it is for good reason, really. I can tell where I didn't agonize
enough (TInterposedMethodValuePackageHolderListener, anyone? :).
I thoroughly sympathise. I have times when I am so deep in the code,
the linguistic bits of the brain shut down, and you end up giving
things stupid names like:
TArrayHandle, TArrayHandleArray,
HandleUnhandledArrayHandleArray(arg: TArrayHandleArray).
Quote
Computers can understand code just fine, they don't care if "Aimfiz"
isn't a good class name :)
I know a guy with an email address, which when spoken is "dot at dot
at dot at" :-)
Quote
Maybe RFID research will help? :)
Dunno, we'll see.
Quote
>There are plenty of OOP inklings, and at lot of the code is object
>based. C++ is used in the kernel, and some of the more experienced
>driver writers write the drivers in C++.

I'm glad to hear there's some OOP in there :)
It's certainly all object based - it has to be to provide flixble
enough models for peopleto use without having the kernel source.
Quote
8< snip 8<

I took a look at MSDN's "Passing IRPs down the driver stack" page. :)
It's bizarre having that sort of control over the 'call stack' - being
able to pretend you don't exist as a caller to return to when you call
the next driver.
Yep, it is a bit odd, but if you follow the rules you're (mostly) OK.
Even worse than that, there's a function somewhere (Something to do
with stopping completion), where when you call it, you get re-entered,
and you pass in a pointer to a flag variable, and use it during
re-entry.
So that means that you're using the return value of a function before
it actually returns...
Quote
Mind you, the async techniques I have work approximately this way. Many
functions and constructors take message futures (equivalent to the
completion routines I'd say). Often, you just call/start something
else, passing on the message future you got (sort of like the
IoSkipCurrentIrpStackLocation scenario) to something else to handle.

Sometimes, you need to intercept what comes back and do something with
it. (e.g. in one case I have, a remoting call needs to open a file on
the other end, but the file access is also asynchronous), akin to the
normal IoCallDriver. Sometimes you just need a "results processor" (e.g.
I have a "call ID" which is only required during remoting), akin to the
IoCopyCurrentIrpStackLocationToNext / IoSetCompletionRoutine /
IoCallDriver scenario (if I understand it correctly)
Yep. to be honest, when I have done this in user mode, I have just passed
a structure through buffers - all the way down, and all the way back,
and the structure has "CommandRequest" and a "CommandResult" fields,
and I don't really distinguish between the two directions of travel
that much .... it just seemed simpler to me: It just means that your
main loop has to pass messages both down and up...
Having said that, I didn't need anything desparately complicated, so
that's why.
Quote
Are you supposed to avoid using STATUS_MORE_PROCESSING_REQUIRED? :)
Nahh, it is OK - there are some situations where you must, but
returning that, and dealing with pending flags are common Gotchas.
Quote
Gotcha - don't really want to do the multiprocessor safety bits on one's
own :)

Actually, there's a whole lot of things in that Executive Support
Routines. How long have these functions been in the kernel?
Dunno. I haven't been around long enough. At the moment, I just rely
on stuff that is in 2K or above, and leave it at that.
Quote
DPCs don't look *too* terrible to use, but it still must be annoying the
restrictions you get. When are you inside an interrupt, and when not?
DPC's are a godsend. because basically, at dispatch_level, you get
code that's:
- In a DPC.
- Called via a Kernel Timer.
- Running in a spinlock.
So if you ever want to do "general" synchronisation, you can
synchronise with just about anything at that point. Just have to
remember to keep the code and data nonpaged.
You're inside an interrupt when you explicitly register to handle one:
IoConnectInterrupt.
Quote
Oh, certainly (*nods head overly vigorously* :)
:-)
Quote
That scenario sounds familiar; the once-in-a-blue-moon plain ol' access
violations if you forget that next check :)
Yep - the reason I was happy with the slight hole was just because of
my inheritance model - it was easier to do that way.
Quote
My thread pool does the occasional zipping around the loop a couple of
times, but the collisions are pretty rare, mostly due to the things that
are queued on the thread pool take a much longer time to execute than
the loop itself does, so the chances of it taking more than one trip
around the loop are vanishingly small :)

Conditional block? Do tell :)
Conditional block as in "if statement". Actually looks like this:
function TMultiThreadQueue.TakeItemFromQHead: TQueueableNotifiable;
begin
EnterCriticalSection(FCrit);
try
result := inherited TakeItemFromQHead;
UpdateEmptyEvent;
finally
LeaveCriticalSection(FCrit);
end;
end;
procedure TFileSysWorkerThread.Execute;
while not Terminated do
begin
FCurrentCommand :=
FParentFilesystem.FWaitingQueue.TakeItemFromQHead as
TFilesysCommand;
if Assigned(FCurrentCommand) then
begin
if not FCurrentCommand.CommandCancelled then
FParentFilesystem.WorkerThreadExecFileSysCmd(FCurrentCommand);
//Might have completed, or might have cancelled.
//However, cannot free it here (notifiable callbacks!)
FParentFilesystem.FCompleteQueue.AtomicGetCountAndAddToTail(FCurrentCommand,
OldCount);
FCurrentCommand := nil;
if OldCount = 0 then
GlobalNotifier.AsynchronousNotify(NotifyHandle);
end;
FParentFilesystem.FWaitingQueue.BlockWhileEmpty;
end;
Quote
I had a communications protocol where things could arrive in the wrong
order, and it wasn't cleaned up by the time it got to the code that
processed it. It was a nightmare!
Sequence numbers are great! - Actually that is a point: part of my
queueing system may span between machines - I end up having sequence
numbers, and a simple window negotiation protocol too :-)
Quote
Understandable - cancelling in a situation like drivers has got to be
{*word*193}. I suppose I should be a little more forgiving when my machine
goes AWOL due to video drivers... but I am not :)
Video drivers are now so specialist that very few companies write
them, and unlike for example PCI or USB drivers, there are few open to
the public seminars or books.
Quote
Similarly, I find disconnecting the rendezvous works for my purposes.
Just keep the items in their state, and instead of doing a lot special
if they're cancelled, the message future simply is disconnected, causing
no activity when it is responded to.
Yep, that would work :-)
Quote
>But unfortunately, is not that hot on useful diagrams.

And MS bought out Visio - and for what!? :)
Probably so that people could put cool diagrams in books ;-)
Quote
>By the way, this is, IMHO the best book I have seen on the subject, when
>it comes to real understanding:

>www.amazon.com/exec/obidos/tg/detail/-/0735618038/qid=1105056486/sr=8-1/ref=sr_8_xs_ap_i1_xgl14/103-2293265-6327068?v=glance&s=books&n=507846

I've never seen that book locally. I hope it shows up in the discount
pile-o'-computer-books that {*word*110}tores and Safeway often seem to have :)
If it does, grab it quick.
Quote
>Even better - the sample code contains a GREAT library module which
>actually implements cancel-safe queues for you.

Wow, a sample CD you can actually USE? :)
Yep - it is got some nice boilerplate drivers. I created a bus driver a
while back, which acts like a "bus" in that it spawns child devices -
the example code made my work a lot easier, and helped me debug stuff
too.
Quote
>Unfortunately, the drivers I am working on got written before the book
>came out ;-)

...and it shows? :)
Nahh, they're really quite good - my predecessors were smart blokes,
but they just take a different approach in places.
Quote
It does give me a good taste of the issues, and hopefully a little bit
of inspiration, because there are certainly a number of driver issues
that mirror the troubles I encounter in the async application-side world
(cancellation, completions, passing between threads, queue safety) -
hopefully I will be able to extract some lessons from this :)

Thanks for the discussion, Martin - really keeps the neurons from
clogging :)
No problemo - hope your app turns out well :-)
mh.