Non-blocking IO and blocking queues
Well performing NIO code is a Good Thing. But getting details of how to do it right is difficult. The examples supplied with the JDK are complete noddy. There's a link on Sun's site to an example but that has a race condition. O'Reilly's NIO book cops out on how to use Selector properly. The thing that could make that book useful.
Real servers have multiple CPUs. Therefore more than one thread is necessary for effective performance. Indeed more than one thread selecting. You could dispatch to other threads, but that seems somewhat wasteful to me.<,/p>
You could partition the channels between single-threaded selectors, but I'm guessing that that is in general hit and miss. Apparently in multiprocessor configurations best performance often comes from splitting threads between reading (network to disk) and writing (disk to network), and hence having separate selectors for each. At least for a trivial web server. That only gets us to dual processor machines.
Consider a connection is accepted and that the new channel requires registration for read operations. If there are two otherwise idle threads, the other thread will be selecting and therefore locking out the registration, both on the same selector. So the registering thread calls wakeup on the selector. Unfortunately there's nothing to stop that discovering there is nothing waiting and regaining the selector lock before the register thread proceeds. My solution is instead of directly registering the new channel directly, put it on a queue, wake the selector and always check the queue for new register requests before every select.
Select operations are notified on absolute state levels. So if a channel can be written to, it wont block a selector. This is opposed to Windows-style edge triggering when the channel becomes writable. (Early Windows Java 1.4 versions had bugs.) So you only want to select on write when you have something to write and the channel cannot be written to (because it's buffer is full). Deregistering is a bad idea. It's really, really slow. However, re-registering with no operations appears to work well.
I'd like to see Tomcat use NIO. Keep-alive connections, parse headers, serve flat files and JSP buffers, upload files, perhaps even do XSLT transforms and some standard JSP tags without dropping down to the blocking model. That and de facto NIO server and client frameworks. Mind you I haven't looked in a while.
The problem of registering a channel while in select is similar to signalling unusual state between a consumer-producer pair of threads. As a simple example, Strings might be passed between the threads and very rarely the producer sends a quit message, say, to the consumer. As it happens the concurrency utils JSR (scheduled for inclusion in 1.5) includes java.util.concurrent.BlockingQueue. How do you signal the unusual message?
As it's rare, perhaps Thread.interupt() could be used. However, there is no knowing how the non-queue part of the consumer will react. You really don't want to interrupt that.
The consumer could just check before every item regardless. This may not be particularly efficient as the queue throughput may be very high. Unlike Selector there is no BlockingQueue.wakeup, so some kind of dummy object would be needed to stop the queue blocking. That object would need to be identified and removed by the consumer.
The obvious way to notify the consumer is to send a null. Unfortunately, unlike Collection, BlockingQueue does not permit null values for any implementation. Presumably as a null returned by poll() would become ambiguous (should polling be encouraged?).
Sending a particular dummy String instance could be more efficient. A particular empty String created with new String() and assigned to a private static final signals update miscellaneous state, rather than consume as usual. The consumer checks for zero length fodder, and then that it is == the special instance.
If instead of String the queue contains a more complex type then perhaps a strategy-style method in the type could take the appropriate action. Usually it would just do the usual consumer type thing, by particular subclasses take different actions. This relies on the queued type being amenable to change. If the type was something like String, then you'd have to wrap it in another object (which could just contain a possibly null String reference), and that would be relatively expensive.
If you consumer or producer is generic you really have no option but to wrap each queued item, with something like:
class Nullable<T> {
private final T obj;
public Nullable() {
this(null);
}
public Nullable(T obj) {
this.obj = obj;
}
public T get() {
return obj;
}
}
But that could cause a significant overhead. Alternatively roll your own blocking queue.
Please feel free to correct me.