What It Does
This article details the threading model that has been employed by the SSHD API. The article details the basic principle behind each of the two choices and aims to highlight why the NIO model was chosen over the thread-per-connection model.

The second part of this article highlights some of the ConfigurationContext threading-related methods that are available and can be used to affect the performance of the SSHD server if the default values are not producing the required results.

By the end of the article the reader should have a good understanding of the threading technique used by the API and why it was chosen. In addition the reader should have a better appreciation of the SSHD API have gained an awareness of at least one way in which the SSHD server performance can be fine tuned.

The Threading Model
A server's ability to handle numerous client requests within a reasonable timeframe is dependent on how effectively the server uses I/O streams. The SSHD API's vision as a scalable high performance API must cater to hundreds of client requests simultaneously, having to use I/O services concurrently. It goes without saying that the API has to incorporate some form of threading policy to handle the plethora of incoming requests.

Thread-per-connection
The two main threading architectures that exist are thread-per-connection, a synchronous based model and the newer NIO threading, a asynchronous model based on the reactor design pattern. The NIO model was chosen as the architecture of choice for the API and a brief summary of the architecture is detailed further in this article. The main problem that was encountered by using thread-per-connection was its inate ability of blocking on essential I/O tasks. The core functions performed by a server can be identified as, accepting incoming connections, reading requests from connected clients and ultimately servicing those requests. With a server using the thread-per-connection threading architecture accepting incoming request is carried out by the ServerSocket::accept() method and writing data through OutputStream::write().

Due to the nature of these methods the running thread has to stop on these calls as these methods block, stopping the thread from performing any other function until either a client has made a connection to the accept method call or the writing buffer is empty of the write method call. Once this condition is satisfied only then is the running thread allowed to continue. In a scenario where connections are serviced one at a time this is not an issue however, if a server is trying to multiplex a number of client connections it does becomes a problem.

When we are trying to service more than one connection only one socket is ever serviced at a time regardless of the fact that the other two ServerSocket objects have clients trying to establish a connection, this might not look too problematic but in a large scale environment with 100s of client connections waiting to be serviced this can be detrimental to performance.

In this threading model even the process of reading data is done in chunks till the assigned buffer is filled, if no data is received or only partial data is received the calling thread will remain blocked pausing the server from other activities.

fig.1 Thread-per-connection blocking processes.

This is not to say that these concerns cannot be overcome, with the ease and simplicity in coding one can quite easily employ a liberal amount of threading activity to resolve any such issues. But with thread-per-connection it has an almost one-to-one ratio of threads to clients and since an SSH server built through the SSH API would probably be expected to serve hundreds of connection this increase in threading activity would seriously affect performance and scalability of the API.

Non-blocking IO
The principal force behind the design of NIO was the reactor design pattern. Server applications in a distributed system must be able to handle multiple client requests, before invoking a specific service the server application must demultiplex and dispatch each incoming request to a corresponding service provider. The reactor pattern serves precisely this function. It enables event-driven applications to demultiplex and dispatch service requests, which are then delivered concurrently to an application from one or more clients.

The NIO threading model uses an entity called a Selector that represents the Reactor, multiplexing events and passing them to associated request handlers. Prior to starting, each request handler aka SocketChannel registers which event it is interested in, such as reading data from connection, with the selector. When events arrive from clients the selector demutliplexes them and dispatches the events to the corresponding SocketChannel to deal with.

It is not this that overcomes the blocking issues present in thread-per-connection rather the concept of an entity called a key, where a key doesn't represent the entire information stream a client sends to a server, but rather just a part. The selector divides the client-data in to sub-requests identified by keys. Consequently, if more clients continuously send data to the server, the selector creates more keys, which are processed according to a time-sharing policy. To emphasize this in Figure 2 the keys have the same color of their related clients.

fig.2 NIO threading model

In addition the SocketChannel objects themselves can also be asynchronously closed and even interrupted allowing any blocked I/O thread to be closed by other SocketChannel objects. This is unlike the thread-per-connection model where a thread will block on a read or a write until the operation is completely finished in this model there is no direct mechanism of interrupting the thread.

This non blocking mechanism enables a thread to read whatever amount of data is available and then return to perform other tasks.

Due to the high grade performance requirements of the SSHD API the need for a robust solution is very important with NIO it encourages a reduction in thread usage which adversely limits the threading issues that arise with a high number of threading activity. Though this threading model is a little more complex to code the net result of a high performance and robust scalable API far outweighs any development complexities. For synchronous and somewhat simpler solutions the thread-per-connection is a viable solution but for the requirements of the API the NIO threading model was best.

Confugration settings
An SSH server is a complex piece of software and depending on the situation in which it is working the performance of the server can be very different. The following section details some of the main performance related methods that can be called and used to affect, in some way, the actions of the server with a net effect of altering the performance statistics of the server itself to some degree.

All the methods listed are taken from the ConfigurationContext object, further details can also be taken from the respective javadoc.

ConfigurationContext::setAsynchronousFileOperations(boolean asynchronousFileOperations)
Flag used by SFTPSubSystem class to determine whether SFTP objects FileSystemOperationThread should run as a new thread (true), spawning a thread for each SFTP session or, (false) synchronously using the selector thread to perform SFTP operations. File operations may block or take a considerable amount of time to complete, if this flag is false then these operations will be executed on the selector thread thus the server will have to wait until these lengthy processes are completed before being able to respond to other selection events.

ConfigurationContext::setIdleServiceRunPeriod(int idleServicePeriod)
By default every 10 seconds idle service run is performed, run evaluates all channels on SelectorThread object to determine if any idle events need triggering. Idle events are used to give time to SelectableChannels to action their received events, reading data, writing data etc. This value is used during initialisation of all 3 SelectorThreadPool objects, the Accept thread pool is used to accept connections from clients, the Transfer thread pool used to process the connections and the Transfer thread pool used to handle all comms for a connection.

ConfigurationContext::setInactiveServiceRunsPerIdleEvent(int inactivePeriodsPerIdleEvent)
An idle event is not passed to an idle event listener until the value of inactiveServiceRunsPerIdleEvent has been reached. With the default service run period of 10 seconds and the default value of 3 for inactive service runs per idle event, then if a channel is inactive for more than 30 seconds an idle event is fired.

ConfigurationContext::setMaximumChannelsPerThread(int maximumChannelsPerThread)
Set maximum number of SelectableChannels that can be assigned to the accept, transfer and connect selectors. Value of 1 effectivley makes server behave in thread-per-connection mode.

ConfigurationContext::setMaximumPublicKeyVerificationAttempts(int maximumPublicKeyVerificationAttempts)
Defines the number of failed public key exchange attempts. If default value of 10 retries is exceeded connection is terminated.

ConfigurationContext::setOptimisticallySendPackets(boolean optimisticallySendPackets)
Changes the way SSHD sends packets to the client. Normally when using NIO the selector object needs to be informed of a write request, a delay is encountered before write event is triggered after this trigger writing of data to socket is performed. If optimistically send packet flag set to true TransportProtocol will attempt to write data packet immediately without waiting for a write event to be triggered.

ConfigurationContext::setPermanentAcceptThreads(int permanentAcceptThreads)
The accept thread services process requests awaiting connection, once accepted thread is registered with a transfer thread which handles necessary I/O operations. This method defines the number of permanent accept threads to be used by the server, though if additional threads are required due to over worked permanent threads server will dynamically create more.

ConfigurationContext:: setPermanentConnectThreads(int permanentAcceptThreads)
The connect thread services connection process, once connection is established the socket is registered with a transfer thread. This method defines the number of permanent connect threads to be used by the server, though if additional threads are required due to over worked permanent threads server will dynamically create more.

ConfigurationContext:: setPermanentTransferThreads(int permanentTransferThreads)
Once a socket has either been accepted or connected, the socket is registered with a transfer thread. Transfer thread asynchronously performs all I/O operations for the socket. This method defines the number of permanent transfer threads to use at start-up; if all threads become fully loaded additional threads will be dynamically created to service additional connections, once done these dynamic threads are removed.

ConfigurationContext::setSelectorProvider(java.nio.channels.spi.SelectorProvider selectorProvider)
Supply a different SelectorProvider class for creating the accept, transfer and connect thread pools other than the default JDK version.

Summary
As can be seen the SSHD API uses the NIO threading model to achieve a more robust and scalable solution that would be otherwise unachievable through the older thread-per-connection model. To maintain a high level of flexibility still the SSHD API enables the workings of the low-level server to be modified through numerous amounts of variables accessible through the ConfigurationContext object, the heart of the SSHD server.

The default values for these variables should be sufficient for most environments but the ConfigurationContext opens these up to allow adjustments if in certain circumstances these are not acceptable. Each adjustment however will affect performance slightly differently in each situation, in some it will increase performance visibly in others, not at all, it really comes down to tweaking parameters in the right manner dependent on your environment.

2005 SSHTools Ltd, All Rights Reserved