The UnboundID LDAP SDK for Java version 0.9.10 has just been released. The release notes provide the complete set of changes, but this post will outline two of the most significant changes in this release.

Synchronous Mode

The LDAP SDK now includes an option to operate in synchronous mode. If an application using the LDAP SDK does not need to perform asynchronous operations, and if it does not attempt to process multiple operations on the same connection at the same time, then you can use the new synchronous mode to achieve a significant increase in throughput and reduction in response time. In my testing, I have generally seen a performance improvement of about 15-20% by enabling the option to use synchronous mode.

Prior to this release, the LDAP SDK only operated in an asynchronous manner, in which each connection had a dedicated reader thread that could be used to read responses from the server. Whenever a request was sent to the server, a blocking queue was used to provide communication between the reader thread and the thread that sent the request. After sending a request to the server, the thread that sent that request would block on this queue waiting for the response from the server to be provided by the reader thread. This design is necessary to be able to handle the possibility of asynchronous operations, in which multiple requests can be outstanding at the same time, but it does incur overhead as a result of thread synchronization and context switching. This asynchronous manner of communication is still the default, but if your application doesn’t need the ability to process multiple operations on the same connection at the same time, then you can choose to use the new synchronous mode for better performance.

To enable synchronous mode, all you need to do is to set the useSynchronousMode connection option on the connection before it is established. For example:

LDAPConnectionOptions options = new LDAPConnectionOptions();
options.setUseSynchronousMode(true);
LDAPConnection connection =
new LDAPConnection(options, host, port, bindDN, password);

You shouldn’t need to make any other changes to your application, since everything else should be completely transparent. If you do attempt to invoke an asynchronous operation (or attempt to perform an abandon or invoke a cancel extended request, which rely on the ability to perform asynchronous operations), then the SDK will throw an LDAP exception with a “not supported” result.

The synchronous mode is completely compatible with the use of connection pools, and in fact if you are currently using connection pools in your application then it’s very likely that your application meets the requirements for using synchronous mode, since it’s not all that easy to invoke asynchronous operations using pooled connections or to use a pooled connection concurrently across multiple threads.

There has been some other restructuring in the LDAP SDK to provide for this capability, and those underlying changes may also provide some level of benefit even for applications using the default asynchronous mode.

Spurious “Server Down” Responses

A user reported periodically getting “server down” responses, especially when reading large result sets. In the UnboundID LDAP SDK for Java, the “server down” response indicates that a previously-established connection is no longer usable, and generally indicates one of two things:

• The connection between the client and the server has been lost unexpectedly, most likely because of a network problem or because the server closed the connection.
• The client encountered an unrecoverable error while decoding a response from the server that prevents it from being able to perform any more communication over that connection.

Upon closer investigation, I did find that there was a corner case in which the LDAP SDK could experience a problem if it was in the process of decoding a response from the server in which the entire LDAP message for the response was not sent all at once but instead came in multiple pieces and there was a delay of at least 50 milliseconds (which is the default socket timeout in asynchronous mode, needed to handle a corner case for StartTLS processing) when trying to read a subsequent piece of the response message. In this case, the LDAP SDK had a problem in which it would try to start reading a completely new response rather than finishing the old response, and it would encounter an unrecoverable decoding error when trying to read in the middle of the old message as if it were the beginning of a new one.

This problem has been corrected, and the LDAP SDK will now properly handle this condition and is more tolerant of delays encountered when reading a response in multiple segments.

The UnboundID LDAP SDK for Java version 0.9.9 has just been released. The release notes provide the complete set of changes, but some of the most significant changes since the 0.9.8 release are listed below.

Maximum Search Response Queue Size

The LDAP SDK now places a maximum size limit on the blocking queue used to hold search result entries and references while waiting for them to be processed. When using a search result listener to consume these types of responses, this can help prevent problems in which the client could potentially run out of memory if a large number of search results are returned faster than the client can process them. With this change, if the client falls behind then the LDAP SDK may eventually create back-pressure against the directory server so that it will only be able to return entries and references as fast as the client can consume them.

Referral Enhancements

It is now possible to control on a per-request basis whether the LDAP SDK should automatically try to follow any referrals returned by the server. This was previously only available on a per-connection basis. In addition, a bug has been fixed that could cause referral following to fail for search operations.

User-Friendly Names for Connections and Connection Pools

It is now possible to assign user-friendly names for individual LDAP connections or LDAP connection pools. These names can be helpful in applications which maintain multiple connections or connection pools for different purposes. These names can be queried from code, and will appear in thread stack traces for the threads used to read responses for those connections.

Debugging Enhancements

It is now possible to indicate that debug log messages should include a stack trace of the thread that generated the message. It is possible to capture a stack trace at the time that a connection is established. The debug log message format has been changed to make the messages more consistent and easier to parse. In the even that a connection is terminated because of an unrecoverable error while attempting to read data from the client, then the underlying cause for that is now more visible.

Additional Connection Pool Statistics

LDAP connection pools now expose additional statistics that can be used to keep track of how a connection was obtained when checking a connection out of the pool (e.g., whether it was obtained immediately without blocking, whether it had to wait for a connection to become available, or whether a new connection was created). This can be helpful when determining whether a connection pool is sized appropriately for the application that is using it.

The UnboundID LDAP SDK for Java version 0.9.8 has just been released. The release notes provide the complete set of changes, but the most significant changes since the 0.9.7 release are listed below. Because of the potential connection leak, anyone using connection pooling in the LDAP SDK is strongly encouraged to upgrade to this release.

StartTLS Bug Fix

Previous versions of the LDAP SDK had a bug in which attempting to process a StartTLS extended operation could fail to return if the server certificate was not trusted. The client connection was terminated, but the processExtendedOperation method would never return. This has been fixed.

LDAP Connection Pool Bug Fixes

Two bugs in the LDAPConnectionPool class have been corrected.

The first bug is that the connection pool could leak a connection when performing a background health check if that connection was examined multiple times in the same health check sequence.

The second bug is that when attempting to retrieve the root DSE from the directory server, two connections were obtained from the pool when only one was needed. Both connections were properly returned, but taking twice as many connections could have created an artificial connection shortage.

Operation Rate Limiting

A new FixedRateBarrier utility class has been added which can be used to help try to perform operations no faster than a specified rate. The authrate, modrate, and searchrate tools have been updated to use this and now provide a “--ratePerSecond” argument that can be used to attempt to perform authentications, modifies, and searches at a specified rate.

The UnboundID LDAP SDK for Java version 0.9.7 has just been released. The release notes provides information about the complete set of changes, but some of the changes since the 0.9.6 release are listed below. If you’d like to be notified by e-mail when a new release is available, send a message to ldapsdk-announce-subscribe@unboundid.com.

LDIF Reader Performance

A change introduced in the 0.9.6 release dramatically improved LDIF reading performance for large entries with attributes with a lot of values (e.g., large static groups). This change has been generalized so that it applies to entries of any size, which significantly benefits performance when reading smaller static groups.

Compact Entries

A new CompactEntry class has been added which can be used to represent a read-only entry in a form that requires much less memory than the existing Entry class, although with a potential performance penalty for some operations. This is primarily useful for the case in which it is necessary to store a large number of entries in memory, since it is possible to hold more entries in a given amount of memory. As an example, for a typical user entry, CompactEntry requires only 25% as much memory as Entry, and for a simple static group with 20 members, CompactEntry requires only 31% as much memory as Entry.

Android Demo Source

The source code for an application demonstrating the use of the UnboundID LDAP SDK for Java (mentioned in my last post) is now included.

Mobile devices are quickly becoming indispensable tools for convenience and productivity wherever you may happen to be. Around the beginning of the year, I started to look at the Android platform for developing applications for mobile devices (and other low-end kinds of systems). I picked up a Google developer phone (basically an unlocked version of the T-Mobile G1) and was impressed by the pretty rich API that it offers. Quite a large portion of Java SE is available for use, although there are some notable exceptions. For example, it’s missing all of the GUI classes, but that makes sense since Android provides its own GUI library. But another significant omission is that of JNDI, which is the API included in Java SE for communicating with LDAP directories.

On one hand, this is a good thing since JNDI is a horrible API that is inconvenient to use and doesn’t provide very good access to the LDAP protocol, and the less opportunity there is to use it the better. But on the other hand, being able to communicate with an LDAP directory server on a mobile device can be extremely helpful. For example, it could be very convenient to have access to the entire corporate directory at your fingertips for phone numbers and e-mail addresses without the need to clutter your contacts list, or maybe to keep local contacts in sync with their corresponding entries from the corporate directory.

The good news, though, is that Android developers don’t need to worry about missing LDAP support for Android because the UnboundID LDAP SDK for Java works just fine on the Android platform. I’ve done quite a bit of testing and made sure that things just work. With a couple of exceptions (primarily CRAM-MD5, DIGEST-MD5, and GSSAPI, which require the Java SASL API that isn’t available on Android) pretty much all of the functionality that we offer in our LDAP SDK can be used on Android. This includes all of the core protocol, all of the controls and extended operations, SSL/StartTLS, connection pooling, and all the other stuff is there, too. Note that there is no special “Android” version of the LDAP SDK — the same SDK that you can use for writing desktop clients or server-side applications works without any changes on the Android platform.

To demonstrate this capability, I’ve put together a simple LDAP client for Android devices. You can see a video online at http://www.youtube.com/watch?v=1NlhwrcCeSQ.

The UnboundID LDAP SDK for Java version 0.9.6 has just been released. This post details some of the changes included since the previous 0.9.5 release. See the release notes for additional information. If you’d like to be notified by e-mail when a new release is available, send a message to ldapsdk-announce-subscribe@unboundid.com.

Some of the changes in this release were based on feedback we received regarding the LDAP SDK. If you have suggestions or other feedback, feel free to send your comments to ldapsdk-support@unboundid.com.

Intermediate Response Messages

The LDAP SDK now includes support for LDAP intermediate response messages as defined in RFC 3771. Intermediate responses make it possible for operations return multiple responses to a single request. A new IntermediateResponseListener interface has been added, and clients can use the LDAPRequest.setIntermediateResponseListener method to be notified of any intermediate responses returned for the operation.

AuthRate Tool

A new authrate tool has been added as an example program. It is similar to the existing SearchRate tool, except that when retrieving an entry it can also perform a bind to verify the user’s password. It supports simple authentication, as well as the CRAM-MD5, DIGEST-MD5, and PLAIN mechanisms.

CommandLineTool Example Usages

The CommandLineTool API has been updated so that it is now possible to provide examples in the usage output. All of the example tools provided with the LDAP SDK now provide example usages.

Connection Pool Connection Expiration

It is now possible to define a maximum age for connections included in a connection pool. If a maximum age is defined and a connection in the pool has been established for greater than this length of time, then the connection will be closed and replaced with a new connection.

Connection and Connection Pool Statistics

A new LDAPConnection.getConnectionStatistics method has been added which can be used to retrieve an LDAPConnectionStatistics object with information about how that connection has been used. This includes the number of times the connection has been established and closed, the number of times each type of request has been sent, the number of times each type of response has been received, and the total and average response time for operations.

A new LDAPConnectionPool.getConnectionPoolStatistics method has been added which can be used to retrieve an LDAPConnectionPoolStatistics object which can be used to retrieve information about how the connection pool has been used. This includes the number of connections currently available, the maximum number of connections that may be available, the number of successful and failed attempts to create new connections, the number of connections that have been closed as defunct, expired, or unneeded, the number of successful and failed attempts to retrieve a connection, and the number of valid connections released back to the pool.

This feature was suggested by Philip Peake.

Connection Pool Health Checking

A new LDAPConnectionPoolHealthCheck API has been added which makes it possible to perform health checking for connections in a connection pool. It can be used to perform checks to determine whether a connection is valid when a new connection is created, checked out, released back to the pool, or after an exception has been encountered while processing an operation in the pool. In addition, a background thread will be used to periodically invoke health checks against all available connections.

A health check implementation has been added which can be used to verify the validity of a connection by retrieving a specified entry from the directory server. It can assess the connection’s availability based on whether the entry can be retrieved and the length of time required to retrieve it.

This feature was suggested by Philip Peake.

Improved Interaction with DNs and RDNs as Strings

A number of convenience methods have been added to the DN and RDN classes to make it easier to deal with them using their string representations. This includes the ability to determine whether a string represents a valid DN or RDN, to compare DNs and RDNs based on their string representations, and to determine whether strings representing DNs have a hierarchical relationship.

This feature was suggested by Felix Gaehtgens.

Improved LDIFReader Performance

The LDIFReader class has been updated to improve its performance when reading entries containing an attribute with a large number of values (e.g., a static group). For a single-threaded test case involving reading group entries with 2000 members, the new implementation was measured at over 360 times faster.

This problem was reported by Felix Gaehtgens.

One of the things that we’ve tried to do with the UnboundID LDAP SDK for Java is to make it as easy as possible to use. Some of this is kind of intangible (e.g., providing convenience methods for doing common things that might take multiple steps in other APIs), but there are things that I think are worth mentioning that we’ve done in this area to help developers write better applications. I’ll describe some of them here.

Extensive Javadoc Documentation

We provide a number of different types of documentation with the LDAP SDK (also available online at http://www.unboundid.com/products/ldapsdk/docs/), but I think that the most helpful documentation is the Javadoc that we generate from the source code. All public and protected methods are documented, including information about each argument and the value returned, and all public and protected fields are also documented.

Other than providing information about the available fields and methods, some of the most useful information included in the Javadoc is the class-level documentation that we provide. In many cases, this is quite extensive, and we include a number of code snippets that demonstrate how to use that class. For example, the class-level documentation for nearly all of the request types as well as the controls and extended operations provide examples for using them.

Another benefit of our Javadoc documentation is that we have generated it with the “-linksource” option, so that means that the entire source code for all of the documented classes is included as part of the documentation. This makes it possible to click on the name of a class or method in the Javadoc and be taken directly to the source code for it. In the event that you have a question about the API that can’t be answered by the Javadoc documentation itself, then using this to look at the source code itself can be very useful.

Annotations

The Javadoc documentation provides extensive information about how to use the LDAP SDK, but by itself there are some questions which a developer might have for which the answers aren’t immediately obvious. For many of these, we have defined annotations that may help make this clearer. Some of those annotations include:

• @ThreadSafety — In many APIs, there isn’t much documentation available about the behavior of the associated code when accessed concurrently by multiple threads. To help make it clearer for our LDAP SDK, all public classes and interfaces have been marked with the @ThreadSafety annotation that indicates how those types of object may be treated in multithreaded contexts.
• @Extensible / @NotExtensible — These annotations are used to mark interfaces and non-final classes to indicate whether they are intended to be extended by third-party code. There are some interfaces and non-final classes in the LDAP SDK which are primarily intended to be extended only by code within the LDAP SDK, and they have been marked with the @NotExtensible annotation. On the other hand, a number of interfaces and non-final classes are safe to extend by third-party code, and they are marked with the @Extensible annotation.
• @Mutable / @NotMutable — These annotations are used to mark classes to indicate whether objects of that type are allowed to be altered by third party code. Most classes which are marked @NotMutable are fully immutable so there is no way to alter their contents at all, but there are a few cases in which it is necessary for an object to be modifiable by code within the LDAP SDK, and in those cases the associated class is marked @NotMutable to indicate that those objects shouldn’t be externally altered. For objects which may be safely altered by third-party code, the classes are marked @Mutable.
• @InternalUseOnly — There are a few cases within the LDAP SDK where constraints enforced by Java require a class or method to be public but we don’t intend for it to be used by external code (e.g., a method required by an internal interface). In such cases, the associated class or method is marked with the @InternalUseOnly annotation to indicate that it’s not intended to be used by third-party code.

These annotations have a “runtime” retention, so they are accessible via reflection (and in fact, we have a number of unit tests to ensure that classes are marked correctly). They also appear in the generated Javadoc documentation, so that helps further improve the quality of the Javadoc.

Generics

Wherever appropriate, the LDAP SDK makes full use of generics. This is particularly true for all collections that are used anywhere in the code. This is very useful for developers because it helps them understand the types of element contained in that collection.

Debugging Support

If something goes wrong with a program using the LDAP SDK, then it may be helpful to debug the processing performed by the LDAP SDK. You can, of course, step through the code using a Java debugger, but you can also enable debugging support within the LDAP SDK itself. This uses the Java logging framework to provide messages about actions performed within the SDK, including:

• Establishing and closing connections to directory servers
• LDAP requests sent and responses received
• ASN.1 elements encoded and decoded
• LDIF entries and change records read and written
• Exceptions caught within the LDAP SDK
• Information about coding errors in which the LDAP SDK was used incorrectly (e.g., an argument that was inappropriately null or had a value outside of an acceptable range)

These types of debugging can be enabled or disabled on an individual basis, and you can control the minimum log level for messages that you want to see.

Debugging can be enabled or disabled programmatically by code using the LDAP SDK by calling methods in the com.unboundid.util.Debug class). However, debugging can also be controlled through the use of Java properties without the need to make any changes in the code using the LDAP SDK. The following properties are defined for use:

• com.unboundid.ldap.sdk.debug.enabled — If this property is set to “true“, then debugging will be enabled.
• com.unboundid.ldap.sdk.debug.level— If this property is set, then it may control the minimum debug level for messages to be written. Allowed debug levels include ALL, SEVERE, WARNING, INFO, CONFIG, FINE, FINER, FINEST, or OFF.
• com.unboundid.ldap.sdk.debug.type — This may be set to a comma-delimited list of the types of debugging to be used within the LDAP SDK. Available debug types include ASN1, CODING_ERROR, CONNECT, EXCEPTION, LDAP, LDIF, and OTHER. If this property is not set, then all debug types will be used.

If debugging is enabled, then it will use a logger name of “com.unboundid.ldap.sdk“. By default, messages will be written to standard error, although the Java logging framework can be configured to use a custom logger to send the messages elsewhere.

Disconnect Notification

With many APIs that provide LDAP communication, if a connection to the directory server is lost, then the application may have no way of knowing until it tries to use that connection, and at that point it is often not possible to understand when the connection was closed or for what reason.

The UnboundID LDAP SDK for Java provides a com.unboundid.ldap.sdk.DisconnectHandler interface, which may be implemented by third-party code to provide a mechanism to notify an application whenever it detects that a connection has been closed. This interface defines the following method:

void handleDisconnect(LDAPConnection connection,
String host,
int port,
DisconnectType disconnectType,
String message,
Throwable cause)

Whenever a connection has been closed, the disconnect handler’s handleDisconnect method will be called to notify it of the disconnect and provide information about it. The DisconnectType element will provide information about the reason for the disconnect, and can be used to indicate whether the closure was expected (e.g., by calling connection.close() to perform an LDAP unbind). If desirable, the disconnect handler may attempt to re-establish the connection (perhaps to a different server), which may help avoid user-visible failures that may result from attempting to use the connection.