UnboundID LDAP SDK for Java 1.1.5 and 2.0.0-rc1

I have just released a new 1.1.5 release of the UnboundID LDAP SDK for Java. It is available for download from the UnboundID LDAP SDK product page, as well as from the SourceForge project page. It should be available in the Maven Central Repository in the near future.

This release has only a few changes over the previous 1.1.4 version, including:

  • I have fixed a bug which could cause problems when using SSL or StartTLS over high-latency networks.

  • I have fixed a bug which could cause LDAPConnection.close to close the connection without an unbind request if the connection had previously been closed and re-established.

  • I have added support for the content synchronization operation as defined in RFC 4533.

I have also released a new 2.0.0-rc1 build of the LDAP SDK, which is available on the SourceForge project page. This is hopefully the last build prior to the official release. It includes the above changes, as well as the following additional changes:

  • The default object encoder in the persistence framework now provides support for URL, URI, and enum objects.

  • Java source code generated by the persistence framework now includes static getPersister and decode convenience methods which make it easier to interact with objects using the persistence framework.

  • LDAP command-line tools now provide an option that allows you to specify the format for the key store and/or trust store when using SSL or StartTLS. It still uses the JKS format by default, but you can request an alternate format like JCEKS or PKCS12 if desired.

  • The LDAP assertion control implementation now includes a generate convenience method that makes it easy to generate an assertion request control based on a provided entry. It can be used to make it easier to ensure that some or all of the attributes in the entry have not been updated since the entry was last retrieved.

  • I have added a new BooleanValueArgument class, which is similar to the BooleanArgument class except that it takes an explicit value (e.g., “true”, or “false”, although it will accept other strings as well, including “yes”, “on”, and “1” for true, and “no”, “off”, and “0” for false) rather than inferring the value based only on whether the argument is present.

  • I have added a new DurationArgument class, which combines an integer with a time unit (e.g., “10ms”). It can be used to specify lengths of time.

  • I have updated the FileArgument class so that it is possible to specify a base DN for relative paths. By default, relative paths will be relative to the current working directory, but you can indicate that they should instead be relative to a specified parent directory.

UnboundID LDAP SDK for Java 1.1.4 and 2.0.0-beta1

I have published two new builds of the UnboundID LDAP SDK for Java. The 1.1.4 build is an official release that contains a number of minor enhancements and bug fixes over the previous 1.1.3 release. The 2.0.0-beta1 build is a preview of our next major release and includes significant new functionality, including frameworks for persisting Java objects in an LDAP directory and for more easily creating applications which accept communication from LDAP clients. The 1.1.4 version is available for immediate download on the UnboundID website, and both the 1.1.4 and 2.0.0-beta1 builds are available for download from the SourceForge project page. The 1.1.4 release is fully supported for production use and has a stable API that will not change in incompatible ways in the future. Although we also believe that the 2.0.0-beta1 build is also production ready, we do reserve the right to change any of the new APIs prior to the official release if there is a compelling reason to do so.

Some of the most notable changes between the 1.1.3 and 1.1.4 releases include:

  • I have fixed a bug that could cause problems when trying to use SSL or StartTLS in conjunction with synchronous communication mode. This problem primarily occurred in deployments in which the client and server were not on the same network and there could be a significant delay between phases of the negotiation.

  • A new search-and-mod-rate command line tool has been added that can be used to test LDAP server performance under a combined load of search and modify operations. The searches will be performed in much the same way as with the searchrate tool, but each entry returned will also be modified.

  • The ldapsearch tool has been updated to provide an option to repeat the search multiple times at regular intervals. This can be useful, for example, if you want to periodically retrieve monitor information from the server.

  • The traditional and thread-local connection pool implementations have been updated to provide an option to avoid closing and re-establishing a large number of connections in a short period of time in the event that a maximum connection time limit has been configured. It is now possible to specify the minimum length of time that should pass between closures of expired connections.

  • If a problem is encountered while trying to decode a response read from the directory server, the exception that is thrown now includes better information about the problem that actually occurred to make it easier to diagnose the problem.

  • The searchrate, modrate, and authrate tools have been updated to better handle the case in which the target directory server becomes unavailable during processing. The tools are now better able to re-establish connections to that server when it becomes available again.

The 2.0.0-beta1 build includes everything in the 1.1.4 release, as well as additional new functionality. In previous blog posts, I have discussed some of the features added to the LDAP SDK persistence framework, and that functionality has been further improved in this new version. Some of those changes include better support for operational attributes, the ability to use lazy initialization for a specified set of attributes, and adding support for simple bind operations. It also includes a new listener API, which makes it possible to easily create applications capable of accepting communication from LDAP clients. A new ldap-debugger example tool makes use of the listener API and can serve as a simple proxy which decodes all LDAP communication that passes through it.

We are very interested in getting feedback on the new APIs included in the upcoming 2.0.0 release because we are committed to maintaining compatibility once they have been officially released. If you are interested in either the persistence or the listener APIs, please at least look over the javadoc and let us know what you think.

How Hard is LDAP?

I’ve recently seen a lot of traffic referencing an image depicting the history of a number of LDAP-related RFCs. That image is available in a number of locations, including http://commons.wikimedia.org/wiki/File:LDAP_RFC_Hist.jpg (I don’t generally like wikipedia, but it seems to be the most professional/reputable of the sites containing the image). Most of the comments associated with this image are along the lines of “look at how hard LDAP is to understand”. While it’s certainly true that you need to read a few RFCs if you want to become an expert in LDAP (especially if you want to write your own fully-compliant directory server), I do think that this is a bit of an unfair conclusion to draw from this image.

First, the sheer number of RFCs pertaining to a topic is not necessarily a direct indicator of how complex that topic is. For example, most people don’t consider telnet to be a particularly complex protocol, but according to http://www.faqs.org/rfcs/np.html there are 105 different RFCs pertaining to telnet versus 89 for LDAP, and both of them are dwarfed by the list of 207 RFCs pertaining to DNS. I’m sure if you created a similar diagram showing the relationships between those telnet or DNS RFCs you’d probably come up with something that puts the LDAP version to shame, however I don’t hear anyone complaining about how hard they are.

Second, I think that the diagram illustrates how LDAP is evolving over time. The arrows in the diagram show changes over time, so it’s not like all of those RFCs cover completely different topics. For example, the LDAPv2 protocol was originally described in RFC 1487, and then revised in RFC 1777, then updated again in RFC 2251 for LDAPv3, and once more in RFC 4511. If you want to understand LDAP at the protocol level, you only need to read RFC 4511 since it is the most comprehensive and up-to-date of the protocol specifications. In fact, reading just RFCs 4510-4519 would provide a pretty comprehensive understanding of most LDAP-related concepts.

Third, the relatively large number of LDAP-related RFCs is a good demonstration of the openness that LDAP exhibits. A handful of RFCs provide me with all the information I need to write a standards-compliant LDAP client, which should be capable of talking to any standards-compliant LDAP directory server, or I could even write my own LDAP server. Contrast that with something like the relational database world. I can’t write my own client for talking with an Oracle database, and even if I could, it wouldn’t be able to communicate with DB2 or MySQL or any other type of server. It’s true that there is an ANSI SQL standard, but even then it’s kind of a crap shoot about which parts of that standard a given database follows.

Ultimately, I think that the number of LDAP-related RFCs and the relationships between them has very little to do with how easy it is to understand, and beyond that how easy it is to use or write an LDAP client or server. It’s definitely true that some directory servers or client APIs are easier to use or more full-featured than others. I spend a lot of my time working to make sure that the UnboundID products are as easy to use as possible, and ensuring that we do so without sacrificing performance, scalability, or functionality, and I know that other vendors have people trying to do the same for their products. There are definitely some poorly-written applications out there that use LDAP, and it’s unfortunate that they can give people a bad impression of LDAP itself, but hopefully that will also improve over time.

Lazy Loading in the LDAP SDK Persistence Framework

I have just committed a new change to the LDAP SDK persistence framework that allows you to perform lazy loading for a specified set of attributes. This feature makes it possible to create fields whose values will normally not be populated when retrieving objects from the directory, and then load them on demand if they are needed. This can be useful for cases in which a particular attribute may only be occasionally needed but could be rather expensive to retrieve, for example because it has a very large value (or a lot of values that are collectively large) or because it is a virtual attribute that is expensive for the server to create.

There are two components to this feature:

  • The @LDAPField annotation type has been updated to provide a new lazilyLoad element. This is false by default, but if it is changed to true, then the associated attribute will not be retrieved when using any of the LDAPPersister.get or LDAPPersister.search methods.

  • A new LDAPPersister.lazilyLoad method has been added that can be used to retrieve the values of all or a specified subset of the lazily-loaded attributes and initialize the corresponding fields.

The generate-source-from-schema tool now has a new --lazyAttribute argument that can be used to request that the specified attribute be marked for lazy initialization in the generated source file. Of course, you can always manually update the @LDAPField annotation of an existing source file, or add it to any source file that you create from scratch rather than using the tool. On a marginally-related note, the generate-source-from-schema tool has also been updated to add an --operationalAttribute argument that can be used to include operational attributes that aren’t part of any of the associated object classes. And if desired, you can have operational attributes lazily loaded in the same way as user attributes.

UnboundID LDAP SDK for Java 2.0.0-alpha1

We have just published an alpha release of the next major version of the UnboundID LDAP SDK for Java. This release includes the first official look at the LDAP persistence framework that I’ve been working on, which should make it much easier for developers to create Java applications that store information in an LDAP directory server. You can download this build on our SourceForge project page at https://sourceforge.net/projects/ldap-sdk/files/.

I’ve talked about the LDAP SDK persistence framework in previous posts, but there have been some significant changes since my last post about it. Further, while we still reserve the right to make changes before the official release if there is a very compelling reason to do so, we believe that the API we have now is suitable for the long term and will allow us to build additional features around it without breaking existing applications. Backward compatibility is very important for us, so we wanted to take some time to get it right. We’ve already gotten third-party feedback on the state of the API, but we’re definitely interested in getting other opinions. If you see something that doesn’t make sense or think could be improved, then please let us know, either via e-mail (ldapsdk-support@unboundid.com or ldap-sdk-discuss@lists.sourceforge.net) or our discussion forums (https://sourceforge.net/projects/ldap-sdk/forums/forum/1001257).

I’ll provide an example that demonstrates the use of this API in an upcoming post. There is some basic documentation, including a simple example, in the docs directory of the zip file, and that will be improved over time.

Some of the changes that have been made in the persistence framework since my last post include:

  • The LDAPPersister class now provides a getInstance method that you can use to obtain an instance for dealing with a specific type of object. This is a little cleaner than requiring you to invoke the constructor because you don’t have to repeat the class name as many times to satisfy Java’s constraints around the generics framework. It also allows instances to be cached and reused if another request is made for the same type of object.

  • Some of the classes used have been renamed to make them less confusing and more developer-friendly. LDAPFieldEncoder has been renamed to ObjectEncoder, since it is responsible for encoding more than just field values. The LDAPFieldGetter and LDAPFieldSetter annotation types have been renamed to LDAPGetter and LDAPSetter, respectively, since they work with methods rather than fields. The PersistSearchListener interface was renamed to ObjectSearchListener to avoid potential confusion with the persistent search control as defined in draft-ietf-ldapext-psearch.

  • The LDAPGetter and LDAPSetter annotation types have been updated so that the attribute element is no longer required to specify the name of the associated attribute. The attribute name can now be inferred from the name of the method. For example, if a method named “getGivenName” is marked with the LDAPGetter annotation, then it will be assumed that the name of the associated LDAP attribute is “givenName” unless that annotation includes an explicit attribute element.

  • The methods for performing searches in the persistence framework have been improved to give the developer better control over the search request that will be generated. It is now possible to include controls in the request, and you can also specify an additional search filter that will be ANDed with the filter generated from the object provided as a template. In addition, a number of searchForObject methods have been added that make it more convenient to perform a search that should only match a single entry.

  • The generate-source-from-schema tool has been updated so that it now provides better support for references to other entries. For any attributes with a DN syntax, the tool will now generate two types of getter methods: one for retrieving the DN(s) stored in that attribute, and another for obtaining the entries that they reference, decoded as objects of a particular type. For example, you could use this capability to retrieve a GroupOfNames object, and then easily iterate across the members as InetOrgPerson objects.

  • The persistence framework now provides a method that can be used to add schema definitions for the type of object you’re working with to an LDAP directory. This is primarily intended to be used as a convenience method, and it may not work with all types of directories since the process for updating directory server schema hasn’t been standardized, but it can be useful during the development and testing phases for an application designed for use with the persistence framework.

  • I have made some of the metadata-related classes public, including LDAPObjectHandler, FieldInfo, GetterInfo, and SetterInfo. These classes are used to perform some of the internal processing and validation, but they can also be used to obtain useful information about the structure of objects used with the persistence framework. You can use them to introspectively discover information about the fields, getter methods, and setter methods used for those objects.

SLAMD 2.0.1

I have just released a new version of SLAMD, version 2.0.1. It is available for download at http://www.slamd.com/. The release notes provide a pretty complete list of the changes included in this release, but some of the most notable updates include:

  • The code used to generate graphs has been improved in a number of ways. Various font sizes and styles are now used to make elements of graphs stand out, and antialiasing has been enabled for the text. The thickness of lines in line graphs has been increased and a light gray background is used for the plot area to help make the lines easier to see, especially for light colors like yellow. Also, the default graph size has been increased from 640×480 to 800×600.
  • When scheduling a job, the job duration and/or statistics collection interval can be specified in a more human-readable format. You can still use the number of seconds like in the past, but you can also use more user-friendly forms like “6 minutes”, “1 hour”, or “1 day 12 hours”.
  • A new version of the NetStat resource monitor is available that provides a number of bug fixes and increased functionality. It is now possible to configure the set of interfaces to monitor and/or to have statistics aggregated across the monitored interfaces.
  • The HTTP GetRate job has been updated so that it provides the ability to authenticate to a proxy server.
  • A bug has been fixed that could cause a problem with the use of the HTTP client if the response does not include a Content-Type header.
  • A bug has been fixed in a number of LDAP-related jobs that could cause them to behave incorrectly if a value pattern was specified using a sequential pattern. That pattern could be repeated by all threads on the client, instead of having the same range shared by all threads so that each value was used by only one of the threads.
  • A bug has been fixed in a number of jobs that could prevent statistics from being collected if a cool-down time was provided but no duration or stop time was defined.
  • Jobs providing rate-limiting capabilities have been updated so that you can control the period of time over which the rate limiting is enforced. This can be used to help make it more reliable over time, with the trade-off of occasionally exceeding the rate limit for a brief period of time to make up for periods of slower activity.
  • New jobs have been added which can be used to perform asynchronous LDAP search and modify operations with multiple concurrent outstanding requests on client connections.
  • The LDAPDecoder tool has been updated to provide the ability to decode LDAP traffic from snoop or tcpdump packets in which LDAP messages do not exactly align with packet boundaries (e.g., packets containing multiple messages, and/or messages spanning multiple packets). Also, when generating a SLAMD script to reproduce the LDAP communication, it is now possible to exclude information about server responses.

UnboundID LDAP SDK for Java 1.1.3

We have just released version 1.1.3 of the UnboundID LDAP SDK for Java. The full release notes can be found at http://www.unboundid.com/products/ldapsdk/docs/release-notes.php, but some of the changes in this release include:

  • It now includes support for LDAP transactions as defined in the recently-released RFC 5805. You will need a directory server that provides support for this capability (like the UnboundID Directory Server) in order to be able to use it.
  • It is now possible to determine the number of operations in progress on a connection that is established and not operating in synchronous mode.
  • A new searchForEntry method has been added to LDAPInterface (implemented by connections and connection pools) to make it more convenient to perform searches that you expect to return at most one entry.
  • The add request object has been updated to make it easier to treat it as if it were an entry.
  • The schema parsing code has been updated to be more lenient about accepting quotes around OIDs and overlooking missing spaces after OID lists. These changes help make it possible for the LDAP SDK to parse the schema presented by Active Directory even though it is in violation of the LDAP specification.
  • It includes a fix for an obscure bug that could cause an exception to be thrown when trying to normalize a value containing only non-ASCII space characters.
  • It includes a fix for a bug that would cause the entry validator to throw an exception when provided with an entry that did not contain any object classes.
  • The searchrate tool has been updated to provide support for processing asynchronous operations so that there can be multiple outstanding requests on each connection.
  • The source for the Android LDAP client has been updated to be the source for the 1.0 version released to the Android Market.

Large result sets in the LDAP SDK

One of the things that I think is particularly nice about the UnboundID LDAP SDK for Java is the way that it allows you to perform a search and have it collect the matching entries in a list that is available in the search result. However, this is really only well suited for cases in which you’re sure that you won’t get a huge number of entries returned because otherwise the need to hold all of the matching entries at once can cause significant memory problems.

However, if you are going to be dealing with large search result sets, then the LDAP SDK provides a couple of additional APIs that may be of use. The SearchResultListener interface defines methods that can be invoked whenever an entry or reference is returned by the server that allows you to act on that entry or reference as soon as it is received. I’ve had a number of people ask for an example of how to use this interface, so I’ve created a simple program, WriteAttrToFileUsingListener.java, that you can use to accomplish this. It’s a pretty simple program that performs a search to retrieve all entries containing a specified attribute, and then writes all of the values for that attribute to a specified output file. It’s a little more complex than it absolutely needs to be in order to demonstrate just the SearchResultListener interface, but it also serves as a nice example of the LDAPCommandLineTool API that you can use to easily write command-line utilities that need to talk to a directory server.

We also have another class, LDAPEntrySource, which can be used to make dealing with large result sets easier. This class provides an implementation of the EntrySource API (which makes it easy to iterate across entries in a common way regardless of how they were obtained, like returned as search results or read from an LDIF file), and you can treat it kind of like an iterator across search entries. I’ve created another version of the example program, WriteAttrToFileUsingEntrySource.java, that demonstrates how to use the LDAPEntrySource as an alternative to SearchResultListener to achieve the same result.

A new “PlugProfile” Android app

Ever since I got my Droid working properly, it is by far the best phone that I’ve ever had. However, one notable deficiency is the fact that it lacks a good way to change ring profiles. There are a lot of applications in the Android Market that can help with this, but none of them did exactly what I wanted, so I decided to write my own.

Several years ago, I got an LG V phone, and it had a really nice and sensible feature that made it possible for you to change how it behaved when a call came in based on whether or not it was plugged in, and I used that to configure the phone to vibrate when it was unplugged (and presumably in my pocket) but to ring at maximum volume when it was plugged in (and obviously not in my pocket).

A couple of years ago, I traded in my V phone for a Blackberry. It didn’t directly support changing the ring profile based on whether it was plugged in or not, but it did support changing the profile based on whether it was in the holster. Since I carried the phone in my pocket and didn’t use the holster, that was good enough because I could put it in the holster whenever I plugged it in to achieve the same effect.

Late last year when I upgraded from the Blackberry to the Droid, it was an upgrade in nearly every way except ring profile management. The phone itself comes with basically nothing for changing the way it rings under different conditions. There are a lot of apps in the Android Market that try to address this problem, and there are some pretty inventive solutions like using information about your location or based on the time of day. However, none of these were a very good fit for my needs. I ended up using “Quick Profiles”, which just let you create different profiles and then manually switch between them, but that wasn’t ideal because I found that I frequently forgot to change the profile when unplugging my phone and putting it in my pocket in the morning and taking it out and plugging it back in in the evening, so it was frequently configured to ring when I didn’t want it to, or vibrate when I wasn’t around to feel it.

The model exhibited by the V phone was just about perfect for me, so I decided to go ahead and write an app to do that for Android. It was a pretty quick and painless process, and the app is now available for free in the Android Market with the name “PlugProfile”. It provides the ability to automatically set the phone to silent mode, vibrate-only mode, ring-only mode, or ring-and-vibrate mode based on whether it’s on battery, on AC power, or on USB power, and if ringing is enabled, you can set the volume from anywhere between 10% and 100% of the maximum volume in 10% increments. I have my phone set to use vibrate-only mode when it’s on battery, ring with 100% volume when it’s plugged into AC power, and ring with 50% volume when it’s plugged into USB (of a presumably nearby computer). It doesn’t mess with your ringtone, so if you’ve got different rings for different people, then that should still be preserved.

LDAP Client now in Android Market

Ever since I started looking at Android a little over a year ago, I’ve had a simple LDAP client in one form or another. Since the UnboundID LDAP SDK for Java works on Android, it wasn’t too difficult to put a simple GUI on top of it that allows you to perform LDAP searches. However, until recently it wasn’t in a state that I felt was suitable for publishing. Prompted by the Android Developer Labs (which I attended earlier tonight), I finally got around to making it presentable, and as of a few minutes ago, the app is now available for free in the Android Market. It’s far from a masterpiece, but it can be pretty useful if you want to access LDAP content. Some of the features it has include:

  • It has support for multiple servers. Each server definition includes an address, port, security mechanism (none, SSL, or StartTLS), optional bind DN and password, and optional base DN.
  • You can customize the type of search to perform. It has a drop-down that allows you to select the type of search (last name, first name, full name, e-mail address, or user ID), or if you want you can enter your own LDAP search filter.
  • If multiple entries are returned, you can see a brief summary of each. Tapping on one of them will take you to a more complete view of the entry. Long-tapping will pop up a menu with options for the entry (view a formatted representation, view an LDIF representation, copy the DN to the clipboard, or copy the LDIF representation to the clipboard).
  • When viewing a single entry, clicking on the header for that entry will allow you to view the entry as LDIF, copy the DN to the clipboard, or copy the LDIF representation to the clipboard.
  • Clicking on a telephone number in an entry will allow you to dial or send an SMS message to that number, or copy the number to the clipboard.
  • Clicking on an e-mail address in an entry will allow you to send an e-mail to that address, or copy the address to the clipboard.
  • Clicking on a postal address or ZIP code in an entry will allow you to show a map of that location, navigate to that location, or copy the address to the clipboard.
  • Clicking on any other attribute in an entry will allow you to copy the value of that attribute to the clipboard.
  • A button at the bottom of the panel for a user entry will allow you to add information about that user to your local contacts.

If you have an Android device, then you can find this application in the market just by searching for “LDAP” (it’s currently the only match). The full name is “LDAP Client” and the author is “Neil Wilson”. I hope to improve it further in the future, but I at least wanted to get this reasonably-functional version out there for people that have a use for it.