Contraband

January 20, 2012 / Neil Wilson

I really don’t understand Mark Wahlberg’s movie selection process. In the past few years, he’s done some great films like The Fighter, The Departed, and Invincible, some perfectly adequate movies like Shooter and The Italian Job, and some absolutely horrible movies like The Happening and Rock Star. I didn’t expect Contraband to be great, but I’d hoped it would still be worth watching. Too bad that wasn’t the case.

Once upon a time, Chris and Kate Farraday (Mark Wahlberg and Kate Beckinsale) were an elite smuggling team able to get just about anything past customs. But after having a couple of kids, it was time to settle down and go straight. Kate became a stay-at-home mom, and Chris started his own company installing security alarms. There were still plenty of bad people who wanted to bring bad stuff into the country, but they just had to find other people to do it. Kate’s younger brother Andy (Caleb Landry Jones) decided to try his hand at it, but when border security agents got a little too close to his boat he dumped the cocaine overboard. This kept him out of jail, but didn’t endear him to Tim (Giovanni Ribisi) who had hired him to do the job. Tim put Andy into the hospital and made it clear that if Andy didn’t come up with the cocaine or its street value in cash, he’d be going to the morgue, and then Tim would come after his family.

Chris had done plenty of work for Tim in the past and tried to reason with him, but the best deal he could get was two weeks to come up with the coke or the cash. Chris wasn’t about to be a drug runner, so he decided to go with counterfeit. As luck would have it, he just happened to have an in with the crew of a cargo ship that was just about to leave for Panama, where he knew a guy who could create “supernotes” that were virtually indistinguishable from legitimate currency. They’d have to work fast, since apparently the ship was only going to be in port in Panama for about four hours before turning right back around to get them back home in time for the deadline, but it was their only shot.

Your ability to enjoy Contraband is probably directly related to your ability to switch off your brain. It is full of logic holes and stupid plot points, but it’s also got a decent amount of action, so it can be fun if you try not to think about it too much. Unfortunately, I’m not very good at doing that, so the utter stupidity of the film hit me with full force while I was watching it. Like Tim’s insistence on continuing to intimidate Kate and the boys while Chris and Andy were doing the job for him. Like how incredibly convenient it was that a ship full of people he knew was ready to take him right where he needed to go right within his very tight time constraints, and how convenient it was that the counterfeiter had no problem coming up with a whole van full of supernotes at the drop of a hat. Like how the thousands of shipping containers would be unloaded and replaced (and the ship refueled and resupplied) in a four hour window. And those are just problems I can mention without spoiling anything.

Since it’s simply not possible to miss the absurdity of the story, you’d think that they would put in enough other content to try to overcome that, but I didn’t really find that to be the case. It’s not boring per se, but there are only a few scenes where there might be enough action to forget how stupid it is. There’s also not much in the way of comedy or interesting dialogue, so it’s pretty much a one-trick pony that doesn’t even do that trick very well.

The Divide

January 18, 2012 / Neil Wilson

Michael Biehn has been in a number of great films, including The Terminator, Aliens, The Abyss, and Tombstone, and a number of others of varied quality. When The Divide came to Austin, the Alamo Drafthouse invited Biehn to introduce it, during which time he gave away a lot of the important plot points and spoiled large portions of the film. But as it turns out, the movie really isn’t worth seeing, so perhaps it’s better to have him tell you want happens than to watch it for yourself.

The film opens with a city (presumably New York) devastated by a massive explosion, probably from some kind of nuclear device. Mickey (Biehn) had been a firefighter during 9/11, and that experience forever changed his life. He gave up being a firefighter to become a building superintendent, and he lived his life under the assumption that it was only a matter of time until the next big thing hit. So as a haggard survivalist, he turned the building’s basement into a bunker where he would be able to ride out whatever might come in relative comfort while the rest of the world was destroyed around him. But he hadn’t expected he’d have to share that space with about nine of the other building’s residents who managed to make their way in before he sealed the door.

For someone so certain of an impending violent end to civilization, he stocked his bunker in a rather unusual way. He had a supply of food and water, although it had been intended for just one man and would be a stretch to sustain ten people until the radiation subsided enough to make it safe to go outside. And yet despite his solo intentions, he had plenty of mattresses for everyone to sleep on (which they made an effort to point out, and yet no effort to explain). He had a gasoline generator, although it was employed in a pretty uneconomical manner to operate a refrigerator and big screen TV. He had lockers and walkie-talkies and plenty of cigarettes, but no guns.

Almost instantly, factions arose within the bunker. Certainly seeing the world they knew come to an end didn’t put them in a great mental state to start with, and cabin fever, hunger, and boredom set in very quickly. It also didn’t help much that they fell under attack pretty early on by a group of soldier-scientists who set up some kind of laboratory right outside their door in a plot line that was never explained and never addressed again. But a complete lack of any form of logic is one of the defining characteristics of the movie, and because it’s about two hours long they have a lot of time to fill with things that don’t make sense and people you don’t care about.

The film is ultimately more psychological thriller than horror, and deals more with the breakdown of civilization in this closed, high-pressure environment. But merely having characters lose their minds and do irrational and impolite things doesn’t really make for much of a thriller. And although they were brought together by a nuclear blast that leveled the city, it really doesn’t really fit in the post-apocalyptic category either. If it is necessary to try to classify The Divide, I’d say it best fits in the “not worth watching” category.

Pariah

January 17, 2012 / Neil Wilson

I know that it’s a subject that I rant about frequently, but I absolutely hate movie trailers. There’s never been a case in which seeing a trailer for a movie before seeing the movie has improved my experience, but there have been many cases in which the trailer has detracted from the film or convinced me to not see it. Pariah is yet another great movie with a horrible trailer, and it really prevented me from experiencing the full effect of the film.

Alike (played by Adepero Oduye) is a smart high school student and aspiring writer. Her name is pronounced uh-leek-ay, but everyone except her father Arthur (Charles Parnell) just calls her Li. She’s also a lesbian, wears mannish clothes, and tries to do whatever she can to avoid looking “girly”. She and her best friend Laura (Pernell Walker) frequent lesbian night clubs, and she doesn’t do anything to hide her orientation at school, but she hasn’t come out to her family and she’s in the habit of changing her clothes after she leaves home in the morning and before she returns at night.

Although Alike’s parents have their suspicions, they’re grasping at whatever straws they can to hold out hope that she’s straight. Her father is a police detective who becomes aggressive if anyone even hints that she might be gay, and he tries to spend as much time as possible away from home to avoid interacting with her. Her mother Audrey (Kim Wayans) is a devout churchgoing woman who is also in denial and tries to buy her cute outfits and force her to spend less time with Laura and more time with Bina (Aasha Davis), the daughter of a coworker and fellow congregation member.

Had I not seen the trailer ahead of time and had it spoil some of the most significant scenes for me, I probably would have been a lot more devastated by this movie than I was. It’s relatively short, but it packs in a lot of emotions like fear and denial and anger and inadequacy. Part of the film’s power comes from the great performances, but it also stems from the realization that her parents are concerned more about how her lesbianism will reflect on them. Her father becomes enraged if anyone even broaches the subject with him, and her mother seems more concerned about Alike’s appearance than her feelings.

The roles are all played expertly, but I was prevented from becoming truly engrossed in the story because I had (based on the trailer) a pretty good idea how it was going to end. It’s a movie that is definitely worth seeing, but I’d recommend trying not to learn much about it before diving in.

Carnage

January 14, 2012 / Neil Wilson

I can’t imagine what it must be like to grow up as a kid in today’s world. When I was a kid, I played on dangerous playground equipment and in homemade tree houses. I rode in the back of station wagons and pickup trucks without being restrained in any way. And with entertainment like The Karate Kid and Teenage Mutant Ninja Turtles, I spent a lot of time fighting with friends. It was all in fun, but occasionally someone would get hurt by a punch or kick that landed just a little too hard. But today’s idiotic “zero tolerance” policies seem like a way of immersing children in the police state culture before they get a chance to have fun or make mistakes they can learn from.

Penelope and Michael Longstreet (Jodie Foster and John C. Reilly) and Nancy and Alan Cowan (Kate Winslet and Christoph Waltz) are just these kinds of overprotective parents, with the added bonus that they’re very concerned about how the behavior of their children reflects on their image. When their sons Ethan and Zachary get into an altercation, it’s imperative that they (the parents, with no sign of the children) get together and discuss it. When Penelope and Michael invited Nancy and Alan to their apartment, they thought it would be a relatively quick ordeal to craft a statement that both parties could agree on. When the film begins, they’re just finishing that statement and Nancy and Alan are about to leave. And they’re still just about to leave for the next hour and a half.

Their interaction starts off in an excessively civilized way, with everyone going far out of their way to be more polite and sophisticated than they really are (with the possible exception of Penelope, who at least believes that she’s as refined as the image she’s trying to present). Penelope and Michael live in the kind of apartment that may be nice to look at, but that would be horrible to live in. The coffee table littered with art books is front and center, the walls are lined with decorative sculptures and at least one well-lit but completely empty frame, and the television is high up in the corner right next to the window, with none of the furniture really facing it.

Nancy and Alan make their way out the door several times, and yet they always seem to be drawn back in for some reason. First, it’s for coffee and cobbler. Then it’s for more coffee. Then it’s a phone call that Alan can’t take in the elevator because there’s no reception. And then it’s the arguing. As they day wears on and the alcohol comes out, the politeness and civility is dropped and their real personalities are exposed.

The film is based on a play, and that really comes through in its direction. With the exception of bookends shot at the nearby park where the fight occurred, the whole movie takes place in Penelope and Michael’s apartment (or in the hall just in front of it), and mostly just in the living room. But they make good use of that space and the film doesn’t really feel constrained in any way. It’s a good “pressure cooker” kind of environment, but they find increasingly interesting (and funny) ways to vent that pressure.

John C. Reilly is a great choice for Michael because his comedy experience really comes through, but so does the more heartfelt performances from recent films like Cyrus and Terri. Christoph Waltz also gives a great performance, although his is the only character that doesn’t really undergo any significant transition in the film. He starts off as a disinterested, self-involved jerk and stays that way throughout the movie, whereas it takes time for the others to drop their personas and devolve to his level. Kate Winslet and Jodie Foster also do well enough in their roles, although in many ways it feels that they’re playing more supporting roles than starring even if everyone has roughly the same amount of screen time and dialogue.

I was pleasantly surprised by the movie’s comedy. Although I was initially concerned about the “sophisticated” humor which is only mildly funny and won’t get more than polite laughs, I was happy to see that as the walls are broken down between them, their interactions get funnier. It’s still not going to go toe-to-toe with the best pure comedies, but there’s enough substance layered underneath to make it a thoroughly enjoyable movie.

UnboundID LDAP SDK for Java 2.3.0

December 1, 2011December 28, 2016 / Neil Wilson

We have just released the UnboundID LDAP SDK for Java 2.3.0. It’s been about six months since the last release, and there are several new features and bug fixes. It is available for download now from the UnboundID website and the SourceForge project page, and it’s also available in the Maven central repository.

The release notes contain a pretty comprehensive set of changes since the 2.2.0 release, but some of the most significant changes are as follows:

It is now possible to use DNS SRV records (as described in RFC 2782) to automatically discover available LDAP servers in the environment. The implementation will respect defined priorities and weights, and can be used for individual connections or connection pools.
Experimental support has been added for the password policy control (as defined in draft-behera-ldap-password-policy) and the no-operation control (as defined in draft-zeilenga-ldap-noop). Even though these drafts are not necessarily finalized, some servers (including the UnboundID Directory Server) have implemented support for them, so it is useful to be able to access them through the LDAP SDK.
The schema caching mechanism (which makes it possible for client-side determinations to use server schema for more appropriate matching) has been made much more efficient so that multiple connections to the same server and with equivalent perceptions of the schema will reference the same object rather than having separate equivalent objects.
Updated the LDAP SDK so that operations invoked via the asynchronous API will still respect client-side timeouts.
A number of schema-related changes have been made to the in-memory directory server. You can now update the schema dynamically through LDAP modify operations. Supported syntaxes and matching rules are now advertised (at least when using the default standard schema). You can configure the server to allow attribute values which violate the associated attribute syntax, or to allow entries with multiple structural object classes (or no structural class at all).
The in-memory directory server has been updated with support for equality indexes to help speed up certain kinds of search operations (particularly when dealing with more than a handful of entries).
The in-memory directory server has been updated to always use the “dn:” form in authorization identity response controls. Previously, it could use the “u:” form in responses to SASL PLAIN binds that used the “u:” form in the request. It will also now use the correct value of “” instead of “dn:” to indicate the anonymous authorization identity.
It is now possible to customize the values that will be displayed for the vendorName and vendorVersion attributes in the root DSE. This can help the server more effectively fool applications which are coded to only work with certain directories.
The LDAP SDK persistence framework has been updated so that it supports attributes with options (e.g., “userCertificate;binary”). It is now also possible to specify superior object classes that should be included in entries that are created.
The connection pool implementation has been updated to provide better closed connection and unsolicited response detection for connections operating in synchronous mode.
The 2.2.0 release added support for using a newly-created connection to retry operations that failed in a manner that indicated the connection may no longer be valid. In the 2.3.0 release, it is now possible to configure that capability based on the type of operation being processed, whereas in the previous version all operation types were handled in an identical manner.
The LDIFReader has new convenience methods that can be used to read the contents of an LDIF file and retrieve the contents as a list of entries. This can be convenient when working with small LDIF files, especially for testing purposes.
The LDAP SDK now supports parsing LDAP URLs with an “ldapi” scheme. The LDAP SDK does not provide support for LDAPI (LDAP over UNIX domain sockets) in the out-of-the-box configuration, but it can now parse URLs using an “ldapi” scheme.
Command-line tools have been updated so that they can specify a tool version. If this is used, then the LDAP SDK can automatically add a “–version” argument to such tools which will cause the version string to be printed to the terminal.
Some changes were made to help the LDAP SDK be more fully functional on IBM Java VMs. This includes necessary changes to support GSSAPI on IBM VMs, and a workaround for an apparent bug that could result in exceptions from concurrent calls to SocketFactory.createSocket methods.

The Problems with Twitter’s Automatic URL Shortening

June 15, 2011December 28, 2016 / Neil Wilson

At the beginning of 2010, I decided to start writing up my thoughts on all of the first-run movies that I see in the theater. It’s debatable about whether those reviews are any good, but I know that at least some people read them. All of my reviews from the last year and a half are available at http://www.viewity.com/.

Last Thursday, I saw (but did not particularly enjoy) J.J. Abrams’ new movie Super 8, and last night I finally got around to writing my review of it, which I posted at http://www.viewity.com/reviews/super-8.html. I use Squarespace to host the reviews, and one of the services it provides is the ability to define a shorter URL that can be used to reference the content. I took advantage of this and created the path “/Super8” instead of “/reviews/super-8.html”. Squarespace also offers support for using multiple domains with the same account, and I have “vwty.us” in addition to “viewity.com”. What this ultimately means is that going to “http://vwty.us/Super8” will take you to “http://www.viewity.com/reviews/super-8.html”.

Whenever I post a new review, one of the ways I let people know about it is by Twitter. The whole reason that I offer the shorter version of the URL is that Twitter limits posts to a maximum of 140 characters, and at 21 characters, the short version of the URL is less than half the size of the 43-character long form. This gives me more space to say something about the movie in addition to merely providing the link, and I try to give at least a hint about whether I liked it. For Super 8, the tweet that I composed was:

Super 8 is super underwhelming. http://vwty.us/Super8

However, what actually got tweeted was:

Super 8 is super underwhelming. http://t.co/TZ43SmY

I will grant you that what Twitter actually made available on my behalf is a whopping two characters shorter. However, it is also much worse than what I had originally written, for many reasons.

First, it’s completely unnecessary. As I mentioned before, Twitter places restrictions on the length of your tweets, but I wasn’t anywhere near that. What I originally wrote was 89 characters, which means that I could have written up to 51 more characters before running out of space. I could have even used the original 43-character URL if I had wanted to and still had plenty of space left.

Second, Twitter’s change dramatically obscures the URL. From the URL that I provided, you can tell that it goes to the vwty.us domain (which is a brand that I control and want to be associated with), and the “/Super8” path gives you a pretty good idea what it might be about. On the other hand, with what Twitter actually provided, you can see that it goes to the “t.co” domain (which is known to be a redirect farm so you have no idea where the content actually resides), and the path “/TZ43SmY” tells you nothing about the content. The original URL is very useful. The shortened version is not.

Another significant problem is that the new URL shortener can have a dramatic impact on the availability of your content. Twitter has such a bad reputation in this area that their “fail whale” page is a well known Internet meme. Because a click on the shortened URL must go through Twitter’s servers before sending you to the ultimate destination, if Twitter is having a problem then it can make your content unavailable. As if by fate, when I clicked on the t.co link earlier this morning, I got exactly that failure page telling me that Twitter was over capacity. Nice. Even if it had worked, it still requires an extra HTTP request and more data over the wire, and an unnecessary delay in getting to the actual content.

The requirement to go through Twitter’s service creates even more ways that the content could become unavailable. It’s likely that tweets will outlive Twitter itself. They’re being archived in the Library of Congress (in addition to a number of other sites), and although future generations probably don’t care how I feel about a movie, there could be long-term value in tweets, and links contained in them. If Twitter goes out of business or is otherwise shut down, then their links won’t work anymore even if the content they referenced is still available. Also, it’s worth pointing out that the “.co” TLD is controlled by the government of Columbia, and that government can shut down such URLs at any time. The government of Lybia has done this for “.ly” domains, so it’s certainly not beyond the realm of possibility.

Twitter’s reason for providing this service is that it can “better protect users from malicious sites that engage in spreading malware, phishing attacks, and other harmful activity”. While this sounds noble, it is also completely ineffective against everyone except the most extreme idiots. They’ve already stated that they won’t shorten URLs that were already shortened using other services like bit.ly, so there’s nothing to prevent people doing suspicious things from using one of them for their posts. Further, there’s nothing to prevent me from serving up different content from my server when I can see that the request is coming from Twitter’s malware detection service versus some other content, so I could still serve up bad stuff to people following the links. On the other hand, the fact that they are trying to verify that content is safe introduces a very real possibility for false positives. My site could have completely legitimate and safe content, but if Twitter thinks that it’s bad for some reason then that may significant inhibit the likelihood of people to go there. Given the unacceptably high percentage of false positives I see from other services like this (e.g., Google mail’s spam detection frequently flags things that aren’t spam), this is far from an impossibility.

Finally, in the ultimate act of inanity, Twitter’s URL shortener can actually produce URLs that are longer than the original URL. For example, when I entered a URL of “http://t.co”, Twitter “shortened” it to be “http://t.co/IzZPmi2”.

I realize that Twitter will show an expanded version of the URL in its web interface, but that doesn’t work for alternate clients. For example, when I use Seesmic on my Android phone, I get the t.co version. And even if I’m using a client that automatically expands that URL, it will only work if the shortening service is available.

Great job, Twitter. This “feature” that I can’t disable has made my links less available, less recognizable, and more likely to be flagged as malicious content. I don’t need any more hurdles to have to get by for people to read the useless drivel that I write.

Comparing Java LDAP SDK Performance

May 31, 2011 / Neil Wilson / 5 Comments

At UnboundID, we take performance seriously and are always trying to improve. This applies just as much for our client-side SDK as for our server products, since a server isn’t very useful without client applications to take advantage of it. There are a number of tools that can be used to measure various aspects of directory server performance, but it’s not as simple to measure the performance of client-side libraries.

To help address this problem, I’ve written a simple tool that can be used to perform a number of different kinds of LDAP operations using various Java-based LDAP SDKs. It’s not particularly elaborate and there’s only a command-line interface, but it provides a range of options, including which SDK to use, the type of operation to test, the number of concurrent threads to use, the length of time to run the test (and to warm-up before beginning to collect data), the type of entries to use, the number of entries to return per search, the result code that should be returned for each operation, and how frequently to close and re-establish connections.

It’s obviously the case that, as the primary developer for the UnboundID LDAP SDK for Java, I am more than a little biased about which SDK I prefer. However, to the best of my knowledge the way that the tool performs the tests is as fair as possible and uses the most efficient mechanism offered by each of the libraries. If anyone believes that there is a more efficient way to use any of the SDKs, then I’d be happy to hear about it and update the results accordingly.

At present, the tool provides at least some level of support for the following SDKs:

Apache LDAP API, version 1.0.0-M3. Although I have written code in the hope of testing this SDK, it does not appear to be fully functional at the present time. For example, when trying to perform searches with multiple threads using a separate connection per thread (which is the only way I have used it to this point), it looks like only a single thread is actually able to perform searches and all the others throw timeout exceptions. If anyone knows how to work around that, I’d be happy to hear about it. Until this problem is resolved, this tool isn’t very useful for testing its performance.
JNDI, as is included in Java SE. For my testing, I used Java SE 1.6.0_25. JNDI is a very abstract API that has the ability to communicate using a number of protocols, and as such was not designed specifically for LDAP. Unfortunately, this means that it’s not ideal for LDAP in a lot of ways. For example, it doesn’t appear that JNDI provides any way to get the actual numeric LDAP result code returned by the server in response to various operations, and it also looks like it does not support bind (for the purpose of authenticating clients) as a distinct type of operation but only in the course of establishing a connection or re-authenticating before performing some other kind of operation. As such, the performance testing tool does not support bind operations, and it does not support testing with operations that return non-successful responses because the result code cannot be verified.
Netscape Directory SDK for Java, version 4.17 (compiled from source, as there does not appear to be a download for a pre-built version of the library). This SDK is fully supported by the performance testing tool.
Novell LDAP Classes for Java, also known as JLDAP, version 2009.10.07-1. This SDK is fully supported by the performance testing tool.
OpenDJ LDAP SDK, version 3.0.0 (snapshot build from May 28, 2011). This appears to be a fork of the OpenDS LDAP SDK that has had the package structure changed, and may have some number of additional changes as well. However, I was not able to successfully use this SDK to run any tests because the code that I used (despite identical code working for the OpenDS SDK, with the exception of changing the package names in import statements) threw an exception when trying to run, indicating that it was attempting to subclass a class that had previously been declared final. It also appeared to be missing an org.forgerock.i18n.LocalizedIllegalArgumentException class, although I worked around that problem by writing my own version of that class.
OpenDS LDAP SDK for Java, 0.9.0 build from May 26, 2011. This SDK is fully supported by the performance testing tool. In addition, because the API provides options for both synchronous and asynchronous connections, the “–useSynchronousMode” option is supported to request using the synchronous version of the API that does not support the use of abandon or multiple concurrent operations on the same connection, while omitting this argument will use a version that does support this capability.
UnboundID LDAP SDK for Java, version 2.2.0. This SDK is fully supported, including the use of the “–useSynchronousMode” option.

These tests obviously require communication with an LDAP directory server. Because the intention is not to measure the performance of the directory server but rather the SDK being used to communicate with that server, it is ideal to use something that is as fast as possible (so that the server is not a bottleneck) and that can be manipulated to give an arbitrary response for any operation. For this purpose, a custom server was created using the LDAP Listener API provided as part of the UnboundID LDAP SDK for Java. It is important to note, however, that even though this API is part of the UnboundID LDAP SDK, it can be used with any kind of client and all interaction with it was over the LDAP protocol using a socket connected over the test system’s loopback interface. The UnboundID LDAP SDK did not have any advantage over any other SDK when interacting with this server.

All of the tests that I ran used Java SE 1.6.0_25 (64-bit version) on a system with a 2.67GHz 8-core Intel Core i7 CPU with 12GB of memory, running a fully-patched version of Ubuntu Linux version 11.04. A detailed description of each of the tests that I ran is provided below, along with the results that I obtained. Each test was run with the JNDI, Netscape, Novell, OpenDS, and UnboundID SDKs, using 1, 2, 4, 8, 16, 32, 64, and 128 client threads. For the OpenDS and UnboundID SDKs, tests were run using both the asynchronous and synchronous modes of operation.

Add Operation Performance

When processing add operations, performance may vary based on the size of the entry being added to the server. As such, I ran two different add performance tests: a “normal-sized” entry (an inetOrgPerson entry with 15 attributes) and a “large” entry (a groupOfUniqueNames entry with 1000 uniqueMember values).

The results I measured when running these tests were:

Add Throughput for Normal-Sized Entries

API	Highest Normal-Sized Entry Add Throughput
JNDI	89,027.954 adds/sec
Netscape	98,916.582 adds/sec
Novell	86,766.964 adds/sec
OpenDS (asynchronous mode)	88,525.069 adds/sec
OpenDS (synchronous mode)	88,586.290 adds/sec
UnboundID (asynchronous mode)	142,105.659 adds/sec
UnboundID (synchronous mode)	174,665.853 adds/sec

Add Throughput for Large Entries

SDK	Highest Large Entry Add Throughput
JNDI	6,472.209 adds/sec
Netscape	8,723.301 adds/sec
Novell	7,437.703 adds/sec
OpenDS (asynchronous mode)	9,454.340 adds/sec
OpenDS (synchronous mode)	9,747.643 adds/sec
UnboundID (asynchronous mode)	17,602.504 adds/sec
UnboundID (synchronous mode)	18,545.810 adds/sec

From these tests, it appears that the UnboundID LDAP SDK for Java is significantly faster than any of the other SDKs when processing add operations, and using the UnboundID LDAP SDK in synchronous mode provides a notable performance improvement over the default asynchronous mode. In contrast, the OpenDS LDAP SDK does not appear to exhibit a significant difference in add performance based on whether the asynchronous or synchronous version of the API is selected.

Search Operation Performance

As for adds, search operation performance can vary significantly based on the size of the entry being returned. As such, I ran tests search tests using the same “normal-sized” and “large” entries as for the add operation testing, and I also tested the case of returning only a single attribute in each entry. Further, because the server can return multiple entries for a single search operation, I ran tests with both operations returning a single entry and 100 identical entries. Results from those tests are provided below:

SDK	Highest Search Throughput for 1 Tiny Entry
JNDI	54,272.423 searches/sec
Netscape	74,601.755 searches/sec
Novell	69,686.323 searches/sec
OpenDS (asynchronous mode)	73,964.204 searches/sec
OpenDS (synchronous mode)	74,572.779 searches/sec
UnboundID (asynchronous mode)	109,159.192 searches/sec
UnboundID (synchronous mode)	168,315.209 searches/sec

SDK	Highest Search Throughput for 1 Normal-Sized Entry
JNDI	46,355.770 searches/sec
Netscape	49,668.681 searches/sec
Novell	55,988.055 searches/sec
OpenDS (asynchronous mode)	55,408.763 searches/sec
OpenDS (synchronous mode)	54,923.308 searches/sec
UnboundID (asynchronous mode)	83,846.853 searches/sec
UnboundID (synchronous mode)	115,738.348 searches/sec

SDK	Highest Search Throughput for 1 Large Entry
JNDI	11,045.600 searches/sec
Netscape	3,849.413 searches/sec
Novell	748.249 searches/sec
OpenDS (asynchronous mode)	10,449.903 searches/sec
OpenDS (synchronous mode)	10,374.687 searches/sec
UnboundID (asynchronous mode)	20,645.026 searches/sec
UnboundID (synchronous mode)	21,341.607 searches/sec

SDK	Highest Search Throughput for 100 Tiny Entries
JNDI	5,749.687 searches/sec
Netscape	2,768.797 searches/sec
Novell	2,739.363 searches/sec
OpenDS (asynchronous mode)	8,295.155 searches/sec
OpenDS (synchronous mode)	8,315.379 searches/sec
UnboundID (asynchronous mode)	5,566.711 searches/sec
UnboundID (synchronous mode)	7,265.108 searches/sec

SDK	Highest Search Throughput for 100 Normal-Sized Entries
JNDI	1,983.319 searches/sec
Netscape	875.249 searches/sec
Novell	1,681.767 searches/sec
OpenDS (asynchronous mode)	1,959.581 searches/sec
OpenDS (synchronous mode)	1,917.131 searches/sec
UnboundID (asynchronous mode)	2,308.414 searches/sec
UnboundID (synchronous mode)	3,278.463 searches/sec

SDK	Highest Search Throughput for 100 Large Entries
JNDI	127.716 searches/sec
Netscape	39.667 searches/sec
Novell	6.731 searches/sec
OpenDS (asynchronous mode)	117.633 searches/sec
OpenDS (synchronous mode)	117.233 searches/sec
UnboundID (asynchronous mode)	225.800 searches/sec
UnboundID (synchronous mode)	237.400 searches/sec

In this case, there is a significant variation in many of the SDKs based on the size and number of entries being returned. The UnboundID LDAP SDK is significantly faster than the other SDKs in most cases (with a notable improvement on top of that when using synchronous mode), but the OpenDS SDK is quite a bit faster than the UnboundID LDAP SDK in the case of a search returning 100 entries with only a single attribute per entry, but that is not the case for normal-sized or large entries. On the other hand, both the Netscape and Novell SDKs appear to be extremely slow when dealing with large search result entries, and the Netscape SDK is also much slower than the others for large entry sets. It is also important to note the significant drop in search performance when using JNDI for larger numbers of threads when returning a single tiny or normal-sized entry.

Modify Operation Performance

For a client SDK, modify performance has fewer variables than for either add or search operations. For a directory server, there are a number of factors, including the size of the target entry, the size of the modified attributes, and whether any of the target attributes is indexed, but none of these has an impact on the client. It is certainly the case that a modify request could update a large number of attributes and/or attribute values, but generally clients modify only one or two values at a time. As such, the only modify test run was for a modify operation replacing a single attribute value. Results for this test are:

Modify Throughput

SDK	Highest Modify Throughput
JNDI	139,593.362 searches/sec
Netscape	139,362.802 searches/sec
Novell	122,199.697 searches/sec
OpenDS (asynchronous mode)	109,556.189 searches/sec
OpenDS (synchronous mode)	109,832.363 searches/sec
UnboundID (asynchronous mode)	196,411.917 searches/sec
UnboundID (synchronous mode)	242,248.797 searches/sec

Again, the UnboundID LDAP SDK is significantly faster than the other SDKs, and again, there is a significant advantage to using synchronous mode. The Novell SDK’s modify performance seems to drop off significantly for higher numbers of threads.

Summary

Download a complete set of results for all tests run as an OpenDocument spreadsheet.

In most cases, the UnboundID LDAP SDK for Java is faster than all other SDKs by a wide margin, and in all cases using the UnboundID LDAP SDK for Java in synchronous mode was faster than using the SDK in the default asynchronous mode. As such, if you are using the UnboundID LDAP SDK for Java and don’t need to perform asynchronous operations, then it is highly recommended that you enable synchronous mode for connections used by that application.

The only case in which the UnboundID LDAP SDK for Java was not the fastest is for a search operation in which a large number of entries were returned with only a single attribute per entry. It was the fastest for both other tests involving a large number of entries, and it was also the fastest for returning only one entry with a single attribute. I will investigate the UnboundID LDAP SDK’s performance in this area to determine whether it can be improved.

The OpenDS LDAP SDK (which was started after I left OpenDS, and for which I have not participated in its development in any way) appears to be the second fastest. It was the only SDK to perform faster than the UnboundID LDAP SDK in any of the tests, and it was never the slowest of any of the tests. There does not appear to be any measurable difference in performance when using the synchronous mode versus asynchronous mode. Across all of the tests, the OpenDS LDAP SDK achieved about 56.0% of the overall performance of the UnboundID LDAP SDK in synchronous mode, and 67.8% of the performance of the UnboundID LDAP SDK in asynchronous mode.

JNDI offers middle-of-the-pack performance in most cases, but its very poor showing for searches returning a single entry with high numbers of threads may be a significant cause for concern, since this is a very common kind of operation.

The Novell SDK performance when dealing with large search result entries is very troublesome, and it is also significantly slower than all other SDKs for modify operations with a high degree of concurrency. The Netscape SDK also appears to have problems with large search result entries, and its search performance for searches returning multiple entries is a problem as well.

UnboundID LDAP SDK for Java 2.2.0

May 23, 2011 / Neil Wilson

UnboundID LDAP SDK for Java 2.2.0 has just been released and is available for download from the UnboundID website or the SourceForge project page, and is also available in the Maven central repository.

The release notes provide a full overview of the changes in this release over the previous 2.1.0 version. There are several bug fixes, but some of the most notable new features include:

A new Minimal Edition has been introduced. The Minimal Edition is available under the same licenses as the Standard Edition and provides support for all LDAP operations, but a number of capabilities have been removed (e.g., support for SASL authentication, a number of controls and extended operations, the persistence framework, the listener framework and in-memory directory server, JNDI and Netscape SDK migration support, etc.). The primary goal of the Minimal Edition is to provide a version of the LDAP SDK with a small jar file size which is desirable for resource-constrained environments like Android applications or other embedded use. The Minimal Edition is available as a separate download, from either the UnboundID website or SourceForge project.
Connection pooling support has been updated to provide the ability to automatically retry operations if the first attempt fails in a way that indicates the connection may no longer be valid. In such cases, a new connection will be established (potentially to a different server, based on the ServerSet in use for the pool) and the operation will be re-attempted on that connection. This can help isolate applications from failures if one of the target directory servers is shut down, crashes, hangs, or begins behaving erratically.
The in-memory directory server has been updated to add support for maintaining referential integrity (e.g., so that if an entry is deleted then that user can be automatically removed from any static groups in which the user was a member), to support LDAP transactions as described in RFC 5805, and to add support for inserting an arbitrary delay before processing operations (which can be useful in simulating environments with slower response times or higher network latencies). There have also been a couple of fixes for bugs that could cause the in-memory directory server to behave incorrectly.
The LDAP SDK persistence framework has been updated to provide better support for searches. Previously, it was difficult to search for entries using anything but equality searches. The generate-source-from-schema tool has been updated so that it will now generate additional methods that can make it easier to perform other kinds of searches, including presence, substring (starts with, ends with, and contains), greater-or-equal, less-or-equal, and approximately-equal-to.
New methods have been added which make it significantly easier to interact with response controls. Each response control class now has one or more static get methods that can be used to extract and decode a response control of that type from a given response object.
Support for GSSAPI authentication has been significantly improved to add support for a number of new options, including the ability to indicate whether to use (or even require) a ticket cache, to specify an alternate location for the ticket cache file, and to request that the TGT be renewed. Changes have also been introduced to make it easier to access GSSAPI debugging information.
A new option has been added that makes it possible to automatically send an abandon request to the directory server if a client-side timeout is encountered while waiting for a response to an operation. Previously, the LDAP SDK would throw an exception but did not have any option to attempt to abandon the operation in the directory server.
The LDAP SDK can now use schema information (if available) in the process of normalizing and comparing DNs and RDNs. This can provide more accurate matching for DNs that use attributes in which something other than case-inexact string matching should be used.
The LDIF reader has been updated to provide the ability to read data from multiple files. This can be useful for cases in which the complete set of data you want to process is broken up into multiple files (e.g., representing different portions of the DIT).

UnboundID LDAP SDK for Java 2.1.0

March 18, 2011 / Neil Wilson / 1 Comment

We have just released the latest version of the UnboundID LDAP SDK for Java, version 2.1.0. You can get it from either the UnboundID website, or from the SourceForge project page.

This is a pretty significant update that has a number of changes in several areas. You can see the release notes for a fairly comprehensive overview of the changes that have been made in this version, but the major focus of the 2.1.0 release was in improving testability, and there are two new features that can significantly help with that:

The LDAP SDK now comes with the ability to create one or more very lightweight yet surprisingly feature-rich LDAP server instances that you can use for testing the directory-enabled applications that you are developing. Although it’s not suitable for production use, it does support quite a few controls and extended operations, SASL authentication, an LDAP-accessible changelog and other features that your application might rely on. It’s also got a lot of nice testing-friendly features like the ability to make assertions about the state of data in the server, the ability to create and restore point-in-time snapshots so that you can revert the server to a known state, and the ability to have multiple listeners that you can control independently for things like testing failover between servers.
It also provides a new LDAPTestUtils class that has lots of helpful methods for use in writing test cases for directory-enabled applications. This includes methods for generating simple entries of various types (e.g., users, groups, organizations, organizationalUnits, etc.), for verifying the contents of a directory server over LDAP, and for verifying results received from processing operations.

These new features have already made it significantly easier for my own testing in developing the LDAP SDK itself, when writing LDAP-based tools, and even in the development of the UnboundID server products like the Directory Server, Directory Proxy Server, and Synchronization Server. Hopefully, it will make it easier for you to test your own applications as well. I’ll be writing additional posts in the near future that go into more detail about how you can use these new features to make it easier to test your directory-enabled applications.

UnboundID LDAP SDK for Java 2.0.1

January 14, 2011 / Neil Wilson / 1 Comment

We have just released UnboundID LDAP SDK for Java version 2.0.1. It is available for download from the UnboundID website or from the SourceForge project page, and should show up in the Maven central repository in the near future. Some of the changes in this release include:

The persistence framework has been updated to support object inheritance. If an object whose class is marked with the @LDAPObject annotation also has a superclass that is marked with @LDAPObject, then the annotated fields and methods of the superclass will be used in the course of converting between objects and entries.
The EntryValidator has been updated to fix a potential bug that could cause an exception to be thrown when processing an entry that did not include all superior classes for all of the object classes in the entry.
Updated the ValuePattern to provide support for back-references, which make it possible to include the same value twice in a generated string (e.g., in the pattern “uid=user.[1-100],ou=org.[ref:1],dc=example,dc=com”, a random number between 1 and 100 will be chosen for the user number, and then that same value will also be used for the organization number). In addition, a bug has been fixed that could interfere with the ability to use the ValuePattern to obtain content from files on Windows systems when the provided path included a drive letter.
The argument parsing framework has been updated to include a new ArgumentListArgument, in which the values of the argument list argument are themselves argument strings (e.g., “–arg1Name arg1value –arg2Name –arg3Name arg3value”), and the argument will have its own parser to perform the validation. Also, the StringArgument has been updated to make it possible to specify a regular expression that can be used to constrain the set of allowed values.
When creating an LDAPConnectionPool using a ServerSet, it is now possible to request that zero connections be initially created. If this is done, then the pool will not have any connections established until the first time an attempt is made to use it.
The Filter class has been updated to expose static encodeValue methods, which may be used to properly escape a value for use in the string representation of a filter. This should only be used if you are manually creating a filter string representation, and is not necessary if you are creating a filter by providing attribute type and value objects.
Debug messages generated by the LDAP SDK have been improved so that they can be more useful, especially for those messages related to LDAP communication in an environment in which many connections may be in use at the same time. Each debug message about LDAP communication will include information about the connection used for that communication and the LDAP message ID for the request or response. Further, result messages will now include the protocol op type.