The LDAP Join Control in the Ping Identity Directory Server

/ Neil Wilson

LDAP is an asynchronous protocol, and most directory servers are very good when it comes to handling multiple requests simultaneously, whether on the same connection or multiple connections. If an application needs to issue multiple independent searches in the course of performing some function, and if it wants to get the results in as little time as possible, then it can issue those searches concurrently rather than sequentially. This means that the application should have all the results it needs in the time required to process the longest operation, as opposed to the sum of the times required to process those operations (not to mention the network round-trip time, which gets magnified when issuing requests in series rather than in parallel).

However, you can’t use this approach if there are dependencies between the searches that you want to perform. For example, let’s say that when an employee logs in, you want to show them a portion of their organizational chart that includes their manager and their peers (that is, the other employees who share the same manager). Normally, you’d need to perform three searches:

  • One search to retrieve the entry for the user who is authenticating and get the manager attribute
  • One to retrieve the entry for the user referenced by the employee’s manager attribute
  • One to retrieve the entries of all users with that same manager value

While you could potentially issue the second and third searches concurrently, you have to wait for the results of the first search to have the information you need for the other two.

In the Ping Identity Directory Server, we provide support for a feature that can allow you to do all of this with a single request: the LDAP join control. As its name implies, it offers an LDAP take on the SQL join you can perform in a relational database. If you issue a search request that includes the join request control, then each entry that matches the search criteria will be joined with “related” entries (in accordance with the criteria in the join request control).

The Join Request Control

The join request control describes the relationship that you want to use to identify other entries that are in some way related to the entries matching your search request. The request control has an OID of 1.3.6.1.4.1.30221.2.5.9 and a value with the following ASN.1 encoding: 

LDAPJoin ::= SEQUENCE {
     joinRule         JoinRule,
     baseObject       CHOICE {
          useSearchBaseDN      [0] NULL,
          useSourceEntryDN     [1] NULL,
          useCustomBaseDN      [2] LDAPDN,
          ... },
     scope            [0] ENUMERATED {
          baseObject             (0),
          singleLevel            (1),
          wholeSubtree           (2),
          subordinateSubtree     (3),
          ... } OPTIONAL,
     derefAliases     [1] ENUMERATED {
          neverDerefAliases       (0),
          derefInSearching        (1),
          derefFindingBaseObj     (2),
          derefAlways             (3),
          ... } OPTIONAL,
     sizeLimit        [2] INTEGER (0 .. maxInt) OPTIONAL,
     filter           [3] Filter OPTIONAL,
     attributes       [4] AttributeSelection OPTIONAL,
     requireMatch     [5] BOOLEAN DEFAULT FALSE,
     nestedJoin       [6] LDAPJoin OPTIONAL,
     ... }

JoinRule ::= CHOICE {
     and               [0] SET (1 .. MAX) of JoinRule,
     or                [1] SET (1 .. MAX) of JoinRule,
     dnJoin            [2] AttributeDescription,
     equalityJoin      [3] JoinRuleAssertion,
     containsJoin      [4] JoinRuleAssertion,
     reverseDNJoin     [5] AttributeDescription,
     ... }

JoinRuleAssertion ::= SEQUENCE {
     sourceAttribute     AttributeDescription,
     targetAttribute     AttributeDescription,
     matchAll            BOOLEAN DEFAULT FALSE }

Most of the fields of the join request should be familiar because they’re similar to the fields of an ordinary search request. But I’ll go ahead and call them all out anyway. Also note that the Javadoc for the JoinRequestControl, JoinRequestValue, JoinRule, and JoinBaseDN classes in the UnboundID LDAP SDK for Java may provide additional information.

The Join Rule

The first, and probably most important, field of a join request control is the join rule. This is used to specify the types of entries that should be joined with the corresponding search result entry. We currently offer six types of join rules (and may add support for more in the future):

  • AND — An AND join rule encapsulates a set of one or more other join rules and will only join a search result entry with entries that match the criteria for all of the encapsulated join rules.
  • OR — An OR join rule encapsulates a set of one or more other join rules and will only join a search result entry with entries that match the criteria for at least one of the encapsulated join rules.
  • DN Join — A DN join rule will join a search result entry with other entries whose DNs are contained in a specified attribute in the search result entry. For example, You could use a DN join rule to join a groupOfNames entry with the entries whose DNs are contained in the member attribute of the group. Or you could join an employee’s entry with the entry of their boss via the manager attribute in the employee’s entry.
  • Equality Join — An equality join rule will join a search result entry with other entries that share a common attribute value. For example, say that a mobile phone service provider has an entry for each account, and an entry for each device linked to an account. If the account entry has an accountNumber attribute, and the devices associated with that account also contain that same accountNumber value, you could use an equality join to associate the device entries with the account. Also note that the value that the joined entries have in common doesn’t necessarily have to be in the same attribute type (for example, it could be in the accountNumber attribute of an account entry and the deviceAccountNumber attribute of a device entry).
  • Contains Join — A contains join rule is much like an equality join rule, except that the server uses a substring match (and more correctly, a subAny match) instead of an equality match when identifying the entries to join with the search result entry. That is, the value of a specified attribute in the search result entry must be equal to or a substring of a value in a specified attribute in the joined entries.
  • Reverse DN Join — A reverse DN join rule will join a search result entry with entries that contain the DN of that search result entry in the value of a specified attribute. For example, you could use a reverse DN join to retrieve the entries for a user’s direct reports via the manager attribute.

The Join Base DN

The base DN field of a join request is expressed a little bit differently than the base DN from a search request. In a search request, you must always specify the topmost entry in the subtree containing the entries you’re interested in retrieving. The base DN field of a join request has the same purpose, but there are three different ways that you can indicate what that base DN should be:

  • You can indicate that the base DN from the search request should also be the base DN used when finding entries to be joined with each search result entry.
  • You can indicate that the DN of each search result entry should be used as the base DN used when finding entries to be joined with that search result entry.
  • You can explicitly specify the base DN that you want to use for the join request.

The Join Scope

The scope field of a join request has basically the same meaning as the scope field of a search request, except that it’s relative to the join base DN rather than the search base DN. Those scopes are:

  • baseObject — This indicates that only the entry specified by the join base DN may be joined with the search result entry. None of its subordinates will be included.
  • singleLevel — This indicates that only the entries that are the immediate subordinates of the entry specified by the join base DN may be joined with the search result entry. The join base entry itself will not be included, nor will entries more than one level below the join base entry.
  • wholeSubtree — This indicates that the join base entry and all of its subordinates, to any depth, may be joined with the search result entry.
  • subordinateSubtree — This indicates that all entries below the join base entry, to any depth, may be joined with the search result entry. The join base entry itself will not be included.

Note that the scope element of a join request control is optional, and if it is not provided, then the scope from the search request will be used.

The Join Alias Dereferencing Policy

The alias dereferencing policy in the join request control has the same meaning as in the search request itself. It tells the server how it should treat any aliases that are encountered during join processing. The allowed values include:

  • neverDerefAliases — Indicates that the server should not attempt to dereference any aliases encountered during join processing.
  • derefInSearching — Indicates that the server should attempt to dereference any aliases encountered below the join base DN, but not the join base DN itself if it happens to be an alias.
  • derefFindingBaseObj — Indicates that the server should attempt to dereference the join base DN if it happens to be an alias, but not any aliases encountered below that entry.
  • derefAlways — Indicates that the server should attempt to dereference any aliases it encounters during join processing.

As with the join scope, this is an optional element in a join request control. If you leave it out, the server will use the same policy as specified in the search request.

The Join Size Limit

This specifies the maximum number of entries that should be joined with each search result entry. If a search result entry would have been joined with more than 1000 entries, then the join result control associated with that entry will include a “size limit exceeded” join result code and will not include any of the joined entries.

Note that the server may impose a size limit that is lower than the one requested by the client. In particular, the effective size limit, cannot be larger than the requester’s maximum search size limit, the maximum size limit imposed by the requester’s client connection policy, and the maximum join request size limit defined in the global configuration.

This is an optional element, and if it is not specified, then the size limit from the search request will be used as the size limit for the join processing.

On a related note, the join request control does not include an element to specify a time limit. The time limit from the search request applies to the entire set of processing for the request.

The Join Filter

This is an optional additional search filter that will be required to match any entry for it to be joined with a search result entry. By default, all entries within the scope of the join request that match the criteria specified by the join rule will be joined with the search result entry. If an additional filter is specified, then entries will also have to match that filter to be included in the join.

The Join Attributes

This specifies the set of attributes that should be included in the entries that are joined with a search result entry. This works in the same way as the set of requested attributes for the search request itself, and it can include special tokens like “*” (to indicate that all user attributes should be included), “+” (to indicate that all operational attributes should be included), and “@” followed by an object class name (to indicate that all attributes associated with that object class should be included). If this element is missing or empty, then the default behavior will be to return all user attributes.

Note that the set of attributes that will actually be included in joined entries may be less than the set of requested attributes. For example, some attributes may be excluded because the requester does not have access control permission to retrieve them, or because returning them would violate a sensitive attribute constraint.

The Require Match Flag

This indicates whether a search result entry must be joined with at least one other entry for it to be included in the search results. If this is set to true and there are no entries that match the join criteria for a given search result entry, then that search result entry will not be returned to the client. If this is false, or if it is omitted from the join request control, then the search result entry will still be returned.

The Nested Join Element

The nested join element allows you to extend the join processing to more levels. Not only can you join each search result entry with a related set of entries, but you can also join each of those joined entries with even more related entries based on another set of join criteria. Note that if there is a nested join element, then each joined entry effectively becomes the search result entry for the next level of the join. And if you want to have more than two levels of joins, nested joins can include their own nested joins, but you probably don’t want to go too deep because it has the potential to make the join result really big.

For example, in the scenario we described above where you want to join an employee to their manager and peers, the search request could be used to find the employee’s entry, and you could use a DN join to join it with their manager’s entry (based on the manager attribute in the employee’s entry), and then you could use a reverse DN join to join the manager with their direct reports (also by the manager attribute in their entries).

The Join Result Control

If the search request includes a join request control, then each search result entry will include a join result control that provides information about the join processing for that entry. The join result control has an OID of 1.3.6.1.4.1.30221.2.5.9 and a value with the following encoding: 

JoinResult ::= SEQUENCE {
     COMPONENTS OF LDAPResult,
     entries     [4] SEQUENCE OF JoinedEntry }

JoinedEntry ::= SEQUENCE {
     objectName            LDAPDN,
     attributes            PartialAttributeList,
     nestedJoinResults     SEQUENCE OF JoinedEntry OPTIONAL }

So basically, the join result control contains a result code, an optional diagnostic message, an optional matched DN, an optional set of referral URLs, and a list of the entries that were joined with the search result entry. And if the join request included a nested join, then each joined entry can have its own set of joined entries.

See the JoinResultControl and JoinedEntry classes in the UnboundID LDAP SDK for Java for more information.

An Example Using the ldapsearch Tool

The ldapsearch command-line tool shipped with the Ping Identity Directory Server (and also with the UnboundID LDAP SDK for Java) includes support for the LDAP join control through the following arguments:

  • --joinRule — The join rule to use. This is the only argument that is required to include the join request control in the search request. The value must be in one of the following formats:

    • dn:{sourceAttribute} — Indicates that each search result entry should be joined with entries whose DNs are contained in the specified source attribute of the search result entry.
    • reverse-dn:{targetAttribute} — Indicates that each search result entry should be joined with entries that contain the DN of the search result entry in the specified target attribute.
    • equals:{sourceAttribute}:{targetAttribute} — Indicates that each search result entry should be joined with entries that have the value of the search result entry’s source attribute in the joined entry’s target attribute.
    • contains:{sourceAttribute}:{targetAttribute} — Indicates that each search result entry should be joined with entries that contain the value of the search result entry’s source attribute as a substring in the joined entry’s target attribute.
  • --joinBaseDN — The join base DN to use. If this is omitted, then the base DN from the search request will be used. If it is provided, the value can be one of the following:

    • The string “search-base”, which indicates that the base DN of the search request should also be used as the join base DN.
    • The string “source-entry-dn”, which indicates that the DN of the search result entry should be used as the join base DN.
    • Any valid LDAP DN, which will be used as the join base DN.
  • --joinScope — The scope for the join processing. If this is omitted, then the scope from the search request will be used. If it is provided, then the value may be one of the following:

    • base — The baseObject scope.
    • one — The singleLevel scope.
    • sub — The wholeSubtree scope.
    • subordinates — The subordinateSubtree scope.
  • --joinSizeLimit — The maximum number of entries that should be joined with each search result entry. If this is omitted, then the search request size limit will be used.
  • --joinFilter — An additional filter that joined entries will be required to match. If this is omitted, then no additional filter will be used.
  • --joinRequestedAttribute — The name or OID of an attribute that should be included in joined entries. This can be specified multiple times to indicate that multiple attributes should be included. If this is omitted, then all user attributes will be requested.
  • --joinRequireMatch — If present, this indicates that search result entries that aren’t joined with any entries should be omitted from the results.

As you might have noticed, the ldapsearch tool doesn’t quite provide full support for all of the LDAP join features. For example, it doesn’t offer the AND or OR join rule types, and it doesn’t allow you to perform a nested join. But it’s still good enough to let you try out the join control in a number of common cases.

The following is an example that demonstrates using ldapsearch to perform a DN join that links an employee’s entry to the entry of their boss via the manager attribute: 

$ bin/ldapsearch --hostname ds.example.com \
     --port 636 \
     --useSSL \
     --bindDN 'cn=LDAP Join Example,ou=Applications,dc=example,dc=com' \
     --joinRule dn:manager \
     --joinBaseDN search-base \
     --joinScope sub \
     --joinRequestedAttribute givenName \
     --joinRequestedAttribute sn \
     --joinRequestedAttribute mail \
     --joinRequestedAttribute telephoneNumber \
     --baseDN dc=example,dc=com \
     --scope sub "(uid=ernest.employee)" \
     givenName \
     sn \
     mail \
     telephoneNumber
Enter the bind password:

The server presented the following certificate chain:

     Subject: CN=ds.example.com,O=Ping Identity Self-Signed Certificate
     Valid From: Friday, February 8, 2019 at 12:23:34 AM CST
     Valid Until: Friday, February 4, 2039 at 12:23:34 AM CST
     SHA-1 Fingerprint: 78:f1:49:a0:06:0b:bd:1e:2c:88:cb:76:60:cb:87:cb:c4:c3:76:97
     256-bit SHA-2 Fingerprint: 55:9c:9a:54:97:48:8c:51:fa:10:da:a0:08:f0:15:dc:f0:92:75:3e:e9:be:56:c5:5c:5c:ec:d5:d4:85:15:a2

WARNING:  The certificate is self-signed.

Do you wish to trust this certificate?  Enter 'y' or 'n': y
# Join Result Control:
#      OID:  1.3.6.1.4.1.30221.2.5.9
#      Join Result Code:  0 (success)
#      Joined With Entry:
#           dn: uid=betty.boss,ou=People,dc=example,dc=com
#           mail: betty.boss@example.com
#           sn: Boss
#           givenName: Betty
#           telephoneNumber: +1 123 456 7891
dn: uid=ernest.employee,ou=People,dc=example,dc=com
mail: ernest.employee@example.com
sn: Employee
givenName: Ernest
telephoneNumber: +1 123 456 7890

# Result Code:  0 (success)
# Number of Entries Returned:  1

An Example Using the UnboundID LDAP SDK for Java

I’ve also written an example that demonstrates the use of the LDAP join control in the UnboundID LDAP SDK for Java. The LDAP SDK does support using nested joins, so this example uses the scenario outlined above, in which we retrieve a user, their manager, and their peers. You can find it at https://github.com/dirmgr/blog-example-source-code/tree/master/ldap-join.