Ping Identity Directory Server 8.3.0.0

/ Neil Wilson

We have just released version 8.3.0.0 of the Ping Identity Directory Server. The release notes provide a pretty comprehensive overview of what’s included, but here’s my summary followed by more detailed explanations for many of the changes.

Summary of Deprecated Functionality

Summary of New Features and Enhancements

Summary of Bug Fixes

  • Fix an issue that could allow users in a “must change password” state to issue requests [more information]
  • Prevent warning messages for unrecognized JVM vendors [more information]
  • Fix an issue that could prevent ds-pwp-modifiable-state-json changes from being replicated right away [more information]
  • Improve the logic for maintaining the entry-balancing global index [more information]
  • Fix an issue that could prevent setting up the server on old JVMs without support for 256-bit AES
  • Fix an issue that could interfere with manage-profile replace-profile when using a StatsD monitoring endpoint [more information]
  • Avoid entering lockdown mode when incorrectly believing that there were missed replication changes [more information]
  • Improve replication for dependent changes that may be received out of order [more information]
  • Fix an issue with incorrectly reporting that certain filters were not indexed [more information]
  • Prevent dsreplication status from listing offline servers under incorrect domains [more information]
  • Allow configuring cipher stream providers in Directory Proxy Server, Synchronization Server, and Metrics Engine [more information]
  • Fix an issue preventing manage-profile replace-profile from updating mirrored configuration [more information]
  • Prevent offline config change warnings when using manage-profile replace-profile [more information]
  • Update manage-profile replace-profile to preserve setup logs [more information]
  • Improve validation and behavior when configuring an explicit set of TLS cipher suites [more information]
  • Improve manage-profile replace-profile detection of changes to files not included in the server profile [more information]
  • Fix an issue when trying to update a topology server group with a server that already exists in that group
  • Fix issues with import-ldif with --addMissingRDNAttributes [more information]
  • Fix an issue with dsjavaproperties with --initialize and --jvmTuningParameter [more information]
  • Fix an issue that could prevent Sync failed ops log publishers from being removed
  • Improve the result code when trying to add an entry through the Directory Proxy Server when no backend servers are available or when adding entries with missing parents [more information]
  • Fix a potentially incorrect warning about duplicate jar files detected during startup
  • Fix an issue that could prevent Server SDK plugins from seeing all content in an add operation [more information]
  • Avoid a potential reverse DNS warning message during setup
  • Fix an issue that could cause the server to provide an incorrect estimate for the number of entries matching a filter using a composite index [more information]
  • Improve prompts when using dsreplication in interactive mode [more information]

Deprecate TLSv1 and TLSv1.1

TLS version 1.0 became a standard over twenty years ago in 1999, and TLSv1.1 became a standard over fifteen years ago (in 2006). While they were undoubtedly improvements over the former SSL protocols, they are pretty ancient in the world of computer security and are no longer considered secure. RFC 8996 officially declared them to be historic protocols that should no longer be used. As such, these protocols will no longer be enabled by default in the Directory Server or related products.

Newer, more secure protocols have been around for many years (TLSv1.2 nearly 13 years ago in 2008, and TLSv1.3 almost 3 years ago in 2018), and these will be the only TLS protocol versions enabled by default. This shouldn’t cause any problems unless you have ancient clients that haven’t been updated in over a decade. You can look at SECURITY-NEGOTIATION messages in the server’s access log to identify which TLS protocol versions clients are using, and we strongly recommend updating any clients still using TLSv1 or TLSv1.1. However, if necessary, you can manually enable support for the legacy protocols in the connection handler configuration.

Changes to TLS Cipher Suite Selection

We have deprecated support for TLS cipher suites that use the SHA-1 message digest algorithm. SHA-1 is no longer considered secure, and we had previously only enabled support for these cipher suites to allow for their use in TLSv1 and TLSv1.1. As we have disabled support for those legacy TLS protocols by default, we have also disabled support for TLS cipher suites that rely on SHA-1.

We have also deprecated support for TLS cipher suites that rely on RSA key exchange. RSA key exchange does not support forward secrecy, which means that if the server certificate’s private key is compromised, then any data transferred over a TLS session negotiated with RSA key exchange using that certificate can be decrypted. Note that this does not prevent using certificates with RSA key pairs, as key agreement algorithms like ECDHE and DHE can still be used with RSA certificates.

Even though we no longer enable support for these cipher suites by default, you can manually enable them if necessary. You can also customize the set of enabled cipher suites on a per-connection-handler basis (for example, if you want to allow different cipher suites for HTTPS connections than LDAPS connections).

And in cases in which you do manually configure the set of TLS cipher suites, we have improved the validation that the server performs for that configuration. Previously, if you manually configured the set of cipher suites, but none of those suites were supported by the JVM, then the server would log a message for each of the unsupported suites and just fall back on a default set of suites. It will not reject an attempt to configure custom suites if none of those suites is available in the underlying JVM. For the sake of preserving compatibility (in cases where you update the server, or if you update the JVM to a version with a different set of supported cipher suites), the server will still allow you to include suites that the JVM doesn’t support, as long as at least one of the configured suites is supported, and it will continue to log a message about each unsupported suite.

Deprecate Incremental Backups

The server has always offered the ability to create incremental backups, which contained only database files that had changed since the previous backup. Unfortunately, there have been numerous problems with this over the years, and we have decided to deprecate this functionality rather than continuing to offer a potentially unreliable backup mechanism.

In the 8.3.0.0 release, the functionality will remain available and with all known issues addressed, but if you run the backup command with the --incremental argument, it will display a warning on the command line indicating that the functionality has been deprecated and will likely be removed in a future release. If you have the server itself initiate an incremental backup (through the tasks interface), then it will log a warning message and generate an administrative alert to provide a more visible warning.

As an alternative to performing incremental backups, we recommend LDIF exports. LDIF data compresses very well, and compressed and encrypted LDIF files are substantially smaller than full backups, allowing you to take them more frequently without consuming more disk space. LDIF exports are also more portable and more useful than backups, allowing you to do things like restore individual entries and perform offline analysis of the data.

Another option is to use the server’s data recovery log in addition to LDIF exports or full backups. The data recovery log has been available since the 7.2.0.0 release, and it is a compressed, encrypted audit log that provides a record of recent changes in reversible form so that they can be easily replayed or reverted if necessary. You can use the extract-data-recovery-log-changes tool to write changes matching desired criteria (for example, all changes recorded since the time of a backup or LDIF export) that can then be replayed using a tool like ldapmodify or parallel-update.

Add a FIPS 140-2-Compliant Mode

FIPS 140-2 is a U.S. government specification that defines requirements around the use of cryptography. U.S. government agencies, and organizations that work with those government agencies, may be required to abide by this specification. In the 8.3.0.0 release, we will allow you to set up the server to operate in a FIPS 140-2-compliant mode, in which only cryptographic operations allowed by the specification will be permitted.

Note that servers operating in FIPS 140-2-compliant mode are not directly compatible with servers not running in FIPS-compliant mode. It is not possible to have a mix of compliant and non-compliant servers in the same topology, nor can you directly replicate between them (although you can use the Synchronization Server to keep a FIPS-compliant topology in sync with a non-compliant topology). You will also not be allowed to update an existing non-FIPS-compliant instance to operate in FIPS 140-2-compliant mode.

When installed in this mode, the server uses the FIPS 140-2-certified Bouncy Castle BCFIPS provider in approved-only mode, along with the FIPS-compliant BCJSSE provider for TLS processing. The server will require secure communication, and certificates must be stored in either BCFKS key stores (which the manage-certificates tool now supports) or PKCS #11 tokens. Data encryption must also be enabled during setup, with at least one encryption settings definition. Because of requirements around the use of PBKDF2 in FIPS-compliant mode, passwords for root users and topology administrators will be required to be at least 14 characters long.

Also note that the Bouncy Castle FIPS 140-2-compliant SecureRandom implementation is very entropy-hungry. If the underlying system does not have enough entropy available, attempts to start the server or launch tools requiring the use of secure random numbers may block for extended periods of time. To avoid this, we strongly recommend installing a hardware random number generator or using an entropy-supplementing daemon like rngd.

Add Support for Passphrase Providers

In some cases, the server may require access to clear-text secrets for use in its processing. For example, it may need clear-text credentials for authenticating to external services or for accessing certificate key and trust stores. Historically, the server has allowed you to provide those secrets by either storing an obscured version of the secret directly in the configuration or by storing it in a file.

In the 8.3.0.0 release, we are introducing a new passphrase provider framework that provides an extensible framework for obtaining access to these kinds of secrets. Initially, we will allow obtaining secrets through the following mechanisms:

  • From an obscured representation of the secret stored directly in the configuration
  • From a file contained on the local filesystem, which may optionally be encrypted with a key from the server’s encryption settings database
  • From an environment variable
  • From a HashiCorp Vault server
  • From the Amazon AWS Secrets Manager service

In addition, the Server SDK provides support for creating custom passphrase provider implementations that use other services or methods for obtaining secrets.

Improved Auditability for SCIM2 Requests

By default, the server authorizes SCIM 2 requests through a combination of two mechanisms:

  • With an authorization identity of “cn=SCIM2 Servlet,cn=Root DNs,cn=config”. Although this is a root account, it does not inherit any of the default root privileges, and it is not assigned any privileges by default. Global ACIs grant that account a minimal level of access to use certain request controls (like server-side sort and virtual list view) and operational attribute (like createTimestamp and modifyTimestamp).
  • With any access control rights granted (via the oauthscope ACI bind rule) for scopes included in the OAuth bearer token used to authenticate to the server.

In the past, the server has not attempted to map the access token used to authorize SCIM2 requests to a local user in the server. As such, the access log would always report that the operations were requested by the “cn=SCIM2 Servlet,cn=Root DNs,cn=config” user, and that DN would also appear in the creatorsName and modifiersName attributes of created and updated entries. This is still the default behavior, but in the 8.3.0.0 release, we’re making it possible to map the access token to a local account and use that account as the authorization identity for any requested operations (along with rights granted to scopes included in the token).

Note that if you use this feature, then you will need to ensure that those mapped users have permission to the same request controls and operational attributes that are available to the SCIM2 Servlet user by default. The easiest way to do this is to ensure that the “scim2” scope is included in any access tokens used to authorize the requests, as this scope will be granted the same default access rights as the SCIM2 Servlet user. Otherwise, you will need to define additional access control rules to grant the necessary rights to the mapped users.

Join Virtual Attribute Types

The Directory Server has provided support for an LDAP join request control for several years. This control allows you to request that the server return “related” entries along with entries that match the search criteria. For example, if you use the manager attribute to hold the DN of a user’s manager, then you can use the join request control to return a user’s manager entry along with the user’s own entry. This is a powerful and useful feature, but because it requires an LDAP control, it’s not been available to non-LDAP clients (like those using SCIM or the Directory REST API). Further, because that control uses a proprietary encoding, it’s not all that convenient to use in LDAP clients unless you’re using the UnboundID LDAP SDK for Java.

In the 8.3.0.0 release, we’re adding support for three new virtual attribute types that make it possible to use some of the power afforded by the join request control but without the need to actually use a control. This makes the content accessible from any LDAP client regardless of the API used to implement it, and also to non-LDAP clients.

The new virtual attribute types include:

  • The DN join virtual attribute allows you to join an entry with zero or more other entries whose DNs are contained in the value of a specified attribute. For example, you could use this to join a user with their supervisor through the “manager” attribute in the user’s entry.
  • The reverse DN join virtual attribute allows you to join an entry with zero or more other entries that contain the DN of the source user’s entry in the value of a specified attribute. For example, you could use this to join a manager with their direct reports.
  • The equality join virtual attribute allows you to join an entry with zero or more other entries that are linked by a common attribute value (that is, the source user entry has a value for one attribute that matches the value of the same or a different attribute in other entries).

In each of these cases, values of the virtual attribute will be a JSON object containing a specified set of attributes from the joined entries.

Admin Console Improvements

In the 8.2.0.0 release, we introduced support for single sign-on authentication for the Admin Console, but this was only supported when authenticating with an OpenID Connect ID token from the PingOne service. In the 8.3.0.0 release, we have expanded that to make it possible to authenticate with tokens from any OpenID Connect provider that the server has been configured to accept.

We have also updated the Admin Console so that it’s possible to invoke either the collect-support-data tool or the manage-profile generate-profile tool against the target server through the web-based interface. The resulting support data archive or server profile will be sent to the client as a zip file download. Note that at present, this is only available when authenticating to the console through basic authentication; it is not yet allowed when authenticating via SSO.

Finally, we have updated the Admin Console so that if it’s running in a separate web application container (rather than as part of the Directory Server itself), it will write its log messages to standard output so that they will appear in the container’s console log by default. When the console is running in the same JVM as the Directory Server, its log output will appear in the logs/webapps/console.log file.

Fixes and Improvements for Accounts in a “Must Change Password” State

When the server receives a request to process an operation under an alternate authorization identity (for example, using the proxied authorization or intermediate client request control), it will make sure that the target account is in a usable state. If the account can’t be used for some reason (for example, because it’s been administratively disabled, the password has expired, or it’s been locked as a result of too many failed attempts), then the server will not allow it to be used as an alternate authorization identity. However, prior to the 8.3.0.0 release, the server did not check to see if an account was in a “must change password” state, which could have incorrectly allowed the server to process an operation as a user via an alternate authorization identity even though it would have been rejected when requested on a connection authenticated as that user. Unfortunately, this also affects requests passing through the Directory Proxy Server, as it uses the intermediate client request control to authorize requests as the end user in backend servers. This has been corrected in the 8.3.0.0 release, and we will be fixing it in updates to older releases in the near future.

We have also added support for a new must-change-password account status notification type. This can be used to trigger some action whenever a user successfully authenticates to the server with an account that is in a “must change password” state. For example, you can have the server send the user an email message indicating that they will need to choose a new password before they will be allowed to request any other operations. While it was already possible to do this in many cases through the existing password-reset account status notification type (which will be triggered if an administrator resets a user’s password), the new notification type allows you to take action even if that previous warning was not heeded, or if the “must change password” state was not triggered by an administrative reset.

In addition, we have updated the server so that when a user successfully binds with an account in a “must change password” state, the bind response will now include a diagnostic message that indicates that the user must choose a new password. This may be beneficial for some clients (although many clients don’t check for or use a diagnostic message in a successful response), but the primary benefit is that it makes it easier to spot this condition in the server’s access log.

Fixes and Improvements for the ds-pwp-modifiable-state-json Operational Attribute

In the 8.2.0.0 release, we introduced a new plugin that makes it possible for clients to change certain aspects of a user’s password policy state through a regular modify operation that provides a JSON-formatted value to the ds-pwp-modifiable-state-json operational attribute. While it was already possible to accomplish this using the password policy state extended operation, this operation uses a proprietary encoding that makes it less convenient for LDAP clients not written with the UnboundID LDAP SDK for Java, and non-LDAP clients (for example, those using SCIM or the Directory REST API) couldn’t request it at all. The server’s support for the ds-pwp-modifiable-state-json attribute addresses all of these issues and makes it easier for appropriately authorized clients to manipulate a user’s password policy state.

Unfortunately, there was a bug in the implementation of this plugin in the 8.2.0.0 release that prevented updates to this operational attribute from being replicated right away. The change would be applied locally, but it would only be replicated to other servers in the topology after another update to the entry. This has been fixed in the 8.3.0.0 release. It should also be addressed in an upcoming 8.2.x patch release, but for now, the best way to work around the problem in 8.2 servers is to ensure that you modify some other attribute in the entry after updating the ds-pwp-modifiable-state-json attribute.

There was also an issue with the out-of-the-box configuration for the plugin that prevented it from being invoked for operations that are part of a multi-update extended operation. This has been corrected in the 8.3.0.0 release. The best way to address this in the 8.2.0.0 release is to remove and re-add the Modifiable Password Policy State plugin, and it will be automatically created with the appropriate configuration.

Further, the implementation we used for the plugin in the 8.2.0.0 release included a couple of limitations that restricted the set of modifications in which it could be updated. Those restrictions include:

  • The server would not allow the ds-pwp-modifiable-state-json attribute to be updated in the same modify request as any other attributes.
  • The server would not allow the ds-pwp-state-json attribute to be updated in a modify operation that was part of an LDAP transaction or an atomic multi-update operation.

Both of those were intentional restrictions based on the way we chose to implement this functionality in the 8.2.0.0 release. But we redesigned the implementation in the 8.3.0.0 release so that these restrictions are no longer imposed.

Cipher Stream Provider Improvements

The server uses cipher stream providers to protect the contents of the encryption settings database. Previous versions of the server included several cipher stream provider implementations, including:

  • Read an encryption passphrase from a file contained on the server filesystem
  • Wait for an administrator to interactively supply the passphrase during the server startup process or when launching a tool that required access to the encryption settings database
  • Obtain an encryption passphrase from a HashiCorp Vault instance
  • Derive an encryption key through interaction with the Amazon Key Management Service
  • Use custom logic implemented in a Server SDK extension

In the 8.3.0.0 release, we have added support for an additional cipher stream provider implementation that obtains an encryption passphrase from the Amazon AWS Secrets Manager service.

We have also updated the Directory Proxy Server, Synchronization Server, and Metrics Engine to expose options for configuring cipher stream providers in those products. Previously, cipher stream provider configuration options were only provided for the Directory Server because they were initially only used for encrypting entries before storing them in the database. However, their usage has expanded to other products, and it is now possible to better protect the contents of the encryption settings database when using those products.

Improvements in PKCS #11 Support

The Directory Server has always provided support for a PKCS #11 key manager provider that makes it possible to access listener certificates in a PKCS #11 token, like an HSM. In the past, it was necessary to pre-configure the JVM with an appropriate security provider (typically by editing its java.security configuration file) with the information needed to access the token. While this worked just fine, it’s not ideal for cases in which it’s not convenient to alter the JVM configuration (for example, if the JVM is provided by the underlying operating system, or if the same installation is shared across multiple applications rather than being dedicated to the Directory Server).

In the 8.3.0.0 release, we’ve updated the server to make it possible to enable PKCS #11 support in a stock JVM by dynamically loading the provider. In such cases, you will likely need to provide a configuration file that provides the information the JVM needs to interact with the PKCS #11 token (for example, the path to the native library that implements support for interacting with the token, and possibly other configuration like the slot to use). You can provide this configuration file during setup if you want to enable PKCS #11 support out of the box, or you can specify it in the PKCS #11 key manager provider configuration. If you’ve already got a JVM that has been manually configured with a PKCS #11 security provider, then it will continue to work as before.

We have also updated the manage-certificates tool so that it can interact with certificates in PKCS #11 key stores. In such cases, the --keystore argument must be provided and must point to the provider configuration file needed to configure the JVM to interact with the PKCS #11 token, and the --keystore-type argument must be provided with a value of PKCS11. You will likely also need to use one of the --keystore-password, --keystore-password-file, and --prompt-for-keystore-password arguments to supply the user PIN for the PKCS #11 token.

Providing Initial TLS Certificates via PEM Files

When setting up the server, if you want to be able to accept secure client communication, then you will need to have a listener certificate to use during TLS negotiation. In the past, we have provided two options for this:

  • You can provide setup with information about existing key and trust stores to use.
  • You can have setup generate a self-signed certificate.

In the 8.3.0.0 release, we are adding support for an additional option: you can provide the listener certificate chain, the listener certificate’s private key, and any trusted certificates in PEM files using the --certificateChainPEMFile, --certificatePrivateKeyPEMFile, and --trustedCertificatePEMFile arguments. This can be useful when migrating from a server that uses PEM files rather than PKCS #12 files or Java key/trust stores. It can also be useful when setting up the server in FIPS 140-2-compliant mode, as you would otherwise need to have an existing BCFKS key store or PKCS #11 token with the desired certificate.

Fixes and Improvements for StatsD Monitoring

The server has provided support for a StatsD monitoring endpoint since the 8.0.0.0 release. In the 8.3.0.0 release, we’re adding the ability to include custom tags (as comma-separated key-value pairs) in each metric message. This can help better differentiate messages from separate instances that are going to the same StatsD endpoint.

We have also fixed an issue that could cause a failure to occur when using manage-profile replace-profile on an instance that is configured with a StatsD monitoring endpoint.

Command-Line Tool Fixes and Improvements

We have made many improvements to command-line tools in the 8.3.0.0 release. This includes:

  • We have updated the default JVM configuration for tools so that most of them will attempt to use less memory.
  • We have added a new oid-lookup tool that can be used to obtain information about a given object identifier that may be used by the server (for example, to identify a schema element, control, extended operation, SNMP trap, etc.), or to obtain the object identifier for something with a given name.
  • We have added a new remove-object-class-from-schema tool that can be used to safely remove an object class from the server schema, even if it has previously been (but is no longer) in use. This is similar to the remove-attribute-type-from-schema tool that was introduced in the 8.2.0.0 release, but for object classes rather than attribute types.
  • We made several improvements to manage-profile replace-profile, including:

    • It is now better able to detect changes in files referenced in the setup-arguments.txt file that reside outside the server profile.
    • The server will no longer warn about offline configuration changes the first time it is started after using manage-profile replace-profile to alter the configuration.
    • We fixed an issue that could cause certain setup log files, which might contain useful troubleshooting information about problems encountered during replace-profile processing, to be replaced by the versions in place before the attempt to run replace-profile.
    • We made it more efficient to apply changes that may require administrative actions.
    • We fixed an issue that could prevent replace-profile from updating information in mirrored configuration.
  • We fixed issues with the import-ldif tool when used in conjunction with the --addMissingRDNAttributes argument. In particular, it could fail to add missing RDN values to an entry if the attribute was required by any of the entry’s object classes, and it could have also resulted in entries with multiple values for single-valued attribute types.
  • We fixed an issue with the dsjavaproperties tool that could prevent it from changing the JVM tuning configuration when the --initialize and --jvmTuningParameter arguments were used together.
  • We have made several improvements to the manage-certificates tool, including:

    • We added a new copy-keystore subcommand to make it possible to copy some or all of the certificates in one key store to another key store (creating it if necessary) of the same or a different type.
    • We added support for BCFKS and PKCS11 key store types.
    • We updated the generate-self-signed-certificate subcommand to add support for optional --output-file and --output-format arguments that can be used to write a PEM-formatted or DER-formatted representation of the generated certificate to a specified file.
    • We updated the list-certificates subcommand display the key store type.
  • We have made several improvements to the dbtest tool, including:

    • We updated the dump-database-container subcommand to provide additional information about an entry’s encoding when dumping the id2entry database, including whether the entry is compressed, whether it is encrypted (and if so, with which encryption settings definition), whether there is a digest, and which attributes (if any) the entry may have. It is also possible to restrict the output to only include entries matching a specified filter.
    • We updated the dump-database-container subcommand to provide more useful information when dumping the state database, including the index name and a user-friendly representation of the trust state.
    • We have updated the dump-database-container subcommand to provide more useful information when dumping the contents of the recent-changes database, including an LDIF representation of the change, the change time, the replication CSN, and information about the original client request.
    • We added new dump-attribute-tokens, dump-object-class-tokens, and dump-dn-tokens subcommands that can display information that the server uses in the course of compacting entry data.
    • We added a new dump-metadata subcommand that can display information from the backend’s metadata database.
  • We updated the ldap-result-code tool to add support for additional output formats, including JSON, CSV, and tab-delimited text.

Index Fixes and Improvements

We also made a number of improvements in index management. These include:

  • We fixed an issue that could cause the server to incorrectly infer that a given search filter might not be indexed in a backend if it was covered by a composite index but not by any attribute indexes. The composite index would actually be used to process applicable searches, but this issue could potentially interfere with certain types of valid configuration changes.
  • We have updated import-ldif to increase the number of threads used to flush intermediate index files to disk, which can dramatically improve import performance. We have also made changes to help reduce the number of intermediate index files that the server generates during an LDIF import, which can help avoid running out of file descriptors when importing very large data sets in deployments with a large number of indexes.
  • We have dramatically improved the performance when removing entry IDs from very large composite indexes.
  • We have dramatically reduced the performance impact incurred when a large exploded index exceeds the index entry limit for a key.
  • We fixed an issue that could cause the server to return an incorrect estimate of the number of entries matching a composite index key with a large ID set after an unclean shutdown.

Performance Improvements

Besides those things already mentioned elsewhere above, we have made the following performance improvements to the server and associated tools:

  • We have updated the server’s support for the get user resource limits request control to make it possible to omit information about the target user’s group membership. The Directory Proxy Server now uses this option when forwarding bind requests to backend servers, which can dramatically improve bind performance in servers with a large number of dynamic groups.
  • We have updated the way the server processes searches that target the isMemberOf virtual attribute when the target entry is a dynamic group. This improvement is particularly significant when the client will be paging through the results rather than retrieving all entries in response to a single request.
  • We have updated setup to provide an --optionCacheDirectory argument that can be used to specify the path to cache files with pre-computed information about which JVM options can be used. This can help improve setup performance, as a significant amount of time during setup is spent automatically selecting an appropriate set of JVM options.
  • We have updated the server to make it possible to minimize the content of conflict prevention details entries that may be generated in response to the uniqueness request control. This can help improve write performance when updating large entries in a request that includes uniqueness constraints.
  • We have updated the purge expired data plugin to allow using multiple concurrent threads when deleting expired entries. The plugin is still single-threaded by default, and we recommend keeping that default configuration unless the plugin isn’t able to keep up with the rate at which entries should be purged.

Reduced DN Escaping

RFC 4514 specifies a minimum set of required escaping that must be used for attribute values used in DNs and RDNs. However, it also allows implementations to optionally escape any other characters as desired. In the Ping Identity Directory Server, we have historically also escaped all ASCII control characters and all non-ASCII characters.

As of the 8.3.0.0 release, we have reduced the amount of escaping that we use for non-ASCII characters. By default, we will no longer escape non-ASCII characters that we believe to be printable. This includes characters from the Unicode letter, number, space, punctuation, and symbol classes. We will continue to escape non-ASCII characters from other Unicode classes, as well as non-ASCII data that does not represent a valid UTF-8 encoding. And we will also continue to escape ASCII control characters.

For example, if an entry has a cn attribute value of “José Núñez 🇩🇴”, it would have previously been encoded as “cn=Jos\c3\a9 N\c3\ba\c3\b1ez \f0\9f\87\a9\f0\9f\87\b4” when returned in search result entries. It will now be returned as “cn=José Núñez 🇩🇴”. Note that LDIF requires all non-ASCII values to be base64-encoded, so if you’re looking at that entry in LDIF (like in the output from ldapsearch), then you’ll now see it as a base64-encoded value, but it will actually be returned by the server in unencoded form.

Improved Logging for Multi-Update Operations

The multi-update extended operation allows you to send multiple add, delete, modify, and modify DN operations in a single request. When you want to apply changes to multiple entries, this can help improve performance by reducing the number of round trips required in communicating with the server, and you can also optionally process the entire multi-update operation as a single atomic unit so that if any of the requests cannot be processed, then no changes will be applied.

In the past, the server’s access logging wasn’t as helpful as it should have been when recording information about operations processed as part of a multi-update extended operation. In particular, messages about the individual operations processed as part of the multi-update were only available if you enabled support for logging internal operations, and the extended result log message did not always include useful information about failures encountered during processing. We have addressed both of these issues in the 8.3.0.0 release so that the server’s processing for the individual operations that are part of a multi-update request will be logged without needing to enable internal operation logging, and if any failure is encountered during processing, then the extended result will provide the result code and diagnostic message from the first failure.

Include the Bouncy Castle Library by Default

When operating in non-FIPS-compliant mode, the Directory Server can use the Bouncy Castle cryptographic library for certain optional functionality, including support for the bcrypt, scrypt, and Argon2 password storage schemes. Previously, the server did not ship with the Bouncy Castle library out of an abundance of caution with regard to U.S. export control laws. If you wanted to use any of this functionality, you needed to download the appropriate jar file from the Bouncy Castle website and copy it into the server’s lib directory.

As of the 8.3.0.0 release, this is no longer necessary. The server now ships with both the FIPS 140-2-compliant and non-FIPS-compliant versions of the Bouncy Castle libraries. Note, however, that these libraries are not compatible with each other, so either one or the other will be activated when setting up the server. When running in FIPS 140-2-compliant mode, features that require the non-FIPS-compliant Bouncy Castle library (namely, the aforementioned bcrypt, scrypt, and Argon2 password storage schemes) will not be available.

Request Control Updates

The 8.3.0.0 includes the following updates in our support for request controls:

  • The assured replication request control (which can be used to delay the response to a change until the server has received confirmation that the change has been replicated) is now allowed through the Directory Proxy Server by default without requiring any special configuration. It was previously only possible to allow the control to be passed through the Directory Proxy Server by adding its OID to the supported-control-oid property in the proxying request processor configuration.
  • We have updated the server to allow the operation purpose request control (which the client can use to provide additional context about the request) to be included in a transaction or an atomic multi-update operation.
  • We have updated the default set of global ACIs to allow the LDAP assertion and permissive modify controls by default. These controls do not allow clients to do anything that they wouldn’t be able to do without them, but they can make processing safer and more efficient.

Unrecognized JVM Vendor Warnings

The server setup process validates the selected JVM to ensure that it is suitable for use. In particular, it examines the Java version (we currently support running on either Java 8 or Java 11) and the Java vendor. setup will refuse to run with an unsupported Java version, but it will merely display a warning message if it encounters a JVM with a supported version but from an unrecognized vendor.

Some Linux distributions, like Red Hat and Ubuntu, have overridden the vendor information for the JVMs that are included as part of their operating systems so that they use vendor strings that we did not previously recognize (and in at least one case, Ubuntu used a vendor of “Private Build”). We have updated the set of recognized vendors so that it will now recognize these vendors, and it will no longer display a warning when setting up the server when using one of those JVMs.

Entry Balancing Global Index Updates

When using the Directory Proxy Server with entry balancing (to shard the data across multiple replicated backend sets for improved performance with very large data sets), it maintains a global index that can be used to help it determine which backend set might have a given attribute value.

We have identified and corrected a number of cases in which the server might not correctly update the global index in response to some types of operations. These include:

  • In some cases, the server could have incorrectly populated the global index when processing an add operation. This could have led it to a scenario in which the Directory Proxy Server might only return a subset of entries containing a given attribute value in response to a search. Each global index configuration object now provides a guaranteed-unique configuration property that can be used to indicate that only a single entry is expected to have any one value for the associated attribute type. The global index will now be updated for add operations only for indexes in which guaranteed-unique is set to true.
  • We fixed an issue that could arise for add operations that used an indexed attribute value in the RDN but not in the content for the entry. In such cases, the attribute value from the RDN would not be populated in the global index for the associated attribute type. This has been corrected, and the global index will be populated for such add operations as long as that index is configured with guaranteed-unique set to true.
  • We fixed an issue that prevented the server from updating the global index for operations that were part of a successful transaction or an atomic multi-update operation.
  • We fixed an issue that could prevent global attribute indexes from being updated as a result of modifications used to remove specific attribute values (as opposed to the entire attribute) from an entry. This now will be properly handled for indexes that are configured with guaranteed-unique set to true.
  • Global attribute indexes were previously not updated for delete operations. We have updated the server so that if the server maintains a global index for any attribute used in a deleted entry’s RDN, and if that index is configured with guaranteed-unique set to true, then that global attribute index will be updated to reflect that the value has been removed.

Replication Improvements

The 8.3.0.0 release includes the following replication-related changes:

  • We fixed an issue that could cause the server to infer an incorrect replication enabled time if no value is provided in a message received from a remote replica. That would cause the server to believe that replication had been enabled in 1970 (the beginning of the epoch used for many OS timekeeping systems), and it would cause the server to enter lockdown mode because it believed it had missed changes from that replica.
  • We improved the processing that the server will perform if it receives interdependent changes in an out-of-order sequence. For example, if a client adds an entry and then immediately modifies it, and if a replica receives the add and modify operations from two different other replicas, then it’s possible that it could try to process the modify operation before the add. This could have previously caused replication of the modify operation to fail.
  • We fixed an issue that could cause dsreplication status to display information about offline servers below all replication domains, even if that server only participated in a subset of those domains. It will now only list them under domains in which they participate.
  • We improved the messages displayed when running dsreplication enable and dsreplication initialize in interactive mode to better highlight the servers and base DNs that will be affected by the operation.

Improved Result Codes through the Directory Proxy Server

We fixed a couple of issues that could cause the Directory Proxy Server to return less desirable result codes in certain cases when using an entry-balanced configuration. These include:

  • When attempting to add an entry when all servers in at least one of the backend sets were unavailable, the server could have returned a response with an inappropriate result code of 81 (server down). It will now use a result code of 52 (unavailable).
  • When attempting to add an entry more than one level below the balancing point, and when the ancestor immediately below the balancing point does not exist, then the server could have returned a response with an inappropriate result code of 80 (other) or 81 (server down). It will now use a result code of 32 (no such object).

Fix for Server SDK Plugins Targeting Add Operations

We fixed an issue that could prevent pre-operation add plugins written with the Server SDK from seeing changes made to the add operation by other plugins that are shipped with the server. For example, a Server SDK pre-operation add plugin was previously unable to see any attribute values generated by the composed attribute plugin.