We have just released version 10.2.0.0 of the Ping Identity Directory Server. See the release notes for a complete overview of changes, but here’s my summary:

Summary of Deprecated Functionality and Potential Compatibility Impacts

• Java 11 support has been deprecated and will be removed in a future release. If you’re currently running on Java 11, we recommend upgrading to Java 17 or 21.
• Java EE provides a framework for HTTP servlets and web applications, along with certain other functionality not directly included in Java SE. Oracle has decided that they will end their role in maintaining Java EE, so that support has been transferred to the Eclipse Foundation. Unfortunately, as part of doing so, Oracle has also revoked the right to use “javax” for base package names for all of those classes, so any code using those classes will need to be updated to replace “javax” package references with “jakarta”. Most of these changes are contained entirely within the server and won’t have any direct customer-facing impact, but some third-party extensions developed using the Server SDK may need to be updated. This primarily includes extensions that specifically deal with functionality formerly provided by Java EE (e.g., HTTP servlet extensions, web application extensions, OAuth token validators, etc.), but it may also apply to additional types of extensions that rely on Java EE functionality.
• The SCIM version 2 API is also affected by the transition of Java EE functionality to the Eclipse Foundation and the need to use “jakarta” instead of “javax” as the base name for affected packages. As such, applications that rely on the SCIMv2 SDK may need to be updated to use the latest release of the SDK.
• Support for SCIM version 1.1 has been deprecated and will be removed in a future release. We recommend upgrading to SCIM version 2.
• Support for SNMP has been deprecated, both for accessing limited monitor data and for generating traps from administrative alerts. We recommend using one of the many alternatives we support, including JMX, Prometheus, or StatsD for access to monitor data, and JMX, SMS, or email messages for alert notifications.

Summary of New Features and Enhancements

• Added the ability to run the server on a Java 21 JVM. Java version 17 is also fully supported, and Java 11 is currently supported but has been deprecated and will be removed in a future release.
• Added support for running in a FIPS 140-3-compliant manner. [more information]
• Added a cache for improving authentication performance when using expensive password storage schemes. [more information]
• Added a new entry counter plugin. [more information]
• Updated the Directory REST API to support making access control decisions based on OAuth scopes. [more information]
• Dramatically improved bind performance in environments with a very large number of dynamic groups.
• Updated the Synchronization Server to support synchronizing changes to a Ping Identity Directory Server for updating both a user’s password and their password policy state at the same time.
• Added the ability to specify a proxy server when defining HTTP external servers in the configuration.
• Added support for pausing database cleaning activity when creating a backup. In some cases, this can increase the speed and reduce the size of the backup.
• Added a new db-on-disk-to-db-cache-size-ratio attribute to database environment monitor entries. Also, added a gauge to raise an alarm if the on-disk database size becomes many times larger than the in-memory cache size, which could lead to performance degradation.
• Added support for caching the contents of key and trust stores for improved performance during TLS negotiation.
• Updated the check-replication-domains tool to better distinguish between deleted and obsolete replicas.
• Updated the dsjavaproperties tool to allow using the new --gcType argument to change type type of garbage collector used for the server.
• Added the ability to use generational ZGC garbage collection when running on Java 21.
• Updated collect-support-data to use the most recent monitor history file if monitor information is not obtained from LDAP.
• Updated the Synchronization Server to use the get changelog batch extended operation as the default mechanism for discovering changes in the Ping Identity Directory Server.

Summary of Bug Fixes

• Fixed an issue in which a Directory REST API error response could potentially allow an unauthorized user to determine whether a specified entry existed in the server.
• Fixed an issue that could cause replication changes to be lost between locations when a remote gateway was in the process of starting or shutting down.
• Fixed an issue that could cause the default topology admin user to be subject to the default password policy when upgrading the server via manage-profile replace-profile.
• Fixed a potential memory leak that could occur in the Synchronization Server in certain failover states when using a Kafka destination.
• Fixed an issue that could result in inconsistency in the metadata for a composite index record. This could cause the server to send errors in response to certain requests, and has the potential to prevent bringing the affected backend online.
• Fixed an issue that could cause upgrade attempts to fail in servers in which the default userRoot backend had been removed.
• Fixed an issue that prevented the server from starting when configured to use a third-party key manager provider created using the Server SDK.
• Fixed an issue in which the Synchronization Server did not always properly encode spaces in HTTP URLs used when communicating with PingOne.
• Fixed an issue in which an untrusted VLV index could interfere with the server’s ability to process certain kinds of searches.
• Fixed an issue in which the server did not always properly use the configured substring-index-entry-limit value when maintaining substring attribute indexes.
• Fixed an issue in which dsjavaproperties did not always properly handle changes to the path to the desired Java runtime.
• Fixed an issue in which the Directory REST API may not use a configured alternative authorization identity when attempting to access data outside the requester’s backend set in an entry-balanced topology.
• Updated the server’s default configuration to prevent going into lockdown mode as a result of missed replication changes from obsolete replicas or as a result of null CSNs.
• Fixed an issue in which the HTTP connection handler’s response-header property was not properly used for certain kinds of error responses.
• Fixed an issue in which the Directory REST API could incorrectly use less-than-or-equal-to matching when comparing JSON fields in cases where strict less-than matching had been requested.
• Fixed an issue in which config-diff could report an error when attempting to compare configuration objects with the same name but different types.
• Fixed an issue in which the Synchronization Server may not properly exclude entries in cases where a configured include-filter targeted a virtual attribute in a NOT component.
• Fixed a potential null pointer exception that could be raised in the Synchronization Server in certain cases in which an operation failed with no additional information about the cause of that failure.
• Fixed an issue that could prevent dsreplication enable from reporting a useful error message when it was unable to establish a connection to one of the server instances.
• Fixed an issue in which isMemberOf values were not automatically updated for groups contained in a subtree that was moved or renamed by a modify DN operation. The server would have continued to use the former group DNs until it was restarted.
• Fixed an issue that allowed the Directory Proxy Server to be configured with attribute mapping proxy transformations for attribute types that were not defined in the schema.
• Fixed an issue in which the server could report an incorrect value for the ds-backend-entry-count attribute in the replicationChanges backend monitor entry if a sequence number counter rolled over after reaching its maximum value.
• Fixed an issue that caused the server to incorrectly indicate that a restart was needed for a change to the LDAP connection handler’s ssl-certificate-nickname property to take effect.
• Fixed an issue that would cause dsconfig or the admin console to suggest a malformed default value when creating a new dictionary-based password validator.
• Reduced the number of error messages generated if the server lost connection to a Prometheus server.
• Updated the server to begin suppressing repeated error log messages of the same type after 200 such messages had been logged. Previously, suppression didn’t kick in until 2000 messages had been logged.
• Fixed an issue in which the server could log information about suppressing duplicate alert messages for alert types that had been disabled.
• Fixed an issue in which the Synchronization Server could incorrectly report errors for all sync pipes when they were only relevant to a single pipe.
• Fixed an issue in which the server could log an irrelevant error message if it was in the process of receiving mirrored topology data when the server began shutting down.
• Fixed an issue with an error message that was generated if an HTTP connection handler could not use a configured key manager provider.

FIPS 140-3 Compliance Support

We have updated the Directory Server, Directory Proxy Server, and Synchronization Server to support operating in a FIPS 140-3-compliant manner using the recently released 2.x version of the Bouncy Castle FIPS-compliant library. The server already provided support for operating in a FIPS 140-2-compliant manner using 1.x versions of the Bouncy Castle library, and that is still an option, but you can now choose to use the newer version of the library when setting up an instance of the server.

If you wish to enable FIPS-compliant mode, use the --fips-provider argument when setting up the server (either when running the setup command in non-interactive mode or when using manage-profile setup), specifying a value of either “BCFIPS1” if you want to use the 1.x version of the Bouncy Castle library for FIPS 140-2-compliant behavior, or “BCFIPS2” if you want to use the 2.x version of the library for FIPS 140-3-compliant behavior. Previously, you could use a value of “BCFIPS” to request FIPS 140-2 compliance, and that is still supported, although there is the potential that could change at some point in the future to defaulting to FIPS 140-3 compliance.

In general, the server’s behavior in FIPS 140-3-compliant mode should be essentially the same as it is in FIPS 140-2-compliant mode.

Encoded Password Caching for Expensive Schemes

Some of the Directory Server’s password storage schemes, including Argon2, PBKDF2, bcrypt, and scrypt, are intentionally designed to be expensive. In the event that an attacker manages to again access to encoded user passwords, passwords encoded with one of those expensive schemes will require significantly greater resources to attack with brute force attacks than passwords encoded with much faster schemes. However, the same holds true for legitimate authentication processing. The server has to do a lot more work to determine whether a bind request should succeed for users with expensively encoded passwords. This increases the length of time for individual authentication attempts, and also dramatically reduces the number of concurrent authentication attempts that the server can process.

To help address this problem, we have introduced a new type of cache that can be used to dramatically speed up authentication performance for users with expensively encoded passwords. Whenever a user attempts to authenticate with a password encoded with one of the aforementioned expensive password storage schemes, the server will check the cache to see if it contains a record for that encoded password. If not, then the server will perform the expensive processing needed to determine whether the provided password is correct. If it is correct, then the server will then encode the clear-text password provided in the bind request with a salted SHA-256 digest and put that in the cache along with the expensive encoding. The next time that user tries to authenticate, the server can retrieve that record from the cache and use the much faster SHA-256 processing to determine whether the provided password is correct.

Note that by default, the server uses PBKDF2 to encode the passwords for root users and topology administrators. As such, even if you’re not using these schemes for regular user accounts, this change can dramatically improve password-based authentication performance for those users. This is especially true for accounts used by applications, including those that the Directory Proxy Server or the Synchronization Server use to authenticate to the Directory Server, since they establish connection pools that can require authenticating multiple connections as that user at the same time.

A few notes about the security of the cache implementation:

• The clear-text password is never stored in the cache. Instead, we store a cryptographically secure SHA-256 digest version of the password that has been combined with a securely generated 128-bit salt.
• The contents of the cache are only stored in memory and are never stored in the database or otherwise persisted. There is also no mechanism for dumping the contents of the cache.
• The encoded password cache only contains a mapping between the original expensive encoding of a password and the faster salted SHA-256 encoding of the same password. It does not include any information that links the password to a particular user entry.
• Because the cache doesn’t do anything to associate an encoded password with the corresponding user, there is no risk that the cache could be used to allow the server to make authentication decisions using stale information. For example, if a user changes their password, the cache can’t be used to allow them to authenticate with their former password. Similarly, if a user’s account is locked or disabled, or if their password is expired, then the cache can’t be used to allow them to successfully authenticate.

By default, the server will cache up to 10,000 passwords using each of the expensive password storage schemes mentioned above. This can be customized using the encoded-password-cache-size property in the storage scheme configuration. If you have a lot of users with passwords encoded using one or more of these schemes, then you may wish to increase the cache size accordingly. Alternatively, if you want to completely disable this caching for some reason, then you can set the value to zero.

The Entry Counter Plugin

In many directory environments, there are a number of potentially expensive queries that are run on a regular basis for reporting purposes. For example, there may be queries to determine the total number of user accounts, and to identify how many of those are active versus inactive. The Directory Server already includes support for a “data security auditor” capability that can be used to identify accounts matching certain criteria, but that results in LDIF files that are written to the server filesystem that need to be processed in some way, and it’s often the case that these kinds of queries are really only needed to determine the number of matching entries rather than to obtain the contents of those entries. To help meet this need, we have introduced a new entry counter plugin in the server.

The entry counter plugin will periodically iterate across entries in the server and count the number of entries matching specified sets criteria. Each set of criteria may include things like:

• The base DN(s) of the subtrees in which to find matching entries.
• A filter to use to identify matching entries.
• An optional additional set of named filters that may be used to further break down matching entries into sub-categories. For example, you may have a set of criteria for identifying all users in the server, with sub-categories for accounts that are considered active (based on last authentication time), those that are considered inactive, and those that have never authenticated.
• Optional warning and/or error thresholds that can be used to cause the server to raise an alarm if the number of matching entries reaches those values. For example, this can be used to notify administrators if the number of users in the server approaches the number allowed for in the product license.
• A flag that indicates whether to track metrics about the size of matching entries, including the average, maximum, and minimum amount of space that encoded representations of the matching entries consume in the underlying database.

Scope-Based Access Control in the Directory REST API

The Directory REST API provides a mechanism for HTTP-based clients to request operations in the server, and it provides substantially greater functionality than is available through SCIM. Clients using the Directory REST API can authenticate using HTTP basic authentication (with a DN and password), but are generally encouraged to use OAuth bearer tokens instead.

Previously, when clients authorized requests with an OAuth bearer token, that token was mapped to a user in the server, and the ACIs applicable to that user were used to determine whether to allow the associated request. However, we have now updated the REST API to make it possible to further consider the set of OAuth scopes in the provided access token when making access control decisions. In particular, the “oauthscope” ACI bind rule can be used to indicate that the ACI applies to requests authorized with a token containing a matching scope. This functionality had previously been available for SCIM requests, as well as for LDAP requests from clients authenticated with the OAUTHBEARER SASL mechanism, but it was not supported for the Directory REST API until the 10.2 release.

Note that for Directory REST API requests that pass through the Directory Proxy Server, we can only support scope-based authorization based in cases in which both the Directory Server and the Directory Proxy Server share a common encryption settings definition. In particular, the Directory Proxy Server must be configured with a preferred encryption settings definition, and that definition must also be available in the Directory Server.